Architecture Description

The goal of this document is to outline the interactions between the various components of the system. It aims to give a fairly concrete overview of all the different pieces of software involved and how they interact.

Prerequisites

We expect a certain baseline understanding of what we are doing in general: building software systems for Shops that sell physical goods. If you are new to this, you might first want to read our introduction blog post. This overview was written with the following assumptions:

Note

As regards mapping Shops and Relays, this is taken care of by Smart Contracts, which maintain Registries that manage configuration and discovery of Shops and Relays.

For the complete terminology reference, please see Terminology.

Architecture Overview

On a high level, the system is composed of a frontend client, a relay, and smart contracts hosted on an ethereum blockchain.

@startuml ArchitectureNetwork actor "Shop Frontend" as UI database Blockchain as Blockchain queue Relay as Relay UI <-> Relay : HTTP/WS & Protocol Buffers UI <-> Blockchain : Ethereum RPC & Contracts ABI Relay <-> Blockchain : Ethereum RPC & Contracts ABI @enduml

“Architecture Diagram”

Creating a Shop

The first thing a merchant needs to do is create the shop itself. This is done by minting an NFT since the Shop Registry is an extension of EIP721. Thus it also supplies the standard conformant mint function.

Note

The merchant will also configure other shop parameters via the same mechanism as outlined in Shop and Relay Synchronization by mutating the Manifest object to e.g. establish accepted currencies and the like. We will not dive deeper into those details and instead remain focused on giving an overview of the system’s different parts and how they interact.

Adding Users and Logging in

Next, we introduce the concept of Users, how they are registered with a Shop, and how they get access to the Listings and Inventory. In terms of interactions, we differentiate Users into Clerks, Admins and Guests:

  • Admins can add or remove Clerks.

  • Registered are all Users that have a Wallet Address (WA) assigned to the Shop Registry.

  • Registered Users can authorise KeyCards (KC) which give them write access to all the objects in the Shop

  • Guests can also obtain KCs, which are needed to file orders, but they don’t need to be added to the shop registry. See Object Visibility for more on this.

Note

There is only one Owner, who also counts as an Admin. Only the Owner can promote Clerks into Admins and down-grade them again.

Onboarding a Clerk

To add a new Clerk, an Admin creates an Invite Secret IS and a corresponding Invite Verifier IV via their Client software and sends a link containing the IS to the Clerk via some confidential side-channel (for example E-Mail or Signal). Its public part, the IV, is submitted to the Shop Registry.

  • IS and IV are the secret and public parts, respectively, of an ECDSA key-pair.

  • The IS is single use.

  • All Admins can see all IVs for their Shop.

Warning

While this process could be modelled with a hash-function, we need to account for what is known as front running in Ethereum and similar systems, where the inputs to function call on a contract are revealed to the public. Using signatures instead ensures only the Clerk had possession of the IS.

When the Clerk opens the IS-containing link they are presented with a registration page. The main goal here is to connect with a Wallet software, such as MetaMask, to get the WA of the new Clerk. We could also prompt for other metadata about the Clerk at this point such as their name, avatar or e-mail address.

The IS is used by calling the contract function redeemInvite, changing the invite status for the Clerk. The contract uses ecrecover to extract the IV and checks whether it is still valid. At the end of the process the new WA is added to the Shop’s configuration, completing the Clerk’s registration.

The “Adding a Clerk” flowchart shows the interaction between the different actors.

@startuml actor Clerk as C actor Admin as A database Relay as R queue "Etherum Chain" as B == Add User == A --> A ++: Wants to allow Clerk to do Clerk things. note left:Here Clerk and Admin are synonymous\nwith different instances of the shop app. A -> A: Generate a fresh invite keypair consisting of: \nInvite _Secret_ (**IS**) and Invite _Verifier_ (**IV**) note right of A: Conceptually this process could be a hashing operation.\nTo protect against front-running a pre-image, we are using a keypair here.\nIS is the secret and IV is the public part. A -> B ++: Add (**IV**) to ShopReg B --> B: Check if **IV** is already taken B --> B: shop **IV** for one-time use B--> A --: Ok A --> C: Send link containing **IS** deactivate A C -> C++: Opens **IS** link to view Registration page C -> C ++ #a00: Connect with MetaMask C --> C: allow connection C --> C --: return WalletAddress **WA** C --> C: compute S = sign_typedData_v4(IS, WA) C --> B ++: Reddem by submitting signed registration info to ShopReg B --> B: check signature against valid **IV**s alt is invalid B --> C: Error: invite expired else is valid B --> B: Invalidate **IV** B --> B: Update ShopRegistry with new Clerk **WA** B --> C --: Ok end @endump

“Adding a Clerk” flowchart

Authorizing Access

To gain access to the Shop, the Clerk needs to authorise a KeyCard (KC) using their WA. The main goal of the KC is to reduce the number of interaction prompts by the Wallet software. The HTTP Request for this is defined in Enroll KeyCard. The process is as follows:

  1. A Clerk generates a new asymmetric key-pair, which represents the KC. The KC is stored in the Shop page using some browser storage mechanism and is regarded as short-lived.

  2. The public part of the key-pair is signed with the WA of the Clerk, prompting a Wallet interaction.

  3. A request containing KC.public, ShopId and the Signature from (2) is sent to a corresponding Relay of the Shop.

  4. The Relay checks the Signature and if valid, calls hasPermission to check if the WA from the request is allowed to access the shop.

  5. If all checks pass, the KC is added to the list of allowed Users and can then be used to initiate remote procedure call (RPC) connections to all Relays.

Logging in

The Login mechanism is necessary for elevated RPC commands that are not public, like changing the Inventory of the Shop.

The “Logging in” flowchart, below, shows the interaction between the different systems.

@startuml actor Clerk as C actor Admin as A database Relay as R queue "Etherum Chain" as B == KeyCard Enrolment == C -> C ++: Generate new keypair for KeyCard (**KC**) C -> C ++ #a00: Sign **KC**.public with __Secret__ of **WA** C --> C: Prompt Wallet C --> C --: Return Signature S C -> R ++: Send EnrollKeyCard{ShopID, KC.public, S} R --> R: extract **WA** from S via ecrecover() R -> B ++: Check ShopRegistry by calling `hasAtLeastAccess` B --> R --: True if **WA** does have access alt Is Not Allowed R -> A: Send Warning notification R --> C: Error else Is Allowed R -> R: Mark **KC**.public as OK for Login R --> C --: OK C --> C: save **KC** in local storage end deactivate C == Login == activate C C -> R ++: Open RPC connection C --> C ++: Begins Shop session\n(i.e. Request Listing, Build Cart, ...) C -> R ++ #00a : LoginRequest{**KC**.public} R -> R: Check **KC** is allowed and remember R -> R: Roll **Ch**allenge and remember R --> C: Send **Ch** C -> C: Sign **Ch** with **KC**.secret C -> R: Send response {Ch, Sig} R -> R: Check **Ch** is correct R -> R: Check signature using previously claimed **KC**.public alt Checks did not pass R --> C: Error R -> R: Close connection to Clerk else Checks passed R -> R: Unlock Session for Shop access R --> C --: Ok end == Do //Clerk Things//™ == C -> R ++ #0a0: Request listing R -> R: check session state R -> R: process request ... R --> C --: Response... ... C -> R ++ #0a0: Write Events R -> R: check session state R -> R: process request ... R --> C --: Response... @endump

“Logging in” flowchart

Shop and Relay Synchronization

Note

Our working assumption is that Relays and their Clients will build the Listing and Inventory state by means of Eventual Consistency. Meaning: all changes to a Listing consist of individual Patches. To gain the current state of a Listing, all patches are replayed and applied locally.

Items in the Shop: Retrieving the Listing

Now that the Clerk can do things in the Shop, let’s discuss the first common action: putting up items for display in the Shop, also known as: the listings.

The “Retrieving the Listing” flowchart, below, shows the interaction between the different systems.

@startuml actor Clerk as C boundary "Shop Page" as S database Relay as R queue "Etherum Chain" as B C -> S ++: Opens Page alt Listing is up2date S --> C: RenderPage(Listing) else Listing expired S -> B ++: LookupRelays(Shop) B --> S --: Relays: [X, Y, ...] S -> R ++: Connect(X) and Authenticate(KC) R --> R: check **KC** validity R --> S --: Response(ok) S --> R ++: SubscriptionRequest(Listing:A, StartSeqNum:N) note right of R: all listings are public.\nSo no access control is done here. R --> R: buffer Patches where seq > N note left of R: TODO: this could be optimized into returning\nthe reduced state with the subscribe result\ninstead of replaying the full log... loop while len(buffer) > 0 R --> R: Es = [E1, E2, ...] = take(buffer, M) R --> S: SubscriptionPushReq(Es) S -> S: applyPatch(E) S --> R: Ok R --> R: check if there are new patches and fill buffer accordingly deactivate R ... end S -> B ++: ReadRootHash(Shop) B --> S --: {root: xyz, ...} S -> S: Verify(Listings) S --> C: RenderPage(Listings) end deactivate S @enduml

“Retrieving the Listing” flowchart

Updating the Listing

To change the Listing we need to create Patches. Patches to a Listing will result in a new reduced state of the Shop, from which we create a root hash. To keep Relays honest, this root hash is saved on-chain, protecting against omissions or other corruptions. See Shop Configuration for more details.

Patches must have a known type. Currently we foresee the following types of patches:

  • Creating and updating Listing objects

  • Changing inventory count of an Item (See Shop Inventory)

  • Creating Order objects and assigning Items to them

  • Creating Tag objects and assigning Items to them

See CBOR References for a full list. For more details on how to create patches, see Patching Objects.

@startuml patches-mutate-listing !pragma useVerticalIf on if (create listing) then (hash state) :empty listing; #green:Hash Alpha; detach elseif (add title) then (hash state) :listing with title; #yellow:Hash Beta; detach elseif (add description) then (hash state) :listing with title and description; #orange:Hash Gamma; detach elseif (delete description) then (hash state) :listing with title; #yellow:Hash Beta; detach endif @enduml

“Replaying the mutations/patches results in a specific state, which computes into a specific hash deterministically.”

Note

We might have a second class of Events in the future: Events that are non-durable. Specifically, these might be order-related events or other session data that will have to be kept in sync between relays but which will not mutate the Listing and thus not change the root hash saved on-chain.

Selling Items

Now that we can add and remove Listings from a Shop, let’s discuss how they are actually sold with our system.

Warning

To protect from selling items twice, the Relays will help with bookkeeping. The Relays will track inventory and mark items as reserved once an order is finalized.

Creating and Completing an Order

In our system, customers can browse a shop and place orders independently. When a customer wants to make a purchase, they first create a Guest KeyCard which allows them to create orders in the shop without requiring a Clerk’s assistance.

When a Guest adds an Item to their Order, the Client creates a patch operation to modify the order object. This is done using our Patching Objects system, which allows for precise modifications to shop objects. For example, a guest might append a new item to their order or increase the quantity of an existing item using patch operations.

The Relay processes these patches, checks inventory availability, and broadcasts the updated state to the Guest and to the Clerks.

The flowchart below shows the interactions that eventually result in an order being filled.

@startuml actor "Guest/Customer" as U boundary "Shop Page" as S database Relay as R queue "Ethereum Chain" as B activate U == Create Guest KeyCard == U -> S ++: Browse shop S -> S: Generate guest KeyCard (**GKC**) == Prepare Order == S -> B ++: getListingState(Shop) B --> S --: {root: d3adb33f, ...} S --> S: check against local listing state opt Item listing out of date S -> R ++: connect(**GKC**) and subscribe R --> S: PatchSetPushReq{patchset1, ...} ... R --> S --: patchsetN S --> S: apply and verify end U -> S: Add items to order S -> S: Create new order object U -> S ++: Add items A, B, and C to order S -> R ++: Patch{append, path:["orders", orderId, "items"], value:[{ListingID: A, Quantity: 1}]} S -> R: Patch{append, path:["orders", orderId, "items"], value:[{ListingID: B, Quantity: 2}]} S -> R: Patch{append, path:["orders", orderId, "items"], value:[{ListingID: C, Quantity: 3}]} R --> S --: Ok S -> S: Update Order UI S --> U --: Items added U -> S: Proceed to checkout S -> R ++: PatchSet{replace, path:["orders", orderId, "State"], value:"Committed"} R --> R: check inventory availability alt All items available R --> S --: Ok S -> S: Update Order UI S --> U: Proceed to payment else Some items unavailable R --> S --: Err:OutOfStock S -> S: Update Order UI S --> U: Show error "Some items out of stock" U -> U: Reconsider selection end note right of R: Inventory is only checked when\norder state is set to Committed deactivate S @enduml

“Filling an Order” flowchart

The Relay keeps track of inventory across all concurrent orders, ensuring that the same item isn’t sold twice. However, items are not reserved until checkout begins.

Warning

Item counts are checked against the total stock. Orders don’t lock up inventory until checkout has begun. This means that the stock count can change between adding an item to the order and checking out. Guests will receive inventory status updates when changes affect their order.

Completing the Purchase

Once the Order is filled with the desired Items, the Guest initiates check-out by submitting patches that add shipping information and chosen currencies, finally changing the order OrderState to Committed. This is also accomplished using the Patching Objects system, which allows for transactional updates to shop objects.

When the order is committed, the Relay processes the patches, locks up the inventory, and provides the Guest with payment information including the payment id and the amount to pay by setting the PaymentDetails object on the order.

The Guest reconstructs the PaymentRequest data structure, which is needed for the pay contract call. If their wallet does not support calling contracts directly, they can use getPaymentAddress to complete payment by manually sending the amount to the returned address, which is a counter-factual CREATE2 deployment that will call pay() once the merchant sweeps it.

The Relay supports both payment modes transparently, for vanilla Ether and for all configured ERC20s. Once the Guest has paid for their purchase, the inventory is updated through additional patches applied by the Relay, and the on-chain data is updated.

The “Completing the Purchase” flowchart, below, shows the interaction between the different systems.

@startuml actor Customer as C boundary "shop Page" as S database Relay as R queue "Etherum Chain" as B == Check out == C -> S ++: Click's "check out" S -> R ++: Send Patch{order: xyz, state: Committed} R --> R: freeze order xyz (see previous, inventory check etc.) R -> R ++ #0d0: reduce order to calculate total amount and payment details R --> R: sum total amount of order\nTotalAmount = 9001 R --> R: Build inventory update patches: U = [A-1, B-2, C-3, ...] R --> R: newRoot: 0x12345... R --> R: recipteHash = 0xdeadbeef R --> R --: {recipteHash, newRoot, InventoryUpdate, totalAmount} R -> R: Call getPaymentAddress(merchantAddr, totalAmount, recipteHash)\nPaymentAddress = b33f... R -> R: Create Patch{set, path:["orders", xyz, "PaymentDetails"],\nvalue: {totalAmount=9001, ttl=24hrs, ...}} R --> S: CommitCardResponse{order_finalized_id=cf1} R -> B++ #f58025: wait for on-chain updates to b33f... S -> S: generate QR code information for payByAddress\nCO={addr: b33f..., chainId: ...,amount: 9001} S -> C ++: showQRcode(CO) C --> C: interacts with their wallet C -> B --: sendFunds(b33f, 9001) B --> R --: funds deposited in block 0x9876... note left of B: the relay might wait N blocks\nas confirmation / protection against re-orgs R --> R: create patch{set, path:["orders", xyz, "TxDetails"],\nvalue: {BlockHash: 0x9876...}} R --> R: create patch{set, path:["orders", xyz, "State"],\nvalue: paid} deactivate B == Update Listing == R --> R: create patch{decrement, path:["inventory", "A", "count"], value: 1} R --> R: create patch{decrement, path:["inventory", "B", "count"], value: 2} R --> R: create patch{decrement, path:["inventory", "C", "count"], value: 3} R -> B ++: shopReg.updateRootHash{shopId, newRoot} B --> R --: new block is X R --> S --: Push order updates [TxDetails, State=paid] S --> S: emit "purchase complete" S --> C--: checkout successful C -> C --: wait for delivery of items @enduml

“Completing the Purchase” flowchart

Privacy Considerations

Warning

Now and next: There is a bunch of things in this current iteration we want to test, knowing that they won’t remain that way. There might be breaking changes in future versions of the protocol.

  • Listing data will be public - we want to make it possible to be indexed and aggregated in the future.

    • Inventory counts, too? This will give a clear transparent picture of how good or bad a shop is doing and is not in the interest of merchants.

  • Users that are running the shop. This should be private/internal company data.

    • We need to think about how to protect this data from being leaked. Probably using separate logs with stricter access control or encryption.

  • The merchantAddr will be where all purchases are collected.

    • Can someone backtrace all sales from that account address?

    • Assuming yes, will we see the paymentAddr of individual sales and thus their account address, too?

  • What exactly hashes into the Order?

    • Can Listing changes be correlated to purchases?