======================== Architecture Description ======================== The goal of this document is to outline the interactions between the various components of the system. It aims to give a fairly concrete overview of all the different pieces of software involved and how they interact. Prerequisites ============= We expect a certain baseline understanding of what we are doing in general: building software systems for :term:`Shops` that sell physical goods. If you *are* new to this, you might first want to read our `introduction blog post `_. This overview was written with the following assumptions: - Readers have conceptual familiarity with `Public-Key signature schemes `_. - **Users** of a :term:`Shop` have a :term:`Wallet`. - The mapping between a :term:`Shop` and its :term:`Relays` is known. .. note:: As regards mapping **Shops** and **Relays**, this is taken care of by :ref:`smart-contracts`, which maintain **Registries** that manage configuration and discovery of :term:`Shops` and :term:`Relays`. .. _registries: For the complete terminology reference, please see :doc:`terms`. .. contents:: Table of Contents :depth: 3 Architecture Overview ===================== On a high level, the system is composed of a :term:`frontend client`, a :term:`relay`, and :term:`smart contracts` hosted on an ethereum blockchain. - The :term:`frontend client` can be used to create and manage :term:`Shops`, and interacts with the :term:`relay` and :term:`smart contracts` under the hood. - The :term:`relay` is responsible for processing requests from the :term:`frontend client` and for maintaining the state of the :term:`Shop`. - The :term:`smart contracts` are supplying discoverability and access control for :term:`Shops`. - The :term:`frontend client` and :term:`relay` communicate with each other via a :term:`WebSocket` connection. See :ref:`transport-overview` for more details. - The :term:`frontend client` and :term:`relay` communicate with the :term:`smart contracts` via the Ethereum JSON-RPC protocol. See :ref:`smart-contracts` for more details. .. uml:: architecture-diagram.puml :name: architecture-diagram :caption: "Architecture Diagram" Creating a Shop ================ The first thing a merchant needs to do is create the shop itself. This is done by minting an **NFT** since the :ref:`sc-registry-shop` is an extension of `EIP721 `_. Thus it also supplies the standard conformant :sol:func:`mint` function. .. note:: The merchant will also configure other shop parameters via the same mechanism as outlined in :ref:`shop-and-relay-synchronization` by mutating the :cbor:ref:`Manifest` object to e.g. establish accepted currencies and the like. We will not dive deeper into those details and instead remain focused on giving an overview of the system's different parts and how they interact. .. _shop-users: Adding Users and Logging in =========================== Next, we introduce the concept of **Users**, how they are registered with a :term:`Shop`, and how they get access to the :term:`Listings` and :term:`Inventory`. In terms of interactions, we differentiate **Users** into **Clerks**, **Admins** and **Guests**: - **Admins** can add or remove **Clerks**. - **Registered** are all **Users** that have a **Wallet Address (WA)** assigned to the :ref:`sc-registry-shop`. - Registered **Users** can authorise **KeyCards (KC)** which give them write access to all the objects in the :term:`Shop` - **Guests** can also obtain **KCs**, which are needed to file orders, but they don't need to be added to the shop registry. See :ref:`object-visibility` for more on this. .. note:: There is only one **Owner**, who also counts as an **Admin**. Only the **Owner** can promote **Clerks** into **Admins** and down-grade them again. .. _adding-clerks: Onboarding a Clerk ------------------ To add a new **Clerk**, an **Admin** creates an **Invite Secret** ``IS`` and a corresponding **Invite Verifier** ``IV`` via their Client software and sends a link containing the ``IS`` to the **Clerk** via some confidential side-channel (for example E-Mail or Signal). Its public part, the ``IV``, is submitted to the :ref:`sc-registry-shop`. - ``IS`` and ``IV`` are the secret and public parts, respectively, of an ECDSA key-pair. - The ``IS`` is single use. - All **Admins** can see all ``IV``\ s for their :term:`Shop`. .. warning:: While this process could be modelled with a hash-function, we need to account for what is known as `front running `_ in Ethereum and similar systems, where the inputs to function call on a contract are revealed to the public. Using signatures instead ensures *only* the Clerk had possession of the ``IS``. When the **Clerk** opens the ``IS``-containing link they are presented with a registration page. The main goal here is to connect with a :term:`Wallet` software, such as MetaMask, to get the **WA of the new Clerk.** We *could* also prompt for other metadata about the **Clerk** at this point such as their name, avatar or e-mail address. The ``IS`` is used by calling the contract function :sol:func:`redeemInvite`, changing the invite status for the Clerk. The contract uses `ecrecover `_ to extract the **IV** and checks whether it is still valid. At the end of the process the new **WA** is added to the Shop's configuration, completing the **Clerk**'s registration. The :ref:`flow-adding-clerk` shows the interaction between the different actors. .. uml:: flow-0.1-registration.puml :name: flow-adding-clerk :caption: "Adding a Clerk" flowchart Authorizing Access ------------------ To gain access to the :term:`Shop`, the **Clerk** needs to authorise a :term:`KeyCard` (**KC**) using their **WA**. The main goal of the **KC** is to reduce the number of interaction prompts by the :term:`Wallet` software. The HTTP Request for this is defined in :ref:`enroll_key_card`. The process is as follows: 1. A Clerk generates a new asymmetric key-pair, which represents the **KC**. The **KC** is stored in the Shop page using some browser storage mechanism and is regarded as short-lived. 2. The public part of the key-pair is signed with the **WA** of the **Clerk**, prompting a Wallet interaction. 3. A request containing **KC.public**, **ShopId** and the **Signature** from (2) is sent to a corresponding :term:`Relay` of the Shop. 4. The Relay checks the Signature and if valid, calls :sol:func:`hasPermission` to check if the **WA** from the request is allowed to access the shop. 5. If all checks pass, the **KC** is added to the list of *allowed* **Users** and can then be used to initiate remote procedure call (RPC) connections to all Relays. Logging in ---------- The Login mechanism is necessary for elevated RPC commands that are not public, like changing the :term:`Inventory` of the :term:`Shop`. - It follows a simple `challenge/response scheme `_ to protect against `replay attacks _`. - It is started by sending :ref:`ref_market.mass.AuthenticateRequest` to the :term:`Relay`. - The **Clerk** proves possession of their **KC** by signing the ``challenge`` the :term:`Relay` responded with. - The **Relay** checks the ``signature`` in :ref:`ref_market.mass.ChallengeSolvedRequest`. - Afterwards, the **Clerk** can use the **KC** to sign :ref:`PatchSets` and perform actions on the :term:`Shop`. The :ref:`flow-logging-in`, below, shows the interaction between the different systems. .. uml:: flow-0.2-auth-and-login.puml :name: flow-logging-in :caption: "Logging in" flowchart .. _shop-and-relay-synchronization: Shop and Relay Synchronization ============================== .. note:: Our working assumption is that :term:`Relays` and their Clients will build the :term:`Listing` and :term:`Inventory` state by means of :term:`Eventual Consistency`. Meaning: all changes to a *Listing* consist of individual :ref:`Patches`. To gain the *current* state of a Listing, all patches are replayed and applied locally. Items in the Shop: Retrieving the Listing ----------------------------------------- Now that the **Clerk** can do things in the :term:`Shop`, let's discuss the first common action: putting up items for display in the :term:`Shop`, also known as: the :term:`listings`. - The :term:`Client` connects to the :term:`Relay` and opens a subscription for some or all listings, using the :ref:`ref_market.mass.SubscriptionRequest`. - The :term:`Relay` will send all applicable :ref:`Patches` via :ref:`ref_market.mass.SubscriptionPushRequest` to the :term:`Client`. - The Client code *should* verify Listing state against on-chain data, by computing and comparing the state root hash. The :ref:`flow-show-listing`, below, shows the interaction between the different systems. .. uml:: flow-1-show-listing.puml :name: flow-show-listing :caption: "Retrieving the Listing" flowchart Updating the Listing -------------------- To change the :term:`Listing` we need to create :ref:`Patches`. Patches to a Listing will result in a new reduced state of the :term:`Shop`, from which we create a **root hash**. To keep :term:`Relays` honest, this root hash is saved on-chain, protecting against omissions or other corruptions. See :ref:`sc-shop-config` for more details. Patches must have a known type. Currently we foresee the following types of patches: - Creating and updating :cbor:ref:`Listing` objects - Changing inventory count of an Item (See :cbor:ref:`Shop` ``Inventory``) - Creating :cbor:ref:`Order` objects and assigning Items to them - Creating :cbor:ref:`Tag` objects and assigning Items to them See :ref:`cbor-references` for a full list. For more details on how to create patches, see :ref:`object-patches`. .. uml:: patches-mutate-listing.puml :name: patches-mutate-listing :caption: "Replaying the mutations/patches results in a specific state, which computes into a specific hash deterministically." .. note:: We might have a second class of Events in the future: Events that are non-durable. Specifically, these might be order-related events or other session data that will have to be kept in sync between relays but which will not mutate the Listing and thus not change the root hash saved on-chain. Selling Items ============= Now that we can add and remove Listings from a :term:`Shop`, let's discuss how they are actually sold with our system. .. warning:: To protect from selling items twice, the :term:`Relays` will help with bookkeeping. The :term:`Relays` will track inventory and mark items as reserved once an order is finalized. Creating and Completing an Order -------------------------------- In our system, customers can browse a shop and place orders independently. When a customer wants to make a purchase, they first create a Guest :term:`KeyCard` which allows them to create orders in the shop without requiring a Clerk's assistance. When a **Guest** adds an Item to their :term:`Order`, the **Client** creates a patch operation to modify the order object. This is done using our :ref:`object-patches` system, which allows for precise modifications to shop objects. For example, a guest might append a new item to their order or increase the quantity of an existing item using patch operations. The :term:`Relay` processes these patches, checks inventory availability, and broadcasts the updated state to the **Guest** and to the **Clerks**. The flowchart below shows the interactions that eventually result in an order being filled. .. uml:: flow-2.1-fill-order.puml :name: flow-fill-order :caption: "Filling an Order" flowchart The :term:`Relay` keeps track of inventory across all concurrent orders, ensuring that the same item isn't sold twice. However, items are not reserved until checkout begins. .. warning:: Item counts are checked against the total stock. Orders don't lock up inventory until checkout has begun. This means that the stock count can change between adding an item to the order and checking out. Guests will receive inventory status updates when changes affect their order. Completing the Purchase ----------------------- Once the :term:`Order` is filled with the desired Items, the **Guest** initiates check-out by submitting patches that add shipping information and chosen currencies, finally changing the order :cbor:ref:`OrderState` to ``Committed``. This is also accomplished using the :ref:`object-patches` system, which allows for transactional updates to shop objects. When the order is committed, the :term:`Relay` processes the patches, locks up the :term:`inventory`, and provides the **Guest** with payment information including the payment id and the amount to pay by setting the :cbor:ref:`PaymentDetails` object on the order. The **Guest** reconstructs the :sol:struct:`PaymentRequest` data structure, which is needed for the :sol:func:`pay` contract call. If their wallet does not support calling contracts directly, they can use :sol:func:`getPaymentAddress` to complete payment by manually sending the amount to the returned address, which is a counter-factual **CREATE2** deployment that will call **pay()** once the merchant sweeps it. The :term:`Relay` supports both payment modes transparently, for vanilla Ether and for all configured ERC20s. Once the **Guest** has paid for their purchase, the inventory is updated through additional patches applied by the :term:`Relay`, and the on-chain data is updated. The :ref:`flow-check-out`, below, shows the interaction between the different systems. .. uml:: flow-2.2-check-out.puml :name: flow-check-out :caption: "Completing the Purchase" flowchart Privacy Considerations ====================== .. warning:: **Now and next**: There is a bunch of things in this current iteration we want to test, knowing that they won't remain that way. There might be breaking changes in future versions of the protocol. - Listing data will be public - we want to make it possible to be indexed and aggregated in the future. - Inventory counts, too? This will give a clear transparent picture of how good or bad a shop is doing and is not in the interest of merchants. - Users that are running the shop. This should be private/internal company data. - We need to think about how to protect this data from being leaked. Probably using separate logs with stricter access control or encryption. - The **merchantAddr** will be where all purchases are collected. - Can someone backtrace all sales from that account address? - Assuming yes, will we see the **paymentAddr** of individual sales and thus their account address, too? - What exactly hashes into the :term:`Order`? - Can Listing changes be correlated to purchases?