In Coda, payments pass through several steps before they are considered verified and complete. This document is meant to walk through what happens to a single payment as it works it's way through our codebase. For a more high-level simple overview aimed at users who want to understand a little bit about how payments work check out the lite lifecycle of a payment.
Let's say you want to send a payment in Coda, assuming you've already made you're account and you have funds.
Your friend gives you her public key -- it's KEFLx5TOqJNzd6buc+dW3HCjkL57NjnZIaplYJ50DO1uTfogKfwAAAAA.
Then you invoke the following command:
$ coda client send-payment -receiver PUBKEY -amount 10 -privkey-path ~/my-private-keyCoda client -- client.ml
client.ml defines the CLI command parser for the coda client subcommands.
We use Jane Street's standard libraries often.
Specifically Core (common datastructures) and Async (asynchronous programming using the composable Deferred.t type) are used very often.
On top of most files you'll see some variant of open Core and open Async.
Here in client.ml we see that too. Async shadows the Command type and lets us declaratively express the details of each command.
If you scroll to the bottom of client.ml, you'll find we register the send-payment command to the function send_payment.
Here we describe the flags this action depends on: receiver a public key, a fee, an amount, and a path to your private key. These flag param kinds are defined in daemon_rpcs.ml.
In the body of send_payment we build a payment and send it over to the daemon.
In payment.mli, you'll see a couple important things. (1) we break down payments into a payment payload (the part that needs to be signed) and the rest. and (2) you see the type defined in what will seem to be a strange manner, but is a common pattern in our codebase.
For more see:
Let's dig into the payment payload:
Check out payment_payload.mli. Recall that a payload is the part of the payment the sender will sign with her private key. We see this is built up out of the receiver public key, amount, fee, and a nonce. Again the payload is SNARKable so it has a type var, and other important Snarkable functions (in a future RFC we will put this in a custom ppx_deriving.
(TODO: @ihm can you correct any details I mess up here)
We use Schnorr signatures. A Schnorr signature is an element in a [group](https://en.wikipedia.org/wiki/Group_(mathematics). Our group is a point on an elliptic curve. So what is a signature? Open up signature lib's checked.ml and scroll to module Signature within module type S. It's a non-zero point on a curve, aka a pair of two curve_scalar values. To sign we give a private key and a message, we can verify a signature on a message with a public key.
This is the first time we see heavily functored code, so see functors if you're confused. This is also the first time we see custom SNARK circuit logic, see custom SNARK circuit logic for more.
In private_key.ml we see a private key is a Tick.Inner_curve.Scalar.t or a scalar on an elliptic curve. Let's break it down more precisely: Because we rely on recursive zkSNARKs we actually have two elliptic curves called Tick and Tock. Most of our logic happens within Tick (TODO: @ihm expand on this). Schnorr signatures demand we use scalars for our private key.
The public key corresponding to a private key p is just include Non_zero_curve_point
Public keys can also be compressed -- see public_key.mli. A point on an elliptic curve can unambiguously be represented by a single scalar field element and a boolean. This is the representation we use into the payment payload because it's more efficient inside of SNARK circuits.
In currency.mli, we define nominal types for fee, amount, and balance that handles overflow and underflow properly. Everything is backed by 64bit unsigned integers for now. Notice, that we again include SNARK circuit operations under the Checked submodules within each of the types.
Payments are applied successfully only if certain properties hold of the account of the sender (and the receiver's balance doesn't overflow).
Checkout account.ml, an account is a record with a public key (the owner of the account), a balance, a nonce, and a receipt chain hash.
A payment is valid if:
- The signature is valid w.r.t. the public key of the sender
- The sender has enough balance to pay out the fee and the amount
- The reciever has enough room for the amount s.t. there won't be an overflow
- The account nonce matches the nonce inside the payment.
When we apply a payment we also cons it onto the receipt chain, and increment the account nonce.
Fees are handled out-of-band see the fee excess system.
This is encoded inside the SNARK in transaction_snark.ml, specifically the apply_tagged_transaction function, although you'll need to look at how the bool flags are set in the is_normal case.
It's captured outside the SNARK here: (TODO: where is this? Ledger_builder somewhere?)
The account_nonce.mli is just a nominal type around a natural number. This is used for protection against double-application of payments. The account nonce is incremented in the sender's the account whenever a payment is applied.
The receipt.mli chain hash is the top hash of a merkle list of payment payloads. This is used to prove that you actually made a payment to someone. Since Coda doesn't keep payment history, this is how you can prove to someone that your payment went through.
How does it work?
A merkle list is like a merkle tree but with one branch. As long as you keep your merkle list hashes, you can prove that any individual piece of data was part of the list if you know for sure what the top hash is.
If it's important to prove your payment went through, you ask the receiver to start recording receipt chain hashs, and hand your payment payload over to the receiver. He can then check the top hash of their receipt chain to see if it includes your payload.
We've fully described all the components of a Payment.t. Congrats on making it this far!
After a break, we'll be ready to dive into the daemon code.
The coda daemon is defined inline in coda.ml. Search for the daemon function to see the CLI flags we use there. The daemon is optionally auto-started by the client if it doesn't already exist. We get configuration from a JSON configuration file (try first from -f, then from $XDG_CONFIG_DIR/coda/daemon.json, then from /etc/coda/daemon.json). We do a lot of setup here which leads up to invoking Coda_main.Coda.Make and then Runing it. The details of those are described below.
When we have the Run module, we can make an instance of the coda daemon at the value level, and set up any background processes and services.
By the time you're reading this, hopefully we've tamed the beast that is coda_main.ml. Here we wire the system together at the module level. What does this mean? We instantiate all the functors for the different subcomponents of the daemon. Eventually we create something that conforms to Main_intf (in this same file).
At the bottom of coda_main.ml, we define a Run functor that finally has the other side of the rpc call that the client makes to send_payment. Run contains the server-side implementations of all the RPC calls the client makes. It also is responsible for logic of setting up any RPC/webservers servers and background processes.
Let's assume we have an instance of Run.t already created, and we'll circle back later.
In daemon_rpcs.ml, we define the concrete RPC calls that the client uses to communicate to the daemon. We use Async's RPC library for this. Send_payments defined the RPC call we use to send the payment: the query type is the input -- the payments we want to send -- and the response is the output -- in this case unit, because we don't get any meaningful feedback other than "the payment has been enqueued" on success.
Back in coda_main.ml, we invoke send_payment in Run, that delegates to schedule_payment -- here we enqueue the payment into the Transaction Pool.
To create a Run instance we'll need to go to coda_lib.ml where we wire all subsystems together at the value level. This is in contrast to coda_main.ml where we wire all the subsystems together at the module level.
It's here where we can trace the path of the payment from the transaction pool forwards. Let's sketch that out before diving deeper into each of the subsystems:
- The transaction pool broadcasts diffs from the transaction pool through to the network
- The proposer reads payments from the transaction pool when it's time to make a transition from one blockchain state to another, those payments are part of a diff to update a ledger builder which is committed to inside the new blockchain state. External transitions are emitted.
- The network and the proposer feed external transitions containing information on how to update a ledger builder with the new payment buffered to the ledger builder controller
- The ledger builder controller figures out where this external transition fits in it's tree of possible forks. If this happens to extend our "best" path (the state upon which we will propose later) then we do an expensive materialization step to create a tip holding the new ledger builder and emit this strongest tip over the network. Healthy clients only forward tips they locally think are the strongest.
Open up transaction_pool.ml... TODO
TODO
TODO
TODO
TODO
A ledger builder can be regarded as a "Pending accounts database" that has transactions(payments, coinbase, and proof-fees) applied for which there are no snarks available yet.
A ledger builder consists of the accounts state (what we currently call ledger) and a data structure called parallel_scan.ml. It keeps track of all the transactions that need to be snarked (grep for Available_job.t) to produce a single transaction snark that certifies a set of transactions. This is exposed as Aux in the ledger builder.
Parallel scan is a tree like structure that stores statements needed to be proved. A statement can be of applying a single transaction Base or of composing other statements Merge. Snarking of these statements is delegated to snark-workers. The snark workers submit snarks for the corresponding statements which are used by the proposer to update the parallel scan state.
When the propser wins a block, the payments read from the transaction pool are sent to the ledger builder to create a diff Ledger_builder_diff.
A diff consists of
- Payments included in the block
- A list of proofs that prove some of the transactions (payments, coinbase, and proof-fees) from previous blocks
- Coinbase
There are two primary operations in ledger builder.
-
Creating a diff : To include a payment from the transaction pool, the proposer needs to include snarks generated by its own snark-workers (or buy it from someone) which certifies some of the transactions added in previous blocks. The number of snarks needs to be twice the number of transactions being included in the block (an invariant of the aux data structure). These proofs are included in the diff along with the payments and coinbase. The diff is then included in the external transition and broadcasted to the network.
-
Applying a diff: Diffs from the node itself (Internal transitions) or from the network (External transitions) are then used to update the ledger builder by applying the payments to the ledger and updating the parallel scan state with the proofs. Applying a diff may produce a proof for a sequence of transactions that were included in the previous blocks.
TODO