Merge pull request #23 from Layr-Labs/depend-on-eigenda-client

Use High-Level EigenDA Client

Makefile

@@ -27,10 +27,10 @@ clean:
 	rm bin/eigenda-proxy
 
 test:
-	go test -v ./... -test.skip ".*E2E.*"
+	go test -v ./...
 
 e2e-test: submodules srs
-	go test -timeout 50m -v ./test/e2e_test.go 
+	go test -timeout 50m -v -testnet-integration ./test/e2e_test.go 
 
 .PHONY: lint
 lint:

README.md

@@ -1,42 +1,58 @@
 # EigenDA Sidecar Proxy
 
 ## Introduction
-This simple DA server implementation is a side-car communication relay between different rollup frameworks and EigenDA. This allows us to keep existing protocol functions (i.e, batch submission, state derivation) lightweight in respect to modification since the server handles key security and data operations like:
+
+This service wraps the high-level EigenDA client, exposing endpoints for interacting with the EigenDA disperser in conformance to the [OP plasma server spec](https://specs.optimism.io/experimental/plasma.html), and adding disperser verification logic. This simplifies integrating EigenDA into various rollup frameworks by minimizing the footprint of changes needed within their respective services. Features of the EigenDA sidecar proxy include:
+
 * blob submission/retrieval to EigenDA
-* data <--> blob encoding/decoding
-* tamper resistance assurance (i.e, cryptographic verification of retrieved blobs)
+* data <--> blob encoding/decoding for BN254 field element compatibility
+* tamper resistance assurance (i.e, cryptographic verification of retrieved blobs) for avoiding a trust dependency on the EigenDA disperser
 
-This allows for deduplication of redundant logical flows into a single representation which can be used cross functionally across rollups.
+In order to disperse to the EigenDA network in production, or at high throughput on testnet, please register your authentication ethereum address through [this form](https://forms.gle/3QRNTYhSMacVFNcU8). Your EigenDA authentication keypair should not be associated with any funds anywhere.
 
 ## EigenDA Configuration
-Additional cli args are provided for targeting an EigenDA network backend:
-- `--eigenda-rpc`: RPC host of disperser service. (e.g, on holesky this is `disperser-holesky.eigenda.xyz:443`)
-- `--eigenda-status-query-timeout`: (default: 25m) Duration for which a client will wait for a blob to finalize after being sent for dispersal.
-- `--eigenda-status-query-retry-interval`: (default: 5s) How often a client will attempt a retry when awaiting network blob finalization. 
-- `--eigenda-use-tls`: (default: true) Whether or not to use TLS for grpc communication with disperser.
-- `--eigenda-g1-path`: Directory path to g1.point file
-- `--eigenda-g2-power-of-tau`: Directory path to g2.point.powerOf2 file
-- `--eigenda-cache-path`: Directory path to dump cached SRS tables
+
+Additional CLI args are provided for targeting an EigenDA network backend:
+
+* `--eigenda-rpc`: RPC host of disperser service. (e.g, on holesky this is `disperser-holesky.eigenda.xyz:443`, full network list [here](https://docs.eigenlayer.xyz/eigenda/networks/))
+* `--eigenda-status-query-timeout`: (default: 30m) Duration for which a client will wait for a blob to finalize after being sent for dispersal.
+* `--eigenda-status-query-retry-interval`: (default: 5s) How often a client will attempt a retry when awaiting network blob finalization.
+* `--eigenda-disable-tls`: (default: false) Whether to disable TLS for grpc communication with disperser.
+* `--eigenda-response-timeout`: (default: 10s) The total amount of time that the client will wait for a response from the EigenDA disperser.
+* `--eigenda-custom-quorum-ids`: (default: []) The quorum IDs to write blobs to using this client. Should not include default quorums 0 or 1.
+* `--eigenda-signer-private-key-hex`: Signer private key in hex encoded format. This key should not be associated with an Ethereum address holding any funds.
+* `--eigenda-put-blob-encoding-version`: The blob encoding version to use when writing blobs from the high level interface.
+* `--eigenda-disable-point-verification-mode`: Point verification mode does an IFFT on data before it is written, and does an FFT on data after it is read. This makes it possible to open points on the KZG commitment to prove that the field elements correspond to the commitment. With this mode disabled, you will need to supply the entire blob to perform a verification that any part of the data matches the KZG commitment.
+* `--eigenda-g1-path`: Directory path to g1.point file
+* `--eigenda-g2-power-of-tau`: Directory path to g2.point.powerOf2 file
+* `--eigenda-cache-path`: Directory path to dump cached SRS tables
 
 ### In-Memory Storage
+
 An ephemeral memory store backend can be used for faster feedback testing when performing rollup integrations. The following cli args can be used to target the feature:
+
 * `--memstore.enabled`: Boolean feature flag
 * `--memstore.expiration`: Duration for which a blob will exist
 
 ## Running Locally
+
 1. Compile binary: `make eigenda-proxy`
 2. Run binary; e.g: `./bin/eigenda-proxy --addr 127.0.0.1 --port 5050 --eigenda-rpc 127.0.0.1:443 --eigenda-status-query-timeout 45m --eigenda-g1-path test/resources/g1.point --eigenda-g2-tau-path test/resources/g2.point.powerOf2 --eigenda-use-tls true`
 
 **Env File**
 An env file can be provided to the binary for runtime process ingestion; e.g:
+
 1. Create env: `cp .env.example .env`
 2. Pass into binary: `ENV_PATH=.env ./bin/eigenda-proxy`
 
 ## Running via Docker
-Container can be built via running `make build-docker`. 
+
+Container can be built via running `make build-docker`.
 
 ## Commitment Schemas
+
 An `EigenDACommitment` layer type has been added that supports verification against its respective pre-images. The commitment is encoded via the following byte array:
+
 ```
             0        1        2        3        4                 N
             |--------|--------|--------|--------|-----------------|
@@ -46,41 +62,48 @@ An `EigenDACommitment` layer type has been added that supports verification agai
 ```
 
 The `raw commitment` for EigenDA is encoding the following certificate and kzg fields:
+
 ```go
 type Cert struct {
-	BatchHeaderHash      []byte
-	BlobIndex            uint32
-	ReferenceBlockNumber uint32
-	QuorumIDs            []uint32
-	BlobCommitment *common.G1Commitment
+ BatchHeaderHash      []byte
+ BlobIndex            uint32
+ ReferenceBlockNumber uint32
+ QuorumIDs            []uint32
+ BlobCommitment *common.G1Commitment
 }
 ```
 
 **NOTE:** Commitments are cryptographically verified against the data fetched from EigenDA for all `/get` calls. The server will respond with status `500` in the event where EigenDA were to lie and provide falsified data thats irrespective of the client provided commitment. This feature isn't flag guarded and is part of standard operation.
 
 ## Testing
+
 Some unit tests have been introduced to assert the correctness of:
+
 * DA Certificate encoding/decoding logic
 * commitment verification logic
 
 Unit tests can be ran via `make test`.
 
-Otherwise E2E tests (`test/e2e_test.go`) exists which asserts that a commitment can be generated when inserting some arbitrary data to the server and can be read using the commitment for a key lookup via the client. These can be ran via `make e2e-test`. Please **note** that this test uses the EigenDA Holesky network which is subject to rate-limiting and slow confirmation times *(i.e, >10 minutes per blob confirmation)*. Please advise EigenDA's [inabox](https://github.com/Layr-Labs/eigenda/tree/master/inabox#readme) if you'd like to spin-up a local DA network for faster iteration testing. 
-
+Otherwise E2E tests (`test/e2e_test.go`) exists which asserts that a commitment can be generated when inserting some arbitrary data to the server and can be read using the commitment for a key lookup via the client. These can be ran via `make e2e-test`. Please **note** that this test uses the EigenDA Holesky network which is subject to rate-limiting and slow confirmation times *(i.e, >10 minutes per blob confirmation)*. Please advise EigenDA's [inabox](https://github.com/Layr-Labs/eigenda/tree/master/inabox#readme) if you'd like to spin-up a local DA network for faster iteration testing.
 
 ## Downloading Mainnet SRS
+
 KZG commitment verification requires constructing the SRS string from the proper trusted setup values (g1, g2, g2.power_of_tau). These values can be downloaded locally using the [operator-setup](https://github.com/Layr-Labs/eigenda-operator-setup) submodule via the following commands.
 
 1. `make submodules`
 2. `make srs`
 
 ## Hardware Requirements
+
 The following specs are recommended for running on a single production server:
+
 * 12 GB SSD (assuming SRS values are stored on instance)
 * 16 GB RAM
 * 1-2 cores CPU
 
 ## Resources
-- [op-stack](https://github.com/ethereum-optimism/optimism)
-- [plasma spec](https://specs.optimism.io/experimental/plasma.html)
-- [eigen da](https://github.com/Layr-Labs/eigenda)
+
+* [op-stack](https://github.com/ethereum-optimism/optimism)
+
+* [plasma spec](https://specs.optimism.io/experimental/plasma.html)
+* [eigen da](https://github.com/Layr-Labs/eigenda)

cmd/server/entrypoint.go

@@ -4,10 +4,10 @@ import (
 	"context"
 	"fmt"
 
-	"github.com/Layr-Labs/eigenda-proxy/eigenda"
 	"github.com/Layr-Labs/eigenda-proxy/metrics"
 	"github.com/Layr-Labs/eigenda-proxy/store"
 	"github.com/Layr-Labs/eigenda-proxy/verify"
+	"github.com/Layr-Labs/eigenda/api/clients"
 	"github.com/ethereum/go-ethereum/log"
 	"github.com/urfave/cli/v2"
 
@@ -30,12 +30,13 @@ func LoadStore(cfg CLIConfig, ctx context.Context, log log.Logger) (proxy.Store,
 		return nil, err
 	}
 
+	client, err := clients.NewEigenDAClient(log, daCfg.ClientConfig)
+	if err != nil {
+		return nil, err
+	}
 	return store.NewEigenDAStore(
 		ctx,
-		eigenda.NewEigenDAClient(
-			log,
-			daCfg,
-		),
+		client,
 		v,
 	)
 }