Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SDK client library #49

Merged
merged 58 commits into from
Jan 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
6d8b693
Add a `BROWSER_BUILD` env variable to the build.
sangaline Jan 12, 2024
0e02ba5
Enable rollup tree shaking in the browser build.
sangaline Jan 12, 2024
f43b50f
Disallow unchecked index access in typescript.
sangaline Jan 12, 2024
4ec78ed
Factor the config path out into the lib.
sangaline Jan 12, 2024
dac5a52
Make the logger use `console` in browser builds.
sangaline Jan 12, 2024
1e9a5ad
Add the start of an API client.
sangaline Jan 12, 2024
62d6da4
Authorize the browser client in tests.
sangaline Jan 12, 2024
3d35cac
Add something closer to real tests for circuit list.
sangaline Jan 12, 2024
214dd49
Record fixtures for the new tests.
sangaline Jan 12, 2024
170c5cf
Check that the API key in browser matches the node value.
sangaline Jan 12, 2024
73c2197
Increase the test timeouts.
sangaline Jan 12, 2024
3e772ad
Make pino work in a browser.
sangaline Jan 12, 2024
79b3562
Add the ability to control the log levels.
sangaline Jan 12, 2024
d0f57f2
Clean up type vs non-type exports.
sangaline Jan 13, 2024
1fa67ed
Add tar-js and gzip-js as dependencies.
sangaline Jan 13, 2024
9813661
Switch to `formdata-node` for formdata handling.
sangaline Jan 13, 2024
d711fad
Implement a rough cut of creating circuits.
sangaline Jan 13, 2024
6997082
Write tests to create circuits.
sangaline Jan 13, 2024
ab06fa6
Take a stab at using an http proxy instead of nock-puppeteer.
sangaline Jan 13, 2024
894e184
Get the http proxy working.
sangaline Jan 14, 2024
0e1719d
Add the missing modules.
sangaline Jan 14, 2024
7356d42
Add patch-package and http-mitm-proxy.
sangaline Jan 14, 2024
26c1e36
Add some progress on using a mitm proxy.
sangaline Jan 14, 2024
ffac439
Get a nock filtering solution to the form boundaries.
sangaline Jan 14, 2024
0ede068
Fix the robots.txt test timeout (was ms not s).
sangaline Jan 14, 2024
c824c8c
Eliminate the polling interval during tests.
sangaline Jan 14, 2024
928e33e
Remove some debug logging.
sangaline Jan 14, 2024
983a426
Update the test data and fixtures.
sangaline Jan 14, 2024
a309830
Add back the mitm proxy.
sangaline Jan 14, 2024
c8c91de
Fix the content-length mismatch issue.
sangaline Jan 14, 2024
0b5cce2
Fix the robots.txt test (was fetching a favicon with navigation).
sangaline Jan 14, 2024
b9a3fc3
Update the fixtures.
sangaline Jan 14, 2024
b14877d
Add some debug logging.
sangaline Jan 14, 2024
1ea93b7
End the nock recording earlier.
sangaline Jan 14, 2024
bf47e90
Re-record fixtures on linux.
sangaline Jan 14, 2024
e1e7a8c
Try running a testing matrix.
sangaline Jan 14, 2024
34ab020
Fix the matrix.
sangaline Jan 14, 2024
6184316
Limit the matrix to 18 and 20.
sangaline Jan 14, 2024
caf6d08
Revert the test matrix.
sangaline Jan 14, 2024
82515dd
Do a deep gzip comparison in form payloads.
sangaline Jan 14, 2024
1459e98
Update the fixtures.
sangaline Jan 14, 2024
1129dfb
Refactor the form matching to be cleaner.
sangaline Jan 14, 2024
8ba848a
Fix typing errors.
sangaline Jan 14, 2024
4d6214d
Add a newline to the circom file.
sangaline Jan 14, 2024
f581b80
Rename `listCircuits` to `getAllCircuits`.
sangaline Jan 15, 2024
b57f07f
Add a method to get all proofs.
sangaline Jan 15, 2024
1a2d3dc
Add a prove circuit method to the client.
sangaline Jan 15, 2024
ee07658
Update the fixtures.
sangaline Jan 15, 2024
4629b5f
Add a get all circuit proofs method.
sangaline Jan 15, 2024
18063cc
Add a `getCircuit` method to the client.
sangaline Jan 15, 2024
992071d
Fix the tests by using unique tags.
sangaline Jan 15, 2024
1c72573
Update fixtures.
sangaline Jan 15, 2024
7513757
Add a getProof method to the client.
sangaline Jan 15, 2024
6d378bd
Update fixtures.
sangaline Jan 15, 2024
0f2e2c6
Increase the test timeouts.
sangaline Jan 15, 2024
a278670
Add some fluffier standins for the docs.
sangaline Jan 15, 2024
0825f42
Exit the browser tests explicitly to work around hanging.
sangaline Jan 15, 2024
544040f
Move the hack to the correct location.
sangaline Jan 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
.npmrc
.http-mitm-proxy/
/dist/
/node_modules/
4 changes: 2 additions & 2 deletions ava.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ export default {
timeout: ["dryrun", "record", "update", "wild"].includes(
process.env.NOCK_BACK_MODE ?? "lockdown",
)
? "60s"
: "10s",
? "240s"
: "60s",
// Use child processes instead of threads because notch isn't thread safe.
workerThreads: false,
};
28 changes: 24 additions & 4 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,12 @@
"cryptography",
"crypto"
],
"files": ["dist/", "sindri-manifest.json", "src/", "templates/"],
"files": [
"dist/",
"sindri-manifest.json",
"src/",
"templates/"
],
"main": "dist/lib/index.js",
"module": "dist/lib/index.mjs",
"bin": {
Expand Down Expand Up @@ -52,7 +57,9 @@
"test:fast": "ava",
"test:record": "NOCK_BACK_MODE=update yarn test",
"test:watch": "NODE_ENV=development NOCK_BACK_MODE=dryrun nodemon --watch src/ --watch test/ --ext js,cjs,mjs,ts,cts,mts --exec 'tsup --silent && ava'",
"type-check": "tsc --noEmit"
"type-check": "tsc --noEmit",
"/***** Hooks *****/": "",
"postinstall": "patch-package"
},
"repository": {
"type": "git",
Expand All @@ -68,39 +75,52 @@
"axios": "^1.6.2",
"commander": "^11.1.0",
"env-paths": "^2.2.1",
"form-data": "^4.0.0",
"formdata-node": "^6.0.3",
"gzip-js": "^0.3.2",
"ignore-walk": "^6.0.4",
"jsonschema": "^1.4.1",
"lodash": "^4.17.21",
"nunjucks": "^3.2.4",
"patch-package": "^8.0.0",
"pino": "^8.16.2",
"pino-pretty": "^10.2.3",
"postinstall-postinstall": "^2.1.0",
"rc": "^1.2.8",
"tar": "^6.2.0",
"tar-js": "^0.3.0",
"zod": "^3.22.4"
},
"devDependencies": {
"@ava/typescript": "^4.1.0",
"@commander-js/extra-typings": "^11.1.0",
"@tsconfig/node18": "^18.2.2",
"@types/get-port": "^4.2.0",
"@types/gzip-js": "^0.3.5",
"@types/ignore-walk": "^4.0.3",
"@types/lodash": "^4.14.202",
"@types/node": "^20.9.1",
"@types/nunjucks": "^3.2.6",
"@types/proxy": "^1.0.4",
"@types/tar": "^6.1.10",
"@types/tar-js": "^0.3.5",
"@typescript-eslint/eslint-plugin": "^6.11.0",
"@typescript-eslint/parser": "^6.11.0",
"ava": "^6.0.1",
"esbuild": "^0.19.11",
"eslint": "^8.53.0",
"eslint-config-prettier": "^9.0.0",
"eslint-plugin-prettier": "^5.0.1",
"get-port": "^7.0.0",
"http-mitm-proxy": "^1.1.0",
"make-synchronous": "^1.0.0",
"mockdate": "^3.0.5",
"nock": "^13.4.0",
"nock-puppeteer": "^14.4.1",
"nodemon": "^3.0.2",
"openapi-typescript-codegen": "^0.25.0",
"parse-multipart-data": "^1.5.0",
"prettier": "^3.1.0",
"puppeteer": "^21.7.0",
"rollup": "^4.9.5",
"tsup": "^7.3.0",
"tsx": "^4.7.0",
"type-fest": "^4.8.2",
Expand Down
13 changes: 13 additions & 0 deletions patches/http-mitm-proxy+1.1.0.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
diff --git a/node_modules/http-mitm-proxy/dist/lib/proxy.js b/node_modules/http-mitm-proxy/dist/lib/proxy.js
index e896595..8623f9f 100644
--- a/node_modules/http-mitm-proxy/dist/lib/proxy.js
+++ b/node_modules/http-mitm-proxy/dist/lib/proxy.js
@@ -361,7 +361,7 @@ class Proxy {
function makeConnection(port) {
const conn = net_1.default.connect({
port,
- host: "0.0.0.0",
+ host: "localhost",
allowHalfOpen: true,
}, () => {
const connectKey = `${conn.localPort}:${conn.remotePort}`;
105 changes: 2 additions & 103 deletions src/cli/config.ts
Original file line number Diff line number Diff line change
@@ -1,108 +1,7 @@
import fs from "fs";
import path from "path";

import { Command } from "@commander-js/extra-typings";
import envPaths from "env-paths";
import { cloneDeep, merge } from "lodash";
import { z } from "zod";

import { logger, print } from "cli/logging";
import { OpenAPI } from "lib/api";

const paths = envPaths("sindri", {
suffix: "",
});
const configPath = path.join(paths.config, "sindri.conf.json");

const ConfigSchema = z.object({
auth: z
.nullable(
z.object({
apiKey: z.string(),
apiKeyId: z.string(),
apiKeyName: z.string(),
baseUrl: z.string().url(),
teamId: z.number(),
teamSlug: z.string(),
}),
)
.default(null),
});

type ConfigSchema = z.infer<typeof ConfigSchema>;

const defaultConfig: ConfigSchema = ConfigSchema.parse({});

const loadConfig = (): ConfigSchema => {
if (fs.existsSync(configPath)) {
logger.debug(`Loading config from "${configPath}".`);
try {
const configFileContents: string = fs.readFileSync(configPath, {
encoding: "utf-8",
});
const loadedConfig = ConfigSchema.parse(JSON.parse(configFileContents));
logger.debug("Config loaded successfully.");
return loadedConfig;
} catch (error) {
logger.warn(
`The config schema in "${configPath}" is invalid and will not be used.\n` +
`To remove it and start fresh, run:\n rm ${configPath}`,
);
logger.debug(error);
}
}
logger.debug(
`Config file "${configPath}" does not exist, initializing default config.`,
);
return cloneDeep(defaultConfig);
};

export class Config {
protected _config!: ConfigSchema;
protected static instance: Config;

constructor() {
if (!Config.instance) {
this._config = loadConfig();
Config.instance = this;
// Prepare API the client with the loaded credentials.
if (this._config.auth) {
OpenAPI.BASE = this._config.auth.baseUrl;
OpenAPI.TOKEN = this._config.auth.apiKey;
}
}
return Config.instance;
}

get auth(): ConfigSchema["auth"] {
return cloneDeep(this._config.auth);
}

get config(): ConfigSchema {
return cloneDeep(this._config);
}

update(configData: Partial<ConfigSchema>) {
// Merge and validate the configs.
logger.debug("Merging in config update:");
logger.debug(configData);
const newConfig: ConfigSchema = cloneDeep(this._config);
merge(newConfig, configData);
this._config = ConfigSchema.parse(newConfig);

// Create the directory if it doesn't exist.
const directory = path.dirname(configPath);
if (!fs.existsSync(directory)) {
fs.mkdirSync(directory, { recursive: true });
}

// Write out the new config.
logger.debug(`Writing merged config to "${configPath}":`, this._config);
fs.writeFileSync(configPath, JSON.stringify(this._config, null, 2), {
encoding: "utf-8",
});
}
}
import { Config } from "lib/config";
import { print } from "lib/logging";

export const configListCommand = new Command()
.name("list")
Expand Down
41 changes: 21 additions & 20 deletions src/cli/deploy.ts
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
import { Blob } from "buffer";
import { existsSync, readFileSync } from "fs";
import path from "path";
import process from "process";

import { Command } from "@commander-js/extra-typings";
import FormData from "form-data";
import { FormData } from "formdata-node";
import walk from "ignore-walk";
import tar from "tar";

import { Config } from "cli/config";
import { logger } from "cli/logging";
import { findFileUpwards } from "cli/utils";
import { ApiError, CircuitsService, CircuitStatus } from "lib/api";
import { Config } from "lib/config";
import { logger } from "lib/logging";

export const deployCommand = new Command()
.name("deploy")
Expand Down Expand Up @@ -103,7 +104,7 @@ export const deployCommand = new Command()
// Always exclude `.git` subdirectories.
!/(^|\/)\.git(\/|$)/.test(file),
);
// Alows include the `sindri.json` file.
// Always include the `sindri.json` file.
const sindriJsonFilename = path.basename(sindriJsonPath);
if (!files.includes(sindriJsonFilename)) {
files.push(sindriJsonFilename);
Expand All @@ -115,23 +116,23 @@ export const deployCommand = new Command()
);
formData.append(
"files",
tar
.c(
{
gzip: true,
onwarn: (code: string, message: string) => {
logger.warn(`While creating tarball: ${code} - ${message}`);
new Blob([
tar
.c(
{
gzip: true,
onwarn: (code: string, message: string) => {
logger.warn(`While creating tarball: ${code} - ${message}`);
},
prefix: `${circuitName}/`,
sync: true,
},
prefix: `${circuitName}/`,
sync: true,
},
files,
)
// @ts-expect-error - @types/tar doesn't handle the `sync` option correctly.
.read(),
{
filename: tarballFilename,
},
files,
)
// @ts-expect-error - @types/tar doesn't handle the `sync` option correctly.
.read(),
]),
tarballFilename,
);

// Attach the tags to the form data.
Expand Down
5 changes: 3 additions & 2 deletions src/cli/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,16 @@ import { argv, exit } from "process";

import { Command } from "@commander-js/extra-typings";

import { Config, configCommand } from "cli/config";
import { configCommand } from "cli/config";
import { initCommand } from "cli/init";
import { deployCommand } from "cli/deploy";
import { lintCommand } from "cli/lint";
import { logger } from "cli/logging";
import { loginCommand } from "cli/login";
import { logoutCommand } from "cli/logout";
import { whoamiCommand } from "cli/whoami";
import { loadPackageJson } from "cli/utils";
import { Config } from "lib/config";
import { logger } from "lib/logging";

export const program = new Command()
.name("sindri")
Expand Down
2 changes: 1 addition & 1 deletion src/cli/init.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ import process from "process";
import { Command } from "@commander-js/extra-typings";
import { confirm, input, select } from "@inquirer/prompts";

import { logger } from "cli/logging";
import { scaffoldDirectory } from "cli/utils";
import { logger } from "lib/logging";

export const initCommand = new Command()
.name("init")
Expand Down
2 changes: 1 addition & 1 deletion src/cli/lint.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ import { Command } from "@commander-js/extra-typings";
import type { Schema } from "jsonschema";
import { Validator as JsonValidator } from "jsonschema";

import { logger } from "cli/logging";
import { findFileUpwards, loadSindriManifestJsonSchema } from "cli/utils";
import { logger } from "lib/logging";

export const lintCommand = new Command()
.name("lint")
Expand Down
14 changes: 0 additions & 14 deletions src/cli/logging.ts

This file was deleted.

4 changes: 2 additions & 2 deletions src/cli/login.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,15 @@ import {
select,
} from "@inquirer/prompts";

import { Config } from "cli/config";
import { logger } from "cli/logging";
import {
ApiError,
AuthorizationService,
InternalService,
OpenAPI,
TokenService,
} from "lib/api";
import { Config } from "lib/config";
import { logger } from "lib/logging";

export const loginCommand = new Command()
.name("login")
Expand Down
4 changes: 2 additions & 2 deletions src/cli/logout.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import { Command } from "@commander-js/extra-typings";
import { confirm } from "@inquirer/prompts";

import { Config } from "cli/config";
import { logger } from "cli/logging";
import { AuthorizationService } from "lib/api";
import { Config } from "lib/config";
import { logger } from "lib/logging";

export const logoutCommand = new Command()
.name("logout")
Expand Down
2 changes: 1 addition & 1 deletion src/cli/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import type { Schema } from "jsonschema";
import nunjucks from "nunjucks";
import type { PackageJson } from "type-fest";

import { logger } from "cli/logging";
import { logger } from "lib/logging";

const currentFilePath = fileURLToPath(import.meta.url);
const currentDirectoryPath = path.dirname(currentFilePath);
Expand Down
4 changes: 2 additions & 2 deletions src/cli/whoami.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ import process from "process";

import { Command } from "@commander-js/extra-typings";

import { Config } from "cli/config";
import { logger, print } from "cli/logging";
import { ApiError, InternalService } from "lib/api";
import { Config } from "lib/config";
import { logger, print } from "lib/logging";

export const whoamiCommand = new Command()
.name("whoami")
Expand Down
Loading