Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev #37

Draft
wants to merge 65 commits into
base: main
Choose a base branch
from
Draft

Dev #37

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
4adf396
chore: change 'strict monotonic' to 'absolute monotonic' in code come…
Nuhvi Sep 17, 2024
4620b94
feat(js): add PubkyClient.fetch()
Nuhvi Sep 17, 2024
3d1a081
refactor(pubky): separate public API modules from internals
Nuhvi Sep 19, 2024
c4af158
feat(pubky): Resolve and connect to a direct server (localhost)
Nuhvi Sep 19, 2024
bcaebc5
feat(pubky): Resolve through intermediate pkarr packet
Nuhvi Sep 20, 2024
a373605
feat(pubky): endpoints return A and AAAA records
Nuhvi Sep 20, 2024
98aa046
feat(pubky): update pkarr v3
Nuhvi Sep 22, 2024
6544a82
Merge branch 'main' of https://github.com/pubky/pubky into feat/pkarr…
Nuhvi Sep 22, 2024
966ccc1
refactor(pubky): simplify session management using 'fetch-cookie' dep…
Nuhvi Sep 22, 2024
1f73b72
refactor(pubky): remove unused session_cookies in PubkyClient
Nuhvi Sep 22, 2024
fff7b6a
feat(js): use pkarr relay Client instead of inline impl
Nuhvi Sep 22, 2024
5aa53f4
refactor(pubky): remove unused pkarr_relays from PubkyClient
Nuhvi Sep 22, 2024
ad9db81
refactor(pubky): replace pkarr_publish/pkarr_resolve with pkarr.publi…
Nuhvi Sep 22, 2024
9fd8501
refactor(pubky): simplify internals mod after using pkarr client
Nuhvi Sep 22, 2024
2ae5435
feat(pubky): use resolve_endpoint from pkarr
Nuhvi Sep 25, 2024
901627d
feat: update pkarr to v3
Nuhvi Sep 30, 2024
72f8684
Merge branch 'main' of https://github.com/pubky/pubky into dev
Nuhvi Sep 30, 2024
1d179c7
Merge branch 'dev' of https://github.com/pubky/pubky into feat/pkarr-…
Nuhvi Oct 1, 2024
6f2a8ce
Merge pull request #36 from pubky/feat/pkarr-reqwest
Nuhvi Oct 1, 2024
b2eaf31
chore(js): publish 0.2.0
Nuhvi Nov 3, 2024
b8fc9b5
chore(js): publish 0.2.0
Nuhvi Nov 3, 2024
f82b301
Merge branch 'main' into dev
Nuhvi Nov 14, 2024
1b7d83a
fix: fix after merging main
Nuhvi Nov 14, 2024
af390e8
feat(pubky): PubkyClient::builder().build() return result
Nuhvi Nov 14, 2024
5972b48
feat(pubky): rename PubkyClient to Client
Nuhvi Nov 14, 2024
d336709
feat(pubky): use Pkarr v3 endpoint resolvers in Client::request
Nuhvi Nov 14, 2024
05cad66
feat(pubky): support GET /pub/*path for native urls
Nuhvi Nov 21, 2024
6109df3
feat(pubky): update HEAD, GET, PUT, and DELETE for pub/ endpoints to …
Nuhvi Nov 24, 2024
1993a73
feat(pubky): add custom cookie jar .. requires https
Nuhvi Nov 25, 2024
bb2799e
fix: clippy
Nuhvi Nov 25, 2024
c9ae345
feat(pubky): add TLS support for homeserver
Nuhvi Nov 26, 2024
e6f7fb3
feat(pubky): add an icann reqwest client
Nuhvi Nov 26, 2024
c088388
fix(pubky): fix signup url
Nuhvi Nov 26, 2024
9b5a4f9
feat(homeserver): separate core to simplify testing
Nuhvi Nov 26, 2024
6f64831
feat(pubky): pass all unit tests after using pubky in host
Nuhvi Nov 26, 2024
b1bef6a
wip(js): migrate wasm bindings to support pubky in host
Nuhvi Nov 26, 2024
6280499
feat(pubky): update pkarr and use anyhow
Nuhvi Dec 3, 2024
6a41b49
feat(homeserver): mark unsafe methods caused by LMDB
Nuhvi Dec 3, 2024
8c40cd2
refactor(homeserver): separate the core and io modules
Nuhvi Dec 3, 2024
1348a18
feat(homeserver): remove pkarr endpoints in HomeserverCore
Nuhvi Dec 3, 2024
da2cc2a
feat(homeserver): add http server as well as https
Nuhvi Dec 3, 2024
1a30f7d
chore: update dependencies
Nuhvi Dec 11, 2024
3ebdeff
feat(homeserver): use Pkarr Relay in testnet and new Config simplific…
Nuhvi Dec 14, 2024
8a799a2
feat(common): add granular crypto errors
Nuhvi Dec 14, 2024
453aa40
fix(pubky): fix authz test with in process http relay
Nuhvi Dec 14, 2024
d0f2ee7
feat(pubky): remove crate::errors.rs and use anyhow
Nuhvi Dec 14, 2024
c97e63a
feat(js): update error handling after removing crate::error
Nuhvi Dec 14, 2024
6262bb7
feat(homeserver,js): connect to testnet from js
Nuhvi Dec 15, 2024
1c0ff6c
feat(js): add logging in wasm and setLogLevel function
Nuhvi Dec 15, 2024
4c6bc9d
test(homeserver): add domain:localhost in unit tests
Nuhvi Dec 15, 2024
6e886f0
fix(js): replace pubky:// with https:// in transform_url
Nuhvi Dec 16, 2024
b4ef0ca
feat(js): basic auth with pubky-host header
Nuhvi Dec 16, 2024
de7d34e
feat(http-relay): add http relay crate to help with local testing
Nuhvi Dec 17, 2024
992607a
feat(homeserver): add http relay in homeserver testnet for js testing
Nuhvi Dec 17, 2024
f42797a
feat(js): pass all unit tests
Nuhvi Dec 17, 2024
f840ce3
Merge branch 'main' into dev
Nuhvi Dec 18, 2024
daa06b6
fix(js): send correct pubky-host header
Nuhvi Dec 19, 2024
0c1326c
feat(homeserver): remove public key in path endpoints
Nuhvi Dec 19, 2024
3cfe656
feat(homeserver): replace session_id with the first 8 bytes of a key
Nuhvi Dec 19, 2024
738bff1
feat(homeserver): add PubkyHost and Authz layers
Nuhvi Dec 20, 2024
fa2a7d2
feat(homeserver): use full public key as cookie name
Nuhvi Dec 20, 2024
96be581
feat(pubky): add handle_http_error! macro
Nuhvi Dec 20, 2024
bcb8d39
feat(homeserver): remove EntryPath extractor and add Authz and PubkyH…
Nuhvi Dec 22, 2024
4ccb060
fix(homeserver): always get sessions from Cookies instead of manually…
Nuhvi Dec 22, 2024
ccbd3ff
feat(js): pass all unit tests
Nuhvi Dec 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
937 changes: 783 additions & 154 deletions Cargo.lock

Large diffs are not rendered by default.

5 changes: 3 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,16 @@ members = [
"pubky",
"pubky-*",

"http-relay",

"examples"
]

# See: https://github.com/rust-lang/rust/issues/90148#issuecomment-949194352
resolver = "2"

[workspace.dependencies]
pkarr = { version = "2.0.0", git = "https://github.com/Pubky/pkarr", branch = "serde", package = "pkarr", features = ["async", "serde"] }
serde = { version = "^1.0.209", features = ["derive"] }
pkarr = { git = "https://github.com/Pubky/pkarr", branch = "v3", package = "pkarr" }

[profile.release]
lto = true
Expand Down
11 changes: 6 additions & 5 deletions examples/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,13 @@ name = "request"
path = "./request/main.rs"

[dependencies]
anyhow = "1.0.86"
anyhow = "1.0.94"
base64 = "0.22.1"
clap = { version = "4.5.16", features = ["derive"] }
clap = { version = "4.5.23", features = ["derive"] }
pubky = { path = "../pubky" }
pubky-common = { version = "0.1.0", path = "../pubky-common" }
reqwest = "0.12.8"
reqwest = "0.12.9"
rpassword = "7.3.1"
tokio = { version = "1.40.0", features = ["macros", "rt-multi-thread"] }
url = "2.5.2"
tokio = { version = "1.42.0", features = ["macros", "rt-multi-thread"] }
tracing-subscriber = { version = "0.3.19", features = ["env-filter"] }
url = "2.5.4"
4 changes: 2 additions & 2 deletions examples/authn/signup.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use anyhow::Result;
use clap::Parser;
use pubky::PubkyClient;
use pubky::Client;
use std::path::PathBuf;

use pubky_common::crypto::PublicKey;
Expand All @@ -24,7 +24,7 @@ async fn main() -> Result<()> {

let homeserver = cli.homeserver;

let client = PubkyClient::builder().build();
let client = Client::new()?;

println!("Enter your recovery_file's passphrase to signup:");
let passphrase = rpassword::read_password()?;
Expand Down
8 changes: 4 additions & 4 deletions examples/authz/3rd-party-app/src/pubky-auth-widget.js
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@ export class PubkyAuthWidget extends LitElement {
this.testnet = false;
this.open = false;

/** @type {import("@synonymdev/pubky").PubkyClient} */
this.pubkyClient = new window.pubky.PubkyClient();
/** @type {import("@synonymdev/pubky").Client} */
this.pubkyClient = new window.pubky.Client();

this.caps = this.caps || ""
}
Expand All @@ -74,9 +74,9 @@ export class PubkyAuthWidget extends LitElement {
console.debug("Switching testnet");

if (this.testnet) {
this.pubkyClient = window.pubky.PubkyClient.testnet()
this.pubkyClient = window.pubky.Client.testnet()
} else {
this.pubkyClient = new window.pubky.PubkyClient();
this.pubkyClient = new window.pubky.Client();
}

console.debug("Pkarr Relays: " + this.pubkyClient.getPkarrRelays())
Expand Down
6 changes: 3 additions & 3 deletions examples/authz/authenticator.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use anyhow::Result;
use clap::Parser;
use pubky::PubkyClient;
use pubky::Client;
use std::path::PathBuf;
use url::Url;

Expand Down Expand Up @@ -66,7 +66,7 @@ async fn main() -> Result<()> {
println!("PublicKey: {}", keypair.public_key());

let client = if cli.testnet.unwrap_or_default() {
let client = PubkyClient::testnet();
let client = Client::testnet()?;

// For the purposes of this demo, we need to make sure
// the user has an account on the local homeserver.
Expand All @@ -78,7 +78,7 @@ async fn main() -> Result<()> {

client
} else {
PubkyClient::builder().build()
Client::new()?
};

println!("Sending AuthToken to the 3rd party app...");
Expand Down
36 changes: 23 additions & 13 deletions examples/request/main.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
use std::env;

use anyhow::Result;
use clap::Parser;
use reqwest::Method;
use url::Url;

use pubky::PubkyClient;
use pubky::Client;

#[derive(Parser, Debug)]
#[command(version, about, long_about = None)]
Expand All @@ -16,23 +18,31 @@ struct Cli {

#[tokio::main]
async fn main() -> Result<()> {
let cli = Cli::parse();
let args = Cli::parse();

let client = PubkyClient::builder().build();
tracing_subscriber::fmt()
.with_env_filter(env::var("TRACING").unwrap_or("info".to_string()))
.init();

match cli.url.scheme() {
"https" => {
unimplemented!();
}
"pubky" => {
let response = client.get(cli.url).await.unwrap();
let client = Client::new()?;

println!("Got a response: \n {:?}", response);
}
_ => {
panic!("Only https:// and pubky:// URL schemes are supported")
// Build the request
let response = client.get(args.url).send().await?;

println!("< Response:");
println!("< {:?} {}", response.version(), response.status());
for (name, value) in response.headers() {
if let Ok(v) = value.to_str() {
println!("< {name}: {v}");
}
}

let bytes = response.bytes().await?;

match String::from_utf8(bytes.to_vec()) {
Ok(string) => println!("<\n{}", string),
Err(_) => println!("<\n{:?}", bytes),
}

Ok(())
}
13 changes: 13 additions & 0 deletions http-relay/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[package]
name = "http-relay"
version = "0.1.0"
edition = "2021"

[dependencies]
anyhow = "1.0.94"
axum = "0.7.9"
axum-server = "0.7.1"
futures-util = "0.3.31"
tokio = { version = "1.42.0", features = ["full"] }
tracing = "0.1.41"
url = "2.5.4"
7 changes: 7 additions & 0 deletions http-relay/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# HTTP Relay

A Rust implementation of _some_ of [Http relay spec](https://httprelay.io/).

Normally you are better off running the [reference implementation's binary](https://httprelay.io/download/).

This implementation, for the time being is meant for having a convenient library to be used in unit tests, and testnets in Pubky.
167 changes: 167 additions & 0 deletions http-relay/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
use std::{
collections::HashMap,
net::{SocketAddr, TcpListener},
sync::{Arc, Mutex},
};

use anyhow::Result;

use axum::{
body::{Body, Bytes},
extract::{Path, State},
response::IntoResponse,
routing::get,
Router,
};
use axum_server::Handle;
use tokio::sync::Notify;

use futures_util::{stream::StreamExt, TryFutureExt};
use url::Url;

// Shared state to store GET requests and their notifications
type SharedState = Arc<Mutex<HashMap<String, (Vec<u8>, Arc<Notify>)>>>;

#[derive(Debug, Default)]
pub struct Config {
pub http_port: u16,
}

#[derive(Debug, Default)]
pub struct HttpRelayBuilder(Config);

impl HttpRelayBuilder {
/// Configure the port used for HTTP server.
pub fn http_port(mut self, port: u16) -> Self {
self.0.http_port = port;

self
}

pub async fn build(self) -> Result<HttpRelay> {
HttpRelay::start(self.0).await
}
}

pub struct HttpRelay {
pub(crate) http_handle: Handle,

http_address: SocketAddr,
}

impl HttpRelay {
pub fn builder() -> HttpRelayBuilder {
HttpRelayBuilder::default()
}

pub async fn start(config: Config) -> Result<Self> {
let shared_state: SharedState = Arc::new(Mutex::new(HashMap::new()));

let app = Router::new()
.route("/link/:id", get(link::get).post(link::post))
.with_state(shared_state);

let http_handle = Handle::new();

let http_listener = TcpListener::bind(SocketAddr::from(([0, 0, 0, 0], config.http_port)))?;
let http_address = http_listener.local_addr()?;

tokio::spawn(
axum_server::from_tcp(http_listener)
.handle(http_handle.clone())
.serve(app.into_make_service())
.map_err(|error| tracing::error!(?error, "HttpRelay http server error")),
);

Ok(Self {
http_handle,
http_address,
})
}

pub fn http_address(&self) -> SocketAddr {
self.http_address
}

/// Returns the localhost Url of this server.
pub fn local_url(&self) -> Url {
Url::parse(&format!("http://localhost:{}", self.http_address.port()))
.expect("local_url should be formatted fine")
}

/// Returns the localhost URL of Link endpoints
pub fn local_link_url(&self) -> Url {
let mut url = self.local_url();

let mut segments = url
.path_segments_mut()
.expect("HttpRelay::local_link_url path_segments_mut");

segments.push("link");

drop(segments);

url
}

pub fn shutdown(&self) {
self.http_handle.shutdown();
}
}

mod link {
use super::*;

pub async fn get(
Path(id): Path<String>,
State(state): State<SharedState>,
) -> impl IntoResponse {
// Create a notification for this ID
let notify = Arc::new(Notify::new());

{
let mut map = state.lock().unwrap();

// Store the notification and return it when POST arrives
map.entry(id.clone())
.or_insert_with(|| (vec![], notify.clone()));
}

notify.notified().await;

// Respond with the data stored for this ID
let map = state.lock().unwrap();
if let Some((data, _)) = map.get(&id) {
Bytes::from(data.clone()).into_response()
} else {
(axum::http::StatusCode::NOT_FOUND, "Not Found").into_response()
}
}

pub async fn post(
Path(id): Path<String>,
State(state): State<SharedState>,
body: Body,
) -> impl IntoResponse {
// Aggregate the body into bytes
let mut stream = body.into_data_stream();
let mut bytes = vec![];
while let Some(next) = stream.next().await {
let chunk = next.map_err(|e| e.to_string()).unwrap();
bytes.extend_from_slice(&chunk);
}

// Notify any waiting GET request for this ID
let mut map = state.lock().unwrap();
if let Some((storage, notify)) = map.get_mut(&id) {
*storage = bytes;
notify.notify_one();
Ok(())
} else {
Err((
axum::http::StatusCode::NOT_FOUND,
"No waiting GET request for this ID",
))
}
}
}
18 changes: 9 additions & 9 deletions pubky-common/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,22 +9,22 @@ repository = "https://github.com/pubky/pubky-core"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
base32 = "0.5.0"
blake3 = "1.5.1"
base32 = "0.5.1"
blake3 = "1.5.5"
ed25519-dalek = { version = "2.1.1", features = ["serde"] }
once_cell = "1.19.0"
once_cell = "1.20.2"
rand = "0.8.5"
thiserror = "1.0.60"
postcard = { version = "1.0.8", features = ["alloc"] }
thiserror = "2.0.6"
postcard = { version = "1.1.1", features = ["alloc"] }
crypto_secretbox = { version = "0.1.1", features = ["std"] }
argon2 = { version = "0.5.3", features = ["std"] }

pubky-timestamp = { version = "0.2.0", features = ["full"] }
serde = { version = "1.0.213", features = ["derive"] }
pkarr = { version = "2.2.1-alpha.2", features = ["serde"] }
serde = { version = "1.0.216", features = ["derive"] }
pkarr = { workspace = true }

[target.'cfg(target_arch = "wasm32")'.dependencies]
js-sys = "0.3.69"
js-sys = "0.3.76"

[dev-dependencies]
postcard = "1.0.8"
postcard = "1.1.1"
Loading
Loading