Skip to content

The Nexus between Pubky homeservers and Pubky-App social features.

License

Notifications You must be signed in to change notification settings

pubky/pubky-nexus

Folders and files

NameName
Last commit message
Last commit date

Latest commit

0efe196 Β· Jan 14, 2025
Nov 5, 2024
Jan 14, 2025
Dec 5, 2024
Nov 21, 2024
Jan 14, 2025
Jan 14, 2025
Jan 14, 2025
Sep 23, 2024
Oct 14, 2024
Jul 22, 2024
Dec 2, 2024
Jan 13, 2025
Jan 13, 2025
Nov 5, 2024
Oct 31, 2024
Nov 12, 2024
Oct 8, 2024
Sep 4, 2024

Repository files navigation

Integration Tests

Pubky Nexus

Pubky Nexus is the central bridge connecting Pubky homeservers with Pubky-App’s social clients. By aggregating events from homeservers into a rich social graph, Nexus transforms decentralized interactions into a high-performance, fully featured social-media-like API. It's designed to support Social-Semantic-Graph (SSG) inference, and more.

🌟 Key Features

  • Real-time Social Graph Aggregation: Nexus ingests events from multiple Pubky homeservers, generating a structured social graph in real time.
  • Full-Content Indexing: Nexus serves content directly, improving latency and user experience. Clients do not need to locate homeservers to retrieve content unless they wish to perform content attestation. We also envision a light-weight Nexus mode that merely point clients to homeserver locations using pubky uris.
  • High Performance & Scalability: Built in Rust, Nexus is optimized for speed and efficiency, handling complex social queries across distributed systems with minimal latency.
  • Powerful Social Semantic Graph: Nexus supports SSG-based interactions, fostering secure and trusted connections between users.
  • Graph-Enhanced Search & Recommendations: Nexus leverages Neo4j to offer deep insights, relationships, and recommendations based on the social graph.
  • Flexible Caching Layer: A Redis cache accelerates access to common queries and minimizes database load, ensuring a smooth user experience. Most requests can be served in less than 1 ms at constant time complexity with respect number of users.
  • Rich Observability: Easily explore the indexed data using Redis Insight and visualize the graph database with Neo4j Browser.

🌐 Accessing the API

⚠️ Warning: The API is currently unstable. We are using the /v0 route prefix while the API undergoes active development and changes. Expect potential breaking changes as we work toward stability.

Nexus provides a REST API, accessible via Swagger UI:

You can explore available endpoints, test queries, and view schema definitions directly within Swagger.

πŸ—οΈ Architecture Overview

Nexus is composed of several core components:

  • service.rs: The REST API server for handling client requests, querying databases, and returning responses to the Pubky-App frontend.
  • watcher.rs: The event aggregator that listens to homeserver events, translating them into social graph updates within the Nexus databases.
  • lib.rs: A library crate containing common functionalities shared by service and watcher, including database connectors, models, and queries.

Data Flow

pubky-nexus-arch

  1. Event Ingestion: The watcher ingests events from Pubky homeservers and indexes them into our social graph.
  2. Indexing and Caching: Relationships, common queries, and graph data are cached in Redis for high-speed access. Complex queries use Neo4j.
  3. API Responses: The service server reads from these indexes to respond to client queries efficiently.

pubky-nexus-graph

Nexus graph schema.

βš™οΈ Setting Up the Development Environment

To get started with Nexus, first set up the required databases: Neo4j and Redis.

1. Configure Databases

  1. Clone the repository and navigate to the project directory.

  2. Copy the environment template and set up the Docker environment:

    cd docker
    cp .env-sample .env
    docker-compose up -d
  3. Populate the Neo4j database with initial data:

    docker exec neo4j bash /db-graph/run-queries.sh

Once the Neo4j graph database is seeded with data, the next step is to populate the Redis database by running the nexus-service

If the Redis cache is empty, the nexus-service will handle it automatically. If not follow the steps of warning section

  1. Run the Nexus service:

    cargo run
  2. Access Redis and Neo4j UIs:

πŸš€ Contributing

To contribute to Nexus, follow these steps:

  1. Fork the Repository and create a feature branch.
  2. Write Tests: Ensure new features and changes are tested and benchmarked.
  3. Submit a Pull Request and provide a description of the changes.

Running Tests

To run all tests:

cargo test

To test specific modules or features:

cargo test watcher:users
cargo test test_homeserver_user_event

Benchmarking:

cargo bench --bench user get_user_view_by_id

⚠️ Troubleshooting

If tests or the development environment seem out of sync, follow these steps to reset:

  1. Reset Neo4j:

    docker exec neo4j bash -c "cypher-shell -u neo4j -p 12345678 'MATCH (n) DETACH DELETE n;'"
    docker exec neo4j bash /db-graph/run-queries.sh
  2. Re-index Redis Cache:

    REINDEX=true cargo run

🌐 Useful Links

About

The Nexus between Pubky homeservers and Pubky-App social features.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages