Skip to content

Disgover

Jump to bottom
Avery-H edited this page Jun 19, 2018 · 1 revision

Table of contents

Enter the world of disgover !

Disgover is part of the Dispatch protocol and is reponsible for node and artifcats disc(G)overy on the network.

Overview

dosgover is a distributed node discovery mechanism that enables locating any entity (server, worker, drone, actor) based on node address.

The intent is to be able to PING and FIND-NODE through some transport protocol (currently using grpc). It is not meant for data storage/distribution mechanism. (That will be another package)

Nodes use a Kademlia Hash Table (KDHT) for the following:

  • store contact information for other nodes
  • provide a list of contacts to new nodes joining the network
  • find specific nodes on the network
  • functions as a gateway to outside local network

In disgover, we refer to other nodes as Contacts. In the course of this wiki, Node and Contact will be used interchangably. When talking about the system in general, the term node makes more sense, while specific references to communication between them in the context of disgovery it makes sense to use contact.

Communication is done through gRPC, which is a object serialization implementation for Remote Procedure Calls (RPC).

Contact is a simple structure that defines the minimum amount of information to lookup a node in the network and provide sufficient information to contact that node. It uses the unique address of the node as a key with the I.P. address and port to communicate with that node.

The approach to finding is accomplished by having all nodes maintain a list of nearby contacts in the network. When a new node joins the network, it will attempt to disgover nodes in the network that it has previous knowlege. If any of those previously known contacts is active, it will provide it's list of its known contacts to the new node on the network. In the even that no previously known contact is available, there must be a fallback node that can provide a list of active contacts in the network.

To find a specific contact, a node sends a FIND request to one of it's known contacts to see if they have it in the list. Nodes receiving a FIND request will search their KDHT for that address and return if it exists; otherwise, it will return the nearest contacts it has to the requested contact and the original requester will continue making requests to the nearest nodes until the desired contact is found. So it continues to get closer and closer until that node is found or is not found.

In a single sentence: it is a communication of .... Do you know this address and the answer may be, no, but this contact might know them or at least know who would know them. It's our own 6 degrees of Kevin Bacon.

Design Approach

One of the main things we want to accomplish is a simple, plugable API. For Disgovery, the we have two major things:

  • Communication protocol. - gRPC is our only currently supported package, but we can add to that as desired.
  • Structure for efficiently finding contacts - KDHT is our currently implementation and at the moment we are tightly bound to using KDHT.

They design follows a plugable implementation for what implementation the node should use on startup. We follow a builder pattern to set this configuration of the system at startup time.

The builder options are specified in the file that has the IService implementation. So you will notice that disgover_service includes a function named withGRPC(). This registers the corresponding transport protocol to use when running the system.

The Packages section provides more details how this is implemented

Getting Started Sample

To use Disgover, you will need to at least have the commons package for loading node configuration

A functional server and client implementation are available to try it out:

A simple service that runs disgover

A simple gRPC client that runs disgover

Run in a local Kubernetes Cluster

Packages

  • proto - protobuf generated code for communication over GRPC

Base Disgover Package

The base disgover package contains the service implementation that handles all of the request / response actions. It follows the Service Implementation pattern.

At this time, disgover supports GRPC and HTTP protocols. GRPC is used for node to node communication and the HTTP protocol is currently intended for use with thin clients.

The package consists of the following files:

  • disgover service

    • Provides the implementation of the IService interface
    • Includes builder functions for the desired transport protocol(s) to use

  • disgover api

    • Provides the concrete implementation of the functions that perform the discovery of other nodes.
    • Find() go func (thisRef *Disgover) Find(contactID string, sender *Contact) (*Contact, error)
    • GetContactList() go func (thisRef *Disgover) Find(contactID string, sender *Contact) (*Contact, error)

  • disgover transport grpc

    • Provides the gRPC functions for communicating to other nodes.
    • Each public function converts the proto structure type to a domain structure type, which supports the ability to have plugable solutions.

  • disgover transport http Add when migrated from disgo to appropriate place in disgover

Clone this wiki locally