fix: Enable sync with all peers for not local documents

[mykola-vrmchk]

Added a test with two peers (Alice, Bob)

• both have a shared policy set to false for all requests
• it fails to replicate the document even with an explicit request .find

Fixed behavior of method .addDocument on synchronizer to send requests for documents we do not create

[acurrieclark]

This is a really interesting case, and one which needs consideration.

A "request" for a document is essentially performed by creating an empty document and then attempting to sync that empty doc with connected peers. As a share policy of false instructs repo not to explicitly sync with other peers unless they request it, that "request" won't ever be sent. That should be a relatively simple case to fix.

However, let's say you have 2 unconnected peers which both connect to a central server. This server would typically have a share policy of false. If one of those peers has a local document which it doesn't want to share (share policies can be set depending on documentId and peerId), the sync server won't know about it. If the 2nd peer tries to request that document from the server, the current outcome is that the sync server doesn't have it, so responds with an "unavailable" message. If we were to change it so that you could request documents with a share policy of false, this would effectively pass on the request to all connected peers. Now suddenly the sync server has that document which the initial peer didn't want to share with it.

While I imagine receiving an explicitly requested a document from one peer might be expected behaviour, I would imagine that being able to essentially request any document from a peer anywhere on your network might not be.

[mykola-vrmchk]

Added test with MessageChannelNetworkAdapter

[acurrieclark]

Looks like there is an only in there. Would you mind removing it?

[mykola-vrmchk]

• Looks like there is an only in there. Would you mind removing it?

I added it intentionally, just not to run all tests on CI for now but I can remove it if you insist

[acurrieclark]

CI won't run the tests at all if there is an only there, sadly

[acurrieclark]

I think there is another side effect here. Let's say that we have a peer with a sharePolicy of false. If it boots up for the first time and tries to find a document, will this now be immediately synced with all peers?

[acurrieclark]

I think there should be a test here for a 3rd peer, to show it can't find a document from across the network.

eg. alice --> bob --> charlie

Alice creates a document.
Charlie requests it. Show that it doesn't resolve.

[dmaretskyi]

Added.

[pvh]

I think this is a relatively significant problem -- we're rather cavalier with how we broadcast documentIds at the moment. If we ask someone who doesn't have a documentId if they've got it... they could make a note and just turn around and request it back.

This is a problem solved by the dat / hypercore project with the idea of a discoveryKey (a hash of the document ID which is used to find peers which claim to have a document before challenging them to prove it.)

We should definitely consider this in the next round of sync & auth{n,z} thinking as we upgrade the network protocol. I don't think there's going to be a great solution in the meantime.

[dmaretskyi]

This test is currently failing