Refactor piece and segment downloading into subspace-data-retrieval

[teor2345]

Purpose

This PR implements piece and segment fetching and reconstruction in the subspace-data-retrieval crate, so we can use that code to fetch objects.

Change List

This PR refactors piece and segment downloading into the subspace-data-retrieval crate.

It also:

• adds a segment piece downloading function
• splits out segment reconstruction code into its own function
• combines the DsnSyncPieceGetter and ObjectPieceGetter traits
• implements segment and piece downloading for ObjectFetcher
• fixes some bugs in piece and segment index calculations

And makes some minor cleanups:

• removes some debugging code that would cause a dependency on subspace-networking
• fixes some error handling bugs
• adds tracing logging

Out of Scope

It doesn't implement piece retries yet, I think that can go in another PR.

Testing

I've manually tested this PR with a proof of concept object fetching RPC on a local node in dev mode. The objects come back correctly and pass a hash check. (I haven't tested anything large or over multiple segments yet.)

Code contributor checklist:

• I have read, understood and followed contributing guide

[teor2345]

Notes from review with Nazar:

• don't change subspace-service
• don't refactor, just copy the code (and note where it came from)

[teor2345]

During my manual testing, I found some bugs in piece and segment index calculations, and added some trace logging.

We can remove the logging later, once we've finished automated testing.

[nazar-pc]

I'd say we don't need these logs if we already return an error explicitly. It balloons the code side and brings little to no benefit.

[teor2345]

I agree the logs don’t need to be in there long-term, but I’d like to keep them temporarily while I’m testing and debugging this code.

[teor2345]

Thanks for the review, just some follow-up questions.

[teor2345]

I agree the logs don’t need to be in there long-term, but I’d like to keep them temporarily while I’m testing and debugging this code.

[nazar-pc]

This will work for now, but long-term this generic function isn't very useful IMO because you will want to do reconstruction of missing pieces instead of failing altogether here. We already do this in farmer for example.

[teor2345]

Thanks, I'll put that on the list.