feat: tar-stream library instead of tar-js (#360)

[tomicvladan]

Using the tar-stream library which creates tar as a stream.

[agazso]

Does this work in a browser?

[tomicvladan]

No, this is just a fix for nodeJS to overcome size limitation.

[AuHau]

You can make work tar-stream in a browser, but the NodeJs APIs (Buffer etc) need to by polyfilled. I have done it here: https://github.com/rsksmart/rif-storage-js/blob/master/src/swarm-mini/index.ts

But not sure if it is good idea. It is quite painful.

[AuHau]

Oh right, exactly for this reason I pushed actually to implement streaming support, but it got never merged :-(

There is draft PR for that here: #372

[tomicvladan]

Using a polyfill for Node structures could be a solution, but I think it's more efficient to use browser streams:

https://developer.mozilla.org/en-US/docs/Web/API/Streams_API

It can work with Axios as well.

[agazso]

From previous experience the Streams API have several issues regarding compatibility, ease of use and maintenance. Also using tar can be considered as a workaround from the Swarm API perspective to allow working with multiple folders and files.

Personally I think it would be more interesting and future-proof to use the Swarm chunk API as an alternative with the help of external projects (bmt-js, mantaray-js) as it matches better the semantics of the problem space and thus could open up new directions, provide better control and create optimization opportunities. But that solution may be a separate library in itself and thus outside of the scope of bee-js.