Cookies & Local Storage

[Kiyoshika]

I know this topic is likely far away from implementation but I was reading the repo's wiki about the topic of cookies/local storage and whether to handle that in the client or the engine.

I did some quick research and seems like cookies/LS can be stored as small text files (maybe in some .cookies folder wherever the browser install directory is.)

If the direction is to possibly have (or at least allow) multiple UI clients to interact with the engine, then I think it makes the most sense to handle cookie/LS in the engine that way we wouldn't have to implement it in every UI client. I don't know the exact details about how to handle cookies/LS with tabs yet but that's far away at this point.

These are just my thoughts

[jaytaph]

I did some quick research and seems like cookies/LS can be stored as small text files (maybe in some .cookies folder wherever the browser install directory is.)

This is basically a user-agent specific. Some will store the info in files, others in an internal database etc.

I reckon we would have some kind of API for transferring cookies and local storage (LS) between the UA en the engine. The engine should not decide on how a UA stores the data, but should be able to use it. We could either transfer all the local data from the UA to the engine (for all sites?), or just send local storage of the given domain.

It could also be a pull action from the engine:

   let mut cookies = self.user_agent.get_cookies_for_domain("example.org");
   let mut local_storage = self.user_agent.get_local_storage("example.org");

Here, the UA would retrieve the cookies and LS from a text-file / database / wherever, and returns it to the engine, which in turn can use it.
After the request (or most likely, during the request), it can send back cookie information and local storage changes to the UA).

I'm sure there are better ways though

[lino-levan]

Storing text files would be really inefficient. Ideally it would use something like sqlite or rocksdb as a backend.

[Kiyoshika]

Curious, how/in what way would it be inefficient? A database is essentially doing the same thing. I don't think we would be storing anything substantially large per file for a given cookie/local storage. I don't know the tradeoff between having a bunch of small files versus one larger "file" (how a DB would typically store its data) but I don't imagine it would be significant

[Kiyoshika]

I guess the only thing I can think of is the IO cost of reading a lot of cookies from files, but I don't know at what point that's a true bottleneck

[lino-levan]

IO is one of the main bottlenecks when it comes to browser engines. There's a reason no (afaik) modern browsers store cookies in individual text files. It just doesn't make sense.

[lino-levan]

Just went out and checked and all three major browser vendors store cookies in SQLite.