Global State Management

[calebjacob]

A pretty common need for larger applications is implementing some type of global state and/or data fetching management. Tools like Zustand and React Query are commonly used in modern React applications to implement these patterns.

The current VM doesn't support any type of global state management - which leaves you with the only option of passing props. Passing props works well at a certain scale, but it can start to become unwieldy when you have multiple levels of components that all need to forward along the same props. Then it becomes even more challenging when a deeply nested component needs to modify that state.

Ideally, the new web engine would support usage of standard tools (like the ones mentioned above) across components. I know this will probably introduce a new set of security challenges, but it would be great to start thinking through how we could achieve this goal.

Thanks!

[mpeterdev]

tracking ticket for this already exists at #18, but we can continue conversation here for multithreaded discussion

[mpeterdev]

simplest example that comes to mind: a top level component represents a dapp, it may have a user preferences tab which affects some things about the application. Those preferences should not need to be passed as props through the entire component tree

[mpeterdev]

potential solution: components can initialize a store keyed by a string e.g. "mydapp-preferences", other components must know the key of the data store they wish to interact with

whether the store actually lives in the initializing component or the OWA is an implementation detail

hurdle: most stores now work on a hooks interface. Could we support cross-component hooks?

[andy-haynes]

What you're describing is well-suited for the current architecture. I think the big challenge for supporting existing off-the-shelf solutions is the iframe barrier between containers; e.g. we couldn't pass zustand methods to components directly because they can't be serialized. Querying state would also need to be asynchronous since all communication with other containers occurs via message passing.

That said, anything we pass as props should be fair game for state values as well, opening the door to storing functions and Promises in state.

A naive solution could be something like:

// in container boilerplate
function useBWEStore() { ... }

// component code
const { getState, setState } = useBWEStore();
const [favoriteDapps, setFavoriteDapps] = useState([]);

useEffect(() => {
  setFavoriteDapps(await getState("dapp.favorites"));
}, []);

useEffect(() => {
  await setState("dapp.preferences", (preferences) => [...preferences, preference]);
}, [preference]);

[gabehamilton]

Global stores would make handling optimistic updates and subscription updates a lot easier: posts & comments could use it at the moment.

[andy-haynes]

Global stores would make handling optimistic updates and subscription updates a lot easier: posts & comments could use it at the moment.

This makes me think what we ultimately want is a query layer for common operations. The cache we're discussing here would be the first step, but it doesn't address potential race conditions when multiple components are fetching the same data, say on initialization:

const fetchPost = (postId) => SocialDB.get({ keys: [`test.near/posts/${postId}`] });

// if multiple components run this for the same post on render, the fetches are duplicated
if (!(await getState(postId))) {
  post = await fetchPost(postId);
  setState(`posts/${postId}`, post);
}

By querying a common layer that initially supports SocialDB, Near RPC, etc., we could abstract the cache behind a query layer that runs in the shared context. Basically just a web worker in the outer application supporting an interface for fetching BOS data, caching the arguments and pushing responses to subscribed components. Now the above example becomes something like:

const post = await BOSQuery.SocialDB.getAndCache({ id: postId, path: 'test.near/posts/' })

[gabehamilton]

Yeah, that's definitely the aspect I'm interested in, something similar to Apollo client.

If further down the road we subscribe to some of that data (posts, notifications) over websockets that could also be a shared socket underneath and components just interact with the cache. A component might make optimistic updates to the cache, other components could receive that state update, then when the same operation trickles through on-chain->indexer->websocket subscription the cache could replace the optimistic with the actual and send out the state update for the final item.

[andy-haynes]

Websockets could be an option, I don't know if there are any challenges with using them across sandboxed iframes but I don't think it's anything that couldn't be overcome. In the current implementation everything uses window.postMessage for communication, which I think would be sufficient for what you're describing, at least in the short term. It would be interesting to see whether one has a distinct performance advantage.

Someday I'd like all that event-based communication to be handled with something like RxJS to simplify integrating events into the component lifecycle. Abstracting it away from the actual mechanism also makes BWE simpler to reason about and maintain.

[mpeterdev]

By querying a common layer that initially supports SocialDB, Near RPC, etc., we could abstract the cache behind a query layer

we might be able to make this a bit less BOS-specific if we are able to hook up a library like React Query or SWR with a custom cache location that bridges components. Both of those have caching+deduping built in

[calebjacob]

Pulling in an existing library like React Query would be great for DX if we can pull that off.