Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEATURE: Speed up UI flow queries with custom queries #3817

Draft
wants to merge 12 commits into
base: 8.4
Choose a base branch
from

Conversation

Sebobo
Copy link
Member

@Sebobo Sebobo commented Jun 26, 2024

What I did

This is a follow up for #3770 in which we load all nodes requested by the UI directly from the database instead of querying each node by itself and its document and content children separately.

How I did it

Extract some methods from the NodeDataRepository and added them to the NodeService in the UI to resolve all requested nodes with one db query and prefetch all child nodes of the resulting nodes to fill them into the 1st-level cache.

How to verify it

Flow query performance for loading a large page in a large project with the page tree, content tree and nodes required for the displayed content. The page tree requests 175 nodes (the 332KB request), the content tree requests 58 nodes ( the 129kb request) and the guest frame requests 191 nodes (the 615kb request) which haven't been loaded by the content tree yet and with the duplicates removed.

Performance UI 8.3.8:
neosui-83-838

Performance with branch #3770:
neosui-83-remove-ui-script-tag

Performance with this branch:
neosui-83-optimise-ui-flowqueries

@Sebobo
Copy link
Member Author

Sebobo commented Jul 2, 2024

Have to rebase, something is off with all those changes

@Sebobo
Copy link
Member Author

Sebobo commented Jul 2, 2024

Also have to check whether I broke the "presets" feature of the document tree

@Sebobo
Copy link
Member Author

Sebobo commented Jul 2, 2024

After discussing this a bit with @mhsdesign, it can make sense to add a "findNodesByIdentifiers" to the NodeDataRepository for 8.4 and use the new method instead of adding queries in this PR. This could also be useful for Neos 9. I will open an issue and link it.

Mark Kuiphuis and others added 12 commits July 4, 2024 09:15
Currently translated at 100.0% (121 of 121 strings)

Translation: Neos/Neos.Ui - Main - 8.3
Translate-URL: https://hosted.weblate.org/projects/neos/neos-ui-main-8-3/nl/
Currently translated at 100.0% (121 of 121 strings)

Translation: Neos/Neos.Ui - Main - 8.3
Translate-URL: https://hosted.weblate.org/projects/neos/neos-ui-main-8-3/es/
…nodedata

With this change the minimal required nodedata for each node in the rendered
content is inserted as data attribute and not as inline script anymore.
This improves performance as no extra function call is executed for each node.

Additonally the reduction in rendered node attributes reduces the output filesize and again improves loading time.

To prevent just-in-time loading of nodes all incomplete nodedata is requested after the guest frame has finished loading.
…r guest frame

The content tree is loaded early and can load fully loaded nodes without
being slower as the response time is the same. But this way we can skip
lots of nodes from being loaded by the guest frame if they are already
present in the store.
The UI filters itself, so the two children queries with
different nodetypefilters have no effect on the final
outcome and are just slowing down the response.
This way the followup methods that add the childnodes to the resulting
noderesults can directly read the children from the 1st level cache.
The speedup is ~50% (1s -> 500ms) in my tests for 175 nodes.
This way we only need one query to get all requested nodes
instead of a query for each one.
@Sebobo Sebobo force-pushed the bugfix/optimise-ui-flowqueries branch from cde3bab to a9b14b2 Compare July 4, 2024 07:16
@Sebobo Sebobo closed this Aug 14, 2024
@Sebobo Sebobo deleted the bugfix/optimise-ui-flowqueries branch August 14, 2024 06:35
@Sebobo Sebobo restored the bugfix/optimise-ui-flowqueries branch August 14, 2024 06:36
@Sebobo Sebobo reopened this Aug 14, 2024
@Sebobo
Copy link
Member Author

Sebobo commented Aug 14, 2024

Needs a rebase when #3770 was unmerged into 8.4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
8.4 Feature Label to mark the change as feature Performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant