Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up lock manager during bulk operations #33

Closed
3 tasks done
overheadhunter opened this issue Mar 25, 2019 · 2 comments
Closed
3 tasks done

Speed up lock manager during bulk operations #33

overheadhunter opened this issue Mar 25, 2019 · 2 comments
Milestone

Comments

@overheadhunter
Copy link
Member

overheadhunter commented Mar 25, 2019

As suggested in this thread, there might be room for improvements in the locking scheme implemented in #15:

Since the modification of the directory resource happens atomically when actually adding/removing a child, it might be worth trying to only get a shared read lock on a directory when adding or removing children. This is more of a eventual consistency model. I.e. a concurrent directory listing will be undefined.

When creating, deleting or moving a directory, we still need an exclusive write lock on that directory. During that time no read lock can be obtained, thus modifications to ancestors will not be possible.

Todo

  • profile application while adding a single large directory (deeply nested)
  • decide whether lock contention can be reduced on directory locks
  • (optionally) only "read-lock" directories when adding or removing children
@overheadhunter
Copy link
Member Author

I added a directory a/b/c/d/e containing 500 4-byte-files. The work was done by 5 FUSE threads, all behaving similar. We see that the actual locking isn't taking much time, but the recursion does:

Bildschirmfoto 2019-03-25 um 16 57 29

@overheadhunter overheadhunter changed the title Reduce lock contention during bulk operations Speed up lock manager during bulk operations Mar 26, 2019
@overheadhunter
Copy link
Member Author

Lock contention is not a problem.

However, the locking mechanism could be sped up significantly by reducing conversions between List<String> pathComponents and String path:

Bildschirmfoto 2019-03-26 um 09 33 21

We now have more than 10k invocations in 927ms, where we had 5.5k before in 986ms before.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant