Browsing "media folder" is slow

[trou]

Hello,
I've been using Slimserver since ages, but I'm finally reporting a "problem" that has been bugging me since I've moved the server to lower power hardware: browsing the "media folder" (which is my preferred way of looking for music) is very slow.

Now my server is running on an apu2c4 board, with a 1ghz AMD GX-412TC and 4GiB of RAM.

Clicking on "Media Folder" in my browser, it took 5s to reach the directory listing (with 270 entries). Note that the root of my library is mounted over CIFS.

However, it seems of the most time is spend in CPU, not I/O (doing some perf/strace checks)

Is there anything I could do to help improve this ?

[mherger]

What LMS version are you running? Make sure you're running 7.9 with the memory setting for the database set to "maximum" (Settings/Advanced/Performance).

BMF has always been a source of complaints about performance, as it's basically running a scan interactively. Whenever you enter a folder, files in that folder would be scanned, checked for updated files, updating data in the database etc. Your low-powered setup, accessing music from another machine over CIFS etc. all don't add to the experience in a positive sense. 7.9 has a bunch of tweaks to improve this behaviour compared to 7.7. Alas, it's not as quick as the other browse modes, due to aforementioned activity the browse mode triggers.

[trou]

I'm running Logitech Media Server Version: 7.9.1 - 1516947667 @ Fri Jan 26 08:25:58 UTC 2018, with perf set to "Maximum". I made some tests with the performance setting and I don't see any visible effect when changing it.

Whenever you enter a folder, files in that folder would be scanned, checked for updated files, updating data in the database etc.

Is this necessary ? I thought the DB was only updated when scanning the media folder. Since I only use BMF, data could be updated only when the music is actually played ?
I'd be happy to help profile the app to see what's so slow. Is there any way to do this ?

[michaelherger]

Is this necessary? No, it's a feature. It's a deliberate design decision (admittedly from years ago).

You might give the "browse file system" a try, which would do a lot of the same work, but not store stuff in the database. Maybe that's faster.

[trou]

You might give the "browse file system" a try, which would do a lot of the same work, but not store stuff in the database. Maybe that's faster.
For those that want to try, you need to activate the "Additional browse mode" plugins, and add a menu in the player settings.

Unfortunately, I mostly browse music through the Web interface, which does not include this browse mode (if I'm not mistaken).

[mherger]

It should be available on any UI - but you might need to enable it in Settings/Player/Advanced Browsemenus or similar.

[trou]

OK, I didn't enable it in the right place, now it is visible on the website too.

I just tried it, it's a tad faster but still slow.

perf record shows the following top functions:

  17,21%  squeezeboxserve  perl               [.] Perl_hv_common                                                                                                                                                                                               
   6,23%  squeezeboxserve  perl               [.] Perl_gv_fetchpvn_flags                                                                                                                                                                                       
   5,58%  squeezeboxserve  perl               [.] Perl_sv_clear                                                                                                                                                                                                
   5,34%  squeezeboxserve  perl               [.] Perl_sv_setsv_flags                                                                                                                                                                                          
   5,17%  squeezeboxserve  perl               [.] Perl_sv_upgrade 

Running the server with --perfwarn=1 gives the following output:

[18-07-24 14:35:11.1644] Request      1.98741 : Slim::Control::Queries::musicfolderQuery
[18-07-24 14:35:13.2342] Template     1.75236 :   /usr/share/squeezeboxserver/HTML/Default/xmlbrowser.html (this templ:  427444 us)
[18-07-24 14:35:13.2437] Page Build   4.17192 : Page: clixmlbrowser/clicmd=browselibrary+items&linktitle=PLUGIN_EXTENDED_BROWSEMODES_BROWSEFS&mode=filesystem/index.html
[18-07-24 14:35:13.2579] IO           4.19268 : Slim::Web::HTTP::processHTTP
[18-07-24 14:35:37.2023] Request      1.30817 : Slim::Control::Queries::musicfolderQuery
[18-07-24 14:35:39.0434] Template     1.61784 :   /usr/share/squeezeboxserver/HTML/Default/xmlbrowser.html (this templ:  387905 us)
[18-07-24 14:35:39.0517] Page Build   3.21070 : Page: clixmlbrowser/clicmd=browselibrary+items&linktitle=PLUGIN_EXTENDED_BROWSEMODES_BROWSEFS&mode=filesystem/index.html
[18-07-24 14:35:39.0655] IO           3.23034 : Slim::Web::HTTP::processHTTP
[18-07-24 14:36:46.9020] Template     1.68874 :   /usr/share/squeezeboxserver/HTML/Default/xmlbrowser.html (this templ:  401861 us)
[18-07-24 14:36:46.9116] Page Build   2.75529 : Page: clixmlbrowser/clicmd=browselibrary+items&linktitle=PLUGIN_EXTENDED_BROWSEMODES_BROWSEFS&mode=filesystem/index.html
[18-07-24 14:36:46.9263] IO           2.77596 : Slim::Web::HTTP::processHTTP
[18-07-24 14:36:52.2206] Request      1.31495 : Slim::Control::Queries::musicfolderQuery
[18-07-24 14:36:54.1456] Template     1.69274 :   /usr/share/squeezeboxserver/HTML/Default/xmlbrowser.html (this templ:  402338 us)
[18-07-24 14:36:54.1536] Page Build   3.30459 : Page: clixmlbrowser/clicmd=browselibrary+items&linktitle=PLUGIN_EXTENDED_BROWSEMODES_BROWSEFS&mode=filesystem/index.html
[18-07-24 14:36:54.1690] IO           3.32644 : Slim::Web::HTTP::processHTTP
[18-07-24 14:37:02.6606] Request      1.32700 : Slim::Control::Queries::musicfolderQuery
[18-07-24 14:37:04.5328] Template     1.64382 :   /usr/share/squeezeboxserver/HTML/Default/xmlbrowser.html (this templ:  390883 us)
[18-07-24 14:37:04.5412] Page Build   3.26113 : Page: clixmlbrowser/clicmd=browselibrary+items&linktitle=PLUGIN_EXTENDED_BROWSEMODES_BROWSEFS&mode=filesystem/index.html
[18-07-24 14:37:04.5566] IO           3.28258 : Slim::Web::HTTP::processHTTP

I also generated a NYTProf output, if you're interested.

[michaelherger]

Oh interesting: how many items per page do you display? Would other UIs (eg. Squeezeplay) be faster? Template rendering is awfully slow. As you can see the musicFolderQuery (which does the I/O, scanning etc.) uses about half of the time, while the xmlbrowser.html spends the rest of the time. Unfortunately due to BMF's nature the pages can't be cached, as with other browse pages. Therefore they have to be re-rendered all the time.

[trou]

I set it to 500, but I only have about 270 items in the root folder.
I did a test with a new folder, with 500 folders inside:

[18-07-24 15:07:26.0406] Request      4.13449 : Slim::Control::Queries::musicfolderQuery
[18-07-24 15:07:29.7528] Template     3.20622 :   /usr/share/squeezeboxserver/HTML/Default/xmlbrowser.html (this templ:  774832 us)
[18-07-24 15:07:29.7658] Page Build   8.23516 : Page: clixmlbrowser/clicmd=browselibrary+items&linktitle=PLUGIN_EXTENDED_BROWSEMODES_BROWSEFS&mode=filesystem/index.html
[18-07-24 15:07:29.7878] IO           8.26700 : Slim::Web::HTTP::processHTTP
[18-07-24 15:08:41.2088] Request      3.11590 : Slim::Control::Queries::musicfolderQuery
[18-07-24 15:08:44.8171] Template     3.19420 :   /usr/share/squeezeboxserver/HTML/Default/xmlbrowser.html (this templ:  753647 us)
[18-07-24 15:08:44.8300] Page Build   7.08658 : Page: clixmlbrowser/clicmd=browselibrary+items&linktitle=PLUGIN_EXTENDED_BROWSEMODES_BROWSEFS&mode=filesystem/index.html
[18-07-24 15:08:44.8548] IO           7.12140 : Slim::Web::HTTP::processHTTP
[18-07-24 15:09:03.4772] Request      3.10965 : Slim::Control::Queries::musicfolderQuery
[18-07-24 15:09:07.0598] Template     3.16907 :   /usr/share/squeezeboxserver/HTML/Default/xmlbrowser.html (this templ:  748674 us)
[18-07-24 15:09:07.0707] Page Build   6.79230 : Page: clixmlbrowser/clicmd=browselibrary+items&linktitle=PLUGIN_EXTENDED_BROWSEMODES_BROWSEFS&mode=filesystem/index.html
[18-07-24 15:09:07.0975] IO           6.82869 : Slim::Web::HTTP::processHTTP

I've also used iPeng to access the library and I think it was faster (I'll have to try tonight to confirm)

[mherger]

The best I can suggest: lower the number of items per page. It should considerably speed up rendering... (though not linearly).

[trou]

Unfortunately, it also makes browsing the root folder of the library a pain :(

[mherger]

You could "Browse Filesystem", which would not import those files.