High CPU usage may impair real-time playback

[klerg]

FluidSynth version

2.4.3

Describe the bug

When playing Altitude.mid in Fluidsynth 2.4.3 with the default audio driver on some Windows 10 systems it will crackle at various points throughout playback of the MIDI.

Expected behavior

When playing Altitude.mid in Fluidsynth 2.4.3 with the default audio driver on some Windows 10 systems Fluidsynth should not crackle throughout playback of the MIDI.

Steps to reproduce

Download Altitude.mid below, open Fluidsynth 2.4.3, load GeneralUser-GS v1.47x.sf2 or GeneralUser-GS.sf2, play Altitude.mid and you may hear it crackle on some Windows 10 systems.

Additional context

It will also crackle with waveout driver, although it is a bit less, also this issue first appeared in FluidSynth 2.4.1 and continued in 2.4.2. AFAIK, cannot hear any crackling on FluidSynth 2.4.0. And turning on/off chorus or reverb has no effect on the crackling. One way to lower most crackling is to set audio buffers at 64, but no need for this in 2.4.0

ALTITUDE.zip

[derselbst]

Sry, but I not matter what audio driver I use, I cannot reproduce this on Windows. These problems are typically very closely bound to the soundcard and driver in use, the system load, and also to the various settings, like period-size and number of buffers that can be specified in fluidsynth, which makes it pretty much impossible to reproduce this or comprehend what's going on.

[klerg]

Well, even if you are not able to reproduce this issue it still exists on some Windows 10 systems. Given that the problem does not happen with Fluidsynth 2.4.0 I doubt the soundcard or driver is at fault here. Also, in 2.4.0 no need to mess with the settings, such as period-size or number of buffers so that pretty much rules out any of those as the cause, can the default value for audio buffers be 64 ?

[derselbst]

I cannot fix an issue that I cannot reproduce. Given the fact that no code changes have been made to the audio drivers between those version, and your vague description of the reproducibility of this issue ("you _may_ hear it crackle on _some_ Windows 10 systems) I am tempted to close this issue.

can the default value for audio buffers be 64 ?

No, because it would impose an unacceptable high latency for all users.

…

Message ID: ***@***.***>

[klerg]

I will give more info so you can reproduce the issue. That is good to know but this issue has nothing to do with the audio drivers and the cause most likely comes from either #1415, #1417 and/or #1424. all from 2.4.1. Yes, here is more detail, the crackling occurs on laptops with Realtek ALC3234 sound chip. I'm sure this will help keep the issue open, yes ?

Ok, I do notice some latency when the audio buffers are set to 64 on my end too

[derselbst]

You should observe the CPU utilization on those systems. Also play around with the -z parameter, which influences audio.period-size. Disabling reverb and chorus would lower CPU utilization, try this as well. And knowing the number of CPU cores, you can try to use multithreaded rendering with -o synth.cpu-cores

[klerg]

Sure, 2.4.0 uses much less CPU than 2.4.1/2/3 6% vs. 22%. This hardly has an effect on the crackling, and causes the MIDI to sound disjointed. I already disabled reverb and chorus but it still crackles and barely uses less CPU. It took some time to get this command, it needs =2 at the end, and that gives 7% CPU usage and no crackling. Is it safe to assume the high CPU is the cause of crackling, and any clue how to fix it ?

[derselbst]

The higher CPU usage could be an effect of #1444. I tried to mitigate that in #1469. Currently, I have no other ideas.

In the meantime, you're advised to use bigger audio buffers (-z) and potentially more of them (-c).

[klerg]

Yes, it seems like the high CPU use can be from #1444. I see too bad #1469 did not help.
I'm sure you will find more ideas soon.

I forgot to mention that -z also eats up more CPU cycles at upwards of 30% with spikes to 40%, -c gives more latency, only -o synth.cpu-cores helps but do not know why

[derselbst]

I forgot to mention that -z also eats up more CPU cycles at upwards of 30% with spikes to 40%, -c gives more latency

Which values did you fed to -z and -c resp?

Also: Which CPU are you using on the affected system(s)?

[klerg]

Well, I need to set -z to 2500 to stop crackling and -c at 8 to avoid high latency

Just 1 CPU core, but I'm not sure if the crackling comes from high CPU, because when I set -o synth.cpu.cores to 3, CPU use jumps to 30-40% but no crackles. Yet, -c 64 gets rid of almost all crackling, how to make sense of this ?

[derselbst]

Which CPU are you using on the affected system(s)? I need the exact model name.

[klerg]

Ok, sure the CPU is Intel Core i3-4010U, but how or why does that matter ?

[derselbst]

This is a dual core CPU, which explains that setting synth.cpu-cores to 2 or 3 works better than 1.

[klerg]

But when I set synth.cpu-cores to 2 or 3 it uses more CPU 30 to 40% than 1 which is in 20s how is that better ?

[derselbst]

Multi-threaded rendering --> higher CPU utilization.

I've compiled a special build which has the IIR filter disabled. This is to confirm whether or not #1444 is to blame. Pls. try to reproduce the "crackle" with this build and let me know the result: https://dev.azure.com/tommbrt/tommbrt/_build/results?buildId=11709&view=artifacts&pathAsName=false&type=publishedArtifacts

[klerg]

I see, but how can a synth.cpu-cores value > 2 work on a dual core CPU ?

Well, my guess is the IIR filter has been enabled pre-2.4.0 too. Yes, that is a good way to see if #1444 is behind this issue. Sure, I get no crackles using the build here, CPU use is at 6% just like 2.4.0, but no filter, what next ?

[derselbst]

Ok, thanks, I'll see what can be done. But pls. don't expect a quick solution.

I see, but how can a synth.cpu-cores value > 2 work on a dual core CPU ?

Because every modern operating system does preemptive multitasking.

[klerg]

Thanks as well, I hope the crackles will stop. Of course this is not an easy bug to fix.

Sure, that helps to know and if you need something from me I'll do my best to lend a hand

[mrbumpy409]

I have been able to reproduce this issue on both Windows and Linux. I happened upon it by accident when encountering a bugged SoundFont preset that was using 34 voices per note due to missing velocity splits. Playing the preset resulted in tremendous crackling and audio dropouts. However, when switching to FluidSynth v2.3.7, v2.4.0, or other SoundFont synths at the same audio latency (BASSMIDI, Polyphone), there were no crackles at all.

So, I did a git bisect, and found the problematic commit:

9a19ab28b0466834189ccde3c1449c1c21e99e93 is the first bad commit
commit 9a19ab28b0466834189ccde3c1449c1c21e99e93
Author: Tom M. <[email protected]>
Date:   Sun Dec 1 14:14:46 2024 +0100

    Smooth linear filter parameter change (#1432)

 src/rvoice/fluid_iir_filter.c | 187 +++++++++++++++++++++++++++++-------------
 src/rvoice/fluid_iir_filter.h |  31 +++++--
 src/rvoice/fluid_rvoice.c     |   4 +-
 src/synth/fluid_voice.c       |   5 --
 4 files changed, 154 insertions(+), 73 deletions(-)

Steps to Reproduce

1. Download and extract high poly audio breakup.zip.
2. Load the SoundFont into bank 0 or 1 and play the MIDI file, e.g.:

   fluidsynth "high poly preset test.sf2" "high poly audio breakup.mid"
   

The MIDI file selects preset 000:002 in the SoundFont and plays a series of jazzy chords on a synth sound that uses 34 voices per note. While it might seem unrealistic to have a preset that plays 34 voices per note, such high polyphony is not an unrealistic demand on a synthesizer when multiple channels of instruments are playing simultaneously.

Result

In FluidSynth 2.4.1 and later, CPU usage is considerably higher, leading to audio dropouts in low latency / low CPU environments. The audio files included in the zip were created on my desktop PC using FluidSynthPlugin running in REAPER with reverb and chorus effects disabled and Pipewire buffer size set to 256 w/ 48000 sample rate. My CPU is an Intel Core i7-990x (3.46 GHz hexa-core) overclocked to 4.15 GHz.

Further Tests

Moving away from the DAW plugin and using command line FluidSynth (via Pipewire driver with the same buffer and sample rate as above) to play the MIDI file resulted in even higher crackling, possibly due to the enabled reverb & chorus. With this configuration, both v2.3.7 and v2.4.0 exhibited very minimal crackling. The amount of crackling between the two versions was identical, as far as I could tell. At these same settings, however, v2.4.1 became an unrecognizable wall of noise.

[derselbst]

An excellent reproducer, thanks Chris! It surprises me that you have bisected 9a19ab2. This commit actually decreases computational complexity compared to 2.3.7. For me, 2.4.1 sounds fine. In my testing I could confirm that the crackling starts with 2.4.2, because of #1444. It's probably a caching issue, I need to profile it again...

Can you pls. double check by using fluidsynth --version that it reports the expected version (for both, runtime and executable) and also reports the same sample type as all the other versions?

[mrbumpy409]

I will do some more testing after my lessons today, as it did seem that the crackling might be a bit worse for me in 2.4.2 than 2.4.1 as well, so there very well could be more than one contributing factor to the current high CPU usage.

[derselbst]

Pls. try recent master as well: https://dev.azure.com/tommbrt/tommbrt/_build/results?buildId=11754&view=artifacts&pathAsName=false&type=publishedArtifacts

[mrbumpy409]

So, I was able to verify my findings. Commit 9a19ab2 does indeed introduce the audio crackling/dropout issue, and the previous commit aed6ba3 is perfectly fine. I have also been verifying the FluidSynth version after each compilation. Recording is still done through FluidSynth Plugin & REAPER, as that is the easiest way for me to capture the exact sound issues. FluidSynth from the command line always has more crackling and dropouts than when used within REAPER. Either way, sample type always showed as "double".

Here is the latest round of recordings. Tonight, I was getting more dropouts rather than crackling, so I re-recorded the tests (except for v2.3.7 / 2.4.0, which was perfect as before). Here are the results:

• FluidSynth 2.3.7 / 2.4.0 / 9a19ab2: No crackling or dropouts, as can be heard in v2.3.7 & 2.4.0.ogg.
• FluidSynth aed6ba3 / 2.4.1: Plenty of audio dropouts, as can be heard in v2.4.1.ogg.
• FluidSynth 2.4.2: Plenty of audio dropouts, as can be heard in v2.4.2.ogg. Is this worse than v2.4.1? It's hard to tell.
• FluidSynth master: There are still dropouts, but perhaps fewer? Can be heard in master.ogg.

[mrbumpy409]

I did even more testing so I could capture CPU performance numbers and recordings of the FluidSynth command line player. Here are the recordings.

I ran the test three times in FluidSynth Plugin (REAPER) and three times in FluidSynth command line for each version and wrote down the maximum CPU usage. Here are the results:

FluidSynth Plugin CPU usage max:

• 2.3.7: 3.8%, 3.9%, 4.0%
• 2.4.0: 3.9%, 4.0%, 4.0%
• 2.4.1: 9.7%, 10.4%, 9.7%
• 2.4.2: 10.3%, 10.1%, 9.7%
• master: 10.8%, 9.8%, 9.7%

FluidSynth command line CPU usage max:

• 2.3.7: 4.1%, 4.2%, 4.2%
• 2.4.0: 3.8%, 4.0%, 3.9%
• 2.4.1: 9.3%, 8.5%, 9.1%
• 2.4.2: 8.5%, 8.5%, 9.0%
• master: 9.0%, 9.4%, 8.5%

FluidSynth versions 2.3.7 and 2.4.0 have nearly identical CPU usage, averaging 3.98%, and then there is a huge jump in CPU usage from 2.4.0 to 2.4.1. This jump happens once commit 9a19ab2 enters the picture, and the CPU usage during the test hits around 9.44% in all subsequent versions. Averaging both sets of numbers shows a jump of 2.37 times higher CPU usage.

The CPU usage of the FluidSynth command line was slightly lower overall than when run as a plugin in REAPER, though this should be no surprise considering the overhead of running within a DAW. REAPER and FluidSynth were also both using the same audio settings (buffer size 256, 48 KHz sample rate, fx disabled) with the exception of synth.polyphony, which was set to 512 for the FluidSynth command line.

Despite similar settings, the FluidSynth command line produced considerably glitchier audio than FluidSynth Plugin running in REAPER, and even produced occasional glitches when using FluidSynth 2.3.7 and 2.4.0. It wasn't until I finished testing that I realized I had the polyphony set to 512 in the FluidSynth configuration file. Setting polyphony back to the default does reduce the distortions somewhat, but they are still far greater than with FluidSynthPlugin.

Anyway, I hope this is useful 🙂

[derselbst]

Ok, thanks Chris! I was previously using float samples. I changed it to double, but still 2.4.1 sounds fine to me. Yet, I might have to revert my previous statement a bit, because when filter parameters are changing, many recalculations of the filter coefficients need to be done. To nail it down a bit and since you know how to compile, can you pls. try the following.

1. git checkout 9a19ab28b0466834189ccde3c1449c1c21e99e93
2. Comment out Line 123 in src/rvoice/fluid_iir_filter.c:

-               fluid_iir_filter_calculate_coefficients(iir_filter, output_rate, &dsp_a1, &dsp_a2, &dsp_b02, &dsp_b1);
+//                fluid_iir_filter_calculate_coefficients(iir_filter, output_rate, &dsp_a1, &dsp_a2, &dsp_b02, &dsp_b1);

3. Compile & Listen to result
4. Undo the change of step 2.
5. Switch to float samples, i.e. cmake -Denable-floats=1
6. Recompile & Listen to result

Step 6. will cause the filter coefficients to calculated in single precision. This would hopefully give a clue of the required optimization that need to be done, i.e. ideally single precision is enough. Otherwise I need to consider using lookup tables or to vectorize the filter coefficient calculation.

Oh, and can you pls. tell me the CPU model you're using? Still trying to figure out what's the difference between our PCs here.

[mrbumpy409]

I will test this out when I am done teaching tonight. My CPU is an Intel Core i7-990x (3.46 GHz hexa-core) overclocked to 4.15 GHz.

[mrbumpy409]

3. Compile & Listen to result

With line 123 commented out, the CPU (and sound) performance is identical to 2.3.7 and 2.4.0:

• FluidSynth Plugin: 3.9%, 3.9%, 3.9%
• FluidSynth CL: 3.9%, 4.0%, 4.1%

6. Recompile & Listen to result

Using float samples, the CPU performance is a slight bit better than with 2.4.1 before, which of course improves the dropouts a bit:

• FluidSynth Plugin: 7.9%, 7.9%, 8.4%
• FluidSynth CL: 8.1%, 8.0%, 8.1%

The average CPU usage with the smooth filter change logic applied improved from 9.44% to 8.07% when using float samples, compared to 3.95% with line 123 commented out.

Note: For consistency with my previous tests, the FluidSynth command line is using set synth.polyphony 512 in the config file.

[klerg]

It's a relief that someone else could find a way to reproduce this bug on Linux too. I do not know if it was actually an accident for you to encounter this particular preset that has so many voices per note as it seems too on the nose. Well, I was just given a lot of crackling and few audio dropouts by chance. Same here 2.4.0 and below has no crackling or noise of any kind going on when playing from BASSMIDI. I assume the same can be said of hardware SF2 synths, but will test this out myself soon then.

I really have no clue what a git bisect means but it seems to help

Yes, this is basically the same results that I was able to obtain although not sure how the high CPU use relates to low latency. At this point, it seems the crackling, and dropouts need both low latency and low CPU to reveal itself. I use REAPER as well but Pipewire I assume is not going to work on my PC as I'm in Windows only, and not on Linux at this time. A Core i7 is not a low-end CPU and it was overclocked to over 4 GHz so it does not matter if the CPU is fast too.

I will update my FluidSynth VST2 plugin in REAPER soon but again Pipewire is not going to be an option for me as using Windows for now and that will stay the same for a while now. Also, not sure if the default setting of Fluidsynth is to turn chorus or reverb on/off, but I know with both off it barely uses less CPU. I'm on the same boat with 2.3.7 and 2.4.0 and hear close to zero crackling or dropouts whatsoever. But with the new test files in 2.4.1 and above is where the noise starts and can be compared like a wall as has been said already. I'm only able to try Fluidsynth command line for now, but once I update the VST2 plugin I will see if it has any difference

Looks like a lot happened and it feels as if this bug has grown and become something of a virus of sorts. And as such, it makes sense to change the title of the bug, but not sure if anyone could predict this chain of events.

I will admit that I'm not totally shocked that #1432 is the problematic "commit" as it is called and behind the audio crackling and dropouts. After all, my hunch had #1415, #1417, #1424 as the culprit, and I forgot that #1432 was the proposed solution for all three of the issues above. But it does make sense for the Fluidsynth command line MIDI player to have more crackling and dropouts than the VSTi plugin in REAPER.

[mrbumpy409]

Yes, this is basically the same results that I was able to obtain although not sure how the high CPU use relates to low latency. At this point, it seems the crackling, and dropouts need both low latency and low CPU to reveal itself. I use REAPER as well but Pipewire I assume is not going to work on my PC as I'm in Windows only, and not on Linux at this time.

In Windows you would use ASIO or WASAPI for low latency. On Linux, you use Pipewire or JACK.

A Core i7 is not a low-end CPU and it was overclocked to over 4 GHz so it does not matter if the CPU is fast too.

Well, it is a high end CPU from 2011 (originally $1,000 retail), but it's definitely not high end anymore.

I will update my FluidSynth VST2 plugin in REAPER soon but again Pipewire is not going to be an option for me as using Windows for now and that will stay the same for a while now. Also, not sure if the default setting of Fluidsynth is to turn chorus or reverb on/off, but I know with both off it barely uses less CPU. I'm on the same boat with 2.3.7 and 2.4.0 and hear close to zero crackling or dropouts whatsoever. But with the new test files in 2.4.1 and above is where the noise starts and can be compared like a wall as has been said already. I'm only able to try Fluidsynth command line for now, but once I update the VST2 plugin I will see if it has any difference

Yeah, I just had effects off since that is the configuration I was using with the plugin, and I wanted to be consistent in all my tests, so I left it off.

[klerg]

I do not know how to use ASIO in Fluidsynth but WASAPI gives some weird error and is incredibly noisy.

Yes, the Core i7 will always be a high end CPU but your model is just old as it is from 2011 so not new by any means or way.

Sure that is a good idea to turn off all effects, but barely uses any less CPU, On the positive side, the latest "master" gives no crackling in Altitude.mid, but your recordings do,

[derselbst]

I've implemented my preferred solution to address this issue, which is to auto-vectorize the calculation of the filter coefficients. Source Code on the cpu-opt branch (530f0e0), Windows binaries here.

While gcc and clang successfully vectorize this on Linux, MinGW and MSVC fail to vectorize this code on Windows. I get no crackling on my Linux desktop PC. I was also testing this on a Windows-Laptop, and I get crackling when it's running on batterie, but no crackling when it's plugged in. So it might not necessarily work better for you. Yet, there have been many code restructurings, so pls. have a test.

[klerg]

I'm not sure what auto-vectorize even means or how it is possible for any of that to help with the high CPU and crackling but so far things have not improved and are worse than the last "master"

I cannot try this on Linux, and do not know anything on MinGW or MSVC or why they do not vectorize this on Windows, whatever that is supposed to do. Altitude.mid uses more CPU, up to 20%, and playback stops or sound is lost at around 1:15. Using waveout, at the same time, playback turns into complete noise and will need to quit. Also, high-poly-audio-breakup.mid has a lot of noise, similar to the recordings above, any idea what is wrong ?

[derselbst]

Ok, thanks. I've implemented the alternative approach, which uses a lookup table. Binaries here. Pls. let me know how it works.

The drawback of this approach is that the cutoff frequency can only be set with an accuracy of 10 cents. I could try using linear interpolation to smooth this out a bit.

Also, the current draft implementation will only work with the default sample rate of 44,1kHz.

[klerg]

Thanks to you. I see, whatever a lookup table does and means my hope is it will sound better. Sure, Altitude.mid plays fully and no crackling the same with high poly audio breakup.mid

Well, to my hears I can barely hear a difference if it is 10 cents or less, but I guess it makes sense to try this linear interpolation but the noise may come back it is hard to know.

Yes, but if I keep the default sample rate of 44.1kHz is my onboard Realtek sound card going to change it to 48 KHz ?

[mrbumpy409]

Testing the cpu-opt branch at commit 530f0e0, I get CPU usage similar to compiling FluidSynth with float samples—perhaps slightly lower:

• FluidSynthPlugin: 7.8%, 7.2%, 7.2%
• FluidSynth: 7.8%, 6.8%, 8.8%

I then updated the cpu-opt branch to the latest commits for lookup table and ran the tests at 44.1 KHz:

• FluidSynthPlugin: 5.4%, 5.4%, 5.4%
• FluidSynth: 5.6%, 5.5%, 5.8%

So definitely better, though still not as performant as 2.3.7 or 2.4.0 and with reduced filter accuracy.

I wonder about something. With 2.3.7, the only pops I ever heard in all of my SoundFont work and composing it turns out were caused by issue #1427. Even when I ran tests with extremely quick filter envelope times at high filter Q, I could not get the filter to pop. It was only once NRPN modulation was able to jerk the filter around abruptly in something like The Nervous Filter that audible popping was noticed.

Would any of the following be worth considering?:

1. Re-evaluate the old algorithm to see if it could provide better results when used with NRPN (and similar) control. What is the modulation envelope doing so special that seemed to make it immune to creating pops?
2. Restore the old filter smoothing algorithm as a "low-CPU" option (or have the new algorithm opt-in, perhaps called something like "safe filter"). It would be understood that the new algorithm eliminates all filter popping at the expense of elevated CPU usage.
3. Only kick in the new algorithm under circumstances shown to cause popping, such as with NRPN control, or something like really fast LFO control (assuming that also creates pops?). Otherwise, use the old algorithm for envelopes, etc. that do not exhibit the problem.

Given the choice, I would personally choose the old algorithm, as the new algorithm is a solution to a problem I never had, and I need to keep CPU usage down for low-latency music production. Either way, I am grateful for all of the work you have been pouring into this project! 😁

[mrbumpy409]

Thinking about this further, if the reduced filter accuracy ends up being unnoticeable, perhaps a lookup table really is the ideal compromise going forward. Does this reduction in filter resolution also affect base cutoff frequency or modulators such as key#-to-filter cutoff? I can think of some cases where reduced filter accuracy might be noticeable, but if it's only affecting the smoothing between two points, then I can't imagine how it would be perceptible, except perhaps with very slow, low modulations at high Q, in which case smoothing probably isn't needed anyway.

Unless I'm not understanding how this all works, which is quite possible. 😆

[derselbst]

It was only once NRPN modulation was able to jerk the filter around abruptly

That's not quite right. NRPN was only the trigger for me to investigate the root cause of these jerks. In the past (2017 - 2019) I played around a lot with modulators and was able to to hear the same clicks and pops back then. Why? Because the old logic is broken by design, so I won't switch back to it, sry.

I do understand that the current lookup based approach is a good way to go, so I'll pursue that one. Pls. also keep in mind that all these exercises only use a single CPU core. We have 2025 by now, so it might be a good idea to set synth.cpu-cores to a reasonable value - possibly fluidsynth should do this automatically at start up... I'll keep this in mind.

---

Does this reduction in filter resolution also affect base cutoff frequency or modulators such as key#-to-filter cutoff? I

To calculate the filter coefficients, we need to calculate sine and cosine depending on the filter's final cutoff frequency, i.e. after modulation has been taken into account, here: (with fres being the cutoff frq. in Hz)

fluidsynth/src/rvoice/fluid_iir_filter.cpp

Lines 51 to 53 in 530f0e0

	R omega = (R)(2.0 * M_PI) * (fres / output_rate);
	R sin_coeff = std::sin(omega);
	R cos_coeff = std::cos(omega);

Instead, now I'm using a lookup table to retrieve the sine and cosine with an accuracy of CENTS_STEPS i.e. 10 cents, to avoid the lookup table from becoming too big, here:

fluidsynth/src/rvoice/fluid_iir_filter.cpp

Lines 80 to 82 in 53ef2d5

	unsigned tab_idx = (fres - 1500) / CENTS_STEP;
	R sin_coeff = sincos_table[tab_idx].sin;
	R cos_coeff = sincos_table[tab_idx].cos;

And I'm planning to use linear interpolation to the next / previous sincos coefficient when filter cutoff is not exactly divisible by CENTS_STEPS.

[mrbumpy409]

It was only once NRPN modulation was able to jerk the filter around abruptly

That's not quite right. NRPN was only the trigger for me to investigate the root cause of these jerks. In the past (2017 - 2019) I played around a lot with modulators and was able to to hear the same clicks and pops back then. Why? Because the old logic is broken by design, so I won't switch back to it, sry.

I am happy with this. Thank you for fixing the broken logic!

I do understand that the current lookup based approach is a good way to go, so I'll pursue that one. Pls. also keep in mind that all these exercises only use a single CPU core. We have 2025 by now, so it might be a good idea to set synth.cpu-cores to a reasonable value - possibly fluidsynth should do this automatically at start up... I'll keep this in mind.

I wonder how much of an issue this is on modern CPUs. My old first-gen i7 has rather weak single-core performance, and I've been able to play MIDI files just fine at low latency and polyphony = 512. With the lookup table method bringing the CPU usage back down so much when the filter is modulated, perhaps it is fine to leave FluidSynth at a single core by default? I'll be able to do proper tests once your latest revisions are complete and 48 KHz sample rate is supported (my primary sound card can't run at 44.1 KHz due to S/PDIF link between cards).

That being said, setting synth.cpu-cores = 2 and reducing polyphony back to the default (256) is enough to completely eliminate audio dropouts in the high poly test on my desktop when using the vectorized build (which I've reverted to so I can run at 48 KHz). I'm thinking single-core usage might still be best when used as a DAW plugin, though? I might have to ask the developer of FluidSynthPlugin (@prof-spock) his thoughts.

And I'm planning to use linear interpolation to the next / previous sincos coefficient when filter cutoff is not exactly divisible by CENTS_STEPS.

I can't wait to try it out!

[derselbst]

I've implemented the mentioned linear interpolation. Proposed by #1494, binaries here. Pls. let me know how it works for you.

The accuracy of the calculated sine and cosine coefficients is actually quite good. The biggest absolute difference between linearly interpolated and truely calculated coefficients I've measured was 0.001010, which doesn't seem to impair audio experience.

[klerg]

Yes, and I recall you said were going to do this soon also. Nice to see the PR for this issue. Sure, Altitude.mid and high-poly-audio-breakup.mid have no crackling but higher CPU usage.

It is good to know all of this is accurate although it does not make sense to me. Well, I have no clue what that number means other than it is a small amount and it is not going to cause crackling or dropouts. But I get a 4% CPU jump in Altitude.mid from 8.2% to 12.3%, and 6% CPU jump in high-poly-audio-breakup.mid from 12% to 18%, is the linear interpolation really necessary ?

[mrbumpy409]

EDIT: I was not paying enough attention to my git navigation, and accidentally tested the latest cpu-opt code rather than the linked pull request. I'm not sure if this makes a difference, so please factor this in when evaluating the test results below. More (and better) test results coming soon.

---

Okay, I have just tested the lookup table with linear interpolation. I have also come to realize that doing some tests at 48 KHz (everything prior to the lookup table implementation) and others at 44.1 KHz (everything involving the lookup table) does not provide properly comparable CPU measurements, as 44.1 KHz does result in slightly lower CPU usage.

With the current latest code in cpu-opt, I get the following CPU measurements at 44.1 KHz:

• FluidSynth Plugin: 7.3%, 7.2%, 7.1%
• FluidSynth CL: 7.0%, 9.5%, 7.4%

This is quite a bit higher than without the interpolation, and seems to be similar to the CPU usage of the previous vectorization method at 48 KHz, which was:

• FluidSynthPlugin: 7.8%, 7.2%, 7.2%
• FluidSynth CL: 7.8%, 6.8%, 8.8%

However, running the vectorized build again at 44.1 KHz shows slightly lower CPU usage than it was at 48 KHz:

• FluidSynthPlugin: 6.7%, 7.4%, 6.9%, 6.6%, 6.7%, 6.6%

Based on an admittedly too-small sample size, it would appear that the linearly-interpolated lookup table actually falls behind the vectorization method. To be sure though, I'm going to re-do all of my previous tests at 44.1 KHz and post back later. To keep things simple, I will only test using FluidSynthPlugin, and will run each test twelve times, discarding highest and lowest value.

[derselbst]

A pull request always contains the latest code of the branch, so don't worry.

You can play around with the CENTS_STEP constant in fluid_iir_filter.cpp and see if it makes a difference. E.g. setting it to 1 would increase the size of the lookup table giving it a datapoint for every single cent (and effectively disable linear interpolation), whereas setting it to e.g. 100 decreases the size of the lookup table only having datapoints every 100 cents.

Btw, CPU usage is good to know, but I'd consider it more crucial that you don't hear any dropouts.

[mrbumpy409]

I have re-done all of my previous tests with the following configuration:

• FluidSynth incarnation: FluidSynth Plugin in REAPER with fx disabled, and other settings at default.
• Audio driver: Pipewire (REAPER accesses this via JACK compatibility)
• Sample rate: 44.1 KHz
• Buffer size: 256
• CPU: Intel Core i7-990x overclocked to 4.15 GHz
• OS: KDE neon User Edition 6.3
• Testing: I ran the test 12 times for each build, recording the highest CPU usage reported by the Plasma system monitor. The highest and lowest values are then eliminated (max one entry each), and the remaining 10 values are averaged for the final score.

Here are the results:

Branch	Build	Description	CPU Usage
master	v2.3.7	old filter smoothing model	3.8%
master	v2.4.0	old model disabled; no smoothing	3.8%
master	v2.4.1	new filter smoothing added	10.21%
master	v2.4.2		9.91%
master	v2.4.3		10.2%
master	current (0461d75)		10.05%
cpu-opt	530f0e0	vectorized filter coefficients	6.69%
cpu-opt	53ef2d5	lookup table w/o interpolation	5.19%
cpu-opt	current (163d0fc)	lookup table w/ linear interpolation	7.02%
master	pull request #1494	lookup table w/ linear interpolation	7.14%
cpu-opt	CENTS_STEP=1	lookup table w/ CENTS_STEP=1	7.11%

Here you can see the results plotted on a chart:

So yes, I can confirm that the latest linear-interpolated lookup table uses higher CPU than vectorizing the filter coefficients. I will update the above chart and graph whenever new tests are run.

Also, I'm not sure if this matters at all for your evaluations, but I also recorded the idle CPU usage in REAPER, in case you wish to subtract it from the results above:

• REAPER empty project: 1.25%
• REAPER test project loaded with FluidSynth Plugin: 1.55%

Update: I have added test results for lookup table + linear interpolation with CENTS_STEP=1.

[mrbumpy409]

I have tested CENTS_STEP=1 and added the results to the above charts. Are you sure this is disabling linear interpolation? The CPU usage is identical to the other tests using lookup + linear interpolation. Perhaps the higher CPU usage is coming from some other code added since 53ef2d5? Please let me know if you'd like me to do a git bisect.