harmonic distortion when using 7th order interpolation

[mrbumpy409]

FluidSynth version

2.4.3

Describe the bug

FluidSynth allows changing the sample interpolation method to balance sound quality vs. CPU usage. This is usually done via FluidSynth's console, using the interp x command, where "x" is one of the following options:

x	Type	Description
0	None	No interpolation: Fastest, but questionable audio quality.
1	Linear	Straight-line interpolation: A bit slower, reasonable audio quality.
4	4th Order	Fourth-order interpolation, good quality, the default.
7	7th Order	Seventh-order interpolation.

In theory, 7th order interpolation should provide the best quality of these options. However, while FluidSynth's 7th order interpolation does provide cleaner interpolation of high frequency content, it also can introduce significant harmonic distortion. This is really noticeable on samples of low-pitched instruments such as electric bass. In these cases, 4th order interpolation provides a superior audio quality.

Steps to reproduce

1. Download and extract sample interpolation test.zip.
2. Load the SoundFont into bank 0 or 1 and play the MIDI file using FluidSynth's different interpolation settings. The MIDI file will play four different tests.

Included in the zip file are audio files of the test using FluidSynth's linear, 4th order, and 7th order resampling methods, as well as BASSMIDI set to linear and sinc interpolation (rendered at 48 KHz using FluidSynth Plugin and BassMidi VSTi). These files are time and volume matched so you can easily compare them in an audio editor. I recommend using Audacity with the waveform view set to "spectrogram".

Test 1: Sine Wave Sweep

The first test plays a descending sine wave sweep across most of the audible frequency range. With the ideal interpolation and sample, you should only see a singular, prominant frequency sweeping down. Here is a spectrogram view comparing BASSMIDI and FluidSynth's interpolation methods (the synthesizer and interpolation method are written at the top of each clip):

The linear interpolation renderings look (and sound) almost identical between BASSMIDI and FluidSynth, with FluidSynth being a slight bit noisier. Both feature significant harmonic distortion that can be seen oscillating up and down in an arch-like pattern.

Comparing the next higher quality interpolation (BASSMIDI's "sinc" and FluidSynth's "4th order"), you can see the harmonic distortion is now barely audible, with BASSMIDI's algorithm once again being a bit cleaner.

Now observe FluidSynth's 7th order interpolation, which has the same harmonic distortion as the linear interpolation examples. This should not be happening with what is supposed to be a superior interpolation method.

Test 2: FM Electric Piano

The second test plays high samples of an FM electric piano moving up and down chromatically. With the ideal interpolation, all frequencies of the note should move together up and down with the notes. Here it is in the spectrogram view:

Of these renderings, only BASSMIDI's sinc interpolation is perfectly clean. All of the other renderings feature varying levels of harmonic distortion. Here, FluidSynth's 7th order interpolation is notably cleaner-sounding than 4th order or linear, but is no match for BASSMIDI's sinc interpolation.

Test 3: Electric Bass

The third test plays an ascending scale of electric bass notes. Here is the spectrogram view:

Here, something strange occurs. Both synthesizers' linear, sinc and 4th order examples sound pretty much the same and are free of artifacts. However, FluidSynth's 7th-order interpolation is creating clear high-frequency distortions which can be seen as horizontal lines sustained through most of the bass notes. These distortions create an audible whine in the sound that is not present using any of the other interpolation methods.

Test 4: Low Quality Timpani Sample

This test attempts to see how an interpolation method handles low-bitrate samples. Low quality interpolation methods will result in audible high-frequency aliasing noise. Here is the spectogram:

Both linear results show the high frequency aliasing noise quite clearly, and this can be heard as a grainy, ringing sound. FluidSynth's 4th order interpolation cuts down on this noise quite a bit, but does not completely eliminate it. BASSMIDI's sinc interpolation completely eliminates the aliasing noise. FluidSynth's 7th-order interpolation seems to have cleaned up most of the aliasing noise, but once again features the same type of high frequency "whine" distortion we saw in the electric bass example.

Expected behavior

A high quality sample interpolation should not be producing these harmonic distortions. Unfortunately, this means that FluidSynth is not currently capable of truly high quality sample playback—at least in terms of what is expected out of a sampler used in professional music production. In many examples, the 7th order resampler sounds worse than 4th order and in some cases even worse than linear.

[derselbst]

I touched this code for 2.4.3 and was a bit worried that it's a regression, but doesn't seem so. However, 2.3.7 also shows some of these ringing artifacts for the sine, but they are much more silent and more similar to bassmidi. Can you confirm? Is that the behavior that would please your ears? If so I would need to investigate what had caused this as I have no clue right off. Here're the renderings: sample interpolation test2.zip

---

I prefer Sonic Visualizer btw ;)

[mrbumpy409]

However, 2.3.7 also shows some of these ringing artifacts for the sine, but they are much more silent and more similar to bassmidi. Can you confirm?

Your 2.3.7 render appears to have been rendered with 4th order interpolation, not 7th. It matches my 4th order rendering perfectly, with any differences (mainly in the second test) being attributable to the fact that your renders are at 44.1 KHz and mine are at 48 KHz.

I prefer Sonic Visualizer btw ;)

I'll have to check it out. 🙂

[derselbst]

Your 2.3.7 render appears to have been rendered with 4th order interpolation, not 7th.

Oh, indeed. I successfully fooled myself. I can now reproduce this behavior even with 2.0.9. This seems to be a problem in the legacy DSP interpolation code. I don't know where though. I went through the sinc code multiple times and the logic looks fine to me...

[derselbst]

I'm trying to understand what's going on and played around with several different window functions. We currently use a Hanning window, which actually yields the best results in my tests. Using different windows (Hamming, Lanczos, Kaiser) make the ringing artifacts sound different, usually more dominant. I still cannot spot an obvious error in the interpolation algorithm itself, which is why I currently believe, that using 7th order is not enough. I dug around in libsamplerate's code a bit but couldn't find which order their sinc filter uses, yet looking at their coefficients I suspect it's a much higher order (340239 coeffs for them, 1792 for us).

Depending on your relationship to the BASSMIDI guy(s) it might be helpful to know a bit more about their approach on sinc interpolation, esp. which window function they use, and the order of their filter (i.e. how many samples are taken into account when interpolating).

I could try to increase the order of the sinc interpolator. But fluidsynth's current implementation doesn't allow this to be done without duplicating lots of code every time it's increased.

[mrbumpy409]

I am working on recreating my test in two open-source applications: sfizz and OpenMPT. I have no idea what interpolation sfizz is using, but I know OpenMPT has some excellent and highly performant interpolation methods to study. Since I'm away on a ski trip right now, it might take me a bit to get the tests completed.

[mrbumpy409]

I have uploaded a new version of my sample interpolation test that works for both SF2 and SFZ synthesizers. The README.md file at that link documents a lot of my findings, but in summary as it pertains to FluidSynth: The sinc interpolation used by sfizz when interpolation level is set to 5 is probably the ideal high-quality interpolation to emulate, in terms of balance between sound quality and CPU usage. Level 6 would only be slightly better, and above that there is no positive difference in any of my tests.

My test documentation provides a more complete picture comparing the interpolation methods of FluidSynth, BASSMIDI and sfizz.

[derselbst]

Thanks! Sounds like your ski trip has been very productive ;)

Sfizz interpolation level 5 is equivalent to a 16 point sinc interpolator, level 6 to 24 point interpolator:

https://github.com/sfztools/sfizz/blob/a708acbe8e82805d60f084d2c3fce64abed1280d/src/sfizz/Voice.cpp#L1444-L1455

This is much more than the 7 point interpolator that fluidsynth uses. I'll think about what can be done: Either changing fluidsynth's current implementation, or adopting the sfizz implementation. I do like the sfizz interpolator. It looks very mature, clean and even uses SSE2. I'll need to check license compatibilty and other implications like filter and volume processing...

[mrbumpy409]

Interesting. It looks like sinc level 3—which had some of the same distortions as FluidSynth's 7th order, but to a lesser degree—is using an 8-point sinc. If you wanted to keep an interpolation level between 4th order and 16-point sinc, perhaps replacing the 7th order interpolater with 8-point sinc might be a good idea. It could be accessed via interp 8, but also have interp 7 point to it for compatibility with existing workflows. And then the 16-point sinc would be interp 16.