Topic: Crackles at low latency when GPU is used (Madi Fx)

Whenever I play a sound in FL Studio with my external synths, I have heavy crackles every few seconds. They only appear at 32 samples and less at 64 samples, at 128 samples it is 99.9% crackle-free if not 100%. Initially I thought my CPU was doing weird stuff like Hyperthreading or switching power modes. On my other machine with a much older Ryzen 5 I can do 32 samples on a Raydat HDSPe with absolutely no issues.

Then I noticed today that the crackles, though random, are very reliably coming when I click on things like the menu or switch the tabs in the settings page. Can this have to do with a conflict with the GPU?

I also noticed that everything works quite fine after a new boot when I start FL studio quickly, then after a minute the crackling starts.

I've tried switching off fast boot in Windows 11, Hyperthreading, Virtualization and the other bios options, startup items, remove services, fresh install...

Could another GPU help? Older drivers? Can I somehow assign different interrupts to the devices?

Thanks for any hint...

HDSPe MADI FX / Euphonix AM713 converters
13700K CPU
16GB RAM
RTX4090

2 (edited by ramses 2024-11-19 21:49:12)

Re: Crackles at low latency when GPU is used (Madi Fx)

Check DPC latencies (LatencyMon), disable VBS in Win11 (performance hog).
It could also be a problem of the VST/VSTi.
Disable energy saving, CPU core parking.
Use the proper energy profile in Windows (highest or ultimate performance).
Tame background processes / system activity (O&O Win10 Shutup).
Prioritize background processes (not foreground processes), might help to reduce the amount of context switches by using a fix and a little longer quantum for the process scheduler.
If you have nVidia GPU, check for threads in this and Steinberg Cubase forum about powermizer tool to deactivate energy saving on nVidia cards (only recommended for desktops, not laptops) plus a few nVidia driver settings according to the guy on the Steinberg forum.

BR Ramses - UFX III, 12Mic, XTC, M-1620 Pro D, RayDAT, ADI-2 Pro FS R BE, X10SRi-F, E5-1680v4, Win10

Re: Crackles at low latency when GPU is used (Madi Fx)

Thank you!! The LatencyMon already did it. I had the red message, but then went to isboxer.com where I got the registry patch that overrides the CPU throttling. And voila, everything is nice and green the crackles are gone now!

Happy wink

HDSPe MADI FX / Euphonix AM713 converters
13700K CPU
16GB RAM
RTX4090

Re: Crackles at low latency when GPU is used (Madi Fx)

dgstr wrote:

Thank you!! The LatencyMon already did it. I had the red message, but then went to isboxer.com where I got the registry patch that overrides the CPU throttling. And voila, everything is nice and green the crackles are gone now!

Happy wink

Well, that was quick, I'm pleased for you. Have fun with the ‘new’ setup wink

BR Ramses - UFX III, 12Mic, XTC, M-1620 Pro D, RayDAT, ADI-2 Pro FS R BE, X10SRi-F, E5-1680v4, Win10

Re: Crackles at low latency when GPU is used (Madi Fx)

No, you, kind sir, have been quick smile

HDSPe MADI FX / Euphonix AM713 converters
13700K CPU
16GB RAM
RTX4090

6 (edited by dgstr 2024-11-23 14:20:32)

Re: Crackles at low latency when GPU is used (Madi Fx)

A further observation for however is interested:

The crackles stopped in my DAW completely, but I still had a little crackle once every 5 minutes when watching youtube. Also LatencyMon had a little hickup every 10 minutes or so, or at least every 10 Minutes the red message appeared (although the sound was alright all the time).

Today I removed the 4090 and connected the onboard GPU instead. Now all crackles are gone completely and the LatencyMon stays green all the time and I have latency in the low teens most of the time.

All the other settings in the BIOS were totally irrelevant: EIST, Turbo boost, Performance mode, Ram profile, Intel Speed shift, Hyperthreading and so on, no matter if I switched it on or off, it didn't change glitches or crackles or Latency as far as I could see.

HDSPe MADI FX / Euphonix AM713 converters
13700K CPU
16GB RAM
RTX4090

7

Re: Crackles at low latency when GPU is used (Madi Fx)

Yeah, we all love nVidia...

Regards
Matthias Carstens
RME

Re: Crackles at low latency when GPU is used (Madi Fx)

Had similar issues.
Although LatencyMon didn't report any big peaks of DPC latency I had occasional audio drops.
This came from the nVidia card.
I had to disable power saving functions for my RTX 4070 using a tool from a russian dev called "powermizer"
and using 2 settings in the nvidia driver.
And the two settings were
- energy saving -> prefer max performance
- threaded optimization -> on
I found this information on Steinberg Forum and since then, no issues anymore.

BR Ramses - UFX III, 12Mic, XTC, M-1620 Pro D, RayDAT, ADI-2 Pro FS R BE, X10SRi-F, E5-1680v4, Win10

Re: Crackles at low latency when GPU is used (Madi Fx)

ramses wrote:

Had similar issues.
Although LatencyMon didn't report any big peaks of DPC latency I had occasional audio drops.
This came from the nVidia card.
I had to disable power saving functions for my RTX 4070 using a tool from a russian dev called "powermizer"
and using 2 settings in the nvidia driver.
And the two settings were
- energy saving -> prefer max performance
- threaded optimization -> on
I found this information on Steinberg Forum and since then, no issues anymore.

 

I was about to go crazy and return my RME UCX II and I have RTX 3090 running on a server board with AMD USB 3 that is mentioned on RME website direct on an AMD Epyc Milan CPU from cable from the board using a header cabe not soldered on the board and have spent 10 days of reinstalling windows doing every low latency mod that I could find.  Even ordered older Sonnet card arriving in few days.  After reading all good things about RME rock solid drivers for win and mac went for it.  I mean Audio stuff should be compatible with world class workstations for video and 3D and plug and play.  I know this is not RME fault but this info should be on the website and a pdf guide in bold.  I mean its crazy situation with USB audio and GPUs.  As I am reading all this even affecting macs.   I had many other cheaper Avid USB devices and honestly never had these issues before.  Why are RME drivers so sensitive or are they just too honest about latency and settings?  If this powermizer setting dont help I have to move on.

Re: Crackles at low latency when GPU is used (Madi Fx)

Possibly for two reasons: a) RME interfaces transmit more channels, with all channels always being transmitted regardless of whether they are in use or not, and b) RME drivers achieve lower latency, which means the computer’s configuration needs to be perfectly aligned, and USB transfer modes must operate flawlessly.

As for Apple, their track record with USB has been less than stellar. There have been hardware and driver issues that Apple initially ignored or delayed addressing, attributing them to "USB 3rd party devices." These issues were only resolved in the drivers after approximately three-quarters of a year, following coverage in the trade press. A few years ago, during the introduction of a new communication chip, USB implementation problems arose, which could only be resolved by purchasing Thunderbolt docks with standard USB3 chips. Despite Apple’s relatively narrow hardware lineup, it is by no means a "perfect world" either.

However, this issue is not exclusive to RME. For decades, companies have offered turnkey systems that are rigorously tested for professional audio (and video) applications—not just for RME components.

Take a look at device lists that rank how laptops perform in terms of DPC latency:
https://www.notebookcheck.com/DPC-Laten … 375.0.html
In this area, only a few systems fall significantly below the critical threshold of 1000 microseconds, which is necessary to handle near-realtime demands for audio processing. Staying well below this limit is essential to avoid audio dropouts under high loads or with smaller buffer sizes.

The issue extends to desktop systems, which present even greater hardware diversity and driver variations.

Hardware must be paired with drivers that don’t occupy CPU cores for extended periods. Audio processes waiting for cores to become available shouldn’t face excessive delays, as this can lead to audio dropouts. While larger buffer sizes can partially offset this, they also increase audible latency for audio transport over digital interfaces (USB, FW, TB).

No mainstream operating system is a real-time operating system. To ensure data integrity, drivers must not be interrupted by the OS’s process scheduler; only the driver itself can release the CPU after its programmed interval. Unfortunately, not all driver developers adhere to the necessary programming conventions. Since audio is a niche market, these aspects are rarely tested. Manufacturers are primarily motivated by benchmark results, where everyone aims to excel. This often leads to longer CPU occupation times for better scores. While this is not an issue for benchmarks or standard applications, it becomes critical for real-time audio processing.

Fortunately, many users don’t encounter major issues, but with the wide variety of hardware combinations, it’s not guaranteed that everything will align perfectly for real-time audio demands.

This longstanding issue is why, despite increasing CPU performance, companies still specialize in providing turnkey solutions for audio processing. Professional recording studios don’t have the time or resources to address these challenges themselves.

Even for Apple, audio processing remains a niche segment. Otherwise, past issues would not have occurred. Unfortunately, manufacturers don’t conduct thorough testing in this area. For Apple, all recording interfaces are considered "3rd party interfaces" by default, instead of recognizing the reality that USB is a multi-purpose interface. Apple also has a responsibility to ensure that its drivers and USB controllers adhere strictly to USB standards and to resolve issues promptly.

Regarding DPC spikes, I haven’t tested it myself, but I frequently read that nVidia GPUs can cause issues for audio processing. AMD cards may not exhibit these problems to the same extent.

In summary, audio processing with near-realtime requirements can be a challenging field. Even powerful CPUs and good benchmark results are no guarantee of excellent audio processing performance.

For maximum reliability, consider investing in a turnkey system for audio processing from a specialized company.
This all is not RME specific...

BR Ramses - UFX III, 12Mic, XTC, M-1620 Pro D, RayDAT, ADI-2 Pro FS R BE, X10SRi-F, E5-1680v4, Win10

11 (edited by FILMTOOLS 2024-12-31 09:10:47)

Re: Crackles at low latency when GPU is used (Madi Fx)

ramses wrote:

Possibly for two reasons: a) RME interfaces transmit more channels, with all channels always being transmitted regardless of whether they are in use or not, and b) RME drivers achieve lower latency, which means the computer’s configuration needs to be perfectly aligned, and USB transfer modes must operate flawlessly.

As for Apple, their track record with USB has been less than stellar. There have been hardware and driver issues that Apple initially ignored or delayed addressing, attributing them to "USB 3rd party devices." These issues were only resolved in the drivers after approximately three-quarters of a year, following coverage in the trade press. A few years ago, during the introduction of a new communication chip, USB implementation problems arose, which could only be resolved by purchasing Thunderbolt docks with standard USB3 chips. Despite Apple’s relatively narrow hardware lineup, it is by no means a "perfect world" either.

However, this issue is not exclusive to RME. For decades, companies have offered turnkey systems that are rigorously tested for professional audio (and video) applications—not just for RME components.

Take a look at device lists that rank how laptops perform in terms of DPC latency:
https://www.notebookcheck.com/DPC-Laten … 375.0.html
In this area, only a few systems fall significantly below the critical threshold of 1000 microseconds, which is necessary to handle near-realtime demands for audio processing. Staying well below this limit is essential to avoid audio dropouts under high loads or with smaller buffer sizes.

The issue extends to desktop systems, which present even greater hardware diversity and driver variations.

Hardware must be paired with drivers that don’t occupy CPU cores for extended periods. Audio processes waiting for cores to become available shouldn’t face excessive delays, as this can lead to audio dropouts. While larger buffer sizes can partially offset this, they also increase audible latency for audio transport over digital interfaces (USB, FW, TB).

No mainstream operating system is a real-time operating system. To ensure data integrity, drivers must not be interrupted by the OS’s process scheduler; only the driver itself can release the CPU after its programmed interval. Unfortunately, not all driver developers adhere to the necessary programming conventions. Since audio is a niche market, these aspects are rarely tested. Manufacturers are primarily motivated by benchmark results, where everyone aims to excel. This often leads to longer CPU occupation times for better scores. While this is not an issue for benchmarks or standard applications, it becomes critical for real-time audio processing.

Fortunately, many users don’t encounter major issues, but with the wide variety of hardware combinations, it’s not guaranteed that everything will align perfectly for real-time audio demands.

This longstanding issue is why, despite increasing CPU performance, companies still specialize in providing turnkey solutions for audio processing. Professional recording studios don’t have the time or resources to address these challenges themselves.

Even for Apple, audio processing remains a niche segment. Otherwise, past issues would not have occurred. Unfortunately, manufacturers don’t conduct thorough testing in this area. For Apple, all recording interfaces are considered "3rd party interfaces" by default, instead of recognizing the reality that USB is a multi-purpose interface. Apple also has a responsibility to ensure that its drivers and USB controllers adhere strictly to USB standards and to resolve issues promptly.

Regarding DPC spikes, I haven’t tested it myself, but I frequently read that nVidia GPUs can cause issues for audio processing. AMD cards may not exhibit these problems to the same extent.

In summary, audio processing with near-realtime requirements can be a challenging field. Even powerful CPUs and good benchmark results are no guarantee of excellent audio processing performance.

For maximum reliability, consider investing in a turnkey system for audio processing from a specialized company.
This all is not RME specific...


Thanks for the extensive write up. Something should really change in this audio department in an industry wide efforts to mitigate these issues.  After digging some more about my AsRock server board and other boards that serve different workloads in server side of things I see there is an effort to create on a BIOS level workloads profiles so that administrators of servers and those building and maintaining these workloads and systems can at least have a good starting point to load certain profiles.  I see there is one for example for low latency java server  for example.  Maybe something like this for audio can be done.  I understand that it is difficult for everyone to come together and play along and motivations are different but it seems like a really simple solution for many problems.  I want to make this work and I got these latencies under double digits with these profiles and tweaking windows.  Looks like windows server 2019 might be a solution here for these server boards as the OS side of things but I am getting closer and yes Nvidia is the problem with occasional spikes even after turning all bloat off and telemetry stuff and using max power.   But we in video and heavy GPU CUDA that need good audio in a single workstation solution are stuck with Nvidia and Win OS or Rocky Linux.  At least RME is doing a driver that is honest and long supported.

12 (edited by ramses 2024-12-31 11:07:06)

Re: Crackles at low latency when GPU is used (Madi Fx)

There are already companies which are using profiles in the BIOS for different purposes.

A few days ago, I stumbled over this:

https://www.xmg.gg/en/xmg-studio-x/

https://www.dropbox.com/scl/fi/almm0nwp1wqlxlubs16ap/2024-12-31-10_43_57-XMG-STUDIO-X-High-End-Desktop-Workstation-Mozilla-Firefox.jpg?rlkey=uhyvljrruehrxt7u085ehqfzg&st=u7xk82hi&dl=1

BR Ramses - UFX III, 12Mic, XTC, M-1620 Pro D, RayDAT, ADI-2 Pro FS R BE, X10SRi-F, E5-1680v4, Win10

Re: Crackles at low latency when GPU is used (Madi Fx)

ramses wrote:

There are already companies which are using profiles in the BIOS for different purposes.

A few days ago, I stumbled over this:

https://www.xmg.gg/en/xmg-studio-x/

https://www.dropbox.com/scl/fi/almm0nwp1wqlxlubs16ap/2024-12-31-10_43_57-XMG-STUDIO-X-High-End-Desktop-Workstation-Mozilla-Firefox.jpg?rlkey=uhyvljrruehrxt7u085ehqfzg&st=u7xk82hi&dl=1

There are some database profiles for my board that in documentation mention latency requirements under 10ųs.  Will play next with these in the bios and see how it goes.