Topic: Win10 1903, any experiences ?

I read on recording.de from a guy telling there shall be big latency issues.

Playback of my 400 tracks with 32 samples ASIO buffer size went fine.

But when editing a forum article it went "red" telling audio issues on this system,
because of some very high DPC routine execution time.

LatencyMon v6.7, Win10 1903.

System has been "silenced" by O&O Shutup Win10 on Administrator and user account (no telemetry, background processes for apps, etc etc ...).

BR Ramses - UFX III, 12Mic, XTC, ADI-2 Pro FS R BE, RayDAT, X10SRi-F, E5-1680v4, Win10Pro22H2, Cub14

Re: Win10 1903, any experiences ?

I googled today and there are a lot of references from people claiming that Win10 1903 introduces DPC latencies in the area of up to 1300 microseconds = 1,3ms which is a lot ...

The best appears to be now to delay the upgrade to 1903 and to stay on 1809 for a while until the issues are fixed.

BR Ramses - UFX III, 12Mic, XTC, ADI-2 Pro FS R BE, RayDAT, X10SRi-F, E5-1680v4, Win10Pro22H2, Cub14

3 (edited by Timur Born 2019-06-14 12:44:10)

Re: Win10 1903, any experiences ?

I am still testing. It's rather likely that single drivers cause these, but we will see. The usual suspects like NVidia drivers, DirectX, HDAudio, TCP/IP, NDIS and the like are still the most prominent culprits. Monitoring hardware sensors increases DPC slightly (HWInfo and the like).

Average interrupt and DPC latencies stay below 10 us (more like below 4 us) with C3 CPU C-states being enabled and allowing the NVidia driver to downclock the GPU, peaks can go over 300 us. Disabling NVidia power saving decreases the averages to below 2 us. But that doesn't mean that there still can be spikes, which only long term tests can catch.

I saw single spikes of over 1.8 ms execution time for NTOSKRNL.exe. But these happen on single CPU cores, so how much that affects your systems depends on various variables.

4 (edited by ramses 2019-06-14 12:50:34)

Re: Win10 1903, any experiences ?

Thanks Timur, btw, you can see reports in the net that the 1.3 ms DPC spikes are caused by

ntoskrnl.exe - NT Kernel & System, Microsoft Corporation

One of the platforms collecting informations: https://www.reddit.com/r/Windows10/comm … cy_issues/

This doesn't look to me like the usual driver problem that can happen
and many ppl seem to be impacted by this.

BR Ramses - UFX III, 12Mic, XTC, ADI-2 Pro FS R BE, RayDAT, X10SRi-F, E5-1680v4, Win10Pro22H2, Cub14

Re: Win10 1903, any experiences ?

Upgraded because of the fixed FLS slot limit, and I was in the middle of projects anyway. We'll see.

Re: Win10 1903, any experiences ?

Highest DPC routine execution time (µs):              1907,661667
Driver with highest DPC routine execution time:       ntoskrnl.exe - NT Kernel & System, Microsoft Corporation

The thing is: This does not happen all the time, but only sporadically. So there ought to be something that is triggering it.

7 (edited by ramses 2019-06-14 12:59:09)

Re: Win10 1903, any experiences ?

What looked strange on my system is, that every 1-1.5 seconds the kernel latency timer jumped from 30 to 990
and sometimes to over 1000 microseconds, instantly .. With no load on the system, pure idle, logged in.
High Performance Power Profile.

BR Ramses - UFX III, 12Mic, XTC, ADI-2 Pro FS R BE, RayDAT, X10SRi-F, E5-1680v4, Win10Pro22H2, Cub14

8 (edited by Timur Born 2019-06-14 16:08:14)

Re: Win10 1903, any experiences ?

Here it is only happens every once in a while, more like minutes rather than seconds. So the real cause would yet have to be found.

Re: Win10 1903, any experiences ?

And you also use the measuring method "kernel timer latency" under Tools -> Options ?
I used 6.70, will re-try with 6.71, a new version came out.

BR Ramses - UFX III, 12Mic, XTC, ADI-2 Pro FS R BE, RayDAT, X10SRi-F, E5-1680v4, Win10Pro22H2, Cub14

Re: Win10 1903, any experiences ?

Kernel Timer Latency does not return useful numbers for me. I get numbers over 20000 us kernel times at default timer resolution. Changing the Windows timer resolution decreases the measurement, but also leads to some erratic jumps at specific resolution settings.

Currently I am checking HPET and DynamicTick settings.

Additionally I am checking how CPU load (Prime95 et al) affects this. I just had 1700 us for the NVidia driver and NDIS, but not NTFSKRNL, so there may be some connections. I have yet to check if the Windows process scheduler was finally fixed, which is partially broken since the very first Creators version (when game mode was introduced). Maybe it got worse and thus causes these high DPC measurement spikes for single threads.

Re: Win10 1903, any experiences ?

Tomorrow I'll try recording something in Studio One. I did a quick run and it all seemed to work well, latency as well.

Re: Win10 1903, any experiences ?

bcdedit /set disabledynamictick yes

might do the trick. I am running LatencyMon for hours already and the highest NTOSKRNL execution time was 0.768 ms.

Both the Nvidia driver (2 ms) and DirectX (1 ms) cause high latencies during gaming, even with GPU power-saving being disabled. No idea how that would translate to productivity load yet.

13 (edited by ramses 2019-06-15 08:48:00)

Re: Win10 1903, any experiences ?

Under steam community I found this:
https://steamcommunity.com/groups/TheUs … 082070040/

This does not help
bcdedit /set disabledynamictick yes

This in addition also not
bcdedit /set useplatformclock No

In the BIOS I deactivated HPET and this also doesnt help

Still getting this peak on each 3rd display of values:

https://www.dropbox.com/s/svrsbgq6vo8j931/2019-06-15-Win10-1809-LatencyMon-Spikejpg.jpg?dl=1

And when disabling HPET in the BIOS, then the kernel latency timer values in Win7 are now constant at
around 975 microseconds.

I reverted the BIOS change to re-enable HPET for my main platform Win7 and currently there seems to be no solution yet to see again the better measuring results under Win10 1903.

I am wondring anyway what the guys are doing there *sigh* .. changes for the changes sake ?
Where are the advantages for the end customer ..

BR Ramses - UFX III, 12Mic, XTC, ADI-2 Pro FS R BE, RayDAT, X10SRi-F, E5-1680v4, Win10Pro22H2, Cub14

14 (edited by Timur Born 2019-06-15 10:16:35)

Re: Win10 1903, any experiences ?

Well, under 1000 us for NTOSKRNL is better than up to 2000 us at least. I still get over 1000 us, though. So testing is still in progress.

And for me it is still only a single thread/core, so depending on your CPU it may not be a real problem in practice. Keep in mind that the main ASIO driver thread only runs on a single (different) core anyway.

I will keep testing after I am finished updating all drivers for 1903. Curiously some Intel drivers from Gigabyte are more recent than Intel drivers from Intel (RST, Chipset), other are more recent from Intel (WIFI, BT, LAN). So it's a matter of matching and mixing them.

Re: Win10 1903, any experiences ?

I have the same problems described by all you guys. Kernel problems all the way. my system blasts 1000 us and once in a while it would jump to 1350 us latency.
I have a Ryzen 2700x, 32 gb ram, Asus ROG Strix x470 with mvme m2 disk and tried both Nvidia and AMD graphics cards.  same problems. The AMD card is better for latency then Nvidia like by 40 us according to LatencyMon.

When I run Latency Checker it blasts out 999 us all the time and my latency was around 1020 with spike up to 1350.
I found an old tool called "TimerResolution.exe" - basically I run that first of all, press the maximum button. then I would run Latency Checker again and it would now blast out 499 all the time. BUT my latency did fall to numbers between 15 and 24 us in latency. Every 15-30 seconds I would see a spike to 499 us but I can run Cubase with 2ms latency no problem with my RME pci on an pci-x to pci adapter. Projects I couldn't even play on 28 ms with asio guard now plays fine on 2ms with this TimerResolution tool.. just google it to download and try for yourself.

having said that, I'd rather have a super stable low latency version where I didn't have to run this app first and didn't get any spikes every so many seconds.

Clearly there is a windows problem somewhere. I suspect some of it is connected to usb3 but it certainly is not the whole solution. I hope someone will find a solution

16 (edited by Timur Born 2019-06-17 00:30:21)

Re: Win10 1903, any experiences ?

Well, things are a bit more complicated and I am not convinced that the measured latencies are "real". There are several different scenarios at play.

1. Device drivers seem to affect the measured latency of NTOSKRNL, on my system the NVidia driver seems to have the biggest impact, though. Turning off various devices via Device-Manager and in addition to some services decreased the *long term* measured latencies of NTOSKRNL.

https://i.postimg.cc/HxXPVbJw/1903-NTOSKRNL-latency.png


2. There are times when the measured system latency is reported higher than any single process/driver latency, including NTOSKRNL's latency. This alone points to measuring issues.

https://i.postimg.cc/mr05LmrZ/1903-nodriver-latency.png


3. Then there are times where small spikes of NTOSKRNL are *not* measured as general system latency. This is what I would expect on a multi-core system anyway.

https://i.postimg.cc/WzVYrtrQ/1903-NTOSKRNL-no-latency.png


4. Then again there are larger spikes of NTOSKRNL that *do* measure as general system latency, despite my system using 16 logical cores that are all just idle and free to service DPCs.

https://i.postimg.cc/BbCYCsBt/1903-NTOSKRNL-latency-nvidia.png


The latter is what other people report here, but there is more to it. Notice the latency spike measuring around 1390 us? I can reproduce this same number every time I restart LatencyMon, but this is only the *first* large spike being measured. If I let LatencyMon run for hours I get spikes well over 3000 us, but as soon as I restart the measurement it's back to those 1390ish spikes and over time it increases over 3000 us again.

This is rather unusual and again might point to a measuring error rather than a real problem. Furthermore I was not able to measure any increase in CPU load or CPU context switches of NTOSKRNL while these supposed spikes happen.

What's next is to test if any audio dropouts even happen at lowest audio buffer settings when these "measured" spikes occur.

17 (edited by Timur Born 2019-06-17 00:56:08)

Re: Win10 1903, any experiences ?

As it turns out I got an even bigger spike some time during my current measurement run. Of course there is DirectX in the mix again, too.

https://i.postimg.cc/hv50PjDF/1903-NTOSKRNL-latency-nvidia-6000.png

Re: Win10 1903, any experiences ?

And now it's at 8200 us. I will let the PC run over night to see how much more the value increases.

Re: Win10 1903, any experiences ?

I assume Microsoft did something wrong. I am using the same HW now since years on Win 7 and 10.

BR Ramses - UFX III, 12Mic, XTC, ADI-2 Pro FS R BE, RayDAT, X10SRi-F, E5-1680v4, Win10Pro22H2, Cub14

Re: Win10 1903, any experiences ?

after the update my pc was always working harder due to a service called delivery optimazation..a was having a 12% cpu use in idle...which is insane for an 8th gen i7...i had to manually turn off this service and everything is back to normal

Re: Win10 1903, any experiences ?

Stayed at 8200 over the night, but the yesterday the increase to 8200 happened while actively using Firefox, so there might be some connection. I will test audio performance vs. reported/measured latencies next.

22 (edited by snoskit 2019-06-17 15:25:29)

Re: Win10 1903, any experiences ?

Microsoft are aware of the issue(s); are requesting reports via the feedback hub (gearslutz thread)

1903 is loaded with tweaks to background services, so I'd guess going to take a few months to iron out wringles

HDSPe AES + HDSPe RayDat + HDSPe MADI
ADI-2 mk.1 x2 + other converters
Cubase + VEP

23 (edited by Timur Born 2019-06-18 13:37:56)

Re: Win10 1903, any experiences ?

This is not entirely correct. As far as I understand, there is one Microsoft guy on the Gearslutz forum is aware that people are reporting high LatencyMon numbers. He is not convinced that this is not a LatencyMon problem and is now waiting for reports of audible glitches that have not happened before the Windows update.

24 (edited by christianh 2019-06-18 23:09:41)

Re: Win10 1903, any experiences ?

I am experiencing the same issue with my Babyface Pro and latency problems with the lowest setting however i did not notice them until after the Windows updates a few days ago. Since the Win10 1903 update been out a while I can't say for certain if it's caused by it but I do experiencing audio clicks which resulting in a lot of buffer size errors. It's caused by any program that uses the audio interface and there are no audio interfaces enabled beside the RME Babyface Pro.

I have never experienced this since i bought the BF Pro several years ago. Something is not right and i I am pretty sure it's not a hardware problem.

25 (edited by loopman 2019-07-30 07:12:05)

Re: Win10 1903, any experiences ?

I installed a new update today, called KB4505903 on windows update, my latency problems/spikes seems to be gone. I would get spike at latest around 3 minutes. Now after 12 minutes still running smooth without spikes smile and with a little lower latency overall

Re: Win10 1903, any experiences ?

I have an issue with Kontakt that didn't happen before 1903. At 48k, 128 samples ASIO buffer, things are clicking in certain cases which never happened before. I have a FL Studio project using Spitfire Chamber Strings that showcases the issue. A friend of mine also reports the same thing happening with 1903, he's using UCX (also latest RME drivers). Didn't happen in 1809 at all.

If I send the same MIDI data to Kontakt standalone, it also happens, so it's not a DAW thing. It's also not a USB thing because I connect my UFX+ via Thunderbolt.

Most interesting thing: if I switch audio output to WASAPI or DirectSound, no clicks at all... Also clicks don't happen in rendered audio, it's "just" a realtime thing.


So I'm not sure if this is MSFT messing something up, or maybe RME drivers needing some sort of an update to make things work better with 1903.

A very happy RME UFX+ & ARC owner smile

27 (edited by ramses 2019-08-29 10:36:45)

Re: Win10 1903, any experiences ?

Windows updates might revert some optimizations which you did in the past.

Did you try O&O Win10 shutup? Some of its settings (deactivations) might help reducing some unwanted background activities (also supports in terms of regaining more privacy).

It might also help to stop the active elements of start menu.

Did you optimize the process scheduler to priotize background tasks? I.e. the ASIO driver should benefit from it.

If you use cubase and have a cpu with high amount of cores/threads then you might ask Steinberg for a registry change as Win 10 has a limitations in terms of the max number of priotized real time processes / threads (MMCSS if I recall this right).

For 6 cores / 12 threads all is fine but I think starting  with 8 cores / 12 threads you could slowly run into issues.
The final solution was that Steinberg would try to fix this in later versions of Cubase.
I read about this when Cubase 9 was out.
I do not know the actual status as I still prefer Win 7 for stability and privacy reasons.

BR Ramses - UFX III, 12Mic, XTC, ADI-2 Pro FS R BE, RayDAT, X10SRi-F, E5-1680v4, Win10Pro22H2, Cub14

28

Re: Win10 1903, any experiences ?

FWIW, I had read not long ago about a kernel performance problem in 1903 for audio, which is not yet fixed. Just can't remember where...

Regards
Matthias Carstens
RME

29 (edited by EvilDragon 2019-08-29 12:07:13)

Re: Win10 1903, any experiences ?

I don't use Cubase, I mentioned FL Studio, but also Kontakt standalone itself. I use Reaper mainly - for some reason the clicks that happen in FL Studio project don't happen there with Spitfire Chamber Strings (but they do happen with a different instrument). However in standalone it's still an issue. CPU here is i7-6700K running at 4.5 GHz.

All the optimizations I did were retained, I checked (I didn't do that many of them in the first place - yet 1809 worked flawlessly!). Also the process scheduler is set to background services. I am not overly bothered with telemetry, I do shut off some of it but not all. I like some of the features W10 has, along with Cortana etc.



MC wrote:

FWIW, I had read not long ago about a kernel performance problem in 1903 for audio, which is not yet fixed. Just can't remember where...

Maybe here?

https://www.techpowerup.com/258611/wind … ged-to-fix


I did that tweak suggested in the article, it didn't help.


I cannot revert back to 1809, 30 days have passed since I upgraded to 1903.

A very happy RME UFX+ & ARC owner smile

Re: Win10 1903, any experiences ?

Can't you simply restore an older disk image and simply deny upgrades for even longer by using O&O Win 10 shutup's options to avoid upgrades for even longer ?

Maybe it also helps to define ie the time window for working hours longer from 5 am to 4 am or something like this to prevent Microsoft to upgrade in case you do not want?

Get Macrium Reflect Home, this is an excellent backup software based on full, differential and incremental backups. Gives you a lot of reliable backup/restore possibilities.

I created ie different Golden Images for Win7 over the years.
For Win10 I am using the upgrade installation which runs very well.

By this I have a very good basement on a stable Win 7 and can change to Win 10 for comparison and benchmarking purposes.

Maybe you can use Win 7 in a similar way for your workflow to ie use it when Win 10 has issues.

At least I recommend to create a golden image that is known to work containing all required applications.

From time to time I restore a golden image to get rid of garbage in the registry from trying new software, then upgrade some applications to known working higher versions and create then a new golden Win7 image.

As newer HW doesn't support Win 7 anymore your golden image might be a proven Win10 installation with network being disabled,  so that Microsoft can not enforce any upgrade.

On a second SSD "for production" you can upgrade this installation to the latest Win10. If it runs well create a new golden image but still keep the previous for safety reasons.

Tip: put your user data to a different disk. The golden image only should contain OS, applications and sample libraries, so that everything matches with Windows registry.
By this those golden images do not become too big.

BTW... Macrium Reflect Home is very fast in restoring backup images because of it's Rapid Delta Restore mechanism. Only those blocks on disk are being changed that need to be changed.

This results in high speed and low wear on SSD  as you have very few write cycles to SSD by this.

I can restore my 1 TB Samsung EVO which is filled to approx   60% in around 10-12 Minutes to the status of last week or when restoring a golden image. Where boot time alone takes quite long because of the long lasting BIOS POST checks of my server board...

So for a normal mainboard with consumer components it will be even faster offering more than excellent turn around times to change to a proven image.

You only need to take care of few things manually. Save ie your Firefox bookmarks before restoring an older image and reload the backup of the bookmarks afterwards.

Some other software might require some additional manual tasks for yoUR backup and restore procedure. Like ie application defaults for Cubase and other applications.

Besides using Macrium for the boot ssd also containing all applications, I am mirroring my users profiles (c:\users) and my user data on k: and r: (dedicated ssd for recording) by using a mirroring called Freefilesync which is reliable and performant and even open source.
As automatic job I execute mirroring on a daily basis.

By this I can easily restore application defaults for certain applicatiobs as needed after a restore of a golden image.

These are the building blocks, I am sure that you can work out something similar to have more options in case of trouble with Windows to come back to work in less than 15 Minutes.

BR Ramses - UFX III, 12Mic, XTC, ADI-2 Pro FS R BE, RayDAT, X10SRi-F, E5-1680v4, Win10Pro22H2, Cub14

31 (edited by EvilDragon 2019-08-29 14:42:45)

Re: Win10 1903, any experiences ?

I don't have an older disk image. Consider that I have about 10 TB of all sorts of data, holding backups of that all would be too expensive for me to get into, as I'd need much more drive space to back up all that (even with just backing up deltas)...

Also, I'm using W10 Pro, so it wasn't a forced upgrade, I delayed it for 6 or so months before updating, and only started noticing these issues when starting to work on a project, but that was over 30 days after I updated, so I cannot revert back to 1809 as mentioned. Also the nature of my work requires network access, so having W10 without networking is not an option. And as I said, it wasn't a forced upgrade, I can delay it quite a lot with W10 Pro.

I cannot use W7 due to some other factors, plus I really don't like it all that much anymore, plus considering it's not gonna have any more updates past 2020 makes it an absolute no go for me. smile



However what I would like to know here is can RME duplicate these issues that I have with my UFX+ on 1903. I can provide the FL Studio project that clearly showcases the problem, but the requirement is to have Spitfire Chamber Strings and Kontakt (and FL obviously)... I have another library that clearly showcases the issue but it's not yet released (it will be cheaper than SCS though, and will work on the free Kontakt Player)...


Maybe there's something they could do with their drivers that could work around these sideeffects that 1903 introduced, I don't know. I'm grasping for straws here. Things are manageable if I increase the buffer size to about, but then forget live tracking or playing through plugins (I use guitar ampsims almost daily)...

A very happy RME UFX+ & ARC owner smile

32

Re: Win10 1903, any experiences ?

We can not reproduce these issues. I also see no chance in fixing this within the driver.

But you still have some options left to try - the old school stuff. Optimization for background processes or not, CPU priorities for audio/ASIO threads, MMCSS on/off in driver and application. One of these might make a difference.

Regards
Matthias Carstens
RME

Re: Win10 1903, any experiences ?

All of those things are set as they should be, and as they were in 1809 where it worked perfectly.

Reaper is already set to MMCSS priority.

A very happy RME UFX+ & ARC owner smile

34 (edited by ramses 2019-08-30 07:25:29)

Re: Win10 1903, any experiences ?

Eventually try bitsum's process lasso.

BTW there are ways to make disk images smaller to make it better manageable.
It's a matter of organizIng your data.

I have only os programs and sample libs on a 1TB SSD.

All my recordings are on a separate dedicated ssd of 512 GB.

My bulk/user data is on a separate 3TB HD with 7200 U/Min.

Backups I perform to 1 internal 10 TB harddisk and to 1 external 10 TB harddisk.

Shall you have so many sample libs, move it to a separate disk and leave it out from disk image.

Shall you have so many recordings, put it on a separate disk as me.

By this you should be able to get a compact compressed disk image like me from the main boot ssd of under 500 GB in size which is easily manageable and quickly restore able by Macrium Reflect.

Networks could also be broken up into an internal and external one to internet if you have mainboard with 2 lan ports. Otherwise try to use perhaps smth like a proxy in front of Internet blocking access to ms upgrade services as good as possible although this might be challenging.

At least for so long as the issue persists once it's fixed you don't need this anymore.

But you need a second SSD as test installation not too loose too much time with rbackup restore.

BR Ramses - UFX III, 12Mic, XTC, ADI-2 Pro FS R BE, RayDAT, X10SRi-F, E5-1680v4, Win10Pro22H2, Cub14

Re: Win10 1903, any experiences ?

Well, not exactly a solution for my problem since I don't have an image to restore to, nor can I go back to 1809. Reinstalling W10 is not an option, I have work to do.

A very happy RME UFX+ & ARC owner smile

Re: Win10 1903, any experiences ?

You will need to find working mechanisms / solutions to protect your business.

Look back how long it took i.e. for Apple to fix issues in Mojaves USB stack (roughly 9 months) where all vendors of recording interfaces were impacted and where the fix could only come from Apple where the issues originated.

Similar things can also happen with Microsoft esp. because they are introducing so many changes to Windows 10 as we all know and there were all kinds of trouble involved up to complete loss of user data with the 1st version of 1809.

I personally regard it as too careless to base a business on computers without any tested backup and fallback procedure.

I understand that you are busy, but I can't recommend to take this much longer as an excuse for not fixing your IT ifrastructure in terms of installing the previous working Windows version newly with all required safety belts.

Get the components now and build up a parallel installation of old Windows 10 with working backup and restore procedures.

All old Win 10 ISO images are still available free of charge. There is a Microsoft application which allows creation of an USB Boot stick, needs 8 GB USB stick.

BR Ramses - UFX III, 12Mic, XTC, ADI-2 Pro FS R BE, RayDAT, X10SRi-F, E5-1680v4, Win10Pro22H2, Cub14