There is no way around it, for lower latency the buffers must be smaller. If you also play the VEP slave, it's latency must be low too. Of course I don't know your machines, but here I can run at very low latencies. Most of the time at 64 and 128 in very busy projects. I don't use VEP slaves much but they can run at 1 buffer setting, when I was testing. My machine is nowhere as fast as yours! A simple core2 quad from maybe 6 years ago.
Running at low latencies where other brands fail is RME's forte! Just set your VEP at one buffer and start lowering the AIOs latency!

Edit, I just saw your machines specs and you should be able to run even at 64!

Please also check if ticking MMCSS to on in the RME driver (last tab) improves performance allowing lower latency.

Just tried turning the mmcss on in rme driver. Unfortunately it didnt improve permormance.

Yes i am on windows 10. I will try disabling ht next, will post results tonight;)

UPDATE:

I disabled the Hyperthreading in BIOS nad it did improve the overall vst-performance in cubase. SO WE ARE GETTING CLOSER HERE

Now i can lower latency in rme driver to 512 samples without any dropouts. Tried 256 and 128 but will get crackling and dropouts when there is a whole lot of midi tracks playing at the same time (whole lot is about 60-70 at the same time, when its just 20-25 its all good).

Also tried removing the mastering Ozone plug from the stereo output track, but to be honest I didn't feel that it made much difference.

So now i feel that the bottleneck might be that I have 40 instances of VE PRO loaded up in the "instruments rack" in Cubase and of course in VE PRO server on my SLAVE. I am pretty sure when working in CUBASE you are only allowed to have 16 midi track per each rack VSTI instrument.... so i have 40x16which 640 midi tracks.... now I wonder i am wrong about this... Is is possible to have the same 640 midi tracks coming from the VE PRO server to be held on less amount of instances? I read somewhere that you can have up to 16 kontakt instances with 16 instruments loaded in each on 1 INSTANCE of VE PRO. (instanception pun here lol), but as many time I tried to do that I just end up with 16 midi track limit from each instrument loaded in the rack in Cubase.

Yep I am using Vst3 version of ve pro. But still i dont know how to have more than 16 midi tracks per vsti instrument

Are you sure the slave can handle the processing? It is a different machine yes? For me it is just as easy to overload the main as the slave machine.

UPDATE:

Just messed with my template so I only have 4 instances of VE Pro instead of 40 of them.... but still cant go lower than 512 samples latency.

Any other recommendations? I wish there would be just one complete post or a video that goes about how to set it all up

Hi!

I am on Windows 10. Yes I have all the C states turned off, hyperthreading turned off, EIST off. I am overclocking the cpu from 3ghz to 4.4ghz so I cant turn off the Intel Turbo Boost. (I tried but it goes back to 3ghz and it drops the performance by a very noticable amount).

Just ran Latencymon. Average latency was 1.79 us and highest latency was 68.28 us. Here is a screenshot:

Also downloaded the cpu parking tool from the link you sent me. Still not sure how it works:) did you use it to decrease latency?

I also get very bad hard pagefaults readings... 663712us is the highest coming from runtimebroker.exe maybe that might be an issue. I will try posting on the latencymon official forum and see if I can get help there.

At least on Win7, most likely on 10, Windows parks CPU cores, which needs time to wake up cores again.
On the other hand a certain amount of cores need to be parked otherwise some of the cores cant reach full turbo speed.
On the other hand No, 2: also turbo speed changes the clock of the CPU, which also takes time.
So .. I think what you did is in total the best. Although I am not a fan of overclocking.

UPDATE:

Thanks guys for the help.

I was able to improve the performance. Now I can go down to 256 samples latency in a super heavy all audio tracks project with a whole lot of vsts and two ozone mastering plugins. And I can go to 256 samples latency on my orchestral template with 640 midi track and 640 output tracks for those with a mastering plugin on stereo out track.

What helped eventually:

So I followed your advices one by one: turned off the hyperthreading, turned off the Speedstep and C States in Bios. Turned off a whole lot of useless background services (printer, fax, telephone, smart card, and etc), turned off the windows defender and the firewall (since the master pc doesnt have an internet connection, there is nothing to defending from). I updated the network and gpu drivers. Also turned of the cpu parking thru the regedit ( I found a youtube video on how to do it and it worked). And now it runs almost perfect haha but still a great improvement from barely pulling on 1024 samples latency to 256 and no crack no pops, vst performance in cubase is at 20-30 with rare random peaks at heavy loaded parts.

Now, something I forgot to mention in this thread is that I 2 reverbs running in the orchestral midi project. 1 is the Vienna Mir Pro that creates realististic placement for instrument sections (violins, vilos, basses, trombones and etc) and the second is AltiVerb 7 that i have set up as an fx send to group busses. Should I try and move those to my slave pc as now i have only instances of ve pro and i can fit the Vienna Mir in there and use one instance of alti verb per instance as a send fx. Do you guys think that will help me reduce the latency buffer to at least 128 on my master daw pc?