Agreed - 100ms is NOT coming from ASIO. If you can distinctly "hear" the headpones as a seperate sound event from the acoustic sound in the air, then the Haas effect basically dictates you are likely hearing more than 20-30ms of latency. Once you get below 10ms, the "realtime" feel is generaly intact, but I prefer to keep it as low as possible 24/7 (32-Sample ASIO for everything but rendering the final mix).
I know I had numerous stumbling blocks on my way to a fully native monitoring setup. The main one was to be VERY CAREFUL with your plug-in selection on such projects. There are lots of plug-ins that add high (or not so high) amounts of delay, and since most DAW's use PDC to keep everything aligned, this delay renders itself as latency across ALL inputs when monitoring!!!
Also, even plug-ins with smaller 32-sample delays can have their latency effects become "additive" if they are bussed in such a manner that requires the PDC latencies to be offset accordingly. However, I run anywhere from 3-4 Plugs that have 32-Sample Delays and my realtime latency is still very tight.
In Cubase, you can easily get an inventory of all of your used plug-ins in a session: Click on "Devices / Plugin information". You can organize the view by clicking on the "# Of Instances" column, and all of the active Plug-ins will be at the top. Then, reference the "Latency" column to see if you have any latent plug-ins running. I generally move the Latency Column next to the "#" column for easy viewing.
If you see some highly-latent plug-ins, that's your problem. Find a lower latency version of the effect to use while tracking - or just find a low-latency version and sick with it for mixing, too (generally what I do). You can also "Constrain Delay Compensation" in Cubase, but that makes the mix sound like crap IMNSHO (and a good sounding "Wet" mix for the talent is what ITB Monitoring is all about!!!)...
You should be fine with a proper session setup IMNSO (Speaking as someone that doesn't even have a mixer in his studio and NEVER uses Direct Monitoring!!!). The CPU limitations at low ASIO latencies can be an issue - but I counter with cost-effective overclocking of "Bang for your buck" PC components and kick them into the stratosphere of performance for pennies on the dollar! :-) A match made in heaven...
Good luck!
MADIface-XT+ARC / 3x HDSP MADI / ADI648
2x SSL Alphalink MADI AX
2x Multiface / 2x Digiface /2x ADI8