Unless you get really "Fast" AD/DA converters, the UCX is likely to edge out the RayDAT with a "slower" AD/DA converter since it (the UCX) has an extremely low-latency AD/DA section.
Edit - Realized you asked about the UFX (not UCX) - The UFX uses "slower" converters than the UCX, so it will be on par with a PCIe setup with equivalent converters (UFX might have a bit more latency than PCIe if you can't squeak out the UFX's lowest USB Latency settings on your PC). I personally have NEVR owned a Firewire or USB interface - I'm 100% PCI/PCIe from day one, and I've been nothing but thrilled with that format overall.
I don't worry a whole lot about conversion latency as long as it isn't much more than 1ms or so, but it can make a difference.
I've been basically doing what you want to do for over 4-5 years now using RME PCI/PCIe Hardware and all different kinds of AD/AD's (currently using SSL Alphalnks, but have used everything from Multiface/Digiface and ADI-8 Pro's, to Presonus Digimax-LT's and Focusrite Octopre's, and now the SSL's over HDSPe MADI). No mixer in sight - and this system is also used to drive the PA and Headphones for band rehearsal w/o any Latency concerns in the slightest (all "wet through the DAW" with effects and all).
I do run at 32-Samples on the RME Driver 24/7, so some of the PC's real-world horsepower is sacraficed to keep the latency that low - but the Sandy/Ivy Bridge CPU's of today are still very capable performers at these latencies IMO. Overclocking never hurts, either (when done correctly and proven to be stable under all circumstances). Even an older Core-2-Duo (4.3GHz e8600) was able to give me lots of VST processing @ 32-Samples 24/7...
Native ITB Mixing/Tracking/Monitoring is the bees knees for me and my somewhat unconventonal Studio setup. Total Recall of every possible setting is bliss...
MADIface-XT+ARC / 3x HDSP MADI / ADI648
2x SSL Alphalink MADI AX
2x Multiface / 2x Digiface /2x ADI8