I don't have a Babyface, although I'd love to have one.
I have a new Fireface UCX, got just a couple of weeks ago.
I normally use a MacBook Pro, but on a single occasion when I hooked it to a spare PC laptop I ran onto the same problem you're describing.

My first piece of advice is: don't lift the ground connection to your PSU! It's dangerous practice, it's a device you have your hands on, and the ground connection is there for a reason: your own safety. You need to solve the problem, not increase your risk.

The second piece of advice is the tiny little thing that saved my day and solved the problem.
It's an impedance adapter box called ZBox manufactured by an American audio interface manufacturer whose 4-letter name starts with letter M and ends with letter U. It sells here in Europe for 50 or 60 €, if mind doesn't play tricks on me. Sorry for the game with the name, I've read forum policy where it hints at not mentioning competing products; you'll find it on the Net quite easily though, on my google.com search it's the 4th link suggestion I get (as of today).

I can't promise it will solve your problem, but in my case it worked like sugar candy: gone is the hum, back is the correct tone spectrum, without the crystal-like, piezo-like high frequencies. Funny enough, the same guitar connected to the same Fireface UCX doesn't show any problem when connected to my Mac.

Hope you find a way to try it and see for yourself if it can cure your problem, too.