Whatever it is, you can passthru it to every output just do the routing in Totalmix

Assumption of course.
Why should one passthru a signal from optical in to optical out thru a audio interface ? A simple toslink cable can do this.

What is connected to the optical input port of the Babyface ?

Is your Dolby Digital signal stereo or surround ?

btw :

Uncompressed surround sound audio is not supported by digital (S/PDIF) connections due to bandwidth limitations. When your lossless 5.1 or 7.1 audio content is sent through the Digital S/PDIF connection, data will be lost which can result in limited 2-channel audio playback.

If I connect the receiver directly to the source, it works.
If the connection passes through the babyface, it doesn't.
Faders are at 0db throughout, no processing of any kind enabled on the spdif in/out.

I don't suppose the BF strips the non-audio flag?

Been trying some more stuff this morning. Brought in a second receiver for sanity check too.

Both receivers work as expected when directly connected to the source.

Receiver A (the one I brought in today) works when passing the signal through the babyface as well.

Totalmix sees the signal as broad spectrum static (as expected) but the receiver connected to the spdif out decodes it just fine.

Receiver B does not. With no changes to totalmix, simply unplugging the cable from receiver A and plugging it into receiver B. Receiver B can't decode the stream, plays static.

Should be noted that receiver A is 20 years old or so, while receiver B is less than 5 years old.

Any ideas what could be the problem?

Probably this is the difference between your receivers. If I understand @MC right Babyface does not send 'Non-Audio' (even if it is on BF input data stream.). Your A probably does not need it, while B needs it.

PCM downmix always worked, yes. Can send stereo sound to both receivers through the babyface with no issue.

Alright then, seems like we figured out the culprit.

Sign me up for a feature request to send that bit on spdif out. Can't be too hard to implement, right?

And the user setting in the driver would not do the trick? Simmilar to SP-DIF Pro/Con flag. Just idea. (I know it is not the same situation as Pro/Con is dependent on HW connected, so sort of 'permanent', the discussed flag is depended on HW connected and also on datatype sent.... So, not so 'permanent'....)

Make sure the signal is passing through absolutely unchanged, i.e. no change to the level and nothing else mixed in. Reset Totalmix and start from scratch...