Sorry, if the previous sounded too complicated. It's difficult to find the right balance.
Some people prefer more detailed information, some less.
I wanted to be sure that I understood your demand well by making it more detailed 
This is now the condensed form.
RME products and TotalMix FX are fine for your work.
Loopback recording is implemented in TotalMix FX for all RME recording interfaces.
As you need EQ and a compressor, you need a recording interface that supports FX, as it needs a dedicated DSP for FX. Not all models have it. Digiface USB doesn't have it, for example, but it has too few channels anyway.
TotalMix FX is the product name of the mixer; it has support for FX, but only if the hardware (recording interface) has support for it. With HDSPe MADI FX, you are fine. BTW this is the only PCIe card with FX, RoomEQ and Crossfeed.
The same applies to the latest features, RoomEQ and Crossfeed; it needs this additional DSP for FX. For the HDSPe MADI FX, the newer/latest versions of this card are needed. This is only meant as a hint, if you should intend to buy it from the used market.
If you want to have 128 channels like you did with the MOTU and maybe even more,
I would recommend the HDSPe MADI FX PCIe card.
PC Case requirements
- 1 free PCIe 1.1 x1 socket (every mainboard fulfills this demand, as PCIe is fully backward compatible)
- 1 free socket in the case for the daughter card, which is connected to the main card per ribbon cable.
Final Solution, HW requirements
You need the HDSPe MADI FX, but this card alone has only a phones output.
It can be used as a line-level port for connecting active monitors, but this is only an unbalanced connection.
For such a solution I would prefer/recommend balanced connections.
Balanced connections are far less susceptible to hum and electrical noise.
The best solution would be to use one of the RME reference converter which can be connected by AES.
AES has the additional benefit of providing a galvanic isolation against hum.
Output Ports which you can use for DAW tracks and loopback recording: 196 (minus a few for monitoring)
HDSPe MADI FX - https://www.thomann.de/de/rme_hdspe_madi_fx.htm € 1349
ADI-2 Pro EX - https://www.thomann.de/de/rme_adi_2_pro_ex.htm € 1989
SUMM € 3338
optional:
ARC USB - https://www.thomann.de/de/rme_arc_usb.htm € 144
TOTAL € 3482
Per SW Playback port cost (to fulfill your needs) excluding AD/DA converter: 1349 / 196 = € 6,88
Per SW Playback port cost including AD/DA converter and ARC USB: 3482 / 196 = € 17,77
Price Tip: if this additional RME Reference Converter (ADI-2 Pro EX) is too expensive, then look at the used market for the older models which are also excellent and up to now all got the same firmware (programmable features):
- ADI-2 Pro FS
- ADI-2 Pro FS R BE
- ADI-2/4 Pro SE
Connection diagram
+---USB----optional: ARC USB (Remote Control for TotalMix FX control room)
/
PC---USB2---------------------------------------------+ optional USB connection for:
| | - remote controlling the ADI-2 Pro with a nice GUI
| PCIe 1.1-5 | - firmware upgrades
| | - DSD playback
| | - alternatively the unit can be connected stand-alone
| |
HDSPe MADI FX---AES OUT------------AES IN---ADI-2 Pro EX---Analog 1/2---Active Monitors
\---AES IN-------------AES OUT---/ \---Analog 3/4---Headphone
Plan B - MADIface XT II
If the additional reference converter is too expensive, then there is a 2nd option to use the MADIface XT II USB3 interface, which also concentrates on MADI and only a few I/O ports.
You should get the XT II (not the XT) because this device also supports RoomEQ. If you do not have the Reference Converter in your solution, then you can use the 9-band PEQ of RoomEQ to linearize your headphones, if needed.
Or to perform room correction by using the EQ, it can, e.g., import the EQ settings of Sonarworks.
If you want, you can also connect an RME reference converter now or later.
You can connect it easily by using the AES port (through a breakout cable).
Connection diagram:
PC---USB------optional: ARC USB
|
| USB3
|
MADIface XT II---Analog Out---Active Monitors
\---Phones Out---Headphones
Price Overview
Output Ports which you can use for DAW tracks and loopback recording: 198 (minus a few for monitoring)
MADIface XT II - https://www.thomann.de/de/rme_madiface_xt_ii.htm € 1949
optional:
ARC USB - https://www.thomann.de/de/rme_arc_usb.htm € 144
TOTAL € 2093
optional:
ADI-2 Pro EX - https://www.thomann.de/de/rme_adi_2_pro_ex.htm € 1989
TOTAL € 4082
Per SW Playback port cost (to fulfill your needs) excluding optional components: 2093 / 198 = € 10,57
Per SW Playback port cost including AD/DA converter and ARC USB: 4082 / 198 = € 20,62
My Conclusions
The best value you will definitively have if you enhance your monitoring by the quality and features of a RME reference converter. If you put all costs together, then the combination of HDSPe MADI FX and reference converter gives you the best price and the best features.
The recording interface (PCIe card) is in the PC case, and you have only the reference converter close to your desk where you can easily connect your headphone.
If the computer is located in a machine room, this poses no problem: AES cable runs of 100 m or more are feasible, depending on the capacitance and attenuation of the cable. Due to AES you have also galvanic isolation which is a perfect solution.
BR Ramses - HDSPe MADI FX, M-1620 Pro D, 12Mic, UFX III, ADI-2 Pro FS R BE, Nuendo 15, Win10 IoT Ent