The analog inputs and outputs of different devices are designed for different use cases.
There are analog ports which are desiged to work at lower consumer levels, which you can find e.g. at HiFi devices (cinch).
There are also analog ports which are designed for studio work where cables usually can get longer why signals work at a higher level and balanced cables are in use.
You device gives you the possibility to select a proper reference level to achieve two things
a) to match with the level of the other side of the cable
it makes neither sense to overload the other circuit, not to send it a too weak signal, so that the other circuit can't work in an optimal range for best SNR and dynamic
b) to make it possible for AD/DA converter to work at optimum SNR and dynamics
Look e.g. at this Excel, how many dB SNR you can loose if you operate your device at a suboptimal reference level:
https://forum.rme-audio.de/viewtopic.ph … 24#p155724
There is also a nice example regarding this in the ADI-2 Pro/DAC manuals, see ch 21.3 here:
http://www.rme-audio.de/downloads/adi2profsr_e.pdf
There are devices that allow you to set the reference level per port or only for a whole bunch of inputs or outputs.
Some devices support up to 4 different reference levels which gives much flexibility.
In your use case you most likely only have to take care of phones and monitors.
For phones you will most likely not need the highest output, so that you can turn the faders towards 0dB in a range between approx -30 - -10 dB. Depends of course on your phones (impedance, effectivenes of drivers).
For monitors you also might need to use the lowest output level, so that 0dB do not roast your ears or speakers.
See also this interesting article from Matthias Carstens about level matching:
https://forum.rme-audio.de/viewtopic.php?id=25399
BR Ramses - UFX III, 12Mic, XTC, ADI-2 Pro FS R BE, RayDAT, X10SRi-F, E5-1680v4, Win10Pro22H2, Cub13