I think your portrayal is technically not correct (looks as if you put a statement out of context), see longer text below *)
In terms of audio loss. More likely is what we read the last months in this forum about all the issues. Partially it were Apple problems, but some problems seemed to be caused on kind of "upgrade installations" of users, where a complete new installation could cure some or all problems.
*) You see it e.g. clear with Cubase under Windows. Decrease ASIO buffersize, and you will see an increase in (EDIT:) CPU and ASIO load. What happens with smaller buffer sizes is simply a higher interrupt load, which is always bad for a system.
The principles are all the same, regardless of which CPU, which OS.
Because the interrupts block CPU cores for other activities and the CPU also has to switch the context much more often and reload all CPU registers with the other context. That's a lot of work for a CPU (management overhead).
Imagine you had to empty a bathtub at a predetermined time interval (e.g. 5 minutes) with a bucket or a glass of water. I think with a water glass you would work up a sweat to get the same amount of water filled in the same amount of time.
All channels are always transmitted on the computer/recording interface. The data must be collected in a timely manner. If you put additional stress on the CPU with small buffers during a high workload, then dropouts are more likely to occur.
The art is
1. to use good drivers that do not occupy a CPU for too long, because the drivers have the highest priority and the process scheduler has no control over the length of time the driver stays on the CPU core (due to data integrity, Windows and MacOS are not real-time systems
2. taming of background processes, to lower the system load, to have more headroom for audio processing in time.
3. to adapt the buffer sizes to the needs of the work/project. When recording, use high buffer sizes (so that nothing is lost). Middle range when mixing/mastering. When playing via VSTs, smaller buffers so that the RTL stays under 10ms.
BR Ramses - UFX III, 12Mic, XTC, ADI-2 Pro FS R BE, RayDAT, X10SRi-F, E5-1680v4, Win10Pro22H2, Cub14