Yes, there is always going to be some latency with your system - the trade off is between having a long delay (as you've already experienced) with a lower CPU load - vs. - shorter delay times and higher demands on your CPU, which may, at shorter and shorter latency settings, cause you to experience clicks, pops, dropouts, etc. in your audio.  What is "optimal" for you might be different than that for another user.

An alternative to hearing the latency in your phones is to set up the RME TotalMix mixer, so that you are monitoring your inputs, rather than the signal that has gone through your DAW.  You can even overdub this way, monitoring the playback of your DAW with the direct sound of your current imput channel.  The DAW software should take care of any timing discrepancies between the new overdub and the existing tracks (if not, you can apply some compensation with delay or manually shifting the track(s)).