My first quibble with the testing is that it uses a loop-back method, which means that it's impossible to tell whether the signal degeneration is the result of the input or output analog electronics, and/or the ADC or the DAC. I suppose the argument could be made about "real world use", but when was the last time you routed audio out and back into your interface ten times? Yes, it piles up the degradation, making it easier to hear and measure, but it does little to identify the source of any degradation.

As well meaning as these kinds of tests are, they do seek to provide one standard answer to the nebulous question, "What is the best?" The answer to that will be at best simplistic and reductive. Best for what? Best at what?

I suppose when you are looking to buy one, and only one audio interface, being reductive is part of the purchase decision, but still I would appreciate more detailed results, and more controlled testing. For example, if I were looking to buy an interface where I knew that I'd be using it most for audio capture, rather than playback, then I might be willing to cut some cost corners by getting an interface with stellar analog in and ADC performance, but lesser quality outputs.

Everything in audio is a trade-off. Seeking to find a single, one-size-fits-all answer ignores this fact of life.

As a follow up to this, I engaged in the thread and undertook a further review of the methodology and results.  I put a number of questions to the originator of the post on the consistency of the results and the meaningfulness of the AudioDiff program analysis.  The response was along the lines of avoidance, obfuscation, and denigration ("shoot the messenger"), and the same response has been observed to others who also sought to dispute with the findings.  The originator also appears to lack the necessary technical background to allow him to undertake such an analysis.  The originator has also acquired cult status, particular from followers whose unit rank high in the testing, and accept the results uncritically.

For the record, I think the AudioDiff tool can produce useful results in detecting differences between input and output files from a particular audio process, but the results depend on the testing configuration and test material. 

What is not clear is whether AudioDiff can carry out the amplitude and phase alignment and cancellation process to sufficient precision to evaluate a precision device like a modern day A-D or D-A convertor.  The original tester does not offer any convincing demonstration that this is the case.  I am currently considering whether it is sufficiently interesting to do some simple testing of the AudioDiff process to reveal its objective limits in this type of application.  I am sure that if it did offer useful information, RME would be using it by now.

My present opinion is that AudioDiff is misapplied in this instance and the results as presented are currently meaningless.  (However, I stand to be corrected.)  The UFX (and most of the others tested) have much better performance than AudioDiff can measure with the necessary accuracy.

I am reminded of the words of the 19th century French-born but American resident writer and political philosopher Alexis de Tocqueville, who observed that the public will prefer to accept a simple lie rather than a complex truth.

It was an early part of my engineering training that "knowledge without measurement is knowledge of a meagre kind", so you will have take that into account with what I say.

I have picked up a separate thread (not as original poster) that focusses more on the AudioDiff program and its capabilities,but I am using my UFX as one of the test subjects.

I buy equipment like the UFX based equally on features and performance, as I infer that you do as well.  I have used some MOTU in the past, and my main quibble is that MOTU do not currently appear to publish any performance specifications for their equipment, in contrast to RME (and others) who provide (usually) comprehensive specifications, against which they are prepared to be held accountable. 

And I am trying to relate performance (as published by RME) against the results obtained using the AudioDiff program, in a scientific manner.  I have to keep reminding myself that GS is not a scientific forum - part of it could be, but at the moment it is not.

But hey - you get to meet some really interesting people on this forum!

@panatrope - totally hear you & I had a very quick look at the reply to your post on that Gearslutz thread where I think you asked some pretty relevant questions & got chided as being apparently unable to understand the software.

A MOTU 828MKII does not sound like a UFX whichever way you slice it & ears not some "measurement software" is plenty enough to tell you that.
Unfortunately the biggest downfall of the internet is the platform it provides for misinformation & at times absolute bs - the people that really know what they are doing/talking about don't have time or the inclination to get involved in huge debates with keyboard warriors who think they know it all based on the ability to spend a short period of time trawling the internet & forming "expert" opinions vs. the traditional route of years of actual experience..

Oh well, never mind, it's a no win situation....like cockroaches you deal with one, another 20 pop up.

Again to the original poster - put it out of your mind, you wisely bought a great bit of gear, spend time learning to use it well vs. dwelling on what someone who is clearly ill-informed & probably spends more time on internet forums than actually being creative thinks of it..

Thank you all, and especially you, Panatrope, for your contribution. i followed the gearslutz thread and the time and effort you invested in those things is very much appreciated on my side. i really like my ufx feature AND soundwise, nonetheless it's making me a bit nervous if someone states poor audio quality on a "scientificic" base which i can't comment. i am a musician and producer and don't dive deep in the technical aspects of my equipment. you clearly presented yourself as an educated, open minded person, and i'm happy you chimed in. after all, i don't want to bother about having to buy a motu 828 mkii for my conversion duties because the ufx sounds poor in comparison. personally i think the ufx sounds great. you all gave me confidence that i still can trust this impression.

Regards,

Fishmac

^^ I'd certainly be interested in seeing what you came up with in a real-world use scenario - i.e. not a 10 pass loopback test :roll
Please post once you have some info.

Seems like perfect common sense - most reference publications on using a DAW advise against leaving the box once you are within the digital domain - A/D in & then possibly D/A - A/D at a push if you are going to using an analogue summing device at final mix.
How can anyone think constant passes of a source through A/D-D/A will not result in some form of signal degradation.

Just like the analogue days when many a session was ruined by constant re-takes, wear on tape stock etc (I recall there being a good documentary on the recording of a Fleetwood Mac album - might have been Rumours - where the drum tracks had deteriorated by the end of tracking to the point where they had to be saved using a backup - which had fortunately been made - of the original takes/comps).

I can't think of any situation where I would "loopback" my source material in a project 10 times - it's just ludicrous as a test subject, somewhat akin to judging an F1 car on how much shopping you can fit in the boot. ed:

A very small one - you can only fit a spare pair of underpants.....

I really must write a proper report about what I have done with AudioDiff.  But suffice to say, I am happy at the moment that the internal time-alignment process is good for sub-sample interval time alignment (it has internal setting to set the alignment resolution down to 120ps).  I used iZotope SRC to up convert a test signal to 192kHz, removed 11 samples (prime number, no ratio to either 192K or 44K1) and converted back, and found a precise alignment of the offset and excellent cancellation.  I also tested the cancellation ability by adding, while at 192kHz, a controlled amount of pink noise (-75dBfs rms).  After SRC back to 44K1, the original test signal was cancelled sufficient to apparently leave only the added pink noise at around -75dBfs and there was no identifiable residual of the original test signal.  This appears to be a good result.

Now this of course was done only in the digital domain.  Things get interesting with a D-A/A-D in the loop.   Jitter comes into question, possibly leading to a loss of "coherence", as one French correspondent on GS puts it.  Previous work as documented in an earlier AES paper (Dunn) came up with a figure of 120ps rms jitter as equivalent to the level of quantisation noise on a 16-bit recording, so as jitter is another form of 'quantising noise" we should see an added noise of this level for this amount of jitter..  Most companies that quote specs have a figure much less than this for their clocks. 

So, I also want to check if jitter is a cause of the inability to completely cancel the test tone in the "THD plus noise" test (given that spectrograms of the harmonic components and noise floor in the output file appear to be as expected). Should we see some evidence of this in the shape of the noise floor, particularly due to jitter components out to say 20 or 100Hz?   (BTW, a loopback test of the UFX - AN out 3,4 high gain to AN in 7,8 low gain) gave results for added harmonics that are about what I expected, based on the published specs.)

Low jitter has previously been linked anecdotally with increased 'transparency'.  If jitter is one of the main reasons for the level of differences detected by AudioDiff, then maybe there is a basis for the tester's contention that such testing can correlate with 'transparency'.  But I think a whole lot more work has to be done to make that conclusion reliable.

That is the raw clock jitter within the AD-conversion process (sampling jitter), which can only be measured by a high-resolution FFT of the digitized signal, and is a quite theoretical value. But: If there would be high jitter, then you would see it in the measurements. The FFT is clean, the noise floor (RMS) is at -112 dBu, far more down than the 96 dB (for 120ps) of that paper, or in other words: the jitter effects are lower than the noise and THD produced by the ADC itself. Jitter can not explain the measurement results.

Thank you for the input.

This was my expectation entirely, but of course I have to observe that for my self.  The figure of 120ps was based on a test tone of 20kHz, and of course the noise for a lower frequency would be correspondingly less, assuming it is a quantising noise-like distortion.  (John Watkinson's "An Introduction to Digital Audio (2nd Ed.)"  has a useful nomogram on p.107 to help point out the noise level one would expect from various levels of jitter.)  Considering it as a form of phase modulation would produce sidebands around the test tone, which would also be revealed by high resolution FFT spectral analysis.

Thankfully, there are many high-quality spectral analysis tools at low cost (some even free) available to assist in this work.  (Not to mention DigiCheck, of course).

op-amp-measurements

Have there been any further revelations on the UFX on this topic? Any clarification from RME? I've been nosing around the GS thread and while interesting I don't think it is revealing a complete and accurate picture of the UFX or any of the units tested, let alone in a real world application setting. I am all for testing/info and any insight into what's going on "under the hood" with the UFX and any of the units shown but partial information is misleading.

I've had my UFX for a while now and it's a great unit, stellar audio quality, stability is excellent, no issues whatsoever in regards to anything, and the ARC is very handy indeed.

Unfortunately I have neither the time nor the technical background to properly address all the issues in the thread. It would be nice as it seems a huge disservice to the recording community at large to provide an list of quality rankings based on partial and inaccurate "ultimate" test results.

As we do not have your ears, can you please describe the differences you hear?

I also have an external reference D/A (not as expensive as yours but still highly regarded) and I have been considering doing the loop-back test described in the GS forum, but using the external D/A in comparison to the UFX outputs. (I can possibly use the A/Ds in the Nagra V to make the opposite comparison.)

I still can't say what the new results might mean in terms of 'transparency'.

Thank you for your explanation.

So, the differences you can describe are "much firmer bass, smoother highs and often more depth to the soundstage."  To me, these could be associated with (among other things) lf phase response, anti-aliasing filter response, and low level phase and amplitude linearity, all characteristics that should be capable of being observed with objective measurements.  However, the history of audio is littered with examples of "I can hear it but I can't measure it"  - the ability to link perceived differences and physical reality.

The proponents of the Audio Difference testing method have postulated that devices can be ranked in terms of 'transparency' by the score achieved in the test.  In my view, they have yet to justify this association, and they have not identified the origin of the "differences" that they have documented.

Their testing methodology does not allow reliable differentiation between the A/D and D/A process included in the chain.  As your observations suggest that the UFX A/D process is satisfactory (to you) but not the D/A process, maybe a comparative DiffTest measurement of a loop comprising the UFX on its own, and the input looped to the output of a Weiss (or other 'reference') D/A could be interesting.

But the problem still remains identifying the cause of any differences measured, and accurately relating these to the subjective observations you have made.   As it has been throughout the history of audio.

So sorry gentlemen! I am not an expert in digital testing and wished only to indicate my experience as to UFX record "suitability" on a live Swing band-with 13 difficult to record horns. Obviously,  I find the UFX' A to D's, encoded jitter more than acceptable.

There are A to Ds beyond my price class (and logistical management ability) but this forum, in large part, is devoted to the uber portable and compact RME gear that I find such a value.  I hope that I more than implied this when I compared the UFX' $2.2K cost with its 24 (is it?) converters to the $7.7K of the Weiss' two (note that the Weiss two channel mastering A to D is about $9K).

My reply post was to a question of how to describe, in words, the difference in sound between two D to A converters-doomed to failure-obviously. My Jitter(y) speculation was my "take" on what I had read. Yes I do have the impression that it's eaiser to A to D for recording than D to A for playback.  However, I really have a limited amout of experience for such an observation and would rely on others for a such a conclusion.

Maybe a more useful comparative could be that my "IT guy" (civilian computer expert) was present when I made the change over to the Weiss DAC.  He knew of my financial ambivalence in buying such an expensive box (the price of two two or more very fine desktop computers to him) but was astonished at the difference in sound-to his untrained ears.

For some decades I had tested analogue devices (notably speaker power amps and small signal amps/preamps) with the best, then, equipment. However, today as then, there are things that I and others can't measure but which do manifest themselves in different and occassionally "better" sound. 

For this I would allude to my 8 amplifer combinations over two years as I escalated in cost, if not measured RMS watts, to drive my B&W monitor speakers. For such acoustic description nicities I would, in the future, refer you to those Golden Ears HiFi wordsmiths-just take your pick of their lush prose.

At least, for now, I am pausing in amplifer upgrades with the stasis of two monoblocked Burmester Mk IIIs for 350 hz up (350hz down is driven by class H "roadie" amps with 4 times the RMS power and 1/100 the cost-because I can't tell the difference in the lower frequencies)).

It's also notable that the wire I use to connect the Burmesters to the B&Ws costs twice as much as the four Tapcos (2500/2800 RMS watts monoblocked-each) used for the low end of the B&Ws and Subs.
Yes, I did try the cheap wire, the expensive wire and the now this more expensive wire.

I paid up for the wire because I could hear the difference in a few seconds and, yes, my IT guy can hear the difference too as well as my Pro Tools engineer.  I am, complaisently, happy with the wire that cost only $2-3k per set ( atmdiscount 2 set in use) and am loath to demo the stuff that's a order of magnititude more.  I also haven't listened to the Weiss ADC 2 (Lavery Gold, etc.) because I would need ten or eleven of them (about $90K+), a rack, a truck and an assistant to help me.

I don't know how you test "wire" either-to predict what it sounds like. So for those of you involved in such ambitious testing, whether digital or analogue, more power to you. I will be interesting in your methods, data and conclusions.

Sampling places a point in both space (voltage) and time.  Accurately rendering the phase of a signal depends on resolving time to the same accuracy as voltage.  Current systems do voltage at around -120dB. 

See John Watkinson's "An Introduction to Digital Audio" (2nd ed.) pp106-107.  For a 20kHz signal in 16-bit system, timing jitter needs to be limited to 120ps.  For lower frequencies, the jitter limit is correspondingly higher.  A 1ns pk-pk jitter on a full-scale 3kHz signal will result in a theoretical signal/noise ratio equivalent to a 18-bit resolution (around -110dB).

Using the AudioDiff test on a single pass through the UFX (AN7-8 out to AN1-2 in) with a digitally-generated single frequency 997Hz signal at -1dBfs, the best cancellation of the fundamental that I could observe with the spectrum analyser (16,384 resolution/Blackman-Harris) of either Sound Forge or iZotope RX II was about -75dB (compared to perfect cancellation on a purely digital path through the same device).  The harmonic distortion components were clearly identifiable at about the expected levels.  Sidebands could be observed at ±200Hz at a level below -100dBfs, suggesting that if they were due to jitter, it might be mains-frequency related.  (No significant mains-related baseband signals were noted.)

I would regard this result as not bad for the capability of the measuring methods available to me.  It broadly supports the specs published for the UFX.  Why AudioDiff cannot more accurately cancel the fundamental when an analog link is included in the chain is a question of interest - whether it is a limitation of the program or whether it is a product of unknown spurious distortion which might (or might not) be jitter related. 

And what jitter contributes to the rating of audio performance ('transparency' or whatever) is what we are concerned with.  Experienced ears can hear different things, but there are so many other factors that influence perception.  It just needs the reasons relating engineering performance and subjective assessment to be discovered, and if you can do that "you're a better man than I am, Gunga Din!"

I was discussing this with a colleague of mine who has long experience in classical musical and audio production. He arranged a comparative test of the DACs of the Prism Orpheus, the Nagra VI and the Weiss with his new Forsell DACs, replaying the same minimalistically recorded digital material into the same set of reference speakers.  His opinion was the Forsell (considerably cheaper than the Weiss, more expensive than Orpheus or UFX) was superior (descriptors "definition" "detail" and "depth") and the Weiss was the inferior ("muffled").  His reported impressions, obviously at odds with your experience. 

What measurements could we possibly devise to objectively explain the different perceptions?  Or do I revert to my old signature quotation - de gustibus non disputandum est - it is useless to argue about taste?

Hi, first time here.

I had already decided to buy UFX or UCX. Then i found that loopback thread, and changed my mind. Well, after all I didn't buy anything, because I was so confused about mixed reaction here and there.

But I was curious about the crucial element of that test: Audio DiffMaker. It appeared to me as a one big mystical element that gives the final truth. 

So what was it and what can it do? I wanted to challenge it before I was ready to trust it as a final word.

I've just  made some tests with DiffMaker. If any body is interested with them, you can find them here:

http://www.slapmedia.com/diffmaker/

There's lots of audio examples and graphs, and everything is in total mess. Hopefully you get better idea of what DiffMaker can do and where it can be used. At the moment I have no answer for that.

what about sample accurate null tests......wouldnt that be the best way?

In essence this is what DiffMaker tries to do. Any difference left after nulling/phase-cancellation is what is measured. Main problem is the "black box" part where users don't know what happens internally versus when you do all this stuff manually.