1 Topic by bisthore (edited by bisthore 2021-08-07 07:42:51)

  • From: Sweden
  • Registered: 2008-11-28
  • Posts: 16

Topic: AVB connections drop after restart of device

I have an issue in a setup with 1 AVB tool, 3 12 Mic and 1 DIGIFACE AVB.

Whenever I switch off any device the AVB setup will not recover. Every setting is like before, but I have to remove all the stream settings of the AVB TOOL and reconnect all devices in the AVB controller before the clock recovers.

AVB is clockmaster as it is our monitoring device in the control room.
I wanted to be able to listen even when the 12 mics are turned off and not risk the clocking of the entire system if I have to restart the computer.

As it is now, I totally have to avoid ever switching off anything during a week of recording. I've several times barely made it on time resetting everything in the morning before the orchestra wanted to start.

Do you think the device is faulty?

Today I updated everything to 1.3 firmware which doesn't seem to solve the problems.

FOLLOW UP:
I will test the update on the AVB TOOL to 1.4 RC  after today's recording session.
Problem is I'm not living nearby and while I have remote access to the system I will not be able to restart the machines remotely. I need to make sure the machines are at least working as well as now. I assume the update is safe (!?)

In order to have a situation that is logical and possible to remember each device has been assigned 1 output stream corresponding with the needs/number of microphone inputs

AVB TOOL: is clock master
4 Inputs
1) 96KHz, AAF, 12ch - connected to 12 Mic 1 Stream 1
2) 96KHz, AAF, 12ch - connected to 12 Mic 2 Stream 1
3) 96KHz, AAF, 12ch - connected to 12 Mic 3 Stream 1
4) 96kHz, AAF, 8ch - connected to DIGIFACE AVB Stream 1
1 Output
1) 96kHz, AAF, 4ch

12 MIC 1, 12 MIC 2, 12 MIC 3 : EACH clock slave to AVB stream 1 (AVB Tool)
1 Inputs
1) 96KHz, AAF, 4ch - connected to AVB TOOL Stream 1
1 Output
1) 96kHz, AAF, 12ch

DIGIFACE AVB: clock slave to AVB stream 1 (AVB Tool)
4 Inputs
1) 96KHz, AAF, 12ch - connected to 12 Mic 1 Stream 1
2) 96KHz, AAF, 12ch - connected to 12 Mic 2 Stream 1
3) 96KHz, AAF, 12ch - connected to 12 Mic 3 Stream 1
4) 96kHz, AAF, 4ch - connected to AVB TOOL Stream 1
1 Output
1) 96kHz, AAF, 8ch

The Maximum recorde channels count with this setup has been 40. Successfully if I leave it unchanged.

We use the AVB TOOL mainly for monitoring in the control room and as source of our communication channel (Talkback).
In the maximum case above we also had the last 4 microphones patched to and inserted via the AVB tool.

Each 12 Mic should provide the number of possible microphone inputs to the network and we want the to logically be available on Ch 1-36 on the DIGIFACE AVB as our main inputs.

I chose 8ch of outputs on the DIGIFACE AVB to have 2 stereo outputs for clean monitoring as well as secondary monitoring that follows SOLO/PFL on our DAW (SAMPLITUDE PRO X5)
The remaining 4 channels are spares e.g. clean program feed to a streaming device via analogue output of the AVB TOOL or MADI to a secondary recording system
----
So, yes, the same outgoing stream (Stream 1) of the AVB TOOL is connected to 4 devices
----
Switch Model is 2* MOTU AVB Switch. One in C/R, one on Stage
----
I have to completely remove the output stream of the AVB TOOL via the AVB controller APP. After that I have to disconnect and reconnect the corresponding Input Stream from all devices.
(It is possible I disconnected the input streams first, then removed the Output Stream from the device, then recreated it, the reconnected the corresponding inputs to really make it work.
It seems sometimes it has been enough to disconnect the inputs and reconnect)
----
Entity Guid of all devices starts with 0x480bb2fffed
(I read it from the AVB controller APP)
also the Stream output is yellow (in the Web Remote Interface) and when I check the Info it states Listener failed behind the string, although the system works at the moment, all inputs on the other devices are green
( I    very  rarely have dropped Audio buffers, both playback and recording, other than that no symptoms of problems at this moment)

Thanks, Max, for helping!
Thore

Max wrote:

Hi Thore,

you mention you updated to 1.3.0 -> please test firmware 1.4 - the one this thread is about. I updated the links above today with the latest release candidate.

To discuss further details regarding your specific problem, please open a separate topic, along with the following information:

- stream format between AVB Tool and other devices (ie. AAF, 96 kHz, 8 Ch.)
- is the same outgoing stream (ie. Stream 1) from the AVB Tool source for all remaining AVB devices, or are you connecting different streams to different devices?
- the switch model used
- you mention that you have to change stream settings of the AVB Tool (Clock Master) in order to be able to reconnect from a controller. Could you explain what setting you are changing?
- could you observe the information AVB Tool > i (third button) > LAN Info > gPTP Grandmaster: in a normal working setup as you describe, the string should not start with "48:0b:b2" while your devices are up and running, and also should not change when you power off one of the other AVB devices. If it does at any time toggle to a string with "48:0b:b2" (the field automatically updates), this would be interesting to know.

Best regards,
Max

  • From: Sweden
  • Registered: 2008-11-28
  • Posts: 16

Re: AVB connections drop after restart of device

Now sitting in front of AVB TOOL
LAN Info Grandmaster is:
00:01:f2:ff:fe:01:47:49

  • Max
  • Max
  • Moderator
  • Offline
  • Registered: 2007-05-09
  • Posts: 315

Re: AVB connections drop after restart of device

Hi Thore,

what a great setup! Really happy to see this in action in a studio like yours (and in the hands of Tonmeisters like us ;-)).

I've re-created your setup, with a difference that all devices use the 1.4.0RC1 and I used one MOTU and one Presonus switch, rather than two MOTU switches (because I only have one). I could not reproduce your issue - I switch off one of the 12Mics, and all other streams continue streaming happily. I switch it back on, the stream to that device is re-established and everything continues, with lock, sync, and no yellow state in any remote control.

I can also power-cycle the AVB Tool, and the system recovers automatically, as they should.

Your issue seems to be related to the stream reservation, in which both the switches and the endpoints take part. Interrupted streams are re-connected automatically by the listeners when they lose the connection, but they should never lose connection because an unrelated listener is switched off.

To investigate further, let's try the following:

1) please flash 1.4.0 RC1 to AVB Tool and all four 12Mics. It is safe to do so. Also confirm that your MOTU switches use the latest firmware v1.1.0+667, the Digiface AVB uses FUT 239, driver 0.9735, and AVB Controller 0.9553.

2) reproduce the issue by powering off one of the three 12Mics while everything is up and running. Please confirm that at this time, the AVB Tool stops receiving from the two other 12Mics, and that the Digiface AVB also stops receiving from the other two 12Mics.

3) if this is the case, please power off only the two MOTU switches and switch them back on. Wait and see if all streams in the network recover.

Please let us know the results. If the issue is still there, a remote support session might be the easiest way to investigate the cause of the issue.

Best regards,
Max

PS: Regarding the grandmaster, a little background info (can be skipped, just a brief explanation why I asked): those numbers are so-called MAC addresses with an additional ff:fe in the middle. Every address is unique. The first three octets are the vendor ID, in your case MOTU (00:01:f2); RME uses 48:0b:b2. The last three are assigned by the vendor, and in case of MOTU they are identical to the last 6 hex numbers of the serial number written on the device (in your case, the serial number of one of the two MOTU switches ends with 014749). The grandmaster sets the clock for the entire network, and all streams use this clock. Any media clock (for example, your 96 kHz coming from the AVB Tool) is streamed over the network "on top of that". In AVB, every stream can have a different media clock (for example 96001 Hz and 176399 Hz can co-exist), whereas in other AoIP technologies, network clock and media clock are tied.

Now, the entire process of grandmaster selection is fully automated in AVB and decided with an algorithm (best master clock). The MOTU switches can be configured to have a higher or lower priority, but even if they have the same priority (factory default), only one of them will be the grandmaster for all devices on the network. Except if the priority was manually configured so low that a connected RME device becomes the grandmaster of the network. In your running setup, I was curious if for some reason a MOTU switch suddenly decided to change the grandmaster when you power off the RME device. This would trigger that the attached devices would change the grandmaster shown in the display, and that might have had an impact on the stream reservation. But in my tests that was all solid, and you observed also the correct MOTU ID as grandmaster, so no need to further observe that number, and no need to worry about gPTP grandmasters at all.

4 Reply by bisthore (edited by bisthore 2021-08-10 07:59:08)

  • From: Sweden
  • Registered: 2008-11-28
  • Posts: 16

Re: AVB connections drop after restart of device

Thank you a lot Max. I will follow up.

I have to work on it remotely together with a local sound engineer, but it may take even a couple of weeks, before I can get back.

What I did manage before packing up:
I had the AVB Tool send 2 identical streams of 4 channels. Each with only one listener connected.

AVB TOOL Output Stream 1, 96kHz, 4ch, AAF feeds now the DIGIFACE AVB input stream 4
AVB TOOL Output Stream 2, 96kHz, 4ch, AAF feeds now the 12 MIC 01 input stream 1

AVB TOOL Output Stream 3, 96kHz, 0ch, CRF (? from my Memory) feeds now the 12 MIC 02 input stream 1, just for clock (it is correct to do so, I hope?)
AVB TOOL Output Stream 4, 96kHz, 0ch, CRF (? from myMemory) feeds now the 12 MIC 03 input stream 1, just for clock

This seems to solve the issue (Still FW 1.3), all connections show green even after power cycling any of the device including the computer (I did not power cycle the network switches, though). I also yet have to update the DIGIFACE AVB and the MOTUs.

-


Funny enough each of the 3 12 MIC are sending only 1 Stream of 12ch which is listened to by 2 devices (AVB Tool for further routing ti MADI /Backup recording and DIGIFACE AVB). They never had this kind of issue. 

-

The problems all circle around the AVB Tool with its 4ch stream format. Maybe it indicates an issue with the format and (maybe) the MOTU.
We also have an expensive LUMINEX AVB switch, which is paired with 1 MOTU when we use the house wiring, this time I used a direct LAN cable. Same issue.
When I first installed the setup, I had thought to send a separate 1 channel stream just for Talkback. That didn't work at all-no clean audio. 2 channel worked but I realized I had to simplify the whole thing and that's where I ended up with the 3 stream formats (4, 8, 12 channels)
I am now thinking, would it be beneficial to use 6ch-streams only throughout the entire system? That is the number that would suit all needs without getting any empty channels in the middle of the the DIGIFACE AVB Inputs

It shouldn't really matter, or could it?

  • Max
  • Max
  • Moderator
  • Offline
  • Registered: 2007-05-09
  • Posts: 315

Re: AVB connections drop after restart of device

Funny enough each of the 3 12 MIC are sending only 1 Stream of 12ch which is listened to by 2 devices (AVB Tool for further routing ti MADI /Backup recording and DIGIFACE AVB). They never had this kind of issue.

You mean if they are used without the clock from the AVB Tool, they operate as expected? But then, you'd only use one at a time, so AVB Tool and Digiface AVB receive the stream from only one of the three devices?

The number of channels per stream should be entirely irrelevant, however MOTU devices only use 8 ch streams, so their switch might treat them differently (no idea, wiill test). Generally speaking, the most common/maximum compatibility stream format is 8 channels AAF. Also, we usually test with certified AVB switches (such as Luminex, Extreme, Cisco, ...), since that allows us to rule out them being the source of the problem.

Did you set the Digiface AVB compatibility to MILAN in the AVB Controller, or did you leave it at IEEE1722.1?

it is correct to do so, I hope

Sure.

  • From: Sweden
  • Registered: 2008-11-28
  • Posts: 16

Re: AVB connections drop after restart of device

What I meant is, it seems necessary to use different streams from the AVB-tool to each of the three 12-Mics and DIGIFACE AVB, whereas it is no problems to connect at least 2 listeners to the same 12-Mics 12ch-stream. Complete setup running. Have to do more testing though



Did you set the Digiface AVB compatibility to MILAN in the AVB Controller, or did you leave it at IEEE1722.1?

Have to check, I think I didn't change it