[NOT ISADORA] Investigation - OSC Lossy at sub 100ms transmission

[liminal_andy]

It's an issue we've discussed before, but would love to re-ignite the conversation on. In our testing, we find that Isadora will "drop" OSC packets from being received if we transmit them at a cadence more frequent than 100ms. This occurs in patches with extremely low LOAD, with no video processing, and with framerate and general service task set to their maximum options. This is an issue because while certain faders, sensors, etc. may be OK to interact with in a lossy way, ZoomOSC/ISO are sending discrete and unique data packets. For example, less than 100ms of delay when receiving our ../list output causes entire zoom participants to be dropped from the corresponding Isadora database. 

Could we continue the conversation on figuring out how to buffer and play out these OSC packets when needed, so that we can increase the speed and reliability of the connection between the ZoomOSC API and Isadora? While small audiences are OK to transmit with the 100ms delay we offer as an option in ZoomOSC, if you have a large database of participants, it can take several minutes for Isadora to build its internal database of participant data.

Thanks!

[Juriaan]

@liminal_andy

Hi there Andy!

Let me tag the tech team here so that this could be discussed further.

@Woland @DusX Could you reach out to Andy / see or it can discussed further with the team?

[DusX]

@liminal_andy We are currently working on new features which will offer powerful options for a wide range of use cases. I will contact you directly to discuss stress testing possibilities. 

[mark]

@liminal_andy

So, I wrote a test program that sends OSC messages a high, fixed rate, e.g. every 20 mS. Then I added code to Isadora to measure the arrival of the OSC Messages on both macOS and Windows. The findings of this test does not concur with your assertion that there is a 100mS limit.

When attempting to send OSC messages every 10 mS (100 times per second), I see this output in the sending program:

delta = 11.019800 mS
delta = 10.973900 mS
delta = 10.913600 mS
delta = 10.973200 mS
delta = 10.969300 mS
delta = 10.972600 mS
delta = 10.971900 mS
delta = 11.086100 mS

so it actually ends up being more like 11 mS. This is mainly because the Windows function to sleep for 1 mS (Sleep(1)) is not accurate down to those timing resolutions.

When receiving those messages in Isadora, in the code right before the message is sent to the OSC Listener actor, I see this

CMessageReceiverList.cpp(315): 01:02:59.244 Message Delta = 12.638300 mS
CMessageReceiverList.cpp(315): 01:02:59.254 Message Delta = 9.963400 mS
CMessageReceiverList.cpp(315): 01:02:59.264 Message Delta = 9.996200 mS
CMessageReceiverList.cpp(315): 01:02:59.276 Message Delta = 12.087800 mS
CMessageReceiverList.cpp(315): 01:02:59.287 Message Delta = 10.845400 mS
CMessageReceiverList.cpp(315): 01:02:59.297 Message Delta = 10.008500 mS
CMessageReceiverList.cpp(315): 01:02:59.309 Message Delta = 12.144200 mS
CMessageReceiverList.cpp(315): 01:02:59.321 Message Delta = 11.751400 mS
CMessageReceiverList.cpp(315): 01:02:59.333 Message Delta = 11.961200 mS

So, while not entirely perfect, it's pretty darn close to 11 mS every time. (The average is 11.2 mS)

For me this proves that Isadora is not limited in the way you've indicated. Is it at all possible that whatever method you're using to send the data is not able to send faster than 100mS per message?

Do I need some sort of patch to put this to a further test?

Best Wishes,
Mark

[DusX]

@mark When these OSC messages are received, how does Isadora pass them to other actors? I assume this is done according to the Cycles schedule, so to pass all these messages without one overriding another (or is there a que) the cycle rate would need to be a minimum of 100?

[liminal_andy]

@mark I've confirmed your results. Right now, ZoomOSC jumps from 10ms to 100ms on the transmit rate menu, so it think the issue is that my 10ms option is too fast, and my 100ms is far too slow. I tested at 20ms and it seemed fine with up to 50 people. I think we can leave this one as solved for now. 

Next time I have 300,000 participants streaming into Isadora all at once, I'll let you know ;)