[SOLVED] Ongoing Depth camera issues

Vidasonik

@woland hi thanks for your reply

Points 1 and 2:

Yes, my first thought was to use Blackhole etc (I did have Loopback but that caused other issues so deleted it) however this introduced very noticeable latency so was not viable. Why won't Izzy receive audio from the Clarett soundcard?

Point 3:

I bought the router on the advice of Izzy specialists but after spending many frustrating hours (on line manuals, helpdesks, customer servces etc) never managed to get it to work. Again, the depth camera should work on a modern mac.

jfg

@vidasonik said:

Point 3:I bought the router on the advice of Izzy specialists but after spending many frustrating hours (on line manuals, helpdesks, customer servces etc) never managed to get it to work. Again, the depth camera should work on a modern mac.

you could try to connect both Mac with a ethernet crossover cable.

Best regards,

Jean-François

Vidasonik

@jfg sounds good! Thanks.

Woland

@vidasonik said:

a 'keyboard watcher' connected to a 'midi send' and 'OSC Transmit'? like this:

Not quite. You'd want the actors to be more like this (assuming you're using Computer 1 to trigger Ableton on Computer 2 with a Keyboard Watcher actor on Computer 1):

For more info on using Isadora to control Ableton, see this article: https://support.troikatronix.com/support/solutions/articles/13000081201-control-ableton-live-sessions-with-midi

You would also need to have the computers networked together so that you could send the OSC between them.

@vidasonik said:

@woland so a capture card is a piece of hardware?

Yes, it's a piece of hardware that, for example, could be plugged into an HDMI cable coming from computer 1 in order to get that video signal into Isadora's Live Capture as a video input device on computer 2.
Example: Computer 1 video output > computer 2's HDMI output port > HDMI cable > video capture device's HDMI input port > video capture device's USB cable > computer 2's USB port > Isadora's Live Capture Settings (selected as video input device for channel 1, for example) > Video In Watcher actor.

@vidasonik said:

Surely using Video In watcher brings us to the same problem that you can't transmit large files over a local network?

No, it wouldn't have anything to do with networking but, depending on the distance between the computers, it may not be viable to use a common video cable like HDMI/DVI and you might have to buy a repeater, extender, or use SDI, all of which is going to end up being more expensive than an ethernet switch and a couple of ethernet cables. Additionally, while it is possible to get the video signal from one computer to another using a video capture device such as a Blackmagic UltraStudio Mini Recorder or Elgato Cam Link 4k, it doesn't give you the capability to transmit any data so it doesn't solve your synch/cueing problem.

@vidasonik said:

Why won't Izzy receive audio from the Clarett soundcard?

The first thing to do is check all four of your Isadora Live Capture channels to make sure that you don't have the Clarett selected as the audio device for channels 2, 3, or 4, because if it is selected on one of those it won't show up as a selectable option in the list of audio devices in Live Capture Channel 1. If that isn't it, then my second guess would be that this might only show up as an audio output device to Isadora for some reason, but that would be very strange, as I have two Focusrite devices that work fine for audio capture into Isadora.

@vidasonik said:

I bought the router on the advice of Izzy specialists but after spending many frustrating hours (on line manuals, helpdesks, customer servces etc) never managed to get it to work.

I am unclear why the recommended router wouldn't work, as it was specifically recommended because it's the exact router a member of our staff uses for precisely this purpose. I completely understand your frustration because both myself and another staff member went well outside the scope of what we officially offer in terms of technical support by providing multiple hours of consulting, video calls, programming, and troubleshooting free of charge in order to try to assist you in getting the router and networked computer setup running, as well as modifying your Isadora files to send and receive OSC and NDI. So, while I apologize that the time and energy spent on trying to get that specific router working for you didn't pay off, I would still like to stress that just because that one device wouldn't work for you doesn't mean you should abandon the entire approach. If your car breaks down and nobody can figure out what's wrong with it, that doesn't mean you should give up on driving forever and never get another car. You could pick up a 4/5-port gigabit ethernet switch for 16-25 euros and try again. You even saw that the setup worked as a proof of concept over your wifi when we were on a call together testing the specific Isadora files I made for each of your computers and at that point I said that all you'd need to do is get your own router or a switch and connect the computers to it with ethernet, and get a LAN up and running.

@vidasonik said:

local internet connection in theatres very unreliable

I also distinctly remember strongly recommending against trying to use wifi or a theatre's existing network for performances because you could easily run into any number of problems that would be outside of your control. You want your network to be locked down and private so that no outside factors can affect it. Additionally, traveling with your own router/switch means you already know that your network setup works, instead of trying to figure out something different at each new venue.

@jfg said:

you could try to connect both Mac with a ethernet crossover cable.

This is definitely worth a shot, but I'm not sure you'll be able to send NDI and OSC this way because two computers connected directly via a network cable don't necessarily always behave exactly the same as two computers both connected to a router/switch. I'm sure someone who knows more about networking than I do could explain it in better detail, but as far as I know, certain aspects of networking require a router or a switch in order to work properly.

@vidasonik said:

Again, the depth camera should work on a modern mac.

We didn't design, manufacture, nor do we sell the hardware and we are not able/allowed to alter the OpenNI code that is used to make it work because it's not ours, so we're neither able to, nor required to make it work on modern Macs. Despite having no obligation to do so, we have done our best to maintain the OpenNI plugins for years but, as far as I understand, the OpenNI code was written nearly a decade before the first M1 Mac computer even existed so there's simply not much we can do anymore. Writing our own library for this from scratch would require more time and resources than we have right now and trying to adapt a different library by somebody else would present its own problems, namely that 1) we'd have to find a library that is both good and allows for commercial use without an enormous licensing fee 2) we'd still have to invest more time and resources into adapting and testing it than we currently have, and 3) it very likely wouldn't work in the exact same way as the OpenNI library (might provide different data, differ in accuracy, not have one or more features we relied on, etc) so an existing Isadora file originally built using the OpenNI plugins would likely need significant adjustments to make it work with the theoretical new tracking plugin or, if the Isadora file required a feature that existed in OpenNI but didn't exist in the new library, it wouldn't work at all.

Best wishes,

Woland

Vidasonik

@woland Thanks so much for this comprehensive reply. Indeed you spent a lot of time with me (as you say, way beyond the call of duty, and I expressed my gratitude at the time) helping to set everything up and I remember feeling confident that a solution had been found which merely needed implementing.

I think I will need to step back from all this and reflect on everything to see if this is worth pursuing further, ie does what I am trying to do justify all this effort from yourselves as well as myself. I would hate to abandon it if I felt a solution was within reach, however. Thanks so much for bearing with me.

All the best, David

Woland

@vidasonik said:

I would hate to abandon it if I felt a solution was within reach, however.

For the record, routers tend to be trickier to set up than ethernet switches. Switches tend to be plug-and-play whereas routers can have more options to fiddle with.

Vidasonik

@woland ok can you recommend an ethernet switch?

DillTheKraut

@woland

Sorry, but as a network engineer, I have to correct your comment. There are complicated switches and simple switches, as well as simple routers/Access points and complicated setups, if used wrong. There are three common types of network connection devices: hubs, switches and routers. But there are different (sub-)types of switches, which can lead to issues, if not known.

- Hubs are very dumb devices, which blow all data from and to all ports, without any logic, which can cause them to slow down in data rich environments. There is only a few to buy nowadays, as switches became the standard.

- Switches are smarter hubs. They know which device is connected to which port and can therefore route data more efficient. But they come in two flavors: managed and UNmanaged. The later is kind of plug and play, where the managed ones try to optimize the data flow on a deeper level by default, which can be a struggle in some cases. In these situations, you need to know how to manage (pun intended ;) ) them, which needs deeper knowledge of network protocols and special network functionality.
Some managed switches as well come with very different types and numbers of extra functionalities and implemented services, which some of are like the ones routers usually provide (like DHCP). But usually these are deactivated by default.

The issue with MANAGED switches is especially true for protocols like NDI. The keyword here is multicast!

- Routers again are a special type of devices, with the main purpose of translating between two or more different networks, e.g. Internet and LAN. Some smaller wifi routers provide connectivity for two computers or even one only. The most bigger ones have an integrated hub or switch, which provides more ports to connect more devices. This should not be more complicated or different to use, like the stand alone hubs/switches, regarding their simple switching functionality.

But on top of this, they provide some extra services, like DHCP, which are active by default in most cases! Those usually do not interfere with the general network functionality. But though some CAN be used to simplify a setup, if used wrong, they can become an issue. Beside this there might be some routers with managing functionality as well as the managed switches and therefore have the same side effects. But they are usually upper class, upper price range.
Basic thing you have to be aware of, is the different types of ports they have (LAN, WAN, etc.). For pure local network functionality as a hub/switch, you would only use those ports with LAN functionality. Not the ones with WAN on it.

@Vidasonik
For a simple setup, have a look for UNmanaged gigabit switches with the number of ports needed. TP-Link, Netgear, Linksys are common brands in that class and don't have big differences in general functionality. Switches with 8 ports and less usually are unmanaged, if not otherwise stated.

If not already done, you should as well get a little bit into basic network knowledge, e.g. IP addresses and subnetmasks, as well as DHCP service.
A little hint: Manually set IPs and DHCP services (e.g. by default provided by most routers) in the same network, can clash, if not set up in a propper way.

DillTheKraut

@jfg

You nowadays don't need any crossover cables anymore as any gigabit device, be it the computers network port or the switch, are able to identify active and passive connections. You can just use any usual RJ45 Cat 5 and up cable.

These cables/adapters are from old 100mbit times, and where called uplink cables. Some hubs had a special uplink port to connect to other hubs without needing an uplink/ crossover cable. Modern switches, doesn't need those anymore.

DillTheKraut

@woland said:

This is definitely worth a shot, but I'm not sure you'll be able to send NDI and OSC this way because two computers connected directly via a network cable don't necessarily always behave exactly the same as two computers both connected to a router/switch. I'm sure someone who knows more about networking than I do could explain it in better detail, but as far as I know, certain aspects of networking require a router or a switch in order to work properly.

This is not true at all! Two computers connected by a standard network cable are just the same as with an unmanaged switch in between.

But you are right in the fact, that you need to do some basic settings on your own in both cases, if it isn't done by a specific device in the network, like a router with an DHCP Service.

DillTheKraut

I don't know what was already done in the support sessions @Woland is referring to. But while reading these comments it seems to me, that there might be some basic network setup issues.

I might state one important basic: If you don't have a DHCP server in your network, which is managing the IPs of all devices connected automatically (most routers provide a service like that), you have to set these IPs for every device your self. To get this right, you have to know a little bit about IP and network addresses and basic networking knowledge. This is especially true, if you add to an already existing network, like those in theaters (where often personnel with specific network knowledge is missing), without knowing its structure and setup.

Just putting devices in a network, without these basic settings might work (there is an auto IP set by default), but then is out of pure luck.

Woland

@dillthekraut said:

Sorry, but as a network engineer, I have to correct your comment.

Yayyyyy! I love learning more, especially when it leads to people getting better advice than I could give them. Thanks so much :)

(This is also why I put a bunch of "I think" and "As far as I know" language into the advice I was giving, as I wasn't entirely certain but was sure there was somebody out there who knew more than me and could correct me if needed.)

@dillthekraut said:

A little hint: Manually set IPs and DHCP services (e.g. by default provided by most routers) in the same network, can clash, if not set up in a propper way.
[...]
I might state one important basic: If you don't have a DHCP server in your network, which is managing the IPs of all devices connected automatically (most routers provide a service like that), you have to set these IPs for every device your self. To get this right, you have to know a little bit about IP and network addresses and basic networking knowledge. This is especially true, if you add to an already existing network, like those in theaters (where often personnel with specific network knowledge is missing), without knowing its structure and setup. Just putting devices in a network, without these basic settings might work (there is an auto IP set by default), but then is out of pure luck.

Using a separate network and static IPs is the way I've always worked on things like this, so everything on the network I'm going to be working with is mine, locked in, simple, and known.

@dillthekraut said:

This is not true at all! Two computers connected by a standard network cable are just the same as with an unmanaged switch in between.

But you are right in the fact, that you need to do some basic settings on your own in both cases, if it isn't done by a specific device in the network, like a router with an DHCP Service.

Aha! I knew there was something to it. Probably when I've tried this and seen people try it, the issue was that I or the person was being tripped up by settings we didn't know we needed to change. So two computers connected and configured correctly can send NDI and OSC back and forth? What setting would need to change? I'd love to know how to do this.

Vidasonik

@woland Thanks everyone. OK I need to take small steps and rule things out or in, one by one. What about establishing a local area network between the two macs? I only need to send data between them not video. I'm resigned to using Mac 2 (old one) for visuals (depth camera and webcam). By local area network I mean selecting 'Create Network' under the Wi Fi menu. It defaults to Channel 11. So is this or is this not a safe/robust/reliable method for linking the two macs without routers, cables etc.? (I have connected the macs in this way before for other purposes)

in hope, David!

Vidasonik

@vidasonik I've solved the main issue myself! Using Ableton's Link facility and setting up a local area network between the 2 macs so when I press play on one it starts both !!

DillTheKraut

@vidasonik

I'm happy to hear you found a solution! Could you tell how you set up the network? Did you use a switch or connected the computers straight up without some distribution device?

Vidasonik

@dillthekraut Hi! Thanks for getting in touch! I just set up a local network on the old mac then connected to that with the new one. Then I set up my dummy start clip in Ableton, selected Link on both macs, activated the scene play button with transport off on both. Then when i hit start on either the other starts.

However, although the first stem tracks are in sync after a 120 sec interval the second tracks are NOT quite in sync. The settings in the Izzy scenes are identical and I've checked both Ableton clips and I cannot find any reason why they are not still in sync at track 2 and after!? The show consists in seven tracks with intervals and midi notes are sent between the two programmes.

Anyone got a clue?

Cheers, David

Vidasonik

@dillthekraut Actually, maybe I should just control everything from Ableton which can synchronise two projects on separate computers perfectly (without routers, cables, switches etc.) Izzy only has seconds for timing whereas Ableton has bars/beats/and sixteenths. Probably best to put Ableton in control and just send midi notes one way.

DillTheKraut

@vidasonik

To know what really is going on and what the reason for the missing sync is, we would need more details about your patch to tell more about how to optimise.

But there are quite some threads about syncing and how to achieve the best possible results in this forum, already. There are (like most of the time with isadora) different approaches and solutions. some are very specific for very specific goals.
There are even some about Ableton and Isadora syncing like this thread from longtime Isadora Specialist Skulpture, who offers some Tutorial videos about this toppic.

But if those doesent fit for your case, or you have fourther questions, don't hesitate to come back here!

DillTheKraut

Hey @Woland,

I just recognised, that I ignored your last part about how to set up the Network on two computers. This wasn't intentional.

Generally, the setup without a distributing device is exactly the same like you do it with a switch inbetween.

Network setup can be irritating and difficult at times. To cover all scenarios and possible issues, would be to much for this thread.
But as I don't know about your basic network knowledge and as well for everyone who wish to know, here is a "short" recap:

Probably most people know, that ip addresses are needed. Every device has to have a distinkt address (IP Address), which must not exist a second time in the same network. All devices which needs to communicate with each other, has to be in the same 'range' of Addresses. These ranges are defined by the 'subnet mask'.
You need to know these addresses, to set them accordingly in the software like Isadora to reach them. (There are other options, but these won't work for all actors/protocols, isadora uses.)

There are (generally) three posibilties to address this.

1. Have them set up automaticaly.
This is the default setting for most OSes and is called Automatic Private IP Addressing (APIPA)
When connected to a network (another computer, or a distributing device like a switch), the OS sets an IP in the range 169.254.0.0 - 169.254.255.255. It automaticaly checks, if an IP allready is occupied and will prevent using it a second time.

The issues with this:
No controll at all about the IP settings. Therefore the need to find them. How easy this is, depends on the OS and other factors.
The IPs might change with every reboot, or even reconnect.
Internet won't work at all in most cases.
Most probably no interaction with computers that has set their addresses in a different way.

2. Using a device which is managing the settings centrally. This service is called DHCP (Dynamic Host Configuration Protocol).
Most routers provide this for easy home LAN and internet connection setup. This DHCP service provides all settings needed by the requesting device and keeps a list of IP adresses, which it gave to the devices.
Most devices which use APIPA, are actually first sending a request to the network, to see if an DHCP service exists. If there is one present, they get the needed settings from it. If they don't get an answer, they go on with APIPA the way described in 1. .
This is working fine for the usual home or office networks, but:

Issues with this:
Control is only managable with the DHCP Service itself and the knowlegde about how to handle it.
By default, IP addresses come with a lease for a set amount of time (usualy 3 days). If the device is not connected in the lease time again, it might loose this ip and will get a new one.
If there are two of those DHCP services present in the same network, they will most likely clash (putting a router in an allready existing network, like in the venues, will most likely have this issue).
Combining this setup with manually setup addresses, needs knowledge about the DHCP server settings (which is the other potential issue, integrating into venue networks).

3. Set a static ip manualy in the settings, provided by the operating system.
Because of the issues listed above, I would strongly recommend going this way. Even if you don't have alot of knowledge about network principles.

Biggest issue: The need for knowledge about the basic IP addressing principles.
This might seem counter intuitive. But with these basics you can tackle issues, which you could only manage with even more diving into network tech when using the other seemingly easier options.

I strongly suggest to have a dip into this with the help of other pages, like this.
At least the parts 'What Is an IP Address?", "The Two Parts of An IP Address" and "The Subnet Mask", will get you going. "DNS" and "The Default Gateway Address" is needed for internet connection, but even if you won't need internet, setting a default gateway will help with issues on Windows, as it takes it as the basis for firewall and other settings.

Other possible issues (There might be a lot more, but these are the most common):

Firewall settings: In generall firewalls and antivirus apps should be put off. At home and most other closed network environment situations, the router will handle this anyway. Firewalls are mainly needed in unsecure situations, like open wifi or other unknown networks.

Port adresses: Protocols like OSC, ArtNet, NDI are using a specific IP port. Every IP address holds more then 65000 possible ports. Some protocols like HTML or certain email protocols, use officially defined ports. If two programms (or actors) use the same port on the same computer, often one of them is blocking it completly. Depending on which one took it first, the following apps/actors can't use them and won't work properly.

Some protocols like NDI, use a "sub protocol", to anounce them self on the network. This is meant to help find compatible devices on the network, without knowing the exact IP address. Some managed switches prevent this "shout outs" to propagate, if not propperly set up.
This discovering method might even work, if the IP adresses are set up wrong. Which might confuse the user, as getting the actual payload (e.g. NDI videostream) still isn't going to work properly, dispite the fact, that the devices are listed as beeing available.

These possible issues are only described in their basics. In reality, they might be even more complex. Like NDI isn't using only one port, but multiple dynamicaly. There are different types of basic network protocols, like TCP and UDP on which are the application protocols (OSC, NDI, etc.) are build on. Specific protocols, like ArtNet are using so called Broadcasting, which can lead to flooding the network, making it very slow.

Beside that, there might be other issues, especially regarding Windows, which sometimes is very confusing about how it handles networks (Keyword here is "private vs. public network", see "Gateway settings" above).

If computers use more then one network card (even wifi in parallel to cable ethernet), one need to make sure, the apps are using the right one. Some offer a setting for it. Isadora e.g. doesn't. Which IP (and therefor network) is used by Isadora, is shown at the lower edge of the 'general' Preferences pane.

This all might seem to be alot to swallow, but with the basics about IP addresses called above, the most cases should be easy to handle.