severe video sync issue

[Lauri]

@Fred Thank you for your reply. And sorry for not responding earlier.

Sony a6300 HDMI out options are ”auto”, 2160p/1080p, 1080p and 1080i. There’s no 720p HDMI out.

So, with Magewell, I’m forced to down-res the signal to 720p. The incoming HDMI signal is 1080p50 and outgoing 720p50 that are both shown in Magewell USB Capture Utility. (So, Magewell accepts the incoming 1080p50, while BM UltraStudio Mini doesn’t.) I can verify the down-ressed signal both from Magewell USB Capture Utility (Capture format: 1280x720, 50fps, YUY2) and Isadora Live Capture Settings (Video Input: Resoluton Native, Format 1280x720).

Yes, it’s true that 2x1080p50 is too much for USB lane.

(With BM UltraStudio down-ressing is not possible, it does not accept 720p setting, black screen. And, as told before, BM UltraStudio Mini does not accept 1080p from Sony HDMI. BM UltraStudio Mini accepts only 1080i50 (or i60) from Sony HDMI, keeps it as that or converts to 1080PsF25 (or PsF30), but you will see the interlacing on both. So, there’s no real 1080p25 or 1080p30.)

However, as mentioned before, with Magewell down-ressed signals, Isadora captures both simultaneous camera signals flawless, no dropped/duplicate frames. So, if it can do that, why it cannot draw the streams properly? Even when only streaming the live cams on stage, no recording involved.

--------

When using BM UltraStudio Minis for capturing live cams (1080i50 or 1080PsF25), there’s also some sort of stuttering that happens cyclically. It looks different, but that’s probably because the framerate is not 50fps.

(P.S. Interestingly, when using BM UltraStudio Minis and cam HDMI out 1080i50 (the only setting that works with BM UltraStudio Minis), I can get cam record to file only if using Apple driver. If using Blackmagic driver, the recording starts but there are no files.)

--------

When replacing the Video In Watcher actors with Movie Player actors and playing the files through the Video Mixer actor, the result is very stuttery. This should not be heavy task at all. The live cam recorded files are Photo-JPEG (720p50, of course), no dropped/duplicate frames. VLC shows Codec details: Motion JPEG video, Planar 4:2:2 YUV full scale, Color Space ITU-R BT.601 Range.

When playing the recorded live cam file outside Isadora, I get strange results (I have checked that the file does not have any dropped/duplicate frames):
- playing the file with QuickTimeX, the file stutters a lot, there’s some kind of cycle that the file doesn’t stutter, but mostly stuttering - playing the file with QuickTime7, the file stutters in cycles, but has longer periods when not stuttering and the cycles are more regular
- when playing the file with VLC, the file doesn’t stutter at all, except that there’s one dropped frame every 72secs
This brings me back to think about a sync issue.

----------

Also, usually when starting the patch (Q4, only playing the superimposed live cam streams, no recording), there’s no stutter for about 2mins and then the stutter starts first in one cam – and then the stutter changes to the other cam and stuttering alternately in a cycle of about 7mins.  So, why would it play well for a while and then start stutter later, if the problem is the ”cross format processing”.

----------

I downloaded OBS but didn't really get it work in a similar way to record (or just show) two superimposed live cam feeds simultaneously.

[Lauri]

@Armando 

No, I use Isadora 2.6.2b3 (or also 2.6.1), so it's not 64bit. However, as stated before, I made the tests both with MacPro OSX10.12.6 and MBP OSX10.14.4. I know Isadora 2.6.x is not compatible with OSX10.14.x, but combination works with no problems in my MBP(2018). So, in this context, forget about the MBP/Mojave, the issue is prevalent in MacPro/Sierra.

[Fred]

@lauri well the mini recorder is only for 1.5 GHz signals, clearly stated in all the specs, on the website and instructions and even in the setup utility. 1080p50 is 3ghz, so if course it will "not work", it adheres to all specs but you are trying to get it to do something it does not, there are other products from black magic for capturing 1080p 50, best to do the proper research before you purchase. The mini recorder is functioning as it should and has no "problems" with the sony output, it is not designed to capture that format and never was.

So even though you are capturing at "720p", your input is 1080p to the magewel and you are using that bandwidth on your usb3 bus so that for sure is part of your problem .There is no magical conversion, your capture device is running at 1080p50 and then running a conversion on every frame, if you were not trying to record and further push the limited resources you have I would suggest just running at 1080p50.

The artefacts you see from the 1080i  and the 1080psf25  capture are more to do with the sony output from this consumer camera than any of your capture devices. If the camera outputs a proper deinterlaced signal it will look fine, however many of these cheaper souped up DSLRs cannot really do on sensor 1080i. Instead it is converted afterwards.

As your cameras do not output 720p you should really run at 1080p, but your system for USB based capture is under spec (as it is for this setup @ 720p,  as you are still capturing at 1080p on the magewel and those frames are uploaded to the CPU for processing and then delivered to Isadora at 720p).

Running a whole system at 1080p is not that easy (which even though you bump down to 720p after capturing, you are actually capturing at 1080p). The Gear you are using is going to be pushed trying to do this, in general apple don't make anything ready for this, no good gfx cards and getting a system equipped with proper 1080p capture is very expensive.

OBS is simple to use and can preview and record 2 streams, I think it is worth reading the manual and checking your system can do what you want it to before looking for faults in isadora, and as well you are running an unsupported system. I know this sucks, Isadora is not ready for Mojave yet and every new computer can run Mojave.

As media artists using this kind of gear with high expectations there is a lot of research involved, we are not running some simple software and it takes good planning and piece by piece testing to make things rub smoothly and reliabily.

[Lauri]

@Fred Thank you for your reply!

First, I'm not running on Mojave! I've said that many times in my posts. I've tried the patch in my Mojave-laptop, BUT I'm running the tests in my MacPro 10.12.6.

Ok, I understand that I'm at/over the limits of the MacPro. And I do understand there's a lot research involved. I would appreciate all the help in finding a solution. It feels just a bit frustrating, I do not know where to head forward – if it's the case for only needing to change the computer, operational system, cameras and capture devices.

[Fred]

@lauri Ok, sorry, I saw you had removed the mojave system from the setup for now.

Lets try to make a few tests to see what the system can handle, sorry if some of these are repeats of what you have already done, or what you have mentioned, but we can try and thorough.

First a few questions about your setup, how clean is your install? Do you have a lot of other software running (especially adobe)? Was the current os 10.12.16 a clean install or upgrade from  previous OS?

For testing lets go with 2 screens, one for the interface and one for the output, set them up to both be 1080p50.

For the following tests leave the system running a while and take some notes while running activity monitor of what Isadora and quicktime are using resource wise (so CPU and ram). Whenever you report with activity monitor also sort by CPU usage and RAM usage and take note if there is anything using more resources than the app we are investigating.

The following tests should be run for 15 minutes or so each, if you see any of the skipping you mention please record it with a phone so we can see it without it causing extra load on the system.

1. Use the utility to reset all the settings on your magewel devices, plug in just one and capture at 1080p50 making sure the output of the camera is 1080p50 (by connecting to a screen briefly) as well and the system is not doing any conversion. Use Isadora with a blank patch and just a video in watcher attached to a projector showing the image full screen on the second monitor. Report what you see on activity monitor for ram and CPU and what you see visually, as well as your Isadora target frame rate and also take note of the cycles.

2. Use the same test but grab at 720p and report what you see on activity monitor for ram and CPU and what you see visually, as well as your Isadora target frame rate and also take note of the cycles.

3. Close isadora and open quicktimeX go to new movie recording and select the magewel capture device  and capture at 1080p50 and report what you see on activity monitor for ram and CPU and what you see visually

4. Open quicktimeX go to new movie recording and select the magewel capture device  and capture at 720p50 and report what you see on activity monitor for ram and CPU and what you see visually

5. Return to Isadora, use the magewel utility to reset all hardware settings to default for the second capture device, make a new isadora patch with 2 video in watchers set to grab at 1080p50 and 2 projectors and use the mapper to show the inputs side by side. Report what you see on activity monitor for ram and CPU and what you see visually, as well as your Isadora target frame rate and also take note of the cycles. 

6. Perform the above test at 720p50.  Report what you see on activity monitor for ram and CPU and what you see visually, as well as your Isadora target frame rate and also take note of the cycles. 

7. Go to your applications folder and duplicate the quicktime Player application so there is 2 copies. Open one at a time and select new movie recording and choose a magewel device, repeat for the second copy of the application and choose the other device. Report what you see on activity monitor for ram and CPU and what you see visually with the previews running one on each screen, you should be able to select different resolutions so try this at 2x720p50 and 2x 1080p50

7b. Try to record from both of these at the same time, trying for both 1080p50 and 720p50

8. Please run blackmagic disk speed test (it comes when you install the blackmagic driver) on the drive you are using as your target to record video from the stage to and take a screen shot and post it.

8b. Please run blackmagic disk speed test (it comes when you install the blackmagic driver) on the drive you are using as your target to record video from the stage while you have Isadora open and showing one live camera full screen, try this at 1080p50 and 720p50 to and take a screen shot and post it.

9. Make a stage recording test, make a new Isadora patch and with some moving text using text draw or some other moving shapes and make it full screen on the second monitor (running 1080p50), report what you see on activity monitor for ram and CPU and disk usage. Try with proress, h264 and photojpeg.

These are the tests I would do to find the bottleneck. Ideally it would be good to do all this with a clean install and no other software running.

If you have the time to do this and let us know the results we can try to see what is getting you bogged down and if there is a way to tune your system to get what you want out of it.

As a last possibility, I am curious if you switch all your system over to 1080i50, all cameras, all capturing and all screens (some screens will not do this, so if possible try find ones that do) I am curious what results you get. I did some checking and the 1080i output of your cameras should be OK, I am guessing you see interlacing as maybe the field order is reversed (I think you can change this in the magewel settings) or because when you switched over your output screens were still progressive, they should also be 1080i for this to work. This is a less desirable pathway for you I know but it would mean you could stop the system doing a real time format conversion from 1080p50 to 720p50 and keep a low data rate that your USB bus should be able to keep up with.

Getting more out of the computer is going to be tough, it has no PCIe expansion and only thunderbolt 2, so your choice of capture cards is pretty limited, even via expansion chassis via thunderbolt.

Have a look at performing these tests and reporting the results, I cannot guarantee that it will reveal the problem. I will re-iterate that there are definitely people that can do multiple channels of capture without issue. 

If you can get everything working without the stage recording maybe there are other ways to get the job done. I dont really trust stage recording and generally hand this off to a hyperdeck hard disk recorder that i can remote control and feed back through a video mixer/router. It is for me a much more peaceful workflow. You would be surprised how much work recording a video is for a computer.

[Lauri]

@Fred 

Thank you for your support and kindly providing steps for tests!

Unfortunately, I’m travelling to France tomorrow and returning in the beginning of May, and cannot go back to my studio before that. So, I need to postpone the testing, my apologies for any inconvenience.

A bit about my MacPro system, I have all my apps in the machine, including Adobe, but Creative Cloud app not running and auto launch always dis-activated. And always when using Isadora, I quit/force quit all adobe processes from Activity Monitor, and other apps are not running.
I believe Sierra was not the original OS installed and was installed over the previous OS. As you recommend, I suppose, I will do a clean install of either 10.12 or 10.13, install Isadora and capture drivers to a clean machine, and make the tests – although in the beginnig of May. Sorry for the delay, hope it's ok.

Thanks again!

Lauri

[DusX]

I have been trying to follow this thread. Unfortunately I do not have the hardware to do a fair test myself.
If Fred is correct in how the Magwell capture works (sends video out the Computers CPU for scaling) then yes, it seems 2 streams is likely at least a peice of the problem. This could be solved by using hardware scalers, before capture. If the feeds are scaled to 720 before reaching the capture device, that will open up your bandwidth.

[Lauri]

@Fred Thousand thanks again for your test instructions! And my deepest apologies for this long long delay in replying – my schedule has gone up and down during these almost 2 months, truly sorry! I hope you would still be willing to go through my test results and give me feedback, if possible. And if needed, propose some further tests to be done.

There are some things I find a bit peculiar, for instance that the cycles usually stay up for a long time and suddenly start to sink and rise back again for long time. But maybe there's a simple explanation... Anyway, the test results became a bit extensive in length, so I thought to put them into a pdf, instead of posting it straight here.

IsadoraTests.pdf

Please, anybody, have a look on these.

@DusX Thank you for your comment too. It would be nice if a hardware scaler would be a solution. But again, while I used 2 Magewells, there was no dropped frames in the captured video files, only on the stage, and even on the stage the channels didn't mostly stutter simultaneously...

Any advice welcome, of course.

Thanks again in advance,

Lauri

P.S. also eagerly waiting for the new version!

[barneybroomer]

Is your wifi or any lan connection to the internet off?

[Lauri]

@barneybroomer Thanks for your reply!

When running a patch, I usually keep the machine disconnected from internet, if I don't need the connection with the patch.

[barneybroomer]

I read the PDF and I am pretty sure it is the Magwell's causing the problem. One thing that bothers me that if you do a recording in QT X it "sees" 720P, while you set the Maxwell to 1080P. That is already a sign. Are you able to set the recording in QT X in high res, next to the red rec button?

Furthermore the encoding from the Maxwell sends the CPU sky high, this is not normal. Sign 2...

Did you try another converter over thunderbolt for example? I have the same machine and capture with Black Magic Design, but all in 1920P, with no issues.

If you can wait for Izzy 3, it is promising.  I test on a Macbook with external GPU with 6 beamers and one BMD input all in 1080P I/O , over one 40Gb cable,  my Macbook is running under 10 percent.... 64 bit crazyness!

I have to test Izzy3 on the AMD700 as well, see if the GPU's will get in effect this time. 

If I were you I would get my hands on a BMD to test over firewire.

[DusX]

@lauri said:

for instance that the cycles usually stay up for a long time and suddenly start to sink and rise back again for long time

This sounds like a potential conflict between your media framerates, capture devices framerate, and your output displays refresh rates.
If all match things should be pretty stable, but if they do not match you can see a rhythmic fluctuation.

Perhaps @Fred can share some insight. This isn't my strongest topic.

[Lauri]

@DusX Thanks for your reply. Yes, let's hope @Fred could comment on the results.

@barneybroomer It's my fault. I made the test3 without acknowledging how to set QT-X capture 1080p, so I have to make the test3 anew.

As told before, I have BM UltraStudio Minis, but I can get only 25fps out (well, 30fps also), there's no chance for 50fps, so they are not straightforward comparable. But yes, I will make a test with them too, and will report here, soon hopefully. I suppose, to compare the 25fps BM UltraStudios, I should make Isadora run 25fps also (instead of 50fps).

[Fred]

@lauri Hi, I don't have much time right now but I read through the PDF, as far as I can tell from the results, your system just cannot do this, it is not a fault with Isadora (seeing as though you get similar errors just using quicktime). I will try find a way to check a little further  in the coming days. I think the next logical step is I can do some tests on my systems with a single magewell and see what results I get, mainly to check if it is a problem with Isadora (which so far from your results it is not). After that it is hard to find the actual bottelneck and with the computer you have you have very few options to use a different pathway (eg some kind of thunderbolt dock with usb 3 to get off the built in usb ports) and the machine is not upgradeable in any way.

[Lauri]

@Fred Hi, thousand thanks for your reply! No worries, take your time and reply when you have time.

I made a mistake with the test, as I wrote in my previous message. So, I made Test 3 anew with the correct capture settings in QT-X – there was no stutter, but please see the test results continued in the attached pdf.

In the same pdf, I added the Test 5, but made with Blackmagic UltraStudio Minis (although 25fps, instead of 50fps). And I get same kind of results as with Magewell. But, again, see the attached pdf.

IsadoraTestsContinued.pdf

Thanks in advance for all the help!

Lauri

[Fred]

@lauri Ill try to check it out in the coming weeks, I do have to say though that I have used multiple blackmagic capture units extensively on mac and PC with Isadora and not experienced any problems.

[Lauri]

@Fred Thanks for your reply! I have no doubt about that, but maybe the fault is somewhere else. @DusX proposed, there might be "a potential conflict between your media framerates, capture devices framerate, and your output displays refresh rates" or maybe the problem is something different. However, I'm still hoping there would be a solution.