Target FPS as related to output device refresh rate
-
Hi, I have a performance question -- I had always assumed that if I work with video that I create at 24 fps, and set Isadora's target fps to 24, I will be able to stay closer to the original frame rate than if I am working at 30 fps because there is less data to be processed per second. However, the Watchout folks advise with their software that when using 60 Hz output devices, performance will be much better if both videos and target frame rate are at 30 fps. Is this true of Isadora as well? Should I be working at 30 fps whenever possible rather than 24 fps if I am sending to an output refreshing at 60 Hz?
-
Hi,
Well, the fact of the matter is that I haven't done a specific test on this topic, so I can't something about this with 100% confidence. But my best practice is to keep all the video frame rates the same, and set the target frame rate to match. Unfortunately that's mostly gut instinct and my insight from knowing the code, it's not based on empirical evidence.Maybe someone is up for giving it a try. Right now I'm a bit swamped.Best WIshes,Mark -
One way to get around this is by using video resolutions for your output. If I connect my output to a TV I can select a variety of video standards, including 24p, I try and keep everything consistent, 108050i cameras, video clips and output settings. 24p is a strange beast, in NTSC land it is actually 23.976 to maintain consistent sync (kind of) with 60i systems. One thing that might be usefull is to also look at how 24p footage is converted to display on systems running at 50 and 60 hz respectively. The 50hz stuff is easy and simple, 24fps footage is just run faster, no frames or fields are dropped- When we in PAL countries watch DVD's of films we see it all 4%faster. I worked at a post house when HD was emerging and we went through a huge number of possible and impossible conversion from and to pretty much every format and source type. Some conversions were always problematic, some just painful.
To get to NTSC from 24 is also difficult, there are several methods, a simple and common one is a 2:3:3:2 field pulldown, (this duplicates and drops fields depending on which way the conversion is going).The same things are going to happen with your videos (well something has to happen). In many programming environments for playing video we are often given the option to do something like setVerticalSync(true) this means the refresh rate of the content (a film or graphics) is locked to the refresh rate of the monitor- this means no tearing and often a smooth image but it can mean you drop fields or you movie is played at the wrong speed.So the ideal situation is to keep everything simple and have field for field and pixel for pixel pathways from camera to beamer. If you are in NTSC land shoot at 30fps and run your beamers/monitors at 60hz (30fps interlaced is 60hz). If you are using monitors that can run at 24p natively then keep your 24p. 24p is great to shoot at if you go to film, otherwise the amount of work that is needed to make it play nice with video is a pain.Maybe not the simple answer you were looking for but some things to take into account.Fred -
Dear Fred,
All of Fred's comments are very well thought out. Thank you for sharing them.But, Fred, I wanted to ask if you think the option to turn off vertical sync is something I should add? It's actually quite simple to do. Sometimes I feel that too many options are confusing to people, especially since -- if they don't full understand it -- the visual "tearing" that might happen will seem like a big negative. Let me know what you think about this.Best WIshes,Mark -
I have to say when I am programming I do use the set vertical sync function, however I dont think I completely understand all of the execution order sometimes. with smaller simpler projects you can really see it make sense, however sometimes I turn it on in a situation that it should be good for and it makes things look worse. I understand the concept but in action there is a lot more at play.
I would be really curious to see this function is Isadora, just because it has changed things for me drastically at times. But yes, more options is more confusion (maybe you could hide it somewhere). In cases where you do have field for field and pixel for pixel video from camera to output it could be good to have this button- if everything is working well- frames coming in, getting unpacked and drawn in time then there should be no tearing and everything should look great.Fred -
A bit late for an answer but just wanted to share my experience with this issue (my experience with isadora is still pretty basic, I use it in a theatre environment - and I love it!). My first isadora project was mapping a cube on stage with first one still on one side of the cube, then 3 times the same video, mapped on 3 sides of the cube via matthew haber's cornerpin actors. It always stuttered when I moved from the scene with the still to the video scene.. The footage was 25p, Isadora was set to the same fps, the projector only did 60fps. I haven't tested changing the target frame rate in isadora for this project, but when I was working on another project where I also had performance issues, I tried setting the target frame rate to 30 fps while exclusively using content with 30fps. Still the same issues... stuttering on many videos (1024x768, photo jpeg). Then I set the target fps to 60, matching the refresh rate of the projector and watched the status window in isadora. When there were more complex scenes or long fades between scenes (many scenes also included 5.1 audio via the sound movie player), the frame rate would drop significantly, but never below 30 (mostly around 40-50 fps), and I had the impression, it worked smoother than before, because the frame rate was always higher than the one of the source material. (before, the frame rate would drop down to roundabout 24 fps, which was clearly visible on the screen..)
I had concerns, that a 60Hz refresh rate would affect Izzy's performance in general, but it didn't at all! So for the future I will try this method again and see how it works. I also have the impression, reencoding the source material to an exact, integer divisor of the target (and the output device) frame rate is better than to have something odd like 25 to 60 fps. On my Retina MBP it's the same. The native resolution is 2880 by 1800, which works perfect for emulating the original resolution of 1440 by 900, but internally, the graphics card renders everything in 3840 by 2400 and then scales down the image to a HiDPI version of 1920x1200 or 1680x1050\. When I watch videos in vimeo for example, the difference is clearly indicated by the speed of the internal fans. Every resolution except the "best for retina" version of 4 times 1440 by 900 has an odd scaling factor and requires much more precessing power. I guess it works similar when trying to play 25p material at a target rate of 60frames. I would be interested to hear if someone else uses this method successfully..All the best,Daniel -
Dear Daniel,
Thanks for these thoughts. But I'm curious about your statement "but internally, the graphics card renders everything in 3840 by 2400 and then scales down the image to a HiDPI version of 1920x1200 or 1680x1050" -- where did you get that information?Also, I'd like to know how many 1024x768 videos you were playing at the same time when this stuttering occurred. (Keep in mind that when transitioning from one scene to the next, Isadora is playing twice as _all_ the videos in both scenes -- i.e., if you have three movies in Scene 1 and two movies in Scene 2, during the crossfade Isadora is playing 5 movies at the same time.)Furthermore, I'm wondering, when mapping on to the cube, if you really need this resolution. Consider the scenario shown in the attached picture. The projector resolution is 1920x1080 -- but the number of pixels occupied by the cube is far less than that. (And even less than the values shown if you consider the individual faces.) Maybe your scenario was different, but in this situation a video resolution of 1024x768 would be overkill -- you'll never see all those pixels. When playing multiple clips, reducing resolution is the single biggest factor that will affect performance.In any case, I appreciate hearing that the 60fps solution worked for you. That's good to hear, and other users will benefit from experimenting with those settings.
-
Dear Mark,
thanks for your thoughts as well! Here's the article from anandtech that explains how scaling works on the retina macbook: http://www.anandtech.com/show/6023/the-nextgen-macbook-pro-with-retina-display-review/6"If you select the 1680 x 1050 or 1920 x 1200 scaling modes, Apple actually renders the desktop at 2x the selected resolution (3360 x 2100 or 3840 x 2400, respectively), scales up the text and UI elements accordingly so they aren’t super tiny (backing scale factor = 2.0), and downscales the final image to fit on the 2880 x 1800 panel. The end result is you get a 3360 x 2100 desktop, with text and UI elements the size they would be on a 1680 x 1050 desktop, all without sacrificing much sharpness/crispness thanks to the massive supersampling. The resulting image isn’t as perfect as it would be at the default setting because you have to perform a floating point filter down to 2880 x 1800, but it’s still incredibly good."When mapping the cube on stage, I had a projector with 1024x768 resolution, the movies and stills were not larger than that. In the first scene there is a picture of a bird mapped on one side of the cube, the next scene has a video in it which is attached to 3 cornerpin projector as it is mapped on 2 sides of the cube, plus the back wall of the stage. so there is one movie and one still photo playing when transitioning between the two scenes. I'm sure the resolution of than picture plus the movies could even be reduced, given a total resolution of 1024x768, but I don't think this is the problem..I haven't worked on this show for a couple of weeks now and will experiment a little with it as soon as I can.Thanks for your insight, I'll try different resolutions and target fps!Best,Daniel
