[SOLVED] FreeFrame actors issues on V3

[antoningm]

@woland

Actually, FrParameters offers these controls:

-Width

-Height

-Scale

-Position X

-Position Y

-Rotate X

-Rotate Y

-Rotate Z

-Anchor X

-Anchor Y

-Anchor Z

-Bypass 

So it offers quite a few more parameters that PanSpinZoom, mostly in scaling and rotation.

On the other hand, Trails is basically a motion blur with the particularity that it -I guess- blends frames using feedback (from what I understand of it).  It's really useful when you slow down, for example, a clip @30fps down to a third or a fifth of its original speed. It blends the existing frames using a soft motion blur that doesn't work with the same decay/accumulation parameters (which can often be a hassle when trying to smooth a slowed down media without boosting the luminosity in the process). For instance, I've used it very often to adjust my playback speed without having to go back in an editing software and render. Big save in time along with some really satisfying results, especially for a GPU based actor (FPS is virtually unaffected by this actor).

I'm attaching both actors to this message so you can have a look yourself.

Trails

FrParameters

A.

[Woland]

@antogmgm said:

@woland Actually, FrParameters offers these controls: -Width -Height -Scale -Position X -Position Y -Rotate X -Rotate Y -Rotate Z -Anchor X -Anchor Y -Anchor Z -Bypass  So it offers quite a few more parameters that PanSpinZoom, mostly in scaling and rotation.

 Try one of these: 

(You can also assign any of these three to a Virtual Stage and then pump the video feed back into your patch by grabbing it from the Virtual Stage with a Get Stage Image actor.)

[Woland]

@antogmgm said:

On the other hand, Trails is basically a motion blur with the particularity that it -I guess- blends frames using feedback (from what I understand of it).  It's really useful when you slow down, for example, a clip @30fps down to a third or a fifth of its original speed. It blends the existing frames using a soft motion blur that doesn't work with the same decay/accumulation parameters (which can often be a hassle when trying to smooth a slowed down media without boosting the luminosity in the process). For instance, I've used it very often to adjust my playback speed without having to go back in an editing software and render. Big save in time along with some really satisfying results, especially for a GPU based actor (FPS is virtually unaffected by this actor).

 Still not entirely sure what to use to replace this. Maybe FFGLBlur, FFGL Motion Blur, or the native Motion Blur/Gaussian Blur actors?

[mark]

It's very possible many of these 32 bit actors could be implemented as shaders. If we put together a list of what's missing for people, maybe I can find a coder to hire to take on that job.

On the 1210 Visuals site, I see a several of people begging for a 64 bit version of "FrParameters" -- so maybe this is one we should put some time into.

If there's something _really_ essential for your workflow, just be squeaky about it because the "squeaky wheel gets the grease." We'll see what we can do.

Best Wishes,
Mark

[jbbarriere]

Congratulations to Mark and the whole team for V3! Wonderful job!

I would like to take the occasion of this discussion to strongly support an implementation of 'Trails' as a shader. I use it these days in many of my patches...

And while making a 'wish list': a GPU version of Displace would be great (I know that there is already a GSL shader, but it is only a partial port, it does not work the same).

Also, in the old FF plugins, 'Wave' or 'TimeBlur' were really useful and are not replaceable with anything else I could find...

Many thanks again for all the efforts!

Best wishes,

Jean-Baptiste Barrière 

[topherbb]

Hi, any solution for the crop actor? It's acting up for me, and the TT GLSLs plugin doesn't have anything to replace it that I can see.

[Woland]

@topherbb 

Try the native crop actor.

[jbbarriere]

@jbbarriere said:

Congratulations to Mark and the whole team for V3! Wonderful job!
I would like to take the occasion of this discussion to strongly support an implementation of 'Trails' as a shader. I use it these days in many of my patches...
And while making a 'wish list': a GPU version of Displace would be great (I know that there is already a GSL shader, but it is only a partial port, it does not work the same).
Also, in the old FF plugins, 'Wave' or 'TimeBlur' were really useful and are not replaceable with anything else I could find...
Many thanks again for all the efforts!
Best wishes,
Jean-Baptiste Barrière 

 Anything new about this? 

I personally cannot really use v3 until I find at least GPU substitutes for Displace and more sophisticated blurs like Trails or the good old Peter's Wave, indeed Gaussian and Motion blurs are really not enough...

[DusX]

@jbbarriere

I would suggest looking thru Shadertoy.com for a glsl effect/s.

The site has thousands of effects that can be used with Isadora. I know I have seen at least a few blurs available.

[Woland]

@dusx said:

The site has thousands of effects that can be used with Isadora.

Just be careful about the license type for the shaders. Some are specifically labeled as being fine for commercial use without credit, some you must credit the creator, some you can't use for commercial work, etc. Each shader is someone's intellectual property so it's always best to keep that in mind.

[jbbarriere]

@woland

Thank you for pointing to Shadertoy.com. Actually, I visit it regularly, it is indeed a very interesting site; takes a lot of time to explore only just a little bit of what is there. Unfortunately, I did not found so far anything satisfying as substitutions of Trails or Peter's Wave and the good old Isadora Displace (presently only CPU, not replaceable exactly by the GLSL code kindly provided by Mark in the Forum).

I also fooled around myself building GLSL blurs, but did not achieve anything that can replace exactly these older things, which is specially problematic when playing older pieces using them. I have indeed now nearly 20 years of pieces made with Isadora using these treatments...

Best wishes,

jbb

[Woland]

@jbbarriere

I don't know what to suggest at this point. If you absolutely can't use Isadora without those 3rd party effects then your best option seems to be sticking with Isadora 2 and freezing your show computer in time (not updating the OS) to dedicate it to use with Isadora 2.

Of course you can also keep looking for GLSL shaders that replicate the 3rd party effects that are integral to your artistic practice (or hire a programmer to write custom GLSL code for you to replicate those effects to your satisfaction).

Best wishes,

Woland

[DusX]

@jbbarriere

Here is a link to a GLSL displace effect: https://community.troikatronix...

At least 1 from the list.

Do a search in the ISF listing, and see if you find any effects that look close.
I think converting ISF to standard GLSL should be possible (I haven't tried it, but I'm willing to try ;)

https://www.interactiveshaderf...

[tomthebom]

@dusx WOW, that ISF  is pretty cool. Somehow it seems that ISF is in GLSL format, but with an interface added. Have you got any hint how to implement ISF into Isadora? I am not the great coder, you know. Best regards, Tom

[mark]

Dear All and @jbbarriere 

I'm jumping in here to see what I can offer, but today I do not have time to review this entire thread, so please excuse me if I'm repeating somethig.

I'll point out at the start that I have the open source code for Pete's Plugins (which includes Wave, TimeBlur and others). So, it is possible to analyze this source code and consider a shader/GPU replacement.

1) Trails -- I can't perceive a visual difference between Trails and Motion Blur. (For example, Trails set with 'feedback' = 90 and 'mode' = 0, and Motion Blur set with 'accum amt' and 'decay amt' both set to 7.5.) Can you explain what the difference is?

Also, while I always want to make all of our users happy, but I must add here that Trails was never an official Isadora plugin (it's from Resolume, right?) and so it's not really my responsibility to offer a replacement. To be tasked with replacing all such third-party solutions is simply more than we can take on.

2) Wave: I went and looked at the source code. This plugin uses a special buffer to hold values from the analysis of previous frames, which is something not so easy (though not impossible) to accomplish in GLSL. (I actually do not have enough experience as a shader programmer to know for sure if it's possible... I'd have to investigate to be sure.) For sure, it cannot be done in the GLSL Shader actor, but I might be able to code a the necessary shader if I did so in a custom actor.

I can, however, imagine converting the C++ code so and offering a "quick and dirty" replacement. If were to do this, the plugin would have to convert the texture input back to a CPU video (always expensive), process it, and then convert the bitmap image back to a texture (inexpensive.) This of course means it will be slow on HD images because the effect would be accomplished on the CPU, not the GPU. Still, if it's still useful with these performance limitations, I could probably put a replacement together quite quickly.

I always thought the Wave effect was really beautiful, and it was always one of the first plugins I'd show when introducing Isadora to students in the early days. In other words, I'd like to have it back too. ;-)

3) The TimeBlur actor keeps as many as fifty previous frames in memory, and then give you the average of some subset of those frames. Depending on your graphics card, this could be problematic if you're feeding in HD (1920x1080), 50 frames would be 395 Mb. I guess most graphics cards could hold this, but put a couple of these actors in a patch and your heading towards 1GB pretty fast. Anyway, this could be coded as a custom shader actor (again, not possible in the GLSL Shader actor alone). I'd be curious to see if performance was any better on a GPU if you had a shader with as many as 50 texture inputs.

4) As discussed extensively in a previous thread, accomplishing what the old Displace actor did in a shader is extremely problematic. You can read that thread if you want to understand why the nature of GLSL makes it difficult if not impossible to accomplish this using a GPU. As with the Wave plugin, if it's simply about providing a GPU input, I could go down the "pull the texture to the CPU/process/convert to back texture" process explained in item #2 about the Wave actor.

Summary:

In the end, there's only so much I can do about plugins that were lost because of the move to 64 bit. I wish we could continue to offer both a 32 bit version of Isadora that would run these old plugins, but that's simply not viable: we do not have the resources be able to fully test both 64 bit and 32 bit releases on both Mac and Windows. Furthermore, on MacOS, the future is sadly clear: Catalina is coming and running anything that is 32 bit will be impossible once people move to that system.

I look forward to your thoughts and reactions to what I've offered above.

Sincerely,
Mark

[DusX]

@tomthebom

yes, I believe it is essentially glsl with a Json interface.
Unfortunately working out the steps to convert isf to standard glsl isn't something I have had time to take on.
perhaps in a few weeks I could jump into this (it will be a bit of a learning curve for me, so no promises)

[tomthebom]

@dusx That would be really cool to get some help from the forum there....;o)  Best regards, Tom

[CitizenJoe]

@DusX , I found this: https://magicmusicvisuals.com/forums/viewtopic.php?t=1017

Cheers,

Hugh

[DusX]

@citizenjoe

Yes. That confirms what I thought. I suspect it should be pretty easy, and I've been meaning to dig into this for some time now.

[mark]

Hi folks,

Let's move any talk about ISF to this thread. See my post there about a possible conversion utility.

Best Wishes,
Mark