[SOLVED] FreeFrame actors issues on V3

[tomthebom]

@dusx WOW, that ISF  is pretty cool. Somehow it seems that ISF is in GLSL format, but with an interface added. Have you got any hint how to implement ISF into Isadora? I am not the great coder, you know. Best regards, Tom

[mark]

Dear All and @jbbarriere 

I'm jumping in here to see what I can offer, but today I do not have time to review this entire thread, so please excuse me if I'm repeating somethig.

I'll point out at the start that I have the open source code for Pete's Plugins (which includes Wave, TimeBlur and others). So, it is possible to analyze this source code and consider a shader/GPU replacement.

1) Trails -- I can't perceive a visual difference between Trails and Motion Blur. (For example, Trails set with 'feedback' = 90 and 'mode' = 0, and Motion Blur set with 'accum amt' and 'decay amt' both set to 7.5.) Can you explain what the difference is?

Also, while I always want to make all of our users happy, but I must add here that Trails was never an official Isadora plugin (it's from Resolume, right?) and so it's not really my responsibility to offer a replacement. To be tasked with replacing all such third-party solutions is simply more than we can take on.

2) Wave: I went and looked at the source code. This plugin uses a special buffer to hold values from the analysis of previous frames, which is something not so easy (though not impossible) to accomplish in GLSL. (I actually do not have enough experience as a shader programmer to know for sure if it's possible... I'd have to investigate to be sure.) For sure, it cannot be done in the GLSL Shader actor, but I might be able to code a the necessary shader if I did so in a custom actor.

I can, however, imagine converting the C++ code so and offering a "quick and dirty" replacement. If were to do this, the plugin would have to convert the texture input back to a CPU video (always expensive), process it, and then convert the bitmap image back to a texture (inexpensive.) This of course means it will be slow on HD images because the effect would be accomplished on the CPU, not the GPU. Still, if it's still useful with these performance limitations, I could probably put a replacement together quite quickly.

I always thought the Wave effect was really beautiful, and it was always one of the first plugins I'd show when introducing Isadora to students in the early days. In other words, I'd like to have it back too. ;-)

3) The TimeBlur actor keeps as many as fifty previous frames in memory, and then give you the average of some subset of those frames. Depending on your graphics card, this could be problematic if you're feeding in HD (1920x1080), 50 frames would be 395 Mb. I guess most graphics cards could hold this, but put a couple of these actors in a patch and your heading towards 1GB pretty fast. Anyway, this could be coded as a custom shader actor (again, not possible in the GLSL Shader actor alone). I'd be curious to see if performance was any better on a GPU if you had a shader with as many as 50 texture inputs.

4) As discussed extensively in a previous thread, accomplishing what the old Displace actor did in a shader is extremely problematic. You can read that thread if you want to understand why the nature of GLSL makes it difficult if not impossible to accomplish this using a GPU. As with the Wave plugin, if it's simply about providing a GPU input, I could go down the "pull the texture to the CPU/process/convert to back texture" process explained in item #2 about the Wave actor.

Summary:

In the end, there's only so much I can do about plugins that were lost because of the move to 64 bit. I wish we could continue to offer both a 32 bit version of Isadora that would run these old plugins, but that's simply not viable: we do not have the resources be able to fully test both 64 bit and 32 bit releases on both Mac and Windows. Furthermore, on MacOS, the future is sadly clear: Catalina is coming and running anything that is 32 bit will be impossible once people move to that system.

I look forward to your thoughts and reactions to what I've offered above.

Sincerely,
Mark

[DusX]

@tomthebom

yes, I believe it is essentially glsl with a Json interface.
Unfortunately working out the steps to convert isf to standard glsl isn't something I have had time to take on.
perhaps in a few weeks I could jump into this (it will be a bit of a learning curve for me, so no promises)

[tomthebom]

@dusx That would be really cool to get some help from the forum there....;o)  Best regards, Tom

[CitizenJoe]

@DusX , I found this: https://magicmusicvisuals.com/forums/viewtopic.php?t=1017

Cheers,

Hugh

[DusX]

@citizenjoe

Yes. That confirms what I thought. I suspect it should be pretty easy, and I've been meaning to dig into this for some time now.

[mark]

Hi folks,

Let's move any talk about ISF to this thread. See my post there about a possible conversion utility.

Best Wishes,
Mark

[jbbarriere]

@MarkyMark136 

Dear Mark,

I apologize for this very late reply: I actually did not read your post until recently (I was busy on other projects), and then when I finally saw it, I wanted to make more tests before to come back to you.

Before anything else, once more thank you for all your work with Isadora. Then coming back to the 'blurs' topic, thank you for taking the time to answer in details.

I completely understand, of course, your arguments about priorities and the impossibility to maintain third-party code.

My personal issue is that, being an Isadora user from the very beginning, I have a number of works made with it (and some of its original objects like Displace etc.), which I perform all around the world all the time, and which are becoming progressively unplayable...

To be more specific about for instance Trails: actually there is really a big quality issue in terms of difference between Trails (and actually also Timeblur) and Motion Blur. For most of what I do, Motion Blur is just unusable (specially in GPU; CPU is slightly better), because it does 'spots', not progressive bluring (see images under).

The obvious constructive approach is to develop new objects with shaders; however, for the time being, there are current limitations that you pointed out and I have not been very successful with it. But I keep it in perspective... 

Best wishes,

jbb

Motion Blur GPU:

Motion blur CPU:

Trails:

Timeblur:

Wave:

[Woland]

@jbbarriere said:

@MarkyMark136 Dear Mark,

 @mark I think @jbbarriere meant to tag you here

[jbbarriere]

@woland (and Mark)

Thank you for correcting this!

I would like to add an information: I have been able to correct similar issues with "spots" in Max/Jitter by forcing textures from "char" to "float32". Would there be a way to do something similar in Isadora?

Thanks!

Best wishes,

jbb