Mapping Video onto Inverted Head

[RWillats]

Hi there,

I have a video of a head talking that is to be mapped onto a scan that's been 3D-printed as a concave imprint. It would be great to "correct" the distortion in Isadora, if possible.

Any tips for mapping this face better onto the surface to reduce as much distortion as possible? I have the mesh itself. I also was thinking I could get pretty close with some inverted version of the bulge distortion, perhaps?

Best,

Rory

[Woland]

@rwillats

Some ideas (though I admit I don't work with 3D much in Isadora):

Use the 3D actors:

1. Import the .3ds file into Isadora and select it with the 3D Model Particles or 3D Player actor in Isadora.
2. Feed your video as the texture input.
3. Adjust the model’s scale, rotation, and translation until the face is aligned with your physical setup.
4. If the print is concave, you can flip the normals (inside-out mesh) or rotate the camera 180° to “project” into the cavity.
5. Once it looks correct virtually, align the projector physically to match this digital view.

(Again, I don't know that what I'm saying here fully makes sense because I don't work with 3D very much, so take all the steps above with a grain of salt.)

---

Use GLSL:

• You could use the existing TT GLSL "TT Bulge Distortion"
• You could also try this new TT Bulge Warp shader that I've just put together (I don't know if it's any good):

/*
TT Bulge Warp - Fragment Shader
version v1.3
*/
// ISADORA_PLUGIN_NAME("TT Bulge Warp")
// ISADORA_PLUGIN_DESC("Applies a controlled radial bulge/concave distortion. Adjustable center, radius, and tiling.")
// ISADORA_FLOAT_PARAM(bulge_amount, bulg, -7.0, 7.0, 0.3, "Amount of bulge. Positive = bulge, Negative = pinch.")
uniform float bulge_amount;
// ISADORA_FLOAT_PARAM(radius, radi, 0.0, 1.0, 0.5, "Radius of effect from center, as 0–1 proportion of image.")
uniform float radius;
// ISADORA_FLOAT_PARAM(horz_center, cx, -100.0, 100.0, 0.0, "Horizontal center point (in % of image width).")
uniform float horz_center;
// ISADORA_FLOAT_PARAM(vert_center, cy, -100.0, 100.0, 0.0, "Vertical center point (in % of image height).")
uniform float vert_center;
// ISADORA_INT_PARAM(tile, tile, 0, 1, 0, "Turn tiling on/off. Clamp if off.")
uniform int tile;
uniform vec2 resolution;
uniform sampler2D tex0;
void main(void) {
    vec2 uv = gl_TexCoord[0].xy;
    // Center position as normalized coordinates
    vec2 center = vec2(0.5 + horz_center / 100.0, 0.5 + vert_center / 100.0);
    vec2 delta = uv - center;
    float dist = length(delta);
    // Only apply within radius
    float effect = smoothstep(radius, 0.0, dist); // 1.0 near center, fades to 0.0 at radius
    float r = dist;
    float theta = atan(delta.y, delta.x);
    // Only modify r inside radius
    float r2 = mix(r, r + bulge_amount * pow(r, 2.0), effect);
    vec2 warped = center + r2 * vec2(cos(theta), sin(theta));
    if (tile == 0) {
        warped = clamp(warped, vec2(0.0), vec2(1.0));
    }
    gl_FragColor = texture2D(tex0, warped);
}

Additionally, here's our GLSL Shader Actor Tutorial Article for Isadora

Since you're trying to modify existing shaders, the part of the article that's probably most relevant for you is Tutorial 3: Adding Real-Time Parameters to a Shader as this tutorial explains the format of variables. In particular, knowing how to adjust the 'min' and 'max' associated with a variable is quite helpful:

Tutorial 3: Adding Real-Time Parameters to A Shader

The really exciting opportunity offered by the GLSL Shader actor is the ability to manipulate shader parameters in real-time. Shaders accept real-time input through a mechanism known as uniform variables. For example, if you wanted the shader to receive a floating point number, you would add a line like this. 

uniform float myVariableName;

This defines a floating point number called myVariableName as an input. Then, you would use that variable in your code as needed. The trick here is to make that variable available to you as an input to the GLSL Shader actor. This is achieved using a special comment line that you add to the shader like this 

// ISADORA_FLOAT_PARAM(name, id, min, max, default, "help text");

In OpenGL Shader Language, any line that starts with "//" is considered to be a comment; in other words, it is ignored by the compiler. But it is not ignored by Isadora. Using comments like the one above, you can create an input for the GLSL Shader actor that sends its value to a uniform variable in the shader 

 Let's go through each part of this special comment in detail 

// ISADORA_FLOAT_PARAM — This portion simply identified what kind of parameter is being defined. In this case, single floating point number. 

name — This is where you define the name of the variable. This name must exactly correspond to the variable name given in the uniform variable — statement in the shader code. You cannot include spaces in this name; if you need a space, use the underscore (_) character instead.

id — This identifier, which can be from 1 to 4 characters long, uniquely identifies the input. Should you re-arrange the order of the inputs, Isadora uses this identifier to ensure the links to that input are maintained. For each shader program, this identifier must be unique.

min — For numeric parameters, this defines the minimum possible value for the input. 

max — For numeric parameters, this defines the maximum possible value for the input. 

default — For numeric parameters, this defines the default value for this input when the actor is added to the scene. This input is only meaningful if you add the source code for this shader to Isadora's GLSL Plugins folder. (More on this feature later.) 

"help text" — This defines the help text that will appear in the information view for this input. If you share your shader code, you can help those who use it by providing useful, descriptive information about this input. Isadora's philosophy has always been to make it easy for the user to use the program; giving useful details in the help text supports that philosophy.