Prevent sawtooth lines when mapping

  • Hi there,
    Our graphic designer did a real good job in creating a very nice animation. It is in Full HD and we've tried both HAP and JGP 75%. I do a quite simple mapping, just adjusting cornerpins and several stright lines in the original video, become a sawtooth pattern. Now she's a bit dissapointed that we're ruining her work. Is there a way to minimize that? Could it be a problem of my graphic card (NVIDIA GEFORCE GT 650 2GB) set to maximum performance. It's a bit old, but this should not be a hard job.

    Any help would be greatly appreciated.


  • Tech Staff


    I think I'd need to see a picture of the projection compared to a screenshot of the animation on a monitor to understand what you're talking about.

    Straight lines certainly should not become sawtooth patterns. Perhaps there's an extreme resolution issue with the projector you're using, or it's an uneven surface, or you're projecting from a strange angle? I can't rule out that it's not actually an issue arising from optics until I can see a comparison between the image on the projection surface and a screenshot of what it's supposed to look like.

    Best wishes,


  • Tech Staff

    straight lines becoming jagged, sounds like a side effect of scaling the image. This is likely to happen especially with fine lines and probably more drastically if the scaling isn't proportional. 

  • @gaspar said:

    several stright lines in the original video, become a sawtooth pattern.

    There are few more details that would help us to analyze this situation.

    1) If you could send a high-quality uncompressed (TIFF) copy of an frame from your video, along with the .izz file in which you've set up the cornerpinning, that would allow us to try to replicate the problem.

    2) You say "HD" but please be specific: 1920 x 1080? Or?

    3) What is the resolution of the output device on which this image is being displayed? 1920 x 1080? Or?

    That said, I did the following test: I imported a 1920x1080 black and white grid, and then used the Stage Setup editor in Isadora to adjust the keystoning (cornerpinning.) This is a comparison of the original image vs. the cornerpinned image when displayed at full resolution (2880 x 1800) on my laptop display.

    So yes, you can see some aliasing of the lines, but to my eye it seems acceptable on my screen.

    The keystoning function must resample the image to perform the required distortion, and you are bound to get some aliasing of the image as a result.

    Is this what you're seeing? Or something else?

    Best Wishes,

  • @mark @DusX @Woland

    Thank you for the responses. I'm attaching the TIF images of the undistorted and distorted view (screenshots of my 1920x1080 display, same as the source video). It is not really that bad there, But I'm projecting on an 8m wide screen, so it becomes more evident. The designer will make the stright lines slightly thicker. I did not really pay attention to that, but since the designer and drama assistant were upset, I decided to ask you if there is a way to improve that.

    Somewhat strange. I took the screenshot twice (one after restarting ISADORA), the second one looks a bit worse. I've found the performance of ISADORA being variable over time (maybe something else going on in the background sometimes, but it is nothing very evident to me). I've attached both. By the way I forgot to set them to uncompressed. But I think it is enough for this.

    I guess we can try these:
    1. Use thick lines.

    2.Avoid too much scaling. In this case we're limited in projector choice and placement, so Maybe doing a lower resolution render of the animation, close to actual mapped resolution, could possibly help.

    3. Any other thoughts to take in count for the future?

    This was rehersal. For the premiere we will have a proper projector and find a better position so scaling and keystoning are minimised. I guess that should improve the outcome.

    Thank you

    PD: I've dragged and dropped the TIFFs twice on the upload file window, but I can't see them in the post. Maybe I'm doing something wrong.

  • Tech Staff


    I don't see any attachments. Perhaps zip the tiff files first.

  • @gaspar said:

    PD: I've dragged and dropped the TIFFs twice on the upload file window, but I can't see them in the post. Maybe I'm doing something wrong.

     .zip the files and then upload them. The TIFF extension might not be allowed by the forum software.

  • Tech Staff

    You could also upload all the files to a folder in Dropbox or Google Drive and share the link to that folder.

  • Tech Staff


    Do you still need help with this?

  • hello.

    I am feeding 4k video from a GH4>Atomos>blackmagic recorder card,

    and I am facing exactly the same aliasing problem when zooming.

    I am not in front of the production machine now, but this is what i intend to test as a fix :

    avoid aliasing artifact with blackmagic 4k live video feed :

    > avoid scaling in the projector node...

    > try FFGLPanSpinZoom freeframe plugin (anyone knows if this freeframe plugin avoid aliasing artifacts when zooming ? i cannot test now because i am not using windows 10, I will test on sunday)

    > or try glsl vertex shader transform
    vertex shader with something like this

    gl_Position = vec4( Position.x * Scale.x, Position.y * Scale.y, 0, 1 )

    > or try glsl
    fragment shader, acting on uv coordinates like in this blur shader :

    uniform sampler2DRect myTexture;
    uniform float blurAmnt;
    void main() {
        vec2 vUv = gl_TexCoord[0].st;
        vec4 color;
        color += 0.5 * texture2DRect(myTexture, vUv + vec2(0.0, blurAmnt * -4.5));

    > in theory antialiasing should only be useful when scaling down ?

    if (scale < 1.0) {
    //in.getTexture().setTextureMinMagFilter(GL_NEAREST, GL_NEAREST); // aliased
    inTexture.setTextureMinMagFilter(GL_LINEAR_MIPMAP_LINEAR, GL_LINEAR); // blurry

    It would be intersting to get other experience with this no-antialiased issue..

    Thank you

  • Tech Staff

    are you also working with a 4K display? If not, bringing in a 4K video feed is overkill because it’ll have to get scaled down for the display.

  • Hello,

    Yes of course,

    or more exactly, 2 edge blended hd projectors (making the picture very nicely bright) on 8 meter wide screen retroprojection.

    Anyway we need to zoom in text documents, which are pretty sensitive to resolution, obviously.

  • @woland

    sorry for the long silence. and thanl you for your interest. The problem still persists, but I've some other priorities right now. Regarding the sawtooth we've made the lines somewhat thicker, this way it is not that obvious, and I'm getting a new machine in a couple of weeks, maybe it works better on the new computer, since my actual graphic card is a bit weak.