Airlock - made with Isadora

n.jones

@kathmandale awesome work again! I love the fact I forget the tech when watching your works. Then go full geek afterwards and try and reverse engineer them :).

jtoenjes

@kathmandale Thank you because the BBC iPlayer won't work in the US

tomthebom

@kathmandale said:

You'll probably be able to figure out most of what we did here but the basics were a skype call with NDI of each performer into the Issy engine. There's then the re-framing, effects and layering with the backgrounds and foregrounds. Happy to answer any questions about what we did and how we did it.

Wow! I am completely blown away! Am I right, if I think, that the graphic look of your backgrounds with its aquarelle touch and the panel layout made their way into Izzy as (scanned/photoshopped) pictures? Or did you create this in Izzy as well?

DusX

Love it!
thank-you for sharing :)

Kathmandale

@tomthebom

You're right Tom, it's a combination of pre-made backgrounds (and sometimes foregrounds) with the live actors slotted in and 'framed' with effects applied to make them look 'hand drawn'.

In the first two episodes the backdrops and animation were made by Simon Wainwright, one of our co-artistic directors. I think it's all done in photoshop and after effects although he does do some bits by hand sometimes. In episode three the bulk of the backgrounds are ink drawings done by Laura Hopkins, a theatre designer who has worked with us for years and designed a lot of our theatre shows. They were scanned and built in to After Effects comps by Simon. Some of the still faces of the actors that appear in the backgrounds were photoshopped to look like they were in that painted style.

Izzy was compiling all these images, bringing in the NDI feeds from Skype, applying effects, colouring and framing/cropping/drawing-shapes-around the live images. We made good use of layers and alpha channels to position things. In episode one there was a lot of freezing going on as well.

We're planning on doing a bit of a short 'making of' video and I'll pop it up here when we've done it.

RIL

@Kathmandale

first of all tell you that your work is excellent and beautiful at the same time. This is a clear example of the use of technologies as tools for a certain purpose, and not the use itself. I can think of many questions but I don't want to be annoying ... maybe you can tell us about the "general environment and circuitry of the device" Thats mean: skype- Isadora- OBS? How did you handle that routing and the sound configuration and rounting again.
Thank you very much for your time

Kathmandale

@ril Thank you.

We just used Skype and Isadora, the recording is straight out of Isadora. We did this on a fairly heavy duty 'trash-can' MacPro (I don't have the exact spec I'm afraid). Originally we conceived of doing this as a live event with Isadora's output going straight back to Skype through a virtual camera and the audience would watch by joining the Skype call. In the end, our funders wanted it to be made for a bigger platform (which is great, we got paid and many many more people get to see our work) so we recorded the output.

As a result we could also 'cheat' a little bit. Because the quality of the sound on Skype isn't great we got the actors to record their voices locally on their phones and we mixed this and the music/foley after the recording. So it's a bit neater than we could get it to be 'live'. We also cheated a tiny bit with the visuals on episode 3; one of the actors internet connection was terrible on the day we recorded so we tidied it up a little bit after the recording. On the whole though what you see is the Isadora output.

This technique would certainly work live though. We did route the Isadora output back in to Skype so that the actors could see themselves in the graphic novel. They often needed to be aware of the edges of the frames, or their positions in relationship to each other and couldn't really do this without being able to see the Isadora output in real time. Our final set up was with my colleague Simon running the patch and recording the output, while in another city, I also ran the patch and output it back to Skype so the actors could see it. Simon kept his video off in Skype to reduce the load on the recording computer; on my computer I turned off some of the transitions and other things the actors didn't need to see to reduce the load and keep my output as smooth as possible for the actors (on my 2014 laptop).

RIL

@kathmandale Thanks a lot !!

[We did route the Isadora output back in to Skype so that the actors could see themselves in the graphic novel.]

Great idea !!

I see sometimes live sound its not entirely necessarie and recording it is a much better solution for the results you aim.

Do you think this kind of performance/device will continue after the isolation period? I mean if this is a new format where is worth to continue investigate ? For a new type of audience ?

Again thank you.

Best,

Maxi

Josafat

Hi, thanks to share by Facebook, since I'm in Mexico I wasn't able to seen it from the BBC platform. It is a really amazing show. And it's very clever the way you did to synchronize your performers so you can mixed in Isadora perfectly between them and with the backgrounds and the foregrounds.

It would be great if you do a live streaming of some chapter, but I understand the issue of the audio using Skype, and also that the BBC is paying for the project. Congrats, I hope that someday I could travel to UK and go to see your work, because I really love all what I saw in the official web page of Imitating the Dog.

Saludos

Ismael

leben

@josafat This is beautiful work...thank you :)

tomthebom

@kathmandale Thanks for your response! What filters did you use in particular to get the "hand drawn" look? And what is your experience with this imitating "hand drawn" look. I find it challenging adapting graphic novel looks and found out, that it works best for mono colours. But I am quite impressed, HOW well it worked in your work...

Kathmandale

@ril said:

Do you think this kind of performance/device will continue after the isolation period? I mean if this is a new format where is worth to continue investigate ? For a new type of audience ?

It's an interesting question that I think we're all asking at the moment. I can't speak for everyone, or even all of imitating the dog, but personally I felt at the start of lockdown as though we were making this kind of work out of necessity; it was impossible to make work the way we normally do, so we were looking for different forms. We were also looking for ways to work together as a company when we were all in different cities (and countries) when normally we come together in one room to do what we do.

When you experiment and find new forms though, there are obviously good, interesting, new things there and it's great to find them. I think going forward that even if we don't make a piece quite like this again, we will take these techniques and this knowledge on to the next piece, what ever that may be. You're always adding to the tool box I guess. If audiences are interested in it then I'd definitely be interested to keep making this kind of work. I'd love to see some merging of this kind of work with more traditional staging, perhaps that works for an audience in a theatre but that is simultaneously creating something like this that can be streamed globally.

Kathmandale

@josafat

Gracias. Or perhaps we will be able to come to you. I might like that idea better ;)

Kathmandale

@tomthebom I'm not on the computer with the show files right now, but I'll check the patch later and see if I can upload some examples. Off the top of my head we did a lot with edge-detection, halftone filters and a kind of two-tone filter. Mostly it was GLSL stuff.

RIL

@kathmandale thanks for your thoughts !!

Best,

Maxi

Skulpture

@kathmandale said:

Airlock is a live action graphic novel in three episodes, each telling a different story exploring themes of isolation, despair and hope.

This is amazing! I have shared with my students!

Kathmandale

@tomthebom sorry for the slow reply Tom, here's the user actor we made to do the effects in episode one and two. There are some GLSL shaders in there, they're standard ones from the TT library or from the other sources shared by Mark on the guru session. I modified two of them to support an alpha channel, there's more on that on this thread if you're interested.

In episode three the 'pencil sketch' effect on the actors was added afterwards Im afraid. This was when the actors internet connection was bad and we tidied it up a bit after the fact (or cheated in other words, in After Effects). That said, on my list of things to work on at the moment is a GLSL shader to do something very similar to that effect. I didn't get it done in time for this project but once I get somewhere with it I'll be sure share it on here.

Airlock - Textures.iua3

tomthebom

@kathmandale Thanks a lot for the follow up. I will look into it and return to you. Best wishes, Tom

tomthebom

@kathmandale Thanks a lot sharing your plugin: I am deeply impressed with the results of it: whatever video I throw into it, the result is sharp and convincing. I managed to build something similar to the "3 frames freeze" you are using. What was your approach to background removing? I am still struggling with a convincing solution there...

UA Capture Video into Panels.iua3

Kathmandale

@tomthebom

When we wanted to remove backgrounds we got the actors to set their 'virtual backgrounds' to a green image and then chroma-keyed it out. It wasn't great as it was using Skypes built in background detection, so even when it's fairly accurate you often get quite a 'blocky' edge . We definitely played around with not always doing a straight chroma key but using that to make a mask that could then be blurred a bit to soften the harsh edges. Something a bit like this (although I've just mocked this up quickly, might not be quite right but you get the idea). Sometimes that worked and sometimes, given the 'graphic' look of the piece, we thought the blocky edges were fine and left it alone.

The user actor you've shared above is pretty much what we did to freeze things. As it was a convention we used a lot I made a fairly comprehensive one to handle all the 'boxing' and freezing. It's similar to yours above but it had some added functionality, doing the 'framing' with crops and shapes to make the outline boxes and desaturating the frozen images as that was a convention we stuck with. It was also built with the idea of putting several inputs in to the same 'box' so it got a bit complex, although looking back I don't know if we ever used it like that in the end. I'll upload it later when I'm on a machine with the show files.