Puppeteering using Chroma Key Compositing

Discussion in 'Fan Productions' started by Matthew Raymond, Jul 29, 2018.

  1. Matthew Raymond

    Matthew Raymond Fleet Captain Fleet Captain

    Joined:
    Oct 24, 2016
    If you've seen any of the behind-the-scenes videos on the movie Automata (2014), you'll know that the robots, rather than being CGI, where mostly puppets, with the puppeteers being dressed in green suits so that could easily be composited out afterwards using chroma key. I was wondering how hard it would be to do something similar using nothing but my cellphone camera and a copy of Final Cut Pro X at the public library, so I donned a green bodysuit from a costume store, shot some video of myself holding a stuffed dragon, and edited the footage to see if I could remove myself from the video. The idea would be that I could remove myself and leave only the dragon, dancing on my living room couch.

    The results weren't great, but the were a bit better than I expected. If you need a cheap and dirty Predator cloaking effect, it certainly works. However, I do think you can make this technique work reasonably well on a low budget. Here's some things I learned from the process:

    1) Use the best quality camera you possibly can and the highest reasonable resolution. (This probably goes for any type of footage you're trying to chroma key.

    2) Keep the footage in the least compressed format you can, preferably RAW. Compression artifacts are the mortal enemy of chroma key compositing.

    3) Make sure you aren't casting any shadows on the background. You may want to consider carefully arranging your lighting to avoid this problem. I'd suggest doing a camera test to be sure. It helps to be as far from a background wall as possible, preferably outside in the middle of a field.

    4) Sadly, the trick of focusing on the subject and letting the keying surface blur in the background won't work here because the puppeteer must necessarily be close to the subject of the camera.

    5) Get as much diffused light on the keying surface as possible. The best conditions will probably be outdoors when it's overcast, where you'll have lots of ambient light but very little direct light.

    6) If possible, don't have the puppeteer on camera at all. Use painted poles, and if the puppeteer must necessarily be in the shot, put up a screen in front of them so you're keying out a flat surface instead of a suit, which is going to have a lot of wrinkles and shadows. Just keep the suit handy as insurance in case the puppeteer keeps accidentally entering the frame.

    7) Make sure that your puppet and background are sufficiently different in tint from your keying color. For example, if you're keying color is green, you might not want to use a puppet with the color cyan, or have a slightly greenish background (although that's less of a problem).

    8) Similarly, if you're going to film in a grassy field, naturally you'll want to use blue instead of green for your chroma key color.

    9) If you can figure out how to do it, it's probably a good idea to lock your camera's settings for a shot so that it doesn't dynamically adjust anything over time. That way, you can record footage to composite, then film the shot with the puppets, and the color balance and focus won't change, giving you a much better result.

    10) Those colored bodysuits are pretty skin-tight, so you might want to work out a bit if you plan to put one on, especially if you're filming outdoors.

    Any thoughts on how this can be improved?
     
  2. Admiral Archer

    Admiral Archer Captain Captain

    Joined:
    Dec 28, 2009
    I actually am fascinated by this, because I was thinking about using a puppet for the main creature in my fantasy film I'm developing, which in this case is also a type of dragon. If I may ask, would you be willing to share your results with us? I'd love to see what you have come up with!
     
  3. Matthew Raymond

    Matthew Raymond Fleet Captain Fleet Captain

    Joined:
    Oct 24, 2016
    mos6507 and Admiral Archer like this.
  4. Admiral Archer

    Admiral Archer Captain Captain

    Joined:
    Dec 28, 2009
    I actually think that was an impressive effort, even for a test. It's VERY difficult to get chroma key to behave properly. I've experimented with it for years, and still haven't gotten it down pat entirely. If I may make a suggestion, have you considered hiding the majority of your body behind the couch, and only cover your hands and arms with some sort of green gloves? You could even use a mirror on the floor behind the couch so you could see what was going on above you. That would minimize the amount of keying work you'd have to do.
     
  5. Matthew Raymond

    Matthew Raymond Fleet Captain Fleet Captain

    Joined:
    Oct 24, 2016
    I think Final Cut Pro X probably deserves most of the credit for the Chroma Key. The tool really does most of the work for you.

    In hindsight, using just the glove from behind the couch would worked out better for the finished product, but I'm not sure I would have done that anyway, because figuring out how to key myself out was sorta the point. The easier I make it to key, the less I learn.

    I've noticed that the suit is kinda glossy. I didn't realize beforehand how poorly it would key. If you want to use this technique, you may want to make your own suit using a cloth with a more even, matte finish. Or just use painted pole, like I said.

    For a particular project I'm planning, I want to have dolphins in special mech suits with legs that allow them to work outside of a aquatic environment. I could use CGI, but I'd just prefer to have some sort of realistic puppet that looks like it's physically there. That's the reason I'm looking into this.
     
    Admiral Archer likes this.
  6. Admiral Archer

    Admiral Archer Captain Captain

    Joined:
    Dec 28, 2009
    You wouldn't happen to be adapting Diane Duane's TNG novel "Dark Mirror", would you? I've been reading it recently for the first time, and that last paragraph in your most recent post sounds a lot like the effects that could be used to create the dolphin-character in the novel. :)
     
  7. Maurice

    Maurice Fact Trekker Premium Member

    Joined:
    Oct 17, 2005
    Location:
    One ferry ride from Starfleet HQ
    Green screen is pain. I've done a lot of it and I mostly hate it. Most of what's suggested here is valid as an approach, but you always want to avoid hotspots and shadows because the color information is low in anything too bright or too dark.
     
    Last edited: Jul 31, 2018
    Admiral Archer likes this.
  8. Matthew Raymond

    Matthew Raymond Fleet Captain Fleet Captain

    Joined:
    Oct 24, 2016
    @Maurice, right. Ideally, you'd either want to frame your shot so that the puppeteer and the controlling armature or strings aren't visible in the shot, or hide them behind a part of the set. For example, you could have the character in front of a crate, then control the puppet from within the crate through a few holes hidden by the puppet's body.

    Thought: If your puppet is a solid color, you could just reverse key the puppet and superimpose it on a clip of the scene without the puppet and the puppeteer. It would be relatively easy to rotoscope out any issues with the keying, since you only need the puppet, and the lighting on the puppet would match the set because you're just repeating the same fixed shot with and without the puppet. You may not even need a suit, so long as the puppeteer isn't wearing the keying color.

    Don't think I've read that one. Haven't read any of the novels since high school (though I read probably a couple dozen back then). I made a concerted effort in college to read more generic science fiction.

    Actually, the dolphins are part of a sci-fi series I've been working on for about two years now. The series deals heavily with transhumanism, and the dolphins themselves represent both uplifting and technogaianism.
     
    Admiral Archer likes this.
  9. YJAGG

    YJAGG Captain Captain

    Joined:
    Jun 3, 2013
    Location:
    Baltimore, MD
    Cool good uck, I agree that maybe a suit made out of a duller fabric would help, but it looks kind of Predator/ Jemhdar-ish or scene in Battle Beyond the Stars where you can see a hand in a black glove holding a model, keep playing and exploring