TMP Deepfake

[Pauln6]

I thought I'd bite the bullet and download a face swap program to experiment with images of TOS Rand. Three days in and it's still learning with a discrepancy of 0.036 (apparently I have to wait for it to get below 0.020). I had to lower the settings as my laptop doesn't have enough working memory so I'm expecting the results to be a bit below par but if it's passable then maybe I'll have another crack after I've upgraded.

As an aside, Grace felt she was singled out by Wise because she played a joke on him with Roddenberry and she wasn't allowed to wear any make-up in her close up scenes, not even eye liner. When you face swap, you really notice what a difference eyeliner makes to her appearance.

 

[Maurice]

This Star Wars fan edit is even more full on with better pacing.

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

It makes me wonder if some version of the memory wall might be possible, or even an alternate version where they sneak a shuttle through the orifice by cruising on thrusters only.

The main problem with that sequence is it's utterly tone-deaf the the original film's style. Lucas shot with a "documentary camera" aesthetic which eschewed the camera movements and framings throughout this. Performance and athletics aside, it's as out of place in Star Wars as Bonnie and Clyde's finale would be in a 30's Warner Bros. gangster pic.

 

[jaime]

The main problem with that sequence is it's utterly tone-deaf the the original film's style. Lucas shot with a "documentary camera" aesthetic which eschewed the camera movements and framings throughout this. Performance and athletics aside, it's as out of place in Star Wars as Bonnie and Clyde's finale would be in a 30's Warner Bros. gangster pic.

I dunno. Original film...yeah maybe...but as a halfway house to knit in with the Kenobi of the prequels, it’s certainly working. Needs to be just a little more ‘kenobi could win this, but is choosing not to’ and maybe dial back the music to be more Williams. Fifty fifty in the sound mix flashbacks. But...I wouldn’t have minded a scene like this back when they added the blinking evokes and what not. It makes it tonally closer to the prequels and helps smooth the joins. I did like the fire on Vader’s arm.
Maybe there should just be a hint that the inner Anakin was holding back a tiny bit, a hint of obi wan trying to see if anything was left of his friend. I think it works.

The rand in TMP thing...not so much tbh.

 

[Pauln6]

I dunno. Original film...yeah maybe...but as a halfway house to knit in with the Kenobi of the prequels, it’s certainly working. Needs to be just a little more ‘kenobi could win this, but is choosing not to’ and maybe dial back the music to be more Williams. Fifty fifty in the sound mix flashbacks. But...I wouldn’t have minded a scene like this back when they added the blinking evokes and what not. It makes it tonally closer to the prequels and helps smooth the joins. I did like the fire on Vader’s arm.
Maybe there should just be a hint that the inner Anakin was holding back a tiny bit, a hint of obi wan trying to see if anything was left of his friend. I think it works.

The rand in TMP thing...not so much tbh.

I've never tried experiments like this before so Rand was a good starting point for proof of concept: plenty of full frontal, limited TOS dialogue to wade through, only a couple of shots where she could logically fit in, plus I have a soft spot for her.

If I had the skills and equipment of whoever made the fight scene above, who knows what monstrosities I could create!

That said, I have now looked through some of the other dialogue and it's a real challenge to find any that fits well.

Part of me would like to add some gallows humour to the V'ger cloud scene but I doubt I can find anything suitable.

 

[jaime]

I've never tried experiments like this before so Rand was a good starting point for proof of concept: plenty of full frontal, limited dialogue to wade through, only a couple of shots where she could logically fit in, plus I have a soft spot for her.

If I had the skills and equipment of whoever made the fight scene above, who knows what monstrosities I could create!

I think rand (soft spot aside) got basically as much as most of the non-triumvirate characters. I don’t remember seeing Sulu doing anything but smile.

 

[Pauln6]

I think rand (soft spot aside) got basically as much as most of the non-triumvirate characters. I don’t remember seeing Sulu doing anything but smile.

Actually Sulu gets a fair amount of subtle stuff that is easily overlooked but he is at least a presence on the bridge and in command on two occasions, whereas Rand doesn't man a station on the bridge or in engineering. I'd love it if I could give every character one extra line though. I fiddled with some dialogue but it sounds terrible stitched together. Uhura has some great lines in TOS and in STIII but it's hard to find something good to go alongside them.

 

[Maurice]

I dunno. Original film...yeah maybe...but as a halfway house to knit in with the Kenobi of the prequels, it’s certainly working.

My point is simply that If you stuck that scene into the film proper it would not work for the reasons stated. As a stand-alone it works okayish.

 

[jaime]

My point is simply that If you stuck that scene into the film proper it would not work for the reasons stated. As a stand-alone it works okayish.

I think it would work. Especially if another bit here or there was jazzed up a little (as they have been) if the film is then in context as ‘episode iv’ basically. But only in that context.

 

[Mudd]

I think it would work.

Nope. Maurice is right.

 

[BeatleJWOL]

Yup. There are pieces of it that would spice up the scene as presented but the whole thing is about as fanwanky as it gets.

 

[Pauln6]

I think that different people like different things in their movies. I mean, somebody actually thought it was important for Greedo to shoot first.

I wouldn't hate this scene but I don't think it's necessary. Similarly, I prefer STII without the lengthy homage to Laurel and Hardy titled droids on conveyor belt.

 

[jaime]

Nope. Maurice is right.

Nope.
But you agree with him.
Opinions.

I dont think it would work in ANH by itself, but when put in context next to the prequels, i think it works. Because this then becomes the equivalent of Lukes salt planet fight in TLJ.

 

[Pauln6]

Deepfake sucks so bad on my not very bright laptop. Worse, I had to reboot on Wednesday and when I restarted the learning process, none of the data re-loaded so I had to start from scratch. I don't think I can commit to running the program for as long as if takes to work if I can't even switch off the laptop. I'm already bored of waiting for it to hit it's 0.02 variance. I'm thinking I would rather wait until I have a PC with a graphics card when the process only takes about 72 hours.

As a final experiment I've loaded on Terrence O'connor's likeness. She's the only obvious choice to be replaced by Rand in the second half of the movie so I might as well see how it looks.

 

[Pauln6]

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

Sixth attempt at lip syncing an extra line into Rand's scene with McCoy. Tiny, but it triples her dialogue in the scene. Baby steps. This wasn't deepfake mind you, this was done manually by pasting her lip movements from the earlier scene - challenging given how little on screen dialogue she has in the movie.

 

[Maurice]

Nope.
But you agree with him.
Opinions.

I dont think it would work in ANH by itself, but when put in context next to the prequels, i think it works. Because this then becomes the equivalent of Lukes salt planet fight in TLJ.

I am right because a film has to be stylistically cohesive, and that sequence is completely at odds with the look of the entirety of the balance of the film's live-action. Like a generic "flesh" colored band-aid on a dark-skinned person, it would stick out like a sore thumb.

BTW, in June I saw a gorgeous 1985 70mm print of Star Wars with no Special Edition nonsense in it and the whole audience cheered when Han blows Greedo away.

 

[Pauln6]

There is a line to be struck and you can't please all of the people all of the time. I find that I like 70% of the CGI in the remastered TOS less than the original.

 

[Pauln6]

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

My first upgrade to the Rand scene in Kirk's quarters importing a close up of Grace using layers. It will be challenging to manually splice the dialogue because of the shadows on the neck. A few more passes to add more transparency might work. Erasing the transporter room will be even more of a challenge because the background is so dark compared to the quarters. I suspect I will end up with inconsistencies around her hair, which are only obvious when the film is running.

Still it's an interesting (for me) challenge. I think I'll revisit and edit some frames later in the year.

 

[Maurice]

Her eyeline is all wrong for the screen. It looks like she's looking at Bones.

I can't even understand what she's saying after the galley bit.

 

[Pauln6]

Her eyeline is all wrong for the screen. It looks like she's looking at Bones.

I can't even understand what she's saying after the galley bit.

Yeah there are serious limitations when you only have one head shot to work with, although McCoy does respond so eyeline is not a deal breaker at this early stage. The longer shot had the right eyeline but doesn't look as clear and crisp. I could move her further to the right but I'm not sure that will help much.

I can't remove the music track fully which is why there is some background noise at this stage. The line from Miri was whispered so it is hard for us old 'uns to hear clearly. I have to make sure I have no background music on when I'm checking for lip sync.

 

[Pauln6]

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

Still playing with this. My laptop has sort of plateaued on the Deepfake. I think it's too stupid to run through a big enough sample size to improve the image so this is hybridised with the original footage. It's shame because the Deepfaked eybrows move more naturally with the speech but they are too blurry at the moment. I'll keep running it occasionally in the background and see how far I can take it.

The Deepfake just doesn't seem to cope with the close up image in the quarters (lip sync looks jerky and she never seems to close her mouth). I think maybe I should go back to the smaller image where her eyeline is correct as Maurice pointed out. I could paste some lip sync from TOS and then run Deepfake to make it look more natural. Smaller image but a bigger job though.

Terrence O'Connor is safe though. She spends too much time with her face to the side and the program is more likely to paste over Scotty. It isn't a good look.