Author Topic: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)  (Read 18811 times)

Offline apollo16uvc

  • Mars
  • ***
  • Posts: 396
  • Where no telescope has gone before.
    • Patreon
All Apollo 13 16mm footage, interpolated and synchronized with audio:

Watch me at: YouTube
Experience the past: Flickr
Support me on Patreon

Offline molesworth

  • Mars
  • ***
  • Posts: 349
  • the curse of st custards
Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
« Reply #1 on: April 11, 2020, 04:17:26 AM »
Excellent, thank you!  Bookmarked to watch later  :)
Days spent at sea are not deducted from one's allotted span - Phoenician proverb

Offline apollo16uvc

  • Mars
  • ***
  • Posts: 396
  • Where no telescope has gone before.
    • Patreon
Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
« Reply #2 on: April 12, 2020, 07:10:22 PM »
I am currently using decent quality sources from the Apollo Flight Journal.

I would rather use ultra-high quality MXF files as provided on archive.org, such as this one: https://archive.org/details/Apollo-15_Onboard-Film-Mags_EE.mxf

Those have a super high bitrate and look stunning, but there is one big problem: They are 59.94fps. This means there are duplicates because the source is either 1, 6, 12 or 24fps.

Duplicates cause a jerking/stuttering effect in the interpolation. I tried merely saving it as a 24fps file, this sometimes worked but duplicate frames still showed up.

I've tried messing around with reverse telecine and changing framerate on VirtualDub, which worked better, and sometimes got rid of duplicates for decent parts of a scene. But for some reason duplicates would start showing up again eventually, as if the pattern changed and VirtualDub couldn't account for it.

I tried exporting the video file to individual frames and then tried several programs to find duplicates and delete those, but that didn't work either.


So if anybody could find a reliable way to reverse-telecine the 59fps MXF files to 24fps without any duplicate frames that would help a lot. From there on I can just put the 24fps video file in my interpolation program and I can set the input framerate to where it was recorded at: 6 or 12fps.
Watch me at: YouTube
Experience the past: Flickr
Support me on Patreon

Offline TippedIceberg

  • Mercury
  • *
  • Posts: 18
Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
« Reply #3 on: April 13, 2020, 11:27:17 AM »
So if anybody could find a reliable way to reverse-telecine the 59fps MXF files to 24fps without any duplicate frames that would help a lot. From there on I can just put the 24fps video file in my interpolation program and I can set the input framerate to where it was recorded at: 6 or 12fps.

I'm not sure how reliable this method is, but I did a test with ffmpeg on Windows, this seems to output only the unique frames as lossless png files:

Code: [Select]
ffmpeg -i "Apollo-15_Onboard-Film-Mags_EE.mxf" -map 0:0 -vf mpdecimate,setpts=N/FRAME_RATE/TB "FrameOutput/frame_%05d.png"^ Requires a folder named FrameOutput in the same folder as your mxf. "FRAME_RATE" is auto, don't change that.

Found in this stackoverflow thread. There are also threshold options if mpdecimate skips or duplicates frames.

Offline apollo16uvc

  • Mars
  • ***
  • Posts: 396
  • Where no telescope has gone before.
    • Patreon
Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
« Reply #4 on: April 14, 2020, 06:51:15 PM »
That seems to work the best out of anything, actually! it doesnt even skip frames on the calibration chart at the start of each mag, which looks the same aside from the grain.

Very good.

I've made a workflow that takes the raw proress422 150mb/s mxf file, crops it, decimates it, and saves as PNGs:

ffmpeg -ss 240 -t 250 -i "D:\Apollo\16mm\AS16\Apollo-16_Onboard-Film-Mags_BBtoT.mxf" -map 0:0 -vf mpdecimate,setpts=N/FRAME_RATE/TB,crop=960:720:156:00 "FrameOutput/frame_%05d.png"

This runs at 39/40fps.

This is the the apollo 16 deep space eva.

THis is handy because I can import the png files directly into the interpolation program.

Thanks.
Watch me at: YouTube
Experience the past: Flickr
Support me on Patreon

Offline TippedIceberg

  • Mercury
  • *
  • Posts: 18
Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
« Reply #5 on: April 15, 2020, 03:48:39 PM »
Good to hear it works! And the crop command is useful - thanks.

Looking forward to the results.

Offline apollo16uvc

  • Mars
  • ***
  • Posts: 396
  • Where no telescope has gone before.
    • Patreon
Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
« Reply #6 on: April 22, 2020, 03:16:34 AM »
Good to hear it works! And the crop command is useful - thanks.

Looking forward to the results.

The Apollo 16 Deep Space EVA, 16mm + TV:

Watch me at: YouTube
Experience the past: Flickr
Support me on Patreon

Offline smartcooky

  • Uranus
  • ****
  • Posts: 1959
Re: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
« Reply #7 on: April 22, 2020, 06:57:11 AM »
Good to hear it works! And the crop command is useful - thanks.

Looking forward to the results.

The Apollo 16 Deep Space EVA, 16mm + TV:



Mate, that is just outstanding. Looks right up there with some of the modern digital stuff from the ISS
If you're not a scientist but you think you've destroyed the foundation of a vast scientific edifice with 10 minutes of Googling, you might want to consider the possibility that you're wrong.

Offline TippedIceberg

  • Mercury
  • *
  • Posts: 18
Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
« Reply #8 on: April 22, 2020, 09:12:08 AM »
The Apollo 16 Deep Space EVA, 16mm + TV:



That looks incredible. If you said this footage was originally shot at 24fps, I'd believe it.

Offline apollo16uvc

  • Mars
  • ***
  • Posts: 396
  • Where no telescope has gone before.
    • Patreon
Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
« Reply #9 on: April 22, 2020, 02:01:43 PM »
Thank you guys!

I am working on tons of new stuff including a tutorial on how to do this yourself. You'll get a week early access on Patreon.

Currently working on a video of the Apollo 14 descent, eva and ascent from the HQ MXF source. Combining it with TV footage and Hasselblad photos. It will be just comms too, no background music.



I am a nut for ambiance music, you can tell from some of my music choices.
Watch me at: YouTube
Experience the past: Flickr
Support me on Patreon

Offline apollo16uvc

  • Mars
  • ***
  • Posts: 396
  • Where no telescope has gone before.
    • Patreon
Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
« Reply #10 on: April 29, 2020, 01:59:07 PM »
Apollo 14: The complete Descent, EVA & Ascent in 16mm, 24fps:

Watch me at: YouTube
Experience the past: Flickr
Support me on Patreon

Offline apollo16uvc

  • Mars
  • ***
  • Posts: 396
  • Where no telescope has gone before.
    • Patreon
Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
« Reply #11 on: April 29, 2020, 04:31:11 PM »
Apollo 17 in 24fps: O R B I T


Watch me at: YouTube
Experience the past: Flickr
Support me on Patreon

Offline Peter B

  • Saturn
  • ****
  • Posts: 1268
Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
« Reply #12 on: April 30, 2020, 07:03:07 PM »
A grim thought, for which I apologise in advance, but would there be any benefit in applying these techniques to footage of the "Challenger" accident?

Having watched the grainy, blurry footage of the Shuttle's breakup I wonder how investigators got as much information from it as they did back in the 1980s. I wonder if there might be more insights to be gained from cleaned up footage, or has that already been done?

Offline apollo16uvc

  • Mars
  • ***
  • Posts: 396
  • Where no telescope has gone before.
    • Patreon
Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
« Reply #13 on: May 09, 2020, 07:16:32 AM »
A grim thought, for which I apologise in advance, but would there be any benefit in applying these techniques to footage of the "Challenger" accident?

Having watched the grainy, blurry footage of the Shuttle's breakup I wonder how investigators got as much information from it as they did back in the 1980s. I wonder if there might be more insights to be gained from cleaned up footage, or has that already been done?
I do not think that would be of any use to analysis. It aren't real frames, after all. No actual new data...
Watch me at: YouTube
Experience the past: Flickr
Support me on Patreon

Offline JayUtah

  • Neptune
  • ****
  • Posts: 3787
    • Clavius
Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
« Reply #14 on: May 09, 2020, 11:55:51 AM »
I'm not sure it would shed any more light on the accident, its causes, and aftermath.  But it's a remarkable set of techniques, and the video is important to our culture and history.  The end of the passenger dirigible era happened with Hindenburg not because it was the first or even worst dirigible accident, but because it was caught on film.  In the words of the relevant industry, there were "visuals."  Visuals tell stories in powerful ways, even stories of our hubris and fallibility.
"Facts are stubborn things." --John Adams