ApolloHoax.net

Apollo Discussions => The Reality of Apollo => Topic started by: apollo16uvc on April 11, 2020, 03:16:53 AM

Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on April 11, 2020, 03:16:53 AM
All Apollo 13 16mm footage, interpolated and synchronized with audio:

Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: molesworth on April 11, 2020, 04:17:26 AM
Excellent, thank you!  Bookmarked to watch later  :)
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on April 12, 2020, 07:10:22 PM
I am currently using decent quality sources from the Apollo Flight Journal.

I would rather use ultra-high quality MXF files as provided on archive.org, such as this one: https://archive.org/details/Apollo-15_Onboard-Film-Mags_EE.mxf

Those have a super high bitrate and look stunning, but there is one big problem: They are 59.94fps. This means there are duplicates because the source is either 1, 6, 12 or 24fps.

Duplicates cause a jerking/stuttering effect in the interpolation. I tried merely saving it as a 24fps file, this sometimes worked but duplicate frames still showed up.

I've tried messing around with reverse telecine and changing framerate on VirtualDub, which worked better, and sometimes got rid of duplicates for decent parts of a scene. But for some reason duplicates would start showing up again eventually, as if the pattern changed and VirtualDub couldn't account for it.

I tried exporting the video file to individual frames and then tried several programs to find duplicates and delete those, but that didn't work either.


So if anybody could find a reliable way to reverse-telecine the 59fps MXF files to 24fps without any duplicate frames that would help a lot. From there on I can just put the 24fps video file in my interpolation program and I can set the input framerate to where it was recorded at: 6 or 12fps.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: TippedIceberg on April 13, 2020, 11:27:17 AM
So if anybody could find a reliable way to reverse-telecine the 59fps MXF files to 24fps without any duplicate frames that would help a lot. From there on I can just put the 24fps video file in my interpolation program and I can set the input framerate to where it was recorded at: 6 or 12fps.

I'm not sure how reliable this method is, but I did a test with ffmpeg (https://www.ffmpeg.org/download.html) on Windows, this seems to output only the unique frames as lossless png files:

Code: [Select]
ffmpeg -i "Apollo-15_Onboard-Film-Mags_EE.mxf" -map 0:0 -vf mpdecimate,setpts=N/FRAME_RATE/TB "FrameOutput/frame_%05d.png"^ Requires a folder named FrameOutput in the same folder as your mxf. "FRAME_RATE" is auto, don't change that.

Found in this stackoverflow thread (https://stackoverflow.com/a/37089629). There are also threshold options (https://www.ffmpeg.org/ffmpeg-filters.html#mpdecimate) if mpdecimate skips or duplicates frames.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on April 14, 2020, 06:51:15 PM
That seems to work the best out of anything, actually! it doesnt even skip frames on the calibration chart at the start of each mag, which looks the same aside from the grain.

Very good.

I've made a workflow that takes the raw proress422 150mb/s mxf file, crops it, decimates it, and saves as PNGs:

ffmpeg -ss 240 -t 250 -i "D:\Apollo\16mm\AS16\Apollo-16_Onboard-Film-Mags_BBtoT.mxf" -map 0:0 -vf mpdecimate,setpts=N/FRAME_RATE/TB,crop=960:720:156:00 "FrameOutput/frame_%05d.png"

This runs at 39/40fps.

This is the the apollo 16 deep space eva.

THis is handy because I can import the png files directly into the interpolation program.

Thanks.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: TippedIceberg on April 15, 2020, 03:48:39 PM
Good to hear it works! And the crop command is useful - thanks.

Looking forward to the results.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on April 22, 2020, 03:16:34 AM
Good to hear it works! And the crop command is useful - thanks.

Looking forward to the results.

The Apollo 16 Deep Space EVA, 16mm + TV:

Title: Re: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: smartcooky on April 22, 2020, 06:57:11 AM
Good to hear it works! And the crop command is useful - thanks.

Looking forward to the results.

The Apollo 16 Deep Space EVA, 16mm + TV:



Mate, that is just outstanding. Looks right up there with some of the modern digital stuff from the ISS
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: TippedIceberg on April 22, 2020, 09:12:08 AM
The Apollo 16 Deep Space EVA, 16mm + TV:



That looks incredible. If you said this footage was originally shot at 24fps, I'd believe it.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on April 22, 2020, 02:01:43 PM
Thank you guys!

I am working on tons of new stuff including a tutorial on how to do this yourself. You'll get a week early access on Patreon.

Currently working on a video of the Apollo 14 descent, eva and ascent from the HQ MXF source. Combining it with TV footage and Hasselblad photos. It will be just comms too, no background music.



I am a nut for ambiance music, you can tell from some of my music choices.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on April 29, 2020, 01:59:07 PM
Apollo 14: The complete Descent, EVA & Ascent in 16mm, 24fps:

Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on April 29, 2020, 04:31:11 PM
Apollo 17 in 24fps: O R B I T


Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: Peter B on April 30, 2020, 07:03:07 PM
A grim thought, for which I apologise in advance, but would there be any benefit in applying these techniques to footage of the "Challenger" accident?

Having watched the grainy, blurry footage of the Shuttle's breakup I wonder how investigators got as much information from it as they did back in the 1980s. I wonder if there might be more insights to be gained from cleaned up footage, or has that already been done?
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on May 09, 2020, 07:16:32 AM
A grim thought, for which I apologise in advance, but would there be any benefit in applying these techniques to footage of the "Challenger" accident?

Having watched the grainy, blurry footage of the Shuttle's breakup I wonder how investigators got as much information from it as they did back in the 1980s. I wonder if there might be more insights to be gained from cleaned up footage, or has that already been done?
I do not think that would be of any use to analysis. It aren't real frames, after all. No actual new data...
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: JayUtah on May 09, 2020, 11:55:51 AM
I'm not sure it would shed any more light on the accident, its causes, and aftermath.  But it's a remarkable set of techniques, and the video is important to our culture and history.  The end of the passenger dirigible era happened with Hindenburg not because it was the first or even worst dirigible accident, but because it was caught on film.  In the words of the relevant industry, there were "visuals."  Visuals tell stories in powerful ways, even stories of our hubris and fallibility.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on May 09, 2020, 03:56:43 PM
Not really comfortable working on that kind of footage...

Feel free to give it your own shot, scroll down some to get a download: https://www.patreon.com/DAINAPP

You need a nvidia card with at least 4GB of memory if you want 360p...

Oh, and a high-quality source is paramount of you want usable results. And its prob going to take a few hours...
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: JayUtah on May 10, 2020, 11:30:36 AM
Not really comfortable working on that kind of footage...

Understandable.

Quote
You need a nvidia card with at least 4GB of memory if you want 360p...

Oh, and a high-quality source is paramount of you want usable results. And its prob going to take a few hours...

I have no problem obtaining computing capacity.  ;D
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on May 16, 2020, 08:36:10 AM
RTX 3000 series cards are just a few months away, those are going to be at least 40 to 60% faster than the 2000 series. And much cheaper.
Thanks to AMD getting back into the game and absolutely crushing INTEL on every front.

Title: Re: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: Count Zero on May 16, 2020, 12:58:09 PM
Beautiful!  I find it much easier to follow the action and understand what he's doing.  As always, I like seeing those fans of dust flying out from his footfalls in the airless, low-gee environment.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on May 17, 2020, 01:39:21 PM
Would like to know your opinions on something:

After how much transformation of the original, public-domain content does material become derivative work on which the transformer can apply his own copyright? By which measurement?
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: JayUtah on May 17, 2020, 03:16:33 PM
The question is a legal one, and as with all legal questions the answer is, "It depends."

Peter Kuran was able to claim copyright on Trinity and Beyond.  That's classified as "restoration," and that's copyrightable.  Applying creative techniques to enhance source material and correcting the effects of age is intellectual property.  Of course Kuran also produced original material to explain and dramatically frame his source footage, so the question of copyright on the entire opus doesn't really present a legal difficulty.  But, for example, if you were to use his cleaned-up and enhanced footage of nuclear-device tests he would have a strong copyright claim against you.

You have taken public domain material and applied various commonly-available tools to produce a cleaned-up, interpolated version.  You're adding material to what was there, which was created by you, and modifying the existing material according to decisions that you make on how to apply the tools.  That's creative content, and you are entitled to claim copyright in it.  The question, "by what measure?" is best answered by stating to what degree you applied expertise or artistic/aesthetic judgment in producing the final result.

The bar you have to clear in order to be an "author" as far as copyright goes is fairly low.  Someone defending an infringement claim you might bring, on the grounds that you have merely repackaged public-domain content, would have to show that the process by which you produced your version was so inconsequential and automatic that it amounted to nothing creative.  I haven't yet looked at the tools, so I have no informed opinion for how much skill it takes to operate them, or what creative choices are possible.  But in general, any substantial transformation you make to public domain material is copyrightable if there is a chance that someone else can make similar choices differently.  For example, Mark Gray is able to claim copyright on many of his Spacecraft Films DVDs because he has edited them into a convenient volumes.  Editions of public-domain material can receive a whole-work copyright on the edition, even if everything that contributed to the edition was originally public domain.  Selecting what is or isn't in your edition is authorship as copyright defines it.

The counterargument is that all you have done is to make public-domain material more useful and appealing toward the purposes for which the original material would have been sought.  Commercial uptake of source material for commercially-exploitable purposes is one of the reasons we like having public-domain material be freely, non-exclusively usable.  We want creators to be able to say, "I don't have the talent or energy to produce new content entirely on my own, but I can do something creative and useful using this other thing as a jumping-off point."  And one of the chronic impediments to this is the difficulty in obtaining high-quality versions of public-domain material.  The exceptions to copyright such as Fair Use and public domain are meant to speak to the larger issue of protecting creativity in general.  That is, a court will balance allegations of infringement against the value of allowing the particular use case to the general climate of creativity.  It is conceivable that a court could rule that, despite the effort you put in, you've merely contributed to making public-domain material more suitable for the kind of reuse it was intended for.

My personal, non-lawyer opinion is that the pro-authorship argument is stronger.  The creative, editorial effort required to produce high-quality renditions of already-existing materials probably qualifies as a copyrightable transformation.  If you really need to know, you need an IP lawyer.  And sadly they're often quite expensive.  They require extra training because they need a degree not only in law but also in the field the intellectual property comes from -- here, image processing and artificial intelligence.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on May 17, 2020, 03:54:26 PM
Thank you jay for such a thorough reply.

I personally don't think my transformations are substantial and creative enough to merit me as a copyright owner. The process to make the interpolations didn't require much creative effort or exceptional skill.

However, the video as a whole does take some time to make. Especially the bigger and more complex ones, as audio, photos and video has to be synchronized and colour-corrected/exposure curves adjusted.

I want to clarify I don't intent to sell any of this, they will always be freely on YT. I only uploaded them under the usual copyright licence because I don't want some documentary or company to just download my vids and sell them on their own, leaving me in the dark about all of it.


I know many people have made mosaics of Apollo panorama's, stitching them together. Many of these can be found in books, as prints or in documentaries. Do these kind of transformations usually have enough merit for the creator to have copyright on them?

It depends, like anything I suppose. Mosaics can be easy, just stitching frames automatically together with photoshop. Or you can put more effort in  it, like adding/changing objects in the composite, cleaning/restoring individual photos.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on May 17, 2020, 03:56:58 PM
And skill is an other thing.

Back in the day, making even minor changes required great darkroom skill, secure timings, special and precise tooling.

It can be argued this has been made much easier in photoshop, with so many features like stitching together photos and adjusting for shadows/highlights now completely automated.

Editing video was even more difficult, requiring large devices and edit rooms for which the user needed special skills and years of experience to master. Even basic things like cutting, changing audio. Let alone complex effects.

Many of these things can now be done on a phone.

SO where does this leave the bar of skill and effort required for a work to be transformed enough?
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: JayUtah on May 17, 2020, 04:30:36 PM
Degree of skill doesn't really matter.  You can spend your life learning to paint and then paint a masterpiece.  And then some parent can have his toddler throw up on a canvas, and both works are equally deserving of copyright protection.  Level of effort sort of matters, but generally only as it employs originality or creativity.  The fact that image correction and video editing have been made so much easier by technology does not mean the products cannot be copyrighted.  If you make creative choices, you can protect them even if they would seem insignificant to others.

In a derived work, the degree to which the derivation departs from the original bears on whether it can be separately copyrighted.  Here there is no copyright on the original, but if your work is markedly different from what can be publicly obtained, it may be a candidate for copyright.  That's where your claim is the shakiest but it's still reasonably strong.  It wouldn't matter how easy or hard it was for you to do.  But it might matter how much your effort relied on creative choices that someone else could make differently.  Copyright is not an assessment of inherent or imbued worth, only of originality.

I think your use of copyright here is appropriate.  You are not doing anything to stifle general creativity or limit the further use of your work unreasonably.  You're correctly employing copyright to prevent others from asserting ownership -- perhaps commercial ownership -- over what is not their work, which aids in promoting creativity.  This is the copyright that Clavius.org operates under.  The original portions of it are copyrighted, but I've ever only had to make one copyright infringement claim in the entire 20-year history of the site.  Because the licensing terms are lenient -- simply give credit to the original source -- I've enjoyed a lot of cordiality.  There is a professor of astronautical engineering somewhere in Georgia who's using many of my illustrations in his classroom.  He asked permission, which I happily granted.  Third parties have translated large portions of the site into other languages.  Again, they asked permission.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on May 17, 2020, 04:55:28 PM
I have no problem with people using my work as long as permission is asked and I am credited appropriately.

And if the end product gets sold or broadcasted a reasonable fee.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: JayUtah on May 18, 2020, 12:54:58 AM
That seems like a reasonable policy.

Copyright law is, I'm told, the hardest for us laymen to grasp correctly.  It costs you nothing nor risks anything to claim copyright.  But acting on those claims really requires the assistance of a properly specialized lawyer.  I might be completely wrong about everything.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on June 10, 2020, 11:17:11 AM
DAIN-APP 0.39 released some days ago.

This adds several features, fixes and a better thought out interface.

Most importantly, it allows me to enter the compression rate for the final mp4 output.

It uses ffmpeg for this, and now has an option to change the CRF value, which was previously fixed to 15.

Setting it to 1 or even 0 greatly enhances output quality.

I wanted to wait for this feature before making more renders and re-doing some old ones I did as tests, such as the APollo 15 ascent.

Some off the tests I will now be rendering to completion, with just comms and the footage as a 4:3 video:










Does anybody knows which 16mm rover traverses were recorded at 6 or 12fps?
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on June 11, 2020, 03:53:39 PM
Apollo 15 Lunar Ascent:

Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on June 17, 2020, 03:57:24 PM
I'm going to call this a proof-of-concept/experiment  :D

Apollo 11 flag deployment interpolated from 1 to 8fps (8x interpolated) . This took like 2-3 days to complete.

Naturally there is a copyright claim on the audio that I have already filed a dispute for.


Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on June 18, 2020, 10:57:41 AM
Apollo 15 Lunar Landing interpolated from 12fps to 60fps, taking 5 hours:

Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on June 21, 2020, 08:53:35 AM
A quick look behind the scenes:

Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on June 22, 2020, 05:49:40 PM


Showing how well the DAIN-AI can interpolate low framerates. Slowed down footage to get the point across. This scene has by far the most extreme movement out of all the footage. Its a worst-case scenario.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: molesworth on June 23, 2020, 05:48:15 AM
That's amazingly good, considering the frame rate of the original.  Having spent many years working in computer graphics (simulators then games) I'm still continually amazed at the progress which continues to be made in all parts of the field.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: TippedIceberg on June 28, 2020, 06:12:59 AM
I saw the Apollo 16 rover 60fps (https://reddit.com/r/space/comments/hgqygy/apollo_16_in_60fps_rover_traverse_to_station_4/) earlier on reddit /r/all - Congratulations! :D It's great that you're breathing new life and interest into this footage.

SO where does this leave the bar of skill and effort required for a work to be transformed enough?
Maybe the new interpolated frames could fall under transformed works? But that is complicated, could others independently create bit-identical interpolated frames with the same software?

There is probably no precedent. Closest thing I could find is https://en.wikipedia.org/wiki/Bridgeman_Art_Library_v._Corel_Corp.
Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on July 03, 2020, 04:45:44 PM
Thank you!



Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: apollo16uvc on July 08, 2020, 08:17:44 AM
Apollo 16 rover traverse to Station 13:

Title: Re: AI Interpolation of NASA footage (Smoothing/Increasing Fps)
Post by: Count Zero on July 08, 2020, 11:28:13 AM
WOW!!!  That has always been my favorite DAC footage, and you rendered it beautifully.

Thank you so much for your time and effort.