Handle "moving photo" video files

[michaelkebe]

Just tested gpth with a takeout. I have a Google Pixel 6a. It has a "Top Shot" feature, which records a video instead of making a single photo.

It looks like in the takeout these top shots are stored as follows:

Photos from 2023/PXL_20230205_164128403.MP.jpg.json
Photos from 2023/PXL_20230205_164128403.MP.jpg
Photos from 2023/PXL_20230205_164128403.MP

Currently these .MP files are ignored by gpth and are kept in the input folder.

I have no concrete idea how to deal with these files. Any discussions are welcome.

The .MP file is a MPEG-4. Here is the output of mediainfo:

$ mediainfo PXL_20230205_164128403.MP
General
Complete name                            : PXL_20230205_164128403.MP
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/mp41)
File size                                : 4.39 MiB
Duration                                 : 2 s 772 ms
Overall bit rate                         : 13.3 Mb/s
Encoded date                             : UTC 2023-02-05 16:41:28
Tagged date                              : UTC 2023-02-05 16:41:28
FileExtension_Invalid                    : braw mov mp4 m4v m4a m4b m4p m4r 3ga 3gpa 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma ismt f4a f4b f4v

Video #1
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : [email protected]@Main
Codec ID                                 : hvc1
Codec ID/Info                            : High Efficiency Video Coding
Duration                                 : 2 s 772 ms
Bit rate                                 : 12.9 Mb/s
Width                                    : 1 440 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 4:3
Rotation                                 : 180°
Frame rate                               : 27.056 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Bits/(Pixel*Frame)                       : 0.306
Stream size                              : 4.26 MiB (97%)
Title                                    : VideoHandle
Language                                 : English
Encoded date                             : UTC 2023-02-05 16:41:28
Tagged date                              : UTC 2023-02-05 16:41:28
Color range                              : Full
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
Codec configuration box                  : hvcC

Video #2
ID                                       : 2
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : [email protected]@Main
Codec ID                                 : hvc1
Codec ID/Info                            : High Efficiency Video Coding
Duration                                 : 747 ms
Bit rate                                 : 1 095 kb/s
Width                                    : 2 048 pixels
Height                                   : 1 536 pixels
Display aspect ratio                     : 4:3
Rotation                                 : 180°
Frame rate mode                          : Constant
Frame rate                               : 1.338 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Bits/(Pixel*Frame)                       : 0.260
Stream size                              : 99.9 KiB (2%)
Title                                    : VideoHandle
Language                                 : English
Encoded date                             : UTC 2023-02-05 16:41:28
Tagged date                              : UTC 2023-02-05 16:41:28
Color range                              : Full
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
Codec configuration box                  : hvcC

Other #1
Type                                     : meta
Duration                                 : 2 s 772 ms
Bit rate mode                            : Constant

Other #2
Type                                     : meta
Duration                                 : 1 s 121 ms
Bit rate mode                            : Constant

More information is available here:
https://support.google.com/googlecamera/answer/9937175?hl=en
https://support.google.com/photos/thread/139943915/open-or-convert-mp-file?hl=en

[TheLastGimbus]

Hm, interesting

but looking into how it's structured, i don't think it will be easy and clean to search for their jsons etc

also, their extension don't point directly to being a video: https://file.org/extension/mp

[michaelkebe]

I should have mentioned that the JSON does not include any information about the .MP file.

[TheLastGimbus]

Soo, you think we can close this for now? Or you think those .mps are valuable?

[michaelkebe]

That's good question.

On the one hand .mp is a non-standard extension and that's a bad thing. One could argue to not support it.

On the other hand they are valuable, since the takeout is a complete backup and gpth should handle it completely. Otherwise potentially data is lost.

Tough call.

[TheLastGimbus]

taking everything into account, i think that it's not worth it :/ 99.99% of people won't care about this (do you?)

but feel free to re-open if anyone really wants this

[TheLastGimbus]

@wicked-head also had a problem with mp4 files from iphone 🤔

altough their case is that it is moved, because its mp4 not .MP, but they have a problem with date detection

[TheLastGimbus]

hey @michaelkebe, i think i know why they are mp not mp4 - the good old #8 :')

[eligibleshield]

Help.

On iOS a photo via Google Photos produces 3 things:

ImageX.HEIC
ImageX.HEIC.json
ImageX.mp4

But the JSON seems to only merge with the HEIC, not the Live Photo mp4

How can I merge it to both the photo and the live photo (video)?

[palijn]

Hi , can you paste here the contents of such a json file ? Thanks ⁣Télécharger TypeApp pour Android ​ Le 8 juin 2023 à 03:32, à 03:32, eligibleshield ***@***.***> a écrit:

…

Help. On iOS a photo via Google Photos produces 3 things: ImageX.HEIC ImageX.HEIC.json ImageX.mp4 But the JSON seems to only merge with the HEIC, not the Live Photo mp4 How can I merge it to both the photo and the live photo (video)? -- Reply to this email directly or view it on GitHub: #180 (comment) You are receiving this because you are subscribed to this thread. Message ID: ***@***.***>

[eligibleshield]

It makes no reference to the video inside the JSON...
Does that mean it's a problem on Google Photos' end? Are they not creating JSons for videos??

[eligibleshield]

I really want the metadata applying to the videos too...

[TheLastGimbus]

Are they not creating JSons for videos??

if you say so then it looks like it... doesn't surprise me tho

I really want the metadata applying to the videos too...

Currently, gpth doesn't really do anything about metadata, besides setting correct lastModified file property - which really helps getting photos chronologically

But no worries! Besides json and photo exif, gpth also tries to guess this date by file name - post a screeshot/something of those files/videos here, and we'll try to figure that out

[TheLastGimbus]

@michaelkebe i have pixel with gcam too, and top shot photos seem to be named like: PXL_20230604_172812686.MP.jpg

there is this MP thing 🤔 - but the image itself contains top shot video

maybe google photos also exports you the extra MP file for whatever reaseon, but please check if maybe image itself still has it

[TheLastGimbus]

Hi guys!

So i've looked at this, and I think i want to do this as good as possible - just as albums - they took long, but now are basically as good as they possible can be implemented 🔥

@BobTheSoftwareDeveloper and @andrewchen5678 - your work and reaserch in #224 is very helpful! But so far it covers only detecting, setting right date and moving the files...

I'm thinking of such situations:

User has most photos from iphone, and is now:

moving iphone+icloud

i need to know how Apple users would manually upload photos to iCloud, and how they could do this to include live photo? As far as i've seen, their livephotos are literally two separate files, jpg and mp4, pointing to each other??

I've seen some libraries for this (https://github.com/LimitPoint/LivePhoto), and they seem to be ment to be used inside the iphone apps... and, this seems to be main way to add any photos to users library...

What i want to say is that it might be impossible for us to prepare the files nicely in the output folder so that when user drag-and-drops them into some apple web app (idk if something like this exists) and sees live-photos nice in-place 👀 but i may be wrong - maybe we just need to upload the image+video pair at once?

What I need

• someone to tell me how apple users would add photos to icloud after using gpth
• someone to try to export live photo from their gallery as those two files and then re-upload it and see that it's still "live" ??

moving to other gphotos / basically any android/non-apple photo solution

I think we could allow ourselves to do such an invasive but convinient thing of converting those live photos to motion photo standard that 99% of modern android phones understand - this one is better documented, has some libraries (https://github.com/googleinterns/libmphoto), and ends up with a single jpg file - making less mess

https://medium.com/android-news/working-with-motion-photos-da0aa49b50c

(Yeah including the above library would require including .so/.dll files with gpth, which i strained from before, but this would be such a nice feature that i could jump over this - and maybe even move gpth to full flutter app, finally 🤔)

OR - maybe platforms like Synology recognize this jpg+mp4 pair too? Since it's apple, and it's common... let me know if anyone knows

User has most photos from android and is now:

Moving to icloud, for some reason

If the format of motions in the takeout is standard (just single jpg), we could extract it with the above library and then try to prepare our DIY apple-motion-photo (two files) - again, i would need some apple person to tell me if it's possible

If it's separate, then maybe it's ready to be imported straight to icloud??

---

I think this is it. Now i need your help :)

@michaelkebe @eligibleshield @aidan-gibson @BobTheSoftwareDeveloper @andrewchen5678 @wicked-head

[dbrennand]

someone to tell me how apple users would add photos to icloud after using gpth

Hi @TheLastGimbus - I'm currently planning a migration from Google Photos to iCloud.

I've recently ran gpth and noticed I have a lot of date-unknown folders. All of the files in them are .MP4 files, which I'm unable to import:

I think these may be from Live Photos? And I may need to wait for #224. I'm going to test the nightly build and see if it makes any difference 🙂

Anyways... regarding your question:

someone to tell me how apple users would add photos to icloud after using gpth

The easiest way based on my research (although, this is dependent on having a Mac) is using the Photos app on Mac.

Once I've used gpth - I'd do the following:

1. Open Photos app.
2. File > Import
3. Select the top level export folder.
4. Check Keep Folder Organization - This creates albums from each folder name.
5. Select Import All New Items

[TheLastGimbus]

thanks for the reaserch!

.MP4 files, which I'm unable to import:

could you try to match them with their og photos, and upload the two of them at once? Will the Photos app figure out and preseve them as live photo 👀 ??

[dbrennand]

thanks for the reaserch!

.MP4 files, which I'm unable to import:

could you try to match them with their og photos, and upload the two of them at once? Will the Photos app figure out and preseve them as live photo 👀 ??

I imported the .JPG and .MP4 for a live photo from the ALL_PHOTOS directory at the same time using the Mac Photos app and it is able to figure it out and preserve the live photo 🙂

[TheLastGimbus]

This is awesome!!!

Knowing this I'm happy to announce that I officially plan full blown live photos support here - not just dates but also going from android to iphone and iphone to android - i want to do this right

Although I have no idea when i will have time to "do it right", so the nightly in the PR must be enough for early adopters for now 😁😁

[rienkvr]

If you need help testing, I am able to test on MacOS / Iphone from an android devices (Pixel 5) with moving photos.

[TheLastGimbus]

@rienkvr thanks! I still don't know when i will sit by this, so if possible, please keep your Takeout zip somewhere to test it in the future

In general - it would help me if you guys would submit here a zips of small subsets of your takeouts that would contain those live photos - both from iphones and androids

If you don't want your zips laying publicly in here, but do trust me to have them - you can email them here: [email protected]

[validatedev]

I've made a little Python script here. It works such that whenever an mp4 file is found, it checks if the corresponding (same name) heic, jpg, or png file has that JSON. If it does, the script copies the JSON file in the same directory for the mp4 as well. My live photos are saved :)

You need to run this first to correct file names:
exiftool -if '$filename=~/(\.[^.]+)(\(\d+\)).json$$/i' '-filename<${filename;s/(\.[^.]+)(\(\d+\)).json$/$2$1.json/}' -progress -ext json -r ./

Then run the script:

import os
import shutil


def process_directory(directory):
    counter = 0
    for filename in os.listdir(directory):
        if filename.lower().endswith('.mp4'):
            json_file = f'{filename}.json'

            # Check if the JSON file for the MP4 exists
            if not os.path.exists(os.path.join(directory, json_file)):
                # Check for corresponding image files and their JSON
                base_name, _ = os.path.splitext(filename)
                for image_ext in ['.heic', '.jpg', '.png']:
                    image_file = f'{base_name}{image_ext}'
                    image_json = f'{base_name}{image_ext}.json'
                    if os.path.exists(os.path.join(directory, image_file)) and os.path.exists(os.path.join(directory, image_json)):
                        # Copy and rename the JSON file with exact name matching
                        shutil.copyfile(os.path.join(directory, image_json), os.path.join(directory, json_file))
                        counter += 1
                        print(f"JSON file copied: {json_file}")
                        break

    return counter


if __name__ == "__main__":
    directory_path = input("Enter the directory path: ")
    counter = process_directory(directory_path)
    print("Processing complete.")
    print(f"Number of JSON files copied: {counter}")

[lorenzopicoli]

Thank you @validatedev your script works perfectly for me. I did a small change to get it to look recursively through the albums folders that Google Takeout creates. Here's the updated version:

import os
import shutil

def process_directory(directory):
    counter = 0

    # Iterate over all files and directories in the current directory
    for entry in os.listdir(directory):
        full_path = os.path.join(directory, entry)

        if os.path.isdir(full_path):
            # If the entry is a directory, recursively process this directory
            counter += process_directory(full_path)
        elif entry.lower().endswith('.mp4'):
            json_file = f'{entry}.json'

            # Check if the JSON file for the MP4 exists
            if not os.path.exists(os.path.join(directory, json_file)):
                # Check for corresponding image files and their JSON
                base_name, _ = os.path.splitext(entry)
                for image_ext in ['.heic', '.jpg', '.png']:
                    image_file = f'{base_name}{image_ext}'
                    image_json = f'{base_name}{image_ext}.json'
                    if os.path.exists(os.path.join(directory, image_file)) and os.path.exists(os.path.join(directory, image_json)):
                        # Copy and rename the JSON file with exact name matching
                        shutil.copyfile(os.path.join(directory, image_json), os.path.join(directory, json_file))
                        counter += 1
                        print(f"JSON file copied: {json_file}")
                        break

    return counter

if __name__ == "__main__":
    directory_path = input("Enter the directory path: ")
    counter = process_directory(directory_path)
    print("Processing complete.")
    print(f"Number of JSON files copied: {counter}")

[djsavvy]

@lorenzopicoli thanks for your updated script. I had to add .jpeg to the list of checked file formats to fix all the import errors, but otherwise this worked perfectly!

[radlinsky]

For others in a similar situation... My partner had an iPhone that wasn't backing up to iCloud (didn't want to pay for the extra storage), so I set up Google Photos on her iPhone. The iPhone died, and she wanted to get her photos back onto her new iPhone. There were years worth of photos from several iPhones. In the end, it was as simple as using the Apple Photos app on a Mac to import the Google Takeout folders. The photos app automatically imported live photos, even though many were separated.

There were some things I noticed in the process that might be useful for this issue (I don't think I'm the first to point this all out, but I'll risk repeating what others said just in case)

• Every iPhone live photo is made up of 2 files, one is always an MP4, and the other is either a HEIC or JPG.
• iPhones seem to reuse the same image name. E.g., IMG_1234.JPG & IMG_1234.MP4 generated in 2020 and IMG_1234.JPG & IMG_1234.MP4 generated in 2024.
• Many (all?) of the photos and movies (MOV) taken on the iPhone camera have an Apple Content ID in the meta information within the file. The Apple Content ID is identical for the two live photo files, both the MP4 and it's associated still image (HEIC or JPG). BTW screenshots don't have an Apple Content ID.
• There are several creation time/date meta tags in MP4s, JPGs, and HEICs that have similar sounding names. But when I look at a single live photo file pair, I could not find any creation time tag that gave the same time. I think it's different formatting. IIRC, the MP4 creation_time has decimals after the seconds, and the tags in the JPG/HEIC round the seconds. I saw the creation_time tag in MP4s and the JPG and HEIC tags were CreationTime and DateTimeOriginal. I didn't dig into this, but I suspect these tags are how you'll have to sort out the actual date a live photo was taken. You might just pick one of these three tags to use for determining when the live photo was taken from the POV of gpth. And maybe ignore the decimals after a second, or round it? But maybe its possible to take two photos in the same second? I don't know...
• Some live photos have very long names with slightly different endings, but they definitely belong together because they share the same Apple Content ID. I'm not sure if this was caused by the iPhone or perhaps by editing in Google Photos. E.g. #########__#######__####-####-####-#######73.MP4 and #########__#######__####-####-####-#######7.HEIC . I found you could still use the filename and get around this problem by looking for files that had "-" and "__" and were at least 30 characters long. Then ignore the part of the filename after the last dash.
• Sometimes, a live photo is missing one of the two files in Google Takeout. I had tens of thousands of media files, many of them live photos. A couple dozen of these were incomplete (either missing the MP4 or missing the HEIC/JPG). I'm not sure if this is a Google Takeout bug, or something got corrupted when I moved files around, or my unzipper messed up... it was rare enough I decided not to dig deeper.

I attached some Python scripts that use ffprobe and exiftools to extract meta data and then sort which pairs of files are live photos. They're pretty rough scripts, but hopefully helpful.

livepairs.py.txt
get_meta.py.txt

[janhoy]

I just spent hours dealing with manual prepping (thanks for the scripts above) for my live photos. Would be great if you added this as a native feature. I sent a donation by PayPal btw as a thank you for this tool that really helps save time in this process.