Preserving Side Data information for 360 video transcoding using FFmpeg

Question

I am trying to transcode a 360 Video using ffmpeg (to play on Gear VR, for now), mainly to reduce the bitrate (this is a requirement). But the output file seems to remove the "Side Data" that has the 360 video metadata.

Input file ffprobe result (note the Side Data section):

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
Metadata:
major_brand     : mp42
minor_version   : 0
compatible_brands: mp42mp41
creation_time   : 2016-10-28T10:41:42.000000Z
Duration: 00:02:09.56, start: 0.000000, bitrate: 20116 kb/s
Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 2560x1280, 19797 kb/s, 50 fps, 50 tbr, 50k tbn, 100 tbc (default)
Metadata:
  creation_time   : 2016-10-28T10:41:42.000000Z
  handler_name    : Alias Data Handler
  encoder         : AVC Coding
Side data:
  spherical: equirectangular (0.000000/0.000000/0.000000) 
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 317 kb/s (default)
Metadata:
  creation_time   : 2016-10-28T10:41:42.000000Z
  handler_name    : Alias Data Handler

Commands tried to transcode (values of bitrate and preset are just for testing):

ffmpeg -i <input_file_name> -vf scale=2560x1280,setdar=16:9 -r 30 -c:v libx264 -b:v 500k -pix_fmt yuv420p -preset ultrafast -c:a aac -b:a 192K -map_metadata 0:s:v:0 -write_id3v2 1 -y test.mp4
ffmpeg -i <input_file_name> -vf scale=2560x1280,setdar=16:9 -r 30 -c:v libx264 -b:v 500k -pix_fmt yuv420p -preset ultrafast -c:a aac -b:a 192K -map_metadata 0:s -y test.mp4
ffmpeg -i <input_file_name> -vf scale=2560x1280,setdar=16:9 -r 30 -c:v libx264 -b:v 500k -pix_fmt yuv420p -preset ultrafast -c:a aac -b:a 192K -map_metadata 0 -y test.mp4
ffmpeg -i <input_file_name> -vf scale=2560x1280,setdar=16:9 -r 30 -c:v libx264 -b:v 500k -pix_fmt yuv420p -preset ultrafast -c:a aac -b:a 192K -y test.mp4

In each case the ffprobe test.mp4 seems to show the missing "Side Data" section:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':
Metadata:
major_brand     : isom
minor_version   : 512
compatible_brands: isomiso2avc1mp41
creation_time   : 2016-10-28T10:41:42.000000Z
encoder         : Lavf57.71.100
Duration: 00:02:09.57, start: 0.000000, bitrate: 708 kb/s
Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 2560x1280 [SAR 8:9 DAR 16:9], 507 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
Metadata:
  creation_time   : 2016-10-28T10:41:42.000000Z
  handler_name    : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 193 kb/s (default)
Metadata:
  creation_time   : 2016-10-28T10:41:42.000000Z
  handler_name    : SoundHandler

Any help or advice to get ffmpeg to preserve the Side Data would be appreciated

@Mulvya thanks, I tried it now as both input and output flag but the ffprobe of output is still same (without Side Data section). `ffmpeg -fflags keepside -i .....` and `ffmpeg -i input.mp4 -vf ..... -fflags keepside -y test.mp4` — Icareus, Jun 26 '17 at 14:23
@jeffcook2150 Do you have a short sample input file that you can share? — llogan, Jan 01 '18 at 19:24
@LordNeckbeard see https://drive.google.com/file/d/1OTd5D0inwN-ir_IlMKo-Fnf23C_ux8BZ/view?usp=sharing for a sample of a video that contains equirectangular side data. This is generated from raw Theta V file by RICOH Theta V Movie Converter: https://theta360.com/en/support/download/movieconverter/ . — jeffcook2150, Jan 01 '18 at 19:58
@LordNeckbeard please see https://drive.google.com/drive/folders/1mbSFgFpDQrU52eFYQpUZttrk8jdcq9RB?usp=sharing for a folder of related samples and ffprobe outputs. — jeffcook2150, Jan 01 '18 at 20:09
@jeffcook2150 yes, see answer, it works with the present git version so make sure you're up to date. — aergistal, Jan 08 '18 at 10:12

score 15 · Answer 1 · edited Jun 01 '20 at 17:53

15

In recent ffmpeg versions the spherical packet side-data is supported but to write it in MP4 you have to set the standard compliance mode to unofficial or experimental:

ffmpeg -i R0010309_er.MP4 -c copy -strict unofficial spherical.mp4

ffprobe spherical.mp4 2>&1 | grep -A1 Side
Side data:
  spherical: equirectangular (0.000000/0.000000/0.000000)

The relevant code in libavformat/movenc.c function mov_write_video_tag:

    if (track->mode == MODE_MP4 && mov->fc->strict_std_compliance <= FF_COMPLIANCE_UNOFFICIAL) {
        AVStereo3D* stereo_3d = (AVStereo3D*) av_stream_get_side_data(track->st, AV_PKT_DATA_STEREO3D, NULL);
        AVSphericalMapping* spherical_mapping = (AVSphericalMapping*)av_stream_get_side_data(track->st, AV_PKT_DATA_SPHERICAL, NULL);

        if (stereo_3d)
            mov_write_st3d_tag(pb, stereo_3d);
        if (spherical_mapping)
            mov_write_sv3d_tag(mov->fc, pb, spherical_mapping);
    }

edited Jun 01 '20 at 17:53

Wayne Piekarski

3,203
18
19

answered Jan 08 '18 at 09:57

aergistal

29,947
5
70
92

Thanks! I tried this with a git build too but I was missing the `-strict unofficial`. Looks like it's been supported since ffmpeg 3.3. – jeffcook2150 Jan 09 '18 at 04:12
1

@jeffcook2150 You're welcome. Please note that if you ever remux to another format (like Matroska) you don't need the compliance option. – aergistal Jan 09 '18 at 08:24
I had the very same issue and `-strict unofficial` did the trick. On my side, I was actually trying to rotate the video 180° and thought that it was the issue of my video not being "360". But no, it was *just* the side data not being copied over :) Full command to rotate 180° a 360 video is: `ffmpeg -i input.mp4 -strict unofficial -metadata:s:v rotate="-180" -codec copy output.mp4` – Christophe Deliens Aug 12 '19 at 20:20
What if our tracks are MOV and not MP4 ? ;) I can't get the Side metadata :( – Hugo H Apr 15 '21 at 09:57

Preserving Side Data information for 360 video transcoding using FFmpeg

1 Answers1

Linked