No support for advanced codecs (for 8K resolution)

[nickkraakman]

Hi,

According to the YouTube uploading specs for 360º video, the input resolution can be up to 8192x8192.

However, this injector only seems to support MPEG4 video, of which the h.264 codec only supports up to 4k and the h.265 codec is not supported as an upload format on YouTube, or MOVs, which are huge.

The only way to achieve the promised 8192x8192 resolution at the moment is to use a .mov file, which will be multiple gigabytes in size and therefore very unpractical to upload.

I therefore suggest to add support for the VP9 codec, which will drastically reduce file size but still support the 8192x8192 resolution.

Thanks!

Nick

[kodawah]

@Kregus you can store vp9 in mp4 (eg https://github.com/Netflix/vp9-dash/blob/master/VPCodecISOMediaFileFormatBinding.docx)

[nickkraakman]

@kodabb Thanks for this suggestion, but it seems like a strange, not so user-friendly workaround. Also, the resulting file was unplayable for me.

VP9 is Google's very own codec, so it seems more than reasonable for YouTube 360 to support this.

[nbirkbeck]

The first version of the spec has always had support for webm/mkv:
https://github.com/google/spatial-media/blob/master/docs/spherical-video-rfc.md
The metadata injector tool doesn't support webm/mp4. Since the metadata lives in the normal key/value pair location within webm/mkv, any tool that allows you to set a key/value pair metadata on an mkv/webm file should work. You can use mkvmerge/ffmpeg to add the metadata.

Do you have a sample video that isn't working that we can look at?

[nbirkbeck]

You can use ffmpeg to insert the V1 version of the spherical metadata. I just tested on your file with the following command:

ffmpeg -i /tmp/example.webm -metadata:s:v spherical-video='<rdf:SphericalVideo xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:GSpherical="http://ns.google.com/videos/1.0/spherical/"> <GSpherical:Spherical>true</GSpherical:Spherical> <GSpherical:Stitched>true</GSpherical:Stitched> <GSpherical:ProjectionType>equirectangular</GSpherical:ProjectionType> </rdf:SphericalVideo>' -acodec copy -vcodec copy /tmp/example_with_metadata_ffmpeg.mkv

and the result worked. I also tried mkvmerge, but I can't figure out how to get the spherical-video xml embedded in a string tag within mkvmerge's xml-based metadata description.

[nbirkbeck]

webm is a subset of mkv. The reason to use mkv extension is that previously the webm spec didn't support the "tag" mechanism used to carry the v1 metadata. It does now, but ffmpeg hasn't been updated to allow tags to appear in webm.

[nbirkbeck]

Alright, I think you are almost there. The windows shell is behaving differently than bash, and the double quotes are getting lost (you can see in the ffmpeg log that the " is missing). So try either removing those fields:

ffmpeg -i /tmp/example.webm -metadata:s:v spherical-video='<rdf:SphericalVideo> <GSpherical:Spherical>true</GSpherical:Spherical> <GSpherical:Stitched>true</GSpherical:Stitched> <GSpherical:ProjectionType>equirectangular</GSpherical:ProjectionType> </rdf:SphericalVideo>' -acodec copy -vcodec copy /tmp/output.mkv

or better yet, try to see if you get get those double quotes inserted in the output. I think to escape them properly in windows you need to use """ for each single ":

ffmpeg -i /tmp/example.webm -metadata:s:v spherical-video='<rdf:SphericalVideo xmlns:rdf="""http://www.w3.org/1999/02/22-rdf-syntax-ns#""" xmlns:GSpherical="""http://ns.google.com/videos/1.0/spherical/"""> <GSpherical:Spherical>true</GSpherical:Spherical> <GSpherical:Stitched>true</GSpherical:Stitched> <GSpherical:ProjectionType>equirectangular</GSpherical:ProjectionType> </rdf:SphericalVideo>' -acodec copy -vcodec copy /tmp/example_with_metadata_ffmpeg.mkv

[evertvorster]

I have noticed that this script does not do .mkv, or even h265 encoded into .mp4 containers either.
360 degree videos are big, and encoding in hevc saves some disk space.
Thanks for the ffmpeg command, I'll be using it. :)

[kodawah]

@evertvorster in general please avoid using the V1 specification as it is deprecated and replaced by V2. If you can deal with mp4 files only you can check my branch https://github.com/kodabb/spatial-media/tree/sphericaltoolsv2 (PR here: #155)

[nbirkbeck]

As @kodabb points out, the v1 spec is deprecated, so it may be good to check if the link he provided can be used to inject v2 metadata for your use case.

I know ffmpeg has support for passing V2 metadata through, but I'm not sure if there is a way to inject it yet.

Also, there are two other tools to inject v2 metadata using other open source tools into webm/mkv.

1. MKVMerge can also be used. There are downloadable packages available. https://mkvtoolnix.download/doc/mkvmerge.html
   The options you will need are "--projection-type=1" "--stereo-mode=2". I haven't tried yet, but something like the following should work:
   mkvmerge -o output.webm --webm --projection-type=1 --stereo-mode=2 input.mkv

2. An alternative approach is available via libwebm (but you would need to compile the source code).
   https://github.com/webmproject/libwebm/blob/master/mkvmuxer_sample.cc#L64
   For example, if you are able to build that application, the arguments of "-projection_type 1 -stereo_mode 2 " should do what you want (but that code will only transmux from webm to webm).

Finally, If you still must use V1, there is a StereoMode tag:
https://github.com/google/spatial-media/blob/master/docs/spherical-video-rfc.md#StereoMode
So adding the stereo-mode tag to the xml should work:
GSpherical:StereoModetop-bottom</GSpherical:StereoMode>

[Boscop]

This works for mkv:

mkvmerge -o vid_360.mkv --projection-type 0:1 vid.mkv

This sets it to 1 for the stream 0.

But is there a way that doesn't require knowing which stream ID the video has? (If I just omit the 0: it complains.)

And is there a way to inject it into an existing file without creating a new file? Because the file is usually huge (several GBs) and I would delete the non-360 file right afterwards anyway.

@nbirkbeck Is there currently a way to inject the metadata for mp4 or mkv using ffmpeg?

I tried this but it's not working:

ffmpeg -i vid.mkv -metadata:s:v spherical-video='<rdf:SphericalVideo xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:GSpherical="http://ns.google.com/videos/1.0/spherical/">GSpherical:Sphericaltrue</GSpherical:Spherical>GSpherical:Stitchedtrue</GSpherical:Stitched>GSpherical:StitchingSoftwareSpherical Metadata Tool</GSpherical:StitchingSoftware>GSpherical:ProjectionTypeequirectangular</GSpherical:ProjectionType></rdf:SphericalVideo>' -c copy vid_360.mkv

I also tried it with a mp4 file, also not working.

@evertvorster What's the current solution for injecting the metadata into h265 mp4 files?

I also tried this:

ffmpeg -i vid.mp4 -c:v copy -bsf:v 'h264_metadata=sei_user_data=ffcc8263-f855-4a93-8814-587a02521fdd+<rdf:SphericalVideo xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:GSpherical="http://ns.google.com/videos/1.0/spherical/">GSpherical:Sphericaltrue</GSpherical:Spherical>GSpherical:Stitchedtrue</GSpherical:Stitched>GSpherical:StitchingSoftwareSpherical Metadata Tool</GSpherical:StitchingSoftware>GSpherical:ProjectionTypeequirectangular</GSpherical:ProjectionType></rdf:SphericalVideo>' vid_360.mp4

(Using this uuid.)

But it gives an error:

[h264_metadata_bsf @ 000000d8043c1d80] No option name near 'SphericalVideo xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:GSpherical="http://ns.google.com/videos/1.0/spherical/">GSpherical:Sphericaltrue</GSpherical:Spherical>GSpherical:Stitchedtrue</GSpherical:Stitched>GSpherical:StitchingSoftwareSpherical Metadata Tool</GSpherical:StitchingSoftware>GSpherical:ProjectionTypeequirectangular</GSpherical:ProjectionType></rdf:SphericalVideo>'
Error parsing options for bitstream filter h264_metadata