Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add video/multimedia support #31

Merged
merged 15 commits into from
Feb 16, 2024
Merged

Conversation

awehrfritz
Copy link
Contributor

This PR adds support for multimedia, and in particular video files.

The implementation follows the approach in LibreOffice for linked and embedded files, as outlined in typst/typst#955.

Specifically, the implementation uses the "Multimedia Features" specified in Section 13.2 of the PDF 1.7 specification. This in turn is based on the "Screen Annotations" described in Section 12.5.6.18, and the "Rendition Actions" described in Section 12.6.4.13 to play the multimedia content, and subsequently the "Media Renditions" (13.2.3.2) and Media Clip Objects (13.2.4).

I only implemented the features required to get the most common use case working, but this can be easily extended if more advanced features are needed.

The video.rs example demonstrates this functionality using the bear video obtained from the Chromium source tree. The example has a flag to create PDFs with embedded (default) or linked videos. This also includes a rudimentary implementation of appearance streams, which are used to provide the visual appearance for the screen annotation when the media is not playing using an image (i.e. the first frame of the video manually extracted using ffmpeg).

This implements the prerequisites for typst/typst#955.

@awehrfritz
Copy link
Contributor Author

awehrfritz commented Feb 12, 2024

I just noticed that the draft PR #25 has an implementation of Appearance Streams already. I have adopted that implementation here now.

@reknih or @laurmaedje, any comments on this implementation would be much appreciated.

Copy link
Member

@reknih reknih left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for using the implementation from #25 for annotation appearance streams. I will set @tingerrr as a co-author.

Please also add that the JS field is allowed for Rendition actions too in the Action::js_stream doc comment.

I also think that users would like some guidance on which codecs and container formats are accepted by which users in the doc-comment and which viewers may not support video at all.

src/actions.rs Outdated Show resolved Hide resolved
src/annotations.rs Show resolved Hide resolved
src/renditions.rs Show resolved Hide resolved
src/renditions.rs Show resolved Hide resolved
src/renditions.rs Outdated Show resolved Hide resolved
src/renditions.rs Outdated Show resolved Hide resolved
src/renditions.rs Show resolved Hide resolved
src/renditions.rs Outdated Show resolved Hide resolved
@tingerrr
Copy link
Contributor

tingerrr commented Feb 12, 2024

I hope my PR being late due to me being busy with uni wasn't too much of a hassle with porting the Appearance Streams impl over! Feel free to change stuff if it makes more sense, and I'll back port it to my PR or simply rebase on top of your changes once I've got time to work on it again.

EDIT: P.S. My handle is tingerrr with three r's, you've mentioned a dead account before.

Copy link
Member

@reknih reknih left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three other remarks on naming and impls after having discussed this with Laurenz

src/renditions.rs Outdated Show resolved Hide resolved
src/renditions.rs Outdated Show resolved Hide resolved
src/renditions.rs Outdated Show resolved Hide resolved
@awehrfritz
Copy link
Contributor Author

awehrfritz commented Feb 15, 2024

I also think that users would like some guidance on which codecs and container formats are accepted by which users in the doc-comment and which viewers may not support video at all.

@reknih which doc-comment did you have in mind specifically? Here some preliminary text, which is the best I can do in this regard at the moment:

PDF reader support for playing video and multimedia formats
===========================================================

Adobe Acrobat
-------------
The Adobe Acrobat documentation [1] lists the following types of videos
supported in PDFs created using Adobe Acrobat or Adobe Acrobat Reader:
  MP3, MP4, MOV, M4V, 3GP, and 3G2 video files that use H.264 compression.

You must have H.264 codecs installed on your computer to play files that use
H.264 compression.


KDE Okular
----------
The KDE Okular document viewer supports a broad range of audio and video files
depending on the respective Phonon backend (e.g. GStreamer or VLC).


[1] https://helpx.adobe.com/acrobat/using/playing-video-audio-multimedia-formats.html#supported_video_audio_and_interactive_formats

Otherwise, I believe I have addressed all comments and this is ready from my side.

@reknih reknih merged commit dd53e58 into typst:main Feb 16, 2024
1 check passed
@reknih
Copy link
Member

reknih commented Feb 16, 2024

I added a version of the note you wrote and ran cargo fmt. Thank you for your contribution!

@awehrfritz
Copy link
Contributor Author

Great, thanks a lot @reknih for your swift response and review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants