Mock-up example of including captions (WebVTT) in Presentation

[jronallo]

Description

Institutions already have caption files available to make their videos accessible. It would be nice to allow implementers to have a smooth on ramp to using IIIF video by allowing them to publish the availability of a caption file within a Presentation manifest.

A client which knows to look for a WebVTT file could then do a number of things, most importantly display the captions to a user.

Proposed Solutions

Though there are other caption formats, start with an example of WebVTT as browsers already understand how to display them with most features.

@azaroth42 and @tomcrane did some initial thinking on what this might look like.

Search inside a video would still require annotations be returned.

[azaroth42]

The key thing here is that the WebVTT is as outside our specs as METS+ALTO or TEI or CIDOC-CRM are. It's not a substitute for the Presentation API modeling, it's a fallback for systems that recognize the content and can do something with it.

Also, it would have to be part of the info.json, not the Presentation API. The distinction with the above specifications is that they're about the object, not the representation. WebVTT is about the video, which could be composited in to a larger canvas. The alternative is to have a single canvas per video, and annotate those canvases into the larger canvas - what has previously been called Zones - but that requires engagement with developers early to get them thinking and building in this direction.

[zimeon]

What other caption formats are there? Or is WebVTT the only game in town?

[jronallo]

There are others. The two W3C efforts are WebVTT and TTML (https://www.w3.org/TR/ttml2/). There are others like SubRip (usually an .srt extension https://en.wikipedia.org/wiki/SubRip) which was widely used and inspiration for WebVTT. There are others I'm less familiar with.