Tt1 to tt3/mspa 705 smpte (#43)

* Validate bindings for EBU-TT-1 documents by dynamically setting superseding class on document creation

* MSPA-702: Check existence of Styling and Layout elements for EBU-TT-1

* MSPA-702: EBU-TT-1 document must contain a tt:region element

* MSPA-702: body/dur attribute is not allowed in EBU-TT-1

* MSPA-702: Check that smpte timeBase is acceptable in EBU-TT-1

* Validate bindings for EBU-TT-1 documents by dynamically setting superseding class on document creation

* MSPA-702: EBU-TT-1 document must contain a tt:region element

* tt3 to ttd conversion (#41)

MSPA-728 ebu-tt-3 to ebu-tt-d conversion

* rebased and fixed ebuttd test files

* extracting ebutt1object base into generic ebuttdocumentbase class

* Add EBU-TT 1 to EBU-TT 3 conversion

Does not handle time conversion.
Sets `ebuttp:sequenceIdentifier` to the value of `tt/head/metadata/documentMetadata/documentIdentifier` if present, otherwise uses "TestConverter".
Resets the `conformsToStandard` to say it is EBU-TT-3 conformant.
Sets the `timeBase` to `media` whether you like it or not, but doesn't do any other conversions.

* Address review comments

Also allows for a setting that specifies whether or not to use `ebuttm:documentIdentifier` element value in the input as the `ebuttp:sequenceIdentifier` attribute value in the output. Adds a test for this.

* Unit test EBUTT1 to EBUTT3 conversion

* Fix validation error messages for unexpected attributes so it doesn't say they are missing.
* Fix cloning of unknown element, and conversion of metadata
* Make EBUTT1Document instantiatable by including required attributes and elements in the constructor
* Add unit test cases for programmatical construction with smpte (skipped) and media timebase
* Add unit test cases for from-document construction with smpte (skipped) and media timebase

* Timedelta converter

Creates a fixed offset SMPTE timecode converter that can return a timedelta equivalent for any valid SMPTE timecode value equal to or after the provided reference start point. Invalid timecodes throw errors. The timedelta returned is the difference between the input timecode and the reference timecode.

* Reimplement ebu#516

See ebu#516 for further details.

Enables an `ebuttm:documentStartOfProgramme` element to be included in a document without causing a crash.

* Make a smpte timecode converter

Make a FixedOffsetSMPTEtoTimedeltaConverter to convert SMPTE times based on the `ebuttm:documentStartOfProgramme` element if present, or assuming `00:00:00:00` otherwise.

* pep8 tidying, where possible

* test smpte time conversion

* Unskip tests that now pass because they're implemented
* add a test to check that using the `ebuttm:documentStartOfProgramme` as the basis for time conversion works.
* pep8 tidying

* Discard elements that end up with negative times

Unit test for this also added.
Tidy out some print statements that had made their way in there.

* Add some documentation

The docs in current state don't build properly because  autodoc throws a "cannot import name `EBUTT1EBUTT3Converter`" exception. This is possibly a problem with  circular `import` references, which should be resolved before we merge. This can be demonstrated by adding `from . import ebutt1_ebutt3` to `ebu_tt_live/bindings/converters/__init__.py` and running `python -m ebu_tt_live.bindings.converters` which shows a stack trace illustrating the issue.

* Resolve circular import loop

This was preventing documentation from building correctly.

* Prune empty output elements

* Add documentation and fix node code

* Add EBUTT1 to EBUTT3 producer node config parameters

* Add example config file for EBUTT1 to EBUTT3 conversion

Also a manifest file for its input, using existing test XML file.

* Add documentation

* Correctly format code blocks

RST uses double backticks, not like markdown's single ones!

* Allow start of programme timecode to be manually overridden

Sometimes we might know better than the document what start of programe timecode to use, so allow any value specified in the EBU-TT 1 document to be overridden.

* Address @danielthepope's excellent  review comments

* Tidy .gitignore
* Fix up documentation - backticks, mainly
* Fix/improve code comments
* Use snake case for method name instead of camelCase, to match PEP8
* Test that negative times raise errors
* Fix dropped frame algorithm typo - it really should check frame values! Update tests to be more rigorous in checking that too, since they shouldn't have passed before.

.gitignore

@@ -104,6 +104,9 @@ ENV/
 out
 gen
 
+# VSCode project files
+.vscode/
+
 # Oxygen XML project files
 *.xpr
 
@@ -134,4 +137,3 @@ export/
 
 # Mac OS X files
 .DS_Store
-

docs/source/configurator.rst

@@ -103,6 +103,11 @@ Node type dependent options for [nodeN] : ::
    └─clock
      └─type : ["local" (default) | "auto" | "clock"]
 
+   type="ebutt1-ebutt3-producer"
+   ├─sequence_identifier : sequence identifier, default "SequenceFromEBUTT1"
+   ├─use_doc_id_as_sequence_id : whether to use the ebuttm:documentIdentifier element contents as the output sequence identifier if it is present, default False
+   └─smpte_start_of_programme : start of programme timecode override in case you know better than the document start of programme metadata, default None
+
    type="ebuttd-encoder"
    ├─media_time_zero : ["current" (default) | clock time at media time zero TODO: check format]
    ├─default_namespace : ["false" (default) | "true"]

docs/source/conversion_from_ebutt.rst

@@ -23,8 +23,25 @@ have a sequence number. EBU-TT part 3 documents must have both.
 
 In order to set the sequence identifier the converter can be
 configured with the desired value, or it can be set to extract the
-document identifier from the `ebuttm:documentIdentifier` element
+document identifier from the ``ebuttm:documentIdentifier`` element
 and use it, if it exists.
 
-TODO: how to convert smpte timebase time expressions into clock or media
-timebase time expressions.
+If the EBU-TT part 1 document uses the ``smpte`` timebase, then all
+the time expressions must be converted to some other timebase.
+Currently they are all converted to ``media``, using a simple fixed
+offset based conversion strategy, encapsulated in the utility class
+:py:class:`ebu_tt_live.bindings.converters.timedelta_converter.FixedOffsetSMPTEtoTimedeltaConverter`. 
+This currently looks for the metadata attribute 
+``tt/head/metadata/ebuttm:documentMetadata/ebuttm:documentStartOfProgramme``
+and if it finds it, uses that as
+the zero point, otherwise it uses ``00:00:00:00``. This can be
+overridden by setting the ``smpte_start_of_programme`` parameter to the
+start of programme timecode to use instead.
+
+The document's frame rate, frame rate multiplier and drop mode are taken into
+account when doing the conversion. This means that illegal frame
+values will cause a
+:py:class:`ebu_tt_live.errors.TimeFormatError` exception to be raised.
+
+Elements with ``begin`` or ``end`` attributes that fall before the start of
+programme are discarded.

docs/source/ebu_tt_live.bindings.converters.rst

@@ -0,0 +1,34 @@
+converters package
+==================
+
+:mod:`converters` Package
+-------------------------
+
+.. currentmodule:: ebu_tt_live.bindings.converters
+
+:mod:`ebutt3_ebuttd` Module
+---------------------------
+
+.. automodule:: ebu_tt_live.bindings.converters.ebutt3_ebuttd
+    :members:
+    :private-members:
+    :undoc-members:
+    :show-inheritance:
+
+:mod:`ebutt1_ebutt3` Module
+---------------------------
+
+.. automodule:: ebu_tt_live.bindings.converters.ebutt1_ebutt3
+    :members:
+    :private-members:
+    :undoc-members:
+    :show-inheritance:
+
+:mod:`timedelta_converter` Module
+---------------------------------
+
+.. automodule:: ebu_tt_live.bindings.converters.timedelta_converter
+    :members:
+    :private-members:
+    :undoc-members:
+    :show-inheritance:

docs/source/ebu_tt_live.bindings.rst

@@ -79,5 +79,6 @@ Subpackages
 .. toctree::
 
     ebu_tt_live.bindings.raw
+    ebu_tt_live.bindings.converters
     ebu_tt_live.bindings.validation
 

docs/source/ebu_tt_live.documents.rst

@@ -44,15 +44,6 @@ documents Package
     :undoc-members:
     :show-inheritance:
 
-:mod:`bindings_converters` Module
----------------------------------
-
-.. automodule:: ebu_tt_live.bindings.converters.ebutt3_ebuttd
-    :members:
-    :private-members:
-    :undoc-members:
-    :show-inheritance:
-
 :mod:`document_converters` Module
 ---------------------------------
 

docs/source/ebu_tt_live.node.rst

@@ -41,6 +41,14 @@ node Package
     :undoc-members:
     :show-inheritance:
 
+:mod:`ebutt1_ebutt3_producer` Module
+------------------------------------
+
+.. automodule:: ebu_tt_live.node.ebutt1_ebutt3_producer
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
 :mod:`encoder` Module
 ---------------------
 

docs/source/scripts_and_their_functions.rst

@@ -128,6 +128,25 @@ Retiming Delay Node is primarily intended for delaying explicitly timed
 documents. Use ``ebu-run`` to start this script, for example ``ebu-run
 --admin.conf=ebu_tt_live/examples/config/retiming_delay.json.``
 
+EBU-TT-1 Producer
+-----------------
+This script produces an EBU-TT Part 3 document from an EBU-TT Part 1 source.
+If SMPTE timecode is used (``ttp:timeBase="smpte"``) then the script looks for
+an ``ebuttm:documentStartOfProgramme`` element in the input document, and if
+present, maps that to the zero media time, and discards any elements that
+begin or end before that time. If that element is absent, then times are
+converted assuming that media time zero is SMPTE timecode ``00:00:00:00``.
+Alternatively both of those values can be overridden by specifying a
+start of programme timecode to use with the ``smpte_start_of_programme``
+configuration parameter.
+The timecode conversion currently assumes that
+the timecode is continuous.
+
+The default output sequence identifier can be specified. There is also a
+parameter to allow the value of the input ``ebuttm:documentIdentifier`` element
+to be used as the output sequence identifier, if present, overriding the
+specified default.
+
 EBU-TT-D Encoder
 ----------------
 This script is an extension of simple consumer and is responsible for

ebu_tt_live/bindings/_ebuttdt.py

@@ -70,20 +70,25 @@ def _ConvertArguments_vx(cls, args, kw):
             # This means we are in XML parsing context. There should be a timeBase and a timing_attribute_name in the
             # context object.
             time_base = context['timeBase']
-            timing_att_name = context['timing_attribute_name']
-            if time_base not in cls._compatible_timebases[timing_att_name]:
-                log.debug(ERR_SEMANTIC_VALIDATION_TIMING_TYPE.format(
-                    attr_name=timing_att_name,
-                    attr_type=cls,
-                    attr_value=args,
-                    time_base=time_base
-                ))
-                raise pyxb.SimpleTypeValueError(ERR_SEMANTIC_VALIDATION_TIMING_TYPE.format(
-                    attr_name=timing_att_name,
-                    attr_type=cls,
-                    attr_value=args,
-                    time_base=time_base
-                ))
+            # It is possible for a timing type to exist as the value of an element not an attribute,
+            # in which case no timing_attribute_name is in the context; in that case don't attempt
+            # to validate the data against a timebase. At the moment this only affects the
+            # documentStartOfProgramme metadata element.
+            if 'timing_attribute_name' in context:
+                timing_att_name = context['timing_attribute_name']
+                if time_base not in cls._compatible_timebases[timing_att_name]:
+                    log.debug(ERR_SEMANTIC_VALIDATION_TIMING_TYPE.format(
+                        attr_name=timing_att_name,
+                        attr_type=cls,
+                        attr_value=args,
+                        time_base=time_base
+                    ))
+                    raise pyxb.SimpleTypeValueError(ERR_SEMANTIC_VALIDATION_TIMING_TYPE.format(
+                        attr_name=timing_att_name,
+                        attr_type=cls,
+                        attr_value=args,
+                        time_base=time_base
+                    ))
         for item in args:
             if isinstance(item, timedelta):
                 result.append(cls.from_timedelta(item))