Input Validation on the Model

[andrewferguson]

Why Input Validation is Needed

Our approach up until now has been to not do too much input validation and checking in the individual chain scripts themselves, on the assumption that breaking APIs will cause crashes that will lead to manual intervention. While this is true for many cases, there are several cases where an API change could lead to invalid data being silently added to the archive, which is a serious issue.

An example of when this could occur is in the Everyman script, where the film year is determined by the following line:

film_year = film_data['ReleaseDate'][:4]

The problem is if the ReleaseDate API changes its format, the film year will silently start being a random string, which will not raise an error.

How to fix this

We need input validation on the model. For example, checking that the year falls within a known likely range of years (1900 - current).

More complex validation

For some of the model attributes, input validation will be more difficult. For example:

• If a showing occurs in a screen that has never been seen before.
• If a chain starts adding in lots of films that have never appeared in the database before (unless it is an arthouse chain).
• If a film title exists in the database but under a different year.

The above anomalies should probably not cause a crash, since they will occur fairly frequently (particularly as we expand the number of chains supported), but they should be recorded and reported (e.g: as a daily email of incidents), so that they can be manually reviewed.

Validation Methods

For the simple validation method, adding checks to the code is one approach. However, the more complex validation will require inspecting the DB, and the time that entries were added to the DB.

This is probably much better served by a script that runs every hour and checks the DB for anomalies. The script would be ideally placed to send the daily reports.