State of Code Before Refactoring

There are 2 databases used by the transient classification server (TCS):

1. Transient DB. This database contains all of the sources discovered by a given transient survey, alongside the classification predicted by the TCS and the catalogued sources matched against the transient sources.
2. Static Cat DB. A database of billions of static sources to crossmatch transient sources against. Catalogues include SDSS, GSC, 2MASS, NED, AGN catalogues etc.

Before refactoring the code in July 2015, the code for the transient server was executed from a main genericPostIngestCrossmatch.py script. Here's the workflow diagram for that code:

The algorithm imported and used by genericPostIngestCrossmatch was a separate python class (classifierAlgorithmFactory.py) with the algorithm buried in the code:

Running the Code

Lets assume I have the code sitting in /Users/Dave/Desktop/ps1

Pythonpath

Setup your pythonpath first:

setenv PYTHONPATH "${PYTHONPATH}:/Users/Dave/Desktop/ps1/code/web/ps1/psdb:/Users/Dave/Desktop/ps1/code/database/reports/python"

SSH Tunnel

ssh -fnN [email protected] -L 9306:psdb2:3306

To test the tunnel, try connecting to the mysql database:

mysql -u dry -h127.0.0.1 -P9306 panstarrs1

Databases

There are two database connections in the code:

1. To the static catalogues (panstarrs1 database)
2. to the transient database (ps13pi database)

At the top of the genericPostIngestCrossmatch.py code you will find:

# Hard wired remote catalogue access.  All the catalogues live on PSDB2 and are
# currently too large to simply copy across to a local machine.

CATUSER = 'kws'
CATPASS = ''
CATNAME = 'panstarrs1'
CATHOST = '127.0.0.1'

These are the credientials for the panstarrs1 database.

In a separate yaml file I can then place the credentials for the transient database like so:

databases:
    local:
        username: "dry"
        password: ""
        database: "ps13pi"
        hostname: "127.0.0.1"

Execute the code

This will run the code on all the transients in the eyeball list:

cd /Users/Dave/Desktop/ps1/code/database/reports/python
python genericPostIngestCrossmatch.py /path/to/settings.yaml

and this will run the code on a listed transient:

python genericPostIngestCrossmatch.py /path/to/settings.yaml 1130918580255220000