Pipeline Improvement Tasks

This is a list of "bite-sized" work can we do on the pipeline that will:

• make it easier to move the code to NERSC
• make a future large-scale pipeline refactor easier
• can be implemented in the current pipeline

Tasks

CVS -> git

Separate some of the CVS code for the pipeline into git repositories. This necessitates updating the installation scripts: the existing build and install scripts sometimes assume various components (e.g., pysnifs and toolbox) live side-by-side, whereas these would become separate git repositories.

The idea would be to move to using git for the "official" development versions: the stuff that gets run in the pipeline.

preprocess & ifuio cleanup

These software have been transferred from CVS to git, but not thoroughly tested.

NAG dependency

Remove it.

cubefit plan (?)

(is it actually a problem still?)

tiger format (?)

Remove hard-coded paths [All over]

Change to using standard environment variables

Clean up try ... except statements [all over]

Change to catch only expected exceptions. Possibly breaking.

Change PBS-specific scripts

First check how much there is that is PBS specific. Make SLURM compatible version.