Skip to content

showrunner.sh

A Bash-based pipeline taking in sub-band Stokes images and generates various spectrapolarimetric analysis and visualisation data products.

This script is availed by initialising Scrappy in an output directory of choice as:

scrappy -i

There's also a -f option in case of the need to overwrite the initial files. Running the above command avails the following files:

  • env-vars : containing pipeline-specific environment variables to be modified.
  • bk_plots.yml : a configuration file for sc-bokehplot.
  • showrunner.sh : the pipeline script.

Showrunner must be accorded execution permissions in order to run. This can be done by:

chmod +x showrunner.sh

There are three options for running showrunner.sh:

  1. ./showrunner.sh -h: to get the help menu.
  2. ./showrunner.sh -run: to run the entire pipeline.

The third option is to run the individual functions in the pipeline by themselves using the syntax 

./showrunner.sh functionName

For example to run the 'makeDirs' function:

./showrunner.sh makeDirs

Setup

This pipeline runs within the Linux shell environment. It has been tested using:

  • Ubuntu 22.04
  • Python 3.10
  • Bash 5.1.16

Software

Disparate software -- both native to Linux and external Python software -- are used within this pipeline to achieve its goals. The following table summarises the software used (including scrappy).

Software Tool Description
montagePy fitsheader Read FITS headers
owlcat fitstool.py Creating Stokes cubes
spimple spimple-imconv Image convolution to the same beam
spimple-spifit Spectral index fitting
scrappy sc-beam-plot Channel auto-selection and beam dimension plotting
sc-rmmap Generate per-pixel RM maps among other things
sc-los Generate independent LoS
sc-los-rm Perform RM-synthesis on LoS from scrappy
sc-bokehplots Generate interactive plots for the LoS
sc-houdini Generate simple image masks given input images
Native Linux cp Copy files and folders
ls Listing files and folders
sed For some text processing
mv Moving and renaming files
rm Deleting files and folders
rename(.ul) Batch file renaming

Output Directory Structure

After a complete run, this pipeline results in an output directory structure similar to the one below.

├── masks                           # Location of the generated FITS masks
├── products
│   ├── conv-selection-cubes        # Location of the selected sub-band images' cubes convolved to the same resolution
│   ├── original-cubes              # Location of the full band images' cubes
│   ├── selection-cubes             # Location of selected images' cubes
|   ├── scrap-outputs               # Outputs from Los data generators
│   │   ├── bokeh-plots             # Interactive plots generated by sc-bokeh-plots from sc-los-rm
│   │   ├── los-data                # LoS data outputs generated by sc-los
│   │   ├── los-rm-data             # LoS data outputs generated by sc-los-rm
│   │   ├── los-rm-data-plots       # Static plots generated by sc-los-rm
│   │   └── regions                 # Location of region files autoselected by sc-los
│   └── spi-fitting                 # Location of the spectral index fitter outputs
├── relevant-images                 # Location of the selected images
└── relevant-images-conv            # Location of the singular sub-band images convolved to the same resolution

Pipeline flow illustration

graph TD s0((Launch showrunner.sh)) --> s1 subgraph Required steps s1(Install python dependencies) --> s2(Initialise directories) end s2 --> s2b(Stack all sub-band images) s2b --> s2c[/All sub-bands cube/] s2 --> s3(Auto-select good sub-bands) s3 -- Selected images --> s3a(Stack selected sub-band images) s3a -->s3b[/Selected sub-bands cube/] s3 -- Selected images --> s4{Convolve before stacking?} s4 -- Yes --> s5a(Convolve individual sub-band images) s5a --> s5ao[/Convolved sub-band images/] s5a --> s5a1(Make image cube) s4 -- No --> s5b(Make image cube) s5b --> s5b1(Convolve image cube) s5a1 --> s5o[/Convolved image cube/] s5b1 --> s5o s3 --> s6(Make Source mask) s6 -- source mask --> s7(Make RM map and other maps) s7 -- Fpol map --> s8(Make Fpol mask) s8 -- Fpol mask + (convolved individual or cube) image --> s9(Generate LoS data) s9 -- LoS data --> s10(RM-synthesis on Los Data) s10 --RM-synth Los Data --> s11(Generate interactive plots) s11 --> s12(Generate depolarization maps) subgraph optional steps s12 --> s13(Generate spectral index maps) end s13 --> s14((End))