Python and CWL code

Rapthor is mainly written in Python and CWL. The Rapthor code tree is organized as follows:

rapthor-master
├── bin
│   └── rapthor
├── docs
├── examples
├── rapthor
│   ├── lib
│   ├── operations
│   ├── pipeline
│   └── scripts
└── test

In the folder structure above:

• rapthor-master/bin contains the rapthor executable used to run Rapthor (see Running Rapthor).

• rapthor-master/docs contains this Sphinx documentation.

• rapthor-master/examples contains an example parset and strategy file.

• rapthor-master/rapthor contains the main Rapthor Python package and CWL files.

• rapthor-master/rapthor/lib contains the main Rapthor classes and modules (see Python classes and modules).

• rapthor-master/rapthor/operations contains the operation subclasses (see Subclasses of the Operation class).

• rapthor-master/rapthor/pipelines contains the CWL workflow templates (see CWL workflows).

• rapthor-master/rapthor/scripts contains the processing scripts (see Python processing scripts).

• rapthor-master/test contains files used for testing.

Python classes and modules

The following Python classes and modules are the principal ones used in Rapthor. The corresponding Python files are located in the rapthor/lib directory of the code tree.

• The Operation class
  • Operation
  • Subclasses of the Operation class
• The Observation class
  • Observation
• The Field class
  • Field
• The Sector class
  • Sector
• The cluster module
  • get_available_memory()
  • get_fast_solve_intervals()
  • get_slow_solve_intervals()
• The context module
  • RedirectStdStreams
  • Timer
• The miscellaneous module
  • approx_equal()
  • calc_theoretical_noise()
  • convert_mjd2mvt()
  • convert_mvt2mjd()
  • create_directory()
  • dec2ddmmss()
  • delete_directory()
  • download_skymodel()
  • find_unflagged_fraction()
  • get_flagged_solution_fraction()
  • get_reference_station()
  • make_template_image()
  • normalize_dec()
  • normalize_ra()
  • ra2hhmmss()
  • rasterize()
  • read_vertices()
  • remove_soltabs()
  • string2bool()
  • string2list()
• The parset module
  • Parset
  • lexical_cast()
  • parset_read()

Python processing scripts

The Rapthor operations call a number of Python scripts to process the solutions, images, etc. The scripts are located in the rapthor/scripts/ directory of the code tree. For details of each script’s function, see the inline documentation in the script’s code. A description of the inputs can also be obtained by running the script with the -h flag.

CWL workflows

The CWL workflow parsets and step definition files are located in the rapthor/pipeline/ directory of the code tree. Each operation in Rapthor has a corresponding workflow parset. An overview of each operation is given in Operations. For details of each step of the workflows, see the inline documentation in the workflow parset files (in rapthor/pipeline/parsets) and step files (in rapthor/pipeline/steps).

Note

The CWL workflow files in rapthor/pipeline/parsets are jinja2 templates, and so are not directly parsable by CWL tools. Rapthor uses the templates to generate the actual CWL workflows that are passed to the CWL runner. These generated files are created in the Rapthor working directory in workdir/pipelines/operation_name at runtime with the following names: the CWL workflow file is named pipeline_parset.cwl (and subpipeline_parset.cwl when there is a subworkflow) and the workflow inputs JSON file is named pipeline_inputs.json.