The SKA Data Challenge 2¶
These pages contain the documentation related to the software developed by the HI-FRIENDS team to analyse SKA simulated data to participate in the second SKA Science Data Challenge (SDC2). The SDC2 is a source finding and source characterisation data challenge to find and characterise the neutral hydrogen content of galaxies across a sky area of 20 square degrees.
The HI-FRIENDS solution to the SDC2¶
HI-FRIENDS is a team participating in the SDC2. The team has implemented a scientific workflow for processing the 1TB SKA simulated data cube and produce a catalog with the properties of detected sources. This workflow, the required configuration and deployment files and this documentation are maintained with version control with git
in GitHub to facilitate its re-execution. The software and parameters are optimized for the solution of this challenge, although the workflow can be used to analyse other radio data cubes because the software can deal with cubes from other observatories. The workflow is intended for SKA community members or any astronomer interested in our approach for HI source finding and characterization. This documentation aims at assisting these scientists to understand and re-use the published scientific workflow as well as to verify it.
The HI-FRIENDS Github repository contains the workflow used to find and characterize the HI sources in the data cube of the SKA Data Challenge 2. This is developed by the HI-FRIENDS team. The execution of the workflow was conducted in the SKA Regional Centre Prototype cluster at the IAA-CSIC (Spain).
Workflow general description¶
The details on the approach of our solution is described in the Methodology section. The workflow is managed and executed using snakemake workflow management system. It uses spectral-cube
based on dask
parallelization tool and astropy
suite to divide the large cube in smaller pieces. On each of the subcubes, we execute Sofia-2 for masking the subcubes, find sources and characterize their properties. Finally, the individual catalogs are cleaned, concatenated into a single catalog, and duplicates from the overlapping regions are eliminated. The catalog is filtered based on the physical properties of the sources to exclude some outliers. Some diagnostic plots are produced using Jupyter notebook. Specific details on how the workflow works can be find in the Workflow section. The workflow is general purpose, but the results from the execution on thte SDC2 data cube are summarized in the SDC2 HI-FRIENDS results section.
The HI-FRIENDS team¶
Mohammad Akhlaghi - Instituto de Astrofísica de Canarias
Antonio Alberdi - Instituto de Astrofísica de Andalucía, CSIC
John Cannon - Macalester College
Laura Darriba - Instituto de Astrofísica de Andalucía, CSIC
José Francisco Gómez - Instituto de Astrofísica de Andalucía, CSIC
Julián Garrido - Instituto de Astrofísica de Andalucía, CSIC
Josh Gósza - South African Radio Astronomy Observatory
Diego Herranz - Instituto de Física de Cantabria
Michael G. Jones - The University of Arizona
Peter Kamphuis - Ruhr University Bochum
Dane Kleiner - Italian National Institute for Astrophysics
Isabel Márquez - Instituto de Astrofísica de Andalucía, CSIC
Javier Moldón - Instituto de Astrofísica de Andalucía, CSIC
Mamta Pandey-Pommier - Centre de Recherche Astrophysique de Lyon, Observatoire deLyon
Manuel Parra - Instituto de Astrofísica de Andalucía, CSIC
José Sabater - University of Edinburgh
Susana Sánchez - Instituto de Astrofísica de Andalucía, CSIC
Amidou Sorgho - Instituto de Astrofísica de Andalucía, CSIC
Lourdes Verdes-Montenegro - Instituto de Astrofísica de Andalucía, CSIC