Radio Astronomy Data Analysis

This use case refers not to a single application but to an architectural component of the Science Data Processor (SDP) of the Square Kilometer Array (SKA). The SDP is planned to be a special-purpose HPC system with a computational power of O(10-100) PFlop/s, which will have to manage the vast amounts of data being generated by the telescopes. It will have to process incoming data at a sufficient high rate to cope with the high data ingest rate.

A key component of the SDP is the Buffer, a distributed storage system that will cope with most of the I/O. The aggregate performance target is 4 TB/s for reading to allow for repeated reading of visibilities for self-calibration major loops. The current design concept does not foresee compute nodes to access the entire Buffer/namespace, but rather a fragmentation in different “Data Islands” is foreseen. A hierarchical approach is foreseen by separating a capacity-optimized “Cold Buffer” (CB) and a performance-optimized “Hot Buffer” (HB).