Fast Transient Detection as a Prototypical ``Big Data'' Problem
Abstract
The detection of fast (< 1 second) transient signals requires a challenging balance between the need to examine vast quantities of high time-resolution data and the impracticality of storing all the data for later analysis. This is the epitome of a ``big data'' issue-far more data will be produced by next generation-astronomy facilities than can be analyzed, distributed, or archived using traditional methods. JPL is developing technologies to deal with ``big data'' problems from initial data generation through real-time data triage algorithms to large-scale data archiving and mining. Although most current work is focused on the needs of large radio arrays, the technologies involved are widely applicable in other areas.
- Publication:
-
New Horizons in Time Domain Astronomy
- Pub Date:
- April 2012
- DOI:
- 10.1017/S1743921312000993
- Bibcode:
- 2012IAUS..285..340J
- Keywords:
-
- methods: data analysis;
- astronomical data bases: miscellaneous