Astro-DISC: Astronomy and Cosmology Applications of Data-Intensive Supercomputing

We are developing new algorithms, data structures, and software tools for the analysis of massive astronomical and cosmological datasets. The goal is to enable the efficient processing of massive astrophysics datasets, with sizes in the tens of terabytes, by scaling up standard algorithms and designing new ones. We are developing programming abstractions and tools to address these challenges and to extend the developed techniques to other fields of science.

RESEARCH AREAS

• Distributed algorithms and data structures
• Indexing and analysis of massive structured data
  
  

CHALLENGES

• Developing distributed versions of standard astronomy applications
• Creating efficient algorithms for new astronomical challenges, in collaboration with astronomers and cosmologists
• Building a general-purpose toolkit for the analysis of massive astronomical observations and cosmological simulations
  

More details: Slide show illustrating motivation and challenges.

SUB-PROJECTS

• DISC-Distances: Analyzing the distribution of distances between galaxies
• DISC-Finder: Identifying clusters of astronomical objects
• DISC-Holes: Indexing black-hole simulations
• DISC-Index: Indexing cosmological simulations
• DISC-Quasars: Identifying distant quasars in sky surveys
• DISC-SkyMap: Indexing astronomical objects
  
  

Acknowledgements

Made possible by seed funding from
The Gordon and Betty Moore Foundation.

We thank the members and companies of the PDL Consortium: Amazon, Bloomberg, Datadog, Google, Honda, Intel Corporation, IBM, Jane Street, Meta, Microsoft Research, Oracle Corporation, Pure Storage, Salesforce, Samsung Semiconductor Inc., Two Sigma, and Western Digital for their interest, insights, feedback, and support.