|
|
|
Large Scale on Demand Cross-Matching with Open SkyQuery
|
|
|
| Summary
This proposal seeks to develop an advanced and versatile all-sky cross-matching engine for large distributed astronomical datasets by bringing together and leveraging the results of two previously funded AISR projects. We wish to extend the present SkyNode and Open SkyQuery functionalities in the following ways:
1. Incorporate parallel data access with Zone partitioning into individual SkyNodes
2. Transport data between SkyNodes and the portal using the Mega Streaming transport protocol developed by Grossman et al.
3. Replace the current synchronous query execution with an asynchronous system that is compatible with emerging IVOA standards.
We will apply the Zone-based partitioning and parallelism that we developed with prior AISR funding to individual SkyNodes in Open SkyQuery (also developed with AISR funding) so that we can parallelize the data access and the cross-match computation by distributing the data and workload among a cluster of database servers.
In order to move the cross-match data quickly between SkyNodes, we will incorporate the mega-streaming network transport protocol for fast data movement developed at NCDM/UIC.
Asynchronous workflow is the third main enhancement that we will make. The size and complexity of the large scale cross-matches will make synchronous response back to the Web client impracticable and inefficient. Asynchronous query execution will require identification of users, tying into the IVOA single sign-on (SSO) protocol. The large outputs resulting from large cross-matches will require direct routing to storage destinations like VOStores.
In addition to producing a large-area cross-matching facility that will allow the NVO to truly federate large distributed archives, this project will also provide high-speed data access to all large datasets supported by the NVO and NASA. It will enable unprecedented science that can be used to constrain cosmological n-point correlation function and power spectrum calculations, determine the AGN luminosity function, find galaxy clusters, etc. These studies will increase the science return from current NASA missions and promote the NASA strategic objective of exploring the universe to understand its origin, structure, evolution, and destiny. A facility that allows for extensive correlation with other catalogs will be extremely useful for near-earth and moving object searches, and for TPF searches related to the NASA strategic objective of searching for Earth-like planets. |
|
|
