3D-HST Data Extraction API

Table of Contents

  1. Introduction
  2. Object Search
  3. Cone Search
  4. Near Object Search
  5. Coverage Check (Object ID)
  6. Coverage Check (Position)
  7. Code Integration Examples
  8. Error Handling
  9. Contacts
  10. References
  11. Note to Users

1. Introduction

In extragalactic astronomy, a tremendous amount of observing time and man power have been dedicated to a small number of 'special' areas in the sky. It was not long that the benefits of having the multiple datasets for the same areas realised. Since then efforts have concentrated to five areas, comprising about 900arcmins^2 : AEGIS, COSMOS, GOODS-N, GOODS-S, and the UDS.

A request can be made to the following URL. Using the API can be learnt via the sample query. The URL for the API is http://narwhal.astro.yale.edu:8080/ from this URL we go to the API endpoints which are as follows:

http://narwhal.astro.yale.edu:8080/object?id=aegis.12387.v41
http://narwhal.astro.yale.edu:8080/conesearch?ra=150.09&dec=2.359&radius=0.05&unit=d
http://narwhal.astro.yale.edu:8080/nearest?ra=150.09&dec=2.359&unit=d
http://narwhal.astro.yale.edu:8080/pointing?ra=150.09&dec=2.359&unit=d
http://narwhal.astro.yale.edu:8080/pointing_id?id=cosmos.16782.v41

2. Object Search

This search is based to find the details of a object in the 3D-HST Database by providing in the 3d_hst_id of the object. The database had various paramenters for a individual objects in the observations including spectrum, redshifts, magnitude, photometry etc. As of now a search on the objects return the RA (degrees), DEC , and Database ID of the object. The calls can be made with the following syntax: url="http://narwhal.astro.yale.edu:8080/object?id=cosmos.17519.v41" for a single object search. this will result a JSON object with some predefined properties.

The first thing to check for in the returned object would be to check for the error field TDHSTAPIError. This field if present indicates a error in the input or with the search on the input. The details of the error can be checked using the Error Handling.

There is a meta field in the returned JSON object which shows the miscellaneous input details.

The results are contained in a results field in the reutrned JSON object. The returned object looks as below:

"results": [
                {
                    "dec": 2.35838413, 
                    "id": 78837, 
                    "3dhst_id": "cosmos.17519.v41", 
                    "ra": 150.15362549
                }
            ]

The above JSON can easily parsed by dumping the returned object after error checks and using it as a Python list with four parameters in each list object. Since this query returns only one object we can straight forward get to results instead of iterating over the list.

for itr in returned_results['results']:
    print( itr['ra'] , itr['dec'] , itr['3dhst_id'] )

3. Cone Search

This search endpoint results in outputting all the objects in the given radial area around the input point. The input includes ra in degrees or hourangle, dec and radius , in degree for API call and arcmins for data extractor input, of the searched area. The query returns the 3dhst_id , ra , dec and id in the DB of the objects qualifying to be in the area around the input point. The input looks like this for a sample query:

url="http://narwhal.astro.yale.edu:8080/conesearch?&ra=150&dec=2.36&radius=0.053"

The result of the above query will result in a output like this:

{
    "results": [{
        "dec": 2.36421847,
        "id": 79475,
        "3dhst_id": "cosmos.18157.v41",
        "ra": 150.05285645
    }, {
        "dec": 2.3663156,
        "id": 79694,
        "3dhst_id": "cosmos.18376.v41",
        "ra": 150.05183411
    },
    ...
    ...
    ...
    {
        "dec": 2.37022376,
        "id": 80121,
        "3dhst_id": "cosmos.18803.v41",
        "ra": 150.05148315
    }],
        "meta": [{
        "ObjectsReturned": 5
    }, {
        "dec": 2.36,
        "radius": 0.053,
        "ra": 150.0
    }]
}

The above resulting JSON has two fields results and meta. The results field cna be iterated over as a python list and results referenced from each iterator object by JSON id as below:

for itr in returned_results['results']:
    print( itr['ra'] , itr['dec'] , itr['3dhst_id'] )

The error handling for this may be refered to using the Error Handling.

4. Near Object Search

The search endpoint is to determine the closest neighbour to the coordinates given as input. It uses Q3C to find the radial distance between two sets of points. Then we sort the results and return the closest object to the input position. This endpoint takes in two mandatory inputs ra dec in either degrees or hourangle and and optional parameter unit to specifiy units of the ra as either d for degree or h for hourangle. The results object is in JSON and contains four fields 3dhst_id, ra, dec and distance from the input point. The units for ra are degrees. There will only be one result per query for this search.

There is also a metafield to the response containing the input details other relevant response codes and fields. More detials will be added to the reponse code as needed.

Make sure to check for the Error fields before proceeding to process the result JSON object using the Error Handling.

5. Coverage Check (Object ID)

This endpoint is to find where a particular object lies in the survey viewing tile. A survey tile is the region in the survey region where the particular object is located and gives a fair idea of the region and object, since a small area contains a lot of objects. The input for the search is the 3dhst_id of the object.

The search returns three properties candels, acs and wfc3 which have seperate meaning. candels property has boolean value true or false meaning is the object in the CANDELS survey or not. The acs property returns a list with all the tiles in which the given object is contained when observed in the ACS field of Hubble Space Telescope. Similarly, wfc3 property returns a list that lists the WFC3 fields of observations and tiles in the fields. These fields (acs and wfc3) may be empty if no tile is found corrosponding to a particular object. Error handling is same as before, Error Handling.

6. Coverage Check (Position)

This endpoint is to find where a particular position lies in the survey viewing tile. A survey tile is the region in the survey region where the particular object is located and gives a fair idea of the region and object, since a small area contains a lot of objects. The input is the ra (in degrees or hourangle), dec compulsory and unit as optional parameter to specifiy units of the ra as either d for degree or h for hourangle.

The search returns three properties candels, acs and wfc3 which have seperate meaning. candels property has boolean value true or false meaning is the object in the CANDELS survey or not. The acs property returns a list with all the tiles in which the given object is contained when observed in the ACS field of Hubble Space Telescope. Similarly, wfc3 property returns a list that lists the WFC3 fields of observations and tiles in the fields. These fields (acs and wfc3) may be empty if no tile is found corrosponding to a particular object. Error handling is same as before, Error Handling.

7. Code Integration Examples

To use this API in your python codes the following examples may be refereed to. This will show how to make some basic calls to the API and parse the results returned. Below is the example to use the Object search. 

import json
import urllib.request

# URL of the API
URL_HTTP = "http://narwhal.astro.yale.edu:8080/" 

# Name of the search
SEARCH = "object?id="

# ID of the obejct to be searched
OBJECT_ID = "cosmos.17423.v41"

# Final Query to be made to the API
QUERY = URL_HTTP+SEARCH+OBJECT_ID

# Getting the response using the urllib library
response = urllib.request.urlopen(QUERY)

# parsing the responses in json and storing in result_search 
result_search = json.loads(response.read().decode('utf-8'))

# Now iterating over the results
# Printing the ra and dec value of the queried object
for itr in result_search['results']:
    print(itr['ra'])
    print(itr['dec'])

Similarly we can write codes for using other searches too. Codes to use other searches will be updates soon.

8. Error Handling

The error handling is done via the TDHSTAPIError property in the returned JSON object. It would be a highly recommended to check for this property for errors before using the results. The error fields look something like this :

{
    "TDHSTAPIError": {
        "type": "No Items Found",
        "detail": "The input led to no results."
    }
}

The TDHSTAPIError has two fields type and detail. The type has a fixed set of responses which are described later in this section. detail is more general and may vary from error to error and has a deeper explanation of the error occuring.
The type field has these properties:

    "Invalid Parameter Combination"
    "Invalid Units"
    "No Items Found"
    "Invalid Parameter Value"
    "Invalid Parameter Type"
    "Required Parameter Missing"
    "Invalid Characters Detected"
    "Functionality Not Yet Implemented"
    "Server Configuration Error"

Each of these errors come with custom error detail fields. The example above it has "The input led to no results." telling the user that the input resulted in no result from the Database.

9. Contacts

For any issues and doubts please contact the following:

10. References

11. Note to Users

We are still in development phase and looking for bugs, user inputs to add more functionality and improve upon the current platform to make more use of the resources and the invaluable data. Thanks.