Saving to Local Files

Now that we have seen some of ATK’s data structures, we can look at the package’s local storage capabilities.

All ATK data structures support local file saving. Returning to our DataStruct example from tutorial two, we can save this to a local file using the savedata() method:

from AstroToolkit.Tools import query

gaia_data = query(kind="data",source=587316166180416640,survey="galex")
gaia_data.showdata()
gaia_data.savedata("test_data.fits")

.kind:       lightcurve
.survey:     ztf
.source:     6050296829033196032
.pos:        [245.44701332846378, -22.886218247040002]
.identifier: J162147.28-225310.39
.figure:     None
.dataname:   J162147.28-225310.39_6050296829033196032_ztf_ATKlightcurve.fits
.plotname:   J162147.28-225310.39_6050296829033196032_ztf_ATKlightcurve.html
.trace:      start -> extracted pos from source query, assumed [2016, 0] -> ztf: [2019, 0] -> ztf query performed -> [2000,0] -> end

.data:
    RAJ2000:  [141.185551]
    DEJ2000:  [8.031037]
    Name:     ['GALEX J092444.5+080151']
    objid:    [6377741628902215075]
    FUVmag:   [19.6878]
    e_FUVmag: [0.113]
    NUVmag:   [19.5523]
    e_NUVmag: [0.0704]
    ...

Available Methods: .savedata(), .showdata()

This generates a .fits file which contains the data structure. We have overriden the file name here, otherwise it would default to the name stored in the dataname attribute. A key feature of file saving in ATK is that it is entirely lossless - i.e. the original data structure can be recreated exactly. This is done by reading the file with the readdata() tool:

from AstroToolkit.Tools import readdata

data=readdata("test_data.fits")
data.showdata()

.kind:       lightcurve
.survey:     ztf
.source:     6050296829033196032
.pos:        [245.44701332846378, -22.886218247040002]
.identifier: J162147.28-225310.39
.figure:     None
.dataname:   J162147.28-225310.39_6050296829033196032_ztf_ATKlightcurve.fits
.plotname:   J162147.28-225310.39_6050296829033196032_ztf_ATKlightcurve.html
.trace:      start -> extracted pos from source query, assumed [2016, 0] -> ztf: [2019, 0] -> ztf query performed -> [2000,0] -> end

.data:
    RAJ2000:  [141.185551]
    DEJ2000:  [8.031037]
    Name:     ['GALEX J092444.5+080151']
    objid:    [6377741628902215075]
    FUVmag:   [19.6878]
    e_FUVmag: [0.113]
    NUVmag:   [19.5523]
    e_NUVmag: [0.0704]
    ...

Available Methods: .savedata(), .showdata()

The above is equally applicable to any ATK data structure returned by query(), although figures must be saved separately if you wish to avoid having to re-plot the data structure.

The query() tool also provides a check_exists flag, which searches for an existing file and recreates the data from this if possible. This can save a significant amount of time when running scripts multiple times. If the file is not found the query will run as normal, and the returned data will be saved to the provided file name for next time.

lightcurve_data = query(kind="lightcurve",source=6050296829033196032,survey="ztf",check_exists="test_lightcurve.fits")

The first time this is run, a query will be performed since the file does not currently exist. The resulting light curve will be saved to local files under the filename “test_lightcurve.fits”. Any future executions of this script will then recreate the data structure from local files instead of running a new query.

Note: The code used in this tutorial can be executed using:

from AstroToolkit.Examples import local_files