.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_tutorials/extension/external_data.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_tutorials_extension_external_data.py: ####################### Importing External Data ####################### Generating a Dataset ==================== ATK also supports the use of external data. The first step is to create an empty :class:`~ATK.Models.DataSet`. This is done with :meth:`~ATK.Models.DataSet.from_target`, which requires any valid target for initialisation (see :doc:`here <../getting_started/data_query>`). For example, using a Gaia Source ID: .. GENERATED FROM PYTHON SOURCE LINES 9-17 .. code-block:: Python from ATK.Models import DataSet target = 4223502720986764672 dataset = DataSet.from_target(target) .. GENERATED FROM PYTHON SOURCE LINES 27-48 | Importing Data -------------- The next step is to generate a data container to add to the :class:`~ATK.Models.DataSet`. For this tutorial, an externally-acquired spectrum will be imported into ATK. The easiest way to do this is to generate a :class:`~ATK.Models.Spectrum` using one of its **IO methods**: :meth:`~ATK.Models.Spectrum.from_table` and :meth:`~ATK.Models.Spectrum.from_dataframe`. **In this tutorial,** :meth:`~ATK.Models.Spectrum.from_dataframe` **will be used.** The :class:`~pandas.DataFrame` (or :class:`~astropy.table.Table` if using :meth:`~ATK.Models.Spectrum.from_table`) must contain a column with the same name as each array-like attribute of the required data container. To generate a :class:`~ATK.Models.Spectrum`, the :class:`~pandas.DataFrame` must therefore have a ``wavelength`` column and a ``flux`` column. | .. note:: When generating data containers with :meth:`~ATK.base.DataFrameIOMixin.from_dataframe`, all columns are assumed to be in specific units - details for which can be found in the container's documentation (e.g. :class:`here ` for spectra). While units could be manually changed after initialisation, it may instead be preferable to work with a :class:`~astropy.table.Table`, as :meth:`~ATK.Models.Spectrum.from_table` preserves all units of the input :class:`~astropy.table.Table`. | Since the format of external data can vary greatly, the process of generating a :class:`~pandas.DataFrame` will not be covered here, but this is what the :class:`~pandas.DataFrame` should look like: .. code-block:: python df = ... .. GENERATED FROM PYTHON SOURCE LINES 48-51 .. code-block:: Python df .. raw:: html
wavelength flux
0 5336.6 -812.069496
1 5336.8 119.795166
2 5337.0 104.242718
3 5337.2 -367.086382
4 5337.4 468.964615
... ... ...
24313 10199.2 11.602664
24314 10199.4 13.233428
24315 10199.6 10.652725
24316 10199.8 8.980172
24317 10200.0 10.517128

24318 rows × 2 columns



.. GENERATED FROM PYTHON SOURCE LINES 68-79 Any other required attributes must be passed as keyword arguments to :meth:`~ATK.Models.Spectrum.from_dataframe` (or :meth:`~ATK.Models.Spectrum.from_table`). A :class:`~ATK.Models.Spectrum` only requires that we specify a ``survey``. | .. note:: For information on the keyword arguments that are required by a data container, see its documentation for the IO method that is being used (for this tutorial, that would be :meth:`~ATK.Models.Spectrum.from_dataframe`). | With everything ready, the following code generates a :class:`~ATK.Models.Spectrum` from the external data: .. GENERATED FROM PYTHON SOURCE LINES 79-85 .. code-block:: Python from ATK.Models import Spectrum spec = Spectrum.from_dataframe(target, data=df, survey="XSHOOTER") spec.show() .. rst-class:: sphx-glr-script-out .. code-block:: none .survey: XSHOOTER .wavelength: [5336.6, 5336.8, ..., 10199.8, 10200.0] Å .flux: [-812.069, 119.795, ..., 8.98, 10.517] 1×10⁻¹⁷ erg Å⁻¹ s⁻¹ cm⁻² Available Methods: .bin(), .crop(), .from_dataframe(), .from_table(), .show(), .to_dataframe(), .to_hdu(), .to_table(), .vspec() .. GENERATED FROM PYTHON SOURCE LINES 89-94 | Adding Data to a DataSet ------------------------ The final step is to add the :class:`~ATK.Models.Spectrum` to the :class:`~ATK.Models.DataSet`, which can be done with :meth:`~ATK.Models.DataSet.add`: .. GENERATED FROM PYTHON SOURCE LINES 94-98 .. code-block:: Python dataset.add(spec) dataset.show() .. rst-class:: sphx-glr-script-out .. code-block:: none .kind: Spectrum .targets: 4223502720986764672 | 302.899° -2.228° (icrs, 2016-01-01T00:00:00.000) .exception: False .data: survey: XSHOOTER wavelength: [5336.6, 5336.8, ..., 10199.8, 10200.0] Å flux: [-812.069, 119.795, ..., 8.98, 10.517] 1×10⁻¹⁷ erg Å⁻¹ s⁻¹ cm⁻² Available Methods: .add(), .apply(), .from_target(), .merge(), .open(), .plot(), .save(), .show(), .split(), .store() .. GENERATED FROM PYTHON SOURCE LINES 102-107 | Using the DataSet ----------------- The newly constructed :class:`~ATK.Models.DataSet` can then be used throughout ATK as if it were retrieved internally: .. GENERATED FROM PYTHON SOURCE LINES 107-111 .. code-block:: Python dataset.apply("crop", cmin=5700, inplace=True) dataset.open(fit=True, prom=2, smooth=5) .. raw:: html
Bokeh Figure


.. GENERATED FROM PYTHON SOURCE LINES 121-130 | Another Example =============== Below is a full example of the process for importing and utilising an externally-acquired light curve (excluding loading of the data): .. code-block:: python df = ... .. GENERATED FROM PYTHON SOURCE LINES 130-143 .. code-block:: Python from ATK.Models import Lightcurve target = 6050296829033196032 dataset = DataSet.from_target(target) lc = Lightcurve.from_dataframe(target, data=df, survey="TNT", band="KG5") dataset.add(lc) folded = dataset.apply("fold", fmin=0, fmax=10, samples=10000, inplace=False) folded.apply("bin", bins=200) folded.open() .. raw:: html
Bokeh Figure


.. GENERATED FROM PYTHON SOURCE LINES 182-187 | | | .. rubric:: Download this Tutorial .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 7.641 seconds) .. _sphx_glr_download_auto_tutorials_extension_external_data.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: external_data.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: external_data.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: external_data.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_