Data Organization¶
home: to keep your light weight & precious files (codes, scripts, figures,…)
workspace: for all personal data files (post-processing runs, diagnostics,…) It can be accessed at:
/local/data/<server>/workspace/<username>
Or by using the corresponding environment variable:
cd $workspace
Observations: if you download, regrid, process,… observations into a dataset that can be useful to others. Please contact admin to move it to the shared space located at:
/local/data/<server>/observations
Quick access with:
cd $observations
Simulations: model inputs/outputs from the group valuable (published, in prep.) experiments, as well as reference experiments from others group (NCAR, GFDL,…) that are of common interest should be moved to:
/local/data/<server>/simulations
Quick access with:
cd $simulations
Discover data with intake:¶
Install intake and intake_xarray installed in your conda environment:
conda install -c intake intake
conda install -c conda-forge intake-xarray
you can explore the datasets available using intake catalogs available here:
/local/data/<server>/catalogs
For example, exploring ASTE data inside python can be done with:
import intake
catalog = intake.Catalog('/local/data/artemis/catalogs/ASTE_catalog.yml')
list(catalog)
Once you have found the dataset you’re interested in, you can load as follows:
ds_biomes = catalog.Biomes_FayMcKinley.to_dask()
# test plot
import matplotlib.pylab as plt
ds_biomes.MeanBiomes.sel(face=4).plot()
plt.show()