Basic Operations

📊 Getting Started with YAXArraysToolbox.jl

Learn the core functions for visualizing and processing Earth System data cubes

Author: Daniel E. Pabon-Moreno

What You'll Learn

In this tutorial, you will learn how to:

✅ Load data from the Earth System Data Cube (ESDC)
✅ Create time series plots with plot_time
✅ Generate spatial maps with plot_space
✅ Aggregate data to different temporal resolutions with aggregate_time

Prerequisites

Basic familiarity with Julia and the concept of multi-dimensional arrays is helpful but not required.

Setup

First, let's load all the required packages:

using Pkg
Pkg.instantiate()
using YAXArrays
Pkg.add(url="https://github.com/dpabon/YAXArraysToolbox.jl")
using YAXArraysToolbox
using CairoMakie
using GeoMakie
using Statistics
using Zarr
using Dates
using PythonCall
using DimensionalData

     Cloning git-repo `https://github.com/dpabon/YAXArraysToolbox.jl`
    Updating git-repo `https://github.com/dpabon/YAXArraysToolbox.jl`
   Resolving package versions...
    Updating `~/work/YAXArraysToolbox.jl/YAXArraysToolbox.jl/docs/Project.toml`
  [fe326ce2] ~ YAXArraysToolbox v0.3.7 `..` ⇒ v0.3.7 `https://github.com/dpabon/YAXArraysToolbox.jl#main` [loaded: `/home/runner/work/YAXArraysToolbox.jl/YAXArraysToolbox.jl/src/YAXArraysToolbox.jl` (v0.3.7) expected `/home/runner/.julia/packages/YAXArraysToolbox/sZQac/src/YAXArraysToolbox.jl` (v0.3.7)]
    Updating `~/work/YAXArraysToolbox.jl/YAXArraysToolbox.jl/docs/Manifest.toml`
  [fe326ce2] ~ YAXArraysToolbox v0.3.7 `..` ⇒ v0.3.7 `https://github.com/dpabon/YAXArraysToolbox.jl#main` [loaded: `/home/runner/work/YAXArraysToolbox.jl/YAXArraysToolbox.jl/src/YAXArraysToolbox.jl` (v0.3.7) expected `/home/runner/.julia/packages/YAXArraysToolbox/sZQac/src/YAXArraysToolbox.jl` (v0.3.7)]
Precompiling packages...
  10012.7 ms  ✓ YAXArraysToolbox
  1 dependency successfully precompiled in 11 seconds. 553 already precompiled.
  1 dependency precompiled but a different version is currently loaded. Restart julia to access the new version. Otherwise, loading dependents of this package may trigger further precompilation to work with the unexpected version.
    CondaPkg Found dependencies: /home/runner/.julia/packages/DimensionalData/FWnw9/CondaPkg.toml
    CondaPkg Found dependencies: /home/runner/.julia/packages/CondaPkg/0UqYV/CondaPkg.toml
    CondaPkg Found dependencies: /home/runner/.julia/packages/PythonCall/83z4q/CondaPkg.toml
    CondaPkg Initialising pixi
             │ /home/runner/.julia/artifacts/cefba4912c2b400756d043a2563ef77a0088866b/bin/pixi
             │ init
             │ --format pixi
             └ /home/runner/work/YAXArraysToolbox.jl/YAXArraysToolbox.jl/docs/.CondaPkg
✔ Created /home/runner/work/YAXArraysToolbox.jl/YAXArraysToolbox.jl/docs/.CondaPkg/pixi.toml
    CondaPkg Wrote /home/runner/work/YAXArraysToolbox.jl/YAXArraysToolbox.jl/docs/.CondaPkg/pixi.toml
             │ [dependencies]
             │ openssl = ">=3, <3.6, >=3, <3.6"
             │ libstdcxx = ">=3.4,<15.0"
             │ libstdcxx-ng = ">=3.4,<15.0"
             │ xarray = "*"
             │ numpy = "*"
             │
             │     [dependencies.python]
             │     channel = "conda-forge"
             │     build = "*cp*"
             │     version = "3.13.*, >=3.10,!=3.14.0,!=3.14.1,<4"
             │
             │ [project]
             │ name = ".CondaPkg"
             │ platforms = ["linux-64"]
             │ channels = ["conda-forge"]
             │ channel-priority = "strict"
             └ description = "automatically generated by CondaPkg.jl"
    CondaPkg Installing packages
             │ /home/runner/.julia/artifacts/cefba4912c2b400756d043a2563ef77a0088866b/bin/pixi
             │ install
             └ --manifest-path /home/runner/work/YAXArraysToolbox.jl/YAXArraysToolbox.jl/docs/.CondaPkg/pixi.toml
✔ The default environment has been installed.

Loading Data

We'll use the Earth System Data Cube (ESDC), an analysis-ready data cube containing dozens of climate and Earth observation variables at global scale.

esdc = open_dataset("https://s3.bgc-jena.mpg.de:9000/esdl-esdc-v2.1.1/esdc-8d-0.25deg-184x90x90-2.1.1.zarr")
esdc = Cube(esdc)
esdc

┌ 1440×720×1840×79 YAXArray{Float64, 4} Downwelling shortwave radiation ┐
├───────────────────────────────────────────────────────────────────────┴ dims ┐
  ↓ lon Sampled{Float64} -179.875:0.25:179.875 ForwardOrdered Regular Points,
  → lat Sampled{Float64} 89.875:-0.25:-89.875 ReverseOrdered Regular Points,
  ↗ time Sampled{Dates.DateTime} [Dates.DateTime("1979-01-05T00:00:00"), …, Dates.DateTime("2018-12-31T00:00:00")] ForwardOrdered Irregular Points,
  ⬔ Variables Categorical{String} ["xco2", …, "Rg"] Unordered
├──────────────────────────────────────────────────────────────────── metadata ┤
  Dict{String, Any} with 27 entries:
  "time_coverage_end"        => "2010-12-31"
  "source_attributes"        => "{'long_name': 'Snow Sublimation', 'project_nam…
  "time_coverage_resolution" => "P8D"
  "time_coverage_start"      => "2000-03-01"
  "time_coverage_duration"   => "P1D"
  "history"                  => "Tue Apr 14 19:57:11 2020 - ESDL data cube gene…
  "standard_name"            => "surface_downwelling_shortwave_flux_in_air"
  "orig_version"             => "15.10.2017"
  "easting"                  => "-180 degrees"
  "_FillValue"               => NaN
  "Conventions"              => "CF-1.6"
  "units"                    => "W m-2"
  "source"                   => "ESDL data cube generation, version 0.3.0.dev1"
  "project_name"             => "BESS"
  "long_name"                => "Downwelling shortwave radiation"
  "esa_cci_path"             => ""
  "processing_level"         => "Level-3"
  "valid_min"                => 0.0
  "standard_name_vocabulary" => "NetCDF Climate and Forecast (CF) Metadata Conv…
  ⋮                          => ⋮
├─────────────────────────────────────────────────────────────── loaded lazily ┤
  data size: 1.1 TB
└──────────────────────────────────────────────────────────────────────────────┘

About the ESDC

The Earth System Data Cube provides harmonized Earth observation and climate data at 0.25° spatial resolution and 8-day temporal resolution, spanning from 1979 to 2021.

1. Time Series Plotting with `plot_time`

The plot_time function creates time series plots by collapsing spatial dimensions using a specified statistic (mean, median, std, etc.).

Selecting a Region

Let's focus on South America and select a time period:

cube_to_plot = esdc[
    lon = -86 .. -35,
    lat = -56 .. 14,
    time = Date(2010) .. Date(2014),
    Variable = At("leaf_area_index", "sensible_heat"),
]
cube_to_plot

┌ 204×280×184×2 YAXArray{Float64, 4} Downwelling shortwave radiation ┐
├────────────────────────────────────────────────────────────────────┴─── dims ┐
  ↓ lon Sampled{Float64} -85.875:0.25:-35.125 ForwardOrdered Regular Points,
  → lat Sampled{Float64} 13.875:-0.25:-55.875 ReverseOrdered Regular Points,
  ↗ time Sampled{Dates.DateTime} [Dates.DateTime("2010-01-05T00:00:00"), …, Dates.DateTime("2013-12-31T00:00:00")] ForwardOrdered Irregular Points,
  ⬔ Variables Categorical{String} ["leaf_area_index", "sensible_heat"] Unordered
├──────────────────────────────────────────────────────────────────── metadata ┤
  Dict{String, Any} with 27 entries:
  "time_coverage_end"        => "2010-12-31"
  "source_attributes"        => "{'long_name': 'Snow Sublimation', 'project_nam…
  "time_coverage_resolution" => "P8D"
  "time_coverage_start"      => "2000-03-01"
  "time_coverage_duration"   => "P1D"
  "history"                  => "Tue Apr 14 19:57:11 2020 - ESDL data cube gene…
  "standard_name"            => "surface_downwelling_shortwave_flux_in_air"
  "orig_version"             => "15.10.2017"
  "easting"                  => "-180 degrees"
  "_FillValue"               => NaN
  "Conventions"              => "CF-1.6"
  "units"                    => "W m-2"
  "source"                   => "ESDL data cube generation, version 0.3.0.dev1"
  "project_name"             => "BESS"
  "long_name"                => "Downwelling shortwave radiation"
  "esa_cci_path"             => ""
  "processing_level"         => "Level-3"
  "valid_min"                => 0.0
  "standard_name_vocabulary" => "NetCDF Climate and Forecast (CF) Metadata Conv…
  ⋮                          => ⋮
├─────────────────────────────────────────────────────────────── loaded lazily ┤
  data size: 160.37 MB
└──────────────────────────────────────────────────────────────────────────────┘

Plotting All Variables

By default, plot_time plots all variables in the cube:

plot_time(
    cube_to_plot;
    time_axis = :time,
    var_axis = :Variables,
    lon_axis = :lon,
    lat_axis = :lat,
    var = nothing,
    fun = "std",
    resolution = (900, 600),
    p = 0.2,
    showprog = true,
    max_cache = "1GB",
    ncol = 1,
    nrow = 2
)

Plotting a Single Variable

You can also target a specific variable:

plot_time(
    cube_to_plot;
    time_axis = :time,
    var_axis = :Variable,
    lon_axis = :lon,
    lat_axis = :lat,
    var = "sensible_heat",
    fun = "std",
    resolution = (900, 600),
    p = 0.2,
    showprog = true,
    max_cache = "1GB",
    ncol = 1,
    nrow = 2
)

Available Statistics

The fun parameter supports multiple aggregation statistics:

Statistic	Description
`"mean"`	Arithmetic mean
`"median"`	Median (50th percentile)
`"std"`	Standard deviation
`"var"`	Variance
`"sum"`	Sum of values
`"min"`	Minimum value
`"max"`	Maximum value
`"quant"`	Quantile (requires `p` parameter)

metrics = ["median", "mean", "std", "var", "sum", "quant", "min", "max"]

for metric in metrics
    fig = plot_time(
        cube_to_plot;
        time_axis = :time,
        var_axis = :Variable,
        lon_axis = :lon,
        lat_axis = :lat,
        var = "sensible_heat",
        fun = metric,
        resolution = (900, 600),
        p = 0.2,
        showprog = true,
        max_cache = "1GB",
        ncol = 1,
        nrow = 2
    )
    display(fig)
end

┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264

2. Spatial Mapping with `plot_space`

The plot_space function creates spatial maps by collapsing the time dimension.

Single Variable Map

cube_to_plot = esdc[
    lon = -86 .. -34,
    lat = -56 .. 14,
    time = Date(2010) .. Date(2014),
    Variable = At("leaf_area_index", "sensible_heat"),
]

plot_space(
    cube_to_plot;
    time_axis = :time,
    resolution = (900, 600),
    var_axis = :Variable,
    var = "leaf_area_index",
    fun = "median"
)

Multiple Variables Side by Side

Set var = nothing to plot all variables:

plot_space(
    cube_to_plot;
    time_axis = :time,
    resolution = (900, 600),
    var_axis = :Variables,
    var = nothing,
    ncol = 2,
    nrow = 1,
    fun = "median"
)

Comparing Different Statistics

metrics = ["median", "mean", "std", "var", "sum", "quant", "min", "max"]

for metric in metrics
    fig = plot_space(
        cube_to_plot;
        time_axis = :time,
        var_axis = :Variable,
        lon_axis = :lon,
        lat_axis = :lat,
        var = "sensible_heat",
        fun = metric,
        p = 0.2,
        showprog = true,
        max_cache = "100MB"
    )
    display(fig)
end


Progress:  50%|████████████████████▌                    |  ETA: 0:00:04
Progress:  75%|██████████████████████████████▊          |  ETA: 0:00:02
Progress: 100%|█████████████████████████████████████████| Time: 0:00:06
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264

Progress:  50%|████████████████████▌                    |  ETA: 0:00:04
Progress:  75%|██████████████████████████████▊          |  ETA: 0:00:02
Progress: 100%|█████████████████████████████████████████| Time: 0:00:07
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264

Progress:  50%|████████████████████▌                    |  ETA: 0:00:04
Progress:  75%|██████████████████████████████▊          |  ETA: 0:00:02
Progress: 100%|█████████████████████████████████████████| Time: 0:00:06
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264

Progress:  50%|████████████████████▌                    |  ETA: 0:00:04
Progress:  75%|██████████████████████████████▊          |  ETA: 0:00:02
Progress: 100%|█████████████████████████████████████████| Time: 0:00:06
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264

Progress:  50%|████████████████████▌                    |  ETA: 0:00:04
Progress:  75%|██████████████████████████████▊          |  ETA: 0:00:02
Progress: 100%|█████████████████████████████████████████| Time: 0:00:06
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264

Progress:  50%|████████████████████▌                    |  ETA: 0:00:04
Progress:  75%|██████████████████████████████▊          |  ETA: 0:00:02
Progress: 100%|█████████████████████████████████████████| Time: 0:00:06
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264

Progress:  50%|████████████████████▌                    |  ETA: 0:00:04
Progress:  75%|██████████████████████████████▊          |  ETA: 0:00:02
Progress: 100%|█████████████████████████████████████████| Time: 0:00:06
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264

Progress:  50%|████████████████████▌                    |  ETA: 0:00:04
Progress:  75%|██████████████████████████████▊          |  ETA: 0:00:02
Progress: 100%|█████████████████████████████████████████| Time: 0:00:06
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/Vn16E/src/scenes.jl:264

3. Temporal Aggregation with `aggregate_time`

The aggregate_time function allows you to resample data to different temporal resolutions.

Monthly Aggregation Example

The ESDC has 8-day temporal resolution. Let's aggregate to monthly means:

lai_month = aggregate_time(
    esdc[Variable = At("leaf_area_index")];
    time_axis = :time,
    new_resolution = "month",
    new_time_step = 1,
    fun = "mean",
    p = nothing,
    skipMissing = true,
    skipnan = true,
    showprog = true,
    max_cache = "1GB"
)
lai_month

┌ 480×1440×720 YAXArray{Union{Missing, Float64}, 3} ┐
├───────────────────────────────────────────────────┴──────────────────── dims ┐
  ↓ Ti Sampled{Dates.DateTime} [Dates.DateTime("1979-01-01T00:00:00"), …, Dates.DateTime("2018-12-01T00:00:00")] ForwardOrdered Irregular Points,
  → lon Sampled{Float64} -179.875:0.25:179.875 ForwardOrdered Regular Points,
  ↗ lat Sampled{Float64} 89.875:-0.25:-89.875 ReverseOrdered Regular Points
├──────────────────────────────────────────────────────────────────── metadata ┤
  Dict{String, Any} with 1 entry:
  "missing_value" => 1.0e32
├─────────────────────────────────────────────────────────────── loaded lazily ┤
  data size: 3.71 GB
└──────────────────────────────────────────────────────────────────────────────┘

The original 8-day temporal resolution has been aggregated to monthly values. The time axis now contains approximately 480 values (one per month over the full time range) instead of the original ~1800+ 8-day values.

Quick Reference

Function	Purpose	Key Parameters
`plot_time`	Time series plot	`fun`, `var`, `time_axis`
`plot_space`	Spatial map	`fun`, `var`, `time_axis`
`aggregate_time`	Temporal aggregation	`new_resolution`, `fun`

Common Parameters

fun = "mean"           # Aggregation statistic
var = "temperature"    # Variable name (or nothing for all)
time_axis = :time      # Name of time dimension
showprog = true        # Show progress bar
max_cache = "1GB"      # Memory limit for caching

Next Steps

Now that you've mastered the basics, explore more advanced analysis:

📖 Space-for-Time Method — Learn how to estimate land cover change impacts
📚 API Reference — Complete documentation of all functions

💡 Tip: All examples in this tutorial use lazy evaluation. Data is only loaded when needed, making it efficient to work with large datasets.