Climate change

In the following sections, we use Python to demonstrate how to access multiples datasets from the Climate change sub-catalog.

Environment setup

from distributed import Client
import intake
import hvplot.xarray
import hvplot.pandas
from dask.distributed import PipInstall
import dask
import xoak
import xarray as xr
import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
import xarray as xr
import numpy as np
import dask
from dask.diagnostics import progress
from tqdm.autonotebook import tqdm
import fsspec
import seaborn as sns

%matplotlib inline
%config InlineBackend.figure_format = 'retina'

/tmp/ipykernel_3903/326864322.py:16: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from tqdm.autonotebook import tqdm

We use a Dask client to ensure all following code compatible with the framework run in parallel

client = Client()
client

2022-12-12 19:54:19,691 - distributed.diskutils - INFO - Found stale lock file and directory '/tmp/dask-worker-space/worker-dggdmyk1', purging
2022-12-12 19:54:19,692 - distributed.diskutils - INFO - Found stale lock file and directory '/tmp/dask-worker-space/worker-ldoowjz8', purging

Client

Client-c3f37eb7-7a56-11ed-8f3f-000d3a3e751f

Connection method: Cluster object	Cluster type: distributed.LocalCluster
Dashboard: http://127.0.0.1:8787/status

Cluster Info

LocalCluster

ed282a95

Dashboard: http://127.0.0.1:8787/status	Workers: 2
Total threads: 2	Total memory: 6.78 GiB
Status: running	Using processes: True

Scheduler Info

Scheduler

Scheduler-bf733540-6596-4315-ae59-a22f696191d1

Comm: tcp://127.0.0.1:39379	Workers: 2
Dashboard: http://127.0.0.1:8787/status	Total threads: 2
Started: Just now	Total memory: 6.78 GiB

Workers

Worker: 0

Comm: tcp://127.0.0.1:37183	Total threads: 1
Dashboard: http://127.0.0.1:34483/status	Memory: 3.39 GiB
Nanny: tcp://127.0.0.1:38359
Local directory: /tmp/dask-worker-space/worker-0l8puexp

Worker: 1

Comm: tcp://127.0.0.1:38445	Total threads: 1
Dashboard: http://127.0.0.1:44455/status	Memory: 3.39 GiB
Nanny: tcp://127.0.0.1:41493
Local directory: /tmp/dask-worker-space/worker-1cmdxfvs

Accessing the data

We are now ready to access our catalog which uses Intake-ESM to organize all our datasets.

Intake is a lightweight package for finding, investigating, loading and disseminating data. A cataloging system is used to organize a collection of datasets and data loaders (drivers) are parameterized such that datasets are opened in the desired format for the end user. In the python context, multi-dimensional xarrays could be opened with xarray’s drivers while polygons (shapefiles, geojson) could be opened with geopandas.

Here is the URL from where we can open the catalog:

a) CMIP6

In order to arrange the collection of datasets, the catalogue itself makes references to various sub-datasets:

col = intake.open_esm_datastore('https://storage.googleapis.com/cmip6/pangeo-cmip6.json')
col

pangeo-cmip6 catalog with 7674 dataset(s) from 514818 asset(s):

	unique
activity_id	18
institution_id	36
source_id	88
experiment_id	170
member_id	657
table_id	37
variable_id	700
grid_label	10
zstore	514818
dcpp_init_year	60
version	736
derived_variable_id	0

col.df['experiment_id'].unique()

array(['highresSST-present', 'piControl', 'control-1950', 'hist-1950',
       'historical', 'amip', 'abrupt-4xCO2', 'abrupt-2xCO2',
       'abrupt-0p5xCO2', '1pctCO2', 'ssp585', 'esm-piControl', 'esm-hist',
       'hist-piAer', 'histSST-1950HC', 'ssp245', 'hist-1950HC', 'histSST',
       'piClim-2xVOC', 'piClim-2xNOx', 'piClim-2xdust', 'piClim-2xss',
       'piClim-histall', 'hist-piNTCF', 'histSST-piNTCF',
       'aqua-control-lwoff', 'piClim-lu', 'histSST-piO3', 'piClim-CH4',
       'piClim-NTCF', 'piClim-NOx', 'piClim-O3', 'piClim-HC',
       'faf-heat-NA0pct', 'ssp370SST-lowCH4', 'piClim-VOC',
       'ssp370-lowNTCF', 'piClim-control', 'piClim-aer', 'hist-aer',
       'faf-heat', 'faf-heat-NA50pct', 'ssp370SST-lowNTCF',
       'ssp370SST-ssp126Lu', 'ssp370SST', 'ssp370pdSST', 'histSST-piAer',
       'piClim-ghg', 'piClim-anthro', 'faf-all', 'hist-nat', 'hist-GHG',
       'ssp119', 'piClim-histnat', 'piClim-4xCO2', 'ssp370',
       'piClim-histghg', 'highresSST-future', 'esm-ssp585-ssp126Lu',
       'ssp126-ssp370Lu', 'ssp370-ssp126Lu', 'land-noLu', 'histSST-piCH4',
       'ssp126', 'esm-pi-CO2pulse', 'amip-hist', 'piClim-histaer',
       'amip-4xCO2', 'faf-water', 'faf-passiveheat', '1pctCO2-rad',
       'faf-stress', '1pctCO2-bgc', 'aqua-control', 'amip-future4K',
       'amip-p4K', 'aqua-p4K', 'amip-lwoff', 'amip-m4K', 'aqua-4xCO2',
       'amip-p4K-lwoff', 'hist-noLu', '1pctCO2-cdr',
       'land-hist-altStartYear', 'land-hist', 'omip1', 'esm-pi-cdr-pulse',
       'esm-ssp585', 'abrupt-solp4p', 'piControl-spinup', 'hist-stratO3',
       'abrupt-solm4p', 'midHolocene', 'lig127k', 'aqua-p4K-lwoff',
       'esm-piControl-spinup', 'ssp245-GHG', 'ssp245-nat',
       'dcppC-amv-neg', 'dcppC-amv-ExTrop-neg', 'dcppC-atl-control',
       'dcppC-amv-pos', 'dcppC-ipv-NexTrop-neg', 'dcppC-ipv-NexTrop-pos',
       'dcppC-atl-pacemaker', 'dcppC-amv-ExTrop-pos',
       'dcppC-amv-Trop-neg', 'dcppC-pac-control', 'dcppC-ipv-pos',
       'dcppC-pac-pacemaker', 'dcppC-ipv-neg', 'dcppC-amv-Trop-pos',
       'piClim-BC', 'piClim-2xfire', 'piClim-SO2', 'piClim-OC',
       'piClim-N2O', 'piClim-2xDMS', 'ssp460', 'ssp434', 'ssp534-over',
       'deforest-globe', 'historical-cmip5', 'hist-bgc',
       'piControl-cmip5', 'rcp26-cmip5', 'rcp45-cmip5', 'rcp85-cmip5',
       'pdSST-piArcSIC', 'pdSST-piAntSIC', 'piSST-piSIC', 'piSST-pdSIC',
       'ssp245-stratO3', 'hist-sol', 'hist-CO2', 'hist-volc',
       'hist-totalO3', 'hist-nat-cmip5', 'hist-aer-cmip5',
       'hist-GHG-cmip5', 'pdSST-futAntSIC', 'futSST-pdSIC', 'pdSST-pdSIC',
       'ssp245-aer', 'pdSST-futArcSIC', 'dcppA-hindcast', 'dcppA-assim',
       'dcppC-hindcast-noPinatubo', 'dcppC-hindcast-noElChichon',
       'dcppC-hindcast-noAgung', 'ssp245-cov-modgreen',
       'ssp245-cov-fossil', 'ssp245-cov-strgreen', 'ssp245-covid', 'lgm',
       'ssp585-bgc', '1pctCO2to4x-withism', '1pctCO2-4xext',
       'hist-resIPO', 'past1000', 'pa-futArcSIC', 'pa-pdSIC',
       'historical-ext', 'pdSST-futArcSICSIT', 'pdSST-futOkhotskSIC',
       'pdSST-futBKSeasSIC', 'pa-piArcSIC', 'pa-piAntSIC', 'pa-futAntSIC',
       'pdSST-pdSICSIT'], dtype=object)

col.df['table_id'].unique()

array(['Amon', '6hrPlev', '3hr', 'day', 'EmonZ', 'E3hr', '6hrPlevPt',
       'AERmon', 'LImon', 'CFmon', 'Lmon', 'fx', 'SImon', 'Ofx', 'Omon',
       'EdayZ', 'Emon', 'CFday', 'AERday', 'Eday', 'Oyr', 'Eyr', 'Oday',
       'SIday', 'AERmonZ', '6hrLev', 'E1hrClimMon', 'CF3hr', 'AERhr',
       'Odec', 'Oclim', 'Efx', 'Aclim', 'SIclim', 'IfxGre', 'ImonGre',
       'Eclim'], dtype=object)

from itables import init_notebook_mode, show
from IPython.display import HTML

init_notebook_mode(all_interactive=False)

show(col.df,
     tags="<caption>Catalog</caption>",
     column_filters="footer",
     dom="lrtip")

Catalog

activity_id	institution_id	source_id	experiment_id	member_id	table_id	variable_id	grid_label	zstore	dcpp_init_year	version
Loading... (need help?)
activity_id	institution_id	source_id	experiment_id	member_id	table_id	variable_id	grid_label	zstore	dcpp_init_year	version

# load a few models to illustrate the problem
query = dict(experiment_id=["ssp370"],
             variable_id="tasmax",
             grid_label="gn",
             table_id='Amon',
             member_id='r1i1p1f1'
            )
cat = col.search(**query)

xarray_kwargs = {'consolidated': True}

with dask.config.set(**{'array.slicing.split_large_chunks': True}):
    dset_dict = cat.to_dataset_dict(xarray_open_kwargs=xarray_kwargs)

--> The keys in the returned dictionary of datasets are constructed as follows:
        'activity_id.institution_id.source_id.experiment_id.table_id.grid_label'

100.00% [10/10 00:06<00:00]

WARNING:google.auth._default:Authentication failed using Compute Engine authentication due to unavailable metadata server.
WARNING:google.auth.compute_engine._metadata:Compute Engine Metadata server unavailable on attempt 1 of 5. Reason: HTTPConnectionPool(host='metadata.google.internal', port=80): Max retries exceeded with url: /computeMetadata/v1/instance/service-accounts/default/?recursive=true (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fec886669a0>: Failed to establish a new connection: [Errno -2] Name or service not known'))
WARNING:google.auth.compute_engine._metadata:Compute Engine Metadata server unavailable on attempt 2 of 5. Reason: HTTPConnectionPool(host='metadata.google.internal', port=80): Max retries exceeded with url: /computeMetadata/v1/instance/service-accounts/default/?recursive=true (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fec88666c70>: Failed to establish a new connection: [Errno -2] Name or service not known'))
WARNING:google.auth.compute_engine._metadata:Compute Engine Metadata server unavailable on attempt 3 of 5. Reason: HTTPConnectionPool(host='metadata.google.internal', port=80): Max retries exceeded with url: /computeMetadata/v1/instance/service-accounts/default/?recursive=true (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fec88666a00>: Failed to establish a new connection: [Errno -2] Name or service not known'))
WARNING:google.auth.compute_engine._metadata:Compute Engine Metadata server unavailable on attempt 4 of 5. Reason: HTTPConnectionPool(host='metadata.google.internal', port=80): Max retries exceeded with url: /computeMetadata/v1/instance/service-accounts/default/?recursive=true (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fec88603130>: Failed to establish a new connection: [Errno -2] Name or service not known'))
WARNING:google.auth.compute_engine._metadata:Compute Engine Metadata server unavailable on attempt 5 of 5. Reason: HTTPConnectionPool(host='metadata.google.internal', port=80): Max retries exceeded with url: /computeMetadata/v1/instance/service-accounts/default/?recursive=true (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fec886036d0>: Failed to establish a new connection: [Errno -2] Name or service not known'))

[key for key in dset_dict.keys()]

['ScenarioMIP.MPI-M.MPI-ESM1-2-LR.ssp370.Amon.gn',
 'ScenarioMIP.DKRZ.MPI-ESM1-2-HR.ssp370.Amon.gn',
 'ScenarioMIP.MRI.MRI-ESM2-0.ssp370.Amon.gn',
 'ScenarioMIP.AWI.AWI-CM-1-1-MR.ssp370.Amon.gn',
 'ScenarioMIP.HAMMOZ-Consortium.MPI-ESM-1-2-HAM.ssp370.Amon.gn',
 'ScenarioMIP.BCC.BCC-CSM2-MR.ssp370.Amon.gn',
 'ScenarioMIP.MIROC.MIROC6.ssp370.Amon.gn',
 'AerChemMIP.BCC.BCC-ESM1.ssp370.Amon.gn',
 'ScenarioMIP.CSIRO-ARCCSS.ACCESS-CM2.ssp370.Amon.gn',
 'ScenarioMIP.CAS.FGOALS-g3.ssp370.Amon.gn']

ds = dset_dict[list(dset_dict.keys())[-6]]
ds

<xarray.Dataset>
Dimensions:         (lat: 96, bnds: 2, lon: 192, member_id: 1,
                     dcpp_init_year: 1, time: 492)
Coordinates:
    height          float64 ...
  * lat             (lat) float64 -88.57 -86.72 -84.86 ... 84.86 86.72 88.57
    lat_bnds        (lat, bnds) float64 dask.array<chunksize=(96, 2), meta=np.ndarray>
  * lon             (lon) float64 0.0 1.875 3.75 5.625 ... 354.4 356.2 358.1
    lon_bnds        (lon, bnds) float64 dask.array<chunksize=(192, 2), meta=np.ndarray>
  * time            (time) datetime64[ns] 2015-01-16T12:00:00 ... 2055-12-16T...
    time_bnds       (time, bnds) datetime64[ns] dask.array<chunksize=(492, 2), meta=np.ndarray>
  * member_id       (member_id) object 'r1i1p1f1'
  * dcpp_init_year  (dcpp_init_year) float64 nan
Dimensions without coordinates: bnds
Data variables:
    tasmax          (member_id, dcpp_init_year, time, lat, lon) float32 dask.array<chunksize=(1, 1, 246, 96, 192), meta=np.ndarray>
Attributes: (12/63)
    Conventions:                      CF-1.7 CMIP-6.2
    activity_id:                      ScenarioMIP AerChemMIP
    branch_method:                    standard
    branch_time_in_child:             60265.0
    branch_time_in_parent:            60265.0
    cmor_version:                     3.5.0
    ...                               ...
    intake_esm_attrs:variable_id:     tasmax
    intake_esm_attrs:grid_label:      gn
    intake_esm_attrs:zstore:          gs://cmip6/CMIP6/ScenarioMIP/HAMMOZ-Con...
    intake_esm_attrs:version:         20190628
    intake_esm_attrs:_data_format_:   zarr
    intake_esm_dataset_key:           ScenarioMIP.HAMMOZ-Consortium.MPI-ESM-1...

xarray.Dataset

• Dimensions:
  • lat: 96
  • bnds: 2
  • lon: 192
  • member_id: 1
  • dcpp_init_year: 1
  • time: 492
• Coordinates: (9)
  • height
    ()
    float64
    ...

    axis :

    Z

    long_name :

    height

    positive :

    up

    standard_name :

    height

    units :

    m

    [1 values with dtype=float64]

  • lat
    (lat)
    float64
    -88.57 -86.72 ... 86.72 88.57

    axis :

    Y

    bounds :

    lat_bnds

    long_name :

    Latitude

    standard_name :

    latitude

    units :

    degrees_north

    array([-88.572169, -86.722531, -84.86197 , -82.998942, -81.134977, -79.270559,
           -77.405888, -75.541061, -73.676132, -71.811132, -69.946081, -68.080991,
           -66.215872, -64.35073 , -62.485571, -60.620396, -58.755209, -56.890013,
           -55.024808, -53.159595, -51.294377, -49.429154, -47.563926, -45.698694,
           -43.833459, -41.96822 , -40.102979, -38.237736, -36.372491, -34.507243,
           -32.641994, -30.776744, -28.911492, -27.046239, -25.180986, -23.315731,
           -21.450475, -19.585219, -17.719962, -15.854704, -13.989446, -12.124187,
           -10.258928,  -8.393669,  -6.528409,  -4.66315 ,  -2.79789 ,  -0.93263 ,
             0.93263 ,   2.79789 ,   4.66315 ,   6.528409,   8.393669,  10.258928,
            12.124187,  13.989446,  15.854704,  17.719962,  19.585219,  21.450475,
            23.315731,  25.180986,  27.046239,  28.911492,  30.776744,  32.641994,
            34.507243,  36.372491,  38.237736,  40.102979,  41.96822 ,  43.833459,
            45.698694,  47.563926,  49.429154,  51.294377,  53.159595,  55.024808,
            56.890013,  58.755209,  60.620396,  62.485571,  64.35073 ,  66.215872,
            68.080991,  69.946081,  71.811132,  73.676132,  75.541061,  77.405888,
            79.270559,  81.134977,  82.998942,  84.86197 ,  86.722531,  88.572169])

  • lat_bnds
    (lat, bnds)
    float64
    dask.array<chunksize=(96, 2), meta=np.ndarray>

    Array Chunk Bytes 1.50 kiB 1.50 kiB Shape (96, 2) (96, 2) Dask graph 1 chunks in 2 graph layers Data type float64 numpy.ndarray

  • lon
    (lon)
    float64
    0.0 1.875 3.75 ... 356.2 358.1

    axis :

    X

    bounds :

    lon_bnds

    long_name :

    Longitude

    standard_name :

    longitude

    units :

    degrees_east

    array([  0.   ,   1.875,   3.75 ,   5.625,   7.5  ,   9.375,  11.25 ,  13.125,
            15.   ,  16.875,  18.75 ,  20.625,  22.5  ,  24.375,  26.25 ,  28.125,
            30.   ,  31.875,  33.75 ,  35.625,  37.5  ,  39.375,  41.25 ,  43.125,
            45.   ,  46.875,  48.75 ,  50.625,  52.5  ,  54.375,  56.25 ,  58.125,
            60.   ,  61.875,  63.75 ,  65.625,  67.5  ,  69.375,  71.25 ,  73.125,
            75.   ,  76.875,  78.75 ,  80.625,  82.5  ,  84.375,  86.25 ,  88.125,
            90.   ,  91.875,  93.75 ,  95.625,  97.5  ,  99.375, 101.25 , 103.125,
           105.   , 106.875, 108.75 , 110.625, 112.5  , 114.375, 116.25 , 118.125,
           120.   , 121.875, 123.75 , 125.625, 127.5  , 129.375, 131.25 , 133.125,
           135.   , 136.875, 138.75 , 140.625, 142.5  , 144.375, 146.25 , 148.125,
           150.   , 151.875, 153.75 , 155.625, 157.5  , 159.375, 161.25 , 163.125,
           165.   , 166.875, 168.75 , 170.625, 172.5  , 174.375, 176.25 , 178.125,
           180.   , 181.875, 183.75 , 185.625, 187.5  , 189.375, 191.25 , 193.125,
           195.   , 196.875, 198.75 , 200.625, 202.5  , 204.375, 206.25 , 208.125,
           210.   , 211.875, 213.75 , 215.625, 217.5  , 219.375, 221.25 , 223.125,
           225.   , 226.875, 228.75 , 230.625, 232.5  , 234.375, 236.25 , 238.125,
           240.   , 241.875, 243.75 , 245.625, 247.5  , 249.375, 251.25 , 253.125,
           255.   , 256.875, 258.75 , 260.625, 262.5  , 264.375, 266.25 , 268.125,
           270.   , 271.875, 273.75 , 275.625, 277.5  , 279.375, 281.25 , 283.125,
           285.   , 286.875, 288.75 , 290.625, 292.5  , 294.375, 296.25 , 298.125,
           300.   , 301.875, 303.75 , 305.625, 307.5  , 309.375, 311.25 , 313.125,
           315.   , 316.875, 318.75 , 320.625, 322.5  , 324.375, 326.25 , 328.125,
           330.   , 331.875, 333.75 , 335.625, 337.5  , 339.375, 341.25 , 343.125,
           345.   , 346.875, 348.75 , 350.625, 352.5  , 354.375, 356.25 , 358.125])

  • lon_bnds
    (lon, bnds)
    float64
    dask.array<chunksize=(192, 2), meta=np.ndarray>

    Array Chunk Bytes 3.00 kiB 3.00 kiB Shape (192, 2) (192, 2) Dask graph 1 chunks in 2 graph layers Data type float64 numpy.ndarray

  • time
    (time)
    datetime64[ns]
    2015-01-16T12:00:00 ... 2055-12-...

    axis :

    T

    bounds :

    time_bnds

    long_name :

    time

    standard_name :

    time

    array(['2015-01-16T12:00:00.000000000', '2015-02-15T00:00:00.000000000',
           '2015-03-16T12:00:00.000000000', ..., '2055-10-16T12:00:00.000000000',
           '2055-11-16T00:00:00.000000000', '2055-12-16T12:00:00.000000000'],
          dtype='datetime64[ns]')

  • time_bnds
    (time, bnds)
    datetime64[ns]
    dask.array<chunksize=(492, 2), meta=np.ndarray>

    Array Chunk Bytes 7.69 kiB 7.69 kiB Shape (492, 2) (492, 2) Dask graph 1 chunks in 2 graph layers Data type datetime64[ns] numpy.ndarray

  • member_id
    (member_id)
    object
    'r1i1p1f1'

    array(['r1i1p1f1'], dtype=object)

  • dcpp_init_year
    (dcpp_init_year)
    float64
    nan

    array([nan])

• Data variables: (1)
  • tasmax
    (member_id, dcpp_init_year, time, lat, lon)
    float32
    dask.array<chunksize=(1, 1, 246, 96, 192), meta=np.ndarray>

    cell_measures :

    area: areacella

    cell_methods :

    area: mean time: maximum within days time: mean over days

    comment :

    maximum near-surface (usually, 2 meter) air temperature (add cell_method attribute 'time: max')

    history :

    2020-05-29T17:28:59Z altered by CMOR: Treated scalar dimension: 'height'. 2020-05-29T17:28:59Z altered by CMOR: replaced missing value flag (-9e+33) and corresponding data with standard missing value (1e+20). 2020-05-29T17:28:59Z altered by CMOR: Inverted axis: lat.

    long_name :

    Daily Maximum Near-Surface Air Temperature

    standard_name :

    air_temperature

    units :

    K

    Array Chunk Bytes 34.59 MiB 17.30 MiB Shape (1, 1, 492, 96, 192) (1, 1, 246, 96, 192) Dask graph 2 chunks in 3 graph layers Data type float32 numpy.ndarray

• Indexes: (5)
  • lat
    PandasIndex

    PandasIndex(Float64Index([ -88.57216851400727,  -86.72253095466814,  -84.86197029204237,
                   -82.99894164283755,  -81.13497683767741,  -79.27055903485967,
                    -77.4058880820788,  -75.54106145287895,  -73.67613231320912,
                   -71.81113211427447,  -69.94608064698343,  -68.08099098565125,
                    -66.2158721139987,  -64.35073040887207,  -62.48557052203639,
                   -60.62039592682648,  -58.75520926937993,  -56.89001260135711,
                   -55.02480753831166,  -53.15959537001968,  -51.29437713895115,
                   -49.42915369712305,  -47.56392574797867, -45.698693877701785,
                   -43.83345857895126,  -41.96822026907538,   -40.1029793042494,
                   -38.23773599056483,  -36.37249059281224,  -34.50724334150103,
                   -32.64199443851768,  -30.77674406172325,  -28.91149236871774,
                   -27.04623949994481, -25.180985581270594,  -23.31573072614093,
                  -21.450475037398185,  -19.58521860882233, -17.719961526447428,
                  -15.854703869694873, -13.989445712356673, -12.124187123455766,
                  -10.258928168006376,  -8.393668907692383, -6.5284094014799905,
                   -4.663149706177884,  -2.797889876956741,  -0.932629967837991,
                    0.932629967837991,   2.797889876956741,   4.663149706177884,
                   6.5284094014799905,   8.393668907692383,  10.258928168006376,
                   12.124187123455766,  13.989445712356673,  15.854703869694873,
                   17.719961526447428,   19.58521860882233,  21.450475037398185,
                    23.31573072614093,  25.180985581270594,   27.04623949994481,
                    28.91149236871774,   30.77674406172325,   32.64199443851768,
                    34.50724334150103,   36.37249059281224,   38.23773599056483,
                     40.1029793042494,   41.96822026907538,   43.83345857895126,
                   45.698693877701785,   47.56392574797867,   49.42915369712305,
                    51.29437713895115,   53.15959537001968,   55.02480753831166,
                    56.89001260135711,   58.75520926937993,   60.62039592682648,
                    62.48557052203639,   64.35073040887207,    66.2158721139987,
                    68.08099098565125,   69.94608064698343,   71.81113211427447,
                    73.67613231320912,   75.54106145287895,    77.4058880820788,
                    79.27055903485967,   81.13497683767741,   82.99894164283755,
                    84.86197029204237,   86.72253095466814,   88.57216851400727],
                 dtype='float64', name='lat'))

  • lon
    PandasIndex

    PandasIndex(Float64Index([    0.0,   1.875,    3.75,   5.625,     7.5,   9.375,   11.25,
                   13.125,    15.0,  16.875,
                  ...
                   341.25, 343.125,   345.0, 346.875,  348.75, 350.625,   352.5,
                  354.375,  356.25, 358.125],
                 dtype='float64', name='lon', length=192))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2015-01-16 12:00:00', '2015-02-15 00:00:00',
                   '2015-03-16 12:00:00', '2015-04-16 00:00:00',
                   '2015-05-16 12:00:00', '2015-06-16 00:00:00',
                   '2015-07-16 12:00:00', '2015-08-16 12:00:00',
                   '2015-09-16 00:00:00', '2015-10-16 12:00:00',
                   ...
                   '2055-03-16 12:00:00', '2055-04-16 00:00:00',
                   '2055-05-16 12:00:00', '2055-06-16 00:00:00',
                   '2055-07-16 12:00:00', '2055-08-16 12:00:00',
                   '2055-09-16 00:00:00', '2055-10-16 12:00:00',
                   '2055-11-16 00:00:00', '2055-12-16 12:00:00'],
                  dtype='datetime64[ns]', name='time', length=492, freq=None))

  • member_id
    PandasIndex

    PandasIndex(Index(['r1i1p1f1'], dtype='object', name='member_id'))

  • dcpp_init_year
    PandasIndex

    PandasIndex(Float64Index([nan], dtype='float64', name='dcpp_init_year'))

• Attributes: (63)

  Conventions :

  CF-1.7 CMIP-6.2

  activity_id :

  ScenarioMIP AerChemMIP

  branch_method :

  standard

  branch_time_in_child :

  60265.0

  branch_time_in_parent :

  60265.0

  cmor_version :

  3.5.0

  contact :

  david.neubauer@env.ethz.ch

  creation_date :

  2020-05-29T17:28:59Z

  data_specs_version :

  01.00.30

  experiment :

  gap-filling scenario reaching 7.0 based on SSP3

  experiment_id :

  ssp370

  external_variables :

  areacella

  forcing_index :

  1

  frequency :

  mon

  further_info_url :

  https://furtherinfo.es-doc.org/CMIP6.HAMMOZ-Consortium.MPI-ESM-1-2-HAM.ssp370.none.r1i1p1f1

  grid :

  gn

  grid_label :

  gn

  history :

  2020-05-29T17:28:59Z ; CMOR rewrote data to be consistent with CMIP6, CF-1.7 CMIP-6.2 and CF standards.

  initialization_index :

  1

  institution :

  ETH Zurich, Switzerland; Max Planck Institut fur Meteorologie, Germany; Forschungszentrum Julich, Germany; University of Oxford, UK; Finnish Meteorological Institute, Finland; Leibniz Institute for Tropospheric Research, Germany; Center for Climate Systems Modeling (C2SM) at ETH Zurich, Switzerland

  institution_id :

  HAMMOZ-Consortium

  license :

  CMIP6 model data produced by the HAMMOZ-Consortium is licensed under a Creative Commons Attribution ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law.

  mip_era :

  CMIP6

  nominal_resolution :

  250 km

  parent_activity_id :

  CMIP

  parent_experiment_id :

  historical

  parent_mip_era :

  CMIP6

  parent_source_id :

  MPI-ESM-1-2-HAM

  parent_time_units :

  days since 1850-1-1 00:00:00

  parent_variant_label :

  r1i1p1f1

  physics_index :

  1

  product :

  model-output

  project_id :

  CMIP6

  realization_index :

  1

  realm :

  atmos

  references :

  MPI-ESM1.2-HAM: Mauritsen, T. et al. (2019), Developments in the MPI‐M Earth System Model version 1.2 (MPI‐ESM1.2) and Its Response to Increasing CO2, J. Adv. Model. Earth Syst.,11, 998-1038, doi:10.1029/2018MS001400, Tegen, I. et al. (2019), The global aerosol-climate model ECHAM6.3-HAM2.3 Part 1: Aerosol evaluation, Geosci. Model Dev., 12, 16431677, https://doi.org/10.5194/gmd-12-1643-2019

  source :

  MPI-ESM1.2-HAM (2017): aerosol: HAM2.3 atmos: ECHAM6.3 (spectral T63; 192 x 96 longitude/latitude; 47 levels; top level 0.01 hPa) atmosChem: sulfur chemistry (unnamed) land: JSBACH 3.20 landIce: none ocean: MPIOM1.63 (bipolar GR1.5, approximately 1.5deg; 256 x 220 longitude/latitude; 40 levels; top grid cell 0-12 m) ocnBgchem: HAMOCC6 seaIce: unnamed (thermodynamic (Semtner zero-layer) dynamic (Hibler 79) sea ice model)

  source_id :

  MPI-ESM-1-2-HAM

  source_type :

  AOGCM BGC

  status :

  2020-09-24;created; by gcs.cmip6.ldeo@gmail.com

  sub_experiment :

  none

  sub_experiment_id :

  none

  table_id :

  Amon

  table_info :

  Creation Date:(09 May 2019) MD5:dff4d5e7b285678699ef52ab1a3cca43

  title :

  MPI-ESM-1-2-HAM output prepared for CMIP6

  tracking_id :

  hdl:21.14100/b3e723f9-c543-4e42-a3ed-544123882211 hdl:21.14100/ece0e2d4-19ef-4a2a-8eae-a71b7d87f46a hdl:21.14100/3973f668-c855-400e-b6ae-86a746a194d9

  variable_id :

  tasmax

  variant_label :

  r1i1p1f1

  netcdf_tracking_ids :

  hdl:21.14100/b3e723f9-c543-4e42-a3ed-544123882211 hdl:21.14100/ece0e2d4-19ef-4a2a-8eae-a71b7d87f46a hdl:21.14100/3973f668-c855-400e-b6ae-86a746a194d9

  version_id :

  v20190628

  intake_esm_vars :

  ['tasmax']

  intake_esm_attrs:activity_id :

  ScenarioMIP

  intake_esm_attrs:institution_id :

  HAMMOZ-Consortium

  intake_esm_attrs:source_id :

  MPI-ESM-1-2-HAM

  intake_esm_attrs:experiment_id :

  ssp370

  intake_esm_attrs:member_id :

  r1i1p1f1

  intake_esm_attrs:table_id :

  Amon

  intake_esm_attrs:variable_id :

  tasmax

  intake_esm_attrs:grid_label :

  gn

  intake_esm_attrs:zstore :

  gs://cmip6/CMIP6/ScenarioMIP/HAMMOZ-Consortium/MPI-ESM-1-2-HAM/ssp370/r1i1p1f1/Amon/tasmax/gn/v20190628/

  intake_esm_attrs:version :

  20190628

  intake_esm_attrs:_data_format_ :

  zarr

  intake_esm_dataset_key :

  ScenarioMIP.HAMMOZ-Consortium.MPI-ESM-1-2-HAM.ssp370.Amon.gn

import holoviews as hv

def preprocess(ds):
    """
    Converts from 0-360 to -180-180 longitude
    """
    ds.coords['lon'] = (ds.coords['lon'] + 180) % 360 - 180
    return ds.sortby(ds.lon)

def plot_graph(ds, title):
    return ds \
    .tasmax \
    .isel(time=0) \
    .squeeze() \
    .hvplot \
    .quadmesh(x='lon', y='lat', cmap='coolwarm',
              title=title, rasterize=True,
              width=550, height=275)


graphs = []
for (title, ds) in dset_dict.items():
    ds = preprocess(ds)
    graphs.append(plot_graph(ds,title))

hv.Layout([graph for graph in graphs]).cols(2)

[eid for eid in col.df['experiment_id'].unique() if 'ssp' in eid]

['ssp585',
 'ssp245',
 'ssp370SST-lowCH4',
 'ssp370-lowNTCF',
 'ssp370SST-lowNTCF',
 'ssp370SST-ssp126Lu',
 'ssp370SST',
 'ssp370pdSST',
 'ssp119',
 'ssp370',
 'esm-ssp585-ssp126Lu',
 'ssp126-ssp370Lu',
 'ssp370-ssp126Lu',
 'ssp126',
 'esm-ssp585',
 'ssp245-GHG',
 'ssp245-nat',
 'ssp460',
 'ssp434',
 'ssp534-over',
 'ssp245-stratO3',
 'ssp245-aer',
 'ssp245-cov-modgreen',
 'ssp245-cov-fossil',
 'ssp245-cov-strgreen',
 'ssp245-covid',
 'ssp585-bgc']

# there is currently a significant amount of data for these runs
expts = ['historical', 'ssp245', 'ssp585']

query = dict(
    experiment_id=expts,
    table_id='Amon',
    variable_id=['tas'],
    member_id = 'r1i1p1f1',
)

col_subset = col.search(require_all_on=["source_id"], **query)
col_subset.df.groupby("source_id")[
    ["experiment_id", "variable_id", "table_id"]
].nunique()

/usr/share/miniconda/envs/catalogs/lib/python3.8/site-packages/intake_esm/_search.py:80: FutureWarning: In a future version of pandas, a length 1 tuple will be returned when iterating over a groupby with a grouper equal to a list of length 1. Don't supply a list with a single grouper to avoid this warning.
  for _, group in grouped:

	experiment_id	variable_id	table_id
source_id
ACCESS-CM2	3	1	1
AWI-CM-1-1-MR	3	1	1
BCC-CSM2-MR	3	1	1
CAMS-CSM1-0	3	1	1
CAS-ESM2-0	3	1	1
CESM2-WACCM	3	1	1
CIESM	3	1	1
CMCC-CM2-SR5	3	1	1
CMCC-ESM2	3	1	1
CanESM5	3	1	1
E3SM-1-1	3	1	1
EC-Earth3	3	1	1
EC-Earth3-CC	3	1	1
EC-Earth3-Veg	3	1	1
EC-Earth3-Veg-LR	3	1	1
FGOALS-f3-L	3	1	1
FGOALS-g3	3	1	1
FIO-ESM-2-0	3	1	1
GFDL-CM4	3	1	1
GFDL-ESM4	3	1	1
IITM-ESM	3	1	1
INM-CM4-8	3	1	1
INM-CM5-0	3	1	1
IPSL-CM6A-LR	3	1	1
KACE-1-0-G	3	1	1
KIOST-ESM	3	1	1
MIROC6	3	1	1
MPI-ESM1-2-HR	3	1	1
MPI-ESM1-2-LR	3	1	1
MRI-ESM2-0	3	1	1
NESM3	3	1	1
NorESM2-LM	3	1	1
NorESM2-MM	3	1	1
TaiESM1	3	1	1

def drop_all_bounds(ds):
    drop_vars = [vname for vname in ds.coords
                 if (('_bounds') in vname ) or ('_bnds') in vname]
    return ds.drop(drop_vars)

def open_dset(df):
    assert len(df) == 1
    ds = xr.open_zarr(fsspec.get_mapper(df.zstore.values[0]), consolidated=True)
    return drop_all_bounds(ds)

def open_delayed(df):
    return dask.delayed(open_dset)(df)

from collections import defaultdict
dsets = defaultdict(dict)

for group, df in col_subset.df.groupby(by=['source_id', 'experiment_id']):
    dsets[group[0]][group[1]] = open_delayed(df)

dsets_ = dask.compute(dict(dsets))[0]

# calculate global means

def get_lat_name(ds):
    for lat_name in ['lat', 'latitude']:
        if lat_name in ds.coords:
            return lat_name
    raise RuntimeError("Couldn't find a latitude coordinate")

def global_mean(ds):
    lat = ds[get_lat_name(ds)]
    weight = np.cos(np.deg2rad(lat))
    weight /= weight.mean()
    other_dims = set(ds.dims) - {'time'}
    return (ds * weight).mean(other_dims)

expt_da = xr.DataArray(expts, dims='experiment_id', name='experiment_id',
                       coords={'experiment_id': expts})

dsets_aligned = {}

for k, v in tqdm(dsets_.items()):
    expt_dsets = v.values()
    if any([d is None for d in expt_dsets]):
        print(f"Missing experiment for {k}")
        continue

    for ds in expt_dsets:
        ds.coords['year'] = ds.time.dt.year

    # workaround for
    # https://github.com/pydata/xarray/issues/2237#issuecomment-620961663
    dsets_ann_mean = [v[expt].pipe(global_mean)
                             .swap_dims({'time': 'year'})
                             .drop('time')
                             .coarsen(year=12).mean()
                      for expt in expts]

    # align everything with the 4xCO2 experiment
    dsets_aligned[k] = xr.concat(dsets_ann_mean, join='outer',
                                 dim=expt_da)

100%|██████████| 34/34 [00:18<00:00,  1.80it/s]

with progress.ProgressBar():
    dsets_aligned_ = dask.compute(dsets_aligned)[0]

We can quickly choose data subsets in both space and time using xarray. Here, we choose July 19–20, 1996, a period when Quebec saw historically extreme precipitation (Canada). The graphic package hvplot can then be used to track the storm throughout the event.

source_ids = list(dsets_aligned_.keys())
source_da = xr.DataArray(source_ids, dims='source_id', name='source_id',
                         coords={'source_id': source_ids})

big_ds = xr.concat([ds.reset_coords(drop=True)
                    for ds in dsets_aligned_.values()],
                    dim=source_da)

big_ds

<xarray.Dataset>
Dimensions:        (year: 451, experiment_id: 3, source_id: 34)
Coordinates:
  * year           (year) float64 1.85e+03 1.851e+03 ... 2.299e+03 2.3e+03
  * experiment_id  (experiment_id) <U10 'historical' 'ssp245' 'ssp585'
  * source_id      (source_id) <U16 'ACCESS-CM2' 'AWI-CM-1-1-MR' ... 'TaiESM1'
Data variables:
    tas            (source_id, experiment_id, year) float64 287.0 287.0 ... nan

xarray.Dataset

• Dimensions:
  • year: 451
  • experiment_id: 3
  • source_id: 34
• Coordinates: (3)
  • year
    (year)
    float64
    1.85e+03 1.851e+03 ... 2.3e+03

    array([1850., 1851., 1852., ..., 2298., 2299., 2300.])

  • experiment_id
    (experiment_id)
    <U10
    'historical' 'ssp245' 'ssp585'

    array(['historical', 'ssp245', 'ssp585'], dtype='<U10')

  • source_id
    (source_id)
    <U16
    'ACCESS-CM2' ... 'TaiESM1'

    array(['ACCESS-CM2', 'AWI-CM-1-1-MR', 'BCC-CSM2-MR', 'CAMS-CSM1-0',
           'CAS-ESM2-0', 'CESM2-WACCM', 'CIESM', 'CMCC-CM2-SR5', 'CMCC-ESM2',
           'CanESM5', 'E3SM-1-1', 'EC-Earth3', 'EC-Earth3-CC', 'EC-Earth3-Veg',
           'EC-Earth3-Veg-LR', 'FGOALS-f3-L', 'FGOALS-g3', 'FIO-ESM-2-0',
           'GFDL-CM4', 'GFDL-ESM4', 'IITM-ESM', 'INM-CM4-8', 'INM-CM5-0',
           'IPSL-CM6A-LR', 'KACE-1-0-G', 'KIOST-ESM', 'MIROC6', 'MPI-ESM1-2-HR',
           'MPI-ESM1-2-LR', 'MRI-ESM2-0', 'NESM3', 'NorESM2-LM', 'NorESM2-MM',
           'TaiESM1'], dtype='<U16')

• Data variables: (1)
  • tas
    (source_id, experiment_id, year)
    float64
    287.0 287.0 287.2 ... nan nan nan

    array([[[287.00749364, 287.01698452, 287.17454216, ...,          nan,
                      nan,          nan],
            [         nan,          nan,          nan, ...,          nan,
                      nan,          nan],
            [         nan,          nan,          nan, ..., 300.68990761,
             300.73173147, 300.76578769]],
    
           [[         nan, 286.82136175, 286.83013242, ...,          nan,
                      nan,          nan],
            [         nan,          nan,          nan, ...,          nan,
                      nan,          nan],
            [         nan,          nan,          nan, ...,          nan,
                      nan,          nan]],
    
           [[287.53316044, 287.57944112, 287.74630242, ...,          nan,
                      nan,          nan],
            [         nan,          nan,          nan, ...,          nan,
                      nan,          nan],
            [         nan,          nan,          nan, ...,          nan,
                      nan,          nan]],
    ...
           [[287.58305201, 287.80131659, 287.70510569, ...,          nan,
                      nan,          nan],
            [         nan,          nan,          nan, ...,          nan,
                      nan,          nan],
            [         nan,          nan,          nan, ...,          nan,
                      nan,          nan]],
    
           [[287.14579909, 287.16514246, 287.06030532, ...,          nan,
                      nan,          nan],
            [         nan,          nan,          nan, ...,          nan,
                      nan,          nan],
            [         nan,          nan,          nan, ...,          nan,
                      nan,          nan]],
    
           [[286.88855234, 287.0176198 , 287.02504101, ...,          nan,
                      nan,          nan],
            [         nan,          nan,          nan, ...,          nan,
                      nan,          nan],
            [         nan,          nan,          nan, ...,          nan,
                      nan,          nan]]])

• Indexes: (3)
  • year
    PandasIndex

    PandasIndex(Float64Index([1850.0, 1851.0, 1852.0, 1853.0, 1854.0, 1855.0, 1856.0, 1857.0,
                  1858.0, 1859.0,
                  ...
                  2291.0, 2292.0, 2293.0, 2294.0, 2295.0, 2296.0, 2297.0, 2298.0,
                  2299.0, 2300.0],
                 dtype='float64', name='year', length=451))

  • experiment_id
    PandasIndex

    PandasIndex(Index(['historical', 'ssp245', 'ssp585'], dtype='object', name='experiment_id'))

  • source_id
    PandasIndex

    PandasIndex(Index(['ACCESS-CM2', 'AWI-CM-1-1-MR', 'BCC-CSM2-MR', 'CAMS-CSM1-0',
           'CAS-ESM2-0', 'CESM2-WACCM', 'CIESM', 'CMCC-CM2-SR5', 'CMCC-ESM2',
           'CanESM5', 'E3SM-1-1', 'EC-Earth3', 'EC-Earth3-CC', 'EC-Earth3-Veg',
           'EC-Earth3-Veg-LR', 'FGOALS-f3-L', 'FGOALS-g3', 'FIO-ESM-2-0',
           'GFDL-CM4', 'GFDL-ESM4', 'IITM-ESM', 'INM-CM4-8', 'INM-CM5-0',
           'IPSL-CM6A-LR', 'KACE-1-0-G', 'KIOST-ESM', 'MIROC6', 'MPI-ESM1-2-HR',
           'MPI-ESM1-2-LR', 'MRI-ESM2-0', 'NESM3', 'NorESM2-LM', 'NorESM2-MM',
           'TaiESM1'],
          dtype='object', name='source_id'))

• Attributes: (0)

df_all = big_ds.sel(year=slice(1900, 2100)).to_dataframe().reset_index()
df_all.head()

	year	experiment_id	source_id	tas
0	1900.0	historical	ACCESS-CM2	287.019917
1	1900.0	historical	AWI-CM-1-1-MR	286.958154
2	1900.0	historical	BCC-CSM2-MR	287.996260
3	1900.0	historical	CAMS-CSM1-0	287.084974
4	1900.0	historical	CAS-ESM2-0	287.263682

sns.relplot(data=df_all,
            x="year", y="tas", hue='experiment_id',
            kind="line", errorbar='sd', aspect=2);

b) Cordex-NA

col = intake.open_esm_datastore('https://ncar-na-cordex.s3-us-west-2.amazonaws.com/catalogs/aws-na-cordex.json')
col

aws-na-cordex catalog with 330 dataset(s) from 330 asset(s):

	unique
variable	15
standard_name	10
long_name	18
units	10
spatial_domain	1
grid	2
spatial_resolution	2
scenario	6
start_time	3
end_time	4
frequency	1
vertical_levels	1
bias_correction	3
na-cordex-models	9
path	330
derived_variable	0

# Show the first few lines of the catalog
show(col.df,
     tags="<caption>Catalog</caption>",
     column_filters="footer",
     dom="lrtip")

Catalog

variable	standard_name	long_name	units	spatial_domain	grid	spatial_resolution	scenario	start_time	end_time	frequency	vertical_levels	bias_correction	na-cordex-models	path
Loading... (need help?)
variable	standard_name	long_name	units	spatial_domain	grid	spatial_resolution	scenario	start_time	end_time	frequency	vertical_levels	bias_correction	na-cordex-models	path

data_var = 'tmax'

col_subset = col.search(
    variable=data_var,
    grid="NAM-44i",
    bias_correction="raw",
    scenario='rcp45'
)

col_subset

aws-na-cordex catalog with 1 dataset(s) from 1 asset(s):

	unique
variable	1
standard_name	1
long_name	1
units	1
spatial_domain	1
grid	1
spatial_resolution	1
scenario	1
start_time	1
end_time	1
frequency	1
vertical_levels	1
bias_correction	1
na-cordex-models	1
path	1
derived_variable	0

col_subset.df

	variable	standard_name	long_name	units	spatial_domain	grid	spatial_resolution	scenario	start_time	end_time	frequency	vertical_levels	bias_correction	na-cordex-models	path
0	tmax	air_temperature	Daily Maximum Near-Surface Air Temperature	degC	north_america	NAM-44i	0.50 deg	rcp45	2006-01-01T12:00:00	2100-12-31T12:00:00	day	1	raw	['MPI-ESM-LR.CRCM5-UQAM', 'CanESM2.CRCM5-UQAM'...	s3://ncar-na-cordex/day/tmax.rcp45.day.NAM-44i...

# Load catalog entries for subset into a dictionary of xarray datasets, and open the first one.
dsets = col_subset.to_dataset_dict(
    xarray_open_kwargs={"consolidated": True}, storage_options={"anon": True}
)
print(f"\nDataset dictionary keys:\n {dsets.keys()}")

# Load the first dataset and display a summary.
dataset_key = list(dsets.keys())[0]
store_name = dataset_key + ".zarr"

ds = dsets[dataset_key]
ds

# Note that the summary includes a 'member_id' coordinate, which is a renaming of the
# 'na-cordex-models' column in the catalog.

--> The keys in the returned dictionary of datasets are constructed as follows:
        'variable.frequency.scenario.grid.bias_correction'

100.00% [1/1 00:01<00:00]

Dataset dictionary keys:
 dict_keys(['tmax.day.rcp45.NAM-44i.raw'])

<xarray.Dataset>
Dimensions:    (lat: 129, lon: 300, member_id: 6, time: 34698, bnds: 2)
Coordinates:
  * lat        (lat) float64 12.25 12.75 13.25 13.75 ... 74.75 75.25 75.75 76.25
  * lon        (lon) float64 -171.8 -171.2 -170.8 ... -23.25 -22.75 -22.25
  * member_id  (member_id) <U21 'MPI-ESM-LR.CRCM5-UQAM' ... 'CanESM2.CanRCM4'
  * time       (time) datetime64[ns] 2006-01-01T12:00:00 ... 2100-12-31T12:00:00
    time_bnds  (time, bnds) datetime64[ns] dask.array<chunksize=(17349, 2), meta=np.ndarray>
Dimensions without coordinates: bnds
Data variables:
    tmax       (member_id, time, lat, lon) float32 dask.array<chunksize=(4, 1000, 65, 150), meta=np.ndarray>
Attributes: (12/41)
    CORDEX_domain:                        NAM-44
    contact:                              {"MPI-ESM-LR.CRCM5-UQAM": "Winger.K...
    creation_date:                        {"MPI-ESM-LR.CRCM5-UQAM": "2012-09-...
    driving_experiment:                   {"MPI-ESM-LR.CRCM5-UQAM": "MPI-M-MP...
    driving_experiment_name:              rcp45
    driving_model_ensemble_member:        {"MPI-ESM-LR.CRCM5-UQAM": "r1i1p1",...
    ...                                   ...
    intake_esm_attrs:vertical_levels:     1
    intake_esm_attrs:bias_correction:     raw
    intake_esm_attrs:na-cordex-models:    ['MPI-ESM-LR.CRCM5-UQAM', 'CanESM2....
    intake_esm_attrs:path:                s3://ncar-na-cordex/day/tmax.rcp45....
    intake_esm_attrs:_data_format_:       zarr
    intake_esm_dataset_key:               tmax.day.rcp45.NAM-44i.raw

xarray.Dataset

• Dimensions:
  • lat: 129
  • lon: 300
  • member_id: 6
  • time: 34698
  • bnds: 2
• Coordinates: (5)
  • lat
    (lat)
    float64
    12.25 12.75 13.25 ... 75.75 76.25

    long_name :

    latitude

    standard_name :

    latitude

    units :

    degrees_north

    array([12.25, 12.75, 13.25, 13.75, 14.25, 14.75, 15.25, 15.75, 16.25, 16.75,
           17.25, 17.75, 18.25, 18.75, 19.25, 19.75, 20.25, 20.75, 21.25, 21.75,
           22.25, 22.75, 23.25, 23.75, 24.25, 24.75, 25.25, 25.75, 26.25, 26.75,
           27.25, 27.75, 28.25, 28.75, 29.25, 29.75, 30.25, 30.75, 31.25, 31.75,
           32.25, 32.75, 33.25, 33.75, 34.25, 34.75, 35.25, 35.75, 36.25, 36.75,
           37.25, 37.75, 38.25, 38.75, 39.25, 39.75, 40.25, 40.75, 41.25, 41.75,
           42.25, 42.75, 43.25, 43.75, 44.25, 44.75, 45.25, 45.75, 46.25, 46.75,
           47.25, 47.75, 48.25, 48.75, 49.25, 49.75, 50.25, 50.75, 51.25, 51.75,
           52.25, 52.75, 53.25, 53.75, 54.25, 54.75, 55.25, 55.75, 56.25, 56.75,
           57.25, 57.75, 58.25, 58.75, 59.25, 59.75, 60.25, 60.75, 61.25, 61.75,
           62.25, 62.75, 63.25, 63.75, 64.25, 64.75, 65.25, 65.75, 66.25, 66.75,
           67.25, 67.75, 68.25, 68.75, 69.25, 69.75, 70.25, 70.75, 71.25, 71.75,
           72.25, 72.75, 73.25, 73.75, 74.25, 74.75, 75.25, 75.75, 76.25])

  • lon
    (lon)
    float64
    -171.8 -171.2 ... -22.75 -22.25

    long_name :

    longitude

    standard_name :

    longitude

    units :

    degrees_east

    array([-171.75, -171.25, -170.75, ...,  -23.25,  -22.75,  -22.25])

  • member_id
    (member_id)
    <U21
    'MPI-ESM-LR.CRCM5-UQAM' ... 'Can...

    array(['MPI-ESM-LR.CRCM5-UQAM', 'CanESM2.CRCM5-UQAM', 'EC-EARTH.HIRHAM5',
           'EC-EARTH.RCA4', 'CanESM2.RCA4', 'CanESM2.CanRCM4'], dtype='<U21')

  • time
    (time)
    datetime64[ns]
    2006-01-01T12:00:00 ... 2100-12-...

    axis :

    T

    bounds :

    time_bnds

    coordinate_defines :

    point

    delta_t :

    0000-00-01 00:00:00

    long_name :

    time

    standard_name :

    time

    array(['2006-01-01T12:00:00.000000000', '2006-01-02T12:00:00.000000000',
           '2006-01-03T12:00:00.000000000', ..., '2100-12-29T12:00:00.000000000',
           '2100-12-30T12:00:00.000000000', '2100-12-31T12:00:00.000000000'],
          dtype='datetime64[ns]')

  • time_bnds
    (time, bnds)
    datetime64[ns]
    dask.array<chunksize=(17349, 2), meta=np.ndarray>

    Array Chunk Bytes 542.16 kiB 271.08 kiB Shape (34698, 2) (17349, 2) Dask graph 2 chunks in 2 graph layers Data type datetime64[ns] numpy.ndarray

• Data variables: (1)
  • tmax
    (member_id, time, lat, lon)
    float32
    dask.array<chunksize=(4, 1000, 65, 150), meta=np.ndarray>

    cell_methods :

    time: maximum

    long_name :

    Daily Maximum Near-Surface Air Temperature

    remap :

    remapped via ESMF_regrid_with_weights: Higher-order Patch

    standard_name :

    air_temperature

    units :

    degC

    Array Chunk Bytes 30.01 GiB 148.77 MiB Shape (6, 34698, 129, 300) (4, 1000, 65, 150) Dask graph 280 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (4)
  • lat
    PandasIndex

    PandasIndex(Float64Index([12.25, 12.75, 13.25, 13.75, 14.25, 14.75, 15.25, 15.75, 16.25,
                  16.75,
                  ...
                  71.75, 72.25, 72.75, 73.25, 73.75, 74.25, 74.75, 75.25, 75.75,
                  76.25],
                 dtype='float64', name='lat', length=129))

  • lon
    PandasIndex

    PandasIndex(Float64Index([-171.75, -171.25, -170.75, -170.25, -169.75, -169.25, -168.75,
                  -168.25, -167.75, -167.25,
                  ...
                   -26.75,  -26.25,  -25.75,  -25.25,  -24.75,  -24.25,  -23.75,
                   -23.25,  -22.75,  -22.25],
                 dtype='float64', name='lon', length=300))

  • member_id
    PandasIndex

    PandasIndex(Index(['MPI-ESM-LR.CRCM5-UQAM', 'CanESM2.CRCM5-UQAM', 'EC-EARTH.HIRHAM5',
           'EC-EARTH.RCA4', 'CanESM2.RCA4', 'CanESM2.CanRCM4'],
          dtype='object', name='member_id'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2006-01-01 12:00:00', '2006-01-02 12:00:00',
                   '2006-01-03 12:00:00', '2006-01-04 12:00:00',
                   '2006-01-05 12:00:00', '2006-01-06 12:00:00',
                   '2006-01-07 12:00:00', '2006-01-08 12:00:00',
                   '2006-01-09 12:00:00', '2006-01-10 12:00:00',
                   ...
                   '2100-12-22 12:00:00', '2100-12-23 12:00:00',
                   '2100-12-24 12:00:00', '2100-12-25 12:00:00',
                   '2100-12-26 12:00:00', '2100-12-27 12:00:00',
                   '2100-12-28 12:00:00', '2100-12-29 12:00:00',
                   '2100-12-30 12:00:00', '2100-12-31 12:00:00'],
                  dtype='datetime64[ns]', name='time', length=34698, freq=None))

• Attributes: (41)

  CORDEX_domain :

  NAM-44

  contact :

  {"MPI-ESM-LR.CRCM5-UQAM": "Winger.Katja@uqam.ca", "CanESM2.CRCM5-UQAM": "Winger.Katja@uqam.ca", "EC-EARTH.HIRHAM5": "obc@dmi.dk", "EC-EARTH.RCA4": "rossby.cordex@smhi.se", "CanESM2.RCA4": "rossby.cordex@smhi.se", "CanESM2.CanRCM4": "cccma_info@ec.gc.ca"}

  creation_date :

  {"MPI-ESM-LR.CRCM5-UQAM": "2012-09-18", "CanESM2.CRCM5-UQAM": "2012-09-16", "EC-EARTH.HIRHAM5": "2012-11-26 19:05:13", "EC-EARTH.RCA4": "2014-02-07-T15:18:17Z", "CanESM2.RCA4": "2014-02-07-T15:20:21Z", "CanESM2.CanRCM4": "2012-06-29 23:48:29"}

  driving_experiment :

  {"MPI-ESM-LR.CRCM5-UQAM": "MPI-M-MPI-ESM-LR,rcp45,r1i1p1", "CanESM2.CRCM5-UQAM": "CCCma-CanESM2,rcp45,r1i1p1", "EC-EARTH.HIRHAM5": "ICHEC-EC-EARTH,rcp45,r3i1p1", "EC-EARTH.RCA4": "ICHEC-EC-EARTH, rcp45, r12i1p1", "CanESM2.RCA4": "CCCma-CanESM2, rcp45, r1i1p1", "CanESM2.CanRCM4": "CCCma-CanESM2, rcp45, r1i1p1"}

  driving_experiment_name :

  rcp45

  driving_model_ensemble_member :

  {"MPI-ESM-LR.CRCM5-UQAM": "r1i1p1", "CanESM2.CRCM5-UQAM": "r1i1p1", "EC-EARTH.HIRHAM5": "r3i1p1", "EC-EARTH.RCA4": "r12i1p1", "CanESM2.RCA4": "r1i1p1", "CanESM2.CanRCM4": "r1i1p1"}

  driving_model_id :

  {"MPI-ESM-LR.CRCM5-UQAM": "MPI-M-MPI-ESM-LR", "CanESM2.CRCM5-UQAM": "CCCma-CanESM2", "EC-EARTH.HIRHAM5": "ICHEC-EC-EARTH", "EC-EARTH.RCA4": "ICHEC-EC-EARTH", "CanESM2.RCA4": "CCCma-CanESM2", "CanESM2.CanRCM4": "CCCma-CanESM2"}

  experiment_id :

  rcp45

  frequency :

  day

  id :

  doi:10.5065/D6SJ1JCH

  institute_id :

  {"MPI-ESM-LR.CRCM5-UQAM": "UQAM", "CanESM2.CRCM5-UQAM": "UQAM", "EC-EARTH.HIRHAM5": "DMI", "EC-EARTH.RCA4": "SMHI", "CanESM2.RCA4": "SMHI", "CanESM2.CanRCM4": "CCCma"}

  institution :

  {"MPI-ESM-LR.CRCM5-UQAM": "Universite du Quebec a Montreal", "CanESM2.CRCM5-UQAM": "Universite du Quebec a Montreal", "EC-EARTH.HIRHAM5": "Danish Meteorological Institute", "EC-EARTH.RCA4": "Swedish Meteorological and Hydrological Institute, Rossby Centre", "CanESM2.RCA4": "Swedish Meteorological and Hydrological Institute, Rossby Centre", "CanESM2.CanRCM4": "CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, Canada)"}

  model_id :

  {"MPI-ESM-LR.CRCM5-UQAM": "UQAM-CRCM5", "CanESM2.CRCM5-UQAM": "UQAM-CRCM5", "EC-EARTH.HIRHAM5": "DMI-HIRHAM5", "EC-EARTH.RCA4": "SMHI-RCA4", "CanESM2.RCA4": "SMHI-RCA4", "CanESM2.CanRCM4": "CCCma-CanRCM4"}

  original_calendar :

  {"MPI-ESM-LR.CRCM5-UQAM": "gregorian", "CanESM2.CRCM5-UQAM": "365_day", "EC-EARTH.HIRHAM5": "gregorian", "EC-EARTH.RCA4": "standard", "CanESM2.RCA4": "365_day", "CanESM2.CanRCM4": "365_day"}

  product :

  output

  project_id :

  CORDEX

  rcm_version_id :

  {"MPI-ESM-LR.CRCM5-UQAM": "v1", "CanESM2.CRCM5-UQAM": "v1", "EC-EARTH.HIRHAM5": "v1", "EC-EARTH.RCA4": "v1", "CanESM2.RCA4": "v1", "CanESM2.CanRCM4": "r2"}

  references :

  {"MPI-ESM-LR.CRCM5-UQAM": "http://www.mrcc.uqam.ca", "CanESM2.CRCM5-UQAM": "http://www.mrcc.uqam.ca", "EC-EARTH.HIRHAM5": "Christensen, et al. \"The HIRHAM regional climate model. Version 5 (beta).\" (2007).", "EC-EARTH.RCA4": "http://www.smhi.se/en/Research/Research-departments/climate-research-rossby-centre", "CanESM2.RCA4": "http://www.smhi.se/en/Research/Research-departments/climate-research-rossby-centre", "CanESM2.CanRCM4": "http://www.cccma.ec.gc.ca/models"}

  title :

  {"MPI-ESM-LR.CRCM5-UQAM": "NA-CORDEX Raw NAM-44i CRCM5-UQAM MPI-ESM-LR RCP4.5 Daily Mean Daily Max 2m Temperature", "CanESM2.CRCM5-UQAM": "NA-CORDEX Raw NAM-44i CRCM5-UQAM CanESM2 RCP4.5 Daily Mean Daily Max 2m Temperature", "EC-EARTH.HIRHAM5": "NA-CORDEX Raw NAM-44i HIRHAM5 EC-EARTH RCP4.5 Daily Mean Daily Max 2m Temperature", "EC-EARTH.RCA4": "NA-CORDEX Raw NAM-44i RCA4 EC-EARTH RCP4.5 Daily Mean Daily Max 2m Temperature", "CanESM2.RCA4": "NA-CORDEX Raw NAM-44i RCA4 CanESM2 RCP4.5 Daily Mean Daily Max 2m Temperature", "CanESM2.CanRCM4": "NA-CORDEX Raw NAM-44i CanRCM4 CanESM2 RCP4.5 Daily Mean Daily Max 2m Temperature"}

  tracking_id :

  {"MPI-ESM-LR.CRCM5-UQAM": "6c0ec5b6-251c-47b2-9279-85f66bcdd483", "CanESM2.CRCM5-UQAM": "6b99c17d-8f83-41d4-8668-f59eed9aedaf", "EC-EARTH.HIRHAM5": "070fd8dd-b9e0-40d4-946c-a1401a84c00c", "EC-EARTH.RCA4": "0e1fcda3-6fb5-45f2-8b6f-d33f0195d2b0", "CanESM2.RCA4": "f70fd2ca-fbb6-4699-a296-ca7570a7044f", "CanESM2.CanRCM4": "8251d932-c8a5-46d9-b83e-5d86f1fec0df"}

  version :

  {"MPI-ESM-LR.CRCM5-UQAM": "1.2", "CanESM2.CRCM5-UQAM": "2.3", "EC-EARTH.HIRHAM5": "1.2", "EC-EARTH.RCA4": "1.1", "CanESM2.RCA4": "1.1", "CanESM2.CanRCM4": "1.2"}

  zarr-dataset-reference :

  For dataset documentation, see DOI https://doi.org/10.5065/D6SJ1JCH

  zarr-version :

  1.0

  intake_esm_vars :

  ['tmax']

  intake_esm_attrs:variable :

  tmax

  intake_esm_attrs:standard_name :

  air_temperature

  intake_esm_attrs:long_name :

  Daily Maximum Near-Surface Air Temperature

  intake_esm_attrs:units :

  degC

  intake_esm_attrs:spatial_domain :

  north_america

  intake_esm_attrs:grid :

  NAM-44i

  intake_esm_attrs:spatial_resolution :

  0.50 deg

  intake_esm_attrs:scenario :

  rcp45

  intake_esm_attrs:start_time :

  2006-01-01T12:00:00

  intake_esm_attrs:end_time :

  2100-12-31T12:00:00

  intake_esm_attrs:frequency :

  day

  intake_esm_attrs:vertical_levels :

  1

  intake_esm_attrs:bias_correction :

  raw

  intake_esm_attrs:na-cordex-models :

  ['MPI-ESM-LR.CRCM5-UQAM', 'CanESM2.CRCM5-UQAM', 'EC-EARTH.HIRHAM5', 'EC-EARTH.RCA4', 'CanESM2.RCA4', 'CanESM2.CanRCM4']

  intake_esm_attrs:path :

  s3://ncar-na-cordex/day/tmax.rcp45.day.NAM-44i.raw.zarr

  intake_esm_attrs:_data_format_ :

  zarr

  intake_esm_dataset_key :

  tmax.day.rcp45.NAM-44i.raw

ds.tmax \
.sel(lat=45, lon=-75, method='nearest') \
.hvplot(x='time',by='member_id', width=750, height=500, grid=True) \
.opts(legend_position='bottom')