diff --git a/README.md b/README.md index 9d6e393..76db4ec 100644 --- a/README.md +++ b/README.md @@ -44,7 +44,7 @@ This repository holds teaching materials for the NCAS Introduction to Scientific | __xarray:__ Introduction to [multidimensional arrays](https://tutorial.xarray.dev/fundamentals/01_data_structures.html), [xarray data structures](https://tutorial.xarray.dev/fundamentals/01_datastructures.html) and [indexing](https://tutorial.xarray.dev/fundamentals/02.1_indexing_Basic.html) | [Exercise 01a](/python-data/exercises/ex01a_xr_intro.ipynb) [Exercise 01b](/python-data/exercises/ex01b_xr_label_based_indexing.ipynb)| [Solution 01a](/python-data/solutions/ex01a_xr_intro.ipynb) [Solution 0b](/python-data/solutions/ex01b_xr_label_based_indexing.ipynb)| | __xarray:__ [Plotting](https://tutorial.xarray.dev/fundamentals/04.1_basic_plotting.html) and [Aggregation](https://tutorial.xarray.dev/fundamentals/03.1_computation_with_xarray.html) | [Exercise 02a](/python-data/exercises/ex02a_xr_plotting.ipynb) [Exercise 02b](/python-data/exercises/ex02b_xr_aggregation.ipynb)| [Solution 02a](/python-data/solutions/ex02a_xr_plotting.ipynb) [Solution 02b](/python-data/solutions/ex02b_xr_aggregation.ipynb)| | __xarray:__ [GroupBy processing](https://tutorial.xarray.dev/fundamentals/03.2_groupby_with_xarray.html) and [masking](https://tutorial.xarray.dev/intermediate/indexing/boolean-masking-indexing.html) | [Exercise 03a](/python-data/exercises/ex03a_xr_groupby.ipynb) [Exercise 03b](/python-data/exercises/ex03b_xr_masking.ipynb)| [Solution 03a](/python-data/solutions/ex03a_xr_groupby.ipynb) [Solution 03b](/python-data/solutions/ex03b_xr_masking.ipynb)| -| [cf-python]() | [Exercise 04](/python-data/exercises/ex04_cf_python.ipynb) | [Solution 04](/python-data/solutions/ex04_cf_python.ipynb) | +| [cf-python](/python-data/notebooks/intro_to_ncas_cf_data_tools.ipynb) | [Exercise 04](/python-data/exercises/ex04_cf_python.ipynb) | [Solution 04](/python-data/solutions/ex04_cf_python.ipynb) | | [matplotlib](https://matplotlib.org/stable/users/explain/quick_start.html) | [Exercise 05](/python-data/exercises/ex05_matplotlib.ipynb) | [Solution 05](/python-data/solutions/ex05_matplotlib.ipynb) | | [numpy](https://numpy.org/doc/stable/user/quickstart.html) | [Exercise 06](/python-data/exercises/ex06_numpy.ipynb) | [Solution 06](/python-data/solutions/ex06_numpy.ipynb) | | [netCDF4 basics](https://unidata.github.io/netcdf4-python/#tutorial) | [Exercise 07](/python-data/exercises/ex07_netcdf4_basics.ipynb) | [Solution 07](/python-data/solutions/ex07_netcdf4_basics.ipynb) | diff --git a/python-data/README.md b/python-data/README.md index 0a24035..d9ae9fa 100644 --- a/python-data/README.md +++ b/python-data/README.md @@ -12,7 +12,7 @@ Presentation material is used from the links listed below: 1. xarray: [Introduction to multidimensional arrays](https://tutorial.xarray.dev/fundamentals/01_data_structures.html), and [intro to xarray data structures](https://tutorial.xarray.dev/fundamentals/01_datastructures.html) and[label-based indexing](https://tutorial.xarray.dev/fundamentals/02.1_indexing_Basic.html). 2. xarray: [Plotting](https://tutorial.xarray.dev/fundamentals/04.1_basic_plotting.html) and [aggregation](https://tutorial.xarray.dev/fundamentals/03.1_computation_with_xarray.html) 3. xarray: [Groupby processing](https://tutorial.xarray.dev/fundamentals/03.2_groupby_with_xarray.html) and [masking](https://tutorial.xarray.dev/intermediate/indexing/boolean-masking-indexing.html) -4. [cf-python]() +4. [cf-python](./notebooks/intro_to_ncas_cf_data_tools.ipynb) 5. [matplotlib](https://matplotlib.org/stable/users/explain/quick_start.html) 6. [numpy](https://numpy.org/doc/stable/user/quickstart.html) 7. [NetCDF4 basics](https://unidata.github.io/netcdf4-python/#tutorial) diff --git a/python-data/cheat_sheets/cfpython_cheatsheet.pdf b/python-data/cheat_sheets/cfpython_cheatsheet.pdf new file mode 100644 index 0000000..64698b5 Binary files /dev/null and b/python-data/cheat_sheets/cfpython_cheatsheet.pdf differ diff --git a/python-data/data/IPSL-CM5A-LR_r1i1p1_tas_n96_rcp45_mnth.nc b/python-data/data/IPSL-CM5A-LR_r1i1p1_tas_n96_rcp45_mnth.nc new file mode 100755 index 0000000..7e3a981 Binary files /dev/null and b/python-data/data/IPSL-CM5A-LR_r1i1p1_tas_n96_rcp45_mnth.nc differ diff --git a/python-data/data/aaaaoa.pmh8dec.pp b/python-data/data/aaaaoa.pmh8dec.pp new file mode 100755 index 0000000..84f5fc9 Binary files /dev/null and b/python-data/data/aaaaoa.pmh8dec.pp differ diff --git a/python-data/data/alpine_precip_DJF_means.nc b/python-data/data/alpine_precip_DJF_means.nc new file mode 100755 index 0000000..95fe762 Binary files /dev/null and b/python-data/data/alpine_precip_DJF_means.nc differ diff --git a/python-data/data/data1.nc b/python-data/data/data1.nc new file mode 100644 index 0000000..2279cd1 Binary files /dev/null and b/python-data/data/data1.nc differ diff --git a/python-data/data/model_precip_DJF_means_low_res.nc b/python-data/data/model_precip_DJF_means_low_res.nc new file mode 100755 index 0000000..9466974 Binary files /dev/null and b/python-data/data/model_precip_DJF_means_low_res.nc differ diff --git a/python-data/data/precip_1D_monthly.nc b/python-data/data/precip_1D_monthly.nc new file mode 100755 index 0000000..f535d52 Binary files /dev/null and b/python-data/data/precip_1D_monthly.nc differ diff --git a/python-data/data/precip_1D_yearly.nc b/python-data/data/precip_1D_yearly.nc new file mode 100755 index 0000000..51f293a Binary files /dev/null and b/python-data/data/precip_1D_yearly.nc differ diff --git a/python-data/data/precip_2010.nc b/python-data/data/precip_2010.nc new file mode 100755 index 0000000..7d44178 Binary files /dev/null and b/python-data/data/precip_2010.nc differ diff --git a/python-data/data/qbo.nc b/python-data/data/qbo.nc new file mode 100755 index 0000000..b93bbdf Binary files /dev/null and b/python-data/data/qbo.nc differ diff --git a/python-data/data/sea_currents.nc b/python-data/data/sea_currents.nc new file mode 100644 index 0000000..7ad8c3f Binary files /dev/null and b/python-data/data/sea_currents.nc differ diff --git a/python-data/exercises/ex04_cf_python.ipynb b/python-data/exercises/ex04_cf_python.ipynb index e93539d..900884c 100644 --- a/python-data/exercises/ex04_cf_python.ipynb +++ b/python-data/exercises/ex04_cf_python.ipynb @@ -2,10 +2,3188 @@ "cells": [ { "cell_type": "markdown", - "id": "2762201c-7657-4894-9e07-aaa62c49efec", + "id": "0cd6019e-c5b3-4dc6-bf86-6fcf10b06851", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "# Practical for 'introduction to the NCAS CF Data Tools, cf-python and cf-plot'" + ] + }, + { + "cell_type": "markdown", + "id": "1e8d7e64-fd9a-4e9b-a369-61d5635cbb62", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "
\n", + "Practical instructions: these green boxes provide instructions and tips about doing this practical (blue boxes are the same as in the teaching notebook and provide useful information). As guidance and for reference, the following are provided below before the practical material starts:\n", + "\n", + "
\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "c4378ad0-8448-44ef-80e2-cd6e67346a1b", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "## A reminder: context, learning objectives and guidance links\n", + "\n", + "### What are the NCAS CF Data Tools and why do they all have 'cf' in the name?\n", + "\n", + "The _NCAS CF Data Tools_ are a suite of complementary Python libraries which are designed to facilitate working with data for research in the earth sciences and aligned domains. The two that are of most relevance to the average user, and those wanting to process, analyse and visualise atmospheric data, are *cf-python* (https://ncas-cms.github.io/cf-python/) and *cf-plot* (https://ncas-cms.github.io/cf-plot/build/). We will be focusing on use of cf-python and cf-plot today.\n", + "\n", + "The 'cf' in the names of the NCAS CF Data Tools corresponds to the _CF Conventions_, a metadata standard, because they are built around this standard in the form of using the CF Data Model, which as well as performance is considered a 'unique selling point' of the tools.\n", + "\n", + "\n", + "### What are the CF Conventions?\n", + "\n", + "The _CF Conventions_, usually referred to in this way but also know by the full name of the **C**limate and **F**orecast (CF) metadata conventions, are a metadata standard which is becoming the de-facto convention to cover the description of geoscientific data so that sharing and intercomparison is simpler. See https://cfconventions.org/ for more information.\n", + "\n", + "\n", + "### What are we going to learn in this session?\n", + "\n", + "Our **learning aim** is to be able to use the NCAS CF Data Tools Python libraries, namely cf-python and cf-plot to process, analyse and visualise netCDF and PP datasets, whilst appreciating the context and 'unique selling point' of the libraries as being built to use the CF Conventions, a metadata standard for earth science data, to make it simpler to do what you want to do with the datasets, by working on top of a Data Model for CF.\n", + "\n", + "We have **six distinct objectives**, matching the sections in this notebook and in the practical notebook you will work through. By the end of this lesson you should be familiar and have practiced using cf-python and cf-plot to:\n", + "\n", + "1. read dataset(s) and view the (meta)data at different detail levels;\n", + "2. edit the (meta)data and write out the edited version to file;\n", + "3. reduce datasets by subspacing and collapsing;\n", + "4. visualise datasets as contour and vector plots;\n", + "5. analyse data: applying mathematical and statistical operations and plotting trends;\n", + "6. change the underlying grid of data through regridding." + ] + }, + { + "cell_type": "markdown", + "id": "26aa2a28-22ce-4e9f-bc1e-6b0d76ab807c", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "### Guidance: where to find more information and resources on the NCAS CF Data Tools\n", + "\n", + "Here are some links relating to the NCAS CF Data Tools and this training.\n", + "\n", + "* This training, with further material, is hosted online and there are instructions for setting up the environment so you can work through it in your own time: https://github.com/NCAS-CMS/cf-tools-training.\n", + "* **The cf-python documentation lives at https://ncas-cms.github.io/cf-python/.**\n", + "* The cf-python code lives on GitHub at https://github.com/NCAS-CMS/cf-python. There is an Issue Tracker to report queries or questions at https://github.com/NCAS-CMS/cf-python/issues.\n", + "* **The cf-plot documentation lives at https://ncas-cms.github.io/cf-plot/build/.**\n", + "* The cf-plot code lives on GitHub at https://github.com/NCAS-CMS/cf-plot. There is an Issue Tracker to report queries or questions at https://github.com/NCAS-CMS/cf-plot/issues.\n", + "* There is a technical presentation about the NCAS CF Data Tools avaialble from https://hps.vi4io.org/_media/events/2020/summer-school-cfnetcdf.pdf.\n", + "* The website of the CF Conventions can be found at https://cfconventions.org/.\n", + "* The landing page for training into the CF Conventions is found here within the website above: https://cfconventions.org/Training/.\n", + "\n", + "If you have any queries after this course, please either use the Issue Trackers linked above or you can email me at: sadie.bartholomew@ncas.ac.uk." + ] + }, + { + "cell_type": "markdown", + "id": "7ca3e79a-e39f-4a04-9dc5-3b56036bef5e", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "
\n", + "Note: In cf-python and when discussing related code and datasets, we use terminology from the CF Data Model (for more detail see: https://ncas-cms.github.io/cf-python/cf_data_model.html). For example cf-python methods are named in relation to concepts from this data model. We don't have time to cover this in detail but for this session it is useful to know the following terms:\n", + "\n", + "\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "1c3beb87-7b0d-44ad-9da3-fa32494b795c", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "df666543-52ee-402a-8fa4-cb4d8389f3a8", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "
\n", + "Practical instructions: run all of the cells in this section to do the set up.\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "196e9e0a-83d1-464c-b53c-a6fc2e0c19ff", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "## Setting up\n", + "\n", + "**In this section we set up this Notebook, import the libraries and check the data we will work with, ready to use the libraries within this notebook.**" + ] + }, + { + "cell_type": "markdown", + "id": "d073760a-6734-4a1d-9832-a70a8e0dd089", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "Import cf-python and cf-plot:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "b4b0c984-1adc-4515-b192-19da265e1f9a", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:07:56.258910Z", + "iopub.status.busy": "2024-11-20T17:07:56.258287Z", + "iopub.status.idle": "2024-11-20T17:08:02.134577Z", + "shell.execute_reply": "2024-11-20T17:08:02.133461Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "import cfplot as cfp\n", + "import cf" + ] + }, + { + "cell_type": "markdown", + "id": "6cfa3e24-6400-4de0-b155-aab85f701441", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "Run some set up for nice outputs in this Jupyter Notebook (not required in interactive Python or a script):" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "aadf1e3e-382f-47c5-aeee-b52f3cd2868d", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:02.141681Z", + "iopub.status.busy": "2024-11-20T17:08:02.141294Z", + "iopub.status.idle": "2024-11-20T17:08:02.154674Z", + "shell.execute_reply": "2024-11-20T17:08:02.153959Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "import warnings\n", + "warnings.filterwarnings('ignore')\n", + "\n", + "%matplotlib inline" + ] + }, + { + "cell_type": "markdown", + "id": "7b00d47b-4497-4540-b1bd-6fa8fa8e3226", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "Inspect the versions of cf-python and cf-plot and the version of the CF Conventions those are matched to:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "ea75b1b6-ba78-428d-b9b2-ca29a8cf6f0d", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:02.158400Z", + "iopub.status.busy": "2024-11-20T17:08:02.158170Z", + "iopub.status.idle": "2024-11-20T17:08:02.165345Z", + "shell.execute_reply": "2024-11-20T17:08:02.164431Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "cf-python version is: 3.16.2\n", + "cf-plot version is: 3.3.0\n", + "CF Conventions version is: 1.11\n" + ] + } + ], + "source": [ + "print(\"cf-python version is:\", cf.__version__)\n", + "print(\"cf-plot version is:\", cfp.__version__)\n", + "print(\"CF Conventions version is:\", cf.CF())" + ] + }, + { + "cell_type": "markdown", + "id": "628af6cc-24f1-4e85-bf8b-654268e72bd3", + "metadata": {}, + "source": [ + "
\n", + "Note: you can work with data compliant by any other version of the CF Conventions, or without (much) compliance, but the CF Conventions version gives the maximum version that these versions of the tools understand the features of.\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "8135f5c6-3d3e-45b6-a84a-3dcea754372f", + "metadata": {}, + "source": [ + "Finally, see what datasets we have to explore:" + ] + }, + { + "cell_type": "markdown", + "id": "cfab63fe-088d-4ca8-ab95-40f61f0c2fd3", + "metadata": {}, + "source": [ + "
\n", + "Note: in a Jupyter Notebook, '!' preceeeds a shell command, so this is a terminal command and not Python\n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "a45afce7-abd8-4edd-b14c-eb033566bd52", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:02.170515Z", + "iopub.status.busy": "2024-11-20T17:08:02.170278Z", + "iopub.status.idle": "2024-11-20T17:08:03.224743Z", + "shell.execute_reply": "2024-11-20T17:08:03.222658Z" + }, + "scrolled": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "IPSL-CM5A-LR_r1i1p1_tas_n96_rcp45_mnth.nc precip_2010.nc\r\n", + "aaaaoa.pmh8dec.pp\t\t\t qbo.nc\r\n", + "alpine_precip_DJF_means.nc\t\t sea_currents.nc\r\n", + "cv-noxy_capeverde_20080301.nc\t\t sensor_data.nc\r\n", + "data1.nc\t\t\t\t tas.nc\r\n", + "ggas2014121200_00-18.nc\t\t\t tas_hadcm3_cf.nc\r\n", + "model_precip_DJF_means_low_res.nc\t tas_rcp45_2055_ann_avg_change.nc\r\n", + "n2o_emissions.nc\t\t\t tas_rcp45_2055_mon_avg_change.nc\r\n", + "precip_1D_monthly.nc\t\t\t xbhubo.pgc0apr.nc\r\n", + "precip_1D_yearly.nc\r\n" + ] + } + ], + "source": [ + "!ls ../data" + ] + }, + { + "cell_type": "markdown", + "id": "e7640cdb-0d6f-4933-86d7-e86e7b9acbe4", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "ef80e6de-8e62-468b-93c8-b41ac768375a", + "metadata": {}, + "source": [ + "
\n", + "Practical instructions: now we can start the practical. We will follow the same sectioning as in the teaching notebook, so please consult the notes there in the matching section for guidance and you can also consult the cf-python and cf-plot documentation linked above.\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "3b7a3c0a-262d-4ac4-8c10-6fa3a42ae73a", + "metadata": {}, + "source": [ + "## 1. Reading dataset(s) and viewing the (meta)data at different detail levels\n", + "\n", + "**In this section we look at a basic use of cf-python, reading in one or more datasets from file and inspecting the data and the metadata at different levels of detail to suit the amount of information you want to see.**" + ] + }, + { + "cell_type": "markdown", + "id": "9da08bf9-a15b-4d65-8c8e-632897daec0b", + "metadata": {}, + "source": [ + "### a) Reading in data and extracting the _field_ of interest" + ] + }, + { + "cell_type": "markdown", + "id": "45be7d09-fe3e-46b3-b35b-dec82b32cc01", + "metadata": {}, + "source": [ + "**1.a.1)** Use `cf` to read in the netCDF dataset `qbo.nc` which is found (as shown at the end of the section above) under the directory `../data`, assigning it to a variable called 'fieldlist'.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "d0a43e35-8627-4a08-ae1e-4fa9ec4f400d", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:03.231603Z", + "iopub.status.busy": "2024-11-20T17:08:03.230895Z", + "iopub.status.idle": "2024-11-20T17:08:03.421103Z", + "shell.execute_reply": "2024-11-20T17:08:03.420285Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "674d8543-fb55-4b2e-bd26-5a006dc9181d", + "metadata": {}, + "source": [ + "**1.a.2)** Use the standard Python function `len` to see how long the read-in fieldlist is." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "0e59c1a6-53ea-43c2-bf57-a405d1f64be9", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:03.426397Z", + "iopub.status.busy": "2024-11-20T17:08:03.426177Z", + "iopub.status.idle": "2024-11-20T17:08:03.432752Z", + "shell.execute_reply": "2024-11-20T17:08:03.432139Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "9dfdb8c4-ac20-4b0c-87b6-5bc3829dee61", + "metadata": {}, + "source": [ + "**1.a.3)** Access the first field in the fieldlist and assign it to the variable name 'field'." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "d92286c4-e62e-4e35-a934-809bc1afe02d", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:03.437468Z", + "iopub.status.busy": "2024-11-20T17:08:03.436839Z", + "iopub.status.idle": "2024-11-20T17:08:03.445834Z", + "shell.execute_reply": "2024-11-20T17:08:03.445204Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "d4ed8112-fc35-496c-84ae-e8a05334c33c", + "metadata": {}, + "source": [ + "### b) Inspecting the _field_ of interest with different amounts of detail" + ] + }, + { + "cell_type": "markdown", + "id": "9e3ad7d7-2657-43d4-95a4-601429b16575", + "metadata": {}, + "source": [ + "**1.b.1)** View the field from (1.a.3) above in minimal detail." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "41528fdd-7f56-4801-95b9-c92b910e3a7d", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:03.449091Z", + "iopub.status.busy": "2024-11-20T17:08:03.448690Z", + "iopub.status.idle": "2024-11-20T17:08:03.466397Z", + "shell.execute_reply": "2024-11-20T17:08:03.465102Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "8a158a41-03f8-47f9-a84e-fd2eaac8a132", + "metadata": {}, + "source": [ + "**1.b.2)** Now try viewing the field from (1.a.3) above at a medium detail level." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "0916e895-c503-47b8-a280-62404bb4d4a1", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:03.470671Z", + "iopub.status.busy": "2024-11-20T17:08:03.470090Z", + "iopub.status.idle": "2024-11-20T17:08:04.234425Z", + "shell.execute_reply": "2024-11-20T17:08:04.233401Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "4653be8e-e953-4a75-a343-93413cee09a6", + "metadata": {}, + "source": [ + "**1.b.3)** OK, finally let's see it in its full glory - with maximal detail. Take a minute or two to compare these outputs and familiarise yourself with the formats of the different views and how they present the metadata (and preview of the data) of a field." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "b07e729d-3ecd-4c1e-b26b-9ff61fc04a70", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.239027Z", + "iopub.status.busy": "2024-11-20T17:08:04.238816Z", + "iopub.status.idle": "2024-11-20T17:08:04.368597Z", + "shell.execute_reply": "2024-11-20T17:08:04.367331Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "0af01357-b0b2-4581-a830-437337565d0d", + "metadata": {}, + "source": [ + "### c) Inspecting a metadata _construct_ e.g. _coordinate_ from the _field_ of interest" + ] + }, + { + "cell_type": "markdown", + "id": "03ce0882-35c1-4f41-9280-33d8b633572b", + "metadata": {}, + "source": [ + "**1.c.1)** Let's assume we want to know about a specific metadata construct, in this case we are intereted in the pressure. Assign to a new variable called 'pressure' the pressure coordinate of the field stored in the variable 'field' from section (1a) as just inspected in section (1b)." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "99b9db17-713f-47c9-82a6-ec9af2213212", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.373344Z", + "iopub.status.busy": "2024-11-20T17:08:04.373048Z", + "iopub.status.idle": "2024-11-20T17:08:04.378587Z", + "shell.execute_reply": "2024-11-20T17:08:04.377522Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "bf10ee1b-d4eb-43ac-839f-1f8066e703ae", + "metadata": {}, + "source": [ + "**1.c.2)** View this coordinate with minimal detail level." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "753ff7ac-2de7-49c7-b501-aee40ddb9090", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.382714Z", + "iopub.status.busy": "2024-11-20T17:08:04.382087Z", + "iopub.status.idle": "2024-11-20T17:08:04.397739Z", + "shell.execute_reply": "2024-11-20T17:08:04.396394Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "23fe4e99-326b-4209-b905-8ca211582b08", + "metadata": {}, + "source": [ + "**1.c.3)** Now use the standard approach to view it with medium detail level." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "ef442fa1-bfe4-4b5c-bfc7-4453c0d32e08", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.401735Z", + "iopub.status.busy": "2024-11-20T17:08:04.401356Z", + "iopub.status.idle": "2024-11-20T17:08:04.411020Z", + "shell.execute_reply": "2024-11-20T17:08:04.409991Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "9b03ffb4-62d7-4e08-a1e5-28591c3fb602", + "metadata": {}, + "source": [ + "**1.c.4)** Finally, let's use the approach for full detail level and see everything about this coordinate." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "ee0cb3a0-5129-456e-8e9e-5f24a598d34d", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.416175Z", + "iopub.status.busy": "2024-11-20T17:08:04.415813Z", + "iopub.status.idle": "2024-11-20T17:08:04.426404Z", + "shell.execute_reply": "2024-11-20T17:08:04.424952Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "bd7fd831-8362-49eb-a03a-281d75596309", + "metadata": {}, + "source": [ + "### d) Inspecting a data array of interest" + ] + }, + { + "cell_type": "markdown", + "id": "16b595ee-6e54-40ec-ba3f-cc28f710eab1", + "metadata": {}, + "source": [ + "**1.d.1)** Access the underlying data of the pressure coordinate from the previous sub-section, (1c), assigning it to a variable called 'pressure_data'." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "f2f748b5-09c3-4382-a6d6-e50a31fca61f", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.431307Z", + "iopub.status.busy": "2024-11-20T17:08:04.430852Z", + "iopub.status.idle": "2024-11-20T17:08:04.441446Z", + "shell.execute_reply": "2024-11-20T17:08:04.440341Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "d7855240-2667-4883-a3fd-8d8b1759122a", + "metadata": {}, + "source": [ + "**1.d.2)** Inspect the pressure coordinate data with minimal detail, noticing the units." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "539107da-25fe-4259-9fb2-0f6419954424", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.445970Z", + "iopub.status.busy": "2024-11-20T17:08:04.445336Z", + "iopub.status.idle": "2024-11-20T17:08:04.460804Z", + "shell.execute_reply": "2024-11-20T17:08:04.459326Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "7288c197-fd6e-405e-b49a-6740bc41992e", + "metadata": {}, + "source": [ + "**1.d.3)** Access the data array of the pressure coordinate. Note that, because it is small, it is not computationally expensive to access this and similarly with other metadata data arrays, but accessing the underlying data array of the whole field (i.e. its main stored variable) could be intensive because for datasets in real usage the data can be very large and/or multi-dimensional." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "e25482db-8342-49aa-bdc1-0911e842759d", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.466289Z", + "iopub.status.busy": "2024-11-20T17:08:04.465723Z", + "iopub.status.idle": "2024-11-20T17:08:04.492133Z", + "shell.execute_reply": "2024-11-20T17:08:04.490549Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "2e2a3d9a-31ec-4f70-964c-212deb4709cc", + "metadata": {}, + "source": [ + "**1.d.4)** Use the standard Python `print` function to view the pressure array." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "aba215ff-f5a7-4b97-9bdb-5a80961cd38f", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.497943Z", + "iopub.status.busy": "2024-11-20T17:08:04.497431Z", + "iopub.status.idle": "2024-11-20T17:08:04.506361Z", + "shell.execute_reply": "2024-11-20T17:08:04.504999Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "09d04ff2-8998-48fb-a0a6-bc5deafe8fc6", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "932d2d38-b8d7-46cc-848f-b6f50b877310", + "metadata": {}, + "source": [ + "## 2. Editing the (meta)data and writing out the edited version to file\n", + "\n", + "**In this section we demonstrate how to change the data that has been read-in from file, both in terms of the data arrays and the metadata that describes it, and then how to write data back out to file with a chosen name, so that you can see how cf-python can be used to edit data or to make new data.**" + ] + }, + { + "cell_type": "markdown", + "id": "bc7c654e-38e5-4b06-a7c0-01828701d92f", + "metadata": {}, + "source": [ + "### a) Changing the underlying data" + ] + }, + { + "cell_type": "markdown", + "id": "1d6b6412-42b4-4d70-9df2-1fd6b0349a56", + "metadata": {}, + "source": [ + "**2.a.1)** Access the data (*not* the data array of the data) of the full field and assign it to a variable called 'data'." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "ad3da187-8515-4a84-95e4-a9c1190dd08d", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.511512Z", + "iopub.status.busy": "2024-11-20T17:08:04.510936Z", + "iopub.status.idle": "2024-11-20T17:08:04.518567Z", + "shell.execute_reply": "2024-11-20T17:08:04.517486Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "b625ea77-a1db-4598-b1c4-6bba6319d1e0", + "metadata": {}, + "source": [ + "**2.a.2)** Inspect the field data with medium detail use the `size` method on it to see its shape." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "4b6e33da-fad2-49a8-86e2-e94331e7e8dd", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.523330Z", + "iopub.status.busy": "2024-11-20T17:08:04.522948Z", + "iopub.status.idle": "2024-11-20T17:08:04.536199Z", + "shell.execute_reply": "2024-11-20T17:08:04.534780Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "86474dfe-a2ac-4b52-a962-eda937b0f58d", + "metadata": {}, + "source": [ + "**2.a.3)** Use the `size` method on it to see how many items (in this case, numbers) there are in it. Can you see how this relates to the `shape` above, and to the structure of the coordinates from the field inspection in (1.b.2)?" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "id": "caaf9db2-c33d-434b-a3fe-ad91e9fc681e", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.540384Z", + "iopub.status.busy": "2024-11-20T17:08:04.539826Z", + "iopub.status.idle": "2024-11-20T17:08:04.547800Z", + "shell.execute_reply": "2024-11-20T17:08:04.547044Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "cf00a0cd-ab33-4c08-94d0-a7c25a2df606", + "metadata": {}, + "source": [ + "**2.a.4)**" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "394dc1e4-3e88-4370-b568-27f031fe1d5d", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.551373Z", + "iopub.status.busy": "2024-11-20T17:08:04.550854Z", + "iopub.status.idle": "2024-11-20T17:08:04.634299Z", + "shell.execute_reply": "2024-11-20T17:08:04.633443Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "8a97f4ab-dfba-4879-ac78-9c0edb36cf2d", + "metadata": {}, + "source": [ + "**2.a.5)** Change all of the values in the first time subarray to the value '-50.0'." + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "c167273d-16b2-4ea8-bc74-be8ac28d9820", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.638835Z", + "iopub.status.busy": "2024-11-20T17:08:04.638616Z", + "iopub.status.idle": "2024-11-20T17:08:04.721739Z", + "shell.execute_reply": "2024-11-20T17:08:04.720894Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "5693c393-4c96-4841-8e4c-f336f9bfebb6", + "metadata": {}, + "source": [ + "**2.a.6)** Access the index item `[1, :, 0, 0]` of the full data array from (2.a.1) and assign it to a variable called 'a_subarray'. Then check what shape it is and try to understand the size that emerges for that sub-array given that specific index." + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "beb5f4b6-5b5e-4761-8c66-2096edc8bfde", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.727137Z", + "iopub.status.busy": "2024-11-20T17:08:04.726900Z", + "iopub.status.idle": "2024-11-20T17:08:04.734753Z", + "shell.execute_reply": "2024-11-20T17:08:04.733471Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "b9f9e1b8-1951-4545-94c1-9d567a0722d5", + "metadata": {}, + "source": [ + "**2.a.7)** Change the values for this sub-array item to all ones i.e. `1.0`. Note to create an array populated all with the value one of the required shape to match the shape of the subarray from (2.a.6), you can use `numpy.ones()` (you will need to import `numpy` first, and let's call the module `np` i.e. use `import numpy as np`)." + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "id": "cd65f487-c6b0-45e0-b093-1ac39f11b2d7", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.739713Z", + "iopub.status.busy": "2024-11-20T17:08:04.738841Z", + "iopub.status.idle": "2024-11-20T17:08:04.750701Z", + "shell.execute_reply": "2024-11-20T17:08:04.749307Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "fe2fe7a5-73b8-4021-8748-e88b3b0d6e5c", + "metadata": {}, + "source": [ + "**2.a.8)** Inspect 'a_subarray' again to confirm it has been set with values of all one." + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "id": "5629a7f3-5fd8-4bfd-a6d7-eb141951a2c7", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.754618Z", + "iopub.status.busy": "2024-11-20T17:08:04.754149Z", + "iopub.status.idle": "2024-11-20T17:08:04.845731Z", + "shell.execute_reply": "2024-11-20T17:08:04.844878Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "0f2d2721-58c2-44d9-b422-3dd81ee8b333", + "metadata": {}, + "source": [ + "### b) Changing some metadata" + ] + }, + { + "cell_type": "markdown", + "id": "ad17957e-9d70-4fcb-9954-e0ff331ac689", + "metadata": {}, + "source": [ + "**2.b.1)** Take the 'pressure' variable coordinate we created in (1.c) and inspected in full detail in (1.c.4). Let's say we want to reverse the direction of this axes, so that the pressures increase in value along it (in order) rather than decreases. In `cf` we can do this using the `flip` method, which will adjust the data accordingly so that it is flipped in the corresponding way to match up to the new direction of the pressure axis.\n", + "\n", + "Reassign `pressure` to itself with the `flip` method applied (note you can, alternatively and equivalently, call the method `flip` with the argument `inplace=True` set without re-assignment because that argument tells `cf` to adjust the variable in-place)." + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "id": "f08327b2-51a1-404b-97fa-e753255eb252", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.850586Z", + "iopub.status.busy": "2024-11-20T17:08:04.850387Z", + "iopub.status.idle": "2024-11-20T17:08:04.855369Z", + "shell.execute_reply": "2024-11-20T17:08:04.854506Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "f240a543-5676-439e-b53e-4384ee5d9171", + "metadata": {}, + "source": [ + "**2.b.2)** Now full inspect the pressure coordinate again with full detail to confirm that the flip of axis direction happened." + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "id": "d4311a56-3252-4b10-9b92-01f00c7e1cdb", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.858774Z", + "iopub.status.busy": "2024-11-20T17:08:04.858427Z", + "iopub.status.idle": "2024-11-20T17:08:04.897379Z", + "shell.execute_reply": "2024-11-20T17:08:04.896240Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "20031178-9faa-4501-b00f-70b0d1c1ae64", + "metadata": {}, + "source": [ + "**2.b.3)** Notice the `standard_name`, which is a CF attribute used to identify the physical quantity in question, that must be one from the thousands of controlled options available in the CF Standard Name table (see https://cfconventions.org/Data/cf-standard-names/current/build/cf-standard-name-table.html, you are encouraged to spend some minutes exploring this resource). At the moment the value in the field is not a valid standard name and it is also not specific enough to identify the variable well. \n", + "\n", + "Let's change it to the more descriptive valid CF Standard Name of 'air_pressure'. To do this, access the `standard_name` attribute of the pressure coordinate and re-set it to the string value desired." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "id": "a6bfddb6-223a-46b0-8adf-725361fe1c64", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.903580Z", + "iopub.status.busy": "2024-11-20T17:08:04.902903Z", + "iopub.status.idle": "2024-11-20T17:08:04.908590Z", + "shell.execute_reply": "2024-11-20T17:08:04.907629Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "d0e0d297-4e71-4a73-a194-f9b151f7a4a3", + "metadata": {}, + "source": [ + "**2.b.4)** Once again view the pressure coordinate with maximal detail to see the two changes we made in this section. Also view the overall field in medium detail to see that these changes propagate through to the field object by Python object rules." + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "id": "155bf824-7ebc-4178-9377-7be91b03393f", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.913103Z", + "iopub.status.busy": "2024-11-20T17:08:04.912610Z", + "iopub.status.idle": "2024-11-20T17:08:04.924621Z", + "shell.execute_reply": "2024-11-20T17:08:04.923383Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "cf7cf826-2d23-40cd-bfca-648de59fd02c", + "metadata": {}, + "source": [ + "### c) Writing a (list of) fields out to a file" + ] + }, + { + "cell_type": "markdown", + "id": "f375ce40-ba55-45cd-83d2-9c5f58df3506", + "metadata": {}, + "source": [ + "**2.c.1)** Let's write out a few fields to disk. We'll write our field from the previous sections as well as a new field all into the same file.\n", + "\n", + "First let's get a new field. We could read in another file to access some new fields, but let's use one of the cf-python ready-made 'example fields', which are designed for exploring cf-python capability without the need for real (and large/memory-intensive) data. Run `cf.example_field()` with any integer from 0 to 11 inclusive to get an example field, assigning it to the variable called 'new_field'." + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "id": "5fb3f0c6-6638-4535-8729-b50d1f70fc29", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.928898Z", + "iopub.status.busy": "2024-11-20T17:08:04.928383Z", + "iopub.status.idle": "2024-11-20T17:08:04.967644Z", + "shell.execute_reply": "2024-11-20T17:08:04.966671Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "3e061bbd-836c-44fb-9614-358585b4c1bf", + "metadata": {}, + "source": [ + "**2.c.2)** Inspect this new field with medium level of detail." + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "id": "4b199ddb-00c6-456e-bb51-58117037c50b", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:04.972642Z", + "iopub.status.busy": "2024-11-20T17:08:04.972443Z", + "iopub.status.idle": "2024-11-20T17:08:05.003772Z", + "shell.execute_reply": "2024-11-20T17:08:05.002950Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "19c295d5-9e45-49a8-bc53-fd9a139aacf8", + "metadata": {}, + "source": [ + "**2.c.3)** Create a two-field FieldList called 'new_fieldlist' containing the field from (1.a.3) that we have been exploring in the previous sections (1.a) and (1.b) along with this new 'new_field' from (2.c.2). You can use the `cf` function `cf.FieldList` with the fields it should contain in a normal Python list as an arugment to set this up." + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "id": "a55c4a55-10bd-4269-a232-32e0d3737a94", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:05.008317Z", + "iopub.status.busy": "2024-11-20T17:08:05.008097Z", + "iopub.status.idle": "2024-11-20T17:08:05.012573Z", + "shell.execute_reply": "2024-11-20T17:08:05.011897Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "c155c131-5367-497f-8590-16bbfdc7c124", + "metadata": {}, + "source": [ + "**2.c.4)** Call `print` with this new fieldlist as an argument to view it in medium detail." + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "id": "d2c71af9-a032-458a-b091-39d776c11025", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:05.015570Z", + "iopub.status.busy": "2024-11-20T17:08:05.015382Z", + "iopub.status.idle": "2024-11-20T17:08:05.028925Z", + "shell.execute_reply": "2024-11-20T17:08:05.027926Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "aee0230d-681c-476e-800e-fa2f83b419ea", + "metadata": {}, + "source": [ + "**2.c.4)** Now we have constructed our desired FieldList, we can write it out to file. Use `cf`'s `write` function to write it to a file called `two_fields.nc` in the `../data` directory." + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "id": "c9775ffb-6830-4102-b3b0-eb11403ad806", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:05.031746Z", + "iopub.status.busy": "2024-11-20T17:08:05.031430Z", + "iopub.status.idle": "2024-11-20T17:08:05.361032Z", + "shell.execute_reply": "2024-11-20T17:08:05.360407Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "fb5aabd2-6e6d-4e28-8e9f-3da1537d981a", + "metadata": {}, + "source": [ + "**2.c.5)** Use the shell command `!ls ../data` to see the contents of that directory to confirm that the file was written there." + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "id": "9f535282-e936-4254-9bc5-fc096bf69e6b", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:05.365590Z", + "iopub.status.busy": "2024-11-20T17:08:05.365298Z", + "iopub.status.idle": "2024-11-20T17:08:06.409829Z", + "shell.execute_reply": "2024-11-20T17:08:06.408062Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "9dd6be4b-60d3-4ca8-8108-fcda62bb7d95", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "4ccfd6a0-567d-43f3-9f06-15bb21afae6a", + "metadata": {}, + "source": [ + "## 3. Reducing datasets by subspacing and collapsing\n", + "\n", + "**In this section we show how multi-dimensional data can be tamed using cf-python so that you can get a reduced form that can be analysed or plotted, by reducing the dimensions by selecting a subset of point(s) along the axes or collapsing down according to some statistic such as the mean or an extrema.**" + ] + }, + { + "cell_type": "markdown", + "id": "410ac6d4-cd8f-43ed-a3f5-eaf8d1d8755a", + "metadata": {}, + "source": [ + "### a) Subspacing using metadata conditions" + ] + }, + { + "cell_type": "markdown", + "id": "cec90546-f200-42b3-af27-6508f367d305", + "metadata": {}, + "source": [ + "**3.a.1)** Read in the file `ggas2014121200_00-18.nc` which is under `../data` and save the corresponding FieldList to a variable called `fieldlist_3`. Inspect it with medium level of detail." + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "id": "a4af5c1c-cdc4-441d-8d2c-17d9cfd6b746", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:06.417881Z", + "iopub.status.busy": "2024-11-20T17:08:06.417161Z", + "iopub.status.idle": "2024-11-20T17:08:06.488448Z", + "shell.execute_reply": "2024-11-20T17:08:06.487599Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "3077b7a8-6f93-4a95-ad6d-58e8b3fc15ec", + "metadata": {}, + "source": [ + "**3.a.2)** Extract the field representing the `cloud_area_fraction` to a variable which we will call `cloud_field`. Inspect that also with medium level of detail." + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "id": "91818c63-a229-491b-aee2-8882ff554746", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:06.493930Z", + "iopub.status.busy": "2024-11-20T17:08:06.493747Z", + "iopub.status.idle": "2024-11-20T17:08:06.506558Z", + "shell.execute_reply": "2024-11-20T17:08:06.505711Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "df537a09-9cc3-43c0-9e6a-7d8d7902344c", + "metadata": {}, + "source": [ + "**3.a.3)** Save the data of the `cloud_field` to a new variable we will call `cloud_field_data`. Do a `print` on the `shape` method of this to confirm the shape of the data, and compare this to the insepction report from the previous cell to see the same information represented in different ways." + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "id": "d54f5edd-abc3-40e7-a2c1-971fb52e12ca", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:06.511333Z", + "iopub.status.busy": "2024-11-20T17:08:06.511106Z", + "iopub.status.idle": "2024-11-20T17:08:06.521767Z", + "shell.execute_reply": "2024-11-20T17:08:06.520513Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "862a5eb4-6a63-4c11-a188-f048c3d27926", + "metadata": {}, + "source": [ + "**3.a.4)** Make a subspace of the `cloud_field` from the cells above to subspace on the *first* time point in order. Note: doing date-time subspaces requires an extra step due to the nature of specifying dates and times which can be ambiguous otherwise: you need to wrap a quoted datetime string in the call `cf.dt()` to notify cf-python that you are providing a valid datetime string, e.g. `field1.subspace(time=cf.dt(\"2020-01-01 12:15:00\"))`.\n", + "\n", + "Assign the subspace operation resulting field to a variable `cloud_field_subspace1` and inspect it with medium detail.\n", + "\n", + "*Extra task, for those who have studied section 4 before doing this practical**: make a contour plot of this subspace of the field to see what it looks like.*" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "id": "e0cdd409-e75d-43bb-96fd-c79b1f5e575b", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:06.528015Z", + "iopub.status.busy": "2024-11-20T17:08:06.527621Z", + "iopub.status.idle": "2024-11-20T17:08:06.596631Z", + "shell.execute_reply": "2024-11-20T17:08:06.594900Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "58adbc35-a63a-4c8a-973d-c28fea109ed3", + "metadata": {}, + "source": [ + "**3.a.5** Make a subspace of the `cloud_field` from the cells above to subspace on the *last* point on the latitude axis.\n", + "\n", + "Assign the subspace operation resulting field to a variable `cloud_field_subspace2` and inspect it with medium detail.\n", + "\n", + "*Extra task, for those who have studied section 4 before doing this practical: make a contour plot of this subspace of the field to see what it looks like.*" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "id": "6db795e1-8eb4-42b8-a6d2-4ad554f7fbb7", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:06.604211Z", + "iopub.status.busy": "2024-11-20T17:08:06.604017Z", + "iopub.status.idle": "2024-11-20T17:08:06.660716Z", + "shell.execute_reply": "2024-11-20T17:08:06.659617Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "bb1db98b-27b3-4c8a-bf63-f67f83dd2bb8", + "metadata": {}, + "source": [ + "### b) Subspacing using indexing, including equivalency to the above" + ] + }, + { + "cell_type": "markdown", + "id": "3d95810b-eb02-4862-938e-91e6634bb2b9", + "metadata": {}, + "source": [ + "**3.b.1)** Take the cloud field from (3.a.2) which we have been subspacing in the previous cells and make a subspace which takes the first time point, leaving all other axes unchanged, but this time do it using indexing. Use the `equals` method of a field to check that the result is the same as that derived from the 'subspacing by metadata' approach in section (3.a.4)." + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "id": "7d6ccf81-0a51-44f4-948c-95a243c751cd", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:06.665740Z", + "iopub.status.busy": "2024-11-20T17:08:06.665392Z", + "iopub.status.idle": "2024-11-20T17:08:06.718877Z", + "shell.execute_reply": "2024-11-20T17:08:06.717805Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "4e2cc4bf-2520-4136-82af-48ac6f974043", + "metadata": {}, + "source": [ + "**3.b.2)** Now make a subspace on the original `cloud_field`, leaving all other axes unchanged, to subspace on the *last* point on the latitude axis, like before, but this time use subspacing by indexing. Use the `equals` method of a field to check that the result is the same as that derived from the 'subspacing by metadata' approach in section (3.a.4)." + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "id": "f15ef080-2d79-4686-8f67-00baee8a98f2", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:06.722878Z", + "iopub.status.busy": "2024-11-20T17:08:06.722526Z", + "iopub.status.idle": "2024-11-20T17:08:06.772906Z", + "shell.execute_reply": "2024-11-20T17:08:06.771872Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "e65d7951-2d10-40d3-8858-87b7210079a8", + "metadata": {}, + "source": [ + "**3.b.3)** Using indexing, do both of the subspaces from the previous sub-questions in one call on the original cloud field.\n", + "\n", + "Extra: do the same operation using the 'subspace by metadata' approach and use the `equals` method to show that the results are the same." + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "id": "0b83671d-442f-41c4-981a-764581a358ec", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:06.778167Z", + "iopub.status.busy": "2024-11-20T17:08:06.777781Z", + "iopub.status.idle": "2024-11-20T17:08:06.818612Z", + "shell.execute_reply": "2024-11-20T17:08:06.817730Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "f5e05393-4c31-4505-8b80-4b8b2bdeba5d", + "metadata": {}, + "source": [ + "**3.b.4)** Do a single subspace on the original cloud field that takes the first 100 latitude and the first 200 longitude values. Use whichever method (subspacing by metadata, or indexing) you prefer, in order to do so." + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "id": "dd33a050-117c-4327-931c-ea8dd2372a70", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:06.825345Z", + "iopub.status.busy": "2024-11-20T17:08:06.825075Z", + "iopub.status.idle": "2024-11-20T17:08:06.842053Z", + "shell.execute_reply": "2024-11-20T17:08:06.840849Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "b870563e-6f45-46ca-9dad-6d8a5cb69ec5", + "metadata": {}, + "source": [ + "### c) Statistical collapses" + ] + }, + { + "cell_type": "markdown", + "id": "05d9602f-744d-4ce7-a4a0-1880a30db393", + "metadata": {}, + "source": [ + "**3.c.1)** Take the original `cloud_field` from (3.a.2) and do a collapse over the time axis to reduce it down to the minimum value. Assign that to the variable name `cloud_field_collapse1`." + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "id": "843d1148-25e1-4cc2-91e3-28085b0ac7f3", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:06.844909Z", + "iopub.status.busy": "2024-11-20T17:08:06.844702Z", + "iopub.status.idle": "2024-11-20T17:08:06.864748Z", + "shell.execute_reply": "2024-11-20T17:08:06.864209Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "1527c1bc-12e3-4708-a544-39fef8d0062b", + "metadata": {}, + "source": [ + "**3.c.2)** Take the original `cloud_field` from (3.a.2) and do a collapse over the latitude axis to reduce it down to the mean value. Assign that to the variable name `cloud_field_collapse2`." + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "id": "f44e6b5e-c3bf-48bf-81b4-d082a437d3c2", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:06.868071Z", + "iopub.status.busy": "2024-11-20T17:08:06.867849Z", + "iopub.status.idle": "2024-11-20T17:08:06.885644Z", + "shell.execute_reply": "2024-11-20T17:08:06.885021Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "592f58a8-77a3-4cc2-beb3-f6247f5f0310", + "metadata": {}, + "source": [ + "**3.c.3)** Take the original `cloud_field` from (3.a.2) and do a collapse over the longitude axis to reduce it down to the maximum value. Assign that to the variable name `cloud_field_collapse3`." + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "id": "62c236eb-c360-47b2-ac20-6d1250ed7a78", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:06.890774Z", + "iopub.status.busy": "2024-11-20T17:08:06.890519Z", + "iopub.status.idle": "2024-11-20T17:08:06.911761Z", + "shell.execute_reply": "2024-11-20T17:08:06.910899Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "22357974-11e2-4820-b0d6-90273cbc4646", + "metadata": {}, + "source": [ + "**3.c.4)** Finally, take the original `cloud_field` from (3.a.2) again and do a collapse over all horizontal space via the pair of horizontal spatial axes, latitude and longitude, to reduce them down to the standard deviation value. Assign that to the variable name `cloud_field_collapse4`." + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "id": "532ae8bf-09a1-4a2c-8800-2802fe186202", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:06.915602Z", + "iopub.status.busy": "2024-11-20T17:08:06.915315Z", + "iopub.status.idle": "2024-11-20T17:08:06.945538Z", + "shell.execute_reply": "2024-11-20T17:08:06.944940Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "59b69a9d-a94c-464c-aa3c-482c699bc404", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "80edfa76-7f87-42b9-8cef-058304ca902a", + "metadata": {}, + "source": [ + "## 4. Visualising datasets as contour and vector plots\n", + "\n", + "**In this section we demonstrate how to plot using cf-plot the data we have read and then processed and/or analysed using cf-python, notably showing how to create contour plots and vector plots as examples of some of the available plot types.**" + ] + }, + { + "cell_type": "markdown", + "id": "794c4940-43fd-45a3-ba9a-69084c0e5ccd", + "metadata": {}, + "source": [ + "### a) Making a contour plot" + ] + }, + { + "cell_type": "markdown", + "id": "d8508032-181e-4a04-ba70-0f0dfd24f601", + "metadata": {}, + "source": [ + "**4.a.1)** Read in the file `alpine_precip_DJF_means.nc` which is in the `../data` directory and assign the first field it contains to a variable called 'field_4'. Inspect that field with medium detail level." + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "id": "e3b39603-c768-472b-bd2b-8aec63e525c0", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:06.951266Z", + "iopub.status.busy": "2024-11-20T17:08:06.950795Z", + "iopub.status.idle": "2024-11-20T17:08:06.990033Z", + "shell.execute_reply": "2024-11-20T17:08:06.989428Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "f780f284-697e-41a3-8984-db53ac970235", + "metadata": {}, + "source": [ + "**4.a.2)** Make a contour plot of the field, noting that because it is already 2D over X and Y axes corresponding to its projection so it can be plotted directly, without any need to reduce via subspacing or collapsing. Just get a plot working - there is no need to customise it yet." + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "id": "94613743-f648-409e-b7a3-22bbf5849c1c", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:06.995599Z", + "iopub.status.busy": "2024-11-20T17:08:06.995445Z", + "iopub.status.idle": "2024-11-20T17:08:09.365699Z", + "shell.execute_reply": "2024-11-20T17:08:09.364898Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "b19186dc-3480-47d1-a859-79008841bd58", + "metadata": {}, + "source": [ + "### b) Customising the (contour) plot" + ] + }, + { + "cell_type": "markdown", + "id": "017e176a-67ab-4580-82e3-245bf30cc585", + "metadata": {}, + "source": [ + "**4.b.1)** Customise the contour plot made in section (4a) to remove the contour lines by running the same call with a new argument set appropriately." + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "id": "b4509ae8-c96c-4d9d-9b41-83e3828d03b7", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:09.380804Z", + "iopub.status.busy": "2024-11-20T17:08:09.380305Z", + "iopub.status.idle": "2024-11-20T17:08:09.903879Z", + "shell.execute_reply": "2024-11-20T17:08:09.903206Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "6b403c8c-5991-44b2-89db-660c58f0abdd", + "metadata": {}, + "source": [ + "**4.b.2)** Customise the contour plot further so, as well as having the contour lines removed as applied above, it is shown using a different colour map to the default colour map, 'viridis'. You can use the link https://ncas-cms.github.io/cf-plot/build/colour_scales.html#colour-scales to explore the different options." + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "id": "3e4e3a79-9e31-4779-9aa3-7cb8d1832819", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:09.911386Z", + "iopub.status.busy": "2024-11-20T17:08:09.911221Z", + "iopub.status.idle": "2024-11-20T17:08:10.792379Z", + "shell.execute_reply": "2024-11-20T17:08:10.791417Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "422253a2-28ab-4d6a-8841-d9cdb0ea527c", + "metadata": {}, + "source": [ + "**4.b.3)** Customise the contour plot further by adding the argument and value `resolution=\"10m\"` to the call to `mapset`, which will increase the resolution of the country borders." + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "id": "7945c4ee-53ed-4c26-b468-b4cd6b0edc88", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:10.800385Z", + "iopub.status.busy": "2024-11-20T17:08:10.800219Z", + "iopub.status.idle": "2024-11-20T17:08:27.841993Z", + "shell.execute_reply": "2024-11-20T17:08:27.840986Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "d01534c9-8c56-4f6f-a550-2e86a5749bed", + "metadata": {}, + "source": [ + "**4.b.4)** Customise the contour plot further so that it focuses in on a smaller region over the Alps, specifically from longitude of 5 to 10 degrees east and from latitude of 45 to 47 degrees. Note, a call to `mapset` will reset any of the previous calls, so you will need to re-apply previous arguments to it that you want to preserve for future plots." + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "id": "70bf4f42-61de-4255-acbc-9235a36f403e", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:27.849301Z", + "iopub.status.busy": "2024-11-20T17:08:27.849095Z", + "iopub.status.idle": "2024-11-20T17:08:33.692729Z", + "shell.execute_reply": "2024-11-20T17:08:33.691631Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "6964260d-84fb-41d9-a1b9-a74e85a79c20", + "metadata": {}, + "source": [ + "### c) Making a vector plot with basic customisation" + ] + }, + { + "cell_type": "markdown", + "id": "5a7610d7-2654-4759-9083-0ae921546f63", + "metadata": {}, + "source": [ + "**4.c.1)** Read in the netCDF file `sea_currents.nc` stored in the usual directory, `../data`, assigning it to a variable 'irish_sea_fieldlist' and inspect it with medium detail." + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "id": "b1ba3732-e318-4e4e-a5e7-8ca17db707b9", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:33.702800Z", + "iopub.status.busy": "2024-11-20T17:08:33.702404Z", + "iopub.status.idle": "2024-11-20T17:08:33.744958Z", + "shell.execute_reply": "2024-11-20T17:08:33.744043Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "920060c3-46e1-420d-a400-f110e9c08b97", + "metadata": {}, + "source": [ + "**4.c.2)** Note that the first two fieds represent perpendicular components of the same directional variable for sea water velocity. The first field is the eastward and the second field is the northward, component of this.\n", + "\n", + "In order to ensure they are compatible to plot together as vectors, we first need to ensure they are defined on the same gridpoints - in cf-python (from CF Data Model) terminology, we say they need to have the same *domain*. Assign to variable names 'eastward_component_field' and 'northward_component_field' the first and second fields respectively, and inspect them with minimal detail as a first step check to see if there is the same number of axes and points along each upon which the data is defined (there are two code blocks here, one for each field since when you inspect ith minimal detail it will only show if it is the final call in the cell)." + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "id": "b0946df9-087f-4460-8519-4b2b4dea2d6f", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:33.750031Z", + "iopub.status.busy": "2024-11-20T17:08:33.749151Z", + "iopub.status.idle": "2024-11-20T17:08:33.756172Z", + "shell.execute_reply": "2024-11-20T17:08:33.755313Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": 58, + "id": "49cd847e-4c92-46d6-9f31-f81b8ec3b4e2", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:33.759433Z", + "iopub.status.busy": "2024-11-20T17:08:33.759075Z", + "iopub.status.idle": "2024-11-20T17:08:33.772911Z", + "shell.execute_reply": "2024-11-20T17:08:33.771509Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "8018e3d9-685b-4a12-93c3-77f3dfcb035e", + "metadata": {}, + "source": [ + "**4.c.3)** You should be able to see from the metadata that the two fields are perpendicular components of the same wind field. But, for us to be sure these represent components of the same vector field, we need to check that they are defined at the same grid points - else the components are defined at different location points in space and therefore can't be combined into a single vector. In cf-python we call the locations that the data is representative of the *domain*.\n", + "\n", + "We can check whether the two fields have the same domain by taking the `domain` attribute of each field and comparing those using the `equals` method, e.g. `field1.domain.equals(field2.domain)` for fields `field1` and `field2`. Do this for the two fields we defined in the pair of cells above." + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "id": "33efb918-44f1-46cc-9018-d7775a108382", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:33.778067Z", + "iopub.status.busy": "2024-11-20T17:08:33.776955Z", + "iopub.status.idle": "2024-11-20T17:08:33.914766Z", + "shell.execute_reply": "2024-11-20T17:08:33.914147Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "b170dab7-7a23-47e9-8dc1-04b448c04ab5", + "metadata": {}, + "source": [ + "**4.c.4)** We know now that the fields are the same variable, with perpendicular northward and eastward components, with the same domain i.e. defined on the same gridpoints. This means they are compatible vector components and can be plotted together to form a vector plot for the variable in question, the sea water velocity. However, we currently have the time coordinate with more than one point, so we need to reduce both fields down in the correpsonding way to get a particular 2D plot.\n", + "\n", + "Let's take the final time point in the series of both of these fields to plot. Subspace both fields down in time to that final time axis point, using indexing - using section (3b) for a reminder about subspacing by index if you need guiance." + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "id": "f94aaa31-8468-4803-869e-878d542a061e", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:33.919287Z", + "iopub.status.busy": "2024-11-20T17:08:33.919127Z", + "iopub.status.idle": "2024-11-20T17:08:33.971845Z", + "shell.execute_reply": "2024-11-20T17:08:33.970672Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": 61, + "id": "9c1640c3-2174-472d-994a-b6a3d7c54ae2", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:33.975629Z", + "iopub.status.busy": "2024-11-20T17:08:33.975429Z", + "iopub.status.idle": "2024-11-20T17:08:34.027042Z", + "shell.execute_reply": "2024-11-20T17:08:34.025422Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "90055498-9f1a-463e-aec1-1cf22cd4b333", + "metadata": {}, + "source": [ + "**4.c.5)** Make a vector plot using the two subspaced component fields you found and defined in the previous section to form the corresponding vectors. Do not worry about customising it yet, just get a plot working even if the vectors are not optimised for readability, however do run `cfp.mapset(resolution=\"10m\")` before you make the call to make the vector plot to reset the X and Y limits we specified in the previous section.\n", + "\n", + "Note that the `u` keyword argument is for eastward vector components and the `v` keyword argument is for northward vector components." + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "id": "c3249180-3c84-4a96-a8f8-f40340769bc2", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:34.033230Z", + "iopub.status.busy": "2024-11-20T17:08:34.032753Z", + "iopub.status.idle": "2024-11-20T17:08:48.591726Z", + "shell.execute_reply": "2024-11-20T17:08:48.590701Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "f99f557c-a4dd-42e4-8b7d-5d9bcc1d5ad9", + "metadata": {}, + "source": [ + "**4.c.6)** Customise the plot you just made, in particular so that the vector lines are legible through adjusting the spacing and size at which they are plotted. This usually takes some trial and error - guess some values for the keyword argument values and increase or decrease them until you find a good balance so the vector lines can be distinguished and clearly interpreted." + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "id": "783e8212-335b-4276-8014-2be54f95ca4b", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:48.603011Z", + "iopub.status.busy": "2024-11-20T17:08:48.602628Z", + "iopub.status.idle": "2024-11-20T17:08:48.920098Z", + "shell.execute_reply": "2024-11-20T17:08:48.918698Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "b191c8b2-a038-4601-b046-997a4566c2f6", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "72524bb7-a643-4753-ad8b-f419656883fa", + "metadata": {}, + "source": [ + "## 5. Analysing data: applying mathematical and statistical operations and plotting trends\n", + "\n", + "**In this section we demonstrate how to do some data analysis including performing arithmetic and statistical calculations on the data, showing how cf-python's CF Conventions metadata awareness means that the metadata is automatically updated to account for the operations that are performed.**" + ] + }, + { + "cell_type": "markdown", + "id": "59637071-edcb-40fe-970a-1d89e0209aa7", + "metadata": {}, + "source": [ + "### a) Applying mathematics e.g. arithmetic and trigonometry on fields" + ] + }, + { + "cell_type": "markdown", + "id": "3ce63f35-c0a3-4c5b-a58c-98856f26ce25", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "**5.a.1)** Take the `fieldlist_3` we have already defined in (3.a.1). This time, we'll work with the *fourth* field in order in the list. Extract this field to a variable called `field_5` and inspect it with medium detail." + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "id": "8846ce13-3ac6-4586-9318-bcf96abe2b97", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:48.930202Z", + "iopub.status.busy": "2024-11-20T17:08:48.930030Z", + "iopub.status.idle": "2024-11-20T17:08:48.941550Z", + "shell.execute_reply": "2024-11-20T17:08:48.940485Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "dfdf903e-1a4b-46e2-9ba5-4118307329c7", + "metadata": {}, + "source": [ + "**5.a.2)** Take a subspace of the first time point, assigning it to a new variable called 'field_5_subspace', and plot it on a contour plot to see what the underlying data is like. Making a plot with cf-plot early on in this way is, for example, a good way to get a quick look at a new dataset to get a feel for the overall scope and patterns." + ] + }, + { + "cell_type": "code", + "execution_count": 65, + "id": "9c7be26b-51d7-4f43-aaa5-ce916b97a468", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:48.945212Z", + "iopub.status.busy": "2024-11-20T17:08:48.945010Z", + "iopub.status.idle": "2024-11-20T17:08:52.163027Z", + "shell.execute_reply": "2024-11-20T17:08:52.162060Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "3d0733cc-b6fa-4889-87ba-c0161b29d8eb", + "metadata": {}, + "source": [ + "**5.a.3)** To normalise data representing a variable `x`, there is the general formula `normalized_x = (x - minimum_x) / x-range`. Use the cf-python methods available on a field, namely `minimum()` and `range()`, to normalise the data in the subspace of `field_5` we defined as `field_5_subspace` from the previous step. Call this result `norm_field_5` as a new Python variable. Hint: you can apply these methods directly on the field which knows to apply them to its underlying data, and the whole data array will be operated on at once, as with array operations in NumPy and similar Python array libraries.\n", + "\n", + "Finally, re-plot the contour plot at the subspace on the first time point to see that the data has been normalised appropriately: the colour bar range should go from 0 to 1 now but the data plot itself (contour pattern and colours) should not have changed because the normalisation adjusts all values to the 0 to 1 range but preserves the relative magnitudes between them." + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "id": "e770cba8-88db-44ac-a143-51e369b9695b", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:52.178478Z", + "iopub.status.busy": "2024-11-20T17:08:52.178200Z", + "iopub.status.idle": "2024-11-20T17:08:53.451814Z", + "shell.execute_reply": "2024-11-20T17:08:53.451010Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "3636fd39-4aa6-4b86-8eaf-0154b285fc2d", + "metadata": {}, + "source": [ + "**5.a.4)** Finally, to explore basic field arithmetic and see the influence on the data via a plot, multiply `field_5_subspace` by negative one and plot this result as a contour plot. Describe in a sentence comment the difference in the plot to the original from ." + ] + }, + { + "cell_type": "code", + "execution_count": 67, + "id": "aed0f47d-cad3-4a80-8ef6-2878ec963fd8", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:53.464767Z", + "iopub.status.busy": "2024-11-20T17:08:53.464298Z", + "iopub.status.idle": "2024-11-20T17:08:55.082284Z", + "shell.execute_reply": "2024-11-20T17:08:55.081625Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "561caf84-0824-4b69-8ad9-82ce28d5bfd1", + "metadata": {}, + "source": [ + "### b) Line plotting" + ] + }, + { + "cell_type": "markdown", + "id": "a42040e2-0bc1-4e25-9168-5dc474e97ab2", + "metadata": {}, + "source": [ + "**5.b.1)** Let's make some line plots. To set this up, read in the file `IPSL-CM5A-LR_r1i1p1_tas_n96_rcp45_mnth.nc` under the usual directory `../data` and assign the *first* field in that FieldList to a variable called `monthly_field`. Inspect it with medium detail." + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "id": "274d1d90-5be7-46ff-8532-5de882998187", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:55.098200Z", + "iopub.status.busy": "2024-11-20T17:08:55.097974Z", + "iopub.status.idle": "2024-11-20T17:08:55.182180Z", + "shell.execute_reply": "2024-11-20T17:08:55.181172Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "5de191f4-d755-43f7-8718-e04a8376b647", + "metadata": {}, + "source": [ + "**5.b.2)** Notice that the `monthly_field` is three-dimensional (3D) with axes sizes all over one for each of the three dimensions. In order to do a lineplot, we need a one-dimensional series, so we need to reduce the field down to size one in two of the dimensions to make a line plot.\n", + "\n", + "Let's do a collapse to reduce the two spatial axes, latitude and longitude, at once. Do a collapse to the maximum over those two axes, assigning it to a variable we'll call `spatial_max_monthly_field`. You can specify both of those axes by name, or you can use `\"area\"` as a shorthand string which means both latitude and longitude at once." + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "id": "34f9b25c-c249-4868-b262-ecadf69367b8", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:55.186828Z", + "iopub.status.busy": "2024-11-20T17:08:55.186632Z", + "iopub.status.idle": "2024-11-20T17:08:55.199230Z", + "shell.execute_reply": "2024-11-20T17:08:55.198599Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "f0e10b7f-b3b5-4dd3-b79b-1d8b6e1bb190", + "metadata": {}, + "source": [ + "**5.b.3)** Use cf-plot to make a line plot of the collapsed field `spatial_max_monthly_field` from the cell above." + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "id": "537ec534-e25e-4dc6-88b6-57e7797a1273", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:55.203523Z", + "iopub.status.busy": "2024-11-20T17:08:55.203384Z", + "iopub.status.idle": "2024-11-20T17:08:55.749436Z", + "shell.execute_reply": "2024-11-20T17:08:55.748424Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "682335b3-5eba-416c-bbd2-182f5915df7d", + "metadata": {}, + "source": [ + "### c) Calculating seasonal trends" + ] + }, + { + "cell_type": "markdown", + "id": "9799e66c-0107-4dd0-8652-5a851a795c8f", + "metadata": {}, + "source": [ + "**5.c.1)** Define a new variable `get_mam_season` and set it to the cf-python function `mam()` which represents a specific collapse over the months of March, April and May, just like `cf.djf()` represents a collapse over December, January and February etc. - there are four such methods to cover each trio of months closely aligned with the seasons, two of which were demonstrated in the teaching Notebook." + ] + }, + { + "cell_type": "code", + "execution_count": 71, + "id": "9f483867-c35f-456b-8314-7c9c006b163a", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:55.754187Z", + "iopub.status.busy": "2024-11-20T17:08:55.753935Z", + "iopub.status.idle": "2024-11-20T17:08:55.758357Z", + "shell.execute_reply": "2024-11-20T17:08:55.757484Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "6cbc9cf7-cfd7-49fc-8244-91612c006cdb", + "metadata": {}, + "source": [ + "**5.c.2)** Do a grouped collapse over the months of March, April and May only using the `group` argument set to `get_mam_season`, for a *maximum* collapse over the *time* axis on the `spatial_max_monthly_field`. Make a line plot of this collapsed field." + ] + }, + { + "cell_type": "code", + "execution_count": 72, + "id": "8763a854-82cf-4fc5-80b5-c0214310fae9", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:55.761922Z", + "iopub.status.busy": "2024-11-20T17:08:55.761596Z", + "iopub.status.idle": "2024-11-20T17:08:57.199843Z", + "shell.execute_reply": "2024-11-20T17:08:57.198784Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "fcb98f04-a03d-4c1c-b0cd-ea153daf6f16", + "metadata": {}, + "source": [ + "**5.c.3)** Define three more variables representing specific collapse over the DJF, JJA and SON trio of months using the appropriate cf-python methods to complete the set across the calendar year (`cf.son()` is for the SON months and use the teaching Notebook for guidance on the other two methods if required but their names follow the same pattern so you can possibly guess them), assigning them to variables `get_djf_season`, `get_jja_season` and `get_son_season` respectively.\n", + "\n", + "Then, using those three variables one-by-one, define three grouped collapses, also for a *maximum* collapse over the *time* axis as above, on the `spatial_max_monthly_field` field using these as the `group`, calling them `djf_season_max`, `jja_season_max` and `son_season_max` corresponding to the collapses in order above." + ] + }, + { + "cell_type": "code", + "execution_count": 73, + "id": "ea2e67a9-1ec6-4d3b-a673-2f06c9f7ae11", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:57.205981Z", + "iopub.status.busy": "2024-11-20T17:08:57.205799Z", + "iopub.status.idle": "2024-11-20T17:08:59.648760Z", + "shell.execute_reply": "2024-11-20T17:08:59.647356Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "b094573f-f690-48da-a8e9-93481ead73fb", + "metadata": {}, + "source": [ + "### d) Plotting the seasonal trends on one (line)plot" + ] + }, + { + "cell_type": "markdown", + "id": "428b074b-d2af-4804-9131-2985d69bf4d9", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "**5.d.1)** Let's make a line plot which shows all of the fields above: the four season maximum trends `djf_season_max`, `mam_season_max`, `jja_season_max` and `son_season_max`, along with the `spatial_max_monthly_field`. Use the code block from the teaching Notebook as a guide, or otherwise: you will need to wrap all of the calls to `lineplot` within `cfp.gopen()` and `cfp.gclose()` so they are plotted on the same canvas. You do not need to set positions." + ] + }, + { + "cell_type": "code", + "execution_count": 74, + "id": "83a29404-cc39-47eb-9909-b2af9b1ab7e9", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:08:59.654463Z", + "iopub.status.busy": "2024-11-20T17:08:59.654283Z", + "iopub.status.idle": "2024-11-20T17:09:02.136961Z", + "shell.execute_reply": "2024-11-20T17:09:02.135829Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "422daf4c-731d-48db-812e-53298d8d80a7", + "metadata": {}, + "source": [ + "**5.d.2)** Finally, copy your code from the previous cell and use the `label` argument to `lineplot` to assign labels to your individual line plot calls, which will result in a legend emerging on the final plot with the labels to identify each line." + ] + }, + { + "cell_type": "code", + "execution_count": 75, + "id": "a6396957-b64b-4c79-9fcd-067463457842", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:09:02.145306Z", + "iopub.status.busy": "2024-11-20T17:09:02.144988Z", + "iopub.status.idle": "2024-11-20T17:09:05.852257Z", + "shell.execute_reply": "2024-11-20T17:09:05.850632Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "3b746f6b-ef82-4aa9-8d39-ca179f15db62", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "2e4c72ef-8631-4749-b4fd-6c79c8d6e455", + "metadata": {}, + "source": [ + "## 6. Changing the underlying grid of data through regridding\n", + "\n", + "**In this section we demonstrate how to change the underlying grid of the data to another grid which could be a higher- or lower- resolution one, or a completely different grid, which is called regridding or interpolation, and indicate various options cf-python supports for doing this.**" + ] + }, + { + "cell_type": "markdown", + "id": "854d7fd4-6d65-4827-b77c-4a89d42da6b0", + "metadata": {}, + "source": [ + "### a) Getting a _source_ field ready to regrid" + ] + }, + { + "cell_type": "markdown", + "id": "65ee428b-cf95-423c-b92e-4cd644b7df41", + "metadata": {}, + "source": [ + "**6.a.1)** Define a new variable 'yearly_field' set to the first field in the FieldList read-in from the netCDF dataset `precip_1D_yearly.nc` which lives in the usual directory, `../data`. Inspect it with medium level of detail." + ] + }, + { + "cell_type": "code", + "execution_count": 76, + "id": "a6ec5550-61a1-47ff-99f1-339f6dff0fa9", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:09:05.858794Z", + "iopub.status.busy": "2024-11-20T17:09:05.857680Z", + "iopub.status.idle": "2024-11-20T17:09:05.915023Z", + "shell.execute_reply": "2024-11-20T17:09:05.914404Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "7955e4b0-a23e-4398-9c09-48375a37e2e2", + "metadata": {}, + "source": [ + "**6.a.2)** This will be our source field for regridding. Note it is a time series, not a dataset defined across different points in space! We can regrid data on any axis or axes, they do not have to represent spatial coordinates, including 1D series. In this case, we have an effectively 1D (3D but with two of the axes size one, hence of size which can be ignored) time series and we are going to regrid it. For a time series, visually this\n", + "\n", + "Make a line plot of the time series in `yearly_field` to see it. Add the argument `marker=\"o\"` to the call which plots markers against the individual data points forming the line. With the markers on we can see the discrete sample points that make up the data represented by the continuous line." + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "id": "17a5908e-ea5a-4b8b-bbf2-980f7bcf0796", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:09:05.918771Z", + "iopub.status.busy": "2024-11-20T17:09:05.918484Z", + "iopub.status.idle": "2024-11-20T17:09:06.174541Z", + "shell.execute_reply": "2024-11-20T17:09:06.173910Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "628e91df-76f0-485e-8288-19ee932a7fb6", + "metadata": {}, + "source": [ + "### b) Getting the _destination_ field: another field in order to regrid the previous _onto its grid_" + ] + }, + { + "cell_type": "markdown", + "id": "ff212883-d45d-4b3a-8d60-5f8816c9acce", + "metadata": {}, + "source": [ + "**6.b.1)** Let's get another time series field, but one with a different sampling resolution. The sampling of time points across the axes is the *grid* in this context. For our destination field, we will take a series which is sampled at higher resolution.\n", + "\n", + "Define a new variable 'monthly_field' set to the first field in the FieldList read-in from the netCDF dataset `precip_1D_monthly.nc` which lives in the usual directory, `../data`. Inspect it with medium level of detail." + ] + }, + { + "cell_type": "code", + "execution_count": 78, + "id": "7f1f0704-f409-4ae5-b6d1-acadd22790d1", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:09:06.177892Z", + "iopub.status.busy": "2024-11-20T17:09:06.177738Z", + "iopub.status.idle": "2024-11-20T17:09:06.219048Z", + "shell.execute_reply": "2024-11-20T17:09:06.218479Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "d9f9d588-5bf4-4911-ba4e-e8e2ece77809", + "metadata": {}, + "source": [ + "**6.b.2)** As the name of the corresponding read-in files suggests, this `dest_field` represents monthly data and the `source_field` represents yearly data, so `dest_field` is around 12 times the resolution of the `source_field`.\n", + "\n", + "Make a line plot of the `dest_field` showing the markers for the discrete data points. Compare this line plot to the one above with respect to the time axis (on the x-axis) to confirm the difference in sampling frequencies of around x12. Note we don't care about the values of the data at each point and whether it is different to the data of the source grid we are considering! All we want to extract from this destination field time series is the time point sampling, our destination grid. (Likewise, in the teaching Notebook where we did horizontal spatial grid regridding, the data defined on the destination grid `lower_res_field` did not matter, just the grid points it was defined on.)" + ] + }, + { + "cell_type": "code", + "execution_count": 79, + "id": "a414b073-0c90-498e-bd71-62bdbc3cc69d", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:09:06.224093Z", + "iopub.status.busy": "2024-11-20T17:09:06.223945Z", + "iopub.status.idle": "2024-11-20T17:09:06.523873Z", + "shell.execute_reply": "2024-11-20T17:09:06.522609Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "dde6c34c-37a9-48c9-8e02-a70913411d7a", + "metadata": {}, + "source": [ + "### c) Performing the regrid operation from the source to the destination fields" + ] + }, + { + "cell_type": "markdown", + "id": "6ce49aba-e16f-4278-ac9f-4fa3e8d4c8c5", + "metadata": {}, + "source": [ + "**6.c.1)** Before we do the regridding operation, let's demonstrate briefly how to remove any size one axes from a field. Note this is not necessary in order to do the regridding, but it can 'tidy up' the fields so they don't have size one axes attached, which can be useful in cases like this because it means the axes are both 1D so it is simpler to interpret the regridding we are doing. To remove the size one axes of a field, apply the `squeeze()` method: you can either re-assign the field with `squeeze()` applied to the same variable name, or use the `inplace=True` argument to the method which changes the field in-place i.e. to the variable storing it without the need to re-assign.\n", + "\n", + "Apply the `squeeze` method to the `monthly_field` and the `yearly_field` and inspect the results of each with medium level of detail." + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "id": "c7d637a8-bdb9-43ad-9e05-0e2abcb58fb9", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:09:06.528995Z", + "iopub.status.busy": "2024-11-20T17:09:06.528651Z", + "iopub.status.idle": "2024-11-20T17:09:06.560668Z", + "shell.execute_reply": "2024-11-20T17:09:06.559248Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "bc533850-f32c-4441-9a4f-f14c3206db14", + "metadata": {}, + "source": [ + "**6.c.2)** Now to do the actual regridding operation! (You could call all the field methods such as reading, indexing and squeezing in the same line as the actual regrid operation, but doing so would be harder to interpret and to debug in case something wasn't speciied correctly, or some other issue, so we have worked these out step-by-step as better practice.)\n", + "\n", + "In the teaching Notebook we used the `regrids` method which performs *spherical* regridding (`s` for spherical) because longitude and latitude are defined on the Earth's surface which forms a spherical coordinate system (you can read more about spherical regridding if desired here: https://ncas-cms.github.io/cf-python/tutorial.html#spherical-regridding).\n", + "\n", + "But that is not appropriate here because we aren't considering coordinates defined on a spherical surface, so we need to use *Cartesian* regridding. We use the `regridc` method of cf-python to do this, where the `c` in the name stands for the 'C' of Cartesian, which is the appropriate type here and in general you should use when considering any axes in Euclidean (ie. flat/non-curved) space (you can read more about Cartesian regridding if desired here: https://ncas-cms.github.io/cf-python/tutorial.html#cartesian-regridding).\n", + "\n", + "With that in mind, take the source and destinations fields we set up in the previous sub-sections and apply the cf-python Cartesian regridding operation to regrid `yearly_field` to the grid (i.e. the sampling resolution in this 1D case) of `monthly_field`, assigning the output to a new variable `linear_regridded_field`. Use the `\"linear\"` argument to `method` to use linear interpolation as the regridding method, this time. You will need to set a further argument for Cartesian regridding, namely `axes=\"T\"` to indicate that we want to regrid over the 'time'/'T' axis." + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "id": "d428cede-8d0f-4c5e-b921-4ab7eb30d6f1", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:09:06.565719Z", + "iopub.status.busy": "2024-11-20T17:09:06.565310Z", + "iopub.status.idle": "2024-11-20T17:09:06.633545Z", + "shell.execute_reply": "2024-11-20T17:09:06.632736Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "01084a42-6564-40cc-9b96-0a945e03715c", + "metadata": {}, + "source": [ + "**6.c.3)** Make a line plot of `linear_regridded_field`, including the argument `marker=\"o\"` so that we can see the discrete points the line is based upon." + ] + }, + { + "cell_type": "code", + "execution_count": 82, + "id": "20e4025c-fe01-4ab4-b92f-74806a30d486", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:09:06.637461Z", + "iopub.status.busy": "2024-11-20T17:09:06.637197Z", + "iopub.status.idle": "2024-11-20T17:09:06.915274Z", + "shell.execute_reply": "2024-11-20T17:09:06.914068Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "7071b0f4-c833-4fe4-a279-55c773979c75", + "metadata": {}, + "source": [ + "**6.c.4)** This time, regrid the inverse way to before, so that our `yearly_field` this time is our destination field and our `monthly_field` is our source field. Again, use `regridc` with the `\"linear\"` argument to `method` to use linear interpolation as the regridding method, and set `axes=\"T\"` to indicate that we want to regrid over the 'time'/'T' axis.\n", + "\n", + "Call this `inverse_linear_regridded_field`. Inspect it with medium detail level and make a line plot of it showing markers." + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "id": "32cabd25-e5c6-48ae-858e-9495751f70e0", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:09:06.920908Z", + "iopub.status.busy": "2024-11-20T17:09:06.920750Z", + "iopub.status.idle": "2024-11-20T17:09:07.235618Z", + "shell.execute_reply": "2024-11-20T17:09:07.234594Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "c3aaa456-411f-437d-92eb-aba7800113d1", + "metadata": {}, + "source": [ + "**6.c.5)** Regrid the reverse way to before, like in (6.c.4), but this time use the `\"nearest_stod\"` method of `regridc` which stands for 'nearest neighbour source to destination' interpolation. This will do the same regridding but using a different approach to the underlying calculation based upon mapping each destination point to the closest source point.\n", + "\n", + "Call this `inverse_nearest_stod_regridded_field`. Inspect it with medium detail level and make a line plot of it showing markers." + ] + }, + { + "cell_type": "code", + "execution_count": 84, + "id": "f5e67ce3-f7eb-47bf-8c85-f1f9da6f4883", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:09:07.242174Z", + "iopub.status.busy": "2024-11-20T17:09:07.241904Z", + "iopub.status.idle": "2024-11-20T17:09:07.604959Z", + "shell.execute_reply": "2024-11-20T17:09:07.604343Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "1286c96e-02f2-406e-a70d-914ae00389b5", + "metadata": {}, + "source": [ + "### d) Finally, some more advanced cf-plot plotting to compare the source, destination, and regridded results" + ] + }, + { + "cell_type": "markdown", + "id": "e2196667-b7dd-4548-89e0-5194aa6170a5", + "metadata": {}, + "source": [ + "**6.d.1)** Use cf-plot to make a plot with *two separate line plots* on one canvas (in either separate rows or columns, your choice), one of the first original field `yearly_field` and the linearly-regridded version of it, `linear_regridded_field`, so we can compare the original (un-regridded) and regridded result.\n", + "\n", + "Use the code block from the teaching Notebook section (6b) as a guide, or otherwise: you will need to wrap all of the calls to `lineplot` within `cfp.gopen()` and `cfp.gclose()` so they are plotted on the same canvas and call `cfp.gpos(N)` with an integer starting at `N=1` to tell cf-plot to move on to the next position in the plot before making a `lineplot` call." + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "id": "910689b2-26d4-4973-bb4e-97cf1cfeb10a", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:09:07.613321Z", + "iopub.status.busy": "2024-11-20T17:09:07.613020Z", + "iopub.status.idle": "2024-11-20T17:09:08.085281Z", + "shell.execute_reply": "2024-11-20T17:09:08.084658Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "7561067c-b9a0-45a8-9c37-87318528d39b", + "metadata": {}, + "source": [ + "**6.d.2)** This time, use cf-plot to make a plot with *multiple line plots plotted together on the same (x- and y-) axes*. Plot both original fields along with the two regridded `inverse_*_regridded_field`, this time, and plot them in different colours of your choosing with sensible descriptive labels:\n", + "1. `monthly_field`\n", + "2. `yearly_field`\n", + "3. `inverse_linear_regridded_field`\n", + "4. `inverse_nearest_stod_regridded_field`\n", + "\n", + "Use the code block from the teaching Notebook section (5d) as a guide, or otherwise: you will need to wrap all of the calls to `lineplot` within `cfp.gopen()` and `cfp.gclose()` and by, not calling `cfp.gpos()` with position integers in the last question, all of the lines will be plotted onto the same canvas to share axes" + ] + }, + { + "cell_type": "code", + "execution_count": 86, + "id": "9e80586f-08a7-44e9-bdde-8898f0a68650", + "metadata": { + "editable": true, + "execution": { + "iopub.execute_input": "2024-11-20T17:09:08.090399Z", + "iopub.status.busy": "2024-11-20T17:09:08.090212Z", + "iopub.status.idle": "2024-11-20T17:09:08.655195Z", + "shell.execute_reply": "2024-11-20T17:09:08.653884Z" + }, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "877fa655-484a-4d24-a31d-b36d93a96ad8", + "metadata": {}, + "source": [ + "**6.d.3)** Finally, study the two compound plots created in this section, in particular paying attention to the marker spacings representing the time data point sampling i.e. resolution which forms our grid in this context, to help you to ensure you understand the process of regridding.\n", + "\n", + "Also, look back to the horizontal spatial regridding example plot in (6d) of the teaching Notebook and pay particular attention to the cell block sizes from before and after regridding, to remind yourself the output from a spatial regridding operation.\n", + "\n", + "We've practiced regridding of fields in two contexts: spherical regridding in a 2D spatial context and Cartesian regridding in a 1D time series context. Hopefully these two examples have allowed you to understand what regridding does, but at least how cf-python can enable you to do it." + ] + }, + { + "cell_type": "markdown", + "id": "08fa89e2-6c21-4118-9cf9-21958bdfa00e", "metadata": {}, "source": [ - "# Exercise 4" + "***" ] } ], diff --git a/python-data/notebooks/intro_to_ncas_cf_data_tools.ipynb b/python-data/notebooks/intro_to_ncas_cf_data_tools.ipynb new file mode 100644 index 0000000..2f34ffa --- /dev/null +++ b/python-data/notebooks/intro_to_ncas_cf_data_tools.ipynb @@ -0,0 +1,2075 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "0cd6019e-c5b3-4dc6-bf86-6fcf10b06851", + "metadata": {}, + "source": [ + "# Introduction to the NCAS CF Data Tools, cf-python and cf-plot" + ] + }, + { + "cell_type": "markdown", + "id": "c4378ad0-8448-44ef-80e2-cd6e67346a1b", + "metadata": {}, + "source": [ + "## Context and learning objectives\n", + "\n", + "### What are the NCAS CF Data Tools and why do they all have 'cf' in the name?\n", + "\n", + "The _NCAS CF Data Tools_ are a suite of complementary Python libraries which are designed to facilitate working with data for research in the earth sciences and aligned domains. The two that are of most relevance to the average user, and those wanting to process, analyse and visualise atmospheric data, are *cf-python* (https://ncas-cms.github.io/cf-python/) and *cf-plot* (https://ncas-cms.github.io/cf-plot/build/). We will be focusing on use of cf-python and cf-plot today.\n", + "\n", + "The 'cf' in the names of the NCAS CF Data Tools corresponds to the _CF Conventions_, a metadata standard, because they are built around this standard in the form of using the CF Data Model, which as well as performance is considered a 'unique selling point' of the tools.\n", + "\n", + "\n", + "### What are the CF Conventions?\n", + "\n", + "The _CF Conventions_, usually referred to in this way but also know by the full name of the **C**limate and **F**orecast (CF) metadata conventions, are a metadata standard which is becoming the de-facto convention to cover the description of geoscientific data so that sharing and intercomparison is simpler. See https://cfconventions.org/ for more information.\n", + "\n", + "\n", + "### What are we going to learn in this session?\n", + "\n", + "Our **learning aim** is to be able to use the NCAS CF Data Tools Python libraries, namely cf-python and cf-plot to process, analyse and visualise netCDF and PP datasets, whilst appreciating the context and 'unique selling point' of the libraries as being built to use the CF Conventions, a metadata standard for earth science data, to make it simpler to do what you want to do with the datasets, by working on top of a Data Model for CF.\n", + "\n", + "We have **six distinct objectives**, matching the sections in this notebook and in the practical notebook you will work through. By the end of this lesson you should be familiar and have practiced using cf-python and cf-plot to:\n", + "\n", + "1. read dataset(s) and view the (meta)data at different detail levels;\n", + "2. edit the (meta)data and write out the edited version to file;\n", + "3. reduce datasets by subspacing and collapsing;\n", + "4. visualise datasets as contour and vector plots;\n", + "5. analyse data: applying mathematical and statistical operations and plotting trends;\n", + "6. change the underlying grid of data through regridding." + ] + }, + { + "cell_type": "markdown", + "id": "b9d89bbc-e583-45cc-a79f-cf5fdb783050", + "metadata": {}, + "source": [ + "
\n", + "Note: much of what you can do with cf-python you can do with the xarray library. Use whichever approach, the cf-python/cf-plot way, or the xarray way, works best for you! However, we want to emphasise that the NCAS CF Data Tools are built around the CF Conventions whereas xarray is not, so cf-python and cf-plot offer better metadata awareness to xarray, which could be a core advantage to our approach for users in/from geoscience. (If you have suggestions for how we can improve cf-python and/or cf-plot for you or your work, please let us know through the Issue Trackers linked at the end of this Notebook.)\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "1c3beb87-7b0d-44ad-9da3-fa32494b795c", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "196e9e0a-83d1-464c-b53c-a6fc2e0c19ff", + "metadata": {}, + "source": [ + "## Setting up\n", + "\n", + "**In this section we set up this Notebook, import the libraries and check the data we will work with, ready to use the libraries within this notebook.**" + ] + }, + { + "cell_type": "markdown", + "id": "d073760a-6734-4a1d-9832-a70a8e0dd089", + "metadata": {}, + "source": [ + "Import cf-python and cf-plot:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9640ef13-50d2-43fe-9a84-5ce701bdfcbc", + "metadata": {}, + "outputs": [], + "source": [ + "import cfplot as cfp\n", + "import cf" + ] + }, + { + "cell_type": "markdown", + "id": "ab792505-5452-4f44-b194-6181e977385a", + "metadata": {}, + "source": [ + "Run some set up for nice outputs in this Jupyter Notebook (not required in interactive Python or a script):" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9bfec50d-94c0-442a-a74d-0a4deb5c4bb0", + "metadata": {}, + "outputs": [], + "source": [ + "import warnings\n", + "warnings.filterwarnings('ignore')\n", + "\n", + "%matplotlib inline" + ] + }, + { + "cell_type": "markdown", + "id": "7b00d47b-4497-4540-b1bd-6fa8fa8e3226", + "metadata": {}, + "source": [ + "Inspect the versions of cf-python and cf-plot and the version of the CF Conventions those are matched to:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ea75b1b6-ba78-428d-b9b2-ca29a8cf6f0d", + "metadata": {}, + "outputs": [], + "source": [ + "print(\"cf-python version is:\", cf.__version__)\n", + "print(\"cf-plot version is:\", cfp.__version__)\n", + "print(\"CF Conventions version is:\", cf.CF())" + ] + }, + { + "cell_type": "markdown", + "id": "628af6cc-24f1-4e85-bf8b-654268e72bd3", + "metadata": {}, + "source": [ + "
\n", + "Note: you can work with data compliant by any other version of the CF Conventions, or without (much) compliance, but the CF Conventions version gives the maximum version that these versions of the tools understand the features of.\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "8135f5c6-3d3e-45b6-a84a-3dcea754372f", + "metadata": {}, + "source": [ + "Finally, see what datasets we have to explore:" + ] + }, + { + "cell_type": "markdown", + "id": "cfab63fe-088d-4ca8-ab95-40f61f0c2fd3", + "metadata": {}, + "source": [ + "
\n", + "Note: in a Jupyter Notebook, '!' preceeeds a shell command, so this is a terminal command and not Python\n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a45afce7-abd8-4edd-b14c-eb033566bd52", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "!ls ../data" + ] + }, + { + "cell_type": "markdown", + "id": "e7640cdb-0d6f-4933-86d7-e86e7b9acbe4", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "3b7a3c0a-262d-4ac4-8c10-6fa3a42ae73a", + "metadata": {}, + "source": [ + "## 1. Reading dataset(s) and viewing the (meta)data at different detail levels\n", + "\n", + "**In this section we look at a basic use of cf-python, reading in one or more datasets from file and inspecting the data and the metadata at different levels of detail to suit the amount of information you want to see.**" + ] + }, + { + "cell_type": "markdown", + "id": "15de8a85-99ae-40a8-a429-34c30a3e1c76", + "metadata": {}, + "source": [ + "
\n", + "Note: In cf-python and when discussing related code and datasets, we use terminology from the CF Data Model (for more detail see: https://ncas-cms.github.io/cf-python/cf_data_model.html). For example cf-python methods are named in relation to concepts from this data model. We don't have time to cover this in detail but for this session it is useful to know the following terms:\n", + "\n", + "\n", + "
\n", + "\n", + "The examples from this section should help you to familiarise yourself with these terms and their practical usage." + ] + }, + { + "cell_type": "markdown", + "id": "9da08bf9-a15b-4d65-8c8e-632897daec0b", + "metadata": {}, + "source": [ + "### a) Reading in data and extracting the _field_ of interest" + ] + }, + { + "cell_type": "markdown", + "id": "60d6c6a4-a3cd-4515-9d7c-e0e5b0fdc1f3", + "metadata": {}, + "source": [ + "Read a chosen data file. Sometimes datasets have descriptive names but this one doesn't, so let's find out what it is!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "67b1cd25-dedb-409b-8a57-3fff5de333dd", + "metadata": {}, + "outputs": [], + "source": [ + "fieldlist = cf.read(\"../data/data1.nc\")" + ] + }, + { + "cell_type": "markdown", + "id": "3e49e1f9-3246-4e67-ac94-a322937c2c42", + "metadata": {}, + "source": [ + "See the 'fieldlist' that cf-python interprets from the data read in:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "70dfc26a-3163-4b56-948d-98a27472dce4", + "metadata": {}, + "outputs": [], + "source": [ + "fieldlist" + ] + }, + { + "cell_type": "markdown", + "id": "b99aa858-bdd3-48d4-b91b-b6fd122fdcb4", + "metadata": {}, + "source": [ + "Select a particular field from the fieldlist of interest:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "269896a7-4f55-4792-b9be-fcbc7d931edc", + "metadata": {}, + "outputs": [], + "source": [ + "field = fieldlist[0]" + ] + }, + { + "cell_type": "markdown", + "id": "d4ed8112-fc35-496c-84ae-e8a05334c33c", + "metadata": {}, + "source": [ + "### b) Inspecting the _field_ of interest with different amounts of detail" + ] + }, + { + "cell_type": "markdown", + "id": "b78cc22c-c52b-4d8d-bcab-15da6752b592", + "metadata": {}, + "source": [ + "View the field with **minimal detail**, i.e. a one-line summary:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "84ab31d2-814d-41a7-8da6-50ab6f9a0969", + "metadata": {}, + "outputs": [], + "source": [ + "field" + ] + }, + { + "cell_type": "markdown", + "id": "9a9af4ea-270a-49e9-bc49-54c36158bf09", + "metadata": {}, + "source": [ + "Or you can view it with a **medium level of detail** with the Python built-in `print` function:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "01618a8c-928a-4250-ba3c-4fef28bfa19e", + "metadata": {}, + "outputs": [], + "source": [ + "print(field)" + ] + }, + { + "cell_type": "markdown", + "id": "f8c2af1e-fd8d-415f-b430-d521e68b98ea", + "metadata": {}, + "source": [ + "A final option is to view it with **maximal detail** using the `dump()` method:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cb1170b2-dd14-4ebf-8bc4-7e0559c8a4a0", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "field.dump()" + ] + }, + { + "cell_type": "markdown", + "id": "0af01357-b0b2-4581-a830-437337565d0d", + "metadata": {}, + "source": [ + "### c) Inspecting a metadata _construct_ e.g. _coordinate_ from the _field_ of interest" + ] + }, + { + "cell_type": "markdown", + "id": "0f348261-09fe-4193-ab6f-3203cbd36e40", + "metadata": {}, + "source": [ + "Use the same approach to view a particular metadata aspect, for example the latitude coordinate:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2d907740-67af-4c28-a13d-6a2a707cc531", + "metadata": {}, + "outputs": [], + "source": [ + "lat = field.coordinate(\"latitude\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ec72414b-cebd-4bb6-bf8b-b7f13ebf394d", + "metadata": {}, + "outputs": [], + "source": [ + "lat" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "307259f7-8528-4fe9-8587-449a70836461", + "metadata": {}, + "outputs": [], + "source": [ + "print(lat)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7e342afb-519f-42c6-889a-aecfb1e1ced6", + "metadata": {}, + "outputs": [], + "source": [ + "lat.dump()" + ] + }, + { + "cell_type": "markdown", + "id": "bd7fd831-8362-49eb-a03a-281d75596309", + "metadata": {}, + "source": [ + "### d) Inspecting a data array of interest" + ] + }, + { + "cell_type": "markdown", + "id": "fb66c5c4-c535-4710-bfdc-bfb6d0fde827", + "metadata": {}, + "source": [ + "Likewise, the same approach works to view the data itself in the field (i.e. the underlying arrays). First, grab the data from the field with the `data` attribute:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "562b2a02-e61a-472e-85e2-7020f704e0cb", + "metadata": {}, + "outputs": [], + "source": [ + "data = field.data" + ] + }, + { + "cell_type": "markdown", + "id": "b385616e-a130-4bf1-a197-f0ae8e193948", + "metadata": {}, + "source": [ + "Then view it in a chosen level of detail as with the above objects:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e650fb5e-319a-41a1-bae9-b9d7e99a43c9", + "metadata": {}, + "outputs": [], + "source": [ + "data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e07f7420-37d1-4ba4-8cd8-f068425301bd", + "metadata": {}, + "outputs": [], + "source": [ + "print(data)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0147639e-56a4-4a50-9580-02ba0b557493", + "metadata": {}, + "outputs": [], + "source": [ + "data.dump()" + ] + }, + { + "cell_type": "markdown", + "id": "4aa94e2b-7226-4846-be7d-696e3996eb8f", + "metadata": {}, + "source": [ + "If you want to see more of the data array itself, you can access it with the `array` attribute. Beware, for real-life datasets:\n", + "\n", + "* this will be large and Python will likely truncate it so your screen isn't spammed with sub-arrays of values!\n", + "* it is computationally intensive to access the underlying data array if it is large, especially if it is multi-dimensional, so your computer will often have to work hard to get the array, so use the `array` method sparingly (only when needed)!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "89d5c9f7-423f-4e03-9517-be40143e5aa4", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "data.array" + ] + }, + { + "cell_type": "markdown", + "id": "09d04ff2-8998-48fb-a0a6-bc5deafe8fc6", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "932d2d38-b8d7-46cc-848f-b6f50b877310", + "metadata": {}, + "source": [ + "## 2. Editing the (meta)data and writing out the edited version to file\n", + "\n", + "**In this section we demonstrate how to change the data that has been read-in from file, both in terms of the data arrays and the metadata that describes it, and then how to write data back out to file with a chosen name, so that you can see how cf-python can be used to edit data or to make new data.**" + ] + }, + { + "cell_type": "markdown", + "id": "cf3c9786-528f-49c8-8388-574062112206", + "metadata": {}, + "source": [ + "Using the same data file from te previous section, let's say we want to change the data and metadata of this. As-is the field and its data are:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7af83b82-0258-4ba2-bb3e-7c0c0ae6e929", + "metadata": {}, + "outputs": [], + "source": [ + "print(\"Field is:\", field, \"Data is:\", data, sep=\"\\n\\n\")" + ] + }, + { + "cell_type": "markdown", + "id": "bc7c654e-38e5-4b06-a7c0-01828701d92f", + "metadata": {}, + "source": [ + "### a) Changing the underlying data" + ] + }, + { + "cell_type": "markdown", + "id": "9acf4da4-881b-46c7-88f3-8fe72972cf1b", + "metadata": {}, + "source": [ + "To change the data, use assignment to the relavant index or indices. For example, to change all values we can use the special index of an ellipsis like so, in this case changing them all to an identical scalar value:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c3983e97-ffc8-4799-9a29-fa88e036a169", + "metadata": {}, + "outputs": [], + "source": [ + "data[...] = 10.0\n", + "data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "dc968051-56f8-4814-a79b-be3402e0b285", + "metadata": {}, + "outputs": [], + "source": [ + "print(data)" + ] + }, + { + "cell_type": "markdown", + "id": "06e5ef92-f3ab-4b5f-b004-4e2fa41429e5", + "metadata": {}, + "source": [ + "Or could change more specifically just one sub-array of these to a different value" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b168b278-0418-4ed0-85cf-de6d0e795638", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "data[0, 0, 0] = 3.0\n", + "data.array" + ] + }, + { + "cell_type": "markdown", + "id": "ba4ebc7f-45c4-4c9c-a7e7-4b7cafe2d305", + "metadata": {}, + "source": [ + "Instead of setting the whole sub-array to one value, you can set the whole array to your precise specification, for example:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bbe84007-811a-4290-bfbe-9e222bded840", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "data[0, 0, 0] = range(320)\n", + "data.array" + ] + }, + { + "cell_type": "markdown", + "id": "0f2d2721-58c2-44d9-b422-3dd81ee8b333", + "metadata": {}, + "source": [ + "### b) Changing some metadata" + ] + }, + { + "cell_type": "markdown", + "id": "3286f679-2c8a-40a8-b7d4-b67f34122e7d", + "metadata": {}, + "source": [ + "To change metadata, first get the metadata you want to change as an object. One of the most flexible ways to do so is to use the `construct` method and as an argument specify the name of the coordinate you are interested in:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4f81cbe5-5d1d-4d30-9388-2750cc81d613", + "metadata": {}, + "outputs": [], + "source": [ + "pressure = field.construct(\"pressure\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2fbbdb05-94e4-432b-b54e-2675901e6d75", + "metadata": {}, + "outputs": [], + "source": [ + "print(pressure)\n", + "print(pressure.data)" + ] + }, + { + "cell_type": "markdown", + "id": "4987c89e-e4c0-4c91-aa2e-fa78ff9f617e", + "metadata": {}, + "source": [ + "You can inspect the units specifically using the `units` attribute:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c2e55b76-f9e7-41cc-b11f-995c0bc0f2eb", + "metadata": {}, + "outputs": [], + "source": [ + "print(pressure.units)" + ] + }, + { + "cell_type": "markdown", + "id": "3051327c-e237-4cdf-adb3-74d5c65d28b4", + "metadata": {}, + "source": [ + "Let's change the units to an equivalent but different unit, the `bar` (out by a factor of 1000), as an example:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f0f6b8a5-ab63-42b9-8b59-1dfe68d48294", + "metadata": {}, + "outputs": [], + "source": [ + "pressure.units = \"bar\"\n", + "print(pressure.units)" + ] + }, + { + "cell_type": "markdown", + "id": "8c15baee-fdaf-4015-b265-41920e6c6d5e", + "metadata": {}, + "source": [ + "Notice how the data has been converted to account for the new units - cf-python's metadata awareness makes contextual changes like this so we don't have to do it manually!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "35d51c76-7c36-4b7b-98c6-85df363259da", + "metadata": {}, + "outputs": [], + "source": [ + "print(pressure.data)" + ] + }, + { + "cell_type": "markdown", + "id": "d3e11a1b-9841-49ff-a74f-5e27e5b08629", + "metadata": {}, + "source": [ + "Note how the pressure units are changed in the field too, since we edited the same object in a Pythonic sense:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e2fa9715-2ac2-4d22-aaec-23edb764708c", + "metadata": {}, + "outputs": [], + "source": [ + "print(field)" + ] + }, + { + "cell_type": "markdown", + "id": "cf7cf826-2d23-40cd-bfca-648de59fd02c", + "metadata": {}, + "source": [ + "### c) Writing a (list of) fields out to a file" + ] + }, + { + "cell_type": "markdown", + "id": "38de7492-13b0-4983-b943-16e23e7851ed", + "metadata": {}, + "source": [ + "We changed some metadata (units) and the data itself from our dataset read-in from file. Let's write the new data out\n", + "as a new file and read it back in to show that it has been changed relative to the original. You write files out to disk using the `write` function with an argument giving the path, including the name (it can _just_ be the name to write a file to the current working directory), you want to create the file to:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e697f71b-3f6c-45ba-b4bc-547d26e1e8e8", + "metadata": {}, + "outputs": [], + "source": [ + "cf.write(field, \"../data/data1-updated.nc\")" + ] + }, + { + "cell_type": "markdown", + "id": "86c8650c-23a4-40f7-9a26-a9aba1b8ee6a", + "metadata": {}, + "source": [ + "See that it was written out to the directory we specified:" + ] + }, + { + "cell_type": "markdown", + "id": "8ce47770-6ee4-482c-8774-16136e235b50", + "metadata": {}, + "source": [ + "
\n", + "Note: in a Jupyter Notebook, '!' preceeeds a shell command, so this is a terminal command and not Python\n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5ef24395-3a11-45d5-b91e-a742e82d80fb", + "metadata": {}, + "outputs": [], + "source": [ + "!ls ../data" + ] + }, + { + "cell_type": "markdown", + "id": "8a0f2ed2-51b2-4cf2-9529-4acf7cf1ff15", + "metadata": {}, + "source": [ + "To check it wrote out the edited version from this Notebook, we can read the file back in and inspect it again:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ac357e9f-85e1-4139-9a22-30c5cc0427da", + "metadata": {}, + "outputs": [], + "source": [ + "updated_fieldlist = cf.read(\"../data/data1-updated.nc\")\n", + "reread_field = updated_fieldlist[0]" + ] + }, + { + "cell_type": "markdown", + "id": "ae8fd9e7-c789-4e7d-8773-074bdbb8c2ce", + "metadata": {}, + "source": [ + "See what `g` is by medium detail inspection:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d08cff69-4bc8-4862-bfc2-06acce62ac19", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "print(reread_field)\n", + "print(reread_field.data)" + ] + }, + { + "cell_type": "markdown", + "id": "0a4f1a78-6dcc-4a65-b927-54b9c044a55d", + "metadata": {}, + "source": [ + "Notice the pressure coordinate units are 'bar' as per our change and the first data array item starts with `0.0` and the final one ends with `10.0` as per our change." + ] + }, + { + "cell_type": "markdown", + "id": "9dd6be4b-60d3-4ca8-8108-fcda62bb7d95", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "4ccfd6a0-567d-43f3-9f06-15bb21afae6a", + "metadata": {}, + "source": [ + "## 3. Reducing datasets by subspacing and collapsing\n", + "\n", + "**In this section we show how multi-dimensional data can be tamed using cf-python so that you can get a reduced form that can be analysed or plotted, by reducing the dimensions by selecting a subset of point(s) along the axes or collapsing down according to some statistic such as the mean or an extrema.**" + ] + }, + { + "cell_type": "markdown", + "id": "d6899662-eaaa-480c-8f0a-4a6f069292a8", + "metadata": {}, + "source": [ + "Often datasets represent highly multi-dimensional data, for example 4D or higher. Usually we want to find a either a sub-space, or a statistical representation (such as an average or extrema), of the full data array in less dimensions, such as in 3D or 2D or even in the form of a 1D time series or 0D statistic.\n", + "\n", + "We'll demonstrate this with another dataset and field selected from it. This serves as a reminder on concepts from the first section of the Notebook:" + ] + }, + { + "cell_type": "markdown", + "id": "e635599e-aa0b-4f48-a49e-28ec8aab9ef7", + "metadata": {}, + "source": [ + "
\n", + "Note: here we are reading in a file in the 'PP' format, which is a file format originating from the Met Office that can often be encountered in geoscience, like netCDF, hence the file extension '.pp'. You can read and process PP files using cf-python exactly the same way you do for netCDF files, so you do not need to concern yourself with the difference in file format in your code.\n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "57d21990-e813-41c5-b686-9c90deba307e", + "metadata": {}, + "outputs": [], + "source": [ + "field2 = cf.read(\"../data/aaaaoa.pmh8dec.pp\")[2]\n", + "print(field2)" + ] + }, + { + "cell_type": "markdown", + "id": "a019f77b-1824-44f8-a534-06445f1ec48f", + "metadata": {}, + "source": [ + "In this case, see the numbers representing axes sizes on the 'Data' line: we have a 3D field where the axes sizes are 17 for air pressure, 30 for grid latitude and 24 for grid latitude. To reduce this to a 2D form, we need to take one of the non-zero axes and convert it to size 1. This can be done either by **subspacing** or by **statistically collapsing** it." + ] + }, + { + "cell_type": "markdown", + "id": "410ac6d4-cd8f-43ed-a3f5-eaf8d1d8755a", + "metadata": {}, + "source": [ + "### a) Subspacing using metadata conditions" + ] + }, + { + "cell_type": "markdown", + "id": "8ed6a2a2-3b35-40b7-9275-eb339eeae23e", + "metadata": {}, + "source": [ + "Use the `subspace` method to find a subspace of a field, the output of which is another field reduced down in the way specified by the method arguments:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d4b98910-09ef-4948-93c7-f85d112ce33b", + "metadata": {}, + "outputs": [], + "source": [ + "field2_subspace1 = field2.subspace(air_pressure=1000.0000610351562) # taking first value\n", + "print(field2_subspace1)" + ] + }, + { + "cell_type": "markdown", + "id": "19ac33bb-384c-413b-965f-992ba0934a3f", + "metadata": {}, + "source": [ + "Let's do the same but to subspace to reduce the data along a different axis, this time `grid_latitude`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1e71740d-1e23-4295-adf3-15dcdae740e0", + "metadata": {}, + "outputs": [], + "source": [ + "field2_subspace2 = field2.subspace(grid_latitude=-5.279999852180481) # taking the last value\n", + "print(field2_subspace2)" + ] + }, + { + "cell_type": "markdown", + "id": "bb1db98b-27b3-4c8a-bf63-f67f83dd2bb8", + "metadata": {}, + "source": [ + "### b) Subspacing using indexing, including equivalency to the above" + ] + }, + { + "cell_type": "markdown", + "id": "918b145a-0fb1-446a-a003-6e7bb75601d1", + "metadata": {}, + "source": [ + "We can also use indexing to do a subspace. So, instead of picking out a given value from the printed information,\n", + "we can use a specific Python index to pick it out.\n", + "\n", + "The `air_pressure` coordinate is listed as first in axes order, so you must specify the index at the first position of the three i.e. `[, :, :]` where `:` means to not take a subspace and leave the whole axis as it was. For example, the following takes the first (position 0 in Python indexing) value of the `air_pressure`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "abdd7e85-9ae2-4d63-b69b-56a50ca7775b", + "metadata": {}, + "outputs": [], + "source": [ + "field2_subspace1_by_index = field2[0, :, :] # taking first value from first coordinate\n", + "print(field2_subspace1_by_index)" + ] + }, + { + "cell_type": "markdown", + "id": "8571fdd2-4262-4979-8430-e3738327faa6", + "metadata": {}, + "source": [ + "To prove that this is the same field as we got using the direct `subspace` method from the sub-section above, namely `h.subspace(air_pressure=1000.0000610351562)`, we can use the `equals` method, which states whether one field is identical to another:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "37028743-07e5-454c-becb-bbb1f9e5c71f", + "metadata": {}, + "outputs": [], + "source": [ + "field2_subspace1_by_index.equals(field2_subspace1)" + ] + }, + { + "cell_type": "markdown", + "id": "85f044f1-f4e6-4110-a50c-acd58838a3c3", + "metadata": {}, + "source": [ + "Similarly, the `grid_latitude` coordinate is listed as second in axes order, so you must specify the index at the second position of the three i.e. `[:, , :]`. Let's take the last (position -1 in Python indexing) value from this, like so:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d42f36cf-4772-4081-94d9-b03a571fdb00", + "metadata": {}, + "outputs": [], + "source": [ + "field2_subspace2_by_index = field2[:, -1, :] # taking last value from second coordinate\n", + "print(field2_subspace2_by_index)" + ] + }, + { + "cell_type": "markdown", + "id": "40204dfd-a51a-4597-876b-70a0127fdd09", + "metadata": {}, + "source": [ + "Again, to prove that this is the same field as we got as using the direct `subspace` approach previously with `h.subspace(grid_latitude=-5.279999852180481)`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c30fcf5e-fc93-49bc-9ab1-41406fa76954", + "metadata": {}, + "outputs": [], + "source": [ + "field2_subspace2_by_index.equals(field2_subspace2)" + ] + }, + { + "cell_type": "markdown", + "id": "1883b43f-521e-4653-ac36-1069e62fe25d", + "metadata": {}, + "source": [ + "You can do multiple subspaces at once via either of the methods above, so for example you can combine the two separate subspaces into one call:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f44273de-e39b-406a-bdbc-02d6e23a1412", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "field2_subspace3 = field2.subspace(air_pressure=1000.0000610351562, grid_latitude=-5.279999852180481)\n", + "field2_subspace3_by_index = field2[0, -1, :]" + ] + }, + { + "cell_type": "markdown", + "id": "36b7314b-cabf-4101-8cdc-3f2ec9c468ff", + "metadata": {}, + "source": [ + "This results in (in both cases):" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "89485fca-8101-4e16-a0c4-13f4f7e38da9", + "metadata": {}, + "outputs": [], + "source": [ + "field2_subspace3.equals(field2_subspace3_by_index)\n", + "print(field2_subspace3)" + ] + }, + { + "cell_type": "markdown", + "id": "b870563e-6f45-46ca-9dad-6d8a5cb69ec5", + "metadata": {}, + "source": [ + "### c) Statistical collapses" + ] + }, + { + "cell_type": "markdown", + "id": "502933e9-f4d9-42d9-bc40-a2b2ffa75958", + "metadata": {}, + "source": [ + "Instead of extracting the data at a particular value from the `air_pressure` dimension coordinate, we might want to collapse the data down according to a representative statistic covering all of those values, such as an average or extrema value. We do this with the `collapse` method.\n", + "\n", + "Say we want to get the _mean_ of all of the air pressure values to reduce that coordinate from having the 17 values to just one mean representation, we would do this as follows:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f6745c82-b11d-4cca-a0f7-4dd825f446cb", + "metadata": {}, + "outputs": [], + "source": [ + "field2_collapse1 = field2.collapse(\"air_pressure: mean\") # taking mean of the 17 values for air pressure\n", + "print(field2_collapse1)" + ] + }, + { + "cell_type": "markdown", + "id": "32834e16-5922-4a3d-a466-630ae3791de0", + "metadata": {}, + "source": [ + "Another example is taking the _minimum_ of all of the values across the grid latitudes:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1e650cc2-455b-462b-b078-ab8c99b0413e", + "metadata": {}, + "outputs": [], + "source": [ + "field2_collapse2 = field2.collapse(\"grid_latitude: minimum\") # taking minimum of the 30 values for the grid latitude\n", + "print(field2_collapse2)" + ] + }, + { + "cell_type": "markdown", + "id": "60b3f3a1-3a17-4a6d-bdaa-a225e7feecb3", + "metadata": {}, + "source": [ + "Equivalently, you can specify the axes via an `axes` argument when the coordinate to collapse along is one of `X`, `Y`, `Z` or `T`. So this `collapse` call will give the same result:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "84caa2f2-1e13-493e-aeb8-5a3c7bd7c763", + "metadata": {}, + "outputs": [], + "source": [ + "field2_collapse3 = field2.collapse(\"minimum\", axes=\"Y\") # taking minimum of the 30 values for the grid latitude\n", + "print(field2_collapse3)" + ] + }, + { + "cell_type": "markdown", + "id": "08377cd7-54d1-4de6-8a9f-82aff981fccf", + "metadata": {}, + "source": [ + "Proving that this gives the same result as with the `\"grid_latitude: minimum\"` argument:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ff981201-b983-4f18-a230-3b9c7a2ebf07", + "metadata": {}, + "outputs": [], + "source": [ + "field2_collapse3.equals(field2_collapse2)" + ] + }, + { + "cell_type": "markdown", + "id": "59b69a9d-a94c-464c-aa3c-482c699bc404", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "80edfa76-7f87-42b9-8cef-058304ca902a", + "metadata": {}, + "source": [ + "## 4. Visualising datasets as contour and vector plots\n", + "\n", + "**In this section we demonstrate how to plot using cf-plot the data we have read and then processed and/or analysed using cf-python, notably showing how to create contour plots and vector plots as examples of some of the available plot types.**" + ] + }, + { + "cell_type": "markdown", + "id": "079820ff-aa40-48c6-8775-46d6b12aec69", + "metadata": {}, + "source": [ + "For this section let's use other fields within the fieldlist we read in for section one, where we investigated the first of the four fields (index 0) in that section:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a71db8e6-d3a7-4bef-b580-143fff2ffe88", + "metadata": {}, + "outputs": [], + "source": [ + "fieldlist" + ] + }, + { + "cell_type": "markdown", + "id": "e227e533-3651-4d92-8c99-c89a8258c310", + "metadata": {}, + "source": [ + "This time let's investigate the other (final) three fields. First we unpack these to variables to save us applying the indexing multiple times (note we assign variable names `fieldN` to a number `N` incremented with fields read and assigned to variable names from the start of the Notebook, not with numbers corresponding to the index position):" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "481c51b9-40cc-4949-bfa6-e77d2a7962c0", + "metadata": {}, + "outputs": [], + "source": [ + "temp_field = fieldlist[1]\n", + "wind_u_field = fieldlist[2]\n", + "wind_v_field = fieldlist[3]" + ] + }, + { + "cell_type": "markdown", + "id": "e400727a-e49e-4d51-84b5-c263594b41f9", + "metadata": {}, + "source": [ + "Now let's see what the fields are like, with medium detail, using a new-line separated print to break up the outputs:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "60cdd349-a7df-4788-8332-5fee2cbe3598", + "metadata": {}, + "outputs": [], + "source": [ + "print(\"'temp_field' is:\", temp_field, \"\\n\")\n", + "print(\"'wind_u_field' is:\", wind_u_field, \"\\n\")\n", + "print(\"'wind_v_field' is:\", wind_v_field, \"\\n\")" + ] + }, + { + "cell_type": "markdown", + "id": "794c4940-43fd-45a3-ba9a-69084c0e5ccd", + "metadata": {}, + "source": [ + "### a) Making a contour plot" + ] + }, + { + "cell_type": "markdown", + "id": "3389b8d4-a3ff-415f-9507-cf3baa82c5a1", + "metadata": {}, + "source": [ + "All of these fields, as we can see from the latitude and longitude value ranges, span the whole globe in extent. Another quick way to gauge this, and plenty of other information about a field notably the data itself and its form and pattern(s), is to plot it.\n", + "\n", + "To make a contour (level) plot of a field, use the `con` function of cf-plot. It requires as its one positional argument a field with only two axes which are greater than size one in order to have an effective 2D field, often a 'slice' i.e. subspace or collapse of a higher-dimensional field, it can plot.\n", + "\n", + "For example, for our fields above we need to reduce one axis down to achieve this, so for example we can reduce the `pressure` coordinate. Let's take a subspace at a given pressure (we take 1000 mbar, the highest in the field), and plot it:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3e9b510f-0acc-4e48-ad2b-cc8b4e56fa13", + "metadata": {}, + "outputs": [], + "source": [ + "cfp.con(temp_field.subspace(pressure=1000.0))" + ] + }, + { + "cell_type": "markdown", + "id": "9e2e7348-0df5-4c97-aab5-73e73a3bca0f", + "metadata": {}, + "source": [ + "Let's see what the equivalent plot would be for the subspace over a different pressure, this time the lowest 1 mbar:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "18589adf-83eb-4ffb-a43c-ca4a19c1a6d8", + "metadata": {}, + "outputs": [], + "source": [ + "cfp.con(temp_field.subspace(pressure=1.0))" + ] + }, + { + "cell_type": "markdown", + "id": "b19186dc-3480-47d1-a859-79008841bd58", + "metadata": {}, + "source": [ + "### b) Customising the (contour) plot" + ] + }, + { + "cell_type": "markdown", + "id": "3f9f60bb-a492-447c-b5ba-d248eec9fe38", + "metadata": {}, + "source": [ + "cf-plot provides numerous ways to customise a specific plot such as those above. We demonstrate just a small number here.\n", + "\n", + "The second plot from the previous sub-section, in particular, has contour lines which are so dense they obscure the world map and data pattern so let's hide those which you do by specifying `lines=False` to the `con` call:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ece996ff-6da0-42be-8d45-02de998756b9", + "metadata": {}, + "outputs": [], + "source": [ + "cfp.con(temp_field.subspace(pressure=1.0), lines=False)" + ] + }, + { + "cell_type": "markdown", + "id": "6285497e-e579-48cf-a10a-032ddf4d944f", + "metadata": {}, + "source": [ + "Let's also do some further customisation of the plot above, as follows:\n", + "\n", + "- use a different map projection;\n", + "- set a different colour map for the data;\n", + "- move the colour bar to be vertical not horizontal;\n", + "- add a title:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fba81794-1652-44cb-bed1-f152cc068a84", + "metadata": {}, + "outputs": [], + "source": [ + "cfp.mapset(proj=\"robin\") # changes the projection\n", + "cfp.cscale(\"plasma\") # changes the colour scale/map\n", + "cfp.con(\n", + " temp_field.subspace(pressure=1.0),\n", + " lines=False,\n", + " colorbar_orientation=\"vertical\", # tells cf-plot to make a vertical colour bar (horizontal is the default)\n", + " title=\"Air temperaure field\" # adds the specified title\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "306d0813-dc38-44ff-8b38-705245be209c", + "metadata": {}, + "source": [ + "We might want to view a smaller section of the data, say over Australia only. We can do this in two ways:\n", + "\n", + "1. by reducing the data further; or\n", + "2. by specifying to cf-plot to only show a smaller window on the data.\n", + "\n", + "For (1), instead of a subspace on just pressure, we also suspace on X and Y i.e. longitude and latitude, to reduce those axes. We can use `cf.wi` which specifies a 'within' condition. First, define the extrema of both of these we want to view, for example these are those longitudes and latitudes which provide a view centered around Australia:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f9e08f06-969d-4019-a82e-210dabe28470", + "metadata": {}, + "outputs": [], + "source": [ + "cfp.mapset() # resets the projection\n", + "lonmin = 110\n", + "lonmax = 155\n", + "latmin = -45\n", + "latmax = -5" + ] + }, + { + "cell_type": "markdown", + "id": "46721929-6a3a-4ad5-8388-fc183d1bd7d1", + "metadata": {}, + "source": [ + "Now do method (1) from above, via a subspace on three axes, pressure (as before) as well as longitude and latitude (new). Note we could do each subspace via a separate call, but they can also be done together in one call like so:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fab363a9-53b5-4338-8572-2fea2fd01029", + "metadata": {}, + "outputs": [], + "source": [ + "# Do further subspacing to reduce the latitude and longitude with cf-python\n", + "fsub_all = temp_field.subspace(\n", + " pressure=1.0,\n", + " longitude=cf.wi(lonmin, lonmax),\n", + " latitude=cf.wi(latmin, latmax)\n", + ")\n", + "\n", + "# Plot the above\n", + "cfp.con(fsub_all, lines=False)" + ] + }, + { + "cell_type": "markdown", + "id": "2620c019-4e45-4d37-a788-cf5eb76c1915", + "metadata": {}, + "source": [ + "Method (2) can be achieved by setting the keyword arguments to the cf-plot `mapset` method like so:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ffe229db-b678-465a-9f13-f06e0d00b120", + "metadata": {}, + "outputs": [], + "source": [ + "cfp.mapset(lonmin=lonmin, lonmax=lonmax, latmin=latmin, latmax=latmax)\n", + "cfp.con(temp_field.subspace(pressure=1.0), lines=False)" + ] + }, + { + "cell_type": "markdown", + "id": "290bdc79-069f-49e1-a997-81413f35c398", + "metadata": {}, + "source": [ + "Note the differences in colour shades between the two plots from the different approaches are because in approach (2) cf-plot normalises the colour map extent of the data to only account for the area shown." + ] + }, + { + "cell_type": "markdown", + "id": "6964260d-84fb-41d9-a1b9-a74e85a79c20", + "metadata": {}, + "source": [ + "### c) Making a vector plot with basic customisation" + ] + }, + { + "cell_type": "markdown", + "id": "8688ff02-0aa0-42ba-a38c-abcf6e3494ea", + "metadata": {}, + "source": [ + "Finally let's make some vector plots. Notice our final two fields from the read-in fieldlist are in eastward and northward components, therefore can be combined into vectors from these vector components:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d493bb8a-f031-4231-90bd-fa2a08d0400a", + "metadata": {}, + "outputs": [], + "source": [ + "print(wind_u_field)\n", + "print(wind_v_field)" + ] + }, + { + "cell_type": "markdown", + "id": "e58a7017-a0d3-4191-b85d-d4e72c251298", + "metadata": {}, + "source": [ + "Again we need to subspace in pressure so that we have an effective 2D not 4D space that can be visualised as a contour plot.\n", + "This time let's take our subspace at 500 mbar pressure:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0cce64a8-2b29-4ac1-9034-f9d82e501827", + "metadata": {}, + "outputs": [], + "source": [ + "u = wind_u_field.subspace(pressure=500.0)\n", + "v = wind_v_field.subspace(pressure=500.0)" + ] + }, + { + "cell_type": "markdown", + "id": "7f7ebfb0-73a9-492b-9e5e-614e26333ac6", + "metadata": {}, + "source": [ + "To make a vector plot using cf-plot, use the `vect` function:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6639ea78-4d27-4c82-81c6-cfee0587dc04", + "metadata": {}, + "outputs": [], + "source": [ + "cfp.mapset() # reset the mapping settings, including map projection\n", + "cfp.vect(u=u, v=v)" + ] + }, + { + "cell_type": "markdown", + "id": "fd721190-98da-4770-9842-fcd006bbe3b0", + "metadata": {}, + "source": [ + "We can mostly just see black here! This is expected and because we need to customise the vector spacing. There are clearly too many vectors shown they overwhelm the plot area, so use the `scale` and `stride` keyword arguments to `vect` to edit the size and spacing of the vector arrows respectively:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "72527e5d-cf24-4beb-9e19-eb90995af0c7", + "metadata": {}, + "outputs": [], + "source": [ + "cfp.vect(u=u, v=v, scale=100, stride=4) # much better - found these values after playing around with values" + ] + }, + { + "cell_type": "markdown", + "id": "3df50206-c6df-4ab0-bbd0-4c09976994b8", + "metadata": {}, + "source": [ + "Like for contour plotting, we can focus in on specific areas of the data in space via the two methods covered there. In the vector plot case, note we may need to adjust the vector spacing and size again for optimal viewing." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f6b8d5c-5c14-4e08-b539-b256a65046e4", + "metadata": {}, + "outputs": [], + "source": [ + "cfp.mapset(lonmin=lonmin, lonmax=lonmax, latmin=latmin, latmax=latmax) # also resets the projection to the default 'cyl'\n", + "cfp.vect(u=u, v=v)" + ] + }, + { + "cell_type": "markdown", + "id": "330207a2-dd98-480d-b828-7ed34b73f39d", + "metadata": {}, + "source": [ + "Trial and error for vector size and spacing gives a nicer result for the above plot with:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5575b4c9-6b78-4c1f-a157-40df44f028bd", + "metadata": {}, + "outputs": [], + "source": [ + "cfp.vect(u=u, v=v, scale=50, stride=1) # better than the above!" + ] + }, + { + "cell_type": "markdown", + "id": "b191c8b2-a038-4601-b046-997a4566c2f6", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "72524bb7-a643-4753-ad8b-f419656883fa", + "metadata": {}, + "source": [ + "## 5. Analysing data: applying mathematical and statistical operations and plotting trends\n", + "\n", + "**In this section we demonstrate how to do some data analysis including performing arithmetic and statistical calculations on the data, showing how cf-python's CF Conventions metadata awareness means that the metadata is automatically updated to account for the operations that are performed.**" + ] + }, + { + "cell_type": "markdown", + "id": "c7378c40-8398-4e3c-9906-d37f7f036ca7", + "metadata": {}, + "source": [ + "As well as the statistics you can calculate and explore from collapsing fields in section (3c), you can ..." + ] + }, + { + "cell_type": "markdown", + "id": "59637071-edcb-40fe-970a-1d89e0209aa7", + "metadata": {}, + "source": [ + "### a) Applying mathematics e.g. arithmetic and trigonometry on fields" + ] + }, + { + "cell_type": "markdown", + "id": "f1eb2eac-2a20-4eae-ba82-1cb3504d497a", + "metadata": {}, + "source": [ + "We will use another dataset to demonstrate this, to remind of the `read` function and fieldlist unpacking from section one:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "53ecf0e8-a0a5-46a2-9b50-df6ea0f2a346", + "metadata": {}, + "outputs": [], + "source": [ + "monthly_field = cf.read(\"../data/IPSL-CM5A-LR_r1i1p1_tas_n96_rcp45_mnth.nc\")[0]\n", + "print(monthly_field)\n", + "print(monthly_field.data)" + ] + }, + { + "cell_type": "markdown", + "id": "7139b32e-c39f-4387-8bec-76e290d70f1c", + "metadata": {}, + "source": [ + "You can perform arithmetical operations on fields using the usual operators, e.g. let's multiply the field's underlying data by 2 and subtract 10 from it to illustrate:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ccd2db42-6dae-48f6-b786-2deb15ec064b", + "metadata": {}, + "outputs": [], + "source": [ + "double_minus_ten_monthly_field = 2 * monthly_field - 10\n", + "print(double_minus_ten_monthly_field)\n", + "print(double_minus_ten_monthly_field.data)" + ] + }, + { + "cell_type": "markdown", + "id": "77db331e-2d98-4094-ab2e-646ccdfb9b3d", + "metadata": {}, + "source": [ + "You can apply mathematical operations too via various available methods. Let's try some rounding and trigonometry as an illustration:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "75c53609-a668-4f14-91f6-5b033e5af493", + "metadata": {}, + "outputs": [], + "source": [ + "round_monthly_field = monthly_field.round()\n", + "print(round_monthly_field.data)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ab014aaa-d778-4c15-abea-fcbe3d1f141c", + "metadata": {}, + "outputs": [], + "source": [ + "cosine_monthly_field = monthly_field.cos()\n", + "print(cosine_monthly_field)" + ] + }, + { + "cell_type": "markdown", + "id": "a1dddf74-b55a-46d5-bb75-17bac0a29747", + "metadata": {}, + "source": [ + "Note the units change appropriately with some operations, again showcasing cf-python's metadata awareness." + ] + }, + { + "cell_type": "markdown", + "id": "561caf84-0824-4b69-8ad9-82ce28d5bfd1", + "metadata": {}, + "source": [ + "### b) Line plotting" + ] + }, + { + "cell_type": "markdown", + "id": "e81d7d20-1e49-4183-838c-5e03993e19bc", + "metadata": {}, + "source": [ + "Often we want to pick out statistical trends from a subspace of the data. cf-python has season-selecting and season-collapsing methods to help you determine information on a per-season basis (as well as month-selecting methods to do the same for specific months, though we don't try any of those here).\n", + "\n", + "Firstly, let's show a line plot of our data averaged across the whole spatial area, so we can see the overall pattern. You can produce a line plot with cf-plot using the `lineplot` function, where the argument should be a field with a 1D series (ignoring any 1D axes such as latitude and longitude in our case which after the collapse become 1D):" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c15c3068-d84f-4719-9529-d856024267ff", + "metadata": {}, + "outputs": [], + "source": [ + "spatial_mean_monthly_field = monthly_field.collapse(\"area: mean\")\n", + "print(spatial_mean_monthly_field)\n", + "\n", + "cfp.lineplot(spatial_mean_monthly_field)" + ] + }, + { + "cell_type": "markdown", + "id": "682335b3-5eba-416c-bbd2-182f5915df7d", + "metadata": {}, + "source": [ + "### c) Calculating seasonal trends" + ] + }, + { + "cell_type": "markdown", + "id": "e7a55b54-c513-4b5a-8e9c-677c051fcf04", + "metadata": {}, + "source": [ + "This data looks like it has trends on different time scales e.g. due to seasons. Let's pick some of those out using cf-python. We select two of the four seasons, using the `djf` and `jja` methods like so:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "eca6c20d-c39d-42db-bc3b-8b676232623a", + "metadata": {}, + "outputs": [], + "source": [ + "get_djf_season = cf.djf() # specific collapse type for the months of December, January, February\n", + "get_jja_season = cf.jja() # specific collapse type for the months of June, July, August" + ] + }, + { + "cell_type": "markdown", + "id": "2a7a2bab-d6a6-4a3d-8793-aee95bfea6f9", + "metadata": {}, + "source": [ + "Then we collapse on these only using the `group` keyword argument to the `collapse` method, which we call a 'grouped collapse':" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7fd95f11-90d9-42f8-b459-1d672d91be81", + "metadata": {}, + "outputs": [], + "source": [ + "djf_season_mean = spatial_mean_monthly_field.collapse(\"T: mean\", group=get_djf_season) # mean across DJF season\n", + "cfp.lineplot(djf_season_mean)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "098f995c-ef89-4b00-87da-8844036e6d06", + "metadata": {}, + "outputs": [], + "source": [ + "jja_season_mean = spatial_mean_monthly_field.collapse(\"T: mean\", group=get_jja_season) # mean across JJA season\n", + "cfp.lineplot(jja_season_mean)" + ] + }, + { + "cell_type": "markdown", + "id": "b094573f-f690-48da-a8e9-93481ead73fb", + "metadata": {}, + "source": [ + "### d) Plotting the seasonal trends on one (line)plot" + ] + }, + { + "cell_type": "markdown", + "id": "b183d71f-437f-405e-b43d-5680ae0a18f3", + "metadata": {}, + "source": [ + "To put those seasonal averages into context from the original data, it would be nice to plot them on top of the original line plot. We can do that using cf-plot, as an example of some more advanced cf-plot plotting capability. If you want to plot multiple aspects on one plot in this way, wrap the calls to the functions to plot such as `con` or `vect` or `lineplot` with the opening and closing functions `gopen` and `gclose`, like so:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f834d38e-bb8d-43c3-a92b-0c68fb573b08", + "metadata": {}, + "outputs": [], + "source": [ + "cfp.gopen()\n", + "# By adding the 'label' argument, we allow labels for the corresponding line on the plot legend\n", + "cfp.lineplot(spatial_mean_monthly_field, label=\"Original monthly data (spatial mean)\")\n", + "cfp.lineplot(djf_season_mean, label=\"Mean over the DJF months of the original spatial mean\")\n", + "cfp.lineplot(jja_season_mean, label=\"Mean over the JJA months of the original spatial mean\")\n", + "cfp.gclose()" + ] + }, + { + "cell_type": "markdown", + "id": "3b746f6b-ef82-4aa9-8d39-ca179f15db62", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "2e4c72ef-8631-4749-b4fd-6c79c8d6e455", + "metadata": {}, + "source": [ + "## 6. Changing the underlying grid of data through regridding\n", + "\n", + "**In this section we demonstrate how to change the underlying grid of the data to another grid which could be a higher- or lower- resolution one, or a completely different grid, which is called regridding or interpolation, and indicate various options cf-python supports for doing this.**" + ] + }, + { + "cell_type": "markdown", + "id": "854d7fd4-6d65-4827-b77c-4a89d42da6b0", + "metadata": {}, + "source": [ + "### a) Getting a _source_ field ready to regrid" + ] + }, + { + "cell_type": "markdown", + "id": "7b387753-9fee-4105-8758-b79f63338423", + "metadata": {}, + "source": [ + "We read in a precipitation field and inspect it:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f4bb94e-4531-4ef6-a7e7-fd46ea06b7fd", + "metadata": {}, + "outputs": [], + "source": [ + "higher_res_field = cf.read(\"../data/precip_2010.nc\")[0]\n", + "print(higher_res_field)" + ] + }, + { + "cell_type": "markdown", + "id": "387322a6-ba11-4a31-8468-c277b6ab4bf5", + "metadata": {}, + "source": [ + "To get a feel for the data we have, let's view a basic subspace as a contour plot. Note we use the `blockfill` argument set to `True` which changes the contour plot to plot the data on a cellular basis rather than filled as smoothed contours (whether or not the contour lines are set to be shown from the `lines` argument, in this case we turn those off too):" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "67e544cd-44e0-48e8-8484-3fc9a6348801", + "metadata": {}, + "outputs": [], + "source": [ + "cfp.mapset() # reset\n", + "cfp.cscale(\"precip_11lev\") # using a colour scale good for precipitation data\n", + "cfp.con(higher_res_field[0], blockfill=True, lines=False)" + ] + }, + { + "cell_type": "markdown", + "id": "628e91df-76f0-485e-8288-19ee932a7fb6", + "metadata": {}, + "source": [ + "### b) Getting the _destination_ field: another field in order to regrid the previous _onto its grid_" + ] + }, + { + "cell_type": "markdown", + "id": "6f5392a3-096a-4d0b-afe6-f4dcdbe29c22", + "metadata": {}, + "source": [ + "Now we read in another precipitation field and inspect it. The key thing to note is that it is lower resolution than the previous one we read:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0dd58b04-f3e6-4cfe-890b-204bd67a3878", + "metadata": {}, + "outputs": [], + "source": [ + "lower_res_field = cf.read(\"../data/model_precip_DJF_means_low_res.nc\")[0]\n", + "print(lower_res_field)" + ] + }, + { + "cell_type": "markdown", + "id": "b398db6d-17c0-4940-ab85-2638b2648056", + "metadata": {}, + "source": [ + "Again, get a feel for the data we have from a view of a basic subspace, to confirm that it is lower resolution (notice the larger cell blocks than before):" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f27a6f45-6212-4547-9a92-0337fdd0a6a6", + "metadata": {}, + "outputs": [], + "source": [ + "cfp.cscale(\"precip_11lev\")\n", + "cfp.con(lower_res_field[0], blockfill=True, lines=False)" + ] + }, + { + "cell_type": "markdown", + "id": "dde6c34c-37a9-48c9-8e02-a70913411d7a", + "metadata": {}, + "source": [ + "### c) Performing the regrid operation from the source to the destination fields" + ] + }, + { + "cell_type": "markdown", + "id": "2f799ab3-462f-4fa7-83ae-41649d65a3aa", + "metadata": {}, + "source": [ + "Now the key step after the previous setup: regridding the first (*source*) field to the grid of the second (*destination*) field. We use the `regrids` method of cf-python to do this, where the `s` in the name stands for spherical for spherical regridding, as opposed to Cartesian regridding, also possible with cf-python.\n", + "\n", + "We are going to do two calls, demonstrating two different methods of interpolation, namely the `patch` and `conservative` methods. Assign these to variables so we can compare them next:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1847882c-0c17-4857-8f92-2da66c11e945", + "metadata": {}, + "outputs": [], + "source": [ + "patch_regridded_field = higher_res_field.regrids(lower_res_field, method=\"patch\")\n", + "conservative_regridded_field = higher_res_field.regrids(lower_res_field, method=\"conservative\")" + ] + }, + { + "cell_type": "markdown", + "id": "f6828e28-d06c-47cf-8bd9-8c7c868e86bf", + "metadata": {}, + "source": [ + "These two methods do not give the same results!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bbb865cc-415c-4e8f-80e9-d2f50e21beaa", + "metadata": {}, + "outputs": [], + "source": [ + "patch_regridded_field.equals(conservative_regridded_field)" + ] + }, + { + "cell_type": "markdown", + "id": "72802b45-73bc-4b3f-adc8-7eda851f01c1", + "metadata": {}, + "source": [ + "Recall how to do field arithmetic. We use subtraction to indicate the difference between the two regridded outcomes which differ only by the interpolation method used to regrid:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1db1455e-b266-4620-a1c4-ec746e03dbec", + "metadata": {}, + "outputs": [], + "source": [ + "difference_field = patch_regridded_field - conservative_regridded_field" + ] + }, + { + "cell_type": "markdown", + "id": "1286c96e-02f2-406e-a70d-914ae00389b5", + "metadata": {}, + "source": [ + "### d) Finally, some more advanced cf-plot plotting to compare the source, destination, and regridded results" + ] + }, + { + "cell_type": "markdown", + "id": "182b8f55-78ad-41c2-ac59-a775f42750ed", + "metadata": {}, + "source": [ + "As with the previous section, let's end by doing more advanced cf-plot plotting to show all of the relevant fields to help us to understand what the regridding did.\n", + "\n", + "Again we are using `gopen` and `gclose` wrapped around our calls to plot. This time we use `gpos` function calls which tell cf-plot exactly where we want to place each plot on the overall canvas. In this case, from our `x` and `y` extrema inputs `xmin`, `xmax`, `ymin` and `ymax`, we are building an effective 4 x 4 grid to place the four contour plots we will show.\n", + "\n", + "There is a lot to take in here, but the main thing is to understand that we use `con` to generate four contour plots corresponding to different stages of our regridding, telling cf-plot to place them at one of the four places in a 4 x 4 grid, with some customisation of colour scales and colour bar levels.\n", + "\n", + "Note here we are taking the [0] index subspace of all of the fields to plot, but we can use any other subspace of the data to view instead:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2bc9fa76-96e6-4ae3-ac1d-95db3d76c95f", + "metadata": {}, + "outputs": [], + "source": [ + "cfp.cscale(\"viridis\") # reset cmap to perceptually uniform scale\n", + "cfp.gopen(rows=2, columns=2, figsize=[7, 15]) #, bottom=0.1, top=0.85)\n", + "\n", + "# Configure first three plots\n", + "cfp.mapset() # reset from previous plots in the notebook - standalone don't need this\n", + "cfp.levs(min=0, max=500, step=50)\n", + "\n", + "# First three plots\n", + "cfp.gpos(xmin=0.1, xmax=0.5, ymin=0.55, ymax=1.0)\n", + "cfp.cscale(\"precip_11lev\")\n", + "cfp.con(\n", + " higher_res_field[0], blockfill=True, lines=False,\n", + " title=\"Precipitation field\\nbefore regridding\",\n", + ")\n", + "cfp.gpos(xmin=0.55, xmax=0.95, ymin=0.55, ymax=1.0)\n", + "cfp.con(\n", + " patch_regridded_field[0], blockfill=True, lines=False,\n", + " title=\"...and after regridding with\\nthe patch recovery method\",\n", + ")\n", + "cfp.gpos(xmin=0.1, xmax=0.5, ymin=0.1, ymax=0.55)\n", + "cfp.con(\n", + " conservative_regridded_field[0], blockfill=True, lines=False,\n", + " title=\"...and after regridding with\\nthe conservative method\",\n", + ")\n", + "cfp.gpos(xmin=0.55, xmax=0.95, ymin=0.1, ymax=0.55)\n", + "\n", + "# This final plot is showing the diff so is a bit different, apply some cf-plot plotting configuration\n", + "cfp.levs()\n", + "cfp.cscale(\"BlueDarkRed18\") # is a diff so use a diverging colour map\n", + "cfp.con(\n", + " difference_field[0], blockfill=True, lines=False,\n", + " title=\"Difference between the two\\nregridding methods is:\",\n", + ")\n", + "cfp.gclose()" + ] + }, + { + "cell_type": "markdown", + "id": "f674f64e-b20e-4d7e-96b4-fac2c719a9ad", + "metadata": {}, + "source": [ + "More generally, using regridding with cf-python you can convert from one grid to another very different grid, e.g. from a tripolar to a regular or rotated latitude-longitude grid or vice-versa, including to/from unstructured grids (UGRID grids)." + ] + }, + { + "cell_type": "markdown", + "id": "6e48ef2e-68bd-434d-99cb-a6c972f24132", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "07c4f4f5-296c-42f1-9885-214577dccab8", + "metadata": {}, + "source": [ + "## Conclusion and recap of learning objectives\n", + "\n", + "The NCAS CF Data Tools are a suite of Python libraries which are designed to facilitate working with data for research in the earth sciences and aligned domains. We learnt today about the cf-python (https://ncas-cms.github.io/cf-python/) and cf-plot (https://ncas-cms.github.io/cf-plot/build/). The 'cf' in the names of the NCAS CF Data Tools corresponds to the CF Conventions, a metadata standard becoming the de-facto convention across geoscience to cover the description of data so that sharing and intercomparison is simpler.\n", + "\n", + "Our **learning aim** was to be able to use the NCAS CF Data Tools Python libraries, namely cf-python and cf-plot to process, analyse and visualise netCDF and PP datasets, whilst appreciating the context and 'unique selling point' of the libraries as being built to use the CF Conventions, a metadata standard for earth science data, to make it simpler to do what you want to do with the datasets, by working on top of a Data Model for CF.\n", + "\n", + "For our **learning objectives**, we practiced using cf-python and cf-plot to:\n", + "\n", + "* read dataset(s) and view the (meta)data at different detail levels;\n", + "* edit the (meta)data and write out the edited version to file;\n", + "* reduce datasets by subspacing and collapsing;\n", + "* visualise datasets as contour and vector plots;\n", + "* analyse data: applying mathematical and statistical operations and plotting trends;\n", + "* change the underlying grid of data through regridding." + ] + }, + { + "cell_type": "markdown", + "id": "8bcb9631-0732-43dd-9fe4-23f891f0d10c", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "e2e0cfb4-8203-48d7-8029-7bfda9ce5be9", + "metadata": {}, + "source": [ + "## Where to find more information and resources on the NCAS CF Data Tools\n", + "\n", + "Here are some links relating to the NCAS CF Data Tools and this training.\n", + "\n", + "* This training, with further material, is hosted online and there are instructions for setting up the environment so you can work through it in your own time: https://github.com/NCAS-CMS/cf-tools-training.\n", + "* The cf-python documentation lives at https://ncas-cms.github.io/cf-python/.\n", + "* The cf-python code lives on GitHub at https://github.com/NCAS-CMS/cf-python. There is an Issue Tracker to report queries or questions at https://github.com/NCAS-CMS/cf-python/issues.\n", + "* The cf-plot documentation lives at https://ncas-cms.github.io/cf-plot/build/.\n", + "* The cf-plot code lives on GitHub at https://github.com/NCAS-CMS/cf-plot. There is an Issue Tracker to report queries or questions at https://github.com/NCAS-CMS/cf-plot/issues.\n", + "* There is a technical presentation about the NCAS CF Data Tools avaialble from https://hps.vi4io.org/_media/events/2020/summer-school-cfnetcdf.pdf.\n", + "* The website of the CF Conventions can be found at https://cfconventions.org/.\n", + "* The landing page for training into the CF Conventions is found here within the website above: https://cfconventions.org/Training/.\n", + "\n", + "If you have any queries after this course, please either use the Issue Trackers linked above or you can email me at: sadie.bartholomew@ncas.ac.uk." + ] + }, + { + "cell_type": "markdown", + "id": "08fa89e2-6c21-4118-9cf9-21958bdfa00e", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "***" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 + Jaspy", + "language": "python", + "name": "jaspy" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/python-data/solutions/ex04_cf_python.ipynb b/python-data/solutions/ex04_cf_python.ipynb index e93539d..0d84791 100644 --- a/python-data/solutions/ex04_cf_python.ipynb +++ b/python-data/solutions/ex04_cf_python.ipynb @@ -2,10 +2,2895 @@ "cells": [ { "cell_type": "markdown", - "id": "2762201c-7657-4894-9e07-aaa62c49efec", + "id": "0cd6019e-c5b3-4dc6-bf86-6fcf10b06851", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "# Practical for 'introduction to the NCAS CF Data Tools, cf-python and cf-plot'" + ] + }, + { + "cell_type": "markdown", + "id": "1e8d7e64-fd9a-4e9b-a369-61d5635cbb62", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "
\n", + "Practical instructions: these green boxes provide instructions and tips about doing this practical (blue boxes are the same as in the teaching notebook and provide useful information). As guidance and for reference, the following are provided below before the practical material starts:\n", + "
    \n", + "
  • the context and learning objectives from the main/presented Notebook below - you are advised to re-read this as a reminder;
    • \n", + "
  • a copy of the final section from the main Notebook which provides links to further information - you might find the documentation links especially useful here;
    • \n", + "
  • the note on terminology from section one is included also in the reminder as a guide to terms used throughout - read this if useful.
    • \n", + "
\n", + "
\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "c4378ad0-8448-44ef-80e2-cd6e67346a1b", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "## A reminder: context, learning objectives and guidance links\n", + "\n", + "### What are the NCAS CF Data Tools and why do they all have 'cf' in the name?\n", + "\n", + "The _NCAS CF Data Tools_ are a suite of complementary Python libraries which are designed to facilitate working with data for research in the earth sciences and aligned domains. The two that are of most relevance to the average user, and those wanting to process, analyse and visualise atmospheric data, are *cf-python* (https://ncas-cms.github.io/cf-python/) and *cf-plot* (https://ncas-cms.github.io/cf-plot/build/). We will be focusing on use of cf-python and cf-plot today.\n", + "\n", + "The 'cf' in the names of the NCAS CF Data Tools corresponds to the _CF Conventions_, a metadata standard, because they are built around this standard in the form of using the CF Data Model, which as well as performance is considered a 'unique selling point' of the tools.\n", + "\n", + "\n", + "### What are the CF Conventions?\n", + "\n", + "The _CF Conventions_, usually referred to in this way but also know by the full name of the **C**limate and **F**orecast (CF) metadata conventions, are a metadata standard which is becoming the de-facto convention to cover the description of geoscientific data so that sharing and intercomparison is simpler. See https://cfconventions.org/ for more information.\n", + "\n", + "\n", + "### What are we going to learn in this session?\n", + "\n", + "Our **learning aim** is to be able to use the NCAS CF Data Tools Python libraries, namely cf-python and cf-plot to process, analyse and visualise netCDF and PP datasets, whilst appreciating the context and 'unique selling point' of the libraries as being built to use the CF Conventions, a metadata standard for earth science data, to make it simpler to do what you want to do with the datasets, by working on top of a Data Model for CF.\n", + "\n", + "We have **six distinct objectives**, matching the sections in this notebook and in the practical notebook you will work through. By the end of this lesson you should be familiar and have practiced using cf-python and cf-plot to:\n", + "\n", + "1. read dataset(s) and view the (meta)data at different detail levels;\n", + "2. edit the (meta)data and write out the edited version to file;\n", + "3. reduce datasets by subspacing and collapsing;\n", + "4. visualise datasets as contour and vector plots;\n", + "5. analyse data: applying mathematical and statistical operations and plotting trends;\n", + "6. change the underlying grid of data through regridding." + ] + }, + { + "cell_type": "markdown", + "id": "26aa2a28-22ce-4e9f-bc1e-6b0d76ab807c", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "### Guidance: where to find more information and resources on the NCAS CF Data Tools\n", + "\n", + "Here are some links relating to the NCAS CF Data Tools and this training.\n", + "\n", + "* This training, with further material, is hosted online and there are instructions for setting up the environment so you can work through it in your own time: https://github.com/NCAS-CMS/cf-tools-training.\n", + "* **The cf-python documentation lives at https://ncas-cms.github.io/cf-python/.**\n", + "* The cf-python code lives on GitHub at https://github.com/NCAS-CMS/cf-python. There is an Issue Tracker to report queries or questions at https://github.com/NCAS-CMS/cf-python/issues.\n", + "* **The cf-plot documentation lives at https://ncas-cms.github.io/cf-plot/build/.**\n", + "* The cf-plot code lives on GitHub at https://github.com/NCAS-CMS/cf-plot. There is an Issue Tracker to report queries or questions at https://github.com/NCAS-CMS/cf-plot/issues.\n", + "* There is a technical presentation about the NCAS CF Data Tools avaialble from https://hps.vi4io.org/_media/events/2020/summer-school-cfnetcdf.pdf.\n", + "* The website of the CF Conventions can be found at https://cfconventions.org/.\n", + "* The landing page for training into the CF Conventions is found here within the website above: https://cfconventions.org/Training/.\n", + "\n", + "If you have any queries after this course, please either use the Issue Trackers linked above or you can email me at: sadie.bartholomew@ncas.ac.uk." + ] + }, + { + "cell_type": "markdown", + "id": "7ca3e79a-e39f-4a04-9dc5-3b56036bef5e", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "
\n", + "Note: In cf-python and when discussing related code and datasets, we use terminology from the CF Data Model (for more detail see: https://ncas-cms.github.io/cf-python/cf_data_model.html). For example cf-python methods are named in relation to concepts from this data model. We don't have time to cover this in detail but for this session it is useful to know the following terms:\n", + "\n", + "
    \n", + "
  • field: a self-contained cf-python object corresponding to a netCDF data variable with all of its (CF) metadata attached;
    • \n", + "
  • field list: a list of lields (see above), stored as its own cf-python object 'FieldList' which is similar to a Python list;
    • \n", + "
  • coordinate: a (CF) metadata concept which corresponds to netCDF coordinate variables. One or more coordinates are defined on every field as either 'dimension' or 'auxiliary' coordinate objects in cf-python.
    • \n", + "
\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "1c3beb87-7b0d-44ad-9da3-fa32494b795c", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "df666543-52ee-402a-8fa4-cb4d8389f3a8", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "
\n", + "Practical instructions: run all of the cells in this section to do the set up.\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "196e9e0a-83d1-464c-b53c-a6fc2e0c19ff", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "## Setting up\n", + "\n", + "**In this section we set up this Notebook, import the libraries and check the data we will work with, ready to use the libraries within this notebook.**" + ] + }, + { + "cell_type": "markdown", + "id": "d073760a-6734-4a1d-9832-a70a8e0dd089", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "Import cf-python and cf-plot:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b4b0c984-1adc-4515-b192-19da265e1f9a", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "import cfplot as cfp\n", + "import cf" + ] + }, + { + "cell_type": "markdown", + "id": "6cfa3e24-6400-4de0-b155-aab85f701441", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "Run some set up for nice outputs in this Jupyter Notebook (not required in interactive Python or a script):" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "aadf1e3e-382f-47c5-aeee-b52f3cd2868d", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "import warnings\n", + "warnings.filterwarnings('ignore')\n", + "\n", + "%matplotlib inline" + ] + }, + { + "cell_type": "markdown", + "id": "7b00d47b-4497-4540-b1bd-6fa8fa8e3226", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "Inspect the versions of cf-python and cf-plot and the version of the CF Conventions those are matched to:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ea75b1b6-ba78-428d-b9b2-ca29a8cf6f0d", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "print(\"cf-python version is:\", cf.__version__)\n", + "print(\"cf-plot version is:\", cfp.__version__)\n", + "print(\"CF Conventions version is:\", cf.CF())" + ] + }, + { + "cell_type": "markdown", + "id": "628af6cc-24f1-4e85-bf8b-654268e72bd3", + "metadata": {}, + "source": [ + "
\n", + "Note: you can work with data compliant by any other version of the CF Conventions, or without (much) compliance, but the CF Conventions version gives the maximum version that these versions of the tools understand the features of.\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "8135f5c6-3d3e-45b6-a84a-3dcea754372f", + "metadata": {}, + "source": [ + "Finally, see what datasets we have to explore:" + ] + }, + { + "cell_type": "markdown", + "id": "cfab63fe-088d-4ca8-ab95-40f61f0c2fd3", + "metadata": {}, + "source": [ + "
\n", + "Note: in a Jupyter Notebook, '!' preceeeds a shell command, so this is a terminal command and not Python\n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a45afce7-abd8-4edd-b14c-eb033566bd52", + "metadata": { + "editable": true, + "scrolled": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "!ls ../data" + ] + }, + { + "cell_type": "markdown", + "id": "e7640cdb-0d6f-4933-86d7-e86e7b9acbe4", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "ef80e6de-8e62-468b-93c8-b41ac768375a", + "metadata": {}, + "source": [ + "
\n", + "Practical instructions: now we can start the practical. We will follow the same sectioning as in the teaching notebook, so please consult the notes there in the matching section for guidance and you can also consult the cf-python and cf-plot documentation linked above.\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "3b7a3c0a-262d-4ac4-8c10-6fa3a42ae73a", + "metadata": {}, + "source": [ + "## 1. Reading dataset(s) and viewing the (meta)data at different detail levels\n", + "\n", + "**In this section we look at a basic use of cf-python, reading in one or more datasets from file and inspecting the data and the metadata at different levels of detail to suit the amount of information you want to see.**" + ] + }, + { + "cell_type": "markdown", + "id": "9da08bf9-a15b-4d65-8c8e-632897daec0b", + "metadata": {}, + "source": [ + "### a) Reading in data and extracting the _field_ of interest" + ] + }, + { + "cell_type": "markdown", + "id": "45be7d09-fe3e-46b3-b35b-dec82b32cc01", + "metadata": {}, + "source": [ + "**1.a.1)** Use `cf` to read in the netCDF dataset `qbo.nc` which is found (as shown at the end of the section above) under the directory `../data`, assigning it to a variable called 'fieldlist'.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d0a43e35-8627-4a08-ae1e-4fa9ec4f400d", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "fieldlist = cf.read(\"../data/qbo.nc\")" + ] + }, + { + "cell_type": "markdown", + "id": "674d8543-fb55-4b2e-bd26-5a006dc9181d", + "metadata": {}, + "source": [ + "**1.a.2)** Use the standard Python function `len` to see how long the read-in fieldlist is." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0e59c1a6-53ea-43c2-bf57-a405d1f64be9", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "len(fieldlist)" + ] + }, + { + "cell_type": "markdown", + "id": "9dfdb8c4-ac20-4b0c-87b6-5bc3829dee61", + "metadata": {}, + "source": [ + "**1.a.3)** Access the first field in the fieldlist and assign it to the variable name 'field'." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d92286c4-e62e-4e35-a934-809bc1afe02d", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "field = fieldlist[0]" + ] + }, + { + "cell_type": "markdown", + "id": "d4ed8112-fc35-496c-84ae-e8a05334c33c", + "metadata": {}, + "source": [ + "### b) Inspecting the _field_ of interest with different amounts of detail" + ] + }, + { + "cell_type": "markdown", + "id": "9e3ad7d7-2657-43d4-95a4-601429b16575", + "metadata": {}, + "source": [ + "**1.b.1)** View the field from (1.a.3) above in minimal detail." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "41528fdd-7f56-4801-95b9-c92b910e3a7d", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "field" + ] + }, + { + "cell_type": "markdown", + "id": "8a158a41-03f8-47f9-a84e-fd2eaac8a132", + "metadata": {}, + "source": [ + "**1.b.2)** Now try viewing the field from (1.a.3) above at a medium detail level." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0916e895-c503-47b8-a280-62404bb4d4a1", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "print(field)" + ] + }, + { + "cell_type": "markdown", + "id": "4653be8e-e953-4a75-a343-93413cee09a6", + "metadata": {}, + "source": [ + "**1.b.3)** OK, finally let's see it in its full glory - with maximal detail. Take a minute or two to compare these outputs and familiarise yourself with the formats of the different views and how they present the metadata (and preview of the data) of a field." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b07e729d-3ecd-4c1e-b26b-9ff61fc04a70", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "field.dump()" + ] + }, + { + "cell_type": "markdown", + "id": "0af01357-b0b2-4581-a830-437337565d0d", + "metadata": {}, + "source": [ + "### c) Inspecting a metadata _construct_ e.g. _coordinate_ from the _field_ of interest" + ] + }, + { + "cell_type": "markdown", + "id": "03ce0882-35c1-4f41-9280-33d8b633572b", + "metadata": {}, + "source": [ + "**1.c.1)** Let's assume we want to know about a specific metadata construct, in this case we are intereted in the pressure. Assign to a new variable called 'pressure' the pressure coordinate of the field stored in the variable 'field' from section (1a) as just inspected in section (1b)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "99b9db17-713f-47c9-82a6-ec9af2213212", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "pressure = field.coordinate(\"pressure\")" + ] + }, + { + "cell_type": "markdown", + "id": "bf10ee1b-d4eb-43ac-839f-1f8066e703ae", + "metadata": {}, + "source": [ + "**1.c.2)** View this coordinate with minimal detail level." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "753ff7ac-2de7-49c7-b501-aee40ddb9090", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "pressure" + ] + }, + { + "cell_type": "markdown", + "id": "23fe4e99-326b-4209-b905-8ca211582b08", + "metadata": {}, + "source": [ + "**1.c.3)** Now use the standard approach to view it with medium detail level." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ef442fa1-bfe4-4b5c-bfc7-4453c0d32e08", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "print(pressure)" + ] + }, + { + "cell_type": "markdown", + "id": "9b03ffb4-62d7-4e08-a1e5-28591c3fb602", + "metadata": {}, + "source": [ + "**1.c.4)** Finally, let's use the approach for full detail level and see everything about this coordinate." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ee0cb3a0-5129-456e-8e9e-5f24a598d34d", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "pressure.dump()" + ] + }, + { + "cell_type": "markdown", + "id": "bd7fd831-8362-49eb-a03a-281d75596309", + "metadata": {}, + "source": [ + "### d) Inspecting a data array of interest" + ] + }, + { + "cell_type": "markdown", + "id": "16b595ee-6e54-40ec-ba3f-cc28f710eab1", + "metadata": {}, + "source": [ + "**1.d.1)** Access the underlying data of the pressure coordinate from the previous sub-section, (1c), assigning it to a variable called 'pressure_data'." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f2f748b5-09c3-4382-a6d6-e50a31fca61f", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "pressure_data = pressure.data" + ] + }, + { + "cell_type": "markdown", + "id": "d7855240-2667-4883-a3fd-8d8b1759122a", + "metadata": {}, + "source": [ + "**1.d.2)** Inspect the pressure coordinate data with minimal detail, noticing the units." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "539107da-25fe-4259-9fb2-0f6419954424", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "pressure_data" + ] + }, + { + "cell_type": "markdown", + "id": "7288c197-fd6e-405e-b49a-6740bc41992e", + "metadata": {}, + "source": [ + "**1.d.3)** Access the data array of the pressure coordinate. Note that, because it is small, it is not computationally expensive to access this and similarly with other metadata data arrays, but accessing the underlying data array of the whole field (i.e. its main stored variable) could be intensive because for datasets in real usage the data can be very large and/or multi-dimensional." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e25482db-8342-49aa-bdc1-0911e842759d", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "pressure_array = pressure_data.array" + ] + }, + { + "cell_type": "markdown", + "id": "2e2a3d9a-31ec-4f70-964c-212deb4709cc", + "metadata": {}, + "source": [ + "**1.d.4)** Use the standard Python `print` function to view the pressure array." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "aba215ff-f5a7-4b97-9bdb-5a80961cd38f", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "print(pressure_array)" + ] + }, + { + "cell_type": "markdown", + "id": "09d04ff2-8998-48fb-a0a6-bc5deafe8fc6", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "932d2d38-b8d7-46cc-848f-b6f50b877310", + "metadata": {}, + "source": [ + "## 2. Editing the (meta)data and writing out the edited version to file\n", + "\n", + "**In this section we demonstrate how to change the data that has been read-in from file, both in terms of the data arrays and the metadata that describes it, and then how to write data back out to file with a chosen name, so that you can see how cf-python can be used to edit data or to make new data.**" + ] + }, + { + "cell_type": "markdown", + "id": "bc7c654e-38e5-4b06-a7c0-01828701d92f", + "metadata": {}, + "source": [ + "### a) Changing the underlying data" + ] + }, + { + "cell_type": "markdown", + "id": "1d6b6412-42b4-4d70-9df2-1fd6b0349a56", + "metadata": {}, + "source": [ + "**2.a.1)** Access the data (*not* the data array of the data) of the full field and assign it to a variable called 'data'." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ad3da187-8515-4a84-95e4-a9c1190dd08d", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "data = field.data" + ] + }, + { + "cell_type": "markdown", + "id": "b625ea77-a1db-4598-b1c4-6bba6319d1e0", + "metadata": {}, + "source": [ + "**2.a.2)** Inspect the field data with medium detail use the `size` method on it to see its shape." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4b6e33da-fad2-49a8-86e2-e94331e7e8dd", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "print(data)\n", + "data.shape" + ] + }, + { + "cell_type": "markdown", + "id": "86474dfe-a2ac-4b52-a962-eda937b0f58d", + "metadata": {}, + "source": [ + "**2.a.3)** Use the `size` method on it to see how many items (in this case, numbers) there are in it. Can you see how this relates to the `shape` above, and to the structure of the coordinates from the field inspection in (1.b.2)?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "caaf9db2-c33d-434b-a3fe-ad91e9fc681e", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "data.size" + ] + }, + { + "cell_type": "markdown", + "id": "cf00a0cd-ab33-4c08-94d0-a7c25a2df606", + "metadata": {}, + "source": [ + "**2.a.4)**" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "394dc1e4-3e88-4370-b568-27f031fe1d5d", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "first_time_subarray = data[0, :, :, :]\n", + "first_time_subarray" + ] + }, + { + "cell_type": "markdown", + "id": "8a97f4ab-dfba-4879-ac78-9c0edb36cf2d", + "metadata": {}, + "source": [ + "**2.a.5)** Change all of the values in the first time subarray to the value '-50.0'." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c167273d-16b2-4ea8-bc74-be8ac28d9820", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "first_time_subarray[0] = -50\n", + "first_time_subarray" + ] + }, + { + "cell_type": "markdown", + "id": "5693c393-4c96-4841-8e4c-f336f9bfebb6", + "metadata": {}, + "source": [ + "**2.a.6)** Access the index item `[1, :, 0, 0]` of the full data array from (2.a.1) and assign it to a variable called 'a_subarray'. Then check what shape it is and try to understand the size that emerges for that sub-array given that specific index." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "beb5f4b6-5b5e-4761-8c66-2096edc8bfde", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "a_subarray = data[1, 0, :, 0]\n", + "a_subarray.shape" + ] + }, + { + "cell_type": "markdown", + "id": "b9f9e1b8-1951-4545-94c1-9d567a0722d5", + "metadata": {}, + "source": [ + "**2.a.7)** Change the values for this sub-array item to all ones i.e. `1.0`. Note to create an array populated all with the value one of the required shape to match the shape of the subarray from (2.a.6), you can use `numpy.ones()` (you will need to import `numpy` first, and let's call the module `np` i.e. use `import numpy as np`)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cd65f487-c6b0-45e0-b093-1ac39f11b2d7", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "np.ones((1, 1, 2, 1))\n", + "data[1, 0, :, 0] = np.ones((1, 1, 2, 1))" + ] + }, + { + "cell_type": "markdown", + "id": "fe2fe7a5-73b8-4021-8748-e88b3b0d6e5c", + "metadata": {}, + "source": [ + "**2.a.8)** Inspect 'a_subarray' again to confirm it has been set with values of all one." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5629a7f3-5fd8-4bfd-a6d7-eb141951a2c7", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "a_subarray" + ] + }, + { + "cell_type": "markdown", + "id": "0f2d2721-58c2-44d9-b422-3dd81ee8b333", + "metadata": {}, + "source": [ + "### b) Changing some metadata" + ] + }, + { + "cell_type": "markdown", + "id": "ad17957e-9d70-4fcb-9954-e0ff331ac689", + "metadata": {}, + "source": [ + "**2.b.1)** Take the 'pressure' variable coordinate we created in (1.c) and inspected in full detail in (1.c.4). Let's say we want to reverse the direction of this axes, so that the pressures increase in value along it (in order) rather than decreases. In `cf` we can do this using the `flip` method, which will adjust the data accordingly so that it is flipped in the corresponding way to match up to the new direction of the pressure axis.\n", + "\n", + "Reassign `pressure` to itself with the `flip` method applied (note you can, alternatively and equivalently, call the method `flip` with the argument `inplace=True` set without re-assignment because that argument tells `cf` to adjust the variable in-place)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f08327b2-51a1-404b-97fa-e753255eb252", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "pressure = pressure.flip()\n", + "# Or: pressure.flip(inplace=True)" + ] + }, + { + "cell_type": "markdown", + "id": "f240a543-5676-439e-b53e-4384ee5d9171", + "metadata": {}, + "source": [ + "**2.b.2)** Now full inspect the pressure coordinate again with full detail to confirm that the flip of axis direction happened." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d4311a56-3252-4b10-9b92-01f00c7e1cdb", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "pressure.dump()" + ] + }, + { + "cell_type": "markdown", + "id": "20031178-9faa-4501-b00f-70b0d1c1ae64", + "metadata": {}, + "source": [ + "**2.b.3)** Notice the `standard_name`, which is a CF attribute used to identify the physical quantity in question, that must be one from the thousands of controlled options available in the CF Standard Name table (see https://cfconventions.org/Data/cf-standard-names/current/build/cf-standard-name-table.html, you are encouraged to spend some minutes exploring this resource). At the moment the value in the field is not a valid standard name and it is also not specific enough to identify the variable well. \n", + "\n", + "Let's change it to the more descriptive valid CF Standard Name of 'air_pressure'. To do this, access the `standard_name` attribute of the pressure coordinate and re-set it to the string value desired." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a6bfddb6-223a-46b0-8adf-725361fe1c64", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "pressure.standard_name = \"air_pressure\"" + ] + }, + { + "cell_type": "markdown", + "id": "d0e0d297-4e71-4a73-a194-f9b151f7a4a3", + "metadata": {}, + "source": [ + "**2.b.4)** Once again view the pressure coordinate with maximal detail to see the two changes we made in this section. Also view the overall field in medium detail to see that these changes propagate through to the field object by Python object rules." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "155bf824-7ebc-4178-9377-7be91b03393f", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "pressure.dump()" + ] + }, + { + "cell_type": "markdown", + "id": "cf7cf826-2d23-40cd-bfca-648de59fd02c", + "metadata": {}, + "source": [ + "### c) Writing a (list of) fields out to a file" + ] + }, + { + "cell_type": "markdown", + "id": "f375ce40-ba55-45cd-83d2-9c5f58df3506", + "metadata": {}, + "source": [ + "**2.c.1)** Let's write out a few fields to disk. We'll write our field from the previous sections as well as a new field all into the same file.\n", + "\n", + "First let's get a new field. We could read in another file to access some new fields, but let's use one of the cf-python ready-made 'example fields', which are designed for exploring cf-python capability without the need for real (and large/memory-intensive) data. Run `cf.example_field()` with any integer from 0 to 11 inclusive to get an example field, assigning it to the variable called 'new_field'." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5fb3f0c6-6638-4535-8729-b50d1f70fc29", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "new_field = cf.example_field(1) # Or: any integer from 0 to 11 works fine as the argument" + ] + }, + { + "cell_type": "markdown", + "id": "3e061bbd-836c-44fb-9614-358585b4c1bf", + "metadata": {}, + "source": [ + "**2.c.2)** Inspect this new field with medium level of detail." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4b199ddb-00c6-456e-bb51-58117037c50b", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "print(new_field)" + ] + }, + { + "cell_type": "markdown", + "id": "19c295d5-9e45-49a8-bc53-fd9a139aacf8", + "metadata": {}, + "source": [ + "**2.c.3)** Create a two-field FieldList called 'new_fieldlist' containing the field from (1.a.3) that we have been exploring in the previous sections (1.a) and (1.b) along with this new 'new_field' from (2.c.2). You can use the `cf` function `cf.FieldList` with the fields it should contain in a normal Python list as an arugment to set this up." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a55c4a55-10bd-4269-a232-32e0d3737a94", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "new_fieldlist = cf.FieldList([field, new_field])" + ] + }, + { + "cell_type": "markdown", + "id": "c155c131-5367-497f-8590-16bbfdc7c124", + "metadata": {}, + "source": [ + "**2.c.4)** Call `print` with this new fieldlist as an argument to view it in medium detail." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d2c71af9-a032-458a-b091-39d776c11025", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "print(new_fieldlist)" + ] + }, + { + "cell_type": "markdown", + "id": "aee0230d-681c-476e-800e-fa2f83b419ea", + "metadata": {}, + "source": [ + "**2.c.4)** Now we have constructed our desired FieldList, we can write it out to file. Use `cf`'s `write` function to write it to a file called `two_fields.nc` in the `../data` directory." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c9775ffb-6830-4102-b3b0-eb11403ad806", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cf.write(new_fieldlist, \"../data/two_fields.nc\")" + ] + }, + { + "cell_type": "markdown", + "id": "fb5aabd2-6e6d-4e28-8e9f-3da1537d981a", + "metadata": {}, + "source": [ + "**2.c.5)** Use the shell command `!ls ../data` to see the contents of that directory to confirm that the file was written there." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9f535282-e936-4254-9bc5-fc096bf69e6b", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "!ls ../data" + ] + }, + { + "cell_type": "markdown", + "id": "9dd6be4b-60d3-4ca8-8108-fcda62bb7d95", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "4ccfd6a0-567d-43f3-9f06-15bb21afae6a", + "metadata": {}, + "source": [ + "## 3. Reducing datasets by subspacing and collapsing\n", + "\n", + "**In this section we show how multi-dimensional data can be tamed using cf-python so that you can get a reduced form that can be analysed or plotted, by reducing the dimensions by selecting a subset of point(s) along the axes or collapsing down according to some statistic such as the mean or an extrema.**" + ] + }, + { + "cell_type": "markdown", + "id": "410ac6d4-cd8f-43ed-a3f5-eaf8d1d8755a", + "metadata": {}, + "source": [ + "### a) Subspacing using metadata conditions" + ] + }, + { + "cell_type": "markdown", + "id": "cec90546-f200-42b3-af27-6508f367d305", + "metadata": {}, + "source": [ + "**3.a.1)** Read in the file `ggas2014121200_00-18.nc` which is under `../data` and save the corresponding FieldList to a variable called `fieldlist_3`. Inspect it with medium level of detail." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a4af5c1c-cdc4-441d-8d2c-17d9cfd6b746", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "fieldlist_3 = cf.read(\"../data/ggas2014121200_00-18.nc\")\n", + "print(fieldlist_3)" + ] + }, + { + "cell_type": "markdown", + "id": "3077b7a8-6f93-4a95-ad6d-58e8b3fc15ec", + "metadata": {}, + "source": [ + "**3.a.2)** Extract the field representing the `cloud_area_fraction` to a variable which we will call `cloud_field`. Inspect that also with medium level of detail." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "91818c63-a229-491b-aee2-8882ff554746", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cloud_field = fieldlist_3[4]\n", + "print(cloud_field)" + ] + }, + { + "cell_type": "markdown", + "id": "df537a09-9cc3-43c0-9e6a-7d8d7902344c", + "metadata": {}, + "source": [ + "**3.a.3)** Save the data of the `cloud_field` to a new variable we will call `cloud_field_data`. Do a `print` on the `shape` method of this to confirm the shape of the data, and compare this to the insepction report from the previous cell to see the same information represented in different ways." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d54f5edd-abc3-40e7-a2c1-971fb52e12ca", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cloud_field_data = cloud_field.data\n", + "print(cloud_field_data.shape)" + ] + }, + { + "cell_type": "markdown", + "id": "862a5eb4-6a63-4c11-a188-f048c3d27926", + "metadata": {}, + "source": [ + "**3.a.4)** Make a subspace of the `cloud_field` from the cells above to subspace on the *first* time point in order. Note: doing date-time subspaces requires an extra step due to the nature of specifying dates and times which can be ambiguous otherwise: you need to wrap a quoted datetime string in the call `cf.dt()` to notify cf-python that you are providing a valid datetime string, e.g. `field1.subspace(time=cf.dt(\"2020-01-01 12:15:00\"))`.\n", + "\n", + "Assign the subspace operation resulting field to a variable `cloud_field_subspace1` and inspect it with medium detail.\n", + "\n", + "*Extra task, for those who have studied section 4 before doing this practical**: make a contour plot of this subspace of the field to see what it looks like.*" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e0cdd409-e75d-43bb-96fd-c79b1f5e575b", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cloud_field_subspace1 = cloud_field.subspace(time=cf.dt(\"2014-12-12 00:00:00\"))\n", + "print(cloud_field_subspace1)\n", + "print(cloud_field_subspace1.data.shape)\n", + "# Extra part: cfp.con(cloud_field_subspace1)" + ] + }, + { + "cell_type": "markdown", + "id": "58adbc35-a63a-4c8a-973d-c28fea109ed3", + "metadata": {}, + "source": [ + "**3.a.5** Make a subspace of the `cloud_field` from the cells above to subspace on the *last* point on the latitude axis.\n", + "\n", + "Assign the subspace operation resulting field to a variable `cloud_field_subspace2` and inspect it with medium detail.\n", + "\n", + "*Extra task, for those who have studied section 4 before doing this practical: make a contour plot of this subspace of the field to see what it looks like.*" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6db795e1-8eb4-42b8-a6d2-4ad554f7fbb7", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cloud_field_subspace2 = cloud_field.subspace(latitude=-89.46282196044922)\n", + "print(cloud_field_subspace2)\n", + "print(cloud_field_subspace2.data.shape)\n", + "# Extra part: cfp.con(cloud_field_subspace2)" + ] + }, + { + "cell_type": "markdown", + "id": "bb1db98b-27b3-4c8a-bf63-f67f83dd2bb8", + "metadata": {}, + "source": [ + "### b) Subspacing using indexing, including equivalency to the above" + ] + }, + { + "cell_type": "markdown", + "id": "3d95810b-eb02-4862-938e-91e6634bb2b9", + "metadata": {}, + "source": [ + "**3.b.1)** Take the cloud field from (3.a.2) which we have been subspacing in the previous cells and make a subspace which takes the first time point, leaving all other axes unchanged, but this time do it using indexing. Use the `equals` method of a field to check that the result is the same as that derived from the 'subspacing by metadata' approach in section (3.a.4)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7d6ccf81-0a51-44f4-948c-95a243c751cd", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cloud_field_subspace1_by_index = cloud_field[0, :, :, :]\n", + "print(cloud_field_subspace1_by_index)\n", + "cloud_field_subspace1_by_index.equals(cloud_field_subspace1)\n", + "# or with the fields swapped i.e. cloud_field_subspace1.equals(cloud_field_subspace1_by_index) is also correct" + ] + }, + { + "cell_type": "markdown", + "id": "4e2cc4bf-2520-4136-82af-48ac6f974043", + "metadata": {}, + "source": [ + "**3.b.2)** Now make a subspace on the original `cloud_field`, leaving all other axes unchanged, to subspace on the *last* point on the latitude axis, like before, but this time use subspacing by indexing. Use the `equals` method of a field to check that the result is the same as that derived from the 'subspacing by metadata' approach in section (3.a.4)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f15ef080-2d79-4686-8f67-00baee8a98f2", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cloud_field_subspace2_by_index = cloud_field[:, :, -1, :]\n", + "print(cloud_field_subspace2_by_index)\n", + "cloud_field_subspace2_by_index.equals(cloud_field_subspace2)\n", + "# or with the fields swapped i.e. cloud_field_subspace2.equals(cloud_field_subspace2_by_index) is also correct" + ] + }, + { + "cell_type": "markdown", + "id": "e65d7951-2d10-40d3-8858-87b7210079a8", + "metadata": {}, + "source": [ + "**3.b.3)** Using indexing, do both of the subspaces from the previous sub-questions in one call on the original cloud field.\n", + "\n", + "Extra: do the same operation using the 'subspace by metadata' approach and use the `equals` method to show that the results are the same." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0b83671d-442f-41c4-981a-764581a358ec", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cloud_field_subspace2_by_index = cloud_field[0, :, -1, :]\n", + "print(cloud_field_subspace2_by_index)\n", + "cloud_field_subspace2_by_index.equals(cloud_field_subspace2)" + ] + }, + { + "cell_type": "markdown", + "id": "f5e05393-4c31-4505-8b80-4b8b2bdeba5d", + "metadata": {}, + "source": [ + "**3.b.4)** Do a single subspace on the original cloud field that takes the first 100 latitude and the first 200 longitude values. Use whichever method (subspacing by metadata, or indexing) you prefer, in order to do so." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "dd33a050-117c-4327-931c-ea8dd2372a70", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cloud_field[:, :, :100, :200]" + ] + }, + { + "cell_type": "markdown", + "id": "b870563e-6f45-46ca-9dad-6d8a5cb69ec5", + "metadata": {}, + "source": [ + "### c) Statistical collapses" + ] + }, + { + "cell_type": "markdown", + "id": "05d9602f-744d-4ce7-a4a0-1880a30db393", + "metadata": {}, + "source": [ + "**3.c.1)** Take the original `cloud_field` from (3.a.2) and do a collapse over the time axis to reduce it down to the minimum value. Assign that to the variable name `cloud_field_collapse1`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "843d1148-25e1-4cc2-91e3-28085b0ac7f3", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cloud_field_collapse1 = cloud_field.collapse(\"time: minimum\")\n", + "print(cloud_field_collapse1)" + ] + }, + { + "cell_type": "markdown", + "id": "1527c1bc-12e3-4708-a544-39fef8d0062b", + "metadata": {}, + "source": [ + "**3.c.2)** Take the original `cloud_field` from (3.a.2) and do a collapse over the latitude axis to reduce it down to the mean value. Assign that to the variable name `cloud_field_collapse2`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f44e6b5e-c3bf-48bf-81b4-d082a437d3c2", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cloud_field_collapse2 = cloud_field.collapse(\"latitude: mean\")\n", + "print(cloud_field_collapse2)" + ] + }, + { + "cell_type": "markdown", + "id": "592f58a8-77a3-4cc2-beb3-f6247f5f0310", + "metadata": {}, + "source": [ + "**3.c.3)** Take the original `cloud_field` from (3.a.2) and do a collapse over the longitude axis to reduce it down to the maximum value. Assign that to the variable name `cloud_field_collapse3`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "62c236eb-c360-47b2-ac20-6d1250ed7a78", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cloud_field_collapse3 = cloud_field.collapse(\"longitude: maximum\")\n", + "print(cloud_field_collapse3)" + ] + }, + { + "cell_type": "markdown", + "id": "22357974-11e2-4820-b0d6-90273cbc4646", + "metadata": {}, + "source": [ + "**3.c.4)** Finally, take the original `cloud_field` from (3.a.2) again and do a collapse over all horizontal space via the pair of horizontal spatial axes, latitude and longitude, to reduce them down to the standard deviation value. Assign that to the variable name `cloud_field_collapse4`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "532ae8bf-09a1-4a2c-8800-2802fe186202", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cloud_field_collapse4 = cloud_field.collapse(\"longitude: latitude: standard_deviation\")\n", + "print(cloud_field_collapse4)" + ] + }, + { + "cell_type": "markdown", + "id": "59b69a9d-a94c-464c-aa3c-482c699bc404", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "80edfa76-7f87-42b9-8cef-058304ca902a", + "metadata": {}, + "source": [ + "## 4. Visualising datasets as contour and vector plots\n", + "\n", + "**In this section we demonstrate how to plot using cf-plot the data we have read and then processed and/or analysed using cf-python, notably showing how to create contour plots and vector plots as examples of some of the available plot types.**" + ] + }, + { + "cell_type": "markdown", + "id": "794c4940-43fd-45a3-ba9a-69084c0e5ccd", + "metadata": {}, + "source": [ + "### a) Making a contour plot" + ] + }, + { + "cell_type": "markdown", + "id": "d8508032-181e-4a04-ba70-0f0dfd24f601", + "metadata": {}, + "source": [ + "**4.a.1)** Read in the file `alpine_precip_DJF_means.nc` which is in the `../data` directory and assign the first field it contains to a variable called 'field_4'. Inspect that field with medium detail level." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e3b39603-c768-472b-bd2b-8aec63e525c0", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "field_4 = cf.read(\"../data/alpine_precip_DJF_means.nc\")[0]\n", + "print(field_4)" + ] + }, + { + "cell_type": "markdown", + "id": "f780f284-697e-41a3-8984-db53ac970235", + "metadata": {}, + "source": [ + "**4.a.2)** Make a contour plot of the field, noting that because it is already 2D over X and Y axes corresponding to its projection so it can be plotted directly, without any need to reduce via subspacing or collapsing. Just get a plot working - there is no need to customise it yet." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "94613743-f648-409e-b7a3-22bbf5849c1c", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cfp.con(field_4)" + ] + }, + { + "cell_type": "markdown", + "id": "b19186dc-3480-47d1-a859-79008841bd58", + "metadata": {}, + "source": [ + "### b) Customising the (contour) plot" + ] + }, + { + "cell_type": "markdown", + "id": "017e176a-67ab-4580-82e3-245bf30cc585", + "metadata": {}, + "source": [ + "**4.b.1)** Customise the contour plot made in section (4a) to remove the contour lines by running the same call with a new argument set appropriately." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b4509ae8-c96c-4d9d-9b41-83e3828d03b7", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cfp.con(field_4, lines=False)" + ] + }, + { + "cell_type": "markdown", + "id": "6b403c8c-5991-44b2-89db-660c58f0abdd", + "metadata": {}, + "source": [ + "**4.b.2)** Customise the contour plot further so, as well as having the contour lines removed as applied above, it is shown using a different colour map to the default colour map, 'viridis'. You can use the link https://ncas-cms.github.io/cf-plot/build/colour_scales.html#colour-scales to explore the different options." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3e4e3a79-9e31-4779-9aa3-7cb8d1832819", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cfp.cscale(\"precip4_11lev\") # or another choice of valid colour map string argument\n", + "cfp.con(field_4, lines=False)" + ] + }, + { + "cell_type": "markdown", + "id": "422253a2-28ab-4d6a-8841-d9cdb0ea527c", + "metadata": {}, + "source": [ + "**4.b.3)** Customise the contour plot further by adding the argument and value `resolution=\"10m\"` to the call to `mapset`, which will increase the resolution of the country borders." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7945c4ee-53ed-4c26-b468-b4cd6b0edc88", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cfp.mapset(resolution=\"10m\")\n", + "cfp.con(field_4, lines=False)" + ] + }, + { + "cell_type": "markdown", + "id": "d01534c9-8c56-4f6f-a550-2e86a5749bed", + "metadata": {}, + "source": [ + "**4.b.4)** Customise the contour plot further so that it focuses in on a smaller region over the Alps, specifically from longitude of 5 to 10 degrees east and from latitude of 45 to 47 degrees. Note, a call to `mapset` will reset any of the previous calls, so you will need to re-apply previous arguments to it that you want to preserve for future plots." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "70bf4f42-61de-4255-acbc-9235a36f403e", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cfp.mapset(lonmin=5, lonmax=10, latmin=45, latmax=47, resolution=\"10m\")\n", + "cfp.con(field_4, lines=False)" + ] + }, + { + "cell_type": "markdown", + "id": "6964260d-84fb-41d9-a1b9-a74e85a79c20", + "metadata": {}, + "source": [ + "### c) Making a vector plot with basic customisation" + ] + }, + { + "cell_type": "markdown", + "id": "5a7610d7-2654-4759-9083-0ae921546f63", + "metadata": {}, + "source": [ + "**4.c.1)** Read in the netCDF file `sea_currents.nc` stored in the usual directory, `../data`, assigning it to a variable 'irish_sea_fieldlist' and inspect it with medium detail." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b1ba3732-e318-4e4e-a5e7-8ca17db707b9", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "irish_sea_fieldlist = cf.read(\"../data/sea_currents.nc\")\n", + "print(irish_sea_fieldlist)" + ] + }, + { + "cell_type": "markdown", + "id": "920060c3-46e1-420d-a400-f110e9c08b97", + "metadata": {}, + "source": [ + "**4.c.2)** Note that the first two fieds represent perpendicular components of the same directional variable for sea water velocity. The first field is the eastward and the second field is the northward, component of this.\n", + "\n", + "In order to ensure they are compatible to plot together as vectors, we first need to ensure they are defined on the same gridpoints - in cf-python (from CF Data Model) terminology, we say they need to have the same *domain*. Assign to variable names 'eastward_component_field' and 'northward_component_field' the first and second fields respectively, and inspect them with minimal detail as a first step check to see if there is the same number of axes and points along each upon which the data is defined (there are two code blocks here, one for each field since when you inspect ith minimal detail it will only show if it is the final call in the cell)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b0946df9-087f-4460-8519-4b2b4dea2d6f", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "eastward_component_field = irish_sea_fieldlist[0]\n", + "eastward_component_field" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "49cd847e-4c92-46d6-9f31-f81b8ec3b4e2", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "northward_component_field = irish_sea_fieldlist[1]\n", + "northward_component_field" + ] + }, + { + "cell_type": "markdown", + "id": "8018e3d9-685b-4a12-93c3-77f3dfcb035e", + "metadata": {}, + "source": [ + "**4.c.3)** You should be able to see from the metadata that the two fields are perpendicular components of the same wind field. But, for us to be sure these represent components of the same vector field, we need to check that they are defined at the same grid points - else the components are defined at different location points in space and therefore can't be combined into a single vector. In cf-python we call the locations that the data is representative of the *domain*.\n", + "\n", + "We can check whether the two fields have the same domain by taking the `domain` attribute of each field and comparing those using the `equals` method, e.g. `field1.domain.equals(field2.domain)` for fields `field1` and `field2`. Do this for the two fields we defined in the pair of cells above." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "33efb918-44f1-46cc-9018-d7775a108382", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "eastward_component_field.domain.equals(northward_component_field.domain)" + ] + }, + { + "cell_type": "markdown", + "id": "b170dab7-7a23-47e9-8dc1-04b448c04ab5", + "metadata": {}, + "source": [ + "**4.c.4)** We know now that the fields are the same variable, with perpendicular northward and eastward components, with the same domain i.e. defined on the same gridpoints. This means they are compatible vector components and can be plotted together to form a vector plot for the variable in question, the sea water velocity. However, we currently have the time coordinate with more than one point, so we need to reduce both fields down in the correpsonding way to get a particular 2D plot.\n", + "\n", + "Let's take the final time point in the series of both of these fields to plot. Subspace both fields down in time to that final time axis point, using indexing - using section (3b) for a reminder about subspacing by index if you need guiance." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f94aaa31-8468-4803-869e-878d542a061e", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "eastward_component_field_last_time = eastward_component_field[-1, :, :, :]\n", + "print(eastward_component_field_last_time)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9c1640c3-2174-472d-994a-b6a3d7c54ae2", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "northward_component_field_last_time = northward_component_field[-1, :, :, :]\n", + "print(northward_component_field_last_time)" + ] + }, + { + "cell_type": "markdown", + "id": "90055498-9f1a-463e-aec1-1cf22cd4b333", + "metadata": {}, + "source": [ + "**4.c.5)** Make a vector plot using the two subspaced component fields you found and defined in the previous section to form the corresponding vectors. Do not worry about customising it yet, just get a plot working even if the vectors are not optimised for readability, however do run `cfp.mapset(resolution=\"10m\")` before you make the call to make the vector plot to reset the X and Y limits we specified in the previous section.\n", + "\n", + "Note that the `u` keyword argument is for eastward vector components and the `v` keyword argument is for northward vector components." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c3249180-3c84-4a96-a8f8-f40340769bc2", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cfp.mapset(resolution=\"10m\")\n", + "cfp.vect(u=eastward_component_field_last_time, v=northward_component_field_last_time)" + ] + }, + { + "cell_type": "markdown", + "id": "f99f557c-a4dd-42e4-8b7d-5d9bcc1d5ad9", + "metadata": {}, + "source": [ + "**4.c.6)** Customise the plot you just made, in particular so that the vector lines are legible through adjusting the spacing and size at which they are plotted. This usually takes some trial and error - guess some values for the keyword argument values and increase or decrease them until you find a good balance so the vector lines can be distinguished and clearly interpreted." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "783e8212-335b-4276-8014-2be54f95ca4b", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cfp.mapset(resolution=\"10m\")\n", + "cfp.vect(u=eastward_component_field_last_time, v=northward_component_field_last_time, scale=3, stride=6)" + ] + }, + { + "cell_type": "markdown", + "id": "b191c8b2-a038-4601-b046-997a4566c2f6", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "72524bb7-a643-4753-ad8b-f419656883fa", + "metadata": {}, + "source": [ + "## 5. Analysing data: applying mathematical and statistical operations and plotting trends\n", + "\n", + "**In this section we demonstrate how to do some data analysis including performing arithmetic and statistical calculations on the data, showing how cf-python's CF Conventions metadata awareness means that the metadata is automatically updated to account for the operations that are performed.**" + ] + }, + { + "cell_type": "markdown", + "id": "59637071-edcb-40fe-970a-1d89e0209aa7", + "metadata": {}, + "source": [ + "### a) Applying mathematics e.g. arithmetic and trigonometry on fields" + ] + }, + { + "cell_type": "markdown", + "id": "3ce63f35-c0a3-4c5b-a58c-98856f26ce25", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "**5.a.1)** Take the `fieldlist_3` we have already defined in (3.a.1). This time, we'll work with the *fourth* field in order in the list. Extract this field to a variable called `field_5` and inspect it with medium detail." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8846ce13-3ac6-4586-9318-bcf96abe2b97", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "field_5 = fieldlist_3[3]\n", + "print(field_5)" + ] + }, + { + "cell_type": "markdown", + "id": "dfdf903e-1a4b-46e2-9ba5-4118307329c7", + "metadata": {}, + "source": [ + "**5.a.2)** Take a subspace of the first time point, assigning it to a new variable called 'field_5_subspace', and plot it on a contour plot to see what the underlying data is like. Making a plot with cf-plot early on in this way is, for example, a good way to get a quick look at a new dataset to get a feel for the overall scope and patterns." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9c7be26b-51d7-4f43-aaa5-ce916b97a468", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "field_5_subspace = field_5[0, :, :, :]\n", + "cfp.con(field_5_subspace)" + ] + }, + { + "cell_type": "markdown", + "id": "3d0733cc-b6fa-4889-87ba-c0161b29d8eb", + "metadata": {}, + "source": [ + "**5.a.3)** To normalise data representing a variable `x`, there is the general formula `normalized_x = (x - minimum_x) / x-range`. Use the cf-python methods available on a field, namely `minimum()` and `range()`, to normalise the data in the subspace of `field_5` we defined as `field_5_subspace` from the previous step. Call this result `norm_field_5` as a new Python variable. Hint: you can apply these methods directly on the field which knows to apply them to its underlying data, and the whole data array will be operated on at once, as with array operations in NumPy and similar Python array libraries.\n", + "\n", + "Finally, re-plot the contour plot at the subspace on the first time point to see that the data has been normalised appropriately: the colour bar range should go from 0 to 1 now but the data plot itself (contour pattern and colours) should not have changed because the normalisation adjusts all values to the 0 to 1 range but preserves the relative magnitudes between them." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e770cba8-88db-44ac-a143-51e369b9695b", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "norm_field_5 = (field_5_subspace - field_5_subspace.minimum()) / field_5_subspace.range()\n", + "print(norm_field_5)\n", + "cfp.con(norm_field_5)" + ] + }, + { + "cell_type": "markdown", + "id": "3636fd39-4aa6-4b86-8eaf-0154b285fc2d", + "metadata": {}, + "source": [ + "**5.a.4)** Finally, to explore basic field arithmetic and see the influence on the data via a plot, multiply `field_5_subspace` by negative one and plot this result as a contour plot. Describe in a sentence comment the difference in the plot to the original from ." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "aed0f47d-cad3-4a80-8ef6-2878ec963fd8", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "field_5_reversed = -1 * field_5_subspace\n", + "cfp.con(field_5_reversed)" + ] + }, + { + "cell_type": "markdown", + "id": "561caf84-0824-4b69-8ad9-82ce28d5bfd1", + "metadata": {}, + "source": [ + "### b) Line plotting" + ] + }, + { + "cell_type": "markdown", + "id": "a42040e2-0bc1-4e25-9168-5dc474e97ab2", + "metadata": {}, + "source": [ + "**5.b.1)** Let's make some line plots. To set this up, read in the file `IPSL-CM5A-LR_r1i1p1_tas_n96_rcp45_mnth.nc` under the usual directory `../data` and assign the *first* field in that FieldList to a variable called `monthly_field`. Inspect it with medium detail." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "274d1d90-5be7-46ff-8532-5de882998187", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "monthly_field = cf.read(\"../data/IPSL-CM5A-LR_r1i1p1_tas_n96_rcp45_mnth.nc\")[0]\n", + "print(monthly_field)" + ] + }, + { + "cell_type": "markdown", + "id": "5de191f4-d755-43f7-8718-e04a8376b647", + "metadata": {}, + "source": [ + "**5.b.2)** Notice that the `monthly_field` is three-dimensional (3D) with axes sizes all over one for each of the three dimensions. In order to do a lineplot, we need a one-dimensional series, so we need to reduce the field down to size one in two of the dimensions to make a line plot.\n", + "\n", + "Let's do a collapse to reduce the two spatial axes, latitude and longitude, at once. Do a collapse to the maximum over those two axes, assigning it to a variable we'll call `spatial_max_monthly_field`. You can specify both of those axes by name, or you can use `\"area\"` as a shorthand string which means both latitude and longitude at once." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "34f9b25c-c249-4868-b262-ecadf69367b8", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "spatial_max_monthly_field = monthly_field.collapse(\"area: maximum\")" + ] + }, + { + "cell_type": "markdown", + "id": "f0e10b7f-b3b5-4dd3-b79b-1d8b6e1bb190", + "metadata": {}, + "source": [ + "**5.b.3)** Use cf-plot to make a line plot of the collapsed field `spatial_max_monthly_field` from the cell above." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "537ec534-e25e-4dc6-88b6-57e7797a1273", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cfp.lineplot(spatial_max_monthly_field)" + ] + }, + { + "cell_type": "markdown", + "id": "682335b3-5eba-416c-bbd2-182f5915df7d", + "metadata": {}, + "source": [ + "### c) Calculating seasonal trends" + ] + }, + { + "cell_type": "markdown", + "id": "9799e66c-0107-4dd0-8652-5a851a795c8f", + "metadata": {}, + "source": [ + "**5.c.1)** Define a new variable `get_mam_season` and set it to the cf-python function `mam()` which represents a specific collapse over the months of March, April and May, just like `cf.djf()` represents a collapse over December, January and February etc. - there are four such methods to cover each trio of months closely aligned with the seasons, two of which were demonstrated in the teaching Notebook." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9f483867-c35f-456b-8314-7c9c006b163a", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "get_mam_season = cf.mam()" + ] + }, + { + "cell_type": "markdown", + "id": "6cbc9cf7-cfd7-49fc-8244-91612c006cdb", + "metadata": {}, + "source": [ + "**5.c.2)** Do a grouped collapse over the months of March, April and May only using the `group` argument set to `get_mam_season`, for a *maximum* collapse over the *time* axis on the `spatial_max_monthly_field`. Make a line plot of this collapsed field." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8763a854-82cf-4fc5-80b5-c0214310fae9", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "mam_season_max = spatial_max_monthly_field.collapse(\"T: maximum\", group=get_mam_season)\n", + "cfp.lineplot(mam_season_max)" + ] + }, + { + "cell_type": "markdown", + "id": "fcb98f04-a03d-4c1c-b0cd-ea153daf6f16", + "metadata": {}, + "source": [ + "**5.c.3)** Define three more variables representing specific collapse over the DJF, JJA and SON trio of months using the appropriate cf-python methods to complete the set across the calendar year (`cf.son()` is for the SON months and use the teaching Notebook for guidance on the other two methods if required but their names follow the same pattern so you can possibly guess them), assigning them to variables `get_djf_season`, `get_jja_season` and `get_son_season` respectively.\n", + "\n", + "Then, using those three variables one-by-one, define three grouped collapses, also for a *maximum* collapse over the *time* axis as above, on the `spatial_max_monthly_field` field using these as the `group`, calling them `djf_season_max`, `jja_season_max` and `son_season_max` corresponding to the collapses in order above." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ea2e67a9-1ec6-4d3b-a673-2f06c9f7ae11", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "get_djf_season = cf.djf()\n", + "get_jja_season = cf.jja()\n", + "get_son_season = cf.son()\n", + "djf_season_max = spatial_max_monthly_field.collapse(\"T: maximum\", group=get_djf_season)\n", + "jja_season_max = spatial_max_monthly_field.collapse(\"T: maximum\", group=get_jja_season)\n", + "son_season_max = spatial_max_monthly_field.collapse(\"T: maximum\", group=get_son_season)" + ] + }, + { + "cell_type": "markdown", + "id": "b094573f-f690-48da-a8e9-93481ead73fb", + "metadata": {}, + "source": [ + "### d) Plotting the seasonal trends on one (line)plot" + ] + }, + { + "cell_type": "markdown", + "id": "428b074b-d2af-4804-9131-2985d69bf4d9", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "**5.d.1)** Let's make a line plot which shows all of the fields above: the four season maximum trends `djf_season_max`, `mam_season_max`, `jja_season_max` and `son_season_max`, along with the `spatial_max_monthly_field`. Use the code block from the teaching Notebook as a guide, or otherwise: you will need to wrap all of the calls to `lineplot` within `cfp.gopen()` and `cfp.gclose()` so they are plotted on the same canvas. You do not need to set positions." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "83a29404-cc39-47eb-9909-b2af9b1ab7e9", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cfp.gopen()\n", + "cfp.lineplot(spatial_max_monthly_field)\n", + "cfp.lineplot(djf_season_max)\n", + "cfp.lineplot(mam_season_max)\n", + "cfp.lineplot(jja_season_max)\n", + "cfp.lineplot(son_season_max)\n", + "cfp.gclose()" + ] + }, + { + "cell_type": "markdown", + "id": "422daf4c-731d-48db-812e-53298d8d80a7", + "metadata": {}, + "source": [ + "**5.d.2)** Finally, copy your code from the previous cell and use the `label` argument to `lineplot` to assign labels to your individual line plot calls, which will result in a legend emerging on the final plot with the labels to identify each line." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a6396957-b64b-4c79-9fcd-067463457842", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cfp.gopen()\n", + "cfp.lineplot(spatial_max_monthly_field, label=\"Original monthly data (spatial mean)\")\n", + "cfp.lineplot(djf_season_max, label=\"Maximum over the DJF months of the original spatial maximum\")\n", + "cfp.lineplot(mam_season_max, label=\"Maximum over the MAM months of the original spatial maximum\")\n", + "cfp.lineplot(jja_season_max, label=\"Maximum over the JJA months of the original spatial maximum\")\n", + "cfp.lineplot(son_season_max, label=\"Maximum over the SON months of the original spatial maximum\")\n", + "cfp.gclose()" + ] + }, + { + "cell_type": "markdown", + "id": "3b746f6b-ef82-4aa9-8d39-ca179f15db62", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [] + }, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "id": "2e4c72ef-8631-4749-b4fd-6c79c8d6e455", + "metadata": {}, + "source": [ + "## 6. Changing the underlying grid of data through regridding\n", + "\n", + "**In this section we demonstrate how to change the underlying grid of the data to another grid which could be a higher- or lower- resolution one, or a completely different grid, which is called regridding or interpolation, and indicate various options cf-python supports for doing this.**" + ] + }, + { + "cell_type": "markdown", + "id": "854d7fd4-6d65-4827-b77c-4a89d42da6b0", + "metadata": {}, + "source": [ + "### a) Getting a _source_ field ready to regrid" + ] + }, + { + "cell_type": "markdown", + "id": "65ee428b-cf95-423c-b92e-4cd644b7df41", + "metadata": {}, + "source": [ + "**6.a.1)** Define a new variable 'yearly_field' set to the first field in the FieldList read-in from the netCDF dataset `precip_1D_yearly.nc` which lives in the usual directory, `../data`. Inspect it with medium level of detail." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a6ec5550-61a1-47ff-99f1-339f6dff0fa9", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "yearly_field = cf.read(\"../data/precip_1D_yearly.nc\")[0]\n", + "print(yearly_field)" + ] + }, + { + "cell_type": "markdown", + "id": "7955e4b0-a23e-4398-9c09-48375a37e2e2", + "metadata": {}, + "source": [ + "**6.a.2)** This will be our source field for regridding. Note it is a time series, not a dataset defined across different points in space! We can regrid data on any axis or axes, they do not have to represent spatial coordinates, including 1D series. In this case, we have an effectively 1D (3D but with two of the axes size one, hence of size which can be ignored) time series and we are going to regrid it. For a time series, visually this\n", + "\n", + "Make a line plot of the time series in `yearly_field` to see it. Add the argument `marker=\"o\"` to the call which plots markers against the individual data points forming the line. With the markers on we can see the discrete sample points that make up the data represented by the continuous line." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "17a5908e-ea5a-4b8b-bbf2-980f7bcf0796", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cfp.lineplot(yearly_field, marker=\"o\")" + ] + }, + { + "cell_type": "markdown", + "id": "628e91df-76f0-485e-8288-19ee932a7fb6", + "metadata": {}, + "source": [ + "### b) Getting the _destination_ field: another field in order to regrid the previous _onto its grid_" + ] + }, + { + "cell_type": "markdown", + "id": "ff212883-d45d-4b3a-8d60-5f8816c9acce", + "metadata": {}, + "source": [ + "**6.b.1)** Let's get another time series field, but one with a different sampling resolution. The sampling of time points across the axes is the *grid* in this context. For our destination field, we will take a series which is sampled at higher resolution.\n", + "\n", + "Define a new variable 'monthly_field' set to the first field in the FieldList read-in from the netCDF dataset `precip_1D_monthly.nc` which lives in the usual directory, `../data`. Inspect it with medium level of detail." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7f1f0704-f409-4ae5-b6d1-acadd22790d1", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "# Read in another precipitation field, with more time axis points, and inspect it\n", + "monthly_field = cf.read(\"../data/precip_1D_monthly.nc\")[0]\n", + "print(monthly_field)" + ] + }, + { + "cell_type": "markdown", + "id": "d9f9d588-5bf4-4911-ba4e-e8e2ece77809", + "metadata": {}, + "source": [ + "**6.b.2)** As the name of the corresponding read-in files suggests, this `dest_field` represents monthly data and the `source_field` represents yearly data, so `dest_field` is around 12 times the resolution of the `source_field`.\n", + "\n", + "Make a line plot of the `dest_field` showing the markers for the discrete data points. Compare this line plot to the one above with respect to the time axis (on the x-axis) to confirm the difference in sampling frequencies of around x12. Note we don't care about the values of the data at each point and whether it is different to the data of the source grid we are considering! All we want to extract from this destination field time series is the time point sampling, our destination grid. (Likewise, in the teaching Notebook where we did horizontal spatial grid regridding, the data defined on the destination grid `lower_res_field` did not matter, just the grid points it was defined on.)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a414b073-0c90-498e-bd71-62bdbc3cc69d", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cfp.lineplot(monthly_field, marker=\"o\")" + ] + }, + { + "cell_type": "markdown", + "id": "dde6c34c-37a9-48c9-8e02-a70913411d7a", + "metadata": {}, + "source": [ + "### c) Performing the regrid operation from the source to the destination fields" + ] + }, + { + "cell_type": "markdown", + "id": "6ce49aba-e16f-4278-ac9f-4fa3e8d4c8c5", + "metadata": {}, + "source": [ + "**6.c.1)** Before we do the regridding operation, let's demonstrate briefly how to remove any size one axes from a field. Note this is not necessary in order to do the regridding, but it can 'tidy up' the fields so they don't have size one axes attached, which can be useful in cases like this because it means the axes are both 1D so it is simpler to interpret the regridding we are doing. To remove the size one axes of a field, apply the `squeeze()` method: you can either re-assign the field with `squeeze()` applied to the same variable name, or use the `inplace=True` argument to the method which changes the field in-place i.e. to the variable storing it without the need to re-assign.\n", + "\n", + "Apply the `squeeze` method to the `monthly_field` and the `yearly_field` and inspect the results of each with medium level of detail." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c7d637a8-bdb9-43ad-9e05-0e2abcb58fb9", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "monthly_field = monthly_field.squeeze() # or just monthly_field.squeeze(inplace=True)\n", + "print(monthly_field)\n", + "yearly_field = yearly_field.squeeze() # or just yearly_field.squeeze(inplace=True)\n", + "print(yearly_field)" + ] + }, + { + "cell_type": "markdown", + "id": "bc533850-f32c-4441-9a4f-f14c3206db14", + "metadata": {}, + "source": [ + "**6.c.2)** Now to do the actual regridding operation! (You could call all the field methods such as reading, indexing and squeezing in the same line as the actual regrid operation, but doing so would be harder to interpret and to debug in case something wasn't speciied correctly, or some other issue, so we have worked these out step-by-step as better practice.)\n", + "\n", + "In the teaching Notebook we used the `regrids` method which performs *spherical* regridding (`s` for spherical) because longitude and latitude are defined on the Earth's surface which forms a spherical coordinate system (you can read more about spherical regridding if desired here: https://ncas-cms.github.io/cf-python/tutorial.html#spherical-regridding).\n", + "\n", + "But that is not appropriate here because we aren't considering coordinates defined on a spherical surface, so we need to use *Cartesian* regridding. We use the `regridc` method of cf-python to do this, where the `c` in the name stands for the 'C' of Cartesian, which is the appropriate type here and in general you should use when considering any axes in Euclidean (ie. flat/non-curved) space (you can read more about Cartesian regridding if desired here: https://ncas-cms.github.io/cf-python/tutorial.html#cartesian-regridding).\n", + "\n", + "With that in mind, take the source and destinations fields we set up in the previous sub-sections and apply the cf-python Cartesian regridding operation to regrid `yearly_field` to the grid (i.e. the sampling resolution in this 1D case) of `monthly_field`, assigning the output to a new variable `linear_regridded_field`. Use the `\"linear\"` argument to `method` to use linear interpolation as the regridding method, this time. You will need to set a further argument for Cartesian regridding, namely `axes=\"T\"` to indicate that we want to regrid over the 'time'/'T' axis." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d428cede-8d0f-4c5e-b921-4ab7eb30d6f1", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "linear_regridded_field = yearly_field.regridc(monthly_field, method=\"linear\", axes=\"T\")\n", + "print(linear_regridded_field)" + ] + }, + { + "cell_type": "markdown", + "id": "01084a42-6564-40cc-9b96-0a945e03715c", + "metadata": {}, + "source": [ + "**6.c.3)** Make a line plot of `linear_regridded_field`, including the argument `marker=\"o\"` so that we can see the discrete points the line is based upon." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "20e4025c-fe01-4ab4-b92f-74806a30d486", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cfp.lineplot(linear_regridded_field, marker=\"o\")" + ] + }, + { + "cell_type": "markdown", + "id": "7071b0f4-c833-4fe4-a279-55c773979c75", + "metadata": {}, + "source": [ + "**6.c.4)** This time, regrid the inverse way to before, so that our `yearly_field` this time is our destination field and our `monthly_field` is our source field. Again, use `regridc` with the `\"linear\"` argument to `method` to use linear interpolation as the regridding method, and set `axes=\"T\"` to indicate that we want to regrid over the 'time'/'T' axis.\n", + "\n", + "Call this `inverse_linear_regridded_field`. Inspect it with medium detail level and make a line plot of it showing markers." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "32cabd25-e5c6-48ae-858e-9495751f70e0", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "inverse_linear_regridded_field = monthly_field.regridc(yearly_field, method=\"linear\", axes=\"T\")\n", + "print(inverse_linear_regridded_field)\n", + "cfp.lineplot(inverse_linear_regridded_field, marker=\"o\")" + ] + }, + { + "cell_type": "markdown", + "id": "c3aaa456-411f-437d-92eb-aba7800113d1", + "metadata": {}, + "source": [ + "**6.c.5)** Regrid the reverse way to before, like in (6.c.4), but this time use the `\"nearest_stod\"` method of `regridc` which stands for 'nearest neighbour source to destination' interpolation. This will do the same regridding but using a different approach to the underlying calculation based upon mapping each destination point to the closest source point.\n", + "\n", + "Call this `inverse_nearest_stod_regridded_field`. Inspect it with medium detail level and make a line plot of it showing markers." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f5e67ce3-f7eb-47bf-8c85-f1f9da6f4883", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "inverse_nearest_stod_regridded_field = monthly_field.regridc(yearly_field, method=\"nearest_stod\", axes=\"T\")\n", + "print(inverse_nearest_stod_regridded_field)\n", + "cfp.lineplot(inverse_nearest_stod_regridded_field, marker=\"o\")" + ] + }, + { + "cell_type": "markdown", + "id": "1286c96e-02f2-406e-a70d-914ae00389b5", + "metadata": {}, + "source": [ + "### d) Finally, some more advanced cf-plot plotting to compare the source, destination, and regridded results" + ] + }, + { + "cell_type": "markdown", + "id": "e2196667-b7dd-4548-89e0-5194aa6170a5", + "metadata": {}, + "source": [ + "**6.d.1)** Use cf-plot to make a plot with *two separate line plots* on one canvas (in either separate rows or columns, your choice), one of the first original field `yearly_field` and the linearly-regridded version of it, `linear_regridded_field`, so we can compare the original (un-regridded) and regridded result.\n", + "\n", + "Use the code block from the teaching Notebook section (6b) as a guide, or otherwise: you will need to wrap all of the calls to `lineplot` within `cfp.gopen()` and `cfp.gclose()` so they are plotted on the same canvas and call `cfp.gpos(N)` with an integer starting at `N=1` to tell cf-plot to move on to the next position in the plot before making a `lineplot` call." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "910689b2-26d4-4973-bb4e-97cf1cfeb10a", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cfp.gopen(rows=1, columns=2) # or could use rows=2, columns=1 for one column instead\n", + "cfp.gpos(1)\n", + "cfp.lineplot(\n", + " yearly_field, marker=\"o\", color=\"red\", title=\"Original time series... before regridding\")\n", + "cfp.gpos(2)\n", + "cfp.lineplot(linear_regridded_field, marker=\"o\", color=\"blue\", title=\"... and after regridding\")\n", + "cfp.gclose()" + ] + }, + { + "cell_type": "markdown", + "id": "7561067c-b9a0-45a8-9c37-87318528d39b", + "metadata": {}, + "source": [ + "**6.d.2)** This time, use cf-plot to make a plot with *multiple line plots plotted together on the same (x- and y-) axes*. Plot both original fields along with the two regridded `inverse_*_regridded_field`, this time, and plot them in different colours of your choosing with sensible descriptive labels:\n", + "1. `monthly_field`\n", + "2. `yearly_field`\n", + "3. `inverse_linear_regridded_field`\n", + "4. `inverse_nearest_stod_regridded_field`\n", + "\n", + "Use the code block from the teaching Notebook section (5d) as a guide, or otherwise: you will need to wrap all of the calls to `lineplot` within `cfp.gopen()` and `cfp.gclose()` and by, not calling `cfp.gpos()` with position integers in the last question, all of the lines will be plotted onto the same canvas to share axes" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9e80586f-08a7-44e9-bdde-8898f0a68650", + "metadata": { + "editable": true, + "slideshow": { + "slide_type": "" + }, + "tags": [ + "clear_answer_cell" + ] + }, + "outputs": [], + "source": [ + "cfp.gopen()\n", + "cfp.lineplot(monthly_field, marker=\"o\", color=\"black\", label=\"Before regridding - source field\")\n", + "cfp.lineplot(yearly_field, marker=\"o\", color=\"red\", label=\"Regridding to - destination field\")\n", + "cfp.lineplot(inverse_linear_regridded_field, marker=\"o\", color=\"blue\", label=\"Linearly regridded\")\n", + "cfp.lineplot(\n", + " inverse_nearest_stod_regridded_field, marker=\"o\", color=\"green\", label=\"Nearest source to destination method regridded\")\n", + "cfp.gclose()" + ] + }, + { + "cell_type": "markdown", + "id": "877fa655-484a-4d24-a31d-b36d93a96ad8", + "metadata": {}, + "source": [ + "**6.d.3)** Finally, study the two compound plots created in this section, in particular paying attention to the marker spacings representing the time data point sampling i.e. resolution which forms our grid in this context, to help you to ensure you understand the process of regridding.\n", + "\n", + "Also, look back to the horizontal spatial regridding example plot in (6d) of the teaching Notebook and pay particular attention to the cell block sizes from before and after regridding, to remind yourself the output from a spatial regridding operation.\n", + "\n", + "We've practiced regridding of fields in two contexts: spherical regridding in a 2D spatial context and Cartesian regridding in a 1D time series context. Hopefully these two examples have allowed you to understand what regridding does, but at least how cf-python can enable you to do it." + ] + }, + { + "cell_type": "markdown", + "id": "08fa89e2-6c21-4118-9cf9-21958bdfa00e", "metadata": {}, "source": [ - "# Exercise 4" + "***" ] } ],