From 9b9f6107ef1782d93cb17c26bbd9a91c1c2a0db8 Mon Sep 17 00:00:00 2001
From: HoWol76
See the jupyter-book docs for info on creating posts - https://jupyterbook.org/intro.html
Creating special blocks (notes, warnings etc.)
-Link to external documentation with “xarray.open_dataset()
”
Link to external documentation with “xarray.open_dataset()
”
To create a new blog post:
I’ve done a .load()
here because we’ll be looping over time. Without the .load()
Dask will try to read from the file at every loop iteration, with it it will only read from the file once - though this does come at the cost of more memory usage so we need to be carefull of how large the grid is that we’re loading.
Now we can do the calculation. numpy.zeros_like()
will create an empty array that’s the same shape as the input array, which we’ll use to store the API values. Then we can just implement the algorithm with a loop
Now we can do the calculation. numpy.zeros_like()
will create an empty array that’s the same shape as the input array, which we’ll use to store the API values. Then we can just implement the algorithm with a loop
# Empty array to store the output
@@ -2242,7 +2242,7 @@ Pure Dask (Advanced)map_blocks runs the function on each chunk independently, which is fine if the data is chunked horizontally but if the chunking is over time then we need some way of passing in the previous time’s API value.
-Let’s re-open the original dataset, adding some horizontal chunking this time, and see how we can deal with the previous timesteps. data
will give you the Dask array from an Xarray object
+Let’s re-open the original dataset, adding some horizontal chunking this time, and see how we can deal with the previous timesteps. data
will give you the Dask array from an Xarray object
ds = xarray.open_mfdataset('/g/data/ua8/Precipitation/FROGs/1DD_V1/ERA5/*.nc', chunks={'lat': 90, 'lon': 90})
@@ -3248,7 +3248,7 @@ Conclusionsscipy.signal.lfilter()
or using numba instead of the loop over timesteps.
+
We’ve focussed more on memory use than speed here, if improved speed is desired you could look at functions like scipy.signal.lfilter()
or using numba instead of the loop over timesteps.
diff --git a/posts/2021-12-08-trajectories.html b/posts/2021-12-08-trajectories.html
index 38222ad..e2aaf54 100644
--- a/posts/2021-12-08-trajectories.html
+++ b/posts/2021-12-08-trajectories.html
@@ -534,7 +534,7 @@
-
- Sample data
Investigating the data#
-We can see how long a single trajectory spends in each region by creating a histogram with numpy.histogram()
:
+We can see how long a single trajectory spends in each region by creating a histogram with numpy.histogram()
:
-If we use this as a mask with numpy.where()
we can find out which region was entered
+If we use this as a mask with numpy.where()
we can find out which region was entered
# Transitions into each state
@@ -820,7 +820,7 @@ Investigating the data
Looking at all datasets#
-numpy.histogram()
only works with 1d data, to use it on all trajectories at once we can combine it with numpy.apply_along_axis()
:
+numpy.histogram()
only works with 1d data, to use it on all trajectories at once we can combine it with numpy.apply_along_axis()
: