Merge pull request #966 from GbotemiB/monte-carlo-docs

Fixes for errors in readthedocs

config.default.yaml

@@ -370,8 +370,6 @@ monte_carlo:
     generators_t.p_max_pu.loc[:, n.generators.carrier == "solar"]:
       type: beta
       args: [0.5, 2]
-  # TODO: Support inputs to simulate outages biggest lines "lines.p_nom_opt.max()": [-3000MW 0MW]
-  # TODO: Support inputs to simulate outages of biggest power plant "generators.p_nom.max()": [-1000MW 0MW]
 
 
 

config.tutorial.yaml

@@ -366,8 +366,6 @@ monte_carlo:
     generators_t.p_max_pu.loc[:, n.generators.carrier == "solar"]:
       type: beta
       args: [0.5, 2]
-  # TODO: Support inputs to simulate outages biggest lines "lines.p_nom_opt.max()": [-3000MW 0MW]
-  # TODO: Support inputs to simulate outages of biggest power plant "generators.p_nom.max()": [-1000MW 0MW]
 
 solving:
   options:

doc/api_reference.rst

@@ -80,12 +80,6 @@ clean_osm_data
 .. automodule:: clean_osm_data
     :members:
 
-config_osm_data
--------------------------------
-
-.. automodule:: config_osm_data
-    :members:
-
 download_osm_data
 -------------------------------
 

doc/conf.py

@@ -183,4 +183,4 @@
 # man_show_urls = False
 
 # Example configuration for intersphinx: refer to the Python standard library.
-intersphinx_mapping = {"https://docs.python.org/": None}
+intersphinx_mapping = {"python": ("https://docs.python.org/3", None)}

doc/index.rst

@@ -168,6 +168,17 @@ Documentation
    rules_overview
    api_reference
 
+**Advanced Usage**
+
+* :doc:`monte_carlo`
+
+.. toctree::
+   :hidden:
+   :maxdepth: 2
+   :caption: Advanced Usage
+
+   monte_carlo
+
 **Help and References**
 
 * :doc:`release_notes`

doc/installation.rst

@@ -129,7 +129,7 @@ To confirm the installation, run the following command in the activated environm
 
 
 Solver Installation
-------------------
+--------------------
 An optimization solver is needed to solve the mathematical problem that is build with the automated workflow of PyPSA-Earth.
 With the goal of supporting completely open source initiative, we focus on relying on Open-Source solvers, such as
 

doc/monte_carlo.rst

@@ -0,0 +1,95 @@
+.. SPDX-FileCopyrightText:  PyPSA-Earth and PyPSA-Eur Authors
+..
+.. SPDX-License-Identifier: CC-BY-4.0
+
+.. _monte_carlo:
+
+##########################################
+Monte Carlo
+##########################################
+
+The Monte Carlo method is a statistical technique that involves running
+multiple simulations with randomly sampled input parameters to estimate
+the distribution of the output variables.
+
+In Africa, navigating decision-making amidst rapid demand growth and
+substantial infrastructure changes poses a significant challenge. To
+tackle this uncertainty, the Monte Carlo method can be employed to
+construct a stochastic interpretation of deterministic model scenarios.
+This approach enhances the ability to account for variability and assess
+potential outcomes, providing a more robust foundation for informed
+decision-making.
+
+To use the Monte-Carlo method in PyPSA-Earth, you have to activate the
+``monte_carlo`` option in the configuration file (``config.yaml``),
+set ``add_to_snakefile`` to true in the monte_carlo section. This will
+enable the monte-carlo method.
+
+There are a few additional options that needs to be set in the monte_carlo
+configuration options.
+
+Set ``options``
+----------------
+- ``samples``: The number of samples to be used in the monte-carlo simulation.
+- ``samppling_strategy``: The method used to sample the input parameters. Either of ``pydoe2``, ``chaospy``, or ``scipy``.
+- ``seed``: The seed for the random number generator. It is useful to set the seed to a fixed value to ensure reproducibility of the results.
+
+Set ``uncertainties``
+---------------------
+The ``uncertainties`` section in the configuration file is used to specify the
+parameters to be sampled in the monte-carlo simulation. The uncertainties
+section is a dictionary with the keys being the ``pypsa object value`` to be
+sampled and the values split into ``type`` and ``args`` of which ``type`` is used to
+select the distribution and ``args`` used to specify the parameters of the selected
+distribution type.
+
+The following is an example of the uncertainties section in the configuration file:
+
+.. code:: yaml
+
+    uncertainties:
+        loads_t.p_set:
+          type: uniform
+          args: [0, 1]
+        generators_t.p_max_pu.loc[:, n.generators.carrier == "onwind"]:
+          type: lognormal
+          args: [1.5]
+        generators_t.p_max_pu.loc[:, n.generators.carrier == "solar"]:
+          type: beta
+          args: [0.5, 2]
+
+.. note::
+    To understand the different distribution types and their parameters,
+    check `Scipy Documentation <https://docs.scipy.org/doc/scipy/reference/stats.html>`_
+
+.. tip::
+    To create reallistic uncertainties, it is important to pay attention to
+    the distribution that is being applied to each ``pypsa object parameter``.
+
+Workflow
+--------
+
+To perform a dry-run of the monte-carlo simulation after setting the config options, use the following command:
+
+.. code:: bash
+
+    .../pypsa-earth % snakemake -j 1 solve_all_networks_monte -n
+
+To create a DAG of the monte-carlo simulation workflow, use the following command:
+
+.. code:: bash
+
+    .../pypsa-earth % snakemake -j 1 solve_all_networks_monte --dag | dot -Tpng > monte_carlo_workflow.png
+
+.. image:: img/monte_carlo_workflow.png
+    :align: center
+
+The monte-carlo simulation can be run using the following rule:
+
+.. code:: bash
+
+    .../pypsa-earth % snakemake -j 1 solve_all_networks_monte
+
+.. note::
+    Increasing the number of cores can make the process run faster. The numbers of cores can be increased by
+    setting the ``-j`` option to the desired number of cores.

doc/plotting.rst

@@ -41,7 +41,7 @@ Rule ``plot_p_nom_max``
 
 |
 
-.. automodule:: plot_p_nom_max
+.. .. automodule:: plot_p_nom_max
 
 .. _summary:
 

doc/populate/build_demand_profiles.rst

@@ -5,7 +5,7 @@
 .. _load_data:
 
 Rule ``build_demand_profiles``
-=============================
+===============================
 
 
 .. automodule:: build_demand_profiles

doc/requirements.txt

@@ -10,11 +10,13 @@ myst-parser  # recommark is deprecated, https://stackoverflow.com/a/71660856/135
 
 pypsa
 vresutils>=0.3.1
-powerplantmatching>=0.4.8
-atlite>=0.2.2
-dask<=2021.3.1
+powerplantmatching>=0.5.5
+atlite>=0.2.9
+dask[distributed]
+matplotlib<=3.5.2
+tabula-py
 
-# cartopy
+cartopy
 scikit-learn
 pycountry
 pyyaml
@@ -23,9 +25,15 @@ memory_profiler
 tables
 descartes
 
-esy-osm-pbf
-esy-osmfilter
 rioxarray
 git+https://github.com/davide-f/google-drive-downloader@master  # if not included will create error in docs `make html`
 
 gitpython
+
+chaospy
+numba
+ruamel.yaml<=0.17.26
+earth-osm>=0.1.0, <0.2.0
+reverse-geocode
+pyDOE2
+# graphviz

doc/solving/solve_operations_network.rst

@@ -38,5 +38,5 @@ Rule ``solve_operations_network``
 
 |
 
-.. automodule:: solve_operations_network
-    :noindex:
+.. .. automodule:: solve_operations_network
+..     :noindex:

scripts/_helpers.py

@@ -459,13 +459,13 @@ def mock_snakemake(rulename, **wildcards):
     the snakemake project. It returns a snakemake.script.Snakemake object,
     based on the Snakefile.
 
-    If a rule has wildcards, you have to specify them in **wildcards.
+    If a rule has wildcards, you have to specify them in **wildcards**.
 
     Parameters
     ----------
     rulename: str
         name of the rule for which the snakemake object should be generated
-    **wildcards:
+    wildcards:
         keyword arguments fixing the wildcards. Only necessary if wildcards are
         needed.
     """
@@ -718,7 +718,7 @@ def read_geojson(fn, cols=[], dtype=None, crs="EPSG:4326"):
     columns specified by the dtype dictionary it not none.
 
     Parameters:
-    ----------
+    ------------
     fn : str
         Path to the file to read
     cols : list
@@ -742,12 +742,12 @@ def read_geojson(fn, cols=[], dtype=None, crs="EPSG:4326"):
 
 def create_country_list(input, iso_coding=True):
     """
-    Create a country list for defined regions in config_osm_data.py.
+    Create a country list for defined regions in osm_config.yaml.
 
     Parameters
     ----------
     input : str
-        Any two-letter country name, regional name, or continent given in config_osm_data.py
+        Any two-letter country name, regional name, or continent given in osm_config.yaml
         Country name duplications won't distort the result.
         Examples are:
         ["NG","ZA"], downloading osm data for Nigeria and South Africa

scripts/build_demand_profiles.py

@@ -23,7 +23,7 @@
 ------
 
 - ``networks/base.nc``: confer :ref:`base`, a base PyPSA Network
-- ``resources/bus_regions/regions_onshore.geojson``: confer :ref:`build_bus_regions`
+- ``resources/bus_regions/regions_onshore.geojson``: confer :mod:`build_bus_regions`
 - ``load_data_paths``: paths to load profiles, e.g. hourly country load profiles produced by GEGIS
 - ``resources/shapes/gadm_shapes.geojson``: confer :ref:`shapes`, file containing the gadm shapes
 

scripts/build_powerplants.py

@@ -40,6 +40,7 @@
 -----------
 
 The configuration options ``electricity: powerplants_filter`` and ``electricity: custom_powerplants`` can be used to control whether data should be retrieved from the original powerplants database or from custom amendmends. These specify `pandas.query <https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.query.html>`_ commands.
+
 1. Adding all powerplants from custom:
 
     .. code:: yaml
@@ -69,20 +70,26 @@
         custom_powerplants: YearCommissioned <= 2015
 
 Format required for the custom_powerplants.csv should be similar to the powerplantmatching format with some additional considerations:
+
 Columns required: [id, Name, Fueltype, Technology, Set, Country, Capacity, Efficiency, DateIn, DateRetrofit, DateOut, lat, lon, Duration, Volume_Mm3, DamHeight_m, StorageCapacity_MWh, EIC, projectID]
 
 Tagging considerations for columns in the file:
+
 - FuelType: 'Natural Gas' has to be tagged either as 'OCGT', 'CCGT'
 - Technology: 'Reservoir' has to be set as 'ror' if hydro powerplants are to be considered as 'Generators' and not 'StorageUnits'
-- Country:  Country name has to be defined with its alpha2 code ('NG' for Nigeria,'BO' for Bolivia, 'FR' for France, etc.)
+- Country:  Country name has to be defined with its alpha2 code ('NG' for Nigeria,'BO' for Bolivia, 'FR' for France, etc.
 
 The following assumptions were done to map custom OSM-extracted power plants with powerplantmatching format.
+
 1. The benchmark PPM keys values were taken as follows:
         'Fueltype': ['Hydro', 'Hard Coal', 'Natural Gas', 'Lignite', 'Nuclear', 'Oil', 'Bioenergy'
             'Wind', 'Geothermal', 'Solar', 'Waste', 'Other']
+
         'Technology': ['Reservoir', 'Pumped Storage', 'Run-Of-River', 'Steam Turbine', 'CCGT', 'OCGT'
             'Pv', 'CCGT, Thermal', 'Offshore', 'Storage Technologies']
+
         'Set': ['Store', 'PP', 'CHP']
+
 2. OSM-extracted features were mapped into PPM ones using a (quite arbitrary) set of rules:
         'coal': 'Hard Coal'
         'wind_turbine': 'Onshore',