Monte carlo case study #78

GbotemiB · 2024-01-20T01:52:46Z

This notebook describes an introduction to the Monte Carlo option in PyPSA-Earth configuration using Nigeria's power data.

… into monte-carlo

ekatef · 2024-03-14T18:08:04Z

notebooks/monte_carlo_case_study_nigeria_data.ipynb

+    "\n",
+    "The goal of this notebook is to analyze the results of the Monte-Carlo simulations and gain insights into the power system of Nigeria. It utilizes the Monte Carlo method to generate multiple scenarios and assess the uncertainty in the power system's performance.\n",
+    "\n",
+    "The Monte Carlo method is a statistical technique that involves running multiple simulations with randomly sampled input parameters to estimate the distribution of the output variables. By applying this method to Nigeria's power data, we can assess the reliability and robustness of the power system under different scenarios and identify potential areas for improvement.\n",


I think the method is applied rather to the model than to the data, right?

Yes, you are right. The method is applied to the model.

Then, it would be good to revise a bit the description ;)

ekatef

Hello @GbotemiB!

Great work! 😄 Added a couple of technical comments across the text. Two main points:

would be great to briefly introduce the model to explain what exactly are we reproducing;
the plots labels should be improved;
we can probably think on how to make a conclusion more impressive.

What is your feeling about that?

ekatef · 2024-03-14T18:11:47Z

notebooks/monte_carlo_case_study_nigeria_data.ipynb

+   "source": [
+    "## Sensitivity Analysis\n",
+    "\n",
+    "Sensitivity analysis helps us explore how variations in input parameters affect the model’s output. These inputs might include energy prices, efficiency factors, renewable energy availability, and demand patterns.\n",


It would be great to shortly introduce the model to clarify what are we actually doing.

ekatef · 2024-03-14T18:13:55Z

notebooks/monte_carlo_case_study_nigeria_data.ipynb

+    }
+   ],
+   "source": [
+    "import os\n",


Just a technical comment: os has been imported before.

ekatef · 2024-03-14T18:19:58Z

notebooks/monte_carlo_case_study_nigeria_data.ipynb

+    "    network = pypsa.Network(path)\n",
+    "    network_list.append(network)\n",
+    "\n",
+    "print(\"All Networks loaded successfully\")"


Doesn't it make sense to use logging instead of prints? We have anyway loaded logging import.

Yh, Totally fine to use logging.

ekatef · 2024-03-14T18:22:28Z

notebooks/monte_carlo_case_study_nigeria_data.ipynb

+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "MONTE_CARLO_PYPSA_FEATURES = [\n",


Have we inherited the capitalised variable names from some previous set-up of Monte-Carlo part? Not sure they look consistent...

I could make it smaller cases.

What I did was to copy the code implementation from the monte_carlo.py script

ekatef · 2024-03-14T18:28:55Z

notebooks/monte_carlo_case_study_nigeria_data.ipynb

+    "sns.barplot(src_beta_coefficients)\n",
+    "plt.title(\"Beta coefficients of the parameters\")\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Here is labelling for the x-axis of the plots with regards to the selected parameters\n",
+    "\n",
+    "0: 'loads_t.p_set <br>\n",
+    "1: 'generators_t.p_max_pu.loc[:, n.generators.carrier == \"solar\"]'<br>\n",
+    "2: 'generators_t.p_max_pu.loc[:, n.generators.carrier == \"onwind\"]'<br>\n",
+    "3: 'generators.capital_cost.loc[n.generators.carrier == \"solar\"]'<br>\n",
+    "4: 'generators.capital_cost.loc[n.generators.carrier == \"onwind\"]'<br>\n",
+    "5: 'generators.weight.loc[n.generators.carrier == \"onwind\"]'<br>\n",
+    "6: 'generators.weight.loc[n.generators.carrier == \"solar\"]"


The plots labels must be fixed on the plot, otherwise it's a bit difficult to comprehend.

We might have a much larger plot if the label is inserted directly into the x-axis. I tried using labels but it took a considerably large amount of space in the plot, covering some of the bars in the plot.

I am open to different ideas.

Fair point. Can we use short yet meaningful version of the labels?

GbotemiB · 2024-03-18T22:41:16Z

Hi @Katia, Thanks for the review so far.

I have updated the notebook based on the review. I think the notebook is ready for another round of review.

ekatef · 2024-03-22T08:18:31Z

Hello @GbotemiB!
Perfect, thanks a lot for introducing the changes. We are converging 🙂

A couple of additional comments, which are mainly follow-ups of our discussion.

Tutorial notebook

It could be probably easier for a reader to get the whole picture if we would organise the plots in a facet-like structure. It can be archived for example by using subplot methods of mathplotlib.
Agree that the negative objective values are of concern, so it could make sense to address this issue (tracked as this PR and issue a dedicated PR ) to make the notebook look nice and avoid confusing notebook users.

Sensitivity analysis notebook

I wonder if it could make sense to briefly explain how exactly the base case Nigeria simulation is designed? Not sure if it's obvious if you are out of this modeling flow (like ourselves in a couple of months ;) ). I see the following possible ways to improve clarity:
- add a link to the paper;
- provide a configuration file;
- insert a small plot of the network.
  The list is by no means exhaustive, and feel free to consider also other ways.
In my perception, replacing plot labels with the text has improved the clarity significantly. Could we probably make the next iteration making the labels "nicer" and explaining what do they mean?

What do you think?

GbotemiB · 2024-03-22T09:46:23Z

Hello @ekatef, Thank you very much for the comment. I will let you know when I have updated the notebook based on the review.

GbotemiB · 2024-04-08T17:01:42Z

Hi @ekatef,

I have made some recent changes to the notebook, happy to have you review them.

GbotemiB · 2024-06-10T10:19:42Z

Following the negative objective issue pypsa-meets-earth/pypsa-earth#961, there seems to be a significant change in the charts for the monte-carlo objectives from the notebook.

ekatef · 2024-06-10T21:23:37Z

Perfect, thanks a lot for the updates @GbotemiB!

Adding a review on the PR (sorry for taking so long for the next iteration):

My feeling is that presentation of the method could be more precise. Currently, we state in the both notebooks that Monte-Carlo approach is needed to estimate the distribution of the output variables. That is true, of course, but may need some clarifications and additions.
Explanations can be probably improved a bit in these fragments:

assess the uncertainty in the power system's performance;
reliability and robustness of the power system.
I'd say that M-C approach allows to address the model features: the uncertainty is basically modeling uncertainty, while reliability and robustness relate rather to the model rather than to the power system itself. Does it make sense for you?

If feels like the flow could be probably made smoother in this paragraph:Decision-makers use this information to understand the implications of different choices on system performance and costs. Energy models inform policy decisions. Sensitivity analysis helps policymakers understand which factors drive energy transitions. (Happy to discuss on that)
A part of the path seems to be hard-coded (mc part) in a few places, e.g. f"mc/pypsa-earth/results/NG/networks/elec_s_10_ec_lcopt_Co2L-24H_m{count}.nc".
Would be nice to have a header when loading a network (just before # load the config file used for the Monte-Carlo simulation).
Probably, we can also add some words here and under Loading simulation results to explain which exactly results are we loading.
Here is labelling for the x-axis of the plots with regards to the selected parameters -- perfect! Looks clear and consistent now. Can we elaborate a bit on human-redable explanation for each of the parameters in Config Parameters section? I mean that it would be nice to
Correlations are calculated against the objective function, right? Would be great to state is explicitly somewhere along the case-study notebook.

ekatef · 2024-06-10T21:26:14Z

@GbotemiB a really great work!

Would be great to address the points above to make the notebooks shine. But the current explanations are still good enough to help start with Monte-Carlo in PyPSA-Earth. So, not much is left to finalise the tutorials :)

GbotemiB · 2024-06-17T11:01:26Z

Thanks for the review @ekatef.

I have factored all the changes into the notebooks.

If feels like the flow could be probably made smoother in this paragraph:Decision-makers use this information to understand the implications of different choices on system performance and costs. Energy models inform policy decisions. Sensitivity analysis helps policymakers understand which factors drive energy transitions. (Happy to discuss on that)

I also made changes to this. But, feel free to revise.

Once again, thanks for the review.

Well, We wont still be able to proceed with the merge until the negative objective issue is solved, which is fine.

GbotemiB · 2024-07-10T13:40:02Z

@ekatef
I have made recent changes to the notebooks with some code revisions to remove hard coding.

I think we are getting closer.

ekatef · 2024-07-15T06:49:30Z

Hello @GbotemiB, brilliant work! Agree, we are almost there 😄

I have a few final comments which mainly relate to polishing of the description in monte_carlo_case_study_nigeria_data.ipynb.

Not sure about presentation of weightings under 5 & 6: Generator Weight (Solar/Wind):

These parameters assign a weighting to solar and wind generators. This weight can be used in optimization to prioritize or penalize certain types of generation based on various criteria, such as cost-effectiveness, reliability, or environmental impact.

Agree that it's a usual and general description of how weights are normally used, but in case of wind and solar, I'd rather say that generator weight rather account for the dynamics of renewable potential.

Description of Sensitivity Analysis sounds great, but it may feel a bit repetitive in some parts, especially when we are talking about decision making. Can you please go through it and make the flow a bit smoother?
The fragment can assess the reliability and robustness of the model under different scenarios and identify potential areas for improvement: not sure what is mean by potential areas of improvement. Can we add a couple of clarifying words, like interprete the results in a meaningful way or exclude and identify potential areas for improvement?
The fragment: The simulation results is based on the monte-carlo configuration defined in the monte_carlo_tutorial notebook focusing on the Nigeria power data. This allows to analyse and interprete the results. It feels like some details can be added there for more clarity, for example interprete the results in a meaningful way.

Could you please look into these points? You have done a really nice work and it would be great to polish it presentation.

GbotemiB · 2024-07-15T09:05:52Z

Thanks @ekatef for the review. I will let you know when the changes are ready.

ekatef · 2024-07-31T20:40:08Z

Hello @GbotemiB!
Thanks a lot for the revisions! Both notebooks look really great now, and the PR is ready to merge.

ekatef · 2024-07-31T20:43:26Z

Merged 😄
Many thanks for an amazing contribution @GbotemiB!! 🎉 🎉 🎉

GbotemiB · 2024-07-31T20:48:05Z

@ekatef
Thanks a lot for the support.

GbotemiB added 2 commits December 21, 2023 11:23

new monte-carlo case study

1def2fa

monte-carlo case study notebook draft

70031b7

GbotemiB mentioned this pull request Jan 23, 2024

Monte carlo Improvement pypsa-meets-earth/pypsa-earth#930

Merged

8 tasks

GbotemiB added 7 commits January 23, 2024 12:48

fixed comments

d1c64bb

adjusted config parameters

6ed9412

added more instruction to setup monte-carlo

d3816d0

Merge branch 'pypsa-meets-earth:main' into monte-carlo

6649030

added sensitvity analysis

8b98615

Merge branch 'monte-carlo' of https://github.com/GbotemiB/documentation…

aafbe26

… into monte-carlo

Merge branch 'main' into monte-carlo

3de0149

GbotemiB marked this pull request as ready for review February 16, 2024 11:29

splitted notebook into two parts

5d4cc24

ekatef reviewed Mar 14, 2024

View reviewed changes

reviewed notebook conclusion

bd09d8b

GbotemiB added 2 commits April 8, 2024 16:16

modified tutorial notebook

1a81f42

modified introduction

e554640

Merge branch 'pypsa-meets-earth:main' into monte-carlo

73bec68

adjusted notebook based on comments

6ed65ce

modified runs with objective value fix

f81c2ab

revised writeups

f86f45b

ekatef approved these changes Jul 31, 2024

View reviewed changes

ekatef merged commit 394eb5a into pypsa-meets-earth:main Jul 31, 2024
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Monte carlo case study #78

Monte carlo case study #78

GbotemiB commented Jan 20, 2024

ekatef Mar 14, 2024

GbotemiB Mar 14, 2024

ekatef Mar 15, 2024

ekatef left a comment

ekatef Mar 14, 2024

ekatef Mar 14, 2024

ekatef Mar 14, 2024

GbotemiB Mar 15, 2024

ekatef Mar 14, 2024

GbotemiB Mar 15, 2024

ekatef Mar 14, 2024

GbotemiB Mar 15, 2024

ekatef Mar 15, 2024

GbotemiB commented Mar 18, 2024

ekatef commented Mar 22, 2024

GbotemiB commented Mar 22, 2024

GbotemiB commented Apr 8, 2024

GbotemiB commented Jun 10, 2024

ekatef commented Jun 10, 2024

ekatef commented Jun 10, 2024

GbotemiB commented Jun 17, 2024

GbotemiB commented Jul 10, 2024

ekatef commented Jul 15, 2024

GbotemiB commented Jul 15, 2024

ekatef commented Jul 31, 2024

ekatef commented Jul 31, 2024

GbotemiB commented Jul 31, 2024

Monte carlo case study #78

Monte carlo case study #78

Conversation

GbotemiB commented Jan 20, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ekatef left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GbotemiB commented Mar 18, 2024

ekatef commented Mar 22, 2024

Tutorial notebook

Sensitivity analysis notebook

GbotemiB commented Mar 22, 2024

GbotemiB commented Apr 8, 2024

GbotemiB commented Jun 10, 2024

ekatef commented Jun 10, 2024

ekatef commented Jun 10, 2024

GbotemiB commented Jun 17, 2024

GbotemiB commented Jul 10, 2024

ekatef commented Jul 15, 2024

GbotemiB commented Jul 15, 2024

ekatef commented Jul 31, 2024

ekatef commented Jul 31, 2024

GbotemiB commented Jul 31, 2024