Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds carbondb #61

Merged
merged 26 commits into from
Apr 10, 2024
Merged

Adds carbondb #61

merged 26 commits into from
Apr 10, 2024

Conversation

ribalba
Copy link
Member

@ribalba ribalba commented Mar 12, 2024

No description provided.

@ribalba ribalba requested review from dan-mm and ArneTR March 12, 2024 16:25
Copy link

github-actions bot commented Mar 12, 2024

Old Energy Estimation

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 5.57227 24.7645 1.7689 24
Measurement #1 5.81 12.3823 1.7689 8
Sleep #2 6.05625 12.3823 1.7689 9

📈 Energy graph:

 
 2.00 ┼─────────────
       Watts over time

Copy link

github-actions bot commented Mar 31, 2024

Old Energy Estimation

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 6.71885 24.7645 1.7689 27
Measurement #1 6.18875 12.3823 1.7689 8
Sleep #2 5.55625 12.3823 1.7689 8

📈 Energy graph:

 
 2.00 ┼─────────────
       Watts over time

Copy link

github-actions bot commented Mar 31, 2024

Old Energy Estimation

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 6.83923 24.7645 1.7689 27
Measurement #1 5.99 12.3823 1.7689 8
Sleep #2 5.72625 12.3823 1.7689 8

📈 Energy graph:

 
 2.00 ┼─────────────
       Watts over time

🌳 CO2 Data:
City: San Francisco, Lat: 37.783, Lon: -122.417
Carbon Intensity for this location: 208
CO2eq emitted for this job: 0.005151

Copy link

github-actions bot commented Apr 1, 2024

Old Energy Estimation

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 7.50417 24.7645 1.7689 26
Measurement #1 5.8525 12.3823 1.7689 8
Sleep #2 6.5625 12.3823 1.7689 8

📈 Energy graph:

 
 2.00 ┼─────────────
       Watts over time

🌳 CO2 Data:
City: Chicago, Lat: 41.8819, Lon: -87.6278
Carbon Intensity for this location: 320
CO2eq emitted for this job: 0.007924

source "$(dirname "$0")/vars.sh" add_var MEASUREMENT_RAN true

if [ -z "$cb_machine_uuid" ]; then
cb_machine_uuid=$(uuidgen)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Github runners can actually be placed on the exact same machine. I do not know how uuidgen works under the hood but I see a fair chance for a lot of collisisone here ...?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The uuidgen command generates a Universally Unique IDentifier (UUID), a 128-bit value guaranteed to be unique over both space and time.

scripts/vars.sh Outdated
get_carbon_intensity() {
latitude=$1
longitude=$2
token="your_electricity_maps_token_here"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this token ever gonna get filled? It is no variable?

Copy link
Member Author

@ribalba ribalba Apr 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never, as electricity maps works without a token. The string is not chose very well. And there is some legacy code there too. Added the option for a var

scripts/vars.sh Outdated

if echo "$response" | jq '.carbonIntensity' | grep -q null; then
echo "Required carbonIntensity is missing. Exiting" >&2
return 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you return a 1 here, but later you check for empty string. Is that intended and you want to proceed with 1 as a valida value?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 is for error. The check for empty string was from before I wrote stuff the stderr

shift
;;

\?)
echo "Invalid option -$OPTARG" >&2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you just want to echo here? Or maybe break?

@@ -35,6 +39,19 @@ inputs:
description: 'Base URL of the Github API to send data to. Default is api.github.com, but can be changed to your hostname if you have Github Enterprise'
default: 'api.github.com'
required: false
company-uuid:
description: 'If you want to add data to the CarbonDB you can set this to your company UUID'
default: ''
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other optional arguments like label are default null. Is it possible to unify that to either?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used empty string here as I can just pass it into the curl post command and not really care as the backend will then ignore it. I am not sure if handling null will make it a lot cleaner

- `send-data`: (optional) (default: true)
- Send metrics data to metrics.green-coding.io to create and display badge, and see an overview of the energy of your CI runs. Set to false to send no data. The data we send are: the energy value and duration of measurement; cpu model; repository name/branch/workflow_id/run_id; commit_hash; source (GitHub or GitLab). We use this data to display in our green-metrics-tool front-end here: https://metrics.green-coding.io/ci-index.html
- Send metrics data to metrics.green-coding.io to create and display badge, and see an overview of the energy of your CI runs. Set to false to send no data. The data we send are: the energy value and duration of measurement; cpu model; repository name/branch/workflow_id/run_id; commit_hash; source (GitHub or GitLab). We use this data to display in our green-metrics-tool front-end here: https://metrics.green-coding.io/ci-index.html
- `show-carbon`: (optional) (default: true)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having this default true is nice, but how will this work without the electricitymaps token?

@ribalba
Copy link
Member Author

ribalba commented Apr 3, 2024

@ArneTR feedback incorporated

Copy link

github-actions bot commented Apr 3, 2024

Old Energy Estimation

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 7.18875 24.7645 1.7689 26
Measurement #1 5.7275 12.3823 1.7689 9
Sleep #2 6.13375 12.3823 1.7689 8

📈 Energy graph:

 
 2.00 ┼─────────────
       Watts over time

🌳 CO2 Data:
City: Boydton, Lat: 36.677696, Lon: -78.37471
Carbon Intensity for this location: 349
CO2eq emitted for this job: 0.008643

Copy link

github-actions bot commented Apr 3, 2024

Old Energy Estimation

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 7.48708 24.7645 1.7689 25
Measurement #1 6.295 12.3823 1.7689 8
Sleep #2 6.20875 12.3823 1.7689 8

📈 Energy graph:

 
 2.00 ┼─────────────
       Watts over time

🌳 CO2 Data:
City: Boydton, Lat: 36.677696, Lon: -78.37471
Carbon Intensity for this location: 348
CO2eq emitted for this job: 0.008618

Copy link

github-actions bot commented Apr 3, 2024

Old Energy Estimation

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 7.16917 24.7645 1.7689 25
Measurement #1 6.095 12.3823 1.7689 8
Sleep #2 6.0225 12.3823 1.7689 9

📈 Energy graph:

 
 2.00 ┼─────────────
       Watts over time

🌳 CO2 Data:
City: San Francisco, Lat: 37.783, Lon: -122.417
Carbon Intensity for this location: 99
CO2eq emitted for this job: 0.002452

@ArneTR
Copy link
Member

ArneTR commented Apr 4, 2024

Thanks for the clarification that the token is optional.

I am not a fan however of the multiple ways to submit input. All other info is supplied via arguments. Only the Electricitymaps Token is now an exported environment var.

What is the rationale behind this?

In any case: I would favor for supplying it as argument and also making it a secret by default. Not a classic ENV.

  1. These two values submitted in the PR info need a dimension.

Carbon Intensity for this location: 99 (g / kg / tons per Wh, J , kwH ?)
CO2eq emitted for this job: 0.002452 (g, kg, ton ?)

@ribalba
Copy link
Member Author

ribalba commented Apr 8, 2024

The problem is that when we do it with a secret/ not as env we will need to add it as a parameter to the display_results.sh and then it will be visible in the logs. Which is not ideal. Secrets are only available in the action yml file.

Copy link

github-actions bot commented Apr 8, 2024

Old Energy Estimation

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 7.28375 24.8097 1.77212 25
Measurement #1 6.03875 12.3823 1.7689 8
Sleep #2 5.9825 12.4274 1.77535 8

📈 Energy graph:

 
 1.81 ┤           ╭╮
 1.81 ┤           ││
 1.81 ┤           ││
 1.80 ┤           ││
 1.80 ┤           ││
 1.79 ┤           ││
 1.79 ┤           ││
 1.78 ┤           ││
 1.78 ┤           ││
 1.77 ┤           ││
 1.77 ┼───────────╯╰
       Watts over time

🌳 CO2 Data:
City: Boydton, Lat: 36.677696, Lon: -78.37471
Carbon Intensity for this location: 365 gCO₂eq/kWh
CO2eq emitted for this job: 0.009055
gCO₂eq

Copy link

github-actions bot commented Apr 8, 2024

Old Energy Estimation

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 7.54708 24.7645 1.7689 25
Measurement #1 5.94875 12.3823 1.7689 8
Sleep #2 5.9525 12.3823 1.7689 9

📈 Energy graph:

 
 2.00 ┼─────────────
       Watts over time

🌳 CO2 Data:
City: Chicago, Lat: 41.8819, Lon: -87.6278
Carbon Intensity for this location: 365 gCO₂eq/kWh
CO2eq emitted for this job: 0.009039 gCO₂eq

@ArneTR
Copy link
Member

ArneTR commented Apr 8, 2024

TBD tomorrow

Copy link

github-actions bot commented Apr 9, 2024

Old Energy Estimation

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 7.242 24.7645 1.7689 27
Measurement #1 5.98375 12.3823 1.7689 9
Sleep #2 5.4625 12.3823 1.7689 9

📈 Energy graph:

 
 2.00 ┼─────────────
       Watts over time

🌳 CO2 Data:
City: Boydton, Lat: 36.677696, Lon: -78.37471
Carbon Intensity for this location: 389 gCO₂eq/kWh
CO2eq emitted for this job: 0.009633 gCO₂eq

Copy link

github-actions bot commented Apr 9, 2024

Old Energy Estimation

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 6.72417 24.7645 1.7689 26
Measurement #1 5.805 12.3823 1.7689 9
Sleep #2 5.80125 12.3823 1.7689 9

📈 Energy graph:

 
 2.00 ┼─────────────
       Watts over time

❌ CO2 Data:
Error in getting value. Please see the detail logs for the error messages!

Copy link

github-actions bot commented Apr 9, 2024

Old Energy Estimation

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 6.99346 24.8966 1.77833 27
Measurement #1 6.18375 12.3823 1.7689 9
Sleep #2 6.48375 12.5143 1.78775 8

📈 Energy graph:

 
 1.90 ┤            ╭
 1.89 ┤            │
 1.87 ┤            │
 1.86 ┤            │
 1.85 ┤            │
 1.83 ┤            │
 1.82 ┤            │
 1.81 ┤            │
 1.80 ┤            │
 1.78 ┤            │
 1.77 ┼────────────╯
       Watts over time

🌳 CO2 Data:
City: San Francisco, Lat: 37.783, Lon: -122.417
Carbon Intensity for this location: 89 gCO₂eq/kWh
CO2eq emitted for this job: 0.002216 gCO₂eq

Copy link

github-actions bot commented Apr 9, 2024

Old Energy Estimation

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 7.0608 24.7645 1.7689 25
Measurement #1 6.06375 12.3823 1.7689 8
Sleep #2 5.79125 12.3823 1.7689 8

📈 Energy graph:

 
 2.00 ┼─────────────
       Watts over time

❌ CO2 Data:
Error in retrieving values. Please see the detailed logs for the exact error messages!

Copy link

github-actions bot commented Apr 9, 2024

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 8.02074 26.7165 1.90832 28
Measurement #1 9.95 14.3342 2.04774 9
Sleep #2 5.7475 12.3823 1.7689 9

📈 Energy graph:

 
 3.72 ┤     ╭╮
 3.53 ┤     ││
 3.33 ┤     ││
 3.14 ┤     ││
 2.94 ┤     ││
 2.74 ┤     ││
 2.55 ┤     ││
 2.35 ┤     ││
 2.16 ┤     ││
 1.96 ┤     ││
 1.77 ┼─────╯╰──────
       Watts over time

🌳 CO2 Data:
City: Phoenix, Lat: 33.448376, Lon: -112.074036
Carbon Intensity for this location: 171 gCO₂eq/kWh
CO2eq emitted for this job: 0.004568 gCO₂eq

@ribalba
Copy link
Member Author

ribalba commented Apr 9, 2024

@ArneTR ready to merge

  • Add error text when apis fail and not fail silently
  • Curl will stop the script if it fails. Add detailed error checking
  • Token as secret

@ArneTR ArneTR merged commit 7732563 into main Apr 10, 2024
3 checks passed
@ArneTR ArneTR deleted the adds-carbondb branch April 10, 2024 22:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants