Uses psutils to get the cpu utilization

[ribalba]

While working on the cloud machine script it was quite annoying to have to keep the c script around. It looks like the c metrics provider and the psutils implementation give similar results. Also I can't see much difference on overhead on my development machine.

I think this will also make the ci scripts easier as we don't need to compile the metric provider all the time. I have the feeling I am missing something though. Why didn't we do this from the beginning?

[ArneTR]

uh, nice! Will merge that in when I do the rest of the auto functionality.

The parameter will need to be renamed than.

A funny note on a typo: "cpu_utalisation" Your autocorrect has created a german crippled variant of the english word that gave me quite a laugh

[ArneTR]

A question on this: I suppose you have made this so you can easily use it on macOS.

How does psutils internally get the CPU % on macOS?

[ribalba]

• Hahah, just inventing new words here.
• The actual reason I added it so that I don't need to create a statically linked version of the metrics provider as I don't want gcc on the ec2 machine just for the reporter. Seemed to be overkill.

I did some research how psutils works.

• On Mac it uses the host_statistics call [0]. I still need to check how htop does it.
• Under linux it just reads the procfs /stats file and parses it as does the metrics provider. Why did you do this in c and not in python. The difference of opening a file and parsing the lines shouldn't be that different?

https://web.mit.edu/darwin/src/modules/xnu/osfmk/man/host_statistics.html

[ArneTR]

sadly the psutil function suffers from the same issues as the C code I wrote (most likely because they use the same syscall).

This is what I get when running with 100ms. A lot of unexpected zeros:

[ArneTR]

I also flagged this here: giampaolo/psutil#1700

[dan-mm]

We can also use this to decouple the XGBoostModel / SDIA metrics providers from our cpu-util metric providers, which would be quite nice. Especially for Mac, where we don't have a cpu-util provider

[ribalba]

@ArneTR can we merge this so I don't have to refer to a branch in my blog article?

[ArneTR]

it still needs macOS code, or not? Currently this only really works under linux

[ArneTR]

splendid! ❤️