Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]: Agent Resource Metrics Redirection Issue #4469

Closed
2 of 4 tasks
mjesperson opened this issue Oct 12, 2023 · 15 comments
Closed
2 of 4 tasks

[BUG]: Agent Resource Metrics Redirection Issue #4469

mjesperson opened this issue Oct 12, 2023 · 15 comments

Comments

@mjesperson
Copy link

What happened?

The recent addition of agent resource metrics makes monitoring log output almost impossible with system.debug enabled. The page while watching the logs constantly switches to the resource metrics.

If we click back on the task to look at the logs it keeps bringing me to the metrics. Can we add a way to disable this? Or use a different flag than system.debug? It makes monitoring out pipeline runs impossible and we value the extra output from system.debug but don't want to continually be redirected to these metrics every few seconds that it updates.

Versions

3.771.1

Both Ubuntu in Docker and Windows

Environment type (Please select at least one enviroment where you face this issue)

  • Self-Hosted
  • Microsoft Hosted
  • VMSS Pool
  • Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

Ubuntu 20.04 / Windows Server 2019

Version controll system

No response

Relevant log output

##[debug]Agent running environment resource - Disk: available:12911852.00MB out of 14532478.00MB, Memory: used 88MB out of 515552MB, CPU: usage 3.99
##[debug]Agent running environment resource - Disk: available:12911845.00MB out of 14532478.00MB, Memory: used 88MB out of 515552MB, CPU: usage 3.93
##[debug]Agent running environment resource - Disk: available:12911831.00MB out of 14532478.00MB, Memory: used 88MB out of 515552MB, CPU: usage 3.81
##[debug]Agent running environment resource - Disk: available:12911823.00MB out of 14532478.00MB, Memory: used 88MB out of 515552MB, CPU: usage 3.70
##[debug]Agent running environment resource - Disk: available:12911813.00MB out of 14532478.00MB, Memory: used 88MB out of 515552MB, CPU: usage 3.63
@kirill-ivlev
Copy link
Contributor

Hi @mjesperson ,
Thank you for reporting this issue, we will take a look.

@HarshdeepGupta
Copy link

Encountering the same issue, is there any ETA on this getting resolved?

@soniaAr
Copy link

soniaAr commented Nov 2, 2023

Hello,
i have the same issue, cannot even check the logs (running stryker mutator testing)

@serbathome
Copy link

Hi @mjesperson , Thank you for reporting this issue, we will take a look.

Any chance it will be fixed soon?

@kirill-ivlev
Copy link
Contributor

Hi @serbathome,
Thank you for raising this, fix was merged and soon will be rolled out.

@misseill
Copy link

Is this still awaiting deployment?

@kirill-ivlev
Copy link
Contributor

@misseill latest agent was rolled out, could you please confirm that issue is resolved for you?

@kirill-ivlev
Copy link
Contributor

Thank you for your feedback, since we haven't received any updates, I closing this item.
Please feel free to post updates if you still facing this issue.

@pbcahill
Copy link

Hi @kirill-ivlev, why was this closed, could you please reopen? This bug still exists and nothing has been merged/deployed to fix it. I think this is the PR to fix it, but no idea if it works - #4486

Monitoring running pipeline logs is still a very poor experience with this resource metrics feature/bug.

@MHebes
Copy link

MHebes commented Feb 20, 2024

I'm not sure which version exactly this was fixed in, but I updated my agent from 3.230.0 to 3.234.0 and it fixed the issue.

Before, my output would switch between showing exclusively Agent environment resources lines and my logs.

After, those Agent environment resources lines are correctly interleaved with my logs.

@RobLSDev
Copy link

RobLSDev commented Mar 5, 2024

I'm not sure which version exactly this was fixed in, but I updated my agent from 3.230.0 to 3.234.0 and it fixed the issue.

Before, my output would switch between showing exclusively Agent environment resources lines and my logs.

After, those Agent environment resources lines are correctly interleaved with my logs.

That wasn't the issue stated for this bug. The problem listed above is that those statements are flooding the logs, making them useless for other purposes.

@Marin73
Copy link

Marin73 commented Jun 10, 2024

Why is this bug closed? It is a spam of messages that seem to be meaningless most of the time.

image

It clogs up the log. Can the issue be reopened please? The agent version we are using is 3.234.0

@omaraladin
Copy link

I'm facing the same issue on a RHEL 8 behind a proxy

@omaraladin
Copy link

just pasting my vstsagent .env file here, for people who works behind the proxy. It is always the number 1 suspect of things not working. after I modified to the below, the issue disappeared

HTTP_PROXY=http://proxy.company.com:3128
HTTPS_PROXY=http://proxy.company.com:3128
http_proxy=http://proxy.company.com:3128
https_proxy=http://proxy.company.com:3128

and I second the comment that encouraged azure team to enhance logging of the vstsAgent!

@JtMotoX
Copy link

JtMotoX commented Oct 9, 2024

Posting here does not help anything. This is a closed issue and will not be reviewed by the developers. Please open a new issue and provide the link here so users can get directed to the new issue and upvote it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests