New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add job/stage durations, expose stage/task ids, add spark docs #47

Merged

sdewitt-newrelic merged 1 commit into main from feat/minor-spark-metric-updates-and-docs

Jan 13, 2025

Contributor

sdewitt-newrelic commented Jan 13, 2025

Description

Adds:

Add Spark job and stage duration metrics
Expose Spark stage id and attempt attributes and Spark task id and attempt attributes
Added Spark metric documentation

Updates:

Change Spark stage and task completed status to complete for consistency
Updated Spark dashboard and screenshots

Fixes:

Incorrectly pulling in peakMemoryMetrics for Spark stages and not peakExecutorMetrics

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
This change requires a documentation update

Checklist:

My code follows the style guidelines of this project
I have performed a self-review of my code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings


          feat: add job/stage durations, expose stage/task ids, add spark docs

9501ce4

aso1124 approved these changes

View reviewed changes

aso1124 left a comment

👍

sdewitt-newrelic merged commit 3f2c13c into main

8 checks passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet