Remove getMarkerSchemaName special cases - look up marker schemas from data.type and nothing else #5293

mstange · 2025-01-06T14:40:34Z

This removes getMarkerSchemaName and simplifies the lookup of the marker schema - the marker schema is now purely based on data.type. This is a simpler and more predictable behavior, and it avoids accidentally matching a marker to the wrong schema.

I took care to keep any desirable effects from the old handling, with an upgrader.

Tests pass after every commit.

The behavior for the following profiles remains the same:

https://share.firefox.dev/3BuOoB3 (deploy)- the tracing markers should remain visible in the timeline-overview area
https://share.firefox.dev/41RKrBg (deploy) - CC / IdleCCSlice / IdleForgetSkippable markers should remain in the memory track

The behavior for the following profiles is changed, to good effect:

https://share.firefox.dev/4fKriEu (deploy) - the first marker should have an empty "Details" column, instead of the broken label "CPUSpeed Speed = GHz"
https://share.firefox.dev/4fx8cSf (deploy) - the Awake marker should have an empty "Details" column, instead of the broken label "Awake - CPU Id = "
https://share.firefox.dev/3BuOoB3 (deploy) - the PCompositorBridge::Msg_FlushRendering markers should have this name in the tooltip, instead of an empty label
https://share.firefox.dev/3P9G4cQ (deploy) - the UserTiming marker should have "Details" in the tooltip as the field label, rather than "User Marker Name". It's just a Text marker, not an actual UserTiming marker.
https://share.firefox.dev/3VQAJLt (deploy) - the Details column of the marker table should be populated with the marker text, and not be empty
https://share.firefox.dev/4j5Nfkb (deploy) - the "PCompositorBridge::Msg_NotifyChildCreated" IPC marker is now visible in the timeline-overview area

The behavior for the following profiles is changed, and it's somewhat unfortunate:

https://share.firefox.dev/4iWCe4C (deploy) - the grey rectangles around the RefreshDriver ticks in the timeline-overview area are no longer visible.

If we want RefreshDriverTick markers to be visible in the timeline-overview area, we should fix them on the Gecko side. They've been missing in all post-schema profiles, and nobody noticed. So I think this regression for old profiles can be accepted.

codecov · 2025-01-06T14:53:14Z

Codecov Report

Attention: Patch coverage is 87.17949% with 5 lines in your changes missing coverage. Please review.

Project coverage is 86.05%. Comparing base (f07cfdc) to head (8227268).
Report is 10 commits behind head on main.

Files with missing lines	Patch %	Lines
src/profile-logic/processed-profile-versioning.js	76.19%	4 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5293      +/-   ##
==========================================
- Coverage   86.07%   86.05%   -0.03%     
==========================================
  Files         311      311              
  Lines       29657    29666       +9     
  Branches     8196     8199       +3     
==========================================
+ Hits        25528    25529       +1     
- Misses       3547     3554       +7     
- Partials      582      583       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

This variable is always non-null. We likely wanted to check markerSchemaName but that's also always non-null at the moment.

…looking up the schema. The usefulness of this special case appears to have been limited to writing tests: Our tests contained many markers which only match a schema due to their name. So this patch moves the special case to the handling of TestDefinedMarkers: When the payload is absent, we synthesize a { type: <markerName> } payload for convenience. Markers from Firefox never needed this special case. And the special handling caused some trouble when markers were mistakenly matched against a schema with a custom label that was referring to fields which weren't present on the marker (because the marker doesn't have a payload). I found two cases of this in the wild, with "CPUSpeed" and "Awake" markers: The first CPUSpeed marker on https://share.firefox.dev/4gmLaif has a null data field. Before this patch, it matched a CPUSpeed schema. This gave it a broken tableLabel ("CPUSpeed Speed = GHz"). For Awake markers (e.g. on https://share.firefox.dev/403suOD ), the code in Firefox which emits these markers uses the typed marker API for the IntervalStart marker, and the payload-less API for the IntervalEnd marker: https://searchfox.org/mozilla-central/rev/43ce3de32b3a946bceac74c3badf442af9f245c0/tools/profiler/core/platform.cpp#7496-7497 So unpaired end markers got a broken tableLabel from the Awake schema before this patch ("Awake - CPU Id = " on https://share.firefox.dev/4fx8cSf ). These two cases are fixed by this patch - the two markers get the default tableLabel, which does not refer to these absent fields.

RefreshDriverTick markers from Firefox are Text markers. Firefox (as of Jan 2025) has never emitted a 'RefreshDriverTick' schema. Only pre-schema profiles have a 'RefreshDriverTick' schema, which they get from the v33 upgrader. Post-schema profiles don't display RefreshDriverTick markers in the timeline-overview area, because Text markers aren't displayed there. This commit makes the marker test consistent with what Firefox outputs. Now it no longer relies on a special case in getMarkerSchemaName which looks up schemas for Text markers based on the marker name. To test timeline-overview filtering, a new marker is introduced, with schema name 'VisibleInTimelineOverview'.

Just use data.type, like for other normal markers. This mostly just affects the RefreshDriverTick marker from upgraded profiles: - Pre-schema profiles which get their schema from the v33 upgrader have a schema with name 'RefreshDriverTick' with 'timeline-overview' in the display array. - Post-schema profiles from Firefox do not have a 'RefreshDriverTick' schema. The RefreshDriverTick markers in such profiles is a regular Text marker, and Text markers do not show up in the 'timeline-overview' area. This change makes it so that the 'RefreshDriverTick' schema from upgraded pre-schema profiles becomes unused, and the markers vanish from the timeline-overview area. This is consistent with post-schema profiles.

This adjusts some markers and schemas to be more consistent with what Firefox outputs: Markers with { type: "Paint" / "Navigation" / "Layout" } aren't present in any existing profiles, they were neither emitted by Firefox nor generated by an upgrader. Instead, these markers have { type: "tracing" } in their payload. DOMEvent markers were tracing markers in the past, and have their own schema today. The test now uses both variants. CC markers were tracing markers in the past, and have their own schema today. This test now uses the former. Post-schema profiles from Firefox have schema with name 'tracing'. This makes tracing markers visible in the 'timeline-overview' area. This patch adds the tracing schema to the test profile. This affects a test which was expecting that 'RandomTracingMarker' was not visible in the timeline-overview area. The test expectation is adjusted. Upgraded pre-schema profiles do not have a 'tracing' schema. This will be fixed in the next commit.

canova

Looks good to me, thanks! And sorry it took a while to get to it.

canova · 2025-01-22T11:28:08Z

src/types/profile.js

-/**
- * This is the base abstract class that marker payloads inherit from. This probably isn't
- * used directly in profiler.firefox.com, but is provided here for mainly documentation
- * purposes.


It's indicated that it's mostly here for the documentation purpose, but do you think it's not needed?

Yes, I think it's not needed. It's also been inaccurate for a long time - stack turned into cause many years ago, and startTime and endTime were moved out of the data object when we migrated to schema markers. Given the inaccuracies, I think it's fair to say that people haven't been relying on this documentation.

canova · 2025-01-22T11:43:20Z

src/test/fixtures/profiles/processed-profile.js

@@ -184,7 +191,7 @@ export function addMarkersToThreadWithCorrespondingSamples(
      markerSchemaForTests,
      stringTable
    );
-    markersTable.data.push(payload);
+    markersTable.data.push((payload: any));


Ah it's a bit unfortunate to use any here. But I guess it's because we don't have any definition of { type: name } in our types?

I haven't thought about it too much because we'll hopefully move the type out of the data object soon, into a dedicated column.

canova · 2025-01-22T11:47:00Z

src/profile-logic/gecko-profile-versioning.js

+        // which get this schema do not have any { type: 'RefreshDriverTick' }
+        // markers - in the past they picked up this schema due to a compat hack,
+        // but this hack is now removed. So this schema is unused. It is kept
+        // here for historical accuracy.


Thanks for the comment! I guess we can directly remove the whole schema from here, but don't mind leaving either.

Hmm, I suppose we could. I'll leave it for now.

src/test/unit/__snapshots__/profile-conversion.test.js.snap

canova · 2025-01-22T12:11:07Z

src/profile-logic/processed-profile-versioning.js

+    const schemaNames = new Set(profile.meta.markerSchema.map((s) => s.name));
+    const kTracingCCSchemaName = 'tracingCCFrom52Upgrader';
+    const shouldMigrateTracingCCMarkers =
+      schemaNames.has('CC') && !schemaNames.has(kTracingCCSchemaName);


Hm. Is it possible to get false from !schemaNames.has(kTracingCCSchemaName)?

Not realistically. I was just trying to be really careful to not mess with any information that was generated by non-Firefox sources. Though it's exceedingly unlikely that somebody happened to pick this exact name for their schema. I will remove it so that it's less confusing.

…emas. This adds an upgrader so that old profiles still get their CC markers displayed in the memory track and non-CC tracing markers displayed in the timeline-overview area. Newer profiles don't need this special case because their CC markers have a CC schema which correctly puts them in the memory track, and because they include a 'tracing' schema which puts non-CC tracing markers into the timeline-overview area.

…om getSchemaFromMarker.

mstange · 2025-01-22T18:16:24Z

Thanks for the review!

Updates: [Julien Wajsberg] Some more small refactorings (#5320) [Markus Stange] Pass the correct sample index offset to getTimingsForCallNodeIndex for the flame graph tooltip. (#5328) [Nisarg Jhaveri] Update docs to include Android Studio/Simpleperf trace file support (#5309) [Markus Stange] Don't pass the preview filtered thread to getTimingsForPath/CallNodeIndex. (#5329) [Nazım Can Altınova] Add a "Sample timestamp" field to the sample tooltip in timeline (#5322) [Markus Stange] Reduce confusion between call tree summary strategy aware samples and regular samples (#5330) [Markus Stange] Rename this getCounter selector to getCounters. (#5337) [Markus Stange] Make sample indexes compatible between the unfiltered and (preview) filtered call tree summary strategy samples when using an allocation strat> [Markus Stange] Remove some code that uses the preview filtered thread (#5336) [Markus Stange] Remove getMarkerSchemaName special cases - look up marker schemas from data.type and nothing else (#5293) [Markus Stange] Remove the makeProfileSerializable step - make the raw in-memory profile match the format that's stored in the file (#5287) [Nicolas Chevobbe] Adapt FilterNavigatorBar to High Contrast Mode. (#5257) [Nicolas Chevobbe] Adapt Tracks to High Contrast Mode. (#5252) [Markus Stange] Adjust string index fields in markers when merging threads (#5344) [Theodoros Nikolaou] Localize title and aria label in ProfileName (#5345) [Julien Wajsberg] Adapt time-slice selection in High Contrast Mode. (#5259) [Markus Stange] Make stackTable (sub)category derived data (#5342) [Markus Stange] Compute cpuRatio values when computing the derived thread (#5288) [Nazım Can Altınova] Add a context menu item to open the JS scripts in DevTools debugger (#5295) Also thanks to our localizers: el: Jim Spentzos fr: Théo Chevalier it: Francesco Lodolo [:flod] zh-TW: Pin-guang Chen

mstange requested a review from canova January 6, 2025 14:40

mstange self-assigned this Jan 6, 2025

mstange force-pushed the use-data-type-for-schema branch from 5295755 to 02732bf Compare January 6, 2025 14:43

mstange force-pushed the use-data-type-for-schema branch 2 times, most recently from 05205e3 to adca93c Compare January 6, 2025 15:01

mstange force-pushed the use-data-type-for-schema branch 2 times, most recently from 2acc632 to 2815f9e Compare January 13, 2025 22:47

mstange added 7 commits January 20, 2025 13:26

Remove unused ProfilerMarkerPayload type.

8e78cfd

Remove useless markerSchemaByName check.

b72025c

This variable is always non-null. We likely wanted to check markerSchemaName but that's also always non-null at the moment.

Allow getMarkerSchemaName to return null.

8d2e3f6

canova approved these changes Jan 22, 2025

View reviewed changes

mstange added 2 commits January 22, 2025 13:15

Remove getMarkerSchemaName and the now-unused marker name argument fr…

8227268

…om getSchemaFromMarker.

mstange force-pushed the use-data-type-for-schema branch from 2815f9e to 8227268 Compare January 22, 2025 18:16

mstange enabled auto-merge January 22, 2025 18:16

mstange merged commit a01fbe7 into firefox-devtools:main Jan 22, 2025
17 of 18 checks passed

canova mentioned this pull request Jan 30, 2025

Deploy Jan 30, 2025 #5348

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove getMarkerSchemaName special cases - look up marker schemas from data.type and nothing else #5293

Remove getMarkerSchemaName special cases - look up marker schemas from data.type and nothing else #5293

mstange commented Jan 6, 2025 •

edited

Loading

codecov bot commented Jan 6, 2025 •

edited

Loading

canova left a comment

canova Jan 22, 2025

mstange Jan 22, 2025

canova Jan 22, 2025

mstange Jan 22, 2025

canova Jan 22, 2025

mstange Jan 22, 2025

canova Jan 22, 2025

mstange Jan 22, 2025

mstange commented Jan 22, 2025

Remove getMarkerSchemaName special cases - look up marker schemas from data.type and nothing else #5293

Remove getMarkerSchemaName special cases - look up marker schemas from data.type and nothing else #5293

Conversation

mstange commented Jan 6, 2025 • edited Loading

codecov bot commented Jan 6, 2025 • edited Loading

Codecov Report

canova left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mstange commented Jan 22, 2025

mstange commented Jan 6, 2025 •

edited

Loading

codecov bot commented Jan 6, 2025 •

edited

Loading