Test crash fixes #14789

pinzart90 · 2023-12-19T21:04:43Z

I changed my approach to the webview2 lifecycle issues:
Instead of trying to catch all misuses of webview2 (not disposing during tests in which they are allocated), I added a WebView2 derived class (to better control disposal and give better debugging logs) so that we can let the gc/finalizer clean them safely.

Added unhandled exception handlers - for better logs during testing
Fixed an issue with the Analytics tests - depending on the order of execution, some tests were hanging
Added better logging for Serialization tests

…sh_logs

github-actions · 2023-12-19T21:21:58Z

UI Smoke Tests

Test: success. 2 passed, 0 failed.
TestComplete Test Result
Workflow Run: UI Smoke Tests
Check: UI Smoke Tests - net8.0

github-actions · 2023-12-20T05:39:05Z

UI Smoke Tests

Test: success. 2 passed, 0 failed.
TestComplete Test Result
Workflow Run: UI Smoke Tests
Check: UI Smoke Tests - net8.0

github-actions · 2023-12-20T14:30:21Z

UI Smoke Tests

Test: success. 2 passed, 0 failed.
TestComplete Test Result
Workflow Run: UI Smoke Tests
Check: UI Smoke Tests - net8.0

github-actions · 2023-12-20T21:19:41Z

UI Smoke Tests

Test: success. 2 passed, 0 failed.
TestComplete Test Result
Workflow Run: UI Smoke Tests
Check: UI Smoke Tests - net8.0

test/DynamoCoreWpfTests/DynamoViewModelUnitTest.cs

github-actions · 2023-12-21T15:18:41Z

UI Smoke Tests

Test: success. 2 passed, 0 failed.
TestComplete Test Result
Workflow Run: UI Smoke Tests
Check: UI Smoke Tests - net8.0

…into test_crash_logs

github-actions · 2023-12-21T19:08:23Z

UI Smoke Tests

Test: success. 2 passed, 0 failed.
TestComplete Test Result
Workflow Run: UI Smoke Tests
Check: UI Smoke Tests - net8.0

github-actions · 2024-01-08T03:17:30Z

UI Smoke Tests

Test: success. 2 passed, 0 failed.
TestComplete Test Result
Workflow Run: UI Smoke Tests
Check: UI Smoke Tests - net8.0

github-actions · 2024-01-08T03:40:50Z

UI Smoke Tests

Test: success. 2 passed, 0 failed.
TestComplete Test Result
Workflow Run: UI Smoke Tests
Check: UI Smoke Tests - net8.0

mjkkirschner · 2024-01-10T19:09:34Z

test/DynamoCoreWpfTests/CoreUITests.cs

@@ -786,7 +787,7 @@ private void RestartTestSetup(bool startInTestMode)

            //create the view
            View = new DynamoView(ViewModel);
-            SynchronizationContext.SetSynchronizationContext(new SynchronizationContext());


these methods are only used for a few tests - correct?
RestartTestSetup and RestartTestSetupWithNewSettings I mean.

yes..just for 2 tests

mjkkirschner · 2024-01-10T19:11:25Z

test/DynamoCoreWpfTests/SerializationTests.cs

@@ -854,7 +855,7 @@ private void CompareWorkspaceViews(WorkspaceViewComparisonData a, WorkspaceViewC
                var valueB = b.NodeViewDataMap[kvp.Key];
                Assert.AreEqual(valueA, valueB,
                string.Format("Node View Data:{0} value, {1} is not equal to {2}",
-                a.NodeViewDataMap[kvp.Key].Name, valueA, valueB));
+                a.NodeViewDataMap[kvp.Key].Name, JsonSerializer.Serialize(valueA), JsonSerializer.Serialize(valueB)));


can this possibly work for all cases? I think it's fragile because we never know what test graphs are going to run.

I did find it useful for investigating some parallel tests. Ex https://master-5.jenkins.autodesk.com/job/Dynamo/job/DynamoSelfServe/job/test_parallel_selfserve/87/testReport/junit/DynamoCoreWpfTests/JSONSerializationTests/SerializationTest__C___Jenkins__workspace__Dynamo__DynamoSelfServe__test_parallel_selfserve__Dynamo__test__core__dsevaluation__regress722_dyn__/
Not sure how much to invest in better visibility here.
I though I would keep it since it does not seem that it would break stuff. (but I can remove it if you think it is dangerous)

mjkkirschner · 2024-01-10T19:12:17Z

test/DynamoCoreWpfTests/Setup.cs

@@ -11,9 +12,24 @@ public class Setup
    {
        private AssemblyHelper assemblyHelper;

+        private void CurrentDispatcher_UnhandledException(object sender, DispatcherUnhandledExceptionEventArgs e)
+        {
+            e.Handled = true;


so, do all these handlers have the potential for hiding real test failures from us?

Yes they do. I will try it again (to see if these actually work during testing - as in runtime). If they do not do anything useful..I will remove them

removed them since they do not seem to add any value to the testing context

mjkkirschner · 2024-01-10T19:12:59Z

test/DynamoCoreWpfTests/SplashScreenTests.cs

+        [SetUp]
+        public void SetUp()
+        {
+            TestUtilities.WebView2Stamp = TestContext.CurrentContext.Test.Name;


yeah I guess it would just be nice if instead of sprinkling this logic all over for any UI test it was just handled in the DynamoWebView2 wrapper.

I could try it. But it would mean to have more testing logic in runtime code. It would probably involve reflection and asembly checking..

let's leave that potential refactor for later, for now it's just one static string.

mjkkirschner

a couple questions.

github-actions · 2024-01-10T20:04:28Z

UI Smoke Tests

Test: success. 2 passed, 0 failed.
TestComplete Test Result
Workflow Run: UI Smoke Tests
Check: UI Smoke Tests - net8.0

pinzart90 · 2024-01-10T20:15:19Z

Passed here https://master-5.jenkins.autodesk.com/job/Dynamo/job/DynamoSelfServe/job/pullRequestValidation/13685/

github-actions · 2024-01-10T21:56:47Z

UI Smoke Tests

Test: success. 2 passed, 0 failed.
TestComplete Test Result
Workflow Run: UI Smoke Tests
Check: UI Smoke Tests - net8.0

github-actions · 2024-01-10T21:57:59Z

UI Smoke Tests

Test: success. 2 passed, 0 failed.
TestComplete Test Result
Workflow Run: UI Smoke Tests
Check: UI Smoke Tests - net8.0

pinzart90 · 2024-01-11T03:22:11Z

also passed here https://master-15.jenkins.autodesk.com/view/DYN/job/DYN-DevCI_Self_Service/1388/

aparajit-pratap · 2024-01-16T19:16:56Z

test/DynamoCoreTests/AnalyticsTests.cs


-            clientMoq.Verify(c => c.CreateCommandEvent("TestCommand", "", null), times);
+            e = Analytics.TrackTaskCommandEvent("TestCommand", "TestCommand description", null, new Dictionary<string, object>() { } ).Result;
+            clientMoq.Verify(c => c.CreateTaskCommandEvent("TestCommand", "", null, null), times);

            e = Analytics.TrackFileOperationEvent(this.TempFolder, Actions.Read, 5);


How about using TrackTaskFileOperationEvent here?

I did not intend to convert all obsolete APIs to the new ones with this PR.

test/ViewExtensionLibraryTests/Setup.cs

test/VisualizationTests/Setups.cs

aparajit-pratap

Some minor comments, then LGTM.

github-actions · 2024-01-19T14:54:30Z

UI Smoke Tests

Test: failure. 1 passed, 1 failed.
TestComplete Test Result
Workflow Run: UI Smoke Tests
Check: UI Smoke Tests - net8.0

sm6srw

LGTM!

* update * Update NodeModelAssemblyLoader.cs * update * update * update * update * Update AnalyticsTests.cs * Update SerializationTests.cs * Update Setup.cs * update * update * Update SplashScreen.xaml.cs * Update PublishPackageViewModelTests.cs * Update PublishPackageViewModelTests.cs * update * Update AssemblyInfo.cs * Update TestUtilities.cs * update * upate * update * update * update * Update AnalyticsTests.cs * Update WebView2Utilities.cs * update * update * Update Setup.cs * update * update * Update PeriodicEvaluationTests.cs --------- Co-authored-by: pinzart <[email protected]>

* update * Update NodeModelAssemblyLoader.cs * update * update * update * update * Update AnalyticsTests.cs * Update SerializationTests.cs * Update Setup.cs * update * update * Update SplashScreen.xaml.cs * Update PublishPackageViewModelTests.cs * Update PublishPackageViewModelTests.cs * update * Update AssemblyInfo.cs * Update TestUtilities.cs * update * upate * update * update * update * Update AnalyticsTests.cs * Update WebView2Utilities.cs * update * update * Update Setup.cs * update * update * Update PeriodicEvaluationTests.cs --------- Co-authored-by: pinzart90 <[email protected]> Co-authored-by: pinzart <[email protected]>

pinzart and others added 6 commits December 7, 2023 15:21

update

c45a3da

Update NodeModelAssemblyLoader.cs

135de06

update

32aaa28

Merge commit 'c45a3dafce611cfe68b25bbaad1c6f5a8c8c9b42' into test_cra…

5129a93

…sh_logs

Merge commit '135de06fd3b7c939b4b692b6bd0319b7a888a65c' into test_cra…

8c2c694

…sh_logs

Merge branch 'master' into test_crash_logs

cf3c37d

update

b0ec268

pinzart90 changed the title ~~Test crash logs~~ [WIP]Test crash logs Dec 20, 2023

pinzart90 added the DNM Do not merge. For PRs. label Dec 20, 2023

pinzart90 marked this pull request as ready for review December 20, 2023 05:21

Merge branch 'master' into test_crash_logs

4cc32b2

pinzart added 2 commits December 20, 2023 14:52

update

7cf5cbd

update

6c3add2

aparajit-pratap reviewed Dec 21, 2023

View reviewed changes

test/DynamoCoreWpfTests/DynamoViewModelUnitTest.cs Outdated Show resolved Hide resolved

pinzart and others added 4 commits December 20, 2023 23:12

Merge branch 'master' into test_crash_logs

5e90045

Update AnalyticsTests.cs

6d062dd

Update SerializationTests.cs

b58be3d

Merge branch 'master' into test_crash_logs

aee8814

pinzart added 2 commits December 21, 2023 13:53

Update Setup.cs

697b33d

Merge branch 'test_crash_logs' of https://github.com/DynamoDS/Dynamo …

e047f1e

…into test_crash_logs

update

a3a9cd7

Merge branch 'master' into test_crash_logs

f85d5f4

mjkkirschner reviewed Jan 10, 2024

View reviewed changes

update

9ebd4b0

pinzart90 removed the DNM Do not merge. For PRs. label Jan 10, 2024

pinzart90 added 2 commits January 10, 2024 16:40

update

eaebcca

Update Setup.cs

6867e6c

pinzart90 requested a review from mjkkirschner January 11, 2024 14:47

aparajit-pratap reviewed Jan 16, 2024

View reviewed changes

test/ViewExtensionLibraryTests/Setup.cs Outdated Show resolved Hide resolved

aparajit-pratap reviewed Jan 16, 2024

View reviewed changes

test/VisualizationTests/Setups.cs Outdated Show resolved Hide resolved

aparajit-pratap approved these changes Jan 16, 2024

View reviewed changes

pinzart90 added 2 commits January 19, 2024 09:34

update

3882214

update

4103d57

Update PeriodicEvaluationTests.cs

b88d49c

pinzart90 mentioned this pull request Jan 24, 2024

Dynamo new homepage implementation #14879

Merged

9 tasks

sm6srw approved these changes Jan 24, 2024

View reviewed changes

pinzart90 merged commit 2a6152a into master Jan 24, 2024
22 checks passed

pinzart90 deleted the test_crash_logs branch January 24, 2024 19:28

QilongTang mentioned this pull request Jan 27, 2024

[CherryPick] Test crash fixes (#14789) #14896

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test crash fixes #14789

Test crash fixes #14789

pinzart90 commented Dec 19, 2023 •

edited

Loading

github-actions bot commented Dec 19, 2023

github-actions bot commented Dec 20, 2023

github-actions bot commented Dec 20, 2023

github-actions bot commented Dec 20, 2023

github-actions bot commented Dec 21, 2023

github-actions bot commented Dec 21, 2023

github-actions bot commented Jan 8, 2024

github-actions bot commented Jan 8, 2024

mjkkirschner Jan 10, 2024 •

edited

Loading

pinzart90 Jan 10, 2024

mjkkirschner Jan 10, 2024

pinzart90 Jan 10, 2024

mjkkirschner Jan 10, 2024 •

edited

Loading

pinzart90 Jan 10, 2024

pinzart90 Jan 11, 2024

mjkkirschner Jan 10, 2024

pinzart90 Jan 10, 2024

mjkkirschner Jan 10, 2024

mjkkirschner left a comment

github-actions bot commented Jan 10, 2024

pinzart90 commented Jan 10, 2024

github-actions bot commented Jan 10, 2024

github-actions bot commented Jan 10, 2024

pinzart90 commented Jan 11, 2024

aparajit-pratap Jan 16, 2024

pinzart90 Jan 19, 2024

aparajit-pratap left a comment

github-actions bot commented Jan 19, 2024

sm6srw left a comment

Test crash fixes #14789

Test crash fixes #14789

Conversation

pinzart90 commented Dec 19, 2023 • edited Loading

github-actions bot commented Dec 19, 2023

UI Smoke Tests

github-actions bot commented Dec 20, 2023

UI Smoke Tests

github-actions bot commented Dec 20, 2023

UI Smoke Tests

github-actions bot commented Dec 20, 2023

UI Smoke Tests

github-actions bot commented Dec 21, 2023

UI Smoke Tests

github-actions bot commented Dec 21, 2023

UI Smoke Tests

github-actions bot commented Jan 8, 2024

UI Smoke Tests

github-actions bot commented Jan 8, 2024

UI Smoke Tests

mjkkirschner Jan 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mjkkirschner Jan 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mjkkirschner left a comment

Choose a reason for hiding this comment

github-actions bot commented Jan 10, 2024

UI Smoke Tests

pinzart90 commented Jan 10, 2024

github-actions bot commented Jan 10, 2024

UI Smoke Tests

github-actions bot commented Jan 10, 2024

UI Smoke Tests

pinzart90 commented Jan 11, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aparajit-pratap left a comment

Choose a reason for hiding this comment

github-actions bot commented Jan 19, 2024

UI Smoke Tests

sm6srw left a comment

Choose a reason for hiding this comment

pinzart90 commented Dec 19, 2023 •

edited

Loading

mjkkirschner Jan 10, 2024 •

edited

Loading

mjkkirschner Jan 10, 2024 •

edited

Loading