-
Notifications
You must be signed in to change notification settings - Fork 142
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update MSTest.Test* packages from 3.6.4 to 3.7.0, follow best practices #848
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I saw in your comments that you used tooling to assist some of these refactors - might make an interesting short and sweet tech talk/demo for the team.
test/Microsoft.Sbom.Adapters.Tests/ComponentDetectionToSBOMPackageAdapterTests.cs
Show resolved
Hide resolved
test/Microsoft.Sbom.Adapters.Tests/ComponentDetectionToSBOMPackageAdapterTests.cs
Show resolved
Hide resolved
test/Microsoft.Sbom.Api.Tests/Config/ConfigurationBuilderTestsForGeneration.cs
Outdated
Show resolved
Hide resolved
test/Microsoft.Sbom.Api.Tests/Entities/FileValidationResultTest.cs
Outdated
Show resolved
Hide resolved
test/Microsoft.Sbom.Api.Tests/Executors/ComponentToPackageInfoConverterTests.cs
Show resolved
Hide resolved
test/Microsoft.Sbom.Api.Tests/Executors/ExternalDocumentReferenceWriterTest.cs
Show resolved
Hide resolved
test/Microsoft.Sbom.Api.Tests/SignValidator/SignValidationProviderTests.cs
Show resolved
Hide resolved
test/Microsoft.Sbom.Api.Tests/Utils/ComponentDetectionCliArgumentBuilderTests.cs
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would also strongly encourage you to enable test parallelization either from runsettings or using the folowing assembly level attribute [assembly: Parallelize(Scope = ExecutionScope.MethodLevel, Workers = 0)]
.
Finally, it'd be good to also consider moving to MSTest runner, which is faster, more ligtweight and the recommended way to run your tests.
@Evangelink, this entire PR was unexpected and frankly work that we hadn't scheduled. It's bloomed to 43 files, which is way too big for a single PR. Trying to add parallelization to this PR seems like it would only lead to pain and more churn. Please feel free to open an issue suggesting the parallelization feature and/or moving to MSTest runner, but I'll honestly say that it's unlikely to find much traction unless there's a compelling reason--our tests are fast enough that the inner dev loop won't be significantly improved by shaving a few seconds from the run time. |
Dependabot tried to update
MSTest.TestFramework
from 3.6.4 to 3.7.0 in #839, andMSTest.TestAdapter
in #841. These PR's both failed, as the 2 pieces need to move together for the tests to work correctly. In addition, the newer version ofMSTest.TestFramework
adds several new warnings, and since our builds treat warnings as errors, the build breaks.This PR touches a lot of files, but since these packages are all test-specific, so are the code changes. It was done as a series of commits, and reviewing each commit in sequence will probably simplify the review process. Commits are as follows:
ExpectedException
is no longer considered a best practice, so I changed them toThrowsException
orThrowsExceptionAsync
. Most of these changes were tool-driven, but some were manual and exposed some dead code in the unit testsAssert.AreEqual
wants the expected parameter first--there were many places in the code where we had the test parameter first. These changes were largely tool-drivenAssert.AreEqual
,Assert.IsNull
, andAssert.IsNotNull
are all preferred overAssert.IsTrue
becuase they are more descriptive. These changes were largely tool-driven.Assert.AreEqual
were in the wrong order, but the analyzer failed to catch the error. These changes were manual.Note: When running from inside Visual Studio, the net472 projects get skipped. They run correctly from the command line, so any potential regressions should get caught in the pipelines. The net80 versions of the tests work as expected in both Visual Studio and from the command line. I opened microsoft/testfx#4487 to report that issue. It was resolved as a dupe of microsoft/testfx#4426