Partial SwiftSyntax support for experimental @abi attribute

[beccadax]

Matches (or will match) swiftlang/swift#76878.

This is mostly a subset of the work in #2882 which adds support for @abi on most of the declarations I intend to permit, but not on nominal types. It's intended to unblock progress on the feature while I work through how to refactor the syntax tree for nominal types without making overly disruptive changes.

I've updated a few things here to respond to review comments. In particular, a review comment asking about #if handling prompted me to add a test for that; I found that I didn't like how it was being diagnosed, so I added a facility that can parse #if syntax into unexpected nodes on its contents.

[beccadax]

@swift-ci please test

[ahoppen]

I’m not a fan of EnclosingUnexpectedHandle because it will cause a memory allocation every time parseDeclaration is called + retain release calls to EnclosingUnexpectedHandle, which I would really like to avoid for performance reasons. Also, when stashing tokens inside EnclosingUnexpectedHandle, we need to remember to eat them later on and put them somewhere in the syntax tree, which is quite easy to forget, which would cause use to loose source accuracy.

Instead, would it be possible to assign the unexpected #if tokens to the node returned by parseResult, similar to what we do here? This will create a copy of the resulting syntax node (and leave the original in the bump allocator) but that should be fine because we expect the error cases to be quite rare.

swift-syntax/Sources/SwiftParser/ParseSourceFile.swift

Lines 170 to 181 in 7aed63b

	let existingUnexpected: [RawSyntax]
	if let unexpectedNode = layout.children[layout.children.count - 1] {
	precondition(unexpectedNode.is(RawUnexpectedNodesSyntax.self))
	existingUnexpected = unexpectedNode.as(RawUnexpectedNodesSyntax.self).elements
	} else {
	existingUnexpected = []
	}
	let unexpected = RawUnexpectedNodesSyntax(elements: existingUnexpected + remainingTokens, arena: self.arena)
	let withUnexpected = layout.replacingChild(at: layout.children.count - 1, with: unexpected.raw, arena: self.arena)
	return R.init(transferTrailingTrivaFromEndOfFileIfPresent(raw: withUnexpected))!

[beccadax]

Ha—I actually did something like that first, but decided it required too many hacks. (Particularly when I added in support for decl headers, which could only be smuggled through the outer IfConfigDeclSyntax by wrapping them in the corresponding decl...)

Fine, though, I'll work it out.

[beccadax]

Changed the implementation to something that "un-parses" the RawIfConfigDeclSyntax nodes after they've been parsed.

[beccadax]

@swift-ci please test

[ahoppen]

Thank you, Becca. I like this, all the non-trivial stuff is nicely localized and thus easy to reason about.

[nkcsgexi]

@swift-ci please test

[nkcsgexi]

testSwiftValidationTestsuite seems to be failing on macOS. @beccadax could you take a look?

[ahoppen]

That failure is unrelated. I’m investigating it.

[ahoppen]

Disabled testSwiftValidationTestsuite while I investigate further: #2909

@swift-ci Please test macOS

[nkcsgexi]

Thank you, Alex!

[beccadax]

@ahoppen The validation test failure may be something I saw when I was working on this PR: the Swift repo's validation-test/compiler_crashers_fixed/26170-swift-parser-skipsingle.swift was causing a stack overflow because the (invalid) braces were too deeply nested. I've worked around it by applying this locally:

--- a/Tests/SwiftParserTest/ParserTests.swift
+++ b/Tests/SwiftParserTest/ParserTests.swift
@@ -134,7 +134,13 @@ class ParserTests: ParserTestCase {
     runParserTests(
       name: "Swift validation tests",
       path: testDir,
-      checkDiagnostics: false
+      checkDiagnostics: false,
+      shouldExclude: { url in
+        [
+          // FIXME: This test is overflowing the stack
+          "26170-swift-parser-skipsingle.swift"
+        ].contains(url.lastPathComponent)
+      }
     )
   }
 }

[nkcsgexi]

🎉