Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reordering autolink-extract runtime orderings #65375

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

etcwilde
Copy link
Contributor

Order is important. The ordering generated by autolink-extract is nonsensical in main and 5.9. I haven't had a chance to actually build the full symbol dependency graph to generate the proper ordering so I expect that I'm missing something, but this gets us closer to the actual library link order, so it might hopefully unblock some folks who are running into the "missing" symbols.

rdar://107643395
rdar://107445812

@etcwilde
Copy link
Contributor Author

@swift-ci please test

"-lswiftSwiftOnoneSupport",
"-lswiftCore",
"-lswift_Concurrency",
"-lswift_StringProcessing",
"-lswift_RegexBuilder",
"-lswift_RegexParser",
"-lswift_Backtracing",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that these should be sunk lower? Right before swiftCore.

"-lswiftGlibc",
"-lBlocksRuntime",
// Dispatch-specific Swift runtime libs
"-lswift_Concurrency",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this have a dependency on swiftCore?

@etcwilde
Copy link
Contributor Author

etcwilde commented Apr 21, 2023

Why does it feel like you're playing 52 card pick up?

Best quote of 2023. We can go home now.

@etcwilde etcwilde force-pushed the ewilde/reorder-autolinks branch from 470a4bc to a2f3443 Compare April 21, 2023 23:32
@etcwilde
Copy link
Contributor Author

@swift-ci please test

@etcwilde etcwilde requested a review from compnerd April 21, 2023 23:55
@etcwilde etcwilde force-pushed the ewilde/reorder-autolinks branch from a2f3443 to d59af2e Compare April 21, 2023 23:56
Copy link
Member

@compnerd compnerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would buy this ordering.

@etcwilde
Copy link
Contributor Author

I would buy this ordering.

Now let's just hope we don't have too many cycles.

@etcwilde
Copy link
Contributor Author

@swift-ci please test

@@ -247,35 +247,31 @@ int autolink_extract_main(ArrayRef<const char *> Args, const char *Argv0,
// in most object files

std::vector<std::string> SwiftRuntimeLibsOrdered = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this benefit from a code comment for posterity that clarifies that ordering matters here and why it's ordered the way it is?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should probably also be a note that it's safe to list all libraries in this list regardless of whether any given link operation actually uses any particular library or whether the library exists in the "current" toolchain (the list only controls ordering and deduplication).

@MaxDesiatov
Copy link
Contributor

swiftlang/swift-integration-tests#78

@swift-ci build toolchain

"-lswift_StringProcessing",
"-lswift_RegexBuilder",
"-lswift_RegexParser",
"-lswift_Backtracing",
Copy link
Contributor

@al45tair al45tair Apr 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just FWIW, _Backtracing is going to depend on _StringProcessing on Linux. (It's using a regex to parse some things from /proc.)

Copy link
Contributor

@al45tair al45tair left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (modulo the comment about the future dependencies of _Backtracing).

@MaxDesiatov
Copy link
Contributor

Can this be closed given that we don't have tests that confirm that changing ordering fixes the issue we were chasing and we've merged alternative fixes since then?

@etcwilde
Copy link
Contributor Author

etcwilde commented Jun 2, 2023

We may still want to fix the ordering since, even though the group makes it so that ordering “doesn’t matter”, it’s basically a while-loop around the linker algorithm until things converge. Putting things in the right order will help them converge faster while consuming less memory. It’s basically an optimization now.

Order is important. The ordering generated by autolink-extract is
nonsensical. I haven't had a chance to actually build the full symbol
dependency graph so I'm definitely missing something, but this gets us
closer to the actual library link order, so it might unblock some folks
who are running into missing symbols.
@etcwilde etcwilde force-pushed the ewilde/reorder-autolinks branch from d59af2e to 0075695 Compare June 2, 2023 23:59
"-lFoundation",
"-lFoundationNetworking",
"-lFoundationXML",
// Foundation support libs
"-lcurl",
"-lxml2",
"-luuid",
// XCTest runtime libs (must be first due to http://github.com/apple/swift-corelibs-xctest/issues/432)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might as well also fix my silly comment typo from way back in my original PR while we're at it:

Suggested change
// XCTest runtime libs (must be first due to http://github.com/apple/swift-corelibs-xctest/issues/432)
// XCTest runtime libs (must come after runtime libs due to https://github.com/apple/swift-corelibs-xctest/issues/432)

@gwynne
Copy link
Contributor

gwynne commented Jun 4, 2023

We may still want to fix the ordering since, even though the group makes it so that ordering “doesn’t matter”, it’s basically a while-loop around the linker algorithm until things converge. Putting things in the right order will help them converge faster while consuming less memory. It’s basically an optimization now.

FWIW, if the group had "fully" helped with ordering, I wouldn't have run into the XCTest issue that made me throw up my hands and add ordering to the deduplication pass in the first place, so a good choice of ordering is definitely a good thing. (Fixing the XCTest issue would also be good! 😆)

It was my impression that #65795 solved the static linking issue; does that include the Radars mentioned by this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants