FreeBSD bootstrap #25
Merged
+1,699
−1,129
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
staticfloat
force-pushed
the
sf/binutils_toolchain
branch
2 times, most recently
from
0abcedb to
f50ce78
Compare
giordano
reviewed
Sep 28, 2025
Member
There was a problem hiding this comment.
I was wondering: are these manifests needed to be able to refer to the other local packages? If so, using the [sources] section of Project.toml is a saner way to do it, and you can get rid of checked in manifests.
This allows us to create a build environment with only Binutils and not a C compiler, which is needed for when we want to do multistage builds for e.g. FreeBSD, where we don't have a convenient GCCBootstrap process.
We need to change how the caching here works to cache both the resolution of version constraints to gitshas as well as the artifact paths themselves. Practically speaking, this means that we need to cache the `tree_hash` of the PackageSpec, and the `artifact_paths` of the `JLLSource`. I admit it is slightly inconvenient that we tie together Pkg resolution and platform selection here; it means that we can't jointly resolve artifacts for different platforms if we wanted to (although that would be a rather unusual case). That being said, I haven't run into a case where that's absolutely necessary yet, so hopefully we never do. The nasty case with the caching here has to do with things like Zlib_jll not having a `tree_hash` after it's loaded from the cache, which is why we need to make sure we mark down the `tree_hash` as well as the artifact paths.
If we've got one of our dependencies built in the universe we're building in, we need to not use the cached JLLSource resolution result, as it will bring in the wrong artifact paths. We don't clear the cache in this case, we just refrain from using it.
This changes our bootstrap technique to use the newly-split out BinutilsToolchain, and re-uses `GCCBoostrapMacOS` (which is now a terrible name) for FreeBSD. It also re-orders when we build ClangBootstrap and Clang for the same.
We have to juggle around how we build Binutils, GCCBootstrapMacOS, etc... to make sure everything builds in the right order.
We need to treat `nothing` just a teensy bit more specially here.
I've run into this error enough times, I need to make the error message clearer so I don't waste more time debugging it.
This most often happens when there's something wrong with one of the JLLs that constitute our toolchains, e.g. we try to deploy for a target we haven't built yet.
This helps when interactively debugging caching issues. NOTE: There was no bug, just a confusing issue with the bootstrap process that was constantly invalidating the cache. :)
We want to drop in to the Julia shell more reliably here
This is used during bootstrap so we can pass `--bootstrap`.
It's nicer to just make these discoverable on the REPL.
staticfloat
force-pushed
the
sf/binutils_toolchain
branch
5 times, most recently
from
d2ca89e to
4393f2f
Compare
This encodes the OS SDK version into the platform triplet for macOS and FreeBSD, to allow for upgrading to newer SDK versions and tracking whether a built artifact is compatible with a particular user's OS version.
This adjusts many things needed to get the final `run_full_bootstrap.jl` script to build all our compilers for everything. It takes ~18 hours to run right now, which could be significantly reduced with some scheduling improvements, but I'm also planning to split the non-bootstrap GCC and LLVM builds out into Yggdrasil proper, now that I have a functioning build.
This is a maintenance command that clears out the sources caches that we use to speed up build environment construction. It's mostly useful for debugging.
staticfloat
force-pushed
the
sf/binutils_toolchain
branch
from
4393f2f to
17267e8
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.