-
-
Notifications
You must be signed in to change notification settings - Fork 274
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
implementation of conda_2_28 (aka centos 8) #1941
Comments
What I understood was:
And maybe I am missing something else, but we can probably merge the discussion with the CentOS 8 thread mentioned in my comment above. |
Someone, I think @isuruf, mentioned listing versions for other things. libm maybe? |
Manylinux uses glibc major.minor for its versioning, and I think it's good to match that (I think in the core call the mood was that That said, not least after the ill-fated Debian-based
That's because manylinux cannot bring its own updated compiler stack along and so is dependent on having the devtoolset backports. Perhaps keeping RHEL X as a base (through one of its ABI-compatible derivatives like Alma,Rocky,UBI) solves the other versioning questions, even if we call it |
Yes. We plan to keep alma8 as the base. |
One plus about just using |
We'll need to tuck the cdt name in the CDTs somewhere maybe? Idk if the CDT package names will conflict or not. |
Can you send an example recipe where people are dealing directly with cdt_name? I thought the jinja2 function took care of that. |
On the other hand, that line in |
Hmmmm. Being able to at least match cdts to the os for our docker containers is inherently useful possibly? This would argue for using conda_2_28 in both places. |
That's not really needed. We use |
I'm not saying we alway have to have them matched. I'm saying that using the same notation in both places is helpful. |
Would it help to start a PR for Docker images? Or are we not ready for that yet? |
Go for it but I don't expect it to be merged anytime soon. |
Gotcha, what do we see as the required steps before they are merged? Asking since they wouldn't be integrated anywhere by just publishing the image. Or is there something else I'm missing? |
I am not sure what goes in them. If we need the sysroots to put in them, then we'd need that. If they don't have anything special, then we can just build it. |
Gotcha Don't think the sysroot is needed The images cache the compiler packages as a convenience, but that can be disabled temporarily or it can use older compilers for now. Not too worried about this Can't think of anything else of concern If something comes up when we start working on them, we can always discuss |
Started adding an image in PR ( conda-forge/docker-images#235 ) |
From Matt, we need to update |
What's the intention here? Being able to switch the image? (I'm asking because I'd be willing to give it a shot...) FWIW, the current 2.17 setup doesn't need any smithy interaction, it's enough to just add Is there a reason we couldn't switch the images to alma8, but keep the sysroot at cos6 (with opt-in upgrade to cos7 & alma8)? |
There is no reason we couldn't bump images. This may break builds using yum requirements if the packages have changed names or conventions upstream. |
According to this search, there's around 170 recipes that use |
Fair point. I'm happy with simply bumping the default image. |
There seems to be something going awry with the new repodata hack. I'm getting Alma 8 kernel headers together with the COS 7 sysroot on aarch:
Interestingly, this is not happening on PPC, where I get:
|
What makes you think it is the repodata hack? |
It must be related to the sysroot, where the kernel-headers are built, and I couldn't see a difference between aarch/ppc in conda-forge/linux-sysroot-feedstock#46. Both variants are pulling in crdh 4, but looking a bit closer, that divergence between aarch & ppc goes back much further. Seems we've been using newer kernel headers on aarch since conda-forge/linux-sysroot-feedstock#15 (corresponding to linux version in RHEL 8, but apparently still being downloaded through CentOS 7 repos1). Seems it's not critical. Footnotes
|
This is worth a read |
Looking at the most recent alma update (about a month ago) again, it does seem like they'll be focussing on CentOS Stream. They're not putting it that directly, but
certainly points in that direction. Following CentOS Stream will keep them ABI-compatible (which is enough for us), but not the previous bug-for-bug compatible. It's also not clear to me how they'll keep going after a given version of CentOS Stream goes EOL (much earlier than the respective RHEL version), but the parts we need are hardly going to keep changing much anyway. I couldn't make the core call today, but I see in the notes that:
I would stay with the RHEL clones. Even CentOS Stream would be good enough, if we just keep using it past its EOL (like we've been doing for CentOS 6 as well). It's not like we get a noticeably longer EOL with OpenSUSE (the latest 15.5 is EOL in Dec. '24, barely half a year more than CentOS Stream 8). Finally, we could still decide to do OpenSUSE independently of whether we finish off the 2.28 Alma sysroots now, because OpenSUSE is on glibc 2.31, so they would be entirely separate. |
@h-vetinari I meant using this: https://www.suse.com/news/SUSE-Preserves-Choice-in-Enterprise-Linux/ |
Yeah okay, sorry for misinterpreting that one. For that initiative, aside from opensuse cannibalising its own Enterprise Linux product (so who knows how long that remains attractive to them), they also only have the sources of CentOS stream to leverage for ABI-compat. Once the updates there stop, I don't see how this is any different from any of the other RHEL clones. More importantly, the clones are all ABI-compatible (which is the number one argument to have a clone in the first place), and likely very tightly derived from RHEL sources and/or CentOS stream sources. So it's very likely that we could change the RHEL clone behind our sysroot (and even CDTs), and it would not be an issue. So I think we can't really do much wrong by going with Alma for now. Either they find a way to keep going past Stream's EOL, or there's another ABI-compatible distro to switch to that does, or we just accept that the image doesn't receive updates anymore. None of these sound terrible IMO. |
Right yeah. My only thought might be that folks could choose to be bug-for-bug compatible with their distro in which case we might have a preference. Only time will tell though. At the ABI level, totally agreed. |
I think no clone will realistically be able stay bug-for-bug. Alma already explicitly abandoned that. It's not realistic to do without knowing the RHEL sources, which no-one gets anymore (well, and if you sneak them out of a subscription and into your distro, RHEL can trivially observe in the open distro sources whether someone is picking up their internal patches, and then come and hound you). The only way to do it would be to reverse engineer the bug fix from the bug report and profiling RHEL behaviour, but ain't nobody got time for dat. It's just too much risk and effort for too little gain I think. Especially as ABI-compat is IMO orders of magnitude more important than whether a given bug behaves exactly the same as on RHEL. Beyond that, I don't see how bug-for-bug compat would be useful to conda-forge. We need the ABI, but I think we can agree that we'd generally like to have less bugs, not more (and most of our linux users will be on way newer distros that carry those fixes already). So from that POV, following CentOS stream would be attractive, were it not for the shorter EOL. Luckily, keeping the ABI while backporting their own bugfixes is something that is at least realistically feasible for the clones, so they actually have a chance to extend that EOL as promised, beyond what they get from stream. Overall, I still think the best choice is going forward with Alma now. |
IIRC in our last conda-forge meeting it sounded like we are ok going ahead with AlmaLinux 8. Did I understand correctly? If so, it sounds like it comes down to doing these next steps (particularly upgrading CDTs). Does that sound right? |
Yes but see the list above. We are trying to get rid of as many CDTs as we can. |
We have merged conda-forge/docker-images#242 recently. What are some next steps here? Are we ready to tackle the first CDTs (modulo conda-forge/cdt-builds#66)? |
We agreed to get rid of as many CDTs as we could. So that's the next step. To go through them and figure out what we can build ourselves. |
Maybe an earlier step is just getting a list of CDTs we use so we can search conda-forge for fuzzy matches |
This issue tracks centos 8 implementation PRs
track_features
.track_features
(ENH glibc 2.28 from alma8 linux-sysroot-feedstock#47)The alma 8 repo is https://repo.almalinux.org/almalinux/8.7/BaseOS/x86_64/os/Packages/ etc.
The text was updated successfully, but these errors were encountered: