Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

all the rubies, plus all the arches #134

Open
wants to merge 162 commits into
base: main
Choose a base branch
from
Open

all the rubies, plus all the arches #134

wants to merge 162 commits into from

Conversation

YOU54F
Copy link

@YOU54F YOU54F commented Jul 8, 2023

builds all the rubies for all the things

  • arm64/aarch64 support on linux/macos
  • x86 & x86_64 support for windows
  • reproducible open source release pipeline for consistent builds
  • builds Ruby 2.6 through to Ruby 3.3.0
    • Only went back to 2.6.10 for unofficial HomeBrew support on Linux aarch64
  • Add's a CLI installer so users can get traveling-ruby at their fingertips, if they just need a ruby runtime, without any additional packaging steps
  • Tests all the things
  • build musl variants of linux, both x86_64 / aarch64
  • OpenSSL 3.2.0 for Ruby >= 3.1.0 / OpenSSL 1.1.1w for Ruby < 3.1.0

Fork:- https://github.com/YOU54F/traveling-ruby

background #133 (comment)

Travelling ruby updates for arm64/aarch64

OS Architecture Supported
OSX x86_64
OSX aarch64 (arm)
Linux (glibc) x86_64
Linux (glibc) aarch64 (arm)
Linux (musl) x86_64
Linux (musl) aarch64 (arm)
Windows x86_64
Windows x86
Windows aarch64 (via x86 emulation)

macos x86_64 binaries work as far back at 10.15 Catalina.

10.14 Mojave fails on libgmp

Serhii Zhuha and others added 30 commits June 1, 2022 11:25
name            pkg_date  ruby_version    os     arch   size
---             --------  --------------  --     ----   ----
traveling-ruby  20230528  2.6.10          linux  arm64  8.5M
traveling-ruby  20230528  3.0.4           linux  arm64  8.9M
traveling-ruby  20230528  3.1.0           linux  arm64  8.9M
traveling-ruby  20230528  3.1.0-preview1  linux  arm64  8.9M
traveling-ruby  20230528  3.1.1           linux  arm64  8.9M
traveling-ruby  20230528  3.1.2           linux  arm64  8.9M
traveling-ruby  20230528  3.2.2           linux  arm64  7.8M
traveling-ruby  20230528  3.3.0-preview1  linux  arm64  7.8M
@YOU54F
Copy link
Author

YOU54F commented Feb 16, 2024

New release is out

Highlights

  • New rubies
  • New gems (mysql2) plus Gem updates
  • Bundler update
  • Runtime dep updates
  • New platforms
    • New Alpine based holy-build box and musl based linux packages
  • New GH Workflows
    • Updated GH action workflows, including macos-14

New Rubies

  • Ruby 3.3.0 Release

Bundler Update

  • Bundler updated to 2.5.3 for Ruby versions > 3
  • Bundled updated to 2.4.22 for Ruby versions < 3

New Gems plus Gem updates

  • mysql2 gem now prebuilt for linux/macos/alpine
    RedCloth (4.3.3)
    bcrypt (3.1.20)
    charlock_holmes (0.7.7)
    curses (1.4.4)
    daemons (1.4.1)
    escape_utils (1.3.0)
    eventmachine (1.2.7)
    fast-stemmer (1.0.2)
    ffi (1.16.3)
    hitimes (2.0.0)
    json (2.7.1)
    kgio (2.11.4)
    mini_portile2 (2.8.5)
    mysql2 (0.5.6)
    nio4r (2.7.0)
    nokogiri (1.16.2)
      mini_portile2 (~> 2.8.2)
      racc (~> 1.4)
    pg (1.5.4)
    posix-spawn (0.3.15)
    puma (6.4.2)
      nio4r (~> 2.0)
    racc (1.7.3)
    rack (2.2.8)
    raindrops (0.20.1)
    redcarpet (3.6.0)
    rexml (3.2.6)
    rugged (1.7.2)
    sqlite3 (1.7.2)
      mini_portile2 (~> 2.8.0)
    thin (1.8.2)
      daemons (~> 1.0, >= 1.0.9)
      eventmachine (~> 1.0, >= 1.0.4)
      rack (>= 1, < 3)
    unf_ext (0.0.9.1)
    unicorn (6.1.0)
      kgio (~> 2.6)
      raindrops (~> 0.7)
    yajl-ruby (1.4.3)

OpenSSL

Ruby versions lower than 3.1.0 are built with OpenSSL 1.1.1w
Ruby versions 3.1.0 or later are build with OpenSSL 3.2.0

Dependency Updates

Linux Runtime updates

CCACHE_VERSION=4.9
CMAKE_VERSION=3.28.1
CMAKE_MAJOR_VERSION=3.28
GCC_LIBSTDCXX_VERSION=9.3.0
ZLIB_VERSION=1.3
OPENSSL_VERSION=3.2.0
CURL_VERSION=8.5.0
GIT_VERSION=2.43.0
SQLITE_VERSION=3450000
SQLITE_YEAR=2024
MYSQL_LIB_VERSION=6.1.9
POSTGRESQL_VERSION=15.5
ICU_RELEASE_VERSION=74-1
ICU_FILE_VERSION=74_1
LIBSSH2_VERSION=1.11.0
LIBFFI_VERSION=3.4.4
LIBYAML_VERSION=0.2.5

MacOS Runtime updates

CCACHE_VERSION=4.9
CMAKE_VERSION=3.28.1
PKG_CONFIG_VERSION=0.29.2
AUTOCONF_VERSION=2.71
AUTOMAKE_VERSION=1.16.5
LIBTOOL_VERSION=2.4.7
OPENSSL_VERSION=3.2.0
NCURSES_VERSION=6.4
LIBEDIT_VERSION=20230828-3.1
LIBEDIT_DIR_VERSION=20230828-3.1
GMP_VERSION=6.3.0
GMP_DIR_VERSION=6.3.0
LIBFFI_VERSION=3.4.4
LIBYAML_VERSION=0.2.5
SQLITE3_VERSION=3450000
SQLITE3_VERSION_YEAR=2024
XZ_VERSION=5.4.5
MYSQL_LIB_VERSION=6.1.9
POSTGRESQL_VERSION=15.5
ICU_RELEASE_VERSION=74-1
ICU_FILE_VERSION=74_1
LIBSSH2_VERSION=1.11.0
LIBXML2_VERSION=2.9.14
LIBXSLT_VERSION=1.1.34

New Platforms

musl based traveling-ruby packages

@YOU54F
Copy link
Author

YOU54F commented Apr 18, 2024

So I took a diversion from Ruby for a bit, to port one of our Ruby apps to Rust.

I thought it would be nice to have a fall back in the cli for each command, to allow it to call the existing ruby cli app, by use of a flag or env var.

With the help of some cross-compilers, I was able to package the rust app, for more supported platform/arch combos than ruby.

That left me with the behaviour of having to tell users who are wanting to fallback to the ruby impl, but don't have a traveling-ruby runtime that they are 💩 out of luck, and stuck with my poorly written rust code. 🤮

What to do 🤔

💡 cross-compilation adventure with qemu, and a switch to ubuntu/alpine over centos 7, as they had multi-platform images available where centos 7 we were stuck only working builds for amd64/arm64.

Updates

New additions

  • Linux - i386 / s390x / ppc64le / riscv64
  • Musl variants of all Linux architectures
  • Windows arm64 (Ruby 3.1.4 only)
  • New gems (bigdecimal / psych / stringio)
  • support for building snapshot from branches or master from ruby releases for (macos/linux) or head from rubyinstaller2 releases (windows)

🚧- Your mileage may vary with new platforms
⚠️ - openssl 1.1 based builds have been dropped, so nothing earlier than 3.1.4 for linux/macos

Check out the pre-release here

Linux

Platform Architecture Musl Glibc
ARMv8 64-bit (arm64v8) ARMv8
Linux x86-64 (amd64) x86-64
x86/i686 (i386) x86/i686
IBM z Systems (s390x) z Systems
IBM POWER8 (ppc64le) POWER8
RISC-V 64-bit (riscv64) RISC-V
  • Alpine binaries built against 3.15
  • Ubuntu binaries built against 14.04 (glibc 2.19)
    • ppc64le / s390x binaries against 16.04
    • riscv64 binaries against 20.04

MacOS

Platform Architecture Supported
MacOS x86-64 (darwin-x86_64) x86-64
MacOS arm64 (darwin-arm64) arm64
  • macos x86_64 binaries - 10.15 Catalina onwards
  • macos arm64 binaries - 11.0 Big Sur onwards

Windows

Platform Architecture Supported
Windows x86-64 (windows-x86_64) x86-64
Windows x86 (windows-x86) x86
Windows arm64 (windows-arm64) arm64 🚧
  • windows-arm64, ruby 3.1.4 only
  • 🚧 Native extensions not currently supported
    • Use ocran or aibika (forks of ocran) to build native extensions
  • Docker Support
    • Nanoserver images, will work if libgmp from package is copied to C:\Windows\System32
  • Wine support
    • x86_64 package fails on darwin-arm64 with unexpected ucrtbase.dll error
      • Workaround, use x86 package on darwin-arm64
  • Windows VM support
    • x86_64 package fails when emulated in vm's on darwin-arm64 with unexpected ucrtbase.dll error
      • Workaround, use x86 package on darwin-arm64
      • Workaround, use arm64 package on darwin-arm64

@YOU54F
Copy link
Author

YOU54F commented Apr 18, 2024

Also I've been working on a traveling-ruby app builder built in rust, for end users, that will package up their app for any or all of the supported TR platforms.

Rust supports embedding files (so an archive) or directory (with rust-embed), and I've managed to eliminate the need for end users app wrappers, by directly invoking the ruby binary with the correct TR environment variables setup.

This means you can distribute your ruby app as a single file which self-extracts its contents and then executes.

It's similar in vein for ocra / ruby_packer which provide a single entry point, however each of those unpack to a temporary directory, which affects speeds (it takes a short while to unpack it every time, and that seems expensive for our use case, cli invocations of which there are highly likely to multiple in quick succession)

In order to support native extensions for windows in my end users app, I cheated a little bit, I used ocran, exploded the temp directory and provided a new wrapper script.

Not sure if it will be of use to others, but it's been scratching my itch of getting to learn more rust and made it easy for me to test out the new TR combos

@utoppo
Copy link

utoppo commented Apr 21, 2024

Also I've been working on a traveling-ruby app builder built in rust, for end users, that will package up their app for any or all of the supported TR platforms.

Rust supports embedding files (so an archive) or directory (with rust-embed), and I've managed to eliminate the need for end users app wrappers, by directly invoking the ruby binary with the correct TR environment variables setup.

This means you can distribute your ruby app as a single file which self-extracts its contents and then executes.

It's similar in vein for ocra / ruby_packer which provide a single entry point, however each of those unpack to a temporary directory, which affects speeds (it takes a short while to unpack it every time, and that seems expensive for our use case, cli invocations of which there are highly likely to multiple in quick succession)

In order to support native extensions for windows in my end users app, I cheated a little bit, I used ocran, exploded the temp directory and provided a new wrapper script.

Not sure if it will be of use to others, but it's been scratching my itch of getting to learn more rust and made it easy for me to test out the new TR combos

That really sounds great! Thank you so much for all the effort!

updates rubygems to 3.5.9
updates bundler to 2.5.9

note need to drop 3.1.7 openssl fails to build in linux, (3.1.4 okay)
plus gem updates

ffi 1.17.0 (was 1.16.3)
racc 1.8.0 (was 1.7.3)
mini_portile2 2.8.7 (was 2.8.6)
hitimes 3.0.0 (was 2.0.0)
nio4r 2.7.3 (was 2.7.1)
curses 1.4.6 (was 1.4.5)
nokogiri 1.16.5 (was 1.16.4)
sqlite3 2.0.2 (was 2.0.1)
rexml 3.2.9 (was 3.2.6)
## fixes
- mini_portile2 pin to 2.8.6 due to errors in 2.8.7 on macos

## gem updates

rexml 3.3.0 (was 3.2.9)
stringio 3.1.1 (was 3.1.0)
nokogiri 1.16.6 (was 1.16.5)
charlock_holmes (0.7.9)
rexml (3.3.2)

drop unmaintained posix-spawn
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants