Rusty Quickemu and more CI

[flexiondotorg]

This topic continues #925, in which we discussed porting Quickemu to Rust and making better use of CI to generate download configurations for Quickget and Quickgui.

• @lj3954 has started work on a Rust re-implementation of Quickemu called quickemu-rs.
  • Other relevant work includes:
    • https://github.com/lj3954/rido
    • https://github.com/lj3954/qg-rust
• @lj3954 has also started on quickgetc

Objectives

Here is my first stab at some high-level short to midterm objectives for Quickemu and Quickgui:

• Transition to a Rust implementation of quickemu; this is the overriding goal
• Add macOS Apple Silicon support Quickemu
• Add multi-architecture (amd64 and aarch64 initially) to Quickemu
• Add macOS support to Quickgui
• Move Quickget to CI so most, or all, download configuration is processed in CI
• Update Quickget and Quickgui to consume CI-generated configuration
• Investigate if Quickemu on Windows is viable.

I'd like to add Apple Silicon and multi-architecture support to Quickemu Bash so that I can figure out the VM integrations and use that as a reference for quickemu-rs.

I've picked up some of the Quickgui development and can make sure we maintain compatible interfaces between quickemu-rs and Quickgui.

@lj3954, are you prepared to incorporate quickemu-rs and quickgetc as projects under the Quickemu Project organisation? If you're on board with this idea, you'd be extended elevated project permissions.

[lj3954]

• https://github.com/lj3954/qg-rust

This is an abandoned project. It essentially consisted of me trying to rewrite quickget bash verbatim, and therefore was very poorly written. It also only attempted to generate configurations for bash quickemu. The logic for windows downloads was moved from that project to the aforementioned rido library a few months ago, which also got a major refactor a couple days ago. The quickemu-rs project also contains a primitive quickget implementation which creates configurations for quickemu-rs. There's a lot that needs to be done for it to be feature complete.

• Add macOS Apple Silicon support Quickemu

A useful resource for this may be the UTM project.

• Add multi-architecture (amd64 and aarch64 initially) to Quickemu

Implementing this is quite a bit harder than it seems. Firmware on different ARM (and other RISC architectures) boards is often quite different, so a distro specifically needs to support the board type being emulated. Therefore, it's likely that a machine type option will have to be added, where quickemu would be able to emulate various SBCs and other relevant devices. Hopefully, this point will become less relevant as other architectures become more prevalent within desktop computing.

• Update Quickget and Quickgui to consume CI-generated configuration

I think it's probably still the best idea to have GUIs wrap some variant of quickget, since some operating systems (notably those from Apple and Microsoft) may be impossible to prepare in CI. The GUI shouldn't have to be updated alongside quickget whenever one of these downloads breaks. The only exception I can see is a potential future rust-based GUI, especially one using the GTK toolkit (as @TuxVinyards has mentioned a couple times), since windows/macOS downloading logic will be in rust libraries regardless, and GTK apps are predominantly packaged via containerized formats, especially Flatpak.

@lj3954, are you prepared to incorporate quickemu-rs and quickgetc as projects under the Quickemu Project organisation? If you're on board with this idea, you'd be extended elevated project permissions.

I certainly am prepared for quickemu-rs and related projects (quickget_configs) to be incorporated into the quickemu project, but the license of the project is GPLv3 and I'm not sure exactly how you feel about that, given that you clearly prefer licensing your software under permissive licenses.

Quickgetc is being written to eventually be a drop-in replacement for bash quickget; I don't view it as a new project, rather just an extension of bash quickemu. Of course, it does depend on a project which currently is separate, long term it would be unwise to merge quickgetc while the source for the JSON data is outside of the project.

[flexiondotorg]

Thanks for the detailed reply; there are excellent points and ideas here. I'll use quickemu and quickget to refer to the functionality, not the current or future state implementation.

• I have looked at UTM and also lima.

Therefore, it's likely that a machine type option will have to be added, where quickemu would be able to emulate various SBCs and other relevant devices.

I agree that the machine-type is a property we will need for ARM and RISC guests. I think quickget should grow that awareness and bake it into the configurations it creates that quickemu can then interpret.

I think it's probably still the best idea to have GUIs wrap some variant of quickget, since some operating systems (notably those from Apple and Microsoft) may be impossible to prepare in CI.

• Yes, the logic for downloading macOS and Windows will likely need to be encapsulated in quickget rather than rely on pre-built configs. What I hope is that quickget get much leaner for the open operating systems where it can simply consume a pre-built download configuration for the majority. The GUIs, such as Quickgui, can still interface with this leaner quickget to broker the downloads.

I certainly am prepared for quickemu-rs and related projects (quickget_configs) to be incorporated into the quickemu project, but the license of the project is GPLv3 and I'm not sure exactly how you feel about that, given that you clearly prefer licensing your software under permissive licenses.

I am absolutely fine with GPLv3 👍

I am most excited about quickget-rs, but I think reworking quickget to incorporate quickgetc and quickget_configs is the better first step.

• It seems that the first project to be migrated to the Quickemu Project org is https://github.com/lj3954/quickget_configs.
  • Do you agree?
  • If so, how shall we migrate quickget_configs? We could use GitHub project transfers or I could simply export and import the quickget_configs repo. Let me know how you want to proceed.
• Are you thinking of renaming quickgetc as quickget when it is a compatible drop-in?

Last night I started better setting up the Quickget and Quickgui projects. Adding more CI/automation, better issue/pull request templates, added CoC and other community docs/policies. Over the next few days I'll be digging through more of the team features and making sure the appropriate teams/permissions are in place for everyone to be effective.

[lj3954]

• It seems that the first project to be migrated to the Quickemu Project org is https://github.com/lj3954/quickget_configs.

The repository is ready to transfer, now that I've gotten done moving quickget_ci to it. The quickemu-rs repository now only hosts code for the user-facing software.
GitHub project transfers would be the most ideal solution, since everything would be redirected to the new repository.

• Are you thinking of renaming quickgetc as quickget when it is a compatible drop-in?

It's already a drop-in replacement, there just aren't enough operating systems available for it. Quickget bash won't be fully replaced until every operating system possible from it is in quickget_ci.

[flexiondotorg]

GitHub project transfers would be the most ideal solution, since everything would be redirected to the new repository.

Good point 👍