Share smoke test config with pacapt

[rami3l]

@icy A quick status report of what we have for now, following our discussion in icy/pacapt#150 (comment) on a common smoke test suite shared between this project and icy/pacapt:

What's good

We already have:

• A docker-based smoke testing procedure.
• A concise DSL for easy code generation in test cases, as defined in README.md and showcased in dpkg.txt:
  • in to input pacapt(r) args
  • in ! to run shell commands
  • ou to pattern match output string
• The ability to run a part of the test suite (eg. ubuntu -> dpkg/apt only).
  • pacapt: This is done by the Makefile with make TESTS="dpkg.txt".
  • pacaptr: This is simply done by cargo test apt.

What's challenging

The following issues should be resolved:

• Choosing certain tests to run in certain environments.
  • On pacapt's side, the im syntax (for choosing images) should be removed able to be ignored, and the choice of images should be done by GitHub Actions config.
• Code generation. Two different implementations of the test framework should be maintained simultaneously.
  • On pacapt's side, the scripts should correctly notice GitHub Actions runner on failure.
  • On pacaptr's side:
    • The DSL should be translated to test suites using macros or build scripts.
    • While grep and pipes are great in *nix shells, pacaptr should always consider Windows support, which means they might have to be manually translated into Rust code.
• Consistency. This shared test suite should NOT enforce the consistency between the two projects to a degree. Both sides are still relatively free to develop new features.

[icy]

On pacapt's side, the im syntax (for choosing images) should be removed, and the choice of images should be done by GitHub Actions config.

@rami3l you may need that to run test on your laptop.

[icy]

I know act from my work, but I never intended to use it :) I think you can get around the im problem easily.

[rami3l]

@icy #130 here's the quick demo that I have been working on for the whole evening XD

PS: The CI didn't pass just yet because I haven't adjusted my test framework according to yours.

[icy]

Great! I will look at them later, it's almost midnight here :D

[rami3l]

@icy After some consideration, I think it's better to enable/disable the im feature with a flag, like --no-im, so that you can deploy your own CI in pacapt and at the same time run your script on your machine as usual.

[icy]

@rami3l It's also fine for me. The im instructions are required in pacapt CI until it's rewritten. You can ignore them in your CI setup (I think your CI system is a lot better than mine with parallel support.) Just go ahead and get the best thing and get it done for you.

Nice weekend.

[rami3l]

@icy I've written some basic tests in #134. Finally I didn't take your text files directly but rather made a framework that allows for easy copy-and-paste of specific tests.

This decision is made in order to make the most of the test structure that I have now. That is, I have multiple GitHub Actions runners running in parallel, each with a different OS, and the execution of tests inside each runner is also concurrent, thanks to cargo test.

Of course, it'll be better if we can somehow share the changes that each of us have made, but first let's get the basics working!

[icy]

Your work is great. Though it's a lot more than I expected :D I'd keep updating my current tests, it's enough and clean for me. (I can have some parallel support for them, but I really want to focus on tests per-se).