Introduce -m/--module flag to execute a `main` function in a package #52103

KristofferC · 2023-11-09T14:42:27Z

This aims to bring similar functionality to Julia as the -m flag for Python which exists to directly run some function in a package and being able to pass arguments to that function.

While in Python, python -m package args runs the file <package>.__main__.py, the equivalent Julia command (julia -m Package args) instead runs <Package>.main(args). The package is assumed to be installed in the environment julia is run in.

An example usage could be:

Add the package:

(@v1.11) pkg> add https://github.com/KristofferC/Rot13.jl
     Cloning git-repo `https://github.com/KristofferC/Rot13.jl`
    Updating git-repo `https://github.com/KristofferC/Rot13.jl`
   Resolving package versions...
    Updating `~/.julia/environments/v1.11/Project.toml`
  [43ef800a] + Rot13 v0.1.0 `https://github.com/KristofferC/Rot13.jl#master`
    Updating `~/.julia/environments/v1.11/Manifest.toml`
  [43ef800a] + Rot13 v0.1.0 `https://github.com/KristofferC/Rot13.jl#master`

And then it can be run (since it has a main function) via:

❯ ./julia/julia -m Rot13 "encrypt this for me" "and this as well"
rapelcg guvf sbe zr
naq guvf nf jryy

I'm not sure if -m/--module is the best choice but perhaps the association to Python makes it worth it.

Seelengrab · 2023-11-09T19:47:08Z

Can we make it so that a "less to write" version doesn't clutter the main environment by default? As is, if you want to keep environments clean you have to do something like

julia --project=@Rot13 -e 'import Pkg; Pkg.add("Rot13")'
julia -m Rot13 --project=@Rot13

which is super verbose. IMO tying executables/apps/<things-with-main> to a seperate environment instead of "whichever Julia happens to run in by default" seems sensible to me, least of which because it'll mean less recompilation for those "isolated" binaries on some update of something unrelated.

KristofferC · 2023-11-09T20:54:52Z

For CLI apps that you would run with just its name (rot13 args) I would agree that perhaps having isolated environments for each app would be a good idea. However, something like that would require a lot more infrastructure work. You would probably have a whole Pkg subcommand for that pkg> app status etc where you could see all the apps you have installed, update them (app update rot13) etc. It would basically be a parallel "universe" from standard packages.

However, for this feature, which is quite a lot more limited in scope, I think it makes sense to basically see it as just a shorthand syntax for loading a package and running a function in it. Having to create and resolve a new environment which might have different versions of what you have installed etc feels like it would be annoying. And then to update things you would have to go activate and update all these environments etc.

Seelengrab · 2023-11-09T21:16:51Z

OK, but isn't this entire PR/idea based around CLI apps? What use other than running CLI apps does this have, since that's also the motivating example above? We already have julia -e 'using Foo; Foo.bar()' for running arbitrary functions after all.

I really don't think citing Python is a good example of how this can be done - python packaging like that is the cause of many problems for sysadmins.

KristofferC · 2023-11-09T21:25:09Z

We already have julia -e 'using Foo; Foo.bar()' for running arbitrary functions after all.

Yes, but it isn't so easy to propagate arguments to that, for example:

~/.julia/packages
❯ ~/julia/julia -m Rot13 `ls`
NofgenpgNytroen
NofgenpgSSGf
NofgenpgYnggvprf
NofgenpgZPZP
NofgenpgGerrf
Npprffbef
Nqncg
NQGlcrf
NqinaprqUZP

or work with stdin + piping etc.

Seelengrab · 2023-11-09T21:29:04Z

That's not a good argument for why the "path of least resistance" for CLI apps should clutter the default environment by default.

mbauman · 2023-11-09T21:33:46Z

It seems to me like julia -m Rot13 "frperg" should use the Rot13 from environment that julia would use normally — if it's there, anyhow.

Then the question is what happens if Rot13 isn't in that environment. It'd be an error here. And perhaps it should automatically do what you suggest (that is, create/use a named env, probably with a prompt on creation/install)... but that seems to be an "extra" feature that's pretty orthogonal, no?

Seelengrab · 2023-11-09T21:41:24Z

but that seems to be an "extra" feature that's pretty orthogonal, no?

I don't think so - we don't really have CLI apps in the ecosystem precisely because it's a pain to use them with environments (not to mention startup time). Isolating them in an environment by default means we can precompile that environment completely for the main invocation, potentially even caching a pkgimage. That's not something that can be done with the ever-changing default environment.

I think it's worthwhile to think the usecase through here, and weigh the up & downsides before settling on an approach. To me, there are very clear UX, update & recompile downsides to making the "default environment" the "easy path", with very little (no?) upside (other than "it's how python does it", which is more of a downside to me..).

Seelengrab · 2023-11-09T21:45:27Z

or work with stdin + piping etc.

This too is highly dependent on the package in question, not on this interface. I could just as well have written julia -e 'using Rot13; Rot13.main(ARGS)' and it'd be exactly the same as what -m in this PR is doing; handling stdin is no easier in a package with this -m approach than one without - after all, this PR does nothing more than pass ARGS to a main it loaded. There's no difference in piping or working with stdin/stdout while writing the CLI.

KristofferC · 2023-11-09T21:46:25Z

I think this would mostly be used for packages that can have both functionality as a library and a package. For example, the Rot13 package could also be used in a non-CLI mode as a standard Julia package. In those cases I think it would be surprising to have e.g. julia -m Rot13 "frperg" to use possibly a different version of Rot13 and different version of dependencies compared to `julia -e 'using Rot13; Rot13.reverse_rot13("input")'.

KristofferC · 2023-11-09T21:50:17Z

I could just as well have written julia -e 'using Rot13; Rot13.main(ARGS)' and it'd be exactly the same as what -m in this PR is doing

Very possible but could you perhaps just show it explicitly to make your point a bit clearer?

Seelengrab · 2023-11-09T21:54:28Z

In those cases I think it would be surprising to have e.g. julia -m Rot13 "frperg" to use possibly a different version of Rot13 and different version of dependencies compared to `julia -e 'using Rot13; Rot13.reverse_rot13("input")'.

That sounds much more like an argument for "don't clutter the default environment with CLI apps because it will be held back by other, unrelated stuff in that environment" than an argument for making this use the default environment by default.

Do we really want to have lots of bug reports about old versions of CLIs floating around in the ecosystem? I think the SciML ecosystem still struggles with those because of old docs pages & old tutorials showing up on google.

Very possible but could you perhaps just show it explicitly to make your point a bit clearer?

Sure:

[sukera@tower ~]$ julia -q --project=@Rot13 -e 'import Pkg; Pkg.add("https://github.com/KristofferC/Rot13.jl")
[sukera@tower ~]$ julia -q --project=@Rot13 -e 'using Rot13; Rot13.main(ARGS)' "encrypt this for me" "and this as well"
rapelcg guvf sbe zr
naq guvf nf jryy

I've put the package into its own environment because I don't want random CLIs to clutter my main environment, and I don't want those to be held back by other random stuff in my main environment.

mbauman · 2023-11-09T21:58:08Z

The key question is what should $julia -m Rot13 "frperg" do if Rot13 exists in the environment that $julia uses (where $julia might use the default env... or it might use a JULIA_PROJECT or have --proj arguments). I agree with Kristoffer — I think it'd be very confusing if it'd come from a different place.

Seelengrab · 2023-11-09T22:04:35Z

I don't think it should. There should be a very clear seperation between uses as a library and uses as a packaged CLI/application. I think this should always use its bespoke environment.

If we want to look for other ecosystems for inspiration, can we take a look at e.g. Rust? Their CLI game is well known to be amazing, so why not take a page out of their book?

On the other hand, how do you propose to deal with held back packages because of CLI apps? Or the other way around?

KristofferC · 2023-11-09T22:09:55Z

Sure:

Thanks!

So to be clear, the concrete suggestion here is for julia -m Foo to default to julia --project=@Foo -m Foo? Some follow up questions to that:

Wouldn't it be annoying having to activate and add the package to that environment? Or do you want a pkg> app install Foo that creates some "internal" environment (e.g. .julia/environments/apps/Foo_$FOO_UUID) and resolves in there and then julia -m activates that environment?
If the environment is internal, how do you check what dependencies are installed / update them etc? pkg> app status Foo, pkg> app update Foo?
Are the dependencies of Foo "locked in" when the package is added or does Foo itself control that with some explicitly checked in manifest?
If I want to actually use a package in my environment would I do julia [email protected] Foo and make sure to match that project env with the julia version?

adienes · 2023-11-09T22:12:21Z

if I understand correctly

julia --project=@Rot13 -e 'import Pkg; Pkg.add("Rot13")'

this only happens once, on install?

I'm no expert on CLI design, but imo makes more sense for julia -m Rot13 to default use the same environment as julia

However, definitely also feels like a reasonable ask to make isolated envs for cli apps ergonomic, but I think that's orthogonal feature and shouldn't be default?

KristofferC · 2023-11-09T22:18:20Z

As I said before, I do think there is a difference between having a "driver" like julia -m Rot13 secret vs something that looks like a standalone executable (rot13 secret). In the first case you really know you are doing something julia specific and it is natural to think that standard julia behavior applies to it. In the second, "anything goes" since in many cases you might not even know you end up executing julia code so then it would be surprising that e.g. changes to the default julia environment would influence how the standalone looking rot13 behaves.

I think there is value in providing both these features but the second one is significantly more design heavy.

fredrikekre · 2023-11-09T22:32:01Z

As I said before, I do think there is a difference between having a "driver" like julia -m Rot13 secret vs something that looks like a standalone executable (rot13 secret). In the first case you really know you are doing something julia specific and it is natural to think that standard julia behavior applies to it. In the second, "anything goes" since in many cases you might not even know you end up executing julia code so then it would be surprising that e.g. changes to the default julia environment would influence how the standalone looking rot13 behaves.

I agree fully, and I had typed up an almost identical comment but got sniped. In particular, the current behavior lets me choose/control the environment, which can be useful sometimes. That wouldn't be possible with the other suggested approach.

Also, people should really relax about the global environment, it is amazingly convenient. I rarely, if ever, run into problems related to this. Maybe I just don't use "troublesome" ecosystems or combinations of packages.

(Disclaimer: this is just a meme that documents my personal development with environments in Julia over the years and not meant to be offensive to anyone 🙂 )

mbauman · 2023-11-09T23:00:02Z

I see this as being as much about ~~python -m http.server 8080~~ julia -m HTTP serve 8080 as it is about julia -m DataAnalysis.Preprocess data/*.csv or julia -m DataAnalysis.Dashboard. In the latter cases, I'd expect to be actively working on an active project — and could see this as being a useful paradigm for accessing lots of entry points therein. It'd be hair-pullingly painful if -m silently checked out a different version from what I'm actively doing!

As far as the pains of add/activate/instantiate and (potentially) dedicated environments, those span many more usages than just -m Module. E.g., JuliaLang/Pkg.jl#1415 and JuliaLang/Pkg.jl#2401. And I do think that -m could do something useful in the package-not-found-error case that helps smooth over some of these pain points... but that's an error case that can change in the future!

PallHaraldsson · 2023-11-09T23:33:28Z

How about on first use:

julia -m Rot13 "encrypt this for me" "and this as well"

would make an empty environment named Rot13, download Rot13 for you, the latest version, and on subsequent use, use that environment (and people could customize it in the meantime)? There's a question where it would be stored, in the current dir or elsewhere.

If Rot13 exists in the global environment (and maybe in each case if there's a conflict, then it could ask, "Do you want to use the global environment (once)? Or copy from it into a local environment? Or download latest version?".

Could a local environment point to the global environment somehow? Or can a local environment program already do that selectively, if it wants?

It seems you might want to support this the same way (at least now since not registered):

julia -m https://github.com/KristofferC/Rot13.jl "encrypt this for me" "and this as well"

It seems to me like julia -m Rot13 "frperg" should use the Rot13 from environment that julia would use normally — if it's there, anyhow.

If you feel like that, and people's ideas differ, should it maybe work like that with -M Rot13?

people should really relax about the global environment, it is amazingly convenient. I rarely, if ever, run into problems related to this.

I run into trouble all the time, I can't install packages, since my environment is too cluttered. FYI: Python has Zip-apps, that I think are though not much used, that allows a whole programs with all dependencies stored in one zipped file with .pyz ending. Julia wouldn't needs such if you can just point to an URL.

Seelengrab · 2023-11-10T08:24:03Z

So to be clear, the concrete suggestion here is for julia -m Foo to default to julia --project=@foo -m Foo? Some follow up questions to that:

All of these issues literally only exist because you've made the julia binary do all that work. The julia binary already has so many overloaded uses that there is hardly any design space left to do a better job - adding even more kitchen sinks to this as with this PR won't help with that.

Taking a page out of the rust book, why must julia be the driver binary for this?

Wouldn't it be annoying having to activate and add the package to that environment? Or do you want a pkg> app install Foo that creates some "internal" environment (e.g. .julia/environments/apps/Foo_$FOO_UUID) and resolves in there and then julia -m activates that environment?

The latter, similar to cargo install.

Though again, there is no need to have julia then also be the driver to run this. There is design opportunity here to actually make CLIs nice to use without having to type all of julia -m Foobar. If something is to be used as a CLI, it ought to be as standalone invokable as possible.

If the environment is internal, how do you check what dependencies are installed / update them etc? pkg> app status Foo, pkg> app update Foo?

It's a regular environment, so you can do something like ]activate app Foobar to activate the app/CLI environment and then do all the regular Pkg operations to your hearts content. It's a regular Pkg environment after all.

Are the dependencies of Foo "locked in" when the package is added or does Foo itself control that with some explicitly checked in manifest?

The dependencies of the CLI resolve as any other package environment does. If Foo compat bounds dependencies, all compat bounds apply as usual. The CLI can be updated as with any other environment, by activating it and calling ]up.

If I want to actually use a package in my environment would I do julia [email protected] Foo and make sure to match that project env with the julia version?

No, you can use julia --project=@., just as you can with any other environment. This defaults to the global environment when not in a project directory, and to a project environment when in a project directory. Since environments stack, a CLI package installed in your global environment will still be accessible, even if a project environment is activated.

Though again, that is only a limitation of being forced to use julia as the driver binary in the first place. This issue doesn't exist if you use a different driver binary (like rust does) for the compiler vs. CLI/Pkg management.

(Disclaimer: this is just a meme that documents my personal development with environments in Julia over the years and not meant to be offensive to anyone 🙂 )

I find this sort of commentary unnecessarily provocative, counterproductive and generally inappropriate. If you don't have anything productive to add to the discussion, please keep your memes to yourself. You're painting any usecase other than your own as being whiny and annoying - perhaps consider that there are different folks with different requirements who have good reasons for wanting to make the path of least resistance more welcoming to traditional software folk.

I find it generally discouraging that every single time I try to advocate for learning from the mistakes of other languages/ecosystems, people seem to double down on reinventing the wheel.

KristofferC · 2023-11-10T09:07:36Z

Though again, there is no need to have julia then also be the driver to run this.

So it seems you are arguing for having (perhaps only) the "standalone" app functionality I described in #52103 (comment) and #52103 (comment), (which is more similar to https://python-poetry.org/docs/pyproject/#scripts and https://doc.rust-lang.org/cargo/reference/cargo-targets.html#binaries.).

I agree that that functionality is useful and it is also something I am actively thinking about (see e.g a screenshot from some thoughts scratch document about this):

Maybe your worry is that CLI support in Julia would end with this PR in which case I understand (and would share) your worry. However, I would only consider this PR as a "little brother" to the standalone executable functionality not a replacement. This is still useful in smaller scale but you would indeed not want to write the next Dropbox using it (julia -m Dropbox lol).

As I wrote in #52103 (comment), there is a lot of design for the standalone CLI support and I would be very grateful for help with the design there. If you write to me I'll invite you to my scratch design document I screenshotted from above and we can iterate there.

KristofferC · 2023-11-10T09:21:17Z

Since environments stack, a CLI package installed in your global environment will still be accessible, even if a project environment is activated.

While true, it will look up dependencies in your project environment first which may be incompatible with other dependencies that gets found from the global env so it can cause quite weird bugs. I wouldn't want any standard use-case to rely on it.

I find it generally discouraging that every single time I try to advocate for learning from the mistakes of other languages/ecosystems, people seem to double down on reinventing the wheel.

Keep fighting the good fight, your input is valuable and appreciated :).

Seelengrab · 2023-11-10T10:07:36Z

Maybe your worry is that CLI support in Julia would end with this PR in which case I understand (and would share) your worry. However, I would only consider this PR as a "little brother" to the standalone executable functionality not a replacement. This is still useful in smaller scale but you would indeed not want to write the next Dropbox using it (julia -m Dropbox lol).

Yes indeed - having this be the "main" thing for apps is part of it, but another is that having multiple ways to do kind-of similar-ish things (run a CLI) with potentially different behaviors is confusing to say the least (as shown in this thread). Thus, I think we either have to reconcile the ideas somehow, or decide on a canonical way this ought to work.

I would be very grateful for help with the design there. If you write to me I'll invite you to my scratch design document I screenshotted from above and we can iterate there.

Sounds good - I'll write you on Slack.

Keno · 2023-11-14T16:29:16Z

base/client.jl

+            mods = split(arg, ".")
+            pkg = popfirst!(mods)
+            # Some more validation of `mods` and `pkg`?
+            m = Base.require(Main, Symbol(pkg))


So, technically this is all that is required if the module exports its main function - that was one of the explicit design goals of @main that you could use it as -e 'using Foo' to get the python app like behavior. I think it's fine to further shortcut that to -m, but I'm not sure about duplicating the main invocation logic here.

Okay I agree with that. So if an app want to have multiple "functionalities" (like the proposed julia -m Foo/Bar to run Foo.Bar.main) would you do that as julia -m Foo --bar or still do julia -m Foo/Bar and then have that still call Foo.main and look at something like Base.PROGRAM_FILE to determine what to execute.

lmiq · 2023-11-23T09:04:43Z

The issue I see with using the global environment is that such a call will be hardly reproducible. Someone incorporating a julia -m Foo arg into a script may have the script broken whenever changes to the global environment occur.

Of course then one can manage a specific environment for that manually, but then the utility of the default behavior is just limited to saving a few keystrokes, and hardly recomendable to a general user.

Since code cashing, using clean temporary (or not) environments became convenient (locally available packages install quickly), and IMHO Julia should move in the direction to encourage keeping the global environment as clean as possible. After you install CUDA, Makie, or another heavyweight precompiled package, getting random precompilations is really annoying. I bet no one wants to run julia -m Foo args and suddenly wait for five minutes.

KristofferC · 2023-12-15T14:15:45Z

@Keno, can you look at this new implementation?

KristofferC · 2023-12-15T14:19:32Z

The issue I see with using the global environment is that such a call will be hardly reproducible. Someone incorporating a julia -m Foo arg into a script may have the script broken whenever changes to the global environment occur.

Having these be reproducible is not the goal of this specific feature though. It is mostly a convenience to be able to write julia -m IJulia or julia -m Pluto.

For "full-scale apps" something more powerful is probably needed yes but that would look completely different (#52103 (comment)).

Seelengrab · 2023-12-15T14:44:51Z

Having these be reproducible is not the goal of this specific feature though.

I think everyone against -m is well aware that this is not a "goal" of your PR. We're just opposed to having this NOT be a goal, since designing only with Pluto or IJulia in mind means that other usecases will suffer disproportionally more than having Pluto or IJulia be in its own environment by default.

KristofferC · 2023-12-28T11:07:02Z

I've been toying a bit with the "standalone app" stuff I've mentioned before and I realize that in the implementation of that this would still be useful to have. The reason for that is that even in that case in the end you want to actually start Julia and run the entry point to the app.

You can either use -e which is inelegant and inefficient since it requires evaluating arbitrary code (and thereby requiring having an interpreter or compiler at hand). Comonicon.jl effectively uses -e since it runs a non-precompiled file with julia code. Or you can use -m (together with suitable load paths etc set up) which does exactly what you want and have the code to execute it available to run in the sysimage.

KristofferC · 2024-01-19T10:24:47Z

Bump... Does the last comment manage to persuade any of the nay-sayers? If not, what is the alternative? If the answer is -e it seems bad to have to add extra latency and force the existence of a julia compiler to run the arbitrary code in -e, as well as removing the possibility for better error messages since we do not know that a main entry point is being executed.

Seelengrab · 2024-01-19T11:55:00Z

From my POV, the concern with dumping everything in the default environment (as this PR currently does) has not been addressed 🤷 I'm aware that in a scenario of a standalone app with LOAD_PATH set up correctly etc. this is not a concern (since there we can easily control the whole chain of invocation), but in naive julia -m Foo usage, it still very much is.

KristofferC · 2024-01-19T12:05:18Z

This is not "dumping anything in the main environment". It provides an option to start up an entry point. This is needed to implement the full feature.

KristofferC · 2024-01-30T13:15:53Z

From my POV, the concern with dumping everything in the default environment (as this PR currently does) has not been addressed

Here is the PR that builds on top of this JuliaLang/Pkg.jl#3772

MasonProtter · 2024-01-30T13:34:50Z

I'm not one of the naysayers, but I'd just like to lend my voice in favour of this implementation and choice of environment semantic.

This is the most natural and straightforward extension of the way that the @main entrypoint currently works: simply extending it to allow for a different package than Main to provide their entrypoint. Having this silently and automatically create a separate environment seems bad.

vchuravy · 2024-01-30T17:52:13Z

base/client.jl

+        elseif cmd == 'm'
+            @eval Main import $(Symbol(arg)).main
+            if !should_use_main_entrypoint()
+                error("`main` in `$arg` not declared as entry point (using `@main`)")


Suggested change

error("`main` in `$arg` not declared as entry point (using `@main`)")

error("`main` in `$arg` not declared as entry point (use `@main` to do so)")

jakobnissen · 2024-01-31T13:07:34Z

That Pkg PR addresses all my usability concerns, and I see it builds on this one. So I'm in favor. And let me say, what an ambitious vision - it looks like the beginning of something amazing!

This aims to bring similar functionality to Julia as the `-m` flag for Python which exists to directly run some function in a package and being able to pass arguments to that function. While in Python, `python -m package args` runs the file `<package>.__main__.py`, the equivalent Julia command (`julia -m Package args`) instead runs `Package.main(args)`. The package is assumed to be installed in the environment `julia` is run in. An example usage could be: Add the package: ``` (@v1.11) pkg> add https://github.com/KristofferC/Rot13.jl Cloning git-repo `https://github.com/KristofferC/Rot13.jl` Updating git-repo `https://github.com/KristofferC/Rot13.jl` Resolving package versions... Updating `~/.julia/environments/v1.11/Project.toml` [43ef800a] + Rot13 v0.1.0 `https://github.com/KristofferC/Rot13.jl#master` Updating `~/.julia/environments/v1.11/Manifest.toml` [43ef800a] + Rot13 v0.1.0 `https://github.com/KristofferC/Rot13.jl#master` ``` And then it can be run (since it has a `main` function) via: ``` ❯ ./julia/julia -m Rot13 "encrypt this for me" "and this as well" rapelcg guvf sbe zr naq guvf nf jryy ```

Seelengrab · 2024-01-31T17:00:26Z

As I said above, if this were exclusive to Apps, I wouldn't have a problem with this. However, the overall commit message and NEWS entry don't make a mention of that, instead likening this to what Python does with having everything that's installed being dumped into the same "environment".

It's this standalone usage, separate from the usage as Apps, that the "naysayers" are worried about. In order to use this feature standalone, the most straightforward way would be to just add everything someone wants to use into the main environment, with the consequences outlined above. This has not been addressed.

Moreover, the current design in JuliaLang/Pkg.jl#3772 simply sets the JULIA_LOAD_PATH to exclusively use the installed app environment. To my understanding, the same thing could be achieved through an empty JULIA_LOAD_PATH, combined with passing the environment to be used through --project. Requiring the use of --project when using -m would completely alleviate my concerns, since that then creates an incentive for new users to go through our App system, instead of just thinking "oh it's like python" and messing up their main environment with potentially outdated packages due to version downgrades.

Seelengrab · 2024-02-01T17:02:26Z

As an alternative - can the flag just be made much longer to disincentivize its use? After all, if its intended for usage from Apps (& thus autogenerated shims), there should be no harm in having a longer flag. @lmiq since you also mentioned opposition to the current behavior, would that be a good compromise for you?

adienes · 2024-02-01T17:07:07Z

I like the current state of the PR and am excited to use -m ; I certainly don't think its use should be disincentivized

if you want to control the project you're running a module in, --project is always right at your fingertips

Seelengrab · 2024-02-01T17:15:57Z

I like the current state of the PR and am excited to use -m ; I certainly don't think its use should be disincentivized

Ok, how do you propose to deal with held back packages because of liberal usage of standalone -m? Or the associated long compile times when that main environment is modified and now all your CLIs have to recompile due to invalidation? Or the non-reproducibility of using julia -m Foo in a quickly thrown together bash script when the global environment changes?

As far as I've noticed, that hasn't been addressed by anyone in favor of liberal usage of standalone -m here. That's all I'm inquiring/trying to solve here. With properly installed Apps, this is of course not an issue because they are (according to the design document!) in their own environment, and not shared in Main.

adienes · 2024-02-01T17:22:57Z

how do you propose to deal with held back packages because of liberal usage of standalone -m?

this does not seem like a likely problem to me so I don't think it needs a solution

Or the associated long compile times when that main environment is modified and now all your CLIs have to recompile due to invalidation?

if it becomes an issue, I'll put each cli into its own environment. but I don't want that to happen by default

MasonProtter · 2024-02-01T17:42:54Z

It's no worse than regular scripts invoked as julia foo.jl, and if the concern is actually about unexpected compilation, then this is actually a lot better since these CLIs can be tracked by the global environment and thus will do their re-compilation when you do ]up.

KristofferC · 2024-02-13T12:40:01Z

I'll wait with merging this until after branching for 1.11. Looking at the comments here I think there is more or less a consensus to not do some behind the scene project modification.

…uliaLang#52103) This aims to bring similar functionality to Julia as the `-m` flag for Python which exists to directly run some function in a package and being able to pass arguments to that function. While in Python, `python -m package args` runs the file `<package>.__main__.py`, the equivalent Julia command (`julia -m Package args`) instead runs `<Package>.main(args)`. The package is assumed to be installed in the environment `julia` is run in. An example usage could be: Add the package: ```julia (@v1.11) pkg> add https://github.com/KristofferC/Rot13.jl Cloning git-repo `https://github.com/KristofferC/Rot13.jl` Updating git-repo `https://github.com/KristofferC/Rot13.jl` Resolving package versions... Updating `~/.julia/environments/v1.11/Project.toml` [43ef800a] + Rot13 v0.1.0 `https://github.com/KristofferC/Rot13.jl#master` Updating `~/.julia/environments/v1.11/Manifest.toml` [43ef800a] + Rot13 v0.1.0 `https://github.com/KristofferC/Rot13.jl#master` ``` And then it can be run (since it has a `main` function) via: ``` ❯ ./julia/julia -m Rot13 "encrypt this for me" "and this as well" rapelcg guvf sbe zr naq guvf nf jryy ``` I'm not sure if `-m/--module` is the best choice but perhaps the association to Python makes it worth it.

KristofferC added the domain:packages Package management and loading label Nov 9, 2023

Keno reviewed Nov 14, 2023

View reviewed changes

KristofferC force-pushed the kc/m branch from baa7dd2 to b61118f Compare December 15, 2023 14:14

KristofferC mentioned this pull request Jan 30, 2024

WIP: App support in Pkg JuliaLang/Pkg.jl#3772

Open

vchuravy reviewed Jan 30, 2024

View reviewed changes

vchuravy approved these changes Jan 30, 2024

View reviewed changes

KristofferC added 2 commits January 31, 2024 17:06

small NEWS update

b4ecf71

KristofferC force-pushed the kc/m branch from dc2df03 to b4ecf71 Compare January 31, 2024 16:07

fix rebase error

dd885e9

vtjnash added this to the 1.12 milestone Feb 13, 2024

KristofferC added 2 commits February 20, 2024 13:20

Merge branch 'master' into kc/m

cdc2abe

Merge branch 'master' into kc/m

5ee0d4d

KristofferC added the status:merge me PR is reviewed. Merge when all tests are passing label Feb 28, 2024

IanButterworth merged commit 962bbf7 into master Feb 29, 2024
8 checks passed

IanButterworth deleted the kc/m branch February 29, 2024 18:01

mbauman removed the status:merge me PR is reviewed. Merge when all tests are passing label Feb 29, 2024

	error("`main` in `$arg` not declared as entry point (using `@main`)")
	error("`main` in `$arg` not declared as entry point (use `@main` to do so)")

Introduce -m/--module flag to execute a main function in a package #52103

Introduce -m/--module flag to execute a main function in a package #52103

Conversation

KristofferC commented Nov 9, 2023 • edited Loading

Seelengrab commented Nov 9, 2023 • edited Loading

KristofferC commented Nov 9, 2023

Seelengrab commented Nov 9, 2023 • edited Loading

KristofferC commented Nov 9, 2023

Seelengrab commented Nov 9, 2023

mbauman commented Nov 9, 2023 • edited Loading

Seelengrab commented Nov 9, 2023

Seelengrab commented Nov 9, 2023 • edited Loading

KristofferC commented Nov 9, 2023

KristofferC commented Nov 9, 2023

Seelengrab commented Nov 9, 2023

mbauman commented Nov 9, 2023 • edited Loading

Seelengrab commented Nov 9, 2023 • edited Loading

KristofferC commented Nov 9, 2023 • edited Loading

adienes commented Nov 9, 2023

KristofferC commented Nov 9, 2023 • edited Loading

fredrikekre commented Nov 9, 2023 • edited Loading

mbauman commented Nov 9, 2023 • edited Loading

PallHaraldsson commented Nov 9, 2023

Seelengrab commented Nov 10, 2023

KristofferC commented Nov 10, 2023

KristofferC commented Nov 10, 2023 • edited Loading

Seelengrab commented Nov 10, 2023

Keno Nov 14, 2023

Choose a reason for hiding this comment

KristofferC Nov 14, 2023 • edited Loading

Choose a reason for hiding this comment

lmiq commented Nov 23, 2023 • edited Loading

KristofferC commented Dec 15, 2023

KristofferC commented Dec 15, 2023

Seelengrab commented Dec 15, 2023

KristofferC commented Dec 28, 2023 • edited Loading

KristofferC commented Jan 19, 2024

Seelengrab commented Jan 19, 2024 • edited Loading

KristofferC commented Jan 19, 2024

KristofferC commented Jan 30, 2024

MasonProtter commented Jan 30, 2024

vchuravy Jan 30, 2024

Choose a reason for hiding this comment

jakobnissen commented Jan 31, 2024

Seelengrab commented Jan 31, 2024

Seelengrab commented Feb 1, 2024

adienes commented Feb 1, 2024

Seelengrab commented Feb 1, 2024 • edited Loading

adienes commented Feb 1, 2024

MasonProtter commented Feb 1, 2024 • edited Loading

KristofferC commented Feb 13, 2024

Introduce -m/--module flag to execute a `main` function in a package #52103

Introduce -m/--module flag to execute a `main` function in a package #52103

KristofferC commented Nov 9, 2023 •

edited

Loading

Seelengrab commented Nov 9, 2023 •

edited

Loading

Seelengrab commented Nov 9, 2023 •

edited

Loading

mbauman commented Nov 9, 2023 •

edited

Loading

Seelengrab commented Nov 9, 2023 •

edited

Loading

mbauman commented Nov 9, 2023 •

edited

Loading

Seelengrab commented Nov 9, 2023 •

edited

Loading

KristofferC commented Nov 9, 2023 •

edited

Loading

KristofferC commented Nov 9, 2023 •

edited

Loading

fredrikekre commented Nov 9, 2023 •

edited

Loading

mbauman commented Nov 9, 2023 •

edited

Loading

KristofferC commented Nov 10, 2023 •

edited

Loading

KristofferC Nov 14, 2023 •

edited

Loading

lmiq commented Nov 23, 2023 •

edited

Loading

KristofferC commented Dec 28, 2023 •

edited

Loading

Seelengrab commented Jan 19, 2024 •

edited

Loading

Seelengrab commented Feb 1, 2024 •

edited

Loading

MasonProtter commented Feb 1, 2024 •

edited

Loading