add --trim option for generating smaller binaries

[JeffBezanson]

This adds a command line option --trim that builds images where code is only included if it is statically reachable from methods marked using the new function entrypoint. Compile-time errors are given for call sites that are too dynamic to allow trimming the call graph (however there is an unsafe option if you want to try building anyway to see what happens).

The PR has two other components. One is changes to Base that generally allow more code to be compiled in this mode. These changes will either be merged in separate PRs or moved to a separate part of the workflow (where we will build a custom system image for this purpose). The branch is set up this way to make it easy to check out and try the functionality.

The other component is everything in the juliac/ directory, which implements a compiler driver script based on this new option, along with some examples and tests. This will eventually become a package "app" that depends on PackageCompiler and provides a CLI for all of this stuff, so it will not be merged here. To try an example:

julia contrib/juliac.jl --output-exe hello --trim test/trimming/hello.jl

When stripped the resulting executable is currently about 900kb on my machine.

Also includes a lot of work by @topolarity

[ViralBShah]

➜  juliac git:(jb/gb/static-call-graph) ✗ ../julia juliac.jl --output-exe hello --trim exe_examples/hello_world.jl
ld: warning: ignoring duplicate libraries: '-ljulia'
ld: warning: reexported library with install name '@rpath/libunwind.1.dylib' found at '/Users/viral/julia/usr/lib/libunwind.1.0.dylib' couldn't be matched with any parent library and will be linked directly
run(`cc $(allflags) -o $outname -Wl,$(Base.Linking.WHOLE_ARCHIVE) $img_path -Wl,$(Base.Linking.NO_WHOLE_ARCHIVE) $init_path $(julia_libs)`) = Process(`cc -std=gnu11 -I/Users/viral/julia/usr/include/julia -fPIC -L/Users/viral/julia/usr/lib -L/Users/viral/julia/usr/lib/julia -Wl,-rpath,/Users/viral/julia/usr/lib -ljulia -o hello -Wl,-all_load /var/folders/qj/54cx7lcs2qv9z1b4f_110xb00000gn/T/jl_fPbFYU/img.a -Wl, /var/folders/qj/54cx7lcs2qv9z1b4f_110xb00000gn/T/jl_fPbFYU/init.a -ljulia -ljulia-internal`, ProcessExited(0))

1M binary on Mac M-series:

➜  juliac git:(jb/gb/static-call-graph) ✗ ls -l hello
-rwxr-xr-x  1 viral  staff  1082296 Jul  6 09:57 hello
➜  juliac git:(jb/gb/static-call-graph) ✗ ./hello
Hello, world!
➜  juliac git:(jb/gb/static-call-graph) ✗ otool -L hello
hello:
	@rpath/libjulia.1.12.dylib (compatibility version 1.12.0, current version 1.12.0)
	@rpath/libjulia-internal.dylib (compatibility version 1.12.0, current version 1.12.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1345.100.2)

[timholy]

#55104 gets the lib example working

[MasonProtter]

Let me know if issues should be raised elsewhere, but I ran into a problem where the compiled binaries don't seem to get ARGS. Here's a MWE:

# args.jl
module ArgsDemo

Base.@ccallable function main()::Cint
    ccall(:jl_, Cvoid, (Any,), ARGS)
    return 0
end

end

❯ ~/julia/./julia --startup=no ~/julia/juliac/juliac.jl --output-exe args --trim args.jl
run(`cc $(allflags) -o $outname -Wl,$(Base.Linking.WHOLE_ARCHIVE) $img_path -Wl,$(Base.Linking.NO_WHOLE_ARCHIVE) $init_path $(julia_libs)`) = Process(`cc -std=gnu11 -I/home/masonp/julia/usr/include/julia -fPIC -L/home/masonp/julia/usr/lib -Wl,--export-dynamic -L/home/masonp/julia/usr/lib/julia -Wl,-rpath,/home/masonp/julia/usr/lib -Wl,-rpath,/home/masonp/julia/usr/lib/julia -ljulia -o args -Wl,--whole-archive /tmp/jl_2vsSFl/img.a -Wl,--no-whole-archive /tmp/jl_2vsSFl/init.a -ljulia -ljulia-internal`, ProcessExited(0))

❯ ./args
Array{String, 1}(dims=(0,), mem=Memory{String}(8, 0x5589c120d310)[#<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>])

❯ ./args arg1 arg2 agr3
Array{String, 1}(dims=(0,), mem=Memory{String}(8, 0x558aac9f6310)[#<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>])

[JeffBezanson]

That should be fixed by integration with PackageCompiler. It knows how to do all that stuff.

[JeffBezanson]

OK, I deleted everything. Now this is barely 1000 lines. Don't worry it's all on a backup branch jb/gb/static-call-graph-backup. I converted the juliac machinery into a test case so we can focus here on just merging the core compiler functionality.

Now there are only a tiny number of controversial/weird changes:

• unoptimize_throw_blocks: Can possibly be changed upstream? I believe we want to do that anyway. Otherwise the test case might need to build its own sysimage. (done by inference: remove throw block deoptimization completely #49260)
• Linalg init changes: these are weird error cases and I think can just be changed not to use logging? Or again we build a custom sysimage for testing.
• Setting max_args of printing functions: weird but harmless; can be upstreamed?

[JeffBezanson]

I think this is mergeable now so we can get the --trim option in. The only difficulties are basically in the tests and docs, which can be updated any time.

[PallHaraldsson]

Please merge this for 1.12. I'm very exited to try this (more) easily out, and for regular users to try, so would it be possible to add this to 1.11.0 or at least some 1.11.x point release (marked as an experimental new feature)? It seems this is largely not affecting regular Julia execution, lives in contrib, but I do see though some changes in codegen and staticdata.c, so I'm not sure how invasive this is, and I understand if this is too late considering RC4...

[MasonProtter]

The v1.11 feature freeze was over half a year ago.