RFC: Julia 1.7 support #496
Conversation
As we don't support the old src-build anymore, there is no need to keep the C++ source code in this repo. `bootstrap.cpp` is moved to a new repo called `libcxxffi`.
|
Thanks for this @Gnimuc , having Cxx back will be awesome! |
`setup_instance` is reworked to use `clang::createInvocationFromCommandLine` for setting compilation arguments. This makes it easier for users to get full control of language features like `RTTI`. System headers can also be configured via common `-isystem` flags.
Great! Feel free to close/"won't fix" #487 (unless you later plan to backport to 1.6) that I opened (and Spot.jl mentioned there, already moved to CxxWrap.jl), and I suppose all other issues on 1.4, 1.5... |
I'm sure it will break again in 1.8, but it should be easier to upgrade. The main problem here is that Cxx.jl uses some Julia internal functions(e.g. There is no intention to resurrect all of the cool features of Cxx.jl in this PR, I just want to see how far it goes and what Julia internal functions we need, so we can open feature requests in the Julia upstream to add new stable APIs for us.
For the reimplementation, I'd like to use the newly introduced incremental compiling feature in LLVM 13, but that will need Julia 1.8. For this upgrade, we don't have the patched source code of LLVM/Clang for Julia 1.6 at this moment(we will never have that if no one would like/has time to contribute). |
|
I like your plan, @Gnimuc - again, thanks for trying to resurrect Cxx! |
|
Now, I can get the following examples to work. Unfortunately, due to some changes in Julia's JIT, it's not easy to resurrect the magic "insert Julia functions into the C++ code" feature(e.g. Example 4 in the README). Any feedback on what features are used in your old Cxx.jl-based projects? # runs on macOS with Julia 1.7.2
julia> using Cxx
julia> cxx""" #include<iostream> """
true
julia> cxx"""
void mycppfunction() {
int z = 0;
int y = 5;
int x = 10;
z = x*y + 2;
std::cout << "The number is " << z << std::endl;
}
"""
true
julia> julia_function() = @cxx mycppfunction()
julia_function (generic function with 1 method)
julia> julia_function()
In file included from /Cxx.h:1:
/Users/gnimuc/.julia/dev/Cxx/src/std.jl:112:38: warning: expression result unused
std::vector<bool> &vr = __juliavar1; vr;
^~
/Users/gnimuc/.julia/dev/Cxx/src/std.jl:43:12: error: implicit instantiation of undefined template 'std::vector<bool>'
__juliavar1.size();
^
/Users/gnimuc/.julia/artifacts/8571aac530057acc3f32ffab9003dc3e00b24723/x86_64-apple-darwin14/sys-root/usr/include/c++/v1/iosfwd:217:28: note: template is declared here
class _LIBCPP_TEMPLATE_VIS vector;
^
In file included from /Cxx.h:1:
/Users/gnimuc/.julia/dev/Cxx/src/std.jl:112:38: warning: expression result unused
std::vector<bool> &vr = __juliavar1; vr;
^~
The number is 52
julia> jnum = 10
10
julia> cxx"""
void printme(int x) {
std::cout << x << std::endl;
}
"""
true
julia> @cxx printme(jnum)
10
julia> cxx"""
void printme(const char *name) {
// const char* => std::string
std::string sname = name;
// print it out
std::cout << sname << std::endl;
}
"""
true
julia> @cxx printme(pointer("John"))
John
julia> cxx"""
class Klassy {
public:
enum Foo { Bar, Baz };
static Foo exec(Foo x) { return x; }
};
"""
true
julia> @cxx Klassy::Bar
Cxx.CxxCore.CppEnum{Symbol("Klassy::Foo"), UInt32}(0x00000000)
julia> @cxx Klassy::exec(@cxx(Klassy::Baz))
Cxx.CxxCore.CppEnum{Symbol("Klassy::Foo"), UInt32}(0x00000001)
julia> cxx"""#include <iostream>
class Hello
{
public:
void hello_world(const char *now){
std::string snow = now;
std::cout << "Hello World! Now is " << snow << std::endl;
}
};"""
true
julia> hello_class = @cxxnew Hello()
(class Hello *) @0x00006000013d5040
julia> using Dates
julia> tstamp = string(Dates.now())
"2022-03-31T01:01:02.900"
julia> @cxx hello_class -> hello_world(pointer(tstamp))
Hello World! Now is 2022-03-31T01:01:02.900 |
|
For those contributors who would like to get involved in the development of this package, this PR is probably a good starting point. To test this PR, you need to 1. build https://github.com/Gnimuc/libcxxffi and |
|
EDIT: NVM just noticed a submodule :/ |
|
We've built & installed julia> using Cxx
[ Info: Precompiling Cxx [a0b5b9ef-44b7-5148-a2d1-f6db19f3c3d2]
ERROR: LoadError: UndefVarError: collectAllHeaders not defined
Stacktrace:
[1] top-level scope
@ ~/.julia/packages/Cxx/cr4BV/src/initialization.jl:232
[2] include(x::String)
@ Cxx.CxxCore ~/.julia/packages/Cxx/cr4BV/src/Cxx.jl:143
[3] top-level scope
@ ~/.julia/packages/Cxx/cr4BV/src/Cxx.jl:173
[4] top-level scope (repeats 2 times)
@ none:1
in expression starting at /home/strazce/.julia/packages/Cxx/cr4BV/src/initialization.jl:232
in expression starting at /home/strazce/.julia/packages/Cxx/cr4BV/src/Cxx.jl:141
ERROR: Failed to precompile Cxx [a0b5b9ef-44b7-5148-a2d1-f6db19f3c3d2] to /home/strazce/.julia/compiled/v1.7/Cxx/jl_w17t2p. |
|
I had a quick test on linux. It managed to compile C++ code, but crashes when the
|
|
Now I am stuck again. It is forbidden to call @Gnimuc have you come across similar issues about constructors? It seems that related functions ( |
|
the function RunGlobalConstructors is implemented in LLVMCGUtils.jl
令和4年10月4日(火) 23:31 melonedo ***@***.***>:
… Now I am stuck again. It is forbidden to call eval inside a generated
function, so we need to find another way to call the constructors.
@Gnimuc <https://github.com/Gnimuc> have you come across similar issues
about constructors? It seems that related functions (RunGlobalConstructors)
are commented out.
—
Reply to this email directly, view it on GitHub
<#496 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABYXW5MFDQ5ZKQUCA6JA3SDWBQ5S7ANCNFSM5OVFOKUQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
I am sorry but I can not find |
Sorry, I was using a phone last night and didn't check the code. What I meant is that the
Indeed. To call |
I guess it is because its signature changed. It accepts an Now |
This is exactly where I got stuck. |
|
I guess there's no need to target 1.7 by now. Would it be easier to target just current stable Julia and its LLVM or possibly only Julia 1.11/1.12 and latest LLVM 18.1.7? I see "memory management" under broken tests. Does the new Julia Memory type help to get Cxx.jl working again, or the opposite since written in pure Julia, and before in C? Is it realistic this project will work again, and not break or would that always be a possibility, with e.g. changes in LLVM? And with full platform support, or even only adding for Windows, unclear to me how bad it was or its issues. |
|
Cxx.jl is mostly a clever extension of Julia's JIT compiler infrastructure into c++. Specifically, Julia 1.3 can track external LLVM modules which correspond to c++ source files enabling a c++ JIT compiler. However, interface to that facility has been removed in 1.4 so a c++ JIT compiler has to be made from scratch, which is the biggest obstacle.
…---Original---
From: "Páll ***@***.***>
Date: Sat, Jun 22, 2024 02:05 AM
To: ***@***.***>;
Cc: ***@***.******@***.***>;
Subject: Re: [JuliaInterop/Cxx.jl] RFC: Julia 1.7 support (PR #496)
I guess there's no need to target 1.7 by now. Would it be easier to target just current stable Julia or its LLVM or possibly only Julia 1.11/1.12 and latest LLVM 18.1.7? I see "memory management" under broken tests. Does the new Julia Memory type help to get Cxx.jl working again, or the opposite since written in pure Julia, and before in C? Is it realistic this project will work again, and not break or would that always be a possibility, with e.g. changes in LLVM? And with full platform support, or even only adding for Windows, unclear to me how bad it was or its issues.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
|
for an alternative solution, see https://github.com/Gnimuc/CppInterOp.jl |
|
@Gnimuc how does CppInterOp.jl work under the hood, compared to Cxx.jl? |
CppInterOp is a C++ interpreter, allowing us to create packages similar to PyCall and RCall. OTOH, Cxx.jl operates at the LLVM IR level. Clang is used to generate LLVM IR, which is then tweaked and sent into Julia's pipeline for execution. The Julia internals are not stable, that's why we stuck at 1.3. A workaround is to use GPUCompiler to build our own pipeline. Those GPUCompiler-based packages are very good examples.(e.g. Enzyme.jl) |
Does it use parts of Clang for that, or is it a standalone interpreter? |
It originates from cling maintained by cling devs and hasn't been fully upstreaming to LLVM yet, see https://compiler-research.org/libinterop/ for the original proposal, which was published 2 years ago. |
As reworking Cxx.jl from the scratch takes more time than I thought, I'm going to try it again to upgrade the old code.
Thanks to https://github.com/JuliaLang/llvm-project, doing a source build is much easier than before.
libcxxffito LLVM 12cleanup_cpp_envsegfaultsclang_constants.jlJLLEnvsmodule for shipping system headers)llvmcallto Julia 1.7inlineexpr, 103