fpclibcmm.pas and signed symbol table.

[fredvs]

Hello.

Not assigning proper signed symbol table for a method from libc.so.6 could give big problems.
See the (hot) topic here:
https://forum.lazarus.freepascal.org/index.php/topic,58888.msg486269.html#msg486269

[synopse]

Thanks for the feedback.

IIRC it is not only about the symbol name itself. It is also about the whole library name.
I use ldd to check the libc versions compatible with a particular executable compiled by FPC.

I usually resolve this potential issue by tuning the actual libc used at compilation time.
What I found the most practical is to cross-compile from Windows to Linux, with some fixed - and old - libc .so file. This is safer than using the libc available on the system, which may be "too" new to run on old systems. Or you can compile on an old system, and it likely to work on newer. ;)

So this is not specific to mORMot, but to the whole FPC tool chain.
Since some of the libc names are shared with FPC RTL, I would rather stick to what the FPC RTL does - even if it is not optimal.

[synopse]

For instance, when compiled on a new Debian stable:

ab@dev-ab:~/dev/lib2/test/fpc/bin/x86_64-linux$ ldd -v mormot2tests
	linux-vdso.so.1 (0x00007ffdbcadb000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f75441d6000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7543ff5000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f7544213000)

	Version information:
	./mormot2tests:
		libc.so.6 (GLIBC_2.33) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.10) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.3) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.17) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.3.2) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.34) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
	/lib/x86_64-linux-gnu/libz.so.1:
		libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
	/lib/x86_64-linux-gnu/libc.so.6:
		ld-linux-x86-64.so.2 (GLIBC_2.2.5) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2

[fredvs]

Hello.

In case you dont want to risk future problem if a new table is assigned for the methods you use, just assign the GLIBC_2.2.5 for x86-64 and GLIBC_2.0 for i386, like this:

{$if defined(linux) and defined(cpux86_64)}  
function malloc(size: PtrUInt): pointer;
  cdecl; external 'c' name 'malloc@GLIBC_2.2.5';

// add all other libc.so.6 methods used

{$else}
{$if defined(linux) and defined(cpui386)}
function malloc(size: PtrUInt): pointer;
  cdecl; external 'c' name 'malloc@GLIBC_2.0';

// add all other libc.so.6 methods used

{$else}
function malloc(size: PtrUInt): pointer;
  cdecl; external 'c' name 'malloc';

// add all other libc.so.6 methods used

{$endif}
{$endif}

[EDIT] Changed defined(x86_64) with defined(cpux86_64) and defined(i386) with defined(cpui386) .

[synopse]

Are you sure it is enough to work, especiallly if e.g. malloc() is defined as plain 'malloc' in another part of the project ?

[fredvs]

Not sure to understand, malloc() is unique but libc.so.6, depending of the version, can deal with many symbol tables for the malloc() method. Each symbol signed table works like a filter to warn the method that it should use the particularities of the assigned table.

But maybe I did not catch the question.

[fredvs]

FPC - libc.pp was developed for x86_64, with the table GLIBC_2.2.5 with his particularities, like the stat record (but sadly FPC did not assign it for each libc.so.6 methods and at linking, because no signature was assigned, the linker assign the last table of the system, that can have other particularities than 2.2.5. and create problems.

For Linux i386, FPC used GLIBC_2.0.

[fredvs]

Are you sure it is enough to work, especiallly if e.g. malloc() is defined as plain 'malloc' in another part of the project ?

Yes, I am totally sure, with lot of deep tests, that assigning the right table works.

Note too that only Linux on x86_64 and i386 give problems.
Till now arm32 and aarch64 Linux do only use the "Base" table, so no problems at the moment for those.

[fredvs]

To check what table was assigned for each method in your binary, just do:

objdump -T mormot2tests | grep "("

If there are some methods with > @GLIBC_2.2.5, like @GLIBC_2.34, it could have problems.

[EDIT] In code of previous post, please note the change defined(x86_64) with defined(cpux86_64) and defined(i386) with defined(cpui386) .

[synopse]

My concern is that even if I change all names to include the suffix, I still get the following error when compiling on a recent Linux and executing on an older Linux:

synopse@sd-73991:~/temp$ ./mormot2tests 
./mormot2tests: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./mormot2tests)
./mormot2tests: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./mormot2tests)

Do you know by chance how to specify the GLIBC needed when linking?

[synopse]

here is the full ldd -v output:

synopse@sd-73991:~/temp$ ldd -v mormot2tests 
./mormot2tests: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./mormot2tests)
./mormot2tests: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./mormot2tests)
	linux-vdso.so.1 =>  (0x00007ffeee180000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fef3087d000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fef304b3000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fef30a97000)

	Version information:
	./mormot2tests:
		libc.so.6 (GLIBC_2.33) => not found
		libc.so.6 (GLIBC_2.10) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.3) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.17) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.3.2) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.34) => not found
		libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
	/lib/x86_64-linux-gnu/libz.so.1:
		libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
		libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
	/lib/x86_64-linux-gnu/libc.so.6:
		ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
		ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2

[synopse]

It sounds like the FPC RTL is the culprit. The GLIBC_2.33 is required for pthread calls, and also at lower level for __libc_start_main resolution.
Since it is hardcoded in the RTL system unit, I don't know how to circumvent this limitation. We need an official RTL fix.

[synopse]

Even the libc did change in its inialization section: they introduced a new __libc_start_main@@GLIBC_2.34 symbol on latest revisions.
https://sourceware.org/git/?p=glibc.git;a=commit;h=035c012e32c11e84d64905efaf55e74f704d3668

[robert-rozee]

synopse: if one does not already exist, try creating a symlink libdl.so that points to libdl.so.2 then compile again:

sudo ln -s /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libdl.so

fredvs told me about this a few weeks back, and i remember (i think) it solved a similar problem.

cheers,
rob :-)

[synopse]

@robert-rozee
sadly, the sudo ln trick did not change the symlink:

0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) strrchr
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.34) dladdr

and the main problem would remain anyway:

0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.34) __libc_start_main

I don't see anything else than (cross-)compile from an old system

[fredvs]

Easy trick if you dont want to wait centuries that fpc fix it.

Use fpc-ootb : https://github.com/fredvs/freepascal-ootb/releases/
For Linux 64 bit: https://github.com/fredvs/freepascal-ootb/releases/download/3.2.2/fpc-ootb-322-x86_64-linux_glibc225.zip

You may test the "timeless" version of LazPaint compiled wit fpc-ootb:
bgrabitmap/lazpaint#531 (comment)

[synopse]

When I look at https://forum.lazarus.freepascal.org/index.php?topic=58888.msg486611#msg486611

I am not confident your changes would ever be included in the trunk.
Even if they sound just safe & fine to me.

[fredvs]

When I look at https://forum.lazarus.freepascal.org/index.php?topic=58888.msg486611#msg486611

Always the same strategy of Marcov, trying to impress with lots of complicated notions that have absolutely nothing to do.

And of course he knows better how to use libc.so.6 than main-dev-libc Forian Weimer.

I never doubted for a second that Marcov would accept these changes.

I think Robert Rozee and I did everything we could to show the Big-Bug.

Now if they don't want to fix it, too bad, I'm the maintainer of the MSEgui project and users are entitled to have a working binary, and the whole thing of compiling on an old distro is pure tinkering.

[synopse]

It is sad, but it makes sense.

Perhaps we could work with @LongDirtyAnimAlf to patch the RTL source directly in fpcupdeluxe?

Do you have any documentation about how to use your stand-alone FPC compiler?

[fredvs]

Perhaps we could work with @LongDirtyAnimAlf to patch the RTL source directly in fpcupdeluxe?

With pleasure but @LongDirtyAnimAlf said that he wanted collaboration of fpc-dev:
I am willing to help, if versioned linking might/will be accepted by FPC devs.

Do you have any documentation about how to use your stand-alone FPC compiler?

1. Download the fpc-ootb.zip file corresponding to your OS.
   See assets in https://github.com/fredvs/freepascal-ootb/releases/

2. Unzip it on a directory of your choice.

3. Run the binary fpc-ootb-64 or fpc-ootb-32, like you do for fpc, nothing needs to be configured, all is already done.

Example:

/home/you/fpc-ootb-322-x86_64-linux_glibc225/fpc-ootb-64 myapp.pas

Note that I will not do publicity for fpc-ootb, it is there only for convenience and, of course, if fpc commit the change, I will be happy to delete that fpc-ootb project.

[synopse]

I tried the OOTB compiler by putting
FPC=/home/ab/fpcup/fpc-ootb/fpc-ootb-64
at the beginning of https://github.com/synopse/mORMot2/blob/master/test/build_fpc.sh
But it fails with message complaining about missing fpcres compiller. :(

My guess is that @LongDirtyAnimAlf knows that any such patch is likely to be never included in the FPC RTL.
I suspect he would probably add a suitable patch. Perhaps enabled by default, with a checkbox to disable it if needed.

[fredvs]

But it fails with message complaining about missing fpcres compiller. :(

Ooops, I did it too fast yesterday, I forgot to add the /tools directory.
Thanks to note it, I will fix this tonight.

[fredvs]

Hello.

I try to run the build_fpc.sh with "original" fpc 3.2.2. and get that error:

> /home/fred/mORMot2-master/test/build_fpc.sh
compiling for x86_64-linux as -O3 -dFPC_NO_DEFAULT_MEMORYMANAGER -dFPC_X64MM -dFPCMM_SERVER -dFPCMM_REPORTMEMORYLEAKS -CX -XX
/usr/bin/ld: cannot find ../../static/x86_64-linux/sha512-x64sse4.o: No such file or directory
/usr/bin/ld: cannot find ../../static/x86_64-linux/crc32c64.o: No such file or directory
/usr/bin/ld: cannot find ../../static/x86_64-linux/quickjs.o: No such file or directory
/usr/bin/ld: cannot find ../../static/x86_64-linux/sqlite3.o: No such file or directory
/usr/bin/ld: cannot find ../../static/x86_64-linux/libdeflatepas.a: No such file or directory
/usr/bin/ld: cannot find ../../static/x86_64-linux/liblizard.a: No such file or directory
Error in /home/fred/mORMot2-master/test/build_fpc.sh on line 105
******Build for x86_64-linux fail******

With ootb and with fpres in /tools, I get the same error.

Sure I miss something.

[synopse]

you need to download the ../../static sub-folder content as detailed by our doc.

[fredvs]

Ha, ok, I will jump into it tonight.
In case you want to try again fpc-ootb before commit in new release, just create a /tools directory in directory of fpc-ootb and add fpres from official fpc.

[fredvs]

OK, I get it and it compiles with fpc-ootb + /tools/fpres:

fred@fred-IdeaPad-5-14IAL7 ~/m/test> /home/fred/mORMot2-master/test/build_fpc.shcompiling for x86_64-linux as -O3 -dFPC_NO_DEFAULT_MEMORYMANAGER -dFPC_X64MM -dFPCMM_SERVER -dFPCMM_REPORTMEMORYLEAKS -CX -XX
 Build for x86_64-linux success. Tests can be executed from
 /home/fred/mORMot2-master/test/bin/fpc-x86_64-linux/mormot2tests

But I am disappointed...

objdump -T mormot2tests | grep "("

There are same methods with 2 different tables assigned, like:

0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) dlopen
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dlopen

0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dlclose
0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) dlclose

0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) dlerror
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dlerror

With official fpc there are only methods with (GLIBC_2.34) and with fpc-ootb, all apps tested have only (GLIBC_2.2.5), maybe it is good to have both (but I would prefer only 2.2.5).

But there are good news only one table for __libstart_main and it is 2.2.5. ;-)
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) __libc_start_main

And, like you noted, all the *.inc of cthreads.pp need to be patched too, this is for tonight.

So remain a shadow, what if 2 tables are defined for a unique method, does libc.so.6 give the choice between 2 tables and use the newer if present?

It should be nice but not ok for fpc, better only one table (or more code).

What a fight!

[fredvs]

Do you know if mormot uses netwlibc ?
/fpc/rtl/netwlibc/libc.pp

Because there are defined dlopen, dlclose, dlerror,... too (without signature).

There is also /fpc/packages/sqlite/src/sqlite3.inc where dlopen, dlclose, dlerror,... are declared too.

If yes, that could explain why 2 tables are assigned.

And those units/inc should be patched too.

Mama mia.

[fredvs]

Some news from the front...

Ok, ootb has all pthreads and friends methods with + LIBC_SUFFIX.

The compiled release for Linux amd64 with /tools is here :
https://github.com/fredvs/freepascal-ootb/releases/download/3.2.2/fpc-ootb-322-x86_64-linux_glibc225.zip

And the compiled release for Linux i386 with /tools is there :
https://github.com/fredvs/freepascal-ootb/releases/download/3.2.2/fpc-ootb-322-i386-linux_glibc20.zip

I did try the mormot build_fpc.sh and it compiles ok.

But doing
objdump -T mormot2tests | grep "2.34"

I still get those:

0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) pthread_join
0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) dlerror
0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) pthread_mutex_trylock
0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) pthread_mutexattr_settype
0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) dlopen
0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) pthread_mutexattr_init
0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) dlclose
0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) pthread_create
0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) dlsym
0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) pthread_mutexattr_destroy

And with this some double table;
objdump -T mormot2tests | grep "dl"

0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) dlerror
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dlclose
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dladdr
0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) dlopen
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dlerror
0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) dlclose
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dlopen
0000000000000000 DF UND 0000000000000000 (GLIBC_2.34) dlsym
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dlsym

I did try in /mORMot2-master/src/core/mormot.core.os.posix.inc adding + LIBC_SUFFIX for each method, appart for sched_getaffinity who needs @GLIBC_2.3.4.

But it did not help, the 2.34 tables are still there.

I dont see why at the moment.

It is very strange because with LazPaint compiled with ootb there is this:

objdump -T lazpaint | grep "("

0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) wcrtomb
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) wcscoll
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) setenv
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dlclose
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) setlocale
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) strcoll
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dladdr
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) iconv_close
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) mbrlen
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dlerror
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) mbrtowc
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dlopen
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) iconv
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) __libc_start_main
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) iconv_open
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sysconf
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) nl_langinfo
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) towlower
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dlsym
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) __errno_location
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) towupper
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sched_yield

And with all my msegui apps, even some using db there is this:

objdump -T ideU | grep "("

0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) realloc
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) getpt
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) ioctl
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_cond_init
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) localtime_r
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_setcanceltype
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_mutex_destroy
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) wcscoll
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) cfsetospeed
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) unlink
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) mkdir
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sscanf
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_mutex_init
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) vfork
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) rmdir
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) setsid
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_setcancelstate
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) tcsetattr
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) setenv
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sem_destroy
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dlclose
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) opendir
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) tcgetattr
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sigfillset
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) ptsname_r
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) grantpt
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_self
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sigaction
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) setlocale
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) strcoll
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pipe
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sigemptyset
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dladdr
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) fsync
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) iconv_close
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) isatty
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) execv
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) __libc_current_sigrtmin
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) waitpid
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sigaddset
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dlerror
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_sigmask
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sem_trywait
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) usleep
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_mutexattr_settype
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) __xstat
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dup
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_cond_wait
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_equal
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dlopen
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) free
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) fcntl
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_mutexattr_init
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) setpgid
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) close
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) iconv
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sem_wait
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_mutex_unlock
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) poll
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_mutex_lock
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) signal
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) getcwd
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) unsetenv
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) open
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) __libc_start_main
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) iconv_open
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) gettimeofday
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) cfsetispeed
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) timelocal
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dup2
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) closedir
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sem_post
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sem_getvalue
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) ftruncate64
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_cond_broadcast
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sigismember
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) nl_langinfo
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) time
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sem_timedwait
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) getenv
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) read
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) readdir
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) unlockpt
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) tcdrain
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) setitimer
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_cond_timedwait
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) towlower
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) calloc
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) kill
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_mutexattr_destroy
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) dlsym
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sem_init
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) malloc
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) __libc_current_sigrtmax
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) _exit
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) write
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_cond_destroy
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) execve
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_mutex_trylock
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) __errno_location
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) pthread_kill
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) rename
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) getpid
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) towupper
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) chdir
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) sched_yield
0000000000000000 DF UND 0000000000000000 (GLIBC_2.2.5) strerror_r

I dont see at the moment why there is still 2.34 with mormot.

I have to pause the combat, I need to sleep now.

Fre;D

[robert-rozee]

use grep to search for occurrences of dlopen in the source tree of both ootb and the project you are compiling. there may be a function called dlopen with an external that has NO name parameter, or an external that specifies an UNVERSIONED dlopen symbol.

fred: i am thinking that we had not considered externals to dlopen in user code or libraries. perhaps the compiler should halt with an error if it sees multiple versions of the same symbol?

cheers,
rob :-)

[synopse]

I have made the modifications.
Note that you need to re-download mormot2static.7z because the sqllite3.o has been rebuilt with _ prefixes to the libc symbol names.

Now I don't have any issue reported, BUT there is an GPF when executing on an old Linux distribution:

synopse@sd-73991:~/temp$ ./mormot2tests 
An unhandled exception occurred at $00007FA83736B3AB:
EAccessViolation: Access violation
  $00007FA83736B3AB

:(

@fredvs
The problem remains about FPC 3.2.2 bug.
IIRC FPC 3.2.4 is about to be published, so maybe FPC 3.2 fixes branch could be used instead.

[LongDirtyAnimAlf]

My guess is that it could be a great addition to fpcupdeluxe to include the fredvs patches of the RTL by default

These patches are huge. Every libc call needs to be changed. It will be nearly impossible to track FPC-compiler and -rtl changes to be sure that this patch will succeed.
So, I am now thinking about a search and replace tactic instead, and using the GLIBC-version include file.

[fredvs]

Hello everybody.
Nice news today!

@fredvs
The only remaining concern seems to be that we use FPC 3.2.2 which is just broken about variant support, so mORMot tests do fail when compiled with this compiler.

Hum, by chance, do you know what commit in FPC 3.3.1 did fix the problem about variant?

Thanks.

Fre;D

[synopse]

My link above in the mORMot/Synopse forum redirected to:
https://gitlab.com/freepascal.org/fpc/source/-/issues/39438
https://gitlab.com/freepascal.org/fpc/source/-/commit/8b21bf1cce2fcf5e62c10a08f0ddd25dfdb6b33b

I still don't understand what is wrong with the patched sqlilte3.o redirection, which triggers a GPF on old distributions.

[fredvs]

I still don't understand what is wrong with the patched sqlilte3.o redirection, which triggers a GPF on old distributions.

I will jump into it asap but what old distribution are you using, with what last symbol table?
Is it possible for you to isolate the problem to know what method-table is the GPF guilty?

My link above in the mORMot/Synopse forum redirected to:

Ha, ok, perfect, I will deeply study it and apply the fix in fpc-ootb if possible,

Thanks.

[fredvs]

@fredvs
The only remaining concern seems to be that we use FPC 3.2.2 which is just broken about variant support, so mORMot tests do fail when compiled with this compiler.

My link above in the mORMot/Synopse forum redirected to:

Yep, many thanks for the links, patch done for fpc-ootb.

./mormot2tests

   mORMot2 Regression Tests
  --------------------------
1. Core units

...

Time elapsed for all tests: 33.65s
Performed 2023-07-27 14:40:47 by fred on fred-IdeaPad-5-14IAL7

Total assertions failed for all test suits:  0 / 77,867,706
! All tests passed successfully.
 
 Flags: SERVER  assumulthrd erms debug repmemleak
 Small:  9K/1MB  including tiny<=128B arenas=8 fed from Medium
 Medium: 15MB/111MB    peak=33MB current=12 alloc=89 free=77 sleep=0
 Large:  0B/3GB    peak=22MB current=0 alloc=1K free=1K sleep=0
 Small Blocks since beginning: 56M/5GB (as small=45/46 tiny=56/56)
  48=20M  64=9M  16=5M  80=5M  96=2M  112=2M  528=2M  32=1M
  224=1M  144=818K  160=711K  128=655K  192=607K  320=200K  256=187K  288=178K
 Small Blocks current: 9K/1MB
  48=4K  64=1K  112=544  352=522  32=506  128=408  96=305  80=248
  288=130  416=129  672=96  624=94  576=67  480=65  384=52  192=50

That's the way I like it!

You may try with the last re-compiled release:
Linux x86_64:
https://github.com/fredvs/freepascal-ootb/releases/download/3.2.2/fpc-ootb-322-x86_64-linux_glibc225.zip

Linux i386:
https://github.com/fredvs/freepascal-ootb/releases/download/3.2.2/fpc-ootb-322-i386-linux_glibc20.zip

[synopse]

@LongDirtyAnimAlf
The patch does not seem very long to me.
And the involved parts of the RTL are unlikely to change. They did not change much since a decade.
My guess is that a regular patch text should be good enough.

Honestly, I don't see the relevance of using a .inc file, because we would need several files. We could just use a local constant. And some of the API entries do not involve the same suffix.
Perhaps the patches could involved only x86_64-linux at first, because i386-linux is more a niche target nowadays, and it won't be the main platform of the developer or build machine. So it is less likely to have such linking issues.

@fredvs
Thanks a lot for including the patch.
Now we can run all regression tests!

About the GPF on my old Ubuntu 16.04 machine, the culprit seems to be pthread_mutexattr_init as redirected from sqlite3.o.
I changed the redirection from a static resolution to dlsym() values... and it seems to work well.
Now I can run all tests on a Linux 4.4 with Ubuntu 16.04:

Software version tested: 2.1.5692 (2023-07-27 15:26:53)

Ubuntu 16.04.7 LTS - Linux 4.4.0-141-generic [utf8 7.7GB 4040010]
    8 x Intel(R) Atom(TM) CPU C2750 @ 2.40GHz 1MB cache (x64)
    on Supermicro SYS-5038MA-H24TRFG-ON002 0123456789
Using mORMot 2.1.5692 x64MMs and OpenSSL 101010BF
    TSqlite3LibraryStatic 3.41.0 with internal MM
Generated with: Free Pascal 3.2.2 64 bit Linux compiler

Time elapsed for all tests: 2m32
Performed 2023-07-27 15:29:51 by synopse

Total assertions failed for all test suits:  0 / 77,819,721
! All tests passed successfully.

I also tried on a good old CentOS Linux 7 (Core) - Linux 3.10.0 and there was not problem for the execution (apart the deprecated OpenSSL 1.0 version included in the system).

[fredvs]

Now I can run all tests on a Linux 4.4 with Ubuntu 16.04:

Yeeep and now you may produce timeless mORMot applications!

Many thanks for your attention, it is a pleasure to explore that jungle with you all.

[fredvs]

apart the deprecated OpenSSL 1.0 version included in the system

Hum, maybe trying with ootb that allows to link libraries with extended so number?

Ok, ok, I leave, I promised to not do propaganda.

[synopse]

About OpenSSL, we just can distribute the OpenSSL 1.1 or 3.x libraries with the executable, and mORMot does late-binding to it anyway.
This is what we do with our LUTI integration.

[fredvs]

About OpenSSL

Hum, now that I do understand better the magic of signed symbol table, it seems to me that it would not be a bad idea if OpenSSL devs had opted for such magic in OpenSSL.

But ok, your LUTI integration does the magic.

[synopse]

@LongDirtyAnimAlf
Note that the packages/libc folder actually needs no suffix fix because it is a deprecated kylix-compatible unit for i386 only.

@fredvs
If you want to propose a patch to the FPC team, perhaps leave the packages/libc folder untouched.
It would help the acceptance, I guess. :)

Also note that rtl\netwlibc should be left untouched because it is for the Netware OS, and never compiled for Linux.

[fredvs]

@fredvs
If you want to propose a patch to the FPC team, perhaps leave the packages/libc folder untouched.
It would help the acceptance, I guess. :)

If I propose a patch, Marcov will directly refuse it.
All the code of fpc-ootb is in Github, each commit is documented and for the patch, just do:

> cd fpc-ootb-main
> git diff bbe0bb85391873789ad846dc702b2377f6ad620f HEAD > ootb.patch

It will do a patch from official fpc 3.2.2 to last fpc-ootb commit.

Also note that rtl\netwlibc should be left untouched because it is for the Netware OS, and never compiled for Linux.

Ha, ok, thanks for this, I will restore the original asap.

Fre;D

[LongDirtyAnimAlf]

I need some advice on the following calls towards libc. The listed function need another GLIBC version, as stated in the list.
All others libc calls for Unix/Linux are covered by GLIBC 2.2.5
How would we proceed to handle this ?

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function epoll_create(size: cint): cint; {$ifdef FPC_USE_LIBC} cdecl; external name 'epoll_create'; {$endif}
GLIBC_2.3.2

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function epoll_create1(flags: cint): cint; {$ifdef FPC_USE_LIBC} cdecl; external name 'epoll_create1'; {$endif}
GLIBC_2.9

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function epoll_ctl(epfd, op, fd: cint; event: pepoll_event): cint; {$ifdef FPC_USE_LIBC} cdecl; external name 'epoll_ctl'; {$endif}
GLIBC_2.3.2

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function epoll_wait(epfd: cint; events: pepoll_event; maxevents, timeout: cint): cint; {$ifdef FPC_USE_LIBC} cdecl; external name 'epoll_wait'; {$endif}
GLIBC_2.3.2

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function epoll_pwait(epfd: cint; events: pepoll_event; maxevents, timeout: cint; sigmask: PSigSet): cint; {$ifdef FPC_USE_LIBC} cdecl; external name 'epoll_pwait'; {$endif}
GLIBC_2.6

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function vmsplice (fdout: cInt; iov: PIOVec; count: size_t; flags: cuInt): cInt; {$ifdef FPC_USE_LIBC} cdecl; external name 'vmsplice'; {$ENDIF}
GLIBC_2.5

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
                             offout: off64_t; len: size_t; flags: cuInt): cInt; {$ifdef FPC_USE_LIBC} cdecl; external name 'splice'; {$ENDIF}
GLIBC_2.5

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function tee(fd_in: cInt; fd_out: cInt; len: size_t; flags: cuInt): cInt; {$ifdef FPC_USE_LIBC} cdecl; external name 'tee'; {$ENDIF}
GLIBC_2.5

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function sync_file_range(fd: cInt; offset, nbytes: off64_t; flags: cuInt): cInt; {$ifdef FPC_USE_LIBC} cdecl; external name 'sync_file_range'; {$ENDIF}
GLIBC_2.6

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function fdatasync (fd: cint): cint; {$ifdef FPC_USE_LIBC} cdecl; external name 'sync_file_range'; {$ENDIF}
GLIBC_2.6

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function inotify_init: cint;  {$ifdef FPC_USE_LIBC} cdecl; external name 'inotify_init'; {$ENDIF}
GLIBC_2.4

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function inotify_init1(flags:cint):cint;  {$ifdef FPC_USE_LIBC} cdecl; external name 'inotify_init1'; {$ENDIF}
GLIBC_2.9

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function inotify_add_watch(fd:cint; name:Pchar; mask:cuint32):cint;  {$ifdef FPC_USE_LIBC} cdecl; external name 'inotify_add_watch'; {$ENDIF}
GLIBC_2.4

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function inotify_rm_watch(fd:cint; wd: cint):cint;  {$ifdef FPC_USE_LIBC} cdecl; external name 'inotify_rm_watch'; {$ENDIF}
GLIBC_2.4

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function clock_getres(clk_id : clockid_t; res : ptimespec) : cint; {$ifdef FPC_USE_LIBC} cdecl; external name 'clock_getres'; {$ENDIF}
GLIBC_2.17

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function clock_gettime(clk_id : clockid_t; tp: ptimespec) : cint;  {$ifdef FPC_USE_LIBC} cdecl; external name 'clock_gettime'; {$ENDIF}
GLIBC_2.17

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
function clock_settime(clk_id : clockid_t; tp : ptimespec) : cint; {$ifdef FPC_USE_LIBC} cdecl; external name 'clock_settime'; {$ENDIF}
GLIBC_2.17

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
Function utimensat(dfd: cint; path:pchar;const times:tkernel_timespecs;flags:cint):cint; {$ifdef FPC_USE_LIBC} cdecl; external name 'utimensat'; {$ENDIF}
GLIBC_2.6

C:\fpcupsystems\trunk\fpcsrc\rtl\linux\linux.pp
Function futimens(fd: cint; const times:tkernel_timespecs):cint; {$ifdef FPC_USE_LIBC} cdecl; external name 'futimens'; {$ENDIF}
GLIBC_2.6


C:\fpcupsystems\trunk\fpcsrc\rtl\netwlibc\libc.pp
function memcpy(__restrict, __restrict1:pointer; _para3:size_t):pointer;cdecl;external libc_nlm name 'memcpy';
GLIBC_2.14

C:\fpcupsystems\trunk\fpcsrc\rtl\netwlibc\libc.pp
function sendfile64(out_fd,in_fd:longint; offset:Poff64_t; count:size_t):ssize_t;cdecl;external libc_nlm name 'sendfile64';
GLIBC_2.3

C:\fpcupsystems\trunk\fpcsrc\rtl\netwlibc\libc.pp
function matherr(_para1:Pexception):longint;cdecl;external libc_nlm name 'matherr';
GLIBC_WRAP_ERROR_SYMBOL_NOT_PRESENT_IN_REQUESTED_VERSION

C:\fpcupsystems\trunk\fpcsrc\rtl\netwlibc\libc.pp
function pthread_cond_init(cond:Ppthread_cond_t; attr:Ppthread_condattr_t):longint;cdecl;external libc_nlm name 'pthread_cond_init';
GLIBC_2.3.2

C:\fpcupsystems\trunk\fpcsrc\rtl\netwlibc\libc.pp
function pthread_cond_destroy(cond:Ppthread_cond_t):longint;cdecl;external libc_nlm name 'pthread_cond_destroy';
GLIBC_2.3.2

C:\fpcupsystems\trunk\fpcsrc\rtl\netwlibc\libc.pp
function pthread_cond_signal(cond:Ppthread_cond_t):longint;cdecl;external libc_nlm name 'pthread_cond_signal';
GLIBC_2.3.2

C:\fpcupsystems\trunk\fpcsrc\rtl\netwlibc\libc.pp
function pthread_cond_broadcast(cond:Ppthread_cond_t):longint;cdecl;external libc_nlm name 'pthread_cond_broadcast';
GLIBC_2.3.2

C:\fpcupsystems\trunk\fpcsrc\rtl\netwlibc\libc.pp
function pthread_cond_wait(cond:Ppthread_cond_t; mutex:Ppthread_mutex_t):longint;cdecl;external libc_nlm name 'pthread_cond_wait';
GLIBC_2.3.2

C:\fpcupsystems\trunk\fpcsrc\rtl\netwlibc\libc.pp
function pthread_cond_timedwait(cond:Ppthread_cond_t; mutex:Ppthread_mutex_t; abstime:Ptimespec):longint;cdecl;external libc_nlm name 'pthread_cond_timedwait';
GLIBC_2.3.2

C:\fpcupsystems\trunk\fpcsrc\rtl\netwlibc\libc.pp
function pthread_mutexattr_getprioceiling(attr:Ppthread_mutexattr_t; prioceiling:Plongint):longint;cdecl;external libc_nlm name 'pthread_mutexattr_getprioceiling';
GLIBC_2.4

C:\fpcupsystems\trunk\fpcsrc\rtl\netwlibc\libc.pp
function pthread_mutexattr_setprioceiling(attr:Ppthread_mutexattr_t; prioceiling:longint):longint;cdecl;external libc_nlm name 'pthread_mutexattr_setprioceiling';
GLIBC_2.4

C:\fpcupsystems\trunk\fpcsrc\rtl\netwlibc\libc.pp
function pthread_mutexattr_getprotocol(attr:Ppthread_mutexattr_t; protocol:Plongint):longint;cdecl;external libc_nlm name 'pthread_mutexattr_getprotocol';
GLIBC_2.4

C:\fpcupsystems\trunk\fpcsrc\rtl\netwlibc\libc.pp
function pthread_mutexattr_setprotocol(attr:Ppthread_mutexattr_t; protocol:longint):longint;cdecl;external libc_nlm name 'pthread_mutexattr_setprotocol';
GLIBC_2.4

C:\fpcupsystems\trunk\fpcsrc\rtl\unix\oscdeclh.inc
    function  FpSchedGetAffinity(pid : pid_t;cpusetsize : size_t;mask : pcpu_set_t) : cint; cdecl; external clib name 'sched_getaffinity';
GLIBC_2.3.4

[fredvs]

@LongDirtyAnimAlf
Note that the packages/libc folder actually needs no suffix fix because it is a deprecated kylix-compatible unit for i386 only.

Hum, not sure for this.
You will need to patch minimum package /libc/src/dlfcnh.inc otherwise you will get some @GLIBC_2.34 for all the dl* methods.

Note that with all the new changes maybe I am wrong now.

[fredvs]

How would we proceed to handle this ?

Maybe using "extended custom" LIBC_SUFFIX?

const
  /// allow to assign proper signed symbol table name for a libc.so.6 method
  {$if defined(linux) and defined(cpux86_64)}
  LIBC_SUFFIX = '@GLIBC_2.2.5';
  LIBC_SUFFIX2 = '@GLIBC_2.3.2';
  LIBC_SUFFIX3 = '@GLIBC_2.9';
  ...
  {$else}
  {$if defined(linux) and defined(cpui386)}
  LIBC_SUFFIX = '@GLIBC_2.0';
  LIBC_SUFFIX2 = '@GLIBC_2.1';
  LIBC_SUFFIX3 = '@GLIBC_2.2';
  ...
  {$else}
  LIBC_SUFFIX = '';
  LIBC_SUFFIX2 = '';
  LIBC_SUFFIX3 = '';
  ...
  {$endif}
  {$endif} 

Example:
function epoll_create(size: cint): cint; {$ifdef FPC_USE_LIBC} cdecl; external name 'epoll_create' + LIBC_SUFFIX2 ; {$endif}

[robert-rozee]

do any of us have the ability to extend the compiler back-end to add a parameter to external? if so, then consider adding an optional parameter useBASE | useAUTO | useNONE.

useBASE would trigger the compiler to search the shared library for all occurrences of the glibc function name, and then match to the BASE one (earliest in list of version numbers). it would use readelf to do this;

useAUTO would do the same, but search for the name containing "@@". this would behave much the same as existing, but take the search for "@@" away from the linker;

useNONE would be for future use, so once we figure out how to stop the linker from versioning any unversioned symbols we can create binaries without any symbol versioning.

then the patch to the RTL would just require appending useBASE after each external statement. see my last posting on the fpc forum where i mention this as an addendum. it is a development from an idea presented by marcov.

cheers,
rob :-)

[fredvs]

@LongDirtyAnimAlf Note that the packages/libc folder actually needs no suffix fix because it is a deprecated kylix-compatible unit for i386 only.

Hum, I think you are right, I disabled in dlfcnh.inc all the + LIBC_SUFFIX, recompile fpc-ootb and indeed no more GLIBC_2.34.

Excellent, so very few code to change (and I did lot of work for nothing ;-) ).

[fredvs]

@LongDirtyAnimAlf Note that the packages/libc folder actually needs no suffix fix because it is a deprecated kylix-compatible unit for i386 only.

Hum, I think you are right, I disabled in dlfcnh.inc all the + LIBC_SUFFIX, recompile fpc-ootb and indeed no more GLIBC_2.34.

Excellent, so very few code to change (and I did lot of work for nothing ;-) ).

Hola, pas trop vite...

Indeed libc.pp is not used for Linux 64 bit but it is not the case for 32 bit.
So maybe better to keep the signature for libc.pp i386. (and all my work is not lost!).

Now, is libc.pp really needed for Linux 32 bit?
Because when you use libc.pp you assign hundreds of methods that will take place in the binary and memory for just nothing if no or very few methods are really used.

I think mainly about the Rpi ARM32 that uses still the 32 bit OS ( 64 bit is under hard development ).

[fredvs]

Re-hello.

Tried this: commented all the *.inc that are in libc.pp.
But lot of complain when trying to compile for i386:

Compiling unixutil/src/unixutils.pp
unixutils.pp(50,45) Error: Identifier not found "TStatBuf"
unixutils.pp(52,46) Error: Identifier not found "__off_t"
unixutils.pp(53,61) Error: Identifier not found "__off_t"
unixutils.pp(54,62) Error: Identifier not found "__off_t"
unixutils.pp(55,59) Error: Identifier not found "__off_t"
unixutils.pp(63,11) Error: Identifier not found "S_IRUSR"
unixutils.pp(63,18) Error: Illegal expression
unixutils.pp(63,19) Error: Identifier not found "S_IWUSR"
....
unixutils.pp(132,24) Fatal: There were 50 errors compiling module, stopping
Fatal: Compilation aborted

So at the moment we may not forget libc.pp for Linux 32 bit...
(But I am sure hundreds of methods are declared and never used.
And it would be a crazy boring work to check what methods are really needed.
Maybe find all the units that use libc.pp and see what methods they use.)

[synopse]

Just don't make any change to the libc unit. It is not allowed at all.

The libc unit is deprecated, and frozen in time for Kylix compatibility, and won't be ported to other architectures or operating systems.
http://wiki.freepascal.org/libc_unit

I don't understand your point about unixutils.pp because this unit is not used by the FPC RTL. It is as deprecated as the libc unit.
And in fact, there is no libc unit use anywhere in the FPC RTL on Linux. Even on i386-linux.
We better stay away from it.

[fredvs]

Hello.

Sorry but libc.pp is used for Linux 32 bit.
There are many units that use libc.pp, like unixutils.pp.

Maybe the doc says libc.pp is not used but it is.

You may try this, in libc.pp, add something wrong like "xxxxxx" and then try to recompile fpc for Linux 32 bit.

You will see that there will be a error.

[synopse]

As I wrote, only unixutils is using libc.pp and this unixutills unit is NEVER used anywhere else in the RTL.
Check the source, Luke! :)

[fredvs]

Here result when compling fpc and in libc.pp added dummy line 'xxxxxxx';

...
Compiling libc/BuildUnit_libc.pp
Compiling ./libc/src/kerneldefs.pp
Compiling ./libc/src/kernelioctl.pp
Compiling ./libc/src/libc.pp
libc.pp(15,1) Fatal: Syntax error, "IMPLEMENTATION" expected but "identifier XXXXXXX" found
Fatal: Compilation aborted

I check the source also when compiling it ;-)

[synopse]

I guess it is not used by the FPC compiler itself (otherwise it would be very weird), but it is part of the RTL.
So it is compiled as part of a RTL packages, bu is never used in practice anywhere else.
As a result, we don't need to change this deprecated unit.

[fredvs]

I agree that maybe the majority of RTL units that use libc.pp are units that nobody will use.

But, in case of somebody wants to use one of those unit that uses libc.pp, there is now a good libc.pp that assign symbols table.

OK, it is not the best argument. ( I try to find one to defend the hard work I did in that f*** libc.pp ).

But you are right, that libc.pp will be used very rarely .

[fredvs]

As a result, we don't need to change this deprecated unit.

Yes, for the patch to fpc I would not add it (but in fpc-ootb it does not hurt to be there).
;-)

[fredvs]

do any of us have the ability to extend the compiler back-end to add a parameter to external? if so, then consider adding an optional parameter useBASE | useAUTO | useNONE.

useBASE would trigger the compiler to search the shared library for all occurrences of the glibc function name, and then match to the BASE one (earliest in list of version numbers). it would use readelf to do this;

useAUTO would do the same, but search for the name containing "@@". this would behave much the same as existing, but take the search for "@@" away from the linker;

useNONE would be for future use, so once we figure out how to stop the linker from versioning any unversioned symbols we can create binaries without any symbol versioning.

then the patch to the RTL would just require appending useBASE after each external statement. see my last posting on the fpc forum where i mention this as an addendum. it is a development from an idea presented by marcov.

cheers, rob :-)

Hello Rob.

Yes, good I dea but, imho, before to think to play with the script for the linker following Marcov idea, first fix the Big-Bug.
At the moment fpc is buggy (__libc_start_main@@GLIBC_2.34 + all dl* methods + not link to libdl.so.2).

Once fpc can run without the Big-Bug, then yes, why not play with a linker-script (that will never work, I did try this, no freedom with the script-options).

I want also thank you for all the hard work we did to explore that jungle (without any help from fpc-devs).

I am very happy now with fpc-ootb that really works like in a Barbie world.

So, I will stop the combat (we have been fighting for this for almost 2 months) and I wish you good luck + big courage with the fpc-devs and your proposals.

Thanks also to everybody here (Rob, AB, DonAlfredo, ...) for your open-mind and patience.

Fre;D