Buildmaster 1.3.x symlink error

[kallisti5]

From https://github.com/haikuports/haikuports
   32ea213..dc57b15  master     -> origin/master
Updating 32ea213..dc57b15
Fast-forward
 net-vpn/tor/tor-0.4.8.12.recipe | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Tue Sep 10 23:26:36 UTC 2024 starting buildmaster update

Already up to date.
moving from 0f4e453839d48eaa6080b671a76a0315fd97d6d6 to dc57b15a54b9bff6b81100c2b99ff300128cb9d2
Checking if any dependency-infos need to be updated ...
Looking for stale dependency-infos ...
ports to be built: abaddon-0.2.1
akregator-23.08.5
ale-0.9.1
apache-2.4.57
audacious-4.3.1
autoconf213-2.13
autosp-2023_10_07
baloo_widgets24-24.08.0
bash-5.2.032
bitchx-1.2.1
boehm_gc-8.2.8
cargo_c-0.10.3
ccls-0.20240202
chktex-1.7.9
cjk_utils-4.8.5
claws_mail-4.3.0
cmake-3.28.3
ctags-6.1.0
dvi2tty-6.0.2
dvipng-1.17
dvisvgm-3.4
eiskaltdcpp-2.4.2
erlang-20.1
ghostscript_gpl-10.03.1
gnash-0.8.11~git
gregorio-6.0.0
gsftopk-1.19.2
gsoap-2.8.135
haiku_userguide-2024_09_09
haiku_welcome-2024_09_09
haikuwebkit-1.9.14
hiredis-1.0.2
itinerary24-24.08.0
kcolorpicker-0.3.1
kf6_kquickimageeditor-0.3.0
khealthcertificate24-24.08.0
khtml-5.115.0
kimageannotator-0.7.1
kio_extras24-24.08.0
kirigami_addons1-1.4.0
kitinerary24-24.08.0
kmime24-24.08.0
kopeninghours24-24.08.0
kosmindoormap24-24.08.0
kpkpass24-24.08.0
kpmcore-21.12.0
kpublictransport24-24.08.0
kquickimageeditor-0.3.0
lcdf_typetools-2.110
libass-0.17.3
libetpan-1.9.4
libevent-2.1.12
libgit2-1.5.0
libiconv-1.17
libkdcraw24-24.08.0
libp11-0.4.10
libpcap-1.10.5
libquotient_qt5-0.8.2
libquotient_qt6-0.8.2
libssh-0.10.5
libtorrent-0.13.8
libwebsockets-4.3.3
libzip-1.9.2
lighttpd-1.4.53
llvm18-18.1.7
m_tx-0.63d
mailnews-2.0.0.25
manaplus-1.8.4.14
marknote-1.3.0
melonds-0.9.5
mesa-22.0.5
minidlna-1.3.3
mpv-0.38.0
musikcube-3.0.0
musixtnt-2016_01_30
neochat-23.08.5
opendetex-2.8.11
openimageio2.2-2.2.16.0
openimageio2.3-2.3.10.0
openimageio2.5-2.5.12.0
openldap2.4-2.4.48
openssl3-3.0.14
otter_browser-1.0.03
phonon_vlc-0.12.0
picard-2.12.3
pigz-2.8
pmx-3.0.0
podofo-0.10.3
postgresql11-11.1
postgresql12-12.0
ps2eps-1.70
psi_plus-1.5.1646
psutils-3.3.5
qbittorrent-4.3.1
qcoro_qt6-0.10.0
qgis-3.24.3
qmmp-2.0.4
qmplay2-24.05.23
qt6_3d-6.7.2
qt6_5compat-6.7.2
qt6_base-6.7.2
qt6_charts-6.7.2
qt6_connectivity-6.7.2
qt6_datavis3d-6.7.2
qt6_declarative-6.7.2
qt6_doc-6.7.2
qt6_examples-6.7.2
qt6_imageformats-6.7.2
qt6_location-6.7.2
qt6_lottie-6.7.2
qt6_multimedia-6.7.2
qt6_networkauth-6.7.2
qt6_positioning-6.7.2
qt6_quick3d-6.7.2
qt6_quick3dphysics-6.7.2
qt6_quicktimeline-6.7.2
qt6_scxml-6.7.2
qt6_sensors-6.7.2
qt6_serialport-6.7.2
qt6_shadertools-6.7.2
qt6_speech-6.7.2
qt6_svg-6.7.2
qt6_tools-6.7.2
qt6_translations-6.7.2
qt6_webchannel-6.7.2
qt6_websockets-6.7.2
qt6_webview-6.7.2
qt_creator-14.0.1
qtkeychain-0.14.1
qtkeychain_qt6-0.14.3
raptor-2.0.15
retroshare-0.6.6
ruby-3.2.5
scribus-1.6.1
scummvm-2.8.1
shanty-0.4
t1utils-1.42
tcpdump-4.99.5
telegram_desktop-5.5.2
tellico-4.0
texlive_core-2024.2
tg_owt-0.0.20240804
tor-0.4.8.12
transmission-3.00
tuxpaint-0.9.33
tuxpaint_config-0.0.24
upmendex-1.09
vice_libretro-3.7_20240827
vlc-3.0.21
vlna-1.5
webkit_gtk-2.40.0
wget2-2.1.0
xar-1.8.0.0~498
xml2pmx-2021_02_07
xpdf-4.05
yaddns-1.1.2
zlib-1.3.1
Local builders count: 0
Reporting to /var/buildmaster/output/buildruns/170/status.json
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/HaikuPorter/Main.py", line 39, in __init__
    self.run(args)
  File "/usr/local/lib/python3.9/dist-packages/HaikuPorter/Main.py", line 116, in run
    self.buildMaster = BuildMaster(self.treePath, packageRepository,
  File "/usr/local/lib/python3.9/dist-packages/HaikuPorter/BuildMaster.py", line 260, in __init__
    sysExit('no builders available')
  File "/usr/local/lib/python3.9/dist-packages/HaikuPorter/Utils.py", line 67, in sysExit
    sys.exit(message)
SystemExit: Error: no builders available
ln: failed to create symbolic link '/var/buildmaster/output/builds/*': File exists
ln: failed to create symbolic link '/var/buildmaster/output/records/*': File exists
build master failed

Tue Sep 10 23:26:55 UTC 2024 buildmaster update failed, waiting 60

root@haikuporter-new-7bf5b89f66-wddb8:/var/buildmaster# ls -la /var/buildmaster/output/builds
total 8
lrwxrwxrwx 1 root root   23 Aug 28 15:27 '*' -> '../buildruns/1/builds/*'
drwxr-xr-x 2 root root 4096 Aug 28 15:27  .
drwxr-xr-x 5 root root 4096 Aug 28 15:27  ..

root@haikuporter-new-7bf5b89f66-wddb8:/var/buildmaster# ls -la /var/buildmaster/output/records
total 8
lrwxrwxrwx 1 root root   24 Aug 28 15:27 '*' -> '../buildruns/1/records/*'
drwxr-xr-x 2 root root 4096 Aug 28 15:27  .
drwxr-xr-x 5 root root 4096 Aug 28 15:27  ..

[kallisti5]

Actually.. I just saw this on our production builders too (which are 1.2.8-6)

Wed Sep 25 08:17:01 UTC 2024 starting buildmaster update

Already up to date.
moving from 327126e40e2e4359407dad0979d7e4575b23cf63 to 578d3832abe9d0ca8e7c2543eafa3349f76232f4
Checking if any dependency-infos need to be updated ...
Looking for stale dependency-infos ...
ports to be built: gemrb_x86-0.9.3
Local builders count: 0
Reporting to /var/buildmaster/output/buildruns/6907/status.json
Checking if any dependency-infos need to be updated ...
Looking for stale dependency-infos ...
======================================================================
games-engines::gemrb_x86-0.9.3
======================================================================
The following build dependencies were found:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/HaikuPorter/Main.py", line 39, in __init__
    self.run(args)
  File "/usr/local/lib/python3.9/dist-packages/HaikuPorter/Main.py", line 493, in run
    self.buildMaster.runBuilds()
  File "/usr/local/lib/python3.9/dist-packages/HaikuPorter/BuildMaster.py", line 335, in runBuilds
    self._runBuilds()
  File "/usr/local/lib/python3.9/dist-packages/HaikuPorter/BuildMaster.py", line 385, in _runBuilds
    self._runBuild(buildToRun)
  File "/usr/local/lib/python3.9/dist-packages/HaikuPorter/BuildMaster.py", line 413, in _runBuild
    sysExit('all builders lost')
  File "/usr/local/lib/python3.9/dist-packages/HaikuPorter/Utils.py", line 67, in sysExit
    sys.exit(message)
SystemExit: Error: all builders lost
ln: failed to create symbolic link '/var/buildmaster/output/records/*': File exists
build master failed

Sat Sep 28 02:48:17 UTC 2024 buildmaster update failed, waiting 60
Sat Sep 28 02:49:17 UTC 2024 updating tree and pruning packages of removed ports
From https://github.com/haikuports/haikuports

[mmlr]

The error is caused by there not being any output records for the buildrun, making the part of the script that links them to the central "all records" directory using a wildcard actually link an asterisk instead of the records. The second time around, that asterisk will already exist and the error is thrown. Overall it is benign and doesn't have any actual effect (besides the bogus symlink). The actual failure is due to lost builders in that case and the error of ln is ignored.

These two places should check for there to be any files before running ln to fix this issue:
https://github.com/haikuports/haikuporter/blob/master/buildmaster/backend/assets/bin/buildmaster#L137
https://github.com/haikuports/haikuporter/blob/master/buildmaster/backend/assets/bin/buildmaster#L145

[Begasus]

32bit buildmaster down for a few days again.

[mtl1979]

AFAIK, shopt -s nullglob should be used when glob can result in no files... using for loop might be safer as it avoids executing ln -rst with empty source when nullglob is set.