Development for LHC@home

This new version contains an updated version of cranky (v0.1.4) which moves the container filesystem from CernVM 3 to CernVM4.

Looks like there's just 1 modified line (line 130):

root="/cvmfs/cernvm-prod.cern.ch/cvm4"

Hence, I tried it on computers running Theory from -prod.

So far no issues except those that sometimes also happen with cvm3.


Container succeeds (and BOINC validates the task):
15:45:58 CET +01:00 2023-11-29: cranky: [INFO] mcplots runspec: boinc ee zhad 133 - - pythia8 8.301 jetset 100000 984
15:45:58 CET +01:00 2023-11-29: cranky: [INFO] ----,^^^^,<<<~_____---,^^^,<<~____--,^^,<~__;_
16:00:45 CET +01:00 2023-11-29: cranky: [INFO] Container Theory_2390-1082687-984_0 finished with status code 0.


Container fails (unknown reason) but BOINC validates the task:
15:45:39 CET +01:00 2023-11-29: cranky: [INFO] mcplots runspec: boinc ppbar jets 1960 100,1960,10,1960 - pythia6 6.423 dwt 100000 984
15:45:39 CET +01:00 2023-11-29: cranky: [INFO] ----,^^^^,<<<~_____---,^^^,<<~____--,^^,<~__;_
15:45:54 CET +01:00 2023-11-29: cranky: [INFO] Container Theory_2390-1087249-984_1 finished with status code 1.

I changed my setup back to cvm3 since it has a much lower container error rate.

CentOS9-native.
No change from my side for this VM:
12:53:19 (5891): wrapper (7.15.26016): starting
12:53:19 (5891): wrapper: running ../../projects/lhcathomedev.cern.ch_lhcathome-dev/cranky-0.1.4 ()
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Detected Theory App
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] This application must have permanent access to
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] online repositories via a local CVMFS service.
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] It supports suspend/resume if a couple of
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] requirements are fulfilled.
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Most important:
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] - init process is systemd
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] - cgroups v2 is enabled and 'freezer' is available
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] - the user running this application is a member of the 'boinc' group
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] - sudo is at least version 1.9.10
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] - sudoer file provided by LHC@home is installed
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Checking local requirements.
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Found Sudo-Version 1.9.5p2.
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] To run this task in new mode
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Sudo-Version must be at least 1.9.10.
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Missing 'CVMFS_HTTP_PROXY="auto;DIRECT"' in '/etc/cvmfs/default.local'.
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Can't find '/etc/cvmfs/domain.d/cern.ch.local'.
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Can't find '/etc/cvmfs/config.d/cvmfs-config.cern.ch.local'.
12:53:47 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Probing /cvmfs/alice.cern.ch... OK
12:53:47 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Probing /cvmfs/cernvm-prod.cern.ch... OK
12:53:47 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Probing /cvmfs/grid.cern.ch... OK
12:53:47 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Probing /cvmfs/sft.cern.ch... OK
12:53:47 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Excerpt from "cvmfs_config stat": VERSION HOST PROXY
12:53:47 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] 2.11.2.0 http://s1ral-cvmfs.openhtc.io/cvmfs/alice.cern.ch http://128.142.161.84:3126
12:53:47 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Found 'runc version spec: 1.0.2-dev' at '/cvmfs/grid.cern.ch/vc/containers/runc.new'.
12:53:47 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Minor requirements are missing. Will try to run this task in legacy mode.
12:53:47 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Checking runc.
12:53:47 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Creating container filesystem.
12:53:48 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Using /cvmfs/cernvm-prod.cern.ch/cvm4

https://lhcathomedev.cern.ch/lhcathome-dev/result.php?resultid=3278198

12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Found Sudo-Version 1.9.5p2.
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] To run this task in new mode
12:53:19 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Sudo-Version must be at least 1.9.10.
.
.
.
12:53:47 CET +01:00 2023-12-05: cranky-0.1.4: [INFO] Minor requirements are missing. Will try to run this task in legacy mode.

Sudo prior to 1.9.10 doesn't support regular expressions which are a must to define the required commands as pattern.

After setting namespaces.
sudo echo "user.max_user_namespaces = 100" >> /etc/sysctl.conf
This task finished with Status zero:https://lhcathomedev.cern.ch/lhcathome-dev/result.php?resultid=3278254

Good:
The task reported a valid scientific result.

Also good:
The fallback mode works.


And since the fallback mode uses cgroups v1 it has no permission to create a directory below cgroup/unified which represents the v2 hierarchy:

mkdir: das Verzeichnis „/sys/fs/cgroup/unified“ kann nicht angelegt werden: Das Dateisystem ist nur lesbar

BTW:
The namespace setting is another requirement.
user.max_user_namespaces = 100 works but 15000 is a suggestion from Fermilab

The namespace setting is another requirement.
user.max_user_namespaces = 100 works but 15000 is a suggestion from Fermilab

Ok, have changed to 15000.

mkdir: das Verzeichnis &#226;&#128;&#158;/sys/fs/cgroup/unified&#226;&#128;&#156; kann nicht angelegt werden: Das Dateisystem ist nur lesbar
How to change it?
This task is successful after changing namespaces:
https://lhcathomedev.cern.ch/lhcathome-dev/result.php?resultid=3278265
Have forgotten to reboot after changing namespaces
https://lhcathomedev.cern.ch/lhcathome-dev/result.php?resultid=3278228

Either run a recent Linux with cgroups v2 plus the sudo modification.
That's the recommended way as it delegates the cgroup handling used for suspend/resume to systemd.

Or disable cgroups v2 and modify the system according to the old suggestions on -prod.
Theory runs without those modifications as the are only there to allow suspend/resume independend from systemd.

This new version contains an updated version of cranky (v0.1.4) which moves the container filesystem from CernVM 3 to CernVM4.

Which is the latest version of CernVM?

On the repository cernvm-prod.cern.ch there are cvm3 and cvm4.

The VM image has also been updated to use the filesystem from CernVM4.

Theory native
15:27:54 CEST +02:00 2024-05-05: cranky-0.1.4: [INFO] mcplots runspec: boinc pp zinclusive 13000 -,-,76,- - pythia6 6.428 349 100000 89
15:30:02 CEST +02:00 2024-05-05: cranky-0.1.4: [INFO] Container 'runc' finished with status code 1.

Other Theory native
14:13:57 CEST +02:00 2024-05-05: cranky-0.1.4: [INFO] mcplots runspec: boinc pp jets 7000 150,-,2360 - pythia8 8.308 tune-A14-CTEQL1 100000 105
15:17:39 CEST +02:00 2024-05-05: cranky-0.1.4: [INFO] Container 'runc' finished with status code 0.

This Status 0 or 1 are Tasks on the same native Computer:
https://lhcathomedev.cern.ch/lhcathome-dev/results.php?hostid=4690