| Info | Message |
|---|---|
| 1) Message boards : Theory Application : Excessive CPU usage
Message 9486 Posted 26 Jun 2026 by computezrmle |
In reply to boboviz's message of 26 Jun 2026:In my stderr file i found:"stats command returned excessive CPU usage" Yes, it is. For BOINC a Theory (docker) task appears to be singlecore => aid.ncpus = 1 while Podman by default allows each container to use all cores of the CPU. This leads to: https://github.com/BOINC/boinc/blob/master/samples/docker_wrapper/docker_wrapper.cpp#L933-L936 if (ru.cpu_frac > aid.ncpus) {
print_once(string("stats command returned excessive CPU usage\n"));
return -1;
} |
| 2) Message boards : Theory Application : Long Runners
Message 9476 Posted 23 Jun 2026 by computezrmle |
In reply to boboviz's message of 23 Jun 2026:In reply to computezrmle's message of 23 Jun 2026: One reason (for BOINC) is to avoid huge local work buffers that can't be processed before the deadline. So, after the deadline BOINC returns unstarted tasks as errors. The vbox app uses snapshots, this doesn't work for the docker app. See: https://github.com/BOINC/boinc/blob/master/samples/docker_wrapper/checkpoint_notes |
| 3) Message boards : Theory Application : Long Runners
Message 9474 Posted 23 Jun 2026 by computezrmle |
In reply to boboviz's message of 22 Jun 2026:In reply to boboviz's message of 19 Jun 2026: No need to get nervous. I have a running task with walltime (so far) >8 days: ===> [runRivet] Mon Jun 15 01:48:38 UTC 2026 [boinc pp z1j 13000 45 - herwig7 7.2.0 nlo-pw-dipole 100000 974] ... 91900 events processed ... and running fine. Looking at long walltimes only(!) does not indicate a task got stuck. Last info from the mcplots team: They do not have a timeout that rejects any result. BOINC also does not reject tasks that started before the deadline. Most important: It is possible to pause/resume Theory docker tasks but each full restart - e.g. due to a reboot - will cause the tasks to start from scratch. |
| 4) Message boards : Theory Application : Long Runners
Message 9468 Posted 20 Jun 2026 by computezrmle |
In reply to boboviz's message of 20 Jun 2026:Interesting log into stderr.txt Should be rewarded with ~160e06 credit points. ;-) |
| 5) Message boards : Theory Application : Long Runners
Message 9465 Posted 19 Jun 2026 by computezrmle |
In reply to computezrmle's message of 19 Jun 2026:It will succeed, no doubt. So, finally as expected: Name Theory_2922-4791921-1128_1781300023_0 Workunit 242136929 Created 12 Jun 2026, 21:33:46 UTC Sent 12 Jun 2026, 21:40:05 UTC Report deadline 23 Jun 2026, 21:40:05 UTC Received 19 Jun 2026, 12:30:05 UTC Server state Over Outcome Success Client state Done Exit status 0 (0x00000000) Computer ID 10598261 Run time 6 days 14 hours 26 min 41 sec CPU time 6 days 4 hours 9 min 5 sec Priority 0 Validate state Valid Credit 7,993.47 Device peak FLOPS 6.05 GFLOPS Application version Theory Simulation v302.10 (docker) x86_64-pc-linux-gnu Peak working set size 55.63 MB Peak swap size 2.45 GB Peak disk usage 1.49 MB |
| 6) Message boards : Theory Application : Long Runners
Message 9464 Posted 19 Jun 2026 by computezrmle |
In reply to boboviz's message of 19 Jun 2026:I have 2 wus running for 3 days. Over at prod (which gets it's work from the same backend): Walltime >6 days. It will succeed, no doubt. ===> [runRivet] Fri Jun 12 22:03:20 UTC 2026 [boinc pp jets 13000 150 - pythia8 8.235 CP1-CR1 100000 1128] ... 97000 events processed |
| 7) Message boards : Theory Application : New version 7.68
Message 9444 Posted 9 Jun 2026 by computezrmle |
As for your CVMFS configuration It looks good except for cvmfs-config.cern.ch. Add the follwing line to /etc/cvmfs/domain.d/cern.ch.local (create the file if it does not exist): CVMFS_CONFIG_REPO_REQUIRED=yes Add the follwing line to /etc/cvmfs/config.d/cvmfs-config.cern.ch.local (create the file if it does not exist): CVMFS_CONFIG_REPO_REQUIRED=no You can then remove cvmfs-config.cern.ch from the repo list in /etc/cvmfs/default.local. Run cvmfs_config reload after the modifications, and may be cvmfs_config probe to check if it works. The reasons to do it that way are as follows: - cvmfs-config.cern.ch is loaded automatically if another repo is configured to use it (and only then) - order matters: it ensures cvmfs-config.cern.ch is loaded before any other repo - updated global parameters can be distributed automatically. In most cases this affects the server list. Changes there happen not very often but they happen. As for lots of failed tasks (at least on some Linux systems) during the last 2 weeks. It looks like they were caused by a buffer in docker_wrapper that was too short. Since the project decided to use longer task ids and those are used to create the container names docker_wrapper's 'is-a-task-alive' monitoring failed and it killed the containers with long names. BOINC provided a patch for docker_wrapper which is now tested with the version here. From that point all is fine if you can run containers for more than a few minutes that look like: boinc__lhcathome.cern.ch_lhcathome__theory_2922-4881423-1107_1780985479_0 The older scheme was like: boinc__lhcathome.cern.ch_lhcathome__theory_2922-4792878-1030_2 |
| 8) Message boards : Theory Application : New version 7.67
Message 9436 Posted 8 Jun 2026 by computezrmle |
On Linux the man page should be available once Podman is installed but you can also use a recent version from the web. As for the containers.conf location on Windows I suggest to read the Podman tutorial here: https://github.com/podman-container-tools/podman/blob/main/docs/tutorials/podman-for-windows.md Based on that I suggest to put the proxy environment from my previous post in a file like containers\containers.conf.d\50-podman-proxy-environment.conf either below '%APPDATA%' or '%ProgramData%'. ... BOINC has to manage this by itself when the User has setup the Proxy-Info in the normal Client... I disagree with this statement. 'http_proxy=xyz' is just an environment variable and Podman has a couple of well defined interfaces (containers.conf, special environment file, command line parameters) to pass arbitrary environment variables to a container. So, there's no need to reinvent the wheel. If you still want it to be solved by BOINC you would need to open an issue here: https://github.com/BOINC/boinc/issues |
| 9) Message boards : Theory Application : New version 7.67
Message 9434 Posted 8 Jun 2026 by computezrmle |
In reply to Yeti's message of 7 Jun 2026:In reply to Laurence CERN's message of 5 Jun 2026: To use a proxy for the podman engine and/or pass a proxy setting into a container use the containers.conf file. Extended information can be found running 'man containers.conf'. In short: [engine]
env = [
"http_proxy=http://proxy_xy.example.com:3128",
"https_proxy=http://proxy_xy.example.com:3128",
"no_proxy=localhost,127.0.0.0/8,192.168.0.0/24"
] |
| 10) Message boards : Theory Application : New version 7.67
Message 9430 Posted 7 Jun 2026 by computezrmle |
In reply to Laurence CERN's message of 5 Jun 2026:Updated the docker wrapper to v22. For me Theory docker (Linux) requires a docker_wrapper that includes pr7113: https://github.com/BOINC/boinc/pull/7113 To ensure this is included please use an artifact/version from this afternoon or newer. Older versions can't deal with the longer task ids as reported here: https://github.com/BOINC/boinc/issues/7111 |
| 11) Message boards : Theory Application : New version 7.67
Message 9424 Posted 6 Jun 2026 by computezrmle |
In reply to boboviz's message of 6 Jun 2026:In reply to computezrmle's message of 6 Jun 2026: Over @prod it has been explained uncountable times that (and why) you must not rely on BOINC's progress meter. Instead, check the runRivet.log in slots/n/shared to get an impression what the scientific app is doing. Depending on the input params a task can easily take several days. |
| 12) Message boards : Theory Application : New version 7.67
Message 9421 Posted 6 Jun 2026 by computezrmle |
In reply to boboviz's message of 6 Jun 2026:Hope this solve the "neverendig-wus" problem What are "neverendig-wus"? Especially those where docker_wrapper is the root cause. |
| 13) Message boards : Theory Application : New version 7.67
Message 9420 Posted 6 Jun 2026 by computezrmle |
In reply to Crystal Pellet's message of 6 Jun 2026:... I don't miss them, because the logging was so extended, that the most important part (building the image) at the beginning of stderr.txt was truncated. This was the intention behind issue #7001: https://github.com/BOINC/boinc/issues/7001 If somebody needs runtime stats I suggest to use 'podman stats ...' in a terminal. It is highly configurable as can be seen here: https://docs.podman.io/en/stable/markdown/podman-stats.1.html |
| 14) Message boards : Theory Application : boinc-buda-runner V4 and BOINC 8.2.8
Message 9215 Posted 3 Dec 2025 by computezrmle |
It's just one out of many open points that need to be solved before docker/podman based apps are ready for production. |
| 15) Message boards : Theory Application : boinc-buda-runner V4 and BOINC 8.2.8
Message 9213 Posted 2 Dec 2025 by computezrmle |
Forward a proxy via environment variable. More details can be found here: https://lhcathomedev.cern.ch/lhcathome-dev//forum_thread.php?id=682&postid=8628 https://lhcathomedev.cern.ch/lhcathome-dev/forum_thread.php?id=682&postid=8607 |
| 16) Message boards : CMS Application : Computation Error
Message 9195 Posted 15 Nov 2025 by computezrmle |
I wonder what you test here for CMS atm. The very same BOINC app is available via prod for roughly a year and even if a new VM image would be in use here it would get the scientific jobs from the same backend queue as prod. As for your log: "The filename or extension is too long." is misleading on Windows. Ever since vboxwrapper reports exit number 206 (EXIT_INIT_FAILURE) when the initialization phase fails. Unlike Linux/Apple Windows doesn't understand it and expands it to something weird. The real error is caused by the missing x509 credential. Unfortunately this is due to a service failure at CERN that happens rather often (too often) in the past weeks. You can do nothing on your side to fix it but you may try a client at prod instead. |
| 17) Message boards : General Discussion : Xtrack Beam simulation 0.05
Message 9185 Posted 12 Nov 2025 by computezrmle |
There's nothing new. At least nothing related to BOINC. Like CMS Theory VMs on dev and prod get their work from the same backend queues. Hence, valid results here deliver useful scientific data. |
| 18) Message boards : General Discussion : Xtrack beam simulation v0.04
Message 9151 Posted 25 Sep 2025 by computezrmle |
There's a clear definition here: https://github.com/BOINC/boinc/wiki/AppVersion https://github.com/BOINC/boinc/wiki/BetaTest Apps on the dev site are always beta. Xtrack on prod is clearly marked as beta test here: https://lhcathome.cern.ch/lhcathome/apps.php To get beta tasks it must be enabled here: https://lhcathome.cern.ch/lhcathome/prefs.php?subset=project q.e.d. |
| 19) Message boards : General Discussion : Xtrack beam simulation v0.04
Message 9149 Posted 25 Sep 2025 by computezrmle |
Even on the prod site Xtrack is clearly marked as beta app which needs explicit opt-in. Cancelling a whole batch may be necessary to avoid badly configured tasks hang around for a long time. Beta apps always need close monitoring by the user and it would be a good idea to start with a small buffer of few tasks instead of many hundreds. If anybody can't deal with it, disable beta apps and wait for a stable version. |
| 20) Message boards : General Discussion : Xtrack beam simulation v0.04
Message 9114 Posted 22 Sep 2025 by computezrmle |
This is a very old BOINC behaviour that can always be observed in connection with new apps/app versions. At least work fetch, credit calculation and estimated time left are based on a couple of input values which are not even constants. Overcommitted computers or very low (sometimes very high!) credits per task are typical results of this. It always takes a while until this stabilises. The more runtimes vary the longer it takes (or becomes even worse). There is not really anything the project can do as the root cause must be solved in the BOINC code (server and client). |
©2026 CERN