Message boards : Theory Application : Theory-tasks are failing since July 26.
Message board moderation

To post messages, you must log in.

AuthorMessage
Crystal Pellet
Volunteer tester

Send message
Joined: 13 Feb 15
Posts: 1252
Credit: 995,923
RAC: 45
Message 8959 - Posted: 30 Jul 2025, 8:23:32 UTC
Last modified: 30 Jul 2025, 8:26:20 UTC

@ Laurence:
Since 26th of July the Theory tasks seems to fail on Docker and VBox.
Atter the successful end of a job the output file seems not be created or can't be found, causing an upload error of no upload file present.

2025-07-30 08:37:24 (9084): Guest Log: job: unpack exitcode=0
2025-07-30 09:54:30 (9084): Guest Log: job: run exitcode=0
2025-07-30 09:54:30 (9084): Guest Log: job: diskusage=16080
2025-07-30 09:54:30 (9084): Guest Log: job: logsize=60 k
2025-07-30 09:54:30 (9084): Guest Log: job: times=
2025-07-30 09:54:30 (9084): Guest Log: 0m0.004s 0m0.008s
2025-07-30 09:54:30 (9084): Guest Log: 72m5.718s 1m57.776s
2025-07-30 09:54:30 (9084): Guest Log: job: cpuusage=4444
2025-07-30 09:54:30 (9084): Guest Log: No output found.
2025-07-30 09:54:30 (9084): Guest Log: Job Failed
2025-07-30 09:54:30 (9084): Guest Log: boinc_shutdown called with exit code 208
ID: 8959 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
TeeVeeEss

Send message
Joined: 2 Jul 25
Posts: 3
Credit: 511,754
RAC: 22,655
Message 8960 - Posted: 30 Jul 2025, 9:12:44 UTC

ID: 8960 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
TeeVeeEss

Send message
Joined: 2 Jul 25
Posts: 3
Credit: 511,754
RAC: 22,655
Message 8961 - Posted: 30 Jul 2025, 9:45:17 UTC - in response to Message 8960.  

Nope, downloaded a few T4T tasks for my host and seeing the same result
Excerpt from "cvmfs_config stat":
VERSION   HOST                           PROXY
2.13.1.0  http://s1ral-cvmfs.openhtc.io  http://192.168.50.138:3128
Environment HTTP proxy: not set
job: htmld=/var/www/lighttpd
job: unpack exitcode=0
job: run exitcode=0
job: diskusage=11380
job: logsize=84 k
job: times=
0m0.003s 0m0.005s
10m34.670s 0m41.044s
job: cpuusage=676
No output found.
Job Failed
boinc_shutdown called with exit code 208
https://lhcathomedev.cern.ch/lhcathome-dev/result.php?resultid=3472806
ID: 8961 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
[AF>Le_Pommier] Jerome_C2005

Send message
Joined: 17 Mar 15
Posts: 106
Credit: 1,038,379
RAC: 432
Message 8962 - Posted: 31 Jul 2025, 9:06:11 UTC

ID: 8962 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
boboviz

Send message
Joined: 24 Oct 19
Posts: 276
Credit: 783,415
RAC: 2,464
Message 8968 - Posted: 6 Aug 2025, 7:48:17 UTC

Is it solved??
ID: 8968 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileLaurence CERN
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1150
Credit: 342,328
RAC: 4
Message 8969 - Posted: 6 Aug 2025, 11:05:33 UTC - in response to Message 8968.  

Sorry for the delay, I am investigating ...
ID: 8969 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileLaurence CERN
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1150
Credit: 342,328
RAC: 4
Message 8971 - Posted: 6 Aug 2025, 14:59:04 UTC - in response to Message 8969.  

With v7.62, the job runs fine but marked as an error. Will continue to investigate.
ID: 8971 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
boboviz

Send message
Joined: 24 Oct 19
Posts: 276
Credit: 783,415
RAC: 2,464
Message 8972 - Posted: 6 Aug 2025, 21:08:57 UTC - in response to Message 8971.  
Last modified: 6 Aug 2025, 21:09:11 UTC

In reply to Laurence CERN's message of 6 Aug 2025:
With v7.62, the job runs fine but marked as an error. Will continue to investigate.


Strange. My 7.62 are validate correctly
Eg: 3474541
ID: 8972 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Crystal Pellet
Volunteer tester

Send message
Joined: 13 Feb 15
Posts: 1252
Credit: 995,923
RAC: 45
Message 8979 - Posted: 7 Aug 2025, 11:24:26 UTC - in response to Message 8971.  

In reply to Laurence CERN's message of 6 Aug 2025:
With v7.62, the job runs fine but marked as an error. Will continue to investigate.
Still tha same problem:

2025-08-07 12:03:17 (12536): Guest Log: No output found.
2025-08-07 12:03:17 (12536): Guest Log: Job Failed
2025-08-07 12:03:17 (12536): Guest Log: boinc_shutdown called with exit code 208
ID: 8979 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
boboviz

Send message
Joined: 24 Oct 19
Posts: 276
Credit: 783,415
RAC: 2,464
Message 8980 - Posted: 7 Aug 2025, 12:07:41 UTC - in response to Message 8979.  
Last modified: 7 Aug 2025, 12:08:03 UTC

In reply to Crystal Pellet's message of 7 Aug 2025:

2025-08-07 12:03:17 (12536): Guest Log: boinc_shutdown called with exit code 208


Up to now 20 wus 7.62, all 20 "completed and validated"
It'a s problem of OS (Win, Linux, Mac)?
ID: 8980 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
maeax

Send message
Joined: 22 Apr 16
Posts: 782
Credit: 4,057,880
RAC: 4,191
Message 8981 - Posted: 7 Aug 2025, 12:19:38 UTC - in response to Message 8979.  

Seeing one PC from you, all Task with no Error.
Do you have Boinc 8.2.5 installed?
ID: 8981 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Crystal Pellet
Volunteer tester

Send message
Joined: 13 Feb 15
Posts: 1252
Credit: 995,923
RAC: 45
Message 8982 - Posted: 7 Aug 2025, 13:29:21 UTC
Last modified: 7 Aug 2025, 14:12:30 UTC

It seems that the failure is now only related to the VBox version.
The docker version transmits the result file correctly:

lhcathome-dev 07 Aug 15:19:11 Started upload of Theory_2922-4868215-142_0_r1342941710_result
lhcathome-dev 07 Aug 15:19:13 Finished upload of Theory_2922-4868215-142_0_r1342941710_result (98126 bytes)


and

lhcathome-dev 07 Aug 16:08:22 Started upload of Theory_2922-4815189-142_0_r1192079509_result
lhcathome-dev 07 Aug 16:08:24 Finished upload of Theory_2922-4815189-142_0_r1192079509_result (11434 bytes)
ID: 8982 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
boboviz

Send message
Joined: 24 Oct 19
Posts: 276
Credit: 783,415
RAC: 2,464
Message 8983 - Posted: 7 Aug 2025, 14:20:07 UTC - in response to Message 8981.  

In reply to maeax's message of 7 Aug 2025:
Seeing one PC from you, all Task with no Error.
Do you have Boinc 8.2.5 installed?


No. 8.2.4
ID: 8983 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
boboviz

Send message
Joined: 24 Oct 19
Posts: 276
Credit: 783,415
RAC: 2,464
Message 8984 - Posted: 7 Aug 2025, 14:27:02 UTC - in response to Message 8982.  

In reply to Crystal Pellet's message of 7 Aug 2025:
It seems that the failure is now only related to the VBox version.


That is still the "old" 7.61 version...
ID: 8984 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileLaurence CERN
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 12 Sep 14
Posts: 1150
Credit: 342,328
RAC: 4
Message 8986 - Posted: 8 Aug 2025, 12:37:04 UTC - in response to Message 8984.  

I have just updated the vbox image to 7.62. Hopefully that will fix the issue.
ID: 8986 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
rbpeake

Send message
Joined: 15 Apr 15
Posts: 46
Credit: 996,408
RAC: 4,549
Message 8987 - Posted: 8 Aug 2025, 14:24:28 UTC

Is this the same problem currently affecting the standard LHC Theory app?
ID: 8987 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Crystal Pellet
Volunteer tester

Send message
Joined: 13 Feb 15
Posts: 1252
Credit: 995,923
RAC: 45
Message 8988 - Posted: 8 Aug 2025, 19:41:46 UTC - in response to Message 8986.  

In reply to Laurence CERN's message of 8 Aug 2025:
I have just updated the vbox image to 7.62. Hopefully that will fix the issue.
I returned one first task (more to come) from this new version and it's successful: https://lhcathomedev.cern.ch/lhcathome-dev/result.php?resultid=3476711

it sent a result file back, that was missing in the previous version.
lhcathome-dev 08 Aug 21:32:18 Started upload of Theory_2922-4861108-145_0_r1059717265_result
lhcathome-dev 08 Aug 21:32:20 Finished upload of Theory_2922-4861108-145_0_r1059717265_result (21676 bytes)
ID: 8988 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
kotenok2000
Avatar

Send message
Joined: 22 Aug 22
Posts: 44
Credit: 82,456
RAC: 167
Message 8991 - Posted: 9 Aug 2025, 13:55:35 UTC

I am getting lots of computation errors.
For some reason tasks finish with exit code 0 but are still reported as failed.
Boinc client 8.3.0 still uploads the result file. I hope it can be analyzed.

https://lhcathomedev.cern.ch/lhcathome-dev/result.php?resultid=3477382
ID: 8991 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Theory Application : Theory-tasks are failing since July 26.


©2025 CERN