Message boards :
News :
Problem writing CMS job results; please avoid CMS tasks until we find the reason
Message board moderation
| Author | Message |
|---|---|
 ivan Send message Joined: 20 Jan 15 Posts: 1139 Credit: 8,181,211 RAC: 2,023  |
Since some time last night CMS jobs appear to have problems writing results to CERN storage (DataBridge). It's not affecting BOINC tasks as far as I can see, they keep running and credit is given. However, Dashboard does see the jobs as failing, hence the large red areas on the job plots. Until we find out where the problem lies, it's best to set No New Tasks or otherwise avoid CMS jobs. I'll let you know when things are back to normal again. 
|
Magic Quantum Mechanic Send message Joined: 8 Apr 15 Posts: 780 Credit: 12,150,930 RAC: 2,185   
|
Thanks Ivan and I will watch for the update and when we can run these again. |
|
ivan Send message Joined: 20 Jan 15 Posts: 1139 Credit: 8,181,211 RAC: 2,023 |
Given the Easter holidays, I'm not sure when someone at CERN will be able to look at it. We're getting "permission denied" trying to write results and logs to the DataBridge, which suggests either something has filled up, or a certificate has expired. We are starting to get some hard failures now so I guess these are jobs which exceeded the re-try limit.
|
|
ivan Send message Joined: 20 Jan 15 Posts: 1139 Credit: 8,181,211 RAC: 2,023 |
We seem to be getting successful jobs again. Unfortunately I'm not able to access a PC until tonight to verify how well we are recovering. Resume tasks with care.
|
©2024 CERN