Message boards : CMS Application : Dip?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · Next

AuthorMessage
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1129
Credit: 7,870,629
RAC: 576
Message 4826 - Posted: 2 Apr 2017, 13:07:56 UTC - in response to Message 4825.  

WMAgent has died again. Set no new tasks if you can...

Looks like we are back up again.
ID: 4826 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1129
Credit: 7,870,629
RAC: 576
Message 4839 - Posted: 10 Apr 2017, 22:45:36 UTC

WMAgent appears to have stopped again. Please set No New Tasks if you can.
ID: 4839 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 4840 - Posted: 11 Apr 2017, 6:18:58 UTC - in response to Message 4839.  

Please set No New Tasks if you can.


Too late!
This REALLY needs too be fixed.
It happened 3 or more times.

I looked at the "Task Tracker" and NOTHING has been done for month.
It should be removed.
ID: 4840 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1129
Credit: 7,870,629
RAC: 576
Message 4841 - Posted: 11 Apr 2017, 8:41:43 UTC

Back again.
ID: 4841 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 4859 - Posted: 26 Apr 2017, 6:15:17 UTC

WMAgent has died again. Set no new tasks if you can...



Déjà vu.
ID: 4859 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1129
Credit: 7,870,629
RAC: 576
Message 4860 - Posted: 26 Apr 2017, 6:46:16 UTC - in response to Message 4859.  
Last modified: 26 Apr 2017, 6:51:31 UTC

WMAgent has died again. Set no new tasks if you can...



Déjà vu.

Investigating...

The queue is fine, WMAgent is running. I'll ping Laurence. We have a LHC@Home meeting in an hour, so let's hope we can sort it by then. In the meantime, yes, set NNT.
ID: 4860 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1129
Credit: 7,870,629
RAC: 576
Message 4862 - Posted: 26 Apr 2017, 7:49:24 UTC - in response to Message 4860.  
Last modified: 26 Apr 2017, 7:51:40 UTC

I'm getting new tasks now, so something must have been fixed.
The squid proxy shows a marked dip in traffic from ~2200 last night so that's when something fell over.
ID: 4862 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1129
Credit: 7,870,629
RAC: 576
Message 4863 - Posted: 26 Apr 2017, 8:02:03 UTC - in response to Message 4862.  

Oops, my mistake -- I'm getting new tasks, not new jobs!
ID: 4863 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1129
Credit: 7,870,629
RAC: 576
Message 4865 - Posted: 26 Apr 2017, 10:24:01 UTC

The problem has been identified. Hopefully a fix will come quickly.
ID: 4865 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1129
Credit: 7,870,629
RAC: 576
Message 4867 - Posted: 26 Apr 2017, 13:06:41 UTC - in response to Message 4865.  

You should be able to pick up some jobs now.
ID: 4867 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1129
Credit: 7,870,629
RAC: 576
Message 4890 - Posted: 3 May 2017, 22:51:05 UTC

Drop in the activity graph. First indication is a Dashboard problem: WMStatus and grafana show no anomalies.
ID: 4890 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 4894 - Posted: 5 May 2017, 6:49:13 UTC - in response to Message 4890.  

Drop in the activity graph.


This time all graphs.
Still a dashboard issue?
ID: 4894 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1129
Credit: 7,870,629
RAC: 576
Message 4895 - Posted: 5 May 2017, 17:08:08 UTC - in response to Message 4894.  

Drop in the activity graph.


This time all graphs.
Still a dashboard issue?

I think so; recovered now.
ID: 4895 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 4898 - Posted: 12 May 2017, 16:50:14 UTC

Cannot get new jobs?
Just a glitch?
ID: 4898 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1129
Credit: 7,870,629
RAC: 576
Message 4899 - Posted: 12 May 2017, 19:09:47 UTC - in response to Message 4898.  

Cannot get new jobs?
Just a glitch?

No, it's Friday night. :-( The WMAgent is down. I've messaged Alan; please set No New Tasks or switch to another subproject until he's kicked it into life again.
ID: 4899 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 4900 - Posted: 12 May 2017, 19:35:21 UTC - in response to Message 4899.  
Last modified: 12 May 2017, 19:37:40 UTC

No, it's Friday night. :-( The WMAgent is down


Is this some anticipated (known to reoccur) incident, that nobody can/want to fix?

Maybe some scheduled maintenance should be implemented. Then, at least, these down-times occur at a known time, rather than happening at more or less random intervals.

Similar to seti downtime every Tuesday.
ID: 4900 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1129
Credit: 7,870,629
RAC: 576
Message 4901 - Posted: 12 May 2017, 19:45:08 UTC - in response to Message 4900.  

No, it's Friday night. :-( The WMAgent is down


Is this some anticipated (known to reoccur) incident, that nobody can/want to fix?

Maybe some scheduled maintenance should be implemented. Then, at least, these down-times occur at a known time, rather than happening at more or less random intervals.

Similar to seti downtime every Tuesday.

No, it just seems to be the nature of Sod's Law. Things don't always fail on Friday afternoon, but when they do there's a higher probability of no-one being around to fix it. Alan has almost always responded quickly in these cases, but there's no guarantee that he's not tied up elsewhere at any given time.
ID: 4901 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 4902 - Posted: 12 May 2017, 20:25:57 UTC - in response to Message 4901.  

Things don't always fail on Friday afternoon, but when they do there's a higher probability of no-one being around to fix it.



Understood.

Thanks,Ivan.
ID: 4902 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ivan
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 20 Jan 15
Posts: 1129
Credit: 7,870,629
RAC: 576
Message 4903 - Posted: 12 May 2017, 22:38:51 UTC - in response to Message 4902.  

Turns out there was an "intervention" on the server this evening, that left it in a bad state. Fixed now.
ID: 4903 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42
Volunteer tester

Send message
Joined: 16 Aug 15
Posts: 966
Credit: 1,211,816
RAC: 0
Message 4935 - Posted: 27 May 2017, 5:00:08 UTC

Yet again!

Could someone pleeease do something about the quota being ruined, when the server has no jobs?

It is getting somewhat annoying.(and counterproductive)
ID: 4935 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · Next

Message boards : CMS Application : Dip?


©2024 CERN