Message boards :
News :
CMS job shortage Wednesday 13th November
Message board moderation
Author | Message |
---|---|
Send message Joined: 20 Jan 15 Posts: 1139 Credit: 8,181,211 RAC: 2,023 |
CMS IT will be installing a new version of WMAgent on Wednesday. This will impact job availability for the duration of the intervention. We might be able to eliminate the little gremlin that's been plaguing us for the last few weeks, too. So, please set your CMS processors to No New Tasks sometime tomorrow, Tuesday 12th, so that current tasks will stop requesting new jobs before the queues get cut. I'll let you know when jobs are available again. Thanks. |
Send message Joined: 8 Apr 15 Posts: 780 Credit: 12,150,930 RAC: 2,185 |
Thanks Ivan I will just finish the ones I have running. |
Send message Joined: 20 Jan 15 Posts: 1139 Credit: 8,181,211 RAC: 2,023 |
OK, jobs are available now, so you can start running CMS tasks again. |
Send message Joined: 8 Apr 15 Posts: 780 Credit: 12,150,930 RAC: 2,185 |
I have several running and a few finished and returned with no problems. Mad Scientist For Life |
Send message Joined: 8 Apr 15 Posts: 780 Credit: 12,150,930 RAC: 2,185 |
edit: well I may have run out of luck since after turning in the finished tasks I started up new ones and on this 8-core I run 2 of the 2-core tasks and the first one was real slow getting connected with Cern and it failed at the VM and watching the Console as usual it was real slow getting beyond *daemon* and then froze up soon after page two started......then the VM killed it. I just started a new one and it was slow getting started too but it looks like it might make it to HTCondor ping before too long (but I wouldn't bet on it) And as far as my end my isp is running at full speed. (oh and I do NOT like the time limit on editing here either since I just typed all this out and it then would not let me so I had to copy and paste it here) -152 (0xFFFFFF68) ERR_NETOPEN [ERROR] Could not connect to lhchomeproxy.cern.ch on port 3125 Mad Scientist For Life |
Send message Joined: 8 Apr 15 Posts: 780 Credit: 12,150,930 RAC: 2,185 |
I just got another one of those errors on a different host Testing CVMFS connection to lhchomeproxy.cern.ch on port 3125 10:47:19.272426 VMMDev: Guest Log: [DEBUG] nc: connect to lhchomeproxy.cern.ch port 3125 (tcp) timed out: Operation now in progress 10:47:19.273231 VMMDev: Guest Log: nc: connect to lhchomeproxy.cern.ch port 3125 (tcp) timed out: Operation now in progress 10:47:19.326805 VMMDev: Guest Log: [DEBUG] 1 10:47:19.398225 VMMDev: Guest Log: [ERROR] Could not connect to lhchomeproxy.cern.ch on port 3125 10:47:19.456350 VMMDev: Guest Log: [INFO] Shutting Down. Both times they already had one of these tasks running (3;30am right now) And both times they got hung up at starting daemon Also both time I sent those back and started a new one and they both are up and running and most likely will be Valids .....goodnight Mad Scientist For Life |
Send message Joined: 8 Apr 15 Posts: 780 Credit: 12,150,930 RAC: 2,185 |
Ivan could you try your Windows OS pc again and see if it has this same problem I keep having? I am either having the Linux server (CentOS) not wanting to connect with Windows or I just need to move my computers 5000 miles closer to have a more reliable communication of servers. Many of mine are working and I have 4 different hosts (3 on Win 10 OS and one on Win7) but I keep watching this sloooooow connection between Cern and my end. -152 (0xFFFFFF68) ERR_NETOPEN this host had 3 Valids and now one of these. [ERROR] Could not connect to lhchomeproxy.cern.ch on port 3125 https://lhcathomedev.cern.ch/lhcathome-dev/result.php?resultid=2835807 This one started real slow on page one of the VM Console right from the start and did make it to page 2 and started to look like it would run but it froze up again where it is testing connection to the Condor. I just started another one and once again it looks like it will run this time. I haven't checked over at LHC yet so I don't know if anyone else had this happen. And I am still running at full speed on the isp (4Mbps) One thing for sure is I am probably the only human who would spend this much time just to get them to run and watching the thousands of VM Console pages and VB Manager Logs |
Send message Joined: 13 Feb 15 Posts: 1188 Credit: 859,751 RAC: 25 |
.... you try your Windows OS pc again and see if it has this same problem I keep having? I loaded 1 CMS on my rather old Windows host. Host heavy loaded with other tasks, cmsRun started 13 minutes after the boot and started running full load after 22 minutes up-time. |
Send message Joined: 8 Apr 15 Posts: 780 Credit: 12,150,930 RAC: 2,185 |
.... you try your Windows OS pc again and see if it has this same problem I keep having? Yes 12 - 13 mins or sooner is the best chance at getting them to run these (or any VB task) They always run if that happens. Like I said I have Valids on all of mine yet they can't be trusted since when the next one starts it may just sit there for 12mins or longer at daemon which means they won;t make it to HTCondor ping or cmsRun It would be nice if I didn't have to watch all of them. They are all running now but it wasn't on their own and just now I used up my day time high-speed (less than 2 days just because I run these) But when all of the current running tasks are finished and ready it will be late at night and I get this goofy *bonus* 50MB data use at 4-15Mbps from 2am to 8am so all will run them that way until I use all of that up. (87% left) (mine are also using all the rest of the cores running all the versions of Sixtracks and some of them run over 2 days) |
©2024 CERN