Development for LHC@home

OK, the database fix appears to have been applied, I have been able to submit a new batch of jobs.

---

Yep, done 2 already.

Yep, done 2 already.

I make it 3 now. :-)

---

From those success jobs:

28 needed 3 attempts to finish and 249 succeeded after 2 attemps.
IMO that number of extra attemps is excessive.

Has anybody carefully followed what happens to those jobs that are abandoned when the host is shutdown/rebooted or whatever? Could they show up like this:-

"N/A / Error return without specification"? Many initial failures do. Maybe Condor puts the "error" bit in because some timeout or heartbeat fails, there certainly wouldn't be a "specification".

They all require at least one retry. At the moment there will probably be a higher rate of these events because people are "poking around", looking for problems and suchlike. Many hosts (mine...) don't run continuously, that tends towards one extra attempt per host per day. IMO you can't reasonably require volunteers to run machines continuously, although I realise many do.

I've followed up two jobs from the 50 event task, 281 and 305. Both were
started but abandoned when the hosts were shut down. Dashboard shows my IP but
the (presumably) final hosts's start and stop times. I can't see any indication
on Dashboard that the jobs weren't run to completion on the hosts which
originally picked them up. No retries are shown. So, at least for successful
jobs, the high retry rate isn't due to these "abandoned" jobs - they show up as
normally successful jobs.

From your today submitted batch 151102_084842:ireid_crab_CMS_at_Home_TTbar_50ev_3 the first 3 returned jobs failed with 8001 / CMS exception (CMSSW)
All 3 from the same IP and running (too) short to be true.

Edit: the 4th job finished with success error code.

From your today submitted batch 151102_084842:ireid_crab_CMS_at_Home_TTbar_50ev_3 the first 3 returned jobs failed with 8001 / CMS exception (CMSSW)
All 3 from the same IP and running (too) short to be true.

Edit: the 4th job finished with success error code.

I see three job logs with 8001 errors, 7, 17, and 23 and they do all come from the same machine. It has 13 errors in all today. I'll see what I can find.

---

Some of our computers are re-assigned to another backfill batch of 200 jobs.
Looks like they are failing, again.