Message boards : Theory Application : Prepare a bigger Theory VDI?
Message board moderation
Author | Message |
---|---|
![]() Send message Joined: 28 Jul 16 Posts: 511 Credit: 400,710 RAC: 160 ![]() ![]() |
A while ago (starting here: https://lhcathomedev.cern.ch/lhcathome-dev/forum_thread.php?id=563&postid=7329) there was a discussion about whether to prepare a bigger Theory vdi file that includes more data in it's CVMFS cache. The idea was to avoid the need to download that data again and again, especially if no local proxy is available. Meanwhile I collected some related numbers: Within less than 2 weeks the Theory related CVMFS cache grows to roughly 5.7 GB. This would result in a vdi file >6.4 GB. Do we really want that (even with differencing images)? Comments (pros/cons) are welcome. |
Send message Joined: 22 Apr 16 Posts: 709 Credit: 2,114,314 RAC: 6,943 ![]() ![]() ![]() |
As we two changed PM years ago, there was also this question. Is it possible to save CVMFS DATA in a local storage (Pirol?). Don't remember exactly. https://cvmfs.readthedocs.io/en/stable/cpt-containers.html |
![]() Send message Joined: 28 Jul 16 Posts: 511 Credit: 400,710 RAC: 160 ![]() ![]() |
That's what ATLAS/Theory native do since they run in a container. ATLAS/Theory vbox can't. At least not yet. |
Send message Joined: 13 Feb 15 Posts: 1207 Credit: 889,924 RAC: 545 ![]() ![]() ![]() |
6.4GB --> as gz-file less than 3GB. The CVMFS data for Theory will not change very often, so I quess you have to download that zipped-VM once a year at the max. As I wrote in your mentioned thread one might consider to reduce the size by only load needed CVMFS-data for e.g. Pythia6 and Pythia8, but for me your measured data for about 2 weeks would reduce the overall data to download for Theory extremely. Specially for the Theory 'only crunchers'. For every Theory task now necessary data for the differencing VM is downloaded from 1 GB up to 1.5 GB every time for every new task. Most of the Theory's runtime is less than 2 hours. Go figure when you run several tasks simultaniously. An extra advantage is that a Theory task will start the event processing earlier. From the above: PRO |
Send message Joined: 22 Apr 16 Posts: 709 Credit: 2,114,314 RAC: 6,943 ![]() ![]() ![]() |
Parrot is the name HPC, SuperComputer.... https://cvmfs.readthedocs.io/en/stable/cpt-hpc.html But, for us small User, not useful. Edit: btw 6.5 TByte Data last month. The experts of my ISP had made a deeper look in it, really! |
©2025 CERN