Introduction
RunJob is a project in the Fermilab Computing Division that seeks to address the common needs of participating experiments in automating the creation of production processing jobs for Monte Carlo generation and for batch oriented data reprocessing and analysis tasks. Based upon existing software used by two experiments at the lab, RunJob will provide modeling tools to describe tree-like multi-application workflows, tools to organize and to track parameters associated with these applications, APIs to work with external control databases, tracking databases, or services, and modules to adapt jobs to a variety of environments including grid environments.
Other projects
The CMS variant, MCRunjob, is part of the
OCTOPUS project. It is
capable of managing the creation of jobs for Monte Carlo simulation and
data reprocessing in cooperation with an external control and tracking
database, and it creates jobs for local farm environments, grid environments
based on the Virtual Data Toolkit (VDT),
and the LHC Computing Grid (LCG).
Development of CMS MCRunjob is a Level 2 deliverable of the USCMS contribution
to the Core Application Software (CAS) subsystem of the Core Computing and Software
(CCS) project of CMS. VDT based grid integration work was done by
Particle Physics Data Grid.
The DZero variant, MCRunjob,
was originally developed to handle the generation of
multi-application jobs for the DZero Monte Carlo challenges
in 1999. An early benefit of using MCRunjob was auto-generation of file metadata for
import of produced Monte Carlo files into the SAM
database. DZero MCRunjob can produce jobs that run in a variety of local farm and
mainframe environments and is being integrated into the DZero
SAM-Grid environment. The grid
integration efforts are a major milestone of the GridPP
group.
The CDF experiment is just beginning to look at RunJob to manage job creation and
submission in farm based environments.
