RunJob March 3, 2004
---------------------

(1) Some news on the Project Status

The Project has been approved.  Ruth will coordinate the project within
the Computing Division.  She had asked for current levels of effort among
people currently on project, and we went around the table and verified:
Eric    , expects to continue
Peter   , expects to ramp up to that soon
Anzar   , 
Jerry   , 
Dave    ,  depends on job situation
-----------
A workshop will be tentatively planned for April 8th, 2004.  The exact
agenda is to be worked out with the stakeholders in the meantime.
-----------
It was pointed out that  if the group is going to grow, we need procedures
within the group to better coordinate developers' efforts.
** - Anzar and Gerry will work on Project Engineering Plan. 
        (Draft in two weeks, final form by April 8th.) 
   -Importance of prototyping over super-strict industry style design in 
     this group was emphasized.  
   -The group will likely expand by several more people.  
-----------
** - Current ShahKar team members should read the "Considerations on
     Constraint Modeling"... paper and provide comments by next week.
       (That's Peter, Dave, Anzar, Jerry, and Eric.) 
-----------

(2) XMLP Feature Review and Code Walkthrough

What is XMLP?
  -It takes a general Python object and converts it to XML and back.
  -Demonstration with some test classes.
  -Uses DOM; a DOM object is a tree.
  -Uses Python miniDOM.  (No PyXML stuff needed.) 
  -Basic types are represented, everything else is mapped to an 
   "InstanceType".
  -Extendible, so that if one defines new "types" in the XMLP, then
   composite objects are handled as well.  (Recursive.) 

Caveats: 
  (A) An XMLP recreated Object has to have its Class "in scope" 
      (or at least in PYTHONPATH)
         -This could blow up later if classes change.
         -What kind of guarantees does a physicist have that there is
          consistency?
            Strategy 1: Emit warnings based on versions
              -Version of the serializer and the version of the class.
              -Make version relevant at the Module level only (?)
              -Check on version numbers: OK if majors agree.
  (B) What are we using XMLP for exactly ?
         -Transmit scriptObjects from site to site
         -Configruring runtime in shREEK.  (Abstract planning...) 
              -Editors...  Do we want to pack up code and ship it around?
         -Shipping monitoring information
         -Transmit macros from site to site.
      -It was noted that for these use cases, then caveat A may not be so 
       serious.  We need to study ways to restrict usage to "safe" use
       cases.
  (C) Unbound method problem.
        -Dave has a fix in mind: save class names along with method names
         in the XML. 
  (D) Overly verbose naming issue in XML
        -Dave will change XML format so that XMLP "name" is saved only once
         and not with each tag.

Other Methods considered: 
  -What about pickling? 
    -Not human readable... 
    -Has exactly the same set of problems.
    -Can't query them easily (I added this after the meeting... -G) 

We branched off into a discussion of the distributed RunJob model: 
    -XMLP running behing Web Services and we ship all of the RunJob code
     every time.  
         -This would address caveat A completely.
         -Probably not considered secure !!!!
    -MCRunjob living at multiple sites 
         -This is more secure.
         -This suffers from caveat A maximally.

We came back down to earth and re-analyzed XMLP in terms of the core 
RunJob use cases running locally or in a well controlled cluster 
environment: 
   -Caveat A is not a problem here.
   -With fixes for Caveats C and D, XMLP is approved for use for the 
    cases enumerated above in section B.
   -We need to carefully consider the distributed use cases for using 
    this because it isn't clear yet what the issues are. 

-----------------------
We added this late to the agenda.

(3) Subpackages

   We would like to make things more easily separable in the RunJob 
 package.  d0Tools may like to use the "scriptObjects" for instance. 
    -Do these as directories within existing ShahKar repository.

 ** -Dave will work with Eric on this.

     What are the subpackages?
          -Logger
          -scriptObject
          -XMLP
          -shREEK
          -FMB
          -Core