Developers and Applications Working Group: Lauri, Stefan, Gabriele + all developers

Deliverables

Developers and Applications Working Group Plan

SAM Developer Documents:
------------------------

- Prerequisites in order to be productive as a sam developer; intended audience for
  further documentation
  (i.e., what we assume you already know, that we will NOT be documenting)
   - cvs
   - understanding of product packaging techniques,
     product environment bootstrapping
     (specifically as related to ups/upd)
   - subprojects may require knowledge about:
      CORBA, oracle (sql in general), x509, globus, condor, grid protocols, xml, java, ???
  >>> this probably needs to be done from scratch.
  >>> Sinisa, Sinisa, Sinisa, Wyatt, Rick, Sinisa, Rob

- Automated Documententation generation:
  We believe it is necessary to spend some time investigating and
  configuring a set of tools for automatic code documentation generation,
  so that the public interfaces, methods, etc., for
  all modules are documented in a consistent way as part of 
  the package release procedure.

  HIGH     o  IDL: idldoc, ?
  MEDIUM   o  python clients:  pydoc, ?
  LOW	 o  c++: ???
  NONE	 o  java: javadoc, ?

  >>> ASSIGNED TO Steve and Sinisa (doxygen, idldoc)
     update 16 apr: doxygen for both c++ and idl

  Once these tools are in place, each component should include
  standardized documentation on:
    a) IDL-based (public) interfaces (via idldoc-like tool)
    b) API-based (public) interfaces (via CommandParser tools)
    c) internal (private) interfaces (via pydoc/javadoc/???-like tool)


- DEVELOPER OVERVIEW DOCUMENTATION:
  --------------------------------
  - Additional documents that need to be written and/or pulled
    together from existing documents, which describe more of the
    technical "how and why" behind-the-scenes issues not covered
    in the automated docs.

HIGH    o  Overview: How the pieces fit together
	   - diagram showing the major SAM components and how
	    they fit together
	    >>> Rick, Wyatt, Sinisa and Rob  not done

MEDIUM    o  Overview: SAM CORBA Infrastructure 
	   - IDL naming conventions
	   - corba structure wrappers
	   - corba exception handling
	   - automarshalling proxies and servants
	   >>> Stefan and Lauri
          Done, visible at http://d0db-dev.fnal.gov/~lauri/DocWeek/Overview_SamCorbaInfrastructure.html,
           needs to be moved to a more permanent home

MEDIUM    o  Overview: Server Instantiation and Configuration Tools
	     - sam_bootstrap/sam_config
	     >>> Art and Lauri
            Art thinks these exist on the CDF workshop page
	     - serverRun/jim_config
	   >>> Andrew/Igor and Gabriele
            We think Gabriele has the links for these

LOW    o  Overview: Packaging Standards
             - sam_util
	     >>> ???
	     - standardized GNUmakefile targets
	     >>> ???

HIGH    o  Overview: API Command Suite ToolChest (for python apps)
	     - sam_common_pylib/SamCommand/ suite of utilities
	     >>> Stefan and Lauri
             Done, will be part of sam_common_pylib package
              documentation at http://d0db.fnal.gov/sam_common_pylib/ when it goes live,
              but until then available at http://d0db-dev.fnal.gov/sam_common_pylib.

MEDIUM    o  Overview: Monitoring Tools
	     - samTV
	     - jim/grid monitoring
	     >>> IGOR AND ADAM

MEDIUM    o  Overview: Test Tools
	  - sam_test_tools
	  >>> Matt and Adam
	  - dbserver unit tests
	  >>> Randy and Steve
	  - sam_common_pylib unit tests
	  >>> Lauri

MEDIUM    o  Overview: Data Movement
	     - sam_cp & protocols
	     >>> SINISA (sam_cp) plus individual protocols:
		 >>> GABRIELE (sam_gridftp)
		 >>> LAURI (sam_bbftp)
		 >>> ANDREW (sam_encp)

HIGH     o  Overview: Local Job Scheduler Adaptation
	     - sam_batch_adapter
	     >>> Robert Illingworth and Sinisa


- COMPONENT DOCUMENTATION:
-------------------------

Preliminary Component Breakdown:
    Core SAM
      INFRASTRUCTURE: (Stefan, Lauri)
        http://d0db[-dev].fnal.gov/sam_common_pylib/Overview_SamCorbaInfrastructure.html
        http://d0db[-dev].fnal.gov/sam_common_pylib/Overview_SamCommon.html
        http://d0db[-dev].fnal.gov/sam_common_pylib/howToBuildAndRun.html
	Corba Infrastructure:    
	  dbProxy (autoMarshall)
	  corbaStruct wrappers
	  exception handling
	python API Infrastructure:
	  commandParsing tools
         http://d0db[-dev].fnal.gov/sam_common_pylib/CommandInterface.html

      DB - (oracle): ???

      Servers:
        DBServer (Randy, Steve)
	  dimension parser
	  dbServerBase
	  dbServerGen 

        Station (Rob, Andrew)
	  smaster
	  fss
	  stagers

        Name Server (Stefan, Sinisa)
	Logger (Andrew, Igor)
	Optimizer (Igor, Andrew)
	(MIS)

      Clients
        Libraries (sam_common_pylib (lauri), sam_client_cpplib (sinisa))
	Packages:
          sam_user_pyapi, sam_admin_pyapi, sam_mis_pyapi
	  Experiment specific: sam_manager, AC++
	  sam_project_editor, sam_farm_request

      Major Utilities:
        samTV (Adam)
	sam_cp + transfer protocols (Sinisa, Gabriele)
	sam_bootstrap (sam_products, init_sam) (Art, Adam)
	sam_config (Art, Sinisa)
	test_harness (Matt, Adam)

    JIM  (Gabriele, Igor, Andrew)
      Infrastructure
        xmldb (client/server)
	configuration (jim_config)
	GSI (sam_gsi_config, cron jobs)
	middleware (globus, condor)
      Client suite
      Submission site suite
      Broker site
      Monitoring suite
        current state (push/pull models)
	history
	web interface (jim_www)
      Execution site suite
        Fabric interface (jim_job_managers)
        Fabric management (jim_sandbox)
        Registration services (jim_advertise)