2014 02 18

• Who: Antons, Ole, AndreM, Shantenu, Matteo

• Agenda:

  • open TODOs
    • TODO OW: plan to replace BJ in BJSimple
    • TODO AM: sync with Antons about testing plan (incl. SJ)
    • TODO OW: Sync with Vivek and MS about BJA/SP
    • TODO AM: add data staging directive parser into utils
    • TODO AM: next call: discuss long term plans for async PS and bulks etc.
  • updates from Antons, Ole, Andre
  • scaling to very large numbers of jobs (see bigjob thread / Scott Michael)
  • programming models for distrib systems
    • many many options
      • async calls, pull state
      • async calls, state notification
      • bulk operations
      • caching for latency hiding
      • caching for bandwidth saving
      • pre-fetching of data
      • guessing of data (speculative execution)
      • redundant components (failover)
    • some options apply to architecture, some to protocols, some to interfaces
    • context in SP
      • current state: sync ops, state pulling, state notifications, bulk ops
      • pro and cons?
      • do we need all? do we need others?
      • coherency of stack...
      • long term approach
    • related:SP error handling in Troy
  • Troy question: who defines default queue? See #41
  • Troy question: error inspection, pilot state notifications.
  • AOB
    • Thursday meeting slot

• Notes:

  • TODO AM: establish troy driven integration tests 'til Thursday
  • TODO SJ: discuss non-zero workloads with Antons
  • TODO AM: $HOME/%(home)s / Troy config docs
  • TODO OW: notifications are coming, needs clarification on state transition (run())
  • TODO OW: possibly start working with Scott Michael
  • TODO ALL: take over early responses on mailing list
  • TODO ALL: keep discussion open on callbacks etc.