Roadmap

• Milestone: 0.11.1

  6 weeks late (10/15/09)

  Bugfix release for autotest 0.11.X series.

• Milestone: 1.0.0

  No date set

  Autotest v1.0 Criteria

  So, if you're area lead, I suggest you take a look at these notes when working on your roadmap.

  Web frontend (TKO + AFE)

  • Tko Requirements
  • New reporting interface (TKO) _(2 months)_
    • Stage 1 - Includes all features of existing spreadsheet report _(1.5 weeks)_
    • Stage 1 - Full SQL filtering capability _(1 day)_
    • Stage 1 - Row and column field selection
    • Stage 1 - Left-click menu with drilldown options
    • Stage 1 - Floating headers
    • Stage 2 - Right-click menu with drilldown options
    • Stage 2 - Multiple cell selection
    • Stage 2 - Powerful test labeling system _(1 week)_
    • Stage 3 - Table-based view for easy job triage _(2 weeks)_
    • Stage 4 - Saved queries per user but shareable _(4 days)_
    • Stage 4 - Test detail view to help users more easily find failure details on their own () _(3 days)_
    • Stage 4 - Composite headers
    • Stage 5 - Intuitive filtering widgets for newer users (POSTPONED to V1.1)
    • Stage 5 - Conversion to SQL with custom editing allowed. (POSTPONED to V1.1)
  • Job creation and tracking frontend (AFE) (/) _(2 weeks)_
    • Parsing and using information from control files
    • Complete and proper ACL support _(3 days)_
    • Creating jobs using previous jobs as templates _(2 days)_
    • Easier management of many jobs/hosts - Adding checkboxes for abort () _(3 days)_
    • Better linking to raw job and host logs () _(1 day)_
  • Unification of frontend data _(1 week)_
    • Databases merged into one (POSTPONED to V1.1)
    • TKO able to include queued and running tests in reports
      • Proper implementation dependent on database unification. (POSTPONED to V1.1)
      • Workaround to provide more information. Necessary for NPI according to.
        • Running: Can implement as desired.
        • Queued: Will implement a status bar at the job level.
        • Queued: Find a way to indicate whether a test has been queued. (POSTPONED to V1.1)
  • Graphing
    • Simple frontend for existing graphing scripts () _(3 days)_
    • Requirements gathering for more sophisticated graphing capabilities ()

  Parser

  • *Status: done*
  • Refactor the parser to enable multiple versions
  • Make the parser a library called by autoserv
  • Update the parser/autoserv to adhere to a stricter spec for status files

  Scheduler

  • *Goal: Support 3000 machines running simulataneously.*
  • Job throttling, tunable via the global config
  • Job timeouts _(2 days)_
  • When machines are dead, do basic triage, then (somehow) send for repairs _(1 week)_
    • Automated repairs ()
    • Determine process for sending machines to repairs. Requires coordination with customers. (POSTPONED to V1.1)
  • Use Django database models (Need to reevaluate this requirement. POSTPONED to V1.1)
  • More user-friendly status messages () _(3 days)_
  • Reinstall without verify () _(1 day)_
  • Having the repair phase going to reinstall the machine ()

  Testing Autotest

  • *Status: In progress.*
  • *Goal: Test 35%-40% of the code.*
  • Unit tests (33% coverage)
    • Everything needs to be testable
    • Need to plan one or more fix-it days
    • All new code must have unit-tests
    • Modifications within testable code should include unit tests
  • Functional tests
    • Automate functional tests. (POSTPONED to V1.1)
  • Automating unit tests
    • Presubmit stuff?
    • Nightly/continuous (build) testing?
    • Dedicated machines for running tests?

  General Code Cleanup

  • Tabs -> spaces
  • Cleanup imports

  Packaging

  • *Status: In progress*
    • Create a program to package and upload tests, gcc, kernels, etc
    • Make it so job.run_test can fetch packages when they don't exist
    • Make it so that tests can specify other dependencies
    • Decrease the dependency on LOAS
  • Refactor the GFS upload/fetch code to be the site specific component of the packaging system _(1 week)_
  • Create a generic tool for packaging and uploading tests, dependencies (like GCC), kernels, etc _(3 days)_
  • Connect tests, deps, kernels, and profilers into the new packaging system _(1 day each)_
  • Debug packaging issues.
  • Handle failures better _(2 Days)_
  • Write unit tests _(2 Days)_

  New CLI

  • Replace old CLI:
    • Done for acl, label, user, host
    • To do:
      • job management
      • results management (will go through RPC layer, waiting for its definition)
  • Maintain feature parity with web frontend
  • Include a 'contrib' section with users' scripts