Architectural enhancements (teamengine v5)
This enhancement proposal suggests an architecture refactoring and the adoption of a standard test reporting language.
Split the current web application into two components:
- a Manager that
- manages makes call to a Test Execution Service
- manages the users logs (database)
- invokes a reporter service
- a "Reporter service" that
- parses the EARL report and creates and presents human-readable reports (e.g. HTML5, PDF) to end users or XML/RDF to clients
- a "Test Execution Service" (TES) that provides a RESTful API for invoking (and managing) test suites
The validator would use the TES, of course, but other software agents (including mobile clients) could do the same.
I recommend adopting the W3C Evaluation and Report Language (EARL) 1.0 Schema, an RDF vocabulary for describing test results:
- Evaluation and Report Language (EARL) 1.0 Schema
- Developer Guide for Evaluation and Report Language (EARL) 1.0
The existing working draft hasn't been updated in a while, but it looks like the Evaluation and Repair Tools Working Group (ERT WG) is seeking to renew its charter. Note that advancing EARL to Recommendation status is one of its goals. The OGC could be an early adopter ;-)
Most test runners have their own custom result formats (usually based on XML), so it would be necessary to transform these to EARL test results (RDF/XML syntax). For example, using TestNG one could implement a custom listener or reporter to do this. The Validator front-end would then take these raw test results and produce something easy on the eyes for human readers :-)