Testing

Goals

We setup the automated Github tests to comprehensively test all pull requests for code merging into the ngen master branch to ensure the commited codes are working correctly without any inadvertently human introduced error. The tests are setup for Linux and MacOS operating systems as of this writing.

Github Actions

All pull requests for code merging into the master branch are tested in Github by running the comprehensive automated test, using Github action and work flow codes. The Github action codes consist of ngen_build, ngen_submodule_build, and clean_build. They are responsible for settting up inputs and outputs, create the general build environment and command structure for building libraries and executable, including dependencies.

The ngen-build actions install MPI, NetCDF, UDUNITS, Boost, Python and dependencies, define compilers flags and download submodules. Then, the action codes setup the general command to build the target executable using cmake with various user defined options, which are managed by CMakelist.txt in the project and subdirectories. The ngen-submodule-build works similar to ngen-build, but only builds a single submodule library, so is also simpler. clean-build actions clean up the build directory.

Github Workflow

Two classes of tests are performed in the workflow.

1. test_and_validation: for this class of tests, the tests cover all the component codes of ngen, or the unit tests. The workflow uses ngen-build and/or ngen-submodule-build to build the target executable, run the executable, then clean up and remove the build directory. This test class covers all the functional components of ngen codes to ensure that they all work as intended without error. In addition, since the ngen framework uses Basic Model Interface (BMI), all BMI related finctionalities are tested.

2. module_integration: This is carried out by running the ngen executable with example realization configuration input This test validates the ngen framework as a whole works correctly as expected. The workflow for this class of automated tests involves checkout and build a number of submodules as dynamic libraries that can be automatically linked in when running ngen. Libraries that are used are defined in the realization condifguration file and the path provided. The main executable, ngen, is checked out and built utilizing the environment and commands defined in ngen-build actions. This produce a target executable, ngen. The test is performed by running ngen with appropriate input hydrofabric data and realization configuration file. The current workflow run a couple of tests with different realization configuration files to cover enough real use cases as well as to be able to complete the automated tests in a reasonable amout of time. For details of running ngen test manually on your local machine, see Quickstart, Building, and Usage sections. In this workflow, we also test ngen integrated with t-route. For detail, see Routing section.