Tests don't depend on external files (#1162)

* Switch from `pytest-vcr` to `pytest-recording`

* Add markers for validation tests

* Add vcr markers and update tests

* Enforce no network on CI

* Write all cassettes in a urllib3-free environment

* Tidy up cassettes and local validation

* Add vcr to docs

* Update changelog

* Mark new extension tests with vcr and write cassettes

CHANGELOG.md

@@ -42,6 +42,7 @@
 - `add_child` and `add_item` return a Link object instead of None ([#1160](https://github.com/stac-utils/pystac/pull/1160))
 - `add_children` and `add_items` return a list of Link objects instead of None ([#1160](https://github.com/stac-utils/pystac/pull/1160))
 - Include collection assets in `make_all_asset_hrefs_relative/absolute` ([#1168](https://github.com/stac-utils/pystac/pull/1168))
+- Use cassettes for all tests that pull files from remote ([#1162](https://github.com/stac-utils/pystac/pull/1162))
 
 ### Deprecated
 

RELEASING.md

@@ -6,26 +6,27 @@ This is a checklist to use when releasing a new PySTAC version.
 2. Create a release branch with the name `release/vX.Y.Z`, where `X.Y.Z` is the next version (e.g. `1.7.0`).
 3. Pull fields-normalized.json from cdn: run `scripts/pull-static`. Note you will need to have [jq](https://stedolan.github.io/jq/) installed.
 4. Update the `__version__` attribute in `pystac/version.py` with the new version.
-5. Update the CHANGELOG.
+5. Update all cassettes: `pytest --record-mode rewrite`
+6. Update the CHANGELOG.
     - Create a new header below `## [Unreleased]` with the new version.
     - Remove any unused header sections.
     - Update the links at the bottom of the page for the new header.
     - Audit the CHANGELOG for correctness and readability.
-6. Audit the changes.
+7. Audit the changes.
    Use the CHANGELOG, your favorite diff tool, and the merged Github pull requests to ensure that:
     - All notable changes are captured in the CHANGELOG.
     - The type of release is appropriate for the new version number, i.e. if there are breaking changes, the MAJOR version number must be increased.
     - All deprecated items that were marked for removal in this version are removed.
-7. Craft draft release notes (<https://github.com/stac-utils/pystac/releases/new>).
+8. Craft draft release notes (<https://github.com/stac-utils/pystac/releases/new>).
    These should be short, readable, and call out any significant changes, especially changes in default behavior or significant new features.
    These should also include a link back to the Github milestone for this release, if there is one.
    These should _not_ be a complete listing of changes -- those will be auto-generated later, after the tag is pushed.
-8. Commit your changes, push your branch to Github, and request a review.
-9. Once approved, merge the PR.
-10. Once the PR is merged, create a tag with the version name, e.g. `vX.Y.Z`.
+9. Commit your changes, push your branch to Github, and request a review.
+10. Once approved, merge the PR.
+11. Once the PR is merged, create a tag with the version name, e.g. `vX.Y.Z`.
    Prefer a signed tag, if possible.
    Push the tag to Github.
-11. Use the tag to finish your release notes, and publish those.
+12. Use the tag to finish your release notes, and publish those.
     The "auto generate" feature is your friend, here.
     When the release is published, this will trigger the build and release on PyPI.
-12. Announced the release in [Gitter](https://matrix.to/#/#SpatioTemporal-Asset-Catalog_python:gitter.im) and on any relevant social media.
+13. Announced the release in [Gitter](https://matrix.to/#/#SpatioTemporal-Asset-Catalog_python:gitter.im) and on any relevant social media.

docs/contributing.rst

@@ -18,33 +18,32 @@ pip as follows:
 
 Testing
 ^^^^^^^
-PySTAC runs tests using ``pytest``. You can find unit tests in the ``tests/``
+tl;dr: Run ``./scripts/test`` to run all tests as they run on CI.
+
+PySTAC runs tests using `pytest <https://docs.pytest.org/en/latest/>`_. You can find unit tests in the ``tests/``
 directory.
 
-Run a single test with:
+To run the tests and generate the coverage report:
 
 .. code-block:: bash
 
-    pytest tests/test_catalog.py::CatalogTest::test_create_and_read
-
-or an entire folder using:
+    $ pytest -v -s --block-network --cov pystac --cov-report term-missing
 
-.. code-block:: bash
+To view the coverage report, you can run
+`coverage report` (to view the report in the terminal) or `coverage html` (to generate
+an HTML report that can be opened in a browser).
 
-    pytest tests/extensions
+The PySTAC tests use `vcrpy <https://vcrpy.readthedocs.io/en/latest/>`_ to mock API calls
+with "pre-recorded" API responses. This often comes up when testing validation.
 
-or the entire project using:
+When adding new tests that require pulling remote files use the ``@pytest.mark.vcr``
+decorator. Record the new responses and commit them to the repository.
 
 .. code-block:: bash
 
-    ./scripts/test
-
-The last command will also check test coverage. To view the coverage report, you can run
-`coverage report` (to view the report in the terminal) or `coverage html` (to generate
-an HTML report that can be opened in a browser).
-
-More details on using ``pytest`` are `here
-<https://docs.pytest.org>`_.
+    $ pytest -v -s --record-mode new_episodes
+    $ git add <new files here>
+    $ git commit -a -m 'new test episodes'
 
 Code quality checks
 ^^^^^^^^^^^^^^^^^^^

pyproject.toml

@@ -62,7 +62,7 @@ test = [
     "pre-commit~=3.2",
     "pytest-cov~=4.0",
     "pytest-mock~=3.10",
-    "pytest-vcr~=1.0",
+    "pytest-recording~=0.12",
     "pytest~=7.3",
     "ruff==0.0.274",
     "types-html5lib~=1.1",

pystac/validation/local_validator.py

@@ -38,24 +38,15 @@ def _validate_from_local(
             )
         return list(validator.iter_errors(stac_dict))
 
-    @staticmethod
-    def catalog_validator(version: str = VERSION) -> Draft7Validator:
-        schema = _read_schema(f"stac-spec/v{version}/catalog.json")
-        return Draft7Validator(schema)
-
-    @staticmethod
-    def collection_validator(version: str = VERSION) -> Draft7Validator:
-        schema = _read_schema(f"stac-spec/v{version}/collection.json")
+    def _validator(self, stac_type: str, version: str) -> Draft7Validator:
+        schema = _read_schema(f"stac-spec/v{version}/{stac_type}.json")
         resolver = RefResolver.from_schema(schema)
+        resolver.store[
+            f"https://schemas.stacspec.org/v{version}/collection-spec/json-schema/collection.json"
+        ] = _read_schema(f"stac-spec/v{version}/collection.json")
         resolver.store[
             f"https://schemas.stacspec.org/v{version}/item-spec/json-schema/item.json"
         ] = _read_schema(f"stac-spec/v{version}/item.json")
-        return Draft7Validator(schema, resolver=resolver)
-
-    @staticmethod
-    def item_validator(version: str = VERSION) -> Draft7Validator:
-        schema = _read_schema(f"stac-spec/v{version}/item.json")
-        resolver = RefResolver.from_schema(schema)
         for name in ("Feature", "Geometry"):
             resolver.store[f"https://geojson.org/schema/{name}.json"] = _read_schema(
                 f"geojson/{name}.json"
@@ -66,6 +57,15 @@ def item_validator(version: str = VERSION) -> Draft7Validator:
             ] = _read_schema(f"stac-spec/v{version}/{name}.json")
         return Draft7Validator(schema, resolver=resolver)
 
+    def catalog_validator(self, version: str = VERSION) -> Draft7Validator:
+        return self._validator("catalog", version)
+
+    def collection_validator(self, version: str = VERSION) -> Draft7Validator:
+        return self._validator("collection", version)
+
+    def item_validator(self, version: str = VERSION) -> Draft7Validator:
+        return self._validator("item", version)
+
 
 def _read_schema(file_name: str) -> Dict[str, Any]:
     with importlib_resources_files("pystac.validation.jsonschemas").joinpath(

scripts/test

@@ -9,10 +9,10 @@ echo
 
 if [[ -z ${CI} || -n ${CHECK_COVERAGE} ]]; then
     echo " -- RUNNING UNIT TESTS (WITH COVERAGE) --"
-    pytest tests --cov
+    pytest tests --block-network --record-mode=none --cov
 else
     echo " -- RUNNING UNIT TESTS (WITHOUT COVERAGE) --"
-    pytest tests
+    pytest tests --block-network --record-mode=none
 fi
 
 echo