Skip to content

Commit

Permalink
Ba/add config lint helper (#338)
Browse files Browse the repository at this point in the history
* move config.example.json

* update config example path in readme

* add lint_config helper script

* update changelog

* remove example config form push

* try fix pipeline

* try fix pipe again
  • Loading branch information
bradleyandrick authored Feb 28, 2024
1 parent c0c9238 commit 753ceb7
Show file tree
Hide file tree
Showing 5 changed files with 138 additions and 7 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/push.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
- name: Run audit (prod, no exclusions)
run: npm run audit-prod
- name: Copy config file
run: cp ./public/config/config.example.json ./public/config/config.json
run: mkdir -p ./public/config && cp config_helper/config.example.json ./public/config/config.json
- name: Run unit tests
run: npm run test
- name: Run dev build
Expand Down
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,16 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).

## Unreleased

### Added

- Added `lint_config.py` helper script to assist in validating `config.json` files.

### Changed

- Moved example config out of public so it doesn't get added to build

## 5.0.0 - 2024-02-27

### Added
Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ Sentinel-2 L2A Mosaic View
### Configuration File

For local development, you should create a `./public/config/config.json` file with appropriate variables outlined in the table below.
The file `./public/config/config.example.json` is included in this repository as representative file structure.
The file `config_helper/config.example.json` is included in this repository as representative file structure.

> NOTE: This project uses a "build-once, deploy-anywhere" approach with config variables. The config is read on application load by a fetch to the `/config/config.json` path.
> There is a cachebreaker included in the request to prevent stale config files from being used.
Expand All @@ -84,21 +84,21 @@ The file `./public/config/config.example.json` is included in this repository as
| DEFAULT_COLLECTION | Default collection option for collection dropdown | Optional |
| COLLECTIONS | Array of strings listing collections to show in dropdown. This is used to filter the collections endpoint from the list fetched from the `STAC_API_URL` defined in the config. Collection property of `id` must be used. If set, only the matched collections will show in the app. If not set, all collections in the STAC API will show in dropdown. | Optional |
| SCENE_TILER_URL | URL for map tiling | Optional |
| SCENE_TILER_PARAMS | Per-collection configuration of TiTiler `assets`, `color_formula`, `bidx`, `rescale`, `expression`, and `colormap_name` parameters. Example in [config.example.json](./public/config/config.example.json) | Optional |
| SCENE_TILER_PARAMS | Per-collection configuration of TiTiler `assets`, `color_formula`, `bidx`, `rescale`, `expression`, and `colormap_name` parameters. Example in [config.example.json](config_helper/config.example.json) | Optional |
| MOSAIC_MIN_ZOOM_LEVEL | Minimum zoom level for mosaic view search results. If not set, the default zoom level will be 7. | Optional |
| ACTION_BUTTON | Button text and redirect URL used to link to external website as a prominent call to action. If not set, the button will not be visible. Should be an object with `text` and `url` keys. Example: [config.example.json](./public/config/config.example.json). | Optional |
| ACTION_BUTTON | Button text and redirect URL used to link to external website as a prominent call to action. If not set, the button will not be visible. Should be an object with `text` and `url` keys. Example: [config.example.json](config_helper/config.example.json). | Optional |
| MOSAIC_TILER_URL | URL for mosaic tiling. If not set, the View Mode selector will not be visible. The app requires the use of the [NASA IMPACT TiTiler fork](https://github.com/NASA-IMPACT/titiler) as it contains the mosaicjson endpoints needed. | Optional |
| MOSAIC_TILER_PARAMS | Per-collection configuration of TiTiler mosaic `assets`, `color_formula`, `bidx`, `rescale`, `expression`, and `colormap_name` parameters. Example in [config.example.json](./public/config/config.example.json) | Optional |
| MOSAIC_TILER_PARAMS | Per-collection configuration of TiTiler mosaic `assets`, `color_formula`, `bidx`, `rescale`, `expression`, and `colormap_name` parameters. Example in [config.example.json](config_helper/config.example.json) | Optional |
| MOSAIC_MAX_ITEMS | Maximum number of items in mosaic. If not set, the default max items will be 100. | Optional |
| SEARCH_MIN_ZOOM_LEVELS | Per-collection configuration for minimum zoom levels needed for grid code aggregated results (medium zoom level) and single scene search results (high zoom level). Example: [config.example.json](./public/config/config.example.json). If no grid code aggregation, set value for `medium` to be the same value as `high` and hex aggregations will be used until the zoom level is reached when individual scenes become available. | Optional |
| SEARCH_MIN_ZOOM_LEVELS | Per-collection configuration for minimum zoom levels needed for grid code aggregated results (medium zoom level) and single scene search results (high zoom level). Example: [config.example.json](config_helper/config.example.json). If no grid code aggregation, set value for `medium` to be the same value as `high` and hex aggregations will be used until the zoom level is reached when individual scenes become available. | Optional |
| CONFIG_COLORMAP | Color map used in low level hex grid search results. Complete list of colormaps are available here: [bpostlethwaite/colormap](https://github.com/bpostlethwaite/colormap). If not set, the default colormap will be "viridis". | Optional |
| BASEMAP_URL | URL to specify a basemap provider used by the leaflet map. Must be a raster tile provider as vector tiles are not supported. If not set, the default colormap will be `https://tile.openstreetmap.org/{z}/{x}/{y}.png`. | Optional |
| BASEMAP_DARK_THEME | Boolean value. If set to `true` or not included in config, a dark theme is applied to the basemap. If set to `false`, the dark theme will not be applied to basemap and the default basemap provider style is used. | Optional |
| BASEMAP_HTML_ATTRIBUTION | String of HTML markup used to set the attribution for the basemap provider used by the leaflet map. Markup is sanitized prior to render with `DOMPurify` and only is retricted to only allow `html`, `'a' tags`, and `'href'` and `'target'` attributes. Custom attribution will not render if `BASEMAP_URL` is not also set. If not set, the default attribution will be `&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a>`. (Note: Raw HTML was used here since attribution is non-standardized.) | Optional |
| SEARCH_BY_GEOM_ENABLED | If set to `true` search options will render and allow users to draw or upload a geojson file to use as search bounds. | Optional |
| CART_ENABLED | If set to `true` cart features will be enabled. These include: rendering cart button in search controls bar, adding cart management buttons to popup results, render buttons in messages to quickly add some or all scenes to cart after search completes. | Optional |
| SHOW_BRAND_LOGO | If set to `true` filmdrop brand logo and clickable hyperlink are rendered at the top right of the UI. If not set or `false`, the logo will not be visible. | Optional |
| POPUP_DISPLAY_FIELDS | Per-collection configuration of popup metadata fields properies to render. Example in [config.example.json](./public/config/config.example.json). Only `Title` field (which maps to the `id` property for STAC items) is rendered if collection used in application but not included in configuration. | Optional |
| POPUP_DISPLAY_FIELDS | Per-collection configuration of popup metadata fields properies to render. Example in [config.example.json](config_helper/config.example.json). Only `Title` field (which maps to the `id` property for STAC items) is rendered if collection used in application but not included in configuration. | Optional |
| APP_NAME | String value used for html title and anywhere else that the text value for app name is used. If not set, default value of `FilmDrop Console` will be used. | Optional |
| APP_FAVICON | If set, custom application favicon is used instead of default FilmDrop favicon. Favicon file of format `.ico` OR `.png` must be used and file must exist next to config in `/config` of the built deployment directory. Place in `public` directory during local development, but can also be added or adjusted post depolyment. File name in `config.json` must match extactly with file in config, see `config.example.json` for example. If not set or error in config/file, default FilmDrop favicon will be used. | Optional |
| MAP_ZOOM | If set, starting map zoom level is set to this integer value. If not set, default value of `3` will be used. | Optional |
Expand Down
File renamed without changes.
121 changes: 121 additions & 0 deletions config_helper/lint_config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
"""
Usage: python3 lint_config.py path/to/config.json
Purpose: Lints a config.json configuration file used by a depolyment of the Filmdrop UI application (https://github.com/Element84/filmdrop-ui).
Checks for missing required keys, extra keys, type errors, and optional keys not included.
"""

import sys
import json
import os

def lint_config(file_path):
# Read the config file
try:
with open(file_path, 'r') as file:
config = json.load(file)
except FileNotFoundError:
print(f"File not found: {file_path}")
sys.exit(1)
except json.JSONDecodeError as e:
print(f"Error parsing JSON: {e}")
sys.exit(1)

# Define the expected keys and their types
expected_keys = {
"STAC_API_URL": str,
"PUBLIC_URL": str,
"LOGO_URL": str,
"LOGO_ALT": str,
"DASHBOARD_BTN_URL": str,
"ANALYZE_BTN_URL": str,
"API_MAX_ITEMS": int,
"DEFAULT_COLLECTION": str,
"COLLECTIONS": list,
"SCENE_TILER_URL": str,
"SCENE_TILER_PARAMS": dict,
"MOSAIC_MIN_ZOOM_LEVEL": int,
"ACTION_BUTTON": dict,
"MOSAIC_TILER_URL": str,
"MOSAIC_TILER_PARAMS": dict,
"MOSAIC_MAX_ITEMS": int,
"SEARCH_MIN_ZOOM_LEVELS": dict,
"CONFIG_COLORMAP": str,
"BASEMAP_URL": str,
"BASEMAP_DARK_THEME": bool,
"BASEMAP_HTML_ATTRIBUTION": str,
"SEARCH_BY_GEOM_ENABLED": bool,
"CART_ENABLED": bool,
"SHOW_BRAND_LOGO": bool,
"POPUP_DISPLAY_FIELDS": dict,
"APP_NAME": str,
"APP_FAVICON": str,
"MAP_ZOOM": int,
"MAP_CENTER": list,
"LAYER_LIST_ENABLED": bool,
"LAYER_LIST_SERVICES": list,
"STAC_LINK_ENABLED": bool,
"SHOW_ITEM_AUTO_ZOOM": bool,
}

print("*********************************************************************")
print("**************** Running Filmdrop UI Config Lint ********************")
print("*********************************************************************")

# Check for missing required keys
required_keys = ["STAC_API_URL"]
missing_required_keys = [key for key in required_keys if key not in config]
if missing_required_keys:
print("Required key(s) missing:")
for key in missing_required_keys:
print(f" - {key}")
print("************************************")

# Check for extra keys that can't be used
extra_keys = [key for key in config.keys() if key not in expected_keys]
if extra_keys:
print("Extra key(s) found that can't be used:")
for key in extra_keys:
print(f" - {key}")
print("************************************")

# Check for optional keys not included
optional_keys = [key for key in expected_keys.keys() if key not in config]
if optional_keys:
print("Optional key(s) not included:")
for key in optional_keys:
print(f" - {key}")
print("************************************")

# Check for type errors
for key, expected_type in expected_keys.items():
if key in config and not isinstance(config[key], expected_type):
print(f"Type error for key '{key}': expected {expected_type.__name__}, got {type(config[key]).__name__}")
print("************************************")

# Perform additional validations as needed

# If everything looks good
if not missing_required_keys and not extra_keys:
print("Configuration looks good!")

if __name__ == "__main__":
# Get the file path from command line arguments
if len(sys.argv) != 2:
print("Usage: ./lint_config.py path/to/config.json")
sys.exit(1)

file_path = sys.argv[1]

# Check if the file is a JSON file
if not file_path.endswith(".json"):
print("Invalid file format. Expected a JSON file.")
sys.exit(1)

# Check if the file exists
if not os.path.exists(file_path):
print(f"File not found: {file_path}")
sys.exit(1)

# Lint the config file
lint_config(file_path)

0 comments on commit 753ceb7

Please sign in to comment.