Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 0.0.1 #26

Merged
merged 63 commits into from
Mar 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
289ed97
re-organize for landsat
jhkennedy Feb 29, 2024
7cb3071
Add some basic niceties to the repo
jhkennedy Feb 29, 2024
09f3fa8
Merge pull request #2 from ASFHyP3/update-lambda
jhkennedy Feb 29, 2024
6dd7ed7
Merge pull request #3 from ASFHyP3/niceties
jhkennedy Feb 29, 2024
f76d9e7
ruff and cf-lint static analysis
jhkennedy Feb 29, 2024
84d4397
setup dependabot
jhkennedy Feb 29, 2024
8779b80
setup makefile
jhkennedy Feb 29, 2024
c82c8a0
fix ruff's complaints
jhkennedy Feb 29, 2024
142f5f8
whitespace
jhkennedy Feb 29, 2024
266f54d
Merge pull request #4 from ASFHyP3/ruff
jhkennedy Feb 29, 2024
f0200c5
Add in landsat functionality
jhkennedy Feb 29, 2024
2694bf5
whitespace
jhkennedy Feb 29, 2024
74e5537
remove todo comments in favor of issues
jhkennedy Feb 29, 2024
faf0fde
Apply suggestions from code review
jhkennedy Mar 1, 2024
98faa44
Merge pull request #5 from ASFHyP3/landsat-part-deux
jhkennedy Mar 1, 2024
3a914c3
prep for deploy
jhkennedy Mar 1, 2024
7c8f144
add units to vairable
jhkennedy Mar 1, 2024
c590610
use reference scene name as project name
jhkennedy Mar 1, 2024
935dffc
Add Prod deployment
jhkennedy Mar 1, 2024
fda2308
check in tile list and only load it once
jhkennedy Mar 1, 2024
2bd4709
remove unused variable
jhkennedy Mar 1, 2024
f3d3ac5
setup test SNS cloudformation, integrations test info
jhkennedy Mar 2, 2024
74523f7
Flesh out README
jhkennedy Mar 2, 2024
e963f9a
pin deployment deps
jhkennedy Mar 2, 2024
15124b7
ruff formatting
jhkennedy Mar 2, 2024
6c595ae
fix typo
jhkennedy Mar 2, 2024
3122e56
Update landsat/cloudformation.yml
jhkennedy Mar 4, 2024
a0fdcef
rename landsat tile list for clarity
jhkennedy Mar 4, 2024
afd1e96
Just HyP3 API is sufficient
jhkennedy Mar 4, 2024
8e5547e
I think lamnda sets up the logging config for us...
jhkennedy Mar 4, 2024
8866081
don't need an sns cf tempate
jhkennedy Mar 4, 2024
15bbafc
add JPL RolesAsCode to CF
jhkennedy Mar 4, 2024
4374c86
Add test topic arn
jhkennedy Mar 4, 2024
2e54f8c
reaname topic arn to landsat topic arn
jhkennedy Mar 4, 2024
ac45cb9
cloudformation role arn isn't needed for JPL
jhkennedy Mar 4, 2024
7696c7c
readme revisions
asjohnston-asf Mar 4, 2024
1e69019
Add deploy environments
jhkennedy Mar 4, 2024
c643abd
Merge pull request #12 from ASFHyP3/readme-tweaks
jhkennedy Mar 4, 2024
448f8d7
set logging level from env variables
jhkennedy Mar 4, 2024
b2683e7
Merge pull request #11 from ASFHyP3/roles-as-code
jhkennedy Mar 4, 2024
ba84946
Merge pull request #10 from ASFHyP3/deploy
jhkennedy Mar 4, 2024
0a3250b
fix path to cloudformation
jhkennedy Mar 4, 2024
3663e05
Merge pull request #13 from ASFHyP3/fix-deploy
jhkennedy Mar 4, 2024
897031b
update integration test payloads
asjohnston-asf Mar 6, 2024
aa48555
add trailing newlines
asjohnston-asf Mar 6, 2024
1ae7e8f
Merge pull request #15 from ASFHyP3/test-cases
asjohnston-asf Mar 6, 2024
fe34a76
refine criteria for which reference/secondary scenes qualify for proc…
asjohnston-asf Mar 6, 2024
a8797fe
add assert to catch non-c2l1 scenes
asjohnston-asf Mar 6, 2024
4209538
ruff formatting
asjohnston-asf Mar 6, 2024
529ee20
add dead letter queue as an offramp for messages that cannot be proce…
asjohnston-asf Mar 6, 2024
2438929
Merge pull request #17 from ASFHyP3/scene-filters
asjohnston-asf Mar 6, 2024
71f44d6
merge branch develop into partial-batch
asjohnston-asf Mar 6, 2024
8c6f907
implement batchItemFailures so only failures are returned to the queue
asjohnston-asf Mar 6, 2024
ff13d46
adjust datetime when querying secondary scenes to exclude reference s…
asjohnston-asf Mar 6, 2024
40b6f5d
ruff formatting
asjohnston-asf Mar 6, 2024
c7ac4fb
Merge pull request #18 from ASFHyP3/dlq
asjohnston-asf Mar 6, 2024
f8c6ad9
Merge pull request #19 from ASFHyP3/partial-batch
asjohnston-asf Mar 6, 2024
281915d
fix environment variable name mismatch
asjohnston-asf Mar 7, 2024
604c42b
Merge pull request #23 from ASFHyP3/log-level
asjohnston-asf Mar 7, 2024
8734d0f
prune to 50 random landsat tiles for production pilot
asjohnston-asf Mar 8, 2024
035a562
Merge pull request #24 from ASFHyP3/tile-subset
asjohnston-asf Mar 8, 2024
fac6a98
add changelog and github workflows for initial release
asjohnston-asf Mar 8, 2024
c90e6f7
Merge pull request #25 from ASFHyP3/release-setup
asjohnston-asf Mar 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# These owners will be requested for review when someone opens a pull request.
* @ASFHyP3/platform
58 changes: 58 additions & 0 deletions .github/actions/deploy/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: Deploy ITS_LIVE Monitoring
description: Deploy the ITS_LIVE Monitoring system

inputs:
STACK_NAME:
required: true
AWS_ACCESS_KEY_ID:
required: true
AWS_SECRET_ACCESS_KEY:
required: true
CF_TEMPLATE_BUCKET:
required: true
LANDSAT_TOPIC_ARN:
required: true
HYP3_API:
required: true
LAMBDA_LOGGING_LEVEL:
required: true
EARTHDATA_USERNAME:
required: true
EARTHDATA_PASSWORD:
required: true

runs:
using: composite
steps:
- uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ inputs.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ inputs.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2

- uses: actions/setup-python@v3
with:
python-version: 3.12

- name: Install
shell: bash
run: make install-lambda-deps

- name: Package and deploy
shell: bash
run: |
aws cloudformation package \
--template-file landsat/cloudformation.yml \
--s3-bucket=${{ inputs.CF_TEMPLATE_BUCKET }} \
--output-template-file packaged.yml

aws cloudformation deploy \
--template-file packaged.yml \
--stack-name=${{ inputs.STACK_NAME }} \
--capabilities CAPABILITY_IAM \
--parameter-overrides \
LandsatTopicArn=${{ inputs.LANDSAT_TOPIC_ARN }} \
Hyp3Api=${{ inputs.HYP3_API }} \
LambdaLoggingLevel=${{ inputs.LAMBDA_LOGGING_LEVEL }} \
EarthdataUsername=${{ inputs.EARTHDATA_USERNAME }} \
EarthdataPassword=${{ inputs.EARTHDATA_PASSWORD }}
19 changes: 19 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file

version: 2
updates:
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "daily"
labels:
- "bumpless"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"
labels:
- "bumpless"
18 changes: 18 additions & 0 deletions .github/workflows/changelog.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: Changelog updated?

on:
pull_request:
types:
- opened
- labeled
- unlabeled
- synchronize
branches:
- main
- develop

jobs:
call-changelog-check-workflow:
uses: ASFHyP3/actions/.github/workflows/[email protected]
secrets:
USER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
34 changes: 34 additions & 0 deletions .github/workflows/deploy-prod.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Deploy ITS_LIVE Monitoring Production

on:
push:
branches:
- main

concurrency: deploy-prod

jobs:
deploy:
environment: prod
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/deploy
with:
STACK_NAME: its-live-monitoring-prod
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
EARTHDATA_USERNAME: ${{ secrets.EARTHDATA_USERNAME }}
EARTHDATA_PASSWORD: ${{ secrets.EARTHDATA_PASSWORD }}
CF_TEMPLATE_BUCKET: cf-templates-3o5lnspmwmzg-us-west-2
LANDSAT_TOPIC_ARN: arn:aws:sns:us-west-2:673253540267:public-c2-notify-v2
HYP3_API: https://hyp3-its-live.asf.alaska.edu
LAMBDA_LOGGING_LEVEL: INFO


call-bump-version-workflow:
needs: deploy
uses: ASFHyP3/actions/.github/workflows/[email protected]
secrets:
USER_TOKEN: ${{ secrets.TOOLS_BOT_PAK }}
27 changes: 27 additions & 0 deletions .github/workflows/deploy-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: Deploy ITS_LIVE Monitoring Test

on:
push:
branches:
- develop

concurrency: deploy-test

jobs:
deploy:
environment: test
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/deploy
with:
STACK_NAME: its-live-monitoring-test
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
EARTHDATA_USERNAME: ${{ secrets.EARTHDATA_USERNAME }}
EARTHDATA_PASSWORD: ${{ secrets.EARTHDATA_PASSWORD }}
CF_TEMPLATE_BUCKET: cf-templates-3o5lnspmwmzg-us-west-2
LANDSAT_TOPIC_ARN: arn:aws:sns:us-west-2:986442313181:its-live-notify-test
HYP3_API: https://hyp3-its-live.asf.alaska.edu
LAMBDA_LOGGING_LEVEL: DEBUG
15 changes: 15 additions & 0 deletions .github/workflows/labeled-pr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: Is PR labeled?

on:
pull_request:
types:
- opened
- labeled
- unlabeled
- synchronize
branches:
- main

jobs:
call-labeled-pr-check-workflow:
uses: ASFHyP3/actions/.github/workflows/[email protected]
16 changes: 16 additions & 0 deletions .github/workflows/release-checklist-comment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: Create Release Comment

on:
pull_request:
types:
- opened
branches:
- main

jobs:
call-release-workflow:
uses: ASFHyP3/actions/.github/workflows/[email protected]
permissions:
pull-requests: write
secrets:
USER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
14 changes: 14 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
name: Create Release

on:
push:
tags:
- 'v*'

jobs:
call-release-workflow:
uses: ASFHyP3/actions/.github/workflows/[email protected]
with:
release_prefix: its-live-monitoring
secrets:
USER_TOKEN: ${{ secrets.TOOLS_BOT_PAK }}
17 changes: 17 additions & 0 deletions .github/workflows/static-anaysis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: Static analysis

on: push

jobs:
call-ruff-workflow:
uses: ASFHyP3/actions/.github/workflows/[email protected]

cfn-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: 3.12
- run: make install
- run: make cfn-lint
Loading
Loading