-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Update account IDs when registrations become historical. (bcgov#1954)
Signed-off-by: Doug Lovett <[email protected]>
- Loading branch information
1 parent
aa83ffc
commit ce98a2f
Showing
25 changed files
with
988 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
Dockerfile | ||
.dockerignore | ||
.pytest_cache | ||
__pycache__ | ||
db | ||
venv | ||
.env | ||
.env* | ||
.eggs | ||
Makefile | ||
devops | ||
k8s | ||
manage.py | ||
migrations | ||
pre-hook-update-db.sh | ||
requirements | ||
test_data | ||
tests |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
.pytest_cache | ||
__pycache__ | ||
|
||
venv | ||
|
||
streaming_pull.py | ||
test_run.py | ||
|
||
.env | ||
.env* | ||
.eggs | ||
Makefile | ||
devops | ||
k8s | ||
manage.py | ||
migrations | ||
requirements | ||
test_data | ||
tests | ||
|
||
flags.json |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
# platform=linux/amd64 | ||
FROM python:3.11-buster | ||
|
||
ARG VCS_REF="missing" | ||
ARG BUILD_DATE="missing" | ||
|
||
ENV VCS_REF=${VCS_REF} | ||
ENV BUILD_DATE=${BUILD_DATE} | ||
|
||
LABEL org.label-schema.vcs-ref=${VCS_REF} \ | ||
org.label-schema.build-date=${BUILD_DATE} | ||
|
||
USER root | ||
|
||
# Create working directory | ||
RUN mkdir /opt/app-root && chmod 755 /opt/app-root | ||
WORKDIR /opt/app-root | ||
|
||
# Install the requirements | ||
COPY ./requirements.txt . | ||
|
||
RUN pip install --upgrade pip | ||
#RUN pip install pip==22.1.2 | ||
RUN pip install --no-cache-dir -r requirements.txt | ||
|
||
COPY . . | ||
|
||
RUN pip install . | ||
|
||
USER 1001 | ||
|
||
# Set Python path | ||
ENV PYTHONPATH=/opt/app-root/src | ||
|
||
CMD [ "python", "-m", "ppr_registrations_historical" ] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
Copyright © 2018 Province of British Columbia | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
|
||
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE) | ||
[![codecov](https://codecov.io/gh/bcgov/lear/branch/master/graph/badge.svg?flag=entityefiler)](https://codecov.io/gh/bcgov/lear/tree/master/queue_services/entity-filer) | ||
|
||
# Application Name | ||
|
||
BC Registries PPR Account Registrations Historical Service | ||
|
||
## Technology Stack Used | ||
* Python | ||
* Postgres - psycopg2-binary | ||
* GCP Artifact Registry | ||
* GCP Cloud Run Jobs | ||
* GCP Cloud Scheduler | ||
|
||
## Project Status | ||
|
||
## Documnentation | ||
|
||
## Security | ||
|
||
## Getting Help or Reporting an Issue | ||
|
||
To report bugs/issues/feature requests, please file an [issue](../../issues). | ||
|
||
## How to Contribute | ||
|
||
If you would like to contribute, please see our [CONTRIBUTING](./CONTRIBUTING.md) guidelines. | ||
|
||
Please note that this project is released with a [Contributor Code of Conduct](./CODE_OF_CONDUCT.md). | ||
By participating in this project you agree to abide by its terms. | ||
|
||
## License | ||
|
||
Copyright 2024 Province of British Columbia | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
|
16 changes: 16 additions & 0 deletions
16
jobs/permanent/ppr-registrations-historical/requirements.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
cachetools==5.3.0 | ||
certifi==2022.12.7 | ||
charset-normalizer==3.1.0 | ||
grpcio-status==1.51.3 | ||
grpcio==1.60.1 | ||
idna==3.4 | ||
proto-plus==1.22.2 | ||
protobuf==4.22.1 | ||
psycopg2-binary==2.9.9 | ||
pyasn1-modules==0.2.8 | ||
pyasn1==0.4.8 | ||
python-dotenv==1.0.0 | ||
requests==2.28.2 | ||
rsa==4.9 | ||
six==1.16.0 | ||
urllib3==1.26.15 |
17 changes: 17 additions & 0 deletions
17
jobs/permanent/ppr-registrations-historical/requirements/dev.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
# Testing | ||
freezegun | ||
pytest | ||
|
||
# Lint and code style | ||
autopep8 | ||
flake8 | ||
flake8-blind-except | ||
flake8-debugger | ||
flake8-docstrings | ||
flake8-isort | ||
flake8-quotes | ||
pep8-naming | ||
pydocstyle | ||
pylint | ||
isort | ||
pytest-cov |
8 changes: 8 additions & 0 deletions
8
jobs/permanent/ppr-registrations-historical/requirements/prod.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
google-auth | ||
google-cloud-secret-manager | ||
google-cloud-storage | ||
google-crc32c | ||
six | ||
psycopg2-binary | ||
python-dotenv | ||
requests |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,117 @@ | ||
[metadata] | ||
name = ppr_registrations_historical | ||
url = https://github.com/thorwolpert/ppr/jobs/permanent/ppr-registrations-historical | ||
author = team-le | ||
author_email = [email protected] | ||
classifiers = | ||
Development Status :: Beta | ||
Intended Audience :: Developers / QA | ||
Topic :: Legal Entities | ||
License :: OSI Approved :: Apache Software License | ||
Natural Language :: English | ||
Programming Language :: Python :: 3.11 | ||
license = Apache Software License Version 2.0 | ||
description = A short description of the project | ||
long_description = file: README.md | ||
keywords = | ||
|
||
[options] | ||
zip_safe = True | ||
python_requires = >=3.9 | ||
include_package_data = True | ||
packages = find: | ||
|
||
[options.package_data] | ||
entity_filer = | ||
|
||
[wheel] | ||
universal = 1 | ||
|
||
[bdist_wheel] | ||
universal = 1 | ||
|
||
[aliases] | ||
test = pytest | ||
|
||
[flake8] | ||
exclude = .git,*migrations* | ||
max-line-length = 120 | ||
docstring-min-length=10 | ||
per-file-ignores = | ||
*/__init__.py:F401 | ||
*.py: B902 | ||
|
||
[pycodestyle] | ||
max_line_length = 120 | ||
ignore = E501 | ||
docstring-min-length=10 | ||
notes=FIXME,XXX # TODO is ignored | ||
match_dir = src/ppr_registrations_historical | ||
per-file-ignores = | ||
*/__init__.py:F401 | ||
good-names= | ||
b, | ||
d, | ||
i, | ||
e, | ||
f, | ||
k, | ||
u, | ||
v, | ||
ar, | ||
cb, #common shorthand for callback | ||
nc, | ||
rv, | ||
sc, | ||
event_loop, | ||
logger, | ||
loop, | ||
|
||
[pylint] | ||
ignore=migrations,test | ||
max_line_length=120 | ||
notes=FIXME,XXX,TODO | ||
disable=C0301,W0511,W0613,R0801,R0902 | ||
|
||
[isort] | ||
line_length = 120 | ||
indent = 4 | ||
multi_line_output = 3 | ||
lines_after_imports = 2 | ||
include_trailing_comma = True | ||
|
||
[tool:pytest] | ||
minversion = 2.0 | ||
testpaths = tests | ||
addopts = --verbose | ||
--strict | ||
-p no:warnings | ||
--cov=src --cov-report html:htmlcov --cov-report xml:coverage.xml | ||
python_files = tests/*/test*.py | ||
norecursedirs = .git .tox venv* requirements* build | ||
log_cli = true | ||
log_cli_level = 1 | ||
filterwarnings = | ||
ignore::UserWarning | ||
markers = | ||
slow | ||
serial | ||
|
||
[coverage:run] | ||
branch = True | ||
source = | ||
src/ppr_discharges_for_mhr | ||
omit = | ||
|
||
[report:run] | ||
exclude_lines = | ||
pragma: no cover | ||
from | ||
import | ||
def __repr__ | ||
if self.debug: | ||
if settings.DEBUG | ||
raise AssertionError | ||
raise NotImplementedError | ||
if 0: | ||
if __name__ == .__main__.: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
# Copyright © 2019 Province of British Columbia. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
"""Installer and setup for this module | ||
""" | ||
import ast | ||
from glob import glob | ||
from os.path import basename, splitext | ||
import re | ||
|
||
from setuptools import setup, find_packages | ||
|
||
_version_re = re.compile(r'__version__\s+=\s+(.*)') # pylint: disable=invalid-name | ||
|
||
with open('src/ppr_registrations_historical/version.py', 'rb') as f: | ||
version = str(ast.literal_eval(_version_re.search( # pylint: disable=invalid-name | ||
f.read().decode('utf-8')).group(1))) | ||
|
||
|
||
def read_requirements(filename): | ||
""" | ||
Get application requirements from | ||
the requirements.txt file. | ||
:return: Python requirements | ||
""" | ||
with open(filename, 'r') as req: | ||
requirements = req.readlines() | ||
install_requires = [r.strip() for r in requirements if r.find('git+') != 0] | ||
return install_requires | ||
|
||
|
||
def read(filepath): | ||
""" | ||
Read the contents from a file. | ||
:param str filepath: path to the file to be read | ||
:return: file contents | ||
""" | ||
with open(filepath, 'r') as file_handle: | ||
content = file_handle.read() | ||
return content | ||
|
||
|
||
REQUIREMENTS = read_requirements('requirements.txt') | ||
|
||
setup( | ||
name='ppr_registrations_historical', | ||
version=version, | ||
author_email='[email protected]', | ||
packages=find_packages('src'), | ||
package_dir={'': 'src'}, | ||
py_modules=[splitext(basename(path))[0] for path in glob('src/*.py')], | ||
include_package_data=True, | ||
license=read('LICENSE'), | ||
long_description=read('README.md'), | ||
zip_safe=False, | ||
install_requires=REQUIREMENTS, | ||
setup_requires=['pytest-runner', ], | ||
tests_require=['pytest', ], | ||
) |
Empty file.
24 changes: 24 additions & 0 deletions
24
jobs/permanent/ppr-registrations-historical/src/ppr_registrations_historical/__main__.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
import json | ||
import os | ||
import sys | ||
|
||
from .config import Config | ||
from .job import job | ||
|
||
# Retrieve Job-defined env vars | ||
TASK_INDEX = os.getenv("CLOUD_RUN_TASK_INDEX", 0) | ||
TASK_ATTEMPT = os.getenv("CLOUD_RUN_TASK_ATTEMPT", 0) | ||
|
||
|
||
# Start script | ||
if __name__ == "__main__": | ||
try: | ||
config = Config() | ||
job(config) | ||
except Exception as err: | ||
message = f"Task #{TASK_INDEX}, " \ | ||
+ f"Attempt #{TASK_ATTEMPT} failed: {str(err)}" | ||
|
||
print(json.dumps({"message": message, "severity": "ERROR"})) | ||
sys.exit(1) # Retry Job Task by exiting the process | ||
|
Oops, something went wrong.