Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rebased aiopenapi3 #69

Open
wants to merge 125 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
125 commits
Select commit Hold shift + click to select a range
b7a9774
using dataclasses
commonism Dec 22, 2021
bcc98a4
dataclasses - …
commonism Dec 22, 2021
880de1a
required_fields - using typing Optional information instead
commonism Dec 23, 2021
de5fc0f
pydantic migration - does not work yet
commonism Dec 23, 2021
c7d1748
pydantic migration - basics working
commonism Dec 27, 2021
4495473
pydantic - more tests work
commonism Dec 27, 2021
babde76
pydantic - use Dict instead of Map
commonism Dec 27, 2021
8fc06c9
Model - remove
commonism Dec 27, 2021
3c4cd76
pydantic - remove relicts
commonism Dec 27, 2021
ff81552
pydantic - all tests work
commonism Dec 27, 2021
eb2f8d6
tests - include specs from bugs #9 & #10
commonism Dec 27, 2021
4c14f99
pydantic - clean up OpjectBase
commonism Dec 27, 2021
0e30074
tests - parameterize tests/data tests
commonism Dec 27, 2021
75b9236
pydantic - cleanups
commonism Dec 27, 2021
aea94b4
pydantic - cleanups
commonism Dec 27, 2021
901da19
pydantic - ObjectBase is not extended, ObjectExtended is
commonism Dec 28, 2021
2b8f232
pydantic objects - avoid ForwardRef
commonism Dec 28, 2021
82372f9
rename resolve_type -> resolve_jr / resolve_path -> resolve_jp
commonism Dec 28, 2021
776cab2
tests - with parameters & callback
commonism Dec 28, 2021
507a830
cleanup - unused imports
commonism Dec 28, 2021
c46bae8
ObjectBase - cleanup private fields
commonism Dec 28, 2021
faf8b16
validation mode - error() return Exception/ValidationError
commonism Dec 28, 2021
ef14b65
errors - drop path attribute
commonism Dec 28, 2021
3bfa8e1
revise models to match v3.0.3 & order fields accordingly
commonism Dec 29, 2021
09e157a
Schema - re-introduce Model
commonism Dec 29, 2021
ae6b33e
paths - split -> media & parameters
commonism Dec 29, 2021
645363a
runtime expression - with tatsu
commonism Dec 29, 2021
85c7d32
schema - get_type does Union[]s
commonism Dec 29, 2021
1f8c42b
model_test -
commonism Dec 30, 2021
c7cfef1
openapi - resolving references is in-situ just links the target
commonism Dec 30, 2021
9e1edc2
runtimeexpression - test escaping / & ~
commonism Dec 30, 2021
6c2e3de
schema - support multi level discrimination
commonism Dec 30, 2021
80c181e
calling - restructure
commonism Dec 30, 2021
07e374f
discrimination - set default values
commonism Dec 30, 2021
9e06494
loader - move
commonism Jan 1, 2022
e903a41
typing
commonism Jan 1, 2022
fc93190
Request - use httpx, add async interface
commonism Jan 1, 2022
a5f6c93
OpenAPI - remove errors() & call_…
commonism Jan 1, 2022
4ff5433
ssl - disabling verification is not easy
commonism Jan 1, 2022
ddec219
tests - httpx changes
commonism Jan 2, 2022
86f0f7c
tests - fastapi based tests using sync api
commonism Jan 2, 2022
41a96b3
request - send auth & headers
commonism Jan 2, 2022
dca487c
cleanups
commonism Jan 2, 2022
7583e9d
rename to aiopenapi3
commonism Jan 2, 2022
a08da44
codecov
commonism Jan 2, 2022
95fbc73
codecov trigger
commonism Jan 2, 2022
d5d7022
README - include codecov banner in README
commonism Jan 2, 2022
0876038
python 3.7 compatiblity
commonism Jan 3, 2022
84d0fec
Loader - tests
commonism Jan 3, 2022
3d00bdd
python 3.8 required due to Literal
commonism Jan 3, 2022
0a69260
python 3.9 required for Annotated
commonism Jan 3, 2022
a3a9694
runtimeexpression - skip tests for >= 3.10
commonism Jan 3, 2022
c7cfda2
README - link Python versions & pre-commit.ci
commonism Jan 3, 2022
6c86e85
setup.cfg - pypi validation
commonism Jan 3, 2022
c58b5a9
schemas - missing type "number"
commonism Jan 3, 2022
f769e58
OperationIndex - Iter the operationIds
commonism Jan 3, 2022
f159ab2
tests - iter the linode api spec
commonism Jan 3, 2022
8ad8018
pre-commit.ci
commonism Jan 3, 2022
cbde1b7
remove the runtime expression code - it's server side
commonism Jan 3, 2022
a0c9433
tests - pre-commit
commonism Jan 3, 2022
1784965
expression - remove
commonism Jan 3, 2022
6d60db7
tests/linode - skip on github, the downloaded file is broken
commonism Jan 3, 2022
3fdf369
setup.cfg - add url
commonism Jan 3, 2022
f4180bb
request - provide .data for easy access
commonism Jan 5, 2022
7d7356d
plugin - dealing with real world implementations
commonism Jan 5, 2022
1a7beb4
plugins - wire into loader & request
commonism Jan 5, 2022
5a172d8
Request/Plugin - coverage
commonism Jan 5, 2022
36c16ab
[pre-commit.ci] pre-commit autoupdate
pre-commit-ci[bot] Jan 3, 2022
2d7785a
validator - use load_file
commonism Jan 6, 2022
659c3e3
request - set user-agent
commonism Jan 6, 2022
b6b0009
validator - use load_file
commonism Jan 6, 2022
e94b15c
v3.1 - adding support
commonism Jan 7, 2022
b5d97a1
v31 - do not to track v30->v31 changes to objects/relations
commonism Jan 10, 2022
5e2a6e4
v30 - fixes
commonism Jan 10, 2022
af41d84
v31/schema - ref
commonism Jan 10, 2022
d0b1f61
RootBase - resolving references
commonism Jan 10, 2022
948c99b
v20 - add support for Swagger/2.0
commonism Jan 11, 2022
74afdb3
loader - do not coerce date times when parsing yaml
commonism Jan 11, 2022
d56b8ef
base/resolve - lists & schema_
commonism Jan 11, 2022
0c781cd
v20 - …
commonism Jan 11, 2022
6df4ae0
schema - default is Any for v20/30/31
commonism Jan 11, 2022
0e42c1f
example - value is Any
commonism Jan 11, 2022
d4b55d2
SecurityRequirement - the validator is wrong
commonism Jan 11, 2022
6e5a037
OpenAPI - allow the default session_factory
commonism Jan 11, 2022
f08a87f
SecurityRequirement - wrong validator
commonism Jan 11, 2022
b779638
v30 - export the glue
commonism Jan 11, 2022
d0e37ec
SecurityScheme - extensions are allowed
commonism Jan 12, 2022
e17b47b
v31 - Schema.additionalProperties can be bool/Schema
commonism Jan 12, 2022
20112ee
v20 - Schema.items can be List
commonism Jan 12, 2022
4d571fe
v20 - empty object can have extensions?
commonism Jan 12, 2022
eb47989
v20/30/31 - paths can have extensions
commonism Jan 12, 2022
0c3bb14
Link - validator behaviour changed
commonism Jan 12, 2022
5c0ae85
loader - WebLoader, NullLoader
commonism Jan 12, 2022
dafb3cb
loader - YAMLCompatibilityLoader remove tags for int/bool/dict
commonism Jan 12, 2022
5b6c43b
v20/30/31 - the PathBase
commonism Jan 12, 2022
eea8e95
tests - fixes
commonism Jan 12, 2022
7626842
Paths - …
commonism Jan 12, 2022
8c51026
loader - fix tests
commonism Jan 12, 2022
432494e
compat - going to 3.7
commonism Jan 12, 2022
4d209ed
compat - use pathlib3x consistently
commonism Jan 12, 2022
9ccbe59
Docker - using containers for Python 3.7
commonism Jan 13, 2022
c9c399e
tests - invalid response
commonism Jan 13, 2022
45121fe
v20 - tests
commonism Jan 13, 2022
c58d0e1
v20/30/31 - authentication
commonism Jan 17, 2022
f1272ca
OperationIndex - operationId is optional
commonism Jan 17, 2022
6ae54d5
README - authentication changes
commonism Jan 17, 2022
44830af
setup.cfg - add v20/30/31 packages to install
commonism Jan 18, 2022
28dfa4c
errors - raise {ContentType,HTTPStatus}Error for unexpected
commonism Jan 19, 2022
5720784
v20/Model - required is Parameter attribute
commonism Jan 19, 2022
a160b86
v20 - OpenAPI.url - include port
commonism Jan 19, 2022
47a0f46
model - schema can be Reference
commonism Jan 19, 2022
2ceb13d
OpenAPI.url - port
commonism Jan 19, 2022
d5178fc
model - primitive types
commonism Jan 20, 2022
ad83b60
tuple - unlist
commonism Jan 20, 2022
ddcd6f5
model - fix type_name creation
commonism Jan 20, 2022
17b3a1a
model - extract functions
commonism Jan 20, 2022
e185891
model - cache type
commonism Jan 20, 2022
1d0eee1
Schema.get_type - initialize by iterating the gc's objects …
commonism Jan 21, 2022
8b691d9
SpecError - sorted names
commonism Jan 24, 2022
59332e4
content-type in encodings
commonism Jan 25, 2022
efe58e3
tests/linode - the github CI …
commonism Jan 25, 2022
a72fdbf
tests - handle DeprecationWarnings
commonism Jan 25, 2022
c64f058
openapi.url - partition
commonism Jan 25, 2022
b5584d8
ObjectExtended - extensions better be Any than object
commonism Jan 28, 2022
8fb9059
model - support pydantic custom formats such as datetime.timedelta
commonism Jan 28, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .github/workflows/codecov.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Codecov
on: [push, pull_request]
jobs:
run:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest]
python: ["3.7","3.8","3.9","3.10"]
env:
OS: ${{ matrix.os }}
PYTHON: ${{ matrix.python }}
PYTHONPATH: "."
steps:
- uses: actions/checkout@master
- name: Setup Python
uses: actions/setup-python@master
with:
python-version: ${{ matrix.python }}
- name: Generate coverage report
run: |
pip install '.[tests]'
pip install '.[compat]'
pytest --cov=./ --cov-report=xml
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v2
with:
token: ${{ secrets.CODECOV_TOKEN }}
directory: ./coverage/reports/
env_vars: OS,PYTHON
fail_ci_if_error: false
files: ./coverage.xml
flags: unittests
name: codecov-aiopenapi3
path_to_write_report: ./coverage/codecov_report.txt
verbose: true
24 changes: 13 additions & 11 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
repos:
- repo: https://github.com/hadialqattan/pycln
rev: v0.0.4 # Possible releases: https://github.com/hadialqattan/pycln/releases
rev: v1.1.0 # Possible releases: https://github.com/hadialqattan/pycln/releases
hooks:
- id: pycln
- repo: 'https://github.com/psf/black'
rev: 21.6b0
rev: 21.12b0
hooks:
- id: black
args:
- "--line-length=120"
- repo: 'https://github.com/pre-commit/pre-commit-hooks'
rev: v4.0.1
rev: v4.1.0
hooks:
- id: end-of-file-fixer
exclude: '^docs/[^/]*\.svg$'
Expand All @@ -20,20 +20,12 @@ repos:
files: |
.gitignore
- id: check-case-conflict
- id: check-json
- id: check-xml
- id: check-executables-have-shebangs
- id: check-toml
- id: check-xml
- id: check-yaml
- id: debug-statements
- id: check-added-large-files
- id: check-symlinks
- id: debug-statements
- id: detect-aws-credentials
args:
- '--allow-missing-credentials'
- id: detect-private-key
- repo: 'https://gitlab.com/pycqa/flake8'
rev: 3.9.2
hooks:
Expand All @@ -42,3 +34,13 @@ repos:
- "--max-line-length=120"
- "--ignore=E203,W503"
- "--select=W504"

ci:
autofix_commit_msg: |
[pre-commit.ci] auto fixes from pre-commit.ci hooks
autofix_prs: true
autoupdate_branch: ''
autoupdate_commit_msg: '[pre-commit.ci] pre-commit autoupdate'
autoupdate_schedule: weekly
skip: []
submodules: false
10 changes: 10 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
FROM python:3.7-slim-buster

WORKDIR /app
COPY setup.cfg pyproject.toml requirements.txt /app/
COPY aiopenapi3/ /app/aiopenapi3
COPY tests /app/tests
RUN ls -al /app
RUN pip install --upgrade pip
RUN pip install ".[compat]"
RUN pip install ".[tests]"
1 change: 1 addition & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
Copyright (c) 2019, William Smith
Copyright (c) 2022, Markus Kötter
All rights reserved.

Redistribution and use in source and binary forms, with or without
Expand Down
220 changes: 220 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,220 @@
# aiopenapi3

A Python [OpenAPI 3 Specification](https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.0.3.md) client and validator for Python 3.

[![Test](https://github.com/commonism/aiopenapi3/workflows/Codecov/badge.svg?event=push&branch=master)](https://github.com/commonism/aiopenapi3/actions?query=workflow%3ACodecov+event%3Apush+branch%3Amaster)
[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/commonism/aiopenapi3/master.svg)](https://results.pre-commit.ci/latest/github/commonism/aiopenapi3/master)
[![Coverage](https://img.shields.io/codecov/c/github/commonism/aiopenapi3)](https://codecov.io/gh/commonism/aiopenapi3)
[![Supported Python versions](https://img.shields.io/pypi/pyversions/aiopenapi3.svg)](https://pypi.org/project/aiopenapi3)


This project is based on [Dorthu/openapi3](github.com/Dorthu/openapi3/).

## Features
* implements …
* Swagger 2.0
* OpenAPI 3.0.3
* OpenAPI 3.1.0
* object parsing via pydantic
* request body model creation via [pydantic](https://github.com/samuelcolvin/pydantic)
* blocking and nonblocking (asyncio) interface via [httpx](https://www.python-httpx.org/)
* tests with pytest
* providing access to methods and arguments via the sad smiley ._. interface


## Usage as a Client

This library also functions as an interactive client for arbitrary OpenAPI 3
specs. For example, using `Linode's OpenAPI 3 Specification`_ for reference:

*Unfortunately I do not have access to the Linode API to validate object creation*

### asyncio
```python
from aiopenapi3 import OpenAPI
url = "https://www.linode.com/docs/api/openapi.yaml"

api = await OpenAPI.load_async(url)

# call operations and receive result models
regions = await api._.getRegions()
```

### blocking io
```python
from aiopenapi3 import OpenAPI
url = "https://www.linode.com/docs/api/openapi.yaml"
my_token = "Gae6aikaegainoor"
api = OpenAPI.load_sync(url)

# call operations and receive result models
regions = api._.getRegions()


```

### objects
pydantic is used for the models.
https://pydantic-docs.helpmanual.io/usage/exporting_models/

```python
from aiopenapi3 import OpenAPI
url = "https://www.linode.com/docs/api/openapi.yaml"

api = await OpenAPI.load_sync(url)

# call operations and receive result models
regions = await api._.getRegions()

regions.__fields_set__
{'results', 'page', 'pages', 'data'}

import json
print(json.dumps((list(filter(lambda x: 'eu-west' in x.id, regions.data))[0]).dict(), indent=2))
{
"id": "eu-west",
"country": "uk",
"capabilities": [
"Linodes",
"NodeBalancers",
"Block Storage",
"Kubernetes",
"Cloud Firewall"
],
"status": "ok",
"resolvers": {
"ipv4": "178.79.182.5,176.58.107.5,176.58.116.5,176.58.121.5,151.236.220.5,212.71.252.5,212.71.253.5,109.74.192.20,109.74.193.20,109.74.194.20",
"ipv6": "2a01:7e00::9,2a01:7e00::3,2a01:7e00::c,2a01:7e00::5,2a01:7e00::6,2a01:7e00::8,2a01:7e00::b,2a01:7e00::4,2a01:7e00::7,2a01:7e00::2"
}
}
```

#### discriminators
discriminators are supported as well, but the linode api can't be used to show how to use them.
look at [aiopenapi3/tests/model_test.py](aiopenapi3/tests/model_test.py) test_model.

### authentication
```python
my_token = "Gae6aikaegainoor"
api.authenticate(personalAccessToken=my_token)

# call an operation that requires authentication
linodes = api._.getLinodeInstances()
```

HTTP basic authentication and HTTP digest authentication works like this:
```python
# authenticate using a securityScheme defined in the spec's components.securitySchemes
# Tuple with (username, password) as second argument
api.authenticate(basicAuth=('username', 'password'))
```

Resetting authentication tokens:
```python
api.authenticate(None)
```

### parameters

```python
# call an opertaion with parameters
linode = api._.getLinodeInstance(parameters={"linodeId": 123})
```

### body
```python
body = api._.createLinodeInstance.args()["data"].model({"region":"us-east", "type":"g6-standard-2"})
print(json.dumps(body.dict(), indent=2))
{
"image": null,
"root_pass": null,
"authorized_keys": null,
"authorized_users": null,
"stackscript_id": null,
"stackscript_data": null,
"booted": null,
"backup_id": null,
"backups_enabled": null,
"swap_size": null,
"type": "g6-standard-2",
"region": "us-east",
"label": null,
"tags": null,
"group": null,
"private_ip": null,
"interfaces": null
}

print(json.dumps(body.dict(exclude_unset=True), indent=2))
{
"type": "g6-standard-2",
"region": "us-east"
}


>>>
new_linode = api._.createLinodeInstance(data=body)
```

## Validation Mode


This module can be run against a spec file to validate it like so::

```
python3 -m aiopenapi3 tests/fixtures/with-broken-links.yaml

6 validation errors for OpenAPISpec
paths -> /with-links -> get -> responses -> 200 -> links -> exampleWithBoth -> __root__
operationId and operationRef are mutually exclusive, only one of them is allowed (type=value_error.spec; message=operationId and operationRef are mutually exclusive, only one of them is allowed; element=None)
paths -> /with-links -> get -> responses -> 200 -> links -> exampleWithBoth -> $ref
field required (type=value_error.missing)
paths -> /with-links -> get -> responses -> 200 -> $ref
field required (type=value_error.missing)
paths -> /with-links-two -> get -> responses -> 200 -> links -> exampleWithNeither -> __root__
operationId and operationRef are mutually exclusive, one of them must be specified (type=value_error.spec; message=operationId and operationRef are mutually exclusive, one of them must be specified; element=None)
paths -> /with-links-two -> get -> responses -> 200 -> links -> exampleWithNeither -> $ref
field required (type=value_error.missing)
paths -> /with-links-two -> get -> responses -> 200 -> $ref
field required (type=value_error.missing)
```

## Real World issues
### YAML
The description document may no be valid yaml.
YAML type coercion can cause this.
```python
>>> yaml.safe_load(str(datetime.datetime.now().date()))
datetime.date(2022, 1, 12)

>>> yaml.safe_load("name: on")
{'name': True}

>>> yaml.safe_load('12_24: "test"')
{1224: 'test'}
```
Those can be turned of using the yload yaml.Loader argument to the Loader.

```python
import aiopenapi3.loader

OpenAPI.load…(…, loader=FileSystemLoader(pathlib.Path(dir), yload = aiopenapi3.loader.YAMLCompatibilityLoader))

```

### description document mismatch
In case the description document does not match the protocol, it may be required to alter the description, objects or data sent/received.
The [Plugin interface](tests/plugin_test.py) can be used to alter any of those.
It can even be used to alter an invalid description document to be usable.



## Running Tests

This project includes a test suite, run via ``pytest``. To run the test suite,
ensure that you've installed the dependencies and then run ``pytest`` in the root
of this project.

```shell
PYTHONPATH=. pytest --cov=./ --cov-report=xml .
```
Loading