ModuleNotFoundError: No module named 'connexion.decorators.validation' while running airflow init #239
Assignees
Comments
|
From pip installation we can see that the connexion version, which was installed, was wrong: My suggestion, just install it separately, since it's coming from from airflow[celery,postgres]==2.6.0. Trace: |
|
Also, I see that the last push to the docker image was done 6 months ago: while the issue regarding the connexion crash was 2 weeks ago (28th of Oct): The change regarding the upper version limit was added also 28th of Oct: |
|
FYI the way we have constraints in the requirements file, doesn't work |
|
Are you sure about it? |
|
Just for information: |
you can check it locally if it has effect (else why have the constraints if they don't constrain ) it should not be part of the so: |
|
Sadly, I tried before, it didn't work on github actions. you can check the following log: However, I found more elegant solution that works: https://stackoverflow.com/questions/34645821/pip-constraints-files Please let me know when you will be done with checking. I will prepare the PR for review: squashing the commits, removing comments, etc. |
ErnestaP
added a commit
to cern-sis/workflows
that referenced
this issue
Nov 8, 2023
For some reason the action didn't even take into considaration the change for the constraints. see: https://github.com/cern-sis/workflows/pull/180/checks#step:5:5 |
|
Yeah we are using constrainst for many years in Airflow and they are designed to specificlly prevent our users from hitting similar problems (aka reproduciibly install airflow) https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html I designed and implemented and had a few talks about it and I was actully thinking that some day this approach could make it into a PEP and some common "reproducible installation" mechanism for python packages, but I thought it's really edge case for complex software like Airflow with 670 (!) dependencies in total and being both a library (where you need to keep dependency flexibility) and application (where you want stuff to install reproducibly. I'd love to hear if others have some thought about it :) |
ErnestaP
added a commit
to cern-sis/workflows
that referenced
this issue
Nov 9, 2023
pamfilos
pushed a commit
to cern-sis/workflows
that referenced
this issue
Nov 9, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I found the same issue posted a few days ago on GitHub.
apache/airflow#35434
The issue appeared on Airflow 2.7.2
We are using Airflow 2.6.0
Discussion of why it is crashing: apache/airflow#35234
In itself, this doesn't lead to any breaking changes if you're using the intended interface of Connexion, which is an application (FlaskApp or AsyncApp in Connexion 3). However, in https://github.com/apache/airflow/pull/29631, Airflow moved away from this intended interface by creating a FlaskApi directly and registering its internal blueprint on the Flask app directly. This worked in Connexion 2, since the FlaskApi created the decorated routes. But it no longer works on Connexion 3, since it doesn't contain the middleware, which is part of the FlaskApp.Installing separately connexion==2.14.2, solved the issue.
However, I see that NEWSEST airflow uses >=2.10.0,<3.0 version of connexion.:
connexion[flask]>=2.10.0,<3.0
https://github.com/apache/airflow/blob/main/setup.cfg#L81-L88
H̶o̶w̶e̶v̶e̶r̶,̶ ̶i̶t̶ ̶m̶i̶g̶h̶t̶ ̶b̶e̶ ̶a̶ ̶b̶a̶d̶ ̶p̶r̶a̶c̶t̶i̶c̶e̶ ̶t̶o̶ ̶i̶n̶s̶t̶a̶l̶l̶ ̶i̶t̶ ̶s̶e̶p̶a̶r̶a̶t̶e̶l̶y̶ ̶f̶r̶o̶m̶ ̶o̶t̶h̶e̶r̶ ̶a̶i̶r̶f̶l̶o̶w̶ ̶d̶e̶p̶e̶n̶d̶e̶n̶c̶i̶e̶s̶.̶ ̶W̶e̶ ̶s̶h̶o̶u̶l̶d̶ ̶i̶n̶s̶t̶a̶l̶l̶ ̶t̶h̶e̶ ̶c̶o̶r̶r̶e̶c̶t̶ ̶c̶o̶n̶n̶e̶x̶i̶o̶n̶ ̶v̶e̶r̶s̶i̶o̶n̶ ̶w̶i̶t̶h̶ ̶a̶i̶r̶f̶l̶o̶w̶ ̶i̶t̶s̶e̶l̶f̶ ̶o̶r̶ ̶w̶i̶t̶h̶ ̶e̶x̶t̶r̶a̶s̶(̶o̶r̶ ̶p̶r̶o̶v̶i̶d̶e̶r̶s̶)̶ ̶b̶y̶ ̶d̶e̶s̶c̶r̶i̶b̶i̶n̶g̶ ̶t̶h̶e̶m̶ ̶i̶n̶ ̶t̶h̶e̶ ̶r̶e̶q̶u̶i̶r̶e̶m̶e̶n̶t̶s̶-̶a̶i̶r̶f̶l̶o̶w̶.̶t̶x̶t̶ ̶f̶i̶l̶e̶:̶
apache-airflow[celery, postgres]==2.6.0 --constraint https://raw.githubusercontent.com/apache/airflow/constraints-2.6.0/constraints-3.10.txt
https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html
Full list of extras and their dependencies:
https://github.com/apache/airflow/blob/main/setup.py
The full trace of error:
Notes: build airflow Dockerfile : https://airflow.apache.org/docs/docker-stack/build.html
The text was updated successfully, but these errors were encountered: