Airflow 2 support #46

kevroes · 2021-03-22T14:07:22Z

kevroes
Mar 22, 2021

Hi there,

I wanted to ask if you are planning to support Airflow 2.0 with your awesome extension. I've been running into some issues, while running your tests, using Airflow 2.0.1 and I guess it's because of the new Airflow version.

Let me know if it is on your roadmap, or not. This would help us coordinate our Airflow implementation on our Platform.

Thanks for the information and have a pleasent day.

Cheers

FYI here is my python test-log:

============================= test session starts =============================
platform win32 -- Python 3.7.6, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: c:\Users\kevin.roeschli\Documents\source\repos\airflow_test
collected 61 items

tests\test_bad_dags.py ....................                              [ 32%]
tests\test_callback_tasks.py ...                                         [ 37%]
tests\test_example_bash_operator.py F                                    [ 39%]
tests\test_good_dags.py ................FFFF.FFF.....F                   [ 88%]
tests\test_integer_callback_arg.py .                                     [ 90%]
tests\test_metadata.py ..                                                [ 93%]
tests\test_template_with_complex_jinja.py .                              [ 95%]
tests\test_template_with_defaults.py .                                   [ 96%]
tests\test_template_with_dicts.py F                                      [ 98%]
tests\test_template_with_from_stdout.py F                                [100%]

================================== FAILURES ===================================
_________________________ test_example_bash_operator __________________________

good_dag_path = <function good_dag_path.<locals>.<lambda> at 0x000001FE35C194C8>

    def test_example_bash_operator(good_dag_path):
        path = good_dag_path("example_bash_operator")
        dags = airflow_declarative.from_path(path)
    
        assert len(dags) == 1
    
        yml_dag = dags[0]
    
        assert isinstance(yml_dag, DAG)
    
        assert yml_dag.dagrun_timeout == datetime.timedelta(minutes=60)
    
        assert yml_dag.default_args["owner"] == "airflow"
    
>       assert yml_dag.start_date == datetime.datetime(2017, 7, 27, 0, 0, 0)
E       AssertionError: assert DateTime(2017, 7, 27, 0, 0, 0, tzinfo=Timezone('UTC')) == datetime.datetime(2017, 7, 27, 0, 0)
E        +  where DateTime(2017, 7, 27, 0, 0, 0, tzinfo=Timezone('UTC')) = <DAG: example_bash_operator>.start_date
E        +  and   datetime.datetime(2017, 7, 27, 0, 0) = <class 'datetime.datetime'>(2017, 7, 27, 0, 0, 0)
E        +    where <class 'datetime.datetime'> = datetime.datetime

tests\test_example_bash_operator.py:42: AssertionError
_________________________ test_serde[check-callbacks] _________________________

path = 'c:\\Users\\kevin.roeschli\\Documents\\source\\repos\\airflow_test\\tests\\dags\\good\\check-callbacks.yaml'

    def test_serde(path):
        schema0 = airflow_declarative.transform(airflow_declarative.schema.from_path(path))
>       content = airflow_declarative.render(path)

tests\test_good_dags.py:48: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv\lib\site-packages\airflow_declarative\__init__.py:62: in render
    return schema.dump(transform(schema.from_path(path)))
.venv\lib\site-packages\airflow_declarative\schema.py:81: in dump
    return yaml.dump(schema, Dumper=Dumper, *args, **kwargs)
.venv\lib\site-packages\yaml\__init__.py:290: in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
.venv\lib\site-packages\yaml\__init__.py:278: in dump_all
    dumper.represent(data)
.venv\lib\site-packages\yaml\representer.py:27: in represent
    node = self.represent_data(data)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:58: in represent_data
    node = self.yaml_representers[None](self, data)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <airflow_declarative.schema.Dumper object at 0x000001FE35CB8EC8>
data = <class 'airflow.operators.dummy.DummyOperator'>

    def represent_undefined(self, data):
>       raise RepresenterError("cannot represent an object", data)
E       yaml.representer.RepresenterError: ('cannot represent an object', <class 'airflow.operators.dummy.DummyOperator'>)

.venv\lib\site-packages\yaml\representer.py:231: RepresenterError
______________________________ test_serde[email] ______________________________

path = 'c:\\Users\\kevin.roeschli\\Documents\\source\\repos\\airflow_test\\tests\\dags\\good\\email.yaml'

    def test_serde(path):
        schema0 = airflow_declarative.transform(airflow_declarative.schema.from_path(path))
>       content = airflow_declarative.render(path)

tests\test_good_dags.py:48: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv\lib\site-packages\airflow_declarative\__init__.py:62: in render
    return schema.dump(transform(schema.from_path(path)))
.venv\lib\site-packages\airflow_declarative\schema.py:81: in dump
    return yaml.dump(schema, Dumper=Dumper, *args, **kwargs)
.venv\lib\site-packages\yaml\__init__.py:290: in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
.venv\lib\site-packages\yaml\__init__.py:278: in dump_all
    dumper.represent(data)
.venv\lib\site-packages\yaml\representer.py:27: in represent
    node = self.represent_data(data)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:58: in represent_data
    node = self.yaml_representers[None](self, data)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <airflow_declarative.schema.Dumper object at 0x000001FE36CE7348>
data = <class 'airflow.operators.dummy.DummyOperator'>

    def represent_undefined(self, data):
>       raise RepresenterError("cannot represent an object", data)
E       yaml.representer.RepresenterError: ('cannot represent an object', <class 'airflow.operators.dummy.DummyOperator'>)

.venv\lib\site-packages\yaml\representer.py:231: RepresenterError
______________________ test_serde[example_bash_operator] ______________________

path = 'c:\\Users\\kevin.roeschli\\Documents\\source\\repos\\airflow_test\\tests\\dags\\good\\example_bash_operator.yaml'

    def test_serde(path):
        schema0 = airflow_declarative.transform(airflow_declarative.schema.from_path(path))
>       content = airflow_declarative.render(path)

tests\test_good_dags.py:48: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv\lib\site-packages\airflow_declarative\__init__.py:62: in render
    return schema.dump(transform(schema.from_path(path)))
.venv\lib\site-packages\airflow_declarative\schema.py:81: in dump
    return yaml.dump(schema, Dumper=Dumper, *args, **kwargs)
.venv\lib\site-packages\yaml\__init__.py:290: in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
.venv\lib\site-packages\yaml\__init__.py:278: in dump_all
    dumper.represent(data)
.venv\lib\site-packages\yaml\representer.py:27: in represent
    node = self.represent_data(data)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:58: in represent_data
    node = self.yaml_representers[None](self, data)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <airflow_declarative.schema.Dumper object at 0x000001FE36D7AA08>
data = <class 'airflow.operators.bash.BashOperator'>

    def represent_undefined(self, data):
>       raise RepresenterError("cannot represent an object", data)
E       yaml.representer.RepresenterError: ('cannot represent an object', <class 'airflow.operators.bash.BashOperator'>)

.venv\lib\site-packages\yaml\representer.py:231: RepresenterError
_________________________ test_serde[executor-config] _________________________

path = 'c:\\Users\\kevin.roeschli\\Documents\\source\\repos\\airflow_test\\tests\\dags\\good\\executor-config.yaml'

    def test_serde(path):
        schema0 = airflow_declarative.transform(airflow_declarative.schema.from_path(path))
>       content = airflow_declarative.render(path)

tests\test_good_dags.py:48: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv\lib\site-packages\airflow_declarative\__init__.py:62: in render
    return schema.dump(transform(schema.from_path(path)))
.venv\lib\site-packages\airflow_declarative\schema.py:81: in dump
    return yaml.dump(schema, Dumper=Dumper, *args, **kwargs)
.venv\lib\site-packages\yaml\__init__.py:290: in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
.venv\lib\site-packages\yaml\__init__.py:278: in dump_all
    dumper.represent(data)
.venv\lib\site-packages\yaml\representer.py:27: in represent
    node = self.represent_data(data)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:58: in represent_data
    node = self.yaml_representers[None](self, data)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <airflow_declarative.schema.Dumper object at 0x000001FE35CBD248>
data = <class 'airflow.operators.dummy.DummyOperator'>

    def represent_undefined(self, data):
>       raise RepresenterError("cannot represent an object", data)
E       yaml.representer.RepresenterError: ('cannot represent an object', <class 'airflow.operators.dummy.DummyOperator'>)

.venv\lib\site-packages\yaml\representer.py:231: RepresenterError
__________________ test_serde[schedule-interval-cron-preset] __________________

path = 'c:\\Users\\kevin.roeschli\\Documents\\source\\repos\\airflow_test\\tests\\dags\\good\\schedule-interval-cron-preset.yaml'

    def test_serde(path):
        schema0 = airflow_declarative.transform(airflow_declarative.schema.from_path(path))
>       content = airflow_declarative.render(path)

tests\test_good_dags.py:48: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv\lib\site-packages\airflow_declarative\__init__.py:62: in render
    return schema.dump(transform(schema.from_path(path)))
.venv\lib\site-packages\airflow_declarative\schema.py:81: in dump
    return yaml.dump(schema, Dumper=Dumper, *args, **kwargs)
.venv\lib\site-packages\yaml\__init__.py:290: in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
.venv\lib\site-packages\yaml\__init__.py:278: in dump_all
    dumper.represent(data)
.venv\lib\site-packages\yaml\representer.py:27: in represent
    node = self.represent_data(data)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:58: in represent_data
    node = self.yaml_representers[None](self, data)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <airflow_declarative.schema.Dumper object at 0x000001FE35C23F08>
data = <class 'airflow.operators.dummy.DummyOperator'>

    def represent_undefined(self, data):
>       raise RepresenterError("cannot represent an object", data)
E       yaml.representer.RepresenterError: ('cannot represent an object', <class 'airflow.operators.dummy.DummyOperator'>)

.venv\lib\site-packages\yaml\representer.py:231: RepresenterError
_____________________ test_serde[schedule-interval-cron] ______________________

path = 'c:\\Users\\kevin.roeschli\\Documents\\source\\repos\\airflow_test\\tests\\dags\\good\\schedule-interval-cron.yaml'

    def test_serde(path):
        schema0 = airflow_declarative.transform(airflow_declarative.schema.from_path(path))
>       content = airflow_declarative.render(path)

tests\test_good_dags.py:48: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv\lib\site-packages\airflow_declarative\__init__.py:62: in render
    return schema.dump(transform(schema.from_path(path)))
.venv\lib\site-packages\airflow_declarative\schema.py:81: in dump
    return yaml.dump(schema, Dumper=Dumper, *args, **kwargs)
.venv\lib\site-packages\yaml\__init__.py:290: in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
.venv\lib\site-packages\yaml\__init__.py:278: in dump_all
    dumper.represent(data)
.venv\lib\site-packages\yaml\representer.py:27: in represent
    node = self.represent_data(data)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:58: in represent_data
    node = self.yaml_representers[None](self, data)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <airflow_declarative.schema.Dumper object at 0x000001FE36D32248>
data = <class 'airflow.operators.dummy.DummyOperator'>

    def represent_undefined(self, data):
>       raise RepresenterError("cannot represent an object", data)
E       yaml.representer.RepresenterError: ('cannot represent an object', <class 'airflow.operators.dummy.DummyOperator'>)

.venv\lib\site-packages\yaml\representer.py:231: RepresenterError
_____________________ test_serde[schedule-interval-none] ______________________

path = 'c:\\Users\\kevin.roeschli\\Documents\\source\\repos\\airflow_test\\tests\\dags\\good\\schedule-interval-none.yaml'

    def test_serde(path):
        schema0 = airflow_declarative.transform(airflow_declarative.schema.from_path(path))
>       content = airflow_declarative.render(path)

tests\test_good_dags.py:48: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv\lib\site-packages\airflow_declarative\__init__.py:62: in render
    return schema.dump(transform(schema.from_path(path)))
.venv\lib\site-packages\airflow_declarative\schema.py:81: in dump
    return yaml.dump(schema, Dumper=Dumper, *args, **kwargs)
.venv\lib\site-packages\yaml\__init__.py:290: in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
.venv\lib\site-packages\yaml\__init__.py:278: in dump_all
    dumper.represent(data)
.venv\lib\site-packages\yaml\representer.py:27: in represent
    node = self.represent_data(data)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:58: in represent_data
    node = self.yaml_representers[None](self, data)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <airflow_declarative.schema.Dumper object at 0x000001FE36D4AB08>
data = <class 'airflow.operators.dummy.DummyOperator'>

    def represent_undefined(self, data):
>       raise RepresenterError("cannot represent an object", data)
E       yaml.representer.RepresenterError: ('cannot represent an object', <class 'airflow.operators.dummy.DummyOperator'>)

.venv\lib\site-packages\yaml\representer.py:231: RepresenterError
_____________________________ test_serde[version] _____________________________

path = 'c:\\Users\\kevin.roeschli\\Documents\\source\\repos\\airflow_test\\tests\\dags\\good\\version.yaml'

    def test_serde(path):
        schema0 = airflow_declarative.transform(airflow_declarative.schema.from_path(path))
>       content = airflow_declarative.render(path)

tests\test_good_dags.py:48: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv\lib\site-packages\airflow_declarative\__init__.py:62: in render
    return schema.dump(transform(schema.from_path(path)))
.venv\lib\site-packages\airflow_declarative\schema.py:81: in dump
    return yaml.dump(schema, Dumper=Dumper, *args, **kwargs)
.venv\lib\site-packages\yaml\__init__.py:290: in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
.venv\lib\site-packages\yaml\__init__.py:278: in dump_all
    dumper.represent(data)
.venv\lib\site-packages\yaml\representer.py:27: in represent
    node = self.represent_data(data)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
.venv\lib\site-packages\yaml\representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
.venv\lib\site-packages\yaml\representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
.venv\lib\site-packages\yaml\representer.py:58: in represent_data
    node = self.yaml_representers[None](self, data)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <airflow_declarative.schema.Dumper object at 0x000001FE36CCFDC8>
data = <class 'airflow.operators.dummy.DummyOperator'>

    def represent_undefined(self, data):
>       raise RepresenterError("cannot represent an object", data)
E       yaml.representer.RepresenterError: ('cannot represent an object', <class 'airflow.operators.dummy.DummyOperator'>)

.venv\lib\site-packages\yaml\representer.py:231: RepresenterError
____________________________ test_callback_params _____________________________

dag = <DAG: dag>

    def test_callback_params(dag):
        foo = dag.task_dict["operator_foo"]
        assert isinstance(foo, GenericOperator)
        foo.execute({})
>       assert foo._callback_instance.param == {"hello": ["привет"]}
E       AssertionError: assert {'hello': ['Ð¿Ñ€Ð¸Ð²ÐµÑ‚']} == {'hello': ['привет']}
E         Differing items:
E         {'hello': ['Ð¿Ñ€Ð¸Ð²ÐµÑ‚']} != {'hello': ['привет']}
E         Use -v to get the full diff

tests\test_template_with_dicts.py:42: AssertionError
____________________________ test_callback_params _____________________________

dag = <DAG: dag>

    def test_callback_params(dag):
>       foo = dag.task_dict["operator_foo"]
E       KeyError: 'operator_foo'

tests\test_template_with_from_stdout.py:39: KeyError
============================== warnings summary ===============================
tests/test_bad_dags.py::test_bad_dags[bad-email-ConfigError]
  C:\Users\kevin.roeschli\AppData\Local\Programs\Python\Python37\lib\importlib\__init__.py:127: DeprecationWarning: This module is deprecated. Please use `airflow.operators.dummy`.
    return _bootstrap._gcd_import(name[level:], package, level)

tests/test_bad_dags.py::test_bad_dags[class-and-callback-ConfigError]
  C:\Users\kevin.roeschli\AppData\Local\Programs\Python\Python37\lib\importlib\__init__.py:127: DeprecationWarning: This module is deprecated. Please use `airflow.providers.http.sensors.http`.
    return _bootstrap._gcd_import(name[level:], package, level)

tests/test_example_bash_operator.py::test_example_bash_operator
  C:\Users\kevin.roeschli\AppData\Local\Programs\Python\Python37\lib\importlib\__init__.py:127: DeprecationWarning: This module is deprecated. Please use `airflow.operators.bash`.
    return _bootstrap._gcd_import(name[level:], package, level)

tests/test_good_dags.py::test_serde[callback-tasks]
tests/test_good_dags.py::test_serde[integer_callback_arg]
tests/test_good_dags.py::test_serde[template-defaults]
tests/test_good_dags.py::test_serde[template-with-complex-jinja]
tests/test_good_dags.py::test_serde[template-with-dicts]
tests/test_good_dags.py::test_serde[template-with-from-stdout]
tests/test_good_dags.py::test_serde[template-with-using]
  c:\Users\kevin.roeschli\Documents\source\repos\airflow_test\tests\test_good_dags.py:49: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
    schema1 = airflow_declarative.schema.ensure_schema(yaml.load(content))

-- Docs: https://docs.pytest.org/en/stable/warnings.html
- generated xml file: C:\Users\KEVIN~1.ROE\AppData\Local\Temp\tmp-8120vPQL6Z01PgO6.xml -
=========================== short test summary info ===========================
FAILED tests/test_example_bash_operator.py::test_example_bash_operator - Asse...
FAILED tests/test_good_dags.py::test_serde[check-callbacks] - yaml.represente...
FAILED tests/test_good_dags.py::test_serde[email] - yaml.representer.Represen...
FAILED tests/test_good_dags.py::test_serde[example_bash_operator] - yaml.repr...
FAILED tests/test_good_dags.py::test_serde[executor-config] - yaml.represente...
FAILED tests/test_good_dags.py::test_serde[schedule-interval-cron-preset] - y...
FAILED tests/test_good_dags.py::test_serde[schedule-interval-cron] - yaml.rep...
FAILED tests/test_good_dags.py::test_serde[schedule-interval-none] - yaml.rep...
FAILED tests/test_good_dags.py::test_serde[version] - yaml.representer.Repres...
FAILED tests/test_template_with_dicts.py::test_callback_params - AssertionErr...
FAILED tests/test_template_with_from_stdout.py::test_callback_params - KeyErr...
================= 11 failed, 50 passed, 10 warnings in 6.21s ==================

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Airflow 2 support #46

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 0 comments

Select a reply

Airflow 2 support #46

kevroes Mar 22, 2021

Replies: 0 comments

kevroes
Mar 22, 2021