Skip to content

Commit

Permalink
V0.6.0 release (wjohnson#120)
Browse files Browse the repository at this point in the history
  • Loading branch information
wjohnson authored May 10, 2021
1 parent 0952ce9 commit bb1a700
Show file tree
Hide file tree
Showing 3 changed files with 48 additions and 25 deletions.
40 changes: 26 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,25 @@
# PyApacheAtlas: API Support for Azure Purview and Apache Atlas

A python package to work with the Apache Atlas API and support bulk loading, custom lineage, and more from a Pythonic set of classes and Excel templates.
A python package to work with the Azure Purview and Apache Atlas API. Supporting bulk loading, custom lineage, and more from a Pythonic set of classes and Excel templates.

The package currently supports:
* Bulk upload of entities.
* Bulk upload of type definitions.
The package supports programmatic interaction and an Excel template for low-code uploads.

The Excel template provides a means to:
* Bulk upload entities
* Supports adding glossary terms to entities.
* Supports adding classifications to entities.
* Supports creating relationships between entities (e.g. columns of a table).
* Creating custom lineage between two existing entities.
* Bulk upload of type definitions.
* Bulk upload of classification definitions (Purview Classification rules are not currently supported).
* Creating custom table and complex column level lineage in the [Hive Bridge style](https://atlas.apache.org/0.8.3/Bridge-Hive.html).
* Supports Azure Purview ColumnMapping Attributes.
* Creating a column lineage scaffolding as in the Hive Bridge Style .
* Performing "What-If" analysis to check if...
* Your entities are valid types.
* Your entities are missing required attributes.
* Your entities are using undefined attributes.

The PyApacheAtlas package itself supports those operations and more for the advanced user:
* Programmatically create Entities, Types (Entity, Relationship, etc.).
* Perform partial updates of an entity (for non-complex attributes like strings or integers).
* Extracting entities by guid or qualified name.
* Creating custom lineage with Process and Entity types.
* Working with the glossary.
* Uploading terms.
* Downloading individual or all terms.
Expand All @@ -23,8 +30,12 @@ The package currently supports:
* Working with relationships.
* Able to create arbitrary relationships between entities.
* e.g. associating a given column with a table.
* Able to upload relationship definitions.
* Deleting types (by name) or entities (by guid).
* Creating a column lineage scaffolding as in the Hive Bridge Style .
* Performing "What-If" analysis to check if...
* Your entities are valid types.
* Your entities are missing required attributes.
* Your entities are using undefined attributes.
* Search (only for Azure Purview advanced search).
* Authentication to Azure Purview via Service Principal.
* Authentication using basic authentication of username and password for open source Atlas.
Expand Down Expand Up @@ -66,7 +77,7 @@ the better way to use this package with Purview.

### Create Entities "By Hand"

You can also create your own entities by hand with the helper `AtlasEntity` class. Convert it with `to_json` to prepare it for upload.
You can also create your own entities by hand with the helper `AtlasEntity` class.

```
from pyapacheatlas.core import AtlasEntity
Expand All @@ -86,7 +97,7 @@ ae = AtlasEntity(
)
# Upload that entity with the client
upload_results = client.upload_entities([ae.to_json()])
upload_results = client.upload_entities( [ae] )
```

### Create Entities from Excel
Expand All @@ -95,8 +106,9 @@ Read from a standardized excel template that supports...

* Bulk uploading entities into your data catalog.
* Creating custom table and column level lineage.
* Creating custom type definitions for datasets
* Creating custom type definitions for datasets.
* Creating custom lineage between existing assets / entities in your data catalog.
* Creating custom classification (Purview Classification rules are not supported yet).

See end to end samples for each scenario in the [excel samples](./samples/excel/README.md).

Expand All @@ -105,6 +117,6 @@ Learn more about the Excel [features and configuration in the wiki](https://gith
## Additional Resources

* Learn more about this package in the [github wiki](https://github.com/wjohnson/pyapacheatlas/wiki/Excel-Template-and-Configuration).
* The [Apache Atlas client in Python](https://pypi.org/project/pyatlasclient/)
* The [Apache Atlas REST API](http://atlas.apache.org/api/v2/)
* The [Purview CLI Package](https://github.com/tayganr/purviewcli) provides CLI support.
* Purview [REST API Official Docs](https://docs.microsoft.com/en-us/azure/purview/tutorial-using-rest-apis)
2 changes: 1 addition & 1 deletion pyapacheatlas/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "0.5.0"
__version__ = "0.6.0"
31 changes: 21 additions & 10 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,26 @@
LONG_DESCRIPTION = """
# PyApacheAtlas: API Support for Azure Purview and Apache Atlas
A python package to work with the Apache Atlas API and support bulk loading, custom lineage, and more from a Pythonic set of classes and Excel templates.
A python package to work with the Azure Purview and Apache Atlas API. Supporting bulk loading, custom lineage, and more from a Pythonic set of classes and Excel templates.
The package currently supports:
* Bulk upload of entities.
* Bulk upload of type definitions.
The package supports programmatic interaction and an Excel template for low-code uploads.
The Excel template provides a means to:
* Bulk upload entities
* Supports adding glossary terms to entities.
* Supports adding classifications to entities.
* Supports creating relationships between entities (e.g. columns of a table).
* Creating custom lineage between two existing entities.
* Bulk upload of type definitions.
* Bulk upload of classification definitions (Purview Classification rules are not currently supported).
* Creating custom table and complex column level lineage in the [Hive Bridge style](https://atlas.apache.org/0.8.3/Bridge-Hive.html).
* Supports Azure Purview ColumnMapping Attributes.
* Creating a column lineage scaffolding as in the Hive Bridge Style .
* Performing "What-If" analysis to check if...
* Your entities are valid types.
* Your entities are missing required attributes.
* Your entities are using undefined attributes.
The PyApacheAtlas package itself supports those operations and more for the advanced user:
* Programmatically create Entities, Types (Entity, Relationship, etc.).
* Perform partial updates of an entity (for non-complex attributes like strings or integers).
* Extracting entities by guid or qualified name.
* Creating custom lineage with Process and Entity types.
* Working with the glossary.
* Uploading terms.
* Downloading individual or all terms.
Expand All @@ -27,8 +34,12 @@
* Working with relationships.
* Able to create arbitrary relationships between entities.
* e.g. associating a given column with a table.
* Able to upload relationship definitions.
* Deleting types (by name) or entities (by guid).
* Creating a column lineage scaffolding as in the Hive Bridge Style .
* Performing "What-If" analysis to check if...
* Your entities are valid types.
* Your entities are missing required attributes.
* Your entities are using undefined attributes.
* Search (only for Azure Purview advanced search).
* Authentication to Azure Purview via Service Principal.
* Authentication using basic authentication of username and password for open source Atlas.
Expand Down

0 comments on commit bb1a700

Please sign in to comment.