forked from airbytehq/airbyte
-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: add public documentation for declarative oauth (airbytehq#52041)
Co-authored-by: Oleksandr Bazarnov <[email protected]>
- Loading branch information
Showing
16 changed files
with
1,592 additions
and
425 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
321 changes: 0 additions & 321 deletions
321
docs/connector-development/config-based/advanced-topics.md
This file was deleted.
Oops, something went wrong.
3 changes: 3 additions & 0 deletions
3
...onnector-development/config-based/advanced-topics/component-schema-reference.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# Component Schema Reference | ||
|
||
A JSON schema representation of the relationships between the components that can be used in the YAML configuration can be found [here](https://github.com/airbytehq/airbyte-python-cdk/blob/main/airbyte_cdk/sources/declarative/declarative_component_schema.yaml). |
58 changes: 58 additions & 0 deletions
58
docs/connector-development/config-based/advanced-topics/custom-components.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
# Custom Components | ||
|
||
:::info | ||
Please help us improve the low code CDK! If you find yourself needing to build a custom component,please [create a feature request issue](https://github.com/airbytehq/airbyte/issues/new?assignees=&labels=type%2Fenhancement%2C+%2Cneeds-triage%2C+area%2Flow-code%2Fcomponents&template=feature-request.md&title=Low%20Code%20Feature:). If appropriate, we'll add it directly to the framework (or you can submit a PR)! | ||
|
||
If an issue already exist for the missing feature you need, please upvote or comment on it so we can prioritize the issue accordingly. | ||
::: | ||
|
||
Any built-in components can be overloaded by a custom Python class. | ||
To create a custom component, define a new class in a new file in the connector's module. | ||
The class must implement the interface of the component it is replacing. For instance, a pagination strategy must implement `airbyte_cdk.sources.declarative.requesters.paginators.strategies.pagination_strategy.PaginationStrategy`. | ||
The class must also be a dataclass where each field represents an argument to configure from the yaml file, and an `InitVar` named parameters. | ||
|
||
For example: | ||
|
||
``` | ||
@dataclass | ||
class MyPaginationStrategy(PaginationStrategy): | ||
my_field: Union[InterpolatedString, str] | ||
parameters: InitVar[Mapping[str, Any]] | ||
def __post_init__(self, parameters: Mapping[str, Any]): | ||
pass | ||
def next_page_token(self, response: requests.Response, last_records: List[Mapping[str, Any]]) -> Optional[Any]: | ||
pass | ||
def reset(self): | ||
pass | ||
``` | ||
|
||
This class can then be referred from the yaml file by specifying the type of custom component and using its fully qualified class name: | ||
|
||
```yaml | ||
pagination_strategy: | ||
type: "CustomPaginationStrategy" | ||
class_name: "my_connector_module.MyPaginationStrategy" | ||
my_field: "hello world" | ||
``` | ||
### Custom Components that pass fields to child components | ||
There are certain scenarios where a child subcomponent might rely on a field defined on a parent component. For regular components, we perform this propagation of fields from the parent component to the child automatically. | ||
However, custom components do not support this behavior. If you have a child subcomponent of your custom component that falls under this use case, you will see an error message like: | ||
``` | ||
Error creating component 'DefaultPaginator' with parent custom component source_example.components.CustomRetriever: Please provide DefaultPaginator.$parameters.url_base | ||
``` | ||
When you receive this error, you can address this by defining the missing field within the `$parameters` block of the child component. | ||
|
||
```yaml | ||
paginator: | ||
type: "DefaultPaginator" | ||
<...> | ||
$parameters: | ||
url_base: "https://example.com" | ||
``` |
Oops, something went wrong.