AIP-81 Central API Communication Mechanism for CLI and Connection Command Integration #45300
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
closes: #42561 and discarding dynamic generation of requests: #42562
Summary
This PR introduces a central API communication mechanism for the CLI and adopts one of the broadest endpoint ranges in the API, focusing on connection commands. After several iterations to automate API call generation and handle edge cases, I couldn't achieve a fully dynamic request generation process. Instead, I landed on a structure similar to TaskSDK <-> ExecutionAPI. This approach simplifies reviews, maintains consistency across clients, and opens the door for better automation in future iterations, possibly in 3.1 or 3.2.
Implementation Details
Note: There are two missing functionalities in this PR for the connection command. Those will be included with follow-up tasks.
Notes on task_command
While working on this, I realized that task_command behaves more like a local_command than a remote_command. This needs further discussion, especially regarding its integration with TaskSDK. From the CLI perspective, depending on TaskSDK seems cleaner than directly calling ExecutionAPI, it avoids duplication and keeps the structure consistent. Even if the CLI calls TaskSDK, the two-hop process to reach the ExecutionAPI feels a bit redundant. What do you think, @ashb (to all, please jump into discussion, I tagged Ash to follow up the previous discussion started on this in Slack)?
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.