taro and apipie

[jaynetics]

Hi @ekohl @h-lame @m-nakamura145 @mathieujobin @nathanpalmer @PanosCodes 👋

As you recently contributed to apipie, I thought you might be interested in taro, which is a new ruby gem that can be considered an alternative or replacement for apipie.

Maybe it is of use for you, or you have some thoughts on it, or you would even like to join in.

I would particularly love to hear if anything is unclear or confusing about the documentation.

If you want to know how taro and apipie compare, there's a short section about that in the readme as well.

If you're wondering why i started this instead of contributing to apipie, i'm happy to explain in detail. The short story is: I was using apipie in a large project and encountered a lot of problems. I didn't want to reinvent the wheel or fracture efforts, but as I looked closely at contributing fixes, it became clear that getting apipie to work well for our use case would be many times as much work as opting for a clean slate.

If you're interested in what specific problems i found, i can provide a list, although i'm not sure which of these are perhaps already covered by the existing open issues.

Feel free to mute this thread, of course, if you're not interested.

Best regards and a happy festive season,
J.

[ekohl]

Thank you for sharing. Right now I care about it because @theforeman uses apipie. In fact, the whole apipie project was started by Foreman developers. Because of that our ecosystem has various crucial tools (like our CLI but also our Ansible modules) that rely on the apipie definitions. Switching over would be a significant effort. Not that I'm principally opposed to it but you can imagine that it's a huge amount of work.

I appreciate you added a section in README to migrate, but I wonder if it makes sense for a controller to opt into the DSL. Right now I see you have unconditional code to extend ActionController:

taro/lib/taro/rails/railtie.rb

Lines 3 to 7 in 288d252

	# The `:action_controller` hook fires for both ActionController::API
	# and ActionController::Base, executing the block in their context.
	ActiveSupport.on_load(:action_controller) do
	extend Taro::Rails::DSL
	end

Now I'm thinking out loud, but what if you hide that behind some setting. Then in Foreman we could create an API v3 which would use some ApiV3Controller` that includes the Taro DSL. It wouldn't interfere with the existing v2 API and allow us to have a migration strategy. If I read the code well you work by inspecting the Rails routes so there's no need for it to be in the base controller.

[jaynetics]

the whole apipie project was started by Foreman developers

That's cool to know, i wasn't aware!

our ecosystem has various crucial tools (like our CLI but also our Ansible modules) that rely on the apipie definitions.

Are these tools directly based on apipie specifics or on apipie's openapi export? If you consider migrating to taro, relying on its openapi export might be the more modular approach.

I just had a look at theforeman/foreman. Its a bit smaller than the project i migrated with regexp search & replace. Building a codemod to do it could also make sense, especially if there are more repos to migrate. I think you're also using some apipie features not present in taro, e.g. example, which we can add, or see (not sure what that does).

I wonder if it makes sense for a controller to opt into the DSL

This sounds like a good idea!

There could also be a "hygienic"/unobtrusive mode where you do something like:

Taro.declare do
  api 'foo'
  param :bar, ...
end
def index
  ...

For now, it is possible to avoid a conflict between taro and apipie with a little hack as described in the README section. I've just clarified it a bit:

Taro uses some of the same DSL as apipie, so for a step-by-step migration, you might want to make taro use a different one. This initializer will change the taro DSL to taro_api, taro_param, and taro_returns and leave api, param, and returns to apipie:

# config/initializers/taro.rb
%i[api param returns].each do |m|
  Taro::Rails::DSL.alias_method("taro_#{m}", m) # `taro_api` etc.
  Taro::Rails::DSL.define_method(m) { |*a, **k, &b| super(*a, **k, &b) }
end

[ekohl]

Are these tools directly based on apipie specifics or on apipie's openapi export? If you consider migrating to taro, relying on its openapi export might be the more modular approach.

The OpenAPI output of apipie was added by a contributor and we don't use it AFAIK.

Our CLI is hammer-cli and that uses apipie-bindings. Note the actual functionality is implemented via plugins

Then we also have foreman-ansible-modules which uses apypi. The latter we also use in some tests.

Those would all need to be rewritten to be OpenAPI based.

I just had a look at theforeman/foreman. Its a bit smaller than the project i migrated with regexp search & replace. Building a codemod to do it could also make sense, especially if there are more repos to migrate. I think you're also using some apipie features not present in taro, e.g. example, which we can add, or see (not sure what that does).

Please realize that we also implement a lot of functionality via plugins. Espcially katello is a large plugin with lots of endpoints.

One complexity is that plugins can extend models from Foreman which also means you need to extend the API output. This isn't nice and I wish we didn't have this, but it's where we are today.

I think you're also using some apipie features not present in taro, e.g. example, which we can add, or see (not sure what that does).

IMHO we don't have enough examples today, but the current structure is also not great IMHO. If you add this, I wouldn't blindly copy the current implementation and critically review the design.

Though I didn't check, the see tag is probably similar to yard's @see and used to refer to other resources (like external links with further documentation).

There could also be a "hygienic"/unobtrusive mode where you do something like:

I like that idea.

Looking at a migration strategy, I'd probably look at creating a V3 API that is Taro based. Then deprecate V2 while we migrate the ecosystem. There are plenty of things in our API I don't like and it could be a good opportunity to clean those up. It does mean shipping both apipie and taro for a few releases, but it gives plugin developers a chance to migrate over time.

[jaynetics]

Ok, I've had a look around the ecosystem now, checking out quite a few of the repos with their different patterns and levels of maintenance, and now i'm actually amazed that you even consider migrating 😄

If there's anything you need, let me know.

One complexity is that plugins can extend models from Foreman which also means you need to extend the API output. This isn't nice and I wish we didn't have this, but it's where we are today.

How is this done? Does the main app know the plugins, or is there something else that merges the results together?

examples [...]. If you add this, I wouldn't blindly copy the current implementation and critically review the design.

Yeah, i've been wondering about it. There are also alternative approaches. Some tools can generate examples based on openapi types (covering permutations e.g. for nullability), though the results can be a bit nondescript. Another one is to record examples from spec runs with a tool like dox, and then merge them into the export.

the see tag is probably similar to yard's @see and used to refer to other resources (like external links with further documentation)

I checked now and see can only reference other declarations via an endpoint-like string, e.g. "users#create". Not a bad idea, and it could work with openapi refs as well, but its probably one of those things i'd rather not add to keep the scope manageable.

[mathieujobin]

For me, the web docs are critical, so we won't be migrating.

Apipie does what we need already.

I hope Apipie remain active and maintained.

[ekohl]

There are many tools that can generate documentation from the OpenAPI specification. https://tools.openapis.org/categories/documentation.html lists some. While I don't have experience with using them, I think some of them are very good. Way better than what apipie generates.

[jaynetics]

We're very happy with RapiDoc. It generates examples automatically as well. (We used it even when we still had apipie because the docs didn't render properly for self-describing properties – maybe related to issue #623.)