Use ocamlfind by default

[hcarty]

MPR discussion: http://caml.inria.fr/mantis/view.php?id=5547

Now that ocamlbuild is being developed and distributed separately from the compiler it would be nice to make this change.

[gasche]

I'm open to pull requests implementing this feature.

Note that the problem with PR#5547 never was that ocamlbuild was distributed with the compiler. The issue was that we tried to make -use-ocamlfind the default before releasing a version with -no-ocamlfind, which made it impossible for user to properly manage the transition. We have had several releases since then, so we could consider again making -use-ocamlfind the default.

This means work to ensure a smooth transition, though. I won't have time in the near future to handle it myself; if you want to push forward, please feel welcome to.

[dbuenzli]

Note that this makes ocamlbuild dependent on ocamlfind which is not the case at the moment.

[gasche]

The default configuration of ocamlbuild would generate calls to ocamlfind, whereas it currently does not. ocamlbuild would remain fully functional in absence of ocamlfind, you would just have to pass -no-ocamlfind explicitly -- just like a majority of our users have to pass -use-ocamlfind explicitly today, I feel okay with the change as long as the no-ocamlfind use-case is preserved (because people have told it is important).

[dbuenzli]

But then if you do opam install ocambuild and you invoke ocamlbuild and it fails because there's no ocamlfind that's not a very good user experience.

[hcarty]

@gasche Is it important to support (a) ocamlbuild without the presence of ocamlfind or (b) without invoking ocamlfind? If it's (a) then we run into the user experience problem @dbuenzli points out. If it's (b) then ocamlbuild should depend on ocamlfind, at least within opam, if -use-ocamlfind becomes the default.

[gasche]

I don't know, we should ask our no-ocamlfind users ( cc @avsm ? ). I would guess that, as you remark, requiring ocamlfind to be installed is okay. On the other hand, this is purely a packaging issue, not an ocamlbuild-development issue; for example, it may be possible to propose ocamlfind as an optional dependency, or to have some kind of virtual "ocamlfind or explicit opt-out" package to depend on.

The user experience I have in mind would be to simply print "please use -no-ocamlfind or install ocamlfind" when -use-ocamlfind is used and ocamlfind is missing. I would think that imposing this inconvenience to the people that wish to not use ocamlfind is reasonable, as those people are experts (we should be clear but we don't need to be hand-holding); this is better in particular than trying to set the default value by dynamically checking whether ocamlfind is present, which would open door to much hard-to-reproduce pain.

The problem we must solve is making sure that people that do not explicitly wish to use ocamlbuild without ocamlfind do not end up in that situation by chance. One easy solution to this problem is to have the default user-facing switches in OPAM install ocamlfind by default.

[dbuenzli]

I think that the tool behaviour should stay as it is now, with the -no-ocamlfind and -use-ocamlfind flags as they exist with their semantics and that the default should be -no-ocamlfind as it has always been.

However what should change is that package(PKG) tags should work regardless without having to issue -use-ocamlfind.

[gasche]

Are you suggesting that using package(...) would automagically set the behaviour as if -use-ocamlfind was present, or that we try to provide an implementation for package without calling ocamlfind in the compiler commands (this has been done in the past, there is logic in ocamlbuild to ask ocamlfind for the paths at build-planning time and generate the correct compiler invocations directly; this was fragile and I would rather not go in this direction again)?

[bluddy]

Would it make sense to have an optional configuration file for ocamlbuild that would specify whether ocamlfind is to be used? 90% of the use-cases will involve using ocamlfind, so why not make it easy for it to be the default?

[dbuenzli]

Are you suggesting that using package(...) would automagically set the behaviour as if -use-ocamlfind was present,

Argh no. This would select the actual compiler binaries depending on whether your _tags file has package tag or not which would be horrible.

(this has been done in the past, there is logic in ocamlbuild to ask ocamlfind for the paths at build-planning time and generate the correct compiler invocations directly; this was fragile and I would rather not go in this direction again)?

I think build systems should go in that direction and avoid the ocamlfind indirection for the compiler tools themselves. The number of logical names and tool indirections we have in the system at the moment is insane and we should aim at taking steps at simplifying this in general.

ocamlfind should be seen as a flag source rather than taking over the compiler tools. (i.e. ocamlc $(ocamlfind flags pkg) rather than ocamlfind ocamlc -pkg pkg). This allows to avoid tangling the tools too much together (and also provide interaction with other tools like pkg-config in an uniform manner) and avoids hiding compilation logic inside ocamlfind itself.

The problem is that ocamfind flag query capabilities are largely broken and noisy (passes flags that are not relevant to certain phases), something that Gerd doesn't seem to be willing to acknowledge and furthermore he doesn't want to move this model (see the discussion before and after this message, not that the "ghost animal" he refers to is not the one I am actually chasing in the discussion). Regarding query the best way seems to be either this or that.

In any case, the current status quo seems good enough and I would wait for the cross-compilation support landing at the opam level before making decisions on this issue. This will allow to lay down a more sensitive and forward looking strategy.

[gasche]

I feel that we should be very prudent about changing default behaviors; for this reason, despite the fact that I personally feel making -use-ocamlfind the default would be the best move, I am tempted to agree on your proposal to wait for a later time. I don't have a definitive opinion on this.

In the grand scheme of things I see your point about avoiding tool entanglement, but in the short to medium term I don't want to pay the price of developing and maintaining a robust ocamlbuild-level logic to replace what we have now. It was tried previously and it was worse than the current state -- it's hard to do well, as it seems you know even better than I do.

[hcarty]

@bluddy You can enable ocamlfind with this line in your myocamlbuild.ml:

let () = Ocamlbuild_plugin.Options.use_ocamlfind := true

If the longer term goal is to either rework or rethink how ocamlfind is used I'm ok with that approach too. Feel free to close this issue if that's the case.

[gasche]

@hcarty note that it would be more robust to set this setting from inside a hook, at either the Before_options or After_options stage (depending on whether or not you wish to override manual flags).

Intuitively I feel that it would be better for everyone to not ask for any explicit mention of ocamlfind to use it, and require the people that explicitly wish to not use it to modify their myocamlbuild.ml to set false here or pass -no-ocamlfind directly -- note that both changes preserve compatibility with older ocamlbuild versions. In practice, changing defaults is complicated.

[hcarty]

For people who just want to build their code easily, is juggling Before_options/After_options really appropriate?

[bluddy]

@dbuenzli: Is there a reason we can't just use pkg-config for ocaml projects?

[whitequark]

Is there a reason we can't just use pkg-config for ocaml projects?

pkg-config is not really appropriate. It does not know, for example, about the OCaml linking semantics, or preprocessors. This would also be an incredibly intrusive change.

[gasche]

Closing this in favor of the duplicate #214 where the discussion is (for now) more focused on the default issue. (Plus I'm embarrassed by the discussion I started on "magically setting use-ocamlfind if package is used" in this thread, it was a really bad idea.)