evaluating PATH in justfile comes too late when using which()

[woutervh]

just 1.39.0

testing the which-command:

use-case: run commands in a python-virtualenv,
some commands might be installed in the .venv and others might not.

when i run a command via just, i want to display the absolute path of the command

this works when ruff is installed globally, outside the .venv:
note: I dont want to "activate my python-env".

# prepend local bin-directory to $PATH
venv_bin_dir := if os_family() == "windows" { justfile_directory() / ".venv/Scripts" } else { justfile_directory() / ".venv/bin" }
export PATH := if os_family() == "windows" { venv_bin_dir + x";${PATH}" } else { venv_bin_dir + x":${PATH}" }


RUFF_EXE := which("ruff")

# show which ruff is used
[group: 'ruff']
ruff-which:
    @ echo -e "Using executable {{RUFF_EXE}}"

# run ruff on python-files
[group: 'ruff']
ruff args="":
    - {{RUFF_EXE}} docs/ etc/ src/ tests/ {{args}}

global ruff on global $PATH is found

 just ruff-which
Using executable /opt/tools/bin/ruff

local ruff in .venv/bin/ruff is not found

 just ruff-which
Using executable         --> not found, empty 

It seems tthat the updated PATH in just comes to late,
after evaluating RUFF_EXE := which("ruff")

[casey]

This is actually quite tricky!

I was thinking about this yesterday, but we have had the same issue with backticks and the shell() command since forever, so I was hoping nobody would notice 😅

The order of assignment evaluation is not defined, so if you have:

FOO := which('foo')
export PATH := "/some/directory"

It is not defined whether or not FOO or PATH is evaluated first. Because of this, export statements don't apply to the scope in which they appear, only inner scopes.

So other assignments in the same scope will not see the export, but recipe defaults and recipe lines will see the export:

export FOO := "BAR"

A := `echo $FOO` # will not see export

foo a=`echo $FOO`: # will see export
  echo $FOO # will see export

This is further complicated by the fact that export statements don't appear to functions at all, including env and which, only sub-processes, like recipes and backticks. env and which only see the environment variables of the just process itself, as well as any defined in .env files.

I would suggest working around this by putting the PATH variable in a .env file, but .env files only work for variables that are not defined, they do not override defined variables, and the PATH variable is certainly defined outside of the .env.

Variables in the same scope are allowed to depend on each other, however. The just analyzer detects these dependencies, and makes sure that dependencies are evaluated and their value available before dependents.

So we could do something where we add an implicit dependency of which() to any export variable named PATH in the same scope, and use its value if available. This would be a huge hack, but it's probably what people expect, so I'm a bit ambivalent about it.

[woutervh]

but .env files only work for variables that are not defined, they do not override defined variables,

I have bad experiences with managing PATH in .env-files. As I can easily screw up external tools like docker/vscode