Twice-included Taskfiles have incorrect directory as TASKFILE_DIR as well as pwd output

[bjoernahrens]

I have a Taskfile.dist.yml file that is checked into the git repository. This Taskfile includes another Taskfile.dist.yml from /frontend.
This file has a variable that depends on the current directory being /frontend.
This setup works fine.

I now want to create a local Taskfile.yml with my custom commands.
In that file, I include Taskfile.dist.yml. I can call the functions from the taskfile in frontend. However, the variable now has an incorrect directory, it uses /.

Reproducable example

Taskfiles

Taskfile.yml

version: "3"

includes:
  dist:
    taskfile: Taskfile.dist.yml

Taskfile.dist.yml

version: "3"

includes:
  frontend:
    taskfile: ./frontend/Taskfile.dist.yml
    dir: ./frontend

frontend/Taskfile.dist.yml

version: "3"

vars:
  PWD_OUTPUT:
    sh: pwd

tasks:
  where-am-I:
    cmds:
      - echo "PWD_OUTPUT  {{ .PWD_OUTPUT }}"
      - echo "TASKFILE    {{.TASKFILE_DIR}}"
      - echo "USER_WORK   {{.USER_WORKING_DIR}}"
      - echo "ROOT_DIR    {{.ROOT_DIR}}"

Shell commands and incorrect result

Actual

When the command is run from Taskfile.dist.yml, the output from pwd as well as the TASKFILE_DIR variable end with /frontend. When it's run from Taskfile.yml, it's the same as ROOT_DIR.

❯ task -t Taskfile.yml dist:frontend:where-am-I
task: [dist:frontend:where-am-I] echo "PWD_OUTPUT  /Users/taskuser/Desktop/task-issue-demo"
PWD_OUTPUT  /Users/taskuser/Desktop/task-issue-demo
task: [dist:frontend:where-am-I] echo "TASKFILE    /Users/taskuser/Desktop/task-issue-demo"
TASKFILE    /Users/taskuser/Desktop/task-issue-demo
task: [dist:frontend:where-am-I] echo "USER_WORK   /Users/taskuser/Desktop/task-issue-demo"
USER_WORK   /Users/taskuser/Desktop/task-issue-demo
task: [dist:frontend:where-am-I] echo "ROOT_DIR    /Users/taskuser/Desktop/task-issue-demo"
ROOT_DIR    /Users/taskuser/Desktop/task-issue-demo

❯ task -t Taskfile.dist.yml frontend:where-am-I
task: [frontend:where-am-I] echo "PWD_OUTPUT  /Users/taskuser/Desktop/task-issue-demo/frontend"
PWD_OUTPUT  /Users/taskuser/Desktop/task-issue-demo/frontend
task: [frontend:where-am-I] echo "TASKFILE    /Users/taskuser/Desktop/task-issue-demo/frontend"
TASKFILE    /Users/taskuser/Desktop/task-issue-demo/frontend
task: [frontend:where-am-I] echo "USER_WORK   /Users/taskuser/Desktop/task-issue-demo"
USER_WORK   /Users/taskuser/Desktop/task-issue-demo
task: [frontend:where-am-I] echo "ROOT_DIR    /Users/taskuser/Desktop/task-issue-demo"
ROOT_DIR    /Users/taskuser/Desktop/task-issue-demo

Expected

The result of the same task (dist:frontend:where-am-I and frontend:where-am-I) from both Taskfiles (Taskfile.yml and Taskfile.dist.yml) should return the correct subfolder for TASKFILE_DIR and pwd instead of the root folder:

/Users/taskuser/Desktop/task-issue-demo/frontend

• Task version: 3.28.0
• Operating system: macOS Ventura 13.5 (22G74)
• Experiments enabled: no

[luckydye]

Came across the same issue!

[A13xB0]

Same issue for me, any update?

[andreynering]

I managed to reproduce the issue, but couldn't quickly find the cause of the problem.

[glenvan]

The issue is with chains of includes in general.

• .TASKFILE_DIR in included Taskfiles seems to be set by the dir: in the top-level Taskfile only
• If you include other Taskfiles within those intermediate Taskfiles, the dir: specified in your intermediate Taskfiles is not used to set the .TASKFILE_DIR in leaf-level Taskfiles
• However, the working directory does track correctly in the included Taskfiles as long as you specify the dir: in the including Taskfile
  • But not for a global variable unless you're careful and able to add dir: to your includes

Here's a little experiment. Download the attachment (taskfiles.zip) or create a folder hierarchy like tasksfiles/sub1/sub2 and add these files:

taskfiles/Taskfile.dist.yml:

version: '3'

output: prefixed

includes:
  sub1:
    taskfile: ./sub1/Sub1Tasks.yml
    dir: ./sub1
  sub2:
    taskfile: ./sub1/sub2/Sub2Tasks.yml
    dir: ./sub1/sub2

tasks:
  default:
    cmds:
      - task: echo
      - task: sub1:echo
      - task: sub2:echo
      - task: sub1:sub2:echo
    silent: true

  echo:
    cmds:
      - 'echo ".TASKFILE_DIR: {{.TASKFILE_DIR}}"'
      - 'echo "         \$PWD: $PWD"'
      - 'echo "          pwd: $(pwd)"'
    silent: true

taskfiles/sub1/Sub1Tasks.yml:

version: '3'

includes:
  sub2:
    taskfile: ./sub2/Sub2Tasks.yml
    dir: ./sub2 # Will be ignored by .TASKFILE_DIR

tasks:
  echo:
    cmds:
      - 'echo ".TASKFILE_DIR: {{.TASKFILE_DIR}}"'
      - 'echo "         \$PWD: $PWD"'
      - 'echo "          pwd: $(pwd)"'
    silent: true

taskfiles/sub1/sub2/Sub2Tasks.yml:

version: '3'

vars:
  PWD_OUTPUT:
    sh: pwd

tasks:
  echo:
    cmds:
      - 'echo ".TASKFILE_DIR: {{.TASKFILE_DIR}}"'
      - 'echo "         \$PWD: $PWD"'
      - 'echo "          pwd: $(pwd)"'
      - 'echo "  .PWD_OUTPUT: {{.PWD_OUTPUT}}"'
    silent: true

Then execute:

$ task
[echo] .TASKFILE_DIR: /home/some.guy/src/taskfiles
[echo]          $PWD: /home/some.guy/src/taskfiles
[echo]           pwd: /home/some.guy/src/taskfiles
[sub1:echo] .TASKFILE_DIR: /home/some.guy/src/taskfiles/sub1
[sub1:echo]          $PWD: /home/some.guy/src/taskfiles/sub1
[sub1:echo]           pwd: /home/some.guy/src/taskfiles/sub1
[sub2:echo] .TASKFILE_DIR: /home/some.guy/src/taskfiles/sub1/sub2
[sub2:echo]          $PWD: /home/some.guy/src/taskfiles/sub1/sub2
[sub2:echo]           pwd: /home/some.guy/src/taskfiles/sub1/sub2
[sub2:echo]   .PWD_OUTPUT: /home/some.guy/src/taskfiles/sub1/sub2
[sub1:sub2:echo] .TASKFILE_DIR: /home/some.guy/src/taskfiles/sub1      <-- Look here!
[sub1:sub2:echo]          $PWD: /home/some.guy/src/taskfiles/sub1/sub2
[sub1:sub2:echo]           pwd: /home/some.guy/src/taskfiles/sub1/sub2
[sub1:sub2:echo]   .PWD_OUTPUT: /home/some.guy/src/taskfiles/sub1/sub2

• Intuitively, sub1:sub2:echo and sub2:echo should have the same .TASKFILE_DIR
• However, .TASKFILE_DIR in the sub1 namespace has the dir: set in the root Taskfile.yml's sub1 include

Now comment out the dir: specifications in only the Taskfile.yml:

$ task
[echo] .TASKFILE_DIR: /home/some.guy/src/taskfiles
[echo]          $PWD: /home/some.guy/src/taskfiles
[echo]           pwd: /home/some.guy/src/taskfiles
[sub1:echo] .TASKFILE_DIR: /home/some.guy/src/taskfiles
[sub1:echo]          $PWD: /home/some.guy/src/taskfiles
[sub1:echo]           pwd: /home/some.guy/src/taskfiles
[sub2:echo] .TASKFILE_DIR: /home/some.guy/src/taskfiles
[sub2:echo]          $PWD: /home/some.guy/src/taskfiles
[sub2:echo]           pwd: /home/some.guy/src/taskfiles
[sub2:echo]   .PWD_OUTPUT: /home/some.guy/src/taskfiles
[sub1:sub2:echo] .TASKFILE_DIR: /home/some.guy/src/taskfiles
[sub1:sub2:echo]          $PWD: /home/some.guy/src/taskfiles/sub1/sub2
[sub1:sub2:echo]           pwd: /home/some.guy/src/taskfiles/sub1/sub2
[sub1:sub2:echo]   .PWD_OUTPUT: /home/some.guy/src/taskfiles

1. All instances of .TASKFILE_DIR resolve to .ROOT_DIR
2. .PWD_OUTPUT (from Sub2Tasks.yml) also resolves to .ROOT_DIR
3. But the dynamically-set working directory still tracks for sub1:sub2:echo because of the dir: in Sub1Tasks.yml's includes
   • If you remove that dir: the PWD will be /home/some.guy/src/taskfiles/sub1, strangely, so it's making some kind of effort...

Circling back to the original complaint:

• The use of an overriding Taskfile.yml that includes Taskfile.dist.yml botches the .TASKFILE_DIR for frontend/Taskfile.dist.yml because Taskfile.dist.yml's intermediate dir: spec isn't being respected
• sh: pwd for variable init returns different results depending on whether there is a full chain of dir: specifications
• There doesn't seem to be a way to set dir: ./ in the root include to at least preserve the variable init
  • I tried a number of ways (., ./, ../task-issue-demo...) and it's just being ignored

[explosivose]

Task version: 3.31.0

For nested includes there is also inconsistent behaviour depending on how you include

Taskfile.yaml

version: '3'
includes: ./tasks

tasks/Taskfile.yaml

version: '3'
includes: ./subject # <-- case 1 included via shorthand

tasks/subject/Taskfile.yaml

version: '3'
tasks:
  hello:
    cmds:
      # expect same output regardless of how Taskfile is included 
      - realpath .
      - echo TASKFILE_DIR {{.TASKFILE_DIR}}

Output case 1 (shorthand)

I expect to see pwd followed by "absolute path of included Taskfile" which I presume should be the "leaf" taskfile if in a tree of inclusions. The output prints out pwd as expected but TASKFILE_DIR seems wrong, expected to be project/tasks/subject.

➜   task common:subject:hello -s
[redacted]/project
TASKFILE_DIR .

Case 2 (longhand)

Then for case 2 I change tasks/Taskfile.yaml to

version: '3'
includes: 
  taskfile: ./subject # <-- case 2 included via longhand

Output case 2

Same expectations, but now both outputs seem wrong, although TASKFILE_DIR changed to an absolute path.

[redacted]/project/tasks
TASKFILE_DIR [redacted]/project/tasks

[Gowiem]

Has anyone found a workaround to this issue? Feel super painful.

[pd93]

An unintended side-effect of the v3.35.1 release is that this is now working as expected for the examples in #1301 (comment) and #1301 (comment). This is because TASKFILE_DIR is no longer calculated, but stored when a Taskfile is read. This should make the result much more reliable.

@explosivose, I was not able to replicate your specific example. Mainly because as far as I can tell, the shorthand syntax for includes that you are describing is not allowed by Task and results in an error.

[Gowiem]

@pd93 bumping to v3.35.1 did the trick -- Thanks for the fix!