Support declaring non-string (bool, maps, slices, etc) variables

[philippgille]

When writing the Taskfile.yml many people probably write it like any other yaml, which means values like false is a boolean, as the yaml spec says: http://yaml.org/type/bool.html

But it turns out to be used as string and when using the template language it's actually "false". This is bad for {{if .SOME_VAR}} for example, which doesn't work, so you have to write {{if eq "true" .SOME_VAR}} for example, or even a much longer expression given the valid forms of a bool in the spec (e.g. "no", "NO", "No", n", ...).

Note: I found the same issue in a different Go repository: concourse/concourse#294
Unfortunately they only link to a Pivotal tracker and the link is 404, so I don't know if and how they dealt with this issue.

If you explicitly don't want to adhere to the yaml spec (and of course there might be good reasons for it), this behaviour should at least be mentioned in the documentation. Currently the "Variables" section doesn't do this: https://taskfile.org/#/usage?id=variables

Example Taskfile, created with task --init and then customized:

# github.com/go-task/task

version: '2'

vars:
  GREETING: Hello, World!
  IS_LIB: false

tasks:
  default:
    cmds:
      - echo "{{.GREETING}}"
      - echo "{{.IS_LIB}}"
      - echo "{{if .IS_LIB}} error {{else}} ok {{end}}"
      - echo "{{if eq "true" .IS_LIB}} error {{else}} ok {{end}}"
    silent: true

The output is:

task: No argument given, trying default task
Hello, World!
false
 error 
 ok 

task Version: Commit 309cfb14995ec511546726ab607d53c55c803d0f

OS: Ubuntu 14.04

[smyrman]

I have met this limitaruon as well. With some effort, I believe not only Boolean, but also array and numeric values could be supported as static values.

Unfortunantly maps would probably be of the table because the syntax for dynamic variables is:

vars:
  myvar:
    sh: cat myfile.txt

For our own use cases, we ended up sticking with string values rather than fixing it (we wanted maps and Boolean).

[andreynering]

Hi @philippgille,

As said by @smyrman, there's a syntax conflict with the sh: keyword, so implementation wise we need to find a workaround.

That said, I agree this would be very useful.

This is slightly related to #82

[smyrman]

Well, there is no conflict for Booleans, numbers and arrays.

vars:
  myvar: <string, bool, number or array>

We can expand that with an explicit value key that will be parse into an equivalent of Var.Static, and also allows maps and dollar ($) prefixes in strings:

vars:
  myvar:
    value: <any yaml value>

---

Implementation wise, at least these changes must be made:

Change the Var type

task/internal/taskfile/var.go

Line 32 in 309cfb1

type Var struct {

To use interface{} over string for Static, plausibly rename Static for increased readability?

type Var struct {
    Value interface{} `yaml:"value"`
    Sh    string      `yaml:"sh"`
}

Replace ToStringMap

task/internal/taskfile/var.go

Line 18 in 309cfb1

func (vs Vars) ToStringMap() (m map[string]string) {

To return a map of interface{} values:

func (vs Vars) Resolve() (m map[string]interface{}) {
    ....
}

Change UnmarshalYAML

task/internal/taskfile/var.go

Line 38 in 309cfb1

func (v *Var) UnmarshalYAML(unmarshal func(interface{}) error) error {

• To parse into an interface{} value, and then do a type-switch.
• To only look for the sh prefix ($) when the parsed root value is a string.

To highlight the last point:

vars:
  myvar:
    value: "$I am not a command"

vars:
  myvar: "$echo 'I am a command'"

From there, there will be lots of follow up changes. If you are interested in doing a PR @philippgille, that would be nice. If not, I suspect it will be fixed eventually.

What's your views @andreynering?

[smyrman]

PS! The same should off-course work the same way for the Taskvars.yml file, and for call variables passed to tasks.

[andreynering]

@smyrman Yeah, I think that's the right way of doing this. Thanks for the suggestion!

[dolwitz-at-sony]

@andreynering I just briefly evaluated this tool as an alternative to make. It looks very promising! Thank you for your effort. What would be a really appreciated feature is to handle variables of type map (or any other complex type). In our case we have a set of variables that should change values depending on if you are deploying to dev, stage, prod etc.

Looking up the values for the current deploy environment in a map would be very neat!

Meanwhile, is there another approach you can use to build up a map like structure as a workaround? Like using a pipe-sign to build up a multi-line string? Are there parsing utilities included that can fetch the values from a text-representation of a map? Perhaps not pretty. Any other suggestion?

[andreynering]

Hi @pettervondolwitz,

Indeed I plan to support non-string variables. It's just a matter of finding free time to do so.

For now I don't have any workaround in mind, but maybe if you explain better your use case, with examples, I can help a bit.

[dolwitz-at-sony]

@andreynering

A slightly simplified description would be the following. Let's say we want to orchestrate deployment to three different environments. Taskvars.yml could look something like below:

TARGETS:
  DEV: 
      ACCOUNT: 11111111111
      AWS_REGION: us-west-1
      BASE_FQDN: dev.acme.com
  STAGE: 
      ACCOUNT: 22222222222
      AWS_REGION: us-west-2
      BASE_FQDN: stage.acme.com
  PROD: 
      ACCOUNT: 33333333333
      AWS_REGION: eu-west-1
      BASE_FQDN: acme.com

Then in the Taskfile.yml we would like to point out the current target environment somehow. By setting a variable perhaps if using the current capabilities of this project and then extracting the relevant account number by using a go template:

{{index .TARGETS "DEV".ACCOUNT}}

where "DEV" represents the current environment. Not sure the syntax is correct here but something like the above. Applying the same type of logic to different environments when deploying should be a fairly common use case. The yaml-file above needs to be more complex in our case, i.e. including additional maps/arrays at a deeper level but if a generic support is added for any complex yaml-variable then this would not present a problem I guess.

[smyrman]

Here are two workarounds you could use (using only one variable in the examples to keep it short):

In Taskvars.yml:

TARGET__DEV__ACCOUNT: "11111111111"
TARGET__STAGE__ACCOUNT: "22222222222"
TARGET__PROD__ACCOUNT: "22222222222"

In Taskfile.yml:

version: "2"
tasks:
  deploy:
    desc: "deply all targets"
    deps:
    - deploy-dev
    - deploy-stage
    - deploy-prod
  deploy-dev:
    desc: "deply dev target"
    cmds:
    - task: _deploy
      vars:
        ACCOUNT: '{{.TARGET__DEV__ACCUOUNT}}'
  deploy-stage:
    desc: "deply stage target"
    cmds:
    - task: _deploy
      vars:
        ACCOUNT: '{{.TARGET__STAGE__ACCUOUNT}}'
  deploy-prod:
    desc: "deply prod target"
    cmds:
    - task: _deploy
      vars:
        ACCOUNT: '{{.TARGET__PROD__ACCUOUNT}}'
  _deploy:
    cmds:
    - 'command {{.ACCOUNT}}'

Alternatively, you could use a folder structure:

• deploy/stage
• deploy/prod
• deploy/dev

Let each of them include the same _deloy task from a "lib" directory, and have their own Taskvars.yml:

ACCOUNT: "11111111111"

[dolwitz-at-sony]

@smyrman Thank you for your suggestions! I think flatten out as you suggest in the first example would be to messy since the real use case is much more involved with maps in maps etc. I will see if we can use your second approach though. Also the first example was a good inspiration on how you can go about similar tasks.

[adrinux]

I was planning a task that created a series of web site base files in a directory per site and was expecting to use a YAML dict object for each site.

From this discussion it seems the best way for now would be to have a sitename1/Taskvars.yml, sitename2/Taskvars.yml style folder structure and a generic 'site' task that reads those?

[smyrman]

Yes, I belie for now, that's a feasible solution. Otherwise it's attempting a PR at the suggested solution above: #140 (comment)

[nick4fake]

Is there any progress on this?

I would also like to ask if there is any suggested way for handling arrays/list in vars? We see them in almost every our Makefile, so looking for a suggested/best way to use them

[treykasada]

@nick4fake You basically have to encode your data as some form of string and decode it at the use-site. Depending on the nature of your data you may be able to go with something like a comma-separated string, but in the general case your best bet is probably the fromJson/toJson functions.

[nick4fake]

@treykasada Yeah, that's something we've figured out (using JSON and jq), though I was thinking about a cleaner solution

[pd93]

Hey all. v3.33.0 of Task was just released including experimental support for any type of variable! Please check out the "Any Variables" experiment docs for details.

I'm going to close this since the experiment is being tracked in #1415. Please feel free to leave your feedback and ideas over there. Thanks!

[mattavos]

@pd93 -- can one specify a default for bools? If one uses the recommended way to provide overridable defaults in Taskfiles

    vars:
      JSON: "{{default false .JSON}}"

it of course stringifies the var such that the nicer style {{if .JSON}} checks won't work as expected; it looks like one needs the old style {{if eq .JSON "true"}} check. This is quite a gotcha.

[pd93]

@mattavos This is a limitation of templating. The output will always be a string. You can use ref if you need to preserve types.

Not sure what you're trying to do in your snippet. A boolean will always default to false.

Also, please avoid using a closed issue to ask questions as it notifies everyone who was subscribed. It is better to open a new discussion or join our discord for support.