Option to exit if modules do not specify a version

[dradetsky]

Terraform Version

`Terraform v1.3.4`

Use Cases

I'm a platform team providing a framework for backend teams to write backend services that use cloud resources. So we have a repo full of terraform modules that the backend services can use, then each service has a root modules that references other modules, and the deployment package for the service is a k8s config which includes some resource that causes terraform to apply some module (like a terranetes resource). Also, we have a gitops-y deployment, so merging changes to the "prod" repo will cause a deploy to prod if necessary, and thus potentially changes to cloud resources.

I want to force the developers to explicitly pin their modules somehow, so that subsequent upstream changes to the modules repo are never applied to the resources used by a specific service unless the developer explicitly declares their intent to change that service's resources (by bumping the pinned version).

So given a root config like

module "one" {
  source  = "terraform-aws-modules/vpc/aws"
}

or

module "two" {
  source = "https://github.com/terraform-aws-modules/terraform-aws-s3-bucket.git"
}

I'd like something like a flag -require-module-versions which would cause the terraform CLI to immediately exit from plan/apply, unless of course the module was pinned some other way, such as the add-versions-to-lockfile proposal (#31301). This could be fixed by replacing the above with

module "three" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "3.14.2"
}

or

module "four" {
  source = "https://github.com/terraform-aws-modules/terraform-aws-s3-bucket.git?ref=v3.1.0"
}

Then the platform team would ensure that terraform was invoked in automation with this flag, although backend developers could do whatever was convenient for them during development.

Attempted Solutions

I haven't actually tried to solve the problem. Essentially, this is an issue that arose at a previous company which I want to ensure does not arise at the next company.

That said, apart from some kind of hacky grep solution (which would probably end up blocking more good deploys than bad ones), I don't think there's any way to solve it. There's different ways to say that a module should be pinned depending on the source type, and the only thing which really knows whether something is providing a version pin to a module invocation is terraform itself (since it's the thing that pulls in all the info and figures out what version it's supposed to get), so it's the only reasonable place to add a rule like "if we didn't pin an explicit version, exit."

Proposal

I've already made like half the proposal, but additionally there are two versions of this: enforcing the must-pin-everything on the root module, and enforcing it transitively. I assume that, absent the introduction of something like #31301, enforcing it only in the root module would be much easier. However, this would be enough for the sort of smaller organizations where it's feasible to just have one big repo full of all the terraform modules.

Additionally, the flag could take an optional argument, such as -require-module-versions=exact, -require-module-versions=semantic, where the latter would allow either pinning to a specific version or a semantic version expr, although I personally have no use for this feature.

References

• Lock module versions, like providers, in .terraform.lock.hcl #31301

[crw]

Thanks for this feature request!

[dradetsky]

I have actually found a potentially reasonable solution, although I have not tested it, so maybe it doesn't actually work correctly, but:

The platform team could add a step to run tflint on the module in automation before attempting to apply it, and stop if the linter emits any warnings. Then the following ruleset (plus probably some extra tflint boilerplate) would, I think, do what I want:

rule "terraform_module_version" {
  enabled = true
  exact = true
}

rule "terraform_module_pinned_source" {
  enabled = true
  style = "semver"
}

Although the latter isn't exactly right. AFAICT, style = flexible isn't restrictive enough, and style = semver is too restrictive. I really want style = either_a_tag_or_a_commit_sha. Still, there's something approximating a solution here.

[devopsrick]

Sentinel can do this. https://github.com/hashicorp/terraform-guides/blob/master/governance/third-generation/cloud-agnostic/require-all-modules-have-version-constraint.sentinel

[apparentlymart]

Hi @dradetsky! Thanks for sharing this feature request.

This sounds like the sort of thing we would typically classify under the general heading of "policy enforcement", which is to say making additional rules that are not crucial to the correct behavior of the Terraform configuration but are nonetheless important to a particular team for other reasons.

So far our position for these has been to say that the actual rules are outside of Terraform's scope (because they are inherently team-specific and subjective) and so Terraform's own role in this process is only to make available the data required to enforce this sort of policy rule outside of Terraform.

(I'm talking specifically about open source Terraform here. Terraform Cloud does offer automatic policy checking as a feature, using the same integration points that are available for other tools to use.)

With that said then, I think our goal here should be to figure out what additional data you would need access to in order to enforce this rule yourself using external software, and then consider adding that to Terraform somehow.

The main integration point right now is to create a plan and save it in a plan file on disk, and then convert that plan to the documentated integration format for analysis by the external software, like this:

terraform plan -out=tfplan
terraform show -json tfplan

The subtree if that result which describes the configuration does already include some information about the module calls in each module, but the documentation doesn't mention any property exposing the version constraints for a call to a registry module and so I don't think it would be possible to implement the desired rule with the current dataset.

I believe we should have enough information in the code which generates the JSON representation of the configuration to include information about the version constraints, though. Do you think that adding that would give what you need to implement this policy rule?

This approach would also make the data available to the policy-checking features in Terraform Cloud, once this features are updated to expose the new data in their own API.