Terraform / OpenTofu efficient variables management

In this note I will share a few tips & tricks that I am using and which I found that it improved my experience as an OpenTofu user as well as my documentation.

Variables documentation using validation rules

When defining variables in OpenTofu or Terraform, adding descriptions is a community accepted good practice since it helps other team members understand the purpose of the variable faster by having the "documentation" right in the code.

However, for variables that only accept certain values, we can go a step further by implementing validation rules. This provides users with clear, actionable, and immediate feedback when they attempt to use invalid values.

Let's look at a few fairly typical examples:

variable "environment" {
  description = "Environment name"
  type        = string

  validation {
    condition     = contains(["dev", "staging", "prod"], var.environment)
    error_message = "Environment must be one of: dev, staging, prod."
  }
}

variable "instance_count" {
  description = "Number of instances to create"
  type        = number

  validation {
    condition     = var.instance_count > 0 && var.instance_count <= 10
    error_message = "Instance count must be between 1 and 10."
  }
}

variable "allowed_ports" {
  type        = set(number)
  description = "Set of allowed incoming ports (80, 443, and 139 only)"

  validation {
    condition     = alltrue([
      for port in var.allowed_ports : contains([80, 443, 139], port)
    ])
    error_message = "Invalid port detected. You may only use port 80 for HTTP, 443 for HTTPS, or 139 for NetBIOS."
  }
}

variable "tags" {
  description = "Resource tags"
  type        = map(string)

  validation {
    condition     = contains(keys(var.tags), "Owner")
    error_message = "Tags must include an 'Owner' key."
  }
}

When using this approach there is a small drawback as the maintainer(s) would have to keep the validations in sync with the system being managed.

To maximize the potential of this approach, a few questions need to be first considered:

how often does the values change in the managed system?
are there other variables that are going to depend on the variable with specific validations?
is this validation used in a module that it is going to impact many other projects when changed?

Variables management

Good variables management is required as OpenTofu projects grow or team sizes increases. So, let's explore a few workflows that would help ease this task for developers.

Splitting into multiple files

Rather than keeping all variables definitions in a single variables definitions file, you can split them into logical groups. This approach improves readability, simplifies maintenance, and makes it easier for teams to collaborate as changes can be more easily identified in CVS systems, ie. git.

A common pattern includes separating variables by resource type or functionality.

# network-vars.tf
variable "vpc_cidr" {
  type        = string
  description = "CIDR block for the VPC"
  default     = "10.0.0.0/16"
}

# compute-vars.tf
variable "instance_type" {
  type        = string
  description = "EC2 instance type"
  default     = "t3.micro"
}

# storage-vars.tf
variable "storage_size" {
  type        = number
  description = "Size of storage in GB"
  default     = 50
}

And not only variables definitions can be split across files, but also breaking into smaller pieces the files that contain the variable's values can help make any maintenance task more enjoyable.

This approach is very useful when managing different environments or configurations since any update to an environment is less error prone and easier to validate.

Let's look at an example deploying to dev and prod environments:

# variables.tf
variable "vpc_settings" {
  type = object({
    cidr_block = string
    name       = string
    subnets    = map(string)
  })
}

# dev.tfvars
vpc_settings = {
  cidr_block = "10.0.0.0/16"
  name       = "dev-vpc"
  subnets    = {
    public  = "10.0.1.0/24"
    private = "10.0.2.0/24"
  }
}

# prod.tfvars
vpc_settings = {
  cidr_block = "172.16.0.0/16"
  name       = "prod-vpc"
  subnets    = {
    public  = "172.16.1.0/24"
    private = "172.16.2.0/24"
  }
}

Using the -var-file flag, the appropriate environment variables can be passed to OpenTofu, like this:

$ tofu plan -o plan.out -var-file dev.tfvars   # for development env
$ tofu plan -o plan.out -var-file prod.tfvars  # for production env

When managing large variables sets, an useful option is to split them based on functionality as well, similar to this:

# networking.tfvars
vpc_cidr = "10.0.0.0/16"
subnet_cidrs = {
  public  = "10.0.1.0/24"
  private = "10.0.2.0/24"
}
route_tables = {
  public  = "public-rt"
  private = "private-rt"
}

# security.tfvars
security_groups = {
  web = {
    name = "web-sg"
    ports = [80, 443]
  }
  database = {
    name = "db-sg"
    ports = [5432]
  }
}
ip_whitelist = ["10.20.0.0/16", "192.168.1.0/24"]

# application.tfvars
app_settings = {
  name = "myapp"
  environment = "production"
  instance_type = "t3.large"
}

Using the same -var-file flag multiple times, will merge together all the variables files to form a single set of variables that will be used by OpenTofu at runtime:

$ tofu plan \
  -var-file=env/dev/networking.tfvars \
  -var-file=env/dev/security.tfvars \
  -var-file=env/dev/application.tfvars"

The drawback to this approach is that when multiple files define the same variable, the value from the last file will overwrite all others. This applies even if the variable is a map and each file specifies a different set of keys for that map.

For example:

# security_web.tfvars
security_groups = {
  web = {
    name = "web-sg"
    ports = [80, 443]
  }
}

# security_db.tfvars
security_groups = {
  database = {
    name = "db-sg"
    ports = [5432]
  }
}

Issuing the following command:

$ tofu plan \
  -var-file=env/dev/security_web.tfvars \
  -var-file=env/dev/security_db.tfvars

will result in the following variable being used at runtime:

# security_db.tfvars
security_groups = {
  database = {
    name = "db-sg"
    ports = [5432]
  }
}

Merging variables

Building on the previous example of splitting variables across files, a solution that can address the potential issue of variable overrides is to use OpenTofu's built-in functions merge() and concat() to combine multiple variables into a single variable.

Here's one example:

# web-security.tfvars
web_security_groups = {
  web = {
    name = "web-sg"
    ports = [80, 443]
  }
}

# db-security.tfvars
db_security_groups = {
  database = {
    name = "db-sg"
    ports = [5432]
  }
}

# main.tf
locals {
  # Merge multiple security group configurations
  security_groups = merge(
    {
      # Default values that can be overridden
      monitoring = {
        name = "monitoring-sg"
        ports = [9090]
      }
    },
    var.web_security_groups,
    var.db_security_groups,
  )
}

This method prevents accidental overwrites, allows for layered configurations, maintains default values as fallbacks and makes it easier to extend configurations without modifying existing files.

Dependencies between variables

Often times systems require specific values to be set based on the values of another configuration option. To achieve this level of flexibility, an OpenTofu's variable value can be made dependent on another variable's value. This is possible because OpenTofu processes the variables at runtime before passing them to the code.

Let's see a few examples:

variable "environment" {
  type    = string
  default = "dev"
}

locals {
  # Instance type depends on environment
  instance_type = {
    dev  = "t3.micro"
    test = "t3.small"
    prod = "t3.large"
  }[var.environment]

  # Storage configuration based on environment
  storage_config = {
    size = var.environment == "prod" ? 1000 : 100
    type = var.environment == "prod" ? "io1" : "gp2"
  }

  # Compute capacity based on multiple variables
  autoscaling_config = {
    min = var.environment == "prod" ? 3 : 1
    max = var.high_availability ? 10 : 3
    desired = var.environment == "prod" ? 5 : 1
  }
}

Here, the instance_type variable's value is actualy a string based on the value of the environment variable which is used as a key to search in a map of possible values.

In the case of the storage_config and autoscaling_config the code uses other variables to set hard values using conditionals. This is useful when there is a need to very tightly constrain certain parameters of the infrastructure - due to compliance or costs for example.

Using this approach can be applied to variable validations to create a more robust documentation and user experience.

# variables.tf
variable "environment_config" {
  type = object({
    name = string      # e.g., "dev", "prod"
    tier = string      # e.g., "free", "premium"
  })

  validation {
    condition     = contains(["dev", "prod"], var.environment_config.name)
    error_message = "Environment name must be either 'dev' or 'prod'"
  }

  validation {
    condition     = contains(
      var.environment_config.name == "dev"
        ? ["free"]
        : ["free", "premium"],
      var.environment_config.tier
    )
    error_message = "Dev environment can only use 'free' tier. Prod can use 'free' or 'premium'"
  }
}

# vars.auto.tfvars

# This is a valid
environment_config = {
  name = "prod"
  tier = "premium"
}

# Invalid - will fail validation
environment_config = {
  name = "dev"
  tier = "premium"  # Error: dev environment can only use 'free' tier
}

Restructuring data

As the OpenTofu project evolves the need may arise to "transform" variables into new formats to match new project code or a third-party module's requirements. This might involve string manipulation, type conversion, or creating completely new variables which combine values from multiple other variables.

Let's take a look at a few examples:

variable "environment" {
  type    = string
  default = "dev"
}

variable "project" {
  type    = string
  default = "myapp"
}

locals {
  # Convert to lowercase and add prefix combining multiple vars values
  resource_prefix = lower("${var.environment}-${var.project}")

  # Create standardized name format
  resource_name = format("%s-%s-%s",
    local.resource_prefix,
    var.project,
    formatdate("YYYY", timestamp())    # resulting in string 2025
  )
}

Closing thoughts

By implementing validation rules, you can ensure that variables adhere to expected values, providing immediate feedback and reducing the likelihood of errors. Splitting variables into logical groups and using multiple files can significantly improve readability and maintainability, making it easier for teams to collaborate and manage complex configurations. Additionally, leveraging OpenTofu's built-in functions like merge() and concat() allows for flexible and dynamic configuration, preventing accidental overwrites and maintaining default values.

Ultimately, these tips and tricks will enhance your development experience, streamline your workflows, and contribute to a more organized and efficient infrastructure management process.