Safer updates with ref*point: catching configuration drift before it bites

index

When I run updates, my biggest fear isn’t that something will break. It’s that something will fail or change silently, and I will only find out days, weeks or months later.

Loud failures are the easy ones. When a server crashes or an app goes offline, a myriad of systems like Grafana or PRTG alert me. When an app opens a new port, or moves to another one because its default was taken, firewalls at the host and network level usually limit the damage. But when an app flips a default setting, moves its config data somewhere else, or one of its components quietly fails to install, nothing catches it.

I ran into that last scenario with a Harbor deployment. During an update I forgot to pass the --with-trivy flag to the installer, so the vulnerability scanner was never installed. From then on, pushed images silently skipped SBOM generation and vulnerability scanning. Every dashboard was green. I only found out by accident, when I pushed a deliberately vulnerable image for a test and watched it sail through.

That is when I knew I needed a way to declare the state I want a system or app to be in, and then validate it every time I change that system. I started with a handful of Bash scripts, but they quickly became cumbersome to manage and extend.

That is how ref*point started. You take a reference point: a declaration of how a system or app should be configured. Then you validate against it on every run. It is written in Go, so it is fast and ships as a single binary. Profiles are written in HashiCorp’s HCL, so they are easy to read and familiar to anyone who has touched Terraform or OpenTofu.

InstallationSection titled Installation

ref*point ships as a single binary, with packages for the most common platforms on the GitHub releases page. Pick your platform below, and set RP_VER to the version you want to install.

1
RP_VER=1.0.0
2
curl -LO "https://github.com/refpoint-dev/refpoint/releases/download/v${RP_VER}/refpoint_${RP_VER}_linux_amd64.deb"
3
sudo apt install "./refpoint_${RP_VER}_linux_amd64.deb"

1
RP_VER=1.0.0
2
curl -LO "https://github.com/refpoint-dev/refpoint/releases/download/v${RP_VER}/refpoint_${RP_VER}_linux_amd64.rpm"
3
sudo dnf install "./refpoint_${RP_VER}_linux_amd64.rpm"

1
RP_VER=1.0.0
2
curl -LO "https://github.com/refpoint-dev/refpoint/releases/download/v${RP_VER}/refpoint_${RP_VER}_darwin_arm64.tar.gz"
3
tar -xzf "refpoint_${RP_VER}_darwin_arm64.tar.gz"
4
sudo mv refpoint /usr/local/bin/

1
$RP_VER = "1.0.0"
2
Invoke-WebRequest -Uri "https://github.com/refpoint-dev/refpoint/releases/download/v${RP_VER}/refpoint_${RP_VER}_windows_amd64.zip" -OutFile "refpoint_${RP_VER}_windows_amd64.zip"
3
Expand-Archive -Path "refpoint_${RP_VER}_windows_amd64.zip" -DestinationPath "$env:LOCALAPPDATA\refpoint"

Then add %LOCALAPPDATA%\refpoint to your PATH.

Check that it works:

1
refpoint version

Writing your first profileSection titled Writing%20your%20first%20profile

A profile is a collection of controls, where each control is one expectation about a system. ref*point provides an init command that bootstraps a new profile with an example control:

1
refpoint init linux-baseline --title "Linux Baseline" --author "Ops Team"

This creates the folder structure and the minimum required files: profile.rpp, which holds the metadata used to manage the profile, and an example control.

1
linux-baseline/
2
├── profile.rpp
3
└── controls/
4
    └── example.rpc

To run the profile against the local system, use the run command:

1
refpoint run linux-baseline

Adding a real controlSection titled Adding%20a%20real%20control

Now let’s add a control that actually matters. In the controls folder, create a file called ssh.rpc (.rpc stands for refpoint controls) with the following content:

1
control "ssh-01" {
2
  title       = "SSH root login is disabled"
3
  description = "Direct root login over SSH removes accountability."
4
  impact      = 0.9
5
  remediation = "Set PermitRootLogin no in /etc/ssh/sshd_config and reload sshd."
6

7
  platform = ["linux"]
8

9
  tags = {
10
    category = "ssh"
11
  }
12

13
  only_if "package" {
14
    name      = "openssh-server"
15
    installed = true
16
  }
17

18
  check "root-login" "sshd_config" {
19
    option "PermitRootLogin" {
20
      value = "no"
21
    }
22
  }
23
}

This control verifies that SSH doesn’t allow direct login as root. Forcing people to log in with their own account first means the audit log records who actually did what. If anyone can log in directly as root, it becomes very hard to tell whether a server was compromised, or who ran the command that destabilized it.

A few things worth pointing out:

impact is a score from 0 to 1 that tells you how much a failure matters, so you can triage a long report quickly.
platform and only_if keep the control from running where it doesn’t make sense. If openssh-server isn’t installed, the control is skipped instead of reported as a failure.
remediation is the fix, written down next to the check. More on why that matters below.

Controls in ref*point are idempotent and read-only, with one exception: the command resource. It exists for cases where ref*point doesn’t have a dedicated resource yet, and it runs whatever command you give it. A carelessly crafted command can write to the target system, so review what your profiles are allowed to execute, especially profiles you didn’t write yourself.

Validating the profileSection titled Validating%20the%20profile

ref*point has a built-in profile validator. refpoint check verifies that the profile has all the required fields and that every control and resource is properly configured:

1
refpoint check linux-baseline

Running the above command will result in the following output:

1
Profile "linux-baseline" is valid (2 controls loaded)

Run it before committing profile changes, or better, as a step in the CI pipeline of your profiles repository, so a broken profile never reaches the servers.

Running the profile against remote systemsSection titled Running%20the%20profile%20against%20remote%20systems

Running locally is useful while writing a profile, but the real value comes from running the same profile against your fleet. ref*point connects over SSH to Unix-like systems and over WinRM to Windows machines. SSH to Windows should work too, but I haven’t tested it yet.

For larger profiles, like the ones I’ve shared on GitHub that covers both Debian and Red Hat based systems, I recommend doing a dry run first. ref*point identifies the target system and shows which controls would run, without running them:

1
refpoint run linux-baseline --target [email protected] --dry-run

1
refpoint 0.1.0
2
config loaded: /home/ops.cafe/.refpoint/config.toml
3
profiles run: /home/ops.cafe/profiles/linux-baseline
4

5
Dry run — execution plan only, no checks are executed
6

7
Profile: Linux Baseline (v0.1.0)
8
════════════════════════════════════════════════════════════
9
Target: [email protected]  ·  linux/ubuntu
10
════════════════════════════════════════════════════════════
11
  ▶ run  [example-01]  Example control
12
  ▶ run  [ssh-01]  SSH root login is disabled  — only_if package "openssh-server" decided at run time
13
────────────────────────────────────────────────────────────
14
  2 to run (1 conditional)

Note: I used the root user above only to keep the example short. In practice, connect with a dedicated, least-privileged user whenever possible.

Reading the resultsSection titled Reading%20the%20results

When the profile runs, ref*point reports every control as passed, failed or skipped. For a failed control, it shows what it expected, what it actually found and, if the control has one, the remediation.

1
refpoint 1.0.0
2
config loaded: /home/ops.cafe/.refpoint/config.toml
3
profiles run: /home/ops.cafe/profiles/linux-baseline
4

5
════════════════════════════════════════════════════════════
6
Target: [email protected]  ·  Linux Baseline v0.1.0
7
════════════════════════════════════════════════════════════
8
  ✓  [example-01]  Example control  impact: 0.5
9
    ✓  file-exists[file: /etc/hostname]
10

11
  ✗  [ssh-01]  SSH root login is disabled  impact: 0.9
12
    ✗  root-login[sshd_config]
13
          ✗ option[permitrootlogin]: got "yes", want "no"
14
    remediation: Set PermitRootLogin no in /etc/ssh/sshd_config and reload sshd.
15

16
════════════════════════════════════════════════════════════
17
Summary
18
════════════════════════════════════════════════════════════
19
Linux Baseline  v0.1.0
20
     ✗  [email protected]                  50.0%  1/2 controls passed · 1 failed
21
════════════════════════════════════════════════════════════
22
Average compliance score: 35.7%

The remediation field pays off most in a team. The controls double as documentation: a senior engineer can write a control once, together with the steps to fix it, and then anyone on the team can run the profile and bring a system back in line when it drifts.

Besides the terminal report, ref*point can produce JSON or JUnit XML. That makes it easy to feed results into a CI pipeline or any tool that already understands test reports.

1
refpoint run linux-baseline --target [email protected] --reporter 'json:report.json'

Back to Harbor: making updates boringSection titled Back%20to%20Harbor%3A%20making%20updates%20boring

This is the workflow that would have saved me with Harbor:

Describe the expected state. Write a profile for the app that covers what matters, not only that the service is up. For Harbor that includes: the Trivy scanner is installed and registered as the default scanner, scan-on-push is enabled for the projects that need it, and the expected ports are listening.
Run it before the update. A clean baseline tells you that any failure afterwards was caused by the update, not by something that was already broken.
Update.
Run it again. Anything that changed silently now shows up as a failed control, with the fix right next to it.

1
refpoint 1.0.0
2
config loaded: /home/ops.cafe/.refpoint/config.toml
3
profiles run: /home/ops.cafe/profiles/harbor-health
4

5
════════════════════════════════════════════════════════════
6
Target: ssh://[email protected]  ·  Harbor Health and Image Check 1.0.0
7
════════════════════════════════════════════════════════════
8

9
 ...
10

11
  ✓  [harbor-health-09]  Log collector container is running and uses the expected goharbor image  impact: 0.7
12
    ✓  running[command: docker inspect -f '{{.State.Running}}' 'harbor-log']
13
    ✓  image[command: docker inspect -f '{{.Config.Image}}' 'harbor-log']
14

15
  ✓  [harbor-health-10]  Trivy adapter container is running and uses the expected goharbor image  impact: 0.7
16
    ✓  running[command: docker inspect -f '{{.State.Running}}' 'trivy-adapter']
17
    ✓  image[command: docker inspect -f '{{.Config.Image}}' 'trivy-adapter']
18

19
 ...
20

21
  ✓  [harbor-health-12]  All components run the same release  impact: 0.5
22
    ✓  single-tag[command: docker ps --format '{{.Names}}' | grep -xE '(nginx|harbor-core|harbor-p…]
23

24
  ✓  [harbor-health-13]  Health endpoint reports the system and every core component healthy  impact: 0.8
25
    ✓  fetch[command: curl -ksS -o '/tmp/refpoint-harbor-health.json' -w '%{http_code}' 'http…]
26
    ✓  components[json: /tmp/refpoint-harbor-health.json]
27

28
  ✓  [harbor-health-14]  Health endpoint reports the Trivy scanner healthy  impact: 0.6
29
    ✓  fetch[command: curl -ksS -o '/tmp/refpoint-harbor-health.json' -w '%{http_code}' 'http…]
30
    ✓  trivy[json: /tmp/refpoint-harbor-health.json]
31

32
════════════════════════════════════════════════════════════
33
Summary
34
════════════════════════════════════════════════════════════
35
Harbor Health and Image Check  1.0.0
36
     ✓  ssh://[email protected]               100.0%  13/13 controls passed · 1 skipped
37
════════════════════════════════════════════════════════════
38
Average compliance score: 100.0%

Had I run this after that Harbor update, the missing scanner would have been one red line in the report, instead of weeks of unscanned images.

ConclusionSection titled Conclusion

ref*point has become an invaluable tool for me as the infrastructure I manage keeps growing. I no longer fear unattended updates, skipped steps or changes that slip past me. It also makes migrations much less risky. Take an older application stuck on an OS that no longer receives security updates: I can capture how it is configured today, rebuild it on a new machine, and run the same profile to confirm nothing got lost in the move.

It also changed when I think about validation. I can set expectations before a system is even deployed: read the app’s documentation, write a profile for it, and wire that profile into the tools I deploy with, be it Ansible or OpenTofu/Terraform.

Finally, it lets me harden systems gradually without fear of breaking them. I capture a profile of the current, working state first, then add a new security layer and confirm that everything that worked before still works. That makes my work faster and a lot less error-prone.

If you give ref*point a try, I’d love to hear which controls you write first.