How Does the Fleetd Client Update? Signature Verification, Version Checks, and Failure Recovery

How Fleetd v0.3.0 checks signed releases, architecture and versions, tests new binaries and replaces agents, with the limits of manual recovery explained.

One of the worst outcomes for a remote management tool is updating a client and then losing access to its machine.

If the server is nearby, you can still log in and fix it. If a soft router is somewhere else, an update that breaks the connection may leave the “Retry” button in the admin console unable to help.

Following the previous five Fleetd development articles, this one covers client updates. An update is more than downloading a new file. It must answer three questions: Can the file be trusted? Is it suitable for this machine? What happens if something goes wrong after replacement?

First, the scope: this article is based on Fleetd v0.3.0, commit c4a066d, and its public source code and documentation. This version supports administrator-initiated remote updates. It does not automatically track new releases on a schedule, and it has no automatic rollback based on runtime health checks. The failure acceptance checks described here are recommendations, not a report of a completed production update exercise.

For Remote Updates, Separate Who Decides What

A Fleetd update starts in the admin console. An administrator uploads a release package and selects a version for a single online machine, or updates online machines of the same architecture that are running older versions.[1]

The server distributes the package and passes along the update request. The client decides whether to accept it.

That distinction matters: a file appearing in your own admin console should not automatically make it trustworthy. The client has a locally configured release public key, release_key, and must verify the package signature itself. If that public key is not configured, it rejects remote updates.[2]

If the server also has a release public key configured, it verifies packages during upload. But the server-side check cannot replace the client-side check. In the source code, when the server has no release public key configured, it can parse only the package manifest for storage and display. “Upload succeeded” therefore does not prove that the client will accept the package.[4]

The release private key determines who can produce trusted packages; the public key determines whom the client trusts. In a self-hosted deployment, you should establish who controls your release keys instead of treating any downloaded release package as installable.

This also differs from browser command signing discussed earlier: command signing authorizes specific operations, while release signing authenticates software packages. Their responsibilities and key management must not be conflated.

A Signature Is More Than a Hash of a File

A v0.3.0 release package contains a manifest, a signature, and a client binary. The manifest records the version, CPU architecture, file size, and SHA-256 hash.[3]

The signature covers the manifest’s original bytes, and the hash in the manifest constrains the binary’s contents. The client first verifies the Ed25519 signature using its trusted public key, then checks that the binary’s size and hash match the manifest.

This addresses two related questions:

  • Was the manifest signed by someone holding the corresponding private key, and has it been changed?
  • Is the downloaded binary actually the file described by that manifest?

A hash without a trusted signature source cannot do the same job. If an attacker can replace both the file and the hash beside it, the two can still “match.”

A valid signature does not prove the software is free of bugs. It proves the file came through a trusted signing process, not that the program will work correctly on every machine. A leaked release private key, a compromised build environment, or a maintainer signing a defective program all require separate responses.

The Client Must Check Version and Architecture Again

After verifying the signature, the client also checks whether the manifest’s architecture matches its own runtime.GOARCH and whether the manifest’s version is newer than the version currently running.[2]

A correctly signed arm64 package therefore cannot be installed on an amd64 client simply because it appears in the admin console’s list.

Version comparison has specific boundaries, too. In v0.3.0, it compares major, minor, and patch numbers as three integers and ignores the suffix after a hyphen.[3]

For example, under this function’s rules, 0.4.0-dev and 0.4.0 have the same version core, so the latter will not be considered newer than the former. This is not the full SemVer prerelease ordering rule; general SemVer intuition cannot substitute for the source code’s behavior.

The same version and older versions are rejected. This prevents arbitrary downgrades through the normal update path, but it is not a rollback defense at every level: someone with permission to modify local files can still replace the program, and restoring an old file is a maintenance procedure explicitly provided in the documentation.

For routine releases, the release manifest should match the version embedded in the binary. The project’s build script sets the program version and, when given a release version and a signing private-key file, also generates signed packages for the corresponding architectures.[5]

Before Replacing Itself, Let the New Program Report Its Version

The client does not overwrite itself as soon as the download finishes.

It first writes the verified binary to a .new file beside the executable. It then runs that new file’s version subcommand, requiring the program to exit within a time limit and its output to match the version in the signed manifest.[2]

This catches some obvious errors, such as a program that cannot start, an incorrect build artifact, or a program that reports a different version.

It is still only a small self-check. Being able to print a version number does not mean that reading configuration, connecting to the server, collecting metrics, and performing authorized operations all work. A program that passes version may still fail during normal startup.

After the self-check passes, the program attempts to preserve the old executable as .old, then renames .new into the executable’s path. Placing the temporary file in the same directory is part of this replacement process; it should not be taken to mean that arbitrary copying across filesystems has the same behavior.

After replacement, the current process reports the result and exits. The subsequent start is left to systemd or procd. The project’s installation templates configure restart behavior for them. If you run the client manually or have modified the service configuration, you need to confirm separately what will start the new program.

An installation success result reaching the admin console and the new process coming online in a healthy state are therefore two different points in time.

Failure Recovery Depends on Where the Failure Occurs

“Failures are rolled back” is too broad a statement. Under the v0.3.0 implementation, at least these stages need to be distinguished:

Failure stage What the current implementation does What still needs checking
Download, signature, content, architecture, or version check fails Returns an error without replacing the executable Whether the reason was recorded and whether the original client is still working normally
New program fails the version self-check Deletes the temporary file and preserves the executable Whether the problem lies in the build, runtime environment, or version marking
Replacement rename fails Returns an error and cleans up the temporary file Directory permissions, filesystem, and disk state
After replacement, the new process fails to start or run its normal functions There is no automatic rollback process based on health status Out-of-band login, the old file, configuration backups, and the ability to recover manually

There is another easily missed detail: the source code first tries to preserve the old program with a hard link, then falls back to copying it by reading and writing. Some failures on that fallback path do not prevent replacement from proceeding.[2]

The mere presence of .old in the implementation therefore does not justify promising that a usable backup is created every time. Before a production release, verify that the old file exists, has the correct contents, and can execute, and keep an independent recovery path available.

Likewise, a rename-based replacement is not a whole-machine upgrade transaction, nor does it guarantee data durability after a power loss. This article does not report testing power loss, disk failures, or actual OpenWrt devices.

Update One Machine Before Considering a Batch

The project documentation explicitly says it does not provide fully automatic updates: pushing a defective version to every machine at once could disconnect every client, at which point the admin console could no longer be relied on to fix them.[1]

I would split update acceptance checks into three stages.

The first is pre-installation rejection testing. In an isolated environment, prepare an invalid signature, modified contents, the wrong architecture, and the same or an older version. Confirm that the client rejects each installation and does not replace the executable.

The second is replacement testing. Check the new program’s version self-check, preservation of the old file, and whether the service manager actually starts the new process.

The third is post-startup operational checking. Confirm that the new version reconnects, continues reporting metrics, and can perform necessary operations. Observe it for a period appropriate to your own operations before expanding the rollout. There is no universal “wait a few minutes and it is safe” threshold here.

The repository includes release-package and installation logic tests covering invalid signatures, file tampering, version and architecture rejection, mismatched version output, and preservation of the old file after a normal replacement, among other paths.[6] I read those tests for this article but did not rerun them or treat them as evidence of a successful upgrade on real hardware.

If the new program has already lost connectivity, recovery depends on the deployment environment. The official guide describes manually replacing the binary with the old one and restarting the service. Linux and OpenWrt use different program paths and service management methods, so commands for one should not be copied verbatim to the other.[1]

A reliable update should end only when the machine is back online and performing the necessary work. Signature and version checks guard the beginning of the process; a recovery path must remain available at the end.

References

Checked on: 2026-11-09. The implementation analysis is fixed to v0.3.0 and does not represent the behavior of later versions. Previous article: After Server Monitoring, How Do You Make Alerts Truly Useful?

  1. Fleetd v0.3.0: Client Update Guide.
  2. Client Update and Self-Check Implementation.
  3. Release Package Signing, Verification, and Version Comparison.
  4. Server-Side Release Package Upload and Distribution.
  5. Build Script and Client Service Installation Template.
  6. Installation Tests and Release Package Tests.

AI-generated concept illustration, not a product test photograph, actual interface or wiring diagram.

Discussion

Share a question or your experience. Comments appear after moderation. Your email is never displayed publicly. Comment privacy

Loading comments…