Skip to content

Rotate the Signing Secret

The JWT signing secret is a server-side 32-byte random key stored in the server configuration. Every agent token is signed with it. Rotating the secret immediately invalidates all existing agent tokens, every enrolled agent loses access simultaneously.

This is a break-glass operation. Use it when:

  • The signing secret (or the server configuration) is suspected to have been exposed.
  • A sweeping token reset is required (e.g., an insider threat, an audit finding, or a policy decision to re-enroll all agents under a fresh credential).

All agents will stop reporting

As soon as the Finch container restarts with the new secret, every agent's current token is rejected. Alloy buffers telemetry locally in its WAL during the outage. Plan to re-enroll all agents promptly to minimise the data gap.


Step 1 - Rotate the secret

finchctl service rotate-secret root@10.19.80.100

The command:

  1. Reads the current server configuration via SSH.
  2. Generates a new 32-byte cryptographically random key and replaces the secret field.
  3. Writes the updated configuration back to the server.
  4. Restarts the finch service to apply the new secret.

From the moment the container comes back up, all existing agent JWTs are invalid.


Step 2 - Re-enroll all agents

Each agent needs a fresh config file containing a token signed with the new secret. Get the list of registered agents first:

finchctl agent list finch.example.com

Then follow Re-enroll an Agent for each agent. Use agent config --agent.rid <rid> to get a fresh token. The agent record still exists in the database, only the token needs refreshing.

Repeat for every enrolled agent. Once Alloy restarts with the new token it reconnects and flushes any buffered WAL data.


What changes and what does not

Changes Does not change
JWT signing secret in the server configuration Agent registrations in the database
All existing agent tokens (invalidated) Agent hostnames, log sources, metrics, labels
Finch container restarts briefly mTLS certificates (finchctl is unaffected)
365-day token window resets for each re-enrolled agent Stack configuration and Grafana data