rescueshell Start trial

The keys on a jump host outlast the people

Most reverse-SSH setups start the same way. Someone generates one key pair, puts the public half on the jump host, copies the private half onto every device so that each can dial in, and writes down who knows the passphrase. It works on the day.

A year later the list of people has changed and the list of keys has not. This post is about that gap: what it looks like on a fleet, and what we built so that fixing it is a small, signed change on the device and not an edit of a shared file. For the separate argument about what a compromised jump host can do, see our earlier post. Here we stay with people and keys.

How the pile grows

Nobody decides to build a mess. It accumulates one reasonable step at a time.

None of this is a failure of any one person. A shared secret on a central host turns every joiner, leaver and lost laptop into a fleet-wide change, and fleet-wide changes get postponed.

What we changed

The first move is to stop having a shared key. In RescueShell each provisioning credential belongs to one team, and the security page says so plainly ("No shared keys"). The second move is to put the list of who may open a shell on the device itself, not on a host in the middle.

Each device keeps its own list of trusted public keys and the accounts those keys may open, under its state directory (trust/keys and trust/users, at most 20 of each). A new install has to be given that list and refuses to run without it. The installer prints each fingerprint and account and asks you to confirm. Our servers cannot add a key or an account to a device (see security).

When someone joins, you add their public key to the devices that should trust it. You can seed a device from an existing account's authorized_keys at install time, which is useful when you are migrating from the old setup and not starting clean.

A change is signed, and it is the device that checks

After install, the list changes in one way: a trust change signed by a key the device already trusts. The payload names the device, the next version number and the challenge that device is holding for its current connection. The signature uses the namespace rescueshell-trust-v1.

The device checks all of it. The signer must be on its current list. The version must be exactly its stored version plus one, so an old change cannot be replayed and a skipped one is refused. The challenge changes on each connection, so a signed change cannot be saved up and applied later. The result has to leave at least one key and one account. The files are written so that a crash leaves each one old or new, never half written.

Our server stores the change and relays it. It never signs. The worst it can do is drop, delay or replay one, and a replay is refused as stale.

For a lost laptop, the runbook example is one command from a key that is still trusted:

rescueshell trust <device> --sign-with ~/.ssh/other_key --add-key new.pub --remove-key lost.pub

For a fleet, the same change can be signed once for a whole group of online devices, from the CLI or from the browser, with one passkey tap or one passphrase. The CLI shows each key fingerprint, each account and the signer before it asks you to confirm, and the browser shows the same review before any unlock. One batch covers at most 50 devices, and the relay counts it as one request. It never signs a change that would remove the key doing the signing, or leave a device with no key or no login account.

That turns rotation from a maintenance window into a command. You add the new key, check that it works, and remove the old one. Nothing has to be copied around, and no file on a shared host is edited.

When a person leaves

Two different things have to happen, and we keep them separate on purpose.

First, remove them from your team. That takes effect at once: they can no longer open a shell through RescueShell on any device, online or not, and their open shells end right away (see the security page, "Your team's data").

Second, take their key off the devices. The runbook calls this hygiene, and we agree with that word. It matters because a key that is still on a device's list is still trusted by that device, whatever happened to the account that used it. Removing it is a signed change, as above.

The audit side is modest. The server records key changes, host key accepts and refused sessions as fingerprints, never key bodies, and each agent keeps its own chained log of the shells it granted or refused (see security). It is a record to read, not an export and not a recording.

Where this does not fit

We would rather list the limits than leave you to find them.

The goal in our threat model is that full control of our backend and relay opens zero shells. We have not run the red-team round that would test it end to end, so it is a goal and not a result.


RescueShell: €0.10 a device a month, billed yearly, at least €6 a year, plus VAT. Thirty-day trial, no card for your first device, and the clock starts when it connects. Until a card is added, shell traffic is capped at 50 MB in total and a second device needs a card. Hosted in the EU, exceptions listed. Try it

All posts · RSS