# Decrypting a Headless Server on Boot... Remotely

November 21, 2020 · David Lick

Like the title says, how do you decrypt a headless server at boot time that has it’s root partition LUKS encrypted.. while not on site? I’ve been running my server for almost two years and I knew this problem would come up eventually so it’s been at the back of my mind for a while. And you know it had to come up at the most inopportune time… 15 minutes before I leave town for a weekend away. After a weekend without access to my network or any of the services I host I finally started working on solving this.

Quick primer on the typical Linux boot process: when you start your computer it loads a minimal OS called `initramfs` that knows how to decrypt your root and data partitions and start the boot process. `initramfs` is configured a few different ways, I’m on a RHEL based distro so I used dracut to configure it. Arch based distros use `mkinitramfs`. Unsure about Ubuntu since I prefer real operating systems ;)

There’s really two routes to take to solving this problem: fully automatic (bad security posture) and manual intervention (good security but inconvenient). The automatic path means writing the keys in plain text in the `initramfs` and the thought of plain text keys makes me full body cringe. It’s like having a safe with your combination on a Sticky Note next to the dial: why even have a safe? So I knew I needed to find a way to reach my box remotely during the initial boot process.

Luckily, this was fairly easy to implement. I found a very good Github repo https://github.com/gsauthof/dracut-sshd with full instructions. It does have a couple different options so here’s what I did:

I initially tried to go the systemd-networkd path but was not able to get it working. I instead had to add a network module to `initramfs`:

    ~$ echo 'add_dracutmodules+=" network "' >> /etc/dracut.conf.d/90-networkd.conf

Next you’ll want to put your public key in `/root/.ssh/authorized_keys`. You should generally generate a new public/private keypair for each new user/service:

    ~$ ssh-keygen -t ed25519 && cat ~/.ssh/id_ed25519.pub >> /root/.ssh/authorized_keys

Then it was as simple as adding the `46sshd` folder from the `dracut-sshd` repo to `/usr/lib/dracut/modules.d/` and rebuilding `initramfs`:

    ~$ git clone https://github.com/gsauthof/dracut-sshd.git ~/dracut-sshd
    ~$ mv ~/dracut-sshd/46sshd /usr/lib/dracut/modules.d
    ~$ dracut -v -f

The `Permission denied (public key)` issue described in the FAQ happened to me. Fixed by:

    ~$ usermod -p '*' root

That’s it! Restart your server and SSH in, run the Systemd ask password agent:

    ~$ systemd-tty-ask-password-agent

I hope this helps someone. If I would’ve known it was this easy I would’ve done it two years ago and never worried about how to reach home after the power goes out!