SSH Setup Guide

Intro

SSH is one of those indispensible tools that quietly becomes the backbone of your entire setup. It’s how you log in, move files, hop between machines, and even tunnel services. Getting it set up correctly is important especially since you tend not to do these steps very often.

But here’s the thing: SSH is deceptively simple on the surface—type ssh user@host and you’re in—yet maddeningly complex once you start juggling multiple machines, keys, configs, and workflows. New users often stumble on permissions, while seasoned pros sometimes end up with messy key sprawl and brittle configs. Either way, pain awaits without a little discipline.

In this guide, I want to capture a clean, repeatable workflow for standing up a new machine with reciprocal SSH trust, a well-structured config, and a few tips and tricks that make everyday use faster, safer, and less painful. Nothing groundbreaking—just the kind of practical guide I wish I’d had when I was first wiring machines together.

We’ll cover:

• Generating modern SSH keys the right way.
• Copying them safely between machines (both directions).
• Setting up a maintainable ~/.ssh/config.
• Using SSH agent and keychains effectively.

For multiplexing, tunneling, and other advanced tricks, check out the companion post

⚔️ Continue to SSH-Fu

Why SSH matters

At its core, SSH (Secure Shell) gives you three superpowers:

1. Secure Remote Access
   Encrypted sessions mean you can log in to a remote system without worrying about someone sniffing your password or keystrokes. This replaced insecure tools like Telnet and rlogin.

2. File Transfer
   With scp and sftp, SSH is a safe way to move files. Add rsync over SSH and the ability to mount paths with sshfs, and you’ve got a flexible toolkit for syncs, backups, and deployments.

3. Tunnels and Proxies
   SSH can forward ports, set up SOCKS proxies, and even “jump” through one machine to reach another. This makes it the Swiss Army knife of your network.

Configuring sshd_config

On most systems the server daemon is sshd, and its main config lives in /etc/ssh/sshd_config. There are a lot of options but the ones we are concerned with are as follows. On the machine you will be connecting to (we’ll call it new-server), make sure you have the following set:

# Only allow modern protocol
Protocol 2

# Enable public key auth
PubkeyAuthentication yes

# Disable password logins once keys are working
PasswordAuthentication no

# Prevent root from password logins (or at all)
PermitRootLogin prohibit-password

# Security hygiene
ChallengeResponseAuthentication no
PermitEmptyPasswords no
UsePAM yes

After making edits, make sure to reload:

sudo systemctl reload sshd    # systemd
# or
sudo service ssh reload       # SysVinit

Pro-Tip Never flip PasswordAuthentication no until you’ve verified that you can log in with a key from another terminal session to avoid getting locked out.

Generating and Installing Keys

Now that you have the machine you will be connecting to ready to accept PubkeyAuthentiation, you’ll need to generate keys on your client and then copy the public key to new-server. You need to have a working user account on new-server as well but in this scenario we’ll assume you are using the same username across your machines.

On your client machine:

ssh-keygen -t ed25519 -a 100 -C "forfaxx@laptop (2025-08)"

Copy the public key to the server:

ssh-copy-id forfaxx@new-server.darkstar.home

If ssh-copy-id isn’t available:

cat ~/.ssh/id_ed25519.pub | ssh forfaxx@new-server.darkstar.home \
 'umask 077; mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys'

On the server, fix permissions:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Client Config (~/.ssh/config)

On your client you’ll want to add an entry to .ssh/config to make it easier to connect. Without an entry you need to remember to connect with a command like:

ssh -i ~/.ssh/id_ed25519 forfaxx@new-server.darkstar.home

So to make it easier, we’ll add an entry to ~/.ssh/config (create it if it doesn’t exist) and include an entry like the following:

Host new-server
  HostName new-server.darkstar.home
  User forfaxx
  IdentityFile ~/.ssh/id_ed25519
  IdentitiesOnly yes
  ServerAliveInterval 30

With that in place you can connect with:

ssh new-server

This is where the real quality of life improvements begin — shorter commands, per-host options, and tricks like multiplexing or jump hosts.

Using an SSH Agent (Stop Skipping This)

If you’ve been running keys without passphrases (guilty 🙋), it’s probably because you thought:
“Why bother typing a passphrase every single time I connect? Connecting without a password is terrific!”

The problem is:

• Without a passphrase, your private key file is just a single factor.
• If a bad guy steals ~/.ssh/id_ed25519, they don’t need anything else — they are you.

So you’re stuck in a dilemma:

• With a passphrase → secure, but annoying to type constantly.
• Without a passphrase → convenient, but dangerously insecure.

That’s exactly what the SSH agent solves.
Think of it as a guard that holds your unlocked key safely in memory for the session. You type the passphrase once, hand it to the guard, and from then on SSH just works — no constant retyping, no unprotected key file.

The best part: you don’t have to sacrifice security for convenience. With an agent, you get both.

How It Works

• Normally, SSH has to decrypt your private key every single time (prompting you for the passphrase).
• The agent is a background process (ssh-agent) that:
  1. Runs once per login/session.
  2. Holds your unlocked key safely in memory.
  3. Answers “signing challenges” on your behalf — without ever exposing the key.
• You unlock the key once with ssh-add, then it’s smooth sailing until you log out or reboot.

One-Time Setup (per session)

# 1. Start the agent (if not already running)
eval "$(ssh-agent -s)"

# 2. Add your key (enter passphrase once)
ssh-add ~/.ssh/id_ed25519

# 3. Verify it's loaded
ssh-add -l

That’s it. Every ssh, scp, or git command will now silently ask the agent instead of bugging you.

How Long Does It Last?

• Until you log out or reboot (or until the agent is killed).
• You can set a timeout if you want the agent to forget automatically:

  ssh-add -t 1h ~/.ssh/id_ed25519
  

• On macOS, you can store keys in the Keychain so they come back on login:

  ssh-add --apple-use-keychain ~/.ssh/id_ed25519
  

Quick Recap

Mental model: like making coffee in the morning → unlock once, good all day.

Make the Agent Automatic (so you don’t have to remember)

Here’s how to set it and forget it on each OS.

macOS (seamless)

One-time:

ssh-add --apple-use-keychain ~/.ssh/id_ed25519

In ~/.ssh/config:

Host *
  UseKeychain yes
  AddKeysToAgent yes

Result: key unlocks at login, no extra steps.

Linux (single line in shell rc)

Add to ~/.bash_profile or ~/.zshrc:

# Start agent if needed, add key if missing (prompts once per login)
ssh-add -l >/dev/null 2>&1 || { eval "$(ssh-agent -s)" >/dev/null; ssh-add ~/.ssh/id_ed25519 </dev/tty; }

Result: each login → one passphrase prompt, then done.

Windows 10/11

One-time enable:

Set-Service ssh-agent -StartupType Automatic
Start-Service ssh-agent

In your PowerShell profile ($PROFILE):

# If agent has no identities, add mine (prompts once per login)
if ((ssh-add -l) -match 'The agent has no identities') {
  ssh-add "$env:USERPROFILE\.ssh\id_ed25519" | Out-Null
}

Result: first terminal after login prompts once, then all SSH/Git just works.

Handy Commands

ssh-add -l        # list loaded keys
ssh-add -D        # remove all keys from agent
ssh-agent -k      # stop agent (Linux/macOS)

Bottom line:
Keep your keys encrypted with a passphrase.
Let the agent do the remembering.
Unlock once, fight battles all day.

Links and stuff

• RFC 4251 - The Secure Shell (SSH) Protocol Architecture
• ssh(1) man page — man7.org
• ssh_config(5) man page — man7.org
• sshd_config(5) man page — man7.org
• Wikipedia: Secure Shell
• OpenSSH Primer
• US-CERT Advisories (search for “SSH”)

Conclusion

SSH is a deceptively simple command with a lot of hidden depth.
A little upfront discipline—clean keys, a modular config, and a few tricks—turns it into one of the most powerful, reliable tools in your kit.

⚔️ Continue to SSH-Fu

Got feedback, corrections, or your own SSH war stories?
I’d love to hear them: feedback@adminjitsu.com.