Hero Commands

I once worked with a woman named Mirya who was, without question, the best Unix admin I’ve ever met. She could rattle off complex pipelines and write `awk` and `sed` scripts from memory, pulling off shockingly powerful feats with *apparent* ease. I asked her for a copy of her shell history once and studied it like it was Merlin’s spellbook. Ever since, I’ve been collecting history files, Perl notebooks, and shell one-liners—the kind of arcane knowledge that can turn you into a command-line hero.

🧾 View Hero-Commands Series

🧙‍♂️ Path Walk

Sometimes you need to troubleshoot the permissions on the whole directory path from your current working directory all the way back to /. Compiling that report manually can really fill up the ‘swear jar’ quickly. Luckily this path walk command makes it easy to see it all together so you can focus on the important stuff.

macOS

Shows permissions, ACLs and xattrs — all in one command.

while [ $PWD != / ]; do ls -aled@ `pwd`; cd ..; done; ls -aled@ `pwd`

Linux

Points to macOS for the elegance of it’s version. This is the POSIX / Linux equivalent.

while [ "$PWD" != "/" ]; do echo; echo "--- $PWD ---"; ls -ld "$PWD"; getfacl -p "$PWD" 2>/dev/null; getfattr -d "$PWD" 2>/dev/null; cd ..; done; echo; echo "--- $PWD ---"; ls -ld "$PWD"; getfacl -p "$PWD" 2>/dev/null; getfattr -d "$PWD" 2>/dev/null

This will produce output like this:

--- /home/grumble/codelab/adminjitsu ---
drwxr-xr-x - grumble  4 Aug 10:39 /home/grumble/codelab/adminjitsu
# file: /home/grumble/codelab/adminjitsu
# owner: grumble
# group: grumble
user::rwx
group::r-x
other::r-x


--- /home/grumble/codelab ---
drwxr-xr-x - grumble  4 Aug 08:49 /home/grumble/codelab
# file: /home/grumble/codelab
# owner: grumble
# group: grumble
user::rwx
group::r-x
other::r-x


--- /home/grumble ---
drwxr-x--- - grumble  4 Aug 20:54 /home/grumble
# file: /home/grumble
# owner: grumble
# group: grumble
user::rwx
group::r-x
other::---


--- /home ---
drwxr-xr-x - root  9 Oct  2024 /home
# file: /home
# owner: root
# group: root
user::rwx
group::r-x
other::r-x


--- / ---
drwxr-xr-x - root  2 Aug 16:38 /
# file: /
# owner: root
# group: root
user::rwx
group::r-x
other::r-x

Bash Function

Since these commands do not require arguments, they make great Bash, alias-like functions. The following will do the same thing with a single command. You just need to add the following to your startup files (e.g., .bashrc,.zshrc) and reload. As a script, it demonstrates a nice pattern for using pushd and popd to perform a sequence of actions and then return to the original directory.

macOS Version

pathwalk() {
  pushd . > /dev/null
  while [ "$PWD" != "/" ]; do
    ls -aled@ "$(pwd)"
    cd ..
  done
  ls -aled@ "$(pwd)"
  popd > /dev/null
}

Linux Version

pathwalk() {
  pushd . > /dev/null
  while [ "$PWD" != "/" ]; do
    echo
    echo "--- $PWD ---"
    ls -ld "$PWD"
    getfacl -p "$PWD" 2>/dev/null
    getfattr -d "$PWD" 2>/dev/null
    cd ..
  done
  echo
  echo "--- $PWD ---"
  ls -ld "$PWD"
  getfacl -p "$PWD" 2>/dev/null
  getfattr -d "$PWD" 2>/dev/null
  popd > /dev/null
}

🜔 Diff a remote file

Sometimes you need to compare a local file with a remote one. Copying files to your machine via scp gets old fast. That’s where this simple but not completely obvious command comes in.

ssh user@host cat /path/to/remotefile | diff /path/to/localfile -

This command streams the contents of the remote file over SSH.
Pipes that remote file into diff using - as an argument to mean “read from stdin”
Compares the local file (/path/to/localfile) against the remote — not temp files, no manual copies, no extra cleanup.
Use sudo if required, ala ssh user@host sudo cat /etc/somefile | diff /path/to/localfile -

An example: ssh user@host sudo cat /etc/somefile | diff /path/to/localfile -

Pro-tip Use colordiff for pretty output

ssh host 'openssl x509 -in /etc/ssl/cert.pem -text' | diff cert.txt -

“Technology is a word that describes something that doesn’t work yet.” — Douglas Adams

There are a few basic commands that really help when you find yourself jumping between directories (web development projects for instance)

Jump to previous directory

No matter how twisty and deep the path, cd - lets you jump to your previous folder and back if you run it again.
```
# toggles between your current and previous directory
cd -
```
Pushd, Popd and Dirs

For when you want to bookmark your current spot before diving into another directory
- pushd <dir>: saves your current location and jumps to
- popd: Jumps back to when you last used pushd (and removes that location from the stack)
- Use these to create a stack of locations for more complex navigation
Example
```
pushd /etc
# Do stuff in /etc
pushd /var/log
# Do stuff in /var/log
popd
# Back to /etc
popd
# Back to where you started
```
Pro-tip You can use the dirs command to view the locations stack

Using these tools you can (somewhat) eliminate the tedium of typing /annoyingly/long/paths

A skeleton wielding a flamethrower — *hero commands: because you never know what kind of monsters you'll encounter*

🐍 Python tools

There are a couple of fantastic tools that “come with” Python and let you do magical things easily. I use them all the time!

1. Validate & Pretty-print JSON

echo '{"json":"obj"}' | python -mjson.tool
# Or with files:
python -mjson.tool < ugly.json > pretty.json

Validates JSON (throws error if invalid).
Indents and formats output for humans.
Saves the day ocassionally

2. HTTP Client & Web Testing

Simple HTTP GET/POST from the command line:

python -m http.client HOST
# Interactive HTTP client (rarely used, but there)

Quickly start a web server (Python 3):

python -m http.server 8000
# Serves current directory on whatever port you specify (8000 in this example)

Great for quick file sharing or local dev.

Run a CGI server (Python 3):

python -m http.server --cgi
# Runs scripts in cgi-bin/

Listen on a specific interface

You can bind to localhost only for instances with the --bind argument, like so →

python -m http.server 8000 --bind 127.0.0.1

3. Encode/Decode Base64

echo 'hello' | python -m base64
# Or:
python -m base64 -d < encoded.txt > decoded.bin

No need for base64 utility, works everywhere Python is installed.

4. Compress/Decompress Files

gzip:

python -m gzip -d file.gz
python -m gzip file

zip:

python -m zipfile -l my.zip    # List contents
python -m zipfile -e my.zip .  # Extract here
python -m zipfile -c out.zip file1 file2  # Create zip

5. Simple File Serving

python -m http.server  # Python 3.x
python -m SimpleHTTPServer  # Python 2.x

Dead simple “share this folder” in seconds.

␀ Empty a file

“The usefulness of a pot comes from its emptiness.”

— Lao Tzu, Tao Te Ching

I frequently want to zero out test files that I am working with. The following are a few ways to do it, easily and safely

# Redirects "nothing" into the file
>filename

# or use truncate. Sometimes you can't use >filename and that's where these less memorable commands come in
truncate -s 0 filename

# you can also do the following just make sure to wrap the comand in single quotes or it will usually fail
sudo bash -c '> filename'

📚 Make ps output easier to read

“If you gaze long into /dev/null, /dev/null gazes into you.

- Friedrich Nietzsche, if he were a sysadmin

I covered this in more detail in another post, ps-for-spelunkers

Basically, you can insert a blank line (or more than one) between each process in the output. This is super useful when you’re trying to make sense of processes with a lot of switches like Java, Databases and Webservers.

echo "$(ps aux)" | awk '{print;} NR % 1 == 0 {print"";}'

Examples

ls -l | awk '{print;} NR % 1 == 0 {print"";}'
cat bigfile.txt | awk '{print;} NR % 1 == 0 {print"";}'

More than one blank line

Just change the modulus in the command:

awk '{print;} NR % 1 == 0 {print"";} NR % 3 == 0 {print"";}'

Pro-tip Specifying just the columns you are interested in conjunction with the awk command makes it even easier to read. ps -eo pid,user,args

a purple wizard — *Sometimes good guys don't wear white*

⇨ Awk: print from a given column to end of line

I had to work with some logs that had optional columns and found myself needing a way to print from a column I wanted to the end of the line. Instead of making a bunch of custom commands for each case, I turned to our old pal, Awk.

Fun fact - AWK is named for its creators, Alfred Aho, Peter Weinberger, and Brian Kernighan

awk '{for(i=4;i<=NF;++i) printf "%s%s", $i, (i<NF?OFS:ORS)}'

You can customize the command easily:

i=4 means start at the 4th field (adjust as needed).
OFS is output field separator (default: space).
ORS is output record separator (default: newline).

Example Usage

cat opendirectoryd.log | awk '{for(i=4;i<=NF;++i) printf "%s%s", $i, (i<NF?OFS:ORS)}'

⌬ Get number of CPU cores

If you ever need to fetch the number of CPU cores on a machine from a script, you can turn to this trusty cross-platform command:

getconf _NPROCESSORS_ONLN

On my 6-core, hyperthreaded Mac, this returns 12.

That comes in handy for things like a yes stress tester

Warning this will max out your processor and produce maximum heat which you probably don’t want to do unless you’re testing your cooling setup. Although if you are testing cooling, then you’re welcome!

for i in $(seq 1 $(getconf _NPROCESSORS_ONLN)); do yes > /dev/null & done

When you’re ready to stop the yes stress test, run:

killall yes

of if you happen to be on a system without killall you can run:

pkill -9 yes

⟡ A listeners function

This is a nice and simple way to display your listening processes and ports. I keep this on all my machines; it’s my go-to for quickly reminding myself what’s listening. Just add the following to your shell session or put it in your startup files (e.g., .bashrc, .zshrc)

listeners() {
  sudo lsof -nP -iTCP -sTCP:LISTEN |
  awk 'BEGIN {
          format = "%-16s %-8s %-22s %-10s\n"
          printf format, "[PROC]", "[PID]", "[PORT]", "[ACCOUNT]"
          printf format, "------", "----", "------", "--------"
      }
      NR > 1 {printf format, $1, $2, $9, $3}'
}

Getting the quoting right is tough, but you can also run it as a one-liner over ssh if you’re feeling kinky.

ssh user@host "sudo lsof -nP -iTCP -sTCP:LISTEN | awk 'BEGIN {format = \"%-16s %-8s %-22s %-10s\\n\"; printf format, \"[PROC]\", \"[PID]\", \"[PORT]\", \"[ACCOUNT]\"; printf format, \"------\", \"----\", \"------\", \"--------\"} NR > 1 {printf format, \$1, \$2, \$9, \$3}'"

Example output:

┌──[ forfaxx@shinobi ]:~/codelab/adminjitsu  (main*)
└─$ listeners
[PROC]           [PID]    [PORT]                 [ACCOUNT]
------           ----     ------                 --------
systemd          1        *:111                  root
systemd          1        *:111                  root
systemd          1        [::1]:2947             root
systemd          1        127.0.0.1:2947         root
rpcbind          208      *:111                  _rpc
rpcbind          208      *:111                  _rpc
sshd             334      *:22                   root
sshd             334      *:22                   root
node             940      127.0.0.1:46267        forfaxx
hugo             315471   127.0.0.1:1313         forfaxx

Conclusion

That’s it for now but more to come ! Have a hero command of your own? I’d love to hear about it! Email me: feedback@adminjitsu.com

🧙‍♂️ Path Walk#

Bash Function#

macOS Version#

Linux Version#

🜔 Diff a remote file#

🝰 Navigation power moves#

🐍 Python tools#

1. Validate & Pretty-print JSON#

2. HTTP Client & Web Testing#

Simple HTTP GET/POST from the command line:#

Quickly start a web server (Python 3):#

Run a CGI server (Python 3):#

Listen on a specific interface#

3. Encode/Decode Base64#

4. Compress/Decompress Files#

5. Simple File Serving#

␀ Empty a file#

📚 Make ps output easier to read#

Examples#

More than one blank line#

⇨ Awk: print from a given column to end of line#

⌬ Get number of CPU cores#

⟡ A listeners function#

Conclusion#