shell-scripting

When this skill is activated, always start your first response with the 🧢 emoji.

Shell Scripting

Shell scripting is the art of automating tasks through the Unix shell - combining built-in commands, control flow, and process management to build reliable CLI tools and automation workflows. This skill covers production-quality bash and zsh scripting: robust error handling, portable argument parsing, safe file operations, and the idioms that separate fragile one-liners from scripts that hold up in production.

When to use this skill

Trigger this skill when the user:

• Asks to write or review a bash or zsh script
• Needs to parse command-line arguments or flags
• Wants to automate a CLI workflow or task runner
• Asks about exit codes, signal trapping, or error handling in shell
• Needs to process files, lines, or streams from the terminal
• Asks about here documents, process substitution, or subshells
• Wants a portable script that works across bash, zsh, and sh

Do NOT trigger this skill for:

• Python or Node.js CLI tools (shell is the wrong tool for complex logic)
• Scripts that require structured data parsing at scale (use a real language instead)

Key principles

1. Always use set -euo pipefail - Start every non-trivial script with this.

-e exits on error, -u treats unset variables as errors, -o pipefail catches failures in pipelines. Without this, silent failures hide bugs for weeks.

1. Quote everything - Always double-quote variable expansions: "$var", "$@",

"${array[@]}". Unquoted variables break on whitespace and glob characters. The only exceptions are intentional word splitting and arithmetic contexts.

1. Check dependencies upfront - Verify required commands exist before the script

runs. Fail fast at the top with a clear error, not halfway through a destructive operation.

1. Use functions for reuse and readability - Extract logic into named functions.

Shell functions support local variables (local), can return exit codes, and make scripts testable. A main() function at the bottom with a guard is idiomatic.

1. Prefer shell built-ins over external commands - [[ ]] over [ ], ${var##*/}

over basename, ${#str} over wc -c. Built-ins are faster, more portable, and avoid spawning subshells. Use printf over echo for reliable output formatting.

Core concepts

Exit codes - Every command returns an integer 0-255. 0 means success; any non-zero value means failure. Use $? to read the last exit code. Use explicit exit N to return meaningful codes from scripts. The || and && operators branch on exit code.

File descriptors - 0 = stdin, 1 = stdout, 2 = stderr. Redirect stderr with 2>file or merge it into stdout with 2>&1. Use >&2 to write errors to stderr so they don't pollute captured output.

Subshells - Parentheses (cmd) run commands in a child process. Changes to variables, cd, or set inside a subshell do not affect the parent. Command substitution $(cmd) also runs in a subshell and captures its stdout.

Variable scoping - All variables are global by default. Use local inside functions to limit scope. declare -r creates read-only variables. declare -a declares arrays; declare -A declares associative arrays (bash 4+).

IFS (Internal Field Separator) - Controls how bash splits words and lines. Default is space/tab/newline. When reading files line by line, set IFS= to prevent trimming of leading/trailing whitespace: while IFS= read -r line.

Common tasks

Robust script template with trap cleanup

Every production script should start with this foundation:

#!/usr/bin/env bash
set -euo pipefail

# --- constants ---
readonly SCRIPT_NAME="$(basename "$0")"
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
readonly TMP_DIR="$(mktemp -d)"

# --- cleanup ---
cleanup() {
  local exit_code=$?
  rm -rf "$TMP_DIR"
  if [[ $exit_code -ne 0 ]]; then
    echo "ERROR: $SCRIPT_NAME failed with exit code $exit_code" >&2
  fi
  exit "$exit_code"
}
trap cleanup EXIT INT TERM

# --- dependency check ---
require_cmd() {
  if ! command -v "$1" &>/dev/null; then
    echo "ERROR: required command '$1' not found" >&2
    exit 1
  fi
}
require_cmd curl
require_cmd jq

# --- main logic ---
main() {
  echo "Running $SCRIPT_NAME from $SCRIPT_DIR"
  # ... your logic here
}

main "$@"

The trap cleanup EXIT fires on any exit - success, error, or signal - ensuring temp files are always removed. BASH_SOURCE[0] resolves the script's real location even when called via symlink.

Argument parsing with getopts and long opts

Use getopts for POSIX-portable short flags. For long options, use a while/case loop with manual shift:

usage() {
  cat >&2 <<EOF
Usage: $SCRIPT_NAME [OPTIONS] <input>

Options:
  -o, --output <dir>   Output directory (default: ./out)
  -v, --verbose        Enable verbose logging
  -h, --help           Show this help
EOF
  exit "${1:-0}"
}

OUTPUT_DIR="./out"
VERBOSE=false

parse_args() {
  while [[ $# -gt 0 ]]; do
    case "$1" in
      -o|--output)
        [[ -n "${2-}" ]] || { echo "ERROR: --output requires a value" >&2; usage 1; }
        OUTPUT_DIR="$2"; shift 2 ;;
      -v|--verbose)
        VERBOSE=true; shift ;;
      -h|--help)
        usage 0 ;;
      --)
        shift; break ;;
      -*)
        echo "ERROR: unknown option '$1'" >&2; usage 1 ;;
      *)
        break ;;
    esac
  done
  # remaining positional args available as "$@"
  INPUT_FILE="${1-}"
  [[ -n "$INPUT_FILE" ]] || { echo "ERROR: input file required" >&2; usage 1; }
}

parse_args "$@"

File processing - read, write, and temp files safely

# Read a file line by line without trimming whitespace or interpreting backslashes
while IFS= read -r line; do
  echo "Processing: $line"
done < "$input_file"

# Read into an array
mapfile -t lines < "$input_file"   # bash 4+; equivalent: readarray -t lines

# Write to a file atomically (avoids partial writes on failure)
write_atomic() {
  local target="$1"
  local tmp
  tmp="$(mktemp "${target}.XXXXXX")"
  # write to tmp, then atomically rename
  cat > "$tmp"
  mv "$tmp" "$target"
}
echo "final content" | write_atomic "/etc/myapp/config"

# Safe temp file with auto-cleanup (cleanup trap handles TMP_DIR removal)
local tmpfile
tmpfile="$(mktemp "$TMP_DIR/work.XXXXXX")"
some_command > "$tmpfile"
process_result "$tmpfile"

String manipulation without external tools

# Substring extraction: ${var:offset:length}
str="hello world"
echo "${str:6:5}"        # "world"

# Pattern removal (greedy ##, non-greedy #; greedy %%, non-greedy %)
path="/usr/local/bin/myapp"
echo "${path##*/}"       # "myapp"     (strip longest prefix up to /)
echo "${path%/*}"        # "/usr/local/bin" (strip shortest suffix from /)

# Search and replace
filename="report-2024.csv"
echo "${filename/csv/tsv}"   # "report-2024.tsv"   (first match)
echo "${filename//a/A}"      # "report-2024.csv" -> "report-2024.csv" (all matches)

# Case conversion (bash 4+)
lower="${str,,}"         # all lowercase
upper="${str^^}"         # all uppercase
title="${str^}"          # capitalise first character

# String length and emptiness checks
[[ -z "$var" ]] && echo "empty"
[[ -n "$var" ]] && echo "non-empty"
echo "length: ${#str}"

# Check if string starts/ends with a pattern (no grep needed)
[[ "$str" == hello* ]] && echo "starts with hello"
[[ "$str" == *world ]] && echo "ends with world"

Parallel execution with xargs and GNU parallel

# xargs: run up to 4 jobs in parallel, one arg per job
find . -name "*.log" -print0 \
  | xargs -0 -P4 -I{} gzip "{}"

# xargs with a shell function (must export it first)
process_file() {
  local f="$1"
  echo "Processing $f"
  # ... work ...
}
export -f process_file
find . -name "*.csv" -print0 \
  | xargs -0 -P"$(nproc)" -I{} bash -c 'process_file "$@"' _ {}

# GNU parallel (more features: pro