Skip to content

Parse JSON in Bash with jq (Stop Using grep and cut on API Responses)

bashjqjsonapiparsing
3 min read
Matching tooljq Filter Builder

Quick Answer

Parsing JSON with grep, cut, and sed feels fine until the API reformats its output — pretty-prints it, reorders keys, or ships a value containing your delimiter — and your pipeline silently pulls the wrong field or an empty string. JSON is not line-oriented, so line-oriented tools are guessing. jq is an actual JSON parser, and three flags do most of the work. -r prints raw strings instead of quoted ones, so a version like 1.2.3 comes back as 1.2.3 and not a quoted string that breaks every comparison. The // operator supplies a default when a key is missing, turning a null into a 0 you can handle. -e sets jq's exit code from the result, so your script can branch on whether a field was actually there. Add .array[] to iterate and select() to filter, and you can pull exactly what you need out of any response without a single regex.

jq is step two of the three-step pattern

The APIs guide is the full fetch → parse → alert script: curl that fails when the API fails, jq with -e so a missing field is an error, and a Slack alert when either breaks. Full guide: Shell Scripts That Talk to APIs.

The wrong-turn theory was that the API was down. A teammate's deploy script had started tagging every release latest instead of the real version number, and we spent an hour blaming the registry before I looked at the actual parse. The script read the version out of a JSON response with grep '"version"' | cut -d'"' -f4. It had worked for two years. Then the API started pretty-printing its output with the version nested one level deeper under a metadata object, grep matched a different "version" field that happened to appear first — a schema version, always "latest" — and cut dutifully handed it back. No error. Just the wrong string, confidently, every time. That's the trap with regex on JSON: it doesn't fail, it lies.

Why regex on JSON is guessing

grep, cut, sed, and awk are line-oriented tools, and JSON has no meaningful lines. {"a":1,"b":2} and the same object pretty-printed across six lines are identical data, but every line-based pattern you write depends on one specific layout. The API is free to change that layout any time — a new field, a reorder, a different indent, a value that contains a " or a : — and your pattern silently starts matching the wrong bytes. jq parses the whole document into a data structure first and then addresses it by path, so .metadata.version means that field regardless of how it's printed.

The three flags that matter

bash
#!/bin/bash # Script: parse-json.sh # Purpose: Read fields out of a JSON API response with jq — grep/cut on JSON breaks the moment the API reformats, and it fails silently # Usage: ./parse-json.sh <owner/repo> set -euo pipefail CHECK="✓" CROSS="✗" REPO="${1:?Usage: parse-json.sh <owner/repo>}" # Fail early and clearly on a box without jq, instead of a confusing error mid-pipeline. command -v jq >/dev/null 2>&1 || { echo "$CROSS jq is not installed (apt install jq / brew install jq)" >&2; exit 1; } response=$(curl -sS --max-time 30 "https://api.github.com/repos/${REPO}") # -e sets jq's exit code from the result: non-zero when .full_name is null/absent, # so a 404 or a rate-limit body branches here instead of returning an empty string. # -r prints the raw value, not a quoted "string" that breaks comparisons and paths. if ! name=$(printf '%s' "$response" | jq -er '.full_name'); then echo "$CROSS no repo returned (rate limited or 404?)" >&2 exit 1 fi # A missing key would print the literal 'null'; // supplies a real default instead. stars=$(printf '%s' "$response" | jq -r '.stargazers_count // 0') echo "$CHECK $name — $stars stars" # Iterate an array field. The ? after [] keeps jq quiet if 'topics' is absent. printf '%s' "$response" | jq -r '.topics[]? | " - " + .'

-r, //, and -e are the whole game for scripting. -r because a quoted string is not the string you want in a variable. // because real APIs omit optional fields and null is not a number. -e because "the field wasn't there" and "the field was empty" are different problems and you want your script to tell them apart.

Filtering with select()

When the response is a list and you only want some of it, select() is the filter:

bash
# Every active item's id, one per line — ready for a while-read loop printf '%s' "$response" | jq -r '.items[]? | select(.active == true) | .id'

Piping that into while IFS= read -r id; do …; done is the clean way to act on each match. Building these expressions by hand gets fiddly fast, which is exactly what the jq Filter Builder is for — paste a real response, click the fields you want, and copy the filter out.

Why jq over python -c

The obvious alternative is python3 -c 'import json,sys; print(json.load(sys.stdin)["name"])', and on a box that's guaranteed to have Python it's a fine fallback. I still reach for jq first because it's built for exactly this: the filters compose left to right like a shell pipe, -r and // and select are one token each instead of a line of Python, and it streams. The tradeoff is honest — jq is a dependency you have to install, and its syntax has a learning curve past the basics. For pulling a handful of fields out of an API response in a bash script, that curve pays for itself the first time an endpoint reformats and your jq path keeps working while a regex would have broken.

The response you're parsing should already be a verified 2xx — see making the request safely with curl. And when a parse fails because the API changed shape, that's exactly the kind of thing worth alerting to Slack rather than discovering downstream.

Raw script, MIT licensed: scripts/bash-parse-json-jq.sh on GitHub

PAID RESOURCE — $9

The Production Bash Toolkit

An operational script system + a 31-function shared library + a 52-page field guide. The production layer the free snippets don't cover.

Get the Toolkit →
curl -O bashlib-starter.sh

Get the bashlib starter

Ten functions I source into every script on my own boxes — strict-mode setup, an ERR trap that names the failing line, lock and timeout wrappers, and cleanup that runs on every exit path. One email, no sequence.

BashSnippets logo

Written by Travis

Creator of BashSnippets.xyz

bashsnippets.xyz/about

Related Snippets

Delete Old Log Files

Unmanaged log files silently fill /var/log until disk writes fail and services crash. find -mtime deletes .log files older than N days — preview with -print before removing from production.

Bash Error Handling with set -euo pipefail

Bash silently continues after failed commands by default — a broken cd followed by rm -rf destroys the wrong directory. set -euo pipefail exits on first failure before damage spreads.

Kill a Process with pkill and pgrep

The ps/grep/copy-PID/kill workflow takes four steps every time you need to stop a process. pkill by name collapses that to one command — pgrep -l previews matches before terminating.

Frequently Asked Questions

faq — snippet

Why shouldn't I parse JSON with grep or cut?

Because JSON has no line structure to rely on. grep '"id"' | cut -d: -f2 works only as long as every id sits alone on its own line with exactly one colon before it. The moment the API pretty-prints differently, reorders keys, nests the field, or a string value contains a colon or a brace, the regex matches the wrong thing — and it does so silently, returning a plausible-looking wrong value instead of an error. jq parses the document into a real structure first, so .id means the id no matter how the bytes are laid out.

faq — snippet

What does the -r flag do in jq?

-r (raw output) strips the surrounding quotes from string results. Without it, jq '.name' on {"name":"web-01"} prints "web-01" with the quotes included, so when you assign that to a variable and compare it to web-01 the comparison fails and file paths built from it are wrong. With -r you get web-01. Any time a jq value is going into a shell variable, a filename, or a comparison, you almost always want -r.

faq — snippet

How do I set a default when a JSON field might be missing?

Use the // operator: jq -r '.stargazers_count // 0' returns the count if the key exists and 0 if it is missing or null. Without it, a missing key prints the literal string null, which then poisons any arithmetic or comparison downstream. // is how you say 'use this field, but fall back to this value if it isn't there' in one expression.

faq — snippet

How do I loop over a JSON array in bash with jq?

Pipe the array through .array[] to emit one element per line, optionally filter with select(), then read the lines in bash. For example, jq -r '.items[]? | select(.active == true) | .id' prints the id of every active item, one per line, ready to feed into a while read loop. The ? after [] keeps jq from erroring if the array is absent.

← All Snippets↑ Back to top