JSON Transform (jq-lite)

What is jq?

jq is a lightweight command-line JSON processor that provides a complete programming language for transforming, filtering, and reshaping JSON data. Created by Stephen Dolan and first released in 2012, jq has become the de facto standard for manipulating JSON in shell scripts, CI/CD pipelines, and data processing workflows. Its expression syntax is compact yet powerful — a single jq filter can extract deeply nested fields, restructure objects, perform arithmetic, apply conditionals, and construct entirely new JSON documents from existing data. This browser-based tool provides a lightweight jq expression evaluator that runs entirely client-side, allowing you to develop and test jq expressions interactively without installing jq locally. Enter your JSON input, write a jq expression, and see the transformed output instantly.

Core jq Operations and Syntax

The simplest jq expression is the identity filter ., which passes input through unchanged. From there, you build expressions by chaining operations with the pipe operator |, which feeds the output of one filter into the next — similar to Unix pipes. Field selection uses dot notation: .name extracts the "name" property from an object, .address.city reaches into nested structures, and .["key-name"] handles keys containing special characters or hyphens.

Array operations: The array iterator .[] explodes an array into its individual elements, producing one output per item. Combined with pipes, this enables powerful transformations: .users[] | .email extracts the email field from every element in the users array. Array indexing uses bracket notation (.[0] for the first element, .[-1] for the last), and slicing follows Python-style syntax (.[2:5] for elements at indices 2 through 4).

Object construction: jq can build new objects from existing data using curly braces. The expression {name: .user.name, age: .user.age} constructs a new object with only the specified fields. When the key name matches the field name, the shorthand {name, age} is equivalent to {name: .name, age: .age}. This capability makes jq ideal for reshaping API responses into the exact structure your application expects, discarding unnecessary fields and renaming others.

String interpolation: jq supports embedded expressions inside strings using the \(expr) syntax. For example, "Hello, \(.name)" inserts the value of .name into the string. This is useful for generating formatted output, constructing URLs from data fields, or creating log messages that combine multiple values.

Built-in Functions: map, select, keys, and More

jq includes a rich standard library of built-in functions for common data operations. The map(f) function applies a filter f to every element of an array and collects the results: map(.price * 1.1) increases all prices by 10%. The select(condition) function filters elements based on a boolean expression — .[] | select(.active == true) keeps only active items, discarding the rest.

Inspection functions: keys returns an array of all property names in an object (sorted alphabetically), useful for exploring unfamiliar JSON structures. length returns the number of elements in an array, the number of keys in an object, or the character count of a string. type returns a string indicating the JSON type of the current value: "object", "array", "string", "number", "boolean", or "null". These introspection functions help write filters that adapt to varying input shapes.

Conditional logic: jq supports if-then-else expressions for branching logic: if .status == "active" then .name else "inactive" end. The alternative operator // provides default values — .nickname // .name returns the nickname if it exists and is not null or false, falling back to the name field otherwise. This is particularly useful when processing data that may have optional fields.

Aggregation and reduction: The add function sums an array of numbers or concatenates an array of strings. Combined with map, it enables aggregation patterns like [.items[] | .price] | add to compute a total. The group_by(f) function partitions an array into groups sharing the same value of f, and sort_by(f) orders elements by the value of an expression. These functions cover the most common data analysis tasks without leaving the jq ecosystem.

Practical Transformation Patterns

Real-world jq usage often involves combining multiple operations into transformation pipelines that reshape data for consumption by other systems. Here are patterns developers frequently need when working with API responses, configuration files, and data exports:

Flattening nested structures: APIs often return deeply nested objects that need flattening for tabular display or CSV export. A filter like .results[] | {id: .id, city: .address.city, country: .address.country} pulls nested address fields up to the top level. When arrays are nested inside arrays, multiple .[] iterators or the flatten function collapse the nesting.

Filtering and transforming simultaneously: The combination of select and object construction handles the common pattern of filtering records and extracting specific fields in one pass: [.users[] | select(.role == "admin") | {name, email, lastLogin: .last_login}] produces a clean array of admin user summaries. Wrapping the expression in square brackets collects the individual outputs back into a single array.

Converting between formats: jq excels at restructuring data between different JSON shapes. Converting an array of key-value pairs into an object uses from_entries, while the reverse uses to_entries. Renaming keys across an entire dataset combines to_entries, map, and from_entries: converting snake_case API responses to camelCase for frontend consumption is a common application of this pattern.

Handling optional and nullable fields: Production JSON data frequently contains null values or missing properties. The try-catch operator ? suppresses errors from missing paths — .config.timeout? returns null silently instead of failing if config or timeout is absent. The alternative operator // then provides sensible defaults: (.config.timeout? // 30) uses 30 when the field is missing.

When to Use jq vs JSONPath

Both jq and JSONPath query JSON documents, but they serve fundamentally different purposes and choosing the right tool depends on what you need to accomplish with the data:

Use jq when you need to transform data: jq is a complete data transformation language. It can reshape objects, compute derived values, merge documents, aggregate arrays, and construct entirely new output structures. If your task involves changing the shape of JSON data — not just reading it — jq is the appropriate tool. Typical jq tasks include: converting API responses to a different schema, computing summaries from raw data, merging multiple JSON files, generating configuration from templates, and reformatting data for import into other systems.

Use JSONPath when you need to extract data: JSONPath is a query language designed for selecting elements from a JSON document without modifying it. It excels at pointing to specific values inside complex structures using concise path expressions. JSONPath is read-only by design — it can filter and select, but it cannot restructure, compute, or construct new output. Typical JSONPath tasks include: extracting a specific field from an API response, writing test assertions about response structure, and configuring monitoring alerts based on specific values in JSON payloads.

Complementary usage: In practice, developers often use both tools in different contexts within the same project. JSONPath handles simple extractions in test frameworks and configuration, while jq handles complex transformations in build scripts and data pipelines. Understanding both helps you choose the minimal tool for each specific task rather than over-engineering simple queries or under-powering complex transformations.

Code Examples

Common jq expressions applied to sample JSON data

// Input JSON
{
  "users": [
    { "name": "Alice", "age": 32, "role": "admin", "active": true },
    { "name": "Bob", "age": 28, "role": "editor", "active": true },
    { "name": "Charlie", "age": 45, "role": "viewer", "active": false },
    { "name": "Diana", "age": 36, "role": "admin", "active": true }
  ],
  "metadata": { "total": 4, "page": 1 }
}

// Expression: .users[] | .name
// Output:
// "Alice"
// "Bob"
// "Charlie"
// "Diana"

// Expression: .users | length
// Output: 4

// Expression: [.users[] | select(.role == "admin")]
// Output: [{"name":"Alice","age":32,"role":"admin","active":true},
//          {"name":"Diana","age":36,"role":"admin","active":true}]

// Expression: [.users[] | select(.active) | {name, role}]
// Output: [{"name":"Alice","role":"admin"},
//          {"name":"Bob","role":"editor"},
//          {"name":"Diana","role":"admin"}]

// Expression: .users | map(.age) | add / length
// Output: 35.25

// Expression: .users | group_by(.role) | map({role: .[0].role, count: length})
// Output: [{"role":"admin","count":2},{"role":"editor","count":1},{"role":"viewer","count":1}]

// Expression: {total_users: .metadata.total, admins: [.users[] | select(.role == "admin") | .name]}
// Output: {"total_users":4,"admins":["Alice","Diana"]}

Reshaping API response data with jq pipes and object construction

# Extract names from a JSON API response
echo '{"results":[{"id":1,"profile":{"name":"Alice"}},{"id":2,"profile":{"name":"Bob"}}]}' \
  | jq '[.results[] | {id, name: .profile.name}]'
# Output: [{"id":1,"name":"Alice"},{"id":2,"name":"Bob"}]

# Filter and transform with conditionals
echo '{"items":[{"product":"A","stock":0},{"product":"B","stock":15},{"product":"C","stock":3}]}' \
  | jq '[.items[] | select(.stock > 0) | {product, status: (if .stock > 10 then "plenty" else "low" end)}]'
# Output: [{"product":"B","status":"plenty"},{"product":"C","status":"low"}]

# Convert an object to key-value pairs
echo '{"host":"localhost","port":5432,"db":"myapp"}' \
  | jq 'to_entries | map("\(.key)=\(.value)") | .[]'
# Output:
# "host=localhost"
# "port=5432"
# "db=myapp"

# Provide defaults for missing fields
echo '{"name":"service","config":{}}' \
  | jq '{name, timeout: (.config.timeout // 30), retries: (.config.retries // 3)}'
# Output: {"name":"service","timeout":30,"retries":3}

Standards & Specifications

• jq Manual — The official jq language reference documenting all built-in operators, functions, and expression syntax
• jq Language Description — Community-maintained formal description of jq semantics including grammar rules and evaluation model
• ECMA-404 (JSON Standard) — The JSON data interchange format standard that defines the input/output data model for jq expressions