Skip to main content

Online edition for BLOGE 0.9.8-RC1 · facts verified 2026-09-15 · 中文

Chapter 4 — Data That Flows

Promise: By the end of this chapter you will understand how data enters, moves through, and exits a BLOGE graph — and you will know when to use an input binding, a transform, or a full operator.

Learning Goals​

  1. Explain the three data sources available inside an input block: context values, upstream outputs, and literal values.
  2. Use path expressions to reach nested fields, including safe navigation (?.) and null coalescing (??).
  3. Reach into lists and maps with index access (xs[i], xs?[i], xs[-1], m["key"]) and use dynamic-member access on expression results ((expr).field, (expr)?.field).
  4. Distinguish a transform (projection without Operator scheduling) from an operator node (business logic with scheduling overhead).
  5. Use lambda expressions in input bindings for collection operations (map, filter, reduce).

Prerequisites​

Source Examples​

FileWhat it shows
order-enrichment-lambda.blogeLambda collection operations in input bindings
order-summary-index-access.blogeIndex, negative-index, and dynamic-member access
basic.bloge (transform)A minimal transform block
safe-navigation.blogeSafe navigation (?.) and null coalescing (??)
lambda-map.blogeLambda map in an input binding
ch03/order-process.blogeTransform block in a realistic workflow

Why This Matters​

In Chapter 3 you learned to think in dependencies. But dependencies only determine when a node runs — not what data it receives.

Getting data flow right is the difference between a clean, maintainable graph and one where every node does too much work reshaping data for the next node. BLOGE gives you a layered toolkit for data:

LayerToolScheduling costWhen to use
RoutingInput binding (input { … })None — resolved at assembly timePassing fields from context or upstream outputs
ProjectionTransform block (transform { … })None — virtual node, no operatorReshaping or renaming fields without business logic
ComputationOperator node (node … : Op { … })Full — scheduled, timed, retriableBusiness logic, I/O, or anything with side effects

Choosing the right layer keeps your graphs fast and readable.

Mental Model​

Diagram: 04-data-that-flows figure 1

  • Input bindings pull data from ctx or from someNode.output.field.
  • Transforms are virtual — the engine never schedules them on a thread. They simply create a new data shape from existing values.
  • Operator nodes execute real code with timeout, retry, and fallback.

First Working Example​

Input bindings — the basics​

Every input { … } block is a set of key-value bindings. The right-hand side is an expression that the engine evaluates at input-assembly time.

// Simplified teaching example — demonstrates all three data sources in one block
node calcPrice : CalcPriceOperator {
  input {
    user     = fetchUser.output           // whole output object
    products = fetchProducts.output       // whole output object
    userId   = fetchUser.output.id        // nested field
    tax      = ctx.taxRate                // context value
    label    = "order"                    // literal string
  }
}

Three data sources, one block:

SourceSyntaxExample
Contextctx.fieldNamectx.taxRate
Upstream outputnodeName.output or nodeName.output.fieldfetchUser.output.id
LiteralA quoted string or number"order", 42

Path expressions​

Path expressions let you navigate nested structures:

input {
  city = fetchUser.output.address.city
}

This traverses the output object: output → address → city.

Safe navigation and null coalescing​

When a field might be null, use the safe navigation operator (?.) to avoid null-pointer errors, and the null coalescing operator (??) to provide a default:

// From safe-navigation.bloge
node normalize : Normalize {
  input {
    profileName = ctx.request?.user?.name
    result      = fetch?.result?.value ?? "missing"
    names       = ctx.items?.map(x -> x.profile?.name)
  }
}

(From safe-navigation.bloge.)

OperatorMeaning
?.If the left side is null, stop and return null instead of throwing
??If the left side is null, use the right-side default

These compose naturally: a?.b?.c ?? "default" means "navigate into a.b.c; if anything along the way is null, use "default"."

Accessing lists and maps​

Path expressions reach fields. To reach elements of a list or entries of a map, use the index access operators:

SyntaxMeaning
xs[i]Element at position i; throws if the index is out of range
xs?[i]Safe element access; yields null instead of throwing
xs[-1]Negative indices count from the tail (-1 is the last element)
m["key"]Look up a map entry by its string key
(expr).fieldRead a field from an arbitrary expression result
(expr)?.fieldSafe field read on an expression result

Index access composes with everything else you already know — safe navigation, null coalescing, lambdas — and is resolved at input-assembly time, with no extra scheduling overhead:

// From ch04/order-summary-index-access.bloge
node fetchOrders : FetchOrdersOperator {
  input {
    customerId = ctx.customerId
  }
}

transform summary {
  firstOrderId = fetchOrders.output.recentOrderIds[0]
  lastOrderId  = fetchOrders.output.recentOrderIds[-1]
  safeFirst    = fetchOrders.output?.recentOrderIds?[0] ?? "none"
  regionLabel  = ctx.regionMap["north-east"]
  totalOrders  = fetchOrders.output.totalOrders
}

Common Trap — xs[i] vs xs?[i]

Both forms read element i, but they behave very differently when the list is shorter than expected:

  • xs[i] is strict. An out-of-range read fails fast with an expression-evaluation error. Use it when an empty or short list is a real bug — for example, when fetching an order you just created.
  • xs?[i] is lenient. An out-of-range read yields null, which pairs with ?? to provide a default. Use it when "no data yet" is a normal outcome — for example, when reading the latest entry from an optional history list.

Reach for the strict form by default; switch to the safe form only when you have a concrete reason to tolerate missing data.

Follow One Order Value Through Four Shapes​

Expression syntax becomes easier when you stop reading it token by token and follow one business object. The order begins as identifiers in GraphContext, gains facts from two Operators, becomes the input to pricing, and ends as a smaller summary for the next step.

Diagram: one order value changing shape

Name the shape at every boundary​

BoundaryShape available thereWhy it exists
Graph entry{userId, productIds}Stable request facts supplied by the caller
After fetch{user} and {products}External capabilities have enriched the request
Pricing input{user, products}Input bindings assemble only what pricing needs
Summary transform{name, email, itemCount, total}A cheap projection for downstream readers

This is data lineage in miniature. When customerEmail is wrong, walk left: the transform reads fetchUser.output.email, so inspect the user output before debugging the price Operator. The graph has already told you which producer owns the fact.

Choose by effect, not by line count​

Use an expression or transform when the operation is a deterministic reshape of values already present. Use an Operator when the step crosses into the outside world, consumes time, can fail independently, or needs retry and timeout policy. A ten-line projection is still a transform; a one-line HTTP call is still an effectful Operator.

The boundary is testable: no Operator invocation should appear for orderSummary, while fetchUser and fetchProducts must have invocation evidence. That observation is more useful than calling transforms "zero cost" without explaining what the engine did not schedule.

Break It Apart​

Transforms — projections without Operator scheduling​

A transform is a virtual node. It reshapes data without running an operator:

// From ch03/order-process.bloge
transform orderSummary {
  customerName  = fetchUser.output.name
  customerEmail = fetchUser.output.email
  itemCount     = fetchProducts.output.items.size
  total         = calcPrice.output.total
}

The engine does not schedule a transform on a thread. It resolves the bindings inline and makes the result available as orderSummary.customerName, orderSummary.itemCount, etc.

A minimal transform from the conformance suite (basic.bloge):

graph g {
  node source : Op {}
  transform summary {
    greeting: String = "hello"
    result: String   = source.output.value
  }
}

Notice that transforms can include type annotations (greeting: String).

When to use a transform vs. an operator:

Use a transform when …Use an operator when …
You only rename or pick fieldsYou call an external service
No side effectsYou run business rules that matter
No need for timeout/retryYou need resilience policies
Pure data reshapingYou need independent observability

Lambda expressions in input bindings​

For collection operations, BLOGE supports lambda expressions directly in input bindings. These are evaluated at input-assembly time with zero scheduling overhead.

From lambda-map.bloge:

node a : OpA {
  input {
    names = ctx.items.map(x -> x.name)
  }
}

The expression ctx.items.map(x -> x.name) takes a list of items from the context and extracts the name field from each one.

A richer example from order-enrichment-lambda.bloge:

node enrichOrders : EnrichOrdersOperator {
  depends_on = [fetchOrders, fetchProducts]
  input {
    enrichedOrders = fetchOrders.output.orders
      .map(o -> {
        orderId:     o.orderId,
        totalValue:  o.price * o.quantity,
        taxAmount:   o.price * o.quantity * 0.1,
        productName: fetchProducts.output.catalogue.associate(p -> p.id, p -> p.name)
      })
      .filter(o -> o.totalValue > ctx.minValue)
      .sortBy(o -> o.totalValue)

    totalRevenue = fetchOrders.output.orders
      .filter(o -> o.price * o.quantity > ctx.minValue)
      .reduce(0, (acc, o) -> acc + o.price * o.quantity)
  }
}

Available collection operations:

OperationSyntaxResult
map.map(x -> expr)New list with each element transformed
filter.filter(x -> condition)List with only matching elements
reduce.reduce(init, (acc, x) -> expr)Single accumulated value
sortBy.sortBy(x -> key)Sorted list

Lambda expressions can produce object literals ({ field: value, … }) and be chained (.map(…).filter(…).sortBy(…)).

Common Trap​

❌ Using an operator node for pure data reshaping​

// WRONG — this is a transform, not business logic
node formatOutput : OutputFormatterOperator {
  input {
    customerName = fetchUser.output.name
    total        = calcPrice.output.total
  }
}

If OutputFormatterOperator just picks fields and renames them, it's paying the cost of a full node (scheduling, timeout tracking) for something a transform does for free:

// CORRECT — projection without an Operator invocation
transform formatOutput {
  customerName = fetchUser.output.name
  total        = calcPrice.output.total
}

Rule of thumb: If there are no side effects, no I/O, and no business rules, use a transform.

What Goes Wrong​

Null path access without safe navigation​

If you write:

input {
  city = fetchUser.output.address.city
}

and address is null at runtime, the node fails while evaluating the binding because the plain .city access tries to walk through a null segment. BLOGE only short-circuits this lookup when you use ?..

Fix: Use safe navigation to handle missing data:

input {
  city = fetchUser.output.address?.city ?? "Unknown"
}

With ?., a null address produces null instead of an error. The ?? operator then substitutes the default.

Guided Rewrite​

Start from this graph that uses too many operator nodes:

graph report {
  node fetchSales   : FetchSalesOperator   { input { region = ctx.region } }
  node fetchReturns : FetchReturnsOperator { input { region = ctx.region } }

  // This node just picks fields — wasteful as an operator
  node buildSummary : SummaryBuilderOperator {
    input {
      totalSales   = fetchSales.output.total
      totalReturns = fetchReturns.output.total
    }
  }
}

Rewrite buildSummary as a transform:

graph report {
  node fetchSales   : FetchSalesOperator   { input { region = ctx.region } }
  node fetchReturns : FetchReturnsOperator { input { region = ctx.region } }

  transform buildSummary {
    totalSales   = fetchSales.output.total
    totalReturns = fetchReturns.output.total
    netRevenue   = fetchSales.output.total
  }
}

Now buildSummary adds no Operator invocation or separately scheduled node. If you need a computed field like netRevenue = totalSales - totalReturns, that can be expressed as an arithmetic expression in the binding — no operator required.

Brain Check​

  1. Name the three data sources available in an input binding. (Context values via ctx.*, upstream node outputs via node.output.*, and literal values.)
  2. What does ?. do in ctx.user?.profile?.name? (If user or profile is null, the entire expression returns null instead of throwing a null-pointer error.)
  3. When should you use a transform instead of an operator node? (When you only need to reshape or rename data without side effects, I/O, or business rules.)
  4. What is the scheduling cost of a lambda expression in an input binding? (Zero — it is evaluated at input-assembly time, not scheduled as a separate node.)
  5. Design question: You have a transform with six fields and an operator node that uses three of those fields. Should the operator read from the transform or from the original upstream outputs? (Either works, but reading from the transform is cleaner if those fields are already projected and named there. The general rule: minimise the number of distinct upstream references in a single input block.)

Lab​

  1. Open order-enrichment-lambda.bloge.
  2. Add a new input binding that computes the average order value using reduce to sum totals and then divides by the list size.
  3. Add a transform node that combines the totalRevenue with a ctx.budgetTarget to produce a percentOfBudget field.
  4. Verify mentally: which parts add no separately scheduled Operator (transform + lambda), and which would require one?

Experiment acceptance card​

  • Expected and observed: An order value crosses context, output, binding, and transform shapes.
  • Failure and recovery: Point a strict path at a missing field; restore the path or use an explicit safe read.
  • Proof boundary: Proves binding and path behavior, not truthful or complete input.
  • Exercise contract: Order sample; change one field path; deliver the value and failure reason; stop when both are attributable.

Recap​

  • Input bindings wire data from context, upstream outputs, and literals into node inputs — evaluated at assembly time with no scheduling overhead.
  • Path expressions navigate nested structures; ?. provides safe navigation and ?? provides null coalescing.
  • Transforms are virtual nodes that reshape data without an Operator invocation or separately scheduled task; expression evaluation still does work.
  • Lambda expressions (map, filter, reduce, sortBy) operate on collections directly in input bindings with no scheduling overhead.
  • Choose the right layer: input bindings for routing, transforms for projection, operator nodes for business logic.

Next Step​

In Chapter 5 — Branches That Decide you will learn how to add conditional control flow to your graphs — routing execution down different paths based on a node's output.

Coding Agent: Open the versioned task guide.