Skip to main content

internal/shadowdiff/diff.go

Source

Overview

What:

Implements the core shadow diff runner.

The runner:

  • replays fixtures to two base URLs (Node and Go)
  • applies optional normalizers to both response bodies
  • compares status codes and bodies
  • returns a structured per-fixture result with latency measurements

Why:

During migrations and refactors, regressions are often subtle:

  • status codes may differ for edge cases
  • JSON bodies may differ only in ordering or volatile fields
  • latency can regress silently

This runner gives a deterministic mechanism to validate behavioral compatibility between two gateway implementations.

How:

  • (*Runner).Run executes the fixture list with bounded concurrency and returns results in the same order as the input.
  • Each fixture is executed by execute, which sends one request to the Node base URL and one to the Go base URL.
  • Bodies are read and normalized, then compared.
  • When a mismatch is detected, diffJSON produces a stable, human-readable diff string.

Notes:

The runner compares only status codes and response bodies. It does not currently diff headers. If you need header comparisons, extend Result and execute accordingly.

Contents

Imports

import block 1

internal/shadowdiff/diff.go#L3
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"sync"
"time"
)

Types

type block 1

internal/shadowdiff/diff.go#L16
type Result struct {
Fixture Fixture
NodeStatus int
GoStatus int
BodyDiff string
LatencyNode time.Duration
LatencyGo time.Duration
Err error
}

Result

What: Result of replaying one fixture against both gateway implementations.

Why: Bundles all useful debugging signals (statuses, diffs, latency, errors) into a single value.

How: Populated by execute and returned to the caller as part of the results slice.

Notes: The BodyDiff field is empty when bodies match after normalization.

type block 2

internal/shadowdiff/diff.go#L27
type Runner struct {
Client *http.Client
Config Config
Normalizers []func([]byte) []byte
}

Runner

What: Configurable runner that executes fixtures and compares responses.

Why: Allows the CLI and tests to provide a custom HTTP client and normalization strategy.

How: Holds an optional *http.Client, the Config (base URLs and concurrency), and a list of normalizers applied to both response bodies.

Notes: Normalizers should be deterministic and safe to run repeatedly.

Functions and Methods

(*Runner).Run

What: Executes all fixtures in a session and returns their results.

Why: Provides the batch entrypoint for the CLI and scripts, while keeping results stable and structured.

How: Ensures an HTTP client exists, uses a buffered channel as a semaphore to enforce Config.Concurrency, spawns one goroutine per fixture, and writes each result into the correct slot in the output slice.

Notes: Results preserve fixture order even though execution happens concurrently.

internal/shadowdiff/diff.go#L34
func (r *Runner) Run(ctx context.Context, fixtures []Fixture) []Result {
client := r.Client
if client == nil {
client = &http.Client{Timeout: 5 * time.Second}
}

results := make([]Result, len(fixtures))
sem := make(chan struct{}, r.Config.Concurrency)
wg := sync.WaitGroup{}

for i, fixture := range fixtures {
sem <- struct{}{}
wg.Add(1)
go func(idx int, fx Fixture) {
defer wg.Done()
defer func() { <-sem }()
results[idx] = r.execute(ctx, client, fx)
}(i, fixture)
}

wg.Wait()
return results
}

Walkthrough

The list below documents the statements inside the function body, including nested blocks and inline closures.

  • L35: client := r.Client
    • What: Defines client.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L36: if client == nil { client = &http.Client{Timeout: 5 * time.Second} }
    • What: Branches conditionally.
    • Why: Handles different execution paths based on runtime state.
    • How: Evaluates the condition and executes the matching branch.
    • Nested steps:
      • L37: client = &http.Client{Timeout: 5 * time.Second}
        • What: Assigns client.
        • Why: Keeps intermediate state available for later steps in the function.
        • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L40: results := make([]Result, len(fixtures))
    • What: Defines results.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L41: sem := make(chan struct{}, r.Config.Concurrency)
    • What: Defines sem.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L42: wg := sync.WaitGroup{}
    • What: Defines wg.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L44: for i, fixture := range fixtures { sem <- struct{}{} wg.Add(1) go func(idx int, fx Fixture) { defer wg.Done() defer func() { <-sem }() resu…
    • What: Iterates over a collection.
    • Why: Processes multiple elements with the same logic.
    • How: Executes a for ... range loop.
    • Nested steps:
      • L45: sem <- struct{}{}
        • What: Sends a value on a channel.
        • Why: Communicates with another goroutine.
        • How: Executes a channel send operation.
      • L46: wg.Add(1)
        • What: Calls wg.Add.
        • Why: Performs side effects or delegates work to a helper.
        • How: Executes the expression statement.
      • L47: go func(idx int, fx Fixture) { defer wg.Done() defer func() { <-sem }() results[idx] = r.execute(ctx, client, fx) }(i, fixture)
        • What: Starts a goroutine.
        • Why: Runs work concurrently.
        • How: Invokes the function call asynchronously using go.
        • Nested steps:
          • L47: func(idx int, fx Fixture) { defer wg.Done() defer func() { <-sem }() results[idx] = r.execute(ctx, client, fx) }
            • What: Defines an inline function (closure).
            • Why: Encapsulates callback logic and may capture variables from the surrounding scope.
            • How: Declares a func literal and uses it as a value (for example, as an HTTP handler or callback).
            • Nested steps:
              • L48: defer wg.Done()
                • What: Defers a call for cleanup.
                • Why: Ensures the deferred action runs even on early returns.
                • How: Schedules the call to run when the surrounding function returns.
              • L49: defer func() { <-sem }()
                • What: Defers a call for cleanup.
                • Why: Ensures the deferred action runs even on early returns.
                • How: Schedules the call to run when the surrounding function returns.
                • Nested steps:
                  • L49: func() { <-sem }
                    • What: Defines an inline function (closure).
                    • Why: Encapsulates callback logic and may capture variables from the surrounding scope.
                    • How: Declares a func literal and uses it as a value (for example, as an HTTP handler or callback).
                    • Nested steps:
                      • L49: <-sem
                        • What: Evaluates an expression.
                        • Why: Performs side effects or delegates work to a helper.
                        • How: Executes the expression statement.
              • L50: results[idx] = r.execute(ctx, client, fx)
                • What: Assigns results[idx].
                • Why: Keeps intermediate state available for later steps in the function.
                • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L54: wg.Wait()
    • What: Calls wg.Wait.
    • Why: Performs side effects or delegates work to a helper.
    • How: Executes the expression statement.
  • L55: return results
    • What: Returns from the current function.
    • Why: Ends the current execution path and hands control back to the caller.
    • How: Executes a return statement (possibly returning values).

(*Runner).execute

What: Replays a single fixture against both Node and Go base URLs and compares the outcomes.

Why: Separates per-fixture logic from the batch concurrency logic in Run.

How: Sends one request to each base URL, captures status and latency, reads and normalizes response bodies, compares status and body bytes, and computes a JSON diff when mismatched.

Notes: The requests are performed sequentially per fixture in the current implementation.

internal/shadowdiff/diff.go#L58
func (r *Runner) execute(ctx context.Context, client *http.Client, fixture Fixture) Result {
res := Result{Fixture: fixture}

nodeResp, nodeLatency, nodeErr := r.send(ctx, client, r.Config.NodeBaseURL, fixture)
goResp, goLatency, goErr := r.send(ctx, client, r.Config.GoBaseURL, fixture)

res.LatencyNode = nodeLatency
res.LatencyGo = goLatency

if nodeErr != nil {
res.Err = fmt.Errorf("node request failed: %w", nodeErr)
return res
}
if goErr != nil {
res.Err = fmt.Errorf("go request failed: %w", goErr)
return res
}

res.NodeStatus = nodeResp.StatusCode
res.GoStatus = goResp.StatusCode

nodeBody, _ := io.ReadAll(nodeResp.Body)
goBody, _ := io.ReadAll(goResp.Body)
nodeResp.Body.Close()
goResp.Body.Close()

for _, normalizer := range r.Normalizers {
nodeBody = normalizer(nodeBody)
goBody = normalizer(goBody)
}

if !bytes.Equal(nodeBody, goBody) || res.NodeStatus != res.GoStatus {
res.BodyDiff = diffJSON(nodeBody, goBody)
}
return res
}

Walkthrough

The list below documents the statements inside the function body, including nested blocks and inline closures.

  • L59: res := Result{Fixture: fixture}
    • What: Defines res.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L61: nodeResp, nodeLatency, nodeErr := r.send(ctx, client, r.Config.NodeBaseURL, fixture)
    • What: Defines nodeResp, nodeLatency, nodeErr.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L62: goResp, goLatency, goErr := r.send(ctx, client, r.Config.GoBaseURL, fixture)
    • What: Defines goResp, goLatency, goErr.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L64: res.LatencyNode = nodeLatency
    • What: Assigns res.LatencyNode.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L65: res.LatencyGo = goLatency
    • What: Assigns res.LatencyGo.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L67: if nodeErr != nil { res.Err = fmt.Errorf("node request failed: %w", nodeErr) return res }
    • What: Branches conditionally.
    • Why: Short-circuits early when a precondition is not met or an error/edge case is detected.
    • How: Evaluates the condition and executes the matching branch.
    • Nested steps:
      • L68: res.Err = fmt.Errorf("node request failed: %w", nodeErr)
        • What: Assigns res.Err.
        • Why: Keeps intermediate state available for later steps in the function.
        • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
      • L69: return res
        • What: Returns from the current function.
        • Why: Ends the current execution path and hands control back to the caller.
        • How: Executes a return statement (possibly returning values).
  • L71: if goErr != nil { res.Err = fmt.Errorf("go request failed: %w", goErr) return res }
    • What: Branches conditionally.
    • Why: Short-circuits early when a precondition is not met or an error/edge case is detected.
    • How: Evaluates the condition and executes the matching branch.
    • Nested steps:
      • L72: res.Err = fmt.Errorf("go request failed: %w", goErr)
        • What: Assigns res.Err.
        • Why: Keeps intermediate state available for later steps in the function.
        • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
      • L73: return res
        • What: Returns from the current function.
        • Why: Ends the current execution path and hands control back to the caller.
        • How: Executes a return statement (possibly returning values).
  • L76: res.NodeStatus = nodeResp.StatusCode
    • What: Assigns res.NodeStatus.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L77: res.GoStatus = goResp.StatusCode
    • What: Assigns res.GoStatus.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L79: nodeBody, _ := io.ReadAll(nodeResp.Body)
    • What: Defines nodeBody, _.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L80: goBody, _ := io.ReadAll(goResp.Body)
    • What: Defines goBody, _.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L81: nodeResp.Body.Close()
    • What: Calls nodeResp.Body.Close.
    • Why: Performs side effects or delegates work to a helper.
    • How: Executes the expression statement.
  • L82: goResp.Body.Close()
    • What: Calls goResp.Body.Close.
    • Why: Performs side effects or delegates work to a helper.
    • How: Executes the expression statement.
  • L84: for _, normalizer := range r.Normalizers { nodeBody = normalizer(nodeBody) goBody = normalizer(goBody) }
    • What: Iterates over a collection.
    • Why: Processes multiple elements with the same logic.
    • How: Executes a for ... range loop.
    • Nested steps:
      • L85: nodeBody = normalizer(nodeBody)
        • What: Assigns nodeBody.
        • Why: Keeps intermediate state available for later steps in the function.
        • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
      • L86: goBody = normalizer(goBody)
        • What: Assigns goBody.
        • Why: Keeps intermediate state available for later steps in the function.
        • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L89: if !bytes.Equal(nodeBody, goBody) || res.NodeStatus != res.GoStatus { res.BodyDiff = diffJSON(nodeBody, goBody) }
    • What: Branches conditionally.
    • Why: Handles different execution paths based on runtime state.
    • How: Evaluates the condition and executes the matching branch.
    • Nested steps:
      • L90: res.BodyDiff = diffJSON(nodeBody, goBody)
        • What: Assigns res.BodyDiff.
        • Why: Keeps intermediate state available for later steps in the function.
        • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L92: return res
    • What: Returns from the current function.
    • Why: Ends the current execution path and hands control back to the caller.
    • How: Executes a return statement (possibly returning values).

(*Runner).send

What: Sends a fixture request to a specific base URL and returns the HTTP response plus latency.

Why: Centralizes request construction, header application, and latency measurement for both Node and Go targets.

How: Joins base URL with the fixture path, constructs a request with context, copies fixture headers, executes the request, and returns the response and elapsed time.

Notes: The response body must be closed by the caller.

internal/shadowdiff/diff.go#L95
func (r *Runner) send(ctx context.Context, client *http.Client, baseURL string, fixture Fixture) (*http.Response, time.Duration, error) {
method := fixture.Method
if method == "" {
method = http.MethodGet
}

target, err := url.JoinPath(baseURL, fixture.Path)
if err != nil {
return nil, 0, fmt.Errorf("build url: %w", err)
}

reqBody := bytes.NewReader(fixture.Body)
req, err := http.NewRequestWithContext(ctx, method, target, reqBody)
if err != nil {
return nil, 0, err
}

for key, value := range fixture.Headers {
req.Header.Set(key, value)
}

start := time.Now()
resp, err := client.Do(req)
latency := time.Since(start)

if err != nil {
return nil, latency, err
}

return resp, latency, nil
}

Walkthrough

The list below documents the statements inside the function body, including nested blocks and inline closures.

  • L96: method := fixture.Method
    • What: Defines method.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L97: if method == "" { method = http.MethodGet }
    • What: Branches conditionally.
    • Why: Handles different execution paths based on runtime state.
    • How: Evaluates the condition and executes the matching branch.
    • Nested steps:
      • L98: method = http.MethodGet
        • What: Assigns method.
        • Why: Keeps intermediate state available for later steps in the function.
        • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L101: target, err := url.JoinPath(baseURL, fixture.Path)
    • What: Defines target, err.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L102: if err != nil { return nil, 0, fmt.Errorf("build url: %w", err) }
    • What: Branches conditionally.
    • Why: Short-circuits early when a precondition is not met or an error/edge case is detected.
    • How: Evaluates the condition and executes the matching branch.
    • Nested steps:
      • L103: return nil, 0, fmt.Errorf("build url: %w", err)
        • What: Returns from the current function.
        • Why: Ends the current execution path and hands control back to the caller.
        • How: Executes a return statement (possibly returning values).
  • L106: reqBody := bytes.NewReader(fixture.Body)
    • What: Defines reqBody.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L107: req, err := http.NewRequestWithContext(ctx, method, target, reqBody)
    • What: Defines req, err.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L108: if err != nil { return nil, 0, err }
    • What: Branches conditionally.
    • Why: Short-circuits early when a precondition is not met or an error/edge case is detected.
    • How: Evaluates the condition and executes the matching branch.
    • Nested steps:
      • L109: return nil, 0, err
        • What: Returns from the current function.
        • Why: Ends the current execution path and hands control back to the caller.
        • How: Executes a return statement (possibly returning values).
  • L112: for key, value := range fixture.Headers { req.Header.Set(key, value) }
    • What: Iterates over a collection.
    • Why: Processes multiple elements with the same logic.
    • How: Executes a for ... range loop.
    • Nested steps:
      • L113: req.Header.Set(key, value)
        • What: Calls req.Header.Set.
        • Why: Performs side effects or delegates work to a helper.
        • How: Executes the expression statement.
  • L116: start := time.Now()
    • What: Defines start.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L117: resp, err := client.Do(req)
    • What: Defines resp, err.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L118: latency := time.Since(start)
    • What: Defines latency.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L120: if err != nil { return nil, latency, err }
    • What: Branches conditionally.
    • Why: Short-circuits early when a precondition is not met or an error/edge case is detected.
    • How: Evaluates the condition and executes the matching branch.
    • Nested steps:
      • L121: return nil, latency, err
        • What: Returns from the current function.
        • Why: Ends the current execution path and hands control back to the caller.
        • How: Executes a return statement (possibly returning values).
  • L124: return resp, latency, nil
    • What: Returns from the current function.
    • Why: Ends the current execution path and hands control back to the caller.
    • How: Executes a return statement (possibly returning values).

diffJSON

What: Produces a stable text diff of two JSON payloads (or raw bytes when not JSON).

Why: Humans need a readable representation of a mismatch to diagnose what changed.

How: Attempts to unmarshal both payloads as JSON, re-encodes them with indentation for canonicalization, and returns a formatted "expected vs actual" string when they differ.

Notes: If either payload is not valid JSON, it falls back to printing raw bytes for both sides.

internal/shadowdiff/diff.go#L127
func diffJSON(expected, actual []byte) string {
var expAny, actAny interface{}
if err := json.Unmarshal(expected, &expAny); err != nil {
if bytes.Equal(expected, actual) {
return ""
}
return fmt.Sprintf("expected raw:\n%s\nactual:\n%s\n", expected, actual)
}
if err := json.Unmarshal(actual, &actAny); err != nil {
if bytes.Equal(expected, actual) {
return ""
}
return fmt.Sprintf("expected raw:\n%s\nactual:\n%s\n", expected, actual)
}

expCanonical, _ := json.MarshalIndent(expAny, "", " ")
actCanonical, _ := json.MarshalIndent(actAny, "", " ")

if bytes.Equal(expCanonical, actCanonical) {
return ""
}

return fmt.Sprintf("expected:\n%s\nactual:\n%s\n", expCanonical, actCanonical)
}

Walkthrough

The list below documents the statements inside the function body, including nested blocks and inline closures.

  • L128: var expAny, actAny interface{}
    • What: Declares local names.
    • Why: Introduces variables or types used later in the function.
    • How: Executes a Go declaration statement inside the function body.
  • L129: if err := json.Unmarshal(expected, &expAny); err != nil { if bytes.Equal(expected, actual) { return "" } return fmt.Sprintf("expected raw:\…
    • What: Branches conditionally.
    • Why: Short-circuits early when a precondition is not met or an error/edge case is detected.
    • How: Evaluates the condition and executes the matching branch.
    • Nested steps:
      • L129: err := json.Unmarshal(expected, &expAny)
        • What: Defines err.
        • Why: Keeps intermediate state available for later steps in the function.
        • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
      • L130: if bytes.Equal(expected, actual) { return "" }
        • What: Branches conditionally.
        • Why: Short-circuits early when a precondition is not met or an error/edge case is detected.
        • How: Evaluates the condition and executes the matching branch.
        • Nested steps:
          • L131: return ""
            • What: Returns from the current function.
            • Why: Ends the current execution path and hands control back to the caller.
            • How: Executes a return statement (possibly returning values).
      • L133: return fmt.Sprintf("expected raw:\n%s\nactual:\n%s\n", expected, actual)
        • What: Returns from the current function.
        • Why: Ends the current execution path and hands control back to the caller.
        • How: Executes a return statement (possibly returning values).
  • L135: if err := json.Unmarshal(actual, &actAny); err != nil { if bytes.Equal(expected, actual) { return "" } return fmt.Sprintf("expected raw:\n%…
    • What: Branches conditionally.
    • Why: Short-circuits early when a precondition is not met or an error/edge case is detected.
    • How: Evaluates the condition and executes the matching branch.
    • Nested steps:
      • L135: err := json.Unmarshal(actual, &actAny)
        • What: Defines err.
        • Why: Keeps intermediate state available for later steps in the function.
        • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
      • L136: if bytes.Equal(expected, actual) { return "" }
        • What: Branches conditionally.
        • Why: Short-circuits early when a precondition is not met or an error/edge case is detected.
        • How: Evaluates the condition and executes the matching branch.
        • Nested steps:
          • L137: return ""
            • What: Returns from the current function.
            • Why: Ends the current execution path and hands control back to the caller.
            • How: Executes a return statement (possibly returning values).
      • L139: return fmt.Sprintf("expected raw:\n%s\nactual:\n%s\n", expected, actual)
        • What: Returns from the current function.
        • Why: Ends the current execution path and hands control back to the caller.
        • How: Executes a return statement (possibly returning values).
  • L142: expCanonical, _ := json.MarshalIndent(expAny, "", " ")
    • What: Defines expCanonical, _.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L143: actCanonical, _ := json.MarshalIndent(actAny, "", " ")
    • What: Defines actCanonical, _.
    • Why: Keeps intermediate state available for later steps in the function.
    • How: Evaluates the right-hand side expressions and stores results in the left-hand variables.
  • L145: if bytes.Equal(expCanonical, actCanonical) { return "" }
    • What: Branches conditionally.
    • Why: Short-circuits early when a precondition is not met or an error/edge case is detected.
    • How: Evaluates the condition and executes the matching branch.
    • Nested steps:
      • L146: return ""
        • What: Returns from the current function.
        • Why: Ends the current execution path and hands control back to the caller.
        • How: Executes a return statement (possibly returning values).
  • L149: return fmt.Sprintf("expected:\n%s\nactual:\n%s\n", expCanonical, actCanonical)
    • What: Returns from the current function.
    • Why: Ends the current execution path and hands control back to the caller.
    • How: Executes a return statement (possibly returning values).

Guides

Neighboring source