Skip to main content

Quickstart: Produce with kcat

This quickstart walks through sending a record to Rafka using kcat — a standard Kafka CLI client that speaks the full Kafka wire protocol. Rafka's gateway accepts these connections on TCP port 9092 with no special configuration.

:::info Verified behavior The produce vertical is fully built and Jaeger-verified in Rafka v2. This quickstart reflects actual live behavior, not a preview. :::

Prerequisites​

  • kcat installed (brew install kcat / apt install kcat / download from edenhill/kcat)
  • Rafka running via the dev compose stack (see below)
  • Optional: Jaeger UI at http://localhost:16686 to observe the trace

Start Rafka​

# From E:/dev/rafka-v2
podman compose -f deployment/dev/compose.yml up -d

This starts:

  • rafka-gateway — TCP listener on :9092, iroh mesh endpoint
  • rafka-broker — receives KafkaOp RPCs from the gateway, stores records
  • jaeger — OTLP trace collector + UI at http://localhost:16686

Wait until both services log their rafka.mesh.boot.* spans (typically under 2 seconds on a LAN).

Send a record​

echo "hello rafka" | kcat -P \
  -b localhost:9092 \
  -t my-first-topic \
  -p 0

kcat will:

  1. Send an ApiVersions request — Rafka responds with its supported API versions.
  2. Send a Metadata request for the topic — Rafka self-advertises as the single broker.
  3. Send a ProduceRequest — Rafka routes it over the iroh mesh to the broker.
  4. Receive a ProduceResponse with error_code: 0 (success).

No output from kcat on success is normal (-P producer mode exits 0 on success).

Observe the trace in Jaeger​

Open http://localhost:16686, select service mesh1.gateway, and click Find Traces. You will see a trace named gateway.kafka.request with child spans:

gateway.kafka.connect
└── gateway.kafka.request
    └── mesh.kafkaop.sent
        └── mesh.kafkaop.handle
            └── broker.produce.store
                └── kafkaop.ack

The full round-trip from TCP accept to broker storage is one correlated trace across two processes. This is Rafka's observability-first design: every hop emits an OpenTelemetry span with W3C trace context propagated in the mesh frame header.

Produce multiple records​

# Pipe three records, one per line
printf "record-1\nrecord-2\nrecord-3\n" | kcat -P \
  -b localhost:9092 \
  -t my-first-topic \
  -p 0

Each ProduceRequest generates a separate correlated trace.

Current limitations​

LimitationWhen it's resolved
Single-tenant — org_id fixed = 1Phase 2: id/slug/RRL multi-tenancy

Records are written to the durable WAL (SingleWal) and survive broker restart. See WAL Durability.

Next steps​