Produce

The produce vertical is Rafka's first fully-built and Jaeger-verified Kafka op. A standard Kafka client (kcat, librdkafka, kafka-go) connects to the gateway on TCP :9092 and sends a ProduceRequest. Rafka routes it to the broker over the iroh mesh and returns a ProduceResponse — all standard Kafka wire, all observable in Jaeger.

The produce flow

Kafka client ──TCP :9092──▶ gateway (kafka_ingress.rs)
    ApiVersions negotiation
    ProduceRequest decode → ProduceOp (postcard-encoded)
    → kafka_op_call(broker_peer, KAFKA_OP_PRODUCE, org_id, payload)
         ▼  iroh QUIC bi-stream  (one frame: tag 0x02 + length + postcard body)
    broker (run_bi_reader dispatch → handle_produce)
    → topic store (in-memory map keyed by topic+partition)
    ◀ ProduceOpResp (correlated by request_id)
    → gateway encodes ProduceResponse ─────────▶ client

Every step emits an OTLP span. The trace chain is:

gateway.kafka.connect → gateway.kafka.request → mesh.kafkaop.sent → mesh.kafkaop.handle → broker.produce.store → kafkaop.ack → kafkaop.ack_received

The KafkaOp carrier

Rafka does not fork a v1 QUIC mesh stack. Kafka ops ride a correlated KafkaOp carrier on top of the existing iroh bi-stream substrate:

• KAFKA_OP_PRODUCE = 0x02 — the produce op code.
• ProduceOp / ProduceOpResp — postcard-encoded request and response types in crates/rafka-mesh-ops.
• kafka_op_call(endpoint, peer, op_code, org_id, payload) — opens an iroh bi-stream, writes InternalMeshFrame::KafkaOp, awaits the correlated response.
• register_kafka_handler(op_code, handler) — the broker's pluggable dispatch table. Adding a new Kafka op is a fixed recipe: new op code + codec in rafka-mesh-ops, new handler registered in broker/main.rs, new decoder in gateway/src/kafka_ingress.rs.

Kafka wire compatibility

Kafka API Key	Name	Supported Versions	Notes
18	ApiVersions	v0–v3	Handled inline at gateway; no mesh hop
0	Produce	v0–v8	Forwarded to broker via KafkaOp carrier
1	Fetch	v0–v11	See Fetch concepts
2	ListOffsets	v1–v5	min=v1 (v0 uses old-style offsets array)
3	Metadata	v0–v5	Self-advertise; gateway is the sole broker

See the centralized Kafka API & Wire Compatibility reference for the full version table and wire behavior notes.

Observability

Rafka's produce vertical emits OTLP spans at every boundary crossing. Attributes include:

• kafka.topic, kafka.partition — on the produce span
• mesh.op_code, mesh.request_id, mesh.org_id — on the kafkaop spans
• W3C traceparent is propagated in the 32-byte frame header so spans from gateway and broker join the same Jaeger trace automatically

See the quickstart to observe a live produce trace in Jaeger.

Current limitations

• Single-tenant — org_id is hardcoded to 1 in the POC. The full id/slug/RRL multi-tenancy model arrives in Phase 2.
• Records are persisted to the SingleWal durable WAL and survive broker restart. See WAL Durability.