Producing and resetting offsets

I'll handle the queues. You call the shots.

— Marshall

Reading never changes anything. Two things do, and both sit behind a sheet that spells out the consequence and asks why.

Producing a message

File a corrected copy on a message (or a new message) opens a form in three parts:

1. Where: the topic.
2. What: the key (optional) and the payload. For a corrected copy, the original is filled in so you fix only what is wrong.
3. Why: the reason, for example Reprocessing after the schema fix.

The sheet reminds you that this adds a real message to a real topic. File it sends it.

The reason travels with the message as a marshall-reason header, so whoever finds it later knows where it came from and why.

You cannot file a decoded Avro record back as a copy: it would put JSON on a topic every consumer reads as Avro. Marshall refuses and says so.

Resetting a group's offsets

Move the position… on a consumer group, for one of its topics:

1. Resume from: an offset, or The beginning. It applies to every partition of the topic.
2. Why: the reason.

If the offset is past the end of some partitions, Marshall names them before the button: Kafka would accept it, and each consumer's next read would be out of range.

Move it applies the change. This makes a real consumer group reprocess.

Refused while the group has members

Kafka's admin API lets you reset a group that still has members, but the members commit over the change within seconds. Marshall refuses it. Stop the consumers first; once nobody is consuming the group, nothing will fight you.