This lesson's interactive features are locked, please to keep using them

Schema

We've serialized objects to JSON and MessagePack using the msgpack-javascript library, but there are many other possible choices like protocol buffers or Avro.

While choosing which serialization format to use is important, it's also important to be careful about the shape or "schema" of the data you're serializing. As a general rule, if you make breaking changes to a schema, make sure you handle backward compatibility.

Updating the Schema

Let's say we have a User interface that we send around in our Pub/Sub system:

interface User {
  id: number;
  name: string;
}

It's usually okay to just add and remove fields willy-nilly:

interface User {
  id: number;
  name: string;
  email: string;
}
// or
interface User {
  id: number;
}

However, if you change a field, you need to be careful. Say we want to make this update:

interface User {
  id: string; // change to string
  name: string;
}

If there are old messages in a queue with the number IDs and we push this change, our new consumers will fail to decode the old messages over and over, resulting in a lot of errors and discarded messages (or retry loops). I have a simple rule:

If you make a breaking change to a schema, use a new routing-key/queue. That way, the old consumers can polish off all the old messages, and the new consumers can start fresh with the new schema.

Note

In JavaScript/TypeScript, you also have to be careful about removing fields because it can result in undefined errors if the client isn't coded in a robust way.