We've serialized structs to JSON and MessagePack using the msgpack-javascript library, but there are many other possible choices like protocol buffers or Avro.
While choosing which serialization format to use is important, it's also important to be careful about the shape or "schema" of the data you're serializing. As a general rule, if you make breaking changes to a schema, make sure you handle backward compatibility.
Let's say we have a User struct that we send around in our Pub/Sub system:
type User struct {
ID int
Name string
}
It's usually okay to just add and remove fields willy-nilly:
type User struct {
ID int
Name string
Email string
}
// or
type User struct {
ID int
}
However, if you change a field, you need to be careful. Say we want to make this update:
type User struct {
ID string // change to string
Name string
}
If there are old messages in a queue with the int IDs and we push this change, our new consumers will fail to decode the old messages over and over, resulting in a lot of errors and discarded messages (or retry loops). I have a simple rule:
If you make a breaking change to a schema, use a new routing-key/queue. That way, the old consumers can polish off all the old messages, and the new consumers can start fresh with the new schema.
In some languages, like JavaScript, you also have to be careful about removing fields because it can result in undefined errors if the client isn't coded in a robust way. In Go, it's usually safer because it defaults to the zero value.