JSON vs Protobuf vs Avro: Size, Speed and Schema Changes

Two things go wrong as traffic grows, and they sit at the heart of the JSON vs Protobuf choice. Every message repeats its field names, so you pay for the same labels millions of times, and nothing stops a renamed field from silently breaking readers.

Picture a common setup. About 1,200 fishing boats each carry a GPS tracker that sends a position ping every 30 seconds over a metered mobile link, billed by the byte. Each ping goes out as one UDP packet, a single send with no connection set up first. The setup is made up, but the problems are real.

A ping is one small message: the boat's id, latitude, longitude, the time, speed and battery level. The harbour office and families on shore watch the pings on a live map. A data team also stores every ping for safety reports.

The first tracker firmware (the program inside the device) sends each ping as JSON text. Three problems show up in the first monsoon season.

• Data bills. Each SIM plan allows 10 MB (10 million bytes) a month. At 2,880 pings a day, JSON payloads (the message bytes themselves) come to about 9.5 MB in a 30-day month. Each packet also carries 28 bytes of IPv4 and UDP headers, the addressing and control data the network adds, which pushes the total near 12 MB.
• Lost pings. Near the edge of coverage the signal drops for seconds at a time. A longer message is on air for longer, so it has more time to be cut off.
• A broken report. New firmware renamed the field battery to battery_pct. The battery report quietly showed blanks for a week before anyone noticed.

The first two problems are about size. The third is about schema evolution: changing the shape of data while old devices and old readers keep running. Boats get new firmware only in harbour, so old and new versions run side by side for months.

You need four words: serialization, schema, binary encoding and field tag. Each gets a plain meaning and an example from the boats.

Serialization means turning an object in your program into bytes you can send or store. Deserialization turns the bytes back into an object. The tracker serializes a ping before sending it.

A schema is a written description of the data: which fields exist, their names and their types. For the ping, boat_id is text, lat is a decimal number and battery_pct is a whole number.

A binary encoding stores values as raw bytes instead of readable characters. The number 81 is two characters in text, but fits in one byte in binary.

A field tag is a small number that stands in for a field name inside the bytes. Protobuf can send the number 6 instead of the eleven letters of battery_pct.

The format decides how many bytes each message carries, and on a metered link you pay for every byte. It also decides how much the sender and receiver must agree on in advance.

Think of an old telegraph office that charges per word. The first way to send a message is to spell everything out: BOAT ID VSK 0412, LATITUDE 17.686815, and so on. Any clerk can read it, but you pay for every label.

The second way uses agreed short codes. Both offices keep the same numbered list, so you send 1 VSK0412, 2 17.686815. You pay far less, and a clerk who sees an unknown number can skip that item.

The third way sends only the values, in a fixed order, with no labels at all. That works only if the receiver holds the key that lists the order. So you either attach the key or send it once for a whole bundle.

Those three ways are JSON, Protobuf and Avro. Where the picture breaks: a clerk can see where an unknown item ends. A Protobuf reader cannot, so each tag carries a hint about the value's length, as you will see.

Now look at where a ping travels, and notice that two very different readers consume the same data.

Kafka is a distributed log: an ordered list of messages that many readers consume at their own pace. A data lake is a big store of raw files kept for later analysis. A schema registry, explained below, keeps every schema version.

JSON carries its field names as text inside every message, while Protobuf carries small numbers and relies on a shared schema file. Protobuf is smaller and stricter; JSON is readable and needs no setup.

Here is the first JSON ping, written compactly with no spaces.

{"boat_id":"VSK-0412","lat":17.686815,"lon":83.218482,"sent_at":1792040400,"speed_knots":6.5,"battery_pct":81}

That is 110 bytes. The quoted field names take 54 bytes, and braces, colons and commas take 13 more. So 61 percent of the message is labels and punctuation, not data.

JSON, short for JavaScript Object Notation, is defined by a short standard, RFC 8259. It covers only syntax: objects, arrays, strings, numbers, true, false and null. It has no schema built in, so nothing stops a sender from renaming a field.

Protobuf, short for Protocol Buffers, works differently. You first write a schema in a .proto file, where each field gets a name, a type and a field number. A tool called protoc then generates code that reads and writes these messages.

The same ping in Protobuf is 41 bytes. The names never travel. Each value starts with a tag, which packs the field number together with a wire type, a small code that tells the reader how to find where the value ends.

Whole numbers use varints (variable-length integers): values from 0 to 127 take one byte and bigger ones take more. The official encoding guide shows the classic case: the value 150 in field 1 becomes three bytes, 08 96 01.

Avro sends only the values, in the order the schema lists them, with no names and no tags. Strings carry a length first, whole numbers use a varint, and floats and doubles take 4 and 8 bytes. Apache Avro is an open-source data format whose schemas are written in JSON.

Pause and predict before reading on. Protobuf spent one tag byte on each of six fields. If Avro drops the tags, how big should the same ping be?

It is 36 bytes, not 35. Avro drops all six tag bytes, but battery 81 takes two bytes instead of one, because Avro stores every whole number in a signed form explained in the engineer section.

The Avro 1.12.0 specification is direct about the cost. Binary Avro data includes no type information or field names, so a reader cannot decode it without the writer's schema. In telegraph terms, the key travels separately.

So where does the schema go? Avro and its tools give you three common places.

• In a file header. An Avro object container file stores the writer's schema once at the top, then records in blocks. Our file of one boat's day (2,880 pings) averaged 36.2 bytes a ping.
• In a fingerprint. Avro's single-object encoding puts a two-byte marker and an eight-byte fingerprint (a short number computed from the schema) before each record, so one ping becomes 46 bytes.
• In a schema registry. A registry is a small service that stores every schema version under an id. In the documented wire format of Confluent Schema Registry, a widely used registry, each message its serializers write starts by default with one zero byte and a four-byte id. A newer opt-in mode moves the id into a message header instead, as of 19 October 2026.

With that five-byte prefix, one Avro ping is 41 bytes. Protobuf sent through the same registry gets the prefix too, plus a short list of message indexes. Avro wins when many records share one schema, as in a day's file.

Schema evolution is changing the shape of your data, such as adding, removing or renaming fields, while old writers and old readers keep working. Each format gives you different rules for doing it safely.

Two more words help. Backward compatible means new readers can read old data. Forward compatible means old readers can read new data. The boats need both, because trackers at sea run old firmware for months.

JSON gives freedom with no guard

JSON lets you send anything, and that is the problem. Adding a field is usually fine if readers ignore keys they do not know. Renaming a field breaks readers silently, which is exactly what happened to the battery report.

You can add checks with JSON Schema, a separate specification for describing JSON. Nothing forces a sender to use it, so the discipline lives in your team and your tests.

Protobuf skips what it does not know

A Protobuf reader reads one field at a time and skips any field number it does not know. The wire type in each tag says how long the value is, so the reader can jump over it safely.

Suppose new firmware adds field 7, an sos flag (a distress button) the crew can press. A server still running the old schema meets field 7 and skips it. Follow the arrows to see how.

The other direction works too. A new server reading a ping from old firmware finds no field 7 and returns the default, which is false for a true-or-false field. The proto3 language guide states both rules.

Protobuf's other rules all protect the field numbers. The guide says changing a field's number is the same as deleting the field and adding a new one.

• Add a field with a brand new number, and decide what its default value means for old data.
• Rename a field freely in the binary format, since names are not sent. Protobuf's JSON form does use names, so take care there.
• Delete a field by marking its number and name as reserved, so nobody reuses them. Old pings may still contain that number.
• Change a type only between pairs the guide lists as compatible, such as int32 and int64, and roll out with care.

Avro matches by name and fills gaps with defaults

Avro reads with two schemas at once. The writer schema is the one used to write the bytes. The reader schema is the one your code expects now. Avro's resolution rules translate between them.

The specification matches fields by name, not position. A field the writer had but the reader dropped is ignored. A field the reader wants but the writer never had takes the reader's default; with no default, the read fails.

Follow the arrows below. A report job reads an old ping, written before the sos field existed, using the new schema.

Renames work differently in Avro. Because fields match by name, a rename needs an alias: the reader's schema lists the old name, so old data still maps to the new field. Without the alias, our test failed the read.

Use JSON when people or browsers read the data, and Protobuf when services or devices exchange messages. Use Avro when you store or stream large volumes that many teams read for years. Many systems use all three.

That is where the boat system ends up. The trackers move to Protobuf, and payloads drop to about 3.5 MB per boat per month. With the 28 header bytes per packet, the total is about 6 MB, under the 10 MB plan.

The ingest server, which receives every ping, turns each Protobuf ping into Avro for Kafka and the data lake. Every schema version lives in a registry set to reject changes that break compatibility. The live map still gets JSON, because browsers read it natively.

Plain JSON is still the right call when the message count is small, outside developers consume the data, or the shape changes weekly and nobody owns schema files yet.

Beginners can stop here. The rest of the guide is for engineers: commands to check every number above, a decode benchmark, and five traps we reproduced on pinned versions.

Run the three files below with the pinned versions, and you should see the same byte counts. We ran them on Python 3.13.16 with protobuf 7.36.2, grpcio-tools 1.84.0 (which bundles protoc 35.1) and fastavro 1.13.1.

First, the Protobuf schema. Save it as ping.proto.

syntax = "proto3";

message Ping {
  string boat_id = 1;
  double lat = 2;
  double lon = 3;
  int64 sent_at = 4;
  float speed_knots = 5;
  int32 battery_pct = 6;
}

Next, sizes.py encodes the same ping three ways and prints the sizes. It also prints the Protobuf bytes in hex (base 16, two digits per byte), so you can match them to the table above.

import io
import json

import fastavro
from ping_pb2 import Ping  # made by protoc from ping.proto

ping = {"boat_id": "VSK-0412", "lat": 17.686815, "lon": 83.218482,
        "sent_at": 1792040400, "speed_knots": 6.5, "battery_pct": 81}

as_json = json.dumps(ping, separators=(",", ":")).encode("utf-8")
as_proto = Ping(**ping).SerializeToString()

schema = fastavro.parse_schema({"type": "record", "name": "Ping", "fields": [
    {"name": "boat_id", "type": "string"},
    {"name": "lat", "type": "double"},
    {"name": "lon", "type": "double"},
    {"name": "sent_at", "type": "long"},
    {"name": "speed_knots", "type": "float"},
    {"name": "battery_pct", "type": "int"}]})
buf = io.BytesIO()
fastavro.schemaless_writer(buf, schema, ping)
as_avro = buf.getvalue()

print("JSON", len(as_json), "bytes")
print("Protobuf", len(as_proto), "bytes")
print("Avro", len(as_avro), "bytes")
print(as_proto.hex(" "))

Then evolve.py writes a ping with the old two-field schema and reads it with the new schema, which adds sos with a default.

import io
import fastavro

v1 = fastavro.parse_schema({"type": "record", "name": "Ping", "fields": [
    {"name": "boat_id", "type": "string"},
    {"name": "battery_pct", "type": "int"}]})
v2 = fastavro.parse_schema({"type": "record", "name": "Ping", "fields": [
    {"name": "boat_id", "type": "string"},
    {"name": "battery_pct", "type": "int"},
    {"name": "sos", "type": "boolean", "default": False}]})

buf = io.BytesIO()
fastavro.schemaless_writer(buf, v1, {"boat_id": "VSK-0412", "battery_pct": 81})
buf.seek(0)
print(fastavro.schemaless_reader(buf, v1, v2))

Install the pinned versions, generate the Protobuf code and run both scripts. The comments show the output we got.

python3 -m venv venv && . venv/bin/activate
pip install protobuf==7.36.2 grpcio-tools==1.84.0 fastavro==1.13.1
python -m grpc_tools.protoc -I. --python_out=. ping.proto
python sizes.py
# JSON 110 bytes
# Protobuf 41 bytes
# Avro 36 bytes
# 0a 08 56 53 4b 2d 30 34 31 32 11 f9 66 9b ... 30 51
python evolve.py
# {'boat_id': 'VSK-0412', 'battery_pct': 81, 'sos': False}

What just happened? The Protobuf bytes start with 0a 08: tag for field 1, then a length of 8. The old Avro ping gained sos as false because v2 supplies a default.

Now break it on purpose. Delete the default from v2 and run evolve.py again: fastavro 1.13.1 raises SchemaResolutionError, saying there is no default value for field sos.

In our Python test, the library mattered more than the format. Protobuf decoded one ping about three times faster than json.loads, even counting reading every field. The official avro package was several times slower than JSON.

The usual reasoning is sound but incomplete. A JSON reader scans every character and turns text such as 17.686815 into a number, while a binary reader gets values whose size is fixed or marked in the bytes. How much that matters depends on where the code runs.

Python's protobuf package uses a C core called upb, and json.loads also has a C core. fastavro ships compiled extension modules. The official avro package 1.12.2 is plain Python, with no compiled parts.

A microsecond is a millionth of a second. Read the ratios, not the raw numbers, because your machine will differ. Note the second protobuf row: reading the six fields after parsing more than doubled the time, so a parse-only benchmark flatters Protobuf.

So binary does not mean fast in every language. In Java, Go or C++ the picture can change again. Benchmark your real payloads with your real library before using speed as the reason to switch.

Five details surprise people who already know the basics. We reproduced each one on protobuf 7.36.2, fastavro 1.13.1 and Node.js 22.22.0, as of 19 October 2026.

Negative numbers can cost ten bytes in Protobuf

A negative int32 or int64 value takes ten bytes, because Protobuf stores it in two's complement, the usual computer form for negatives, which sets the highest bits. In our test, int32 -1 made an 11-byte message, and sint32 -1 made a 2-byte one.

The sint32 and sint64 types use ZigZag encoding, which maps small negative numbers to small positive ones. Avro uses ZigZag for every int and long, so -1 is one byte, and 81 becomes 162, which needs two.

A zero may not be sent at all

In a proto3 file, a plain number, text or true-or-false field without the optional label, set to its default, such as battery_pct of 0, is left out of the bytes. A reader cannot tell a flat battery from a missing reading.

Mark the field optional to track presence; then 0 is sent and the HasField check reports it. Protobuf editions, the newer syntax that replaces the proto2 and proto3 labels, track presence by default. Our edition 2023 test sent the zero; the latest edition is 2024, as of 19 October 2026.

Protobuf's JSON form changes names and drops unknown fields

When you print a Protobuf message as JSON, field names become lowerCamelCase (no underscores, later words capitalised), so battery_pct turns into batteryPct. 64-bit integers become strings, so sent_at appears as "1792040400" in quotes.

Unknown fields are lost too. In our test, an old server converted a new ping to JSON and back, and the sos flag set to true came out false.

Big ids lose digits in JavaScript

RFC 8259 says whole numbers between -(2 to the power 53) + 1 and 2 to the power 53 - 1 are interoperable. Outside that range, readers may disagree.

In Node.js 22.22.0, JSON.parse turned the id 9007199254740993 into 9007199254740992. Python's json module kept it exact. Send large ids as strings in JSON.

Reusing a field number gives wrong data, not an error

The proto3 guide warns that reusing a field number can cause parse errors, leaked personal data or data corruption. Our test showed the quiet version of that.

We read an old ping, where field 6 meant battery, with a schema where field 6 meant heading_deg. The reader reported a heading of 81 degrees without complaint. When we redefined field 6 as text, Python's protobuf 7.36.2 showed an empty field with no error and kept the 81 only as an unknown field.

That last number is flattering, since our 2,880 pings were identical copies. Real pings differ, so compress them in batches and measure on real data.

Explain them as three points on one scale: how much each message describes itself, versus how much both sides must agree in advance. Then tie your choice to size, speed and change for each hop.

1. 1

   Start with the scale

   JSON carries names, Protobuf carries numbers, and Avro carries only values. Less in the message means more agreement needed beforehand.

2. 2

   Put numbers on size

   Do one estimate out loud. 1,200 devices sending 110-byte pings every 30 seconds is 4,400 payload bytes a second; at 41 bytes it is 1,640. Then add the headers each packet carries.

3. 3

   Talk about speed honestly

   Say decode speed depends on the library and language, and that you would benchmark real payloads before switching for speed alone.

4. 4

   Explain schema evolution

   Protobuf never reuses field numbers. Avro matches names and needs defaults. JSON has no rules unless your team adds them.

5. 5

   Show the mixed design

   Protobuf from devices, Avro in Kafka with a registry, JSON to the browser. Explain why each hop got its format.

6. 6

   Name the failure you prevent

   Mention one real risk, such as a renamed field breaking a report or a reused field number, and the rule that stops it.