Protobuf Code Generation with protoc
On this page
From .proto schema to generated code
A .proto file does nothing on its own. A compiler parses it, and plugins generate code for each language: message types with serializers, a client with one method per RPC, and a server interface to implement.
Generating code with protoc
protoc is the reference compiler. Each language needs its plugin installed (for Go, protoc-gen-go for messages and protoc-gen-go-grpc for services):
protoc \
--proto_path=proto \
--go_out=gen --go_opt=paths=source_relative \
--go-grpc_out=gen --go-grpc_opt=paths=source_relative \
proto/shop/v1/orders.protoIt works, but flags multiply with each language, include paths for imports are fiddly, and every developer needs the same plugin versions.
Generating code with buf
buf (from Buf Technologies) wraps compilation, linting, breaking-change detection and generation in one tool with config files. The course's lab uses this setup:
# buf.yaml: where the protos are, and which rules apply
version: v2
modules:
- path: proto
lint:
use:
- STANDARD
breaking:
use:
- FILE# buf.gen.yaml: what to generate
version: v2
plugins:
- local: protoc-gen-go
out: gen
opt: paths=source_relative
- local: protoc-gen-go-grpc
out: gen
opt: paths=source_relativebuf lint && buf generate && find gen -name '*.go'gen/shop/v1/orders.pb.go
gen/shop/v1/orders_grpc.pb.goorders.pb.go holds the message types; orders_grpc.pb.go holds OrderServiceClient, OrderServiceServer and registration functions. Versions in the lab: buf 1.73, protoc-gen-go 1.36.12, protoc-gen-go-grpc 1.6.2, grpc-go 1.84.
Remote plugins (remote: buf.build/protocolbuffers/go) run generation on Buf's servers, so nobody installs plugins locally. The Buf Schema Registry can also host shared protos and generated SDKs.
Using the generated Go code
The server embeds a generated base type, which returns Unimplemented for any method you haven't written yet, so adding an RPC to the proto doesn't break the build:
type server struct {
shopv1.UnimplementedOrderServiceServer
// your fields
}
func (s *server) GetOrder(ctx context.Context, req *shopv1.GetOrderRequest) (*shopv1.GetOrderResponse, error) {
// ...
}The client is created from a connection (Day 6):
client := shopv1.NewOrderServiceClient(conn)Managing generated code: commit it or generate in CI
- Commit it or generate it in CI? Committing generated code makes builds simple and reviews show exactly what changed; generating in CI keeps the repo smaller. Either way, CI must fail if the generated code is stale (regenerate and check
git diff --exit-code). - Pin plugin versions in CI and a tools file, so everyone generates identical code.
- One proto repository or per-service protos? Many organizations keep all protos in one place (or the Buf Schema Registry) and publish generated packages per language, so consumers depend on a versioned package rather than copying files.
- Never edit generated files; changes are lost on the next generation.
Check yourself
The .pb.go file has message types and serialization; the _grpc.pb.go file has the client, the server interface and registration helpers.
New RPCs added to the proto get a default Unimplemented implementation, so the server still compiles until you implement them.
Regenerate in CI and fail the build if the result differs from what's committed (or always generate during the build).
A config file that describes generation instead of long protoc command lines, linting against style rules, breaking change detection against a previous version, and dependency management for shared protos. Together they make schema work repeatable in CI.
Either works. Committing makes builds simpler and diffs visible; generating in CI keeps the repo clean. Pick one and enforce it.