Kafka Security: SASL, ACLs and TLS Encryption
On this page
Kafka security is off by default
Out of the box, a Kafka listener uses PLAINTEXT: no encryption, no authentication, no authorization. Anyone who can reach port 9092 can read customer data from any topic, write fake orders, or delete topics. Securing Kafka means three separate things:
| Concern | Question | Mechanism |
|---|---|---|
| Encryption | Can someone read traffic on the network? | TLS |
| Authentication | Who is this client? | SASL (SCRAM, OAUTHBEARER, GSSAPI) or mutual TLS |
| Authorization | What may this client do? | ACLs evaluated by an authorizer |
Encrypting Kafka traffic with TLS
Configure listeners with SSL or SASL_SSL instead of PLAINTEXT, for client traffic and for broker-to-broker replication. TLS has a CPU cost, which modern hardware handles well; it also prevents Kafka's zero-copy optimization for consumers, which slightly reduces read efficiency.
Kafka authentication: SASL/SCRAM, OAUTHBEARER and mTLS
Common choices:
- SASL/SCRAM-SHA-512: username and salted password, stored by Kafka. Simple and widely supported. Always over TLS.
- SASL/OAUTHBEARER: clients present OAuth tokens from your identity provider. Good for central identity management and short-lived credentials.
- Mutual TLS: each client has a certificate; the certificate's subject becomes the principal. Strong, but you need certificate issuance and rotation.
- Cloud IAM: managed services such as Amazon MSK and Azure Event Hubs authenticate with the cloud's identity system.
A listener for SASL over TLS:
listeners=SASL_SSL://0.0.0.0:9093
security.inter.broker.protocol=SASL_SSL
sasl.enabled.mechanisms=SCRAM-SHA-512
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-512
ssl.keystore.location=/etc/kafka/tls/broker.p12
authorizer.class.name=org.apache.kafka.metadata.authorizer.StandardAuthorizer
allow.everyone.if.no.acl.found=false
super.users=User:adminStandardAuthorizer is the KRaft-era authorizer. allow.everyone.if.no.acl.found=false makes access deny-by-default.
Clients connecting to that listener need matching settings:
consumer = Consumer({
"bootstrap.servers": "kafka:9093",
"security.protocol": "SASL_SSL",
"sasl.mechanism": "SCRAM-SHA-512",
"sasl.username": "billing-service",
"sasl.password": os.environ["KAFKA_PASSWORD"], # from a secret store, never in code
"ssl.ca.location": "/etc/kafka/tls/ca.pem", # trust the cluster's certificate authority
"group.id": "billing",
})In .NET's Confluent.Kafka, the same settings are SecurityProtocol = SecurityProtocol.SaslSsl, SaslMechanism = SaslMechanism.ScramSha512, SaslUsername, SaslPassword and SslCaLocation. A client still using PLAINTEXT against this listener fails with a connection or handshake error rather than an authorization error, which is the first thing to check when a newly secured cluster "stops working".
Kafka ACLs for least-privilege authorization
Each service gets its own principal and only the permissions it needs:
# the order service may write to orders
bin/kafka-acls.sh --bootstrap-server kafka:9093 --command-config admin.properties \
--add --allow-principal User:order-service --operation Write --operation Describe --topic orders
# the billing service may read orders as consumer group "billing"
bin/kafka-acls.sh --bootstrap-server kafka:9093 --command-config admin.properties \
--add --allow-principal User:billing-service --operation Read --operation Describe --topic orders
bin/kafka-acls.sh --bootstrap-server kafka:9093 --command-config admin.properties \
--add --allow-principal User:billing-service --operation Read --group billing
bin/kafka-acls.sh --bootstrap-server kafka:9093 --command-config admin.properties --list --topic ordersConsumers need Read on the group as well as the topic, a common source of "authorization failed" errors. Transactional producers need Write on their transactional.id, and idempotent producers on older versions needed IdempotentWrite on the cluster. Prefix ACLs (--resource-pattern-type prefixed --topic billing.) keep rules manageable for teams that own a topic namespace.
What TLS, SASL and ACLs don't cover
- Data at rest: broker disks and tiered-storage objects aren't encrypted by Kafka. Use disk or volume encryption, and encrypted object storage.
- Sensitive fields: anyone allowed to read a topic sees every field. For card numbers or health data, encrypt fields in the producer (client-side field-level encryption) or keep them out of shared topics.
- The schema registry and Connect: secure their REST APIs too; Connect configs often hold database credentials (use config providers, Day 12).
Check yourself
No. TLS encrypts traffic between clients and brokers, but if anyone who can reach the cluster can connect without proving who they are, they can still read and write every topic. You need authentication, such as SASL/SCRAM, OAuth or mTLS, plus ACLs that grant each principal only what it needs.
Read permission on its consumer group. Consumers must join a group and commit offsets, which requires a separate Read ACL on the group resource, so topic permissions alone aren't enough.
With one identity per service, ACLs can grant each exactly the topics and groups it needs, audit logs show which service did what, and a compromised credential can be revoked without affecting everyone else. A shared superuser makes all of that impossible.
Encryption of data at rest on broker disks, protection against a compromised broker host, schema-level validation of payloads, and personal data inside messages. Those need disk encryption, host hardening, a schema registry and data handling policies.