Configure Access to Kafka Cluster

The following information describes how to use credentials to configure access to the Kafka cluster.

Kafka mTLS authentication

Communication with your Kafka cluster is TLS secured, meaning both the client and the Kafka cluster authenticate each other. The client authenticates the server by verifying the server's certificate, and the server authenticates the client by verifying the client's certificate. As the Kafka cluster does not have publicly signed certificates, you must validate them with the cluster's certificate authority. Authentication happens via mutual TLS (mTLS). Therefore, your cluster maintains a client certificate authority to sign authenticated user certificates.

Get certificates and key

To connect and authenticate to your Kafka cluster, you must fetch the required two certificates and a key from the user's API endpoint. Below are the steps to get the required certificates and key with curl commands for a cluster created in Frankfurt (de-fra) region.

# Get the cluster's CA certificatecurl--locationhttps://kafka.de-fra.ionos.com/clusters/${clusterId}/users/${userId}/access--header"Authorization: Bearer ${personalToken}"|yq-r'.metadata.certificateAuthority'>ca-cert.pem# verifyopensslx509-inca-cert.pem-text-noout# Get the (admin) users client certificatecurl--locationhttps://kafka.de-fra.ionos.com/clusters/${clusterId}/users/${userId}/access--header"Authorization: Bearer ${personalToken}"|yq-r'.metadata.certificate'>admin-cert.pem# verifyopensslx509-inadmin-cert.pem-text-noout# Get the (admin) users client keycurl--locationhttps://kafka.de-fra.ionos.com/clusters/${clusterId}/users/${userId}/access--header"Authorization: Bearer ${personalToken}"|yq-r'.metadata.privateKey'>admin-key.pem# verifyopensslrsa-inadmin-key.pem-check

Convert certificates & key

You will need different file formats for the certificates depending on the consumer/producer's implementation. The following sections show how to create and use them with the Kafka Command-Line Interface (CLI) Tools.

PKCS#12 (.p12 / .pfx)

# Create a ca-cert.p12 (with openssl >3.2 )openssl pkcs12 -export -nokeys -in ca-cert.pem -out ca-cert.p12 -passout "pass:changeit" -jdktrust anyExtendedKeyUsage# Create a ca-cert.p12 (with keytool)keytool -importcert -storetype PKCS12 -keystore ca-cert.p12 -storepass changeit -alias cluster-ca -file ca-cert.pem -noprompt# verifyopenssl pkcs12 -info -in ca-cert.p12# Create an admin.p12openssl pkcs12 -export -in admin-cert.pem -inkey admin-key.pem -out admin.p12 -passout "pass:admin_p12_pass"# verifyopenssl pkcs12 -info -nodes -in admin.p12

Your admin.properties files should look like this:

security.protocol=SSLssl.truststore.type=PKCS12ssl.truststore.location=ca-cert.p12ssl.truststore.password=changeitssl.endpoint.identification.algorithm=ssl.keystore.type=PKCS12ssl.keystore.location=admin.p12ssl.keystore.password=admin_p12_pass

bin/kafka-topics.sh --list --bootstrap-server=clusterIp:Port --command-config admin.properties

Java KeyStore (JKS)

# Create a Java Truststorekeytool -import -alias cluster-ca -file ca-cert.pem -keystore truststore.jks -storepass changeit -noprompt# verifykeytool -list -keystore truststore.jks -rfc -storepass changeit# Create a Java Keystoreopenssl pkcs12 -export -in admin-cert.pem -inkey admin-key.pem -out admin.p12 -passout "pass:admin_p12_pass"keytool -importkeystore -srckeystore admin.p12 -srcstorepass admin_p12_pass -destkeystore admin.ks -storepass admin_jks_pass# verifykeytool -list -keystore admin.ks -rfc -storepass admin_jks_pass# verify including the keykeytool -importkeystore -srckeystore admin.ks -srcstorepass admin_jks_pass -deststoretype PKCS12 -destkeystore filename.p12 -storepass p12_pass; openssl pkcs12 -info -nodes -in filename.p12 -passin "pass:p12_pass"; rm -f filename.p12

Your admin.properties files should look similar to the following:

security.protocol=SSLssl.truststore.location=truststore.jksssl.truststore.password=changeitssl.endpoint.identification.algorithm=ssl.keystore.location=admin.ksssl.keystore.password=admin_jks_pass

bin/kafka-topics.sh --list --bootstrap-server=clusterIp:Port --command-config admin.properties

PKCS#8 PEM

# No need to do anything with the ca-cert.pem it can be used without any modification# verifyopenssl x509 -in ca-cert.pem -text -noout# Create a admin.pem containing key and cert# as the Kafka CLI tool requires the key in PKCS#8 and to be secured with a passphrase we need to convert it firstopenssl pkcs8 -in admin-key.pem -passout "pass:admin_pem_pass" -topk8 -v1 PBE-SHA1-3DES -out admin.pemcat admin-cert.pem >> admin.pem# verifyopenssl x509 -in admin.pem -text -nooutopenssl pkey -in admin.pem -check

Your admin.properties files should look similar to the following:

security.protocol=SSLssl.truststore.type=PEMssl.truststore.location=ca-cert.pemssl.endpoint.identification.algorithm=ssl.keystore.type=PEMssl.keystore.location=admin.pemssl.key.password=admin_pem_pass

bin/kafka-topics.sh --list --bootstrap-server=clusterIp:Port --command-config admin.properties