I have an issue producing messages to a Kafka topic (named secure.topic) secured with ACL.
My Groovy-based producer throws this error:
Error while fetching metadata with correlation id 9 : {secure.topic=LEADER_NOT_AVAILABLE}
Some notes about the configuration:
1 Kafka server, version 2.11_1.0.0 (both server and Java client libs)
topic ACL is set to All (also tested with --producer) and the user is the full name specified in the certificate
client auth enabled using self generated certificates
Additional server config:
security.inter.broker.protocol = SSL
ssl.client.auth = required
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
If I remove the authorizer.class.name property, then my client can produce messages (so, no problem with SSL and certificates).
Also, the kafka-authorizer.log produces the following message:
[2018-01-25 11:57:02,779] INFO Principal = User:CN= User,OU=XXX,O=XXX,L=XXX,ST=Unknown,C=X is Denied Operation = ClusterAction from host = 127.0.0.1 on resource = Cluster:kafka-cluster (kafka.authorizer.logger)
Any idea what can cause the LEADER_NOT_AVAILABLE error when enabling ACL?
From the authorizer logs, it looks like the Authorizer denied ClusterAction on the Cluster resource.
If you check your topic status (for example using kafka-topic.sh), I'd expect to see it without a Leader (-1).
When enabling authorizations, they are applied to all Kafka API messages reaching your cluster including inter-broker messages like StopReplica, LeaderAndIsr, ControlledShutdown, etc. So it looks like you only added ACLs for your client but forgot to add the ACLs required for the brokers to function.
So you need to at least add an ACL granting ClusterAction on the Cluster resource for your broker's principals. IIRC that's the only required ACL for inter-broker messages.
Following that, your cluster should be able to correctly elect a leader for the partition enabling your client to produce.
Related
I have 3 nodes of Kafka cluster in the Windows environment. I have recently added security to this existing cluster with the SASL_SSL mechanism.
Here is my server.properties security configurations on each node:
authroizer.class.name=kafka.security.auth.SimpleAclAuthorizer
security.inter.broker.protocol=SASL_SSL
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN
ssl.client.auth=required
ssl.enabled.protocols=TLSv1.2
ssl.endpoint.identification.algorithm=
ssl.truststore.location=kafka-truststore.jks
ssl.truststore.password=******
ssl.keystore.location=kafka.keystore.jks
ssl.keystore.password=******
ssl.key.password=******
Everything is working fine. I am able to store and retrieve messages. Kafka stream applications are properly connected. But from yesterday I am getting continuous logs on all three nodes as
INFO [SocketServer brokerId=2] Failed authentication with host.docker.internal/ip (SSL handshake failed) (org.apache.kafka.common.network.Selector)
As the log says broker with id 2 is refusing the SSL handshake from the other brokers i.e. 1 & 3.
I have verified the jks certificates and they all are valid.
Did anyone know the reason for such logs?
I'm getting this exception:
org.apache.pulsar.client.api.PulsarClientException$AuthorizationException: Authorization failed example_ingest_producer on topic persistent://myTenant/myNamespace/myTopicName with error Don't have permission to administrate resources on this tenant
when trying to connect to Pulsar from our client application. I'm running Pulsar 2.4.2.
I confirmed that I'm connecting to the correct endpoint (pulsar+ssl://pulsar-ms-tls.mydomain.com:6651), and we're using SSL+TLS.
What could be causing this problem?
This error is occurring because you're not using the correct token to connect, or the role associated with your token lacks sufficient permission. You will need to ensure that you are using the correct token (and that it has the right permissions) to enable you to connect.
Running on RHEL 7.5 with Java 8. Kerberos 5 release 1.15.1.
We are seeing a strange behaviour with this set-up that has been seen in all versions since 2.11.10.
Note, I can't post direct logs or config as it my company blocks this.
Steps to reproduce
1) Configure gerrit to use kerberos
gerrit.config
[container]
javaHome = <path to JRE>
javaOptions = -Djava.security.auth.login.config=<path to jaas.conf>
[auth]
type = LDAP
[ldap]
authentication = GSSAPI
server = ldap://<AD Realm>
<.. other AD related stuff..>
jaas.conf
KerberosLogin {
com.sun.security.auth.module.Krb5LoginModule
required
useTicketCache=true
doNotPrompt=true
renewTGT=true;
};
which is direct from the documentation.
2) kinit the keytab to create a ticket in the cache.
3) Try to login. It fails with "Server not found in Kerberos database (7)".
It will also fail if you change the jaas.conf to try and use the keytab directly.
You can access LDAP directly using the username/password but due to Company restrictions we can't have an unencrypted password at rest on a device so this is not a viable long, term solution.
We have taken packet captures of the traffic to the AD Realm and we see the same behaviour whether we use the keytab or the cache.
1) For the kinit we see one request to AD with the SPN field set to the SPN from the keytab. This, of course, works fine.
2) For any request from Gerrit we see TWO requests to AD, the first has the correct SPN from the cache/keytab the second tries to send an SPN of "ldap/" no matter what value of SPN is set. This second request is what is causing the error as that SPN is not recognised b AD.. Note, we have tried keytabs with various SPN's (HTTP/device, host/device, HTTP/device# etc etc). The same thing happens every time.
This may well be something very simple is wrong in our config but we have been banging our heads on this for weeks now.
The second request most likely shows up because you specified an LDAP server ldap://<AD Realm> in Gerrit's configuration. HTTP GSSAPI authentication may very well have succeeded at this point, but now the application needs to authenticate itself against the LDAP server before it can retrieve information about the user. That happens independently from the HTTP authentication itself.
It's normal that the SPN is not recognized because Active Directory generally doesn't use <AD Realm> to pick a domain controller – instead the individual server names have to be specified, e.g. ldap://dc01.ad.example.com. (Real AD clients choose a server automatically via DNS SRV records, but plain LDAP clients often don't support that.)
Note also that a keytab is essentially an unencrypted password at rest.
I have a unsecured kafka instance with 2 brokers everything was running fine until I decided to configure ACL for topics, after ACL configuration my consumers stopped polling data from Kafka and I keep getting warning Error while fetching metadata with correlation id , my broker properties looks like below:-
listeners=PLAINTEXT://localhost:9092
advertised.listeners=PLAINTEXT://localhost:9092
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
allow.everyone.if.no.acl.found=true
And my client configuration looks like below:-
bootstrap.servers=localhost:9092
topic.name=topic-name
group.id=topic-group
I've used below command to configure ACL
bin\windows\kafka-acls.bat --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:* Read --allow-host localhost --consumer --topic topic-name --group topic-group
After having all above configuration when I start consumer it stopped receiving messages. Can someone point where I'm mistaking. Thanks in advance.
We are using ACLs successfully, but not with PLAINTEXT protocol.
IMHO you shall use SSL protocol and instead of localhost use the real machine name.
I am having an issue with connecting AS400 MQ Local Queue, its rejecting with code JMSWMQ2013.
My appserver has a username as mquser#mydomain.com but in AS400 I am not able to give the specified username in MQ Object Authority.
Is there any way to connect to the queue defined in AS400 machine from Websphere Appserver on windows machine?
Below is the error i am facing while connecting:
FFDC Exception:com.ibm.msg.client.jms.DetailedJMSSecurityException SourceId:com.ibm.ejs.jms.JMSManagedQueueConnection.createConnection ProbeId:116 Reporter:com.ibm.ejs.jms.JMSManagedQueueConnection#db6f33e4
com.ibm.msg.client.jms.DetailedJMSSecurityException: JMSWMQ2013: The security authentication was not valid that was supplied for QueueManager 'TESTQMGR' with connection mode 'Client' and host name 'AS400T(1416)'.
Please check if the supplied username and password are correct on the QueueManager to which you are connecting.
Root cause:
JMSCMQ0001: WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2035' ('MQRC_NOT_AUTHORIZED').
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:204)
Maybe you need to define separate J2C authentication alias and map it to the connection factory. What does it mean that My appserver has a username as mquser#mydomain.com?
Check this page 2035 MQRC_NOT_AUTHORIZED Connecting to WebSphere MQ for more details:
The two most likely reasons why the connection is refused by MQ are as follows:
1. The user identifier passed across the client connection from the application
server to MQ is not known on the server where the MQ queue manager is running,
is not authorised to connect to MQ, or is longer than 12 characters and has
been truncated.
For queue managers running on Windows, the following error might be seen in
the MQ error logs for this scenario:
AMQ8075: Authorization failed because the SID for entity 'wasuser'
cannot be obtained.
For UNIX no entry in the MQ error logs would be seen by default.
See technote MQS_REPORT_NOAUTH environment variable can be used
to better diagnose return code 2035 for details of enabling error log
entries on all platforms.
2. The user identifier passed across the client connection from the
application server to MQ is a member of the 'mqm' group on the server
hosting the MQ queue manager, and a Channel Authentication Record (CHLAUTH)
exists that blocks administrative access to the queue manager.
WebSphere MQ configures a CHLAUTH record by default in WebSphere MQ
Version 7.1 and later that blocks all MQ admins from connecting
as a client to the queue manager.
The following error in the MQ error logs would be seen for
this scenario: AMQ9777: Channel was blocked
You indicate in a later comment that you are using MQ V7.0 on AS/400.
Your question details that you have a user ID mquser#mydomain.com which will not be recognised by the AS/400 O/S.
Therefore you are looking for a way to assign a user ID for the connection which you are making from the App Server on Windows so that it can run using a recognised user ID on the AS/400 queue manager.
Since you are pre-V7.1, you cannot use CHLAUTH rules, so your choices are
Write a security exit to do it (or buy/download one)
Give this connection its own channel and set the MCAUSER on the SVRCONN to something that is known and recognised by the AS/400 O/S. In this case, please also make sure you have some form of authentication, e.g. SSL/TLS so that no-one else can use this channel.
Yes, Username and password can be passed with setStringProperty on MQQueueConnectionFactory
MQQueueConnectionFactory mqConFactory = new MQQueueConnectionFactory();
mqConFactory.setStringProperty(WMQConstants.USERID, "username");
mqConFactory.setStringProperty(WMQConstants.PASSWORD, "password");
//other configs
mqConFactory.setHostName("MQ_HOST");
mqConFactory.setChannel("MQ_CHANNEL");//communications link
mqConFactory.setPort("MQ_PORT");
mqConFactory.setQueueManager("MQ_MANAGER");//service provider
mqConFactory.setTransportType(JMSC.MQJMS_TP_CLIENT_MQ_TCPIP);
Needed imports:
import com.ibm.mq.jms.MQQueueConnectionFactory;
import com.ibm.msg.client.wmq.WMQConstants;
Dependency jars:
compile('com.ibm.mq:com.ibm.mq.allclient:9.0.5.0')
Part of code took from this page