Hadoop enable simple authentication - java

I tried the steps listed here How to disable Hadoop Kerberos but my datanode wouldn't start because of this exception
java.lang.IllegalArgumentException: Does not contain a valid host:port authority: 50010
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:212)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153)
I have apache hadoop 2.6 installed on a CentOS vm. I started hadoop with start-dfs.sh and start-yarn.sh. I'm trying to connect to hadoop from a java client with hdfs protocol but I am getting the below exception.
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): SIMPLE authentication is not enabled. Available:[TOKEN]
at org.apache.hadoop.ipc.Client.call(Client.java:1468)
at org.apache.hadoop.ipc.Client.call(Client.java:1399)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
I've seen a similar exception posted a few times by other people but it usually ends in [TOKEN, KERBEROS]. Do I need to enable KERBEROS authentication or is TOKEN something else? Ideally I would like to just use simple authentication. If I can't use simple authentication would installing CDH4 satisfy the TOKEN authentication?

The problem was that I was sending my request to the node address port I got from localhost:8088 and not port 8020 or whatever port is specified in fs.defaultFS in core-site.xml.

Related

Start jmeter distributed testing from clouds

I want to run distributed load testing by jmeter, where servers(linux) are in the cloud and I have ability to start test from my local pc.
Is it even possible or client and servers should be in same subnet?
And in case its possible, may you provide me step by step guide how to achive this goal?
I have tired
https://jmeter.apache.org/usermanual/remote-test.html
https://cloud.google.com/community/tutorials/ssh-port-forwarding-set-up-load-testing-on-compute-engine?hl=ja
but I got error after error.
Before I disabled ssl verification, next error appeared:
error during JRMP connection establishment; nested exception is:
javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake
With disabled ssl verification I have next error:
Error in rconfigure() method java.rmi.MarshalException: error marshalling argume
nts; nested exception is:
java.net.SocketException: Connection reset by peer: socket write error
I have feeligs, like I making smth wrong with setting ip, port or firewall, but cant understand what
It is definitely possible to starts load agents (jmeter-server) from your local system. The only condition is that the traffic should not be blocked by firewall.
Based on your error, it looks like jmeter process is not having permission to open a port, port is already open (although there is a specific exception for this)
You can also try allowing all incoming traffic to your slaves (cloud linux boxes) and try running the test again.
According to the manual you mentioned all you need to do is just to take the following steps:
On your local machine set up the SSH tunnel:
ssh -L 24000:127.0.0.1:24000 -R 25000:127.0.0.1:25000 -L 26000:127.0.0.1:26000 username#hostname_of_cloud_machine
On the cloud machine launch the JMeter slave
./jmeter -s -Jserver.rmi.ssl.disable=true -Jserver_port=24000 -Jserver.rmi.localhostname=127.0.0.1 -Jserver.rmi.localport=26000 -j jmeter.log
Again on the local machine launch the JMeter master:
./jmeter -Jserver.rmi.ssl.disable=true -Jremote_hosts=127.0.0.1:24000 -Jclient.rmi.localport=25000 -Jmode=Statistical -n -t test.jmx -l result.jtl
You don't need to configure any firewalls as all the communication is happening over the SSH tunnel
More information:
Remote hosts and RMI configuration
Apache JMeter Properties Customization Guide

Disable Host name verification in Solr SSL setup using self-signed certificate

I am using Solr as search server and saving all my data in Solr. I am using Solr version 5.5.2.
Solr is running in SSL mode with self signed certificate with hostname as localhost. I can not add hostname here as at the time of certificate generation I don't have machine name where its going be installed.
I am connecting to this solr server from wildfly 10. On wildfly 10 I have added my solr certificate as trusted so it is not complaining about self signed certificate but as my wildfly is on different host I am calling solr using hostname in URL and wildfly is raising below error.
Caused by: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://SOLR_HOST:8983/solr/default_core
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:589)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:974)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:990)
at org.springframework.data.solr.core.SolrTemplate$11.doInSolr(SolrTemplate.java:512)
at org.springframework.data.solr.core.SolrTemplate$11.doInSolr(SolrTemplate.java:509)
at org.springframework.data.solr.core.SolrTemplate.execute(SolrTemplate.java:168)
... 66 more
Caused by: javax.net.ssl.SSLException: Certificate for <SOLR_HOST> doesn't match common name of the certificate subject: localhost
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:173)
at org.apache.http.conn.ssl.BrowserCompatHostnameVerifier.verify(BrowserCompatHostnameVerifier.java:61)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:141)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:114)
at org.apache.http.conn.ssl.SSLSocketFactory.verifyHostname(SSLSocketFactory.java:580)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:554)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:412)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:328)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:481)
... 74 more
As I am not exposing my solr server outside my network I just want to ignore this error and disable hostname verification. HttpClient here is created by Solrj and managed within SolrTemplate so we do not have control to add custom strategy as suggested here
Basically I want to disable hostname verification for my setup. Any suggestion on this will be highly appreciated.
I figure out way from their source code. Solr is checking for below property when creating HttpClient object. Default value is true if we set it to false then Solr wont check hostname in SSL.
-Dsolr.ssl.checkPeerName=false
Set this property into your wildfly startup script or whatever program you are running and it should work fine. Solr will no more verify hostname of URL with hostname of self-signed certificate.

Weblogic, Tibco SSL with ejb3 :javax.jms.JMSSecurityException: Failed to connect to any server

EJB 3
Weblogic: 10.3.x
JDK: 1.7.x
We are facing below error when we enabled SSL for existing TIBCO connection factories/queues. Our requirement was to enable SSL for tibco connection factories / queues.
javax.jms.JMSSecurityException: Failed to connect to any server at: ssl://server_name:44343,ssl://server_name:44343 [Error: invalid name or password: url that returned this exception = SSL://server_name:44343 ]
at com.tibco.tibjms.TibjmsConnection._create(TibjmsConnection.java:1389)
at com.tibco.tibjms.TibjmsConnection.<init>(TibjmsConnection.java:4127)
at com.tibco.tibjms.TibjmsXAConnection.<init>(TibjmsXAConnection.java:30)
at com.tibco.tibjms.TibjmsXAQueueConnection.<init>(TibjmsXAQueueConnection.java:29)
at com.tibco.tibjms.TibjmsxCFImpl._createImpl(TibjmsxCFImpl.java:164)
at com.tibco.tibjms.TibjmsxCFImpl._createConnection(TibjmsxCFImpl.java:253)
at com.tibco.tibjms.TibjmsXAQueueConnectionFactory.createXAQueueConnection(TibjmsXAQueueConnectionFactory.java:129)
at weblogic.ejb.container.internal.JMSConnectionPoller.getXAConnection(JMSConnectionPoller.java:438)
at weblogic.ejb.container.internal.JMSConnectionPoller.createJMSConnection(JMSConnectionPoller.java:2233)
at weblogic.ejb.container.internal.JMSConnectionPoller.connect(JMSConnectionPoller.java:808)
at weblogic.ejb.container.internal.MDConnectionManager.startConnectionPolling(MDConnectionManager.java:265)
at weblogic.ejb.container.manager.MessageDrivenManager.start(MessageDrivenManager.java:657)
at weblogic.ejb.container.deployer.MessageDrivenBeanInfoImpl$DestinationResovler.activateNoneDDMDManager(MessageDrivenBeanInfoImpl.java:2356)
at weblogic.ejb.container.deployer.MessageDrivenBeanInfoImpl$QueueConnectionHandler.handleNoneDD(MessageDrivenBeanInfoImpl.java:2798)
at weblogic.ejb.container.deployer.MessageDrivenBeanInfoImpl$DestinationResovler.resolveDestnationWorkMode(MessageDrivenBeanInfoImpl.java:2289)
at weblogic.ejb.container.deployer.MessageDrivenBeanInfoImpl$DestinationEventHandler.onDestinationsAvailable(MessageDrivenBeanInfoImpl.java:2112)
at weblogic.jms.extensions.JMSDestinationAvailabilityHelper$DestinationAvailabilityListenerWrapper$2.run(JMSDestinationAvailabilityHelper.java:386)
at weblogic.jms.extensions.JMSDestinationAvailabilityHelper$DestinationAvailabilityListenerWrapper.callOutListener(JMSDestinationAvailabilityHelper.java:402)
at weblogic.jms.extensions.JMSDestinationAvailabilityHelper$DestinationAvailabilityListenerWrapper.onDDMembershipChange(JMSDestinationAvailabilityHelper.java:383)
at weblogic.jms.common.CDS$DD2Listener.run(CDS.java:1264)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:545)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
The error message shared does show "invalid name or password" as the cause of the error, so I believe that is a good hint to what is causing the error.
Make sure the correct username and password has been configured both in the TIBCO EMS configuration file for the particular instance you are trying to connect to and on Weblogic.
The only other reason I can think of is if the ssl url is incorrect which I believe is not the case.

Got AmazonClientException "hostname in certificate didn't match” when using SQS JAVA API

When using JAVA API of SQS (sdk version: 1.9.30), sometimes I got following exceptions:
com.amazonaws.AmazonClientException: Unable to execute HTTP request: hostname in certificate didn't match: <sqs.us-west-1.amazonaws.com> != <*.mycompany.com> OR <*.mycompany.com> OR <mycompany.com>
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:472)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:296)
at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2422)
at com.amazonaws.services.sqs.AmazonSQSClient.sendMessage(AmazonSQSClient.java:1015)
There are some question like mine (i.e
SSL problems with S3/AWS using the Java API: "hostname in certificate didn't match"), but seems not the same one. Because in my situation the exception is like:
[sqs url] != [my company's url]
looks like when I try to connect to AWS server, the sdk connect to my company's server for some reason, and make the SSL handshake fail.
There are other one got the same question (https://github.com/Upplication/Amazon-S3-FileSystem-NIO2/issues/40), but still have no answer.
Is this a bug of SDK and any solution for this?

How to find weblogic t3 adminurl

I have a weblogic folder, From this - How can i find the adminurl that I can use with the below command
java weblogic.Deployer –adminurl t3://server:7001 –username system –password weblogic –listapps
I tried looking into setupinfo.txt, portlist.ini - but somehow this did not help/work. How can get this working?
I get errors/exceptions like these (after few attempts)
The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object
Unable to connect to 'http://server:7001': Destination unreachable; java.net.ConnectException: Tried all: 1 addresses, but could not connect over HTTPS to server: localhost port: 7102; No available router to destination
based on setupinfo.txt, I am able to login to the web console - but I need to know how to get this command working with adminurl !!!
Thanks,
Got it, thanks Alex.
Steps
Find startscript.xml under your weblogic domain , search this file for "ADMIN_URL"
The same can be done by web console UI ..... Admin Console Login to AdminConsole->Server->Configuration->ListenPort (enable and note down the port)
Bingo.

Categories