We Keep Coding

Could not create File Client. Mismatch found for java and native libraries java build version

I am trying to access mapr path remotely, using a spring boot application. I have set the fs.mapr.bailout.on.library.mismatch property to false, to avoid the error on version mismatch. Still whenever the getFileStatus() function gets called, the application stops with the following error:
2020-12-15 10:07:18,7377 ERROR JniCommon fs/client/fileclient/cc/jni_MapRClient.cc:691 Thread: 123145425235968 Mismatch found for java and native libraries java build version 6.0.1.20180404222005.GA, native build version 6.0.1.20190808152212.GA java patch vserion $Id: mapr-version: 6.0.1.20180404222005.GA 1aeeb6d3c17c777fcba0, native patch version $Id: mapr-version: 6.0.1.20190808152212.GA 1aeeb6d3c17c777fcba0
2020-12-15 10:07:18,7378 ERROR JniCommon fs/client/fileclient/cc/jni_MapRClient.cc:708 Thread: 123145425235968 Client initialization failed.
Code:
FileSystem fileSystem = FileSystem.newInstance(new Configuration());
Configuration conf = fileSystem.getConf();
conf.set("fs.mapr.bailout.on.library.mismatch", "false");
Path OffsetPath = new Path(filePath);
FileStatus file = fileSystem.getFileStatus(filePath); ====> This statement gives error
hbase dependencies used:
<dependency>
<groupId>com.mapr.fs</groupId>
<artifactId>mapr-hbase</artifactId>
<version>6.0.1-mapr</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>1.1.8-mapr-1710</version>
</dependency>
How can I correct this?

So it isn't a great idea to just disable the version mismatch error. Some mismatch is OK for some tasks, but some mismatch can be fatal.
To say much more, it would be helpful to have a little bit of clarification.
when you say remote access, are you talking from one cluster to another? Or from a client machine that is outside the cluster?
assuming that you mean the second case (client outside the cluster), what did you install on the client machine?
why did you disable the version mismatch? What versions are you worried will not match?
does the POSIX interface or the local loopback NFS work from the problem node?
does the hadoop shell work from the problem node?

Apache Beam Streaming from Pub/Sub to ElasticSearch

I'm writing a java streaming pipeline with Apache Beam that reads messages from Google Cloud PubSub and should write them into an ElasticSearch instance. Currently, I'm using the direct runner, but the plan is to deploy the solution on Google Cloud Dataflow.
First of all, I wrote a pipeline that reads from PubSub and writes to text files and it works. Then, I sat up the ElasticSearch instance and also this works. I wrote some documents with curl and it was easy.
Then, when I tried to perform the write with Beam's ElasticSearch connector, I started to get some error. Actually, I get ava.lang.NoSuchMethodError: org.elasticsearch.client.RestClient.performRequest, in spite of the fact that I added the dependency on my pom.xml file.
What I'm doing is essentially this:
messages.apply(
"TwoMinWindow",
Window.into(FixedWindows.of(new Duration(120*1000)))
).apply(
"ElasticWrite",
ElasticsearchIO.write()
.withConnectionConfiguration(
ElasticsearchIO.ConnectionConfiguration
.create(new String[]{"http://xxx.xxx.xxx.xxx:9200"}, "streaming_data", "string")
.withUsername("xxxx")
.withPassword("xxxxxxxx")
)
);
Using the DirectRunner, I'm able to connect to PubSub, but I get an exception when the pipeline tries to connect with the ElasticSearch instance:
java.lang.NoSuchMethodError: org.elasticsearch.client.RestClient.performRequest(Ljava/lang/String;Ljava/lang/String;[Lorg/apache/http/Header;)Lorg/elasticsearch/client/Response;
at org.apache.beam.sdk.util.UserCodeException.wrap (UserCodeException.java:34)
at org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO$Write$WriteFn$DoFnInvoker.invokeSetup (Unknown Source)
at org.apache.beam.sdk.transforms.reflect.DoFnInvokers.tryInvokeSetupFor (DoFnInvokers.java:50)
at org.apache.beam.runners.direct.DoFnLifecycleManager$DeserializingCacheLoader.load (DoFnLifecycleManager.java:104)
at org.apache.beam.runners.direct.DoFnLifecycleManager$DeserializingCacheLoader.load (DoFnLifecycleManager.java:91)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture (LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync (LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad (LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get (LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get (LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad (LocalCache.java:3974)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get (LocalCache.java:4958)
at org.apache.beam.runners.direct.DoFnLifecycleManager.get (DoFnLifecycleManager.java:61)
at org.apache.beam.runners.direct.ParDoEvaluatorFactory.createEvaluator (ParDoEvaluatorFactory.java:129)
at org.apache.beam.runners.direct.ParDoEvaluatorFactory.forApplication (ParDoEvaluatorFactory.java:79)
at org.apache.beam.runners.direct.TransformEvaluatorRegistry.forApplication (TransformEvaluatorRegistry.java:169)
at org.apache.beam.runners.direct.DirectTransformExecutor.run (DirectTransformExecutor.java:117)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
at java.lang.Thread.run (Thread.java:748)
Caused by: java.lang.NoSuchMethodError: org.elasticsearch.client.RestClient.performRequest(Ljava/lang/String;Ljava/lang/String;[Lorg/apache/http/Header;)Lorg/elasticsearch/client/Response;
at org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO.getBackendVersion (ElasticsearchIO.java:1348)
at org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO$Write$WriteFn.setup (ElasticsearchIO.java:1200)
What I added in the pom.xml is :
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
<version>${beam.version}</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.elasticsearch.client/elasticsearch-rest-client -->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
<version>${elastic.version}</version>
</dependency>
I'm stuck with this problem and I don't know how to solve it. If I use a JestClient, I'm able to connect to ElasticSearch without any issue.
Have you any suggestion?

You are using a newer version of RestClient that does not have the method performRequest(String, Header). If you look at the latest source code, you can see that the method takes a Request now, whereas in older versions there were methods that took Strings and Headers.
These methods were deprecated and then removed from the code on September 1, 2018.
Either change your code to use the newer Elastic Search library, or specify an older version of the library (it needs to be before 7.0.x, e.g. 6.8.4) that is compatible with your code.

How to force JavaMailSenderImpl to use TLS1.2?

Have a JDK7 app running on Tomcat and it does have the following env settings:
-Dhttps.protocols=TLSv1.1,TLSv1.2
The above setting ensures that we don't use TLS 1.0 when connecting over HTTPS while making API calls etc.
We also use the org.springframework.mail.javamail.JavaMailSenderImpl class to send outgoing SMTP email, and use these props:
mail.smtp.auth=false;mail.smtp.socketFactory.port=2525;mail.smtp.socketFactory.fallback=true;mail.smtp.starttls.enable=true
The problem is that the connection to the SMTP email server is failing when it's upgraded to TLS1.2.
javax.net.ssl.SSLHandshakeException: Remote host closed connection
during handshake
Is there a settings or code change that will force the TLS1.2 protocol?
I did some searching and it looks like these env settings are only for applet and web clients, not for server side apps
-Ddeployment.security.SSLv2Hello=false -Ddeployment.security.SSLv3=false -Ddeployment.security.TLSv1=false

This is the fix for the next guy looking:
mail.smtp.starttls.enable=true;
mail.smtp.ssl.protocols=TLSv1.2;

It didn't work for me in one pretty old app and I couldn't realize why. After some research I found that the javax.mail version in the app dependencies was 1.4. You must upgrade to at least 1.5.

I needed both Vojtech Zavrel and Sunny's answer in my case. I was running Java 1.8 Spring Boot 1.2.5 and running on Big Sur 11.2.3 and spring version 4.2.1.RELEASE.
After I updated my dependency like this
<dependency>
<groupId>javax.mail</groupId>
<artifactId>mail</artifactId>
<version>1.5.0-b01</version>
</dependency>
and I updated my JavaMailSenderImpl with
Properties prop = new Properties();
prop.setProperty("mail.smtp.auth", "true");
prop.setProperty("mail.smtp.starttls.enable", "true");
prop.setProperty("mail.smtp.ssl.protocols", "TLSv1.2"); // Added this line
prop.setProperty("mail.smtp.ssl.trust", mailUri.getHost());
mailSender.setJavaMailProperties(prop);
I saw the Received fatal alert: protocol_version error resolve.

An update to the most recent version (1.6.2.) of Java Mail also fixes the issue. In my case I upgraded from:
<dependency>
<groupId>javax.mail</groupId>
<artifactId>mail</artifactId>
<version>1.5.0-b01</version>
</dependency>
to:
<dependency>
<groupId>com.sun.mail</groupId>
<artifactId>javax.mail</artifactId>
<version>1.6.2</version>
</dependency>
This fixed the error
javax.mail.AuthenticationFailedException: 421 4.7.66 TLS 1.0 and 1.1 are not supported. Please upgrade/update your client to support TLS 1.2
I was getting from an Outlook SMTP-Server. No property changes needed.

websocket code not working on Prod Tomcat Server - 404 Error

I am trying to run my first websocket app and refered this link to get some sample code . I simply created CustomEndPoint , WSClient class , html file and then ran it on netbeans IDE and it was working like a charm.
I tried to deploy it on tomcat server whose url is accessible using https by changing ws:// with wss:// and it worked on my dev environment but when I deployed the same code on Production env its throwing below error in console:
WebSocket connection to 'wss://xxxxxx-xxx.xxxx.com/websoc/ratesrv' failed: Error during WebSocket handshake: Unexpected response code: 404
For dev environment I below WS call is working :
wsocket = new WebSocket("wss://dev_ip:8443/websoc/ratesrv");
For Prod I am using(note the .com in url):
wss://xxxxx-xxxxx.xx.com/websoc/ratesrv
Do I need to explicitly provide the port number as well in PROD ?

Does your Tomcat version includes Websockets Runtime?
If it does you must delete all the Websockets dependencies from your WAR. Ensure that you call mvn clean after change scope to provided.
If not, you should include it. If you want to use Tyrus just put
<dependency>
<groupId>org.glassfish.tyrus</groupId>
<artifactId>tyrus-container-servlet</artifactId>
<version>1.12</version>
</dependency>
<dependency>
<groupId>org.glassfish.tyrus</groupId>
<artifactId>tyrus-client</artifactId>
<version>1.12</version>
</dependency>
And check that that there are no errors in the Tomcat console when deploy.

Try to use the latest version of tomcat.
tomcat8.5.20 worked for me.

JDBC via JT400 works on local Apache Tomcat 8 install but fails on Apache Tomcat 8 running on server

I have a Java application running and tested on my development workstation running Apache Tomcat 8. I have an established IBM i (AS/400) database connection working locally using the JT400.jar file. When I build and deploy the application to our production server running Apache Tomcat 8 with the same JT400.jar file, the database connection seems to fail and I cannot figure out why.
I get the following "HTTP Status 500 - Servlet execution threw an exception" error:
<code>
java.lang.AbstractMethodError
org.apache.tomcat.dbcp.dbcp2.DelegatingConnection.isValid(DelegatingConnection.java:913)
org.apache.tomcat.dbcp.dbcp2.PoolableConnection.validate(PoolableConnection.java:218)
org.apache.tomcat.dbcp.dbcp2.PoolableConnectionFactory.validateConnection(PoolableConnectionFactory.java:302)
org.apache.tomcat.dbcp.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2164)
org.apache.tomcat.dbcp.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2147)
org.apache.tomcat.dbcp.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:1902)
org.apache.tomcat.dbcp.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1412)
utilities.SerialSearch.doSearch(SerialSearch.java:81)
processes.ProcessScan.getSerialScreenDetail(ProcessScan.java:66)
processes.ProcessScan.ProcessScanRequest(ProcessScan.java:103)
controller.MetricServlet.performTask(MetricServlet.java:145)
controller.MetricServlet.doPost(MetricServlet.java:43)
javax.servlet.http.HttpServlet.service(HttpServlet.java:644)
javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
</code>
What is different and how do I resolve the issue?
Thanks!

I had the same error while using the current latest version (as of today 9.3).
The problem was that I was using the JDK 8, and using maven to fetch the package I received a version that was not appropriate for this JDK.
To solve this problem I used the proper maven qualifier:
<dependency>
<groupId>net.sf.jt400</groupId>
<artifactId>jt400</artifactId>
<version>9.3</version>
<classifier>jt400_jdk8</classifier>
</dependency>
Note that if you download the zip with the full suite, all version of the jar do have the same name (i.e. jt400.jar) which makes them hard to distinguish.

Problem solved. Rookie mistake. I inadvertently left the old jt400 jar files in the lib folder on the Apache Tomcat 8 server by renaming them. Once I deleted the old jar files my issues were solved with the new jt400 jar that I installed last week that started the conflict to begin with. Thanks!