We Keep Coding

TLS on apache server

I want to start Apache HTTP server in Java.
After a while I found this answer. And did as indicated there.
The KeyStore was created according to this instruction.
My code now:
import me.project.Main;
import org.apache.http.ExceptionLogger;
import org.apache.http.config.SocketConfig;
import org.apache.http.impl.bootstrap.HttpServer;
import org.apache.http.impl.bootstrap.ServerBootstrap;
import org.apache.http.ssl.SSLContexts;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import javax.net.ssl.SSLContext;
import java.io.IOException;
import java.net.URL;
import java.security.KeyManagementException;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.UnrecoverableKeyException;
import java.security.cert.CertificateException;
public class Server {
private static final Logger logger = LoggerFactory.getLogger(Server.class);
private static HttpServer server;
public static void createAndStart() {
if (server != null) return;
SSLContext sslContext = prepareSSLContext();
SocketConfig config = SocketConfig.custom().setSoTimeout(1500).setTcpNoDelay(true).build();
server = ServerBootstrap.bootstrap()
.setListenerPort(43286)
.setServerInfo("Test")
.setSocketConfig(config)
.setSslContext(sslContext)
.setExceptionLogger(ExceptionLogger.STD_ERR)
.registerHandler("/", new HelloPage())
.create();
System.out.println("Ok");
try {
server.start();
} catch (IOException ioException) {
logger.error("Cannot start server: ", ioException);
}
}
private static SSLContext prepareSSLContext() {
URL keyStoreFile = Main.class.getClassLoader().getResource("keystore.jks");
if (keyStoreFile == null) {
logger.error("Key store not found");
System.exit(1);
}
SSLContext sslContext = null;
try {
sslContext = SSLContexts.custom()
.loadKeyMaterial(keyStoreFile, "mysuperpass".toCharArray(), "mysuperpass".toCharArray()).build();
} catch (NoSuchAlgorithmException | KeyManagementException | KeyStoreException | UnrecoverableKeyException | CertificateException | IOException e) {
logger.error("Cannot init ssl context", e);
}
return sslContext;
}
}
It starts sucessfully and prints "Ok".
But a few things I still don't understand:
It seems to me that this is still SSL protocol, not TLS
How can I make sure that the server really communicates using the encrypted TLS protocol (except to intercept traffic and watch)
How does the loadKeyMaterial (keyStore, storePass, KeyPass) method know which key to take from the store without the key alias?
I found another example of enabling TLS here, but I don't understand where to use the object conn after:
DefaultBHttpClientConnection conn = new DefaultBHttpClientConnection(8 * 1204);
conn.bind(socket);
Sorry, I don't know English well and translated this with Google Translate

It seems to me that this is still SSL protocol, not TLS
The classes were originally implemented for SSL(v3) in the 1990s, but they have implemented TLS since about 2000 (first 1.0, later 1.1 1.2 and now 1.3) while the names have remained the same for compatibility. In fact since 8u31 in 2014 SSLv3 has been disabled by default in the JVM configuration because it was broken by the "POODLE" attack and the 'SSL' classes in fact provide only TLS.
How can I make sure that the server really communicates using the encrypted TLS protocol (except to intercept traffic and watch)
Try connecting to it with TLS and non-TLS tools.
How does the loadKeyMaterial (keyStore, storePass, KeyPass) method know which key to take from the store without the key alias?
It uses the JSSE-default KeyManager which implements chooseServerAlias to return the first key-and-cert entry (i.e. PrivateKeyEntry) it finds that is suitable for a client-enabled ciphersuite -- or for TLS 1.3 where this is no longer in the ciphersuite, a client-enabled sigalg. IIRC 'first' is in a hashmap-defined order which means for practical/human purposes it is arbitrary. But most servers use a keystore that contains only one PrivateKeyEntry so all choice sequences are the same.

Jetty 9 WebSocket client issue when using secure connection

I'm writing a WebSocket client in a java application, using the Jetty 9.4.18 libraries.
I'm pretty new to WebSockets, so I started testing using the two sample classes from the Jetty documentation, connecting to echo.websocket.org
The test runs fine when I connect without SSL, but if fails when the connection is done to wss://echo.websocket.org
I always get the same exception:
java.io.EOFException: HttpConnectionOverHTTP#50371e9d::DecryptedEndPoint#6dc65fc2{echo.websocket.org/174.129.224.73:443<->/192.168.1.34:60521,OPEN,fill=-,flush=C,to=226/0}
at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.earlyEOF(HttpReceiverOverHTTP.java:338)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1551)
at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.shutdown(HttpReceiverOverHTTP.java:209)
at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:147)
at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:73)
at org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:133)
at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:155)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
It looks like the server closes without answering to the handshake request.
I am aware of SslContextFactory, but my understanding is that it should be used only if you need your own TrustStore or KeyStore, or for other special cases.
Note also that after some failed attempt I downloaded another websocket implementation from https://github.com/TooTallNate/Java-WebSocket, and it works fine with both ws and wss, without setting anything specific for SSL.
However for this project I'm bind to use Jetty.
The code I'm using is exactly the sample from Jetty documentation at https://www.eclipse.org/jetty/documentation/9.4.x/jetty-websocket-client-api.html
the only change I did was to add an onError method to SimpleEchoSocket, that dumps the full exception stack.
Am I missing something?
Thanks in advance!

Unfortunately, websocket.org (and Kaazing host/proxy) has a bunch of TLS issues at this point in time, so using their public servers are not sane choice right now.
Here's a different demo, also using TLS and WebSocket, against a stackexchange server with a proper and sane TLS/SSL implementation.
This was written against Jetty 9.4.18.v20190429
package org.eclipse.jetty.demo;
import java.net.URI;
import java.util.concurrent.Future;
import org.eclipse.jetty.client.HttpClient;
import org.eclipse.jetty.util.component.LifeCycle;
import org.eclipse.jetty.util.log.Log;
import org.eclipse.jetty.util.log.Logger;
import org.eclipse.jetty.util.ssl.SslContextFactory;
import org.eclipse.jetty.websocket.api.Session;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketClose;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketConnect;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketError;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketMessage;
import org.eclipse.jetty.websocket.api.annotations.WebSocket;
import org.eclipse.jetty.websocket.client.WebSocketClient;
#WebSocket
public class SecureClientSocket
{
private static final Logger LOG = Log.getLogger(SecureClientSocket.class);
public static void main(String[] args)
{
String url = "wss://qa.sockets.stackexchange.com/";
SslContextFactory ssl = new SslContextFactory.Client();
ssl.setEndpointIdentificationAlgorithm("HTTPS");
HttpClient http = new HttpClient(ssl);
WebSocketClient client = new WebSocketClient(http);
try
{
http.start();
client.start();
SecureClientSocket socket = new SecureClientSocket();
Future<Session> fut = client.connect(socket, URI.create(url));
Session session = fut.get();
session.getRemote().sendString("Hello");
session.getRemote().sendString("155-questions-active");
}
catch (Throwable t)
{
LOG.warn(t);
}
finally
{
stop(http);
stop(client);
}
}
private static void stop(LifeCycle lifeCycle)
{
try
{
lifeCycle.stop();
}
catch (Exception e)
{
e.printStackTrace();
}
}
#OnWebSocketConnect
public void onConnect(Session sess)
{
LOG.info("onConnect({})", sess);
}
#OnWebSocketClose
public void onClose(int statusCode, String reason)
{
LOG.info("onClose({}, {})", statusCode, reason);
}
#OnWebSocketError
public void onError(Throwable cause)
{
LOG.warn(cause);
}
#OnWebSocketMessage
public void onMessage(String msg)
{
LOG.info("onMessage() - {}", msg);
}
}

Apache Nifi Processor for Signature Verification

I have developed a JAVA application for signature verification as follows:
package read_key_pck;
import static java.nio.charset.StandardCharsets.UTF_8;
import java.util.Scanner;
import javax.xml.bind.DatatypeConverter;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.security.KeyFactory;
import java.security.NoSuchAlgorithmException;
import java.security.NoSuchProviderException;
import java.security.PrivateKey;
import java.security.PublicKey;
import java.security.Security;
import java.security.Signature;
import java.security.spec.InvalidKeySpecException;
import java.security.spec.PKCS8EncodedKeySpec;
import java.security.spec.X509EncodedKeySpec;
import java.util.Base64;
import org.bouncycastle.jce.provider.BouncyCastleProvider;
public class Main {
public final static String RESOURCES_DIR = "C:\\Users\\KX5710\\eclipse-workspace\\read_key\\src\\read_key_pck\\";
public static boolean verify(String plainText, String signature, PublicKey publicKey) throws Exception {
Signature publicSignature = Signature.getInstance("SHA256withRSA");
publicSignature.initVerify(publicKey);
publicSignature.update(plainText.getBytes(UTF_8));
byte[] signatureBytes = DatatypeConverter.parseHexBinary(signature);
return publicSignature.verify(signatureBytes);
}
public static void main(String[] args) throws Exception {
Security.addProvider(new BouncyCastleProvider());
PublicKey pub = null;
KeyFactory factory = KeyFactory.getInstance("RSA", "BC");
try {
pub = generatePublicKey(factory, RESOURCES_DIR
+ "rsa_2048_pub.pem");
} catch (InvalidKeySpecException e) {
e.printStackTrace();
}
String encodedPublicKey = Base64.getEncoder().encodeToString(pub.getEncoded());
System.out.println("publickey: " + encodedPublicKey);
boolean isCorrect = verify(" 4 5.00 1.80", "54c5645616a8567d605bd990df0456913420fb630860316cf29bf57e19ef3102933ac948e672a4e3dfa8b7e20cc9eb44520aa8d8dc69143a5bc718a17a1d3f2dfc0084f172dbda6dd38e411e75100136d95d46e3d563c3a0aed062c40c6d17a5038ef0ad681622245bd2332008b1edca693b3418df1f5f86f09a6585d4af7da0d2ab44f5ce1c7ec8eed51696697b2674a881868be7163eda7c6562d02bdf069fcd66d483e8b22c5c976d27c04ffe38e62800eec17f1786b2f2161fcf44b91ba85a4486edb48a5c5a2f58dc1fa3969917eadabd7af753947de4a7f03034de09adb3b1cacb77e21444eee2d90944037606f84d0aa99f4794e8df8a2d0fbfff55c7", pub);
System.out.println("Signature verification: " + isCorrect);
}
private static PublicKey generatePublicKey(KeyFactory factory,
String filename) throws InvalidKeySpecException,
FileNotFoundException, IOException {
PemFile pemFile = new PemFile(filename);
byte[] content = pemFile.getPemObject().getContent();
X509EncodedKeySpec pubKeySpec = new X509EncodedKeySpec(content);
return factory.generatePublic(pubKeySpec);
}
}
package read_key_pck;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStreamReader;
import org.bouncycastle.util.io.pem.PemObject;
import org.bouncycastle.util.io.pem.PemReader;
public class PemFile {
private PemObject pemObject;
public PemFile(String filename) throws FileNotFoundException, IOException {
PemReader pemReader = new PemReader(new InputStreamReader(
new FileInputStream(filename)));
try {
this.pemObject = pemReader.readPemObject();
} finally {
pemReader.close();
}
}
public PemObject getPemObject() {
return pemObject;
}
}
Now I want to develop an Apache Nifi Processor for this JAVA application; I have a sample code here :
https://github.com/pcgrenier/nifi-examples
However I do not know that I have to modify which parts of this sample code, since it is the first time i develop an Apache Nifi Processor.
Also When I want to run .bat file in cmd i.e. run-nifi.bat I get this error :
"The JAVA_HOME environment variable is not defined correctly. Instead
the PATH will be used to find the java executable."

Here is detailed guide for beginners https://community.hortonworks.com/articles/4318/build-custom-nifi-processor.html .
The above article is based on this video https://www.youtube.com/watch?v=3ldmNFlelhw
Briefly explanation:
You should create the new project from one of the NiFi maven archetype.
Modify generated processor to your needs.
Build the nar file and put it to NiFi.
Use it in NiFi.

Great that you're writing your own custom processor on NiFi. Couple of things that you have to keep in mind:
The core functionality that your processor intends to do goes inside the overridden onTrigger method.
Handling relationships of your processor (Failure/Success)
Couple of guides that would help in your quest:
http://www.nifi.rocks/developing-a-custom-apache-nifi-processor-json/
https://bryanbende.com/development/2015/02/04/custom-processors-for-apache-nifi
The first one provides you some overview on common terminologies & functionalities in the form of annotation and classes that are widely used in NiFi processors and also provide relevant links to the same on Apache NiFi developer guide. That tutorial itself should be able to get you started with the actual implementation since it has examples in the form of code and POM.

Java ssl performance issue in connection with downloads

i'm currently working on a small swift-client and got some hard performance issues by downloading files.
After thousands of checks i localized the problem by downloading files through ssl.
Normal downloads(http) works fine without any problems(or performance issues).
But SSL downloads blow up my CPU ... a single core goes up to 100 % load (for a single thread)
i wrote a small testclass without my entire program and can confirm my previous observations.
package testDownload;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.URL;
import java.net.URLConnection;
import java.nio.ByteBuffer;
import java.nio.channels.Channels;
import java.nio.channels.ReadableByteChannel;
import java.nio.channels.WritableByteChannel;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.X509Certificate;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;
public class Downloader {
public Downloader(boolean sslTrigger) throws IOException {
try {
this.allowSelfSignCerts();
} catch (KeyManagementException | NoSuchAlgorithmException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
URL url;
// Http Download
if (sslTrigger) {
System.out.println("HTTPS-Mode");
url = new URL("HTTPS LINK");
} else {
System.out.println("HTTP-Mode");
url = new URL("HTTP LINK");
}
URLConnection conn = url.openConnection();
InputStream input = conn.getInputStream();
OutputStream output = new FileOutputStream(new File(System.getProperty("user.home")+System.getProperty("file.separator")+"test.dat"));
ReadableByteChannel inputChannel = Channels.newChannel(input);
WritableByteChannel outputChannel = Channels.newChannel(output);
System.out.println("start download");
this.fastChannelCopy(inputChannel, outputChannel);
inputChannel.close();
outputChannel.close();
System.out.println("finish");
}
private void fastChannelCopy(ReadableByteChannel src, WritableByteChannel dest) throws IOException {
ByteBuffer buffer = ByteBuffer.allocateDirect(16 * 1024);
while (src.read(buffer) != -1) {
buffer.flip();
dest.write(buffer);
buffer.compact();
}
buffer.flip();
while (buffer.hasRemaining()) {
dest.write(buffer);
}
}
private void allowSelfSignCerts() throws NoSuchAlgorithmException, KeyManagementException {
TrustManager[] trustAllCerts = new TrustManager[] {new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(X509Certificate[] certs, String authType) {
}
}
};
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, new java.security.SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
HostnameVerifier allHostsValid = new HostnameVerifier() {
public boolean verify(String hostname, SSLSession session) {
return true;
}
};
HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);
}
}
used java-version
java -version
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
Has anyone an idea for solving this issue?

Normal downloads(http) works fine without any problems(or performance issues). But SSL downloads blow up my CPU ... a single core goes up to 100 % load (for a single thread)
This might depend on the cipher client and server agreed on. Very common are AES ciphers. But while AES is fast with special hardware and it terrible slow and CPU bound when done in software. Historically hardware acceleration was not available for Java. But it looks like they've added it in Java 8, even though only for the server JVM.
For more information see AES acceleration for Java and AES-NI intrinsics enabled by default?.

How to check if proxy is working in Java?

I searched google, this site and JavaRanch and I can not find an answer.
My program needs to obtain proxies from a selected file(I got that done using java gui FileChooser class and RandomAccessFile)
Then I need to verify the proxies starting with the one that is first in the txt file. It will try to connect to some site or port to verify if the connection was successful.If the connection was successful (I got a positive response) it will add the proxy to a list of proxies and then get and check next one in the list until it is done.
I know how to do this but I got a little problem. My Problem is that this process needs to be independent of connection speed because someone may set 15000(milliseconds) timeout for the connection to be dealt with and set 100 threads and then none of the proxies would come out working because connection is too slow.
I heard of a method called pinging to check proxies,but I do not know how to use it in java.
Could anyone give me solution or at least classes I could use.

Ok I found a solution and it is easy.
What I used it InetAddress.isReachable() method along with some HttpClient by Apache. For proxy checking I used blanksite.com because all I need is check connectability and not speed of proxies.
So here is the code(Including input from file, but it is not gui, YET):
/* compile with
java -cp .;httpclient-4.5.1.jar;httpcore-4.4.3.jar ProxyMat
run with
java -cp .;httpclient-4.5.1.jar;httpcore-4.4.3.jar;commons-logging-1.2.jar ProxyMat
put one proxy to check per line in the proxies.txt file in the form
some.host.com:8080
*/
import java.io.File;
import java.io.FileNotFoundException;
import java.io.RandomAccessFile;
import java.net.InetAddress;
import org.apache.http.params.CoreConnectionPNames;
import org.apache.http.HttpEntity;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.conn.params.ConnRoutePNames;
import org.apache.http.impl.client.DefaultHttpClient;
public class ProxyMat{
File file=null;
static RandomAccessFile read=null;
public ProxyMat(){
file=new File("proxies.txt");
try {
read=new RandomAccessFile(file,"rw");
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
public void checkproxies(){
try{
String line;
for(int i=0;i<25;i++){
if((line=read.readLine())!=null){
System.out.println(line);
String[] hp=line.split(":");
InetAddress addr=InetAddress.getByName(hp[0]);
if(addr.isReachable(5000)){
System.out.println("reached");
ensocketize(hp[0],Integer.parseInt(hp[1]));
}
}
}
}catch(Exception ex){ex.printStackTrace();}
}
public void ensocketize(String host,int port){
try{
File pros=new File("working.txt");
HttpClient client=new DefaultHttpClient();
HttpGet get=new HttpGet("http://blanksite.com/");
HttpHost proxy=new HttpHost(host,port);
client.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
client.getParams().setParameter(CoreConnectionPNames.SO_TIMEOUT, 15000);
HttpResponse response=client.execute(get);
HttpEntity enti=response.getEntity();
if(response!=null){
System.out.println(response.getStatusLine());
System.out.println(response.toString());
System.out.println(host+":"+port+" ## working");
}
}catch(Exception ex){System.out.println("Proxy failed");}
}
public static void main(String[] args){
ProxyMat mat=new ProxyMat();
mat.checkproxies();
}
}