I'm writing a standalone application that implements a Web Service, for which the Endpoint is published using the embedded Sun HttpServer. I have an odd issue with this, where in a specific deployment situation, there is an apparent delay between the server processing /sending the reply and the client receiving the reply.
Let me give a few scenarios:
Case 1) Working: server is running inside Eclipse, which uses OpenJDK 1.6.0_23 as runtime. Client is implemented with axis (not axis2!) and is running on Solaris x86 inside JBoss (must admit I don't know the exact Java version used, but I suspect a Java 5 version).
Case 2) Working: server is running on Solaris x86 with java 1.6.0_26, client is running inside Eclipse with OpenJDK 1.6.0_23.
Case 3) Not working: server is running on Solaris x86 with java 1.6.0_26, client is on Solaris x86 with axis on Solaris x86 (again, suspect it's Java 5, not 6).
I'm wondering if I could be suffering from the following Java bug, which is fixed in 1.6.0_30 (assuming that OpenJDK 1.6.0_xx does not suffer the same bug)?
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068416
But if that's the case, then why would case 2 work? Can the client somehow control the TCP_NODELAY on the server side?
On the exact delays that i have observed: I have 2 web services, published on different contexts. Eg 2 different WSDLs. The client has (obviously) separate (axis 1) bindings for each service. For one service, I see a consistent delay of exactly 150 seconds, for the other service the delay is consistently 300 seconds. Do these values ring a bell to anybody?
Maarten
Edit
I am now leaning towards the cause and solution in Eclipse Generated Web Service Client Extremely Slow. Can't test at the moment as I'm sitting in a hotel room without access to the system.
OK, managed to solve this, by telling Axis to use the CommonsHttpSender instead of the default HttpSender. Since the relevant application already had the prerequisite jars for that inside its WEB-INF/lib directory, that wasn't such a big deal.
To make Axis (1.4) use the CommonsHttpSender, create a "client-config.wsdd" file in the following location (note: this was the non-obvious part that caused me quite a few headaches):
MY.ear/MY.war/WEB-INF/classes/org/apache/axis/client/client-config.wsdd
with the following contents:
<?xml version="1.0" encoding="UTF-8"?>
<deployment
name="commonsHTTPConfig"
xmlns="http://xml.apache.org/axis/wsdd/"
xmlns:java="http://xml.apache.org/axis/wsdd/providers/java">
<!-- use CommonsHTTPSender instead of the default HTTPSender -->
<transport name="http" pivot="java:org.apache.axis.transport.http.CommonsHTTPSender" />
<transport name="local" pivot = "java:org.apache.axis.transport.local.LocalSender" />
<transport name="java" pivot="java:org.apache.axis.transport.java.JavaSender" />
</deployment>
Restart your application. After this change, Axis will use HTTP/1.1 to make web service calls, which seems to be all that was needed to correct this annoying delay. Seems that there is something in the HTTP protocol specs (or perhaps Axis' implementation) that doesn't like a HTTP/1.1 response to a HTTP/1.0 request.
Maarten
Related
I have a Producer Consumer based application based on Netty. The basic requirement was to build a message oriented middleware (MOM)
MOM
So the MOM is based on the concept of queuing (Queuing makes systems loosely coupled and that was the basic requirement of the application).
The broker understands the MQTT protocol. We performed stress testing of the application on our local machine. These are the specs of the local machine.
We were getting great results. However, our production server is AWS Ubuntu based. So we stress tested the same application on AWS Ubuntu server. The performance was 10X poor than the local system. This is the configuration of the AWS server.
We have tried the following options to figure out where the issue is.
Initially we checked for bugs in our business logic. Did not find any.
Made the broker, client and all other dependencies same on mac as well as aws. What I mean by same dependencies is that we installed the same versions on aws as on mac.
Increased the ulimit on AWS.
Played with sysctl settings.
We were using Netty 4.1 and we had a doubt that it might be a Netty error as we do not have stable release for Netty 4.1 yet. So we even built the entire application using Netty 3.9.8 Final (Stable) and we still faced the same issue.
Increased the hardware configurations substantially of the AWS machine.
Now we have literally run out of options. The java version is the same on both machines.
So the last resort for us is to build the entire application using NodeJS but that would require a lot of effort rather than tweaking something in Netty itself. We are not searching for Java based alternatives to Netty as we think this might even be a bug in JVM NIO's native implementation on Mac and Ubuntu.
What possible options can we try further to solve this bug. Is this a Netty inherent issue. Or is this something to do with some internal implementations on Mac and Ubuntu which are different and are leading to perfomance differences as we see them ?
EDIT
The stress testing parameters are as follows.
We had 1000 clients sending 1000 messages per second (Global rate).
We ran the test for about 10 minutes to note the latency.
On the server side we have 10 consumer threads handling the messages.
We have a new instance of ChannelHandler per client.
For boss pool and worker pool required by Netty, we used the Cached Thread pool.
We have tried tuning the consumer threads but to no avail.
Edit 2
These are the profiler results provided by jvmtop for one phase of load testing.
I have an web application from which multiple users loads a Java applet. Now there is a problem that the loadbalancer does not support TLS1.2 which is the default for Java8 and it seems that Java8 does not automatically try lower version.
How can I force the applet to be loaded using TLS 1.0/1.1? I have tried to put this into the <applet>:
<PARAM name="java_arguments" value="-Dhttps.protocols=TLSv1">
Any help is appreciated, not very keen on solution where hundreds of users need to configure their Java clients.
This the starting point from which this question was brought up:
Java applet not loading on Java8/HTTPS
I have an web application from which multiple users loads a Java applet... How can I force the applet to be loaded using TLS 1.0/1.1?
The applet is loaded by the browser, not by Java. So it does not help to make any Java related settings here. These settings are only relevant if the applet itself communicates with the server.
Edit: The download is done by the Java plugin. This does not affect the rest of the answer i.e. that the problem must be fixed at the load balancer.
Now there is a problem that the loadbalancer does not support TLS1.2
Unless the load balancer is broken it will negotiate to a lower protocol version. It is inherent behavior of TLS that both parties agree to the best version both support. But, there are broken load balancers out there which simply do not understand TLS1.2 or behave strange when confronted with larger packets which are more likely with TLS1.2 (older F5, long fixed).
Unfortunately, if this happens to be such an old broken F5 you might be out of luck because a bug in these load balancers caused the packet to be dropped, so that the connection would stay open until timeout. In this case most browsers do not downgrade to a lower TLS version, because they only downgrade on immediate errors like a connection close from the peer.
All you can do in this case is to fix the broken load balancer.
I have a soap client that calls a web service through ssl, when I add this line:
System.setProperty("java.protocol.handler.pkgs","com.sun.net.ssl.internal.www.protocol");
the client works with a rate of 15 calls per second, when removing it the speed goes down to 1.5 per second (10 times slower), I am using java 4 and tomcat 6 on a windows machine for my development environment
I'd be happy with this but when deploying the same code to oracle application server 10g on a unix machine the speed is always 1.5 per second weather I am setting the property or not!
Can any one figure out what is going on here?!
In Oracle App Server, try using oracle.mds.net.protocol instead of com.sun.net.ssl.internal.www.protocol. This is the value used by default in WebLogic, I have never used OAS so I can't advise.
I am facing a strange issue with apache MINA. I have server application which is based on apache Mina framework. I am using ExecutorFilter at the server side. the application works perfectly on my machine but when I am installing it on client's machine its "throwing exception".
Here is the error I am getting on client's machine:
java.lang.NoSuchMethodError: org.apache.mina.filter.executor.ExecutorFilter.<init>(II[Lorg/apache/mina/core/session/IoEventType;)V
This is how I am referring it in my code-
new ExecutorFilter(1, 100, IoEventType.MESSAGE_RECEIVED)
There are 2 differences I found between my system and client's machine -
Java Version ==> My 1.6.0_26 / Client's 1.6.0_24
Operating system ==> my XP SP3/ Client's Windows 2003 server SP 2
Can any one shed any light on any possible solution?
thanks in advance.
In my experience, a NoSuchMethodError (or NoClassDefFoundError, AbstractMethodError) at runtime almost always indicates a library version mismatch. Make sure your client uses the exact same Mina version, and make sure there does not happen to be a second version of the Mina JAR on the classpath confusing things.
In production we run tomcat behind apache, in development bare tomcat.
How can I know inside the Servlet class if it is running behind apache or not?
This depends on how your Apache is talking to Tomcat.
If it's connected via HTTP (mod_proxy), you can check request.getRemoteAddr(). It will be the IP of the Apache, probably your internal IP. You can also check "Via" header to see if Apache is there.
If it's connected via AJP (mod_jk/mod_proxy_ajp), you can check request.getLocalPort() to see if it's the port of your JK connector.
If this is to distinguish between development and production environment, then you will probably be better off by having this completely under your control instead of ad-hoc guessing (which will eventually break).
This could be "Is feature X set in JNDI?" or "Is property foo.bar set in c:/ourproject.properties". You should not rely on artifacts like "is class X loaded from a file or found inside a jar" (since that will break if you change application servers) or "Is http header line X present" since that is out of your control plus it may break if somebody else is using Apache as a frontend accellerator.
So, explicit configuration - it can be done quite easily :)