I am creating a test app to load test a comet servlet by creating a bunch of outbound https connections (each looping in its own thread).
Is there a limit on the number of concurrent outbound http requests? If so, is it an OS level issue (xp 32bit) or Java? I get the following exception when running around 100 connections. The error does not occur instantly but after anywhere between 50-150 loops (sometimes more), leading me to suspect that it actually might be me not releasing resources correctly:
java.net.BindException: Address already in use: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:559)
at com.sun.net.ssl.internal.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:141)
at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:272)
at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:329)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:172)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:916)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:158)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1177)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:234)
at fqexconnectiontesting.FQEXHttpClient.run(FQEXHttpClient.java:158)
at java.lang.Thread.run(Thread.java:662)
My code:
while(true){
try{
connection =(HttpsURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setRequestProperty("keep-alive", "true");
connection.setRequestMethod("GET");
input = new DataInputStream( connection.getInputStream() );
b = new byte[4096];
totalBytes += input.read(b);
input.close();
b = null;
connection.disconnect();
connection = null;
Thread.sleep(100);
}
Any ideas appreciated.
Thanks,
Dazz
By default Java opens only up to 5 persistent connections to the same host.
But your code uses the disconnect() method which AFAIK prevents persistent connections.
You should better read the documentation about Persistent Connections. Take the sample code near the end for getting a fully working persistent connection.
May be this fixes your problem.
Related
I have to work with Amazon MQ. Amazon MQ is based on ActiveMQ. I found some code, and it should put a blob message (PDF size 230kB) on a queue. But if I run the program it errors out in the error stack below.
This is my code:
private final static String WIRE_LEVEL_ENDPOINT = "ssl://<examplednsname>-1.amazonaws.com:61617";
private final static String ACTIVE_MQ_USERNAME = "test123";
private final static String ACTIVE_MQ_PASSWORD = "test123";
public static void sendFileViaQueue(String uri, String queueName) throws JMSException {
File file = new File("test.pdf");
ConnectionFactory connectionFactory = null;
Connection connection = null;
Session session = null;
BlobMessage blobMsg = null;
MessageProducer producer = null;
try {
connectionFactory = new ActiveMQConnectionFactory(ACTIVE_MQ_USERNAME, ACTIVE_MQ_PASSWORD, WIRE_LEVEL_ENDPOINT);
connection = connectionFactory.createConnection();
connection.start();
session = connection.createSession(Boolean.TRUE, Session.AUTO_ACKNOWLEDGE);
producer = session.createProducer(session.createQueue(queueName));
blobMsg = ((ActiveMQSession) session).createBlobMessage(file);
blobMsg.setStringProperty("FILE.NAME", file.getName());
blobMsg.setLongProperty("FILE.SIZE", file.length());
producer.send(blobMsg);
session.commit();
} finally {
closeQuietly(producer);
closeQuietly(session);
closeQuietly(connection);
}
}
It seems like it want something to upload to 8080 but I didn't configure anything locally. It only should upload a PDF to a queue thats it.
Has anybody an idea to fix this?
It shouldn't be that complicated just upload a blob to a queue.
This is the stack-trace I am getting:
javax.jms.JMSException: PUT failed to: http://localhost:8080/uploads/ID:bpSligro-PC-50920-1584558692848-1:1:1:1:1
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:72)
at org.apache.activemq.command.ActiveMQBlobMessage.onSend(ActiveMQBlobMessage.java:177)
at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1952)
at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:288)
at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:223)
at org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:241)
at nl.bpittens.mq.AmazonMQExample.sendFileViaQueue(AmazonMQExample.java:81)
at nl.bpittens.mq.AmazonMQExample.main(AmazonMQExample.java:52)
Caused by: java.io.IOException: PUT failed to: http://localhost:8080/uploads/ID:bpSligro-PC-50920-1584558692848-1:1:1:1:1
at org.apache.activemq.blob.DefaultBlobUploadStrategy.uploadStream(DefaultBlobUploadStrategy.java:67)
at org.apache.activemq.blob.DefaultBlobUploadStrategy.uploadFile(DefaultBlobUploadStrategy.java:44)
at org.apache.activemq.blob.BlobUploader.upload(BlobUploader.java:53)
at org.apache.activemq.command.ActiveMQBlobMessage.onSend(ActiveMQBlobMessage.java:174)
... 6 more
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
at sun.net.www.http.HttpClient.New(HttpClient.java:339)
at sun.net.www.http.HttpClient.New(HttpClient.java:357)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1220)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:984)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1334)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1309)
at org.apache.activemq.blob.DefaultBlobUploadStrategy.uploadStream(DefaultBlobUploadStrategy.java:60)
... 9 more
Sending a normal JMS TextMessage works without a problem.
As noted in the documentation, a "blob" message:
allows massive BLOBs (Binary Large OBjects) to be sent around in some out-of-band transport mechanism. Possible out-of-band mechanisms could be HTTP or FTP or SCP or some other point-to-point protocol.
Notice that the actual binary data must be sent "in some out-of-band transport mechanism." In other words, the blob doesn't actually go to the queue. The blob is uploaded somewhere else and the message that goes to the queue simply points to that location.
You need to configure the transfer policy using the jms.blobTransferPolicy.uploadUrl parameter on the client URL. The default upload URL of the default transfer policy is http://localhost:8080/uploads/ which is what your client is trying to use to upload the binary data.
If you want to send an arbitrarily large message directly to a queue rather than using some out of band mechanism consider moving to ActiveMQ Artemis which supports that functionality.
If you're stuck using Amazon MQ then I don't think you have any other solution other than some kind of manual solution where you break the file into smaller chunks that you can put into individual messages and then re-assemble those chunks later in the consuming application.
This question already has answers here:
What is a connection timeout during a http request
(2 answers)
Closed 27 days ago.
I get an exception when I run this code. Why?
Exception in thread "main" java.net.ConnectException: Connection timed out: connect
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;
public class MainClass {
public static void main(String[] args) throws Exception {
System.setProperty("java.protocol.handler.pkgs", "com.sun.net.ssl.internal.www.protocol");
URL url = new URL("https://www.verisign.com/");
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
String line;
while ((line = in.readLine()) != null) {
System.out.println(line);
}
in.close();
}
}
Exception:
Exception in thread "main" java.net.ConnectException: Connection timed out: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:525)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:550)
at com.sun.net.ssl.internal.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:141)
at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:272)
at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:329)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:172)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:801)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:158)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1049)
at com.sun.net.ssl.internal.www.protocol.https.HttpsURLConnectionOldImpl.getInputStream(HttpsURLConnectionOldImpl.java:204)
at java.net.URL.openStream(URL.java:1010)
at https.ssl.MainClass.main(MainClass.java:13)
We can't diagnose your networks for you. You need to do it yourself, or get your local admins to look at.
Things you should check before you bug your admins:
can you ping the host?
can you connect to http://www.verisign.com using a web browser?
can you connect to https://www.verisign.com using a web browser?
can you connect to http://www.verisign.com using your program?
can you connect to anything using your program?
The chances are that your problem is firewall related. My first guess would be that you don't have the correct environment variables or Java system properties set to tell the JVM to use a local proxy server for outgoing HTTP / HTTPS requests.
If it is not a problem with your settings, you will need to get help from someone local who can help you diagnose the problem.
Below is a simple code snippet that shows how to connect to a VoltDB server.
ClientConfig clientConfig = new ClientConfig();
Client client = ClientFactory.createClient(clientConfig);
String server = "192.168.43.32";
client.createConnection(server);
Based on my experiments, if the server is down or just not connectable from network layer, it will take about 75 seconds to get the response.
SEVERE: Failed to connect to 192.168.43.32, in 75,359 ms
java.net.ConnectException: Operation timed out
at sun.nio.ch.Net.connect0(Native Method)
at sun.nio.ch.Net.connect(Net.java:458)
at sun.nio.ch.Net.connect(Net.java:450)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
at org.voltdb.client.ConnectionUtil.getAuthenticatedConnection(ConnectionUtil.java:154)
at org.voltdb.client.ConnectionUtil.getAuthenticatedConnection(ConnectionUtil.java:142)
at org.voltdb.client.ConnectionUtil.getAuthenticatedConnection(ConnectionUtil.java:134)
at org.voltdb.client.Distributer.createConnectionWithHashedCredentials(Distributer.java:878)
at org.voltdb.client.ClientImpl.createConnectionWithHashedCredentials(ClientImpl.java:189)
at org.voltdb.client.ClientImpl.createConnection(ClientImpl.java:682)
at src.java.tutorial.voltdb.integration.ConnectionTest.main(ConnectionTest.java:27)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
Is there any ways to set the time out time, so the application needs not to wait for such a long time. A successful connection normally takes just tens of milliseconds, so I think if the connection cannot be established within 1000 milliseconds, something is definitely wrong already.
I have tried the setting of below
clientConfig.setConnectionResponseTimeout(1000);
In this case, it has no effects at all. So I guess it is not for this purpose.
Normally when the database is down and your client tries to connect it will get an immediate Connection refused exception, for example:
Exception in thread "main" java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect0(Native Method)
at sun.nio.ch.Net.connect(Net.java:364)
at sun.nio.ch.Net.connect(Net.java:356)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:623)
at java.nio.channels.SocketChannel.open(SocketChannel.java:184)
at org.voltdb.client.ConnectionUtil.getAuthenticatedConnection(ConnectionUtil.java:165)
at org.voltdb.client.ConnectionUtil.getAuthenticatedConnection(ConnectionUtil.java:153)
at org.voltdb.client.ConnectionUtil.getAuthenticatedConnection(ConnectionUtil.java:145)
at org.voltdb.client.Distributer.createConnectionWithHashedCredentials(Distributer.java:890)
at org.voltdb.client.ClientImpl.createConnectionWithHashedCredentials(ClientImpl.java:191)
at org.voltdb.client.ClientImpl.createConnection(ClientImpl.java:684)
at benchmark.Benchmark.<init>(Benchmark.java:17)
at benchmark.Benchmark.main(Benchmark.java:78)
In general, a "java.net.ConnectException: Connection timed out" can occur if there is a firewall that prevents the client from receiving any sort of response, or there could be other causes. The first thing to check might be if you have any firewall or network settings that would prevent access to port 21212 (the default VoltDB database connection port).
The ClientConfig setConnectionResponseTimeout() setting is used to cause a live connection to be closed if it hasn't received a response from a procedure call or a ping for the given number of milliseconds, but it is not used for creating a new connection.
If I set a socket SoTimeout, and read from it. when read time exceed the timeout limit, I'll get an "SocketTimeoutException: Read timed out".
and here is the stack in my case:
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:150)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:277)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
at java.io.DataInputStream.readInt(DataInputStream.java:387)
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:527)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:462)
but here I encountered "IOExcetion: Connection timed out", i don't know how it happened.
Stacks:
java.io.IOException: Connection timed out
at sun.nio.ch.FileDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:198)
at sun.nio.ch.IOUtil.read(IOUtil.java:171)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:245)
at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:277)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
at java.io.DataInputStream.readInt(DataInputStream.java:370)
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:527)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:462)
Can someone tell me what's the differences between the two exceptions, Thanks.
A connection timeout means you attempted to connect to the remote IP/port pair and failed to do so: it did not answer at all. Another possible error at that stage would be connection refused, in which this pair is available but rejected your connection attempt. Both of these errors appear on the initial setup of a socket. Note that these errors only occur with TCP, since a TCP connection requires the establishment of a session.
When you have a socket read timeout, it means you are connected, but failed to read data in time. Timeouts on sockets are configurable. You may also get a connection reset error, which means you did connect successfully, but the other end decided that after all you're not worth it :p
Simple answer:
In one case (Connection timed out) your application cannot connect to the server in a timely manner. In the other case (Read timed out) the connection can be established but during read the connection times out.
'Connection timed out' after the connect phase means that something has gone seriously wrong with the connection and it must be closed. 'Read timeout' just means that no data arrived within the specified receive timeout period: it isn't fatal.
Sample of the code:
ServiceTemplate tmp1 = new ServiceTemplate(null, classes, null);
try {
/* if(System.getSecurityManager() == null)
System.setSecurityManager(new RMISecurityManager());*/
thisIp = InetAddress.getLocalHost();
LookupLocator locator = new LookupLocator("jini://"+thisIp.getHostName().toString());
ServiceRegistrar sr = locator.getRegistrar();
JavaSpace space = (JavaSpace)sr.lookup(tmp1);
System.out.println("Success");
}
Encountering error at [ ServiceRegistrar sr = locator.getRegistrar();]
which states
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:519)
at java.net.Socket.connect(Socket.java:469)
at java.net.Socket.<init>(Socket.java:366)
at java.net.Socket.<init>(Socket.java:209)
at net.jini.core.discovery.LookupLocator.getRegistrar(LookupLocator.java:328)
at net.jini.core.discovery.LookupLocator.getRegistrar(LookupLocator.java:286)
at MessageEntry.main(MessageEntry.java:34)
I am new to JavaSpace tech, kindly assist me.
Thank You in Advance
Jeetesh.N
The ConnectException is a quite fundamental error which simply means that your Java process could not establish a socket connection to the target machine and port, because that machine wasn't accepting connections on the target port. In this case it's almost certainly trying to connect to your localhost IP address using the default JINI port.
I'd suggest that this error means you don't have a JINI service/registrar running on your machine at the time you executed the code.