Resetting heartbeat timestamps because of huge system clock jump - java

I am running hazelcast application, and I am getting below error, after keeping my machine in sleep mode/log off for sometime.
2021-06-21 14:41:07.854 INFO 8288 --- [cached.thread-7] c.h.i.c.impl.ClusterHeartbeatManager
: [192.168.181.51]:5701 [APP] [4.2] System clock apparently jumped from 2021-06-21 14:10:28.569 to 2021-06-21 14:41:07.832 since last heartbeat (+1834263 ms)
2021-06-21 14:41:07.855 INFO 8288 --- [cached.thread-9] c.h.i.server.tcp.TcpServerConnection : [192.168.181.51]:5701 [APP] [4.2] Connection[id=1, /127.0.0.1:5701->/127.0.0.1:5702, qualifier=null, endpoint=[127.0.0.1]:5702, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Client heartbeat is timed out, closing connection to Connection[id=1, /127.0.0.1:5701->/127.0.0.1:5702, qualifier=null, endpoint=[127.0.0.1]:5702, alive=true, connectionType=JVM, planeIndex=-1]. Now: 2021-06-21 14:41:07.833. LastTimePacketReceived: 2021-06-21 14:10:29.314
2021-06-21 14:41:07.915 WARN 8288 --- [cached.thread-7] c.h.i.c.impl.ClusterHeartbeatManager : [192.168.181.51]:5701 [APP] [4.2] Resetting heartbeat timestamps because of huge system clock jump! Clock-Jump: 1834263 ms, Heartbeat-Timeout: 60000 ms
2021-06-21 14:41:08.208 WARN 8288 --- [onMonitorThread] c.h.s.i.o.impl.InvocationMonitor : [192.168.181.51]:5701 [APP] [4.2] MonitorInvocationsTask delayed 1836451 ms
2021-06-21 14:41:08.213 WARN 8288 --- [onMonitorThread] c.h.s.i.o.impl.InvocationMonitor : [192.168.181.51]:5701 [APP] [4.2] BroadcastOperationControlTask delayed 1834623 ms
2021-06-21 14:41:08.539 INFO 8288 --- [cached.thread-9] c.h.i.server.tcp.TcpServerConnection : [192.168.181.51]:5701 [APP] [4.2] Connection[id=2, /127.0.0.1:5701->/127.0.0.1:5703, qualifier=null, endpoint=[127.0.0.1]:5703, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Client heartbeat is timed out, closing connection to Connection[id=2, /127.0.0.1:5701->/127.0.0.1:5703, qualifier=null, endpoint=[127.0.0.1]:5703, alive=true, connectionType=JVM, planeIndex=-1]. Now: 2021-06-21 14:41:08.539. LastTimePacketReceived: 2021-06-21 14:10:29.949
2021-06-21 14:41:08.551 WARN 8288 --- [ached.thread-36] c.h.i.cluster.impl.MulticastService : [192.168.181.51]:5701 [APP] [4.2] Sending multicast datagram failed. Exception message saying the operation is not permitted usually means the underlying OS is not able to send packets at a given pace. It can be caused by starting several hazelcast members in parallel when the members send their join message nearly at the same time.
java.net.NoRouteToHostException: No route to host: Datagram send failed
at java.net.TwoStacksPlainDatagramSocketImpl.send(Native Method) ~[na:1.8.0_251]
at java.net.DatagramSocket.send(Unknown Source) ~[na:1.8.0_251]
at com.hazelcast.internal.cluster.impl.MulticastService.send(MulticastService.java:291) ~[hazelcast-all-4.2.jar!/:4.2]
at com.hazelcast.internal.cluster.impl.MulticastJoiner.searchForOtherClusters(MulticastJoiner.java:113) [hazelcast-all-4.2.jar!/:4.2]
at com.hazelcast.internal.cluster.impl.SplitBrainHandler.searchForOtherClusters(SplitBrainHandler.java:75) [hazelcast-all-4.2.jar!/:4.2]
at com.hazelcast.internal.cluster.impl.SplitBrainHandler.run(SplitBrainHandler.java:42) [hazelcast-all-4.2.jar!/:4.2]
at com.hazelcast.spi.impl.executionservice.impl.DelegateAndSkipOnConcurrentExecutionDecorator$DelegateDecorator.run(DelegateAndSkipOnConcurrentExecutionDecorator.java:77) [hazelcast-all-4.2.jar!/:4.2]
at com.hazelcast.internal.util.executor.CachedExecutorServiceDelegate$Worker.run(CachedExecutorServiceDelegate.java:217) [hazelcast-all-4.2.jar!/:4.2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.8.0_251]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.8.0_251]
at java.lang.Thread.run(Unknown Source) [na:1.8.0_251]
at com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76) [hazelcast-all-4.2.jar!/:4.2]
at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) [hazelcast-all-4.2.jar!/:4.2]
My client config is as below:
ClientConfig clientConfig = new ClientConfig();
clientConfig.setClusterName("abc");
clientConfig.getNetworkConfig().addAddress("localhost");
clientConfig.getNetworkConfig().setSmartRouting(true);
clientConfig.getNetworkConfig().addOutboundPortDefinition("5701-5720");
ClientConnectionStrategyConfig connectionStrategyConfig = clientConfig.getConnectionStrategyConfig();
ConnectionRetryConfig connectionRetryConfig = connectionStrategyConfig.getConnectionRetryConfig();
connectionRetryConfig.setInitialBackoffMillis(1000)
.setMaxBackoffMillis(60000)
.setMultiplier(2)
.setClusterConnectTimeoutMillis(1000)
.setJitter(0.2);
HazelcastClient hc = HazelcastClient.newHazelcastClient(clientConfig);
Please let me know what wrong configuration am I doing or why this is happening?

This is a common issue for socket based applications. Ideally you disable the sleep / power save mode. You can try using the SystemParametersInfo API:
SystemParametersInfo( SPI_SETPOWEROFFACTIVE, 0, NULL, 0 );
But typically this would be seen as ill behaved since you should disable the Power Off during installation of your application thus requesting permission.

Related

Kafka on the server failed the next day。(EndOfStreamException: Unable to read additional data from client,)

The Kafka service I deployed on the server passed the test a few days ago and was running normally. Later, no other operations were carried out.
Then the service call failed today. I went online and looked at it. It's normal these days. Just this afternoon, the service hung up. Looking at the log, it seems to say that the Kafka service is restarted automatically, and then conflicts with the ID of zookeeper.
Restart the service. Everything is normal. Google doesn't have a good solution. What's the reason?
The following is the relevant log, please help to see, thank you
[2021-11-06 14:43:29,139] WARN Unexpected exception (org.apache.zookeeper.server.NIOServerCnxn)
EndOfStreamException: Unable to read additional data from client, it probably closed the socket: address = /159.89.4.236:59168, session = 0x0
at org.apache.zookeeper.server.NIOServerCnxn.handleFailedRead(NIOServerCnxn.java:163)
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:326)
at org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:522)
at org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:154)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2021-11-06 15:00:18,301] INFO Completed load of Log(dir=/tmp/kafka-logs/__consumer_offsets-46, topicId=dPs0Kq4ERHKDaR440Nt8qA, topic=__consumer_offsets, partition=46, highWatermark=0, lastStableOffset=0, logStartOffset=0, logEndOffset=0) with 1 segments in 20ms (53/53 loaded in /tmp/kafka-logs) (kafka.log.LogManager)
[2021-11-06 15:00:18,303] INFO Loaded 53 logs in 2350ms. (kafka.log.LogManager)
[2021-11-06 15:00:18,303] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2021-11-06 15:00:18,304] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2021-11-06 15:00:18,838] INFO [BrokerToControllerChannelManager broker=0 name=forwarding]: Starting (kafka.server.BrokerToControllerRequestThread)
[2021-11-06 15:00:19,148] INFO Updated connection-accept-rate max connection creation rate to 2147483647 (kafka.network.ConnectionQuotas)
[2021-11-06 15:00:19,165] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
[2021-11-06 15:00:19,237] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Created data-plane acceptor and processors for endpoint : ListenerName(PLAINTEXT) (kafka.network.SocketServer)
[2021-11-06 15:00:19,282] INFO [BrokerToControllerChannelManager broker=0 name=alterIsr]: Starting (kafka.server.BrokerToControllerRequestThread)
[2021-11-06 15:00:19,400] INFO [ExpirationReaper-0-DeleteRecords]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2021-11-06 15:00:19,400] INFO [ExpirationReaper-0-Fetch]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2021-11-06 15:00:19,400] INFO [ExpirationReaper-0-Produce]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2021-11-06 15:00:19,409] INFO [ExpirationReaper-0-ElectLeader]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2021-11-06 15:00:19,451] INFO [LogDirFailureHandler]: Starting (kafka.server.ReplicaManager$LogDirFailureHandler)
[2021-11-06 15:00:19,530] INFO Creating /brokers/ids/0 (is it secure? false) (kafka.zk.KafkaZkClient)
[2021-11-06 15:00:19,566] ERROR Error while creating ephemeral at /brokers/ids/0, node already exists and owner '72057654275276801' does not match current session '72057654275276802' (kafka.zk.KafkaZkClient$CheckedEphemeral)
[2021-11-06 15:00:19,572] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists
at org.apache.zookeeper.KeeperException.create(KeeperException.java:126)
at kafka.zk.KafkaZkClient$CheckedEphemeral.getAfterNodeExists(KafkaZkClient.scala:1904)
at kafka.zk.KafkaZkClient$CheckedEphemeral.create(KafkaZkClient.scala:1842)
at kafka.zk.KafkaZkClient.checkedEphemeralCreate(KafkaZkClient.scala:1809)
at kafka.zk.KafkaZkClient.registerBroker(KafkaZkClient.scala:96)
at kafka.server.KafkaServer.startup(KafkaServer.scala:319)
at kafka.Kafka$.main(Kafka.scala:109)
at kafka.Kafka.main(Kafka.scala)
[2021-11-06 15:00:19,596] INFO [KafkaServer id=0] shutting down (kafka.server.KafkaServer)
[2021-11-06 15:00:19,597] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Stopping socket server request processors (kafka.network.SocketServer)```

Hazelcast client is not reconnecting

In my application I using Hazelcast two clients and one server, everything till now is working properly but when machine goes to sleep and after starting machine clients tries to reconnect to the server but one client get connected and second tries to connect 2 times and fails:
Received auth from Connection[id=3, /192.168.0.6:5701->/192.168.0.6:5704, qualifier=null, endpoint=[192.168.0.6]:5704, alive=true, connectionType=JVM, planeIndex=-1], successfully authenticated, clientUuid: e99c4f2e-efe2-4bf7-93f8-1910ffdcf241, client version: 4.2
2021-07-12 17:09:37.356 INFO 8060 --- [ration.thread-3] c.h.c.i.p.t.AuthenticationMessageTask : [192.168.0.6]:5701 [APP] [4.2] Received auth from Connection[id=4, /192.168.0.6:5701->/192.168.0.6:5705, qualifier=null, endpoint=[192.168.0.6]:5705, alive=true, connectionType=JVM, planeIndex=-1], successfully authenticated, clientUuid: e99c4f2e-efe2-4bf7-93f8-1910ffdcf241, client version: 4.2
[{taskList=[{recurid=17, timeschedulerid=57, status=RUNNING}, {recurid=21, timeschedulerid=61, status=RUNNING}], taskcount=2, type=worker, uniqueid=4bbcda77-3286-4828-939a-a277ddda46a2}, {taskList=[{recurid=18, timeschedulerid=58, status=RUNNING}], taskcount=1, type=worker, uniqueid=e99c4f2e-efe2-4bf7-93f8-1910ffdcf241}]
2021-07-12 17:09:37.396 INFO 8060 --- [.IO.thread-in-0] c.h.i.server.tcp.TcpServerConnection : [192.168.0.6]:5701 [APP] [4.2] Connection[id=4, /192.168.0.6:5701->/192.168.0.6:5705, qualifier=null, endpoint=[192.168.0.6]:5705, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the other side
[{taskList=[{recurid=17, timeschedulerid=57, status=RUNNING}, {recurid=21, timeschedulerid=61, status=RUNNING}], taskcount=2, type=worker, uniqueid=4bbcda77-3286-4828-939a-a277ddda46a2}, {taskList=[{recurid=18, timeschedulerid=58, status=RUNNING}], taskcount=1, type=worker, uniqueid=e99c4f2e-efe2-4bf7-93f8-1910ffdcf241}]
Transfer Size
0
2021-07-12 17:09:37.703 WARN 8060 --- [.IO.thread-in-1] c.h.i.server.tcp.TcpServerConnection : [192.168.0.6]:5701 [APP] [4.2] Connection[id=2, /127.0.0.1:5701->/127.0.0.1:5703, qualifier=null, endpoint=[127.0.0.1]:5703, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Exception in Connection[id=2, /127.0.0.1:5701->/127.0.0.1:5703, qualifier=null, endpoint=[127.0.0.1]:5703, alive=true, connectionType=JVM, planeIndex=-1], thread=hz.APP1.IO.thread-in-1
java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.SocketDispatcher.read0(Native Method) ~[na:1.8.0-262]
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43) ~[na:1.8.0-262]
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[na:1.8.0-262]
at sun.nio.ch.IOUtil.read(IOUtil.java:197) ~[na:1.8.0-262]
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:377) ~[na:1.8.0-262]
at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:119) ~[hazelcast-all-4.2.jar!/:4.2]
at com.hazelcast.internal.networking.nio.NioThread.processSelectionKey(NioThread.java:383) [hazelcast-all-4.2.jar!/:4.2]
at com.hazelcast.internal.networking.nio.NioThread.processSelectionKeys(NioThread.java:368) [hazelcast-all-4.2.jar!/:4.2]
at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:294) [hazelcast-all-4.2.jar!/:4.2]
at com.hazelcast.internal.networking.nio.NioThread.executeRun(NioThread.java:249) [hazelcast-all-4.2.jar!/:4.2]
at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) [hazelcast-all-4.2.jar!/:4.2]
Why it is not reconnecting and getting closed and why it is trying to connect twice

Mongodb Atlas "Got Socket exception on Connection To Cluster"

I'm using Java & Springboot and MongoDB Atlas and created a database which response to many Object's CURD
When I do the post on uploadingImage, I got this error Got Socket exception on Connection [connectionId{localValue:4, serverValue:114406}] to cluster0-shard-00-02.1c6kg.mongodb.net:27017
However when I call other object's CRUD, it works totally fine. I don't why it raise this exception. BTW all my CRUD operation of all objects works well on localhost when not connecting to MongoDB Atlas, That means my ImageDAO should be fine, I just used mongoTemplate.insert(Image).
I search online, and they said might be IP whitelist of Atlas, So I setup my Cluster open to any IP Address.
Also I set my timeout and socket configure like this in my .properties file:
spring.data.mongodb.uri=mongodb+srv://username:password#cluster0.1c6kg.mongodb.net/database?retryWrites=true&w=majority&keepAlive=true&pooSize=30&autoReconnect=true&socketTimeoutMS=361000000&connectTimeoutMS=3600000
it still not work, I think the problem definitely related to the timeout of socket, But I don't know where else I can config
The Error log is here:
2020-11-01 12:25:34.275 WARN 20242 --- [nio-8088-exec-1] org.mongodb.driver.connection : Got socket exception on connection [connectionId{localValue:4, serverValue:114406}] to cluster0-shard-00-02.1c6kg.mongodb.net:27017. All connections to cluster0-shard-00-02.1c6kg.mongodb.net:27017 will be closed.
2020-11-01 12:25:34.283 INFO 20242 --- [nio-8088-exec-1] org.mongodb.driver.connection : Closed connection [connectionId{localValue:4, serverValue:114406}] to cluster0-shard-00-02.1c6kg.mongodb.net:27017 because there was a socket exception raised by this connection.
2020-11-01 12:25:34.295 INFO 20242 --- [nio-8088-exec-1] org.mongodb.driver.cluster : No server chosen by WritableServerSelector from cluster description ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=cluster0-shard-00-00.1c6kg.mongodb.net:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=8, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=46076648, setName='atlas-d9ovwb-shard-0', canonicalAddress=cluster0-shard-00-00.1c6kg.mongodb.net:27017, hosts=[cluster0-shard-00-00.1c6kg.mongodb.net:27017, cluster0-shard-00-01.1c6kg.mongodb.net:27017, cluster0-shard-00-02.1c6kg.mongodb.net:27017], passives=[], arbiters=[], primary='cluster0-shard-00-02.1c6kg.mongodb.net:27017', tagSet=TagSet{[Tag{name='nodeType', value='ELECTABLE'}, Tag{name='provider', value='AWS'}, Tag{name='region', value='US_WEST_2'}, Tag{name='workloadType', value='OPERATIONAL'}]}, electionId=null, setVersion=1, lastWriteDate=Sun Nov 01 12:25:29 PST 2020, lastUpdateTimeNanos=104428017411386}, ServerDescription{address=cluster0-shard-00-02.1c6kg.mongodb.net:27017, type=UNKNOWN, state=CONNECTING}, ServerDescription{address=cluster0-shard-00-01.1c6kg.mongodb.net:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=8, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=41202444, setName='atlas-d9ovwb-shard-0', canonicalAddress=cluster0-shard-00-01.1c6kg.mongodb.net:27017, hosts=[cluster0-shard-00-00.1c6kg.mongodb.net:27017, cluster0-shard-00-01.1c6kg.mongodb.net:27017, cluster0-shard-00-02.1c6kg.mongodb.net:27017], passives=[], arbiters=[], primary='cluster0-shard-00-02.1c6kg.mongodb.net:27017', tagSet=TagSet{[Tag{name='nodeType', value='ELECTABLE'}, Tag{name='provider', value='AWS'}, Tag{name='region', value='US_WEST_2'}, Tag{name='workloadType', value='OPERATIONAL'}]}, electionId=null, setVersion=1, lastWriteDate=Sun Nov 01 12:25:29 PST 2020, lastUpdateTimeNanos=104428010234368}]}. Waiting for 30000 ms before timing out
2020-11-01 12:25:34.316 INFO 20242 --- [ngodb.net:27017] org.mongodb.driver.cluster : Discovered replica set primary cluster0-shard-00-02.1c6kg.mongodb.net:27017
2020-11-01 12:25:34.612 INFO 20242 --- [nio-8088-exec-1] org.mongodb.driver.connection : Opened connection [connectionId{localValue:5, serverValue:108547}] to cluster0-shard-00-02.1c6kg.mongodb.net:27017
2020-11-01 12:25:34.838 WARN 20242 --- [nio-8088-exec-1] org.mongodb.driver.connection : Got socket exception on connection [connectionId{localValue:5, serverValue:108547}] to cluster0-shard-00-02.1c6kg.mongodb.net:27017. All connections to cluster0-shard-00-02.1c6kg.mongodb.net:27017 will be closed.
2020-11-01 12:25:34.838 INFO 20242 --- [nio-8088-exec-1] org.mongodb.driver.connection : Closed connection [connectionId{localValue:5, serverValue:108547}] to cluster0-shard-00-02.1c6kg.mongodb.net:27017 because there was a socket exception raised by this connection.
2020-11-01 12:25:34.876 INFO 20242 --- [ngodb.net:27017] org.mongodb.driver.cluster : Discovered replica set primary cluster0-shard-00-02.1c6kg.mongodb.net:27017
2020-11-01 12:25:34.878 ERROR 20242 --- [nio-8088-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.data.mongodb.UncategorizedMongoDbException: Exception sending message; nested exception is com.mongodb.MongoSocketWriteException: Exception sending message] with root cause

Jmeter: TCP Sampler error: 500 ReadException: Error reading from server, bytes read: 0

While checking TCP connection from Jmeter TCP Sampler, we are getting below error:
Response code: 500
Response message: org.apache.jmeter.protocol.tcp.sampler.ReadException: Error reading from server, bytes read: 0
We tried changing EOL, Connect & Response timeouts, Re-use connection & Close connection settings, but no luck.
Jmeter TCP Sampler log:
2018-04-24 12:09:09,740 INFO o.a.j.e.StandardJMeterEngine: Running the test!
2018-04-24 12:09:09,741 INFO o.a.j.s.SampleEvent: List of sample_variables: []
2018-04-24 12:09:09,745 INFO o.a.j.g.u.JMeterMenuBar: setRunning(true, *local*)
2018-04-24 12:09:09,755 INFO o.a.j.e.StandardJMeterEngine: Starting ThreadGroup: 1 : DS Requests
2018-04-24 12:09:09,756 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group DS Requests.
2018-04-24 12:09:09,764 INFO o.a.j.e.StandardJMeterEngine: Thread will continue on error
2018-04-24 12:09:09,765 INFO o.a.j.t.ThreadGroup: Starting thread group... number=1 threads=1 ramp-up=5 perThread=5000.0 delayedStart=false
2018-04-24 12:09:09,765 INFO o.a.j.t.ThreadGroup: Started thread group number 1
2018-04-24 12:09:09,765 INFO o.a.j.e.StandardJMeterEngine: All thread groups have been started
2018-04-24 12:09:09,766 INFO o.a.j.t.JMeterThread: Thread started: DS Requests 1-1
2018-04-24 12:09:09,767 INFO o.a.j.p.t.s.TCPClientImpl: Using platform default charset:windows-1252
2018-04-24 12:09:09,767 INFO o.a.j.p.t.s.TCPSampler: Using eolByte=10
2018-04-24 12:09:09,787 ERROR o.a.j.p.t.s.TCPSampler:
org.apache.jmeter.protocol.tcp.sampler.ReadException: Error reading from server, bytes read: 0
at org.apache.jmeter.protocol.tcp.sampler.TCPClientImpl.read(TCPClientImpl.java:131) ~[ApacheJMeter_tcp.jar:3.3 r1808647]
at org.apache.jmeter.protocol.tcp.sampler.TCPSampler.sample(TCPSampler.java:403) [ApacheJMeter_tcp.jar:3.3 r1808647]
at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:498) [ApacheJMeter_core.jar:3.3 r1808647]
at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:424) [ApacheJMeter_core.jar:3.3 r1808647]
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:255) [ApacheJMeter_core.jar:3.3 r1808647]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_144]
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source) ~[?:1.8.0_144]
at java.net.SocketInputStream.read(Unknown Source) ~[?:1.8.0_144]
at java.net.SocketInputStream.read(Unknown Source) ~[?:1.8.0_144]
at org.apache.jmeter.protocol.tcp.sampler.TCPClientImpl.read(TCPClientImpl.java:114) ~[ApacheJMeter_tcp.jar:3.3 r1808647]
... 5 more
2018-04-24 12:09:09,788 INFO o.a.j.t.JMeterThread: Thread is done: DS Requests 1-1
2018-04-24 12:09:09,788 INFO o.a.j.t.JMeterThread: Thread finished: DS Requests 1-1
2018-04-24 12:09:09,789 INFO o.a.j.e.StandardJMeterEngine: Notifying test listeners of end of test
2018-04-24 12:09:09,789 INFO o.a.j.g.u.JMeterMenuBar: setRunning(false, *local*)
Please suggest how to fix this issue.
Checking connectivity using TCP Sampler is not the best idea, I would recommend going for JSR223 Sampler and Socket class instead:
def sock = new Socket()
def host = "example.com" // change it to your host
def port = 80 // change it to your port
def timeout = 1000 // change it to your timeout (in milliseconds)
sock.setSoTimeout(1000)
sock.connect(new InetSocketAddress(host, port))
if (sock.isConnected()) {
log.info('Connection established')
SampleResult.setSuccessful(true)
}
else {
log.info('Server is not listening')
SampleResult.setSuccessful(false)
}
See Apache Groovy - Why and How You Should Use It article for more information on enhancing your JMeter test with Groovy scripting.
In the majority of cases it is much easier to use HTTP Raw Request plugin than the TCP Sampler.

SolrJ hanging when connecting to zookeeper

I have a local two instance Solr Cloud setup with a single zookeeper instance. I am trying to connect via SolrJ to execute a query however my code hangs for 2mins or so when executing the query and then fails. I have followed the basic example on the Solr wiki. The logs/code is below
2016-07-24 13:29:01.932 INFO 83666 --- [qtp699221219-28] org.apache.zookeeper.ZooKeeper : Initiating client connection, connectString=localhost:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$3#496eab9
2016-07-24 13:29:01.948 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ConnectionManager : Waiting for client to connect to ZooKeeper
2016-07-24 13:29:01.953 INFO 83666 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-07-24 13:29:01.955 INFO 83666 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Socket connection established to localhost/127.0.0.1:2181, initiating session
2016-07-24 13:29:01.967 INFO 83666 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x1561cdd875e0004, negotiated timeout = 10000
2016-07-24 13:29:01.972 INFO 83666 --- [back-3-thread-1] o.a.solr.common.cloud.ConnectionManager : Watcher org.apache.solr.common.cloud.ConnectionManager#4bb95d56 name:ZooKeeperConnection Watcher:localhost:2181 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
2016-07-24 13:29:01.972 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ConnectionManager : Client is connected to ZooKeeper
2016-07-24 13:29:01.973 INFO 83666 --- [qtp699221219-28] o.apache.solr.common.cloud.SolrZkClient : Using default ZkACLProvider
2016-07-24 13:29:01.974 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ZkStateReader : Updating cluster state from ZooKeeper...
2016-07-24 13:29:01.990 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ZkStateReader : Loaded empty cluster properties
2016-07-24 13:29:01.995 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ZkStateReader : Updated live nodes from ZooKeeper... (0) -> (2)
2016-07-24 13:31:24.653 ERROR 83666 --- [qtp699221219-28] o.a.s.client.solrj.impl.CloudSolrClient : Request to collection foo failed due to (0) java.net.ConnectException: Operation timed out, retry? 0
and my code is:
String zkHostString = "localhost:2181";
CloudSolrClient solr = new CloudSolrClient.Builder().withZkHost(zkHostString).build();
solr.setDefaultCollection("foo");
SolrQuery query = new SolrQuery();
query.set("q", "*:*");
QueryResponse response = null;
try {
response = solr.query(query);
} catch (SolrServerException e) {
return null;
}
//Do Something with the results...
Urgh, I'm an idiot, the zookeeper instance and solr instances are inside docker, the code posted above is not. So Zookeeper reported back the solr urls using the docker containers ip...The host needs to connect via localhost and not the docker container ip.
Eg: Zookeeper responds [http://172.17.0.5:8983/solr/foo_shard1_replica2, http://172.17.0.6:8984/solr/foo_shard1_replica1]
but my code needs to call [http://localhost:8983/solr/foo_shard1_replica2, http://localhost:8984/solr/foo_shard1_replica1]

Categories