Route loadfile is getting started automatically when I start main class.
On exception, when process should finish. It starts loadfile again and again.
It should get start from timer and then should call loadfile route, but loadfile is starting independent as well as from timer.
CamelContext context = new DefaultCamelContext(sr);
try {
context.addRoutes(new RouteBuilder() {
#Override
public void configure() throws Exception {
onException(Exception.class)
.log(LoggingLevel.INFO, "Extype:${exception.message}")
.stop();
from("timer://alertstrigtimer?period=60s&repeatCount=1")
.startupOrder(1)
.log(LoggingLevel.INFO, "*******************************Job-Alert-System: Started: alertstrigtimer******************************")
.to("direct:loadFile").stop();
from("direct:loadFile").routeId("loadfile")
.log(LoggingLevel.INFO, "*******************************Job-Alert-System: Started: direct:loadFile******************************")
.from(getTriggerFileURI(getWorkFilePath(), getWorkFileName())).choice()
.
.
});
context.start();
Thread.sleep(40000);
Following is log:
[main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.21.1 (CamelContext: camel-1) is starting
[main] INFO org.apache.camel.management.ManagedManagementStrategy - JMX is enabled
[main] INFO org.apache.camel.impl.converter.DefaultTypeConverter - Type converters loaded (core: 194, classpath: 14)
[main] INFO org.apache.camel.impl.DefaultCamelContext - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route1 started and consuming from: timer://alertstrigtimer?period=60s&repeatCount=1
[main] INFO org.apache.camel.impl.DefaultCamelContext - Skipping starting of route loadfile as its configured with autoStartup=false
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: loadDataAndAlerts started and consuming from: direct://loadDataAndAlerts
[main] INFO org.apache.camel.impl.DefaultCamelContext - Total 4 routes, of which 2 are started
[main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.21.1 (CamelContext: camel-1) started in 0.761 seconds
[Camel (camel-1) thread #1 - timer://alertstrigtimer] INFO route1 - *******************************Job-Alert-System: Started: alertstrigtimer******************************
[Camel (camel-1) thread #2 - timer://alertstrigtimer] INFO loadfile - *******************************Job-Alert-System: Started: direct:loadFile******************************
[Camel (camel-1) thread #1 - file://null] INFO loadfile - *******************************Job-Alert-System: Started: direct:loadFile******************************
The problem could be cause by this line .from(getTriggerFileURI(getWorkFilePath(), getWorkFileName())) in loadfile route. Route with multiple from endpoint is known as Multiple Input and this pattern is removed in Camel 3.x.
From RedHat,
from("URI1").from("URI2").from("URI3").to("DestinationUri");
..., exchanges from each of the input endpoints,
URI1, URI2, and URI3, are processed independently of each other and in
separate threads. In fact, you can think of the preceding route as
being equivalent to the following three separate routes:
from("URI1").to("DestinationUri");
from("URI2").to("DestinationUri");
from("URI3").to("DestinationUri");
Rather than using multiple from endpoint (extra independent input), try content enricher pattern (pollEnrich for file component).
Related
The Kafka service I deployed on the server passed the test a few days ago and was running normally. Later, no other operations were carried out.
Then the service call failed today. I went online and looked at it. It's normal these days. Just this afternoon, the service hung up. Looking at the log, it seems to say that the Kafka service is restarted automatically, and then conflicts with the ID of zookeeper.
Restart the service. Everything is normal. Google doesn't have a good solution. What's the reason?
The following is the relevant log, please help to see, thank you
[2021-11-06 14:43:29,139] WARN Unexpected exception (org.apache.zookeeper.server.NIOServerCnxn)
EndOfStreamException: Unable to read additional data from client, it probably closed the socket: address = /159.89.4.236:59168, session = 0x0
at org.apache.zookeeper.server.NIOServerCnxn.handleFailedRead(NIOServerCnxn.java:163)
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:326)
at org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:522)
at org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:154)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2021-11-06 15:00:18,301] INFO Completed load of Log(dir=/tmp/kafka-logs/__consumer_offsets-46, topicId=dPs0Kq4ERHKDaR440Nt8qA, topic=__consumer_offsets, partition=46, highWatermark=0, lastStableOffset=0, logStartOffset=0, logEndOffset=0) with 1 segments in 20ms (53/53 loaded in /tmp/kafka-logs) (kafka.log.LogManager)
[2021-11-06 15:00:18,303] INFO Loaded 53 logs in 2350ms. (kafka.log.LogManager)
[2021-11-06 15:00:18,303] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2021-11-06 15:00:18,304] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2021-11-06 15:00:18,838] INFO [BrokerToControllerChannelManager broker=0 name=forwarding]: Starting (kafka.server.BrokerToControllerRequestThread)
[2021-11-06 15:00:19,148] INFO Updated connection-accept-rate max connection creation rate to 2147483647 (kafka.network.ConnectionQuotas)
[2021-11-06 15:00:19,165] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
[2021-11-06 15:00:19,237] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Created data-plane acceptor and processors for endpoint : ListenerName(PLAINTEXT) (kafka.network.SocketServer)
[2021-11-06 15:00:19,282] INFO [BrokerToControllerChannelManager broker=0 name=alterIsr]: Starting (kafka.server.BrokerToControllerRequestThread)
[2021-11-06 15:00:19,400] INFO [ExpirationReaper-0-DeleteRecords]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2021-11-06 15:00:19,400] INFO [ExpirationReaper-0-Fetch]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2021-11-06 15:00:19,400] INFO [ExpirationReaper-0-Produce]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2021-11-06 15:00:19,409] INFO [ExpirationReaper-0-ElectLeader]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2021-11-06 15:00:19,451] INFO [LogDirFailureHandler]: Starting (kafka.server.ReplicaManager$LogDirFailureHandler)
[2021-11-06 15:00:19,530] INFO Creating /brokers/ids/0 (is it secure? false) (kafka.zk.KafkaZkClient)
[2021-11-06 15:00:19,566] ERROR Error while creating ephemeral at /brokers/ids/0, node already exists and owner '72057654275276801' does not match current session '72057654275276802' (kafka.zk.KafkaZkClient$CheckedEphemeral)
[2021-11-06 15:00:19,572] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists
at org.apache.zookeeper.KeeperException.create(KeeperException.java:126)
at kafka.zk.KafkaZkClient$CheckedEphemeral.getAfterNodeExists(KafkaZkClient.scala:1904)
at kafka.zk.KafkaZkClient$CheckedEphemeral.create(KafkaZkClient.scala:1842)
at kafka.zk.KafkaZkClient.checkedEphemeralCreate(KafkaZkClient.scala:1809)
at kafka.zk.KafkaZkClient.registerBroker(KafkaZkClient.scala:96)
at kafka.server.KafkaServer.startup(KafkaServer.scala:319)
at kafka.Kafka$.main(Kafka.scala:109)
at kafka.Kafka.main(Kafka.scala)
[2021-11-06 15:00:19,596] INFO [KafkaServer id=0] shutting down (kafka.server.KafkaServer)
[2021-11-06 15:00:19,597] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Stopping socket server request processors (kafka.network.SocketServer)```
I'm trying to create Kafka topics using Java. But I get a Exception in thread "main" java.lang.NoSuchFieldError: DEFAULT_SSL_PRINCIPAL_MAPPING_RULES and I can't fix it.
My goal is to create a topic so that when I run my Kafka server to display my topics using this command bin/kafka-topics.sh --list --bootstrap-server localhost:9092, I can actually see my topic in the list.
(the command is from Kafka's official website https://kafka.apache.org/quickstart)
I looked up to this problem How to create a Topic in Kafka through Java which actually inspired my code, but not only it doesn't really help, but it seems like it uses deprecated classes and methods.
I tried to use what I believed to be more recent classes such as ZooKeeperClient, KafkaZkClient and AdminZkClient, but from what I understand, the method adminZkClient.createTopic(topic, noOfPartitions, noOfReplication, topicConfiguration, RackAwareMode.Disabled$.MODULE$); is what brings the exception. I don't know what it is about that function that creates the exception, whether I forgot something from my application.properties file or something else.
Here's my code
import java.util.Properties;
import org.I0Itec.zkclient.ZkClient;
import org.I0Itec.zkclient.ZkConnection;
import org.apache.kafka.common.utils.Time;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.data.jpa.repository.config.EnableJpaAuditing;
import kafka.admin.AdminUtils;
import kafka.admin.RackAwareMode;
import kafka.utils.ZkUtils;
import kafka.zk.AdminZkClient;
import kafka.zk.KafkaZkClient;
import kafka.zookeeper.ZooKeeperClient;
import kafka.utils.*;
#SpringBootApplication
#EnableJpaAuditing
public class ChatApplication {
#SuppressWarnings("deprecation")
public static void main(String[] args) {
try {
String zookeeperHost = "127.0.0.1:2181";
int sessionTimeOutInMs = 15 * 1000;
int connectionTimeOutInMs = 10 * 1000;
ZooKeeperClient zooKeeperClient = new ZooKeeperClient(zookeeperHost, sessionTimeOutInMs, connectionTimeOutInMs, 2, Time.SYSTEM, "BytesInPerSec", "BytesOutPerSec");
KafkaZkClient kafkaZkClient = new KafkaZkClient(zooKeeperClient, true, Time.SYSTEM);
String topic = "superTopic";
int noOfPartitions = 2;
int noOfReplication = 1;
Properties topicConfiguration = new Properties();
AdminZkClient adminZkClient = new AdminZkClient(kafkaZkClient);
adminZkClient.createTopic(topic, noOfPartitions, noOfReplication, topicConfiguration, RackAwareMode.Disabled$.MODULE$);
} catch (Exception ex) {
ex.printStackTrace();
}
}
//SpringApplication.run(ChatApplication.class, args);
}
Here's the output I get :
06:50:06.796 [main] INFO kafka.utils.Log4jControllerRegistration$ - Registered kafka:type=kafka.Log4jController MBean
06:50:07.101 [main] INFO kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Initializing a new session to 127.0.0.1:2181.
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 00:39 GMT
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:host.name=kunta
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.version=11.0.3
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Oracle Corporation
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/lib/jvm/java-11-openjdk-amd64
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=/home/robscientist/STS-WORKSPACE/poc_chat/target/classes:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-web/2.1.3.RELEASE/spring-boot-starter-web-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter/2.1.3.RELEASE/spring-boot-starter-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-logging/2.1.3.RELEASE/spring-boot-starter-logging-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar:/home/robscientist/.m2/repository/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar:/home/robscientist/.m2/repository/org/apache/logging/log4j/log4j-to-slf4j/2.11.2/log4j-to-slf4j-2.11.2.jar:/home/robscientist/.m2/repository/org/apache/logging/log4j/log4j-api/2.11.2/log4j-api-2.11.2.jar:/home/robscientist/.m2/repository/org/slf4j/jul-to-slf4j/1.7.25/jul-to-slf4j-1.7.25.jar:/home/robscientist/.m2/repository/javax/annotation/javax.annotation-api/1.3.2/javax.annotation-api-1.3.2.jar:/home/robscientist/.m2/repository/org/yaml/snakeyaml/1.23/snakeyaml-1.23.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-json/2.1.3.RELEASE/spring-boot-starter-json-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.9.8/jackson-datatype-jsr310-2.9.8.jar:/home/robscientist/.m2/repository/com/fasterxml/jackson/module/jackson-module-parameter-names/2.9.8/jackson-module-parameter-names-2.9.8.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-tomcat/2.1.3.RELEASE/spring-boot-starter-tomcat-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/org/apache/tomcat/embed/tomcat-embed-core/9.0.16/tomcat-embed-core-9.0.16.jar:/home/robscientist/.m2/repository/org/apache/tomcat/embed/tomcat-embed-el/9.0.16/tomcat-embed-el-9.0.16.jar:/home/robscientist/.m2/repository/org/apache/tomcat/embed/tomcat-embed-websocket/9.0.16/tomcat-embed-websocket-9.0.16.jar:/home/robscientist/.m2/repository/org/hibernate/validator/hibernate-validator/6.0.14.Final/hibernate-validator-6.0.14.Final.jar:/home/robscientist/.m2/repository/javax/validation/validation-api/2.0.1.Final/validation-api-2.0.1.Final.jar:/home/robscientist/.m2/repository/org/jboss/logging/jboss-logging/3.3.2.Final/jboss-logging-3.3.2.Final.jar:/home/robscientist/.m2/repository/com/fasterxml/classmate/1.4.0/classmate-1.4.0.jar:/home/robscientist/.m2/repository/org/springframework/spring-web/5.1.5.RELEASE/spring-web-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-webmvc/5.1.5.RELEASE/spring-webmvc-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-aop/5.1.5.RELEASE/spring-aop-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-expression/5.1.5.RELEASE/spring-expression-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/io/debezium/debezium-core/0.9.5.Final/debezium-core-0.9.5.Final.jar:/home/robscientist/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.9.8/jackson-core-2.9.8.jar:/home/robscientist/.m2/repository/io/debezium/debezium-core/0.9.5.Final/debezium-core-0.9.5.Final-tests.jar:/home/robscientist/.m2/repository/org/springframework/spring-jdbc/5.1.5.RELEASE/spring-jdbc-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-beans/5.1.5.RELEASE/spring-beans-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-core/5.1.5.RELEASE/spring-core-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-jcl/5.1.5.RELEASE/spring-jcl-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-tx/5.1.5.RELEASE/spring-tx-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-websocket/2.1.3.RELEASE/spring-boot-starter-websocket-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-messaging/5.1.5.RELEASE/spring-messaging-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-websocket/5.1.5.RELEASE/spring-websocket-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-data-jpa/2.1.3.RELEASE/spring-boot-starter-data-jpa-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-aop/2.1.3.RELEASE/spring-boot-starter-aop-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/org/aspectj/aspectjweaver/1.9.2/aspectjweaver-1.9.2.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-jdbc/2.1.3.RELEASE/spring-boot-starter-jdbc-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/com/zaxxer/HikariCP/3.2.0/HikariCP-3.2.0.jar:/home/robscientist/.m2/repository/javax/transaction/javax.transaction-api/1.3/javax.transaction-api-1.3.jar:/home/robscientist/.m2/repository/javax/xml/bind/jaxb-api/2.3.1/jaxb-api-2.3.1.jar:/home/robscientist/.m2/repository/javax/activation/javax.activation-api/1.2.0/javax.activation-api-1.2.0.jar:/home/robscientist/.m2/repository/org/hibernate/hibernate-core/5.3.7.Final/hibernate-core-5.3.7.Final.jar:/home/robscientist/.m2/repository/javax/persistence/javax.persistence-api/2.2/javax.persistence-api-2.2.jar:/home/robscientist/.m2/repository/org/javassist/javassist/3.23.1-GA/javassist-3.23.1-GA.jar:/home/robscientist/.m2/repository/net/bytebuddy/byte-buddy/1.9.10/byte-buddy-1.9.10.jar:/home/robscientist/.m2/repository/antlr/antlr/2.7.7/antlr-2.7.7.jar:/home/robscientist/.m2/repository/org/jboss/jandex/2.0.5.Final/jandex-2.0.5.Final.jar:/home/robscientist/.m2/repository/org/dom4j/dom4j/2.1.1/dom4j-2.1.1.jar:/home/robscientist/.m2/repository/org/hibernate/common/hibernate-commons-annotations/5.0.4.Final/hibernate-commons-annotations-5.0.4.Final.jar:/home/robscientist/.m2/repository/org/springframework/data/spring-data-jpa/2.1.5.RELEASE/spring-data-jpa-2.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/data/spring-data-commons/2.1.5.RELEASE/spring-data-commons-2.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-orm/5.1.5.RELEASE/spring-orm-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-aspects/5.1.5.RELEASE/spring-aspects-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/com/h2database/h2/1.4.197/h2-1.4.197.jar:/home/robscientist/.m2/repository/org/webjars/stomp-websocket/2.3.3/stomp-websocket-2.3.3.jar:/home/robscientist/.m2/repository/org/webjars/bootstrap/3.3.7/bootstrap-3.3.7.jar:/home/robscientist/.m2/repository/org/webjars/jquery/1.11.1/jquery-1.11.1.jar:/home/robscientist/.m2/repository/org/springframework/kafka/spring-kafka/2.2.4.RELEASE/spring-kafka-2.2.4.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-context/5.1.5.RELEASE/spring-context-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/retry/spring-retry/1.2.4.RELEASE/spring-retry-1.2.4.RELEASE.jar:/home/robscientist/.m2/repository/org/apache/kafka/kafka-clients/2.0.1/kafka-clients-2.0.1.jar:/home/robscientist/.m2/repository/org/lz4/lz4-java/1.4.1/lz4-java-1.4.1.jar:/home/robscientist/.m2/repository/org/xerial/snappy/snappy-java/1.1.7.1/snappy-java-1.1.7.1.jar:/home/robscientist/.m2/repository/org/apache/kafka/kafka_2.12/2.2.0/kafka_2.12-2.2.0.jar:/home/robscientist/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.9.8/jackson-databind-2.9.8.jar:/home/robscientist/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.9.0/jackson-annotations-2.9.0.jar:/home/robscientist/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-jdk8/2.9.8/jackson-datatype-jdk8-2.9.8.jar:/home/robscientist/.m2/repository/net/sf/jopt-simple/jopt-simple/5.0.4/jopt-simple-5.0.4.jar:/home/robscientist/.m2/repository/com/yammer/metrics/metrics-core/2.2.0/metrics-core-2.2.0.jar:/home/robscientist/.m2/repository/org/scala-lang/scala-library/2.12.8/scala-library-2.12.8.jar:/home/robscientist/.m2/repository/org/scala-lang/scala-reflect/2.12.8/scala-reflect-2.12.8.jar:/home/robscientist/.m2/repository/com/typesafe/scala-logging/scala-logging_2.12/3.9.0/scala-logging_2.12-3.9.0.jar:/home/robscientist/.m2/repository/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25.jar:/home/robscientist/.m2/repository/com/101tec/zkclient/0.11/zkclient-0.11.jar:/home/robscientist/.m2/repository/org/apache/zookeeper/zookeeper/3.4.13/zookeeper-3.4.13.jar:/home/robscientist/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar:/home/robscientist/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/home/robscientist/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar:/home/robscientist/.m2/repository/org/apache/yetus/audience-annotations/0.5.0/audience-annotations-0.5.0.jar:/home/robscientist/.m2/repository/io/netty/netty/3.10.6.Final/netty-3.10.6.Final.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-reactor-netty/2.1.3.RELEASE/spring-boot-starter-reactor-netty-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/io/projectreactor/netty/reactor-netty/0.8.5.RELEASE/reactor-netty-0.8.5.RELEASE.jar:/home/robscientist/.m2/repository/io/netty/netty-codec-http/4.1.33.Final/netty-codec-http-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-common/4.1.33.Final/netty-common-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-buffer/4.1.33.Final/netty-buffer-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-transport/4.1.33.Final/netty-transport-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-resolver/4.1.33.Final/netty-resolver-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-codec/4.1.33.Final/netty-codec-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-codec-http2/4.1.33.Final/netty-codec-http2-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-handler/4.1.33.Final/netty-handler-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-handler-proxy/4.1.33.Final/netty-handler-proxy-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-codec-socks/4.1.33.Final/netty-codec-socks-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-transport-native-epoll/4.1.33.Final/netty-transport-native-epoll-4.1.33.Final-linux-x86_64.jar:/home/robscientist/.m2/repository/io/netty/netty-transport-native-unix-common/4.1.33.Final/netty-transport-native-unix-common-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/projectreactor/reactor-core/3.2.6.RELEASE/reactor-core-3.2.6.RELEASE.jar:/home/robscientist/.m2/repository/org/reactivestreams/reactive-streams/1.0.2/reactive-streams-1.0.2.jar:/home/robscientist/.m2/repository/org/postgresql/postgresql/42.2.5/postgresql-42.2.5.jar:/home/robscientist/.m2/repository/org/projectlombok/lombok/1.18.6/lombok-1.18.6.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-devtools/2.0.0.RELEASE/spring-boot-devtools-2.0.0.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot/2.1.3.RELEASE/spring-boot-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-autoconfigure/2.1.3.RELEASE/spring-boot-autoconfigure-2.1.3.RELEASE.jar
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=/tmp
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
06:50:07.117 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.version=4.15.0-50-generic
06:50:07.117 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.name=robscientist
06:50:07.117 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.home=/home/robscientist
06:50:07.117 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/home/robscientist/STS-WORKSPACE/poc_chat
06:50:07.118 [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=15000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$#5bda8e08
06:50:07.120 [main] DEBUG org.apache.zookeeper.ClientCnxn - zookeeper.disableAutoWatchReset is false
06:50:07.129 [main] DEBUG kafka.utils.KafkaScheduler - Initializing task scheduler.
06:50:07.131 [main] INFO kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Waiting until connected.
06:50:07.138 [main-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
06:50:07.144 [main-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/127.0.0.1:2181, initiating session
06:50:07.145 [main-SendThread(localhost:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Session establishment request sent on localhost/127.0.0.1:2181
06:50:07.403 [main-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x1000016739e0016, negotiated timeout = 15000
06:50:07.409 [main-EventThread] DEBUG kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Received event: WatchedEvent state:SyncConnected type:None path:null
06:50:07.415 [main] INFO kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Connected.
06:50:07.469 [main-SendThread(localhost:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply sessionid:0x1000016739e0016, packet:: clientPath:/brokers/ids serverPath:/brokers/ids finished:false header:: 1,12 replyHeader:: 1,237,0 request:: '/brokers/ids,F response:: v{'0},s{5,5,1559484903664,1559484903664,0,5,0,0,0,1,191}
06:50:07.492 [main-SendThread(localhost:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply sessionid:0x1000016739e0016, packet:: clientPath:/brokers/ids/0 serverPath:/brokers/ids/0 finished:false header:: 2,4 replyHeader:: 2,237,0 request:: '/brokers/ids/0,F response:: #7b226c697374656e65725f73656375726974795f70726f746f636f6c5f6d6170223a7b22504c41494e54455854223a22504c41494e54455854227d2c22656e64706f696e7473223a5b22504c41494e544558543a2f2f6b756e74613a39303932225d2c226a6d785f706f7274223a2d312c22686f7374223a226b756e7461222c2274696d657374616d70223a2231353539353332323533363131222c22706f7274223a393039322c2276657273696f6e223a347d,s{191,191,1559532253652,1559532253652,1,0,0,72057690466942976,180,0,191}
06:50:07.690 [main-SendThread(localhost:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply sessionid:0x1000016739e0016, packet:: clientPath:/brokers/topics/superTopic serverPath:/brokers/topics/superTopic finished:false header:: 3,3 replyHeader:: 3,237,-101 request:: '/brokers/topics/superTopic,F response::
Exception in thread "main" java.lang.NoSuchFieldError: DEFAULT_SSL_PRINCIPAL_MAPPING_RULES
at kafka.server.Defaults$.<init>(KafkaConfig.scala:224)
at kafka.server.Defaults$.<clinit>(KafkaConfig.scala)
at kafka.log.Defaults$.<init>(LogConfig.scala:36)
at kafka.log.Defaults$.<clinit>(LogConfig.scala)
at kafka.log.LogConfig$.<init>(LogConfig.scala:219)
at kafka.log.LogConfig$.<clinit>(LogConfig.scala)
at kafka.zk.AdminZkClient.validateTopicCreate(AdminZkClient.scala:128)
at kafka.zk.AdminZkClient.createTopicWithAssignment(AdminZkClient.scala:86)
at kafka.zk.AdminZkClient.createTopic(AdminZkClient.scala:56)
at com.predisurge.kafka.KafkaTopicCreator.createTopic(KafkaTopicCreator.java:41)
at com.predisurge.ChatApplication.main(ChatApplication.java:32)
After running the command to display my topics, I don't see the one I want to add.
IDE : STS 4,
Kafka version : 2.2.0,
ZK version : 3.5.5
not sure why you get this exception. Here's a solution that works for me, and just uses the KafkaAdminClient:
#SpringBootApplication
#EnableJpaAuditing
public class TopicCreator {
public static void main(String[] args) throws InterruptedException, ExecutionException {
Properties properties = new Properties();
properties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
AdminClient kafkaAdminClient = KafkaAdminClient.create(properties);
CreateTopicsResult result = kafkaAdminClient.createTopics(
Stream.of("foo", "bar", "baz").map(
name -> new NewTopic(name, 3, (short) 1)
).collect(Collectors.toList())
);
result.all().get();
}
}
As the above code shows, for newer versions of Kafka you can create topics directly through Kafka. Does this help?
i have a Camel process which pulls file from Azure container and process it in Azure environment.
I expect the process to run continuously, but it shuts down after random interval.
Logs:
CamelHangupInterceptor: INFO (MainSupport.java:87) - Received hang up - stopping the main instance.
CamelHangupInterceptor: DEBUG (Main.java:187) - Stopping Spring ApplicationContext: org.springframework.context.support.ClassPathXmlApplicationContext#47ef968d
2019-01-29 21:39:50,782: main: INFO (MainSupport.java:502) - MainSupport exiting code: 0
...
closes all the routes after inflight exchange is compeleted since using DefaultShutdownStrategy.
Spring context route:
- Start with a scheduler for initial delay,
- then <dealy> component to randomly generate time (logic used for scalability to avoid race condition)
- invoking the custom class implementing Process class, which has Azure container url with credentials and fetch the file from container
- then using wireTap component to downloading the file
- finally invoking another class implementing Process class.
The Camel (v 2.20) process starts and executes as expected, but after a random interval process shuts down.
I see hangup signal received, but not sure how it happens. Logs shows graceful shutdown. Is there a way to send hangup signal to Camel process from external process?
in one of the route i am using to stop the exchange to stop the route forcibly.
exchange.setProperty(Exchange.ROUTE_STOP, Boolean.TRUE);
More logs:
2019-01-29 21:40:17,838: Camel Thread #0 - CamelHangupInterceptor: DEBUG (DefaultExecutorServiceManager.java:363) - Forcing shutdown of ExecutorService: org.apache.camel.util.concurrent.RejectableThreadPoolExecutor#647b9364[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0][WireTap]
...
2019-01-29 21:40:17,840: Camel Thread #0 - CamelHangupInterceptor: DEBUG (RouteService.java:289) - Shutting down services on route: route1
2019-01-29 21:40:17,841: Camel Thread #0 - CamelHangupInterceptor: DEBUG (BeanComponent.java:72) - Clearing BeanInfo cache[size=1, hits=1, misses=1, evicted=0]
2019-01-29 21:40:17,849: Camel Thread #0 - CamelHangupInterceptor: DEBUG (SimpleLanguage.java:136) - Clearing simple language predicate cache[size=0, hits=0, misses=0, evicted=0]
2019-01-29 21:40:17,849: Camel Thread #0 - CamelHangupInterceptor: DEBUG (SimpleLanguage.java:142) - Clearing simple language expression cache[size=4, hits=1, misses=4, evicted=0]
2019-01-29 21:40:17,849: Camel Thread #0 - CamelHangupInterceptor: DEBUG (DefaultManagementAgent.java:358) - Unregistered MBean with ObjectName: org.apache.camel:context=camelAzureBlobContext,type=routecontroller,name="camelAzureBlobContext"
2019-01-29 21:40:17,849: Camel Thread #0 - CamelHangupInterceptor: DEBUG (DefaultManagementAgent.java:358) - Unregistered MBean with ObjectName: org.apache.camel:context=camelAzureBlobContext,type=health,name="camelAzureBlobContext"
2019-01-29 21:40:17,850: Camel Thread #0 - CamelHangupInterceptor: DEBUG (TimerListenerManager.java:128) - Removed TimerListener: org.apache.camel.management.mbean.ManagedCamelContext#473692b
2019-01-29 21:40:17,850: Camel Thread #0 - CamelHangupInterceptor: DEBUG (DefaultManagementAgent.java:358) - Unregistered MBean with ObjectName: org.apache.camel:context=camelAzureBlobContext,type=context,name="camelAzureBlobContext"
2019-01-29 21:40:17,850: Camel Thread #0 - CamelHangupInterceptor: INFO (MainLifecycleStrategy.java:44) - CamelContext: camelAzureBlobContext has been shutdown, triggering shutdown of the JVM.
2019-01-29 21:40:17,850: Camel Thread #0 - CamelHangupInterceptor: DEBUG (DefaultExecutorServiceManager.java:363) - Forcing shutdown of ExecutorService: org.apache.camel.util.concurrent.RejectableThreadPoolExecutor#72b68833[Running, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 1][ShutdownTask]
2019-01-29 21:40:17,851: Camel Thread #0 - CamelHangupInterceptor: DEBUG (DefaultInflightRepository.java:183) - Shutting down with no inflight exchanges.
2019-01-29 21:40:17,851: Camel Thread #0 - CamelHangupInterceptor: DEBUG (DefaultServicePool.java:110) - Stopping service pool: org.apache.camel.impl.SharedPollingConsumerServicePool#13ed066e
2019-01-29 21:40:17,851: Camel Thread #0 - CamelHangupInterceptor: DEBUG (DefaultServicePool.java:110) - Stopping service pool: org.apache.camel.impl.SharedProducerServicePool#151ab2b9
...
2019-01-29 21:40:17,856: Camel Thread #0 - CamelHangupInterceptor: DEBUG (MBeanInfoAssembler.java:79) - Clearing cache[size=30, hits=12, misses=30, evicted=0]
2019-01-29 21:40:17,864: Camel Thread #0 - CamelHangupInterceptor: DEBUG (IntrospectionSupport.java:134) - Clearing cache[size=93, hits=192, misses=93, evicted=0]
2019-01-29 21:40:17,869: Camel Thread #0 - CamelHangupInterceptor: INFO (DefaultCamelContext.java:3575) - Apache Camel 2.20.0 (CamelContext: camelAzureBlobContext) uptime 32 minutes
-- ROUTE INFORMATION--
<camel:camelContext id="camelAzureBlobContext" xmlns="http://camel.apache.org/schema/spring" typeConverterStatisticsEnabled="true" autoStartup="true">
<endpoint id="listBlobendpoint"
uri="azure-blob://storageaccount/containerName?credentials=containercredientiasl&operation=listBlobs" /> <!-- changed the actual values -->
<dataFormats>
<json id="inputMsg" library="Jackson" unmarshalTypeName="pacakage.requiredinputpojojacksonclass" /> <!-- renamed the class name-->
</dataFormats>
<onException>
<exception>com.fasterxml.jackson.core.JsonParseException</exception>
<exception>com.fasterxml.jackson.databind.JsonMappingException</exception>
<handled>
<constant>true</constant>
</handled>
<process ref="parseExceptionResponse" />
</onException>
<route>
<from uri="scheduler://tempScheduler?initialDelay=5000&delay=50000" /> <!-- changed the actual values -->
<setHeader headerName="BlobListingDetails">
<simple resultType="com.microsoft.azure.storage.blob.BlobListingDetails">METADATA</simple>
</setHeader>
<delay>
<method ref="blobcamelprocess" method="randomDelayToPoll"></method> <!-- Method which has some random number generation-->
</delay>
<to uri="ref:listBlobendpoint" /> <!-- bean which actually sets the metadata value-->
<process ref="blobcamelprocess" />
<!-- creating recipient list to update metadata of the container blob file -->
<recipientList>
<header>update_metadata</header>
</recipientList>
<log message="Message | $simple{in.header[filename]}" loggingLevel="INFO"></log>
<wireTap uri="file://location?fileName=$simple{in.header[filename]}"/>
<unmarshal ref="inputMsg" />
<process ref="messageconversionprocess" /> <!-- bean which actually converts uses the parsed json to construct java object-->
<process ref="deleteblobProcess" /> <!-- bean that will be used to delete the file from the blob store -->
<recipientList>
<header>delete_blob</header> <!-- endpoint details is set from the above bean and passed here -->
</recipientList>
</route>
</camel:camelContext>
As per the logs, your whole Camel context is shutting down. Its quite unlikely that the context shutdown is initiated by your ROUTE_STOP call.
Looking at MainSupport.java source, the INFO log that you see in your log files is executed as a part of hang up notification hook, notified by a JVM Shutdown in progress. When this happens, does your JVM die as well? This documentation says hooks are notified when a JVM Abnormal termination is initiated by SIGINT, SIGTERM or SIGHUP
Perhaps we can look into further details if you could provide answers to the following
Is Camel running in a web container or something else? I see MainSupport.java :)
What JVM version do you use?
What OS platform is it running on?
When Camel context shuts down, is the JVM still running or JVM exits too?
Do you see additional exceptions in the log files?
Can we correlate this CamelContext shutdown to a system resource exhaustion event? (like running short of memory or something like that). Quite unlikely to be the OOMKiller since its is a graceful shutdown.
On a side note, don't be misled by the name CamelHangupInterceptor it doesn't mean Camel received a SIGHUP.
I have set up a class extending AbstractJavaSamplerClientcontaining a setupTest, runTest and getDefaultParameters method, all written accordingly to the jMeter's templates and online examples.
Have added a Logger, tried setting it up normally (LoggingManager.getLoggerForClass();) and using super as well (super.getLogger();) and added multiple Logger.info and Logger.error
I'm using a Java request and giving a Jar containing my classes have completed the user.properties with the classpath. The right class is selected when running the tests.
When I run my tests, no custom logs appear. No error either. And I'm getting this:
2016/02/03 13:34:42 INFO - jmeter.engine.StandardJMeterEngine: Running the test!
2016/02/03 13:34:42 INFO - jmeter.samplers.SampleEvent: List of sample_variables: []
2016/02/03 13:34:42 INFO - jmeter.gui.util.JMeterMenuBar: setRunning(true,*local*)
2016/02/03 13:34:42 INFO - jmeter.engine.StandardJMeterEngine: Starting ThreadGroup: 1 : Group Test
2016/02/03 13:34:42 INFO - jmeter.engine.StandardJMeterEngine: Starting 1 threads for group Group Test.
2016/02/03 13:34:42 INFO - jmeter.engine.StandardJMeterEngine: Thread will start next loop on error
2016/02/03 13:34:42 INFO - jmeter.threads.ThreadGroup: Starting thread group number 1 threads 1 ramp-up 1 perThread 1000.0 delayedStart=false
2016/02/03 13:34:42 INFO - jmeter.threads.ThreadGroup: Started thread group number 1
2016/02/03 13:34:42 INFO - jmeter.engine.StandardJMeterEngine: All thread groups have been started
2016/02/03 13:34:42 INFO - jmeter.threads.JMeterThread: Thread started: Group Test 1-1
2016/02/03 13:34:42 INFO - jmeter.threads.JMeterThread: Thread is done: Group Test 1-1
2016/02/03 13:34:42 INFO - jmeter.threads.JMeterThread: Thread finished: Group Test 1-1
2016/02/03 13:34:42 INFO - jmeter.engine.StandardJMeterEngine: Notifying test listeners of end of test
2016/02/03 13:34:42 INFO - jmeter.gui.util.JMeterMenuBar: setRunning(false,*local*)
I've checked the user.properties and the logs are written to jmeter.log
I feel like my class is not even tested since the logs are not displaying anything beside "Starting/Done/Finshed Thread".
When I add a result tree or any other result thread, it stays empty.
What could be set up wrong here?
EDIT:
Didn't use the right JAVa request (Configurations>Default Java Request != Single Java Request). From that point I saw I had an error with my Jonas thus not connecting and explaining the non existent results.
As per example given in WebSocket Testing With Apache JMeter article:
Necessary imports:
import org.apache.jorphan.logging.LoggingManager;
import org.apache.log.Logger;
Initialization:
private static final Logger log = LoggingManager.getLoggerForClass();
Usage:
log.info("your log message");
log.error("your error message", exception);
I have barely started on Camel and I am trying out the file component. I want to process each file in a directory recursively. I tried to chain from("file:") to a processor and I don't understand why it's not working. In the trace, I can see that the camel context started the route.
public static void main(String[] args) throws Exception {
System.out.println("Starting camel");
final CamelContext camelContext = new DefaultCamelContext();
camelContext.addRoutes(new RouteBuilder() {
#Override
public void configure() throws Exception {
from(
"file://Users/abc123/Documents?recursive=true&noop=true&idempotent=true")
.process(new Processor() {
public void process(Exchange exchange)
throws Exception {
System.out.println("exchange=" + exchange);
}
});
}
});
camelContext.setTracing(true);
camelContext.start();
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
try {
camelContext.stop();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
});
Thread.currentThread().join();
}
Here's the log trace:
Starting camel
2015-06-03 00:00:59 INFO DefaultCamelContext - Apache Camel 2.15.2 (CamelContext: camel-1) is starting
2015-06-03 00:00:59 INFO DefaultCamelContext - Tracing is enabled on CamelContext: camel-1
2015-06-03 00:00:59 INFO ManagedManagementStrategy - JMX is enabled
2015-06-03 00:00:59 INFO DefaultTypeConverter - Loaded 182 type converters
2015-06-03 00:00:59 INFO DefaultCamelContext - AllowUseOriginalMessage is enabled. If access to the original message is not needed, then its recommended to turn this option off as it may improve performance.
2015-06-03 00:00:59 INFO DefaultCamelContext - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
2015-06-03 00:00:59 INFO FileEndpoint - Using default memory based idempotent repository with cache max size: 1000
2015-06-03 00:00:59 INFO DefaultCamelContext - Route: route1 started and consuming from: Endpoint[file://Users/abc123/Documents?idempotent=true&noop=true&recursive=true]
2015-06-03 00:00:59 INFO DefaultCamelContext - Total 1 routes, of which 1 is started.
2015-06-03 00:00:59 INFO DefaultCamelContext - Apache Camel 2.15.2 (CamelContext: camel-1) started in 0.422 seconds
Be sure to specify the correct path to the folder. The best way is to specify the absolute path:
"file:/Users/abc123/Documents?recursive=true&noop=true&idempotent=true"
You can use the relative path, but then make sure that it really is the relative path for the current working directory of the application.