I'm trying to create Kafka topics using Java. But I get a Exception in thread "main" java.lang.NoSuchFieldError: DEFAULT_SSL_PRINCIPAL_MAPPING_RULES and I can't fix it.
My goal is to create a topic so that when I run my Kafka server to display my topics using this command bin/kafka-topics.sh --list --bootstrap-server localhost:9092, I can actually see my topic in the list.
(the command is from Kafka's official website https://kafka.apache.org/quickstart)
I looked up to this problem How to create a Topic in Kafka through Java which actually inspired my code, but not only it doesn't really help, but it seems like it uses deprecated classes and methods.
I tried to use what I believed to be more recent classes such as ZooKeeperClient, KafkaZkClient and AdminZkClient, but from what I understand, the method adminZkClient.createTopic(topic, noOfPartitions, noOfReplication, topicConfiguration, RackAwareMode.Disabled$.MODULE$); is what brings the exception. I don't know what it is about that function that creates the exception, whether I forgot something from my application.properties file or something else.
Here's my code
import java.util.Properties;
import org.I0Itec.zkclient.ZkClient;
import org.I0Itec.zkclient.ZkConnection;
import org.apache.kafka.common.utils.Time;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.data.jpa.repository.config.EnableJpaAuditing;
import kafka.admin.AdminUtils;
import kafka.admin.RackAwareMode;
import kafka.utils.ZkUtils;
import kafka.zk.AdminZkClient;
import kafka.zk.KafkaZkClient;
import kafka.zookeeper.ZooKeeperClient;
import kafka.utils.*;
#SpringBootApplication
#EnableJpaAuditing
public class ChatApplication {
#SuppressWarnings("deprecation")
public static void main(String[] args) {
try {
String zookeeperHost = "127.0.0.1:2181";
int sessionTimeOutInMs = 15 * 1000;
int connectionTimeOutInMs = 10 * 1000;
ZooKeeperClient zooKeeperClient = new ZooKeeperClient(zookeeperHost, sessionTimeOutInMs, connectionTimeOutInMs, 2, Time.SYSTEM, "BytesInPerSec", "BytesOutPerSec");
KafkaZkClient kafkaZkClient = new KafkaZkClient(zooKeeperClient, true, Time.SYSTEM);
String topic = "superTopic";
int noOfPartitions = 2;
int noOfReplication = 1;
Properties topicConfiguration = new Properties();
AdminZkClient adminZkClient = new AdminZkClient(kafkaZkClient);
adminZkClient.createTopic(topic, noOfPartitions, noOfReplication, topicConfiguration, RackAwareMode.Disabled$.MODULE$);
} catch (Exception ex) {
ex.printStackTrace();
}
}
//SpringApplication.run(ChatApplication.class, args);
}
Here's the output I get :
06:50:06.796 [main] INFO kafka.utils.Log4jControllerRegistration$ - Registered kafka:type=kafka.Log4jController MBean
06:50:07.101 [main] INFO kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Initializing a new session to 127.0.0.1:2181.
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 00:39 GMT
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:host.name=kunta
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.version=11.0.3
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Oracle Corporation
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/lib/jvm/java-11-openjdk-amd64
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=/home/robscientist/STS-WORKSPACE/poc_chat/target/classes:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-web/2.1.3.RELEASE/spring-boot-starter-web-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter/2.1.3.RELEASE/spring-boot-starter-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-logging/2.1.3.RELEASE/spring-boot-starter-logging-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar:/home/robscientist/.m2/repository/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar:/home/robscientist/.m2/repository/org/apache/logging/log4j/log4j-to-slf4j/2.11.2/log4j-to-slf4j-2.11.2.jar:/home/robscientist/.m2/repository/org/apache/logging/log4j/log4j-api/2.11.2/log4j-api-2.11.2.jar:/home/robscientist/.m2/repository/org/slf4j/jul-to-slf4j/1.7.25/jul-to-slf4j-1.7.25.jar:/home/robscientist/.m2/repository/javax/annotation/javax.annotation-api/1.3.2/javax.annotation-api-1.3.2.jar:/home/robscientist/.m2/repository/org/yaml/snakeyaml/1.23/snakeyaml-1.23.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-json/2.1.3.RELEASE/spring-boot-starter-json-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.9.8/jackson-datatype-jsr310-2.9.8.jar:/home/robscientist/.m2/repository/com/fasterxml/jackson/module/jackson-module-parameter-names/2.9.8/jackson-module-parameter-names-2.9.8.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-tomcat/2.1.3.RELEASE/spring-boot-starter-tomcat-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/org/apache/tomcat/embed/tomcat-embed-core/9.0.16/tomcat-embed-core-9.0.16.jar:/home/robscientist/.m2/repository/org/apache/tomcat/embed/tomcat-embed-el/9.0.16/tomcat-embed-el-9.0.16.jar:/home/robscientist/.m2/repository/org/apache/tomcat/embed/tomcat-embed-websocket/9.0.16/tomcat-embed-websocket-9.0.16.jar:/home/robscientist/.m2/repository/org/hibernate/validator/hibernate-validator/6.0.14.Final/hibernate-validator-6.0.14.Final.jar:/home/robscientist/.m2/repository/javax/validation/validation-api/2.0.1.Final/validation-api-2.0.1.Final.jar:/home/robscientist/.m2/repository/org/jboss/logging/jboss-logging/3.3.2.Final/jboss-logging-3.3.2.Final.jar:/home/robscientist/.m2/repository/com/fasterxml/classmate/1.4.0/classmate-1.4.0.jar:/home/robscientist/.m2/repository/org/springframework/spring-web/5.1.5.RELEASE/spring-web-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-webmvc/5.1.5.RELEASE/spring-webmvc-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-aop/5.1.5.RELEASE/spring-aop-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-expression/5.1.5.RELEASE/spring-expression-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/io/debezium/debezium-core/0.9.5.Final/debezium-core-0.9.5.Final.jar:/home/robscientist/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.9.8/jackson-core-2.9.8.jar:/home/robscientist/.m2/repository/io/debezium/debezium-core/0.9.5.Final/debezium-core-0.9.5.Final-tests.jar:/home/robscientist/.m2/repository/org/springframework/spring-jdbc/5.1.5.RELEASE/spring-jdbc-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-beans/5.1.5.RELEASE/spring-beans-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-core/5.1.5.RELEASE/spring-core-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-jcl/5.1.5.RELEASE/spring-jcl-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-tx/5.1.5.RELEASE/spring-tx-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-websocket/2.1.3.RELEASE/spring-boot-starter-websocket-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-messaging/5.1.5.RELEASE/spring-messaging-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-websocket/5.1.5.RELEASE/spring-websocket-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-data-jpa/2.1.3.RELEASE/spring-boot-starter-data-jpa-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-aop/2.1.3.RELEASE/spring-boot-starter-aop-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/org/aspectj/aspectjweaver/1.9.2/aspectjweaver-1.9.2.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-jdbc/2.1.3.RELEASE/spring-boot-starter-jdbc-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/com/zaxxer/HikariCP/3.2.0/HikariCP-3.2.0.jar:/home/robscientist/.m2/repository/javax/transaction/javax.transaction-api/1.3/javax.transaction-api-1.3.jar:/home/robscientist/.m2/repository/javax/xml/bind/jaxb-api/2.3.1/jaxb-api-2.3.1.jar:/home/robscientist/.m2/repository/javax/activation/javax.activation-api/1.2.0/javax.activation-api-1.2.0.jar:/home/robscientist/.m2/repository/org/hibernate/hibernate-core/5.3.7.Final/hibernate-core-5.3.7.Final.jar:/home/robscientist/.m2/repository/javax/persistence/javax.persistence-api/2.2/javax.persistence-api-2.2.jar:/home/robscientist/.m2/repository/org/javassist/javassist/3.23.1-GA/javassist-3.23.1-GA.jar:/home/robscientist/.m2/repository/net/bytebuddy/byte-buddy/1.9.10/byte-buddy-1.9.10.jar:/home/robscientist/.m2/repository/antlr/antlr/2.7.7/antlr-2.7.7.jar:/home/robscientist/.m2/repository/org/jboss/jandex/2.0.5.Final/jandex-2.0.5.Final.jar:/home/robscientist/.m2/repository/org/dom4j/dom4j/2.1.1/dom4j-2.1.1.jar:/home/robscientist/.m2/repository/org/hibernate/common/hibernate-commons-annotations/5.0.4.Final/hibernate-commons-annotations-5.0.4.Final.jar:/home/robscientist/.m2/repository/org/springframework/data/spring-data-jpa/2.1.5.RELEASE/spring-data-jpa-2.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/data/spring-data-commons/2.1.5.RELEASE/spring-data-commons-2.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-orm/5.1.5.RELEASE/spring-orm-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-aspects/5.1.5.RELEASE/spring-aspects-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/com/h2database/h2/1.4.197/h2-1.4.197.jar:/home/robscientist/.m2/repository/org/webjars/stomp-websocket/2.3.3/stomp-websocket-2.3.3.jar:/home/robscientist/.m2/repository/org/webjars/bootstrap/3.3.7/bootstrap-3.3.7.jar:/home/robscientist/.m2/repository/org/webjars/jquery/1.11.1/jquery-1.11.1.jar:/home/robscientist/.m2/repository/org/springframework/kafka/spring-kafka/2.2.4.RELEASE/spring-kafka-2.2.4.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/spring-context/5.1.5.RELEASE/spring-context-5.1.5.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/retry/spring-retry/1.2.4.RELEASE/spring-retry-1.2.4.RELEASE.jar:/home/robscientist/.m2/repository/org/apache/kafka/kafka-clients/2.0.1/kafka-clients-2.0.1.jar:/home/robscientist/.m2/repository/org/lz4/lz4-java/1.4.1/lz4-java-1.4.1.jar:/home/robscientist/.m2/repository/org/xerial/snappy/snappy-java/1.1.7.1/snappy-java-1.1.7.1.jar:/home/robscientist/.m2/repository/org/apache/kafka/kafka_2.12/2.2.0/kafka_2.12-2.2.0.jar:/home/robscientist/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.9.8/jackson-databind-2.9.8.jar:/home/robscientist/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.9.0/jackson-annotations-2.9.0.jar:/home/robscientist/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-jdk8/2.9.8/jackson-datatype-jdk8-2.9.8.jar:/home/robscientist/.m2/repository/net/sf/jopt-simple/jopt-simple/5.0.4/jopt-simple-5.0.4.jar:/home/robscientist/.m2/repository/com/yammer/metrics/metrics-core/2.2.0/metrics-core-2.2.0.jar:/home/robscientist/.m2/repository/org/scala-lang/scala-library/2.12.8/scala-library-2.12.8.jar:/home/robscientist/.m2/repository/org/scala-lang/scala-reflect/2.12.8/scala-reflect-2.12.8.jar:/home/robscientist/.m2/repository/com/typesafe/scala-logging/scala-logging_2.12/3.9.0/scala-logging_2.12-3.9.0.jar:/home/robscientist/.m2/repository/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25.jar:/home/robscientist/.m2/repository/com/101tec/zkclient/0.11/zkclient-0.11.jar:/home/robscientist/.m2/repository/org/apache/zookeeper/zookeeper/3.4.13/zookeeper-3.4.13.jar:/home/robscientist/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar:/home/robscientist/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/home/robscientist/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar:/home/robscientist/.m2/repository/org/apache/yetus/audience-annotations/0.5.0/audience-annotations-0.5.0.jar:/home/robscientist/.m2/repository/io/netty/netty/3.10.6.Final/netty-3.10.6.Final.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-starter-reactor-netty/2.1.3.RELEASE/spring-boot-starter-reactor-netty-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/io/projectreactor/netty/reactor-netty/0.8.5.RELEASE/reactor-netty-0.8.5.RELEASE.jar:/home/robscientist/.m2/repository/io/netty/netty-codec-http/4.1.33.Final/netty-codec-http-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-common/4.1.33.Final/netty-common-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-buffer/4.1.33.Final/netty-buffer-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-transport/4.1.33.Final/netty-transport-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-resolver/4.1.33.Final/netty-resolver-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-codec/4.1.33.Final/netty-codec-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-codec-http2/4.1.33.Final/netty-codec-http2-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-handler/4.1.33.Final/netty-handler-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-handler-proxy/4.1.33.Final/netty-handler-proxy-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-codec-socks/4.1.33.Final/netty-codec-socks-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/netty/netty-transport-native-epoll/4.1.33.Final/netty-transport-native-epoll-4.1.33.Final-linux-x86_64.jar:/home/robscientist/.m2/repository/io/netty/netty-transport-native-unix-common/4.1.33.Final/netty-transport-native-unix-common-4.1.33.Final.jar:/home/robscientist/.m2/repository/io/projectreactor/reactor-core/3.2.6.RELEASE/reactor-core-3.2.6.RELEASE.jar:/home/robscientist/.m2/repository/org/reactivestreams/reactive-streams/1.0.2/reactive-streams-1.0.2.jar:/home/robscientist/.m2/repository/org/postgresql/postgresql/42.2.5/postgresql-42.2.5.jar:/home/robscientist/.m2/repository/org/projectlombok/lombok/1.18.6/lombok-1.18.6.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-devtools/2.0.0.RELEASE/spring-boot-devtools-2.0.0.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot/2.1.3.RELEASE/spring-boot-2.1.3.RELEASE.jar:/home/robscientist/.m2/repository/org/springframework/boot/spring-boot-autoconfigure/2.1.3.RELEASE/spring-boot-autoconfigure-2.1.3.RELEASE.jar
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=/tmp
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux
06:50:07.116 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
06:50:07.117 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.version=4.15.0-50-generic
06:50:07.117 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.name=robscientist
06:50:07.117 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.home=/home/robscientist
06:50:07.117 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/home/robscientist/STS-WORKSPACE/poc_chat
06:50:07.118 [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=15000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$#5bda8e08
06:50:07.120 [main] DEBUG org.apache.zookeeper.ClientCnxn - zookeeper.disableAutoWatchReset is false
06:50:07.129 [main] DEBUG kafka.utils.KafkaScheduler - Initializing task scheduler.
06:50:07.131 [main] INFO kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Waiting until connected.
06:50:07.138 [main-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
06:50:07.144 [main-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/127.0.0.1:2181, initiating session
06:50:07.145 [main-SendThread(localhost:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Session establishment request sent on localhost/127.0.0.1:2181
06:50:07.403 [main-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x1000016739e0016, negotiated timeout = 15000
06:50:07.409 [main-EventThread] DEBUG kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Received event: WatchedEvent state:SyncConnected type:None path:null
06:50:07.415 [main] INFO kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Connected.
06:50:07.469 [main-SendThread(localhost:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply sessionid:0x1000016739e0016, packet:: clientPath:/brokers/ids serverPath:/brokers/ids finished:false header:: 1,12 replyHeader:: 1,237,0 request:: '/brokers/ids,F response:: v{'0},s{5,5,1559484903664,1559484903664,0,5,0,0,0,1,191}
06:50:07.492 [main-SendThread(localhost:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply sessionid:0x1000016739e0016, packet:: clientPath:/brokers/ids/0 serverPath:/brokers/ids/0 finished:false header:: 2,4 replyHeader:: 2,237,0 request:: '/brokers/ids/0,F response:: #7b226c697374656e65725f73656375726974795f70726f746f636f6c5f6d6170223a7b22504c41494e54455854223a22504c41494e54455854227d2c22656e64706f696e7473223a5b22504c41494e544558543a2f2f6b756e74613a39303932225d2c226a6d785f706f7274223a2d312c22686f7374223a226b756e7461222c2274696d657374616d70223a2231353539353332323533363131222c22706f7274223a393039322c2276657273696f6e223a347d,s{191,191,1559532253652,1559532253652,1,0,0,72057690466942976,180,0,191}
06:50:07.690 [main-SendThread(localhost:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply sessionid:0x1000016739e0016, packet:: clientPath:/brokers/topics/superTopic serverPath:/brokers/topics/superTopic finished:false header:: 3,3 replyHeader:: 3,237,-101 request:: '/brokers/topics/superTopic,F response::
Exception in thread "main" java.lang.NoSuchFieldError: DEFAULT_SSL_PRINCIPAL_MAPPING_RULES
at kafka.server.Defaults$.<init>(KafkaConfig.scala:224)
at kafka.server.Defaults$.<clinit>(KafkaConfig.scala)
at kafka.log.Defaults$.<init>(LogConfig.scala:36)
at kafka.log.Defaults$.<clinit>(LogConfig.scala)
at kafka.log.LogConfig$.<init>(LogConfig.scala:219)
at kafka.log.LogConfig$.<clinit>(LogConfig.scala)
at kafka.zk.AdminZkClient.validateTopicCreate(AdminZkClient.scala:128)
at kafka.zk.AdminZkClient.createTopicWithAssignment(AdminZkClient.scala:86)
at kafka.zk.AdminZkClient.createTopic(AdminZkClient.scala:56)
at com.predisurge.kafka.KafkaTopicCreator.createTopic(KafkaTopicCreator.java:41)
at com.predisurge.ChatApplication.main(ChatApplication.java:32)
After running the command to display my topics, I don't see the one I want to add.
IDE : STS 4,
Kafka version : 2.2.0,
ZK version : 3.5.5
not sure why you get this exception. Here's a solution that works for me, and just uses the KafkaAdminClient:
#SpringBootApplication
#EnableJpaAuditing
public class TopicCreator {
public static void main(String[] args) throws InterruptedException, ExecutionException {
Properties properties = new Properties();
properties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
AdminClient kafkaAdminClient = KafkaAdminClient.create(properties);
CreateTopicsResult result = kafkaAdminClient.createTopics(
Stream.of("foo", "bar", "baz").map(
name -> new NewTopic(name, 3, (short) 1)
).collect(Collectors.toList())
);
result.all().get();
}
}
As the above code shows, for newer versions of Kafka you can create topics directly through Kafka. Does this help?
Related
Recently I was tasked to perform the following actions on our (single node) Tableau Server:
Harden the installation as per: https://help.tableau.com/current/server/en-us/security_harden.htm
Increase the disk size of the attached volume (AWS EBS)
Change the instance type of the server.
Having no previous experience of Tableau, I:
Logged into the EC2 instance and ran:
tsm configuration set -k ssl.protocols -v "all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1"
tsm pending-changes apply
Waited for the changes to apply, and stopped the server
Increased the EBS volume size
Changed the EC2 instance type
Restarted the server.
Upon restart of the server, it took a while for everything to load. And everything appeared to be okay.... except for the admin panel and tsm command.
Running tsm status revealed:
$ tsm status
Could not connect to server. Make sure that Tableau Server is running and try again.
Okay. Checked the syslog (/var/log/syslog), which was pointed me towards tabadmincontroller as per:
systemd[1172]: Started Tableau Server Administration Controller.
tabadmincontroller_0[11354]: [11354] [INFO] 2021-11-24 13:13:47.106 +0000 : Loading configuration from /var/opt/tableau/tableau_server/data/tabsvc/services/tabadmincontroller_0.20204.20.1116.1810/bin/tabadmincontroller.runjavaservice.json
tabadmincontroller_0[11354]: [11354] [INFO] 2021-11-24 13:13:47.107 +0000 : Loading configuration from /var/opt/tableau/tableau_server/data/tabsvc/services/tabadmincontroller_0.20204.20.1116.1810/config/tabadmincontroller.runjavaservice.json
tabadmincontroller_0[11354]: [11354] [INFO] 2021-11-24 13:13:47.107 +0000 : Loading manifest from /var/opt/tableau/tableau_server/data/tabsvc/services
/tabadmincontroller_0.20204.20.1116.1810/bin/tabadmincontroller.jar
tabadmincontroller_0[11354]: [11354] [INFO] 2021-11-24 13:13:47.107 +0000 : Starting malloc_trim thread. Run every 60 sec. Heap pad MB: 1
tabadmincontroller_0[11354]: [11354] [INFO] 2021-11-24 13:13:47.109 +0000 : Loading JVM library /var/opt/tableau/tableau_server/data/tabsvc/services/tabadmincontroller_0.20204.20.1116.1810/repository/jre/lib/server/libjvm.so
tabadmincontroller_0[11354]: [11354] [INFO] 2021-11-24 13:13:47.192 +0000 : Java class name: com.tableausoftware.tabadmin.webapp.TabadminController; Method name: main; Arguments: run
nlp_0[10759]: time="2021-11-24T13:13:54.323621681Z" level=info msg="[core] grpc: Server.Serve failed to create ServerTransport: connection error: desc = \"transport: http2Server.HandleStreams failed to receive the preface from client: EOF\"" logger="pegasus/server:server" system=system
nlp_0[10759]: time="2021-11-24T13:14:05.775776702Z" level=info msg="get job counts" blockedJobCount=0 inProgressJobCount=0 logger="semanticmodel/metrics:Monitor" queuedJobCount=0 waitingJobCount=0
systemd[1172]: tabadmincontroller_0.service: Main process exited, code=exited, status=1/FAILURE
systemd[1172]: tabadmincontroller_0.service: Failed with result 'exit-code'.
systemd[1172]: tabadmincontroller_0.service: Service hold-off time over, scheduling restart.
systemd[1172]: tabadmincontroller_0.service: Scheduled restart job, restart counter is at 7.
systemd[1172]: Stopped Tableau Server Administration Controller.
systemd[1172]: Started Tableau Server Administration Controller.
With an ongoing crash/restart loop going on forever.
Checking the logs at /var/opt/tableau/tableau_server/data/tabsvc/logs/tabadmincontroller
shows the following:
2021-11-24 13:17:37.372 +0000 main : INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=ip-172-33-30-206:8591 sessionTimeout=120000 watcher=org.apache.curator.ConnectionState#3657201c
2021-11-24 13:17:37.381 +0000 main : INFO org.apache.zookeeper.common.X509Util - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
2021-11-24 13:17:37.387 +0000 main : INFO org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 4194304 Bytes
2021-11-24 13:17:37.397 +0000 main : INFO org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=
2021-11-24 13:17:37.402 +0000 main : INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Default schema
2021-11-24 13:17:37.408 +0000 main-SendThread(ip-172-33-30-206:8591) : INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server ip-address-redacted. Will not attempt to authenticate using SASL (unknown error)
20redacted:8591) : INFO org.apache.zookeeper.ClientCnxn - Socket connection established, initiating session, client: /ip-redacted:45082, server: ip/ip:8591
2021-11-24 13:17:37.423 +0000 main-SendThread(ip:8591) : INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server, sessionid = 0x10000005ed60050, negotiated timeout = 120000
2021-11-24 13:17:37.432 +0000 main-EventThread : INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED
2021-11-24 13:17:37.780 +0000 main : ERROR com.tableausoftware.telemetry.helper.Environment - [xdev-edpa-007] process_type is not present in the configuration
2021-11-24 13:17:38.298 +0000 ZKWorker-ScheduledTask-0 : INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
2021-11-24 13:17:38.298 +0000 ZKWorker-ScheduledTask-0 : INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=ip sessionTimeout=120000 watcher=org.apache.curator.ConnectionState#48dddf5a
2021-11-24 13:17:38.298 +0000 ZKWorker-ScheduledTask-0 : INFO org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 4194304 Bytes
2021-11-24 13:17:38.299 +0000 ZKWorker-ScheduledTask-0 : INFO org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=
2021-11-24 13:17:38.300 +0000 ZKWorker-ScheduledTask-0 : INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Default schema
2021-11-24 13:17:38.300 +0000 ZKWorker-ScheduledTask-0-SendThread(ip:8591) : INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server ip/ip:8591. Will not attempt to authenticate using SASL (unknown error)
2021-11-24 13:17:38.301 +0000 ZKWorker-ScheduledTask-0-SendThread(ip:8591) : INFO org.apache.zookeeper.ClientCnxn - Socket connection established, initiating session, client: ip:45088, server: ip/ip:8591
2021-11-24 13:17:38.303 +0000 ZKWorker-ScheduledTask-0-SendThread(ip-172-33-30-206:8591) : INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server ip-172-33-30-206/ip-address:8591, sessionid = 0x10000005ed60051, negotiated timeout = 120000
2021-11-24 13:17:38.303 +0000 ZKWorker-ScheduledTask-0-EventThread : INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED
2021-11-24 13:17:39.357 +0000 main : INFO org.eclipse.jetty.server.session - DefaultSessionIdManager workerName=node0
2021-11-24 13:17:39.357 +0000 main : INFO org.eclipse.jetty.server.session - No SessionScavenger set, using defaults
2021-11-24 13:17:39.358 +0000 main : INFO org.eclipse.jetty.server.session - node0 Scavenging every 660000ms
2021-11-24 13:17:39.364 +0000 main : INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.s.b.w.e.j.JettyEmbeddedWebAppContext#38e0ccf3{application,/,[file:///var/opt/tableau/tableau_server/data/tabsvc/temp/tabadmincontroller_0.20204.20.1116.1810/jetty-docbase.7353433006274726961.8850/],AVAILABLE}
2021-11-24 13:17:39.365 +0000 main : INFO org.eclipse.jetty.server.Server - Started #9503ms
2021-11-24 13:17:39.755 +0000 main : INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
2021-11-24 13:17:39.756 +0000 main : INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=ip-172-33-30-206:8591 sessionTimeout=120000 watcher=org.apache.curator.ConnectionState#6376534c
2021-11-24 13:17:39.756 +0000 main : INFO org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 4194304 Bytes
2021-11-24 13:17:39.757 +0000 main : INFO org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=
2021-11-24 13:17:39.757 +0000 main : INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Default schema
2021-11-24 13:17:39.758 +0000 main-SendThread(ip-172-33-30-206:8591) : INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server ip-172-33-30-206/ip-address:8591. Will not attempt to authenticate using SASL (unknown error)
2021-11-24 13:17:39.758 +0000 main-SendThread(ip-172-33-30-206:8591) : INFO org.apache.zookeeper.ClientCnxn - Socket connection established, initiating session, client: /ip-address:45098, server: ip-172-33-30-206/ip-address:8591
2021-11-24 13:17:39.760 +0000 main-SendThread(ip-172-33-30-206:8591) : INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server ip-172-33-30-206/ip-address:8591, sessionid = 0x10000005ed60052, negotiated timeout = 120000
2021-11-24 13:17:39.760 +0000 main-EventThread : INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED
2021-11-24 13:17:50.803 +0000 main : WARN org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'webSecurityConfig' defined in URL [jar:file:/opt/tableau/tableau_server/packages/bin.20204.20.1116.1810/tabadmincontroller.jar!/com/tableausoftware/tabadmin/webapp/WebSecurityConfig.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cookieAuthenticationService' defined in URL [jar:file:/opt/tableau/tableau_server/packages/bin.20204.20.1116.1810/tabadmincontroller.jar!/com/tableausoftware/tabadmin/webapp/auth/CookieAuthenticationService.class]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.tableausoftware.tabadmin.webapp.auth.CookieAuthenticationService]: Constructor threw exception; nested exception is org.springframework.core.convert.ConversionFailedException: Failed to convert from type [java.lang.String] to type [int] for value 'warn'; nested exception is java.lang.NumberFormatException: For input string: "warn"
2021-11-24 13:17:50.806 +0000 Curator-Framework-0 : INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - backgroundOperationsLoop exiting
2021-11-24 13:17:50.910 +0000 main : INFO org.apache.zookeeper.ZooKeeper - Session: 0x10000005ed60052 closed
2021-11-24 13:17:50.910 +0000 main-EventThread : INFO org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x10000005ed60052
2021-11-24 13:17:50.917 +0000 Curator-Framework-0 : INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - backgroundOperationsLoop exiting
2021-11-24 13:17:50.930 +0000 Thread-15 : ERROR com.tableausoftware.tabadmin.webapp.EmbeddedLicenseActivator - Caught exception in EmbeddedLicenseActivator.
org.springframework.beans.factory.BeanCreationNotAllowedException: Error creating bean with name 'nativeApiInitializer': Singleton bean creation not allowed while singletons of this factory are in destruction (Do not request a bean from a BeanFactory in a destroy method implementation!)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:208) ~[spring-beans-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) ~[spring-beans-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:310) ~[spring-beans-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1290) ~[spring-beans-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.context.annotation.ContextAnnotationAutowireCandidateResolver$1.getTarget(ContextAnnotationAutowireCandidateResolver.java:90) ~[spring-context-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:192) ~[spring-aop-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at com.sun.proxy.$Proxy144.hasEnoughRoleLicenses(Unknown Source) ~[?:?]
at com.tableausoftware.tabadmin.webapp.impl.ProductKeyService.getLicensingState(ProductKeyService.java:373) ~[tab-tabadmin-controller-latest.jar:?]
at com.tableausoftware.tabadmin.webapp.EmbeddedLicenseActivator$1.run(EmbeddedLicenseActivator.java:43) [tabadmincontroller.jar:?]
2021-11-24 13:17:50.935 +0000 Thread-15 : ERROR com.tableausoftware.tabadmin.webapp.EmbeddedLicenseActivator - Failed to activate the OEM embedded license.
2021-11-24 13:17:51.020 +0000 main-EventThread : INFO org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x10000005ed60050
2021-11-24 13:17:51.020 +0000 main : INFO org.apache.zookeeper.ZooKeeper - Session: 0x10000005ed60050 closed
2021-11-24 13:17:51.021 +0000 main : WARN com.tableausoftware.tabadmin.agent.zookeeper.ConnectionHolder - Object ZooKeeperWorker#74b99b0e. was not a consumer of this connection.
2021-11-24 13:17:51.022 +0000 Curator-Framework-0 : INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - backgroundOperationsLoop exiting
2021-11-24 13:17:51.124 +0000 ZKWorker-ScheduledTask-0-EventThread : INFO org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x10000005ed60051
2021-11-24 13:17:51.124 +0000 main : INFO org.apache.zookeeper.ZooKeeper - Session: 0x10000005ed60051 closed
2021-11-24 13:17:51.142 +0000 main : INFO org.eclipse.jetty.server.session - node0 Stopped scavenging
2021-11-24 13:17:51.144 +0000 main : INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped o.s.b.w.e.j.JettyEmbeddedWebAppContext#38e0ccf3{application,/,[file:///var/opt/tableau/tableau_server/data/tabsvc/temp/tabadmincontroller_0.20204.20.1116.1810/jetty-docbase.7353433006274726961.8850/],UNAVAILABLE}
2021-11-24 13:17:51.154 +0000 main : INFO org.springframework.boot.autoconfigure.logging.ConditionEvaluationReportLoggingListener -
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2021-11-24 13:17:51.156 +0000 main : ERROR org.springframework.boot.SpringApplication - Application run failed
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'webSecurityConfig' defined in URL [jar:file:/opt/tableau/tableau_server/packages/bin.20204.20.1116.1810/tabadmincontroller.jar!/com/tableausoftware/tabadmin/webapp/WebSecurityConfig.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cookieAuthenticationService' defined in URL [jar:file:/opt/tableau/tableau_server/packages/bin.20204.20.1116.1810/tabadmincontroller.jar!/com/tableausoftware/tabadmin/webapp/auth/CookieAuthenticationService.class]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.tableausoftware.tabadmin.webapp.auth.CookieAuthenticationService]: Constructor threw exception; nested exception is org.springframework.core.convert.ConversionFailedException: Failed to convert from type [java.lang.String] to type [int] for value 'warn'; nested exception is java.lang.NumberFormatException: For input string: "warn"
Can anyone help point me in the right direction of the next steps here ?
Route loadfile is getting started automatically when I start main class.
On exception, when process should finish. It starts loadfile again and again.
It should get start from timer and then should call loadfile route, but loadfile is starting independent as well as from timer.
CamelContext context = new DefaultCamelContext(sr);
try {
context.addRoutes(new RouteBuilder() {
#Override
public void configure() throws Exception {
onException(Exception.class)
.log(LoggingLevel.INFO, "Extype:${exception.message}")
.stop();
from("timer://alertstrigtimer?period=60s&repeatCount=1")
.startupOrder(1)
.log(LoggingLevel.INFO, "*******************************Job-Alert-System: Started: alertstrigtimer******************************")
.to("direct:loadFile").stop();
from("direct:loadFile").routeId("loadfile")
.log(LoggingLevel.INFO, "*******************************Job-Alert-System: Started: direct:loadFile******************************")
.from(getTriggerFileURI(getWorkFilePath(), getWorkFileName())).choice()
.
.
});
context.start();
Thread.sleep(40000);
Following is log:
[main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.21.1 (CamelContext: camel-1) is starting
[main] INFO org.apache.camel.management.ManagedManagementStrategy - JMX is enabled
[main] INFO org.apache.camel.impl.converter.DefaultTypeConverter - Type converters loaded (core: 194, classpath: 14)
[main] INFO org.apache.camel.impl.DefaultCamelContext - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route1 started and consuming from: timer://alertstrigtimer?period=60s&repeatCount=1
[main] INFO org.apache.camel.impl.DefaultCamelContext - Skipping starting of route loadfile as its configured with autoStartup=false
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: loadDataAndAlerts started and consuming from: direct://loadDataAndAlerts
[main] INFO org.apache.camel.impl.DefaultCamelContext - Total 4 routes, of which 2 are started
[main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.21.1 (CamelContext: camel-1) started in 0.761 seconds
[Camel (camel-1) thread #1 - timer://alertstrigtimer] INFO route1 - *******************************Job-Alert-System: Started: alertstrigtimer******************************
[Camel (camel-1) thread #2 - timer://alertstrigtimer] INFO loadfile - *******************************Job-Alert-System: Started: direct:loadFile******************************
[Camel (camel-1) thread #1 - file://null] INFO loadfile - *******************************Job-Alert-System: Started: direct:loadFile******************************
The problem could be cause by this line .from(getTriggerFileURI(getWorkFilePath(), getWorkFileName())) in loadfile route. Route with multiple from endpoint is known as Multiple Input and this pattern is removed in Camel 3.x.
From RedHat,
from("URI1").from("URI2").from("URI3").to("DestinationUri");
..., exchanges from each of the input endpoints,
URI1, URI2, and URI3, are processed independently of each other and in
separate threads. In fact, you can think of the preceding route as
being equivalent to the following three separate routes:
from("URI1").to("DestinationUri");
from("URI2").to("DestinationUri");
from("URI3").to("DestinationUri");
Rather than using multiple from endpoint (extra independent input), try content enricher pattern (pollEnrich for file component).
I have a local two instance Solr Cloud setup with a single zookeeper instance. I am trying to connect via SolrJ to execute a query however my code hangs for 2mins or so when executing the query and then fails. I have followed the basic example on the Solr wiki. The logs/code is below
2016-07-24 13:29:01.932 INFO 83666 --- [qtp699221219-28] org.apache.zookeeper.ZooKeeper : Initiating client connection, connectString=localhost:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$3#496eab9
2016-07-24 13:29:01.948 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ConnectionManager : Waiting for client to connect to ZooKeeper
2016-07-24 13:29:01.953 INFO 83666 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-07-24 13:29:01.955 INFO 83666 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Socket connection established to localhost/127.0.0.1:2181, initiating session
2016-07-24 13:29:01.967 INFO 83666 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x1561cdd875e0004, negotiated timeout = 10000
2016-07-24 13:29:01.972 INFO 83666 --- [back-3-thread-1] o.a.solr.common.cloud.ConnectionManager : Watcher org.apache.solr.common.cloud.ConnectionManager#4bb95d56 name:ZooKeeperConnection Watcher:localhost:2181 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
2016-07-24 13:29:01.972 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ConnectionManager : Client is connected to ZooKeeper
2016-07-24 13:29:01.973 INFO 83666 --- [qtp699221219-28] o.apache.solr.common.cloud.SolrZkClient : Using default ZkACLProvider
2016-07-24 13:29:01.974 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ZkStateReader : Updating cluster state from ZooKeeper...
2016-07-24 13:29:01.990 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ZkStateReader : Loaded empty cluster properties
2016-07-24 13:29:01.995 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ZkStateReader : Updated live nodes from ZooKeeper... (0) -> (2)
2016-07-24 13:31:24.653 ERROR 83666 --- [qtp699221219-28] o.a.s.client.solrj.impl.CloudSolrClient : Request to collection foo failed due to (0) java.net.ConnectException: Operation timed out, retry? 0
and my code is:
String zkHostString = "localhost:2181";
CloudSolrClient solr = new CloudSolrClient.Builder().withZkHost(zkHostString).build();
solr.setDefaultCollection("foo");
SolrQuery query = new SolrQuery();
query.set("q", "*:*");
QueryResponse response = null;
try {
response = solr.query(query);
} catch (SolrServerException e) {
return null;
}
//Do Something with the results...
Urgh, I'm an idiot, the zookeeper instance and solr instances are inside docker, the code posted above is not. So Zookeeper reported back the solr urls using the docker containers ip...The host needs to connect via localhost and not the docker container ip.
Eg: Zookeeper responds [http://172.17.0.5:8983/solr/foo_shard1_replica2, http://172.17.0.6:8984/solr/foo_shard1_replica1]
but my code needs to call [http://localhost:8983/solr/foo_shard1_replica2, http://localhost:8984/solr/foo_shard1_replica1]
I have written an encoder that does encode the message I am sending on the wire. In my handler I issued the the ctx.writeAndFlush() method but nothing seems to be written to the remote endpoint. These are my code snippets:
Encoder
#ChannelHandler.Sharable
public class FreeSwitchEncoder extends MessageToByteEncoder<BaseCommand> {
/**
* Logger property
*/
private final Logger _log = LoggerFactory.getLogger(this.getClass());
/**
* Character set delimiting the end of each FreeSwitch message parts
*/
private final String MESSAGE_END_STRING = "\n\n";
private final Charset _encoding = StandardCharsets.UTF_8;
#Override
protected void encode(ChannelHandlerContext ctx, BaseCommand msg, ByteBuf out) throws Exception {
// Get the string representation of the BaseCommand
String toSend = msg.toString().trim();
// Let us check whether the command ends with \n\n
if (!StringUtils.isEmpty(toSend)) {
if (!toSend.endsWith(MESSAGE_END_STRING)) toSend = toSend + MESSAGE_END_STRING;
_log.debug("Encoded message sent [{}]", toSend);
ByteBuf encoded = Unpooled.copiedBuffer(toSend.getBytes());
// encoded = encoded.capacity(encoded.readableBytes());
out.writeBytes(encoded);
}
}
}
The code that sends the message
public FreeSwitchMessage sendCommand(ChannelHandlerContext ctx,
final BaseCommand command) {
ManuelResetEvent manuelResetEvent = new ManuelResetEvent();
syncLock.lock();
try {
manuelResetEvents.add(manuelResetEvent);
_log.debug("Command sent to freeSwitch [{}]", command.toString());
ChannelFuture future = ctx.writeAndFlush(command);
future.addListener(new ChannelFutureListener() {
public void operationComplete(ChannelFuture future) throws Exception {
if(!future.isSuccess()){
future.cause().printStackTrace();
}
}
});
} finally {
syncLock.unlock();
}
// Block until the response is available
return manuelResetEvent.get();
}
I do not know what I am doing wrong. The code hangs at the writting to the wire. Please assist. This is the log:
06:53:44.417 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as the default logging framework
06:53:44.421 [main] DEBUG i.n.c.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 16
06:53:44.432 [main] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
06:53:44.432 [main] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
06:53:44.433 [main] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
06:53:44.433 [main] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: true
06:53:44.433 [main] DEBUG i.n.util.internal.PlatformDependent - Platform: Windows
06:53:44.434 [main] DEBUG i.n.util.internal.PlatformDependent - Java version: 8
06:53:44.434 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noUnsafe: false
06:53:44.434 [main] DEBUG i.n.util.internal.PlatformDependent - sun.misc.Unsafe: available
06:53:44.434 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noJavassist: false
06:53:44.435 [main] DEBUG i.n.util.internal.PlatformDependent - Javassist: unavailable
06:53:44.435 [main] DEBUG i.n.util.internal.PlatformDependent - You don't have Javassist in your class path or you don't have enough permission to load dynamically generated classes. Please check the configuration for better performance.
06:53:44.435 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.tmpdir: C:\Users\smsgh\AppData\Local\Temp (java.io.tmpdir)
06:53:44.436 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
06:53:44.436 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
06:53:44.458 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.noKeySetOptimization: false
06:53:44.459 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.selectorAutoRebuildThreshold: 512
06:53:44.487 [main] INFO io.freeswitch.OutboundTest - Client connecting ..
06:53:44.517 [main] DEBUG i.n.util.internal.ThreadLocalRandom - -Dio.netty.initialSeedUniquifier: 0xf77d8a3de122380b (took 9 ms)
06:53:44.559 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: unpooled
06:53:44.559 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 65536
06:53:44.591 [nioEventLoopGroup-2-1] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetectionLevel: simple
06:53:44.596 [nioEventLoopGroup-2-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.maxCapacity.default: 262144
06:53:44.599 [nioEventLoopGroup-2-1] DEBUG i.f.codecs.FreeSwitchDecoder - read header line [Content-Type: auth/request]
06:53:44.601 [nioEventLoopGroup-2-1] DEBUG i.f.message.FreeSwitchMessage - adding header [CONTENT_TYPE] [auth/request]
06:53:44.601 [nioEventLoopGroup-2-1] DEBUG i.f.codecs.FreeSwitchDecoder - read header line []
06:53:44.602 [nioEventLoopGroup-2-1] DEBUG io.netty.util.internal.Cleaner0 - java.nio.ByteBuffer.cleaner(): available
06:53:44.602 [nioEventLoopGroup-2-1] DEBUG i.n.channel.DefaultChannelPipeline - Discarded inbound message FreeSwitchMessage: contentType=[auth/request] headers=1, body=0 lines. that reached at the tail of the pipeline. Please check your pipeline configuration.
06:53:44.603 [nioEventLoopGroup-2-1] DEBUG i.f.outbound.OutboundHandler - Received message: [FreeSwitchMessage: contentType=[auth/request] headers=1, body=0 lines.]
06:53:44.607 [nioEventLoopGroup-2-1] DEBUG i.f.outbound.OutboundHandler - Auth requested, sending [auth *****]
06:53:44.683 [nioEventLoopGroup-2-1] DEBUG i.f.outbound.OutboundHandler - Command sent to freeSwitch [auth ClueCon]
06:53:44.683 [nioEventLoopGroup-2-1] DEBUG i.f.codecs.FreeSwitchEncoder - Encoded message sent [auth ClueCon
]
I am embedding the apache ftpserver in my java web app, I have tried a couple different things and everytime I try to connect to the server via FTP, it says Authentication failed. I have attached my code, and also the console output when I try to do the logging in. Does anyone have any suggestions? I tried running this on OS X and Centos Linux.
PropertiesUserManagerFactory userManagerFactory = new PropertiesUserManagerFactory();
userManagerFactory.setFile(new File("/ftpdata/myusers.properties"));
//userManagerFactory.setPasswordEncryptor(new SaltedPasswordEncryptor());
userManagerFactory.setPasswordEncryptor(new ClearTextPasswordEncryptor());
UserManager um = userManagerFactory.createUserManager();
BaseUser user = new BaseUser();
user.setName("andy");
user.setPassword("12345678");
user.setHomeDirectory("/ftpdata/");
um.save(user);
FtpServerFactory serverFactory = new FtpServerFactory();
ListenerFactory factory = new ListenerFactory();
factory.setPort(2225);
serverFactory.addListener("default", factory.createListener());
FtpServer server = serverFactory.createServer();
server.start();
12:03:28,768 INFO [STDOUT] 12:03:28,768 INFO [DefaultFtpServer] FTP server started
12:04:14,200 INFO [STDOUT] 12:04:14,200 INFO [FtpLoggingFilter] CREATED
12:04:14,208 INFO [STDOUT] 12:04:14,208 INFO [FtpLoggingFilter] OPENED
12:04:14,228 INFO [STDOUT] 12:04:14,228 INFO [FtpLoggingFilter] RECEIVED: USER andy
12:04:14,230 INFO [STDOUT] 12:04:14,230 INFO [FtpLoggingFilter] SENT: 220 Service ready for new user.
12:04:14,232 INFO [STDOUT] 12:04:14,232 INFO [FtpLoggingFilter] SENT: 331 User name okay, need password for andy.
12:04:14,233 INFO [STDOUT] 12:04:14,233 INFO [FtpLoggingFilter] RECEIVED: PASS *****
12:04:14,236 INFO [STDOUT] 12:04:14,236 WARN [PASS] User failed to log in
12:04:14,736 INFO [STDOUT] 12:04:14,736 WARN [PASS] Login failure - andy
12:04:14,740 INFO [STDOUT] 12:04:14,740 INFO [FtpLoggingFilter] RECEIVED: QUIT
12:04:14,746 INFO [STDOUT] 12:04:14,746 INFO [FtpLoggingFilter] SENT: 530 Authentication failed.
12:04:14,746 INFO [STDOUT] 12:04:14,746 INFO [FtpLoggingFilter] SENT: 221 Goodbye.
12:04:14,747 INFO [STDOUT] 12:04:14,746 INFO [FtpLoggingFilter] CLOSED
my question may sound stupid but
i can see in the log
12:04:14,233 INFO [STDOUT] 12:04:14,233 INFO [FtpLoggingFilter] RECEIVED: PASS *****
These are 5 stars , your password length is 8 :D
I can see that you pass a property file, is there any chance that you have a password in there?
At last, you create a UserManager, you dont have to pass this object somewhere?