When trying to submit my topology through StormSubmitter, I am getting -
Caused by: java.lang.NoSuchFieldError: INSTANCE
at org.apache.http.impl.io.DefaultHttpRequestWriterFactory.<init>(DefaultHttpRequestWriterFactory.java:52)
I am using Spring.
I am not initializing HttpClient in Spout/Bolt Constructor. Instead its initialized in constructor of a class that is being fetched from Spring Context in prepare() method of bolt
Code is structured as follows -
SomeBolt.java
#Component
public class SomeBolt extends BaseRichBolt {
private OutputCollector _collector;
private SomeClient someClient;
#Override
public void prepare(Map conf, TopologyContext context, OutputCollector collector) {
_collector = collector;
someClient = AppContext.getBean(SomeClient.class);
}
}
SomeClient.java
#Component
public class SomeClient {
private final CloseableHttpClient httpClient;
public SomeClient() {
this.httpClient = (httpClient == null ? HttpClients.createDefault() : httpClient);
}
}
AppContext.java
#Component
public class AppContext implements ApplicationContextAware {
private static ApplicationContext applicationContext;
#Override
public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
AppContext.applicationContext = applicationContext;
}
public static <T> T getBean(Class<T> c) {
return applicationContext.getBean(c);
}
}
This is probably a dependency issue.
It's a very unclear error message but I found something similar here:
Hibernate NoSuchFieldError INSTANCE but only with Struts 1?
I faced similar issue like this, In my class path there was two jar contains same class, httpcore-4.3 and apache-httpcomponents-httpcore, I have removed apache-httpcomponents-httpcore from class path solved the issue.
Harsh is right its in the storm class path.
So what I did to make this work was remove the httpclient and httpcore that comes with storm and replaced them with newer version 4.3.3 and 4.3.2 respectively. This changes the classpath the works/nimbus/supervisor uses to start. You can run storm classpath and it print the class path out.
[nimbus ~]$ storm classpath
...../storm-0.8.2/lib/httpclient-4.3.3.jar:..../storm-0.8.2/lib/httpcore-4.3.2.jar.....
I am not sure this is a very good work around, I am not sure what part of storm uses this jar.
if you look at the python storm code you see that it will put all jars in the storm root and storm/lib
def get_classpath(extrajars):
ret = get_jars_full(STORM_DIR)
ret.extend(get_jars_full(STORM_DIR + "/lib"))
ret.extend(extrajars)
return normclasspath(":".join(ret))
I had the below jar files in the path inside the plugin folder:
./var/lib/jenkins/plugins/build-pipeline-plugin/WEB-INF/lib/httpcore-4.2.1.jar
./var/lib/jenkins/plugins/git-client/WEB-INF/lib/httpcore-4.3.2.jar
./var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/httpcore-4.2.4.jar
After, I removed the below file, it worked for me
/var/lib/jenkins/plugins/build-pipeline-plugin/WEB-INF/lib/httpcore-4.2.1.jar
Related
I have a bean with a constructor as follows. The password argument is resolved from the placeholder my.password, with a default value of DEFAULT. If the value of DEFAULT is passed, a warning is logged. Note - this Bean is contained within an imported third-party library.
#Bean
public class EncryptionBean {
public EncryptionBean(#Value("${my.password}") String password) {
if "DEFAULT".equals(password) {
// log warning message
} else {
// do stuff with the password
}
}
}
The password is retrieved at startup from an external system using a client SDK. This SDK object is itself provided as a Bean (also from a third-party library). After retrieving the password, I am setting it as a System property for the above EncryptionBean to have access to at the time of instantiation:
#Configuration
public class MyConfiguration {
#Autowired
public SDKObject sdkObject;
#PostConstruct
public void init() {
System.setProperty("my.password", sdkObject.retrievePassword());
// #Value("${my.password"}) should now be resolvable when EncryptionBean is instantiated
}
}
However, EncryptionBean is still being instantiated with a value of DEFAULT for my.password. I'm wondering if System.setProperty in #PostConstruct might be getting executed AFTER Spring has already instantiated the instance of EncryptionBean?
If so, is there a way to guarantee this property has been set before Spring instantiates EncryptionBean? I came across #DependsOn as a way to control the order Beans get instantiated by Spring, but since EncryptionBean comes from a third-party library, I haven't been able to make this annotation work.
Instead of setting a system property, you should create a Spring EnvironmentPostProcessor class to retrieve the password from the external source and add it to the Spring Environment. That would look something like this:
public class PasswordEnvironmentPostProcessor implements EnvironmentPostProcessor, ApplicationContextAware {
private ApplicationContext applicationContext;
#Override
public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
SDKObject sdkObject = applicationContext.getBean(SDKObject.class);
Map<String, Object> properties = Collections.singletonMap("my.password", sdkObject.retrievePassword());
MapPropertySource propertySource = new MapPropertySource("password", properties);
environment.getPropertySources().addFirst(propertySource);
}
#Override
public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
this.applicationContext = applicationContext;
}
}
Then you'll need to register this class with Spring by adding an entry to the file META-INF/spring.factories that looks like this:
org.springframework.boot.env.EnvironmentPostProcessor=com.example.PaswordEnvironmentPostProcessor
Documentation for this is available here: https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#howto.application.customize-the-environment-or-application-context.
I could not figure out a nice clean way to inject a property at runtime without a lot of boilerplate. But I came up with a clean way to do what you want to do without having to refresh the application context or messing with the 3rd party library implementation.
First we exclude the 3rd party bean from our application context:
#ComponentScan(excludeFilters = #ComponentScan.Filter(value = EncryptionBean.class, type = FilterType.ASSIGNABLE_TYPE))
#SpringBootApplication
public class SandboxApplication {
public static void main(String[] args) {
SpringApplication.run(SandboxApplication.class, args);
}
}
Then we create the Bean ourselves with the values we want.
#Configuration
public class MyConfiguration {
public final SDKObject sdkObject;
public MyConfiguration(SDKObject sdkObject) {
this.sdkObject = sdkObject;
}
#Bean
public EncryptionBean encryptionBean() {
return new EncryptionBean(sdkObject.retrievePassword());
}
}
I developed a kind of wrapper to make it work as a custom logger. I'm instantiating this class using #CustomLog Lombok annotation just to make it easier and cleaner. The tricky thing comes next: the idea behind this wrapper is to use a common logger (as org.slf4j.Logger) along with a custom monitor class that each time I call log.error(), the proper message gets logged in the terminal and the event is sent to my monitoring tool (Prometheus in this case).
To achieve this I did the following classes:
CustomLoggerFactory the factory called by Lombok to instantiate my custom logger.
public final class CustomLoggerFactory {
public static CustomLogger getLogger(String className) {
return new CustomLogger(className);
}
}
CustomLogger will receive the class name just to then call org.slf4j.LoggerFactory.
public class CustomLogger {
private org.slf4j.Logger logger;
private PrometheusMonitor prometheusMonitor;
private String className;
public CustomLogger(String className) {
this.logger = org.slf4j.LoggerFactory.getLogger(className);
this.className = className;
this.monitor = SpringContext.getBean(PrometheusMonitor.class);
}
}
PrometheusMonitor class is the one in charge of creating the metrics and that kind of things. The most important thing here is that it's being managed by Spring Boot.
#Component
public class PrometheusMonitor {
private MeterRegistry meterRegistry;
public PrometheusMonitor(MeterRegistry meterRegistry) {
this.meterRegistry = meterRegistry;
}
}
As you may noticed, to access PrometheusMonitor from CustomLogger I need an additional class in order to get the Bean / access the context from a non Spring managed class. This is the SpringContext class which has an static method to get the bean by the class supplied.
#Component
public class SpringContext implements ApplicationContextAware {
private static ApplicationContext context;
public static <T extends Object> T getBean(Class<T> beanClass) {
return context.getBean(beanClass);
}
#Override
public void setApplicationContext(ApplicationContext context) throws BeansException {
SpringContext.context = context;
}
}
So all this works just fine when running the application. I ensure to load SpringContext class before anything else, so once each CustomLogger gets instantiated it just works.
But the BIG issue comes here: this is not working while unit testing my app. I tried many things and I saw some solutions that may help me but that I'm trying to avoid (e.g. using PowerMockito). Lombok is processing #CustomLog annotation before any #Before method I add to my test class. Once getBean() method is called I get an exception cause context is null.
My guesses are that I could solve it if I can force the SpringContext to be loaded before Lombok does its magic, but I'm not sure that's even possible. Many thanks for taking your time to read this. Any more info I can provide just let me know.
NOTE: It sounds like your custom logging needs are better served by logging to slf4j as normal, and registering an additional handler with the slf4j framework so that slf4j will forward any logs to you (in addition to the other handlers, such as the one making the log files).
Lombok is processing #CustomLog
The generated log field is static. If an annotation is going to help at all, you'd need #BeforeClass, but that probably also isn't in time. Lombok's magic doesn't seem relevant here. Check out what delombok tells you lombok is doing: It's just.. a static field, being initialized on declaration.
Well I managed to solve this issue changing a little how the CustomLogger works. Meaning that instead of instantiating monitor field along with the logger, you can do it the first time you'll use it. E.g.:
public class CustomLogger {
private org.slf4j.Logger logger;
private Monitor monitor;
public CustomLogger(String className) {
this.logger = org.slf4j.LoggerFactory.getLogger(className);
}
public void info(String message) {
this.logger.info(message);
}
public void error(String message) {
this.logger.error(message);
if (this.monitor == null) {
this.monitor = SpringContext.getBean(PrometheusMonitor.class);
}
this.monitor.send(message);
}
}
But after all I decided to not follow this approach because I don't think it's the best one possible and worth it.
I am new to dropwizard, and am using GuiceBundle and MongoBundle in my application.
The MongoClient is wrapped in a dropwizard Managed object and is tied to the lifecycle of the application.
public class SalApplication extends Application<SomeConf> {
...
private GuiceBundle<SomeConf> guiceBundle;
private MongoBundle<SomeConf> mongoBundle;
...
#Override
public void initialize(Bootstrap<SomeConf> bootstrap) {
// build bundles and add to bootstrap
...
}
#Override
public void run(SomeConf someConf, Environment env) throws Exception{
...
MongoClient client = mongoBundle.getClient();
MongoClientManager mongoDB = new MongoClientManager(client);
env.lifecycle().manage(mongoDB); //MongoClientManager implements Managed
}
My hiccup is, how do I get hold of the MongoClient object.
The object is supposed to be injected into my DAOs.
But how can I access the MongoClient object from inside guice Module.
If I construct another MongoClient object inside guice module, then what is the point of the Managed Object. I'm really confused.
I would recommend writing your own Guice module. While the dropwizard-guice is quite useful it has not been updated since Feb 2017 and only supports up to version 1.0.0 of DW. A basic version of your module could look like this:
public class CustomModule implements Module {
private final MongoClient mongoClient;
public CustomModule(MongoClient mongoClient) {
this.mongoClient = mongoClient;
}
#Override
public void configure(Binder binder) {
binder.bind(MongoClient.class).toInstance(mongoClient);
}
}
Which then you can initialize from the "run" method in your Application class using the Guice standard methods:
public class SalApplication extends Application<SomeConf> {
...
#Override
public void run(SomeConf someConf, Environment env) throws Exception{
Guice.createInjector(new CustomModule(mongoBundle.getClient());
...
}
}
I found a simpler way to achieve what I needed.
I was previously using com.meltmedia.dropwizard.dropwizard-mongo, whose MongoBundle constructs the MongoClient, which had to be passed to my ManagedObject.
I stopped using it. Instead I'm constructing the MongoClient object myself using mongo-java-driver inside guice module, and is injected into the constructor of my managed object.
We recently migrated from java 7 to java 8 and tomcat 6 to tomcat 8. All was well before migration and now we are getting below issue during deployment of our services and deployment fails and tomcat does not start.
What is crazy is that, this does not happen always and is intermittent (occurs once in three or four deployments).
Below is the stack trace that we see during deployment.
Caused by: java.lang.NullPointerException
at com.async.ClassB.createClassC(ClassB.java)
at com.async.ClassA.afterPropertiesSet(ClassA.java)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1573)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1511)
Existing code:
In our code we have a singleton classA, injected with singleton ClassB. And also a prototype ClassC used in ClassB.
In ClassA, we have afterPropertiesSet(), where-in we are getting instance of ClassB using applicationContext.
In ClassB we are trying to get the instance of ClassC using BeanFactory.
Exact code looks like this.
public ClassA implements ApplicationContextAware, InitializingBean {
private ApplicationContext applicationContext;
public void afterPropertiesSet() throws Exception {
ClassB classB =
(ClassB)applicationContext.getBean("classB");
registry.registerAsyncListener(this);
}
public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
this.applicationContext = applicationContext;
}
}
public ClassB implements Lifecycle, DisposableBean, BeanFactoryAware {
private AsyncMessageListener createClassC(ClassA classA) {
logger.warn("bean factory value ... " + getBeanFactory());
ClassC classC = (ClassC) getBeanFactory().getBean("classC", ClassC.class);
return classC;
}
public void setBeanFactory(BeanFactory beanFactory) throws BeansException {
this.beanFactory = beanFactory;
}
public start(){ // some code
}
public stop(){ // some code
}
public isRunning(){ // some code
}
public destroy(){ // some code
}
}
When we added more loggers, what we saw is that beanFactory ggets the null value intermittently and hence the NullpointerException.
I am not very much into springs, and did some basic research. It looks like like there is no code issue and still i wonder why BeanFactory is getting null value.
We tried replacing BeanFactory with ApllicationContext the still the same issue.
Any help will be much appreciated.
Considering a Spring Boot CommandLineRunner Application, I would like to know how to filter the "switch" options passed to Spring Boot as externalized configuration.
For example, with:
#Component
public class FileProcessingCommandLine implements CommandLineRunner {
#Override
public void run(String... strings) throws Exception {
for (String filename: strings) {
File file = new File(filename);
service.doSomething(file);
}
}
}
I can call java -jar myJar.jar /tmp/file1 /tmp/file2 and the service will be called for both files.
But if I add a Spring parameter, like java -jar myJar.jar /tmp/file1 /tmp/file2 --spring.config.name=myproject then the configuration name is updated (right!) but the service is also called for file ./--spring.config.name=myproject which of course doesn't exist.
I know I can filter manually on the filename with something like
if (!filename.startsWith("--")) ...
But as all of this components came from Spring, I wonder if there is not a option somewhere to let it manage it, and to ensure the strings parameter passed to the run method will not contain at all the properties options already parsed at the Application level.
Thanks to #AndyWilkinson enhancement report, ApplicationRunner interface was added in Spring Boot 1.3.0 (still in Milestones at the moment, but will soon be released I hope)
Here the way to use it and solve the issue:
#Component
public class FileProcessingCommandLine implements ApplicationRunner {
#Override
public void run(ApplicationArguments applicationArguments) throws Exception {
for (String filename : applicationArguments.getNonOptionArgs())
File file = new File(filename);
service.doSomething(file);
}
}
}
There's no support for this in Spring Boot at the moment. I've opened an enhancement issue so that we can consider it for a future release.
One option is to use Commons CLI in the run() of your CommandLineRunner impl.
There is a related question that you may be interested.
Here is another solution :
#Component
public class FileProcessingCommandLine implements CommandLineRunner {
#Autowired
private ApplicationConfig config;
#Override
public void run(String... strings) throws Exception {
for (String filename: config.getFiles()) {
File file = new File(filename);
service.doSomething(file);
}
}
}
#Configuration
#EnableConfigurationProperties
public class ApplicationConfig {
private String[] files;
public String[] getFiles() {
return files;
}
public void setFiles(String[] files) {
this.files = files;
}
}
Then run the program :
java -jar myJar.jar --files=/tmp/file1,/tmp/file2 --spring.config.name=myproject