Unable to connect to S3 using spark-redshift library in java - java

I am trying to create a table in Redshift based on the spark dataset. I am using spark-redshift driver in jdbc to achieve this locally.
The code snippet to do this
data.write()
.format("com.databricks.spark.redshift")
.option("url", "jdbc:redshift://..")
.option("dbtable", "test_table")
.option("tempdir", "s3://temp")
.option("aws_iam_role", "arn:aws:iam::..")
.option("extracopyoptions", "region 'us-west-1'")
.mode(SaveMode.Append).save();
My maven pom.xml has following dependency:
<dependency>
<groupId>com.databricks</groupId>
<artifactId>spark-redshift_2.11</artifactId>
<version>2.0.1</version>
</dependency>
I am using java 1.8.
I am getting the following error:
java.io.IOException: No FileSystem for scheme: s3
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2660)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
at com.databricks.spark.redshift.Utils$.assertThatFileSystemIsNotS3BlockFileSystem(Utils.scala:156)
at com.databricks.spark.redshift.RedshiftWriter.saveToRedshift(RedshiftWriter.scala:340)
at com.databricks.spark.redshift.DefaultSource.createRelation(DefaultSource.scala:106)
at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:45)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:86)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)
at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:80)
at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:80)
at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:668)
at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:668)
at org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:78)
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73)
at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:668)
at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:276)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:270)
at com.peak.spark.jobs.SparkDataIngestJob.writeData(SparkDataIngestJob.java:196)
at com.peak.spark.jobs.SparkDataIngestJob.exec(SparkDataIngestJob.java:123)
at com.peak.spark.core.AbstractSparkJob.run(AbstractSparkJob.java:74)
at com.peak.spark.core.SparkAppLauncher.onApplicationEvent(SparkAppLauncher.java:40)
at com.peak.spark.core.SparkAppLauncher.onApplicationEvent(SparkAppLauncher.java:16)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:151)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:128)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:331)
at org.springframework.context.support.AbstractApplicationContext.start(AbstractApplicationContext.java:1174)
at com.peak.spark.core.SparkApp.launch(SparkApp.java:38)
at com.peak.spark.core.SparkApp.main(SparkApp.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Exception in thread "main" java.lang.RuntimeException: Spark Job FailedNo FileSystem for scheme: s3
at com.peak.spark.jobs.SparkDataIngestJob.exec(SparkDataIngestJob.java:162)
at com.peak.spark.core.AbstractSparkJob.run(AbstractSparkJob.java:74)
at com.peak.spark.core.SparkAppLauncher.onApplicationEvent(SparkAppLauncher.java:40)
at com.peak.spark.core.SparkAppLauncher.onApplicationEvent(SparkAppLauncher.java:16)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:151)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:128)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:331)
at org.springframework.context.support.AbstractApplicationContext.start(AbstractApplicationContext.java:1174)
at com.peak.spark.core.SparkApp.launch(SparkApp.java:38)
at com.peak.spark.core.SparkApp.main(SparkApp.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Please help me figure out what's wrong here.

I suppose you forget to include hadoop-aws package to your project. This package will allow you to work with s3:// schema
<!-- https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-aws -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-aws</artifactId>
<version>2.6.0</version>
</dependency>

Since you are trying to execute this code on your local system, your code won't know how to access s3 file system.
You can do one of the two things to solve this problem:
Configure AWS credentials in your system so that your code will somehow try to reach s3 bucket. I will not recommend this approach for various reasons.
Keep your file path in configuration file. Use 2 configuration files - one for testing code and other for production environment. In test environment, use paths like c:\path\to\your\dummy\folder\ and in production environment configuration file use paths like s3:\your_bucket_name\path\in\bucket.
Hope it helps.

Related

" Could not find API definition for name "JDO"" when launched through jar

So when I try to connect to my hive metastore I'm getting this exception
javax.jdo.JDOFatalInternalException: Unexpected exception caught.
at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1193)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:412)
at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:441)
at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:336)
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:292)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:60)
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:69)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:681)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:659)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:708)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:507)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6286)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:207)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:187)
at com.bouygtel.jupiter.dic.catalogue.AspirationMetastore.aspirer(AspirationMetastore.java:146)
at com.bouygtel.jupiter.dic.integration.adaptateurs.AdaptateurUpdate.lambda$updateDatabase$0(AdaptateurUpdate.java:68)
at com.bouygtel.commun.summer.socle.coeur.threads.SummerExecutorServiceFactory$3.run(SummerExecutorServiceFactory.java:160)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
at java.security.AccessController.doPrivileged(Native Method)
at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
... 25 common frames omitted
Caused by: org.datanucleus.exceptions.NucleusUserException: Error : Could not find API definition for name "JDO". Perhaps you dont have the requisite datanucleus-api-XXX jar in the CLASSPATH?
at org.datanucleus.api.ApiAdapterFactory.getApiAdapter(ApiAdapterFactory.java:93)
at org.datanucleus.AbstractNucleusContext.<init>(AbstractNucleusContext.java:118)
at org.datanucleus.PersistenceNucleusContextImpl.<init>(PersistenceNucleusContextImpl.java:170)
at org.datanucleus.PersistenceNucleusContextImpl.<init>(PersistenceNucleusContextImpl.java:159)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.<init>(JDOPersistenceManagerFactory.java:436)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:314)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:223)
... 33 common frames omitted
But I have this in my pom.xml
<dependency>
<groupId>org.datanucleus</groupId>
<artifactId>datanucleus-api-jdo</artifactId>
<version>5.0.3</version>
</dependency>
The weird thing is that this only happens when I use my code through the jar, when I run it from eclipse, the thing works perfectly.
I double checked through "jar tf" and the jar that I run does contain the "datanucleus-api-jdo-5.0.3.jar".
I've searched for conflicts in my dependency hierarchy and there is one dependency that also use an older version of the "datanucleus-api-jdo", but it's omitted for conflict according to eclipse and excluding it does not solve the problem
Not sure if that helps but the run configuration within eclipse is "clean package spring-boot:run".
Any ideas of what is causing this?
If you think any information is missing please ask

Java "InvalidAlgorithmParameterException"?

Currently i am using web3j to create an account in my private block chain.
Like:
Web3j web3 = Web3j.build(new HttpService());
Web3ClientVersion web3ClientVersion;
String walletFileName = WalletUtils.generateFullNewWalletFile("password",new File("/opt/test"));
LOGGER.info("walletFileName>>>>>" + walletFileName.substring(0));
This code works fine in my local machine.But i am getting this exception in my test server:
java.security.InvalidAlgorithmParameterException: parameter object not a ECParameterSpec
at org.bouncycastle.jce.provider.JDKKeyPairGenerator$EC.initialize(Unknown Source)
at org.web3j.crypto.Keys.createSecp256k1KeyPair(Keys.java:49)
at org.web3j.crypto.Keys.createEcKeyPair(Keys.java:55)
at org.web3j.crypto.WalletUtils.generateNewWalletFile(WalletUtils.java:44)
at org.web3j.crypto.WalletUtils.generateFullNewWalletFile(WalletUtils.java:29)
at com.belrium.service.UserWalletService.createNewWallet(UserWalletService.java:91)
at com.belrium.service.UserAuthenticationService.verifyEmailAddress(UserAuthenticationService.java:126)
at com.belrium.controller.UserAuthenticationController.verifyEmailAddress(UserAuthenticationController.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
I have searched a lot and tried all things but not able to resolve it.Please help.
This error is typically related to your current JDK/ceritificate/configurations installations :
Solution : RE-install the JDK/configurations on your test server.
Other possible Issue :
If you have any SSL certificate on your test server ? please refer to below URL to understand the issue :
Please go through http://iwang.github.io/support/2014/03/14/cxf-cause-https-error.html
I was using this maven plugin
<!-- flying saucer pdf -->
<!-- <dependency>
<groupId>org.xhtmlrenderer</groupId>
<artifactId>flying-saucer-pdf</artifactId>
<version>9.1.4</version>
</dependency> -->
for creating pdf. By removing this plugin from my pom.xml my issue is ressolved.But why it was creating that exception not know.

Spring Boot JSP

Is there any simple way to generate Spring Boot Mvc project for .jsp views?
I've tried to create project in similar way like Spring Boot Rest project, but it's not soo easy for me. I've tried to use official Spring Boot Mvc jsp sample, but when I try to run this I get exception like this:
org.springframework.beans.factory.BeanDefinitionStoreException: Failed to parse configuration class [sample.jsp.SampleWebJspApplication]; nested exception is java.lang.IllegalStateException: Failed to introspect annotated methods on class org.springframework.boot.context.web.SpringBootServletInitializer
at org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:187) ~[spring-context-4.3.0.BUILD-20160427.201731-353.jar:4.3.0.BUILD-SNAPSHOT]
at org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:321) ~[spring-context-4.3.0.BUILD-20160427.201731-353.jar:4.3.0.BUILD-SNAPSHOT]
at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:243) ~[spring-context-4.3.0.BUILD-20160427.201731-353.jar:4.3.0.BUILD-SNAPSHOT]
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanDefinitionRegistryPostProcessors(PostProcessorRegistrationDelegate.java:273) ~[spring-context-4.3.0.BUILD-20160427.201731-353.jar:4.3.0.BUILD-SNAPSHOT]
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:98) ~[spring-context-4.3.0.BUILD-20160427.201731-353.jar:4.3.0.BUILD-SNAPSHOT]
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:681) ~[spring-context-4.3.0.BUILD-20160427.201731-353.jar:4.3.0.BUILD-SNAPSHOT]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:523) ~[spring-context-4.3.0.BUILD-20160427.201731-353.jar:4.3.0.BUILD-SNAPSHOT]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:765) [spring-boot-1.4.0.BUILD-20160427.203128-289.jar:1.4.0.BUILD-SNAPSHOT]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:370) [spring-boot-1.4.0.BUILD-20160427.203128-289.jar:1.4.0.BUILD-SNAPSHOT]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.BUILD-20160427.203128-289.jar:1.4.0.BUILD-SNAPSHOT]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1182) [spring-boot-1.4.0.BUILD-20160427.203128-289.jar:1.4.0.BUILD-SNAPSHOT]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1171) [spring-boot-1.4.0.BUILD-20160427.203128-289.jar:1.4.0.BUILD-SNAPSHOT]
at sample.jsp.SampleWebJspApplication.main(SampleWebJspApplication.java:33) [classes/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_65]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_65]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_65]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_65]
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) [idea_rt.jar:na]
Caused by: java.lang.IllegalStateException: Failed to introspect annotated methods on class org.springframework.boot.context.web.SpringBootServletInitializer
at org.springframework.core.type.StandardAnnotationMetadata.getAnnotatedMethods(StandardAnnotationMetadata.java:163) ~[spring-core-4.3.0.BUILD-20160427.201731-353.jar:4.3.0.BUILD-SNAPSHOT]
at org.springframework.context.annotation.ConfigurationClassParser.doProcessConfigurationClass(ConfigurationClassParser.java:300) ~[spring-context-4.3.0.BUILD-20160427.201731-353.jar:4.3.0.BUILD-SNAPSHOT]
at org.springframework.context.annotation.ConfigurationClassParser.processConfigurationClass(ConfigurationClassParser.java:237) ~[spring-context-4.3.0.BUILD-20160427.201731-353.jar:4.3.0.BUILD-SNAPSHOT]
at org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:204) ~[spring-context-4.3.0.BUILD-20160427.201731-353.jar:4.3.0.BUILD-SNAPSHOT]
at org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:173) ~[spring-context-4.3.0.BUILD-20160427.201731-353.jar:4.3.0.BUILD-SNAPSHOT]
... 17 common frames omitted
Caused by: java.lang.NoClassDefFoundError: javax/servlet/ServletContext
at java.lang.Class.getDeclaredMethods0(Native Method) ~[na:1.8.0_65]
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) ~[na:1.8.0_65]
at java.lang.Class.getDeclaredMethods(Class.java:1975) ~[na:1.8.0_65]
at org.springframework.core.type.StandardAnnotationMetadata.getAnnotatedMethods(StandardAnnotationMetadata.java:152) ~[spring-core-4.3.0.BUILD-20160427.201731-353.jar:4.3.0.BUILD-SNAPSHOT]
... 21 common frames omitted
Caused by: java.lang.ClassNotFoundException: javax.servlet.ServletContext
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_65]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_65]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[na:1.8.0_65]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_65]
... 25 common frames omitted
This blog post explains what steps you should take and why. Its fairly easy and I've fixed it myself.
TD:LR
Add Maven Dependencies
In order to enable JSP support, we need to add a few dependencies to our new project in pom.xml.
<dependency>
<groupId>org.apache.tomcat.embed</groupId>
<artifactId>tomcat-embed-jasper</artifactId>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
</dependency>
Place the JSP templates under src/main/resources/META-INF/resources/WEB-INF/jsp/
As I know, the spring boot default page technology is thymeleaf and the official recommends us to use it.
If you want to use jsp, you could do some configuration.
The following steps may be useful.
1. Creating src/main/webapp/WEB-INF/jsp folder.
2. Modify application.properties file.
spring.mvc.view.prefix: /WEB-INF/jsp/
spring.mvc.view.suffix: .jsp
3.Add jar support for jsp. You could add the followings maven repository to your project pom.xml file.
<dependency>
<groupId>org.apache.tomcat.embed</groupId>
<artifactId>tomcat-embed-jasper</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
</dependency>
Hope it helps. Any concerns, please feel free to let me know.
Add below dependencies to your POM file:
<dependency>
<groupId>org.apache.tomcat.embed</groupId>
<artifactId>tomcat-embed-jasper</artifactId>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
</dependency>
Create below structure
/src/main/resources/META-INF/resources
Now the above folder is equivalent to your WebContent folder. Inside this you can put your static contents. Add WEB-INF folder here and now you can place your JSP files like any other web project.
In your application.properties file add below:
spring.mvc.view.prefix=/WEB-INF/jsp/
spring.mvc.view.suffix=.jsp
Any chance you're running in IntelliJ? I've noticed this issue as well and the solution is to remove the <scope>provided</scope> from the spring-boot-starter-tomcat dependency, like this:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
<!-- <scope>provided</scope> -->
</dependency>
There's a bug in IntelliJ, causing it not to add provided dependencies to the classpath when running locally.
EDIT: Removing the provided scope might cause issues when you're going to run it within a servlet container, so be cautious about that, an explanation about how to fix that could be found in Run Spring-boot's main using IDE.
I have had similar problems, see the link to github.
jsf-spring-boot-2.1.6 parent use spring boot 1.4.0
everything has worked and there is no problems, believe it or not!
https://github.com/joinfaces/joinfaces-example-war
Adding dependency in POM file below
<dependency>
<groupId>org.apache.tomcat.embed</groupId>
<artifactId>tomcat-embed-jasper</artifactId>
<scope>provided</scope>
</dependency>
Full code avaliable on github link click here
Download the code and import the eclipse/STS using exiting maven project
Next to run the project.
Go to the package on com.chandrakumar.test.demo.
it has "DemoApplication.java" and right click the mouse and select run as
choose the java application.
The browser address bar type "http://localhost:8080/".
Following johnny's reply, I actually wasn't aware that JSP files could be ran while deploying the application as a JAR. Though apparently there are some extensions that I haven't heard of... Typically, a JSP application is deployed as a WAR, and all JSP-related files are located in the webapp/WEB-INF/ folder. You may need to add <packaging>war</packaging> to your POM if if you are comfortable packaging your application as a WAR. This doesn't sound like a bad idea if your application is not a microservice or does not utilize cloud-based technologies. But the majority of cloud-based technologies (excluding cloud-foundry and maybe some others) only support JAR packaging. In that case, you may want to use alternative templates such as Thymeleaf or FreeMarker, over JSP, if you decide to go the JAR route.
Use a maven run configuration with spring-boot:run as the "Command line" value.

java ee: class not found despite valid import?!

I use jersey to create some sort of RESTful API. To get information about the current request, I use the HttpServletRequest Object. I am able to compile the project without problems or errors, but when I run it, I get the following error:
[INFO] --- exec-maven-plugin:1.2.1:java (default-cli) # backend ---
[WARNING]
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: javax/servlet/http/HttpServletRequest
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2570)
at java.lang.Class.getDeclaredMethods(Class.java:1855)
at org.glassfish.jersey.server.model.IntrospectionModeller$2.run(IntrospectionModeller.java:236)
at java.security.AccessController.doPrivileged(Native Method)
at org.glassfish.jersey.server.model.IntrospectionModeller.getAllDeclaredMethods(IntrospectionModeller.java:230)
at org.glassfish.jersey.server.model.IntrospectionModeller.checkForNonPublicMethodIssues(IntrospectionModeller.java:170)
at org.glassfish.jersey.server.model.IntrospectionModeller.doCreateResourceBuilder(IntrospectionModeller.java:118)
at org.glassfish.jersey.server.model.IntrospectionModeller.access$000(IntrospectionModeller.java:80)
at org.glassfish.jersey.server.model.IntrospectionModeller$1.call(IntrospectionModeller.java:111)
at org.glassfish.jersey.server.model.IntrospectionModeller$1.call(IntrospectionModeller.java:108)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.processWithException(Errors.java:255)
at org.glassfish.jersey.server.model.IntrospectionModeller.createResourceBuilder(IntrospectionModeller.java:108)
at org.glassfish.jersey.server.model.Resource.from(Resource.java:744)
at org.glassfish.jersey.server.ApplicationHandler.initialize(ApplicationHandler.java:400)
at org.glassfish.jersey.server.ApplicationHandler.access$500(ApplicationHandler.java:163)
at org.glassfish.jersey.server.ApplicationHandler$3.run(ApplicationHandler.java:323)
at org.glassfish.jersey.internal.Errors$2.call(Errors.java:289)
at org.glassfish.jersey.internal.Errors$2.call(Errors.java:286)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.processWithException(Errors.java:286)
at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:320)
at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:285)
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.<init>(GrizzlyHttpContainer.java:331)
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpServerFactory.createHttpServer(GrizzlyHttpServerFactory.java:116)
at com.getbro.api.Main.startServer(Main.java:32)
at com.getbro.api.Main.main(Main.java:42)
... 6 more
Caused by: java.lang.ClassNotFoundException: javax.servlet.http.HttpServletRequest
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 36 more
I think it's strange, because I import this class per maven:
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
<scope>provided</scope>
</dependency>
... so it should work like expected, but it does not!
You import it in provided scope. That way you just promise don't worry, it'll be there, and nothing is added to war. Use scope compile to actually import dependancies.
Your dependency has a provided scope, which means that the artifact will be available for compiling, but not during runtime. More info on this topic in the Maven documentation or this StackOverflow question.
That's the right thing to do, because the Servlet API is not something you want to have within your WAR, EAR, or whatever package you're building. Instead, this servlet-api JAR will be provided by your application server (Tomcat, Jetty...).
If you're running your web service project from Eclipse, you need to tell him which server runtime it has to use. (If you're using an IDE other than Eclipse, you'll have to do something similar):
Window > Preferences > Server > Runtime environments.
If there is already a runtime environment defined for your server (i.e. Tomcat 6.x if that's what you're using), cancel this window. Otherwise, create it by using the Add... button.
Once there is a runtime environment defined:
Right click on your project > Build path > Configure build path....
Libraries tab > Add library.
Select the appropiate server runtime and finish.

unable to upload file to azure blob using jclouds APIs

i am using the same example given here to upload my files tu Azure blob. when i run it as a plain Java project code, it is working fine but when i create a OSGi bundle out of it and try the same thing, i am getting following exception.
2013-09-28 14:05:11,578[qtp26511982-145]|ERROR|<my_package>.fileupload.service.uploadservice.impl.UploadServiceAzureProcess|IOException java.util.NoSuchElementException: key [azureblob] not in the
list of providers or apis: {apis=[transient]}
java.util.NoSuchElementException: key [azureblob] not in the list of providers or apis: {apis=[transient]}
at org.jclouds.ContextBuilder.newBuilder(ContextBuilder.java:178)
at <my_package>.fileupload.service.uploadservice.impl.UploadServiceAzureProcess.uploadFileWithDetails(UploadServiceAzureProcess.java:99)
at <my_package>.fileupload.service.uploadservice.impl.UploadServiceImpl.uploadCaseFiles(UploadServiceImpl.java:113)
at <my_package>.fileupload.service.uploadservice.impl.UploadServiceImpl.uploadFile(UploadServiceImpl.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180)
at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:193)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:102)
at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58)
at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:94)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:271)
at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.serviceRequest(JettyHTTPDestination.java:355)
at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:319)
at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:72)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:363)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:931)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:992)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:856)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:627)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:51)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:722)
can someone please help me with this?
That is because you might not have the azureblob dependency in the application classpath. Make sure you have this dependency in your pom.xml file:
<dependency>
<groupId>org.apache.jclouds.provider</groupId>
<artifactId>azureblob</artifactId>
<version>${jclouds.version}</version>
</dependency>
Have a look at the troubleshooting guide for more information on this error.

Categories