I am trying to use java to write a MapReduce function to count the words in a csv file.
I am able to successfully build the project and I manage to create a jar file.
I'm using Hadoop for running everything and I successfully install it on my PC, manage to upload the csv file to hadoop server but when I try to run this command:
hadoop jar C:/MRProgramsDemo.jar PackageDemo.WordCount wordCountFile MRDir1
I keep getting the following error message:
try to run other programs I found online and other jar file but without succses. can anybody no what I do wrong?
Two problems.
First, you put the file at the root of HDFS, not your user folder, therefore, the parameter must be /wordCountFile
Second, it looks like you've made an executable JAR, so you can remove the class name from the parameters
Related
I want to run a fat client delivered as a Java web start application without Java web start. I launched it via javaws and managed to get all the jar files mentioned in the JNLP file from the cache after they were downloaded.
I tried running the jar file that contains the main class according to the JNLP file, but I get the 'Could not find or load main class' error. Were I just trying to run a class I'd set the classpath accordingly, but since I'm running a jar file with java -jar, as far as I know the classpath settings will be ignored anyway. Now I'm not sure what to do, does anybody know how to tackle this?
I'll answer this myself now, turns out it is stupidly simple: Get all the jar files, unzip them to get the content, merge all the content (best done with rsync), create a new MANIFEST.MF file that contains the main class to be loaded and the merged hashes for all existing files from all MANIFEST.MF files, zip again to create a jar. That's it.
I have one spring boot application which exposes some rest services.It contains some sql files and some python files which are stored in files inside resources folder in folder structure resources/scripts/.
Some of them are sql whereas some of them are python scripts.
I am creating an executable jar files using spring boot application. So when I package the application using maven, all those files go inside jar files as expected.Now, In a function inside code , I have a command as follow
Process p = Runtime.getRuntime().exec(run generator/generate.py)
generate.py is basically a python file containing stuffs to do.When I run the application within intellij , it works fine . When I package it and try to execute the jar file , it is not able to find the file and complaining .
Error running generator/generate.py: Cannot run program "generator/generate.py" (in directory "file:/Users/bpokharel/code/oprime_fm_service/target/oprime-fmservice-V1.jar!/BOOT-INF/classes!/scripts"): error=2, No such file or directory
failed to parse or execute scripts/commands.txt:45 [run generator/generate.py]
My resource folder is as follow:-
resources/scripts/commands.txt
resources/scripts/generator/generate.py
commands.txt contains other commands. One of the command is generator/generate.py. So I am basically reading commands.txt and executing each scripts as defined in commands.txt
I am reading commands.txt using inputstream and then executing each commands.
I believe Runtime.getRunTime tries to execute the command in a directory.
, but for executable jar file I am not sure how I can supply folder name where to execute the command
Any help would be appreciated .
Here is somewhat related but I have to go one step further than this .
Classpath resource not found when running as jar
I need to run my Selenium(Java) script in different machine. So I'm exporting the script to a jar and executing it from a batch file. I would like to know if there is anyway I can create a log for my script while executing it from a batch file and store it somewhere.
While running the script in my local(using Eclipse) I'm able to do that with the help of log4j but I'm not sure how to do the same while executing the script in a different machine. (Note: It would be great if there is any way I can do this, by using log4j itself.)
yourcommand &>filename
where yourcommand is executing the batch file from command line, something like
file.bat &>logfile.log
(redirects both stdout and stderr to filename).
If you use log4j that means that you have a properties file. When you create the executable jar it should also include that file.
In the properties file, for your logs you should have a line like this:
log4j.appender.dest1.File=C:\\Users\\Your_User\\workspace\\Manual.logs
Edit the line to be like:
log4j.appender.dest1.File=Manual.logs
Add the "Manual.logs" (or the name that you defined) file next to the jar executable. This way, as long as both files are in the same folder, your logs are going to be generated.
I built a java program that is based on the Youtube Data API command line Examples. The program works fine and I now tried to convert the java class into a JAR file in order for me not having to start eclipse each time I want to use the program. To build the jar file I used the Eclipse export function "File->Export->Java->runnable JAR file".
I also chose to extract all required Libraries into the generated JAR.
All seems to work fine up to the point where I execute the program, which even creates the output file I wanted it to create but the JAR file doesn't seem to be able to retrieve data from the Youtube Data API, which means the created file is empty.
Is there anyone who can help me or at least is experiencing silimar issues?
Below I attached two sample Outputs:
This is what I get when executing the code within eclipse which gives me the correct answer
Channel; Subscribercount
Channelname; 12345
This is what I get when executing the JAR file
Channel; Subscribercount
Channelname; null
It seems that the JAR fails to connect to the API
Here I have a console log:
None of these errors occur when executing within eclipse
java.lang.NullPointerException
at java.io.Reader.<init>(Reader.java:78)
at java.io.InputStreamReader.<init>(InputStreamReader.java:72)
at com.google.api.services.samples.youtube.cmdline.Auth.authorize(Auth.java:51)
at com.google.api.services.samples.youtube.cmdline.data.getChannelsVideos.retrieveChannelInfo(getChannelsVideos.java:220)
at com.google.api.services.samples.youtube.cmdline.data.getChannelsVideos.main(getChannelsVideos.java:49)
java.lang.NullPointerException
at java.io.Reader.<init>(Reader.java:78)
at java.io.InputStreamReader.<init>(InputStreamReader.java:72)
at com.google.api.services.samples.youtube.cmdline.Auth.authorize(Auth.java:51)
at com.google.api.services.samples.youtube.cmdline.data.getChannelsVideos.getChannelsVideo(getChannelsVideos.java:92)
at com.google.api.services.samples.youtube.cmdline.data.getChannelsVideos.main(getChannelsVideos.java:50)
Solved it:I copied the client_secrets.json and the youtube. properties into the same folder as the JAR. Obviously it wasn't possible to import these files from within the JAR structure
I have a Java jar file which takes two csv files as an input. I tried to run this jar via Linux command line and it works fine.
Here is what i tried on Linux command line:
java -jar /home/test/Download.jar
I am trying to do the same via Jenkins in Execute shell but I am getting error:
Couldn't load file: test1.csv
Couldn't load file: test2.csv
information possible empty
These are the csv files that Jar takes as input.
I have given chmod 777 permission to all files.
Probably that's because of the location of the test[12].csv files. Try printing out the getAbsolutePath() for those File entries that you want to load: I'm sure they will point to a non-existing location.
I suppose those files are "next to" your Download.jar file. Now when you're executing a Jenkins job, the actual working directory is the workspace of the job (check the console log of the job on the web interface for the details). Either copy the files there or use absolute references.