FileNotFoundException from new CSVReaderHeaderAware - java

Referencing the quick start for OpenCSV, having trouble opening a file which shows as present through the OS and by using exists to demonstrate.
code:
package net.bounceme.dur.basexfromjaxb.csv;
import com.opencsv.CSVReaderHeaderAware;
import java.io.File;
import java.io.FileReader;
import java.net.URI;
import java.util.Map;
import java.util.logging.Logger;
public class ReaderForCVS {
private static final Logger LOG = Logger.getLogger(ReaderForCVS.class.getName());
public ReaderForCVS() {
}
public void unmarshal(URI inputURI) throws Exception {
LOG.info(inputURI.toString());
File encyptFile = new File(inputURI);
System.out.println(encyptFile.exists());
Map<String, String> values = new CSVReaderHeaderAware(new FileReader("file:/home/thufir/jaxb/input.csv")).readMap();
}
}
file not found:
thufir#dur:~/NetBeansProjects/BaseXFromJAXB$
thufir#dur:~/NetBeansProjects/BaseXFromJAXB$ gradle run
> Task :run FAILED
Jan 10, 2019 1:47:50 PM net.bounceme.dur.basexfromjaxb.csv.ReaderForCVS unmarshal
INFO: file:/home/thufir/jaxb/input.csv
true
Exception in thread "main" java.io.FileNotFoundException: file:/home/thufir/jaxb/input.csv (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileInputStream.<init>(FileInputStream.java:93)
at java.io.FileReader.<init>(FileReader.java:58)
at net.bounceme.dur.basexfromjaxb.csv.ReaderForCVS.unmarshal(ReaderForCVS.java:23)
at net.bounceme.dur.basexfromjaxb.App.marshalCSV(App.java:24)
at net.bounceme.dur.basexfromjaxb.App.main(App.java:16)
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 1s
3 actionable tasks: 2 executed, 1 up-to-date
thufir#dur:~/NetBeansProjects/BaseXFromJAXB$
thanks to:
https://stackoverflow.com/a/18552188/262852

Why this works I don't know:
public void unmarshal(URI inputURI) throws Exception {
FileReader f = new FileReader(new File(inputURI));
Map<String, String> values = new CSVReaderHeaderAware(f).readMap();
}

Related

Converting Docx4j to pdf via documents4j

java code to generate the conversion of word to pdf
package com.sonakshi;
import com.documents4j.api.DocumentType;
import com.documents4j.api.IConverter;
import com.documents4j.job.LocalConverter;
import org.apache.commons.io.output.ByteArrayOutputStream;
import java.io.*;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
public class Hello{
public static void main(String[] args) throws IOException, ExecutionException, InterruptedException {
ByteArrayOutputStream bo = new ByteArrayOutputStream();
InputStream in = new BufferedInputStream(new FileInputStream("//home//sonakshi_user//Documents//WordDocToConvert.docx"));
IConverter converter = LocalConverter.builder()
.baseFolder(new File("//home//sonakshi_user//Documents//"))
.workerPool(20, 25, 2, TimeUnit.SECONDS)
.processTimeout(5, TimeUnit.SECONDS).build();
Future<Boolean> conversion = converter
.convert(in).as(DocumentType.MS_WORD)
.to(bo).as(DocumentType.PDF)
.prioritizeWith(1000) // optional
.schedule();
conversion.get();
try (OutputStream outputStream = new FileOutputStream("//home//sonakshi_user//Documents//Output.pdf")) {
bo.writeTo(outputStream);
} catch (IOException e) {
e.printStackTrace();
}
in.close();
bo.close();
}
}
I am getting this below exception generated while conversion
log4j:WARN No appenders could be found for logger (org.zeroturnaround.exec.ProcessExecutor).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.lang.IllegalStateException: class com.documents4j.conversion.msoffice.MicrosoftWordBridge could not be created by a (File, long, TimeUnit) constructor
at com.documents4j.conversion.ExternalConverterDiscovery.make(ExternalConverterDiscovery.java:33)
at com.documents4j.conversion.ExternalConverterDiscovery.makeAll(ExternalConverterDiscovery.java:43)
at com.documents4j.conversion.ExternalConverterDiscovery.loadConfiguration(ExternalConverterDiscovery.java:86)
at com.documents4j.conversion.DefaultConversionManager.<init>(DefaultConversionManager.java:22)
at com.documents4j.job.LocalConverter.makeConversionManager(LocalConverter.java:79)
at com.documents4j.job.LocalConverter.<init>(LocalConverter.java:51)
at com.documents4j.job.LocalConverter$Builder.build(LocalConverter.java:186)
at com.sonakshi.Hello.main(Hello.java:20)
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at com.documents4j.conversion.ExternalConverterDiscovery.make(ExternalConverterDiscovery.java:31)
... 7 more
Caused by: com.documents4j.throwables.ConverterAccessException: Unable to run script: /home/sonakshi_user/Documents/word_start1713780447.vbs
at com.documents4j.conversion.AbstractExternalConverter.runNoArgumentScript(AbstractExternalConverter.java:76)
at com.documents4j.conversion.msoffice.AbstractMicrosoftOfficeBridge.runNoArgumentScript(AbstractMicrosoftOfficeBridge.java:51)
at com.documents4j.conversion.msoffice.AbstractMicrosoftOfficeBridge.tryStart(AbstractMicrosoftOfficeBridge.java:34)
at com.documents4j.conversion.msoffice.MicrosoftWordBridge.startUp(MicrosoftWordBridge.java:44)
at com.documents4j.conversion.msoffice.MicrosoftWordBridge.<init>(MicrosoftWordBridge.java:39)
... 12 more
Caused by: org.zeroturnaround.exec.ProcessInitException: Could not execute [cmd, /S, /C, ""/home/sonakshi_user/Documents/word_start1713780447.vbs""] in /home/sonakshi_user/Documents. Error=2, No such file or directory
at org.zeroturnaround.exec.ProcessInitException.newInstance(ProcessInitException.java:80)
at org.zeroturnaround.exec.ProcessExecutor.invokeStart(ProcessExecutor.java:1002)
at org.zeroturnaround.exec.ProcessExecutor.startInternal(ProcessExecutor.java:970)
at org.zeroturnaround.exec.ProcessExecutor.execute(ProcessExecutor.java:906)
at com.documents4j.conversion.AbstractExternalConverter.runNoArgumentScript(AbstractExternalConverter.java:72)
... 16 more
Caused by: java.io.IOException: Cannot run program "cmd" (in directory "/home/sonakshi_user/Documents"): error=2, No such file or directory
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
at org.zeroturnaround.exec.ProcessExecutor.invokeStart(ProcessExecutor.java:997)
... 19 more
Caused by: java.io.IOException: error=2, No such file or directory
at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:340)
at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:271)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)
... 21 more
Exception in thread "Shutdown hook: com.documents4j.job.LocalConverter" java.lang.NullPointerException
at com.documents4j.job.LocalConverter.shutDown(LocalConverter.java:100)
at com.documents4j.job.ConverterAdapter$ConverterShutdownHook.run(ConverterAdapter.java:134)
I have included many jar files as well.
Kindly specify if I need to use specific one

Question: Some messages have been simplified; recompile with -Xdiags:verbose to get full output

package com.example.flutter_app;
import android.os.Bundle;
import io.flutter.app.FlutterActivity;
import io.flutter.plugins.GeneratedPluginRegistrant;
import io.flutter.plugin.common.MethodCall;
import io.flutter.plugin.common.MethodChannel;
import io.flutter.plugin.common.MethodChannel.Result;
import io.flutter.plugin.common.MethodChannel.MethodCallHandler;
import android.content.ContextWrapper;
import android.content.Intent;
import android.content.IntentFilter;
import android.os.BatteryManager;
import android.os.Build.VERSION;
import android.os.Build.VERSION_CODES;
import android.os.Bundle;
public class MainActivity extends FlutterActivity {
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
GeneratedPluginRegistrant.registerWith(this);
new MethodChannel(getFlutterView(), "course.flutter.dev/battery").setMethodCallHandler(
new MethodCallHandler() {
#Override
public void onMethodCall(MethodCall call, Result result) {
if (call.method.equals("getBatteryLevel")) {
int batteryLevel = getBatteryLevel();
if (batteryLevel != -1) {
result.success(batteryLevel);
} else {
result.error("UNAVAILABLE", "Could not fetch battery level.", null);
}
} else {
result.notImplemented();
}
}
}
);
}
private int getBatteryLevel() {
int batteryLevel = -1;
if (VERSION.SDK_INT >= VERSION_CODES.LOLLIPOP) {
BatteryManager batteryManager = (BatteryManager) getSystemService(BATTERY_SERVICE);
batteryLevel = batteryManager.getIntProperty(BatteryManager.BATTERY_PROPERTY_CAPACITY);
} else {
Intent intent = new ContextWrapper(getApplicationContext()).registerReceiver(null, new IntentFilter(Intent.ACTION_BATTERY_CHANGED));
batteryLevel = (intent.getIntExtra(BatteryManager.EXTRA_LEVEL, -1) * 100) / intent.getIntExtra(BatteryManager.EXTRA_SCALE, -1);
}
return batteryLevel;
}
}
My java code is given above. I was following a tutorial of writing native java code to get the battery level in my flutter app.
But it gives me the error given below:
(Debug console output)
Launching lib\main.dart on LDN L21 in debug mode...
lib\main.dart:1
C:\Users\ARPC\flutter_app\android\app\src\main\java\com\example\flutter_app\MainActivity.java:22: error: incompatible types: MainActivity cannot be converted to FlutterEngine
GeneratedPluginRegistrant.registerWith(this);
^
Note: Some messages have been simplified; recompile with -Xdiags:verbose to get full output
1 error
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':app:compileDebugJavaWithJavac'.
> Compilation failed; see the compiler error output for details.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 3s
Exception: Gradle task assembleDebug failed with exit code 1
Exited (sigterm)
Kindly tell me what I am doing wrong I thought that I have to turn verbose to true but I don't know how to do that if someone knows the solution please tell me.
Remove the below code in manifest file
meta-data
android:name="flutterEmbedding"
android:value="2"
and run the project it will work

CalendarFX in combination with JavaFX: Module javafx.controls not found

I am trying to use CalendarFX as a gradle dependency in my JavaFX project with the gradle javafx plugin, but I get the error that Module javafx.controls is nout found, while clearly specifying in the build.gradle file that it should use the javafx.controls module. My setup is this:
build.gradle:
plugins {
id 'java'
id 'application'
id 'org.openjfx.javafxplugin' version '0.0.8'
}
group 'org.example'
version '1.0-SNAPSHOT'
mainClassName = "org.example.MainApp"
sourceCompatibility = 13
repositories {
mavenCentral()
}
dependencies {
implementation 'com.calendarfx:view:11.8.3'
}
javafx {
version = "13"
modules = ['javafx.controls', 'javafx.fxml']
}
MainApp.java:
package org.example;
import com.calendarfx.model.Calendar;
import com.calendarfx.model.CalendarSource;
import com.calendarfx.view.CalendarView;
import javafx.application.Application;
import javafx.application.Platform;
import javafx.scene.Scene;
import javafx.scene.layout.StackPane;
import javafx.stage.Stage;
import java.time.LocalDate;
import java.time.LocalTime;
public class MainApp extends Application {
#Override
public void start(Stage primaryStage) throws Exception {
CalendarView calendarView = new CalendarView();
Calendar test = new Calendar("Test");
test.setShortName("T");
test.setStyle(Calendar.Style.STYLE1);
CalendarSource familyCalendarSource = new CalendarSource("Source");
familyCalendarSource.getCalendars().add(test);
calendarView.getCalendarSources().setAll(familyCalendarSource);
calendarView.setRequestedTime(LocalTime.now());
StackPane stackPane = new StackPane();
stackPane.getChildren().add(calendarView);
Thread updateTimeThread = new Thread("Calendar: Update Time Thread") {
#Override
public void run() {
while (true) {
Platform.runLater(() -> {
calendarView.setToday(LocalDate.now());
calendarView.setTime(LocalTime.now());
});
try {
// update every 10 seconds
sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
};
updateTimeThread.setPriority(Thread.MIN_PRIORITY);
updateTimeThread.setDaemon(true);
updateTimeThread.start();
Scene scene = new Scene(stackPane);
primaryStage.setTitle("Calendar");
primaryStage.setScene(scene);
primaryStage.setWidth(1300);
primaryStage.setHeight(1000);
primaryStage.centerOnScreen();
primaryStage.show();
}
}
And the error I get when I try to run is:
$ ./gradlew run
> Task :run FAILED
Error occurred during initialization of boot layer
java.lang.module.FindException: Module javafx.controls not found
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':run'.
> Process 'command '/Library/Java/JavaVirtualMachines/jdk-13.0.2.jdk/Contents/Home/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 738ms
3 actionable tasks: 2 executed, 1 up-to-date
Try this in your VM arguments:
--module-path \path\to\javafx-sdk\lib --add-modules=javafx.controls,javafx.fxml
JavaFx does not come with CalendarFx ,
Download the lib and import it in your project.
https://github.com/dlsc-software-consulting-gmbh/CalendarFX/releases/download/11.8.3/view-11.8.3.jar

how to authenticate with Twitter4J?

I have some parameters stored in environment variables -- certainly those environment variables work with powershell.
Do I need slightly different parameters to authenticate with Twitter4J, perhaps?
powershell script output:
thufir#dur:~$
thufir#dur:~$ pwsh /home/thufir/powershell/helloPSTwitterAPI/twitter.ps1
RT #adamdriscoll: Today is the last day to sign up for the #Powershell #UniversalDashboard #udemy course at a discount rate. Check it out h…
RT #DirectoryRanger: Invoke-CommandAs
htt...
RT #adamdriscoll: Just added some #UniversalAutomation documentation about pre-defined variables in UA jobs. htt....
RT #adamdriscoll: #PowerShell #UniversalDashboard 2.8.2 is now available on the PowerShell Gallery. Lots of fixes, some improvements to Adm…
#psdevuk #adamdriscoll #psdbatools 👀
#adamdriscoll 🔥
#BillKindle #McDonalds #Wendys Sad, but that’s what I’m going to do next time. It should be ‘BigMac with Bacon Bits… htt...
I was excited to try out the new BigMac with Bacon... but horrible portion.. looks like cesar salad bacon bits...… htt...
#WindosNZ PSTwitterAPI? ;)
#Marioperator Thanks for the shoutout ❤️
RT #adamdriscoll: Nice! Financial charts for UD! htt.... #powershell htt...
#TomatoApp1 Constantly having to bind/unbind MiaoMiao device. And now the app won’t even open after trying reinstal… htt....
#adamdriscoll It shall get indexed and searchable in 15 minutes! I can just imagine your amazon shopping suggestions...
#adamdriscoll #LeeAlanBerg Pics or it didn’t happen
#SwiftOnSecurity #adbertram Did you end up finding a more elegant solution?
RT #racheldianeb: Had cake and wine tonight. 2 things I said I wouldn’t consume in Jan and would generally limit in 2020. It’s Jan 1st. So…
#adilio #sstranger Someone would probably be wrong.. 😝
#AndrewPlaTech #sstranger You have nothing to lose.. I mean, clearly I lost.. ;)
Someone’s mother has four sons. North, South and East. What is the name of the fourth son. Private message me the n… htt....
RT #EssentialSign_: For whoever needs this this evening. htt....
done
thufir#dur:~$
powershell script source:
thufir#dur:~$
thufir#dur:~$ cat /home/thufir/powershell/helloPSTwitterAPI/twitter.ps1
Import-Module PSTwitterAPI
#Set-TwitterOAuthSettings -ApiKey $env:ApiKey -ApiSecret $env:ApiSecret -AccessToken $env:AccessToken -AccessTokenSecret $env:AccessTokenSecret
Set-TwitterOAuthSettings -ApiKey $env:oAuthConsumerKey -ApiSecret $env:oAuthConsumerSecret -AccessToken $env:oAuthAccessToken -AccessTokenSecret $env:oAuthAccessTokenSecret
$TwitterStatuses = Get-TwitterStatuses_UserTimeline -screen_name 'mkellerman'
Foreach ($status in $TwitterStatuses) {
Write-Host $status.text
}
Write-Host "done"
thufir#dur:~$
java crash:
thufir#dur:~/java/helloTwitter4J$
thufir#dur:~/java/helloTwitter4J$ gradle clean run
> Task :run FAILED
Jan. 29, 2020 4:16:31 P.M. helloTwitter4J.App runQuery
INFO: {oAuthAccessToken=abc, oAuthConsumerKey=def, oAuthConsumerSecret=ghi, oAuthAccessTokenSecret=jkl}
Exception in thread "main" java.lang.IllegalStateException: Authentication credentials are missing. See http://twitter4j.org/en/configuration.html for details
at twitter4j.TwitterBaseImpl.ensureAuthorizationEnabled(TwitterBaseImpl.java:201)
at twitter4j.TwitterImpl.get(TwitterImpl.java:1966)
at twitter4j.TwitterImpl.search(TwitterImpl.java:293)
at helloTwitter4J.App.runQuery(App.java:52)
at helloTwitter4J.App.main(App.java:60)
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':run'.
> Process 'command '/home/thufir/.sdkman/candidates/java/12.0.1-zulu/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 1s
4 actionable tasks: 4 executed
thufir#dur:~/java/helloTwitter4J$
java source:
package helloTwitter4J;
import java.io.IOException;
import java.util.InvalidPropertiesFormatException;
import java.util.Properties;
import java.util.Set;
import java.util.logging.Logger;
import twitter4j.Query;
import twitter4j.QueryResult;
import twitter4j.Status;
import twitter4j.Twitter;
import twitter4j.TwitterException;
import twitter4j.TwitterFactory;
import twitter4j.conf.ConfigurationBuilder;
public class App {
private static final Logger log = Logger.getLogger(App.class.getName());
private final Properties properties = new Properties();
private void loadProperties() throws InvalidPropertiesFormatException, IOException {
properties.loadFromXML(App.class.getResourceAsStream("/twitter.xml"));
log.fine(properties.toString());
Set<Object> keySet = properties.keySet();
String key = null;
String value = null;
for (Object obj : keySet) {
key = obj.toString();
value = System.getenv(key);
log.fine(key + value);
properties.setProperty(key, value);
}
}
private void runQuery() throws TwitterException, InvalidPropertiesFormatException, IOException {
loadProperties();
log.info(properties.toString()); //this matches what powershell uses
ConfigurationBuilder configurationBuilder = new ConfigurationBuilder();
configurationBuilder.setDebugEnabled(true)
.setOAuthConsumerKey(properties.getProperty("oAuthConsumerKey"))
.setOAuthConsumerSecret(properties.getProperty("oAuthConsumerSecret"))
.setOAuthAccessToken(properties.getProperty("oAuthAccessToken"))
.setOAuthAccessTokenSecret(properties.getProperty("oAuthAccessTokenSecret"));
Twitter twitter = TwitterFactory.getSingleton();
Query query = new Query("source:twitter4j yusukey");
QueryResult result = twitter.search(query);
// for (Status status : result.getTweets()) {
// log.info("#" + status.getUser().getScreenName() + ":" + status.getText());
// }
}
public static void main(String[] args) throws InvalidPropertiesFormatException, IOException, TwitterException {
new App().runQuery();
}
}
I've also printed the properties file to the console and it looks correct. Certainly the variables are getting picked up, I'm just not entirely sure they're what Twitter4J requires. Perhaps a specific twitter.properties file would help.
whoops, need to actually build the factory properly:
https://www.dummies.com/web-design-development/mobile-apps/how-to-make-a-configurationbuilder-to-talk-to-the-twitter-server-with-your-android-app/
working hello world type code:
package helloTwitter4J;
import java.io.IOException;
import java.util.List;
import java.util.Properties;
import java.util.Set;
import java.util.logging.Logger;
import twitter4j.Status;
import twitter4j.Twitter;
import twitter4j.TwitterException;
import twitter4j.TwitterFactory;
import twitter4j.conf.ConfigurationBuilder;
public class App {
private static final Logger log = Logger.getLogger(App.class.getName());
private Properties loadProperties() throws IOException {
Properties properties = new Properties();
properties.loadFromXML(App.class.getResourceAsStream("/twitter.xml"));
log.fine(properties.toString());
Set<Object> keySet = properties.keySet();
String key = null;
String value = null;
for (Object obj : keySet) {
key = obj.toString();
value = System.getenv(key);
log.fine(key + value);
properties.setProperty(key, value);
}
return properties;
}
private TwitterFactory configTwitterFactory() throws IOException {
Properties properties = loadProperties();
log.info(properties.toString()); //this matches what powershell uses
ConfigurationBuilder configurationBuilder = new ConfigurationBuilder();
configurationBuilder.setDebugEnabled(true)
.setOAuthConsumerKey(properties.getProperty("oAuthConsumerKey"))
.setOAuthConsumerSecret(properties.getProperty("oAuthConsumerSecret"))
.setOAuthAccessToken(properties.getProperty("oAuthAccessToken"))
.setOAuthAccessTokenSecret(properties.getProperty("oAuthAccessTokenSecret"));
TwitterFactory twitterFactory = null;
twitterFactory = new TwitterFactory(configurationBuilder.build());
return twitterFactory;
}
private void getHomeTimeLine() throws TwitterException, IOException {
Twitter twitter = configTwitterFactory().getInstance();
List<Status> statuses = null;
statuses = twitter.getHomeTimeline();
System.out.println("Showing home timeline.");
if (statuses != null) {
for (Status status : statuses) {
System.out.println(status.getUser().getName() + ":"
+ status.getText());
}
}
}
public static void main(String[] args) throws TwitterException, IOException {
new App().getHomeTimeLine();
}
}
also see https://github.com/nisrulz/twitterbot-java/blob/master/src/github/nisrulz/bot/TwitterBot.java

Apache Spark eventLog configuration on Windows Giving error

I am using Spark 1.5 on Windows. I haven't installed any separate binaries of Hadoop.
I running a Master and a single worker.
It's a simple HelloWorld Program as below :
package com.java.spark;
import java.io.Serializable;
import java.util.Arrays;
import java.util.List;
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.api.java.function.VoidFunction;
public class HelloWorld implements Serializable{
/**
*
*/
private static final long serialVersionUID = -7926281781224763077L;
public static void main(String[] args) {
// Local mode
//SparkConf sparkConf = new SparkConf().setAppName("HelloWorld").setMaster("local");
SparkConf sparkConf = new SparkConf().setAppName("HelloWorld").setMaster("spark://192.168.1.106:7077")
.set("spark.eventLog.enabled", "true")
.set("spark.eventLog.dir", "file:///D:/SparkEventLogsHistory");
//.set("spark.eventLog.dir", "/work/");
//tried many combinations above but all gives error.
JavaSparkContext ctx = new JavaSparkContext(sparkConf);
String[] arr = new String[] { "John", "Paul", "Gavin", "Rahul", "Angel" };
List<String> inputList = Arrays.asList(arr);
JavaRDD<String> inputRDD = ctx.parallelize(inputList);
inputRDD.foreach(new VoidFunction<String>() {
public void call(String input) throws Exception {
System.out.println(input);
}
});
}
}
The exception I am getting is :
Exception in thread "main" java.io.IOException: Cannot run program "cygpath": CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessBuilder.start(Unknown Source)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:206)
at org.apache.hadoop.util.Shell.run(Shell.java:188)
at org.apache.hadoop.fs.FileUtil$CygPathCommand.<init>(FileUtil.java:412)
at org.apache.hadoop.fs.FileUtil.makeShellPath(FileUtil.java:438)
at org.apache.hadoop.fs.FileUtil.makeShellPath(FileUtil.java:465)
at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:592)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:584)
at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:420)
at org.apache.spark.scheduler.EventLoggingListener.start(EventLoggingListener.scala:130)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:541)
at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:61)
at com.java.spark.HelloWorld.main(HelloWorld.java:28)
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(Unknown Source)
at java.lang.ProcessImpl.start(Unknown Source)
... 13 more
16/04/01 20:13:24 INFO ShutdownHookManager: Shutdown hook called
Does anyone has any idea how to resolve this exception, so that Spark can pick the eventLogs from local directory.
If I dont give configure eventLog.dir then exception changes to :
Exception in thread "main" java.io.FileNotFoundException: File file:/H:/tmp/spark-events does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:468)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:373)
at org.apache.spark.scheduler.EventLoggingListener.start(EventLoggingListener.scala:100)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:541)
at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:61)
at com.java.spark.HelloWorld.main(HelloWorld.java:28)

Categories