I am new to Java and Docker, so this may be very simple.
The program reads user input and passes it to a function that does a dfs for broken links with a depth limit.
import java.util.Scanner;
public class CrawlerTest {
public static void main(String [ ] args)
{
Scanner reader = new Scanner(System.in);
System.out.println("Enter full website url to crawl, starting with http://");
String domain = reader.next();
System.out.println("Enter max crawl depth: ");
int maxDepth = reader.nextInt();
reader.close();
Crawler crawler = new Crawler();
crawler.crawl(domain, maxDepth);
}
}
and the Crawler class imports the following libraries
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Stack;
import org.jsoup.Connection;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import javafx.util.Pair;
I exported a runnable jar file in eclipse
I created a Docker file with contents below:
FROM openjdk:12-alpine
WORKDIR / ADD Test.jar Test.jar
EXPOSE 8080
CMD java -jar Test.jar
I built the docker image with docker image build .. This succeeds and I get docker image id
Next, I just run this image with docker run -it
I am prompted to enter user input, which I successfully do. Then on hitting Enter second time I see the following errors, which I don't see when just running jar file in console:
Add --attach to the docker run command.
Without --attach, there is no console for the Java program to use, so any use of System.in will fail.
JavaFX was removed from default Java distribution starting from JDK11. It needs to be added explicitly via Java module system. That's why you are getting NoClassDefFoundError for the Pair class. Either change Java version prior to 11 or remove Pair class to resolve the issue. You can also add the JavaFX module to your module path.
Related
My distro doesn't provide a 'selenium' package
$ apt search selenium p libtest-www-selenium-perl - Perl test framework using Selenium Remote Control i A python3-selenium - Python3 bindings for Selenium p qunit-selenium - Run QUnit tests through Selenium WebDriver p ruby-selenium-webdriver.
I've tried this and this approaches and the browser does get invoked successfully. This is my code.
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
//import selenium.webdriver;
//import selenium.webdriver.support.ui.WebDriverWait;
//import selenium.webdriver.common.by.By;
//import selenium.webdriver.support.expected_conditions;
import static org.openqa.selenium.By.*;
//import static sun.security.util.KnownOIDs.EC;
public class Login {
public static void main(String[] args) throws InterruptedException {
System.setProperty("webdriver.gecko.driver", "./geckodriver");
WebDriver dr = new FirefoxDriver();
dr.get("https://google.com/");
dr.manage().window().maximize();
//Click on "Join now"
WebElement join = dr.findElement(xpath("//a[#class='newUser green']"));
//WebDriverWait(driver, 20).until(EC.element_to_be_clickable((xpath("//a[#class='newUser green']")))).click();
join.click();
}
}
My problem is uncommenting any of the imports breaks because java has no idea what a selenium is. How do I add it? Linux Mint.
Go to https://chromedriver.chromium.org/downloads page
Depends on your Chrome version, Download Linux chrome driver zip package
Unip package and locate under any folder that is in system PATH or add your folder to PATH
Go to https://www.selenium.dev/downloads/ and download Selenium Java package and locate it under your project's library package(depends on your project/IDE settings)
Welcome. There is a lot going on in your quesrion: intelij; Java class paths; import statements.
I recommend tackling it one at a time.
copy the java example on selenium.dev doco page.
Run commandline javac.exe directly javac command examples (code Java.net) look for the 'Compile a source file which depends on multiple libraries' example. NOTE: the import statements are class references... Don't use the filename (is different to python).
Search Web for how to configure classpath for intelij.
I have:
Oracle 19c
java 8 on its machine
What i did:
I write simple class with one method in Java 8.
package <mypackage>;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.networknt.schema.JsonSchema;
import com.networknt.schema.JsonSchemaFactory;
import com.networknt.schema.SpecVersion;
import com.networknt.schema.ValidationMessage;
import java.io.IOException;
import java.io.Reader;
import java.sql.Clob;
import java.sql.SQLException;
import java.util.Set;
import java.util.stream.Collectors;
public class <classname> {
<some simple validation function>
}
I compile my project with maven and maven-assembly-plugin to build .jar file with dependencies.
I upload it with loadtool: loadjava -f -r -v -synonym -oracleresolver -resolve -grant <user> -thin -user <credentials> <filename>.jar
There were 0 errors during upload. All uploaded classes (including dependencies) have 'VALID' status in dba_objects table.
I write PL/SQL wrapper over my Java function.
create FUNCTION <funcname>(P_IN_BODY_TEXT CLOB, P_IN_BODY_SCHEMA_TEXT CLOB)
RETURN VARCHAR2
AS LANGUAGE JAVA NAME '<packagename>.<classname>.<funcname>(java.sql.Clob, java.sql.Clob) return java.lang.String';
/
I use this function in my ORDS REST Service.
When doing request to ORDS i am getting this exception:
The request could not be processed because an error occurred whilst attempting to evaluate
the SQL statement associated with this resource.
Please check the SQL statement is correctly formed and executes without error. SQL Error Code: 29532,
Error Message: ORA-29532: Java call terminated by uncaught Java exception:
java.lang.NoClassDefFoundError ORA-06512: at <rest stacktrace that gives me nothing>
Quiestion is:
What is root of this problem? By -synonym flag tool creates synonyms for me, all classes is valid. Do i need some permisions to use java packages liike java.sql that in my class imports? I upload some others open sources java libraries into my Oracle, but they doesn't hava dependencies - is that the problem?
Problem was in slf4j library that throws this exception. slf4j was dependency of library that i used.
Didn't dig the problem, I just pick another labrary with less dependencies and its works.
I'm using following package for calling Weka functions from within Matlab https://github.com/NicholasMcCarthy/wekalab
and my code is
close all; clear all; clc;
dbstop if error
%%
javaclasspath('C:\Program Files (x86)\Weka-3-8\weka.jar');
javaaddpath('C:\Users\PC\wekafiles\packages\imageFilters\imageFilters.jar');
%%
import weka.filters.*
import weka.filters.Filter.*
import weka.filters.unsupervised.instance.imagefilter.*
import weka.filters.unsupervised.instance.imagefilter.BinaryPatternsPyramidFilter.*
import weka.classifiers.Classifier.*
import weka.classifiers.functions.SMO.*
import weka.classifiers.Evaluation.*
import weka.core.Attribute.*
import weka.core.FastVector.*
import weka.core.Instances.*
import weka.core.DenseInstance.*
import weka.classifiers.Classifier.*
import weka.classifiers.Evaluation.*
import weka.core.converters.ArffLoader.*
import weka.filters.unsupervised.instance.imagefilter.*
import weka.core.converters.ConverterUtils.*;
D = wekaLoadData('E:\pro\program\selectedPics\character\test.arff', 'ARFF');
myFilter = wekaFilter('weka.filters.unsupervised.instance.imagefilter.BinaryPatternsPyramidFilter');
filteredData = wekaApplyFilter(D, myFilter);
when i use the default filters of weka,
myFilter = wekaFilter('weka.filters.unsupervised.attribute.Standardize');
it works fine but when i use the installed package of weka (imageFilters) it give me this error
Error using javaObject
No class weka.filters.unsupervised.instance.imagefilter.BinaryPatternsPyramidFilter can be located on the Java class path
image filter (package) path: C:\Users\PC\wekafiles\packages\imageFilters
weka path: C:\Program Files (x86)\Weka-3-8
Environment variable & their paths:
CLASSPATH
C:\Program Files (x86)\Weka-3-8\weka.jar;
C:\Program Files(x86)\Weka-3-8\imageFilters\imageFilters.jar;
C:\Users\PC\wekafiles\packages\imageFilters\src\main\java;
C:\Users\PC\wekafiles\packages\imageFilters\src\main\java\filters\unsupervised\instance\imagefilter;
C:\Users\PC\wekafiles\packages\imageFilters\src\main\java\filters\unsupervised\instance;
java
C:\Program Files\Java\jre1.8.0_181\bin;
PATH
C:\Program Files\Java\jdk1.8.0_181\bin;
PATH_HOME
C:\Program Files\Java\jdk1.8.0_181;
You can use weka.Run in order to use additional packages in weka (weka.Run details)
On the Terminal:
Before using it you may want to add your weka.jar file into CLASSPATH:
export CLASSPATH=path_to_weka.jar
For example, a way to use this command:
java weka.Run weka.filters.unsupervised.instance.imagefilter.BinaryPatternsPyramidFilter -D "directory were files to filter are located" -i "input arff" -o "output arff"
For more info, check: Weka official documenation
In my case, I used the manual for Weka 3-7-8. The information mentioned is on the page 26 under Running installed learning algorithms. This could change from version to version.
Here you have a list of any manual you could need: Weka manuals for every version
About using it on Java, I don't have information about it. I wanted to answer this question because there might be people out there needing help with command line Weka.
All I am trying is to create a directory in HDFS using Java programmatically.
I am getting this error.
Exception in thread "main" java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider org.apache.hadoop.fs.s3a.S3AFileSystem could not be instantiated
Caused by: java.lang.NoClassDefFoundError: com/amazonaws/AmazonServiceException
Caused by: java.lang.ClassNotFoundException: com.amazonaws.AmazonServiceException
Not sure where all these Amazon S3 came over here. Please help.
Here's the code. This is Hadoop 2.7
package tas.module1;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.BlockLocation;
import org.apache.hadoop.fs.FSDataInputStream;
import org.apache.hadoop.fs.FSDataOutputStream;
import org.apache.hadoop.fs.FileStatus;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
public class JavaClient {
public JavaClient() {
}
public static void main(String[] args) throws IOException {
JavaClient jc = new JavaClient();
Configuration config = new Configuration();
config.addResource(new Path("/usr/local/hadoop-2.7.1/etc/hadoop/core-site.xml"));
config.addResource(new Path("/usr/local/hadoop-2.7.1/etc/hadoop/hdfs-site.xml"));
config.set("fs.hdfs.impl",
org.apache.hadoop.hdfs.DistributedFileSystem.class.getName()
);
config.set("fs.file.impl",
org.apache.hadoop.fs.LocalFileSystem.class.getName()
);
FileSystem dfs = FileSystem.get(config);
String dirName = "TestDirectory";
System.out.println(dfs.getWorkingDirectory() +" this is from /n/n");
Path src = new Path(dfs.getWorkingDirectory()+"/"+dirName);
dfs.mkdirs(src);
System.out.println("created dir");
dfs.close();
}
}
ah, this is a bug fixed a while back in HADOOP-12636, which relates to the java service API and classpaths. Hadoop 2.7.2 enumerates all available filesystem implementation classes in JARs, and fails here due to the transient classpath problems.
If you drop the hadoop-aws JAR from your CP this will go away, or just upgrade to Hadoop 2.7.3
It seems that you are missing some dependency related to the working with S3 file system. In order to work with it, you need to have aws java sdk jars deployed in your cluster. You can download aws-java-sdk from http://sdk-for-java.amazonwebservices.com/latest/aws-java-sdk.zip. Then you need to unzip it, and copy every jar in aws-java-sdk/lib/ and aws-java-sdk/third-party/ to your datanodes.
Another option is to create uber jar and include this dependency directly into your jar through maven:
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.11.91</version>
</dependency>
How can we use external Jython modules with JSR-223? (Assuming some foo python module is installed using pip or easy_install and the following Python code is running with Jython)
Python code: (jsr223_test.py)
import pyfoo
pyfoo.do_sth()
Java code:
import javax.script.*;
import java.io.*;
import org.python.core.Py;
import org.python.core.PySystemState;
public class Main() {
//BEGIN EDIT
PySystemState engineSys = new PySystemState();
engineSys.path.append(".");
Py.setSystemState(engineSys);
//END EDIT
ScriptEngineManager mgr = new ScriptEngineManager();
ScriptEngine jython = mgr.getEngineByName("jython");
jython.eval(new FileReader(new File("jsr223_test.py")));
}
This results in ImportError: no module named pyfoo. However executing the following in a terminal succeeds.
jython jsr223_test.py
Any advice?
Edit:
I guessed it was a classpath issue and copied the py files from the eggs to my applications working directory. Following the http://wiki.python.org/jython/UserGuide#using-jsr-223 I've also added that path to the engine environment (as seen on the code, between //BEGIN EDIT and //END EDIT) and it worked.
I've also tried adding $JYTHON_HOME/Lib/site-packages to the path but it did not work, I don't know why.