I have the following three classes :
I tried making the routine of 1 & 2 and used tjava to call the main class and the method from 1 & 2 but I am unable to fetch those methods.
1)
package page_scraper;
import com.gargoylesoftware.htmlunit.Page;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.WebClientOptions;
import com.gargoylesoftware.htmlunit.html.FrameWindow;
import com.gargoylesoftware.htmlunit.html.HtmlButtonInput;
import com.gargoylesoftware.htmlunit.html.HtmlElement;
import com.gargoylesoftware.htmlunit.html.HtmlOption;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
import com.gargoylesoftware.htmlunit.html.HtmlSelect;
import com.gargoylesoftware.htmlunit.html.HtmlTextInput;
import java.io.FileWriter;
import java.io.IOException;
import java.io.PrintStream;
import java.io.Writer;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.Date;
import java.util.List;
import page_scraper.UnitArray;
public class PageScraper {
public void Scrape() throws IOException {
try {
UnitArray object = new UnitArray();
ArrayList<String> unitList = object.getUnitArray();
WebClient webClient = new WebClient();
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
HtmlPage page = (HtmlPage)webClient.getPage("http://www.bmreports.com/servlet/com.logica.neta.bwp_PanBMUData");
List frames = page.getFrames();
HtmlPage page1 = (HtmlPage)((FrameWindow)frames.get(0)).getEnclosedPage();
HtmlTextInput settlementDay = (HtmlTextInput)page1.getHtmlElementById("param5");
HtmlSelect period = (HtmlSelect)page1.getHtmlElementById("param6");
HtmlOption periodOption = period.getOption(1);
HtmlTextInput unitId = (HtmlTextInput)page1.getHtmlElementById("param1");
HtmlButtonInput button = (HtmlButtonInput)page1.getHtmlElementById("go_button");
String outputLocation = String.valueOf(System.getProperty("user.home")) + "/Documents/output.csv";
FileWriter fileWriter = new FileWriter(outputLocation);
String errorLocation = String.valueOf(System.getProperty("user.home")) + "/Documents/error.csv";
FileWriter errorWriter = new FileWriter(errorLocation);
int i = 0;
while (i < unitList.size()) {
int x = 0;
while (x < 365) {
String errorData;
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");
Calendar cal = Calendar.getInstance();
cal.add(5, - x);
String dateValue = dateFormat.format(cal.getTime());
System.out.println(dateValue);
settlementDay.setValueAttribute(dateValue);
period.setSelectedAttribute(periodOption, true);
unitId.setValueAttribute(unitList.get(i));
System.out.println(unitList.get(i));
try {
button.click();
HtmlPage page2 = (HtmlPage)((FrameWindow)frames.get(1)).getEnclosedPage();
String pageSource = page2.asXml();
int firstIndex = pageSource.indexOf("csv=") + 38;
int secondIndex = pageSource.indexOf("n\"") + 1;
String csvData = pageSource.substring(firstIndex, secondIndex);
fileWriter.append(csvData);
}
catch (ClassCastException e) {
errorData = String.valueOf(dateValue) + " " + unitList.get(i) + System.getProperty("line.separator");
System.out.println(errorData);
errorWriter.append(errorData);
continue;
}
catch (StringIndexOutOfBoundsException e) {
errorData = String.valueOf(dateValue) + " " + unitList.get(i) + System.getProperty("line.separator");
System.out.println(errorData);
errorWriter.append(errorData);
continue;
}
++x;
}
++i;
}
webClient.close();
fileWriter.close();
errorWriter.close();
}
catch (IOException e) {
e.printStackTrace();
}
}
}
2)
package page_scraper;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
public class UnitArray {
public ArrayList<String> getUnitArray() {
String csvList = "abc,xyz";
ArrayList<String> list = new ArrayList<String>(Arrays.asList(csvList.split(",")));
return list;
}
}
3)
package page_scraper;
import page_scraper.PageScraper;
public class main {
public static void main(String[] args) throws Exception {
PageScraper test = new PageScraper();
test.Scrape();
}
}
I made the routines for the above code(1) & 2)) in Talend and then used tjava to call the method but unable to do so..I also tried using tjava for all and did a onSubjob ok on each of the tjava.
How can I call these classes in talend and call the method ?
Firstly, routines classes in Talend need to be in routines package
package routines;
public class PageScraper {
public void Scrape() {
System.out.println("PageScraper.Scrape");
}
}
Secondly, to use it in Job you need to drag'n'drop routine to opened job area.
Then you can use your class in that way
You can easily make a jar file that contains the three classes then load the jar using tLibraryLoad or include the jar in your routine if you want to get more reusability.
As suggested in the other answers, you need to define classes under routines package.
in case you are using Takend 7.3 & above, Right click on your routine and add it as Dependent package
Get routines as a jar and in case using in bigData jobs, you may need to use tLibraryLoad to package it together with other dependencies..
Related
I am simply trying to call my testImage() method in Wavemaker. I imported the .jar file after running the application perfectly in Eclipse. However when I call the same method in the .jar file in Wavemaker it gives my this error:
Error
Compile failed with output: [{"filename" : "master/services/MyJavaService1/src/com/demo_jquery/myjavaservice1/MyJavaService1.java","type" : "ERROR","lineNumber" : 134,"columnNumber" : 22,"startPosition" : 4743,"endPosition" : 4751,"message" : "The method testImage() in the type pictures.TestUrl is not applicable for the arguments (java.lang.String)"}]
I will now show you the TestUrl class which I call to invoke my two methods testImage() and getImage():
package pictures;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import javax.imageio.ImageIO;
/*
* By: Victor Foning
*
* This program will consist of two Methods:
*
* The First will Test the Validity and reachability of an validity
* of an image URL.
*
* The second methods will log4j
*
*
*
*
*
*/
public class TestUrl {
public Boolean testImage (String l) {
// String urlString = "http://www.eurobiopark.org/sites/default/files/EurobioparkMashups5.1.png";
System.out.println("Using " + l);
// Open connection
URL u = null;
try {
u = new URL( l);
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
URLConnection connection = null;
try {
connection = u.openConnection();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// Check if response code is HTTP_OK (200)
HttpURLConnection httpConnection
= (HttpURLConnection) connection;
int code = 0;
try {
code = httpConnection.getResponseCode();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
String message = null;
try {
message = httpConnection.getResponseMessage();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println(code + " " + message);
if (code == HttpURLConnection.HTTP_OK) {
return true;
} else {
return false;
}
}
public void getImage (String u) {
BufferedImage image =null;
try{
URL url =new URL(u);
// read the url
image = ImageIO.read(url);
ImageIO.write(image, "jpg",new File("C://Users//Foning//Desktop//GeoDataLab//mash7.jpg"));
}catch(IOException e){
e.printStackTrace();
}
}
}
The Picture is downloaded locally and the Console out is this:
Using http://www.eurobiopark.org/sites/default/files/EurobioparkMashups5.1.png
200 OK
true
Here is my TestDownload main Class through which I call my two Methods:
package pictures;
import java.io.IOException;
/*
* By: Victor Foning 17/Septembre/2019
*
* From this Main Methods we will call:
*
* TestUrl.java and the GetImage.java Methods
*
*
*
*/
public class TestDownload {
public static void main(String[] args) {
String path = "http://www.eurobiopark.org/sites/default/files/EurobioparkMashups5.1.png";
// We Begin encapsulating the TestUrl Methods
TestUrl im = new TestUrl();
boolean image = im.testImage(path);
if(image){
im.getImage(path);
System.out.print("true");
}
else{
System.out.print(" victor_WakeUP_false");
}
}
}
Here I then export the .jar file (in Bold) into Wavemaker and Make the same method invocation through my JavaService Class:
package com.demo_jquery.myjavaservice1;
import javax.servlet.http.HttpServletRequest;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import pictures.TestUrl;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import org.springframework.beans.factory.annotation.Autowired;
import com.wavemaker.runtime.security.SecurityService;
import com.wavemaker.runtime.service.annotations.ExposeToClient;
import com.wavemaker.runtime.service.annotations.HideFromClient;
Here you will find the class getImageFromWaveMaker()I create inside my javaService1 class to invoke my two methods:
package com.demo_jquery.myjavaservice1;
import javax.servlet.http.HttpServletRequest;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import pictures.TestUrl;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import org.springframework.beans.factory.annotation.Autowired;
import com.wavemaker.runtime.security.SecurityService;
import com.wavemaker.runtime.service.annotations.ExposeToClient;
import com.wavemaker.runtime.service.annotations.HideFromClient;
import com.demo_jquery.myjavaservice1.model.*;
#ExposeToClient
public class MyJavaService1 {
private static final Logger logger = LoggerFactory.getLogger(MyJavaService1.class);
#Autowired
private SecurityService securityService;
public void getImageFromWavemaker( String p) {
String path =
"http://www.eurobiopark.org/sites/default/files/EurobioparkMashups5.1.png";
// We Begin encapsulating the TestUrl Methods
TestUrl im = new TestUrl();
boolean image = im.testImage(path);
if(image){
//im.getImage(url);
logger.info("true");
}
else{
logger.info("victor_WakeUP_false");
}
}
}
Please help me figure out why I get and Error when I run this code?
Thanks to a bit of meditation and positive re-orientation from some good friends I was able to solve this issue.
I came to realize that even though camelcase is accepted when declaring method. I re-declared both methods in my TestUrl Class by starting them with a Capital Letter.
With reference to the Error Code:
Aslo, understanding that all classes in java are derived from the object class and that the String class is one of the most important class in Java. I made that java.lang.String class the Super Class when I created all my Classes in Eclipse.
I did a maven Compile, which generated a .jar file. I proceeded to upload the .jar file in my resource lib file on Wavemaker, Binded my variables accordingly with my JavaService class and the results were good.
Please feel free to express any further perspective on this issue, I would be glad to hear it. Thank you!
Can anyone tell me how to store copied data into String in selenium?
example:
driver.findElement(By.xpath("//*[#id="x")).sendKeys(Keys.CONTROL, "a"));
driver.findElement(By.xpath("//*[#id="xy")).sendKeys(Keys.CONTROL, "c"));
driver.findElement(By.xpath("//*[#id="xy")).sendKeys(Keys.CONTROL, "v"));
I need put the copied data into String
Something like this
String text = driver.findElement(By.xpath("//*[#id="xy")).sendKeys(Keys.CONTROL, "v"));
It seems you want to send ctrl + c keys, which will copy some data in clipboard.. want to store that data in String variable..right?
You have to use Clipboard class to do so..See implementation below...
package resources;
import java.awt.Toolkit;
import java.awt.datatransfer.Clipboard;
import java.awt.datatransfer.DataFlavor;
import java.awt.datatransfer.Transferable;
import java.awt.datatransfer.UnsupportedFlavorException;
import java.io.IOException;
import org.openqa.selenium.By;
import org.openqa.selenium.Keys;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.Test;
public class One {
#Test
public void getClipboardContents() {
String result = "";
System.setProperty("webdriver.chrome.driver", "C://WebDrivers/chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.get("http://www.google.com");
String copy = Keys.chord(Keys.CONTROL,Keys.chord("c"));
driver.findElement(By.linkText("Images")).sendKeys(copy);
Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard();
Transferable contents = clipboard.getContents(null);
boolean hasTransferableText = (contents != null)
&& contents.isDataFlavorSupported(DataFlavor.stringFlavor);
if (hasTransferableText) {
try {
result = (String) contents.getTransferData(DataFlavor.stringFlavor);
} catch (UnsupportedFlavorException | IOException ex) {
System.out.println(ex);
ex.printStackTrace();
}
}
System.out.println(result);
}
}
#Starlord ..modify locators as per your need.
#Gaurav Thanks for the code.. I have modified some part of the code.. now I am getting what i want exactly
public void getClipboardContents()
throws UnsupportedFlavorException, IOException {
String result = "google.com";
System.setProperty("webdriver.chrome.driver", "E:\\New folder\\chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.get("https://www.google.co.in/");
String copy = Keys.chord(Keys.CONTROL,Keys.chord("c"));
driver.findElement(By.xpath("//*[#id=\"lst-ib\"]")).sendKeys("google.com");
driver.findElement(By.xpath("//*[#id=\"lst-ib\"]")).sendKeys(Keys.CONTROL+"a");
driver.findElement(By.xpath("//*[#id=\"lst-ib\"]")).sendKeys(copy);
Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard();
Transferable contents = clipboard.getContents(null);
String x = (String) contents.getTransferData(DataFlavor.stringFlavor);
System.out.println(x);
int a= result.length();
int b = x.length();
System.out.println(a);
System.out.println(b);
if(a<=b)
{
System.out.println("Matched Character length")
}else
{
System.out.println("Issue In Character length");
}
}
}
I am not able to get properties from an ontology file. I am new to ontology and apache jena. And also I am not able to use 'getProperty' command in proper way.
Till these code I am able to get classes but I don't know how to use 'getProperty','listObjectsOfProperty' command to get properties.
package onto1;
import java.io.InputStream;
import org.semarglproject.vocab.OWL;
import com.hp.hpl.jena.ontology.OntClass;
import com.hp.hpl.jena.ontology.OntModel;
import com.hp.hpl.jena.ontology.OntModelSpec;
import com.hp.hpl.jena.rdf.model.ModelFactory;
import com.hp.hpl.jena.util.FileManager;
import com.hp.hpl.jena.util.iterator.ExtendedIterator;
public class ontolo {
public static void main(String[] args) {
OntModel model = ModelFactory.createOntologyModel(OntModelSpec.OWL_MEM);
String inputFileName = "file:///F:/apache-jena-2.12.1/travel.owl";
InputStream in = FileManager.get().open( inputFileName );
if (in == null) {
throw new IllegalArgumentException("File: " + inputFileName + " not found");
}
//Property hasTime = model.createProperty( "file:///F:/apache-jena-2.12.1/shopping.owl#" );
//Property getProperty ( inputFileName, hastime) ;
model.read(in, null);
com.hp.hpl.jena.rdf.model.Property irrr = model.getProperty(OWL.ON_PROPERTIES);
com.hp.hpl.jena.rdf.model.NodeIterator iter1 = model.listObjectsOfProperty(irrr);
com.hp.hpl.jena.rdf.model.ResIterator i = model.listSubjectsWithProperty (irrr);
//com.hp.hpl.jena.rdf.model.Statement iir = model.getRequiredProperty(inputFileName, irrr);
//com.hp.hpl.jena.rdf.model.NodeIterator iter2 = model.listObjectsOfProperty(inputFileName.subClassOf);
ExtendedIterator<OntClass> iter = ((OntModel) model).listClasses();
while ( iter.hasNext()){
System.out.println(iter.next().toString());
}
// write it to standard out
model.write(System.out);
}
}
I am facing the following problem:
I will receive one or more .java files. The idea is to automate the compile and execute process.
I have not written nor viewed .java source files i receive.
There may be 1 file or multiple files in multiple directories.
All of this is done under linux ( Debian / CentOS ).
Here is an example case:
2 files are received:
SomeFile.java and SomeOtherFile.Java ( This one has the static public void main(String args[]){} method but i do NOT know that !)
A process picks up the files and compiles them in this way:
javac -encoding UTF-8 -sourcepath . -d . *.java
So my problem is: I do not know which package(if any) contains the Main method so i do not know what do execute ?
java packageName.SomeOtherFile
Plenty of ways:
Get a java source code parser, parse the source code, find the method
Compile everything, go over the resulting *.class files using reflection, find the method.
For a small enough number of files, just try them all
I have written something like this before:
package vonbulow.nicki;
import java.io.ByteArrayInputStream;
import java.io.DataInputStream;
import java.io.File;
import java.io.FilenameFilter;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.jar.Manifest;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
/**
*
* #author Nicki
*/
public class AppLoader extends ClassLoader {
private static final String userdir = System.getenv("USERPROFILE");
private static final AppLoader instance = new AppLoader();
private static HashMap<String, Class> loaded = new HashMap<String, Class>();
public static void loadapp(final String name) {
if(loaded.containsKey(name)) {
Thread d = new Thread(new Runnable(){
public void run(){
try {
Class c = loaded.get(name);
Method m = c.getDeclaredMethod("main", String[].class);
m.invoke(null, (Object[])new String[]{null});
} catch (NoSuchMethodException | SecurityException | IllegalAccessException | IllegalArgumentException | InvocationTargetException ex) {
}
}
});
d.start();
return;
}
File ud = new File(userdir+"\\nvbapp");
ud.mkdir();
File[] fa = ud.listFiles(new FilenameFilter() {
public boolean accept(File dir, String name) {
return name.toLowerCase().endsWith(".appn");
}
});
for(final File f:fa) {
if(f.getName().split("\\.")[0].equalsIgnoreCase(name)) {
Thread t = new Thread(new Runnable() {
public void run() {
runapp(f, name);
}
});
t.start();
}
}
}
private static void runapp(File f, String nam) {
List<Class> classes = new ArrayList<Class>();
ZipFile jf;
String name = "";
try {
jf = new ZipFile(f);
Enumeration<? extends ZipEntry> eze = jf.entries();
while(eze.hasMoreElements()){
ZipEntry ze = eze.nextElement();
if(ze.getName().endsWith(".class")&&!ze.isDirectory()){
InputStream fis = jf.getInputStream(ze);
byte[] bytes = new byte[(int)ze.getSize()];
fis.read(bytes);
classes.add(instance.defineClass(getClassName(bytes), bytes, 0, bytes.length));
}
if(ze.getName().equalsIgnoreCase("META-INF/MANIFEST.MF")) {
Manifest manifest = new Manifest(jf.getInputStream(ze));
name = manifest.getMainAttributes().getValue("Main-Class");
}
}
Iterator<Class> classit = classes.iterator();
while(classit.hasNext()) {
Class c = classit.next();
if(c.getName().equals(name)) {
try {
loaded.put(nam, c);
Method m = c.getDeclaredMethod("main", String[].class);
m.invoke(null, (Object[]) new String[]{null});
} catch (NoSuchMethodException | SecurityException | IllegalAccessException | IllegalArgumentException | InvocationTargetException ex) {
ex.printStackTrace();
}
}
}
} catch (IOException ex) {
Logger.getLogger(AppLoader.class.getName()).log(Level.SEVERE, null, ex);
}
}
private static String getClassName(byte[] is) {
try {
DataInputStream dis = new DataInputStream(new ByteArrayInputStream(is));
dis.readLong(); // skip header and class version
int cpcnt = (dis.readShort()&0xffff)-1;
int[] classes = new int[cpcnt];
String[] strings = new String[cpcnt];
for(int i=0; i<cpcnt; i++) {
int t = dis.read();
if(t==7) classes[i] = dis.readShort()&0xffff;
else if(t==1) strings[i] = dis.readUTF();
else if(t==5 || t==6) { dis.readLong(); i++; }
else if(t==8) dis.readShort();
else dis.readInt();
}
dis.readShort(); // skip access flags
return strings[classes[(dis.readShort()&0xffff)-1]-1].replace("/", ".");
} catch (IOException ex) {
return null;
}
}
}
I have not edited it; you will need to edit it so it loads your own classes.
You also need to compile the files first with the JavaCompiler class. This also assumes that the classes are in the zip file.
guys..i has a simple question.
is it possible to display each class console result on java GUI ?
Each class has different console results..
import org.jsoup.Jsoup;
import org.jsoup.helper.Validate;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
public class SimpleWebCrawler {
public static void main(String[] args) throws IOException {
try {
URL my_url = new URL("http://theworldaccordingtothisgirl.blogspot.com/");
BufferedReader br = new BufferedReader(new InputStreamReader(
my_url.openStream()));
String strTemp = "";
while (null != (strTemp = br.readLine())) {
System.out.println(strTemp);
}
} catch (Exception ex) {
ex.printStackTrace();
}
System.out.println("\n");
System.out.println("\n");
System.out.println("\n");
Validate.isTrue(args.length == 0, "usage: supply url to crawl");
String url = "http://theworldaccordingtothisgirl.blogspot.com/";
print("Fetching %s...", url);
Document doc = Jsoup.connect(url).get();
Elements links = doc.select("a[href]");
System.out.println("\n");
BufferedWriter bw = new BufferedWriter(new FileWriter("abc.txt"));
for (Element link : links) {
print(" %s ", link.attr("abs:href"), trim(link.text(), 35));
bw.write(link.attr("abs:href"));
bw.write(System.getProperty("line.separator"));
}
bw.flush();
bw.close();
}
private static void print(String msg, Object... args) {
System.out.println(String.format(msg, args));
}
private static String trim(String s, int width) {
if (s.length() > width)
return s.substring(0, width - 1) + ".";
else
return s;
}
}
Example output :
Fetching http://theworldaccordingtothisgirl.blogspot.com/...
http://theworldaccordingtothisgirl.blogspot.com/2011/03/in-time-like-this.html
https://lh5.googleusercontent.com/-yz2ql0o45Aw/TYBNhyFVpMI/AAAAAAAAAGU/OrPZrBjwWi8/s1600/Malaysian-Newspaper-Apologises-For-Tsunami-Cartoon.jpg
http://ireport.cnn.com/docs/DOC-571892
https://lh3.googleusercontent.com/-nXOxDT4ZyWA/TX-HaKoHE3I/AAAAAAAAAGQ/xwXJ-8hNt1M/s1600/ScrnShotsDesktop-1213678160_large.png
http://theworldaccordingtothisgirl.blogspot.com/2011/03/in-time-like-this.html#comments
http://www.blogger.com/share-post.g?blogID=3284083343891767749&postID=785884436807581777&target=email
http://www.blogger.com/share-post.g?blogID=3284083343891767749&postID=785884436807581777&target=blog
http://www.blogger.com/share-post.g?blogID=3284083343891767749&postID=785884436807581777&target=twitter
http://www.blogger.com/share-post.g?blogID=3284083343891767749&postID=785884436807581777&target=facebook
http://www.blogger.com/share-post.g?blogID=3284083343891767749&postID=785884436807581777&target=buzz
If you want to separate the standard output (System.out.println) based on which class it comes from, the answer is, no, (not easily at least).
I suggest you let each class that wants to do output get a PrintWriter as argument to the constructor, then use that PrintWriter instead of the System.out print writer.