jsoup Not Catching Full Webpage - java

Trying to make a super simple bit of code using jsoup to see if a webpage contains a specific word (checks to see the availability of a class to take in-residence). jsoup seems to be catching lots of the webpage specified, but won't capture all of it - specifically the area I'm interested in. Is there a reason for this? Am I doing something wrong?
import org.jsoup.Jsoup;
import java.io.IOException;
import org.jsoup.nodes.Document;
public class main {
public static void main(String[] args) throws IOException{
String html = Jsoup.connect("https://www.afit.edu/CE/Course_Desc.cfm?p=WENG%20481").maxBodySize(0).get().html();
System.out.print(html);
if (html.contains("Resident")) {
System.out.print("\nAVAILABLE!");
}
}
}

Related

Spanish language encoding issue with Java Properties class load method

I am trying to encode the Spanish language for internationalization and use the Java Properties class load method to populate it to pass it to frontend.
I have tried to encode it using UTF-8 but still the accent characters are not coming correctly.
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.nio.charset.Charset;
import java.util.Properties;
public class Message extends Properties {
public static void main(String[] args) throws IOException {
String spanish = "label=Sí";
Message messages = new Message();
messages.load(new ByteArrayInputStream(spanish.getBytes(Charset.forName("UTF-8"))));
System.out.println(messages.get("label"));
}
}
When i run the above code I am getting the text as "Sí-". How can I retrieve the same text as "Sí"?
Try using ISO-8859-1 it gets expected result:
messages.load(new ByteArrayInputStream(spanish.getBytes(Charset.forName("ISO-8859-1"))));
More about ISO-8859-1 can be obtained from this link:
https://en.wikipedia.org/wiki/ISO/IEC_8859-1

Google Translate from Java Application

I am trying to do a simple translator by NetBeans. Firstly, I tried to implement the code below from a forum page:(https://www.java-forums.org/java-applets/38563-language-translation.html)
import com.google.api.translate.Language;
import com.google.api.translate.Translate;
public class Main {
public static void main(String[] args) throws Exception {
// Set the HTTP referrer to your website address.
Translate.setHttpReferrer("http://code.google.com/p/google-api-translate-java");
String translatedText = Translate.execute("Bonjour monde le",
Language.FRENCH, Language.ENGLISH);
System.out.println(translatedText);
}
}
I cannot compile the code. I got cannot resolve symbol for setHttpReferrer() although I added related jar.
Secondly, I tried to implement another solution from the page (https://www.java-forums.org/java-applets/61655-language-translation-using-google-api.html). I got my API key and set it.
import com.google.api.GoogleAPI;
import com.google.api.translate.Language;
import com.google.api.translate.Translate;
public class Translation
{
public static void main(String[] args) throws Exception {
GoogleAPI.setHttpReferrer("http://code.google.com/p/google-api-translate-java");
GoogleAPI.setKey("i have set my Api key");
String translatedText = Translate.DEFAULT.execute("Bonjour le monde", Language.FRENCH, Language.ENGLISH);
System.out.println(translatedText);
}
}
When I try to run this I got 403 error as null. Is there a simple way to call Google Translator from Java application?
403 error is documented on the faq as "exceeding your quota". https://cloud.google.com/translate/faq
I suspect however, you get the error because you haven't initialised the API properly, i.e authenticated, ...
Have a look at the setup in this code. Also search for hello welt.
https://github.com/GoogleCloudPlatform/google-cloud-java/blob/master/google-cloud-translate/src/test/java/com/google/cloud/translate/TranslateImplTest.java
Hope this helps.

How do I use j2html without rendering everything

I am converting my html rendering code to use j2html. Whilst I like the library it is not easy for me to convert all the code in one go so sometimes I may convert the outer html to use j2html but not able to convert the inner html to j2html at the same time. So I would like j2html to be able to accept text passed to it as already rendered, but it always re-renders it so
System.out.println(p("<b>the bridge</b>"));
returns
<p><b>the bridge</b></p>
is there a way I get it to output
<p><b>the bridge</b></p>
Full Test Case
import j2html.tags.Text;
import static j2html.TagCreator.b;
import static j2html.TagCreator.p;
public class HtmlTest
{
public static void main(String[] args)
{
System.out.println(p(b("the bridge")));
System.out.println(p("<b>the bridge</b>"));
}
}
import static j2html.TagCreator.b;
import static j2html.TagCreator.p;
import static j2html.TagCreator.rawHtml;
public class HtmlTest
{
public static void main(String[] args)
{
System.out.println(p(b("the bridge")));
System.out.println(p(rawHtml("<b>the bridge</b>")));
}
}
Result:
<p><b>the bridge</b></p>
<p><b>the bridge</b></p>
In j2html 1.1.0 you can disable text-escaping by writing
Config.textEscaper = text -> text;
Be careful though..

How to download an online file directory (localhost) using Java

I have a static file server at "localhost:8888/fileserver".
I am trying to write a program in java to download the files from the server. The file server consists of three folders, therefore I am trying to write a script that automatically goes through the directory and copies it to my computer.
I know there is a wget function for linux that accomplishes this recursively. Is there a way to do this in Java?
Please could you advise on how I should go about doing this or proceed.
Thank you
Below is a code to go through an online directory and return back all the links needed to download.
After that I just need to download each individual link.
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
public class WebCrawler {
//Created a global list variable to save the links
static List<String> createList = new ArrayList<String>();
public static void main(String[] args) throws IOException {
String url = "http://localhost:8888";
System.out.println(myCrawler(url)+"\n"+"Size: "+myCrawler(url).size());
}
public static List<String> myCrawler(String url) throws IOException{
//Creates an open connection to a link
Document doc = Jsoup.connect(url).ignoreContentType(true).get();
Elements links = doc.select("a[href]");
//Recursively iterates through all the links provided on the initial url
for (Element i : links) {
String link = print("%s", i.attr("abs:href"));
if (link.endsWith("/")){myCrawler(link);} //Recursive part, calls back on itself
else {createList.add(link);}
}
return createList;
}
//Translates the link into a readable string object
private static String print(String msg, Object... args){return String.format(msg, args);}
}

Apache Camel - not moving file

I cannot figure out what I am doing wrong here. I have tried all sorts of things, including absolute paths, relative, enabling logging (which also doesnt seem to be working, using Main, using DefaultCamelContext, adding threadsleep, but I cannot get camel to move a file from one folder to another.
Here is my code:
package scratchpad;
import org.apache.camel.CamelContext;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.dataformat.beanio.BeanIODataFormat;
import org.apache.camel.impl.DefaultCamelContext;
import org.apache.camel.main.Main;
import org.apache.camel.spi.DataFormat;
public class CamelMain {
private static Main main;
public static void main(String[] args) throws Exception {
main = new Main();
main.addRouteBuilder( new RouteBuilder() {
#Override
public void configure() throws Exception {
// DataFormat format = new BeanIODataFormat(
// "org/apache/camel/dataformat/beanio/mappings.xml",
// "orderFile");
System.out.println("starting route");
// a route which uses the bean io data format to format a CSV data
// to java objects
from("file://input?noop=true&startingDirectoryMustExist=true")
.to("file://output");
}
});
//main.run();
main.start();
Thread.sleep(5000);
main.stop();
}
}
Can someone spot anything wrong with the above?
Thanks
You can for example read from the free chapter 1 for the Camel in Action book, as it has a file copied example it covers from top to bottom.
The pdf can be downloaded here: http://manning.com/ibsen/

Categories