Elements elements = doc.select("span.st");
for (Element e : elements) {
out.println("<p>Text : " + e.text()+"</p>");
}
Element e contains text with some email id in it. How to extract the maild id from it. I have seen the Jsoup API doc which provides :matches(regex), but I didn't understand how to use it. I'm trying to use
^[a-zA-Z0-9_!#$%&’*+/=?`{|}~^.-]+#[a-zA-Z0-9.-]+$
which I found while googling.
Thank in advance for your help.
:matches(regex) is useful if you want to find something based on a specified regex (e.g. find all nodes that contain email).
I think this is not what you want. Instead, you need to extract the email from e.text() using regex. In your case:
Elements elements = doc.select("span.st");
for (Element e : elements) {
out.println("<p>Text : " + e.text()+"</p>");
out.println(extractEmail(e.text()));
}
// ...
public static String extractEmail(String str) {
Matcher m = Pattern.compile("[a-zA-Z0-9_.+-]+#[a-zA-Z0-9-]+\\.[a-zA-Z0- 9-.]+").matcher(str);
while (m.find()) {
return m.group();
}
return null;
}
Related
Code:
Document doc = Jsoup.connect("things.com").get();
Elements jpgs = doc.select("img[src$=.jpg]");
String links = jpgs.attr("src");
System.out.print("all: " + jpgs);
System.out.print("src: " + links);
Output:
all:
<img alt="Apple" src="apple.jpg">
<img alt="Cat" src="cat.jpg">
<img alt="Boat" src="boat.jpg">
src: apple.jpg
Jsoup gave the attribute value for first element. How can I get the others (cat.jpg and boat.jpg)?
Thank you.
You loop through links and get it from each one via Element#attr, since Elements#attr (note the s) says:
Get an attribute value from the first matched element that has the attribute.
(My emphasis.)
So for instance:
for (Element e : jpgs) {
// use e.attr("src") here
}
Using Java 8's new Stream stuff, you can probably get a List<String> of them if you like:
List<String> links = jpgs.stream<Element>()
.map(element -> element.attr("src"))
.collect(Collectors.toList());
...but my Java 8 streams-fu is very weak, so that may not be quite right. Yeah, that isn't right. But that's the general idea.
The boring old-fashioned way is:
List<String> links = new ArrayList<String>(links.size());
for (Element e : jpgs) {
srcs.add(e.attr("src"));
}
Elements#attr will only return the first match.
Elements#attr Source Code
public String attr(String attributeKey) {
for (Element element : this) {
if (element.hasAttr(attributeKey))
return element.attr(attributeKey);
}
return "";
}
Solution
To obtain the result you want, you should loop over your Elements
for (Element e : jpgs) {
System.out.println(e.attr("src"));
}
Is there an efficient way to optimize this code, as most part of it look like identical, I just started learning jsoup and dont know how really can do that ://
Document doc = Jsoup.connect("http://www.blocket.se/hela_sverige/bilar?ca=11&cg=1020&w=3&md=th").get();
Elements partOne = doc.select("a[title=Flera bilder]");
for (Element element : partOne) {
String myElementOne = element.attr("abs:href");
System.out.println(myElementOne);
}
Elements partTwo = doc.select("a[title=\"\"]");
for (Element element : partTwo) {
String myElementTwo = element.attr("abs:href");
System.out.println(myElementTwo);
}
Elements partThree = doc.select("a[title=Bild]");
for (Element element : partThree) {
String myElementThree = element.attr("abs:href");
System.out.println(myElementThree);
}
The partOne, partTwo and partThree blocks are basically identical; just replace all of the parameter differences with variables and extract to a method:
void someMethodName(Document doc, String selector) {
Elements partOne = doc.select(selector);
for (Element element : partOne) {
String myElementOne = element.attr("abs:href");
System.out.println(myElementOne);
}
}
Example invocation:
someMethodName(doc, "a[title=Flera bilder]");
Alternatively, if you have access to Guava:
Iterable<Element> it = Iterables.concat(
doc.select("a[title=Flera bilder]"),
doc.select("a[title=\"\"]"),
doc.select("a[title=Bild]"));
for (Element element : it) {
String myElement = element.attr("abs:href");
System.out.println(myElement);
}
Andy's solution is of course doing the job. However, since you asked specifically for ways optimizing the JSoup calls, I would suggest to learn more about CSS selectors and regular expressions. For example this will do fine in your case:
Elements allParts = doc.select("a[title~=^Flera bilder$|^$|^Bild$]");
for (Element element : allParts) {
String elStr = element.attr("abs:href");
System.out.println(elStr);
}
Here, I use the ~= operator for attribute texts. It allows me to use a common regular expression to combine all three of your select statements into one.
An alternative way of doing this would be to use the , operator for adding all selectors into one:
Elements allParts2 = doc.select("a[title=Flera bilder],a[title=\"\"],a[title=Bild]");
I am trying to parse XML from URL using Jsoup.
In this given XML there are nodes with namespace.
for ex: <wsdl:types>
Now I want to get all nodes which contain text as "types" but can have any namespace.
I am able to get this nodes using expression as "wsdl|types".
But how can I get all nodes containing text as "types" having any namespace. ?
I tried with expression as "*|types" but it didn't worked.
Please help.
There is no such selector (yet). But you can use a workaround - a not as easy to read like a selector, but it's a solution.
/*
* Connect to the url and parse the document; a XML Parser is used
* instead of the default one (html)
*/
final String url = "http://www.consultacpf.com/webservices/producao/cdc.asmx?wsdl";
Document doc = Jsoup.connect(url).parser(Parser.xmlParser()).get();
// Elements of any tag, but with 'types' are stored here
Elements withTypes = new Elements();
// Select all elements
for( Element element : doc.select("*") )
{
// Split the tag by ':'
final String s[] = element.tagName().split(":");
/*
* If there's a namespace (otherwise s.length == 1) use the 2nd
* part and check if the element has 'types'
*/
if( s.length > 1 && s[1].equals("types") == true )
{
// Add this element to the found elements list
withTypes.add(element);
}
}
You can put the essential parts of this code into a method, so you get something like this:
Elements nsSelect(Document doc, String value)
{
// Code from above
}
...
Elements withTypes = nsSelect(doc, "types");
I'm using JSoup to retrive reviews from a particular webpage in Amazon and what I have now is this:
Document doc = Jsoup.connect("http://www.amazon.com/Presto-06006-Kitchen-Electric-Multi-Cooker/product-reviews/B002JM202I/ref=sr_1_2_cm_cr_acr_txt?ie=UTF8&showViewpoints=1").get();
String title = doc.title();
Element reviews = doc.getElementById("productReviews");
System.out.println(reviews);
This gives me the block of html which has the reviews but I want only the text without all the tags div etc. I want to then write all this information into a file. How can I do this? Thanks!
Use text() method
System.out.println(reviews.text());
While text() will get you a bunch of text, you'll want to first use jsoup's select(...) methods to subdivide the problem into individual review elements. I'll give you the first big division, but it will be up to you to subdivide it further:
public static List<Element> getReviewList(Element reviews) {
List<Element> revList = new ArrayList<Element>();
Elements eles = reviews.select("div[style=margin-left:0.5em;]");
for (Element element : eles) {
revList.add(element);
}
return revList;
}
If you analyze each element, you should see how amazon further subdivides the information held including the title of the review, the date of the review and the body of the text it holds.
I have requirement of selecting s subset of data from a collection using a part of a key word
Assume that I have a collection consist of following entries,
"Pine Grosbeak"
"House Crow"
"Hume`s Leaf Warbler"
"Great Northern Loon"
"Long-tailed Duck"
"Lapland Longspur"
"Northern Gannet"
"Eastern Imperial Eagle"
"Little Auk"
"Lesser Spotted Woodpecker"
"Iceland Gull"
When I provide a search string as “ro” following should be filtered;
"House Crow"
"Pine Grosbeak"
(This is something similar to “LIKE ‘%ro%’ “ clause in SQL)
Can some one help me on this?
You could simply do something like that:
for (String s : strings)
{
if (s.contains(search))
{
// match
}
}
And if you want to be case insensitive:
String lowerSearch = search.toLowerCase();
for (String s : strings)
{
if (s.toLowerCase().contains(lowerSearch))
{
// match
}
}
Iterate through and use
contains() method to filter
List<String> lst = new ArrayList<String>();
lst.add("Pine Grosbeak");
lst.add("House Crow");
.
.
.
for(String str:lst){
if(str.contains(keyword)){
System.out.println("matched : "+ str);
}
}