We Keep Coding

extract URL location from an xml link in java

I'm new in java and i have a link "https://moz.com/blog-sitemap.xml" that has URLs ,i want to get them and save them in a string vector/array.
i tried this first to see how i'm going to get the links
URL robotFile = new URL("https://moz.com/blog-sitemap.xml");
//read robot.txt line by line
Scanner robotScanner = new Scanner(robotFile.openStream());
while (robotScanner.hasNextLine()) {
System.out.println(robotScanner.nextLine());
}
this is the sample output
my answer is ,is there a simple easier way to get these links instead of looping on each line checking if it contains "https" so i can extract the link from it ?

You can use Jsoup to do this more easly:
List<String> urlList = new ArrayList<>();
Document doc = Jsoup.connect("https://moz.com/blog-sitemap.xml").get();
Elements urls = doc.getElementsByTag("loc");
for (Element url : urls) {
urlList.add(url.text());
}

How to remove byte order mark using scala.io.Source?

Byte order mark is making my regex fail when using scala.io.Source to read from a file. This answer is a lightweight solution using java.io. Is there anything similar for scala.io.Source, or will I have to revert back to Java because of a single byte?

Based on Joe K's idea in his comment, and using Andrei Punko's answer for the problem in Java and Alvin Alexander's Scala code, the simplest solution to read a file possibly containing byte order mark into an array of string is:
#throws[IOException]
def skip(reader: Reader): Unit = {
reader.mark(1)
val possibleBOM = new Array[Char](1)
reader.read(possibleBOM)
if (possibleBOM(0) != '\ufeff') reader.reset
}
val br = new BufferedReader(new InputStreamReader(new FileInputStream(file)))
skip(br)
val lines = {
val ls = new ArrayBuffer[String]()
var l: String = null
while ({l= br.readLine; l != null}) {
ls.append(l)
}
br.close
ls.toArray
}

xpath: write to a file

I'm developing Java code to get data from a website and store it in a file. I want to store the result of xpath into a file. Is there any way to save the output of the xpath? Please forgive for any mistakes; this is my first question.
public class TestScrapping {
public static void main(String[] args) throws MalformedURLException, IOException, XPatherException {
// URL to be fetched in the below url u can replace s=cantabil with company of ur choice
String url_fetch = "http://www.yahoo.com";
//create tagnode object to traverse XML using xpath
TagNode node;
String info = null;
//XPath of the data to be fetched.....use firefox's firepath addon or use firebug to fetch the required XPath.
//the below XPath will display the title of the company u have queried for
String name_xpath = "//div[1]/div[2]/div[2]/div[1]/div/div/div/div/table/tbody/tr[1]/td[2]/text()";
// declarations related to the api
HtmlCleaner cleaner = new HtmlCleaner();
CleanerProperties props = new CleanerProperties();
props.setAllowHtmlInsideAttributes(true);
props.setAllowMultiWordAttributes(true);
props.setRecognizeUnicodeChars(true);
props.setOmitComments(true);
//creating url object
URL url = new URL(url_fetch);
URLConnection conn = url.openConnection(); //opening connection
node = cleaner.clean(new InputStreamReader(conn.getInputStream()));//reading input stream
//storing the nodes belonging to the given xpath
Object[] info_nodes = node.evaluateXPath(name_xpath);
// String li= node.getAttributeByName(name_xpath);
//checking if something returned or not....if XPath invalid info_nodes.length=0
if (info_nodes.length > 0) {
//info_nodes[0] will return string buffer
StringBuffer str = new StringBuffer();
{
for(int i=0;i<info_nodes.length;i++)
System.out.println(info_nodes[i]);
}
/*str.append(info_nodes[0]);
System.out.println(str);
*/
}
}
}

You can "simply" print the nodes as strings, to console/or a file --
example in Perl:
my $all = $XML_OBJ->find('/'); # selecting all nodes from root
foreach my $node ($all->get_nodelist()) {
print XML::XPath::XMLParser::as_string($node);
}
note: this output however may not be nicely xml-formatted/indented

The output of an XPath in Java is a nodeset, so yes, once you have a nodeset you can do anything you want with it, save it to a file, process it some more.
Saving it to a file would involve the same steps in java that saving anything else to a file involve, there is no difference between that and and any other data. Select the nodeset, itterate through it, get the parts you want from it and write them to some kind of file stream.
However, if you mean is there a Nodeset.SaveToFile(), then no.

I would recommend you to take the NodeSet, which is a collection of Nodes, iterate on it, and add it to a created DOM document object.
After this, you can use the TransformerFactory to get a Transformer object, and to use its transform method. You should transform from a DOMSource to a StreamResult object which can be created based on FileOutputStream.

Is there an easy way to parse JSON in Java without using some extra library?

I'm really new to JavaScript and JSON. I am writing a Java servlet that needs to parse JSON into Java objects, but the trick is that I cannot use external libraries such as json-simple or gson.
So I've already saved the contents of the JavaScript file in a String called "contents"...
contents = "myVar = " + contents + ";";
BufferedWriter bWriter = new BufferedWriter(new OutputStreamWriter(response.getOutputStream(), "UTF-8"));
ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("JavaScript");
Object returnVal = engine.eval(contents);
NativeObject a = (sun.org.mozilla.javascript.internal.NativeObject) engine.get("myVar");
Object[] ids = a.getAllIds();
for (Object id : ids) {
String newId = (String) id;
bWriter.write("\n" + newId);
}
This will print out all of the names of the various attributes of my object correctly. Thing is, I have no idea what to do from here. I've tried all sorts of different methods of NativeObject to no avail.
There must be something really obvious I'm missing. Can anyone help me out?
Thanks.

Android java load images from url

I have a question
private final String images[] = {"http://developer.android.com/images/dialog_custom.png", "http://developer.android.com/images/dialog_progress_bar.png"};
how can i change this
to load all of the links from a file from my website?
like a .txt on my site just has like
http://link.com/1.png
http://link.com/2.png
http://link.com/3.png
http://link.com/4.png
http://link.com/5.png
http://link.com/6.png
http://link.com/7.png
http://link.com/8.png
http://link.com/9.png
http://link.com/10.png
so it will just load all those then later on i can add more links without having to update the app

Use a BufferedReader to readLine() to a String and then fetch the image...

Just put the space or "," or "/n" in file then get all data in a string one by one
String reader = "";
String separated ;
BufferedReader buffer = new BufferedReader(new FileReader("YourFileName.txt"));
while ((reader = buffer.readLine()) != null) {
separated = reader.split(" "); or reader.split(","); or reader.split("/n");
//Here at each itration separated will have your single link
}
buffer.close;

Try out the following links:
How to load an ImageView by URL in Android?
http://russenreaktor.wordpress.com/2009/08/11/android-imageloader-load-images-sequencially-in-the-background/
There are plenty of links that provide solutions--enjoy.