We Keep Coding

Get EC2 Instance XML Description using AWS Java SDK?

We have a scenario in which we need to retrieve the description info for EC2 instances running on AWS. To accomplish this, we are using the AWS Java SDK. In 90% of our use case, the com.amazonaws.services.ec2.model.Instance class is exactly what we need. However, there is also a small use-case where it would be beneficial to get the raw XML describing the instance. That is, the XML data before it is converted into the Instance object. Is there any way to obtain both the Instance object and the XML string using the AWS Java SDK? Is there a way to manually convert from one to the other? Or, would we be forced to make a separate call using HttpClient or something similar to get the XML data?

Make an EC2Client by adding request handler and override the beforeUnmarshalling() method like below
AmazonEC2ClientBuilder.standard().withRegion("us-east-1")
.withRequestHandlers(
new RequestHandler2() {
#Override
public HttpResponse beforeUnmarshalling(Request<?> request, HttpResponse httpResponse) {
// httpResponse.getContent() is the raw xml response from AWS
// you either save it to a file or to a XML document
return new HTTPResponse(...);
// if you consumed httpResponse.getContent(), you need to provide new HTTPResponse
}
}
).build():

If you have xml (e.g. from using AWS rest API directly), then you can use com.amazonaws.services.ec2.model.transform.* classes to convert xml to java objects. Unfortunately, it only provides classes required for SDK itself. So you, for example, can convert raw XML to an Instance using InstanceStaxUnmarshaller, but can't convert Instance to XML unless you write such converter.
Here is an example how to parse an Instance XML:
XMLEventReader eventReader = XMLInputFactory.newInstance().createXMLEventReader(new StringReader(instanceXml));
StaxUnmarshallerContext suc = new StaxUnmarshallerContext(eventReader, new TreeMap<>());
InstanceStaxUnmarshaller isu = new InstanceStaxUnmarshaller();
Instance i = isu.unmarshall(suc);
System.out.println(i.toString());
You probably can try to intercept raw AWS response, so that you can keep raw XML while still using SDK most of the time. But I wouldn't call that easy as it will require quite a bit of coding.

You could use JAXB.marshal like following. JAXB (Java Architecture for XML Binding) could convert Java object to / from XML file.
StringWriter sw = new StringWriter();
JAXB.marshal(instance, sw);
String xmlString = sw.toString();

You can use AWS rest API to replace Java SDK. A bonus will be slight performance gain because you'll not send statistic data to Amazon as the SDK does.

Looking for a less verbose way to parse JSON in Android (Java)

I'm writing an app that uses JSON (from a Kimonolabs API) and while I was able to utilize my API effortlessly in Python as such:
results = json.loads(urllib.request.urlopen(KimonoAPIlink).read().decode('utf-8'))
Which returns a useable JSON dictionary:
title = results['results']['suart'][0]['stitle']['text']
href = results['results']['suart'][0]['stitle']['href']
In Java (Android specifically) I have to do it this way (Using Gson):
URL url = new URL(KimonoAPIlink);
HttpURLConnection request = (HttpURLConnection) url.openConnection();
request.connect();
JsonParser jp = new JsonParser(); //from gson
JsonElement root = jp.parse(new InputStreamReader((InputStream) request.getContent()));
JsonObject rootobj = root.getAsJsonObject(); //May be an array, may be an object.
JsonElement x = rootobj.getAsJsonObject("results").getAsJsonArray("suart").get(0);
JsonObject y = x.getAsJsonObject();
title.setText(y.getAsJsonObject("stitle").getAsJsonPrimitive("text").toString().replace("\"", "").replace("\\","\""));
Is there any way to do this that isn't so verbose and complicated? (my code works, I'd just like it to be more simple and clean)
Moreover, can I somehow parse the whole nested JSON object into something useful (like python does) and not reparse it each layer I access?
Thanks in advance.
Edit: I've also seen people use Apache commons.io on here for this, but I don't think it returns a different JSON object (i.e. I'd still have to parse every layer)

Check out GSON, Jackson or Moshi

What you are looking for is lookup using JsonPath, wherein JSON coordinates are represented by a string with syntax pretty similar to the one you are using in Python (as well as other featuress).
For example, in your case it would be $.results.suart[o].stitle.text.
A quick look in the docs and the gson github shows it is not implemented in gson, although the JsonReader class does support a method to return that value.
There is a library implementation JsonPath in java (called JsonPath). Here's a one line code for your case (ignoring connection detail):
String result = JsonPath.parse("[some json]").read("$.results.suart[o].stitle.text");

How do you get the xml version of a document using org.jdom2?

I am using org.jdom2 to parse xml files. I need to know if the file is marked as version 1.1 or version 1.0. How do I access the xml declaration?
Also how do I set the version when writing the output using the XMLOutputter?

The XML Version is parsed and used by the XML parser (SAX). Some parsers support the SAX2 API, and that allows some of the parsers to supply extended parsing information. If the parser does this, the XML version may be available in the Locator2 implementation getXMLVersion(). JDOM does not have a hook on this information, so the data is not yet available in JDOM. It would make a good feature request.
JDOM also outputs data in XML 1.0 version. The differences between 1.0 and 1.1 from JDOM's perspective are slight. The most significant difference is the slightly different handling between different supported characters.
If you want to specify a different XML version for your output you can force the declaration by disabling the XMLOutputter's declaration (setOmitDeclaration() and then dump the declaration yourself on to the stream before outputting the XML.
Alternatively you can extend the XMLOutputProcessor and override the processDelcaration() method to outpuit the declaration you want.
None of these options are easy, and the support for XML 1.1 in JDOM is limited. Your mileage may vary, but please keep me updated on your success, and file issues on the Github issues if you have suggestions/problems: https://github.com/hunterhacker/jdom/issues

I fully believe that rolfl's answer is correct. It isn't the approach I finally took. I decided to just do a quick parsing of the document myself. This probably needs further testing with documents with BOM.
private static Pattern xmlDeclaration = Pattern.compile("<?xml.* version=\"([\\d|\\.]+)\".*?>");
private static boolean isXml10(InputStream inputStream) throws IOException
{
boolean result = true;
InputStreamReader is = null;
BufferedReader br = null;
try
{
is = new InputStreamReader(inputStream);
br = new BufferedReader(is);
String line = br.readLine();
Matcher declarationMatch = xmlDeclaration.matcher(line);
if (declarationMatch.find())
{
String version = declarationMatch.group(1);
result = version.equals("1.0");
}
}
finally
{
is.close();
br.close();
}
return result;
}

What is the fastest way to get a HTML Content using java?

I have this, but I was wondering if there is a faster way:
URL url=new URL(page);
InputStream is = new BufferedInputStream(url.openConnection().getInputStream());
BufferedReader in=new BufferedReader(new InputStreamReader(is));
String tmp="";
StringBuilder sb=new StringBuilder();
while((tmp=in.readLine())!=null){
sb.append(tmp);
}

Probably network is the biggest overhead, there isn't much you can do on Java code side. But using IOUtils is at least much faster to implement:
String page = IOUtils.toString(url.openConnection().getInputStream());
Remember to close underlying stream.

if you need manipulating with your html, find some library. Like for example jsoup.
jsoup is a Java library for working with real-world HTML. It provides
a very convenient API for extracting and manipulating data, using the
best of DOM, CSS, and jquery-like methods.
Example:
Document doc = Jsoup.connect("http://en.wikipedia.org/").get();
Elements newsHeadlines = doc.select("#mp-itn b a");

If you're using Apache Commons IO's IOUtils as Tomasz suggests, there's an even simpler method: toString(URL), or its preferred cousins that take a charset (of course that requires knowing the resource's charset in advance).
String string = IOUtils.toString( new URL( "http://some.url" ));
or
String string = IOUtils.toString( new URL( "http://some.url" ), "US-ASCII" );

How to use cURL in Java?

I want to use curl in java. Is curl built-in with Java or I have to install it from any 3rd party source to use with Java? If it needs to be separately installed, how can that be done?

You can make use of java.net.URL and/or java.net.URLConnection.
URL url = new URL("https://stackoverflow.com");
try (BufferedReader reader = new BufferedReader(new InputStreamReader(url.openStream(), "UTF-8"))) {
for (String line; (line = reader.readLine()) != null;) {
System.out.println(line);
}
}
Also see the Oracle's simple tutorial on the subject. It's however a bit verbose. To end up with less verbose code, you may want to consider Apache HttpClient instead.
By the way: if your next question is "How to process HTML result?", then the answer is "Use a HTML parser. No, don't use regex for this.".
See also:
How to use java.net.URLConnection to fire and handle HTTP requests?
What are the pros and cons of the leading Java HTML parsers?

Some people have already mentioned HttpURLConnection, URL and URLConnection. If you need all the control and extra features that the curl library provides you (and more), I'd recommend Apache's httpclient.

The Runtime object allows you to execute external command line applications from Java and would therefore allow you to use cURL however as the other answers indicate there is probably a better way to do what you are trying to do. If all you want to do is download a file the URL object will work great.

Using standard java libs, I suggest looking at the HttpUrlConnection class
http://java.sun.com/javase/6/docs/api/java/net/HttpURLConnection.html
It can handle most of what curl can do with setting up the connection.
What you do with the stream is up to you.

Curl is a non-java program and must be provided outside your Java program.
You can easily get much of the functionality using Jakarta Commons Net, unless there is some specific functionality like "resume transfer" you need (which is tedious to code on your own)

Use Runtime to call Curl. This code works for both Ubuntu and Windows.
String[] commands = new String {"curl", "-X", "GET", "http://checkip.amazonaws.com"};
Process process = Runtime.getRuntime().exec(commands);
BufferedReader reader = new BufferedReader(new
InputStreamReader(process.getInputStream()));
String line;
String response;
while ((line = reader.readLine()) != null) {
response.append(line);
}

Paste your curl command into curlconverter.com/java/ and it'll convert it into Java code using java.net.URL and java.net.URLConnection.