We Keep Coding

Is using url.openConnection() function safe in java?

I have a shortened URL. Now I am using HttpUrlConnection to open the connection with the shortened link.
URL url = new URL(myshortened url);
Now I open the connection by calling:
HttpURLConnection httpurlconnection = url.openConnection();
Finally I am extracting the location header containing the actual destination URL by calling:
String expandedurl = httpurlconnection.getHeaderField("Location");
At the end I disconnect the httpurlconnection by calling:
httpurlconnection.disconnect();
I want to know if the URL I have used is of a malicious website, can it cause any harm to the calling host? If yes, then what are the possible ways it can attack the calling host?
Edit: I have even disabled redirect by calling:
httpurlconnection.setInstanceFollowRedirects(false);

It depends on what you do with the result. For example if you use it to query a database, it could be vulnerable for SQL injection.

Taking text from a response web page using Java

I am sending commands to a server using http, and I currently need to parse a response that the server sends back (I am sending the command via the command line, and the servers response appears in my browser).
There are a lot of resources such as this: Saving a web page to a file in Java, that clearly illustrate how to scrape a page such as cnn.com. However, since this is a response page that is only generated when the camera receives a specific command, my attempts to use the method described by Mike Deck (in the link above) have met with failure. (Specifically, when my program requests the page again the server returns a 401 error.)
The response from the server opens a new tab in my browser. Essentially, I need to know how to save the current web page using java, since reading in a file is probably the most simple way to approach this. Do any of you know how to do this?
TL;DR How do you save the current webpage to a webpage.html or webpage.txt file using java?
EDIT: I used Base64 from the Apache commons codec, which solved my 401 authentication issue. However, I am still getting a 400 error when I attempt to connect my InputStream (see below). Does this mean a connection isn't being established in the first place?
URL url = new URL ("http://"+ipAddress+"/axis-cgi/record/record.cgi?diskid=SD_DISK");
byte[] encodedBytes = Base64.encodeBase64("root:pass".getBytes());
String encoding = new String (encodedBytes);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("POST");
connection.setDoInput (true);
connection.setRequestProperty ("Authorization", "Basic " + encoding);
connection.connect();
InputStream content = (InputStream)connection.getInputStream();
BufferedReader in = new BufferedReader (new InputStreamReader (content));
String line;
while ((line = in.readLine()) != null) {
System.out.println(line);
}
EDIT 2: Changing the request to a GET resolved the issue.

So while scrutinizing my code above, I decided to change
connection.setRequestMethod("POST");
to
connection.setRequestMethod("GET");
This solved my problem. In hindsight, I think the server was not recognizing the HTTP because it is not set up to handle the various trappings that come along with post.

Java set accept on http get

I am trying to send a request to a server using GET that will respond with XML. I am told that I need to set the "Accept" property, code follows:
StringBuffer url = new StringBuffer(BASE_URL);
url.append(DRS_SERVICE_RELATIVE_URL);
url.append("?").append(DOC_PARAM_NAME).append("=").append(docId);
url.append("&").append(DOB_PARAM_NAME).append("=").append(dob);
try
{
this.server = new URL(url.toString());
URLConnection urlCon = this.server.openConnection();
HttpURLConnection con = (HttpURLConnection)urlCon;
con.addRequestProperty("Accept", "text/xml, application/*+xml, application/xml, text/xml, application/*+xml");
con.connect();
input = new BufferedReader(new InputStreamReader(con.getInputStream()));
String line = null;
while((line = input.readLine()) != null)
System.out.println(line);
I get response code 500. When I talk to the developers of the URL I am trying to access they say I am not setting the "Accept" property to XML? What am I doing wrong? How are you supposed to set that property?
EDIT:
OK this is embarassing. The problem had to do with my development enviroment, specifically the way I set up a TCP/IP monitoring tool. When I stopped monitoring the network messages it worked as expected.

The problem had to do with my development enviroment, specifically the way I set up a TCP/IP monitoring tool. When I stopped monitoring the network messages it worked as expected.

HttpURLConnection returns 503 error when accessed through proxy

I am working on creating a Video sitemap for a site that has hosted videos on Brightcove video cloud. In order to get all the video information from the site, Brightcove suggests to read the response from their url of following form
http://api.brightcove.com/services/library?token="+accountToken+"&page_size=1&command=find_all_videos&output=JSON&get_item_count=true
the output of the url is in JSON, where accountToken is just an identifier of the account.
When I hit the above url with Token in the browser, it gives me the correct response.
I wrote below program snippet to read from that url
URL jsonURL = new URL("http://api.brightcove.com/services/library?token="+accountToken+"&page_size=1&command=find_all_videos&output=JSON&get_item_count=true");
HttpURLConnection connection = (HttpURLConnection) jsonURL.openConnection();
connection.setRequestMethod("GET");
connection.connect();
BufferedReader reader = new BufferedReader(
new InputStreamReader(connection.getInputStream()));
String lineRead = "";
while (reader.ready()) {
lineRead = lineRead + reader.readLine();
}
As my browser uses proxy, I added below code to include proxy settings
System.setProperty("http.proxyHost", "my.proxyurl.com");
System.setProperty("http.proxyPort", "80");
Without using proxy settings, it returns java.net.ConnectException: Connection refused: connect and with proxy it gives me java.io.IOException: Server returned HTTP response code: 503
So my question is , why is it giving me a 503(Service Unavailable) error ? From the browser its working fine.
Update 1:
It seems like an issue with the Network. I pinged the domain and it said "Request Timed out". Working via HTTP though. Looks like an issue with the Firewall.

I think, it may due to your internet connection, I have tried your code I didn't get any 503(Service Unavailable). Check out with different connection connection(without proxy) and it should work. Or you can try it with slightly different approach:
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("host", "port));
conn = new URL(jsonURL).openConnection(proxy);
If you have SOCKS type proxy, change Proxy's constructor parameter to Proxy.Type.SOCKS.

Minor correction to Jamas code
String host="myproxy.com";
int port=8080;
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(host, port));

Proxy With Java URLConnection class

I am very new with Java. I am using following code for the calling REST API, its working fine in simple environment but when I used with proxy environment Its throwing the NullPointerException. I found result on google that we have to set proxy setting for that. I set proxy according to that http://www.javaworld.com/javaworld/javatips/jw-javatip42.html article but this is not working + base64Encode( password ) creating syntax error.
URL url = new URL("http://examplerestapi/get/user");
URLConnection yc = url.openConnection();
in = new BufferedReader(new InputStreamReader(yc.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null) {
sb.append(inputLine);
}
String res = sb.toString();
please help me to set proxy Host, port , username and password.

I suspect your NullPointerException is occurring because yc.getInputStream() is returning null. You need to check that it is returning some non-null value before you attempt to create a reader to read bytes from it.
As for the proxy issue, you can pass a Proxy object to the connection, e.g.:
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("my.proxy.example.com", 3128));
URLConnection yc = url.openConnection(proxy);
This might at least allow you to interrogate the Proxy and rule out potential sources for the problem (there are several, as it stands).
This thread might have some useful hints for getting your proxy username and password string working properly. The article you linked looks slightly out of date.