We Keep Coding

Java URL Without Protocol

I'm trying to open an InputStream to a certain URL as given by the service's API. However, it does not have a set protocol (it's not http or https) and without one, I am getting the following error.
Is there any way to get a request without a protocol?
Exception:
Exception in thread "main" java.net.MalformedURLException: no protocol.
Code:
String url = "maple.fm/api/2/search?server=1";
InputStream is = new URL(url).openStream();
UPDATE: I now updated the code to:
Code:
String url = "http://maple.fm/api/2/search?server=1";
InputStream is = new URL(url).openStream();
and now I'm getting the following error:
Exception:
Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: http://maple.fm/api/2/search?server=1

A URL without a protocol is not a valid URL. It is actually a relative URI, and you can only use a relative URI if you have an absolute URI (or equivalent) to provide the context for resolving it.
Is there any way to [make] a request without a protocol?
Basically .... No. The protocol tells the client-side libraries how to perform the request. If there is no protocol, the libraries would not know what to do.
The reason that "urls without protocols" work in a web browser's URL bar is that the browser is being helpful, and filling in the missing protocol with "http:" ... on the assumption that that is what the user probably means. (Plus a whole bunch of other stuff, like adding "www.", adding ".com", escaping spaces and other illegal characters, ... or trying a search instead of a normal HTTP GET request.)
Now you could try to do the same stuff in your code before passing the URL string to the URL class. But IMO, the correct solution if you are writing code to talk to a service is to just fix the URL. Put the correct protocol on the front ...
The 403 error you are now getting means Forbidden. The server is saying "you are not permitted to do this".
Check the documentation for the service you are trying to use. (Perhaps you need to go through some kind of login procedure. Perhaps what you are trying to do is only permitted for certain users, or something.)
Try the example URL on this page ... which incidentally works for me from my web browser.

When you say it does not have a set protocol, I am a little bit suspicious of what that means. If it can use multiple protocols, I would hope the API documentation mentions some way of determining what the protocol should be.
I hit the URL http://maple.fm/api/2/search?server=1 and it is simply returning JSON over http. I think your actual problem is that you are trying to open an InputStream to talk to a web server. I believe the solution to your problem, of trying to handle JSON over http, can be found here.
I decided to dig into this because I was curious. Combining this answer and this answer, we have the following code which will print out the JSON output from your URL. Of course, you still need a JSON library to parse it, but that's a separate problem.
import java.net.*;
import java.io.*;
public class Main{
public static String getHTML(String urlToRead) {
URL url;
HttpURLConnection conn;
BufferedReader rd;
String line;
String result = "";
try {
url = new URL(urlToRead);
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.95 Safari/537.11");
rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
while ((line = rd.readLine()) != null) {
result += line;
}
rd.close();
} catch (IOException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
return result;
}
public static void main(String[] args) {
String url = "http://maple.fm/api/2/search?server=1";
System.out.println(getHTML(url));
}
}

you need to surround it with a try/catch block
try {
String url = "maple.fm/api/2/search?world=1";
InputStream is = new URL(url).openStream();
catch(MalformedURLException e) {
e.printStackTrace();

Java simple code: java.net.SocketException: Unexpected end of file from server

I wrote some simple code in Java, the method should connect to the website and return the BufferedReader.
private BufferedReader getConnection(String url_a) {
URL url;
try {
System.out.println("getting connection");
url = new URL(url_a);
HttpURLConnection urlConnection = (HttpURLConnection)
url.openConnection();
urlConnection.addRequestProperty("User-Agent",
"Mozilla/5.0 (X11; U; Linux i586; en-US; rv:1.7.3) Gecko/20040924"
+ "Epiphany/1.4.4 (Ubuntu)");
inStream = new InputStreamReader(urlConnection.getInputStream());
return new BufferedReader(inStream);
} catch (Exception ex) {
Logger.getLogger(Reader.class.getName()).log(Level.SEVERE, null, ex);
}
return null;
}
When I use it on my PC, it works fine but when I put .jar file on the server I get this error:
java.net.SocketException: Unexpected end of file from server
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:718)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:579)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:715)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:579)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1322)
at dataconverter.Reader.getConnection(Reader.java.260)
Problem is quite strange because the exception isn't thrown each time, sometimes everything is OK and program works fine.
Has anybody got any ideas?

"Unexpected end of file" implies that the remote server accepted and closed the connection without sending a response. It's possible that the remote system is too busy to handle the request, or that there's a network bug that randomly drops connections.
It's also possible there is a bug in the server: something in the request causes an internal error, and the server simply closes the connection instead of sending a HTTP error response like it should. Several people suggest this is caused by missing headers or invalid header values in the request.
With the information available it's impossible to say what's going wrong. If you have access to the servers in question you can use packet sniffing tools to find what exactly is sent and received, and look at logs to of the server process to see if there are any error messages.

Summary
This exception is encountered when you are expecting a response, but the socket has been abruptly closed.
Detailed Explanation
Java's HTTPClient, found here, throws a SocketException with message "Unexpected end of file from server" in a very specific circumstance.
After making a request, HTTPClient gets an InputStream tied to the socket associated with the request. It then polls that InputStream repeatedly until it either:
Finds the string "HTTP/1."
The end of the InputStream is reached before 8 characters are read
Finds a string other than "HTTP/1."
In case of number 2, HTTPClient will throw this SocketException if any of the following are true:
The HTTP method is CONNECT
The HTTP method is POST and the client is set to streaming mode
Why would this happen
This indicates that the TCP socket has been closed before the server was able to send a response. This could happen for any number of reasons, but some possibilities are:
Network connection was lost
The server decided to close the connection
Something in between the client and the server (nginx, router, etc) terminated the request
Note: When Nginx reloads its config, it forcefully closes any in-flight HTTP Keep-Alive connections (even POSTs), causing this exact error.

I do get this error when I do not set the Authentication header or I set wrong credentials.

I would suggest using wire shark to trace packets. If you are using Ubuntu, sudo-apt get wireshark. Like Joni stated the only way to figure out whats going wrong is to follow the GET requests and their associated responses.
http://www.wireshark.org/download.html

In my case it was solved just passing proxy to connection. Thanks to #Andreas Panagiotidis.
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("<YOUR.HOST>", 80)));
HttpsURLConnection con = (HttpsURLConnection) url.openConnection(proxy);

In my case url contained wrong chars like spaces . Overall log your url and in some cases use browser.

Most likely the headers you are setting is incorrect or not acceptable.
Example:
connnection.setRequestProperty("content-type", "application/json");

I got this exception too. MY error code is below
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod(requestMethod);
connection.setRequestProperty("Content-type", "JSON");
I found "Content-type" should not be "JSON",is wrong!
I solved this exception by update this line to below
connection.setRequestProperty("Content-type", "application/json");
you can check up your "Content-type"

Error 503 in HTTP during page parsing in java

Today I'm developing a java RMI server (and also the client) that gets info from a page and returns me what I want. I put the code right down here. The problem is that sometimes the url I pass to the method throws an IOException that says that the url given makes a 503 HTTP error. It could be easy if it was always that way but the thing is that it appears sometimes.
I have this method structure because the page I parse is from a weather company and I want info from many cities, not only for one, so some cities works perfectly at the first chance and others it fails. Any suggestions?
public ArrayList<Medidas> parse(String url){
medidas = new ArrayList<Medidas>();
int v=0;
String sourceLine;
String content = "";
try{
// The URL address of the page to open.
URL address = new URL(url);
// Open the address and create a BufferedReader with the source code.
InputStreamReader pageInput = new InputStreamReader(address.openStream());
BufferedReader source = new BufferedReader(pageInput);
// Append each new HTML line into one string. Add a tab character.
while ((sourceLine = source.readLine()) != null){
if(sourceLine.contains("<tbody>")) v=1;
else if (sourceLine.contains("</tbody>"))
break;
else if(v==1)
content += sourceLine + "\n";
}
........................
........................ NOW THE PARSING CODE, NOT IMPORTANT
}

HTTP 500 errors reflect server errors so it has likely nothing to do with your client code.
You would get a 400 error if you were passing invalid parameters on your request.
503 is "Service Unavailable" and may be sent by the server when it is overloaded and cannot process your request. From a publicly accessible server, that could explain the erratic behavior.
Edit
Build a retry handler in your code when you detect a 503. Apache HTTPClient can do that automatically for you.
List of HTTP Status Codes

Check that the IOException is really not a MalformedURLException. Try printing out the URLs to verify a bad URL is not causing the IOException.
How large is the file you are parsing? Perhaps your JVM is running out of memory.

Webservice call returns error 500

I have started a small project in Java.
I have to create a client which will send xml to a url as a HTTP POST request.
I try it using java.net.* package (Following is the piece of code) but I am getting error as follows:
java.io.IOException: Server returned HTTP response code: 500 for URL: "target url"
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at newExample.main(newExample.java:36)
My code is as follows:
try {
URL url = new URL("target url");
URLConnection connection = url.openConnection();
if( connection instanceof HttpURLConnection )
((HttpURLConnection)connection).setRequestMethod("POST");
connection.setRequestProperty("Content-Length", Integer.toString(requestXml.length()) );
connection.setRequestProperty("Content-Type","text/xml; charset:ISO-8859-1;");
connection.setDoOutput(true);
connection.connect();
// Create a writer to the url
PrintWriter writer = new PrintWriter(new
OutputStreamWriter(connection.getOutputStream()));
// Get a reader from the url
BufferedReader reader = new BufferedReader(new
InputStreamReader(connection.getInputStream()));
writer.println();
writer.println(requestXml);
writer.println();
writer.flush();
String line = reader.readLine();
while( line != null ) {
System.out.println( line );
line = reader.readLine();
}
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Please help with suitable examples or any other ways of doing this.
Point errors/mistakes in above code or other possibilities.
My Web Service is in spring framework
xml to send is in the string format: requestXml

The problem lies in below code
// Get a reader from the url
BufferedReader reader = new BufferedReader(new
InputStreamReader(connection.getInputStream()));
As the service might not always return you the proper response... as you are calling a service through http, it can be possible that the server itself is not available or the service is not available. So you should always check for the response code before reading response from streams, based on the response code you've to decide whether to read it from inputStream for success response or from errorStream for failure or exception condition.
BufferedReader reader = null;
if(connection.getResponseCode() == 200)
{
reader = new BufferedReader(new
InputStreamReader(connection.getInputStream()));
}
else
{
reader = new BufferedReader(new
InputStreamReader(connection.getErrorStream()));
}
This would resolve the problem

The problem is inside your server code or the server configuration:
10.5.1 500 Internal Server Error
The server encountered an unexpected condition which prevented it from fulfilling the request.
(w3c.org/Protocols)
If the server is under your control (should be, if I look at the URL [before the edit]), then have a look at the server logs.

Well, you should close your streams and connections. Automatic resource maangement from Java 7 or http://projectlombok.org/ can help. However, this is probably not the main problem.
The main problem is that the server-side fails. HTTP code 500 means server-side error. I can't tell you the reason, because I don't know the server side part. Maybe you should look at the log of the server.

I think that your problem is that you are opening the input stream before you have written and closed the output stream. Certainly, the Sun Tutorial does it that way.
If you open the input stream too soon, it is possible that the output stream will be closed automatically, causing the server to see an empty POST request. This could be sufficient to cause it to get confused and send a 500 response.
Even if this is not what is causing the 500 errors, it is a good idea to do things in the order set out in the tutorial. For a start, if you accidentally read the response before you've finished writing the request, you are likely to (at least temporarily) lock up the connection. (In fact, it looks like your code is doing this because you are not closing the writer before reading from the reader.)
A separate issue is that your code does not close the connection in all circumstances, and is therefore liable to leak network connections. If it does this repeatedly, it is likely to lead to more IOExceptions.

If you are calling an External Webservice and passing a JSON in the REST call, check the datatype of the values passed.
Example:
{ "originalReference":"8535064088443985",
"modificationAmount":
{ "amount":"16.0",
"currency":"AUD"
},
"reference":"20170928113425183949",
"merchantAccount":"MOM1"
}
In this example, the value of amount was sent as a string and the webservice call failed with Server returned HTTP response code: 500.
But when the amount: 16.0 was sent, i.e an Integer was passed, the call went through. Though you have referred API documentation while calling such external APIs, small details like this could be missed.

Can't download with HttpClient with a different server port

I am using Apache's HttpClient via httpclient-fluent-builder to download a http page.
This is the code:
...
try {
response = Http.get("http://fr3.ah.fm:9000/played.html")
.use(client) // use this HttpClient (required)
.charset("windows-1252") // set the encoding... (optional)
.asString();
} catch (IOException e) {
Log.d(TAG, "Error: "+e.toString());
e.printStackTrace();
}
Log.i(TAG, ""+ response );
...
Problem is that I get org.apache.http.client.ClientProtocolException
It's something with the host:port/url, beacause it works with urls without ports. I also get this same error with another Httphelper class than fluent-builder. Firewall is off.
Logcat: http://pastebin.com/yMMvvdQ3

Found out what it was via this post that it was the Shoutcast server...
You should be able to connect to 8000
with your web browser and get the DNAS
status page. If, on the other hand,
you connect to that port with a media
player, it'll return the direct MP3
stream. (Unfortunately, in an
incredibly boneheaded piece of design,
the way SHOUTcast decides which to
respond with is by sniffing your
User-Agent header for something
beginning with Mozilla, so if you're
using an alternative browser or
blocking your UA you'll not be able to
get the status, and if the stream's
down you might just get nothing.)
Drove me crazy.. But it's an easy fix. I just added.
.header("User-Agent", "UserAgent: Mozilla/5.0")

try to set the uri using HttpHost and use it in HttpClient.execute().
I have not tried it..