We Keep Coding

How to make REST API calls that have filter added

I am new to REST API and I want to make a REST API call which returns a JSON object
http://smlookup.dev/sec/products?search={"ABC.CP":"123A5"} - Runs fine in a browser and gives a JSON object
how do i get '?search={"ABC.CP":"12345"}' this expression to work as it filter the records
Code i am using is
String serverUrl="http://smlookup.dev/sec/products?search=";
String search=URLEncoder.encode("={\"ABC.CP\":\"12345\"}");
URL url = new URL(serverUrl+search);
HttpURLConnection httpCon = (HttpURLConnection) url.openConnection();
httpCon.setDoOutput(true);
httpCon.setRequestMethod("GET");
OutputStream out = httpCon.getOutputStream();
//FAILS GIVING 405 STATUS CODE
int responseCode = httpCon.getResponseCode();
All help or suggestions are helpful
Thanks!

Not sure if its normal but you dont send any data in your POST.
Furthermore you should urlencode your url, the inverted comma are not accepted like that.
URLEncoder.encode("={\"Xref.CP\":\"XAW3123A5\"}");

Retrieve the final location of a given URL in Java

I am trying to retrieve the final location of a given URL (String ref) as follows:
HttpURLConnection con = (HttpURLConnection)new URL(ref).openConnection();
con.setInstanceFollowRedirects(true);
con.setRequestProperty("User-Agent","");
int responseCode = con.getResponseCode();
return con.getURL().toString();
It works in most cases, but rarely returns a URL which yet contains another redirection.
What am I doing wrong here?
Why do I get responseCode = 3xx, even after calling setInstanceFollowRedirects(true)?
UPDATE:
OK, responseCode can sometimes be 3xx.
If it happens, then I will return con.getHeaderField("Location") instead.
The code now is:
HttpURLConnection con = (HttpURLConnection)new URL(ref).openConnection();
con.setInstanceFollowRedirects(true);
con.setRequestProperty("User-Agent","");
int responseType = con.getResponseCode()/100;
while (responseType == 1)
{
Thread.sleep(10);
responseType = con.getResponseCode()/100;
}
if (responseType == 3)
return con.getHeaderField("Location");
return con.getURL().toString();
Will appreciate comment should anyone see anything wrong with the code above.
UPDATE
Removed the handling of code 1xx, as according to most commenters it is not necessary.
Testing if the Location header exists before returning it, in order to handle code 304.
HttpURLConnection con = (HttpURLConnection)new URL(ref).openConnection();
con.setInstanceFollowRedirects(true);
con.setRequestProperty("User-Agent","");
if (con.getResponseCode()/100 == 3)
{
String target = con.getHeaderField("Location");
if (target != null)
return target;
}
return con.getURL().toString();

HttpURLConnection will not follow redirects if the protocol changes, such as http to https or https to http. In that case, it will return the 3xx code and you should be able to get the Location header. You may need to open a connection again in case that new url also redirects. So basically, use a loop and break it when you get a non-redirect response code. Also, watch out for infinite redirect loops, you could set a limit for the number of iterations or check if each new url has been visited already.

If you just want the redirect url, the response header should give you that:
if (con.getResponseCode() == 301) {
String redirectUrl = con.getHeaderField("Location");
}

There probably can easily be multiple levels of redirection - imagine a bit.ly pointing to a youtu.be address pointing to youtube.com. Perhaps you need to loop until you get your 200 OK or until you hit a redirection cycle.
I have trouble locating the source code to check but I believe what I said is true. See e.g. java urlconnection get the final redirected URL
You also might need to handle protocol redirects, e.g. HTTP -> HTTPS: URLConnection Doesn't Follow Redirect

I think I now understand what you want. I now think that you are trying to retrieve the final address, not the content of the final address. Please correct me if my assumption is wrong.
For doing this (not the content, but the address), you need a different approach. You need to switch off follow-redirects and you then need to handle the iterational redirect-following on your own until you find a non-redirecting response. Bear in mind that you can not reuse a URLConnection.
The approaches for finding the final address and the other approach for retrieving the content of the final address are so different, because URLConnection does not reveal the followed-to address if you switch on follow-redirects.
In your code, you seem to expect URLConnection.getURL() to return the followed-to address. This is not the behavior of this method. It returns the original URL which you used to create the URLConnection. It does this no matter if you switch on follow-redirects or not.
However, if you switch it on, you will not be able to get the followed-to URL address. This is because getHeaderField("Location"), with follow-redirects, makes no sense: it returns the redirection-target of the final redirect, which should not exist, since it's the final address.

Sometime it is loading in the field of requestURI. Use like this code:
val declaredField = con.javaClass.getDeclaredField("requestURI")
declaredField.isAccessible=true
val loc = declaredField.get(con).toString()

Getting an ajax response in java from a web method (java.io.FileNotFoundException)

Don't know how to explain it better but i'm trying to get a response from an URL containing a function (right?).
I'm working on this for a lot of hours and progressing a little every time but can't get this finally working.
This is the request and response headers from chrome dev tools:
Headers
My code is:
String params = "{\"prefixText\":\"" + city
+ "\",\"count\":10,\"contextKey\":\"he\"}";
conn = (HttpURLConnection) new URL(
"http://bus.gov.il/WebForms/wfrmMain.aspx/GetCompletionList")
.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("POST");
conn.setChunkedStreamingMode(0);
// conn.setFixedLengthStreamingMode(params.length());
conn.addRequestProperty("Accept", "*/*");
conn.addRequestProperty("Content-Type", "application/json; charset=UTF-8");
conn.addRequestProperty("Content-Length", String.valueOf(params.length()));
conn.addRequestProperty("Host", "bus.gov.il");
conn.addRequestProperty("Origin", "http://bus.gov.il");
conn.addRequestProperty("X-Requested-With", "XMLHttpRequest");
conn.addRequestProperty("Referer",
"http://bus.gov.il/WebForms/wfrmMain.aspx?width=1024&company=1&language=he&state=");
OutputStream os = new BufferedOutputStream(conn.getOutputStream());
os.write(params.getBytes());
String answer = readStream(conn.getInputStream());
I get the exception (I see in the stack trace) when calling "getinputstream" on this line:
String answer = readStream(conn.getInputStream());
before entering the readStream function!
I don't know how to solve it...
Tried searching about xmlhttprequest but understood that it's only in JS.
Also: I know I have a lot of unnecessary request properties but I can't figure out which are unnecessary until the code will work.
Thanks in advance :)

Sadly, it used to be (and probably still is) that the HttpURLConnection throws a FileNotFoundException when you get a 404 error. When you are doing the getInputStream() that's when it's first connecting, so any error from the server will show up there.
Get Wireshark or something if you want to see what's really going on in HTTP land as you make the request.

Multiple Http requests and responses using HttpURLConnection

I am developing a very simple http bot. I am using the javax.net.ssl.HttpsURLConnection class and I have to make multiple requests.
Snippet of code :
HttpURLConnection urlConnection =
(HttpURLConnection) new URL(url+"?"+firstParameters).openConnection();
urlConnection.setRequestProperty("Accept-Charset", "UTF-8");
headerFields = urlConnection.getHeaderFields();
keys = headerFields.keySet();
for(String key : keys){
if(key != null && key.contains("ookie")){
cookies = urlConnection.getHeaderField(key);
break;
}
}
for(String cookie : cookies.split(";")){
if(cookie.contains("JSESSION")){
JSESSION = cookie.split("=")[1];
break;
}
}
document = new InputSource(urlConnection.getInputStream());
parser.setDocument(document);
attributesId.put("name",new ArrayList<String>(Arrays.asList(attributesNames)));
elementsIds.put("INPUT",attributesId);
elements = parser.getValues(elementsIds);
for(String attr : attributesNames){
secondParameters = secondParameters.replaceAll("#r"+index,elements.get(attr));
}
urlConnection.getInputStream().close();
//Second call
urlConnection = (HttpURLConnection) new URL(url2).openConnection();
urlConnection.setRequestMethod("POST");
urlConnection.setRequestProperty("Cookie", "JSESSIONID="+JSESSION);
urlConnection.setDoInput(true);
urlConnection.setDoOutput(true);
payload = new PrintWriter(urlConnection.getOutputStream());
payload.print(secondParameters);
payload.flush();
payload.close();
Summarizing the code above, first i do a request without any payload and i am able to see the correct response from the server, but the problem is when i make the second request (now with payload and with the JSESSION cookie), what i receive it his the same response that i received in the first request, it looks like i am making the first request again.
So my question is , what i am doing wrong ?
I just need to open one connection, and then change the headers and payload ?
There is any tutorial related with multiple http requests(with mixed methods , post and get)?
Thanks in advance

I've never used HttpURLConnection before. I usually use Apache's HTTPClient code. There are a lot of docs and tutorials about it on their home page.
Couple of things that I noticed about your code:
You code does not handle multiple Cookie headers on the response. Mine seems to handle that better.
Are you sure that all you need is JSESSION? Maybe there are other cookies you are missing?
Have you debugged your code to make sure that your JSESSION cookie gets set appropriately? I added some trim() calls in my cookie processing code to make sure some spaces didn't slip in there.
I can't see the real value of your secondParameters. I have no idea if they are valid. Have you debugged your code to verify the secondParamters value looks good. You can see in my code what I'm posting to the server. Btw, I'd use a StringBuilder instead of + to build them.
Hope this helps.

Better way to send lots of text to a server?

I have a java application that sends text to a sql database on a server. Currently my java application takes the text, puts it into the url, then sends it to a php page on the server that takes it with GET and puts it in the database. that works fine to an extent, the problem is, that i need to be able to send lots of text, and i keep getting 414, uri to long errors. is there a better way to do this?
ok, i tried what you said, and read the tutorial, but something is not working. here is my code that i tried
public void submitText(String urls,String data) throws IOException{
URL url = new URL(urls);
URLConnection con = url.openConnection();
con.setRequestProperty("Content-Type", "text/plain; charset=utf-8");
con.setDoOutput(true);
OutputStreamWriter out = new OutputStreamWriter(con.getOutputStream(), "UTF-8");
out.write(data);
out.flush();
}
submitText(server + "publicPB.php", "param=" + text);
here is my php code
$param = $_POST['param'];
$sql = "UPDATE table SET cell='{$param}' WHERE 1";
mysql_query($sql);
...
im pretty sure its not a problem with the php as the php worked fine with GET, and thats all i change with it, my problem i think is that im not 100% sure how to send data to it with the java

Use a POST instead of a GET and send the text as the request body. You can only pass so much data to a URL. E.g.:
// Assuming 'input' is a String and contains your text
URL url = new URL("http://hostname/path");
URLConnection con = url.openConnection();
con.setRequestProperty("Content-Type", "text/plain; charset=utf-8");
con.setDoOutput(true);
OutputStreamWriter out = new OutputStreamWriter(con.getOutputStream(), "UTF-8");
out.write(input);
out.close();
See Reading from and Writing to a URLConnection for more details.

Why don't you use POST to send data across to PHP page? GET does have a smaller limit of content.

Use POST requests, which do not have content length limits.

POST requests do not have length content limits and are much secure than GET requests ;)

If using SQL Server I would look into leveraging BCP. You can write the file and call BCP from within Java, and it will send the information directly to your database.