I have an android app and REST server made in Spring Tool Suite. REST is connecting to database and returns list of Json objects depending on the query in the GET request.
This is my REST response to GET method
#RequestMapping(value = "/short", method = RequestMethod.GET)
public Map<String,List<Temperature>> shortPeriod(#RequestParam(value = "time",required = false)
#DateTimeFormat(pattern="yyyy:MM:dd HH:mm:ss") Date time){
Map<String, List<Temperature>> results = new HashMap<String, List<Temperature>>();
List<Temperature> temperatureList = new ArrayList<Temperature>();
System.out.println("TEST!");
try {
temperatureList = dataServices.getEntityList(time,"short");
results.put("data", temperatureList);
} catch (Exception e) {
e.printStackTrace();
}
return results;
}
and this is how i connect on android
String tempUrl = new String("http://" + IP + ":8080/" + "RestServer/temperature/" + tableName + "?time=" + urlQueryTimestamp);
String encodedUrlTemp = tempUrl.replace(" ", "%20");
String encodedUrl = encodedUrlTemp.replace("-", ":");
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setDoInput(true);
connection.setDoOutput(true);
connection.connect();
I have put test prints and It's working ok till connection.connect();
The url looks something like this
http://192.168.1.106:8080/RestServer/temperature/short?time=2015:06:05%2010:20:03
I'm printing "url" variable before connecting and when I take that url and paste it into chrome I get normal response as I should. But when I'm trying to connect in my app I get "method not allowed". I have found some solutions about that exception and it's usually problem with POST method not implemented on server, but with same url i get normal response on chrome on my computer and even on my mobile chrome on smartphone.
Other thing is that I have one other server with hardcoded json data for testing and i can connect normally there when there is no query in http request, so only this
http://192.168.1.106:8080/RestServer/temperature/short
So my assumption is that chrome escapes some strings internally or sets up some header infromation by itself when using a query, and android doesn't. Server doesn't even records request as one I pasted in code above, it's like im sending wrong url.
One other thing, I tried using url encoder from android library but it escapes some letters that shouldn't be escaped and I read that it escapes it for html, so I'm doing it manually(maybe I'm doing it wrong).
I turned off the firewall, checked ip adress dozen of times and I have no ideas more.
After trying absolutely every solution I found with escapeing, changing library's and reading about URI's and URL's, I have found the solution on android developer documentation page where it says:
HttpURLConnection uses the GET method by default. It will use POST if
setDoOutput(true) has been called. Other HTTP methods (OPTIONS, HEAD,
PUT, DELETE and TRACE) can be used with setRequestMethod(String).
Hope this helps to someone
Related
I'm trying to send an http request to bing's spell checking api using a GET request. I checked my parameters and headers on https://www.hurl.it/ and it returned a proper json with the spelling errors properly, however when I send the request from my java app it returns this json with NO spelling errors detected (therefore, text parameter HAS to be empty somehow). I'm definitely passing the correct key in the header because that part isn't sending an error and the code is 200 (success).
My string: "my funger is harting me"
My code returned:
{"_type":"SpellCheck","flaggedTokens":[]}
Hurl.it returned:
{
"_type":"SpellCheck",
"flaggedTokens":[
{
"offset":3,
"token":"funger",
"type":"UnknownToken",
"suggestions":[
{
"suggestion":"finger",
"score":0.903614003311793
}
]
},
{
"offset":13,
"token":"harting",
"type":"UnknownToken",
"suggestions":[
{
"suggestion":"hurting",
"score":0.903614003311793
}
]
}
]
}
This is my java code using Apache's HTTPClient library:
(note: "command.getAfter()" is the passed string I mentioned above. I debugged it and even hard coded a string to test it out. Same output obviously.)
HttpClient httpclient = HttpClients.createDefault();
try {
URIBuilder builder = new URIBuilder("https://api.cognitive.microsoft.com/bing/v5.0/spellcheck/");
builder.setParameter("text", command.getAfter());
URI uri = builder.build();
HttpGet request = new HttpGet(uri);
request.setHeader("Ocp-Apim-Subscription-Key", "XXXXXXXX");
HttpResponse response = httpclient.execute(request);
HttpEntity entity = response.getEntity();
if (entity != null) {
System.out.println(EntityUtils.toString(entity));
}
} catch (Exception e) {
System.out.println(e.getMessage());
}
EDIT: It turns out the URI returned in the request object returns this:
https://api.cognitive.microsoft.com/bing/v5.0/spellcheck/?text=my+funger+is+harting+me
So the parameter is not empty? But when fed no text parameter in hurl.it, the api returns an error of no parameters. When the text parameter is a space " ", it returns an identical result to mine. Unsure what this means since the URI seems to be valid and not empty and my subscription key is working because i would get an error if it weren't...
EDIT: I'm starting to suspect the Apache library is ignoring the parameters I'm passing in HttpGet(uri). I'm unsure, but I'm going to try a different solution to send the request with a header and see what happens.
EDIT: I tried the following code below:
String url = "https://api.cognitive.microsoft.com/bing/v5.0/spellcheck/?text=" + command.getAfter().replace(" ", "+");
try {
URL request_url = new URL(url);
//URIBuilder uri = new URIBuilder("https://api.cognitive.microsoft.com/bing/v5.0/spellcheck/");
//uri.setParameter("text", command.getAfter());
HttpURLConnection con = (HttpURLConnection) request_url.openConnection();
con.setRequestMethod("GET");
con.setRequestProperty("Ocp-Apim-Subscription-Key", Keys.BING_SPELL_CHECK_API);
con.setConnectTimeout(100000);
con.setReadTimeout(100000);
con.setInstanceFollowRedirects(true);
String theString = IOUtils.toString(con.getInputStream(), "UTF-8");
System.out.println(theString);
} catch (IOException e) {
e.printStackTrace();
}
It returned the same result as the Apache one... :/ What else should I try?
EDIT:
This is the output of the request as well:
https://api.cognitive.microsoft.com/bing/v5.0/spellcheck/?text=my+funger+is+hartingme - [Ocp-Apim-Subscription-Key: <XXXXXXXXXXXX>]
HTTP/1.1 200 OK - en_US
{"_type": "SpellCheck", "flaggedTokens": []}
I don't get it.... Why is the json outputted empty when hurl.it returns the correct json for this same request? Is this a java issue or something?
EDIT:
I just tried UniRest's api. Exact same result... What am I doing wrong here?!
I'm so lost...
Separate Issue:
I do want to note the following: When I set the bing api's version to 7.0, I get the following error:
Received http status code 401 with message Access Denied and body {"message":"Access denied due to invalid subscription key. Make sure to provide a valid key for an active subscription.","statusCode":401}
This is not the case with v5.0. I'm getting the correct key from my Azure portal. (The page called Keys and lists 2 keys you can use and regenerate)
Answer to getting v7.0 to work:
This is not the case with v5.0. I'm getting the correct key from my Azure portal. (The page called Keys and lists 2 keys you can use and regenerate)
You get 2 keys per version. So if you are seeing 2 keys, they are likely both for v5.0. It should explicitly mention v7.0.
There should be different sections, also with different endpoints.
Use these in combination with each other to get the desired result.
i'm running a simple java program to get HttpResponse codes, however for some reason not all codes happen to be 200. I find this odd because when checking the network tab for certain URLs like www.reddit.com, the Response is 200, but my program is returning a different value.
The code below...
try{
String urlName = "http://www.reddit.com";
URL url = new URL(urlName);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.connect();
String message = connection.getResponseMessage();
System.out.println("Message: " + message);
int code = connection.getResponseCode();
System.out.println(Integer.toString(code));
}
catch(Exception e){
e.printStackTrace();
}
Lastly, is there a reason to set the RequestMethod to GET and connection again? I get the response code whether or not I have that code because the connection executes openConnection();
Goal - make all valid connections return 200
You said that you're seeing a 301 for Reddit and a 302 for Facebook. Those status codes mean that you're getting redirected. Your browser's following them; your code isn't.
Java's built-in HTTP support is not great for end-users. I strongly recommend using a better HTTP client library, such as Apache's HttpClient, or Horizon, which is built on top of Apache (for synchronous requests) and Ning (for async).
Full disclosure: I work for HubSpot; Horizon is one of our open-source libraries.
It would be nice if you posted the error code it did give...
I ran your code myself and the error was 301, meaning moved permanently.
If you go to http://www.reddit.com yourself, you will see that you get redirected to the httpS version of reddit. Changing this in the urlName will fix your problem.
Edit: same goes for facebook as i saw in comments to your question, google does not require https always so that does work.
How can I get the redirected URL of http://at.atwola.com/?adlink/5113/1649059/0/2018/AdId=4041444;BnId=872;itime=15692006;impref=13880156912668385284; in Java?
My code (given below) is constructed according to answers to similar questions on stack-overflow (https://stackoverflow.com/a/5270162/1382251 in particular).
But it just yields the original URL. I suspect that there are other similar cases, so I would like to resolve this one in specific and use the solution in general.
String ref = "http://at.atwola.com/?adlink/5113/1649059/0/2018/AdId=4041444;BnId=872;itime=15692006;impref=13880156912668385284;";
try
{
URLConnection con1 = new URL(ref).openConnection();
con1.connect();
InputStream is = con1.getInputStream();
URL url = con1.getURL();
is.close();
String finalPage = url.toString();
if (finalPage.equals(ref))
{
HttpURLConnection con2 = (HttpURLConnection)con1;
con2.setInstanceFollowRedirects(false);
con2.connect();
if (con2.getResponseCode()/100 == 3)
finalPage = con2.getHeaderField("Location");
}
System.out.println(finalPage);
}
catch (Exception error)
{
System.out.println("error");
}
I played a bit with your URL with telnet, wget, and curl and I noticed that in some cases the server returns response 200 OK, and sometimes 302 Moved Temporarily. The main difference seems to be the request User-agent header. Your code works if you add the following before con1.connect():
con1.setRequestProperty("User-Agent","");
That is, with empty User-Agent (or if the header is not present at all), the server issues a redirect. With the Java User-Agent (in my case User-Agent: Java/1.7.0_45) and with the default curl User-Agent (User-Agent: curl/7.32.0) the server responds with 200 OK.
In some cases you might need to also set:
System.setProperty("http.agent", "");
See Setting user agent of a java URLConnection
The server running the site is the Adtech Adserver and apparently it is doing user agent sniffing. There is a long history of user agent sniffing. So it seems that the safest thing to do would be to set the user agent to Mozilla:
con1.setRequestProperty("User-Agent","Mozilla"); //works with your code for your URL
Maybe the safest option would be to use a user agent used by some of the popular web browsers.
I have the following code, it works totally fine on my local development server, but when I uploaded to the deployment server, I always hit file not found Exception
String urlStr = "http://" + getContext().getRequest().getServerName() +
getContext().getServletContext().getContextPath() + "test.action";
URL url = new URL(urlStr);
InputStream input = url.openStream(); //Error always occurs here, it gives me the correct URL but it says file not found.
Can anyone help me with this?
Because its a HTTP URL the correct way would be as follows.
String urlStr = "http://" + getContext().getRequest().getServerName() +
getContext().getServletContext().getContextPath() + "test.action";
URL url = new URL(urlStr);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
if (conn.getResponseCode() == HttpURLConnection.HTTP_ACCEPTED) {
InputStream input = conn.getInputStream();
}
I think that #deadlock's comments is probably the key to solving this.
You are getting a FileNotFoundException because the remote server is sending a 404 Not Found response. The most likely explanation is that you are attempting to connect using the wrong URL. Print out the URL string before you try to connect.
All the evidence is pointing to the fact that the server is sending "404 Not Found" responses ... for both versions of the code. This normally means that your URL is wrong. But it is also possible for it to be other things:
You may be using different proxies in the Java and browser cases, resulting in the Java case reaching some server that doesn't understand the URL.
It is conceivable that the server is implementing some anti web scraping mechanism, and sending you 404 responses `cos this thinks (rightly) that your requests aren't coming from a web browser,
I am developing a very simple http bot. I am using the javax.net.ssl.HttpsURLConnection class and I have to make multiple requests.
Snippet of code :
HttpURLConnection urlConnection =
(HttpURLConnection) new URL(url+"?"+firstParameters).openConnection();
urlConnection.setRequestProperty("Accept-Charset", "UTF-8");
headerFields = urlConnection.getHeaderFields();
keys = headerFields.keySet();
for(String key : keys){
if(key != null && key.contains("ookie")){
cookies = urlConnection.getHeaderField(key);
break;
}
}
for(String cookie : cookies.split(";")){
if(cookie.contains("JSESSION")){
JSESSION = cookie.split("=")[1];
break;
}
}
document = new InputSource(urlConnection.getInputStream());
parser.setDocument(document);
attributesId.put("name",new ArrayList<String>(Arrays.asList(attributesNames)));
elementsIds.put("INPUT",attributesId);
elements = parser.getValues(elementsIds);
for(String attr : attributesNames){
secondParameters = secondParameters.replaceAll("#r"+index,elements.get(attr));
}
urlConnection.getInputStream().close();
//Second call
urlConnection = (HttpURLConnection) new URL(url2).openConnection();
urlConnection.setRequestMethod("POST");
urlConnection.setRequestProperty("Cookie", "JSESSIONID="+JSESSION);
urlConnection.setDoInput(true);
urlConnection.setDoOutput(true);
payload = new PrintWriter(urlConnection.getOutputStream());
payload.print(secondParameters);
payload.flush();
payload.close();
Summarizing the code above, first i do a request without any payload and i am able to see the correct response from the server, but the problem is when i make the second request (now with payload and with the JSESSION cookie), what i receive it his the same response that i received in the first request, it looks like i am making the first request again.
So my question is , what i am doing wrong ?
I just need to open one connection, and then change the headers and payload ?
There is any tutorial related with multiple http requests(with mixed methods , post and get)?
Thanks in advance
I've never used HttpURLConnection before. I usually use Apache's HTTPClient code. There are a lot of docs and tutorials about it on their home page.
Couple of things that I noticed about your code:
You code does not handle multiple Cookie headers on the response. Mine seems to handle that better.
Are you sure that all you need is JSESSION? Maybe there are other cookies you are missing?
Have you debugged your code to make sure that your JSESSION cookie gets set appropriately? I added some trim() calls in my cookie processing code to make sure some spaces didn't slip in there.
I can't see the real value of your secondParameters. I have no idea if they are valid. Have you debugged your code to verify the secondParamters value looks good. You can see in my code what I'm posting to the server. Btw, I'd use a StringBuilder instead of + to build them.
Hope this helps.