How do I do an HTTP PUT request using the UrlFetch service for Google App Engine's Java runtime?
The following code in my application will send a POST request:
URL url = ...;
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setReadTimeout(MAX_TIMEOUT); // Maximum allowable timeout on GAE of 60 seconds
conn.setRequestMethod("POST");
conn.setDoOutput(true);
conn.setRequestProperty("Content-Type", "application/json; charset=utf-8");
OutputStreamWriter writer = null;
try {
writer = new OutputStreamWriter(conn.getOutputStream(), Charset.forName("utf-8"));
writer.write(jsonContent);
} finally {
if (writer != null)
writer.close();
}
}
InputStream inStream = null;
E response = null; // Unmarshalled using Jackson
try {
inStream = conn.getInputStream();
if (status != 204) { // NO CONTENT => No need to use Jackson to deserialize!
response = mapper.readValue(inStream, typeRef);
}
} finally {
if (inStream != null) {
inStream.close();
}
}
I try doing the same as POST for PUT, but I keep on getting a 405 HTTP error for Method Not Allowed. Thanks for the help.
Reference: https://developers.google.com/appengine/docs/java/urlfetch/overview
Unfortunately, GAE Java doesn't have as good of documentation as Python's version of GAE URL fetch(https://developers.google.com/appengine/docs/python/urlfetch/overview). Anyone from Google wanna say why?
I think your code works. The HTTP 405 you are seeing is coming from the server you're contacting (not all services support PUT). You can try using curl to hit the URL if you'd like confirmation that it's not your code at fault:
curl -X PUT -d "foo=bar" http://your_site.com/path/to/your/resource
Related
I have been trying to upload large files by generating a signed URL. Here is the documentation I was following to generate the signed URL: https://cloud.google.com/storage/docs/access-control/signing-urls-with-helpers#code-samples
It was working fine for files in 100's of MB's but as soon as the file size went up to 1 GB, the curl command started timing out even after increasing the expiration time. I tried looking at the answer here: https://stackoverflow.com/a/63789297/7466551, but I am still unable to get the URL working to upload the URL.
I am using this command to upload the file:
curl -X POST -H 'x-goog-resumable: start' --upload-file file-name 'pre_signed_google_url'. I am adding the 'x-goog-resumable: start' header as I am having "x-goog-resumable", "start" header as a part of my code to generate the URL.
Can someone please let me know if I need to do any additional thing to generate the URL to upload large files?
Answering my own question as I had to use two separate sources to reach a solution.
On top of the java code here: https://stackoverflow.com/a/63789297/7466551, I referred to the medium article here: https://medium.com/google-cloud/google-cloud-storage-signedurl-resumable-upload-with-curl-74f99e41f0a2
So you need the following additional lines of code on top of the StackOverflow answer to get a signed URL for resumable uploads:
// Open a HTTP connection and add header
URL obj = new URL(url.toString());
HttpURLConnection connection = (HttpURLConnection) obj.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("x-goog-resumable", "start");
connection.setDoOutput(true);
// Connect to the URL and post headers
DataOutputStream writer = new DataOutputStream(
connection.getOutputStream());
writer.writeBytes("");
writer.flush();
writer.close();
// Checking the responseCode to
if (connection.getResponseCode() == connection.HTTP_CREATED) {
connection.disconnect();
System.out.println("Location: " + connection.getHeaderField("Location"));
}
else {
// Throw error
BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = reader.readLine()) != null) {
response.append(inputLine);
}
reader.close();
String errorMessage = response.toString();
connection.disconnect();
throw new IOException(errorMessage);
}
I added an open extension to an event in a calendar and am trying to read it back.
Here is the url:
https://graph.microsoft.com/v1.0/users/{userid}/calendars/{calendarId}=/events?$expand=Extensions($filter=Id eq 'c.i.m.p.server.entities.outlook.Event')
I cannot get this to work in a Java program. The following combinations do work:
It works my Java program if I remove the $expand... parameter. I can also ask for certain fields, that works too.
The request works in Postman (I just have to set the token)
The request works in Graph Explorer when I log in as the owner of the calendar
Here is the extension (inside one of the events) when I use Postman to read the event. It is the last item in the event:
"extensions#odata.context": "https://graph.microsoft.com/v1.0/$metadata#users('{userid}')/calendars('{calendarId}')/events('{eventId})/extensions",
"extensions": [
{
"#odata.type": "#microsoft.graph.openTypeExtension",
"id": "Microsoft.OutlookServices.OpenTypeExtension.c.i.m.p.server.entities.outlook.Event",
"extensionName": "c.i.m.p.server.entities.outlook.Event",
"adherentId": "12346",
"timeSlotID": "346463"
}
]
Here is the Java code (Java 8, using java.io and java.net libraries):
private static void doSomething(String _accessToken) throws IOException {
String urlString = "https://graph.microsoft.com/v1.0/users/{userId}/calendars/{calendarId}/events?$expand=Extensions($filter=Id eq 'c.i.m.p.server.entities.outlook.Event')";
URL url = new URL(urlString);
Proxy webProxy
= new Proxy(Proxy.Type.HTTP, new InetSocketAddress({proxy-address}, {port}));
HttpURLConnection connection = (HttpURLConnection) url.openConnection(webProxy);
// Set the appropriate header fields in the request header.
connection.setRequestProperty("Authorization", "Bearer " + _accessToken);
connection.setRequestProperty("Accept", "application/json");
connection.setDoOutput(true);
connection.setReadTimeout(5000);
connection.setRequestMethod(HttpMethod.GET);
try {
connection.connect();
int responseCode = connection.getResponseCode();
System.out.println("execute(), response code = " + responseCode);
String responseMessage = connection.getResponseMessage();
System.out.println("execute(), response Message = " + responseMessage);
String responseString = null;
try {
InputStream ins = connection.getInputStream();
BufferedReader br=new BufferedReader(new InputStreamReader(ins));
StringBuffer sb=new StringBuffer();
String line;
while ((line=br.readLine()) != null) {
sb.append(line);
}
responseString = sb.toString();
} catch (Exception e) {
System.out.println("Could not get input stream from response, error is " + e.toString());
}
System.out.println("execute(), httpResult = " + responseString);
} catch (IOException e) {
System.out.println(".execute(), IOException : " + e.toString());
} finally {
connection.disconnect();
}
}
How do I fix this? Thanks!
400 means bad request. It could be because of url encoding. Url encode the query string.
Something like
String query = "Extensions($filter=Id eq 'c.i.m.p.server.entities.outlook.Event'";
String url = "https://graph.microsoft.com/v1.0/users/{userId}/calendars/{calendarId}/events?
$expand=" + URLEncoder.encode(query, StandardCharsets.UTF_8.name());
Alternatively you could use graph service java api based on your need which will help abstract all the interactions for you or you could use any of the rest clients available.
First of all, you should provide more info on the error - Stacktrace and error message. But 400 code indicates that was a user mistake, meaning that you are sending an invalid request. Since you say that postman request works then compare all the headers that are sent by postman and see if your code misses some hearer. As for the code, instead of coding your own Http client functionality I would suggest using 3d party Http client. Here are a few suggestions:
Apache Http client - very popular and well known 3d party Http Client
OK Http client - Open-source Http client. Here is tutorial
MgntUtils Http client - very simple 3d party HttpClient: Provided in MgntUtils Open source library (written by me). Very simple in use. Take a look at Javadoc. Library itself provided as Maven artifacts and on Git (including source code and Javadoc).
I am consuming a web services using HttpURLConnection which took 60 seconds to return the response but when I use CURL(command line) for same operation with same parameters then it took only 20 - 25 seconds to return the response.
what could be the issue in API service call through HttpURLConnection because it's taking longer time to return the response.
HttpURLConnection API call code :
`
url = new URL(this._serviceURL);
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "text/xml; charset=utf-8");
connection.setRequestProperty("Accept", "application/xml;");
connection.setRequestProperty("SOAPAction", "http://www.xxtest.com/Request");
connection.setDoInput(true);
connection.setDoOutput(true);
// Send request
DataOutputStream wr = new DataOutputStream(connection.getOutputStream());
wr.writeBytes(xmlRequest);
wr.flush();
wr.close();
// Get Response
responseCode = connection.getResponseCode();
String xmlResponse = "";
if (responseCode == HttpURLConnection.HTTP_OK) { // success
is = connection.getInputStream();
xmlResponse = IOUtils.toString(is);
// Decode base64 and Decompress
final GZIPInputStream gzipInputStream = new GZIPInputStream(new ByteArrayInputStream(Base64.decodeBase64(xmlResponse.getBytes())));
xmlResponse = IOUtils.toString(gzipInputStream);
}`
CURL command :
curl -XPOST -H "Content-type: text/xml" -H "SOAPAction: http://www.xxtest.com/Request" -H "Accept: application/xml;" -d #request_soap.xml 'http://www.xxtest.com/xmlservices.asmx' > response.xml
Update :
Above mentioned HttpURLConnection API call java code - when executed from a Web Application (Tomcat) then it's taking longer time(60 seconds) to return the response but when I run the same java code as standalone java program on same server then it is returning response in 20 seconds. Exactly same code. Now, I don't understand why the same code is taking longer time when it is getting executed from a Web Application.
The performance problem might be at this point
DataOutputStream wr = new DataOutputStream(connection.getOutputStream());
wr.writeBytes(xmlRequest);
wr.close();
I would guess following will get better Performance.
OutputStreamWriter out = new OutputStreamWriter(connection.getOutputStream());
out.write(xmlRequest);
out.close();
I have added Proxy.NO_PROXY in openConnection method in a HttpURLConnection API call code(mentioned in my question) and delayed response issue solved. The services which I was consuming were having proxy setting and because of that I was facing delayed response issue.
Please check below mentioned code snippet.
connection = (HttpURLConnection) url.openConnection(Proxy.NO_PROXY);
I have a service deployed in Heroku that produces a pdf file output. When I hit the URL in the browser, I am able to download the pdf file (I am not prompting to save (as per my requirement), it auto save to defined path in the code). So service is up and available. But when I am accessing it using HttpURLConnection I am getting 404 error. . Could anyone help me out on this?
Following is the link I am accessing:
http://quiet-savannah-7144.herokuapp.com/services/time/temp
Here is the service code, deployed in Heroku server:
#jawax.ws.rs.core.Context
ServletContext context;
#GET
#path("/temp")
#Produces("application/pdf")
public Response getPdf() throws IOException{
InputStream is = context.getResourceAsStream("/static/temp.pdf");
ResponseBuilder res = Response.ok(is);
res.header("Content-Disposition","attachment; filename=temp.pdf");
return res.build();
}
Note: I have my file in the location webapp/static/temp.pdf
Client code is as follows:
try {
URL url = new URL("http://quiet-savannah-7144.herokuapp.com/sevices/time/temp");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("GET");
conn.setRequestProperty("Content-Type", "application/json");
int code = conn.getResponseCode();
System.out.println("code>>"+code+"<<");
if (conn.getResponseCode() == 200) {
System.out.println("*************************done****************************");
InputStream inputStream = conn.getInputStream();
OutputStream output = new FileOutputStream("D:/copyOfTest.pdf");
byte[] buffer = new byte[1024];
int bytesRead;
while ((bytesRead = inputStream.read(buffer)) != -1) {
output.write(buffer, 0, bytesRead);
}
output.close();
} else {
Scanner scanner = new Scanner(conn.getErrorStream());
while(scanner.hasNext())
System.out.println(scanner.next());
scanner.close();
}
conn.disconnect();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
I tried content type with pdf and x-pdf as shown below, nothing is working
conn.setRequestProperty("Content-Type", "application/pdf");
conn.setRequestProperty("Content-Type", "application/x-pdf");
When I deploy the service locally in Tomcat server in my machine, then things are absolutely fine. I am struggling from the past 6 hours to resolve this, but no clue. Actually I have to fetch it in the android AsyncTask. If I am able to do it in java, then I could achieve the same in android. Could someone help me out on this.
Thanks in advance
I see a few problems.
First, if you do a GET you should not write conn.setDoOutput(true); cause you're not outputting from your application to the server.
Second, the Content-Type header is the content-type of what YOU send to the server, not the opposite, so since you're not sending anything but just doing a get, you should not set it.
Instead, maybe, if you want, you can set the Accept header.
Content-Type is a server header. You should send an Accept header, maybe you could try something generic like Accept: *.
Though this thread is old, may be useful for some:
Came up with similar problem and solved by adding the "Accept-Encoding: gzip, deflate, br" header
I'm having problem with sending XML-data using HTTP POST to an API.
If I send well formatted XML, I get an error message:
Server Exception: Cannot access a closed Stream
If the XML isn't well formatted, I get HTTP 500. And if I just send an empty string instead of a string with XML, I get back an error message: EMPTY REQUEST.
I don't have many ideas about what the error could be, but the connection works because the error message is returned in XML format. I'm just sending the XML data as a string. Is it possible that I am required to send an EOF or something in the end? And how do I do that in my Java code? Any other ideas about what the problem can be?
The API is made in .NET
Here is the Java code I'm using to POST the XML data:
Authenticator.setDefault(new MyAuthenticator());
String xmlRequestStatus =
"<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?><test><data>32</data></test>";
System.out.println(xmlRequestStatus);
String contentType = "text/xml";
String charset = "ISO-8859-1";
String request = null;
URL url = null;
HttpURLConnection connection = null;
OutputStream output = null;
InputStream response = null;
try {
url = new URL("http://127.0.0.1/test");
} catch (MalformedURLException e) {
e.printStackTrace();
}
try {
connection = (HttpURLConnection)url.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setRequestProperty("Accept-Charset", charset);
connection.setRequestProperty("Content-Type", contentType);
output = connection.getOutputStream();
output.write(request.getBytes("ISO-8859-1"));
if(output != null) try { output.close(); } catch (IOException e) {}
response = connection.getInputStream();
....
It looks fine and should work fine. The connection.setRequestMethod("POST"); is however entirely superfluous when you already did connection.setDoOutput(true);.
Since this error is coming straight from the .NET webservice hosted at localhost, are you sure that it is written without bugs? I don't do .NET, but Google learns me that it's related to MemoryStream. I'd concentrate on the .NET code and retest/debug it. Maybe those related SO questions may help.
You need to specify method POST by doing something like this,
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Length", "" + length);
Otherwise, it's treated as a GET and some server doesn't expect body with GET so the stream is closed.
Maybe close the OutputStream later in the control flow. So instead of this:
output.write(request.getBytes("ISO-8859-1"));
if(output != null) try { output.close(); } catch (IOException e) {}
response = connection.getInputStream();
Try this (and maybe add the flush)?
output.write(request.getBytes("ISO-8859-1"));
output.flush();
response = connection.getInputStream();
if(output != null) try { output.close(); } catch (IOException e) {}
Shouldn't it be <32 instead of <32?
It looks like request is initialized to null, but afterwards not set. Should it not be
output.write(xmlRequestStatus.getBytes("ISO-8859-1"));