We Keep Coding

Java 8 HttpUrlConnection fail to parse response with http protocol 2

We are maintaining a project which currently run on java 8. Now we have to call a partner server through their api to migrate some data into our database.
When we call their server, we got an exception:
Caused by: org.apache.http.ProtocolException: The server failed to respond with a valid HTTP response
Response code in this case is -1.
So i try to use curl to get data from their server, i saw that their response status line is:
HTTP/2 200
While HttpURLConnection check like this:
if (statusLine.startsWith("HTTP/1.")) {
int codePos = statusLine.indexOf(' ');
if (codePos > 0) {
int phrasePos = statusLine.indexOf(' ', codePos+1);
if (phrasePos > 0 && phrasePos < statusLine.length()) {
responseMessage = statusLine.substring(phrasePos+1);
}
// deviation from RFC 2616 - don't reject status line
// if SP Reason-Phrase is not included.
if (phrasePos < 0)
phrasePos = statusLine.length();
try {
responseCode = Integer.parseInt
(statusLine.substring(codePos+1, phrasePos));
return responseCode;
} catch (NumberFormatException e) { }
}
}
return -1;
Because status line here is 2, not 1., it fail to read and parse response from server.
But we can't just upgrade our Java to higher version as it will require a lot of testing to make sure nothing break, anyone know to fix this in java 8?

Can you make use of HttpClient 5.0 ? This supports HTTP/2 on Java 7 and above (note that I'm somewhat surprised that your partner service doesn't support a downgrade to HTTP/1.x in some form?)

Microservice communication with python in Asynchronous manner

I want to trigger Python script from my Spring boot microservices im Asynchronous manner, SO that my Microservice will be notified once the execution of python script completes.Can any one suggest the best approach for this? appreciated if any one provide some reference to sample code.
Thanks in advance!!!
Thanks,
Sudheer

here is very good Example for something like that:
Source: http://code.activestate.com/recipes/511454-an-asynchronous-http-server-with-cgi-support/
import SimpleAsyncServer
# =============================================================
# An implementation of the HTTP protocol, supporting persistent
# connections and CGI
# =============================================================
import sys
import os
import traceback
import datetime
import mimetypes
import urlparse
import urllib
import cStringIO
class HTTP(SimpleAsyncServer.ClientHandler):
# parameters to override if necessary
root = os.getcwd() # the directory to serve files from
cgi_directories = ['/cgi-bin'] # subdirectories for cgi scripts
logging = True # print logging info for each request ?
blocksize = 2 << 16 # size of blocks to read from files and send
def request_complete(self):
"""In the HTTP protocol, a request is complete if the "end of headers"
sequence ('\r\n\r\n') has been received
If the request is POST, stores the request body in a StringIO before
returning True"""
terminator = self.incoming.find('\r\n\r\n')
if terminator == -1:
return False
lines = self.incoming[:terminator].split('\r\n')
self.requestline = lines[0]
try:
self.method,self.url,self.protocol = lines[0].strip().split()
except:
self.method = None # indicates bad request
return True
# put request headers in a dictionary
self.headers = {}
for line in lines[1:]:
k,v = line.split(':',1)
self.headers[k.lower().strip()] = v.strip()
# persistent connection
close_conn = self.headers.get("connection","")
if (self.protocol == "HTTP/1.1"
and close_conn.lower() == "keep-alive"):
self.close_when_done = False
# parse the url
scheme,netloc,path,params,query,fragment = urlparse.urlparse(self.url)
self.path,self.rest = path,(params,query,fragment)
if self.method == 'POST':
# for POST requests, read the request body
# its length must be specified in the content-length header
content_length = int(self.headers.get('content-length',0))
body = self.incoming[terminator+4:]
# request is incomplete if not all message body received
if len(body)<content_length:
return False
f_body = cStringIO.StringIO(body)
f_body.seek(0)
sys.stdin = f_body # compatibility with CGI
return True
def make_response(self):
"""Build the response : a list of strings or files"""
if self.method is None: # bad request
return self.error_resp(400,'Bad request : %s' %self.requestline)
resp_headers, resp_body, resp_file = '','',None
if not self.method in ['GET','POST','HEAD']:
return self.err_resp(501,'Unsupported method (%s)' %self.method)
else:
file_name = self.file_name = self.translate_path()
if not os.path.exists(file_name):
return self.err_resp(404,'File not found')
elif self.managed():
response = self.mngt_method()
else:
ext = os.path.splitext(file_name)[1]
c_type = mimetypes.types_map.get(ext,'text/plain')
resp_line = "%s 200 Ok\r\n" %self.protocol
size = os.stat(file_name).st_size
resp_headers = "Content-Type: %s\r\n" %c_type
resp_headers += "Content-Length: %s\r\n" %size
resp_headers += '\r\n'
if self.method == "HEAD":
resp_string = resp_line + resp_headers
elif size > HTTP.blocksize:
resp_string = resp_line + resp_headers
resp_file = open(file_name,'rb')
else:
resp_string = resp_line + resp_headers + \
open(file_name,'rb').read()
response = [resp_string]
if resp_file:
response.append(resp_file)
self.log(200)
return response
def translate_path(self):
"""Translate URL path into a path in the file system"""
return os.path.join(HTTP.root,*self.path.split('/'))
def managed(self):
"""Test if the request can be processed by a specific method
If so, set self.mngt_method to the method used
This implementation tests if the script is in a cgi directory"""
if self.is_cgi():
self.mngt_method = self.run_cgi
return True
return False
def is_cgi(self):
"""Test if url in a cgi directory"""
for path in self.cgi_directories:
if self.path.startswith(path):
rest = self.path[len(path):]
if not rest or rest.startswith('/'):
return True
return False
def run_cgi(self):
# set CGI environment variables
self.make_cgi_env()
# redirect print statements to a cStringIO
save_stdout = sys.stdout
sys.stdout = cStringIO.StringIO()
# run the script
try:
execfile(self.file_name)
except:
sys.stdout = cStringIO.StringIO()
sys.stdout.write("Content-type:text/plain\r\n\r\n")
traceback.print_exc(file=sys.stdout)
response = sys.stdout.getvalue()
if self.method == "HEAD":
# for HEAD request, don't send message body even if the script
# returns one (RFC 3875)
head_lines = []
for line in response.split('\n'):
if not line:
break
head_lines.append(line)
response = '\n'.join(head_lines)
sys.stdout = save_stdout # restore sys.stdout
# close connection in case there is no content-length header
self.close_when_done = True
resp_line = "%s 200 Ok\r\n" %self.protocol
return [resp_line + response]
def make_cgi_env(self):
"""Set CGI environment variables"""
env = {}
env['SERVER_SOFTWARE'] = "AsyncServer"
env['SERVER_NAME'] = "AsyncServer"
env['GATEWAY_INTERFACE'] = 'CGI/1.1'
env['DOCUMENT_ROOT'] = HTTP.root
env['SERVER_PROTOCOL'] = "HTTP/1.1"
env['SERVER_PORT'] = str(self.server.port)
env['REQUEST_METHOD'] = self.method
env['REQUEST_URI'] = self.url
env['PATH_TRANSLATED'] = self.translate_path()
env['SCRIPT_NAME'] = self.path
env['PATH_INFO'] = urlparse.urlunparse(("","","",self.rest[0],"",""))
env['QUERY_STRING'] = self.rest[1]
if not self.host == self.client_address[0]:
env['REMOTE_HOST'] = self.host
env['REMOTE_ADDR'] = self.client_address[0]
env['CONTENT_LENGTH'] = str(self.headers.get('content-length',''))
for k in ['USER_AGENT','COOKIE','ACCEPT','ACCEPT_CHARSET',
'ACCEPT_ENCODING','ACCEPT_LANGUAGE','CONNECTION']:
hdr = k.lower().replace("_","-")
env['HTTP_%s' %k.upper()] = str(self.headers.get(hdr,''))
os.environ.update(env)
def err_resp(self,code,msg):
"""Return an error message"""
resp_line = "%s %s %s\r\n" %(self.protocol,code,msg)
self.close_when_done = True
self.log(code)
return [resp_line]
def log(self,code):
"""Write a trace of the request on stderr"""
if HTTP.logging:
date_str = datetime.datetime.now().strftime('[%d/%b/%Y %H:%M:%S]')
sys.stderr.write('%s - - %s "%s" %s\n' %(self.host,
date_str,self.requestline,code))
if __name__=="__main__":
# launch the server on the specified port
port = 8081
print "Asynchronous HTTP server running on port %s" %port
print "Press Ctrl+C to stop"
server = SimpleAsyncServer.Server('', port)
HTTP.logging = False
try:
SimpleAsyncServer.loop(server,HTTP)
except KeyboardInterrupt:
for s in server.client_handlers:
server.close_client(s)
print 'Ctrl+C pressed. Closing'

One way of doing asynchronous execution in Python would be to use the celery framework. It's really simple to install and setup (basically just pip install), and the semantic for calling a method asynchronously is super clean.
If you are not too bound to Spring you could even switch to the pymacaron microservice framework (basically a flask app that uses swagger to spawn a REST API). Pymacaron (http://pymacaron.com/) supports asynchronous calls by default, via the pymacaron-async module. See the examples here: http://pymacaron.com/async.html

java.io.IOException: Incomplete parts with embedded Jetty Server

I`m programming a little file server which gets documents via HTTP-POST requests from another software.
The requests are always "multipart/form-data" types, so I`d like to split it via .getParts();
Unfortunately I always get a "java.io.IOException: Incomplete parts" or it does not find the part.
Is there something wrong with my code or is there a problem with the request?
I`m using a embedded Jetty server with Eclipse
public void create_document() {
String lv_path = gr_request.getParameter("contRep") + File.separator + gr_request.getParameter("docId");
Part lr_part = null;
try {
System.out.println(gr_request.getContentType());
//for testing
Part lr_test = gr_request.getPart("data");
System.out.println("1");
System.out.println(lr_test);
//the actual part
Collection<Part> lr_parts = gr_request.getParts();
for (Iterator<Part> i = lr_parts.iterator(); i.hasNext();) {
lr_part = ((Iterator<Part>) lr_parts).next();
//again for testing
System.out.println("content Type" + lr_part.getContentType());
System.out.println("name" + lr_part.getName());
System.out.println("content Type" + lr_part.getContentType());
String test = lv_path + ".jpg";
lr_part.write(test);
the log is
2017-11-28 11:07:47.941:INFO:oejs.Server:main: jetty-9.0.4.v20130625
2017-11-28 11:07:48.222:INFO:oejs.ServerConnector:main: StartedServerConnector#7165cbeb{HTTP/1.1}{0.0.0.0:1090}
Erkannte Aktion: CREATE_DOCUMENT
2017-11-2811:07:54.469:WARN:oejs.Request:qtp424058530-15:java.io.IOException:Incomplete parts
multipart/form-data; boundary=KoZIhvcNAQcB
1
null
The MultiPartConfig was done by
MultipartConfigElement multipartConfigElement = newMultipartConfigElement((String)null);
ir_request.setAttribute(Request.__MULTIPART_CONFIG_ELEMENT, multipartConfigElement);
Beginning of the body of a transmitted PDF file:
--KoZIhvcNAQcB
Content-Disposition: form-data; filename="data"
X-compId: data
Content-Type: application/pdf
Content-Length: 182370
%PDF-1.7
%µµµµ
1 0 obj
...and so on...
182188
%%EOF
--KoZIhvcNAQcB--

It seems that there is a problem with the request.
I changed the "filename" tag to "name" while receiving the request.
Now it's running

Running Project Wonder REST example throws an exception

I am trying to reproduce the example from the Wiki tutorial for Project Wonder REST:
community.org/display/WEB/Your+First+Rest+Project#YourFirstRestProject-Addingpostsandauthorswithcurl
I am the point where you add entries in the DB with curl (I couldn't do it, I added them via SQL).
I am trying to run the curl command to retrieve entries and get an error "Empry reply from server". The console reports the following:
Request start for URI /cgi-bin/WebObjects/BlogTutorial.woa/ra/blogEntries.json
Headers{accept = ("*/*"); host = ("127.0.0.1:45743"); user-agent = ("curl/7.38.0"); }
[2015-8-14 17:20:19 CEST] <WorkerThread14> <er.rest.routes.ERXRouteRequestHandler>: Exception while handling action named "index" on action class "your.app.rest.controllers.BlogEntryController" :com.webobjects.foundation.NSForwardException [java.lang.reflect.InvocationTargetException] null:java.lang.reflect.InvocationTargetException
_ignoredPackages:: ("com.webobjects", "java.applet", "java.awt", "java.awt.datatransfer", "java.awt.event", "java.awt.image", "java.beans", "java.io", "java.lang", "java.lang.reflect", "java.math", "java.net", "java.rmi", "java.rmi.dgc", "java.rmi.registry", "java.rmi.server", "java.security", "java.security.acl", "java.security.interfaces", "java.sql", "java.text", "java.util", "java.util.zip")
Headers{cache-control = ("private", "no-cache", "no-store", "must-revalidate", "max-age=0"); expires = ("Fri, 14-Aug-2015 15:20:19 GMT"); content-type = ("text/html"); content-length = ("9296"); pragma = ("no-cache"); x-webobjects-loadaverage = ("1"); date = ("Fri, 14-Aug-2015 15:20:19 GMT"); set-cookie = (); }
The request start and both Headers messages are mine, through an override of dispatchRequest.
Any ideas?

Java Play framework parsing JSON error

I want to setup Janrain authentication to my Play! project which is hosted on GAE and uses GAE module. But I get the following error while I try to login:
RuntimeException occured : Cannot parse JSON (check logs)
And Play highlighs the following line as error:
JsonElement rpxJson = rpxRequest.get().getJson();
Here is method that I use for token callback:
public static void tokenCallback(String token) {
Properties p = Play.configuration;
// Try the driver
String rpxApi = p.getProperty("login.rpx.apiKey");
WSRequest rpxRequest = WS.url("http://rpxnow.com/api/v2/auth_info");
// get RPX
rpxRequest.setParameter("token", token);
rpxRequest.setParameter("apiKey", rpxApi);
JsonElement rpxJson = rpxRequest.get().getJson();
JsonElement profile = rpxJson.getAsJsonObject().get("profile");
String identifier = profile.getAsJsonObject().getAsJsonPrimitive("identifier").getAsString();
welcome(identifier);
}
And here is the error that I get from terminal:
Internal Server Error (500) for request POST /login/tokencallback
Execution exception (In /app/controllers/Login.java around line 27)
RuntimeException occured : Cannot parse JSON (check logs)
play.exceptions.JavaExecutionException: Cannot parse JSON (check logs)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:237)
at Invocation.HTTP Request(Play!)
Caused by: java.lang.RuntimeException: Cannot parse JSON (check logs)
at play.libs.WS$HttpResponse.getJson(WS.java:668)
at controllers.Login.tokenCallback(Login.java:27)
at play.mvc.ActionInvoker.invokeWithContinuation(ActionInvoker.java:557)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:508)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:484)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:479)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:161)
... 1 more
Caused by: com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Expected EOF at line 2 column 1
at com.google.gson.JsonParser.parse(JsonParser.java:65)
at com.google.gson.JsonParser.parse(JsonParser.java:45)
at play.libs.WS$HttpResponse.getJson(WS.java:665)
... 7 more
Caused by: com.google.gson.stream.MalformedJsonException: Expected EOF at line 2 column 1
at com.google.gson.stream.JsonReader.syntaxError(JsonReader.java:1310)
at com.google.gson.stream.JsonReader.peek(JsonReader.java:390)
at com.google.gson.JsonParser.parse(JsonParser.java:60)
... 9 more
What can I do? Please, help me to solve this problem.
Thanks in advance.

OK, Here is my first suggestion. Try using the HTTPS connection for the URL. I ran into some problems with the HTTP connection. Here is how I do the Janrain connection:
WSRequest rpxRequest = WS.url("https://rpxnow.com/api/v2/auth_info");
// get RPX
rpxRequest.setParameter("token", token);
rpxRequest.setParameter("apiKey", rpxApi);
HttpResponse res = null;
try {
res = rpxRequest.post();
} catch (JavaExecutionException ex) {
Log.error("unknown error ", ex);
Validation.addError("", "Unknown Error: please try again");
Validation.keep();
Secure.login();
} catch (Exception ex) {
Log.error("Most likely SSL error", ex);
Validation.addError("", "SSL Error: please try again");
Validation.keep();
Secure.login();
}
if (res.getStatus() != 200) {
Log.error("status 200 error");
Validation.addError("", "Status 200 error: please try again");
Validation.keep();
Secure.login();
}
JsonElement rpxJson = res.getJson();
JsonElement profile = rpxJson.getAsJsonObject().get("profile");
JsonObject profileJson = profile.getAsJsonObject();

Having called the URL http://rpxnow.com/api/v2/auth_info , it immediately redirects to https://rpxnow.com/api/v2/auth_info (http s ). I suspect you don't get the JSON answer, but a http redirect code in your call to the web service.
Two possibilites:
1) Change the web service call to https://rpxnow.com/api/v2/auth_info , this probably solves your problem, failing that;
2) Change the line JsonElement rpxJson = rpxRequest.get().getJson(); into something like
HttpResponse httpResponse = rpxRequest.get();
Logger.log ( httpResponse.getString() );
if ( httpResponse.success() ) {
JsonElement rpxJson = httpResponse.getJson();
} else {
// fail gracefully
}
and report back on the contents of the answer which gets logged in the second line.