We are maintaining a project which currently run on java 8. Now we have to call a partner server through their api to migrate some data into our database.
When we call their server, we got an exception:
Caused by: org.apache.http.ProtocolException: The server failed to respond with a valid HTTP response
Response code in this case is -1.
So i try to use curl to get data from their server, i saw that their response status line is:
HTTP/2 200
While HttpURLConnection check like this:
if (statusLine.startsWith("HTTP/1.")) {
int codePos = statusLine.indexOf(' ');
if (codePos > 0) {
int phrasePos = statusLine.indexOf(' ', codePos+1);
if (phrasePos > 0 && phrasePos < statusLine.length()) {
responseMessage = statusLine.substring(phrasePos+1);
}
// deviation from RFC 2616 - don't reject status line
// if SP Reason-Phrase is not included.
if (phrasePos < 0)
phrasePos = statusLine.length();
try {
responseCode = Integer.parseInt
(statusLine.substring(codePos+1, phrasePos));
return responseCode;
} catch (NumberFormatException e) { }
}
}
return -1;
Because status line here is 2, not 1., it fail to read and parse response from server.
But we can't just upgrade our Java to higher version as it will require a lot of testing to make sure nothing break, anyone know to fix this in java 8?
Can you make use of HttpClient 5.0 ? This supports HTTP/2 on Java 7 and above (note that I'm somewhat surprised that your partner service doesn't support a downgrade to HTTP/1.x in some form?)
Related
I have written an application on Android which realises sending simply requests (using Volley) to the server. The server is stood up on the NodeMCU (ESP8266) microcontroller, written in Lua. The problem is, that after sending the request, application not always is able to print the response. If the address is e.g. "http://www.google.com" it correctly sends request and receive and display response, but if it is the address from the code below - it correctly sends request (the server reacts) but does not (?) receive response (does not display it, displays: "That didn't work!"). Do you have any ideas, how can I fix it and be able to print the response?
Android (part responsible for sending requests):
buttonSynchro.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View view) {
// Instantiate the RequestQueue.
String url = "http://192.168.1.12/";
// Request a string response from the provided URL.
StringRequest stringRequest = new StringRequest(Request.Method.GET, url,
new Response.Listener<String>() {
#Override
public void onResponse(String response) {
// Display the first 500 characters of the response string.
testTextView.setText("Response is: "+ response.substring(0,500));
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
testTextView.setText("That didn't work!");
}
});
// Add the request to the RequestQueue.
RequestQueue queue = Volley.newRequestQueue(SettingsActivity.this);
queue.add(stringRequest);
}
});
NodeMCU, Lua:
station_cfg={}
station_cfg.ssid="Dom"
station_cfg.pwd="lalala"
wifi.sta.config(station_cfg)
function receive(conn, request)
print(request)
print()
local buf = "";
buf = buf.."<!doctype html><html>";
buf = buf.."<h1> ESP8266 Web Server</h1>";
buf = buf.."</html>";
conn:send(buf);
conn:on("sent", function(sck) sck:close() end);
collectgarbage();
end
function connection(conn)
conn:on("receive", receive)
end
srv=net.createServer(net.TCP, 30)
srv:listen(80, connection)
The code by nPn works in some user agents (Chrome/Firfox/curl/wget on macOS) but not in others (Safari on macOS & iOS, Firefox Klar on iOS). That likely is due to missing HTTP headers.
I advise you stick to the example we have in our documentation at https://nodemcu.readthedocs.io/en/latest/en/modules/net/#netsocketsend.
srv = net.createServer(net.TCP)
function receiver(sck, data)
print(data)
print()
-- if you're sending back HTML over HTTP you'll want something like this instead
local response = {"HTTP/1.0 200 OK\r\nServer: NodeMCU on ESP8266\r\nContent-Type: text/html\r\n\r\n"}
response[#response + 1] = "<!doctype html><html>"
response[#response + 1] = "<h1> ESP8266 Web Server</h1>"
response[#response + 1] = "</html>"
-- sends and removes the first element from the 'response' table
local function send(localSocket)
if #response > 0 then
localSocket:send(table.remove(response, 1))
else
localSocket:close()
response = nil
end
end
-- triggers the send() function again once the first chunk of data was sent
sck:on("sent", send)
send(sck)
end
srv:listen(80, function(conn)
conn:on("receive", receiver)
end)
Also, your code (and nPn's for that matter) makes assumptions about WiFi being available where it shouldn't.
wifi.sta.config(station_cfg) (with auto-connect=true) and wifi.stat.connect are asynchronous and thus non-blocking - as are many other NodeMCU APIs. Hence, you should put the above code into a function and only call it once the device is connected to the AP and got an IP. You do that by e.g. registering a callback for the STA_GOT_IP event with the WiFi event monitor. You'll find a very elaborate example of a boot sequence that listens to all WiFi events at https://nodemcu.readthedocs.io/en/latest/en/upload/#initlua. For starters you may want to trim this and only listen for got-IP.
Based on your comment above and the link you posted showing the traceback, your android app is crashing in the onResponse() method because you are asking for a substring longer than the actual string length.
You can fix this in a number of ways, but one would be to make the ending index be the minimum of the length of the response and 500 (which I assume is the max you can take in your TextView?). You can try changing
testTextView.setText("Response is: "+ response.substring(0,500));
to
testTextView.setText("Response is: "+ response.substring(0, Math.min(response.length(), n)));
or whatever other way you think is more appropriate to limit the length of the response that does not cause the IndexOutOfBoundsException
See the substring method here
public String substring(int beginIndex,
int endIndex)
Returns a new string that is a substring of this string. The substring
begins at the specified beginIndex and extends to the character at
index endIndex - 1. Thus the length of the substring is
endIndex-beginIndex.
Examples:
"hamburger".substring(4, 8) returns "urge"
"smiles".substring(1, 5) returns "mile"
Parameters:
beginIndex - the beginning index, inclusive.
endIndex - the ending index, exclusive. Returns:
the specified substring. Throws:
IndexOutOfBoundsException - if the beginIndex is negative, or endIndex is larger than the length of this String object, or
beginIndex is larger than endIndex.
I am not a Lua expert, but I think you are registering your "sent" callback after you send the response.
I think you should move it into the connection function:
station_cfg={}
station_cfg.ssid="Dom"
station_cfg.pwd="lalala"
wifi.sta.config(station_cfg)
function receive(conn, request)
print(request)
print()
local buf = "";
buf = buf.."<!doctype html><html>";
buf = buf.."<h1> ESP8266 Web Server</h1>";
buf = buf.."</html>";
conn:send(buf);
collectgarbage();
end
function connection(conn)
conn:on("receive", receive)
conn:on("sent", function(sck) sck:close() end);
end
srv=net.createServer(net.TCP, 30)
srv:listen(80, connection)
I have tried to implement range request video playback on a system that has webservlet and UI, that sends the range request from chrome starting with bytes :0- to the backend dataserver. Now I have been sending the full stream as I was under the impression the jetty server handles the response range. I see it work for the first and second request but then fails as the next request has a range that is less than what the previous range was.
(Request 1) Range:bytes=0-
(Response 1) Accept-Ranges:bytes
Content-Length:6748748
Content-Range:bytes 0-10005/6748748
(Request 2) Range:bytes=6717440-
(Response 2) Accept-Ranges:bytes
Content-Length:6748748
Content-Range:bytes 6717440-6718465/6748748
(Request 3) Range:bytes=3932160-
(Response 3) Accept-Ranges:bytes
Content-Length:6748748
Content-Range:bytes 3932160-3933185/6748748
(Request 4) Range:bytes=5701632-
(Response 4) Fails -
Can anyone make sense of this? With short videos this does not occur, so is there some timeout issue but then why is the chrome request with a smaller range? This is what I specify in the headers but again do not explicitly send the bytes as requested as I thought jetty handles it.
if(inputStream != null) {
if (parameters.containsKey("Range")) {
String range =parameters.get("Range").toString();
String[] ranges = range.split("=")[1].split("-");
final int from = Integer.parseInt(ranges[0]);
if(parameters.containsKey("Content-Length")) {
int sLength = (int) parameters.get("Content-Length");
int to = 10005 + from;
if (to >= sLength) {
to = (int) (sLength - 1);
}
if (ranges.length == 2) {
to = Integer.parseInt(ranges[1]);
}
final String responseRange = String.format("bytes %d-%d/%d", from, to, sLength);
parameters.put("Responserange", responseRange);
}
}
EDIT:
My logs in the dataserver side show the following consistently with each request being handled:
java.nio.channels.ClosedChannelException,
Added additional stack trace
java.nio.channels.ClosedChannelException
at
org.eclipse.jetty.util.IteratingCallback.close(IteratingCallback.java:427)
at org.eclipse.jetty.server.HttpConnection.onClose(HttpConnection.java:489)
at org.eclipse.jetty.io.ssl.SslConnection.onClose(SslConnection.java:217)
We're having some trouble trying to implement a Pool of SftpConnections for our application.
We're currently using SSHJ (Schmizz) as the transport library, and facing an issue we simply cannot simulate in our development environment (but the error keeps showing randomly in production, sometimes after three days, sometimes after just 10 minutes).
The problem is, when trying to send a file via SFTP, the thread gets locked in the init method from schmizz' TransportImpl class:
#Override
public void init(String remoteHost, int remotePort, InputStream in, OutputStream out)
throws TransportException {
connInfo = new ConnInfo(remoteHost, remotePort, in, out);
try {
if (config.isWaitForServerIdentBeforeSendingClientIdent()) {
receiveServerIdent();
sendClientIdent();
} else {
sendClientIdent();
receiveServerIdent();
}
log.info("Server identity string: {}", serverID);
} catch (IOException e) {
throw new TransportException(e);
}
reader.start();
}
isWaitForServerIdentBeforeSendingClientIdent is FALSE for us, so first of all the client (we) send our identification, as appears in logs:
"Client identity String: blabla"
Then it's turn for the receiveServerIdent:
private void receiveServerIdent() throws IOException
{
final Buffer.PlainBuffer buf = new Buffer.PlainBuffer();
while ((serverID = readIdentification(buf)).isEmpty()) {
int b = connInfo.in.read();
if (b == -1)
throw new TransportException("Server closed connection during identification exchange");
buf.putByte((byte) b);
}
}
The thread never gets the control back, as the server never replies with its identity. Seems like the code is stuck in this While loop. No timeouts, or SSH exceptions are thrown, my client just keeps waiting forever, and the thread gets deadlocked.
This is the readIdentification method's impl:
private String readIdentification(Buffer.PlainBuffer buffer)
throws IOException {
String ident = new IdentificationStringParser(buffer, loggerFactory).parseIdentificationString();
if (ident.isEmpty()) {
return ident;
}
if (!ident.startsWith("SSH-2.0-") && !ident.startsWith("SSH-1.99-"))
throw new TransportException(DisconnectReason.PROTOCOL_VERSION_NOT_SUPPORTED,
"Server does not support SSHv2, identified as: " + ident);
return ident;
}
Seems like ConnectionInfo's inputstream never gets data to read, as if the server closed the connection (even if, as said earlier, no exception is thrown).
I've tried to simulate this error by saturating the negotiation, closing sockets while connecting, using conntrack to kill established connections while the handshake is being made, but with no luck at all, so any help would be HIGHLY appreciated.
: )
I bet following code creates a problem:
String ident = new IdentificationStringParser(buffer, loggerFactory).parseIdentificationString();
if (ident.isEmpty()) {
return ident;
}
If the IdentificationStringParser.parseIdentificationString() returns empty string, it will be returned to the caller method. The caller method will keep calling the while ((serverID = readIdentification(buf)).isEmpty()) since the string is always empty. The only way to break the loop would be if call to int b = connInfo.in.read(); returns -1... but if server keeps sending the data (or resending the data) this condition is never met.
If this is the case I would add some kind of artificial way to detect this like:
private String readIdentification(Buffer.PlainBuffer buffer, AtomicInteger numberOfAttempts)
throws IOException {
String ident = new IdentificationStringParser(buffer, loggerFactory).parseIdentificationString();
numberOfAttempts.incrementAndGet();
if (ident.isEmpty() && numberOfAttempts.intValue() < 1000) { // 1000
return ident;
} else if (numberOfAttempts.intValue() >= 1000) {
throw new TransportException("To many attempts to read the server ident").
}
if (!ident.startsWith("SSH-2.0-") && !ident.startsWith("SSH-1.99-"))
throw new TransportException(DisconnectReason.PROTOCOL_VERSION_NOT_SUPPORTED,
"Server does not support SSHv2, identified as: " + ident);
return ident;
}
This way you would at least confirm that this is the case and can dig further why .parseIdentificationString() returns empty string.
Faced a similar issue where we would see:
INFO [net.schmizz.sshj.transport.TransportImpl : pool-6-thread-2] - Client identity string: blablabla
INFO [net.schmizz.sshj.transport.TransportImpl : pool-6-thread-2] - Server identity string: blablabla
But on some occasions, there were no server response.
Our service would typically wake up and transfer several files simultaneously, one file per connection / thread.
The issue was in the sshd server config, we increased maxStartups from default value 10
(we noticed the problems started shortly after batch sizes increased to above 10)
Default in /etc/ssh/sshd_config:
MaxStartups 10:30:100
Changed to:
MaxStartups 30:30:100
MaxStartups
Specifies the maximum number of concurrent unauthenticated connections to the SSH daemon. Additional connections will be dropped until authentication succeeds or the LoginGraceTime expires for a connection. The default is 10:30:100. Alternatively, random early drop can be enabled by specifying the three colon separated values start:rate:full (e.g. "10:30:60"). sshd will refuse connection attempts with a probability of rate/100 (30%) if there are currently start (10) unauthenticated connections. The probability increases linearly and all connection attempts are refused if the number of unauthenticated connections reaches full (60).
If you cannot control the server, you might have to find a way to limit your concurrent connection attempts in your client code instead.
I am trying to reproduce the example from the Wiki tutorial for Project Wonder REST:
community.org/display/WEB/Your+First+Rest+Project#YourFirstRestProject-Addingpostsandauthorswithcurl
I am the point where you add entries in the DB with curl (I couldn't do it, I added them via SQL).
I am trying to run the curl command to retrieve entries and get an error "Empry reply from server". The console reports the following:
Request start for URI /cgi-bin/WebObjects/BlogTutorial.woa/ra/blogEntries.json
Headers{accept = ("*/*"); host = ("127.0.0.1:45743"); user-agent = ("curl/7.38.0"); }
[2015-8-14 17:20:19 CEST] <WorkerThread14> <er.rest.routes.ERXRouteRequestHandler>: Exception while handling action named "index" on action class "your.app.rest.controllers.BlogEntryController" :com.webobjects.foundation.NSForwardException [java.lang.reflect.InvocationTargetException] null:java.lang.reflect.InvocationTargetException
_ignoredPackages:: ("com.webobjects", "java.applet", "java.awt", "java.awt.datatransfer", "java.awt.event", "java.awt.image", "java.beans", "java.io", "java.lang", "java.lang.reflect", "java.math", "java.net", "java.rmi", "java.rmi.dgc", "java.rmi.registry", "java.rmi.server", "java.security", "java.security.acl", "java.security.interfaces", "java.sql", "java.text", "java.util", "java.util.zip")
Headers{cache-control = ("private", "no-cache", "no-store", "must-revalidate", "max-age=0"); expires = ("Fri, 14-Aug-2015 15:20:19 GMT"); content-type = ("text/html"); content-length = ("9296"); pragma = ("no-cache"); x-webobjects-loadaverage = ("1"); date = ("Fri, 14-Aug-2015 15:20:19 GMT"); set-cookie = (); }
The request start and both Headers messages are mine, through an override of dispatchRequest.
Any ideas?
I want to setup Janrain authentication to my Play! project which is hosted on GAE and uses GAE module. But I get the following error while I try to login:
RuntimeException occured : Cannot parse JSON (check logs)
And Play highlighs the following line as error:
JsonElement rpxJson = rpxRequest.get().getJson();
Here is method that I use for token callback:
public static void tokenCallback(String token) {
Properties p = Play.configuration;
// Try the driver
String rpxApi = p.getProperty("login.rpx.apiKey");
WSRequest rpxRequest = WS.url("http://rpxnow.com/api/v2/auth_info");
// get RPX
rpxRequest.setParameter("token", token);
rpxRequest.setParameter("apiKey", rpxApi);
JsonElement rpxJson = rpxRequest.get().getJson();
JsonElement profile = rpxJson.getAsJsonObject().get("profile");
String identifier = profile.getAsJsonObject().getAsJsonPrimitive("identifier").getAsString();
welcome(identifier);
}
And here is the error that I get from terminal:
Internal Server Error (500) for request POST /login/tokencallback
Execution exception (In /app/controllers/Login.java around line 27)
RuntimeException occured : Cannot parse JSON (check logs)
play.exceptions.JavaExecutionException: Cannot parse JSON (check logs)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:237)
at Invocation.HTTP Request(Play!)
Caused by: java.lang.RuntimeException: Cannot parse JSON (check logs)
at play.libs.WS$HttpResponse.getJson(WS.java:668)
at controllers.Login.tokenCallback(Login.java:27)
at play.mvc.ActionInvoker.invokeWithContinuation(ActionInvoker.java:557)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:508)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:484)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:479)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:161)
... 1 more
Caused by: com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Expected EOF at line 2 column 1
at com.google.gson.JsonParser.parse(JsonParser.java:65)
at com.google.gson.JsonParser.parse(JsonParser.java:45)
at play.libs.WS$HttpResponse.getJson(WS.java:665)
... 7 more
Caused by: com.google.gson.stream.MalformedJsonException: Expected EOF at line 2 column 1
at com.google.gson.stream.JsonReader.syntaxError(JsonReader.java:1310)
at com.google.gson.stream.JsonReader.peek(JsonReader.java:390)
at com.google.gson.JsonParser.parse(JsonParser.java:60)
... 9 more
What can I do? Please, help me to solve this problem.
Thanks in advance.
OK, Here is my first suggestion. Try using the HTTPS connection for the URL. I ran into some problems with the HTTP connection. Here is how I do the Janrain connection:
WSRequest rpxRequest = WS.url("https://rpxnow.com/api/v2/auth_info");
// get RPX
rpxRequest.setParameter("token", token);
rpxRequest.setParameter("apiKey", rpxApi);
HttpResponse res = null;
try {
res = rpxRequest.post();
} catch (JavaExecutionException ex) {
Log.error("unknown error ", ex);
Validation.addError("", "Unknown Error: please try again");
Validation.keep();
Secure.login();
} catch (Exception ex) {
Log.error("Most likely SSL error", ex);
Validation.addError("", "SSL Error: please try again");
Validation.keep();
Secure.login();
}
if (res.getStatus() != 200) {
Log.error("status 200 error");
Validation.addError("", "Status 200 error: please try again");
Validation.keep();
Secure.login();
}
JsonElement rpxJson = res.getJson();
JsonElement profile = rpxJson.getAsJsonObject().get("profile");
JsonObject profileJson = profile.getAsJsonObject();
Having called the URL http://rpxnow.com/api/v2/auth_info , it immediately redirects to https://rpxnow.com/api/v2/auth_info (http s ). I suspect you don't get the JSON answer, but a http redirect code in your call to the web service.
Two possibilites:
1) Change the web service call to https://rpxnow.com/api/v2/auth_info , this probably solves your problem, failing that;
2) Change the line JsonElement rpxJson = rpxRequest.get().getJson(); into something like
HttpResponse httpResponse = rpxRequest.get();
Logger.log ( httpResponse.getString() );
if ( httpResponse.success() ) {
JsonElement rpxJson = httpResponse.getJson();
} else {
// fail gracefully
}
and report back on the contents of the answer which gets logged in the second line.