I'm am developing an application with the Jersey framework that acts as a proxy to the docker HTTP API. I'm able to upload images successfully to Docker but I am unable to forward the response from Docker to the client without an error. I believe it has something to do with the Transfer-Encoding=[chunked] header in the response.
Response
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> POST /proxy/images/load HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:8080
> Accept: */*
> Content-Type: application/octet-stream
> Content-Length: 477637632
> Expect: 100-continue
>
< HTTP/1.1 100
< HTTP/1.1 200
< Transfer-Encoding: chunked
< Docker-Experimental: false
< Api-Version: 1.26
< Server: Docker/1.13.1 (linux)
< Date: Mon, 23 Jul 2018 11:20:14 GMT
< Content-Type: application/json
< Content-Length: 54
<
* Problem (2) in the Chunked-Encoded data
* Closing connection 0
curl: (56) Problem (2) in the Chunked-Encoded data
code
#POST
#Path("/load2")
//#Consumes(MediaType.APPLICATION_OCTET_STREAM)
public Response loadimages(byte[] input, #Context UriInfo uriInfo) {
WebTarget webTarget = ClientBuilder.newClient().target("http://127.0.0.1:5555/images").path("load");
Response response = webTarget.request(MediaType.APPLICATION_JSON).post(Entity.entity(input, MediaType.APPLICATION_OCTET_STREAM));
return response;
}
Related
I'm using Jetty to deploy a WebSocket and an Angular application. On development it's all working but on production I have the issue that when refreshing the frontend or typing an url I get a 404 from the server that the given resource does not exist. Now I'm trying to create a rewrite rule to redirect requests to my index.html. Initializing my server looks like this:
server = new Server();
ServerConnector connector = new ServerConnector(server);
connector.setPort(config.getServer().getPort());
server.addConnector(connector);
RewriteHandler rewriteHandler = new RewriteHandler();
rewriteHandler.setRewriteRequestURI(true);
rewriteHandler.setRewritePathInfo(false);
rewriteHandler.setOriginalPathAttribute("requestedPath");
/*
RedirectRegexRule rule1 = new RedirectRegexRule();
rule1.setRegex("/(.+)");
rule1.setLocation("/index.html");
rewriteHandler.addRule(rule1);
*/
URL webRootLocation = this.getClass().getResource("/frontend/index.html");
if (webRootLocation == null)
throw new IllegalStateException("Unable to determine webroot URL location");
URI webRootUri = URI.create(webRootLocation.toURI().toASCIIString().replaceFirst("/index.html$","/"));
logger.debug("Web Root URI: {}",webRootUri);
ServletContextHandler contextHandler = new ServletContextHandler(ServletContextHandler.SESSIONS);
contextHandler.setContextPath("/");
contextHandler.setBaseResource(Resource.newResource(webRootUri));
contextHandler.setWelcomeFiles(new String[]{ "index.html" });
rewriteHandler.setHandler(contextHandler);
ServerContainer container = WebSocketServerContainerInitializer.initialize(contextHandler);
List<Class<? extends Encoder>> encoders = new ArrayList<>();
encoders.add(MessageEncoder.class);
List<Class<? extends Decoder>> decoders = new ArrayList<>();
decoders.add(MessageDecoder.class);
ServerEndpointConfig endpointConfig = ServerEndpointConfig.Builder
.create(AppEndpoint.class, "/wss-test")
.encoders(encoders)
.decoders(decoders)
.configurator(new AppEndpointConfig(config, factory))
.build();
container.addEndpoint(endpointConfig);
//server.setHandler(contextHandler);
HandlerList handlerList = new HandlerList();
handlerList.setHandlers(new Handler[]{rewriteHandler, contextHandler});
server.setHandler(handlerList);
contextHandler.addServlet(DefaultServlet.class, "/");
server.start();
// server.dump(System.err);
server.join();
My Angular frontend is compiled in a resource folder of the application which is served by the server on - for example: localhost:8080/. If my app routs to localhost:8080/some/path everything is fine, but when refreshing the page I get a 404 that some/path is unknown. Using the rewirte rule (commented out) I get ERR_TOO_MANY_REDIRECTIONS also lazy-loaded resources / modules are not found. Any suggestions on how to use Jetty rewrite for an Angular application as I have no glue on how to continue here or does Jetty not support something like the apache rewrite that is suggested by angular deployment
RewriteEngine On
# If an existing asset or directory is requested go to it as it is
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -f [OR]
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -d
RewriteRule ^ - [L]
# If the requested resource doesn't exist, use index.html
RewriteRule ^ /index.html
For anyone looking for a solution for this I made it work with Jettys RewriteHandler. My servers setup now looks like the following:
server = new Server(config.getServer().getPort());
RewriteHandler rewriteHandler = new RewriteHandler();
rewriteHandler.setRewriteRequestURI(true);
rewriteHandler.setRewritePathInfo(false);
rewriteHandler.setOriginalPathAttribute("requestedPath");
RewriteRegexRule rule1 = new RewriteRegexRule();
rule1.setRegex("^((?!"+wsPath+"|\\.js|\\.css|\\.jpe?g|\\.png).)*$");
rule1.setReplacement("/index.html");
rewriteHandler.addRule(rule1);
URL webRootLocation = this.getClass().getResource("/frontend/index.html");
if (webRootLocation == null)
throw new IllegalStateException("Unable to determine Web-Root URL location");
URI webRootUri = URI.create(webRootLocation.toURI().toASCIIString().replaceFirst("/index.html$","/"));
logger.debug("Web-Root URI: {}",webRootUri);
ServletContextHandler contextHandler = new ServletContextHandler(ServletContextHandler.SESSIONS);
contextHandler.setContextPath("/");
contextHandler.setBaseResource(Resource.newResource(webRootUri));
contextHandler.setWelcomeFiles(new String[]{ "index.html" });
rewriteHandler.setHandler(contextHandler);
// Initialize WebSocket
ServerContainer container = WebSocketServerContainerInitializer.initialize(contextHandler);
List<Class<? extends Encoder>> encoders = new ArrayList<>();
encoders.add(MessageEncoder.class);
List<Class<? extends Decoder>> decoders = new ArrayList<>();
decoders.add(MessageDecoder.class);
ServerEndpointConfig endpointConfig = ServerEndpointConfig.Builder
.create(AppEndpoint.class, "/" + wsPath)
.encoders(encoders)
.decoders(decoders)
.configurator(new AppEndpointConfig(config, factory))
.build();
container.addEndpoint(endpointConfig);
server.setHandler(rewriteHandler);
contextHandler.addServlet(DefaultServlet.class, "/");
server.start();
server.join();
It's not ideal as only specific file endings match the regex but it does work with a builded and deployed Angular application inside of Jetty. If a RESTful API is used some regex must be defined to match a path of the API like the wsPath (a simple string) in my example.
The only way to accomplish this in the Servlet world is to have a error page handling the 404 status code and issuing the redirect yourself.
This can only be done within a specific webapp, and not from a generic rewrite handling routine.
Why?
Well, the "If the requested resource doesn't exist, use index.html" is the key.
What happens.
The webapp was entered, and no url-pattern matched, so the default url-pattern (of "/") is used.
The default url-pattern is mapped to what's known as "The Default Servlet"
The Default Servlet is responsible for looking in the webapp's base resource for a matching resource and returning that content as a static resource request.
If the static resource doesn't exist, and the request was for a directory (eg: /js/ then use the list of welcome files to look up a welcome file. (this list is configured in the WEB-INF/web.xml)
<welcome-file-list>
<welcome-file>index.html</welcome-file>
<welcome-file>index.htm</welcome-file>
<welcome-file>index.jsp</welcome-file>
</welcome-file-list>
If the static resource still doesn't exist, process it as a 404 response.
Meanwhile, the error page handling routines of the servlet spec kick in.
This will result in a lookup for a path declared for 404 in your WEB-INF/web.xml.
<servlet>
<servletname>404Handler</servlet-name>
<servlet-class>com.acme.My404Servlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>404Handler</servlet-name>
<url-pattern>/404status</url-pattern>
</servlet-mapping>
<!-- ... then later ... -->
<error-page>
<error-code>404</error-code>
<location>/404status</location>
</error-page>
That path can be a servlet, a static resource, a jsp, etc. Pretty much anything that you can reference with a path.
If it's a servlet (or jsp) you can interrogate the original request via the request attributes to know why you are handling this error.
See: https://stackoverflow.com/a/32910916/775715
An example of this in embedded-jetty would be ...
package jetty.errors;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import javax.servlet.DispatcherType;
import javax.servlet.RequestDispatcher;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.handler.DefaultHandler;
import org.eclipse.jetty.server.handler.HandlerList;
import org.eclipse.jetty.servlet.DefaultServlet;
import org.eclipse.jetty.servlet.ErrorPageErrorHandler;
import org.eclipse.jetty.servlet.ServletContextHandler;
import org.eclipse.jetty.servlet.ServletHolder;
import org.eclipse.jetty.util.resource.PathResource;
public class EmbeddedWelcomeErrorDemo
{
public static void main(String[] args) throws Exception
{
Server server = new Server(8080);
String baseDir = System.getProperty("user.home");
if (args.length > 0)
baseDir = args[0];
Path basePath = Paths.get(baseDir);
if (!Files.exists(basePath) || !Files.isDirectory(basePath))
{
throw new IOException("Not a valid directory: " + basePath);
}
ServletContextHandler context = new ServletContextHandler();
context.setContextPath("/");
context.setBaseResource(new PathResource(basePath));
context.setWelcomeFiles(new String[]{
"index.html"
});
// Add error page mapping for context
context.addServlet(ErrorHandling.class, "/errorpage");
ErrorPageErrorHandler errorMapper = new ErrorPageErrorHandler();
errorMapper.addErrorPage(404, "/errorpage");
context.setErrorHandler(errorMapper);
// to handle static resources against base resource (always last)
// always named "default" (per spec)
ServletHolder defaultHolder = new ServletHolder("default", DefaultServlet.class);
// assigned to default url-pattern of "/" (per spec)
context.addServlet(defaultHolder, "/");
HandlerList handlers = new HandlerList();
handlers.addHandler(context);
handlers.addHandler(new DefaultHandler()); // for non-context errors
server.setHandler(handlers);
server.start();
server.join();
}
public static class ErrorHandling extends HttpServlet
{
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException
{
if (req.getDispatcherType() != DispatcherType.ERROR)
{
// we didn't get here from a error dispatch.
// somebody attempted to use this servlet directly.
resp.setStatus(404);
return;
}
String requestedResource = (String)req.getAttribute(RequestDispatcher.ERROR_REQUEST_URI);
log("[ErrorHandling] Requested resource was " + requestedResource);
int statusCode = (int)req.getAttribute(RequestDispatcher.ERROR_STATUS_CODE);
switch (statusCode)
{
case 404:
// let handle it by a redirect
resp.sendRedirect("/");
break;
default:
// pass the other errors through
resp.setStatus(statusCode);
break;
}
}
}
}
Some example of what happens.
$ mkdir $HOME/tmp-base
$ mdkir css
$ echo "this is the index.html" > index.html
$ echo "this is my other html" > myother.html
$ echo "this is my fancy css" > css/main.css
Then run the server example with the command line to this directory
$ java ... jetty.errors.EmbeddedWelcomeErrorDemo $HOME/tmp-base
2019-09-24 14:17:55.540:INFO::main: Logging initialized #190ms to org.eclipse.jetty.util.log.StdErrLog
2019-09-24 14:17:55.621:INFO:oejs.Server:main: jetty-9.4.20.v20190813; built: 2019-08-13T21:28:18.144Z; git: 84700530e645e812b336747464d6fbbf370c9a20; jvm 1.8.0_202-b08
2019-09-24 14:17:55.661:INFO:oejsh.ContextHandler:main: Started o.e.j.s.ServletContextHandler#7921b0a2{/,file:///home/joakim/tmp-base/,AVAILABLE}
2019-09-24 14:17:55.674:INFO:oejs.AbstractConnector:main: Started ServerConnector#7cef4e59{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2019-09-24 14:17:55.674:INFO:oejs.Server:main: Started #325ms
And then make a few requests ...
$ curl -L -vv http://localhost:8080/
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 24 Sep 2019 19:26:28 GMT
< Last-Modified: Tue, 24 Sep 2019 19:12:21 GMT
< Content-Type: text/html
< Accept-Ranges: bytes
< Content-Length: 23
< Server: Jetty(9.4.20.v20190813)
<
this is the index.html
* Connection #0 to host localhost left intact
this was welcome file handling
$ curl -L -vv http://localhost:8080/myother.html
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> GET /myother.html HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 24 Sep 2019 19:21:10 GMT
< Last-Modified: Tue, 24 Sep 2019 19:13:46 GMT
< Content-Type: text/html
< Accept-Ranges: bytes
< Content-Length: 22
< Server: Jetty(9.4.20.v20190813)
<
This is my other html
* Connection #0 to host localhost left intact
this was normal static file serving
$ curl -L -vv http://localhost:8080/css/main.css
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> GET /css/main.css HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 24 Sep 2019 19:22:22 GMT
< Last-Modified: Tue, 24 Sep 2019 19:22:16 GMT
< Content-Type: text/css
< Accept-Ranges: bytes
< Content-Length: 21
< Server: Jetty(9.4.20.v20190813)
<
this is my fancy css
* Connection #0 to host localhost left intact
This was normal static file serving
If I make some requests to non-existent resources or directories ....
$ curl -L -vv http://localhost:8080/css/bogus.css
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> GET /css/bogus.css HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 302 Found
< Date: Tue, 24 Sep 2019 19:22:46 GMT
< Location: http://localhost:8080/
< Content-Length: 0
< Server: Jetty(9.4.20.v20190813)
<
* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://localhost:8080/'
* Found bundle for host localhost: 0x5647e1581a50 [can pipeline]
* Re-using existing connection! (#0) with host localhost
* Connected to localhost (::1) port 8080 (#0)
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 24 Sep 2019 19:22:46 GMT
< Last-Modified: Tue, 24 Sep 2019 19:12:21 GMT
< Content-Type: text/html
< Accept-Ranges: bytes
< Content-Length: 23
< Server: Jetty(9.4.20.v20190813)
<
this is the index.html
* Connection #0 to host localhost left intact
This was handled by the ErrorHandling servlet
$ curl -L -vv http://localhost:8080/this/directory/does/not/exist
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> GET /this/directory/does/not/exist HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 302 Found
< Date: Tue, 24 Sep 2019 19:23:02 GMT
< Location: http://localhost:8080/
< Content-Length: 0
< Server: Jetty(9.4.20.v20190813)
<
* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://localhost:8080/'
* Found bundle for host localhost: 0x561eefa8b020 [can pipeline]
* Re-using existing connection! (#0) with host localhost
* Connected to localhost (::1) port 8080 (#0)
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 24 Sep 2019 19:23:02 GMT
< Last-Modified: Tue, 24 Sep 2019 19:12:21 GMT
< Content-Type: text/html
< Accept-Ranges: bytes
< Content-Length: 23
< Server: Jetty(9.4.20.v20190813)
<
this is the index.html
* Connection #0 to host localhost left intact
This was handled by the ErrorHandling servlet
[joakim#hyperion tmp]$ curl -L -vv http://localhost:8080/non-existant.jpeg
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> GET /non-existant.jpeg HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 302 Found
< Date: Tue, 24 Sep 2019 19:21:18 GMT
< Location: http://localhost:8080/
< Content-Length: 0
< Server: Jetty(9.4.20.v20190813)
<
* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://localhost:8080/'
* Found bundle for host localhost: 0x563f476b6a50 [can pipeline]
* Re-using existing connection! (#0) with host localhost
* Connected to localhost (::1) port 8080 (#0)
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 24 Sep 2019 19:21:18 GMT
< Last-Modified: Tue, 24 Sep 2019 19:12:21 GMT
< Content-Type: text/html
< Accept-Ranges: bytes
< Content-Length: 23
< Server: Jetty(9.4.20.v20190813)
<
this is the index.html
* Connection #0 to host localhost left intact
This was handled by the ErrorHandling servlet
I am wondering why Spring Boot is handling 404 Not Found differently.
None existing path example
I make a request on a none existing path with curl:
$ curl -v -H "Authorization: Basic YWRtaW46YWRtaW4xMjM=" http://localhost:8080/no/endpoint | python -m json.tool
* Adding handle: conn: 0x69aa30
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x69aa30) send_pipe: 1, recv_pipe: 0
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* About to connect() to localhost port 8080 (#0)
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /no/endpoint HTTP/1.1
> User-Agent: curl/7.33.0
> Host: localhost:8080
> Accept: */*
> Authorization: Basic YWRtaW46YWRtaW4xMjM=
>
< HTTP/1.1 404
< Set-Cookie: JSESSIONID=62B5B02F18842F3BD46BCE57F2EAB017;path=/;HttpOnly
< X-Application-Context: Rechnungsservice Gateway:dev
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: 0
< Content-Type: application/json;charset=UTF-8
< Transfer-Encoding: chunked
< Date: Mon, 20 Feb 2017 11:59:19 GMT
<
{ [data not shown]
100 115 0 115 0 0 569 0 --:--:-- --:--:-- --:--:-- 614
* Connection #0 to host localhost left intact
{
"error": "Not Found",
"message": "No message available",
"path": "/no/endpoint",
"status": 404,
"timestamp": 1487591959599
}
Same operation with a browser:
And now I request an endpoint for an none existing entity:
$ curl -v -H "Authorization: Basic YWRtaW46YWRtaW4xMjM=" http://localhost:8080/api/v1/settings/123 | python -m json.tool
* Adding handle: conn: 0x62aa40
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x62aa40) send_pipe: 1, recv_pipe: 0
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* About to connect() to localhost port 8
080 (#0)
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /api/v1/settings/123 HTTP/1.1
> User-Agent: curl/7.33.0
> Host: localhost:8080
> Accept: */*
> Authorization: Basic YWRtaW46YWRtaW4xMjM=
>
< HTTP/1.1 404
< Set-Cookie: JSESSIONID=BCD5ADDA48EB03C235E6573A36860F7D;path=/;HttpOnly
< X-Application-Context: Rechnungsservice Gateway:dev
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: 0
< Content-Length: 0
< Date: Mon, 20 Feb 2017 12:05:29 GMT
<
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
* Connection #0 to host localhost left intact
No JSON object could be decoded
Same operation in browser:
So my question is: Why is Spring Boot only sending a simple 404 Status in case of the not found entity? I would like to see a nice message: Entity not found or something like this on my error page for text/html or a json error object in case of a json request... For the path example it is working like a charm out of the box...
I am using Spring Boot 1.4.3.RELEASE.
UPDATE:
Sorry I forgot to say that I am using Spring Data Rest.
And here I debugged unti I found this code snippet:
class RepositoryEntityController ... {
public ResponseEntity<Resource<?>> getItemResource(...) {
Object domainObj = getItemResource(resourceInformation, id);
if (domainObj == null) {
return new ResponseEntity<Resource<?>>(HttpStatus.NOT_FOUND);
}
}
In my opinion a request to a single entity that doesn't exists should throw a "NotFoundException", leading to a better response.
The first request never reaches your Spring application, your WAR is not deployed at that location, so instead you get the generic 404 message from the web server (Tomcat perhaps).
I've created a simple server Java application using Gradle. As an embedded server , I am using Jetty . I am also using a Gretty plugin as it supports the latest Jetty version.
The project runs just fine . And I have tried to stress test it. As a part of my test I need to check the response time and therefore I am sending "Connection:Close" header via curl.
My response is a long JSON string , and I see only part of it , after which the connection hangs . I would like to know Why is it happening , and how can I work around it.
NOTE :
When sending Connection:Keep-alive header , everything is fine
When response from the server is not a long string , but smaller . It works just fine (doesn't hang)
Tried the standard Jetty plugin from gradle , the result was the same.
HOW TO TEST :
Build and run my project from console ./gradlew appRun
From bash console run curl -H "Connection:Close" -i "http://localhost:8080/Environment/example"
See the partial response and the connection still alive...
Seems like you are confusing the persistent connection modes between HTTP/1.0 and HTTP/1.1.
Either that, or you are using a really old version of curl that still defaults to HTTP/1.0.
HTTP/1.0 has no persistent connections by default, so to use persistent connections we send Connection: keep-alive.
HTTP/1.1 uses persistent connections by default, so to disable it we can send Connection: close
Using HTTP/1.0, with Connection: close is like sending this ...
GET /Environment/example HTTP/1.0
Host: localhost:8080
Connection: close
... which produces an invalid header value for Connection per the HTTP/1.0 spec
Lets use the verbose features of curl to see what's really going on Connection wise...
Example: HTTP/1.1 with normal operation:
$ curl --verbose --http1.1 http://apache.org/ -so /dev/null
* Trying 88.198.26.2...
* Connected to apache.org (88.198.26.2) port 80 (#0)
> GET / HTTP/1.1
> Host: apache.org
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 06 May 2016 12:05:39 GMT
< Server: Apache/2.4.7 (Ubuntu)
< Last-Modified: Fri, 06 May 2016 11:10:20 GMT
< ETag: "cf64-5322a812896a8"
< Accept-Ranges: bytes
< Content-Length: 53092
< Vary: Accept-Encoding
< Cache-Control: max-age=3600
< Expires: Fri, 06 May 2016 13:05:39 GMT
< Content-Type: text/html
<
{ [1125 bytes data]
* Connection #0 to host apache.org left intact
Notice that it says it kept the connection intact?
Example: HTTP/1.1 with manual Connection: close operation:
$ curl --verbose --http1.1 --header "Connection: close" http://apache.org/ -so /dev/null
* Trying 140.211.11.105...
* Connected to apache.org (140.211.11.105) port 80 (#0)
> GET / HTTP/1.1
> Host: apache.org
> User-Agent: curl/7.43.0
> Accept: */*
> Connection: close
>
< HTTP/1.1 200 OK
< Date: Fri, 06 May 2016 12:06:35 GMT
< Server: Apache/2.4.7 (Ubuntu)
< Last-Modified: Fri, 06 May 2016 11:10:20 GMT
< ETag: "cf64-5322a812896a8"
< Accept-Ranges: bytes
< Content-Length: 53092
< Vary: Accept-Encoding
< Cache-Control: max-age=3600
< Expires: Fri, 06 May 2016 13:06:35 GMT
< Connection: close
< Content-Type: text/html
<
{ [1106 bytes data]
* Closing connection 0
Ah, the HTTP response headers say that the server will close, and curl saw the connection being closed. What we wanted.
Example: HTTP/1.0 with normal operation:
$ curl --verbose --http1.0 http://apache.org/ -so /dev/null
* Trying 140.211.11.105...
* Connected to apache.org (140.211.11.105) port 80 (#0)
> GET / HTTP/1.0
> Host: apache.org
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 06 May 2016 12:08:27 GMT
< Server: Apache/2.4.7 (Ubuntu)
< Last-Modified: Fri, 06 May 2016 11:10:20 GMT
< ETag: "cf64-5322a812896a8"
< Accept-Ranges: bytes
< Content-Length: 53092
< Vary: Accept-Encoding
< Cache-Control: max-age=3600
< Expires: Fri, 06 May 2016 13:08:27 GMT
< Connection: close
< Content-Type: text/html
<
{ [4002 bytes data]
* Closing connection 0
See how the HTTP response headers say that the server will close?
Curl also saw the connection being closed.
That's what we expect with normal HTTP/1.0 operation.
Example: HTTP/1.0 with persistent connection:
$ curl --verbose --http1.0 --header "Connection: keep-alive" http://apache.org/ -so /dev/null
* Trying 88.198.26.2...
* Connected to apache.org (88.198.26.2) port 80 (#0)
> GET / HTTP/1.0
> Host: apache.org
> User-Agent: curl/7.43.0
> Accept: */*
> Connection: keep-alive
>
< HTTP/1.1 200 OK
< Date: Fri, 06 May 2016 12:08:37 GMT
< Server: Apache/2.4.7 (Ubuntu)
< Last-Modified: Fri, 06 May 2016 11:10:20 GMT
< ETag: "cf64-5322a812896a8"
< Accept-Ranges: bytes
< Content-Length: 53092
< Vary: Accept-Encoding
< Cache-Control: max-age=3600
< Expires: Fri, 06 May 2016 13:08:37 GMT
< Keep-Alive: timeout=30, max=100
< Connection: Keep-Alive
< Content-Type: text/html
<
{ [3964 bytes data]
* Connection #0 to host apache.org left intact
Yup, the server indicates that it will use Keep-Alive too (per HTTP/1.0 spec), and curl even concurs and says the connection is left intact.
I have implemented an application on GAE which can be accessible through https://<my_app_id>.appspot.com. Now I have a custom domain registered with Register.com. As described in GAE documentation I have mapped my custom domain to https://<my_app_id>.appspot.com and I see my application getting served from my custom domain. But I see requests are failed with error "SSL required to perform this operation". But I don't have any SSL certificate. And Can I serve my application without SSL? I mean just using http
UPDATED:
Response on executing url curl -v example.org/_ah/api/myapi/v1/package/ -o /dev/null
* Adding handle: conn: 0x1fa7e80
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x1fa7e80) send_pipe: 1, recv_pipe: 0
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* About to connect() to mydomain.com port 80 (#0)
* Trying ipaddress...
* Connected to mydomain.com (ipaddress) port 80 (#0)
> GET /_ah/api/myapp/v1/package/ HTTP/1.1
> User-Agent: curl/7.30.0
> Host: mydomain.com
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: Fri, 01 Jan 1990 00:00:00 GMT
< Date: Tue, 23 Jun 2015 12:26:50 GMT
< Vary: X-Origin
< Content-Type: text/html; charset=UTF-8
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
* Server GSE is not blacklisted
< Server: GSE
< Alternate-Protocol: 80:quic,p=0
< Accept-Ranges: none
< Vary: Origin,Accept-Encoding
< Transfer-Encoding: chunked
<
{ [data not shown]
Warning: Failed to create the file /dev/null: No such file or directory
* Failed writing body (0 != 9)
* Failed writing data
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
* Closing connection 0
curl: (23) Failed writing body (0 != 9)
There two things you've missed from docs:
Google Cloud Endpoints requires SSL. If you need to access your backend API in a system not supporting SSL, you'll need to either update the system to support SSL or use a proxy.
and
Google Cloud Endpoints does not support custom domains.
See https://cloud.google.com/appengine/docs/java/endpoints/
I'm doing a POST to a restlet and need to return a zip file. But although the created file is zip, the method returns gibberish.
I tried wrapping the FileRepresentation as was suggested here:
new org.restlet.engine.application.EncodeRepresentation(org.restlet.data.Encoding.ZIP, representation);
And also tried adding a Produces annotation like this:
#Produces({"application/x-zip-compressed"})
But neither works. The representation returns as gibberish string, and the Content-Type header stays application/octet-stream. What am I missing?
These are the request headers. Note the Accept-Encoding: gzip, deflate:
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36
Origin: chrome-extension://hgmloofddffdnphfgcellkdfbfbjeloo
Content-Type: application/json
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8,he;q=0.6
The response headers:
Vary: Accept-Encoding
Last-Modified: Wed, 06 May 2015 14:49:03 GMT
Content-Disposition: attachment; filename=_backup_20150506.zip; size=162191
Date: Wed, 06 May 2015 14:49:03 GMT
Accept-Ranges: bytes
Server: Restlet-Framework/2.2.1
Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept
Set-Cookie: JSESSIONID=5F10BBBDC58D5C3D6C0474FA12C44FB9; Path=/AppName/; Domain=localhost
Content-Encoding: gzip
Content-Type: application/octet-stream
Transfer-Encoding: chunked
EDIT: I also tried changing the media type when creating the representation:
MediaType mt = MediaType.APPLICATION_ZIP;
FileRepresentation fr = new FileRepresentation(file, mt);
The response content type changed to Content-Type: application/zip but the returned value is still a gibberish string.
The right way to do that is what you used:
public class MyServerResource extends ServerResource {
#Post
public Representation test(Representation repr) {
FileRepresentation outputRepresentation
= new FileRepresentation(new File("(...)"),
MediaType.APPLICATION_ZIP);
return outputRepresentation;
}
}
So this should work.
Using curl with such code, here is what I have:
$ curl -X POST http://localhost:8182/test > mycontent.zip
$ unzip mycontent.zip
Archive: mycontent.zip
extracting: test.txt
In addition, here is what I have with the verbose mode of curl:
curl -X POST --verbose http://localhost:8182/test
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8182 (#0)
> POST /test HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:8182
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-type: application/zip
< Last-modified: Thu, 07 May 2015 08:08:59 GMT
< Content-length: 134
* Server Restlet-Framework/2.3.1 is not blacklisted
< Server: Restlet-Framework/2.3.1
< Accept-ranges: bytes
< Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept
< Date: Thu, 07 May 2015 08:19:26 GMT
<
Notice that you can use the header Disposition if you want to configure hints within the download dialog of your browser.
Otherwise, "enable GZIP compression of the JSON response entity on Resltet" corresponds to automatic compression of the whole response content by Restlet. Browsers support this and can directly uncompress the content before displaying it. I don't think that isn't really what you need / expect. If it's the case, you could be interested in this link: https://templth.wordpress.com/2015/02/23/optimizing-restlet-server-applications/.
Hope it helps you,
Thierry