We Keep Coding

Proper Http status codes for uploading files on rest end point

We are developing an application that allows a user to upload file on rest end point.
Could someone please guide if it is correct to send 400 error code for the failure of following validation scenario:
1) The Length of file name exceeds permissible limit.
2) File name contains special characters
3) Uploaded file was empty
4) The System failed to read the uploaded file from disk.
Regards,
Tarun

The Length of file name exceeds permissible limit.
I think the 400 is not an appropriate because syntax of the request is correct in this case. The 422 Unprocessable Entity is better in this case.
File name contains special characters
Illegal characters mean the syntax is broken. So 400 Bad Request is a proper response in this case. Someone may claim that a definition of illegal characters is needed so the server may authoritatively send 400.
Uploaded file was empty
I think it is not an error because an empty file is a legal file.
The System failed to read the uploaded file from disk.
Does the system mean the server? Then the server should return a 5xx response because it is not a client failure. In case of general read error the server should return 500.
EDIT:
Uploaded file was empty.
When application semantic forbids an empty file the 400 or 422 appropriate. More details about them is at 400 vs 422 response to POST of data

4xx statuses are for client-side errors, 5xx are for server-side errors. So, generally you need 4xx codes for your cases 1) to 3), while 4) should be a 5xx error.
Let’s first say that for your case 4), a simple HTTP 500 seems appropriate. If you want to indicate that the client could try again later, HTTP 503 would be more suitable.
Now for 1) to 3): According to RFC 2616, HTTP 400 indicates syntax errors; this would usually be protocoll errors, e.g. invalid headers. Semantical or payload errors aren’t really defined in this generic RFC, however, (as Zaboj mentions) WebDAV offers HTTP 422, which seems suitable, though it’s not really meant for generic HTTP.
In the end, it doesn’t really matter which particular codes you send. If your upload fails with HTTP 400 or 422, in either case the client will perform some error routine (e.g. show or log the error message).
The important thing to know is that some codes can trigger client behaviour (e.g. HTTP 401 combined with certain headers can trigger an authentication dialog in a browser), and you should be aware of these side effects.
In my opinion, it is much more important to send a useful error description in the response body to help the client fix their problem, than finding the “perfect” HTTP status code. I know that REST zealots will disagree, but none of them will be able to give you the right HTTP status code for every situation.
That said, if you want to issue fine-grained error codes/messages for automated processing, you can introduce custom HTTP header fields, e.g.
X-MyApp-Error-Code: 2.1.6
X-MyApp-Error-Message: The uploaded file is empty
Then you would provide a documentation and/or SDK which reveals all possible error code values for X-MyApp-Error-Code to your API consumers.

jclouds IOExpection: Error writing request body to server

We are using jclouds with Rackspace and when uploading lots of files via cloudfile api (multi threaded)
Once in while we are getting an exception on objectApi.put line (see example code at bottom)
Exception
16-Jul-2015 11:58:00.811 SEVERE [threadsPool-1] org.jclouds.logging.jdk.JDKLogger.logError error after writing 8192/streaming bytes to https://*****/****.jpg
java.io.IOException: Error writing request body to server
at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.checkError(HttpURLConnection.java:3478)
at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpURLConnection.java:3461)
at com.google.common.io.CountingOutputStream.write(CountingOutputStream.java:53)
at com.google.common.io.ByteStreams.copy(ByteStreams.java:74)
at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.writePayloadToConnection(JavaUrlHttpCommandExecutorService.java:297)
at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:160)
at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:64)
at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:91)
at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
at com.sun.proxy.$Proxy176.put(Unknown Source)
at
Similar issue with S3
can be found here
Example Code
ObjectApi objectApi = cloudFiles.getObjectApi(REGION, container);
ByteSource byteSource = Files.asByteSource(file);
Payload payload = Payloads.newByteSourcePayload(byteSource);
objectApi.put(hashedName, payload);
The question:
Any one has experience some behavior like that? maybe someone has workaround for that kind of issue?
Thanks
Alon

Networks are unreliable, so expect some exceptions when using cloud services, especially when dealing with many files. Specifically for jclouds uploads, we have some example code here:
https://github.com/jclouds/jclouds-examples/tree/master/blobstore-uploader
Edit: I have also added a JIRA issue to make sure we add a test specifically for this situation in swift:
https://issues.apache.org/jira/browse/JCLOUDS-965

Java Web service client can't connect to BPEL webservice when SoapUI can

I am experiencing the following error and after extensive research on the matter with a few similar issues, none seem to help.
My error:
com.sun.xml.internal.ws.client.ClientTransportException: HTTP transport error: java.net.ConnectException: Connection refused: connect
at com.sun.xml.internal.ws.transport.http.client.HttpClientTransport.getOutput(HttpClientTransport.java:121)
at com.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.process(HttpTransportPipe.java:142)
at com.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.processRequest(HttpTransportPipe.java:83)
at com.sun.xml.internal.ws.transport.DeferredTransportPipe.processRequest(DeferredTransportPipe.java:105)
at com.sun.xml.internal.ws.api.pipe.Fiber.__doRun(Fiber.java:587)
at com.sun.xml.internal.ws.api.pipe.Fiber._doRun(Fiber.java:546)
at com.sun.xml.internal.ws.api.pipe.Fiber.doRun(Fiber.java:531)
at com.sun.xml.internal.ws.api.pipe.Fiber.runSync(Fiber.java:428)
at com.sun.xml.internal.ws.client.Stub.process(Stub.java:211)
at com.sun.xml.internal.ws.client.sei.SEIStub.doProcess(SEIStub.java:124)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:98)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:78)
at com.sun.xml.internal.ws.client.sei.SEIStub.invoke(SEIStub.java:107)
at com.sun.proxy.$Proxy33.onboardPerson(Unknown Source)
at com.paychex.corp.fileloader.FileloaderMain.callWebService(FileloaderMain.java:202)
at com.paychex.corp.fileloader.FileloaderMain.main(FileloaderMain.java:104)
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:411)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:525)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:208)
at sun.net.www.http.HttpClient.New(HttpClient.java:291)
at sun.net.www.http.HttpClient.New(HttpClient.java:310)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:987)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:966)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:841)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1031)
at com.sun.xml.internal.ws.transport.http.client.HttpClientTransport.getOutput(HttpClientTransport.java:109)
My client is generated from wsimport using the WSDL file that IS visible from the active service. Code connection sample: URL and service name, etc are correct, can't list for security
URL url = new URL("http://correctserviceurl?WSDL");
QName qname = new QName("http://correct servicelocation","nameofService");
CorporateEmployeeOnboardingService service = new CorporateEmployeeOnboardingService(url, qname);
//CorporateEmployeeOnboardingService service = new CorporateEmployeeOnboardingService();
CorporateEmployeeOnboardingProcess process = service.getICorporateEmployeeOnboardingProcess();
OnboardRequestType onboardRequest = new OnboardRequestType();
/**
* TESTING SECTION FOR PROCESS CALL
*/
//create necessary complex types:
GregorianCalendar gregorianCalendar = new GregorianCalendar();
DatatypeFactory datatypeFactory;
datatypeFactory = DatatypeFactory.newInstance();
XMLGregorianCalendar testDate =
datatypeFactory.newXMLGregorianCalendar(gregorianCalendar);
testDate.setYear(1991);
testDate.setMonth(4);
testDate.setDay(3);
AssignmentComplexType testAssignment = new AssignmentComplexType();
testAssignment.setHiringManagerID("123445");
//set person to send
person.setSSN("333-33-3333");
person.setFirstName("testFname");
person.setLastName("testLname");
person.setDOB(testDate);
person.setHireDate(testDate);
person.setAssignment(testAssignment);
/**
* end testing section of the person creation
*/
onboardRequest.getPerson().add(person);//add a person to the array of people for the request
ServiceResponse response = process.onboardPerson(onboardRequest);
As stated SoapUI can send this same test information and get a perfectly valid request from the service. All information the same. I have been digging around for awhile and confirmed that there is no proxy, both are same bit (32/64 in this case both 32), service name and WSDL are correct and objects are named properly, etc. I did further debugging trying to get around this awful error code that has seemingly no usable information and using wireshark compared the SoapUI request to the java request to find that SoapUI was using POST and the java client was using GET which has me leaning on this right now as the source of the generic refused connection error.
The service is using BPEL to run and was created using BPEL so all code was basically generated. I will be looking at the Get/post find as of now but wanted to get this out there as I am doubtful it could be the cause.
Any help is greatly appreciated. Let me know if I forgot any key information I might be able to share.
EDIT
I wanted to add that I am able to access the endpoint URL in the web browser from the machine that I am using to try and connect to the BPEL service. It seems the issue is solely with the client generated from the WSDL stubs I got using wsimport for the service's WSDL.
Edit 2: Development/Progress
The issue seems to be that the client is sending the response to local host even after setting up the qname to the correct address. I am now trying to figure out where this local host is being set. Any ideas on this development would be greatly appreciated.

It's been a while, but I believe this line: URL url = new URL("http://correctserviceurl?WSDL");
Needs to read: URL url = new URL("http://correctserviceurl");

K! This one was an issue with BPEL I believe. Quite frustrating. Apparently after initializing the service in the following lines:
URL url = new URL("http://serviceaddresshere?WSDL");
QName qname = new QName("http://servicelocationaddress","NameOfService");
CorporateEmployeeOnboardingService service = new CorporateEmployeeOnboardingService(url, qname);
CorporateEmployeeOnboardingProcess process = service.getICorporateEmployeeOnboardingProcess();
BPEL gets the request with the service WSDL and the CORRECT address. However, BPEL seems to think "O? you want this service, here you go I'll give you that, only the WSDL you want is wrapper, so here's the REAL WSDL." In plain english, you get back the BPEL WSDL that it thinks is for the service, and this has the Localhost address in it. Now I tried changing the address in this WSDL but BPEL through errors up the wazzoo. So the work around was adding in this line after the process variable is make: (AKA THE FIX)
((BindingProvider) process).getRequestContext().put(Dispatch.ENDPOINT_ADDRESS_PROPERTY, "http://the serviceaddress");
Bassically you set the endpoint AGAIN after you make the process, because BPEL overwrites what you sent it the first time. This is of course a sort of work around, but the process now works good. If you can think of a way to bypass this work around and dynamically fix the service so we don't have to keep changing the service WSDL when we move ti to new enviornments like Dev, etc. I am all ears. Thanks and as always I will read comments for suggestions. =)

You can use bpel configuration plans to switch between environments.
Configuration plans are the recommended option to use for making composites environment specific.
Efficient use of configuration plans can greatly reduce the work required per process.
A configuration plan can be generated from the composite.xml file.
Please consider this link:
http://biemond.blogspot.com.br/2010/02/web-service-references-and-soa.html

RESTHeart issue with filters

I've been using RESTHeart on top of mongodb to have crud support with Mongodb's REST interface. Its working fine when I'm not using any filter, however when I tried to apply filter with the HTTP GET request as provided in the documentation I got error with the stacktrace as provided below.
Request: http://127.0.0.1:8080/inBeta/donor?filter="{'name':'john'}"
14:15:49.373 [XNIO-1 task-1] ERROR c.s.restheart.handlers.ErrorHandler
- error handling the request java.lang.ClassCastException: java.lang.String cannot be cast to org.bson.BSONObject
at com.softinstigate.restheart.db.CollectionDAO.lambda$getCollectionData$45(CollectionDAO.java:178)
~[restheart.jar:0.9.7]
at com.softinstigate.restheart.db.CollectionDAO$$Lambda$20/1288164368.accept(Unknown
Source) ~[na:na]
at java.util.ArrayDeque$DeqSpliterator.forEachRemaining(Unknown
Source)~[na:1.8.0_31]
at java.util.stream.ReferencePipeline$Head.forEach(Unknown Source) ~[na:1.8.0_31]
at com.softinstigate.restheart.db.CollectionDAO.getCollectionData(CollectionDAO.java:177)
~[restheart.jar:0.9.7]
When I didn't apply any filter its returning JSON object, however with filters its returning BSON object which RESTHeart is unable to convert as JSON response. Will appreciate any help or direction to look into the issue.
P.S. There is no tag for RESTHeart, so it would be helpful if someone could create a tag for the same.

Finally, issue resolved :)
I tried using other APIs provided in documentation and found that everything is working except filter and hence trying out the request using the below request:
http://127.0.0.1:8080/inBeta/donor?filter=%7B'username':'john'%7D
it worked. Hence the culprit is double quotes around filter query.

Error while using part="...,processingDetails" in a youtube.videos.insert() request

We're developing in java 7 and YouTube Data API v3.
When we are calling the upload service with "processingDetails" part, we are getting a 503 Service Unavailable Error. What are we doing wrong? If we remove it, we can upload the video just fine. But we want/need to know about processingDetails.
Here is the code:
YouTube.Videos.Insert videoInsert = this.youtube.videos().insert("snippet,statistics,status,processingDetails", videoObjectDefiningMetadata, mediaContent);
everything else it's the same as the java client provided in google API.
Here is the request:
https://www.googleapis.com/upload/youtube/v3/videos?part=snippet,statistics,status,processingDetails
And here is the exception stack trace:
2013-06-14 17:50:40,375 ERROR [] [main] (...)despegar.client.impl.YouTubeMediaClient (YouTubeMediaClient.java:264) : Error
com.google.api.client.googleapis.json.GoogleJsonResponseException: 503 Service Unavailable
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:145)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:111)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:38)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:423)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:343)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:460)
at com.despegar.client.impl.YouTubeMediaClient.uploadVideoWithFailureRecovery(YouTubeMediaClient.java:285)
at com.despegar.client.impl.YouTubeMediaClient.uploadAttempt(YouTubeMediaClient.java:260)
Thanks!

Two things:
The part parameter values you include in your youtube.videos.insert() request should correspond to the parts you're specifying in the upload metadata. Normally this would be snippet (since that's the part containing the required title and description) and sometimes status (if you're also setting the public/private status of the video). You can't set statistics or processingDetails during an upload, so you shouldn't include them in the list of parts. You can still request the statistics or processingDetails parts after the upload completes, via a youtube.videos.list() call.
The API should return a meaningful error message explaining this to you instead of returning a generic 503. I'll file a bug with the engineering team asking them to take care of that.