I am basically working on creating a restful API which produces surveys and consumes survey responses.
I want to use Morphia library for modelling the HTTP Request document in MongoDB.
I want to log incoming HTTP request data for a survey application and later planning to analyze the data by performing various kind of "rule matching" on that request data.
HTTP requests have a quite a lot of information in headers. I want to basically store those headers in a meaningful way.
There are some standard headers with specific values, and there are some custom headers with custom values. I want to treat standard headers like Referer,Cookie,URL etc as first class citizens for query the data later.
I am not intending to store request body in this model as it will have a variable size ranging from body of KiBs to many MiBs.
So following are kind of queries i'll be performing
Match HTTP Requests with Referer as major search engines. Match HTTP
Match requests where URL param is some kind of regex Match HTTP Request
Match requests where some Cookie and it's given value is present
and list goes on..
What should a Morhpia model look like to achieve I described above ?
Morphia : http://code.google.com/p/morphia/
Related
So I am building a REST APi that will be responsible for running various jobs on a special hardware.
So I understand that REST is used for accessing resources and not calling functions.
So what are the recommendations and best way to design an API for responsible for calling functions.
For example I will have an api job/run which will return the PID of job if job was sucesfully ran.
I'll also have a job/{pid} for accessing information about a given job. and job/cancel/{pid} for stopping said job.
So what are the recommendations and best way to design an API for
responsible for calling functions
Create a user: POST /users
Delete a user: DELETE /users/1
Get all users: GET /users
Get one user: GET /users/1
To GET Record
Bad designs
GET /FetchUsers // To fetch all records
GET /getAllUsers/12 // To fetch specific records
Preferred Designs
GET /users //To fetch all records
GET /users/12 // To fetch specific records
To Crete Record
Bad designs
POST /createUsers //To create users
GET /createrecordforUsers //To fetch all records
Preferred designs
POST /users //To create users records
To Update Record
Bad designs
PUT /updateUsersid // To update user
POST /id/modifyuser // To update users
Preferred designs
PUT /users/:id // To update users
To Delete Record
Bad designs
DELETE /deleteuser/id //To delete users
POST /id/removeusers //To delete users
Preferred designs
DELETE /users/:id // To delete users
Below points should be considered
Platform independence . (Any client should be able to call the API, regardless of how the API is implemented internally)
Service evolution .(The web API should be able to evolve and add functionality independently from client applications.)
A resource has an identifier, which is a URI that uniquely identifies that resource. For example, the URI for a particular customer order might be:
Request : GET -> https://domain/orders/1
Response :
JSON
{"orderId":1111,"amount":99.90,"productId":1,"quantity":1}
The most common operations are GET, POST, PUT, PATCH, and DELETE.
GET retrieves a representation of the resource at the specified URI. The body of the response message contains the details of the requested resource.
POST creates a new resource at the specified URI. The body of the request message provides the details of the new resource. Note that POST can also be used to trigger operations that don't actually create resources.
PUT either creates or replaces the resource at the specified URI. The body of the request message specifies the resource to be created or updated.
PATCH performs a partial update of a resource. The request body specifies the set of changes to apply to the resource.
DELETE removes the resource at the specified URI.
REST APIs use a stateless request model. HTTP requests should be independent and may occur in any order, so keeping transient state information between requests is not feasible.
We can return response with hypermedia links as spring boot having this feature
[Spring Boot:] https://spring.io/guides/gs/rest-hateoas/
https://domain/orders (Better)
https://domain/create-order (Avoid)
A resource doesn't have to be based on a single physical data item. For example, an order resource might be implemented internally as several tables in a relational database, but presented to the client as a single entity. Avoid creating APIs that simply mirror the internal structure of a database.
A client should not be exposed to the internal implementation.
Avoid requiring resource URIs more complex than (order/collection/item/details)
Summary
- Pagination Support : /orders?limit=25&offset=50
- Error handing :
- API Version (avoid as much as if possible)
Refer here https://www.openapis.org/blog/2017/03/01/openapi-spec-3-implementers-draft-released
I have WCA 3.5.0 server and I need to get documents from the site, using web crawler.
The problem is in the fact that I have to send a POST request to the site to get some data (initialy my site consists only of a form with some fields and submit button to send the request to the server). So, my POST request body should be something like that:
{"DateFrom":"2000-01-01T00:00:00","DateTo":"2030-01-01T23:59:59","Bundles":[{"Name":"the test name that i passed","Type":-1}],"Company":[],"Transaction":[],"Text":""}
I was thinking about making a a prefetch plugin for a web crawler.
But from the documentation I've found it looks like it is hardly possible:
"The first element ([0]) in the argument array that is passed to your
plug-in is an object of type PrefetchPluginArg1, which is an interface
that extends the interface PrefetchPluginArg. This is the only
argument and the only argument type that is passed to the prefetch
plug-in."
PrefetchPluginArg1 class has only getHTTPHeader(), setHTTPHeader(), getURL(), setURL(), doFetch(), setFetch(),
where:
The getHTTPHeader method returns a String that contains the all of
the content of the HTTP request header that the crawler sends so that
the crawler can download the document.
The getURL method returns the URL (in String form) of a document that
the crawler downloads. You can use this URL to decide if the document
requires additional information in the request header, such as a
cookie.
And it looks like there is no way to change request body.
So, is it realy possible to control POST request body, but not only header, and if it is so, can you please, share some information about the ways of solving this task?
I am working on a Ticket Reservation application. I am required to build a web-service which takes in request for cancellation, checks with a downstream system (via SOAP), whether it can be cancelled, and returns the appropriate response to the calling system after updating my tables.
I have to build a bunch of similar web-services for cancellation, rescheduling, et for which I'm planning to use RESTful web-services
This I'm planning to achieve using the POST method of the REST API, using JAX-RS. I could see that all the web-services I'm going to build suit the POST verb, and the other HTTP methods (GET, POST and DELETE) doesn't seem like they need to be used here. In that case what is the actual use case of these HTTP methods in REST? Am I right in using only POST?
Your REST resource is the ticket. Use GETs to read the ticket, PUT to modify the ticket state, and DELETE to do a logical delete (cancellation).
For any other operation, use the wildcard operation POST.
Have a look at this blog post Brief Introduction to REST, where the author goes through the core REST principles.
For the purpose of a cancellation, I would actually use the DELETE method. In the case of a successful cancellation, return a 200 with either response body confirming success or a response body containing the details of the reservation. Or a 204 if you don't want to return a body.
If it can't be canceled, you have your choice from the 400 series of errors (e.g. 404 if there is no such reservation, 403 if the request is forbidden, etc.) or the 500 series if there is a bug or malfunction.
You will certainly use the other HTTP verbs for all the other reservation actions--making a reservation, changing it, etc.
Consider your Ticket Reservations as your application resources.
Clients to your application would:
submit GETrequest to retrieve a representation (XML, JSON or anything else) one or many ticket reservations (given the URL they use, eg: /tickets to retrieve all of them - or all they are allowed to see, /tickets/1to retrieve a representation of the Ticket Reservation whose id is 1, tickets/?start=0&size=10 for paginated content, etc.)
submit POST requests on /ticketsto create a new reservation (and the response would provide the client with a location giving the location of the created reservation)
submit a PUT request to /tickets/1 (with a representation of the new reservation in the request body) to update the reservation identified by 1
submit a DELETE request to /tickets/1 to delete the reservation identified by 1
The main idea behind the use of such verbs is that the URLs should identify resources (and not actions), and the HTTP verb would be the action.
If you use everything with POST, you'll be more or less doing SOAP.
This article will give you more information on hat I tried to hihlight above: http://martinfowler.com/articles/richardsonMaturityModel.html
There's quite a lot to say about designing RESTful applications. You really should read one or more of these books: "Restful Web Services" - http://amzn.com/0596529260, "RESTful Web APIs" - http://amzn.com/B00F5BS966, "REST in Practice: Hypermedia and Systems Architecture" - http://amzn.com/B0046RERXY or "RESTful Web Services Cookbook" - http://amzn.com/B0043D2ESQ.
Get a cup of coffee and enjoy some reading :-)
We are building small test simulators that need to respond to restful requests injected into a platform based on content we inject into the message:
Example: GET http://server/example-app/users
Content-Headers and/or query params = some value of pass with 200
Server respond with 200 and content
Example: GET http://server/example-app/users
Content-Headers and/or query params = some value of fail with 400
Server respond with 400 and error
I am looking to see if anyone knows of an open source tool that would be of usefulness to inspect / parse the http request and figure out response based on the look-up criteria. I'm sure I could write some parsers easily, but just interested if the community has used or knows of something that is available to do the parsing and response mappings.
It's possible that the OWASP Zed Attack Proxy covers your needs.
Fiddler is an excellent proxy to inspect code and perform filtering: http://fiddler2.com/
I'm reasonably confident I know the answer to this, but am struggling to find any concrete information out there. I'm aware that the client submits requests to an http server optionally supplying reqeust parameters. The server has the additional capability to store information in request attributes via Objects. My question is, does the client have any access to the attributes in an http request object? We have a lot of poorly written code which looks something like this:
if (request.getAttribute("name") != null)
name = request.getAttribute("name);
else if (request.getParameter("name") != null)
name = request.getParameter("name");
I'm guess this is because the original developer didn't fully understand how client side http requests submitted data to the server. In any case, I'm currently working on implementing additional valiadation and encoding of request data to prevent XSS vulnerabilities and wondered if it was possible for the client to corrupt/hack/take advantage of request attributes (assuming they aren't ever populated with data sourced from the client)?
No. The attributes are something the servlet spec adds, and can be used to communicate between different entities operating on the request. They don't travel over the wire, so they don't exist client side.
The client can set the body, the parameters (i.e. the url) and the headers, and that's pretty much it.
See:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5
How the attribute field of a HttpServletRequest maps to a raw HTTP request?