I'm new to this and I need my Java program to send http or https requests on the different websites(for example, Amazon)
so it can get the HTML code and than I would be able to find information(price on some goods) that I need via Pattern class.
Did anybody faced with that task? Maybe you used JSON or other tools? Thank you.
It seems that Amazon have an API. You should using instead of trying to parse their website.
Regarding libs to call webservices in JAVA, you could use Retrofit.
There are several parts to what you are asking:
Constructing / determining what to include in the HTTP request
Issuing the actual HTTP request
Parsing the response
The first and last are dependent on the particular service / API you are invoking, though if the API response is in a standard format (e.g. JSON), there are libraries that can help you interpret the response (though exactly which fields in the response mean something to you, will depend on the particular API and your application). Issuing the HTTP request, itself, is something that can be done with a number of different libraries, including the builtin HttpURLConnection / URL classes, as well as third party libraries such as the Apache HttpComponents or the Google HTTP Java Client Library, the latter of which includes libraries for parsing common output formats, as well.
I am coming from servlet/web application world and started learning web services(SOAP based). I have gone through some of the webservice
tutorials. I am trying to draw the parallel between normal http request and webservice request. Here are my observations ;-
1)Both are HTTP request. Webservice is a also post request which contains soap envelope as request body. Soap envelope
is just a normal xml which contains the data
2)java Stub internally marshal the XML , creates HTTP request and send it to consumer
3)Servlet at consumer side intercpets that request and unrmashal it to java object and send it to corresponding service.
Is my observation correct ? I know there may be other complexities but i tried to put the comparison in simple manner.
Your assumptions are generally correct. Yet, the subtelties can lead to huge differences.
Claim 1 : both are HTTP.
SOAP is generally used with an HTTP "binding". Yet it does not have to be that way. SOAP is designed to be fairly transport agnostic. It is not uncommon to have SOAP being used over JMS (although one might consider this an overuse of JMS, and an over architected protocol), it is certainly in production in many places. Rarely seen are SOAP/SMTP or SOAP/TCP without HTTP, but these exist too.
Webservice is a also post request which contains soap envelope as request body
A SOAP call over HTTP is a POST request. It may not be of content-type xml, though, as some variants such as SwA (SOAP with attachments) or XOP+MTOM variants may produce HTTP payloads that are MIME/Multipart (the first part of which is the SOAP Enveloppe in its pure XML form).
This use cas is most common when on is to send large binary content over a SOAP Call, and for which binary encoding may add a large weight to the request (base64 is a 1.3x factor in weight).
java Stub internally marshal the XML, creates HTTP request and send it to consumer
That is the usual way it is done, Axis framework and JAXWS frameworks work primarily this way.
The older SAAJ API, a standard EE API, requires you to build your SOAP Message by hand, using the DOM APIs, (see SOAPMessageFactory), and then send it.
If you look at Spring WS, you'll have something close to your claim, but where each part is fairly exposed and in your control (you may elect to build certain calls with a DOM Api, others by using JAXB marshalling, ...).
3)Servlet at consumer side intercpets that request and unrmashal it to java object and send it to corresponding service
Again, this is how things generaly work. But you can also have an implementation that works outside a servlet container. (See Endpoint Service API in JAX WS).
Your assumptions are right:-
Yes, Servlet request and Web service request both are normal HTTP request and yes, SOAP web service internally use HTTP POST.
Yes,java internally marshal the XML. Also, at client end one java web service client(may be a wrapped in servlet) un-marshal it. A SOAP message or SOAP type web service has many characteristics like:-
SOAP message mostly send data using XML format. XMLs are technology independent. So, SOAP can interact with two heterogeneous web applications built in two separate technologies and exchange data using XML.
SOAP web services send XML using HTTP protocol. Data are sent wrapped in an XML using the payload of HTTP.
SOAP web services can be secured. For an example, all the payment related transactions using credit card and bank info are done using secured SOAP web services.
A SOAP web service accepts XML in request and returns XML in response. In case of errors this return XMLs can also contain SOAP faults. SOAP faults contain the description of error and an error code.
Web services can carry attachment document also like PDF, Word etc. with its XML payload. Java provides separate API for this type of web services. There is an API available in java called SAAJ to accomplish this.
I think that you can find a very good response in this blog post by Ben Klopfer.
Mainly the difference between XML/SOAP vs HTTP/REST is that the former is most response verbose while the latter is lighter.
But this is not the only aspect you have to take into account.
REST represents the state of the resource and is a little bit easier to use and to understand, and always remember that it comes afterwards compared to SOAP.
In addition SOAP it is not limited to using HTTP/HTTPS, but can be also used with other transports like SMTP, JMS, etc.
Resuming the post reminds you:
Use SOAP when:
All you need is simple operations, like read only methods
Implementing a one-way or one-object service for something like data exchange or transfer
You want finer control over the specific transport of data, or can’t always use HTTP
Rigid specifications need to be enforced on incoming requests, and you want to minimize additional documentation needed for use
You can rely on client ability to parse XML, or more preferably SOAP itself
You need built-in error handling when things go wrong with requests
Use REST when:
Operations are complex, like create/read/update/delete on objects
Implementing a multi-faceted service for a variety of different objects
You want to easily and quickly target a variety of consumer end user devices as clients
Requests are generally stateless, like call and response compared to a conversation
Your clients may have limited bandwidth or processing power
You can leave it up to the client to get their requests correct and to deal with problems
I am making a automation website to send multiple files to an another site to prevent filling form every time to send a file.
I want to to make the POST request from server, because AJAX doesn't allow request to other domains.
How I can make this?
I am using Spring MVC3
Use apache http components - it allows you to perform http requests. You can also use (without 3rd party libraries) new URL(..).openConnection(), but it's less pleasant to code with it.
You can use Apache HTTP Components to implement pretty much any HTTP calls you want in your application. Also, note that it is possible to do cross domain AJAX with certain helper technologies like Flash ... if you really needed too.
I have a Spring-MVC webapp (3.0.5-RELEASE) which needs to access JSON webservices from another webapp on a different sub-domain (from the client/browser via AJAX).
I've solved this in the past by either:
writing a simple controller that proxies the requests, using Apache Commons HttpClient to handle the requests. Thus overcoming the cross-site/cross-origin request security limitations of most browsers
implementing a JSONP service on the server side (when calling our own JSON services) - not always possible
In the case where JSONP is not possible, is there a better way of doing (1.)?
and/or
Is there a library that will handle this for me? So I don't have to write all the HttpClient code myself - its not a lot of code, but I wonder if I'm (badly) re-inventing the wheel.
I have often had to consume third party web services (API), and as you mentioned, JSONP is not always an option. This is how I go about designing:
If the API is user centric, it has to provide a jsonp interface and that's what I will use. User centric means that you cannot perceive any reason to call the API, do some computations with the response, may be call one of your ajax services and then combine the response and show the user.
If my use case includes calling the API, and then acting on the response, like calling additional services from my application, combining the data and then showing it to the user, I would prefer not doing this in the browser. I would instead use RestTemplate and make backend api calls to the service. In which case there are no cross domain restrictions.
The only case where using a Server Proxy to bypass jsonp is when you are creating a Product that allows people to build custom plugins, plugins which are hosted on your page, but need to make Ajax calls to the app developers servers. This is a very involved case! (As as example look at how Apigee creates Public Facing REST API around your existing urls, or how Zendesk allows you to develop apps)
Hope this helps.
This question already has answers here:
SOAP vs REST (differences)
(13 answers)
Closed 5 years ago.
For now I have a slight idea about the differences between SOAP and RESTful services.
My question is when I should use SOAP, and when I should use RESTful; which one is "better" when it comes to performance/speed or request handling?
I'm implementing it for the first time in RESTful (Java) and I want know more about it; I've dealt with SOAP before.
This is a follow-up question to this post.
REST is almost always going to be faster. The main advantage of SOAP is that it provides a mechanism for services to describe themselves to clients, and to advertise their existence.
REST is much more lightweight and can be implemented using almost any tool, leading to lower bandwidth and shorter learning curve. However, the clients have to know what to send and what to expect.
In general, When you're publishing an API to the outside world that is either complex or likely to change, SOAP will be more useful. Other than that, REST is usually the better option.
REST vs. SOAP Web Services
I am seeing a lot of new web services are implemented using a REST
style architecture these days rather than a SOAP one. Lets step back a
second and explain what REST is.
What is a REST web service?
The acronym REST stands for representational state transfer, and this
basically means that each unique URL is a representation of some
object. You can get the contents of that object using an HTTP GET, to
delete it, you then might use a POST, PUT, or DELETE to modify the
object (in practice most of the services use a POST for this).
Who's using REST?
All of Yahoo's web services use REST, including Flickr and Delicious.
APIs use it, pubsub, bloglines, Technorati, and both eBay, and Amazon
have web services for both REST and SOAP.
Who's using SOAP?
Google seams to be consistent in implementing their web services to
use SOAP, with the exception of Blogger, which uses XML-RPC. You will
find SOAP web services in lots of enterprise software as well.
REST vs. SOAP
As you may have noticed the companies I mentioned that are using REST
APIs haven't been around for very long, and their APIs came out this
year mostly. So REST is definitely the trendy way to create a web
service, if creating web services could ever be trendy (lets face it
you use soap to wash, and you rest when your tired). The main
advantages of REST web services are:
Lightweight - not a lot of extra XML markup Human Readable Results
Easy to build - no toolkits required. SOAP also has some advantages:
Easy to consume - sometimes Rigid - type checking, adheres to a
contract Development tools For consuming web services, its sometimes a
toss up between which is easier. For instance Google's AdWords web
service is really hard to consume (in ColdFusion anyway), it uses SOAP
headers, and a number of other things that make it kind of difficult.
On the converse, Amazon's REST web service can sometimes be tricky to
parse because it can be highly nested, and the result schema can vary
quite a bit based on what you search for.
Whichever architecture you choose make sure its easy for developers
to access it, and well documented.
Freitag, P. (2005). "REST vs SOAP Web Services". Retrieved from http://www.petefreitag.com/item/431.cfm on June 13, 2010
SOAP
Simple Object Access Protocol (SOAP) is a standard, an XML language, defining a message architecture and message formats. It is used by Web services. It contains a description of the operations.
WSDL is an XML-based language for describing Web services and how to access them. It will run on SMTP, HTTP, FTP, etc. It requires middleware support and well-defined mechanism to define services like WSDL+XSD and WS-Policy.
SOAP will return XML based data
REST
Representational State Transfer (RESTful) web services. They are second-generation Web services.
RESTful web services communicate via HTTP rather than SOAP-based services and do not require XML messages or WSDL service-API definitions. For REST middleware is not required, only HTTP support is needed. It is a WADL standard, REST can return XML, plain text, JSON, HTML, etc.
REST is an architecture.
REST will give human-readable results.
REST is stateless.
REST services are easily cacheable.
SOAP is a protocol. It can run on top of JMS, FTP, and HTTP.
REST has no WSDL (Web Description Language) interface definition.
REST is over HTTP, but SOAP can be over any transport protocols such as HTTP, FTP, SMTP, JMS, etc.
REST stands for representational state transfer whereas SOAP stands for Simple Object Access Protocol.
SOAP defines its own security where as REST inherits security from the underlying transport.
SOAP does not support error handling, but REST has built-in error handling.
REST is lightweight and does not require XML parsing. REST can be consumed by any client, even a web browser with Ajax and JavaScript. REST consumes less bandwidth, it does not require a SOAP header for every message.
REST is useful over any protocol which provide a URI. Ignore point 5 for REST as mentioned below in the picture.
REST vs. SOAP
SOAP:
► SOAP is simple object access protocol that run on TCP/UDP/SMTP.
► SOAP read and write request response messages in XML format.
► SOAP uses interface in order to define the services.
► SOAP is more secure as it has its own security and well defined standards.
► SOAP follows RPC and Document style to define web services.
► SOAP uses SOAP-UI as client tools for testing.
REST
► REST is representational state transfer that uses underlying HTTP protocols.
► REST is stateless.
► REST is an architectural style that is used to describe and define web services.
► REST can read and write request response messages in JSON/XML/Plain HTML.
► REST uses URI for each resource that is used in web service.A resource can be image text method etc.
► REST uses set of verbs, like HTTP's GET, POST, PUT, DELETE.
► REST is easy to develop and easy to manage as compared to SOAP UI.
► REST has light-weight client tools or plugins that can easily be integrated inside a browser.
► REST services are cacheable.
Difference between REST and SOAP:
SOAP Web services:
If your application needs a guaranteed level of reliability and security then SOAP offers additional standards to ensure this type of operation.
If both sides (service provider and service consumer) have to agree on the exchange format then SOAP gives the rigid specifications for this type of interaction.
RestWeb services:
Totally stateless operations: for stateless CRUD (Create, Read, Update, and Delete) operations.
Caching situations: If the information needs to be cached.
SOAP web service always make a POST operation whereas using REST you can choose specific HTTP methods like GET, POST, PUT, and DELETE.
Example: to get an item using SOAP you should create a request XML, but in the case of REST you can just specify the item id in the URL itself.
REST is easier to use for the most part and is more flexible. Unlike SOAP, REST doesn’t have to use XML to provide the response. We can find REST-based Web services that output the data in the Command Separated Value (CSV), JavaScript Object Notation (JSON) and Really Simple Syndication (RSS) formats.
We can obtain the output we need in a form that’s easy to parse within the language we need for our application.REST is more efficient (use smaller message formats), fast and closer to other Web technologies in design philosophy.