Can someone explain to me what is actually the need to Url-encode
data sent in the body part of http requests when using
content-type: application/x-www-form-urlencoded
thanks
By "the need", do you mean "the purpose"?
If you're after the purple - it is simply there to tell the server what to expect: URL-encoded key=value pairs. It also allows the server to know what is not coming its way - the likes of multipart/form-data!
This allows the server to unambiguously know how to read incoming data.
The data is sent as one header (this is also why it has a size limit). As such, you definitely want to avoid stuff like: newlines, colons. In addition to this, you definitely want to escape = in data, so that it doesn't mess with the key=value structure. You also want to escape & for the same reason. URL-encoding does all that - so it only makes sense that whoever designed the HTTP protocol went for it!
There are multiple ways to send data to the server in a POST request; URL Encoded data is just one of several possible formats.
The client and server have to agree on the format of data in the POST body. URL Form encoded data is the easiest to use because of its universal support. Browsers support it natively. Every programming language allows you to read url encoded post parameters using a familiar syntax.
But of course, there is no need to use url form encoded. You can as easily send json or xml encoded POST body. As long as the client and server are in sync, you can even create a totally different encoding and use it.
Related
I am making an application in Java, that will log into my school diary using web api, so I will be able to make my own UI. As the title says, Java at some moment changes the cyrillic to unicode like \uXXXX symbolds. Here is the code on the Russian Stackoverflow: https://ru.stackoverflow.com/questions/1452959/%d0%a1%d0%b5%d1%80%d0%b2%d0%b5%d1%80-%d0%be%d1%82%d0%b2%d0%b5%d1%80%d0%b3%d0%b0%d0%b5%d1%82-%d0%b7%d0%b0%d0%bf%d1%80%d0%be%d1%81. Try to translate it, to understand more. When I am sending my request to https://httpbin.org/post instead of my LOGIN_URL with cyrillic symbols it returns them transformed, if I send request with ascii symbols, I get them back, and, in the linked post I mentioned the python project, which does exactly the same thing I want. And when I modify it to make it send request to httpbin, the cyrillic symbols are returned back! What do I do to fix my java code? P.S. Currently I am switched to okhttp3 from apache http client (same problem), but, I can go back.
Well, I solved my problem. It consisted not in the character encoding, but in the absence of two http headers, namely
httpPost.setHeader("X-Requested-With", "XMLHttpRequest");
httpPost.setHeader("Referer", Constants.BASE_URL);
(added to login request)
I have a curl that will make a post request. I have tried it in the command line and it works fine, so I know all my values that I should use. However, I am quite lost when it comes to translating this into Java.
I need to send form data and header data. The header is my authorization, and the form data contains both Strings and Dates, but again, the problem is that I don't know how to translate this into Java.
I have tried using URIBuilder and adding my form data with the addParameter, but it seems as if that one can only accept Strings and does not have any function for setting header information?
So I tried using HttpPost instead where I found the addHeader function that enables me to add my header data, but I did not find any function for adding the form data with this option.
I would be thankful if anyone could help me in any way possible.
thank you for reading my question. I have read numerous documents about HTTP/2 and tried to understand most important concepts for constructing a APNs server-side API in Java.
So far, I have a working code for creating Payload and initiating ssl handshake with APNs.
I am stuck with figuring out a way to send Push Notification to APNs, specifically sending a HTTP/2 POST request to the Apple server.
APNs requires the use of HPACK (header compression for HTTP/2), which
prevents repeated header keys and values. APNs maintains a small
dynamic table for HPACK. To help avoid filling up the APNs HPACK table
and necessitating the discarding of table data, encode headers in the
following way—especially when sending a large number of streams:
If I have understood correctly, I need to use Huffman Encoding for the header compression. If I am wrong on this, please correct me.
The :path value should be encoded as a literal header field without
indexing
The authorization request header, if present, should be encoded as a
literal header field without indexing
I read RFC 7541 for this, but have no idea what they are talking about. Please note that I am trying to understand the system and requirements to gain knowledge through this post, not just answers to the specific questions.
The appropriate encoding to employ for the apns-id, apns-expiration,
and apns-collapse-id request headers differs depending on whether it
is part of the initial or a subsequent POST operation, as follows:
The first time you send these headers, encode them with incremental
indexing to allow the header names to be added to the dynamic table
Subsequent times you send these headers, encode them as literal header
fields without indexing
What do they mean when they say, "encode them with incremental indexing to allow the header names to be added to the dynamic table" and "Subsequent times you send these headers, encode them as literal header fields without indexing" afterwards. I guess understanding one of two literal header fields with/out indexing will help me comprehend this better.
Thank you again for reading the question and please help me with this!!
I think you will be better off using a Java library that does HTTP/2 for you.
The Jetty HttpClient (disclaimer, I'm the maintainer) does exactly that, see here.
If instead you want to implement HPACK because you want to have fun, then you have to take your time and read carefully RFC 7541.
As an starting point for implementing it, you can read the many Java implementations of HPACK out there, from Jetty's to Netty's, Undertow's, etc.
All the questions you are asking (e.g. "what is a 'literal header field without indexing') are detailed in the RFC in their own sections.
Very coarsely, HPACK defines a mapping table that maps numbers to strings.
Both peer maintain this table in sync, so that the two tables always contain the same data (at rest).
When one peer sends a HPACK block, it sends the numbers, so that the receiving peer can use the numbers to access the HPACK table to obtain the strings.
For new/custom headers (think cookies), the sending peer sends the number and the string, so that the receiving peer can update its HPACK table. For the first time there is no compression, but the second time that the same header is sent, the sending peer just sends the number, since it knows that the other peer will have the string mapped already, and this gives good compression of HTTP headers.
I want to write an application in Java that will communicate with Google App Engine app written in Go by sending and receiving dynamic data. The data is not human readable (as in, not ASCII, Unicode or the like) and ranges from a couple bytes to about 1MB.
I am wondering if it is possible to send such data to and from GAE using Post method directly, or is it better to just encode it as a hex dump and transfer is at text (thusly increasing its size a couple times)?
Yes, this is possible, of course. Just like an HTTP response, an HTTP request can contain a payload of any sort (unless it's a GET or other method that doesn't permit a body); just set the content-type appropriately and send the data in the body of the HTTP request.
If people can POST 10mb photos to Facebook, then I don't see why you can't do that with your data :)
I don't know a lot about this area so please excuse me if my question is vague or stupid.
I have a webpage which uses javascript and AJAX to display live data. Every few seconds, a request is made and a JSON response is returned and the data on the webpage is updated.
What I want to do is create a program in Java that will basically capture every response and interpret the data. I have found libraries which handle the JSON format already. However, I don't know how to get the response using Java.
So for example, a live news feed. I would like to log the data as it appears.
Thanks
Basically what you need to do is make an HTTP GET request to the page that hosts the JSON. You can do this by using a Java HTTP client. The one in the link is from Apache Commons but I believe there is actually one built into Java that is relatively straight-forward to use. When you make a request, it will return a result object that you can then use to access the response data and information such as response headers, etc.