I am trying to create HTTP client using netty and everything work, but i have hard time parsing the body. My pipeline looks like this:
pipeline.addLast(new HttpClientCodec())
pipeline.addLast(new HttpContentDecompressor())
pipeline.addLast(new HttpObjectAggregator(1024*10))
pipeline.addLast(new HttpClientHandler[A](key, metrics))
and client handler (written in scala)
class HttpClientHandler[A: BodyParser](key: AttributeKey[Callback[A]], metrics: Metrics)
extends SimpleChannelInboundHandler[FullHttpResponse]
with LazyLogging {
override def channelRead0(ctx: ChannelHandlerContext, msg: FullHttpResponse): Unit = {
val callback = ctx.channel().attr(key).get()
if (callback != null) {
val response = buildResponse(msg)
callback(response)
} else {
throw new Exception("Callback not present in channel context ... this is a bug")
}
}
private def buildResponse(msg: FullHttpResponse): Either[Throwable, Response[A]] = {
val result = {
try {
val parsedBody = BodyParser[A].parse(msg.content().asReadOnly())
if (msg.status() == HttpResponseStatus.OK) {
Right(Response.Ok(parsedBody))
} else {
Right(Response.Other(msg.status().code(), parsedBody))
}
} catch {
case e: Throwable =>
Left(e)
}
}
result.fold(metrics.bodyParseFailure, metrics.successfulResponse)
result
}
override def exceptionCaught(ctx: ChannelHandlerContext, cause: Throwable): Unit = {
logger.warn(s"error observed for channel ${ctx.channel()}, closing", cause)
ctx.channel().attr(key).get().apply(Left(cause))
}
}
The main issue is that msg.content() also contains Http data (method, version, headers ...) but im only interested in body. What am i doing wrong? Thanks!
That's super strange... msg.content() is a ByteBuf which only should have the payload of the request / response included.
Turns out this was caused by testing this against echo server which repeats the whole http request back, as proven by:
$ curl -X POST localhost:3000/a/b/c -v
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 3000 (#0)
> POST /a/b/c HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sat, 19 Dec 2020 09:50:56 GMT
< X-Http-Echo-Server-Id: 9fcf5d57-2ac7-4a92-8aed-775879bf3ac2
< Content-Type: text/plain;charset=utf-8
< Transfer-Encoding: chunked
< Server: Jetty(9.4.24.v20191120)
<
POST /a/b/c HTTP/1.1
Accept: */*
User-Agent: curl/7.64.1
Host: localhost:3000
* Connection #0 to host localhost left intact
* Closing connection 0
Related
i have this code to connect to my NTRIP Server,
String requestmsg = "GET /" + nMountpoint + " HTTP/1.0\r\n";
requestmsg += "User-Agent: NTRIP Client\r\n";
requestmsg += "Accept: */*\r\n";
requestmsg += "Connection: close\r\n";
if (nUsername.length() > 0) {
requestmsg += "Authorization: Basic " + ToBase64(nUsername + ":" + nPassword);
}
requestmsg += "\r\n";
os = nsocket.getOutputStream();
os.write(requestmsg.getBytes());
and got this reply, that indicate that i was connected to the server
ICY 200 OK
Server: GNSS Spider 7.7.0.9065/1.0
Date: Wed, 09 Feb 2022 13:25:27 GMT
but After I try to send my String message:
private String RecentGGA = "$GPGGA,154223,4000,N,08312,W,4,10,1,200,M,1,M,8,0*7F";
Using this Code:
private void SendGGAToServer() {
SendDataToNetwork(RecentGGA + "\r\n");
//Log.d(TAG, "SendGGAToCaster: GGA Data Sent to Caster");
}
public void SendDataToNetwork(String NewGGA) { // You run this from the main thread.
try {
if (nsocket != null) {
if (nsocket.isConnected()) {
if (!nsocket.isClosed()) {
Log.i("SendDataToNetwork", "SendDataToNetwork: Writing message to socket");
os.write(NewGGA.getBytes());
} else {
Log.i("SendDataToNetwork", "SendDataToNetwork: Cannot send message. Socket is closed");
}
} else {
Log.i("SendDataToNetwork", "SendDataToNetwork: Cannot send message. Socket is not connected");
}
}
} catch (Exception e) {
Log.i("SendDataToNetwork", "SendDataToNetwork: Message send failed. Caught an exception " + e);
}
}
The Problem is, i can not send the string and shows this error:
I/NTRIP_Service: Creating socket
I/System.out: [socket]:check permission begin!
W/System: ClassLoader referenced unknown path: system/framework/mediatek-cta.jar
I/System.out: [socket] e:java.lang.ClassNotFoundException: com.mediatek.cta.CtaUtils
I/NTRIP_Service: Socket created, streams assigned
This is a NTRIP connection
I/Request: GET /max-rtcm3 HTTP/1.0
User-Agent: NTRIP Client
Accept: /
Connection: close
Authorization: Basic R2VvMTpHZW8x
D/NTRIP_Service: Data Mode: 0
D/NTRIP: ParseNetworkDataStream: ICY 200 OK
Server: GNSS Spider 7.7.0.9065/1.0
Date: Wed, 09 Feb 2022 13:25:27 GMT
I/SendDataToNetwork: SendDataToNetwork: Writing message to socket
SendDataToNetwork: Message send failed. Caught an exception android.os.NetworkOnMainThreadException
i'm still a newbie, so i dont know how to fix this. does anyone know how to fix this?
null: [HTTP/1.1 405 HTTP method
PUT is not supported by this
URL]
Server:
[Jetty(6.1.26.cloudera.4)]
Pragma: [no-cache]
Content-Length: [0]
X-FRAME-OPTIONS: [SAMEORIGIN]
Date: [Wed, 13 Oct 2021
17:05:16 GMT]
'''
private static void putTest() throws IOException {
System.out.println("putTest : Entry");
URL url = new URL("http://localhost:50070/webapps/v1/longdata.txt&op=CREATE");
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("PUT");
con.setDoOutput(true);
con.getOutputStream().write("Hey Guys".getBytes(StandardCharsets.UTF_8));
Map<String, List<String>> headers = con.getHeaderFields();
for (String header : headers.keySet())
{
System.out.println(header + ": " + headers.get(header));
}
System.out.println("putTest : Exit");
}
'''
You need two requests - one to the namenode port, which creates the file on the namenode, then that returns a datanode address where you send the second address to write the content stream.
However, you are using Java, so just use hadoop-client dependency with the FileSystem.create method, do not use HttpURLConnection
I'm trying to post an entity to a webservice, here the code:
EmptyTagWrapper wrap = webResource.get(EmptyTagWrapper.class);
client = Client.create();
client.addFilter(new LoggingFilter(System.out));
webResource = client.resource(MyWebservice.CREATETAGS.getUrl());
ClientResponse clientResponse = webResource.post(ClientResponse.class, wrap);
System.out.println(clientResponse);
If I try to do this manually it works:
http://www.mysite.de/api/tags/?xml=<?xml version="1.0" encoding="UTF-8"?><prestashop xmlns:xlink="http://www.w3.org/1999/xlink"><tag><name>Testtag</name><id_lang>1</id_lang></tag></prestashop>
but if I run my program, I get this error:
<?xml version="1.0" encoding="UTF-8"?>
<prestashop xmlns:xlink="http://www.w3.org/1999/xlink">
<errors>
<error>
<code><![CDATA[127]]></code>
<message><![CDATA[XML error : String could not be parsed as XML
XML length : 0
Original XML : ]]></message>
</error>
</errors>
</prestashop>
I think the reason is the linebreak that is appendet between the URL and the marshaled entity, here the sysout of the post-request:
1 * Client out-bound request
1 > POST http://www.mysite.de/api/tags/?xml=
<?xml version="1.0" encoding="UTF-8"?><prestashop xmlns:xlink="http://www.w3.org/1999/xlink"><tag><name>Testtag</name><id_lang>1</id_lang></tag></prestashop>
1 * Client in-bound response
1 < 500
1 < Execution-Time: 0.006
1 < Server: Apache
1 < Access-Time: 1404134558
1 < Connection: close
1 < Vary: Host
1 < PSWS-Version: 1.5.4.1
1 < Content-Length: 282
1 < Date: Mon, 30 Jun 2014 13:22:38 GMT
1 < Content-Type: text/xml;charset=utf-8
1 < X-Powered-By: PrestaShop Webservice
1 <
Any Ideas how to solve this?
Background
I've got different services and fields which are constructed depending on the entity and the service:
public enum PrestaWebservice {
PRODUCTLINKS("Key1","products"),
FULLPRODUCTLIST("Key1","products?display=["+PrestaProduct.FIELDS+"]"),
CATEGORIESLIST("Key1","categories?display=["+PrestaCategory.FIELDS+"]"),
CUSTOMERSLIST("Key2","customers?display=["+PrestaCustomer.FIELDS+"]"),
TAGSLIST("Key2","tags?display=full"),
EMPTYTAG("Key2","tags?schema=synopsis"),
CREATETAGS("Key2","tags/?xml=");
private final String SHOPADDRESS="http://www.mySite.de/api";
private String key;
private String url;
private PrestaWebservice(String key, String url){
this.key = key;
this.url = url;
}
public String getKey(){
return key;
}
public String getUrl(){
return SHOPADDRESS+"/"+url;
}
}
In your question the URL is being constructed in your own code. You need to example in the following area why the carriage return is being added.
MyWebservice.CREATETAGS.getUrl()
Is there a reason you are send the XML as the value of a query parameter instead of in the body of the HTTP request?
I am trying to read the Status header of a bounced email. This site explains better what I am trying...
The original email is composed by several MultiParts objects, so I am reading it in java code:
private void test(MimeMessage message) throws IOException, MessagingException {
if (message.getContent() != null && message.getContent() instanceof Multipart) {
Multipart content = (Multipart) message.getContent();
for (int i = 0; i < content.getCount(); i++) {
BodyPart bodyPart = content.getBodyPart(i);
Enumeration headers = bodyPart.getAllHeaders();
while(headers.hasMoreElements()){
Header header = (Header) headers.nextElement();
LOGGER.info("Header: " + header.getName() + " value: " + header.getValue());
}
}
}
}
The email part I am analyzing:
Content-Description: Delivery report Content-Type: text/plain;
charset=utf-8 Content-Transfer-Encoding: 7bit
Reporting-MTA: dns; someLink.com
X-Postfix-Queue-ID: EC862F00D0 X-Postfix-Sender: rfc822;
receiver#email.com Arrival-Date: Wed, 7 Aug 2013
13:52:43 +0200 (CEST)
Final-Recipient: rfc822; noexisting#email.com
Original-Recipient: rfc822;noexisting#email.com Action:
failed Status: 5.1.1 Remote-MTA: dns; [somelink.com
Diagnostic-Code: smtp; 550-5.1.1 The email account that you tried to
reach does
not exist. Please try 550-5.1.1 double-checking the recipient's email
address for typos or 550-5.1.1 unnecessary spaces.
In my log file I can see only the 3 first headers:
> Header: Content-Description value: Delivery report
> Header: Content-Type value: text/plain; charset=us-ascii INFO
> Header: Content-Transfer-Encoding value: 7bit
Does anyone know why? How could I get the status header? Thanks
I couldn´t find the Status information in the header, and I will take it from the content. It is not an elegant solution, but at least it works.
If someone finds a better one, please let me know!
Java code:
StringWriter writer = new StringWriter();
IOUtils.copy(bodyPart.getInputStream(), writer);
LOGGER.info("Content inputstream: " + writer.toString());
Logs:
Content inputstream: Reporting-MTA: dns; srvvie-mx3.styria-multi-media.com
X-Postfix-Queue-ID: 2A1A8F00CF X-Postfix-Sender: rfc822;
Arrival-Date: Fri, 9 Aug 2013
11:14:02 +0200 (CEST)
Final-Recipient: rfc822; MAILER-DAEMON#domain.com
Original-Recipient: rfc822;MAILER-DAEMON#domain.com
Action: failed Status: 5.1.1 Remote-MTA: dns;
Diagnostic-Code: smtp; 550 5.1.1 Mailbox
does not exist
I am attempting to upload images to SmugMug via HTTP Post as per their documentation. I have all the headers correct, but the part that is confusing me is setting the binary data in the body as stated:
This method requires a POST request with the binary data in the body
and all other metadata in the headers.
I have tried:
SMResponse response = builder.post(SMResponse.class, Files.readAllBytes(image.toPath()));
SMResponse response = builder.post(SMResponse.class, new String(Files.readAllBytes(image.toPath())));
SMResponse response = builder.post(SMResponse.class, new String(Base64.encode(Files.readAllBytes(image.toPath()))));
SMResponse response = builder.post(SMResponse.class, Base64.encode(Files.readAllBytes(image.toPath())));
My best guess was the first one would work, but all of these return:
{"stat":"fail","method":"smugmug.images.upload","code":5,"message":"system error"}
Here is the full method that does the uploading, in case I missed something:
public boolean upload(File image, int albumId, String caption, String keywords,
Boolean hidden, Integer imageId, Integer altitude, Float latitude,
Float longitude, boolean pretty) throws IOException, InvalidKeyException, NoSuchAlgorithmException, SmugMugException {
logger.debug("upload() called");
byte[] imageBytes = Files.readAllBytes(image.toPath());
WebResource resource = SmugMugAPI.CLIENT.resource("http://upload.smugmug.com/");
LoggingFilter logFilter = new LoggingFilter();
resource.addFilter(logFilter);
OAuthSecrets secrets = new OAuthSecrets().consumerSecret(smugmug.getConsumerSecret());
OAuthParameters oauthParams = new OAuthParameters().consumerKey(smugmug.getCosumerKey()).
signatureMethod("HMAC-SHA1").version("1.0");
// Create the OAuth client filter
OAuthClientFilter filter = new OAuthClientFilter(SmugMugAPI.CLIENT.getProviders(), oauthParams, secrets);
// Add the filter to the resource
if (smugmug.getToken() != null){
secrets.setTokenSecret(smugmug.getToken().getSecret());
oauthParams.token(smugmug.getToken().getId());
}
resource.addFilter(filter);
WebResource.Builder builder = resource.getRequestBuilder();
//User agent
builder = builder.header("User-Agent", smugmug.getAppName());
//API Version header
builder = builder.header("X-Smug-Version", "1.3.0");
//Response Type header
builder = builder.header("X-Smug-ResponseType", "JSON");
//Content-Length header
builder = builder.header("Content-Length", Long.toString(image.length()));
//Content-MD5 header
builder = builder.header("Content-MD5", DigestUtils.md5Hex(imageBytes));
//X-Smug-FileName header
builder = builder.header("X-Smug-FileName", image.getName());
//X-Smug-AlbumID header
builder = builder.header("X-Smug-AlbumID", Integer.toString(albumId));
//X-Smug-Caption header
if(caption != null){
builder = builder.header("X-Smug-Caption", caption);
}
//X-Smug-Caption header
if(keywords != null){
builder = builder.header("X-Smug-Keywords", keywords);
}
//X-Smug-Hidden header
if(hidden != null){
builder = builder.header("X-Smug-Hidden", hidden.toString());
}
//X-Smug-ImageID header
if(imageId != null){
builder = builder.header("X-Smug-ImageID", imageId.toString());
}
//X-Smug-Altitude header
if(altitude != null){
builder = builder.header("X-Smug-Altitude", altitude.toString());
}
//X-Smug-Latitude header
if(latitude != null){
builder = builder.header("X-Smug-Latitude", latitude.toString());
}
//X-Smug-Latitude header
if(longitude != null){
builder = builder.header("X-Smug-Longitude", longitude.toString());
}
//X-Smug-Pretty header
if(pretty){
builder = builder.header("X-Smug-Pretty", Boolean.toString(pretty));
}
SMResponse response = builder.post(SMResponse.class, new String(imageBytes));
if (!"ok".equals(response.getStat())) {
throw new SmugMugException(response);
}
return true;
}
Where have I gone wrong?
Tried just to see the response:
SMResponse response = builder.entity(image).post(SMResponse.class);
It actually sent back a blank response (no json) which is odd in itself, as I would have expected some message back. Here is the output:
Nov 21, 2012 11:55:48 PM com.sun.jersey.api.client.filter.LoggingFilter log
INFO: 1 * Client in-bound response
1 < 200
1 < Edge-Control: no-store
1 < X-SmugMug-Hiring: How to love what you do: http://www.smugmug.com/jobs/
1 < Date: Thu, 22 Nov 2012 05:55:48 GMT
1 < Content-Length: 0
1 < X-SmugMug-Values: 4/4 - It's the product, stupid
1 < Expires: Thu, 22 Nov 2012 05:55:49 GMT
1 < Connection: keep-alive
1 < Content-Type: application/json; charset=utf-8
1 < X-Powered-By: SmugMug/0.9
1 < Server: Apache
1 < Cache-Control: private, no-store, no-cache, max-age=1, must-revalidate
1 <
I am not exactly sure what happened, but I was able to get it working after finding the Upload Log in the SmugMug Account Settings (To get there go to Tools -> Account Settings -> Stats -> Uploads -> Details). Note that in the upload log there is a toggle to show only errors or all uploads.
Now on to the actual answer to how to set the actual "body" of the Post Request. The actual format should have been the first one I posted:
SMResponse response = builder.post(SMResponse.class, Files.readAllBytes(image.toPath()));
So either I messed up, and thought it was not working when it was, there was a problem on smugmug's end at the time, or there was something else in my code that was wrong, that got fixed in the process of me trying to fix this non-issue.