We Keep Coding

How receive xml response from SOAP endpoint?

My problem is that I have one endpoint url which if I put to the browser I can see the website. When I use this endpoint in postman program with correct request (xml) I am receiving xml response. Which is what I want. But I am trying to receive the same response in my java application but what I am receiving is the website(html).
I am not sure what is wrong
JAXBContext jaxbContext = JAXBContext.newInstance(Envelope.class);
Marshaller jaxbMarshaler = jaxbContext.createMarshaller();
jaxbMarshaler.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
Date now = new Date();
SimpleDateFormat formater = new SimpleDateFormat("ddMMyyhhmmss");
fileName = "EMCS_" + tNumber.trim() + "_" + formater.format(now) + "_" + obtDto.getTransactionIdHost()
+ ".in";
jaxbMarshaler.marshal(envelope, new File("C:\\Temp\\" + fileName));
url = new URL(urlString);
connection = (HttpsURLConnection) url.openConnection();
connection.setUseCaches(false);
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setRequestProperty("Connection", "Keep-Alive");
connection.setRequestProperty("Accept-Encoding", "gzip,deflate");
connection.setRequestProperty("Accept", "*/*");
connection.setRequestProperty("SOAPAction", "");
connection.setRequestProperty("Cache-Control", "no-cache");
connection.setRequestProperty("Authorization",
"Basic " + Base64.encode("USER:PASSWORD".getBytes()));
connection.setRequestProperty("Content-Type", "text/xml");
BufferedOutputStream dos = new BufferedOutputStream(connection.getOutputStream());
File file = new File("C:\\Temp\\" + fileName);
BufferedInputStream fileOutput = new BufferedInputStream(new FileInputStream(file));
byte[] b = new byte[(int) file.length()];
for (int i = 0; i < b.length; i++) {
b[i] = (byte) fileOutput.read();
System.out.printf((char) b[i] + "");
}
dos.write(b);
dos.flush();
dos.close();
fileOutput.close();
System.out.println("RESPONSE: " + connection.getResponseCode());
BufferedOutputStream responseWebsite = new BufferedOutputStream(
new FileOutputStream("C:\\Temp\\response.html"));
InputStream in = url.openStream(); // IMPORTANT TO READ PROPERLY DATA
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
StringBuilder result2 = new StringBuilder();
String line;
while ((line = reader.readLine()) != null) {
responseWebsite.write(line.getBytes());
result2.append(line + "\n");
}
responseWebsite.close();
By using code above I am posting request and getting HTML response instead of XML. What I am doing wrong?
EDIT
I edited post cause I need give more explanation I think. My problem is that in
'reader' variable bytes which I am receiving are HTML not XML. I want XML.
EDIT2
So I can not give a link to repo cause is not me private project. But when I am using postman and sending request to endpoint with
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:de.aeb.xnsg.emcs.bf" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body><ns2:request>...
body which I genereate with my app I am receiving
<?xml version='1.0' encoding='UTF-8'?>
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"><S:Body><ns2:response>...
And that is good.
And response in postman is pure XML which I want.
But when I put this endpoint to chrome for example I am receiving website where I can find other endpoints/wsdls to this service but in different language(French, German etc).
EDIT 3
SOLVED
My problem was that I have specified Accept-Encoding
connection.setRequestProperty("Accept-Encoding", "gzip,deflate");
when I removed it I could read proper response from stream.
But also I was able to read this message by using GZIPInputStream, but not sure which solution is better.

Maybe problem is in this line new FileOutputStream("C:\\Temp\\response.html"));?and you should change extension to response.xml

Remote file transfer using java nio

I am trying to use JAVA NIO to transfer a file from host A to client B without having to download the file locally and then giving the client B a link to download the file.
I am running a spark Apache framework and using maven project.
I mapped the request http://localhost:8080/download/hello in Spark using :
get("/download/:id",RequestHandler::downloadHandler);
Inside of the function is the code that downloads the file from :
"https://download.springsource.com/release/STS/3.8.1.RELEASE/dist/e4.6/spring-tool-suite-3.8.1.RELEASE-e4.6-linux-gtk-x86_64.tar.gz"
try {
URL url = new URL("https://download.springsource.com/release/STS/3.8.1.RELEASE/dist/e4.6/spring-tool-suite-3.8.1.RELEASE-e4.6-linux-gtk-x86_64.tar.gz");
HttpURLConnection httpURLConnection = (HttpURLConnection) url.openConnection();
int respCode = +httpURLConnection.getResponseCode();
System.out.println("response code : "+respCode);
if (respCode == HttpURLConnection.HTTP_OK){
String fileName = "";
String disposition = httpURLConnection.getHeaderField("Content-Disposition");
String contentType = httpURLConnection.getContentType();
int contentLength = httpURLConnection.getContentLength();
if (disposition != null) {
// extracts file name from header field
int index = disposition.indexOf("filename=");
if (index > 0) {
fileName = disposition.substring(index + 10,
disposition.length() - 1);
}
} else {
// extracts file name from URL
fileName = url.toString().substring(url.toString().lastIndexOf("/") + 1,
url.toString().length());
}
System.out.println("Content-Type = " + contentType);
System.out.println("Content-Disposition = " + disposition);
System.out.println("Content-Length = " + contentLength);
System.out.println("fileName = " + fileName);
httpURLConnection.disconnect();
System.out.println("other stuff : ");
System.out.println(url.getHost());
ReadableByteChannel readableByteChannel = Channels.newChannel(url.openStream());
FileOutputStream fileOutputStream = new FileOutputStream(fileName);
FileChannel fileChannel = fileOutputStream.getChannel();
fileChannel.transferFrom(readableByteChannel, 0, Long.MAX_VALUE);
fileOutputStream.close();
readableByteChannel.close();
}
} catch (IOException e) {
e.printStackTrace();
}
I fetch the filename and file size using httpURLConnection and then processed to download the file. what I am trying to do is, instead of downloading the file locally using fileChannel.transferFrom(readableByteChannel, 0, Long.MAX_VALUE) transfer the file directly to the client.
I did some research and I think it is possible with using Socketchannels but I didn't understand how it is supposed to work.
I also read this article
https://examples.javacodegeeks.com/core-java/nio/java-nio-large-file-transfer-tutorial/
and tried to understand the class Reciever, but it is still not clear to me how.
I would appreciate some guidance. Thank you

How to convert HTML String to PDF using ConvertAPI (without a physical file)

Previously I was using http://do.convertapi.com/Web2Pdf to convert HTML String to PDF using a simple GET request (not POST) using Java. The entire content was passed using curl parameter.
However, that API seems to have stopped working recently. I'm trying to port over to https://v2.convertapi.com/web/to/pdf but I cannot find a sample to do the same using the new API either with GET or POST.
Can someone provide an example to make a GET or POST request using Java?
UPDATE: I have managed to make it work.
private static final String WEB2PDF_API_URL = "https://v2.convertapi.com/html/to/pdf";
private static final String WEB2PDF_SECRET = "secret-here";
String htmlContent = "valid HTML content here";
URL apiUrl = new URL(WEB2PDF_API_URL + "?secret=" + WEB2PDF_SECRET + "&download= attachment&PageOrientation=landscape&MarginLeft=0&MarginRight=0&MarginTop=0&MarginBottom=0");
HttpURLConnection connection = null;
ByteArrayOutputStream buffer = null;
connection = (HttpURLConnection) apiUrl.openConnection();
connection.setRequestProperty("Content-Disposition", "attachment; filename=\"data.html\"");
connection.setRequestProperty("Content-Type", "application/octet-stream");
connection.setRequestMethod("POST");
connection.setConnectTimeout(60000);
connection.setReadTimeout(60000);
connection.setDoOutput(true);
/* write request */
OutputStreamWriter writer = new OutputStreamWriter(connection.getOutputStream());
writer.write(htmlContent);
writer.flush();
writer.close();
/* read response */
String responseMessage = connection.getResponseMessage();
logger.info("responseMessage: " + responseMessage);
int statusCode = connection.getResponseCode();
logger.info("statusCode: " + statusCode);
if (statusCode == HttpURLConnection.HTTP_OK) {
logger.info("HTTP status code OK");
// parse output
InputStream is = connection.getInputStream();
buffer = new ByteArrayOutputStream();
int nRead;
byte[] data = new byte[16384];
while ((nRead = is.read(data, 0, data.length)) != -1) {
buffer.write(data, 0, nRead);
}
buffer.flush();
byte[] attachmentData = buffer.toByteArray();
Multipart content = new MimeMultipart();
...
MimeBodyPart attachment = new MimeBodyPart();
InputStream attachmentDataStream = new ByteArrayInputStream(attachmentData);
attachment.setFileName("filename-" + Long.toHexString(Double.doubleToLongBits(Math.random())) + ".pdf");
attachment.setContent(attachmentDataStream, "application/pdf");
content.addBodyPart(attachment);
...
}

You can easily push HTML string as file. I do not have JAVA example but the C# demo will give you right path.
using System;
using System.IO;
using System.Net.Http;
using System.Text;
class MainClass {
public static void Main (string[] args) {
var url = new Uri("https://v2.convertapi.com/html/to/pdf?download=attachment&secret=<YourSecret>");
var htmlString = "<!doctype html><html lang=en><head><meta charset=utf-8><title>ConvertAPI test</title></head><body>This page is generated from HTML string.</body></html>";
var content = new StringContent(htmlString, Encoding.UTF8, "application/octet-stream");
content.Headers.Add("Content-Disposition", "attachment; filename=\"data.html\"");
using (var resultFile = File.OpenWrite(#"C:\Path\to\result\file.pdf"))
{
new HttpClient().PostAsync(url, content).Result.Content.CopyToAsync(resultFile).Wait();
}
}
}

Why does when sending a file in multipart/form-data in java we have to use both a Writer and a OutputStream?

Hi I have seen many sample codes working for sending a file in multipart/form-data in java.
But they have used both Writer and an OutputStream.
Why can't they use just use one of them?
Here is the sample code they have sent
import java.io.*;
import java.net.HttpURLConnection;
import java.net.URL;
public class MainClass_External2 {
public static void main(String[] args){
try{
// Connect to the web server endpoint
URL serverUrl =
new URL("http://posttestserver.com/post.php?dir=example");
HttpURLConnection urlConnection = (HttpURLConnection)serverUrl.openConnection();
String boundaryString = "----SomeRandomText";
String fileUrl = "abc.txt";
File logFileToUpload = new File(fileUrl);
// Indicate that we want to write to the HTTP request body
urlConnection.setDoOutput(true);
urlConnection.setRequestMethod("POST");
urlConnection.addRequestProperty("Content-Type","multipart/form-data; boundary=" + boundaryString);
// Indicate that we want to write some data as the HTTP request body
urlConnection.setDoOutput(true);
OutputStream outputStreamToRequestBody = urlConnection.getOutputStream();
BufferedWriter httpRequestBodyWriter =
new BufferedWriter(new OutputStreamWriter(outputStreamToRequestBody));
// Include value from the myFileDescription text area in the post data
httpRequestBodyWriter.write("\n\n--" + boundaryString + "\n");
httpRequestBodyWriter.write("Content-Disposition: form-data; name=\"myFileDescription\"");
httpRequestBodyWriter.write("\n\n");
httpRequestBodyWriter.write("Log file for 20150208");
// Include the section to describe the file
httpRequestBodyWriter.write("\n--" + boundaryString + "\n");
httpRequestBodyWriter.write("Content-Disposition: form-data;"
+ "name=\"myFile\";"
+ "filename=\""+ logFileToUpload.getName() +"\""
+ "\nContent-Type: text/plain\n\n");
httpRequestBodyWriter.flush();
// Write the actual file contents
FileInputStream inputStreamToLogFile = new FileInputStream(logFileToUpload);
int bytesRead;
byte[] dataBuffer = new byte[1024];
while((bytesRead = inputStreamToLogFile.read(dataBuffer)) != -1){
outputStreamToRequestBody.write(dataBuffer, 0, bytesRead);
}
// Mark the end of the multipart http request
httpRequestBodyWriter.write("\n--" + boundaryString + "--\n");
httpRequestBodyWriter.flush();
// Close the streams
outputStreamToRequestBody.close();
httpRequestBodyWriter.close();
// Read response from web server, which will trigger the multipart HTTP request to be sent.
BufferedReader httpResponseReader =
new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
String lineRead;
while((lineRead = httpResponseReader.readLine()) != null) {
System.out.println(lineRead);
}
}catch (Exception e){
e.printStackTrace();
}
}
}

Basically, the response contains both text and binary data, so using both a Writer and an OutputStream makes perfect sense.
The writer just wraps the output stream, and is used to write text. The output stream itself is used to write the binary data.
Why can't they use just use one of them?
Using just an OutputStream would make it more painful to write the text. Using just a Writer would be inappropriate when binary data needs to be written.

Image writing over URLConnection

I am trying to write an image over an HttpURLConnection.
I know how to write text but I am having real problems trying
to write an image
I have succeeded in writing to the local HD using ImageIO:
But I am trying to write Image by ImageIO on url and failed
URL url = new URL(uploadURL);
connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setDoInput(true);
connection.setUseCaches(false);
connection.setRequestProperty("Content-Type", "multipart/form-data;
boundary=" + boundary);
output = new DataOutputStream(connection.getOutputStream());
output.writeBytes("--" + boundary + "\r\n");
output.writeBytes("Content-Disposition: form-data; name=\"" + FIELD_NAME + "\";
filename=\"" + fileName + "\"\r\n");
output.writeBytes("Content-Type: " + dataMimeType + "\r\n");
output.writeBytes("Content-Transfer-Encoding: binary\r\n\r\n");
ImageIO.write(image, imageType, output);
the uploadURL is the url to an asp page on the server which will upload the image with the file name given in "content-Disposition: part.
now when I send this then asp page find the request and find the name of file. but does not find the file to be uploaded.
The problem is that when writing by ImageIO on URL what will the name of the file on which the ImageIO is writing,
So please help me how ImageIO will write an image on URLConnection and how can I know the name of the file which I have to use in the asp page to upload the file
Thanks for taking the time to read this post
Dilip Agarwal

First I believe that you should call io.flush() and then io.close() after writing image.
Second content type seems strange for me. It seems that you are trying to submit form while it is actually image. I do not know what does your asp expect but typically when I write code that should transfer file over HTTP I send appropriate content type, e.g. image/jpeg.
Here is for example code snippet I extracted from one small utility that I wrote and I am using during my current work:
URL url = new URL("http://localhost:8080/handler");
HttpURLConnection con = (HttpURLConnection)url.openConnection();
con.setDoInput(true);
con.setDoOutput(true);
con.setUseCaches(false);
con.setRequestProperty("Content-Type", "image/jpeg");
con.setRequestMethod("POST");
InputStream in = new FileInputStream("c:/temp/poc/img/mytest2.jpg");
OutputStream out = con.getOutputStream();
copy(in, con.getOutputStream());
out.flush();
out.close();
BufferedReader r = new BufferedReader(new InputStreamReader(con.getInputStream()));
// obviously it is not required to print the response. But you have
// to call con.getInputStream(). The connection is really established only
// when getInputStream() is called.
System.out.println("Output:");
for (String line = r.readLine(); line != null; line = r.readLine()) {
System.out.println(line);
}
I used here method copy() that I took from Jakarta IO utils. Here is the code for reference:
protected static long copy(InputStream input, OutputStream output)
throws IOException {
byte[] buffer = new byte[12288]; // 12K
long count = 0L;
int n = 0;
while (-1 != (n = input.read(buffer))) {
output.write(buffer, 0, n);
count += n;
}
return count;
}
Obviously the server side must be ready to read the image content directly from POST body.
I hope this helps.

The OP seems lost into oblivion but for the benefit of Mister Kite :
// main method
URL url = new URL(uploadURL);
connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true); // triggers "POST"
// connection.setDoInput(true); // only if needed
connection.setUseCaches(false); // dunno
final String boundary = Long.toHexString(System.currentTimeMillis());
connection.setRequestProperty("Content-Type", "multipart/form-data; boundary="
+ boundary);
output = new DataOutputStream(connection.getOutputStream());
try {
// image must be a File instance
flushMultiPartData(image, output, boundary);
} catch (IOException e) {
System.out.println("IOException in flushMultiPartData : " + e);
return;
}
// ...
private void flushMultiPartData(File file, OutputStream serverOutputStream,
String boundary) throws FileNotFoundException, IOException {
// SEE https://stackoverflow.com/a/2793153/281545
PrintWriter writer = null;
try {
// true = autoFlush, important!
writer = new PrintWriter(new OutputStreamWriter(serverOutputStream,
charsetForMultipartHeaders), true);
appendBinary(file, boundary, writer, serverOutputStream);
// End of multipart/form-data.
writer.append("--" + boundary + "--").append(CRLF);
} finally {
if (writer != null) writer.close();
}
}
private void appendBinary(File file, String boundary, PrintWriter writer,
OutputStream output) throws FileNotFoundException, IOException {
// Send binary file.
writer.append("--" + boundary).append(CRLF);
writer.append(
"Content-Disposition: form-data; name=\"binaryFile\"; filename=\""
+ file.getName() + "\"").append(CRLF);
writer.append("Content-Type: "
+ URLConnection.guessContentTypeFromName(file.getName()))
.append(CRLF);
writer.append("Content-Transfer-Encoding: binary").append(CRLF);
writer.append(CRLF).flush();
InputStream input = null;
try {
input = new FileInputStream(file);
byte[] buffer = new byte[1024];
for (int length = 0; (length = input.read(buffer)) > 0;) {
output.write(buffer, 0, length);
}
output.flush(); // Important! Output cannot be closed. Close of
// writer will close output as well.
} finally {
if (input != null) try {
input.close();
} catch (IOException logOrIgnore) {}
}
writer.append(CRLF).flush(); // CRLF is important! It indicates end of
// binary boundary.
}
You may want to add Gzip compression - see file corrupted when I post it to the servlet using GZIPOutputStream for a working class with or without Gzip. The ImageIO has no place here - just write the bytes past the wire and use ImageIO to your heart's content on the server. Based on #BalusC answer