We Keep Coding

How to export utf-8 content in octet stream response for REST API endpoint?

I have a Quarkus based REST API project in which one endpoint is supposed to serve exported data as .csv files. Since i do not want to create temporary files, i was writing to a ByteArrayInputStream to be used in an octet stream response for my webservice.
However, although this works fine for latin character content we also have content that may be in Chinese. The downloaded .csv file does not view the characters properly or rather does not write them properly (they only show up as question marks, even in plain text view e.g. with notepad).
We already checked the source of the problem not being how the data is stored, for example the encoding in the database is correct and it works fine when we export it as .json (here we can set charset utf-8).
As far as i understand a charset or encoding cannot be set for an octet stream.
So how can we export/stream this content as a file download without creating an actual file?
Some code examples below on how we do it currently. We use the apache common library component CSVPrinter to create the CSV format in text in a custom CSV streamer class:
#ApplicationScoped
public class JobRunDataCsvStreamer implements DataFormatStreamer<JobData> {
#Override
public ByteArrayInputStream streamDataToFormat(List<JobData> dataList) {
try {
ByteArrayOutputStream out = getCsvOutputStreamFor(dataList);
return new ByteArrayInputStream(out.toByteArray());
} catch (IOException e) {
throw new RuntimeException("Failed to convert job data: " + e.getMessage());
}
}
private ByteArrayOutputStream getCsvOutputStreamFor(List<JobData> dataList) throws IOException {
ByteArrayOutputStream out = new ByteArrayOutputStream();
CSVPrinter csvPrinter = new CSVPrinter(new PrintWriter(out), getHeaderFormat());
for (JobData jobData : dataList) {
csvPrinter.printRecord(extractStringRowData(jobData));
}
csvPrinter.flush();
csvPrinter.close();
return out;
}
private CSVFormat getHeaderFormat() {
return CSVFormat.EXCEL
.builder()
.setDelimiter(";")
.setHeader("ID", "Source term", "Target term")
.build();
}
private List<String> extractStringRowData(JobData jobData) {
return Arrays.asList(
String.valueOf(jobData.getId()),
jobData.getSourceTerm(),
jobData.getTargetTerm()
);
}
}
Here is the quarkus API endpoint for the download:
#Path("/jobs/data")
public class JobDataResource {
#Inject JobDataRepository jobDataRepository;
#Inject JobDataCsvStreamer jobDataCsvStreamer;
...
#GET
#Path("/export/csv")
#Produces(MediaType.APPLICATION_OCTET_STREAM)
public Response getAllAsCsvExport() {
List<JobData> jobData = jobDataRepository.getAll();
ByteArrayInputStream stream = jobDataCsvStreamer.streamDataToFormat(jobData);
return Response.ok(stream, MediaType.APPLICATION_OCTET_STREAM)
.header("content-disposition", "attachment; filename = job-data.csv")
.build();
}
}
Screenshot of result in the downloaded file for chinese characters in the second column:
We tried setting headers etc. for encoding, but none of it worked. Is there a way to stream content which requires specific encoding as a file in Java web services? We tried using PrintWriter which works, but requies creating a local file on the server.
Edit: We tried using PrintWriter(out, false, StandardCharsets.UTF_8) for the PrintWriter to write to a byte array out stream for the response, which yields a different result but still with broken view in both Excel and plain text:
Screenshot:
Code for endpoint:
#GET
#Path("/export/csv")
#Produces(MediaType.APPLICATION_OCTET_STREAM)
public Response getAllAsCsvExport() {
List<JobData> jobData = jobRunDataRepository.getAll();
ByteArrayOutputStream out = new ByteArrayOutputStream();
try{
PrintWriter pw = new PrintWriter(out, false, StandardCharsets.UTF_8);
pw.println(String.format("%s, %s, %s", "ID", "Source", "Target"));
for (JobData item : jobData) {
pw.println(String.format("%s, %s, %s",
String.valueOf(item.getId()),
String.valueOf(item.getSourceTerm()),
String.valueOf(item.getTargetTerm()))
);
}
pw.flush();
pw.close();
} catch (Exception e) {
throw new RuntimeException("Failed to convert job data: " + e.getMessage());
}
return Response.ok(out).build();
}

How to display DynamicReports in browser without download to cliend drive?

I have to display some reports with Dynamic Reports. I use NetBeans and Tomcat 7. Eventually, all must be uploaded to cloud OpenShift. I used DynamicReports to create simple report (code snippet):
Connection conn=null;
try {
Class.forName(DBConnStrings.driver);
conn = DriverManager.getConnection(DBConnStrings.url + DBConnStrings.dbName+DBConnStrings.sslState, DBConnStrings.userName, DBConnStrings.password);
} catch (Exception e) {
e.printStackTrace();
}
JasperReportBuilder report = DynamicReports.report();
report
.columns(
Columns.column("Tank Id", "id", DataTypes.integerType()),
Columns.column("Tank Name", "name", DataTypes.stringType()),
Columns.column("Label", "label", DataTypes.stringType()),
Columns.column("Description", "descrshort", DataTypes.stringType()));
report.setDataSource("SELECT id, name, label, descrshort FROM "+ DBConnStrings.dbName +".tbltankslist", conn);
try {
//show the report
//report.show();
//export the report to a pdf file
report.toPdf(new FileOutputStream("c:/report.pdf"));
} catch (DRException e) {
e.printStackTrace();
} catch (FileNotFoundException e) {
e.printStackTrace();
}
This code located in a Servlet. It works. I get JasperViewer at first and a report.pdf on my HDD. But I don't want it. First I do not want to see JasperViewer, second I do not want to download file to client HDD. How to display report inside web-browser only?
Here is the question Jasper Reports. It is about jasper reports + iReport and I have no idea how to use that information for DynamicReports - at first, second there is also "download pdf to client drive" approach, but I need to show it inside the browser.

use the following code in your file which redirect towards jasper invocation page, so that your jasperPDF should open in new tab instead of downloading.
JasperInvocation.jsp => file in which you invoke jasperReport
<form method="POST" action="JasperInvocation.jsp" target="_blank">

Please find following code , I have implemented in Dynamic report(Jasper Api) , Its working for me :-
#RequestMapping(value="/pdfDownload", method = RequestMethod.GET)
public void getPdfDownload(HttpServletResponse response) {
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
report().columns().setDataSource().show()
.toPdf(buffer);
byte[] bytes = buffer.toByteArray();
InputStream inputStream = new ByteArrayInputStream (bytes);
IOUtils.copy(inputStream, response.getOutputStream());
response.setHeader("Content-Disposition", "attachment; filename=Accepted1.pdf");
response.flushBuffer();
}

Apache FTPClient : 530 You aren't logged in

I'm using FTPClient of apache for getting size of specific file from my server. So, the error that I faced in android is 530 You aren't logged in. while my code is such below and I've tested it in pure Java. I don't know why this error occurred on Android but all things are okay in Java.
client.connect("my-server-ip");
client.login("username", "password");
client.setKeepAlive(true);
client.setFileType(FTP.BINARY_FILE_TYPE, FTP.BINARY_FILE_TYPE);
client.setFileTransferMode(FTP.BINARY_FILE_TYPE);
client.sendCommand("SIZE " + "file-path");
try {
sizeStr = client.getReplyString().trim();
this.M_fileData.M_contentLength = Long.parseLong(sizeStr.split(" ")[1]);
System.out.println(sizeStr);
} catch (NumberFormatException e) {
e.printStackTrace();
client.disconnect();
}
Java pure result is : 213 1757682, while android result is 530 You aren't logged in.
Could any one explain me how to solve this ?
Thanks in advance.

- First of all i want you to make sure you have given the android.permission.INTERNET permission.
Well for further reference, i am giving you the code that i used to Download music clip in my android application.
I have used the Apache's commons library
public void goforIt(){
FTPClient con = null;
try
{
con = new FTPClient();
con.connect("50.xx.xx.xx");
if (con.login("Adminxxxxx", "KUjWbk361wobbyl-xxxxxx"))
{
con.enterLocalPassiveMode(); // important!
con.setFileType(FTP.BINARY_FILE_TYPE);
String data = "/sdcard/vivek.m4a";
OutputStream out = new FileOutputStream(new File(data));
boolean result = con.retrieveFile("vivekm4a.m4a", out);
out.close();
if (result) Log.v("download result", "succeeded");
con.logout();
con.disconnect();
}
}
catch (Exception e)
{
Log.v("download result","failed");
e.printStackTrace();
}
}

Whenever 530 error code return it means your username or password is incorrect. And due to login failed you cant upload file.

Why uploaded file through Apache FTPClient is smaller then original file on the local?

I am uploading files(.cvs,.zip,.rar,.doc,.png,.jpg...) to ftp server. The strange is everything is successfully but I miss some data.
Does any body know why it happens and how to fix it?
public static void uploadWithCommonsFTP(File fileToBeUpload) {
FTPClient f = new FTPClient();
try {
f.connect(server.getServer());
f.login(server.getUsername(), server.getPassword());
f.changeWorkingDirectory("user");
f.setFileType(FTP.BINARY_FILE_TYPE);
f.setFileTransferMode(FTP.BINARY_FILE_TYPE);//this is part of Mohammad Adil's solutions
f.enterLocalPassiveMode();
ByteArrayInputStream in = new ByteArrayInputStream(FileUtils.readFileToByteArray(fileToBeUpload));
boolean reply = f.storeFile(fileToBeUpload.getName(), in);
if(!f.completePendingCommand()) {
f.logout();
f.disconnect();
System.err.println("File transfer failed.");
System.exit(1);
}
if(reply){
JOptionPane.showMessageDialog(null,"uploaded successfully.");
}else{
JOptionPane.showMessageDialog(null,"Upload failed.");
}
}
//Logout and disconnect from server
in.close();//this is part of Mohammad Adil's solutions
f.logout();
f.disconnect();
} catch (IOException e) {
e.printStackTrace();
}
}

It's often forgotten that FTP has two modes of operation - one for text files and the other for binary(jpg,csv,pdf,zip) files.
Your code doesn't work because the default transfer mode for FTPClient is FTP.ASCII_FILE_TYPE. You just need to update the configuration to transfer in binary mode.
Add this in your code :
f.setFileTransferMode(FTP.BINARY_FILE_TYPE);
just put that line after f.setFileType(FTP.BINARY_FILE_TYPE);
and it should work then.
EDIT:
You are not closing inputStream in your code,Just call in.close() before calling logout()

HttpServletResponse response: Ask user to download file instead of auto downloading

This is my download code. It just starts downloading the file without asking user. I've searched multiple forums and nothing seems to work. This is code is in a backing bean attached to a commandButton.
public void doDownloadFile() {
PrintWriter out = null;
try {
HttpServletResponse response = (HttpServletResponse) FacesContext.getCurrentInstance().getExternalContext().getResponse();
response.setContentType("application/vnd.ms-excel");
response.setHeader("Content-disposition", "attachment;filename=test.csv");
out = response.getWriter();
CSVWriter writer = new CSVWriter(out);
List<String[]> stringList = new ArrayList<String[]>();
for (User user : userList) {
String[] string = {user.getEmail(), user.getName(), user.getPassword()};
stringList.add(string);
}
writer.writeAll(stringList);
out.flush();
} catch (IOException ex) {
Logger.getLogger(ViewLines.class.getName()).log(Level.SEVERE, null, ex);
} finally {
out.close();
}
}

This is most likely due to the fact your browser is configured to download files of these types without prompt. The code has nothing to do with it.

The behavior of what to do with a download is 100% local, meaning it's the browser, not you, that determines what to do in that case. Whether the user's browser just dumps the file in a download folder or allows him to save it to a particular spot is entirely up to the browser.
Not much to be done.