We Keep Coding

How to pass multiple parameters in controller?

I am having trouble passing multiple parameters to GET resource in my controller. I have created a named query in my repository. When i call this GET endpoint, it should execute the named query by passing parameters.
Below code should take multiple parameters as input for example ID = 1,2,3,4 etc. It only takes single input as param.
#GetMapping("/message/{Ids}")
#CrossOrigin(origins = "*")
public void multidownload(#PathVariable Long[] Ids , HttpServletResponse response)throws Exception {
List<MessageRepository> messageRepository = Repository.findbyId(Ids);
String xml = new ObjectMapper().writeValueAsString(messageRepository);
String fileName = "message.zip";
String xml_name = "message.xml";
byte[] data = xml.getBytes();
byte[] bytes;
try (ByteOutputStream bout = new ByteOutputStream();
ZipOutputStream zout = new ZipOutputStream(bout)) {
zout.setLevel(1);
ZipEntry ze = new ZipEntry(xml_name);
ze.setSize(data.length);
zout.putNextEntry(ze);
zout.write(data);
zout.closeEntry();
bytes = bout.getBytes();
}
response.setContentType("application/zip");
response.setContentLength(bytes.length);
response.setHeader("Content-Disposition", "attachment; " + String.format("filename=" + fileName));
ServletOutputStream outputStream = response.getOutputStream();
FileCopyUtils.copy(bytes, outputStream);
outputStream.close();
}
downloaded zip file should contain multiple ID records which were passed as parameter when calling the GET endpoint.
can someone look into my code and point out what needs changing?

You can rewrite it to List of Ids - `List Ids
#GetMapping("/message/{Ids}")
#CrossOrigin(origins = "*")
public void multidownload(#PathVariable List<Long> Ids , HttpServletResponse response)throws Exception {
...

You achieve the multiple input parameters in POST request method.
In the request payload, please add this array of integer in your Request payload.
[1,2,3,4,5]
To achieve same thing in GET request method convert your array of integer into string.
Example:
localhost:8080/user/str=1,2,3

How to deserialize avro files

I would like to read a hdfs folder containing avro files with spark . Then I would like to deserialize the avro events contained in these files. I would like to do it without the com.databrics library (or any other that allow to do it easely).
The problem is that I have difficulties with the deserialization.
I assume that my avro file is compressed with snappy because at the begining of the file (just after the schema), I have
avro.codecsnappy
written. Then it's followed by readable or unreadable charaters.
My first attempt to deserialize the avro event is the following :
public static String deserialize(String message) throws IOException {
Schema.Parser schemaParser = new Schema.Parser();
Schema avroSchema = schemaParser.parse(defaultFlumeAvroSchema);
DatumReader<GenericRecord> specificDatumReader = new SpecificDatumReader<GenericRecord>(avroSchema);
byte[] messageBytes = message.getBytes();
Decoder decoder = DecoderFactory.get().binaryDecoder(messageBytes, null);
GenericRecord genericRecord = specificDatumReader.read(null, decoder);
return genericRecord.toString();
}
This function works when I want to deserialise an avro file that doesn't have the avro.codecsbappy in it. When it's the case I have the error :
Malformed data : length is negative : -50
So I tried another way of doing it which is :
private static void deserialize2(String path) throws IOException {
DatumReader<GenericRecord> reader = new GenericDatumReader<>();
DataFileReader<GenericRecord> fileReader =
new DataFileReader<>(new File(path), reader);
System.out.println(fileReader.getSchema().toString());
GenericRecord record = new GenericData.Record(fileReader.getSchema());
int numEvents = 0;
while (fileReader.hasNext()) {
fileReader.next(record);
ByteBuffer body = (ByteBuffer) record.get("body");
CharsetDecoder decoder = Charsets.UTF_8.newDecoder();
System.out.println("Positon of the index " + body.position());
System.out.println("Size of the array : " + body.array().length);
String bodyStr = decoder.decode(body).toString();
System.out.println("THE BODY STRING ---> " bodyStr);
numEvents++;
}
fileReader.close();
}
and it returns the follwing output :
Positon of the index 0
Size of the array : 127482
THE BODY STRING --->
I can see that the array isn't empty but it just return an empty string.
How can I proceed ?

Use this when converting to string:
String bodyStr = new String(body.array());
System.out.println("THE BODY STRING ---> " + bodyStr);
Source: https://www.mkyong.com/java/how-do-convert-byte-array-to-string-in-java/

Well, it seems that you are on a good way. However, your ByteBuffer might not have a proper byte[] array to decode, so let's try the following instead:
byte[] bytes = new byte[body.remaining()];
buffer.get(bytes);
String result = new String(bytes, "UTF-8"); // Maybe you need to change charset
This should work, you have shown in your question that ByteBuffer contains actual data, as given in the code example you might have to change the charset.
List of charsets: https://docs.oracle.com/javase/7/docs/api/java/nio/charset/Charset.html
Also usful: https://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer.html

MarkLogic: No stream to write

I'm having a problem where I have a method that gets parameters from the AngularJs front-end, creates an object with them, writes the object as XML file in a folder, and then is supposed to write that XML file into the MarkLogic database.
However, the part where it's supposed to write to the database appears to see as if the file doesn't exist, even though it does:
Here's the code:
#RequestMapping(value = "/add/korisnik", method = RequestMethod.POST)
public String addKorisnik(#RequestParam String ime, #RequestParam String prezime, #RequestParam String username, #RequestParam String password, #RequestParam String orcid, #RequestParam String role) throws JAXBException, FileNotFoundException{
Korisnik.Roles roles = new Korisnik.Roles();
roles.setRole(role);
Korisnik k = new Korisnik();
k.setIme(ime);
k.setPrezime(prezime);
k.setUsername(username);
k.setPassword(password);
k.setOrcid(orcid);
k.setRoles(roles);
System.out.println(k.toString());
// create JAXB context and instantiate marshaller
JAXBContext context = JAXBContext.newInstance(Korisnik.class);
Marshaller m = context.createMarshaller();
m.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
StringWriter sw = new StringWriter();
m.marshal(k, sw);
// Write to File
File f = new File("src/main/resources/data/korisnici/" + k.getUsername() + ".xml");
if (f.exists()) {
return "Username already taken.";
}
else {
m.marshal(k, new File("src/main/resources/data/korisnici/" + k.getUsername() + ".xml"));
}
// acquire the content
InputStream docStream = ObjavaNaucnihRadovaApplication.class.getClassLoader().getResourceAsStream(
"data/korisnici/" + k.getUsername() + ".xml");
// create the client
DatabaseClient client = DatabaseClientFactory.newClient(MarkLogicConfig.host,
MarkLogicConfig.port, MarkLogicConfig.admin,
MarkLogicConfig.password, MarkLogicConfig.authType);
// create a manager for XML documents
XMLDocumentManager docMgr = client.newXMLDocumentManager();
// create a handle on the content
InputStreamHandle handle = new InputStreamHandle(docStream);
// write the document content
docMgr.write("http://localhost:8011/korisnici/" + k.getUsername()+".xml", handle);
//release the client
client.release();
return "OK";
}

Several issues.
First, the file you write to is not the same as you read from. You are writing to "src/main/resources/data/korisnici/.." which is relative to the current directory of the JVM (application server). You are reading from the classpath resource directory -- not likely to be the same. You could simply reuse the same File object then they would be the same.
Second, you don't need to write to disk this small of an object, just write it to a in-memory stream (like ByteArrayStream() ).

It sounds as if the classloader isn't finding your file.
If you already have a File object that you have just written, why not construct an InputStream from it, rather than attempt to find it from a differently constructed relative path?
InputStream docStream = new FileInputStream(f);

Contents of csv file getting displayed on webpage instead of a proper file download

Problem: Contents of a CSV file are getting displayed as page contents if the number of records is large.
Expected: File download.
The request gets fired from another page via window.open(url); Following is the url handler.
#RequestMapping(value = "/url", method = RequestMethod.GET)
public void handler(HttpServletRequest request,ModelMap map, #RequestParam("itemPk") long pk, HttpServletResponse response) throws Exception {
CSVWriter csvWriter = new CSVWriter(new OutputStreamWriter(response.getOutputStream()));
try {
List<String[]> data = new ArrayList<String[]>();
for (int i = 0; i < 500; i++) {
String[] row = new String[] {"asdr", "10", "sourajit", "34.567", "06-DEC-2015", "awx445"};
data.add(row);
}
csvWriter.writeAll(data);
response.setContentType("text/csv");
response.setHeader("Content-disposition", "attachment; filename=volcs.csv");
} finally {
csvWriter.flush();
csvWriter.close();
}
}
Same behavior for GET or POST.
(For POST, opening a new window with a html form)

This is because a Writer instance is used for generating the response.
We need to treat it as bytes.
ServletOutputStream out = response.getOutputStream();
out.write(baos.toByteArray());
baos.writeTo(out);
out.flush();

You need to add headers to tell browser that its attachment in the response. Check the below!
String filename = "volcs.csv";
response.setContentType("application/force-download");
response.setContentLength((int)f.length());
//response.setContentLength(-1);
//response.setHeader("Content-Transfer-Encoding", "binary");
response.setHeader("Content-Disposition","attachment; filename=" + filename);

How to extract .gz file Dynamically in Java?

In http://www.newegg.com/Siteindex_USA.xml lots of urls of .gz-files are given, like this:
<loc>
http://www.newegg.com//Sitemap/USA/newegg_sitemap_product01.xml.gz
</loc>
I want to extract these dynamically. I don't want to store them locally, I just want to extract them and store the contained data in a database.
Modify:
I am getting exception
private void processGzip(URL url, byte[] response) throws MalformedURLException,
IOException, UnknownFormatException {
if (DEBUG) System.out.println("Processing gzip");
InputStream is = new ByteArrayInputStream(response);
// Remove .gz ending
String xmlUrl = url.toString().replaceFirst("\\.gz$", "");
if (DEBUG) System.out.println("XML url = " + xmlUrl);
InputStream decompressed = new GZIPInputStream(is);
InputSource in = new InputSource(decompressed);
in.setSystemId(xmlUrl);
processXml(url, in);
decompressed.close();
}

Simply wrap the input stream in GZIPInputStream, and it'll decompress the data as you're reading it.