I am using spark to run the server side for a web application I am writing. I searched the documentation a bit, but I came up empty.. is there a way to serve data to the frontend such that it automatically downloads for the user as a csv file? My data that I am attempting to serve as csv looks something like this.
// example data... returned by "getData()"
JSONArray test = new JSONArray()
.put(
new JSONArray().put("foo").put("bar")
)
.put(
new JSONArray().put(1).put(2)
);
// route
get("/csv/:path", "application/json", (req, res) -> {
res.type("text/csv");
JSONArray reply = getData(req);
return data;
});
I was taking a look at the ResponseTransformers section of the documentation, but I couldn't figure out how to serve my data as a downloadable csv file instead of a json object. I'm assuming the ResponseTransformer would somehow need to be subclassed, but I couldn't find an example to do what I want. Could anyone provide me with an example, or point me in the direction of some docs that explain how to do this?
EDIT : I was able to, on the javascript side, call my route like this.
window(route);
which allowed me to select a program on my computer to download the response. However, the data looks like this in notepad
[["foo","bar"],[1,2]]
So, close.. but not quite a csv file. I was hoping the output would look more like this.
foo,bar
1,2
I think you could use a StringBuilder to render your csv file, as this answer does. I also think that the second parameter of your request "application/json" could also be removed. It would look like this:
// route
get("/csv/:path", (req, res) -> {
res.type("text/csv");
StringBuilder sb = new StringBuilder();
sb.append("id");
sb.append(',');
sb.append("Name");
sb.append('\n');
sb.append("1");
sb.append(',');
sb.append("Zack");
sb.append('\n');
return sb.toString();
});
Related
I am creating a Spring Cloud Function that I want to give two inputs, an id and a Multipart file (CSV file) but I am having trouble.
If I choose to send a post with a multipart file the function won't recognise this and gives an error like Failed to determine input for function call with parameters:
With the Postman request being this:
#Bean
public Function<MultipartFile, String> uploadWatchlist() {
return body -> {
try {
return service.convert(body);
}
}
}
I have tried using something more akin to Spring MVC like a request entity object but no luck.
The backup I have (other than Python haha) will be using the binary data post so it will just be a string that has the contents of the file which does work, but requires me to append the id inside to each row of the csv which is a bit messy.
There are other solutions but trying to get this working as Java lambdas are what we want to try and use as first choice.
The infrastructure will be to fix up a manual file upload/verification process that is tedious at the moment and looks like: postman -> load balancer -> lambda -> ecs
The postman/load balancer part will be replaced in future. Ideally have the lambda sorted in Java taking in a file and id.
Thanks for any help :)
The question can seem simple, but I didn't find a good answer yet. I need to send a JSon structure (build with an unspecified libretry I'm currently developing) from a Servlet to a remote page.
I'm interested in the best way to send the structure.
I mean, in my Servlet, inside the doPost() event, how should I manage the send?
I was thinking about 2 scenarios:
try (PrintWriter out = response.getWriter()) {
out.print(myJSon.toString(); // <- recursive function that overrides
// toString() and returns the entire JSon
// structure
} (...)
or
try (OutputStream os = response.getOutputStream()) {
myJSon.write(os, StandardCharsets.UTF8); // <- function that
// recursively writes chunk of my JSon structure
// in a BufferWriter created inside the root write function
// forcing UTF-8 encoding
} (...)
Or something different, if there's a better approch.
Note that the JSon structure contains an array of objects with long text fields (descriptions with more than 1000 characterd), so it can be quite memory consuming.
For why I'm not using standard JSon libreries, it's because I don't know them and I don't know if I can trust them yet. And also I don't know if I will be able to install them on the production server.
Thanks for your answers.
From your question i see multiple points to adress:
How to send your JSon
What JSon library can you use
How to use the library in production
How to send your JSon
From your code this seems to be an HTTP response rather than a POST on your Servlet so you need to know how to send a JSON string as an HTTP response's body
Do you use a framework for your web server or are you handling everything manually ? If you use a framework it usually does it for you, just pass the JSON String
If your doing it manually:
try (PrintWriter pw = response.getWriter()) {
pw.write(myJson.toString());
}
or
try (OutputStream os = response.getOutputStream()) {
os.write(myJson.toString().getBytes());
}
Both are valid, see Writer or OutputStream?
Your JSON's size shouldn't matter given what your saying, it's just text so it won't be big enough to matter.
What libraries can you use
There are a lot of JSON libraries for Java, mainly:
Jackson
GSon
json-io
Genson
Go for the one you prefer, there will be extensive documentation and resources all over google
How to use in production
If you are not sure you are able to install dependencies on the production server, you can always create an uber-jar (See #Premraj' answer)
Basically, you bundle the dependency in your Jar
Using Gson is good way to send json
Gson gson = new Gson();
String jsonData = gson.toJson(student);
PrintWriter out = response.getWriter();
try {
out.println(jsonData);
} finally {
out.close();
}
for detail json response from servlet in java
Is there a way (any jmeter plugin) by which we can have the JMeter script read all the contents(String) from external text file ?
I have a utility in java which uses Jackson ObjectMapper to convert a arraylist to string and puts it to a text file in the desktop. The file has the JSON info that i need to send in the jmeter Post Body.
I tried using ${__FileToString()} but it was unable to deserialize the instance of java.util.ArrayList. It was also not reading all the values properly.
I am looking for something like csv reader where i just give the file location. I need all the json info present in the file. Need to extract it and assign to the post body.
Thanks for your help !!!
If your question is about how to deserialize ArrayList in JMeter and dynamically build request body, you can use i.e. Beanshell PreProcessor for it.
Add a Beanshell PreProcessor as a child of your request
Put the following code into the PreProcessor's "Script" area:
FileInputStream in = new FileInputStream("/path/to/your/serialized/file.ser");
ObjectInput oin = new ObjectInputStream(in);
ArrayList list = (ArrayList) oin.readObject();
oin.close();
in.close();
for (int i = 0; i < list.size(); i++) {
sampler.addArgument("param" + i, list.get(i).toString());
}
The code will read file as ArrayList, iterate through it and add request parameter like:
param1=foo
param2=bar
etc.
This is the closest answer I'm able to provide, if you need more exact advice - please elaborate your question. In the meantime I recommend you to get familiarized with How to use BeanShell: JMeter's favorite built-in component guide to learn about scripting in JMeter and what do pre-defined variables like "sampler" in above code snippet mean.
(After months of surfing the internet, talking to the school's computing department and try code out, I still don't get how to do it, but I do know more specific about what I trying to do)
Previously I said I want to "Add lines" to a existing JSON file.
What I want to do is simply add an element to an JSON object from a file, then save the file.
However I am still confused about how to do it.
The process I am guessing is to use ajax to load the content of the file (the JSON code in the file) into a variable then add the new element into the object then save the file.
I have seen a lot of code but are all just too confusing and looks like its for webpages. I am trying to edit a file on the computer as a program which I think webpage related code such as xmlhttp requests are irrelevant as the file is in a folder in appdata.
I have been confused and thought Java and Javascript were the same thing, I know now they're not.
What code or functions would I look for and how would it be used in the code?
(Please don't post pseudocode because I have no idea how to write the code for them since I have literally no idea how to code anything other than a html webpage and some php. Other coding language like Java, Javascript and Python I have little knowledge with but not enough to write a program alone.)
I think it would be best to use code that somebody else has already written to manipulate the JSON. There are plenty of libraries for that, and the best would be the officially specified one, JSON-P. What you would do is this:
Go to http://jsonp.java.net/ and download JSON-P. (You will have to examine the page carefully to find the link to "JSON Processing RI jar".) You will need to include this JAR in your class path while you write your program.
Add imports to your program for javax.json.*.
Write this code to do the job (you will have to catch JsonExceptions and IOExceptions):
JsonReader reader = Json.createReader(new FileReader("launcher_profiles.json"));
JsonObject file = reader.readObject();
reader.close();
JsonObject profiles = file.getJsonObject("profiles");
JsonObject newProfile = Json.createObjectBuilder()
.add("name", "New Lines")
.add("gameDir", "New Lines")
.add("lastVersionId", "New Lines")
.add("playerUUID", "")
.build();
JsonObjectBuilder objectBuilder = Json.createObjectBuilder()
.add("New Profile Name", newProfile);
for (java.util.Map.Entry<String, JsonValue> entry : profiles.entrySet())
objectBuilder.add(entry.getKey(), entry.getValue());
JsonObject newProfiles = objectBuilder.build();
// Now, figure out what I have done so far and write the rest of the code yourself! At the end, use this code to write out the new file:
JsonWriter writer = Json.createWriter(new FileWriter("launcher_profiles.json"));
writer.writeObject(newFile);
writer.close();
I have to do a HTTP post in java every second after building a json object.
The json object is built from reading a CSV file which is huge (200Mbs+), so my problem is
how do I read x number of lines build x objects and post that every second(as it is not possible to parse the whole 200mb file in less than a second) and continue reading the next x lines.
Please let me know your thoughts..
Can I use Java timer class, and keep reading the CSV file and at the same time post the json object to the server every second with the formed json?
It is hardly possible to read, parse, convert and send a 200 MB file once per second.
So you need to change your design:
My suggestion would be to only send changed lines, something like this:
{
"1" : {"field1":"value1","field2":"value2"},
"17" : {"field1":"value1","field2":"value2"}
}
Which of course gives you new problems:
The client needs to figure out which lines have changed, and the server needs to integrate the changed lines with the existing data.
I would make it depending on the file size and not depending on time.
BufferedReader fin = null; //create it
Gson gson=new Gson(); //Google code open source library for JSON in Java
ArrayList<JSONObject> jsonList=new ArrayList<JSONObject>();
while (((line = fin.readLine()) != null)) {
if ( line.length()==0 ){
//"Blank line;
}else{
currJSON=loadJSON(line);//You have to load it in a Java Object
if ( jsonList.size()<MAX_JSON){
jsonList.add(currJSON);
}
if (JsonList.size()==MAX_JSON){ //Define the maximum size of the list you want to post
gson.toJson(jsonList); //Convert to JSON
//You should post your Json with some Http Connection to your server
jsonList.clear();