We Keep Coding

Specification and XML parsing with lot of entities -> out of memory error

I have a request to find a lot of entities in my database with a Specification.
After getting them I need to parse them into another entity in order to export them into a XML file. When my Specification get to much entities (~6 000) I get an out of memory error.
Does anyone have a solution ? I'm out of ideas ...(increase heap size is not an option)
#RequestMapping(method = RequestMethod.GET, value = "/cmc")
public String getXmlExport(CmcCriteria cmcCriteria, Model model) throws CmcCommonsException {
Specification<Cmc> cmcSpec = cmcService.getSpec(cmcCriteriaResource);
List<Cmc> cmcList = cmcRepository.findAll(cmcSpec);
model.addAttribute("requestCmcXml", getCmcXmlListFromCmcList(cmcList));
return "cmc/list";
}
#Override
protected final void renderMergedOutputModel(Map<String, Object> model, HttpServletRequest request, HttpServletResponse response) throws Exception {
List<CmcXml> listeCmcXml = (List<CmcXml>) model.get("requestCmcXml");
CmcXmlExport export = new CmcXmlExport();
export.setPersonnels(listeCmcXml);
response.setContentType(CONTENT_TYPE_XML);
ServletOutputStream out = response.getOutputStream();
JAXBContext context = JAXBContext.newInstance(CmcXmlExport.class);
Marshaller marshaller = context.createMarshaller();
marshaller.marshal(export, out);
out.flush();
}
#XmlRootElement(name = "CMC")
public class CmcXmlExport {
private List<CmcXml> personnels;
[...]
}
#XmlType(propOrder = { "id", "nom", "prenom", "genre", [...] })
public class CmcXml implements Serializable {
private static final long serialVersionUID = 1L;
private int id;
private String nom;
private String prenom;
private String genre;
[...]
}

Try to use Apache Xerces lib for Java: Official website. It encourages to write more efficient code for XML parcing and handling.
This lib provides you the methods for handling XML blocks and you will have to write some additional code that parse your entities.

Jackson double serialized json string to Java object

I have a Json string which has a string message field.
String:
{ "Type" : "Text",
"Subject" : "data received",
"Message" :"{\\"language\\":\\"US\\",\\"data\\":\\"signature\\"}"
}
I want to convert it into the following structure:
Notification.java
public class Notification {
String type;
String subject;
Message message;
}
Message.java
public class Message {
String language;
String data;
}
Is there a way in which I can directly convert the string to a Java object of the above structure? I want to avoid deserializing twice.

You can create a custom Deserializer to deserialize the Message text into Message object and annotate the Message class with #JsonDeserialize:
#JsonDeserialize(using = MessageDeserializer.class)
public class Message {
String language;
String data;
}
public class MessageDeserializer extends JsonDeserializer<Message> {
public MessageDeserializer() {
super();
}
#Override
public Message deserialize(
final JsonParser jsonParser, final DeserializationContext deserializationContext) throws
IOException, JsonProcessingException {
final String messageText = jsonParser.getText();
// parse messageText into Message object
}
}

I am not sure my solution is acceptable since it does require additional explicit call to ObjectMapper to perform deserialization of the string value of Message.
However, this is it is done during the buildup of Notification object and does not require a String message property.
You need to add a ctor with String argument to Message class, where you can deserialize the String into Map and extract the instance propertieds:
public Message(String str) {
try {
#SuppressWarnings("unchecked")
Map<String, Object> map =
(Map<String, Object>)new ObjectMapper().readValue(str, Map.class);
language = map.containsKey("language") ? map.get("language").toString() : null ;
data = map.containsKey("data") ? map.get("data").toString() : null ;
} catch (Exception e) {
e.printStackTrace();
}
}
the new ctor will be called by Jackson when you deserialize a Notification object:
Notification n = (Notification)new ObjectMapper().readValue(reader, Notification.class);

You can convert json string into key-value pairs in Map.You will have to do twice as the Message value is again a json string.Use org.json for JSONObject
Map<String, String> map = new HashMap<String, String>();
JSONObject j = new JSONObject(str);
Iterator<String> keys = j.keys();
while( keys.hasNext() ){
String key = (String)keys.next();
String val = j.getString(key);
map.put(key, val);}
Then retrieve the values by iterating over the keys and pass the values into the class constructor
Then map.get(key) can be used to retrieve the values and will be passed into constructors of the classes.

The org.json library is easy to use:
//Create Json object to parse string
// str is input string
JSONObject obj = new JSONObject(str);
//Create Message
Message mess = new Message();
JSONObject obj2 = new JSONObject(obj.getString("Message"));
mess.data = obj2.getString("data");
mess.language = obj2.getString("language");
//Create Notification
Notification noti = new Notification();
noti.message = mess;
noti.subject = obj.getString("Subject");
noti.type = obj.getString("Type");

Java REST service. Info is not saved to file and my code doesnt read the file

I am creating my first Rest service using JSON objects for the data
transfer between user and server, with the help of the Gson library 2.5.
I am not using any frameworks like Jersey or anything like that. (That was my
project requirment). The java version i use is 1.6 (part of my requirment)
jboss server and Eclipse as IDE.
At the moment i have 2 small functions from a simple HTML form. The first is
suposed to requests the data from the JSON file and the second is suposed to
add a new json information to the json document.
Problem is: When i try to acces the JSON file, a array its returned with the
last submited Person. When i save a new Person information, that information is
not saved in the personsJsonFile but someplace else [have no ideea where].
My json file is found in the Projects main folder.
Any help is deeply apreciated.
GetData class:
#Path("/data")
public class GetDataClass {
#GET
#Produces("text/plain")
public ArrayList<PersonConstructor> displayJsonFile() throws IOException{
ArrayList<PersonConstructor> newLib = new ArrayList<PersonConstructor>();
File jsonFile = new File("personsJsonFile.json");
Scanner fileInput = new Scanner(jsonFile);
Gson gson = new Gson();
while(fileInput.hasNextLine()){
String jsonLine = fileInput.nextLine();
PersonConstructor singlePerson = gson.fromJson(jsonLine, PersonConstructor.class);
newLib.add(singlePerson);
}
fileInput.close();
return newLib;
}
}
AddData Class:
#Path("/add")
public class AddPersonsClass {
#POST
public String addPersons(
#FormParam("idInput") int idInput,
#FormParam("surnameInput") String surnameInput,
#FormParam("nameInput") String nameInput
) throws IOException
{
Gson gson = new Gson();
PersonConstructor newPerson = new PersonConstructor();
newPerson.setPersonId(idInput);
newPerson.setPersonNume(nameInput);
newPerson.setPersonPrenume(surnameInput);
File jsonFile = new File("personsJsonFile.json");
FileWriter jsonWriter = new FileWriter(jsonFile);
System.out.println(newPerson);
String jsonLine = gson.toJson(newPerson);
System.out.println(newPerson);
jsonWriter.write(jsonLine+"\n");
jsonWriter.close();
return "Element: " + newPerson + "has been added";
}
}
PersonConstructor Class:
public class PersonConstructor {
private int personId;
private String personNume;
private String personPrenume;
public PersonConstructor(int personId, String personNume,String personPrenume){
this.personId = personId;
this.personPrenume = personPrenume;
this.personNume = personNume;
}
public PersonConstructor() {
}
public int getPersonId(){
return personId;
}
public void setPersonId(int personId){
this.personId = personId;
}
public String getPersonNume(){
return personNume;
}
public void setPersonNume(String personNume){
this.personNume = personNume;
}
public String getPersonPrenume(){
return personPrenume;
}
public void setPersonPrenume(String personPrenume){
this.personPrenume = personPrenume;
}
public String toString(){
return String.format("\n%s %s %s\n", this.personId, this.personNume, this.personPrenume);
}
}
Json file contains:
{"personId":5,"personNume":"Ursu","personPrenume":"Niculae"},
{"personId":6,"personNume":"Ivan","personPrenume":"Claudiu"},
{"personId":7,"personNume":"Hap","personPrenume":"Dorel"}

Your problem seems to that you have not specified the path where to save the file.
Add the path when creating a file.
final String jsonDirectory = "path to file";
File file = new File(jsonDirectory + "\\results.txt");

Formatting header field for bean generator - Java

I have written a program that parses a csv file and creates a bean from the data to be put into a database. Everything works perfectly however now that this will be moved out of a testing environment, the real header names from the csv's will have to be added. These headers contain spaces and /. I am searching for a way to allow my parser to read these headers. When I define the header names, I have to use camelCasing and I am unable to insert spaces or other characters. Is there anyway to alter this?
Here is my constructor(integrationTeam needs to be Integration Team, softwareHardware needs to be Hardware/Software -- as is in csv header)
public class BeanGen {
public BeanGen(
final String name,
final String manufacturer,
final String model,
final String owner,
final String integrationTeam,
final String shipping,
final String hardwareSoftware,
final String subsystem,
final String plane,
final String integrationStandalone,
final String integrationInterface,
final String function,
final String helpLinks,
final String installationInstructions,
final String testSteps,
final String leadEngineer)
{
this.name = name;
this.manufacturer = manufacturer;
this.model = model;
this.owner = owner;
this.integrationTeam = integrationTeam;
this.shipping = shipping;
this.hardwareSoftware = hardwareSoftware;
this.subsystem = subsystem;
this.plane = plane;
this.integrationStandalone = integrationStandalone;
this.integrationInterface = integrationInterface;
this.function = function;
this.helpLinks = helpLinks;
this.installationInstructions = installationInstructions;
this.testSteps = testSteps;
this.leadEngineer = leadEngineer;
}
Here is the parser that handles the constructor
public class ParseHandler {
private static CellProcessor[] getProcessors() {
final CellProcessor[] processors = new CellProcessor[] {
new Optional(),
new Optional(),
new Optional(),
new Optional(),
new Optional(),
new Optional(),
new Optional(),
new Optional(),
new Optional(),
new Optional(),
new Optional(),
new Optional(),
new Optional(),
new Optional(),
new Optional(),
new Optional(),
};
return processors;
}
public static BeanGen readWithCsvBeanReader(Path path) throws IOException {
ICsvBeanReader beanReader = null;
BeanGen projectBean = null;
System.out.println("Processing File: " + path);
try {
beanReader = new CsvBeanReader(new FileReader(path.toString()), CsvPreference.STANDARD_PREFERENCE);
// the header elements are used to map the values to the bean (names
// must match)
final String[] header = beanReader.getHeader(true);
final CellProcessor[] processors = getProcessors();
if ((projectBean = beanReader.read(BeanGen.class, header, processors)) != null) {
System.out.println(String.format("%s", projectBean.toString()));
}
} finally {
if (beanReader != null) {
beanReader.close();
}
} return projectBean;
}
}

See the Super CSV documentation, section Reading with CsvBeanReader:
This relies on the fact that the column names in the header of the CSV file [...] match up exactly with the bean's field names, and the bean has the appropriate setters defined for each field.
If your header doesn't match (or there is no header), then you can simply define your own name mapping array.
You read the header and pass it to beanReader.read() as the second parameter. But according to the API reference the second parameter is a string array containing the bean property names. So you should pass something like
new String[] { "name", "manufacturer", "model", "owner", "integrationTeam", ... }
as the second parameter. So the first CSV column matches to bean field name, the second field matches to bean field manufacturer, etc.

populate a jtable with json string results from a RESTful web service using java

I have a web service that returns JSON strings one by one based on query inputs, a GET request to the service returns this (there's only one entry in the database)
[{"checked":false,"dateposted":"2014-10-23T00:00:00","itemnumber":1,"quantity":5,"stockcheckid":1}]
at the moment I just have this System.out.println in a while loop.
what I want to do is be able to access these results in a way that I can input them into a jtable to display on a client application. I've read some guides about reading from JSON files etc but I cant find anything specific to REST web services. I hear GSON mentioned a lot, i tried that but i cant quite work out how to make it work in this context
I should also mention that the service can also send this data in XML format.
Do I somehow create a JSON file appending each new entry to it? and then populate the table from that file?
anyway here's the code that initiates the GET request.
public static void getRequest(String dataGet) {
try {
URL url = new URL("http://localhost:8080/nXXXXXXXXc/webresources/entities.stockchecks/" + dataGet);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed : HTTP error code : "
+ conn.getResponseCode());
}
BufferedReader br = new BufferedReader(new InputStreamReader(
(conn.getInputStream())));
String output;
while ((output = br.readLine()) != null) {
System.out.println(output);
}
conn.disconnect();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}`

The response is a map. You can serialise a JSON map to a Java map with Jackson like this:
import static org.junit.Assert.assertEquals;
import java.io.IOException;
import java.util.Map;
import java.util.TreeMap;
import org.junit.Test;
import com.fasterxml.jackson.annotation.JsonAnyGetter;
import com.fasterxml.jackson.annotation.JsonAnySetter;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
public class JSONtoMap {
public static final String json = "{\"B\":\"b\",\"C\":\"c\"}";
public static class POJO{
private Map<String,String> map = new TreeMap<String,String>();
#JsonAnyGetter
public Map<String, String> get() {
return map;
}
#JsonAnySetter
public void set(String name, String value) {
map.put(name, value);
}
}
#Test
public final void test() throws JsonProcessingException, IOException {
ObjectMapper jsonmapper = new ObjectMapper();
POJO p = jsonmapper.readValue(json, POJO.class);
assertEquals(jsonmapper.writeValueAsString(p),json);
}
}
I imagine you could do something similar with GSON. Another option is if you know the structure of the JSON object - in that case you make a simple POJO version and de-serialise into that instead of something like the POJO class I've defined above.
More detail and a similar version that supports XML as well as JSON mapping

Irrespective of how you obtain the data, use it to construct a suitable TableModel and use that model to construct the JTable. In this example, the model accesses a Map<String, String> to fulfill the TableModel contract; you can substitute the Map obtained using the approach shown here. Because loading the data may take an indeterminate amount of time, use a SwingWorker, as shown here.

This is simply a combination of trashgod's and tom's answer, with an example, using Jackson and a TableModel. I really just want to give camickr's BeanTableModel/RowTableModel (which is a generic class to help us easily create table models to from pojos) a try (seems to work great).
For more information/details please see this post
Entity class (properties mapped to the keys in your json)
public class Entity {
// field/property names must match to your json keys (with some exceptions)
// other wise we are required to use further annotations
private boolean checked;
private Date dateposted;
private int itemnumber;
private int quantity;
private int stockcheckid;
/*** ----- DO NOT FORGET GETTERS AND SETTERS ---- ***/
}
Main class. Note the use BeanTableModel. You will need to download this class along with RowTableModel from the link above.
public class JsonTableDemo {
public static void main(String[] args) throws Exception {
ObjectMapper mapper = new ObjectMapper();
String json = "[{\"checked\":false,\"dateposted\":\"2014-10-23T00:00:00\",\"itemnumber\":1,\"quantity\":5,\"stockcheckid\":1}]";
List<Entity> response = mapper.readValue(json,
TypeFactory.defaultInstance().constructCollectionType(
List.class, Entity.class));
RowTableModel model = new BeanTableModel<>(Entity.class, response);
JTable table = new JTable(model) {
#Override
public Dimension getPreferredScrollableViewportSize() {
return getPreferredSize();
}
};
JOptionPane.showMessageDialog(null, new JScrollPane(table));
}
}
Result
Note, for long running task (many io task included), such as requesting a rest resource, you should use a SwingWorker as explained by trashgod. When the repsonse comes in, you can basically addRow to the RowTableModel instance. For instance if we use the same json response and model as above, we could simply do something like
response = mapper.readValue(json,
TypeFactory.defaultInstance().constructCollectionType(
List.class, Entity.class));
for (Entity entity : response) {
model.addRow(entity);
}
UPDATE
I should also mention that the service can also send this data in XML format.
Looking at your json, it is an array of objects. With XML, the format is a little different, as you must have a root document element. So you can't just have say
<entity>
<prop></prop>
</entity>
<entity>
<prop></prop>
</entity>
It would need to be something like
<entities>
<entity>
<prop></prop>
</entity>
<entity>
<prop></prop>
</entity>
</entities>
That being said, using data binding, the easiest way would be to create another class to wrap a List<Entity>. Now I'm not too familiar with Jackson's XML capabilities/features, but using JAXB, you can have a class like:
#XmlRootElement
#XmlAccessorType(XmlAccessType.FIELD)
public class Entities {
#XmlElement(name = "entity")
protected List<Entity> entities;
public List<Entity> getEntities() {
if (entities == null) {
entities = new ArrayList<>();
}
return entities;
}
public void setEntities(List<Entity> entities) {
this.entities = entities;
}
}
Then you can unmarshal the below XMl into the Entities class. Here's an update demo to show both json and xml
public class JsonTableDemo {
public static void main(String[] args) throws Exception {
ObjectMapper mapper = new ObjectMapper();
String json = "[{\"checked\":false,\"dateposted\":\"2014-10-23T00:00:00\",\"itemnumber\":1,\"quantity\":5,\"stockcheckid\":1}]";
List<Entity> response = mapper.readValue(json,
TypeFactory.defaultInstance().constructCollectionType(
List.class, Entity.class));
RowTableModel jsonModel = new BeanTableModel<>(Entity.class, response);
JTable jsonTable = new JTable(jsonModel) {
#Override
public Dimension getPreferredScrollableViewportSize() {
return getPreferredSize();
}
};
String xml = "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n"
+ "<entities>\n"
+ " <entity>\n"
+ " <checked>false</checked>\n"
+ " <dateposted>2014-10-22T17:00:00-07:00</dateposted>\n"
+ " <itemnumber>1</itemnumber>\n"
+ " <quantity>5</quantity>\n"
+ " <stockcheckid>1</stockcheckid>\n"
+ " </entity>\n"
+ "</entities>";
JAXBContext context = JAXBContext.newInstance(Entities.class);
Unmarshaller unmarshaller = context.createUnmarshaller();
Entities entities = (Entities)unmarshaller.unmarshal(new StringReader(xml));
RowTableModel<Entity> xmlModel = new BeanTableModel<>(
Entity.class, entities.getEntities());
JTable xmlTable = new JTable(xmlModel){
#Override
public Dimension getPreferredScrollableViewportSize() {
return getPreferredSize();
}
};
JPanel panel = new JPanel(new GridLayout(0, 1));
JPanel jsonPanel = new JPanel(new BorderLayout());
jsonPanel.add(new JLabel("JSON Table", SwingConstants.CENTER), BorderLayout.PAGE_START);
jsonPanel.add(new JScrollPane(jsonTable));
panel.add(jsonPanel);
JPanel xmlPanel = new JPanel(new BorderLayout());
xmlPanel.add(new JLabel("XML Table", SwingConstants.CENTER), BorderLayout.PAGE_START);
xmlPanel.add(new JScrollPane(xmlTable));
panel.add(xmlPanel);
JOptionPane.showMessageDialog(null, new JScrollPane(panel));
}
}
See more about using JAXB