I have the following code:
List<Details> detailsList = new ArrayList<>();
List<String[]> csv = csvReader.readAll();
final Map<String, Integer> mappedHeaders = mapHeaders(csv.get(0));
List<String[]> data = csv.subList(1, csv.size());
for (String[] entry : data) {
Details details = new Details(
entry[mappedHeaders.get("A")],
entry[mappedHeaders.get("B")],
entry[mappedHeaders.get("C")]);
detailsList.add(details);
I'm essentially reading in a CSV file as a list of string arrays where the first list item is the CSV file headers and all remaining elements correspond to the data rows. However, since different CSV files of the same features might have different feature column ordering I don't know the ordering in advance. For that, I have a mapHeaders method which maps the headers to indices so I can later properly put together the Details object (for example, if headers are ["B", "A", "C"], the mappedHeaders would correspond to {B: 0; A: 1; C: 2}.
I also have some test data files of different column orderings and all but one of them work as they should. However, the one that doesn't work gives me
java.lang.NullPointerException: cannot unbox null value
when trying to evaluate entry[mappedHeaders.get("A")]. Additionally, when running the code in debugging mode, the mappedHeaders contains the correct keys and values and the value for "A" isn't null.
I have also tried entry[mappedHeaders.getOrDefault("A", Arrays.asList(csv.get(0)).indexOf("A"))] which returns -1. The only thing that works is entry[mappedHeaders.getOrDefault("A", 0)] since A is the first column in the failing case, but that workaround don't seem very feasible as there might be more failing cases that I don't know about, but where the ordering is different. What might be the reason for such behavior? Might it be some weird encoding issue?
That's because you are trying to unbox a null value.
A method like intValue, longValue() or doubleValue() is being called on a null object.
Integer val = null;
if (val == 1) {
// NullPointerException
}
Integer val = null;
if (val == null) {
// This works
}
Integer val = 0;
if (val == 1) {
// This works
}
Related
I know Set and Map accepts null values and today I just found out LinkedList also accepts null values like
map.put(null, 1);
set.add(null);
linkedList.add(null)
Is there any other collections that allow null values to be stored?. Just posting to get a comprehensive list in one place and reason for each for them.
Set and Map are interfaces. They have several implementations in Java.
Popular implementations of the Map are:
HashMap - accepts one null key
Hashtable - doesn't accept any null key
TreeMap - doesn't accept any null key
LinkedHashMap - accepts one null key
Any number of null values can be added as value in any of above implementations
Popular implementations of the Set are:
HashSet - accepts one null element
TreeSet - doesn't accept any null element
LinkedHashSet - accepts one null element
Any implementations of List, like ArrayList or LinkedList can accept nulls.
This problem can be resolved by using placeholder class Optional from JDK.
The principle is to package a nullable value within an wrapper instance using Optional class that will carry values (null values also).
It can be annoying and little heavy in the code but surely can be suitable in many cases.
Here is an example with a non-null value :
Optional<Integer> valuedInt = Optional.ofNullable(123);
assertTrue(valuedInt.isPresent());
assertEquals(Integer.valueOf(123), valuedInt.get());
Here is an example with a null value :
Optional<Integer> nullInt = Optional.ofNullable(null);
assertTrue(nullInt.isEmpty());
try {
var val = nullInt.get();
} catch (Exception e) {
// Invocation of nullInt.get() throws an exception.
assertTrue(e instanceof NoSuchElementException);
}
// However, the method Optional.orElse(T) can be used as
// a getter that supplies the value wether it is valued or null.
assertEquals(Integer.valueOf(123), valuedInt.orElse(null));
assertEquals(null, nullInt.orElse(null));
We can initialize a list as so :
// Our hacked list
List<Optional<Integer>> integersOpt;
First way to populate the list :
integersOpt = new ArrayList<>();
integersOpt.add(Optional.ofNullable(1));
integersOpt.add(Optional.ofNullable(null));
integersOpt.add(Optional.ofNullable(2));
integersOpt.add(Optional.ofNullable(null));
integersOpt.add(Optional.ofNullable(3));
Second way to populate the list (unmodifiable) :
integersOpt =
Arrays.asList(1, null, 2, null, 3).stream()
.map(x -> Optional.ofNullable(x))
.collect(Collectors.toList());
Third way to populate the list (modifiable) :
integersOpt =
new ArrayList<>(
Arrays.asList(1, null, 2, null, 3).stream()
.map(x -> Optional.ofNullable(x))
.collect(Collectors.toList())
);
Count and print values :
int countNulls = 0, countNonNulls = 0;
for(Optional<Integer> opt : integersOpt) {
Integer optVal = opt.orElse(null); // Our famous "getter" function.
if(optVal == null) {countNulls++;}
else {countNonNulls++;}
System.out.println("Original value = " + optVal);
}
assertEquals(2, countNulls);
assertEquals(3, countNonNulls);
integersOpt.toString() will return this value :
[Optional[1], Optional.empty, Optional[2], Optional.empty, Optional[3]]
Accepting nulls was a design mistake, as time has proven. First of all this complicates implementations, because at some point in time you might need to be able to compare null to null, for example; and this sometimes require special handling (or special branches in the code at least).
Since java-9, immutable collections all reject null and even document it properly. One minor example (but there are many more) is Set:of, that says:
#throws NullPointerException if the element is null
Not only that, but they also don't allow null checks, so you might be surprised by this a bit:
Set<Object> oldS = new HashSet<>();
oldS.add(new Object());
System.out.println(oldS.contains(null)); // false
Set<Object> newS = Set.of(new Object());
System.out.println(newS.contains(null)); // NPE
Every new collection or implementation that comes to JDK prohibits null.
My sample request
{
"requestModel":{
"CUSTID": "100"
},
"returnParameters":[
{
"name":"NETWORK/NETID",
"datatype":"String",
"order":"asc",
"sequence":1
},
{
"name":"INFODATA/NAME",
"datatype":"String",
"order":"asc",
"sequence":1
},
{
"name":"SOURCE/SYSTEM",
"datatype":"int",
"order":"asc",
"sequence":2
},
]
}
Sample Response
Below is my dynamically generated Map format of json response[Response parameters will be different each time based on the request params],
"responseModel":{
"documents": [
{
"NETWORK":[
{"NETID":"1234"},
{"ACT":"300"}
],
"SOURCE": {
"SYSTEM":"50"
},
"INFODATA":{
"NAME":"PHIL"
}
},
{
"NETWORK":[
{"NETID":"1234"},
{"ACT":"300"}
],
"SOURCE": {
"SYSTEM":"100"
},
"INFODATA":{
"NAME":"PHIL"
}
}
]
}
Problem Statement
I need to do multi level sorting based on the "returnParameters" in the request which is dynamic...
"order" indicates ascending (or) descending and sequence indicates the the priority for ordering like (group by in sql query)
Code
Map<String,Object> documentList = new HashMap<String,Object>();
JSONObject jsonObject= new JSONObject(response.getContent());
response.getContent() -> is nothing but it contains the above json response in Map format.
Now I converting the map to list of json object
JSONArray jsonArray= (JSONArray)jsonObject.get("documents");
ArrayList<JSONObject> list = new ArrayList<>();
for(int i=0;i<jsonArray.length();i++){
list.add((JSONObject) jsonArray.get(i));
}
Collections.sort(list, new ResponseSorter());
public class ResponseSorter implements Comparator<JSONObject> {
#Override
public int compare(JSONObject o1,JSONObject o2){
String s1= (String)((JSONObject) o1.get("NETWORK")).get("NETID");
String s2= (String)((JSONObject) o2.get("NETWORK")).get("NETID");
int i1=Integer.parseInt(s1);
int i2=Integer.parseInt(s2);
return i1-i2;
}
}
I'm stuck here to proceed further. Created one for Integer comparator, .Should I create for each dataType? also
I need to dynamically construct the composite comparator by parsing the "retunrParameters" , below sample is hard coded, how to create dynamically??
(String)((JSONObject) o1.get("NETWORK")).get("NETID"); -> this should be dynamically framed , since "returnParameters" are also dynamic in nature.[NETWORK & NETID may not be come in another request],so my comparator should be capable enough to frame the keys in runtime
Would anyone able to assist me to create composite comparator in runtime for sorting?
NOTE:- Java Pojo cannot be created as the response is dynamic nature
In your case a simple comparator that's provided with the sort parameters might be easier to understand than a bunch of nested comparators.
Basically you'd do something like this:
class ReturnParameterComparator implements Comparator<JSONObject> {
private List<ReturnParameter> params; //set via constructor
public int compare( JSONObject left, JSONObject right) {
int result = 0;
for( ReturnParameter p : params ) {
//how exactly you get those values depends on the actual structure of your data and parameters
String leftValueStr = left.get( p );
String rightValueStr = right.get( p );
switch( p.datatype ) {
case "String":
result = String.compare( leftValueStr, rightValueStr );
break;
case "int":
//convert and then compare - I'll leave the rest for you
}
//invert the result if the order is descending
if( "desc".equals(p.order ) {
result += -1;
}
//the values are not equal so return the order, otherwise continue with the next parameter
if( result != 0 ) {
return result;
}
}
//at this point all values are to be considered equal, otherwise we'd have returned already (from the loop body)
return 0;
}
}
Note that this is just a stub to get you started. You'll need to add quite a few things:
how to correctly use the parameters to extract the values from the json objects
how to convert the data based on the type
how to handle nulls, missing or incompatible data (e.g. if a value should be sorted as "int" but it can't be parsed)
Adding all those would be way too much for the scope of this question and depends on your data and requirements anyway.
EDITED after additional questions in comments and additional info in description
You have a couple of steps you need to do here to get to the solution:
You want to have the sorting be dynamic based on the value of the property sequence in the request. So you need to parse the names of those returnParameters and put them in order. Below I map them to a List where each String[] has the name and order (asc/desc). The list will be ordered using the value of sequence:
List<String[]> sortParams = params.stream() // params is a List<JSONObject>
.filter(json -> json.containsKey("sequence")) // filter those that have "sequence" attribute
.sorted( sequence ) // sorting using Comparator called sequence
.map(jsonObj -> new String[]{jsonObj.get("name").toString(), jsonObj.get("order").toString()} )
.collect(Collectors.toList());
Before this you'll map the objects in the returnParameters array in the request to a List first.Then the stream is processed by 1. filtering the JSONObjects to only keep those that have prop sequence, 2. sorting the JSONObjects using comparator below. 3. from each JSONObject get "name" & "order" and put them in a String[], 4. generate a list with those Arrays. This list will be ordered in the order of attributes with priority 1 first, then priority 2, etc, so it will be ordered in the same way you want the JSONObjects ordered in the end.
Comparator<JSONObject> sequence = Comparator.comparingInt(
jsonObj -> Integer.valueOf( jsonObj.get("sequence").toString() )
);
So for your example, sortParams would look like: List( String[]{"NETWORK/NETID", "asc"}, String[]{""INFODATA/NAME", "asc"}, String[]{"SOURCE/SYSTEM", "asc"} )
Then you need to write a method that takes two params: a JSONObject and a String (the path to the property) and returns the value of that property. Originally I advised you to use JSONAware interface and then figure out the sub-class, but let's forget about that for now.
I am not going to write this method for you. Just keep in mind that .get(key) method of JSON.Simple always yields an Object. Write a method with this signature:
public String findSortValue(JSONObject doc, String path){
// split the path
// find the parent
// cast it (parent was returned as an Object of type Object)
// find the child
return value;
}
Write a generic individual comparator (that compares values of just one sort attribute at a time) and figures out if it's an Int, Date or regular String. I would write this as a regular method so it'll be easier to combine everything later on. Since you had so many questions about this I've made an example:
int individualComparator(String s1, String s2){
int compResult = 0;
try{
int numeric1 = Integer.parseInt(s1);
int numeric2 = Integer.parseInt(s2);
compResult = numeric1 - numeric2; // if this point was reached both values could be parsed
} catch (NumberFormatException nfe){
// if the catch block is reached they weren't numeric
try{
DateTime date1 = DateTime.parse(s1);
DateTime date2 = DateTime.parse(s2);
compResult = date1.compareTo(date2); // compareTo method of joda.time, the library I'm using
} catch (IllegalArgumentException iae){
//if this catch block is reached they weren't dates either
compResult = s1.compareTo(s2);
}
}
return compResult;
};
Write an overall Comparator that combines everything
Comparator<JSONObject> overAllComparator = (jsonObj1, jsonObj2) -> {
List<String[]> sortValuesList = sortParams.stream()
.map(path -> new String[]{ findValueByName(jsonObj1, path), findValueByName(jsonObj2, path) } )
.collect(Collectors.toList());
//assuming we always have 3 attributes to sort on
int comp1 = individualComparator(sortValuesList.get(0)[0], sortValuesList.get(0)[1]);
int comp2 = individualComparator(sortValuesList.get(1)[0], sortValuesList.get(1)[1]);
int comp3 = individualComparator(sortValuesList.get(2)[0], sortValuesList.get(2)[1]);
int result = 0;
if (comp1 != 0){
result = comp1;
} else if (comp2 != 0){
result = comp2;
} else{
result = comp3;
}
return result;
};
This Comparator is written lambda-style, for more info https://www.mkyong.com/java8/java-8-lambda-comparator-example/ .
First it takes the ordered list of sortParams we made in step 1 and for each returns an array where position 0 has the value for jsonObj1, and position 1 has the value for jsonObj2 and collects it in sortValuesList. Then for each attribute to sort on, it get the result of the individualComparatormethod. Then it goes down the line and returns as result of the overall comparison the first one that doesn't result in 0 (when a comparator results in 0 both values are equal).
The only thing that's missing now is the asc/desc value from the request. You can add that by chainingint comp1 = individualComparator(sortValuesList.get(0)[0], sortValuesList.get(0)[1]); with a simple method that takes an int & a String and multiplies the int by -1 if the String equals "desc". (Remember that in sortParams we added the value for order on position 1 of the array).
Because the first list we made, sortParams was ordered based on the priority indicated in the request, and we always did evertything in the order of this list, the result is a multi-sort in this order. It is generic & will be determined dynamically by the contents of returnParams in the request. You can apply it to your list of JSONObjects by using Collections.sort()
My suggestion: learn about:
Comparator.comparing which allows you to build your comparator by specifying the key extractor
Comparator.thanComparing which allows you to chain multiple comparators. The comparators later in the chain are called only if predecessors say the objects are equal
A tutorial if you need one: https://www.baeldung.com/java-8-comparator-comparing
I have a test.csv file that is formatted as:
Home,Owner,Lat,Long
5th Street,John,5.6765,-6.56464564
7th Street,Bob,7.75,-4.4534564
9th Street,Kyle,4.64,-9.566467364
10th Street,Jim,14.234,-2.5667564
I have a hashmap that reads a file that contains the same header contents such as the CSV, just a different format, with no accompanying data.
In example:
Map<Integer, String> container = new HashMap<>();
where,
Key, Value
[0][NULL]
[1][Owner]
[2][Lat]
[3][NULL]
I have also created a second hash map that:
BufferedReader reader = new BufferedReader (new FileReader("test.csv"));
CSVParser parser = new CSVParser(reader, CSVFormat.DEFAULT);
Boolean headerParsed = false;
CSVRecord headerRecord = null;
int i;
Map<String,String> value = new HashMap<>();
for (final CSVRecord record : parser) {
if (!headerParsed = false) {
headerRecord = record;
headerParsed = true;
}
for (i =0; i< record.size(); i++) {
value.put (headerRecord.get(0), record.get(0));
}
}
I want to read and compare the hashmap, if the container map has a value that is in the value map, then I put that value in to a corresponding object.
example object
public DataSet (//args) {
this.home
this.owner
this.lat
this.longitude
}
I want to create a function where the data is set inside the object when the hashmaps are compared and when a value map key is equal to a contain map key, and the value is placed is set into the object. Something really simply that is efficient at handling the setting as well.
Please note: I made the CSV header and the rows finite, in real life, the CSV could have x number of fields(Home,Owner,Lat,Long,houseType,houseColor, ect..), and a n number of values associated to those fields
First off, your approach to this problem is too unnecessarily long. From what I see, all you are trying to do is this:
Select a two columns from a CSV file, and add them to a data structure. I highlighted the word two because in a map, you have a key and a value. One column becomes the key, and the other becomes the value.
What you should do instead:
Import the names of columns you wish to add to the data structure into two strings. (You may read them from a file).
Iterate over the CSV file using the CSVParser class that you did.
Store the value corresponding to the first desired column in a string, repeat with the value corresponding to the second desired column, and push them both into a DataSet object, and push the DataSet object into a List<DataSet>.
If you prefer to stick to your way of solving the problem:
Basically, the empty file is supposed to hold just the headers (column names), and that's why you named the corresponding hash map containers. The second file is supposed to contain the values and hence you named the corresponding hash map values.
First off, where you say
if (!headerParsed = false) {
headerRecord = record;
headerParsed = true;
}
you probably mean to say
if (!headerParsed) {
headerRecord = record;
headerParsed = true;
}
and where you say
for (i =0; i< record.size(); i++) {
value.put(headerRecord.get(0), record.get(0));
}
you probably mean
for (i =0; i< record.size(); i++) {
value.put(headerRecord.get(i), record.get(i));
}
i.e. You iterate over one record and store the value corresponding to each column.
Now I haven't tried this code on my desktop, but since the for loop also iterates over Home and Longitude, I think it should create an error and you should add an extra check before calling value.put (i.e. value.put("Home", "5th Street") should create an error I suppose). Wrap it inside an if conditional and check of the headerRecord(i) even exists in the containers hash map.
for (i =0; i< record.size(); i++) {
if (container[headerRecord.get(i)] != NULL) {
value.put(headerRecord.get(i), record.get(i));
}
}
Now thing is, that the data structure itself depends on which values from the containers hash map you want to store. It could be Home and Lat, or Owner and Long. So we are stuck. How about you create a data structure like below:
struct DataSet {
string val1;
string val2;
}
Also, note that this DataSet is only for storing ONE row. For storing information from multiple rows, you need to create a Linked List of DataSet.
Lastly, the container file contains ALL the column names. Not all these columns will be stored in the Data Set (i.e. You chose to NULL Home and Long. You could have chosen to NULL Owner and Lat), hence the header file is not what you need to make this decision.
If you think about it, just iterate over the values hash map and store the first value in string val1 and the second value in val2.
List<DataSet> myList;
DataSet row;
Iterator it = values.entrySet().iterator();
while (it.hasNext()) {
Map.Entry pair = (Map.Entry)it.next();
row.val1 = pair.getKey();
row.val2 = pair.getValue();
myList.add(row);
it.remove();
}
I hope this helps.
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
How can I obtain values of key-value pair from a Hashmap?
QueryStats queryStats=new QueryStats();
Map parameterMap = request.getParameterMap()==null? null:new HashMap(request.getParameterMap());
System.out.println("Query2:" + parameterMap);
Collection newParamsValue=parameterMap.values();
Object newParams[]=newParamsValue.toArray();
StringBuffer strParam=new StringBuffer() ;
int l=newParams.length;
for(int i=0;i<l;i++){
strParam=strParam.append(newParams[i].toString());
}
output: Query2:{sortPrefix=[Ljava.lang.String;#1d66aa9, searsOnly=[Ljava.lang.String;#1f4bdca, globalPrefix=[Ljava.lang.String;#d81cda, indent=[Ljava.lang.String;#4e57ba, qt=[Ljava.lang.String;#1619137, wt=[Ljava.lang.String;#84c1f9, fq=[Ljava.lang.String;#1dae27f}
From this parameterMap, I want to create a query? How is it possible?
If the request is a HTTP request (and I suspect it is) and the query is a database query, then what you're trying to do is a big no-no. Always, always, always sanitize your input. Just stop for a moment and think with the mind of a malicious person trying to wreck or obtain your database content, trying to forge a HTTP request that fools your unsuspecting application into doing exactly that.
Another thing about HTTP request parameters is that they come in string arrays. If you want the string value of an input, you need to cast your values to String[] first, then grab the first element of that array. Please note that the array can be empty though, and this method only works for single value inputs, multiple selection inputs will return multiple elements.
The problem you face is that the map returned from getParameterMap maps String keys to arrays of String values. So each element of newParamsValue is a String[]. You need to do a little more work to get a sensible (comma-separated) string out of these than calling toString(). Something like this (untested) might do what you want:
ArrayList<String> toStrings(Collection requestValues) {
ArrayList<String> strings;
for (Object object : requestValues) {
String[] vals = (String[]) object;
if (vals.length() == 1) {
strings.add(vals[0]);
} else if (vals.length() > 1) {
StringBuilder sb = new StringBuilder(vals[0]);
for (int i = 1; i < vals.length(); ++i) {
sb.append(',');
sb.append(vals[i]);
}
strings.add(sb.toString());
}
}
return strings;
}
I need to be able to sort multiple intermediate result sets and enter them to a file in sorted order. Sort is based on a single column/key value. Each result set record will be list of values (like a record in a table)
The intermediate result sets are got by querying entirely different databases.
The intermediate result sets are already sorted based on some key(or column). They need to be combined and sorted again on the same key(or column) before writing it to a file.
Since these result sets can be massive(order of MBs) this cannot be done in memory.
My Solution broadly :
To use a hash and a random access file . Since the result sets are already sorted, when retrieving the result sets , I will store the sorted column values as keys in a hashmap.The value in the hashmap will be a address in the random access file where every record associated with that column value will be stored.
Any ideas ?
Have a pointer into every set, initially pointing to the first entry
Then choose the next result from the set, that offers the lowest entry
Write this entry to the file and increment the corresponding pointer
This approach has basically no overhead and time is O(n). (it's Merge-Sort, btw)
Edit
To clarify: It's the merge part of merge sort.
If you've got 2 pre-sorted result sets, you should be able to iterate them concurrently while writing the output file. You just need to compare the current row in each set:
Simple example (not ready for copy-and-paste use!):
ResultSet a,b;
//fetch a and b
a.first();
b.first();
while (!a.isAfterLast() || !b.isAfterLast()) {
Integer valueA = null;
Integer valueB = null;
if (a.isAfterLast()) {
writeToFile(b);
b.next();
}
else if (b.isAfterLast()) {
writeToFile(a);
a.next();
} else {
int valueA = a.getInt("SORT_PROPERTY");
int valueB = b.getInt("SORT_PROPERTY");
if (valueA < valueB) {
writeToFile(a);
a.next();
} else {
writeToFile(b);
b.next();
}
}
}
Sounds like you are looking for an implementation of the Balance Line algorithm.