How Should I read CSV By Grouping Column of Same Value - java

I have a task to read a CSV file then do some logic and then create a JSON for that.
I am kind of stuck with required logic before creating JSON, I need to set Max PR VALUE against the SK as PR for all the same SK's.
My Requirement:
CSV:
SK,VR,ST,PR
1000,1000-Q1,10,187
1000,1000-Q2,20,925 // MAX PR against SK
1001,1001-Q1,10,112
1001,1001-Q2,30,120 // MAX PR against SK
Note: Max PR against SK will always be in the last row of its SK.
I have to read CSV here and need to write JSON data as below :
[
{
"SK": "1000",
"VR": "1000-Q1",
"ST": "10",
"PR": "925"
},
{
"SK": "1000",
"VR": "1000-Q2",
"ST": "20",
"PR": "925"
},
{
"SK": "1001",
"VR": "1001-Q1",
"ST": "10",
"PR": "120"
},
{
"SK": "1001",
"VR": "1001-Q2",
"ST": "30",
"PR": "120"
}
]
Edit:
Code
File input = new File("input.csv");
File output = new File("output.json");
CsvSchema csvSchema = CsvSchema.builder().setUseHeader(true).build();
CsvMapper csvMapper = new CsvMapper();
// Read data from CSV file
List<Object> readAll = csvMapper.readerFor(Map.class).with(csvSchema).readValues(input).readAll();
ObjectMapper mapper = new ObjectMapper();
// Write JSON formated data to output.json file
mapper.writerWithDefaultPrettyPrinter().writeValue(output, readAll);
// Write JSON formated data to stdout
System.out.println(mapper.writerWithDefaultPrettyPrinter().writeValueAsString(readAll));

One Approach would be to first group your CSV Records per SK
String[] HEADERS = { "SK","VR","ST","PR"};
Reader in = new FileReader("mycsvfile.csv");
Iterable<CSVRecord> records = CSVFormat.DEFAULT
.withHeader(HEADERS)
.withFirstRecordAsHeader()
.parse(in);
// Group the records by SK
Map<String, List<CSVRecord>> recordListBySK = StreamSupport
.stream(records.spliterator(), false).
collect(Collectors.groupingBy(record -> record.get("SK")));
Then you need to add another a Mapping this time, which keeps the MAX-PR per
each Sk
Map<String, Integer> skMaxMap = recordListBySK
.entrySet()
.stream()
.collect(Collectors
.toMap( e -> e.getKey(),
e -> e.getValue()
.stream()
.mapToInt(v -> Integer.parseInt(v.get("PR")))
.max()
.getAsInt()
)
);
Now you simply build you json Sk list like this:
// Building the new sk (JSON ) objects
List<NewSk> newSkList = new ArrayList<>();
recordListBySK
.values()
.stream()
.flatMap(v -> v.stream())
.forEach(csvRecord -> {
NewSk newSk = new NewSk(csvRecord.get("SK"),
csvRecord.get("VR"),
csvRecord.get("ST"),
skMaxMap.get(csvRecord.get("SK"))
);
newSkList.add(newSk);
});
if you try to print them out:
newSkList.forEach(sk -> {
System.out.print(" "+sk.getSk());
System.out.print(" "+sk.getVr());
System.out.print(" "+sk.getSt());
System.out.print(" "+sk.getPr());
System.out.println(" ");
});
you'll get this
1001 1001-Q1 10 120
1001 1001-Q2 30 120
1000 1000-Q1 10 925
1000 1000-Q2 20 925
No you can write your List to your JSON file using your JSON Object Mapper.
Hope it helps
EDIT:
public class NewSk {
private String sk;
private String vr;
private String st;
private String pr;
public NewSk(String sk, String vr, String st, String pr) {
this.sk = sk;
this.vr = vr;
this.st = st;
this.pr = pr;
}
public String getSk() {
return sk;
}
public void setSk(String sk) {
this.sk = sk;
}
public String getVr() {
return vr;
}
public void setVr(String vr) {
this.vr = vr;
}
public String getSt() {
return st;
}
public void setSt(String st) {
this.st = st;
}
public String getPr() {
return pr;
}
public void setPr(String pr) {
this.pr = pr;
}
}

First, I've made a pojo to manipulate easily data :
public class Model {
private final int sk;
private final String vr;
private final int st;
private final int pr;
public Model(int sk, String vr, int st, int pr) {
this.sk = sk;
this.vr = vr;
this.st = st;
this.pr = pr;
}
public int getSk() { return sk; }
public String getVr() { return vr; }
public int getSt() { return st; }
public int getPr() { return pr; }
#Override
public String toString() {
return "Model{" +
"sk=" + sk +
", vr='" + vr + '\'' +
", st=" + st +
", pr=" + pr +
'}';
}
}
After, with Java 8 streaming you can do chat you want :
List<Model> models = new ArrayList<>();
models.add(new Model(1000, "1000-Q1", 10, 187));
models.add(new Model(1000, "1000-Q2", 10, 925));
models.add(new Model(1001, "1001-Q1", 10, 112));
models.add(new Model(1001, "1001-Q2", 30, 120));
List<Model> collect = models.stream()
.collect(Collectors.groupingBy(Model::getSk))
.entrySet()
.stream()
.flatMap(
v -> {
int max = v.getValue()
.stream()
.map(m -> m.getPr())
.reduce((a, b) -> Math.max(a, b))
.get();
return v.getValue().stream()
.map(m -> new Model(m.getSk(), m.getVr(), m.getSt(), max));
}
)
.collect(Collectors.toList());
System.out.println(collect);
Result:
[
{sk=1000, vr='1000-Q1', st=10, pr=925},
{sk=1000, vr='1000-Q2', st=10, pr=925},
{sk=1001, vr='1001-Q1', st=10, pr=120},
{sk=1001, vr='1001-Q2', st=30, pr=120}
]

Related

Java/Stream Help: Convert a nested list of maps to a map using streams only

I have a nested list of maps as below
Example List of Maps:
[{owner=trader1, ord_id=[ord1, ord2], watchlist=trader1_WL}, {owner=trader2, ord_id=[ord3, ord4], watchlist=trader2_WL}]
I need to convert it into a map of ord_id:owner
Output Required:
{ord1:trader1, ord2:trader1, ord3:trader2, ord4.trader2}
Sample Code:[Doesn't work as ord_id has list of items]
import java.util.*;
import java.util.stream.Collectors;
public class Main
{
public static void main (String[]args)
{
Map < String, Object > order1 = new HashMap <> ();
order1.put ("ord_id", Arrays.asList ("ord1", "ord2"));
order1.put ("owner", "trader1");
order1.put ("watchlist", "trader1_WL");
Map < String, Object > order2 = new HashMap <> ();
order2.put ("ord_id", Arrays.asList ("ord3", "ord4"));
order2.put ("owner", "trader2");
order2.put ("watchlist", "trader2_WL");
List < Map < String, Object >> open_order_list = new ArrayList <> ();
open_order_list.add (order1);
open_order_list.add (order2);
System.out.println (open_order_list);
Map < String, String > result =
open_order_list.stream ().collect (Collectors.
toMap (s->(String) s.get ("ord_id"),
s->(String) s.get ("owner")));
System.out.println (result);
}
}
It's ugly, but you can do it like this:
Map<String, String> result = open_order_list.stream()
.flatMap(obj -> {
String owner = (String) obj.get("owner");
#SuppressWarnings("unchecked")
List<String> ord_id = (List<String>) obj.get("ord_id");
return ord_id.stream().map(id -> new String[] { owner, id });
})
.collect(Collectors.toMap(a -> a[1], a -> a[0]));
The code would be a lot better if you parsed the JSON text into POJOs, instead of generic collection objects.
List<Order> open_order_list = Arrays.asList(
new Order("trader1", "trader1_WL", "ord1","ord2"),
new Order("trader2", "trader2_WL", "ord3","ord4") );
System.out.println(open_order_list);
Map<String, String> result = open_order_list.stream()
.flatMap(ord -> ord.getOrd_id().stream().map(id -> new String[] { ord.getOwner(), id }))
.collect(Collectors.toMap(a -> a[1], a -> a[0]));
System.out.println (result);
class Order {
private final String owner;
private final String watchlist;
private final List<String> ord_id;
public Order(String owner, String watchlist, String... ord_id) {
this.owner = owner;
this.watchlist = watchlist;
this.ord_id = Arrays.asList(ord_id);
}
public String getOwner() {
return this.owner;
}
public String getWatchlist() {
return this.watchlist;
}
public List<String> getOrd_id() {
return this.ord_id;
}
#Override
public String toString() {
return "Order[owner=" + this.owner + ", watchlist=" + this.watchlist + ", ord_id=" + this.ord_id + "]";
}
}

How can I simplify if else using java 8 aproach

public Pair<String, String> getSalesChannelDisplayData(DiscountRule rule, List<SalesChannelDto> allSalesChannels) {
String salesChannelDisplayNames = "";
String salesChannelDefaultCountryCodes = "";
Set<String> storeCodes = new HashSet<>();
if(rule.getConditions() != null) {
for (Condition condition : rule.getConditions()) {
if (condition instanceof ValueCondition) {
if (((ValueCondition) condition).getField() == Field.SALES_CHANNEL) {
Set<String> salesChannelIds = new HashSet<>();
if(((ValueCondition) condition).getOperator().equals(Operator.IN)){
salesChannelIds = ((ValueCondition) condition).getValues();
}else if (((ValueCondition) condition).getOperator().equals(Operator.NOT_IN)) {
salesChannelIds = allSalesChannels.stream().map(SalesChannelDto::getId).collect(Collectors.toSet());
salesChannelIds.removeAll(((ValueCondition) condition).getValues());
}
for (String salesChannelId : salesChannelIds) {
SalesChannelDto salesChannel = Beans.find(allSalesChannels, s-> s.getId().equals(salesChannelId));
salesChannelDisplayNames += salesChannel.getDisplayName() + ", ";
storeCodes.add(salesChannel.getDefaultCountryCode());
}
}
}
}
if (salesChannelDisplayNames.length()>1) {
salesChannelDisplayNames = salesChannelDisplayNames.substring(0,salesChannelDisplayNames.length()-2);
salesChannelDefaultCountryCodes = Joiner.on(", ").join(storeCodes);
}
return new Pair<>(salesChannelDisplayNames, salesChannelDefaultCountryCodes);
}
I want to simplify the above code using java stream API. Is that possible for replace the if, else if with java 8 approach?
The stream API is not a good choice to simplify your code. There are some parts in your code that you can modify them.
1- Not to need to check rule.getConditions() nullity.
if(rule.getConditions() != null) {...}
2- Don't repeat yourself by this: ((ValueCondition) condition) instead you can define a variable for it and use it.
ValueCondition vCondition = (ValueCondition) condition;
3- Instead concatenating salesChannelDisplayNames declare a List<String> salesChannelNames = new ArrayList<>(); and add channelName into it.
salesChannelNames.add(salesChannel.getDisplayName());
at the end use String.join(",", salesChannelNames) to add , delimeter between them.
This is a sample you can try out. I have tried to completely eliminate if-else.
public class FunctionalIfElse {
public static void main(String[] args) {
Product product1 = new Product(1, "Audi A8");
String category1 = "car";
System.out.println(ProductProxy.getEnrichedProduct.apply(product1, category1).toString());
Product product2 = new Product(2, "OnePlus 8 Pro");
String category2 = "mobile";
System.out.println(ProductProxy.getEnrichedProduct.apply(product2, category2).toString());
Product product3 = new Product(3, "Macbook Pro");
String category3 = "laptop";
System.out.println(ProductProxy.getEnrichedProduct.apply(product3, category3).toString());
Product product4 = new Product(4, "Emaar Palm Heights");
String category4 = "home";
System.out.println(ProductProxy.getEnrichedProduct.apply(product4, category4).toString());
}
}
#AllArgsConstructor
#Data
class Product {
private int productId;
private String productName;
}
class ProductProxy {
static BiFunction<Product, String, Product> getEnrichedProduct = (inputProduct, category) -> {
AtomicReference<Product> outputProduct = new AtomicReference<>();
Objects.requireNonNull(category, "The category is null");
Predicate<String> checkIsCar = productCategory -> productCategory.equalsIgnoreCase("car") ? true : false;
Predicate<String> checkIsMobile = productCategory -> productCategory.equalsIgnoreCase("mobile") ? true : false;
Predicate<String> checkIsLaptop = productCategory -> productCategory.equalsIgnoreCase("laptop") ? true : false;
Optional.ofNullable(category).filter(checkIsCar).map(input -> ProductService.enrichProductForCar.apply(inputProduct)).map(Optional::of).ifPresent(returnedProduct -> outputProduct.set(returnedProduct.get()));
Optional.ofNullable(category).filter(checkIsMobile).map(input -> ProductService.enrichProductForMobile.apply(inputProduct)).map(Optional::of).ifPresent(returnedProduct -> outputProduct.set(returnedProduct.get()));
Optional.ofNullable(category).filter(checkIsLaptop).map(input -> ProductService.enrichProductForLaptop.apply(inputProduct)).map(Optional::of).ifPresent(returnedProduct -> outputProduct.set(returnedProduct.get()));
Optional.ofNullable(outputProduct.get()).orElseThrow(() -> new RuntimeException("This is not a valid category"));
return outputProduct.get();
};
}
class ProductService {
static Function<Product, Product> enrichProductForCar = inputProduct -> {
inputProduct.setProductName(inputProduct.getProductName() + ":Car");
return inputProduct;
};
static Function<Product, Product> enrichProductForMobile = inputProduct -> {
inputProduct.setProductName(inputProduct.getProductName() + ":Mobile");
return inputProduct;
};
static Function<Product, Product> enrichProductForLaptop = inputProduct -> {
inputProduct.setProductName(inputProduct.getProductName() + ":Laptop");
return inputProduct;
};
}

Using array as key for hashmap java

i have a method that puts some value(obtained from an excel file) into a hashmap with an array as the key
public HashMap<List<String>, List<String[]>> sbsBusServiceDataGnr() throws
IOException
{
System.out.println(engine.txtY + "Processing HashMap "
+ "sbsBusServiceData..." + engine.txtN);
int counterPass = 0, counterFail = 0, stopCounter = 0;
String dataExtract, x = "";
String[] stopInfo = new String[3];
List<String[]> stopsData = new ArrayList<String[]>();
List<String> serviceNum = new Vector<String>();
HashMap<List<String>, List<String[]>> sbsBusServiceData =
new HashMap<List<String>, List<String[]>>();
String dataPath = this.dynamicPathFinder(
"Data\\SBS_Bus_Routes.csv");
BufferedReader sbsBusServiceDataPop = new BufferedReader(
new FileReader(dataPath));
sbsBusServiceDataPop.readLine();
//Skips first line
while ((dataExtract = sbsBusServiceDataPop.readLine()) != null) {
try {
String[] dataParts = dataExtract.split(",", 5);
if (!dataParts[4].equals("-")){
if (Double.parseDouble(dataParts[4]) == 0.0){
sbsBusServiceData.put(serviceNum, stopsData);
String serviceNum1 = "null", serviceNum2 = "null";
if(!serviceNum.isEmpty()){
serviceNum1 = serviceNum.get(0);
serviceNum2 = serviceNum.get(1);
}
System.out.println("Service Number " + serviceNum1
+ ":" + serviceNum2 + " with " + stopCounter
+ " stops added.");
stopCounter = 0;
//Finalizing previous service
serviceNum.Clear();
serviceNum.add(0, dataParts[0]);
serviceNum.add(1, dataParts[1]);
//Adding new service
}
}
stopInfo[0] = dataParts[2];
stopInfo[1] = dataParts[3];
stopInfo[2] = dataParts[4];
stopsData.add(stopInfo);
//Adding stop to service
stopCounter++;
counterPass++;
}
catch (Exception e) {
System.out.println(engine.txtR + "Unable to process "
+ dataExtract + " into HashMap sbsBusServiceData."
+ engine.txtN + e);
counterFail++;
}
}
sbsBusServiceDataPop.close();
System.out.println(engine.txtG + counterPass + " number of lines"
+ " processed into HashMap sbsBusServiceData.\n" + engine.txtR
+ counterFail + " number of lines failed to process into "
+ "HashMap sbsBusServiceData.");
return sbsBusServiceData;
}
//Generates sbsBusServiceDataGnr HashMap : 15376 Data Rows
//HashMap Contents: {ServiceNumber, Direction},
// <{RouteSequence, bsCode, Distance}>
this method work for putting the values into the hashmap but i cannot seem to get any value from the hashmap when i try to call it there is always a nullpointerexception
List<String> sbsTest = new Vector<String>();
sbsTest.add(0, "10");
sbsTest.add(1, "1");
System.out.println(sbsBusServiceData.get(sbsTest));
try{
List<String[]> sbsServiceResults = sbsBusServiceData.get(sbsTest);
System.out.println(sbsServiceResults.size());
String x = sbsServiceResults.get(1)[0];
System.out.println(x);
} catch(Exception e){
System.out.println(txtR + "No data returned" + txtN + e);
}
this is a sample of the file im reading the data from:
SBS
How can i get the hashmap to return me the value i want?
Arrays are not suitable as keys in HashMaps, since arrays don't override Object's equals and hashCode methods (which means two different array instances containing the exact same elements will be considered as different keys by HashMap).
The alternatives are to use a List<String> instead of String[] as the key of the HashMap, or to use a TreeMap<String[]> with a custom Comparator<String[]> passed to the constructor.
If you are having fixed array size then the example I'm posting might be useful.
Here I've created two Object one is Food and Next is Product.
Here Food object is use and added method to get string array.
public class Product {
private String productName;
private String productCode;
public Product(String productName, String productCode) {
this.productName = productName;
this.productCode = productCode;
}
public String getProductName() {
return productName;
}
public void setProductName(String productName) {
this.productName = productName;
}
public String getProductCode() {
return productCode;
}
public void setProductCode(String productCode) {
this.productCode = productCode;
}
}
Food Model Class: Use as a Object instead of String[] and achieve String[] functionality.
public class Food implements Comparable<Food> {
private String type;
private String consumeApproach;
public Food(String type, String consumeApproach) {
this.type = type;
this.consumeApproach = consumeApproach;
}
public String getType() {
return type;
}
public void setType(String type) {
this.type = type;
}
public String getConsumeApproach() {
return consumeApproach;
}
public void setConsumeApproach(String consumeApproach) {
this.consumeApproach = consumeApproach;
}
public String[] FoodArray() {
return new String[] { this.type, this.consumeApproach };
}
//Implement compareTo method as you want.
#Override
public int compareTo(Food o) {
return o.getType().compareTo(this.type);
}
}
Using HashMap example
public class HashMapKeyAsArray {
public static void main(String[] args) {
HashMap<Food,List<Product>> map = dataSetLake();
map.entrySet().stream().forEach(m -> {
String[] food = m.getKey().FoodArray();
Arrays.asList(food).stream().forEach(f->{
System.out.print(f + " ");
});
System.out.println();
List<Product> list = m.getValue();
list.stream().forEach(e -> {
System.out.println("Name:" + e.getProductName() + " Produc Code:" + e.getProductCode());
});
System.out.println();
});
}
private static HashMap<Food,List<Product>> dataSetLake(){
HashMap<Food,List<Product>> data = new HashMap<>();
List<Product> fruitA = new ArrayList<>();
fruitA.add(new Product("Apple","123"));
fruitA.add(new Product("Banana","456"));
List<Product> vegetableA = new ArrayList<>();
vegetableA.add(new Product("Potato","999"));
vegetableA.add(new Product("Tomato","987"));
List<Product> fruitB = new ArrayList<>();
fruitB.add(new Product("Apple","123"));
fruitB.add(new Product("Banana","456"));
List<Product> vegetableB = new ArrayList<>();
vegetableB.add(new Product("Potato","999"));
vegetableB.add(new Product("Tomato","987"));
Food foodA = new Food("Fruits","Read To Eat");
Food foodB = new Food("Vegetables","Need To Cook");
Food foodC = new Food("VegetablesC","Need To Cook C");
data.put(foodA, fruitB);
data.put(foodB, vegetableB);
data.put(foodA, fruitA);
data.put(foodC, vegetableA);
return data;
}
Using TreeMap example
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.TreeMap;
public class TreeMapKeyAsArray {
public static void main(String[] args) {
TreeMap<Food, List<Product>> map = dataSetLake();
map.entrySet().stream().forEach(m -> {
String[] food = m.getKey().FoodArray();
Arrays.asList(food).stream().forEach(f->{
System.out.print(f + " ");
});
System.out.println();
List<Product> list = m.getValue();
list.stream().forEach(e -> {
System.out.println("Name:" + e.getProductName() + " Produc Code:" + e.getProductCode());
});
System.out.println();
});
}
private static TreeMap<Food, List<Product>> dataSetLake() {
TreeMap<Food, List<Product>> data = new TreeMap<>();
List<Product> fruitA = new ArrayList<>();
fruitA.add(new Product("Apple", "123"));
fruitA.add(new Product("Banana", "456"));
List<Product> vegetableA = new ArrayList<>();
vegetableA.add(new Product("Potato", "999"));
vegetableA.add(new Product("Tomato", "987"));
List<Product> fruitB = new ArrayList<>();
fruitB.add(new Product("Apple", "123"));
fruitB.add(new Product("Banana", "456"));
List<Product> vegetableB = new ArrayList<>();
vegetableB.add(new Product("Potato", "999"));
vegetableB.add(new Product("Tomato", "987"));
Food foodA = new Food("Fruits", "Read To Eat");
Food foodB = new Food("Vegetables", "Need To Cook");
data.put(foodA, fruitB);
data.put(foodB, vegetableB);
data.put(foodA, fruitA);
data.put(foodB, vegetableA);
return data;
}
}

Elasticsearch retrieves data from bad the index

I do not know why elasticsearch does not set the appropriate values for the index, and type. He needs to retrieve data from the index = auctions and type = auctions like this is in Model:
AuctionIndex.java:
#Document(indexName = "auctions", type = "auctions")
public class AuctionIndex {
#Id
private String id;
private Long cat;
private Long tcat;
private String curr;
private Long price;
private Long start_date;
private Long end_date;
private String title;
private String pow;
private String woj;
private String loc;
private String cat_name;
private Long catdec;
private Long uid;
private Long qty;
...getters and setters...
}
This code works when downloading data as follows:
public Map searchByIndexParams(AuctionIndexSearchParams searchParams, Pageable pageable) {
Map response = new HashMap();
NativeSearchQuery searchQuery = this.getSearchQuery(searchParams, pageable).build();
final FacetedPage<AuctionIndex> search = auctionIndexRepository.search(searchQuery);
List<AuctionIndex> content = search.getContent();
response.put("content", content.stream().map(row -> {
return Auction.builder()
.cat(row.getCat())
.item(Long.parseLong(row.getId()))
.endts(row.getEnd_date())
.startts(row.getStart_date())
.loc(row.getLoc())
.pow(row.getPow())
.woj(row.getWoj())
.price(row.getPrice())
.qty(row.getQty())
.title(row.getTitle())
.user(row.getUid())
.catName(row.getCat_name())
.build();
}).collect(Collectors.toList()));
response.put("first", search.isFirst());
response.put("last", search.isLast());
response.put("number", search.getNumber());
response.put("numberOfElements", search.getNumberOfElements());
response.put("size", search.getSize());
response.put("sort", search.getSort());
response.put("totalElements", search.getTotalElements());
response.put("totalPages", search.getTotalPages());
return response;
}
By downloading all the records in this way:
public Map findAllByIndexParams(AuctionIndexSearchParams searchParams, Pageable pageable) {
List rows = new ArrayList();
Map response = new HashMap();
final List<FilterBuilder> filters = Lists.newArrayList();
final NativeSearchQueryBuilder searchQuery = new NativeSearchQueryBuilder().withQuery(matchAllQuery());
Optional.ofNullable(searchParams.getCategoryId()).ifPresent(v -> filters.add(boolFilter().must(termFilter("cat", v))));
Optional.ofNullable(searchParams.getCurrency()).ifPresent(v -> filters.add(boolFilter().must(termFilter("curr", v))));
Optional.ofNullable(searchParams.getTreeCategoryId()).ifPresent(v -> filters.add(boolFilter().must(termFilter("tcat", v))));
Optional.ofNullable(searchParams.getUid()).ifPresent(v -> filters.add(boolFilter().must(termFilter("uid", v))));
final BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
//access for many uids
if (searchParams.getUids() != null) {
if (searchParams.getItemId() != null || searchParams.getTitle() != null) {
Optional.ofNullable(searchParams.getUids().split(",")).ifPresent(v -> {
filters.add(boolFilter().must(termsFilter("uid", v)));
});
} else {
for (String user : searchParams.getUids().split(",")) {
boolQueryBuilder.should(queryStringQuery(user).field("uid"));
}
}
}
//access for many categories
if (searchParams.getCategories() != null) {
Optional.ofNullable(searchParams.getCategories().split(",")).ifPresent(v -> {
filters.add(boolFilter().must(termsFilter("cat", v)));
});
}
if (searchParams.getItemId() != null) {
boolQueryBuilder.must(queryStringQuery(searchParams.getItemId()).field("_id"));
}
if (Optional.ofNullable(searchParams.getTitle()).isPresent()) {
boolQueryBuilder.must(queryStringQuery(searchParams.getTitle()).analyzeWildcard(true).field("title"));
}
if (searchParams.getLoc() != null) {
filters.add(boolFilter().must(termsFilter("loc", searchParams.getLoc())));
// boolQueryBuilder.must(queryStringQuery(searchParams.getLoc()).field("loc"));
}
if (searchParams.getPow() != null) {
filters.add(boolFilter().must(termsFilter("pow", searchParams.getPow())));
// boolQueryBuilder.must(queryStringQuery(searchParams.getPow()).field("pow"));
}
if (searchParams.getWoj() != null) {
filters.add(boolFilter().must(termsFilter("woj", searchParams.getWoj())));
// boolQueryBuilder.must(queryStringQuery(searchParams.getWoj()).field("woj"));
}
if (searchParams.getCatdec() != null) {
boolQueryBuilder.must(queryStringQuery(String.valueOf(searchParams.getCatdec())).field("catdec"));
}
if (Optional.ofNullable(searchParams.getStartDateFrom()).isPresent()
|| Optional.ofNullable(searchParams.getStartDateTo()).isPresent()) {
filters.add(rangeFilter("start_date").from(searchParams.getStartDateFrom()).to(searchParams.getStartDateTo()));
}
if (Optional.ofNullable(searchParams.getEndDateFrom()).isPresent()
|| Optional.ofNullable(searchParams.getEndDateTo()).isPresent()) {
filters.add(rangeFilter("end_date").from(searchParams.getEndDateFrom()).to(searchParams.getEndDateTo()));
}
if (Optional.ofNullable(searchParams.getPriceFrom()).isPresent()
|| Optional.ofNullable(searchParams.getPriceTo()).isPresent()) {
filters.add(rangeFilter("price").from(searchParams.getPriceFrom()).to(searchParams.getPriceTo()));
}
searchQuery.withQuery(boolQueryBuilder);
FilterBuilder[] filterArr = new FilterBuilder[filters.size()];
filterArr = filters.toArray(filterArr);
searchQuery.withFilter(andFilter(filterArr));
if (searchParams.getOrderBy() != null && searchParams.getOrderDir() != null) {
if (searchParams.getOrderDir().toLowerCase().equals("asc")) {
searchQuery.withSort(SortBuilders.fieldSort(searchParams.getOrderBy()).order(SortOrder.ASC));
} else {
searchQuery.withSort(SortBuilders.fieldSort(searchParams.getOrderBy()).order(SortOrder.DESC));
}
}
String scrollId = searchTemplate.scan(searchQuery.build(), 100000, false);
System.out.println(scrollId);
Page<AuctionIndex> page = searchTemplate.scroll(scrollId, 500000, AuctionIndex.class);
System.out.println(page.getTotalElements());
if (page.hasContent()) {
while (true) {
for (AuctionIndex hit : page.getContent()) {
Auction row = Auction.builder()
.cat(hit.getCat())
.item(Long.parseLong(hit.getId()))
.endts(hit.getEnd_date())
.startts(hit.getStart_date())
.loc(hit.getLoc())
.pow(hit.getPow())
.woj(hit.getWoj())
.price(hit.getPrice())
.qty(hit.getQty())
.title(hit.getTitle())
.user(hit.getUid())
.catName(hit.getCat_name())
.build();
rows.add(row);
}
page = searchTemplate.scroll(scrollId, 500000, AuctionIndex.class);
if (page.hasContent() == false) {
break;
}
}
}
response.put("content", rows);
return response;
}
AuctionService.java:
private AuctionRepository auctionRepository;
private AuctionIndexRepository auctionIndexRepository;
#Autowired
public AuctionService(AuctionRepository auctionRepository, AuctionIndexRepository auctionIndexRepository) {
this.auctionRepository = auctionRepository;
this.auctionIndexRepository = auctionIndexRepository;
}
#Autowired
private ElasticsearchTemplate searchTemplate;
AuctionIndexRepository.java:
public interface AuctionIndexRepository extends ElasticsearchRepository<AuctionIndex, Integer> {
}
AuctionRepository.java:
#Repository
public class AuctionRepository {
private final AerospikeClient aerospikeClient;
#Autowired
public AuctionRepository(AerospikeClient aerospikeClient) {
this.aerospikeClient = aerospikeClient;
}
/**
*
* #param auctionId
* #param transactionIndexId
* #return
*/
public Map findTransactionAuctionById(Long auctionId, String transactionIndexId) {
final Statement stmt = new Statement();
stmt.setNamespace(NAMESPACE_ALLEK);
stmt.setSetName(SET_U);
final Map<String, Object> aMap = findAuctionUserInSetA(auctionId);
final Key uKey = new Key(NAMESPACE_ALLEK, SET_U, aMap.get("u") + "_" + aMap.get("p"));
final Object uRecord = aerospikeClient.execute(null, uKey, NAMESPACE_ALLEK, FUN_FIND_U_ITEM, Value.get(auctionId));
return parseTransactionAuction((HashMap) uRecord, auctionId, transactionIndexId);
}
/**
*
* #param r
* #return
*/
private Map parseTransactionAuction(HashMap r, Long auctionId, String transactionIndexId) {
return parseTransactionAuction(new Record(r, 0, 0), auctionId, transactionIndexId);
}
/**
*
* #param r rekord z aerospike
* #return
* #return
*/
private Map parseTransactionAuction(Record r, Long auctionId, String transactionIndexId) {
Map response = new HashMap();
final Object recordTrans = r.getValue("t");
final ArrayList<HashMap> trans = Optional.ofNullable(recordTrans).isPresent() ? (ArrayList<HashMap>) recordTrans : new ArrayList<>();
Object qty = 0;
Object price = 0;
for (HashMap hit : trans) {
if (transactionIndexId.equals(auctionId + "_" + hit.get("buyer") + "_" + hit.get("ts"))) {
qty = hit.get("qty");
price = hit.get("price");
break;
}
}
response.put("qty", qty);
response.put("price", price);
response.put("startts", r.getLong("startts"));
response.put("endts", r.getLong("endts"));
response.put("qty_auction", r.getLong("qty"));
return response;
}
public AuctionRaw findAuctionRawById(Long auctionId) {
final Statement stmt = new Statement();
stmt.setNamespace(NAMESPACE_ALLEK);
stmt.setSetName(SET_U);
final Map<String, Object> aMap = findAuctionUserInSetA(auctionId);
final Key uKey = new Key(NAMESPACE_ALLEK, SET_U, aMap.get("u") + "_" + aMap.get("p"));
final Object uRecord = aerospikeClient.execute(null, uKey, NAMESPACE_ALLEK, FUN_FIND_U_ITEM, Value.get(auctionId));
return parseAuctionRaw((HashMap) uRecord);
}
private AuctionRaw parseAuctionRaw(HashMap r) {
return parseAuctionRaw(new Record(r, 0, 0));
}
private AuctionRaw parseAuctionRaw(Record r) {
return AuctionRaw.builder()
.cat(r.getLong("cat"))
.len(r.getInt("len"))
.start(r.getLong("start"))
.build();
}
public Auction findAuctionById(Long auctionId) {
final Statement stmt = new Statement();
stmt.setNamespace(NAMESPACE_ALLEK);
stmt.setSetName(SET_U);
final Map<String, Object> aMap = findAuctionUserInSetA(auctionId);
final Key uKey = new Key(NAMESPACE_ALLEK, SET_U, aMap.get("u") + "_" + aMap.get("p"));
final Object uRecord = aerospikeClient.execute(null, uKey, NAMESPACE_ALLEK, FUN_FIND_U_ITEM, Value.get(auctionId));
return parseAuction((HashMap) uRecord);
}
public Map<String, Object> findAuctionUserInSetA(Long auctionId) {
final Statement stmt = new Statement();
stmt.setNamespace(NAMESPACE_ALLEK);
stmt.setSetName(SET_U);
final Key aKey = new Key(NAMESPACE_ALLEK, SET_A, Value.get(auctionId / 1024));
final Map<String, Object> aMap = (Map<String, Object>) aerospikeClient.execute(null, aKey, NAMESPACE_ALLEK, FUN_FIND_A_ITEM, Value.get(auctionId));
return aMap;
}
public List<Auction> findAuctionByUserId(Long userId) {
final Statement stmt = new Statement();
stmt.setNamespace(NAMESPACE_ALLEK);
stmt.setSetName(SET_U);
stmt.setFilters(Filter.equal("u", userId));
final RecordSet records = aerospikeClient.query(null, stmt);
return StreamSupport.stream(records.spliterator(), true)
.flatMap(l -> {
final ArrayList<HashMap> auctionsFromRecord = (ArrayList<HashMap>) l.record.getValue("v");
return Optional.ofNullable(auctionsFromRecord).isPresent() ? auctionsFromRecord.stream() : Stream.<HashMap>empty();
})
.map(r -> parseAuction(r))
.collect(Collectors.toList());
}
private Auction parseAuction(HashMap r) {
return parseAuction(new Record(r, 0, 0));
}
private Auction parseAuction(Record r) {
// final Object recordTrans = r.getValue("t");
// final ArrayList<HashMap> trans = Optional.ofNullable(recordTrans).isPresent() ? (ArrayList<HashMap>) recordTrans : new ArrayList<>();
// final List<Transaction> transactions = trans.stream()
// .map(m -> {
// HashMap recordComment = (HashMap) m.get("c");
// Comment comment = null;
// if (recordComment != null && recordComment.size() > 0) {
// comment = Comment.builder()
// .id((Long) recordComment.get("id"))
// .ts((Long) recordComment.get("ts"))
// .text((String) recordComment.get("text"))
// .type((Long) recordComment.get("type"))
// .build();
// }
// return Transaction.builder()
// .ts((Long) m.get("ts"))
// .qty((Long) m.get("qty"))
// .price((Long) m.get("price"))
// .c(comment)
// .buyer((Long) m.get("buyer"))
// .build();
// })
// .collect(Collectors.toList());
return Auction.builder()
.item(r.getLong("item"))
.startts(r.getLong("startts"))
.endts(r.getLong("endts"))
.user(r.getLong("user"))
.qty(r.getLong("qty"))
.price(r.getLong("price"))
.title(r.getString("title"))
.cat(r.getLong("cat"))
// .tcat(r.getLong("tcat"))
// .curr(r.getString("curr"))
.loc(r.getString("loc"))
.woj(r.getString("woj"))
.pow(r.getString("pow"))
.catName(r.getString("cat_name"))
// .t(transactions)
// .len(r.getInt("len"))
// .detSt(r.getLong("det_st"))
// .detLen(r.getLong("det_len"))
// .start(r.getLong("start"))
.build();
}
}
I do not know why but scroll retrieves the data from the old the index = allek and type = auctions.
How do I know that the old index? And so the result in the old Index is equal to 16k (there is just more data and there are other fields than in the new Index) while in the new Index is the records of about 400.
My question is why is this happening? What I should change to be able to use the scrollbar configuration index = auctions and type = auctions?
I ask you for help I have no idea why this is happening.

How to remove duplicate in list based on two custom objects

I have ProductDetail class which contains two variables productName and productVersion. I need to replace duplicate items in the list according to the latest productVersion.
Example:
class ProductDetails {
string productName;
string productVersion;
}
new ProductDetails("ios", "9.1.0")
new ProductDetails("android", "6.0.1")
new ProductDetails("android", "5.1.1")
new ProductDetails("ios", "10.0.0")
Result:
android 6.0.1
ios 10.0.0
Thanks in advance.
This should work out for you:
import java.util.ArrayList;
import java.util.List;
import java.util.regex.Pattern;
public class ProductDetails {
private String productName;
private String productVersion;
private List<ProductDetails> productData = null;
public ProductDetails(String productName,String productVersion){
this.productName = productName;
this.productVersion = productVersion;
if(productData == null) {
productData = new ArrayList<ProductDetails>();
adNewProduct(this);
}
}
private void adNewProduct(ProductDetails p){
String[] s1 = p.productVersion.split(Pattern.quote("."));
for (ProductDetails dpp: productData) {
if (dpp.productName.equals(p.productName)) {
int index = productData.indexOf(dpp);
String[] s2 = dpp.productVersion.split(Pattern.quote("."));
for(int i = 0; i < s1.length; i++) {
int v1 = Integer.valueOf(s1[i]);
int v2 = Integer.valueOf(s2[i]);
if (v1 > v2) {
productData.set(index,p);
return;
}
}
}
}
productData.add(p);
}
#Override // you can modify it to how you want
public String toString(){
String s = "";
for (ProductDetails p: productData){
s += "ProductName: " + p.productName + " ProductVersion: " + p.productVersion + "\n";
}
return s;
}
//the main method
public static void main(String[] args) {
ProductDetails k = new ProductDetails("ios", "9.1.1");
k.adNewProduct(new ProductDetails("android", "5.1.1"));
k.adNewProduct(new ProductDetails("android", "6.0.1"));
k.adNewProduct(new ProductDetails("ios", "10.0.0"));
System.out.println(k);
}
}
Output:
ProductName: ios ProductVersion: 10.0.0
ProductName: android ProductVersion: 6.0.1
You can put all those objects into a Map<String, ProductDetails> and keep the one with the latest version.
List<ProductDetails> details = Arrays.asList(
new ProductDetails("ios", "9.1.0"),
new ProductDetails("android", "6.0.1"),
new ProductDetails("android", "5.1.1"),
new ProductDetails("ios", "10.0.0"));
// get part of version string
Function<Integer, Function<ProductDetails, Integer>> version =
n -> (pd -> Integer.valueOf(pd.getProductVersion().split("\\.")[n]));
// chain to comparator
Comparator<ProductDetails> versionComp = Comparator.comparing(version.apply(0))
.thenComparing(version.apply(1)).thenComparing(version.apply(2));
Map<String, ProductDetails> latest = new HashMap<>();
for (ProductDetails pd : details) {
String name = pd.getProductName();
if (! latest.containsKey(name) || versionComp.compare(latest.get(name), pd) < 0) {
latest.put(name, pd);
}
}
Afterwards, latest is:
{android=Sandbox.ProductDetails(productName=android, productVersion=6.0.1),
ios=Sandbox.ProductDetails(productName=ios, productVersion=10.0.0)}
Or, you can use Collectors.groupingBy and then use the same Comparator:
details.stream()
.collect(Collectors.groupingBy(ProductDetails::getProductName))
.values().stream().map(list -> Collections.max(list, versionComp))
.forEach(System.out::println);

Categories