I want to get all items from a dynamo db table. i have written a query in java as below and it works. but the problem is that it doesn't add all the columns to the AttributeValue Map. it has only the first column (key). so what i do here is searching every item by key in the loop. so it is not efficient if your table has millions of data as i search every item in the loop. what can i do to avoid this situation ?
note: 'name' is the first column in the table. (primary key)
i tried to get the 'count' as i did like 'name' but it doesn't return anything for 'count' but null.
Table table = dynamoDb.getTable(DYNAMODB_TABLE_NAME);
ScanRequest scanRequest = new ScanRequest(DYNAMODB_TABLE_NAME);
ArrayList<DeviceResponse>deviceResponses=new ArrayList<>();
Map<String, AttributeValue> exclusiveStartKey = null;
do {
final ScanResult scanResult = client.scan(scanRequest);
List<Map<String, AttributeValue>> items = scanResult.getItems();
for (int i = 0; i < items.size(); i++) {
Map<String, AttributeValue> map = items.get(i);
AttributeValue value = map.get("name");
String name = value.getS();
Item item = table.getItem("name", name); //Searching item
int count = item.getInt("count"); //getting count from item
DeviceResponse deviceResponse = new DeviceResponse();
deviceResponse.setName(name);
deviceResponse.setCount(count);
deviceResponses.add(deviceResponse);
}
exclusiveStartKey = scanResult.getLastEvaluatedKey();
// Reusing same request object, just setting the start key
scanRequest.setExclusiveStartKey(exclusiveStartKey);
} while(exclusiveStartKey != null);
return deviceResponses;
i tried to get the 'count' as i did like 'name'
No you didn't. This code where you get name:
AttributeValue value = map.get("name");
String name = value.getS();
Is not equivalent to this code where you attempt to get count:
Item item = table.getItem("name", name); //Searching item
int count = item.getInt("count"); //getting count from item
You are taking the name field, and then executing another query against the database for some reason. All the attributes are in the Map<String, AttributeValue> map object. If you wanted to get an attribute named count the same way you got the attribute named name then the code would be the following:
AttributeValue value = map.get("count");
int count = value.getN();
Or just simplify it like so:
int count = map.get("count").getN();
Related
I have a test.csv file that is formatted as:
Home,Owner,Lat,Long
5th Street,John,5.6765,-6.56464564
7th Street,Bob,7.75,-4.4534564
9th Street,Kyle,4.64,-9.566467364
10th Street,Jim,14.234,-2.5667564
I have a hashmap that reads a file that contains the same header contents such as the CSV, just a different format, with no accompanying data.
In example:
Map<Integer, String> container = new HashMap<>();
where,
Key, Value
[0][NULL]
[1][Owner]
[2][Lat]
[3][NULL]
I have also created a second hash map that:
BufferedReader reader = new BufferedReader (new FileReader("test.csv"));
CSVParser parser = new CSVParser(reader, CSVFormat.DEFAULT);
Boolean headerParsed = false;
CSVRecord headerRecord = null;
int i;
Map<String,String> value = new HashMap<>();
for (final CSVRecord record : parser) {
if (!headerParsed = false) {
headerRecord = record;
headerParsed = true;
}
for (i =0; i< record.size(); i++) {
value.put (headerRecord.get(0), record.get(0));
}
}
I want to read and compare the hashmap, if the container map has a value that is in the value map, then I put that value in to a corresponding object.
example object
public DataSet (//args) {
this.home
this.owner
this.lat
this.longitude
}
I want to create a function where the data is set inside the object when the hashmaps are compared and when a value map key is equal to a contain map key, and the value is placed is set into the object. Something really simply that is efficient at handling the setting as well.
Please note: I made the CSV header and the rows finite, in real life, the CSV could have x number of fields(Home,Owner,Lat,Long,houseType,houseColor, ect..), and a n number of values associated to those fields
First off, your approach to this problem is too unnecessarily long. From what I see, all you are trying to do is this:
Select a two columns from a CSV file, and add them to a data structure. I highlighted the word two because in a map, you have a key and a value. One column becomes the key, and the other becomes the value.
What you should do instead:
Import the names of columns you wish to add to the data structure into two strings. (You may read them from a file).
Iterate over the CSV file using the CSVParser class that you did.
Store the value corresponding to the first desired column in a string, repeat with the value corresponding to the second desired column, and push them both into a DataSet object, and push the DataSet object into a List<DataSet>.
If you prefer to stick to your way of solving the problem:
Basically, the empty file is supposed to hold just the headers (column names), and that's why you named the corresponding hash map containers. The second file is supposed to contain the values and hence you named the corresponding hash map values.
First off, where you say
if (!headerParsed = false) {
headerRecord = record;
headerParsed = true;
}
you probably mean to say
if (!headerParsed) {
headerRecord = record;
headerParsed = true;
}
and where you say
for (i =0; i< record.size(); i++) {
value.put(headerRecord.get(0), record.get(0));
}
you probably mean
for (i =0; i< record.size(); i++) {
value.put(headerRecord.get(i), record.get(i));
}
i.e. You iterate over one record and store the value corresponding to each column.
Now I haven't tried this code on my desktop, but since the for loop also iterates over Home and Longitude, I think it should create an error and you should add an extra check before calling value.put (i.e. value.put("Home", "5th Street") should create an error I suppose). Wrap it inside an if conditional and check of the headerRecord(i) even exists in the containers hash map.
for (i =0; i< record.size(); i++) {
if (container[headerRecord.get(i)] != NULL) {
value.put(headerRecord.get(i), record.get(i));
}
}
Now thing is, that the data structure itself depends on which values from the containers hash map you want to store. It could be Home and Lat, or Owner and Long. So we are stuck. How about you create a data structure like below:
struct DataSet {
string val1;
string val2;
}
Also, note that this DataSet is only for storing ONE row. For storing information from multiple rows, you need to create a Linked List of DataSet.
Lastly, the container file contains ALL the column names. Not all these columns will be stored in the Data Set (i.e. You chose to NULL Home and Long. You could have chosen to NULL Owner and Lat), hence the header file is not what you need to make this decision.
If you think about it, just iterate over the values hash map and store the first value in string val1 and the second value in val2.
List<DataSet> myList;
DataSet row;
Iterator it = values.entrySet().iterator();
while (it.hasNext()) {
Map.Entry pair = (Map.Entry)it.next();
row.val1 = pair.getKey();
row.val2 = pair.getValue();
myList.add(row);
it.remove();
}
I hope this helps.
I have a spinner that has an item and a sub item, it's populated programatically with hashmaps. I need to be able to grab a value out of the selected item by its key. I've gotten as far as getting the entire hashmap out but I can't figure out how to get just the one value based on the key I need.
JSONArray recordsArray = json.getJSONArray("record");
for (int i = 0; i < recordsArray.length(); i++) {
JSONObject record = recordsArray.getJSONObject(i);
Map<String, String> datum = new HashMap<String, String>(2);
datum.put("code", record.getString("id") + " - " + record.getString("heading"));
datum.put("description", record.getString("body"));
spinneritems.add(datum);
adapter.notifyDataSetChanged();
spinner.setSelection(0);
populateList();
}
The above code shows how I populate my spinner, I then need to grab the value in the populateList() method.
Spinner was populated by an array of hashmaps. I used
String spinnerItem = spinneritems.get(spinner.getSelectedItemPosition()).get("key");
to get the value out of the hashmap at the index in the array I needed
Using the methods in Get class in the org.apache.hadoop.hbase.client package, I am able to retrieve only one record at a time based on the row key.
Is there any way that multiple rows can be retrieved based on the value of any other non primary column(In java) .
public ArrayList<HashMap<String, String>> getDataQualifierListByRowkeyList(String tableName,
ArrayList<String> rowkeyList) throws IOException {
Table table = connection.getTable(TableName.valueOf(tableName));
List<Get> getList = new ArrayList<Get>();
for (String rowkey : rowkeyList) {
Get get = new Get(Bytes.toBytes(rowkey));
getList.add(get);
}
Result[] results = table.get(getList);
var funcResult = new ArrayList<HashMap<String, String>>();
for (Result result : results) {
var tempHashMap = new HashMap<String, String>();
for (Cell cell : result.rawCells()) {
tempHashMap.put(
String.format("%s:%s", Bytes.toString(CellUtil.cloneFamily(cell)),
Bytes.toString(CellUtil.cloneQualifier(cell))),
Bytes.toString(CellUtil.cloneValue(cell)));
}
funcResult.add(tempHashMap);
}
return funcResult;
}
Get is only for one record which matches row key. For others, you can use scan with column value filters. This approach will have more latency time when compared to get. Get can also be used with list of gets(list of row keys). But you need get with field other than row key.
If possible, best approach would be, to design row key and add frequently searched field along with main row key. This is only if you need better latency.
I am getting data for a particular user id from 14 tables as shown below. As part of data, I am extracting user_id, record_name and record_value and then I get timestamp from record_name (by splitting on it) and then populate my TreeMap with key as timestamp and value as record_value.
After that I am extracting 100 most recent record_value from valueTimestampMap and then populating it in my recordValueHolder LinkedList.
In my case 100 most recent means by looking at the timestamp not the way they are coming.
Below is my code -
public List<String> getData(String userId) {
List<String> recordValueHolder = new LinkedList<String>();
Map<Long, String> valueTimestampMap = new TreeMap<Long, String>(Collections.reverseOrder());
for (int tableNumber = 0; tableNumber < 14; tableNumber++) {
String sql = "select * from table_" + tableNumber + " where user_id='" + userId + "';";
SimpleStatement query = new SimpleStatement(sql);
query.setConsistencyLevel(ConsistencyLevel.QUORUM);
ResultSet res = session.execute(query);
Iterator<Row> rows = res.iterator();
while (rows.hasNext()) {
Row r = rows.next();
String user_id = r.getString("user_id"); // get user id
String record_name = r.getString("record_name"); // get record name
String record_value = r.getString("record_value"); // get record value
long timestamp = Long.parseLong(record_name.split("\\.")[1]);
// populate my tree map
valueTimestampMap.put(timestamp, record_value);
}
}
// now extract 100 most recent record_value since
// valueTimestampMap is already sorted basis on key in
// descending order
for (Map.Entry<Long, String> entry : valueTimestampMap.entrySet()) {
if (recordValueHolder.size() > 99)
break;
recordValueHolder.add(entry.getValue());
}
return recordValueHolder;
}
I am sorting TreeMap in descending order of the keys by using Collections.reverseOrder() so that I have most recent timestamps at the top and then I can simply extract 100 most recent record_value from it and that's what my above code does.
Problem Statement:-
I have 100 most recent record_value in recordValueHolder List. Now I also need to find out which tableNumber each record_value out of 100 came from and what was the record_name for that record_value as well?
So I was thinking to make a data structure something like below which can hold 100 most recent record_value along with their tableNumber, record_name and timestamp as well.
public class RecordValueTimestampTableHolder {
private long timestamp;
private String recordName;
private String recordValue;
private Integer tableNumber;
// setters and getters
}
So the size of List<RecordValueTimestampTableHolder> should be 100. Is this possible to do with my current setup? I am not able to understand how to make this work?
Now my return data type of getData method will change and instead of returning List<String>, now it will return List<RecordValueTimestampTableHolder> which will have 100 most recent record_values along with other values as well.
Instead of using a TreeMap<Long, String>, use TreeMap<Long, RecordValueTimestampTableHolder>
Instead of using
valueTimestampMap.put(timestamp, record_value);
use:
valueTimestampMap.put(timestamp, new RecordValueTimestampTableHolder(timestamp, record_name, record_value, tableNumber));
Of course, this means you will have to add a constructor to RecordValueTimestampTableHolder that accepts the four parameters and assigns them to the internal fields.
As you said recordValueHolder will have to be defined as a List<RecordValueTimestampTableHolder> and this will also have to be the return type from this method.
Filling it will be exactly like you fill it now. Though personally I'd use valueTimestampMap.values() to iterate.
int i = 0;
for (RecordValueTimestampTableHolder item : valueTimestampMap.values()) {
recordValueHolder.add(item);
if (++i == 100)
break;
}
I have a module where I have to update attendance of each student.
The code is as below:
public void updateDailyAttendance(ActionRequest areq, ActionResponse aRes) throws Exception{
int totalEmployees = EmployeeLocalServiceUtil.getEmployeesCount();
String attendanceValue = getAttendanceValue(areq);
//long attPKey = CounterLocalServiceUtil.increment(Employee.class.getName());
for (int i = 0; i < totalEmployees; i++) {
// use attendanceValue to update employee entry
//String attendanceValue = getAttendanceValue(areq);
// parameterValue is value of radio button parameter
long attPKey = CounterLocalServiceUtil.increment(Employee.class.getName());
Attendance newAttendanceInstance = new AttendanceImpl();
newAttendanceInstance.setAttId(attPKey);
newAttendanceInstance.setAttStatus(attendanceValue);
AttendanceLocalServiceUtil.addAttendance(newAttendanceInstance);
}
}
private String getAttendanceValue(ActionRequest areq) {
Enumeration parameters = areq.getParameterNames();
while (parameters.hasMoreElements()) {
String parameterName = parameters.nextElement().toString();
if (parameterName.startsWith("updateattendance")) {
return areq.getParameter(parameterName);
}
}
throw new IllegalStateException("Parameter updateattendance cannot be found!!");
}
When I use the above code my database gets updated but the attendance (present/absent entered for first employee is taken as the value for other employees even though I mark different values(Present/absent) for other employees
How should I modify the above code so that the radio button value entred for each employee is stored ?
You get the attendanceValue only once before the loop. getAttendanceValue returns only one string, so how can the attendanceValue can be different between two students?
String attendanceValue = getAttendanceValue(areq);
That's why all attendanceValues are the same. What you need to do is updating the value for every employee inside the for loop.
It is because you are fetching only one value for attendence. In this line "String attendanceValue = getAttendanceValue(areq);" you get the attendnce for first employee. But in getAttendanceValue method
while (parameters.hasMoreElements()) {
String parameterName = parameters.nextElement().toString();
if (parameterName.startsWith("updateattendance")) {
return areq.getParameter(parameterName);
}
}
This code just check if its starts with updateattendance which is always satisfied for first record and that value is returned. You can Store record for different employees by checking the suffix you used for making "updateattendance" unique if it is some number then you can store it in the ArrayList with Index as the number after updateattendance. And instead of returning the Strig you can return that ArrayList
Here is the better way instead use Map with EmpId as Key and updateattendanc as value.
Map<String, String> mapEmpAttendance = new HashMap<String, String>();
while (parameters.hasMoreElements()) {
String parameterName = parameters.nextElement().toString();
String value = "";
if (parameterName.startsWith("updateattendance")) {
value = areq.getParameter(parameterName);
map.put(parameterName.substring(16, parameterName.length), value)
}
}
return mapEmpAttendance;