We Keep Coding

How to get specific data from a CSV file

I have a very big CSV file, I have managed to put all this into an ArrayList using Scanner
Path filepath = Paths.get("./data.csv");
try{
Scanner InputStream = new Scanner(filepath);
while (InputStream.hasNext()){
wholefile.add(String.valueOf(InputStream.next()));
} InputStream.close();
System.out.println(wholefile);
} catch (IOException e) {
e.printStackTrace();
}
}
and my array looks like this :
wholefile = [id,property, address,first_name,last_name,email,Owner, contact, address,Price,Date, sold, 1,94032, Mockingbird, Alley,Brander,Verillo,bverillo0#sogou.com,,435587.57,, 2,293, Haas, Lane,Maxy,Reynalds...........]
Here is a screenshot of the csv file in excel
https://plus.google.com/photos/photo/115135191238195349859/6559552907258825106?authkey=CIu-hovf5pj29gE
There are some things that I would like to do with this data but I am confused what methods I need to write:
Get a property record by ID
Get a list of n number of top priced properties
Total sales for a month.
any help or guidance would be much appreciated, I'm not sure if I'm goign about this the right way
https://plus.google.com/photos/photo/115135191238195349859/6559637333893665186

Don't waste time by reinventing the wheel.
I suggest to use Apache Commons CSV library to manipulate .csv files.
you can find official doc here.
And some examples here.

I had to roll out a custom CSV parser for some proof of concept we were trying to do and I think you could re purpose it here:
CSVReader.java
public class CSVReader implements Iterable<CSVRow> {
private List<String> _data;
private int _itPos = 0;
private int _skip = 0;
private FileIterator _it;
private boolean _hasTrailingComma = false;
public CSVReader(Path path, boolean hasTrailingComma) throws IOException {
this(Files.readAllLines(path), hasTrailingComma);
}
public CSVReader(Path path) throws IOException {
this(path, false);
}
public CSVReader(List<String> data, boolean hasTrailingComma) {
_data = data;
_it = new FileIterator();
_hasTrailingComma = hasTrailingComma;
}
public CSVReader(List<String> data) {
this(data, false);
}
public CSVRow getHeaders() {
return new CSVRow(_data.get(0), _hasTrailingComma);
}
public void skip(int rows) {
_skip = rows;
}
#Override
public Iterator<CSVRow> iterator() {
_itPos = _skip;
return _it;
}
private class FileIterator implements Iterator<CSVRow> {
#Override
public boolean hasNext() {
return _itPos < _data.size();
}
#Override
public CSVRow next() {
if (_itPos == _data.size()) {
throw new NoSuchElementException();
}
return new CSVRow(_data.get(_itPos++), _hasTrailingComma);
}
}
}
CSVRow.java
public class CSVRow implements Iterable<String> {
private String[] _data;
private int _itPos = 0;
private int _skip = 0;
private RowIterator _it = null;
private int _actualLength = 0;
public CSVRow(String row, boolean trailingComma) {
// Minor hack
// in case the data doesn't end in commas
// we check for the last character and add
// a comma. Ideally, the input file should be fixed;
if(trailingComma && !row.endsWith(",")) {
row += ",";
}
_data = row.split("\\s*,\\s*", -1);
_actualLength = trailingComma ? _data.length - 1 : _data.length;
_it = new RowIterator();
}
public CSVRow(String row) {
this(row, false);
}
public void skip(int cells) {
_skip = cells;
}
#Override
public Iterator<String> iterator() {
_itPos = _skip;
return _it;
}
public String[] toArray() {
return Arrays.copyOf(_data, _actualLength);
}
private class RowIterator implements Iterator<String> {
#Override
public boolean hasNext() {
return _itPos < _actualLength;
}
#Override
public String next() {
if (_itPos == _actualLength) {
throw new NoSuchElementException();
}
return _data[_itPos++];
}
}
}
Usage
public static void main(String[] args) {
Path filepath = Paths.get("./data.csv");
CSVReader reader = new CSVReader(filepath);
for (CSVRow row : reader) {
for (String str : row) {
System.out.printf("%s ", str);
}
System.out.println();
}
}
Now it will be useful to model each row as an object so that you can do stuff with it in Java. You can define a class Property that models each row
public class Property {
private int id;
private String address;
private String firstName;
private String lastName;
private String email;
private String ownerContactAddress;
private BigDecimal price;
private java.sql.Date dateSold;
public Property() {
}
// Setters and getters
public long getId() {
return this.id;
}
public void setId(String id) {
this.id = Long.parseLong(id);
}
public String getAddress() {
return this.address;
}
public void setAddress(String address) {
this.address = address;
}
// TODO: setter/getters for firstName, lastName, email, ownerContactAddress
public BigDecimal getPrice() {
return this.price;
}
public void setPrice(String price, Locale locale) throws ParseException {
NumberFormat format = NumberFormat.getNumberInstance(locale);
if (format instanceof DecimalFormat) {
((DecimalFormat) format).setParseBigDecimal(true);
}
this.price = (BigDecimal) format.parse(amount.replaceAll("[^\\d.,]",""));
}
public java.sql.Date getDateSold() {
return this.dateSold;
}
public void setDateSold(String date, String format) throws ParseException {
SimpleDateFormat sdf = new SimpleDateFormat(format);
this.dateSold = new Date(sdf.parse(date).getTime());
}
}
Bringing everything together (Not tested)
public static void main(String[] args) {
// Collection to store properties
// You could also write a class to wrap this
// map along with the methods you need to implement
// Say PropertyTable {
// private Map<Long, Property> properties ...
// Property getPropertyById(long id);
// getHighestPriced() // sort the map by price
// }
Map<Long, Property> properties = new HashMap<>();
Path filepath = Paths.get("./data.csv");
CSVReader reader = new CSVReader(filepath);
for (CSVRow row : reader) {
Iterator<String> it = row.iterator();
Property p = new Property();
p.setId(it.next());
p.setAddress(it.next());
// ... set the remaining properties
p.setPrice(it.next(), new Locale("en", "GB"));
p.seDateSold(it.next(), "MM/dd/yyyy");
properties.put(p.getId(), p);
}
// At this point, you should have all the properties read
// let's try to get property with id 5
Property prop = properties.get(5L);
}
I hope this helps.

With an ArrayList of Strings will have a bad Performance at time of doing what do you want.
First Create an Object that Match your CVS Header. Then at time of reading the File start adding to an ArrayList of the Object you created, and for sorting, search and a Total sales just make a stream over the ArrayList.

Room #Insert does not insert all records

I am running into an issue where only 1 record is being inserted into my Room SQLite DB.
When I perform a getAll(); the result only returns 1 record.
FOUND ISSUE: Genre[] genres = gson.fromJson(jsonArray.toString(), Genre[].class);
This line above is setting all "gid" values to 0, and I am not sure how to change that.
Genre.java
#Entity(indices = {#Index(value = {"id", "name"}, unique = true)})
public class Genre {
#PrimaryKey
private int gid;
//#ColumnInfo(name = "id") By Default - No need to annotate
#NonNull
private int id;
private String name;
public int getGid() {
return gid;
}
public void setGid(int gid) {
this.gid = gid;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
GenreDao.java
#Dao
public interface GenreDao {
#Query("SELECT * FROM Genre")
LiveData<List<Genre>> getAll();
#Insert(onConflict = OnConflictStrategy.REPLACE) //If there is a conflict, replace the record.
void insertAll(Genre... genres);
}
GenreRepository.java
public class GenreRepository {
private final GenreDao genreDao;
public GenreRepository(GenreDao genreDao) {
this.genreDao = genreDao;
}
//Database Methods
public void insertAll(Genre... genres) {
AsyncTask.execute(() -> { //Same as new Runnable()
genreDao.insertAll(genres);
});
}
public LiveData<List<Genre>> getAll() {
return genreDao.getAll();
}
}
APIUtil.java - getGenres() Method
This class makes an API call, returns the proper results, converts the JSONArray to a Genre[]. I can successfully loop through the Genre[] and confirm 10+ results come back.
public static void getGenres(Context context) {
APIWrapper wrapper = new APIWrapper(context, API_KEY);
Parameters params = new Parameters();
params.addFields(GENRE_FIELDS);
params.addLimit("50");
wrapper.genres(params, new onSuccessCallback() {
#Override
public void onSuccess(JSONArray jsonArray) {
Gson gson = new Gson();
Genre[] genres = gson.fromJson(jsonArray.toString(), Genre[].class);
//Insert DB
AppDatabase db = AppDatabase.getAppDatabase(context);
GenreRepository genreRepository = new GenreRepository(db.genreDao());
genreRepository.insertAll(genres);
}
#Override
public void onError(VolleyError volleyError) {
Log.e("GENRES ERROR:", volleyError.toString());
}
});
}
GenreViewModel.java
public class GenreViewModel extends ViewModel {
private GenreRepository genreRepository;
public GenreViewModel(GenreRepository genreRepository) {
this.genreRepository = genreRepository;
}
public void insertAll(Genre... genres){
genreRepository.insertAll(genres);
}
public LiveData<List<Genre>> getAll(){
return genreRepository.getAll();
}
}
SearchFragment.java
This is where I am retrieving the database values. This for loop only returns 1 result.
AppDatabase db = AppDatabase.getAppDatabase(getActivity());
GenreRepository genreRepository = new GenreRepository(db.genreDao());
GenreViewModel genreViewModel = new GenreViewModel(genreRepository);
genreViewModel.getAll().observe(this, genres -> { //new Observer<List<Genre>>()
for(Genre g : genres){
Log.e("GENRE", g.getName());
}
});

public void insertAll(Genre... genres){
genreRepository.insertAll(genres);
}
here is your mistake , what you provide as method definition and what you provide at call. see you make some thing wrong.
Solution
void insertAll(List<T> obj);
you can try with convert your array to list and put above in definition

I had this problem too.
And Solved it this way.
The problem was that the id that comes from server was mongoId and String so I should create a int primary key and pass currentTime as value to it so the database can insert all of them not replace them.
But you should consider using System.nanoTime() method instead of System.currentTimeMillis() cause sometimes it generates same value and then room replace them instead of inserting each one of them.

How do I remove objects from an arraylist which are present in another arraylist based on one member variable of the object in Java?

I have a POJO class A with the below structure
class A{
private String var1;
private int var2;
private String var3;
}
I have two ArrayList<A> List1 and List2 with different size. I want to remove all elements in List1 which are already present in List2 and this equality needs to be checked with respect to the value stored in var2.
I have already checked making the List a Hashset and using removeAll(). But this wont give the desired output since for the same var2, var1 values differ.
Please help me solve this problem.
Edit 1 - Requested by Murat
public class HistoryDto implements Serializable,Comparable<HistoryDto> {
private Integer id;
private String sId;
public Integer getId() {
return id;
}
public void setId(Integer id) {
this.id = id;
}
public String getSId() {
return sId;
}
public void setSId(String sId) {
this.sId = sId;
}
public String getTrackName() {
return trackName;
}
public void setTrackName(String trackName) {
this.trackName = trackName;
}
public String getTrackDescription() {
return trackDescription;
}
public void setTrackDescription(String trackDescription) {
this.trackDescription = trackDescription;
}
public Integer getUsedNo() {
return usedNo;
}
public void setUsedNo(Integer usedNo) {
this.usedNo = usedNo;
}
public String getExtraInfo() {
return extraInfo;
}
public void setExtraInfo(String extraInfo) {
this.extraInfo = extraInfo;
}
public String getImageUrl() {
return imageUrl;
}
public void setImageUrl(String imageUrl) {
this.imageUrl = imageUrl;
}
public Integer getPartyId() {
return partyId;
}
public void setPartyId(Integer partyId) {
this.partyId = partyId;
}
private String trackName;
private String trackDescription;
private Integer usedNo;
private String extraInfo;
private String imageUrl;
private Integer partyId;
public int compareTo(HistoryDto other) {
return this.sId.compareTo(other.sId);
}
}
Removing Items
ListA.removeAll(new HashSet(listB));

You need 2 loops nested.
pseudocode:
for each in A do
for each in B do
if (current item of A equals to current item of B)
say yes!
done
done
You just need to translate it to Java.

You may do it using the Stream API:
Set<Integer> var2 = list2.stream()
.map(a -> a.var2)
.collect(Collectors.toSet());
list1.stream()
.filter(a -> !var2.contains(a.var2))
.collect(Collectors.toList());

Inserting items in DynamoDB Range key

I have created a DynamoDB table with following details:
and I'm trying to insert items in my table:
public static void insertItems() {
AmazonDynamoDBClient ddb = UserPreferenceDemoActivity.clientManager
.ddb();
DynamoDBMapper mapper = new DynamoDBMapper(ddb);
try {
UserPreference userPreference = new UserPreference();
userPreference.setNavn("SalonSol");
for (int i = 800; i <= 1600; i = i + 50) {
userPreference.setTid(i);
userPreference.setMandag("Ledig");
userPreference.setTirsdag("Ledig");
userPreference.setOnsdag("Ledig");
userPreference.setTorsdag("Ledig");
userPreference.setFredag("Ledig");
userPreference.setLørdag("Ledig");
userPreference.setSøndag("Ledig");
Log.d(TAG, "Inserting Tid and Dage");
mapper.save(userPreference);
Log.d(TAG, "Tid and Dage inserted");
}
} catch (AmazonServiceException ex) {
Log.e(TAG, "Error inserting users");
UserPreferenceDemoActivity.clientManager
.wipeCredentialsOnAuthError(ex);
}
}
But AWS keeps returning following exception:
AmazonserviceException: The provided key element does not match the schema
Status Code: 400
I'm actually inserting a String value as a Hash Key and int values as range, so I don't really understand why I'm getting this exception.
My definition of UserPreference class:
#DynamoDBTable(tableName = Constants.TEST_TABLE_NAME)
public static class UserPreference {
private String Navn;
private int Tid;
private String Mandag;
private String Tirsdag;
private String Onsdag;
private String Torsdag;
private String Fredag;
private String Lørdag;
private String Søndag;
#DynamoDBHashKey(attributeName = "Navn")
public String getNavn() {
return Navn;
}
public void setNavn(String Navn) {
this.Navn = Navn;
}
#DynamoDBRangeKey(attributeName = "Tid")
public int getTid() {
return Tid;
}
public void setTid(int Tid) {
this.Tid = Tid;
}
#DynamoDBAttribute(attributeName = "Mandag")
public String getMandag() {
return Mandag;
}
public void setMandag(String Mandag) {
this.Mandag = Mandag;
}
#DynamoDBAttribute(attributeName = "Tirsdag")
public String getTirsdag() {
return Tirsdag;
}
public void setTirsdag(String Tirsdag) {
this.Tirsdag = Tirsdag;
}
#DynamoDBAttribute(attributeName = "Onsdag")
public String getOnsdag() {
return Onsdag;
}
public void setOnsdag(String Onsdag) {
this.Onsdag = Onsdag;
}
#DynamoDBAttribute(attributeName = "Torsdag")
public String getTorsdag() {
return Torsdag;
}
public void setTorsdag(String Torsdag) {
this.Torsdag = Torsdag;
}
#DynamoDBAttribute(attributeName = "Fredag")
public String getFredag() {
return Fredag;
}
public void setFredag(String Fredag) {
this.Fredag = Fredag;
}
#DynamoDBAttribute(attributeName = "Lørdag")
public String getLørdag() {
return Lørdag;
}
public void setLørdag(String Lørdag) {
this.Lørdag = Lørdag;
}
#DynamoDBAttribute(attributeName = "Søndag")
public String getSøndag() {
return Søndag;
}
public void setSøndag(String Søndag) {
this.Søndag = Søndag;
}
}
and the table name has following definition in the "Constants" class:
public static final String TEST_TABLE_NAME = "EkstraTable";

I tried to reproduce the problem by copying and pasting your code then change it to write to my own test table. The problem did not occur, I successfully put all items into the table as per your code:
http://i.imgur.com/lIPgn7P.png
I would suggest checking to make sure you have the newest version of the AWS Java SDK, and that the table was created correctly (I created mine through the console to be safe, you can try that if code still doesn't work after upgrading).

I found the issue. I did a huge mistake by importing:
com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBRangeKey
instead of:
com.amazonaws.mobileconnectors.dynamodbv2.dynamodbmapper.DynamoDBRangeKey
for #DynamoDBRangeKey annotation since I'm using Android mobile SDK. I found the issue by downloading and adding the DynamoDBMapper.jar for 2.1.8, and this version shows the
com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBRangeKey
as deprecated.

Update specific item in arraylist

I want to update specific item in arraylist.
This is Conversation class:
class Conversation
{
String sender,to,name,bio,picture;
Integer id,time,unread;
public Conversation() {
}
public Conversation (int id,String sender,String to,String name,String bio,String picture,int time,int unread) {
this.sender=sender;
this.to=to;
this.id=id;
this.name=name;
this.bio=bio;
this.picture=picture;
this.time=time;
this.unread=unread;
}
public void setSender(String sender) {
this.sender=sender;
}
public void setTo(String to) {
this.to=to;
}
public void setId(int id) {
this.id=id;
}
public void setTime(int time) {
this.time=time;
}
public void setUnread(int unread) {
this.unread=unread;
}
public void setName(String name) {
this.name=name;
}
public void setBio(String bio) {
this.bio=bio;
}
public void setPicture(String picture) {
this.picture=picture;
}
public String getSender() {
return this.sender;
}
public String getTo() {
return this.to;
}
public int getId() {
return this.id;
}
public int getTime() {
return this.time;
}
public int getUnread() {
return this.unread;
}
public String getName() {
return this.name;
}
public String getBio() {
return this.bio;
}
public String getPicture() {
return this.picture;
}
}
I am adding items from database to this list with following lines:
public List<Conversation> getAllConversations() {
List<Conversation> conversationsList=new ArrayList<Conversation>();
String selectQuery = "SELECT * FROM " + TABLE_CONVERSATIONS+" order by id desc";
SQLiteDatabase db = this.getWritableDatabase();
Cursor cursor = db.rawQuery(selectQuery, null);
if (cursor.moveToFirst()) {
do {
Conversation Conversation = new Conversation();
Conversation.setId(Integer.parseInt(cursor.getString(0)));
Conversation.setSender(cursor.getString(1));
Conversation.setTo(cursor.getString(2));
Conversation.setName(cursor.getString(3));
Conversation.setBio(cursor.getString(4));
Conversation.setPicture(cursor.getString(5));
Conversation.setTime(Integer.parseInt(cursor.getString(6)));
Conversation.setUnread(Integer.parseInt(cursor.getString(7)));
conversationsList.add(Conversation);
} while (cursor.moveToNext());
}
return conversationsList;
}
I want to use setUnread method for specific item but how ? I know I can change like this:
conversationsList.get(location).setUnread(1);
But I don't know the location.I need to get the item with another parameter.e.g can I get the item by sender value ?
I need something like this:
conversationsList.getByUsername("username").setUnread(1);

An ArrayList can only be accessed by using a zero-based index. If you want to acces the elements by using another key (id or username) you need to use a Map or SparseArray (if you use a numeric key).
Since you wanted to lookup elements by "username", I'll use a map in the following example:
public Map<String, Conversation> getAllConversations() {
final Map<String, Conversation> conversations = new HashMap<>();
Cursor cursor = ...;
while (cursor.moveToNext()) {
Conversation conversation = new Conversation();
...
conversations.put(conversation.getSender(), conversation);
}
cursor.close(); // don't forget to close your cursors!
return conversations;
}
Then you can look up conversations like this:
conversations.get("John Doe").setUnread(1);
Note: You can use conversation.setTime(cursor.getInt(6)); instead of conversation.setTime(Integer.parseInt(cursor.getString(6)));. The SQLite database does not really care whether you store a string or an integer.