We Keep Coding

Is there a way loop through 2 arrays and print the element of array 1 if it contains the substring of any element from array 2?

The problem I am trying to solve is how to read lines from a text file and add it to an array. Then sort each element from this new array by the date that is also in each element. I will explain so its easier to understand but will explain what I am doing.
My text file (First column is name, second is Date of birth and last is the date the person died):
sarah jones,1966-12-02,2018-12-04
matt smith,1983-02-03,2020-03-02
john smith,1967-03-04,2017-04-04
I want to sort this file and output it to another file (testing by printing to console at the moment) by sorting it by the date the person died. A way I thought of doing this is to read each line and pass it to an array. Then read each element within the array, split it and then save the date the person died to another array. Then sort the array that has the death dates, loop through both arrays by seeing if the first element of the death date array matches the first element of the first line in the text file, if so then write it to another file. If not then go to the next line.
For example
BufferedReader reader = new BufferedReader(new FileReader("input_text.txt"));
PrintWriter outputStream = new PrintWriter(new FileWriter("output.txt",true));
ArrayList<String> lines = new ArrayList<String>();
ArrayList<String> substr_date = new ArrayList<String>();
String currentline = reader.readLine();
while(currentline !=null){
String a_line[] = currentline.split(",");
substr_date.add(a_line[2])
lines.add(currentline);
currentline = reader.readLine();
}
Collections.sort(substr_date);
for(String date : substr_date){
for(String line : lines){
if(line.contains(date)){
System.out.println(line);
}
}
}
I expect the output to be:
john smith,1967-03-04,2017-04-04
sarah jones,1966-12-02,2018-12-04
matt smith,1983-02-03,2020-03-02
The results are initially in order but then some lines are repeated multiple times and then the whole text file in repeated to the console and becomes a mess. I am not sure how to go about doing this. I am new to java and not sure if I asked this question properly either so if you need any more info please ask.

I would create class for objects which you can insert into a list and then define a comparator on this class which you can use to sort.
Here is an example of the class you could define:
static class DeceasedPerson {
String name;
LocalDate birthDate;
LocalDate deathDate;
DeceasedPerson(String name, LocalDate birthDate, LocalDate deathDate) {
this.name = name;
this.birthDate = birthDate;
this.deathDate = deathDate;
}
#Override
public String toString() {
return name + ", " + birthDate + ", " + deathDate;
}
}
Then you could simply load objects based on this class into a list which you sort using a comparator. Here is some sample code you can run with the class defined above:
public static void main(String[] args) {
String input =
"matt smith,1983-02-03,2020-03-02\n" +
"sarah jones,1966-12-02,2018-12-04\n" +
"john smith,1967-03-04,2017-04-04\n";
List<DeceasedPerson> deceasedPersonList = new ArrayList<>();
try (Scanner scanner = new Scanner(input)) {
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
String[] array = line.split(",");
DeceasedPerson deceasedPerson = new DeceasedPerson(array[0],
LocalDate.parse(array[1]), LocalDate.parse(array[2]));
deceasedPersonList.add(deceasedPerson);
}
}
deceasedPersonList.sort(Comparator.comparing(o -> o.deathDate));
deceasedPersonList.forEach(System.out::println);
}
If you run the code above using the DeceasedPerson class you should see on the console the following output:
john smith, 1967-03-04, 2017-04-04
sarah jones, 1966-12-02, 2018-12-04
matt smith, 1983-02-03, 2020-03-02
You could actually also use a TreeSet instead of a List in the main method above and achieve the same results. Here is a move concise alternative:
public static void main(String[] args) {
String input =
"matt smith,1983-02-03,2020-03-02\n" +
"sarah jones,1966-12-02,2018-12-04\n" +
"john smith,1967-03-04,2017-04-04\n";
Set<DeceasedPerson> deceasedPersonList = new TreeSet<>(Comparator.comparing(o -> o.deathDate));
try (Scanner scanner = new Scanner(input)) {
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
String[] array = line.split(",");
DeceasedPerson deceasedPerson = new DeceasedPerson(array[0],
LocalDate.parse(array[1]), LocalDate.parse(array[2]));
deceasedPersonList.add(deceasedPerson);
}
}
deceasedPersonList.forEach(System.out::println);
}

The way you are doing is a long shot. You can do this in much simpler way. You could pass a comparator to the Collections.sort() method like this.
Collections.sort(substr_date, new Comparator<String>{
#Override
public int compare(String str1, String str2){
String dd1 = str1.split(",")[2];
String dd2 = str2.split(",")[2];
return dd1.compareTo(dd2);
}
});
Comparing dates like this, though, is not a good approach. You should convert the date string to LocalDateTime and then use isBefore() or isAfter() to compare them. For ex,
public int compare(String str1, String str2){
DateTimeFormatter format = DateTimeFormatter.ofPattern("yyyy-MM-dd")
LocalDateTime d1 = LocalDateTime.parse(str1.split(",")[2],format);
LocalDateTime d2 = LocalDateTime.parse(str2.split(",")[2],format);
return d1.isBefore(d2)?-1:(d1.isAfter(d2)?1:0);
}

Best method for parsing date formats during import datas

I created method for parsing a view different date formats during data import (400 K records). My method catches ParseException and trying to parse date with next format when it's different.
Question: Is better way(and faster) to set correct date format during data import?
private static final String DMY_DASH_FORMAT = "dd-MM-yyyy";
private static final String DMY_DOT_FORMAT = "dd.MM.yyyy";
private static final String YMD_DASH_FORMAT = "yyyy-MM-dd";
private static final String YMD_DOT_FORMAT = "yyyy.MM.dd";
private static final String SIMPLE_YEAR_FORMAT = "yyyy";
private final List<String> dateFormats = Arrays.asList(YMD_DASH_FORMAT, DMY_DASH_FORMAT,
DMY_DOT_FORMAT, YMD_DOT_FORMAT);
private Date parseDateFromString(String date) throws ParseException {
if (date.equals("0")) {
return null;
}
if (date.length() == 4) {
SimpleDateFormat simpleDF = new SimpleDateFormat(SIMPLE_YEAR_FORMAT);
simpleDF.setLenient(false);
return new Date(simpleDF.parse(date).getTime());
}
for (String format : dateFormats) {
SimpleDateFormat simpleDF = new SimpleDateFormat(format);
try {
return new Date(simpleDF.parse(date).getTime());
} catch (ParseException exception) {
}
}
throw new ParseException("Unknown date format", 0);
}

If you're running single threaded, an obvious improvement is to create the SimpleDateFormat objects only once. In a multithreaded situation using ThreadLocal<SimpleDateFormat> would be required.
Also fix your exception handling. It looks like it's written by someone who shouldn't be trusted to import any data.

For a similar problem statememt , i had used time4j library in the past. Here is an example. This uses the following dependencies given below as well
import java.text.ParseException;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import net.time4j.PlainDate;
import net.time4j.format.expert.ChronoFormatter;
import net.time4j.format.expert.MultiFormatParser;
import net.time4j.format.expert.ParseLog;
import net.time4j.format.expert.PatternType;
public class MultiDateParser {
static final MultiFormatParser<PlainDate> MULTI_FORMAT_PARSER;
static {
ChronoFormatter<PlainDate> style1 = ChronoFormatter.ofDatePattern("dd-MM-yyyy", PatternType.CLDR,
Locale.GERMAN);
ChronoFormatter<PlainDate> style2 = ChronoFormatter.ofDatePattern("dd.MM.yyyy", PatternType.CLDR, Locale.US);
ChronoFormatter<PlainDate> style3 = ChronoFormatter.ofDatePattern("yyyy-MM-dd", PatternType.CLDR, Locale.US);
ChronoFormatter<PlainDate> style4 = ChronoFormatter.ofDatePattern("yyyy.MM.dd", PatternType.CLDR, Locale.US);
//this is not supported
//ChronoFormatter<PlainDate> style5 = ChronoFormatter.ofDatePattern("yyyy", PatternType.CLDR, Locale.US);
MULTI_FORMAT_PARSER = MultiFormatParser.of(style1, style2, style3, style4);
}
public List<PlainDate> parse() throws ParseException {
String[] input = { "11-09-2001", "09.11.2001", "2011-11-01", "2011.11.01", "2012" };
List<PlainDate> dates = new ArrayList<>();
ParseLog plog = new ParseLog();
for (String s : input) {
plog.reset(); // initialization
PlainDate date = MULTI_FORMAT_PARSER.parse(s, plog);
if (date == null || plog.isError()) {
System.out.println("Wrong entry found: " + s + " at position " + dates.size() + ", error-message="
+ plog.getErrorMessage());
} else {
dates.add(date);
}
}
System.out.println(dates);
return dates;
}
public static void main(String[] args) throws ParseException {
MultiDateParser mdp = new MultiDateParser();
mdp.parse();
}
}
<dependency>
<groupId>net.time4j</groupId>
<artifactId>time4j-core</artifactId>
<version>4.19</version>
</dependency>
<dependency>
<groupId>net.time4j</groupId>
<artifactId>time4j-misc</artifactId>
<version>4.19</version>
</dependency>
The case yyyy will have to be handled differently as it is not a date. May be similar logic that you have used (length ==4) is a choice.
The above code returns , you can check a quick perf run to see if this scales for the 400k records you have.
Wrong entry found: 2012 at position 4, error-message=Not matched by any format: 2012
[2001-09-11, 2001-11-09, 2011-11-01, 2011-11-01]

Talking about 400K records, it might be reasonable to do some "bare hands" optimization here.
For example: if your incoming string has a "-" on position 5, then you know that the only (potentially) matching format would be "yyyy-MM-dd". If it is "."; you know that it is the other format that starts yyyy.
So, if you really want to optimize, you could fetch that character and see what it is. Could save 3 attempts of parsing with the wrong format!
Beyond that: I am not sure if sure if "dd" means that your other dates start with "01" ... or if "1.1.2016" would be possible, too. If all your dates always use two digits for dd/mm; then you can repeat that game - as you would fetch on position 3 - to choose between "dd...." and "dd-....".
Of course; there is one disadvantage - if you follow that idea, you are very much "hard-coding" the expected formats into your code; so adding other formats will become harder. On the other hand; you would save a lot.
Finally: the other thing that might greatly speed up things would be to use stream operations for reading/parsing that information; because then you could look into parallel streams, and simply exploit the ability of modern hardware to process 4, 8, 16 dates in parallel.

Only take most recent line from CSV when a value appears twice

I'm working with a CSV file in Mule that could look something like the following:
ID|LastUpdated
01|01/12/2016 09:00:00
01|01/12/2016 09:45:00
02|01/12/2016 09:00:00
02|01/12/2016 09:45:00
03|01/12/2016 09:00:00
I'm trying to find a way of stripping out all duplicate occurrences of an ID value by taking only the most recent one, determined by the LastUpdated column. I'm trying to achieve this using DataWeave but have so far had no luck. I'm open to writing the logic in to a custom Java class but have limited knowledge of how to do that as well.
My desired output is something like the following:
ID|LastUpdated
01|01/12/2016 09:45:00
02|01/12/2016 09:45:00
03|01/12/2016 09:00:00
Any help or guidance would be appreciated.
Edit: it's worth noting that I expect the inbound file to be quite large (up to 000's of rows) so I need to be aware of performance in my solution
Edit: a solution using DataWeave can be found on the Mulesoft forum here.

If the dates/hours are always sorted into your CSV like in the example you gave the you can keep a reference on all your ID as keys into a Map and just update the value corresponding to the ids:
public static void main(String[] arg){
// I replace all the CSV reading by this list for the example
ArrayList<String> lines = new ArrayList<>();
lines.add("01|01/12/2016 09:00:00");
lines.add("01|01/12/2016 09:45:00");
lines.add("02|01/12/2016 09:00:00");
lines.add("02|01/12/2016 09:45:00");
lines.add("03|01/12/2016 09:00:00");
Iterator it = lines.iterator();
Map<String, String> lastLines = new HashMap<String, String>();
while (it.hasNext()) { // Iterator on the CVS lines here
String s = (String)it.next();
String id = s.substring(0, s.indexOf("|"));
String val = s.substring(s.indexOf("|") + 1 , s.length());
lastLines.put(id, val);
}
Iterator<String> keys = lastLines.keySet().iterator();
while (keys.hasNext()) {
String id = (String) keys.next();
System.out.println(id + "|" + lastLines.get(id));
}
}
This produce :
01|01/12/2016 09:45:00
02|01/12/2016 09:45:00
03|01/12/2016 09:00:00
If the CSV records can be in any order then you need to add a validation of the dates to keep only the most recent for each id.
private static final SimpleDateFormat sdf = new SimpleDateFormat("dd/MM/yyyy hh:mm:ss");
public static void main(String... args) {
// I replace all the CSV reading by this list for the example
ArrayList<String> lines = new ArrayList<>();
lines.add("01|01/12/2016 09:45:00");
lines.add("01|01/12/2016 09:00:00");
lines.add("02|01/12/2016 09:00:00");
lines.add("02|01/12/2016 09:45:00");
lines.add("03|01/12/2016 09:00:00");
Iterator it = lines.iterator();
Map<String, String> lastLines = new HashMap<String, String>();
while (it.hasNext()) { // Iterator on the CVS lines here
String s = (String)it.next();
String id = s.substring(0, s.indexOf("|"));
String val = s.substring(s.indexOf("|") + 1 , s.length());
if(lastLines.containsKey(id)){
try{
Date storeDate = sdf.parse(lastLines.get(id));
Date readDate = sdf.parse(val);
if(readDate.getTime() > storeDate.getTime())
lastLines.put(id, val);
}catch(ParseException pe){
pe.printStackTrace();
}
}else{
lastLines.put(id, val);
}
}
Iterator<String> keys = lastLines.keySet().iterator();
while (keys.hasNext()) {
String id = (String) keys.next();
System.out.println(id + "|" + lastLines.get(id));
}
}
I'm not sure about the date format you are currently using. You may need to change the format of the parser"dd/MM/yyyy hh:mm:ss". You can find the related documentation here

Just saw this one and I believe #danw had asked this question on Mule forum too. There is a better way to achieve it with DataWeave.
Check out my answer on mule forum -
http://forums.mulesoft.com/questions/40897/only-take-most-recent-line-from-csv-when-a-value-a.html#answer-40975

TimeZone customized display name

I need a timezone display values as follows :
(UTC + 05:30) Chennai, Kolkata, Mumbai, New Delhi
But by using following method I am getting bit different output. How should I get the timezone display name as above ? (if required, I can use JODA).
public class TimeZoneUtil {
private static final String TIMEZONE_ID_PREFIXES =
"^(Africa|America|Asia|Atlantic|Australia|Europe|Indian|Pacific)/.*";
private static List<TimeZone> timeZones;
public static List<TimeZone> getTimeZones() {
if (timeZones == null) {
timeZones = new ArrayList<TimeZone>();
final String[] timeZoneIds = TimeZone.getAvailableIDs();
for (final String id : timeZoneIds) {
if (id.matches(TIMEZONE_ID_PREFIXES)) {
timeZones.add(TimeZone.getTimeZone(id));
}
}
Collections.sort(timeZones, new Comparator<TimeZone>() {
public int compare(final TimeZone t1, final TimeZone t2) {
return t1.getID().compareTo(t2.getID());
}
});
}
return timeZones;
}
public static String getName(TimeZone timeZone) {
return timeZone.getID().replaceAll("_", " ") + " - " + timeZone.getDisplayName();
}
public static void main(String[] args) {
timeZones = getTimeZones();
for (TimeZone timeZone : timeZones) {
System.out.println(getName(timeZone));
}
}
}

This code may do the trick for you:
public static void main(String[] args) {
for (String timeZoneId: TimeZone.getAvailableIDs()) {
TimeZone timeZone = TimeZone.getTimeZone(timeZoneId);
// Filter out timezone IDs such as "GMT+3"; more thorough filtering is required though
if (!timeZoneId.matches(".*/.*")) {
continue;
}
String region = timeZoneId.replaceAll(".*/", "").replaceAll("_", " ");
int hours = Math.abs(timeZone.getRawOffset()) / 3600000;
int minutes = Math.abs(timeZone.getRawOffset() / 60000) % 60;
String sign = timeZone.getRawOffset() >= 0 ? "+" : "-";
String timeZonePretty = String.format("(UTC %s %02d:%02d) %s", sign, hours, minutes, region);
System.out.println(timeZonePretty);
}
}
The output looks like this:
(UTC + 09:00) Tokyo
There are, however, a few caveats:
I only filter out timezones whose ID matches the format "continent/region" (e.g. "America/New_York"). You would have to do a more thorough filtering process to get rid of outputs such as (UTC - 08:00) GMT+8 though.
You should read the documentation for TimeZone.getRawOffSet() and understand what it's doing. For example, it doesn't DST effects into consideration.
On the whole, you should know that this is a messy approach, primarily because the timezone ID can be of so many different formats. Maybe you could restrict yourself down to the timezones that matter for your application, and just have a key value mapping of timezone IDs to display names?

Java String Parsing and Evaluating

All,
I am in the process or rewriting some code that I wrote a while back. The objective of the code was to calcualte a date and time based on a string in the following formats:
DayStart+2Hour+1Day-2Minutes
NOW+20Day
MonthStart+1Month
Which would take the start of the day (in local time), e.g. 2011-09-15 00:00:00 BST (2011-09-15 23:00 GMT) then add 2 hours, add 1 day, and subtract 2 minutes.
The implementation is in Java and the original algorithm was pretty basic. It iterated through each character in the string and appended to a buffer. The buffer was then checked to see if it ended with the strings I was looking (date specifier e.g MINUTE, HOUR, DAYSTART, etc.) for then extracted the number and added to an ArrayList where DateOffset was a simple class with a int and String which was date specifier. Here is some sample code:
// hard coded for sample
String s = "DayStart+2Hour+1Day-2Minutes";
StringBuilder sbBuffer = new StringBuilder();
String buffer;
// iterate through date string
for (char c : s.toCharArray()) {
sbBuffer.append(c);
buffer = sbBuffer.toString();
// check to see the end of the buffer string is what we expect
if (buffer.endsWith("DAYSTART")) {
offsets.add(new DateOffset(0, "DAYSTART"));
sbBuffer = new StringBuilder();
} else if (buffer.endsWith("DAY") && buffer.length() > 3) {
String numberStringPart = buffer.substring(0, buffer.length() - 3);
numberStringPart = numberStringPart.replaceAll("[+]", "").trim(); // need as parseInt does not like the +.
offsets.add(new DateOffset(Integer.parseInt(numberStringPart), "DAY"));
sbBuffer = new StringBuilder();
} ... and so on ...
else {
}
}
After the string was parsed I iterated through ArrayList to calculate my datetime.
The problem with the above is probably not efficient although we have experienced no problems. It also does not pick up any errors so you could enter DayStart+2GKGKER.
I'm just trying to come up with some fresh and neat ideas on what to use to rewrite it. I have done a little regex but not too sure if this would be the best route.
Any thoughts?
Thanks,
Andez

Define a grammar for your expressions. Take a look at the ANTLR framework to help you construct a grammar and process your expressions.

Woohoo, that was fun! Thank you! :-)
public class DateExpressions {
private Map<String, Date> dateVariables;
private Map<String, Integer> temporalUnits;
private Map<Character, Integer> temporalOperations;
public static DateExpressions createInstance() {
DateExpressions de = new DateExpressions();
Calendar c = Calendar.getInstance();
de.setVariable("NOW", c.getTime());
c.set(Calendar.HOUR_OF_DAY, 0);
c.set(Calendar.MINUTE, 0);
c.set(Calendar.SECOND, 0);
c.set(Calendar.MILLISECOND, 0);
de.setVariable("DayStart", c.getTime());
c.set(Calendar.DAY_OF_MONTH, 1);
de.setVariable("MonthStart", c.getTime());
return de;
}
public DateExpressions() {
this.dateVariables = new HashMap<String, Date>();
this.temporalUnits = new HashMap<String, Integer>();
this.temporalUnits.put("Second", Calendar.SECOND);
this.temporalUnits.put("Minute", Calendar.MINUTE);
this.temporalUnits.put("Hour", Calendar.HOUR_OF_DAY);
this.temporalUnits.put("Day", Calendar.DATE);
this.temporalUnits.put("Month", Calendar.MONTH);
this.temporalUnits.put("Year", Calendar.YEAR);
this.temporalOperations = new HashMap<Character, Integer>();
this.temporalOperations.put('+', 1);
this.temporalOperations.put('-', -1);
}
public void setVariable(String key, Date value) {
this.dateVariables.put(key, value);
}
public Date parseExpression(String expr) throws IOException {
StringReader sr = new StringReader(expr);
String s;
int n;
char c;
int offset;
int unit;
int op = 1;
Calendar base = null;
StringBuilder sb1 = new StringBuilder();
StringBuilder sb2 = new StringBuilder();
while ((n = sr.read()) != -1) {
c = (char) n;
if (base == null && temporalOperations.containsKey(c)) {
s = sb2.toString();
if (!dateVariables.containsKey(s)) {
throw new IOException("Unknown variable '" + s + "' used");
}
base = Calendar.getInstance();
base.setTime(dateVariables.get(sb2.toString()));
op = temporalOperations.get(c);
sb1.setLength(0);
sb2.setLength(0);
} else if (temporalOperations.containsKey(c)) {
if (!temporalUnits.containsKey(sb2.toString())) {
throw new IOException(
"Parse error: unknown temporal unit used '"
+ sb2.toString() + "'");
}
offset = Integer.parseInt(sb1.toString());
unit = temporalUnits.get(sb2.toString());
base.add(unit, op * offset);
op = temporalOperations.get(c);
sb1.setLength(0);
sb2.setLength(0);
} else if (Character.isDigit(c)) {
sb1.append(c);
} else {
sb2.append(c);
}
}
if (!temporalUnits.containsKey(sb2.toString())) {
throw new IOException("Parse error: unknown temporal unit used '"
+ sb2.toString() + "'");
}
offset = Integer.parseInt(sb1.toString());
unit = temporalUnits.get(sb2.toString());
base.add(unit, op * offset);
return base.getTime();
}
public static void main(String[] args) throws IOException {
DateExpressions de = DateExpressions.createInstance();
System.out.println(de.parseExpression("DayStart+2Hour+1Day-2Minute"));
System.out.println(de.parseExpression("NOW+20Day"));
System.out.println(de.parseExpression("MonthStart+1Month"));
}
}

If you're after rapid experimentation, sometimes a literate API combined with on the fly compilation is an easy way to go.
So, your example could look like (given appropriate static imports)
daystart().plus()
.hours(2).plus()
.days(1).minutes(2)
or even (given milliseconds as the basic units)
daystart() + hours(2) + days(1) - minutes(2)

Regex seems to be the best bet for such a scenario. Although, I'm puzzled why would you want to interpret strings in this manner, rather than having sophisticated APIs.