We Keep Coding

Java: Most efficient way to loop through CSV and sum values of one column for each unique value in another Column

I have a CSV file with 500,000 rows of data and 22 columns. This data represents all commercial flights in the USA for one year. I am being tasked with finding the tail number of the plane that flew the most miles in the data set. Column 5 contains the airplain's tail number for each flight. Column 22 contains the total distance traveled.
Please see my extractQ3 method below. First, created a HashMap for the whole CSV using the createHashMap() method. Then, I ran a for loop to identify every unique tail number in the dataset and stored them in an array called tailNumbers. Then for each unique tail number, I looped through the entire Hashmap to calculate the total miles of distance for that tail number.
The code runs fine on smaller datasets, but once the sized increased to 500,000 rows the code becomes horribly inefficient and takes an eternity to run. Can anyone provide me with a faster way to do this?
public class FlightData {
HashMap<String,String[]> dataMap;
public static void main(String[] args) {
FlightData map1 = new FlightData();
map1.dataMap = map1.createHashMap();
String answer = map1.extractQ3(map1);
}
public String extractQ3(FlightData map1) {
ArrayList<String> tailNumbers = new ArrayList<String>();
ArrayList<Integer> tailMiles = new ArrayList<Integer>();
//Filling the Array with all tail numbers
for (String[] value : map1.dataMap.values()) {
if(Arrays.asList(tailNumbers).contains(value[4])) {
} else {
tailNumbers.add(value[4]);
}
}
for (int i = 0; i < tailNumbers.size(); i++) {
String tempName = tailNumbers.get(i);
int miles = 0;
for (String[] value : map1.dataMap.values()) {
if(value[4].contentEquals(tempName) && value[19].contentEquals("0")) {
miles = miles + Integer.parseInt(value[21]);
}
}
tailMiles.add(miles);
}
Integer maxVal = Collections.max(tailMiles);
Integer maxIdx = tailMiles.indexOf(maxVal);
String maxPlane = tailNumbers.get(maxIdx);
return maxPlane;
}
public HashMap<String,String[]> createHashMap() {
File flightFile = new File("flights_small.csv");
HashMap<String,String[]> flightsMap = new HashMap<String,String[]>();
try {
Scanner s = new Scanner(flightFile);
while (s.hasNextLine()) {
String info = s.nextLine();
String [] piecesOfInfo = info.split(",");
String flightKey = piecesOfInfo[4] + "_" + piecesOfInfo[2] + "_" + piecesOfInfo[11]; //Setting the Key
String[] values = Arrays.copyOfRange(piecesOfInfo, 0, piecesOfInfo.length);
flightsMap.put(flightKey, values);
}
s.close();
}
catch (FileNotFoundException e)
{
System.out.println("Cannot open: " + flightFile);
}
return flightsMap;
}
}

The answer depends on what you mean by "most efficient", "horribly inefficient" and "takes an eternity". These are subjective terms. The answer may also depend on specific technical factors (speed vs. memory consumption; the number of unique flight keys compared to the number of overall records; etc.).
I would recommend applying some basic streamlining to your code, to start with. See if that gets you a better (acceptable) result. If you need more, then you can consider more advanced improvements.
Whatever you do, take some timings to understand the broad impacts of any changes you make.
Focus on going from "horrible" to "acceptable" - and then worry about more advanced tuning after that (if you still need it).
Consider using a BufferedReader instead of a Scanner. See here. Although the scanner may be just fine for your needs (i.e. if it's not a bottleneck).
Consider using logic within your scanner loop to capture tail numbers and accumulated mileage in one pass of the data. The following is deliberately basic, for clarity and simplicity:
// The string is a tail number.
// The integer holds the accumulated miles flown for that tail number:
Map<String, Integer> planeMileages = new HashMap();
if (planeMileages.containsKey(tailNumber)) {
// add miles to existing total:
int accumulatedMileage = planeMileages.get(tailNumber) + flightMileage;
planeMileages.put(tailNumber, accumulatedMileage);
} else {
// capture new tail number:
planeMileages.put(tailNumber, flightMileage);
}
After that, once you have completed the scanner loop, you can iterate over your planeMileages to find the largest mileage:
String maxMilesTailNumber;
int maxMiles = 0;
for (Map.Entry<String, Integer> entry : planeMileages.entrySet()) {
int planeMiles = entry.getValue();
if (planeMiles > maxMiles) {
maxMilesTailNumber = entry.getKey();
maxMiles = planeMiles;
}
}
WARNING - This approach is just for illustration. It will only capture one tail number. There could be multiple planes with the same maximum mileage. You would have to adjust your logic to capture multiple "winners".
The above approach removes the need for several of your existing data structures, and related processing.
If you still face problems, put in some timers to see which specific areas of your code are slowest - and then you will have more specific tuning opportunities you can focus on.

I suggest you use the java 8 Stream API, so that you can take advantage of Parallel streams.

Verifying the data from two arraylist and delete it

I have a master arraylist call toBeDeleted which stored timestamp and email. The following are the sample data inside the toBeDeleted arraylist
[1507075234, bunny#outlook.com]
I have one arraylist call logData1 which stored status,email,timestamps and ID. The following are the sample data inside the logData1 arraylist.
[16, bunny#outlook, 1507075234, 0OX9VQB-01-00P-02]
I hope to delete the data inside the logData1 arraylist by verifying the timestamp first with timestamps stated in toBeDeleted1 arraylist, if the timestamp matched, I will check the email for both arraylist. If both of them are matched, I would like to delete away all the data (status,email,timestamp,ID). But I cant make it work
this is my sample output from my source code
[16, bunny#outlook.com, 1507075234, 0OX9VQB-01-00P-02]
The data inside toBeDeleted1 is :[1507075234, bunny#outlook.com]
The time1 is :1507075234
The email1 is :bunny#outlook.com
The time is :1507075234
The emails is :bunny#outlook.com
The data is :bunny#outlook.com
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -3
at java.util.ArrayList.elementData(Unknown Source)
at java.util.ArrayList.get(Unknown Source)
at EmailReporting.main(EmailReporting.java:83)
This is my sample program
System.out.println(logData1);
System.out.println("The data inside toBeDeleted1 is :"+toBeDeleted1);
for(int v = 0;v<toBeDeleted1.size();v++) //look through the logdata1 for removing the record base on timestamp
{
String time1 = toBeDeleted1.get(v);
String email1 = toBeDeleted1.get(v+1);
System.out.println("The time1 is :"+time1);
System.out.println("The email1 is :"+email1);
for(int f = logData1.size();f>logData1.size()-1;f--)
{
// System.out.println(logData1.size());
// System.out.println("The data in logdata1 is "+logData1.get(f-2));
if(time1.equals(logData1.get(f-2)))
{
System.out.println("The time is :"+logData1.get(f-2));
System.out.println("The emails is :"+logData1.get(f-3));
if(email1.equals(logData1.get(f-3)))
{
System.out.println("The data is :"+logData1.get(f-3));
logData1.remove(f-1);
logData1.remove(f-2);
logData1.remove(f-3);
logData1.remove(f-4);
f-=4;
}
}
}
}
The error occurred after this line of code executed
System.out.println("The data is :"+logData1.get(f-3));

You can find elements in the list in order using Collections.indexOfSubList:
List<String> toFind = Arrays.asList(time1, email1);
int emailIndex = Collections.indexOfSubList(logData1, toFind);
A similar lastIndexOfSubList method also exists. That might be more appropriate for your use case.
You can then use this to remove the elements from toFind:
int emailIndex = Collections.lastIndexOfSubList(logData1, toFind);
if (emailIndex >= 1) {
logData1.subList(emailIndex-1, emailIndex+3).clear();
}
Just do this in a loop to keep going until all occurrences have been removed.
Note that just doing this in a loop naively will keep on searching over the tail of the list repeatedly. Instead, you can use subList to "chop" the end of the list, to avoid re-searching it:
List<String> view = logData1;
int emailIndex;
while ((emailIndex = Collections.lastIndexOfSubList(view, toFind)) >= 1) {
logData1.subList(emailIndex-1, emailIndex+3).clear();
view = logData1.subList(0, emailIndex-1);
}
Additionally, note that deleting from the middle of an ArrayList is inefficient, because the elements after the ones you delete have to be shifted down. This is why using subList(...).clear() is better, because it does all of those shifts at once. But if you are removing lots of 4-element batches, you can do better.
Instead of the subList(...).clear(), you can set the bits of elements to be deleted into a BitSet:
List<String> view = logData1;
BitSet bits = new BitSet(logData1.size());
int emailIndex;
while ((emailIndex = Collections.lastIndexOfSubList(view, toFind)) >= 1) {
bits.set(emailIndex-1, emailIndex+3);
view = logData1.subList(0, emailIndex-1);
}
And then shift all the elements down at once, discarding the elements you want to delete:
int dst = 0;
for (int src = 0; src < logData1.size(); ++src) {
if (!bits.get(src)) {
logData1.set(dst++, logData1.get(src));
}
}
And now truncate the list:
logData1.subList(dst, logData1.size());

how to iterate over a text file to perform different tasks (including creating an unknown number of objects) depending on which line I'm reading

Hello I'm a low level comp sci student that's really struggling/unfamiliar with file i/o.
I'm attempting to read in a text file using buffered reader. I understand how to use a while loop to continue scanning until the end of the file is reached but how can I instruct my reader to read just one line and do something until the end of that one line is reached, then read the next line and do something until that line ends, etc?
basically my input text file is going to repeat every three lines. The text file represents nodes in a weighted directed graph.
The input text file would supposedly look like the following:
each node is represented by two lines of text. For example the on the very top line, the first 'S' is the name of the node, the second 'S' indicates that it's a start node, the third 'n' indicates that it's a regular node, not a goal node, which would be indicated by a 'g'.
On the second line are the two nodes connected to 'S' the first being 'B' with a weighted distance of 1, and the second being 'E' with a weighted distance of 2.
The third line is supposed to be blank, and the pattern is repeated.
S S n
B 1 E 2
B N n
C 2 F 3
C N n
D 2 GA 4
D N n
GA 1
E N n
B 1 F 3 H 6
F N n
I 3 GA 3 C 1
GA N g
H N n
I 2 GB 2 F 1
I N n
GA 2 GB 2
GB N g
My code is as follows:
public void actionPerformed(ActionEvent e)
{
if(e.getSource() == openButton)
{
returnVal = fileChooser.showOpenDialog(null);
if(returnVal == JFileChooser.APPROVE_OPTION)
{
selected_file = fileChooser.getSelectedFile();
String file_name = fileChooser.getSelectedFile().getName();
file_name = file_name.substring(0, file_name.indexOf('.'));
try
{
BufferedWriter buff_writer = null;
File newFile = new File("."+file_name+"_sorted.txt");
boolean verify_creation = newFile.createNewFile();
//if (verify_creation)
// System.out.println("file created successfully");
//else
// System.out.println("file already present in specified location");
file_reader1 = new BufferedReader(new FileReader(selected_file));
file_reader2 = new BufferedReader(new FileReader(selected_file));
FileWriter file_writer = new FileWriter(newFile.getAbsoluteFile());
buff_writer = new BufferedWriter(file_writer);
//find the number of nodes in the file
while( (currentLine = file_reader1.readLine()) != null)
{
k++;
//System.out.println("value of k: " + k);
}
nodeArray = new Node[k];
while( (currentLine = file_reader2.readLine()) != null)
{
//System.out.print(currentLine);
String[] var = currentLine.split(" ");
nodeArray[x] = new Node(var[0]);
if (var[1].equals('S') || var[1].equals('s'))
nodeArray[x].setType(NodeType.START);
else if (var[2].equals('g') || var[2].equals('G'))
nodeArray[x].setType(NodeType.GOAL);
else
nodeArray[x].setType(NodeType.NORMAL);
x++;
}
buff_writer.close();
file_writer.close();
}
catch (Exception e1)
{
e1.printStackTrace();
}
}
}
My node class is as follows:
import java.util.*;
enum NodeType
{
START, GOAL, NORMAL;
}
public class Node
{
private String name;
private NodeType typeOfNode;
private final Map<Node, Integer> neighbors = new HashMap<>();
public Node(String name)
{
this.name = name;
}
public void setType(NodeType type)
{
typeOfNode = type;
}
public void addAdjacentNode(Node node, int distance)
{
neighbors.put(node, distance);
}
public String toString()
{
String output = "";
output += "node name: " + name + ",\n";
return output;
}
}
My other major problem is how to handle the second line in the repeating three line sequence. The second line gives all the adjacent nodes and their weighted distances from the node described on the first line. The problem is, I don't know how many adjacent nodes will exist for any given node. Technically there could be none, or maybe a large number.
A kind programmer here suggested that I use a hash map to record adjacent nodes but I'm not sure how to structure a line of code to account for an indeterminate number of such adjacencies
Note: this question is in reference to this earlier question I asked: how to create an adjacency matrix, using an input text file, to represent a directed weighted graph [java]?
if anyone could point me in the right direction I would be eternally grateful

As far as the input problem goes, your while loop is treating every line it reads identically. You will have to add a variable to keep track of which line in the 3-line sequence you're dealing with.

As for the adjacent nodes use an ArrayList, which is a dynamically sized array.
You need an ArrayList for each node, which stores information about that node's adjacent nodes.
So you will need an array containing (k divided by 3) ArrayLists.

Genetic Algorithm - Partially Mapped Crossover - Java

I'm working on something quite interesting, the TSP in Genetic Algorithms, more specifically looking at Partially Mapped Crossover. For background on the code, it receives two arrays of type int which correspond to the relevant cities so for example, first and second could be 1,2,3,4,5,6,7 and 2,3,4,5,2,4,3. What happens next is I try and cross over the cities without any duplication, however when I'm executing the while loop, it doesn't seem to be able to solve my issue as it gets stuck in an infinite loop.
Essentially, I'm baffled as to why it would get stuck in a loop when eventually it should just cross Over the city and get rid of all the duplicates, but for some reason I'm forever stuck in the while!
Background of code:
SIZE = size of cities in array, parent one and parent two contain random cities of size SIZE.
Any help would be greatly aprechiated!
private int[][] partiallyMappedCrossover(int first, int second){
//Used to return an array of type int
int[][] tempArray = new int[2][SIZE];
//Used to represent the selected individuals
ArrayList<Integer> parentOne = new ArrayList<Integer>();
ArrayList<Integer> parentTwo = new ArrayList<Integer>();
ArrayList<Integer> parentOneExchange = new ArrayList<Integer>();
ArrayList<Integer> parentTwoExchange = new ArrayList<Integer>();
//Used to generate crossOverPoints
ArrayList<Integer> crossOverPoints = new ArrayList<Integer>();
crossOverPoints.add(random.nextInt(SIZE));
crossOverPoints.add(random.nextInt(SIZE));
Collections.sort(crossOverPoints);
//Used for checking the parents contents
int currentCity = 0;
int arrayIndex = 0;
int newCity = 0;
//Assign the contents of the selected parents to my parentArrays
for(int i = 0; i < SIZE; i++){
parentOne.add(population[first][i]);
parentTwo.add(population[second][i]);
}
//used to gather cities from tours and swap between randomly selected crossoverpoints
for(int k = crossOverPoints.get(0) ; k < crossOverPoints.get(1) ; k++){
//declare ints to store the city value
int a = parentOne.get(k);
int b = parentTwo.get(k);
//excahnge cities between the two crossOverPoints
parentOneExchange.add(b);
parentTwoExchange.add(a);
}
for(int i = 0; i < crossOverPoints.get(0); i++){
//get the first city from the parentOne
currentCity = parentOne.get(i);
//Check the cities
if(parentOneExchange.contains(currentCity)){
//If it does contain the city, give one the index from the exchange
arrayIndex = parentOneExchange.indexOf(currentCity);
// get the city where we have a repitition
newCity = parentTwo.get(arrayIndex);
//if the new city is also a duplicated one, do another check
while(parentOneExchange.contains(newCity)){
// get the index of the city to replace the repeated city
arrayIndex = parentOneExchange.indexOf(newCity);
// get the city that is intended to replace the repeated city
newCity = parentTwo.get(arrayIndex);
}
//replace the duplicated city with the new city
parentOne.set(i,newCity);
}
currentCity = parentTwo.get(i);
if(parentTwoExchange.contains(currentCity)){
//If it does contain the city, give one the index from the exchange
arrayIndex = parentTwoExchange.indexOf(currentCity);
// get the city where we have a repitition
newCity = parentOne.get(arrayIndex);
//if the new city is also a duplicated one, do another check
while(parentTwoExchange.contains(newCity)){
// get the index of the city to replace the repeated city
arrayIndex = parentTwoExchange.indexOf(newCity);
// get the city that is intended to replace the repeated city
newCity = parentOne.get(arrayIndex);
}
//replace the duplicated city with the new city
parentTwo.set(i,newCity);
}
}
//loop the second crosschange
for(int i = crossOverPoints.get(1); i < SIZE; i++){
//get the first city from the parentOne
currentCity = parentOne.get(i);
//Check the cities
if(parentOneExchange.contains(currentCity)){
//If it does contain the city, give one the index from the exchange
arrayIndex = parentOneExchange.indexOf(currentCity);
// get the city where we have a repitition
newCity = parentTwo.get(arrayIndex);
//if the new city is also a duplicated one, do another check
while(parentOneExchange.contains(newCity)){
// get the index of the city to replace the repeated city
arrayIndex = parentOneExchange.indexOf(newCity);
// get the city that is intended to replace the repeated city
newCity = parentTwo.get(arrayIndex);
}
//replace the duplicated city with the new city
parentOne.set(i,newCity);
}
currentCity = parentTwo.get(i);
if(parentTwoExchange.contains(currentCity)){
//If it does contain the city, give one the index from the exchange
arrayIndex = parentTwoExchange.indexOf(currentCity);
// get the city where we have a repitition
newCity = parentOne.get(arrayIndex);
//if the new city is also a duplicated one, do another check
while(parentTwoExchange.contains(newCity)){
// get the index of the city to replace the repeated city
arrayIndex = parentTwoExchange.indexOf(newCity);
// get the city that is intended to replace the repeated city
newCity = parentOne.get(arrayIndex);
}
//replace the duplicated city with the new city
parentTwo.set(i,newCity);
}
}
//Assign the new offspring to the temp array for return
for(int i = 0; i<SIZE; i++){
tempArray[0][i] = parentOne.get(i);
tempArray[1][i] = parentTwo.get(i);
}
//return the contents of my tempArray
return tempArray;
}

Reading code to find errors like this is notoriously difficult and laborious. There are lots of easier ways you can find these types of errors. I'll give you four to consider (in my personal rough order of preference):
Split the various operations in your method into separate methods then write unit tests for each of those methods making sure they do exactly what you expect before moving onto the next. Once they are all working then you write a method that uses them all. Debugging a small method is much easier than debugging a large one.
Add assert statements that check that the conditions that you expect to be true actually are true. I'll give an example of that below.
An interactive debugger can find why your loop is not completing. That way you can see exactly what values the variables have at each point in your loop.
Add log statements to record interim values as the method progresses. This allows you to ensure expected conditions are met as the algorithm progresses.
Looking at one of your while loops:
while(parentOneExchange.contains(newCity)){
// get the index of the city to replace the repeated city
arrayIndex = parentOneExchange.indexOf(newCity);
// get the city that is intended to replace the repeated city
newCity = parentTwo.get(arrayIndex);
}
This will infinitely loop any time parentTwo.get returns a city that has been previously encountered. I expect that's what's happening due to a logic error earlier in the code. You could add an assertion to ensure that's not the case:
List<Integer> previous = new ArrayList<>();
while(parentOneExchange.contains(newCity)){
assert !previous.contains(newCity): previous;
previous.add(newCity);
arrayIndex = parentOneExchange.indexOf(newCity);
newCity = parentTwo.get(arrayIndex);
}
When this assertion fails you can see the list of previously visited cities and try to understand why it is looping.

Transforming google maps waypoint directions to via-directions

I am trying to change the URL of google maps directions from directions which have multiple waypoints to directions where these intermediate waypoints are deleted but the route remains the same.
Specifically from: https://www.google.nl/maps/dir/51.804323,5.8061076/51.8059489,5.7971745/51.8095767,5.8032703/#51.8068221,5.806553,16.5z/data=!4m2!4m1!3e2
to:
https://www.google.nl/maps/dir/51.804323,5.8061076/51.8095767,5.8032703/#51.8069622,5.8023697,17z/data=!4m9!4m8!1m5!3m4!1m2!1d5.7971218!2d51.8060231!3s0x47c7061292e15b39:0x4d7bcd7484c71cf3!1m0!3e2
EDIT: because I had to drag the route manually in the second URL the coordinates of the middle marker are not exactly the same as in the first URL; this difference can be ignored.
the start part of these URLS seem pretty obvious as to what they are doing, however the data parameter is still unclear to me (without it the route is not correct). I tried the Google Maps API, but these return an XML or JSON file, but I just need the corresponding URL which I would also get using the webinterface of Google Maps.
How can I tranform the first URL to the second??

So after a long time trying to figure out how the URL scheme works, I finally figured out how it works (for the directions interface).
The URL consists of the following steps:
You start off with "https://www.google.nl/maps/dir/"
This is followed by the start coordinates in the form "[LAT],[LONG]", the coordinates of intermediate waypoints in the same format,and then the coordinates of end points. All these coordinates are seperated by a "/" character.
This is followed by "#[LAT],[LONG],[ZOOM]/" where LAT LONG are the coordinates of the viewbox and ZOOM is the level of zoom (lower means more zoomed out).
This is followed by "data=" and then "!4m[(5x+4+y)]!4m[(5x+3+y)]!" where x is the amount of VIA-points and y is the amount of intermediate waypoints in the route. So if you have a route from A to D with intermediate destinations B and C and VIA points Q, W and R you have x=3 and y=2 so you get the string "!4m21!4m20"
Next we get all VIA points. This done in the following scheme: you append "!1m[(5x)]" where x is the amount of VIA-points between the current waypoint and the next. So "!1m5...[data]...!1m0" means that between the start and first waypoint there is one VIA-point and between the first waypoint and the end there are no VIA-points. Each "!1m[(5x)]" is followed by x instances of "!1d[LONG]!2d[LAT]!3s[COORDINATE]". I am not entirely sure what COORDINATE does, but is has to be in the format "0x[HEX]:0x[HEX]" where HEX is a hexadecimal number; I simply take the number 0 for this. This seems to work in all my test cases and does not seem to influence anything.
This is then followed by "!1m0". I believe this is necessary to indicate that after the last waypoint (the finish) there are no more VIA points, which is useless information but needed nevertheless.
Finally, we get the last parameter which looks like "!3e[n]" where n is a discrete variable to indicate the type of navigation: n=0 for driving by car,n=1 is for bicycle riding, n=2 is for walking, and n=3 for public transportation.
That is mostly it for what I found out about the URL scheme by testing it relentlessly. There are more parameters you can add, but that needs more work testing.
Finally, I included my implementation for transforming a URL with 0 or more waypoints and 0 or more VIA-points to a URL containing only VIA-points. Feel free to use it and please let me know if you have found any mistakes so I can fix them.
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
System.out.print("Enter URL: ");
String originalURL = br.readLine();
//get start of URL
String start = "https://www.google.nl/maps/dir/";
//get navigation type
String type = "!3e1";
Matcher t = getMatcher(originalURL, "!3e\\d");
if (t.find()) {
type = t.group();
}
//get viewbox parameter
Matcher v = getMatcher(originalURL, "#[-]?[\\d]+\\.[\\d]+,[-]?[\\d]+.[\\d]+,[-]?[\\d]+[[.]+[\\d]+]*z");
v.find();
String viewbox = v.group();
//get order of points when using VIA
String data = originalURL.substring(originalURL.indexOf("/data=") + 6);
ArrayList<String> order = new ArrayList<>();
Matcher o = getMatcher(data, "!1m[\\d]+");
while (o.find()) {
order.add(o.group());
}
if (order.size() > 0) {
//remove the last element which is always m0 as this should not be
//displayed in the VIA-list
order.remove(order.size() - 1);
}
//!1m2 does not represent the order but indicates that coordinates that are coming up
order.removeIf(a -> a.equals("!1m2"));
//get coordinates of via-points
ArrayList<String> originalViaPoints = new ArrayList<>();
Matcher c = getMatcher(data, "!1d[-]?[\\d]+.[\\d]+!2d[-]?[\\d]+.[\\d]+");
while (c.find()) {
String[] g = c.group().substring(3).split("!2d");
originalViaPoints.add(g[1] + "," + g[0]);
}
//get coordinates of start, end and intermediate points
originalURL = originalURL.substring(0, v.start());
ArrayList<String> waypoints = new ArrayList<>();
Matcher p = getMatcher(originalURL, "[-]?[\\d]+\\.[\\d]+,[-]?[\\d]+.[\\d]+");
while (p.find()) {
waypoints.add(p.group());
}
//start and end must be displayed seperately
String bound = waypoints.get(0) + "/" + waypoints.get(waypoints.size() - 1);
//add intermediate waypoints and via-points to a list of VIA points
ArrayList<String> viaPoints = new ArrayList<>();
//we have VIA points to process
if (!order.isEmpty()) {
int via_index = 0;
int wp_index = 1;
for (String step : order) {
int iter = Integer.valueOf(step.substring(3)) / 5;
for (int i = 0; i < iter; i++) {
viaPoints.add(originalViaPoints.get(via_index++));
}
viaPoints.add(waypoints.get(wp_index++));
}
} else //There are only waypoints in the URL
{
for (int i = 1; i < waypoints.size() - 1; i++) {
viaPoints.add(waypoints.get(i));
}
}
//calculate prefix according to the amount of nodes of the via points
int nodes = viaPoints.size();
String prefix = "!4m" + (5 * nodes + 4) + "!4m" + (5 * nodes + 3) + "!1m" + (5 * nodes);
//get nodes string
String viaString = "";
for (String node : viaPoints) {
viaString += "!3m4!1m2";
String[] pieces = node.split(",");
viaString += "!1d" + pieces[1]; //ALERT: the coordinates are flipped!
viaString += "!2d" + pieces[0];
viaString += "!3s0x0:0x0";
}
String url = start + bound + "/" + viewbox + "/data=" + prefix + viaString + "!1m0" + type;

According to this site, in the old url scheme, there should be 3 ways to add a via point or a route, and they are:
https://www.google.com/maps?dirflg=w&saddr=51.804323,5.8061076&daddr=51.8059489,5.7971745+to:51.8095767,5.8032703
https://www.google.com/maps?dirflg=w&saddr=51.804323,5.8061076&daddr=51.8095767,5.8032703&mrad=51.8059489,5.7971745
https://www.google.com/maps?dirflg=w&saddr=51.804323,5.8061076&daddr=51.8095767,5.8032703&via=51.8059489,5.7971745
But it seems they dropped support to mrad and via. And for using to, it shows the address as if it would be shown in the new url scheme.
For the new URL scheme.. it does not seems to have a lot of documentation on it, so I am not sure if Google wants you to play with it. but... here it is: How to do it with the new scheme.
according to this blog post:
the !xx, is a separator. Looking at your url:
data=
!4m9
!4m8
!1m5
!3m4
!1m2
!1d5.7971218
!2d51.8060231
!3s0x47c7061292e15b39:0x4d7bcd7484c71cf3
!1m0
!3e2
it is really unclear what it is doing, but, at least we see your via lat, and via lng in the !1d and !2d fields;
Also the !3s, in a hex format looks like some kind of lat/lng, might be the area of search. This is how it looks like in dec 5172109373901724473:5583282063383403763
Well, in short, just change the !1d and !2d fields and it seems to work fine. like this:
https://www.google.nl/maps/dir/51.804323,5.8061076/51.8095767,5.8032703/#51.8769532,5.8550939,7.58z/data=!4m9!4m8!1m5!3m4!1m2!1d5.871218!2d52.8060231!3s0x47c7061292e15b39:0x4d7bcd7484c71cf3!1m0!3e2