We Keep Coding

Java: Most Efficient Way to Get Input Integer Array

I'm working on a problem that requires me to store a very large amount of integers into an integer array. The input is formatted so that one line displays the amount of integers and the next displays all of the values meant to be stored. Ex:
3
12 45 67
In the problem there is closer to 100,000 integers to be stored. Currently I am using this method of storing the integers:
Scanner scanner = new Scanner(System.in);
int n = scanner.nextInt();
int[] iVau = new int[n];
String[] temp = scanner.nextLine().split(" ");
for(int i = 0; i < n; i++) {
iVau[i] = Integer.parseInt(temp[i]);
}
This works fine, however the problem I am solving has a strict time limit and my current solution is exceeding it. I know that there is a more efficient way to store this input using buffered readers and input streams, but I don't know how to do it, can someone please show me.

The way you are using Scanner makes your program save a String containing the whole numbers at once, in memory. With 100000 numbers in the 2nd line of your input, it is not so efficient, you could read numbers one after the other without keeping the previous one in memory. So, this way, avoiding using Scanner.readLine() should make your program run faster. You will not have to read the whole line one time, and read a 2nd time this String to parse the integers from it: you will do both of these operations only once.
Here is an example. The method testing() does not use any Scanner. The method testing2() is the one you provided. The file tst.txt contains 100000 numbers. The output from this program, on my Mac Mini (Intel Core i5#2.6GHz) is:
duration without reading one line at a time, without using a Scanner instance: 140 ms
duration when reading one line at a time with a Scanner instance: 198 ms
As you can see, not using Scanner makes your program 41% faster (integer part of (198-140)/140*100 equals 41).
package test1;
import java.io.*;
import java.util.*;
public class Test {
// Read and parse an Int from the stream: 2 operations at once
private static int readInt(InputStreamReader ir) throws IOException {
StringBuffer str = new StringBuffer();
int c;
do { c = ir.read(); } while (c < '0' || c > '9');
do {
str.append(Character.toString((char) c));
c = ir.read();
} while (!(c < '0' || c > '9'));
return Integer.parseInt(str.toString());
}
// Parsing the input step by step
private static void testing(File f) throws IOException {
InputStreamReader ir = new InputStreamReader(new BufferedInputStream(new FileInputStream(f)));
int n = readInt(ir);
int [] iVau = new int[n];
for (int i = 0; i < n; i++) iVau[i] = readInt(ir);
ir.close();
}
// Your code
private static void testing2(File f) throws IOException {
Scanner scanner = new Scanner(f);
int n = scanner.nextInt();
int[] iVau = new int[n];
scanner.nextLine();
String[] temp = scanner.nextLine().split(" ");
for(int i = 0; i < n; i++)
iVau[i] = Integer.parseInt(temp[i]);
scanner.close();
}
// Compare durations
public static void main(String[] args) throws IOException {
File f = new File("/tmp/tst.txt");
// My proposal
long t = System.currentTimeMillis();
testing(f);
System.out.println("duration without reading one line at a time, without using a Scanner instance: " + (System.currentTimeMillis() - t) + " ms");
// Your code
t = System.currentTimeMillis();
testing2(f);
System.out.println("duration when reading one line at a time with a Scanner instance: " + (System.currentTimeMillis() - t) + " ms");
}
}
NOTE: creating the input file is done this way, with bash or zsh:
echo 100000 > /tmp/tst.txt
for i in {1..100000}
do
echo -n $i" " >> /tmp/tst.txt
done

I believe this is what you're looking for. A BufferedReader can only read a line at a time, so it is necessary to split the line and cast Strings to ints.
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
try {
int n = Integer.parseInt(br.readLine());
int[] arr = new int[n];
String[] line = br.readLine().split(" ");
for (int i = 0; i < n; i++) {
arr[i] = Integer.parseInt(line[i]);
}
} catch (IOException e) {
e.getStackTrace();
}

Just a thought, String.split returns an array of Strings. You say the input can be around 100,000 values. So in order to split the array in this way, String.split must be iterating through each element. Now in parsing the new array of strings to Integers you have iterated through the collection twice. You could do this in one iteration with a few small tweaks.
Scanner scanner = new Scanner(System.in);
String tmp = scanner.nextLine();
scanner = new Scanner(tmp);
for(int i = 0; scanner.hasNextInt(); i++) {
arr[i] = scanner.nextInt();
}
The reason for linking the scanner to a String instead of leaving it on System.in is so that it ends properly. It doesn't open System.in for user input on the last token. I believe in big O notation this is the difference between O(n) and O(2n) where the original snippet is O(2n)

I am not quite sure why OP has to use Integer.parseInt(s) here since Scanner can just do the parsing directly by new Scanner(File source).
Here is a demo/test for this idea:
public class NextInt {
public static void main(String... args) {
prepareInputFile(1000, 500); // create 1_000 arrays which each contains 500 numbers;
Timer.timer(() -> readFromFile(), 20, "NextInt"); // read from the file 20 times using Scanner.nextInt();
Timer.timer(() -> readTest(), 20, "Split"); // read from the file 20 times using split() and Integer.parseInt();
}
private static void readTest() {
Path inputPath = Paths.get(Paths.get("").toAbsolutePath().toString().concat("/src/main/java/io/input.txt"));
try (Scanner scanner = new Scanner(new File(inputPath.toString()))) {
int n = Integer.valueOf(scanner.nextLine());
int[] iVau = new int[n];
String[] temp = scanner.nextLine().split(" ");
for (int i = 0; i < n; i++) {
iVau[i] = Integer.parseInt(temp[i]);
}
} catch (IOException ignored) {
ignored.printStackTrace();
}
}
private static void readFromFile() {
Path inputPath = Paths.get(Paths.get("").toAbsolutePath().toString().concat("/src/main/java/io/input.txt"));
try (Scanner scanner = new Scanner(new File(inputPath.toString()))) {
while (scanner.hasNextInt()) {
int arrSize = scanner.nextInt();
int[] arr = new int[arrSize];
for (int i = 0; i < arrSize; ++i) {
arr[i] = scanner.nextInt();
}
// System.out.println(Arrays.toString(arr));
}
} catch (IOException ignored) {
ignored.printStackTrace();
}
}
private static void prepareInputFile(int arrCount, int arrSize) {
Path outputPath = Paths.get(Paths.get("").toAbsolutePath().toString().concat("/src/main/java/io/input.txt"));
List<String> lines = new ArrayList<>();
for (int i = 0; i < arrCount; ++i) {
int[] arr = new int[arrSize];
for (int j = 0; j < arrSize; ++j) {
arr[j] = new Random().nextInt();
}
lines.add(String.valueOf(arrSize));
lines.add(Arrays.stream(arr).mapToObj(String::valueOf).collect(Collectors.joining(" ")));
}
try {
Files.write(outputPath, lines);
} catch (IOException ignored) {
ignored.printStackTrace();
}
}
}
Locally tested it with 1_000 arrays while each array has 500 numbers, reading all the elements cost about: 340ms using Scanner.nextInt() while OP's method about 1.5ms.
NextInt: LongSummaryStatistics{count=20, sum=6793762162, min=315793916, average=339688108.100000, max=618922475}
Split: LongSummaryStatistics{count=20, sum=26073528, min=740860, average=1303676.400000, max=5724370}
So I really have doubt the issue lies in the input reading.

Since in your case you are aware of the total count of elements all that you have to do is to read X integers from the second line. Here is an example:
public static void main(String[] args) {
Scanner in = new Scanner(System.in);
int count = in.nextInt();
int array[] = new int[count];
for (int i = 0; i < count; i++) {
array[i] = in.nextInt();
}
}
If this is not fast enough, which I doubt, then you could switch to the use of a BufferedReader as follows:
public static void main(String[] args) throws IOException {
BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
int count = Integer.parseInt(in.readLine());
int array[] = new int[count];
for (int i = 0; i < count; i++) {
int nextInteger = 0;
int nextChar = in.read();
do {
nextInteger = nextInteger * 10 + (nextChar - '0');
nextChar = in.read();
} while (nextChar != -1 && nextChar != (int)' ');
array[i] = nextInteger;
}
}
In your case the input will be aways valid so this means that each of the integers will be separated by a single whitespace and the input will end up with EoF character.
If both are still slow enough for you then you could keep looking for more articles about Reading Integers in Java, Competative programming like this one: https://www.geeksforgeeks.org/fast-io-in-java-in-competitive-programming/
Still my favorite language when it comes to competitions will always be C :) Good luck and enjoy!

How to Merge N sort two text files

I have to read in two texts files that look something like this....
FileOne: 10 1 2 3 4 5 67 75 47 18
FileTwo: 5 65 74 57 68 28 38
The first number represents the count of how many of these integers I should use to populate my array.
I need to read the first line of the file then use that number to populate the two arrays with however many elements specified.
I have figured out how to read in both text files and concatenate them together, however I can't figure out how to only get the first number from the file and use it to determine how many numbers I should use after that.
import java.io.File;
import java.util.Arrays;
import java.util.Scanner;
public class ReadData {
public static void main(String[] args) {
Scanner s = new Scanner(System.in);
System.out.println("Please enter the first file name: ");
String input = s.nextLine();
System.out.println("Please enter the second file name: ");
String input2 = s.nextLine();
int[] data = readFiles(input);
int[] data2 = readFiles(input2);
//System.out.println(Arrays.toString(data));
//System.out.println(Arrays.toString(data2));
System.out.println(Arrays.toString(concat(data, data2)));
}
public static int[] readFiles(String file) {
try {
File f = new File(file);
Scanner s = new Scanner(f);
int counter = 0;
while(s.hasNextInt()) {
counter++;
s.nextInt();
}
int[] arr = new int[counter];
Scanner s1 = new Scanner(f);
for(int i = 0; i < arr.length; i++)
arr[i] = s1.nextInt();
return arr;
}
catch(Exception e) {
return null;
}
}
static int[] concat(int[]... arrays) {
int length = 0;
for (int[] array : arrays) {
length += array.length;
}
int[] result = new int[length];
int pos = 0;
for (int[] array : arrays) {
for (int element : array) {
result[pos] = element;
pos++;
}
}
return result;
}
}

For starters, I'd suggest reading the entire line as a single string, then parse it using a parser which will use space as a delimiter. You will get a DS containing all of your numbers in a more elegant solution.
(Java parsing simple solution : http://pages.cs.wisc.edu/~hasti/cs302/examples/Parsing/parseString.html)
Once you have the DS containing the numbers, you can iterate on it at your pleasure, remembering that the first value determines the amount of future iterations on the DS.
Assuming you use java 8, I'd suggest using the stream functionalities in order to do so : http://winterbe.com/posts/2014/07/31/java8-stream-tutorial-examples/
Comment if you require any furthor elaborations and good luck.

Saturday Night Java Fun with 2D arrays, and reading a text based tab delimited file to populate such an array

Hey guys and gals hope everyones Saturday night is going as swimmingly (preferably more) than mine own.
I'm a java noob so bear with me.
We are told to export the an excel sheet from Open Office into a .txt (Tab Delimited)
It ends up looking like this
1 2 3
4 5 6
7 8 9
10 11 12
Where all values are separated by a tab.. (something I haven't encountered yet and are Integer values)
I can see one option, as i type this as I could capture each line, then string split the line by?? whitespace or /t ... and then assign the values to the respective positions in the [30][10] array...
(which ends up being a .csv and load it into java.
So Job 1 is to populate a 2D array with the files from the tab delimited file.
import java.util.Scanner;
import java.io.File;
import java.util.Arrays;
public class Nitrogen
{
private int elevations[][] = null;
private String filename = "location.txt";
public Nitrogen()
{
int [][]elevations = new int[30][10];
}
public void run()
{
try{
File file = new File(filename);
Scanner input = new Scanner(file);
int rows = 30;
int columns = 10;
int[][] elevations = new int[30][10];
for(int i = 0; i < rows; ++i)
{
for(int j = 0; j < columns; ++j)
{
if(input.hasNextInt())
{
elevations[i][j] = input.nextInt();
}
}
for (int h=0; h < rows; h++) {
for (int g=0; g < columns; g++)
System.out.print(elevations[h][g] +" ");
}
System.out.println("");
}
}
catch (java.io.FileNotFoundException e) {
System.out.println("Error opening "+filename+", ending program");
System.exit(1);}
}
public static void main(String[] args)
{
Nitrogen n = new Nitrogen();
n.run();
}
}
So, this prints out 30 lines of line 1, then a line of 0's on top of 29 lines of line 2, then 2 lines of 0's on top of 28 lines of line 3, You get the point....all moving left to right.
Not quite sure....tis getting late and i might give up for the evening.
Alright!! Here is the solution... persistence pays of thanks for the help everyone
public void populate()
{
try{
File file = new File(filename);
Scanner input = new Scanner(file);
int rows = 30;
int columns = 10;
int[][] elevations = new int[30][10];
for(int i = 0; i < rows; ++i){
for(int j = 0; j < columns; ++j)
{
if(input.hasNextInt())
{
elevations[i][j] = input.nextInt();
}
}
}
for (int h=0; h < rows; h++){ //This was just to show I had it
for (int g=0; g < columns; g++) { //in there correctly
System.out.print(elevations[h][g] +" ");
}
System.out.println(""); }
}
catch (java.io.FileNotFoundException e) {
System.out.println("Error opening "+filename+", ending program");
System.exit(1);}
}

If you are trying to print out the array, I would suggest the following amendment to your code:
for (int h=0; h < rows; h++) {
for (int g=0; g < columns; g++)
System.out.print(elevations[h][g] + " ");
}
System.out.println("");
}
The above will produce an output such as:
Row1, Row1, Row1
Row2, Row2, Row2
Because it prints out the columns, with a space in between each element, then a new line between the rows.

I know this is a simple assignment for a class, but if you can use Collections for IO, please do so. It makes the code much more generally usable.
If you want to read values from a tab delimited file (in OpenOffice, this is called a {Tab} delimited CSV file in the Save dialog window), the easiest you can do, is split each line according to a tab, like this:
public static ArrayList<ArrayList<Integer>> readFile(String filename) throws IOException {
File file = new File(filename);
BufferedReader br = new BufferedReader(new FileReader(file));
ArrayList<ArrayList<Integer>> list = new ArrayList<>(); // list of lines
String buffer;
while ((buffer = br.readLine()) != null) {
String[] splitted = buffer.split("\t"); // split the lines by tabs
ArrayList<Integer> line = new ArrayList<>();
for (String str : splitted) {
line.add(Integer.parseInt(str)); // cast and add all the integers to a list
}
list.add(line); // add the line to the list of lines
}
return list;
}
We are able to create a list of lines, with each line containing the Integer values from the text file. Now, we need to create a 2D list out of it, so concatenate all the values from every line to the line before.
public static ArrayList<Integer> concatenateAll(ArrayList<ArrayList<Integer>> lines) {
ArrayList<Integer> result = new ArrayList<>(); // create an empty 2D list for all the values
for(ArrayList<Integer> line : lines) {
result.addAll(line); // add all the values from every line to the 2D list
}
return result;
}
We can create a 2D list of all the values in the file. To use these two methods, we need to invoke them like this:
public static void main(String[] args) {
ArrayList<ArrayList<Integer>> lineList = null;
try {
lineList = readFile("table.csv");
} catch (IOException ioe) {
ioe.printStackTrace();
}
ArrayList<Integer> allLines = concatenateAll(lineList);
System.out.println(allLines);
}
If we absolutely need an array, we can add this at the end:
Integer[] linesArray = new Integer[allLines.size()];
allLines.toArray(linesArray);
System.out.println(Arrays.toString(linesArray));

For non-class assignments, here's a one liner:
private string[][] Deserialize(string data)
{
return ( from string line
in data.Split('\r\n')
select line.Split(' ')
).ToArray();
}

Java - Read lines of file into different variables

I've worked with Java for a few years but during that time I've almost never had to do anything with text files. I need to know how to read lines of a text file into different variables as two-digit integers, along with several lines of said text file into a 2D integer array. Every text file is to be written like this:
5 5
1 2
4 3
2 4 2 1 4
0 1 2 3 5
2 0 4 4 1
2 5 5 3 2
4 3 3 2 1
The first three lines should all be separate integers, but the first line is indicative of the 2D array's dimensions. The last segment needs to go into that integer array. This is what I've got so far in terms of code.
import java.util.*;
import java.io.*;
public class Asst1Main {
public static void main(String[]args){
try {
x = new Scanner(new File("small.txt"));
} catch (FileNotFoundException e) {
System.out.println("File not found.");
}
while(x.hasNext()){
}
}
}
I'm completely at a loss of how to do this.

Here is some psuedoish code
Scanner input = new Scanner(new File("blammo.txt"));
List<String> data = new ArrayList<String>();
String line1;
String line2;
String line3;
line1 = readALine(input);
line2 = readALine(input);
line3 = readALine(input);
... process the lines as you see fit. perhaps String.split(line1);
while (input.hasNextLine())
{
String current = input.nextLine();
data.add(current);
}
private String readALine(final Scanner input)
{
String returnValue;
if (input.hasNextLine())
{
returnValue = input.nextLine();
}
else
{
returnValue = null; // maybe throw an exception instead.
}
return returnValue;
}
Once you have the data (or perhaps while reading it), you can split it and process it as you see fit.

Start with an ArrayList of integer arrays instead. It'll be easier to do this:
ArrayList<Integer[]> list = new ArrayList<Integer>();
String line = scanner.nextLine();
String[] parts = line.split("[\\s]");
Integer[] pArray = new Integer[parts.length];
for (Integer x = 0; x < parts.length; x++) {
pArray[x] = Integer.parseInt(parts[x]);
}
list.add(pArray);
Do the bulk of that inside of a loop obviously.

Here is a full version.
import java.util.*;
import java.io.*;
public class Asst1Main {
public static void main(String[] args) {
Scanner in;
try {
in = new Scanner(new File("small.txt"));
} catch (FileNotFoundException e) {
System.out.println("File not found.");
return;
}
int rows = in.nextInt();
int cols = in.nextInt();
int startRow = in.nextInt();
int startCol = in.nextInt();
int endRow = in.nextInt();
int endCol = in.nextInt();
int[][] map = new int[rows][cols];
for (int row = 0; row < rows; row++) {
for (int col = 0; col < cols; col++) {
map[row][col] = in.nextInt();
}
}
}
}

Reading a file into a multidimensional array

I want to read in a grid of numbers (n*n) from a file and copy them into a multidimensional array, one int at a time. I have the code to read in the file and print it out, but dont know how to take each int. I think i need to splitstring method and a blank delimiter "" in order to take every charcter, but after that im not sure. I would also like to change blank characters to 0, but that can wait!
This is what i have got so far, although it doesnt work.
while (count <81 && (s = br.readLine()) != null)
{
count++;
String[] splitStr = s.split("");
String first = splitStr[number];
System.out.println(s);
number++;
}
fr.close();
}
A sample file is like this(the spaces are needed):
26 84
897 426
4 7
492
4 5
158
6 5
325 169
95 31
Basically i know how to read in the file and print it out, but dont know how to take the data from the reader and put it in a multidimensional array.
I have just tried this, but it says 'cannot covernt from String[] to String'
while (count <81 && (s = br.readLine()) != null)
{
for (int i = 0; i<9; i++){
for (int j = 0; j<9; j++)
grid[i][j] = s.split("");
}

Based on your file this is how I would do it:
Lint<int[]> ret = new ArrayList<int[]>();
Scanner fIn = new Scanner(new File("pathToFile"));
while (fIn.hasNextLine()) {
// read a line, and turn it into the characters
String[] oneLine = fIn.nextLine().split("");
int[] intLine = new int[oneLine.length()];
// we turn the characters into ints
for(int i =0; i < intLine.length; i++){
if (oneLine[i].trim().equals(""))
intLine[i] = 0;
else
intLine[i] = Integer.parseInt(oneLine[i].trim());
}
// and then add the int[] to our output
ret.add(intLine):
}
At the end of this code, you will have a list of int[] which can be easily turned into an int[][].

private static int[][] readMatrix(BufferedReader br) throws IOException {
List<int[]> rows = new ArrayList<int[]>();
for (String s = br.readLine(); s != null; s = br.readLine()) {
String items[] = s.split(" ");
int[] row = new int[items.length];
for (int i = 0; i < items.length; ++i) {
row[i] = Integer.parseInt(items[i]);
}
rows.add(row);
}
return rows.toArray(new int[rows.size()][]);
}

EDIT: You just updated your post to include a sample input file, so the following won't work as-is for your case. However, the principle is the same -- tokenize the line you read based on whatever delimiter you want (spaces in your case) then add each token to the columns of a row.
You didn't include a sample input file, so I'll make a few basic assumptions.
Assuming that the first line of your input file is "n", and the remainder is the n x n integers you want to read, you need to do something like the following:
public static int[][] parseInput(final String fileName) throws Exception {
BufferedReader reader = new BufferedReader(new FileReader(fileName));
int n = Integer.parseInt(reader.readLine());
int[][] result = new int[n][n];
String line;
int i = 0;
while ((line = reader.readLine()) != null) {
String[] tokens = line.split("\\s");
for (int j = 0; j < n; j++) {
result[i][j] = Integer.parseInt(tokens[j]);
}
i++;
}
return result;
}
In this case, an example input file would be:
3
1 2 3
4 5 6
7 8 9
which would result in a 3 x 3 array with:
row 1 = { 1, 2, 3 }
row 2 = { 4, 5, 6 }
row 3 = { 7, 8, 9 }
If your input file doesn't have "n" as the first line, then you can just wait to initialize your final array until you've counted the tokens on the first line.