What is the fastest way to retrieve values from String? - java

During my app development one performance question came to my mind:
I have a lot of lines of data that can looks like that:
!ANG:-0.03,0.14,55.31
!ANG:-0.03,-0.14,305.31
!ANG:-234.03,-0.14,55.31
in general: !ANG:float,float,float
Between those lines there are also "damaged" lines - they don't start with ! or are too short/have extra signs and so on.
To detect lines that are damaged at the begining I simply use
if(myString.charAt(0) != '!')//wrong string
What I can do to detect lines that are damaged at the end? It is very important to mention that I need not only to check if the line is correct but also get those 3 float numbers to use it later.
I've found three options for this:
use regexp
split twice (first ":" and second ",") and count elements
use Scanner class
I am not sure which one of this (or maybe there are other) methods will be the best from the performance point of view. Can you please give me some advice?
EDIT:
After some comments I see that it is worth to write how damage lines an look:
NG:-0.03,0.14,55.31
.14,55.31
!ANG:-0.03,0.14,
!A,-0.02,-0.14,554,-0.12,55
It is quite difficult to talk about number of lines because I am getting them from readings from other device so I get packets of around 20 lines at a time with a frequency of 50Hz.
What I've found out so far is the big drawback of using scanner - for each line I need to create new object and after some time my device is starting to get short on resources.

Benchmark them, then you will know.
The likely fastest way is to write your own tiny state machine to match your format and find the float boundaries. Theoretically a regex will have the same performance, but it's likely to have additional overhead.

As an intermediate solution I'd do something like that :
private static class LineObject {
private float f1, f2, f3;
}
private LineObject parseLine(String line) {
LineObject obj = null;
if (line.startsWith("!ANG:")) {
int i = line.indexOf(',', 5);
if (i != -1) {
int j = line.indexOf(',', i+1);
if (j != -1) {
try {
obj = new LineObject();
obj.f1 = Float.parseFloat(line.substring(5, i));
obj.f2 = Float.parseFloat(line.substring(i+1, j));
obj.f3 = Float.parseFloat(line.substring(++j));
} catch (NumberFormatException e) {
return null;
}
}
}
}
return obj;
}
After you can copy/paste only usefull jdk code of startsWith, indexOf and parseFloat in your own state machine...

Related

Resizing an Array with a Dedicated method for upsize and downsizing

I'm fairly new to coding and am struggling with an assignment for my class. The program takes a user input for the size of an Array and prompts the user to enter each value 1 at a time. The array size starts at 3 and if the array needs to be bigger when the array has filled a new array that's 2x size is created and all info is copied into it. I was able to figure out this part but I just can't see what I'm doing wrong in the downsizing part. After the info is copied I have to remove the trailing zeroes. I think I have the downsize method right but I don't know if I'm calling it right
import java.util.Scanner;
public class Lab6 {
public static void main(String args[]) {
int[] myarray = new int[3];
int count = 0;
int limit, limitcount = 1;
Scanner kbd = new Scanner(System.in);
System.out.print("How many values would you like to enter? ");
limit = kbd.nextInt();
while (limitcount <= limit) {
System.out.println("Enter an integer value ");
int input = kbd.nextInt();
limitcount++;
if (count < myarray.length) {
myarray[count] = input;
}
else {
myarray = upsize(myarray);
myarray[count] = input;
}
count++;
}
myarray = downsize(myarray, count)
printArray(myarray);
System.out.println("The amount of values in the arrays that we care about is: " + count);
}
static int[] upsize(int[] array) {
int[] bigger = new int[array.length * 2];
for (int i =0;i<array.length; i++) {
bigger[i] = array[i];
}
return bigger;
}
static void printArray( int[] array ) {
for ( int number : array ) {
System.out.print( number + " ");
}
System.out.println();
}
static int[] downsize(int[] array,int count) {
int[] smaller = new int[count];
for (int i =0; i<count; i++) {
smaller[i] = array[i];
}
return array;
}
}
Giving you a full response rather than a comment since you're new here and I don't want to discourage you with brevity which could be misunderstood.
Not sure what happened to your code when you pasted it in here, you've provided everything but the format is weird (the 'code' bit is missing out a few lines at the top and bottom). Might be one to double-check before posting. After posting, I see that someone else has already edited your code to fix this one.
You're missing a semi-colon. I'm not a fan of handing out answers, so I'll leave you to find it :) If you're running your code in an IDE, it should already be flagging that one up for you. If you're not, why on earth not??? IntelliJ is free, easy to get going with, and incredibly helpful. There are others out there as well which different folk prefer :) An IDE will help you spot all sorts of useful things quickly.
I have now run your code, and you do have a problem! It's in your final method, downsize(). Look very, very carefully at the return statement ;) Your questions suggests you aren't actually sure whether or not this method is right, which makes me wonder: have you actually run this code with different inputs to see what results you get? Please do that.
Style-wise: blank lines between methods would make the code easier to look at, by providing a visual gap between components. Please be consistent with putting your opening { on the same line as the method signature, and with having spaces between items, e.g. for (int i = 0; i < count; i++) rather than for (int i =0; i<count; i++). The compiler couldn't care less, but it is easier for humans to look at and just makes it look like you did care. Always a good thing!
I think it is awesome that you are separating some of the work into smaller methods. Seriously. For extra brownie points, think about how you could move that while() block into its own method, e.g. private int[] getUserData(int numberOfItems, Scanner scanner). Your code is great without this, but the more you learn to write tiny units, the more favours you will be doing your future self.
Has your class looked at unit testing yet? Trust me, if not, when you get to this you will realise just how important point 5 can be. Unit tests will also help a lot with issues such as the one in point 3 above.
Overall, it looks pretty good to me. Keep going!!!
Simple mistake in your downsize method. If you have an IDE like Eclipse, Intellij, etc. you would have seen it flagged right away.
return array; // should return smaller
I have a few suggestions since you mentioned being new to coding.
The "limitcount" variable can be removed and substituted with "count" at every instance. I'll leave it to you to figure that out.
Try using more descriptive and understandable variable names. Other people will read your code (like now) and appreciate it.
Try to use consistent spacing/indentation throughout your code.
Your upsize method can be simplified using a System.arraycopy() call which generally performs better and avoids the need for writing out a for loop. You can rewrite downsize in a similar manner.
static int[] upsize(int[] array) {
int[] bigger = new int[array.length * 2];
System.arraycopy(array, 0, bigger, 0, array.length);
return bigger;
}
Edit: All good points by sunrise above - especially that you've done well given your experience. You should set up an IDE when you have the time, they're simple to use and invaluable. When you do so you should learn to step through a debugger to explore the state of your program over time. In this case you would have noticed that the myarray variable was never reassigned after the downsize() call, quickly leading you to a solution (if you had missed the warning about an unused "smaller" array).

Java "Quicksave" Execution

The problem I seem to have hit is one relating to loading times; I'm not running on a particularly fast machine by any means, but I still want to dabble into neural networks. In short, I have to load 336,600,000 integers into one large array (I'm using the MNIST database; each image is 28x28, which amounts to 748 pixels per image, times 45,000 images). It works fine, and surprisingly I don't run out of RAM, but... it takes 4 and a half hours, just to get the data into an array.
I can supply the rest of the code if you want me to, but here's the function that runs through the file.
public static short[][] readfile(String fileName) throws FileNotFoundException, IOException {
short[][] array = new short[10000][784];
BufferedReader br = new BufferedReader(new FileReader(System.getProperty("user.dir") + "/MNIST/" + fileName + ".csv"));
br.readLine();
try {
for (short i = 1; i < 45000; i++) {
String line = br.readLine();
for (short j = 0; j < 784; j++) {
array[i][j] = Short.parseShort(line.split(",")[j]);
}
}
br.close();
} catch (IOException e) {
e.printStackTrace();
}
return array;
}
What I want to know is, is there some way to "quicksave" the execution of the program so that I don't have to rebuild the array for every small tweak?
Note: I haven't touched Java in a while, and my code is mostly chunked together from a lot of different sources. I wouldn't be surprised if there were some serious errors (or just Java "no-nos"), it would actually help me a lot if you could fix them if you answer.
Edit: Bad question, I'm just blind... sorry for wasting time
Edit 2: I've decided after a while that instead of loading all of the images, and then training with them one by one, I could simply train one by one and load the next. Thank you all for your ideas!
array[i][j] = Short.parseShort(line.split(",")[j]);
You are calling String#split() for every single integer.
Call it once outside the loop and copy the value into your 2d array.

UVa Online Judge 100th (3n+1) uDebug shows everything as correct but wrong answer

This is my first UVa submission so I had a few problems in the way. The biggest hurdle that took my time so far was probably getting all the formats correctly (I know, shouldn't have been too hard but I kept getting runtime error without knowing what that actually meant in this context). I did finally get past that runtime error, but I still get "Wrong answer."
Listed below are the things I've done for this problem. I've been working on this for the last few hours, and I honestly thought about just dropping it altogether, but this will bother me so much, so this is my last hope.
Things I've done:
considered int overflow so changed to long at applicable places
got the whole list (1-1000000) in the beginning through memorization for computation time
submitted to uDebug. Critical input and Random input both show matching output.
submitted to to UVa online judge and got "Wrong Answer" with 0.13~0.15 runtime.
Things I'm not too sure about:
I think I read that UVa doesn't want its classes to be public. So I left mine as class Main instead of the usual public class Main. Someone from another place mentioned that it should be the latter. Not sure which one UVa online judge likes.
input. I used BufferedReader(new InputStreaReader (System.in)) for this. Also not sure if UVa online judge likes this.
I thought my algorithm was correct but because of "Wrong answer," I'm not so sure. If my code is hard to read, I'll try to describe what I did after the code.
Here is my code:
class Main {
public static int mainMethod(long i, int c, List<Integer> l) {
if (i==1)
return ++c;
else if (i%2==0) {
if (i<1000000&&l.get((int)i)!=null)
return l.get((int)i)+c;
else {
c++;
return mainMethod(i/2, c, l);
}
}
else {
if (i<1000000&&l.get((int)i)!=null)
return l.get((int)i)+c;
else {
c++;
return mainMethod(i*3+1, c, l);
}
}
}
public static int countMax(int x, int y, List<Integer> l) {
int max=0;
if (x>y) {
int temp = x;
x= y;
y = temp;
}
for (int i=x; i<=y; i++) {
if (l.get(i)>max)
max = l.get(i);
}
return max;
}
public static void main(String[] args) {
List<Integer> fixed = Arrays.asList(new Integer[1000000]);
for (long i=1; i<1000000; i++) {
fixed.set((int)i, mainMethod(i,0,fixed));
}
String s;
try {
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
while ((s = br.readLine())!=null) {
int x = -1;
int y = -1;
for (String split : s.split("\\s+")) {
if (!split.equals("\\s+") && x==-1) {
x = Integer.parseInt(split);
} else if (!split.equals("\\s+") && x!=-1) {
y = Integer.parseInt(split);
}
}
if (x!=-1&&y!=-1)
System.out.println(Integer.toString(x) + " " + Integer.toString(y) + " " + Integer.toString(countMax(x,y,fixed)));
}
} catch (IOException e) {
} catch (NumberFormatException e) {
}
}
}
I apologize for generic names for methods and variables. mainMethod deals with memorization and creating the initial list. countMax deals with the input from the problem (15 20) and finding the max length using the list. The for loop within the main method deals with potential empty lines and too many spaces.
So my (if not so obvious) question is, what is wrong with my code? Again, this worked perfectly fine on uDebug's Random Input and Critical Input. For some reason, however, UVa online judge says that it's wrong. I'm just clueless as to where it is. I'm a student so I'm still learning. Thank you!
Haven't spotted your error yet, but a few things that may make it easier to spot.
First off:
int goes to 2^31, so declaring i in mainMethod to be long is unnecessary. It also states in the problem specification that no operation will overflow an int, doesn't it? Getting rid of the extraneous longs (and (int) casts) would make it easier to comprehend.
Second:
It's probably clearer to make your recursive call with c + 1 than ++c or doing c++ before it. Those have side effects, and it makes it harder to follow what you're doing (because if you're incrementing c, there must be a reason, right?) What you're writing is technically correct, but it's unidiomatic enough that it distracts.
Third:
So, am I missing something, or are you never actually setting any of the values in the List in your memoization function? If I'm not blind (which is a possibility) that would certainly keep it from passing as-is. Wait, no, definitely blind - you're doing it in the loop that calls it. With this sort of function, I'd expect it to mutate the List in the function. When you call it for i=1, you're computing i=4 (3 * 1 + 1) - you may as well save it.

Assigning values to zip codes

What I am trying to do is assign a number value to a group of zip codes and then use that number to do a calculation later on in the program. What I have now is
if (zip == 93726 || zip == 93725 || zip == 92144) {
return 100;
}
else if (zip == 94550 || zip == 34599 || zip == 77375) {
return 150;
}
and then I take the variable and use it in a calculation. Assigning the number to the variable and the calculation all work but what I have ran into is apparently android only allows you to have so many lines of code and I have ran out of lines with just using if else statements. My question is what would be a better what to go about this?
I am not trying to assign a city to each zip because I have seen that they have services that do that from other posters.
a. You can either use a
switch (zip)
{
case 93726: case 93725: case 92144: return 100;
case 94550: case 34599: case 77375: return 150;
}
-
b. or create a HashMap<Integer,Integer> and fill it with 'zip to value' entries, which should give you a much better performance if you have that many cases.
Map<Integer,Integer> m = new HashMap<>();
m.put(93726,100);
later you could call
return m.get(zip);
-
c. If your zip count is in the tens of thousands and you want to work all in memory, then you should consider just holding a hundred-thousand int sized array:
int[] arr=new int[100000];
and filling it with the data:
arr[93726]=100;
.
.
.
You should probably use String constants for you ZIP codes since
in some places, some start with 0
in some places, they may contain letters as well as numbers
If you are using an Object (either String or Integer) for your constants, I have often used Collection.contains(zip) as a lazy shortcut for the condition in each if statement. That collection contains all the constants for that condition, and you would probably use a subclass that is geared towards finding, perhaps HashSet. Keep in mind that if you use a HashMap solution as suggested elsewhere, your keys will be Integer objects too, so you will do hashing on the keys in any case, you just won't need to store the result values in the collection suggestion.
I suspect that for a large collection of constants, hashing may turn out to be faster than having to work through the large number of == conditions in the if statement until you get to the right condition. (It may help a bit if the most-used constants come first, and in the first if statement...)
On a completely different note (i.e. strategy instead of code), you should see if you could group your ZIPs. What I mean is for example, that if you know that all (or most) ZIPs of the forms "923xx" and "924xx" result in a return of 250, you could potentially shorten your conditionals considerably. E.g. zip.startsWith("923") || zip.startsWith("923") for String ZIPs, or (zip / 100) == 923 || (zip / 100) == 924 for int.
A small number of more specific exceptions to the groups can still be handled, you just need to have the more specific conditionals before the more general conditionals.
Use declarative data. Especially as the zip codes might get updated, extended, corrected:
Using for instance a zip.properties
points = 100, 150
zip100 = 93726, 93725, 92144
zip150 = 94550, 34599, 77375,\
88123, 12324, 23424
And
Map<Integer, Integer> zipToPoints = new HashMap<>();
If you got ZIP codes with leading zeroes maybe better use String or take care to parse them with base 10 (leading zero is base 8, octal).
Whether there really exists such a limitation I do not know, but the extra effort of a bit of coding is worth having all as data.
Have a map of the zip codes.just return the value.
Map<Integer,Integer> zipValues = new HashMap<Integer,Integer>();
zipValues.put(93726,100);
.
.
And so on. or u can read from a prop file and populate the map.
then instead of using the if(s),
return zipValues.get(zipCode);
so say zipCode=93726, it will return u 100.
Cheers.
You could create a Map which maps each zip code to an integer. For example:
Map<Integer, Integer> zipCodeMap = new HashMap<>();
zipCodeMap.put(93726, 100);
And later you can retrieve values from there.
Integer value = zipCodeMap.get(93726);
Now you still have to map each zipcode to a value. I would not do that in Java code but rather use a database or read from a text file (csv for example). This depends mostly on your requirements.
Example csv file:
93726, 100
93727, 100
93726, 150
Reading from a csv file:
InputStream is = this.class.getResourceAsStream("/data.csv");
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
try {
String line;
while ((line = reader.readLine()) != null) {
String[] row = line.split(",");
int zip = Integer.parseInt(row[0].trim());
int value = Integer.parseInt(row[1].trim());
zipCodeMap.put(zip, value);
}
}
catch (IOException ex) {
// handle exceptions here
}
finally {
try {
is.close();
}
catch (IOException e) {
// handle exceptions here
}
}

Java use a piece of code from a text file

I am currently trying to figure something out. For my world editor I want my program to read a text file and use its content as code material. I've already made a decent file reader but now I've got a problem. In the console I am getting the right output, the file has only one line that says:
this.makeGrass(new Vector3f(0, 1, 2));
this is actually part of a code that tells my program to render a specific object to the scene, in this case it's a grass model. However instead of just printing this information to the console with
System.out.println(aryLines[i]);
I want to be able to use the information stored on the .txt file so I can actually add it to my rendering code. The entire method that prints the lines on the text file to the console is:
public void TextOutput()
{
String file_name = "C:/Text.txt";
try
{
StoreCoords file = new StoreCoords(file_name);
String[] aryLines = file.OpenFile();
int i;
for (i = 0; i < aryLines.length; i++)
{
System.out.println(aryLines[i]);
// !! How to use the information as part of my code ??
}
} catch(IOException e)
{
System.out.println(e.getMessage());
}
}
I hope you understand what I want: The content of my text file is a piece of code that I want to use further instead of having it just print to the console, I'm sure this is possible but I wouldn' know how.
As Java is a compiled language, you'd have to recompile at runtime and I am not sure that is even possible. If I were you, I'd hardcode in my own commands. You want to call a function called makeGrass, hardcode it in. Maybe in your text file you can have this:
makeGrass:0,1,2
Then have this right after the println:
if(aryLines[i].startsWith("makeGrass:")) {
String Arguments = aryLines[i].substring(aryLines[i].indexOf(":")+1, aryLines[i].length());
ArgArray = Arguments.split(",");
this.makeGrass(new Vector3f(Double.parseDouble(ArgArray[0]), Double.parseDouble(ArgArray[1]), Double.parseDouble(ArgArray[2])));
}
I'm going to leave my answer like this, assuming you are an experienced programmer. If I am wrong feel free to ask and I will explain it to you. I can also explain how to modify it to add different commands if you want.
Also, this is rather unsafe because if the input is in the wrong format it will crash the app. If you plan on letting users edit the file, then I can show you how to add on safeties.
Hope this helped,
Joseph Meadows
Okay, thanks to Joseph Meadows for the hint, I'm doing the following thing, right after the println statement I've added the code provided by him. To make ArgArray work I had to put String[] before it and also I had to create a new constructor in my Vector3f class to match the Double.parseDouble thingy..
public void TextOutput()
{
String file_name = "C:/Users/Server/Desktop/textText.txt";
try
{
StoreCoords file = new StoreCoords(file_name);
String[] aryLines = file.OpenFile();
int i;
for (i = 0; i < aryLines.length; i++)
{
System.out.println(aryLines[i]);
if(aryLines[i].startsWith("makeGrass:")) {
String Arguments = aryLines[i].substring(aryLines[i].indexOf(":")+1, aryLines[i].length());
String[] ArgArray = Arguments.split(",");
this.makeGrass(new Vector3f(Double.parseDouble(ArgArray[0]),
Double.parseDouble(ArgArray[1]),
Double.parseDouble(ArgArray[2])));
}
}
} catch(IOException e)
{
System.out.println(e.getMessage());
}
}
my original Vector3f constructor is:
public Vector3f(float x, float y, float z)
{
this.m_x = x;
this.m_y = y;
this.m_z = z;
}
and to make the code in the TextOutput method work I've added another constructor right below the original one..
public Vector3f(double parseDouble, double parseDouble2, double parseDouble3) {
this.m_x = (float) parseDouble;
this.m_y = (float) parseDouble2;
this.m_z = (float) parseDouble3;
}
Now everything works great, the console gives me the apropriate statement
makeGrass:0,1,2
and the rendering system creates the grass model at the apropriate coordinates, the only thing I want to change now is that I don't have to add an additional constructor to the Vector3f class, I'm sure I'll figure that out too.
In the picture provided in this link you can see exactly what's going on:
http://www.pic-upload.de/view-27720774/makeGrassEx.png.html
As you can see, the content of the text file is printed out in the console (the numbers below is the fps counter) and the coordinates provided by the text file are interpreted correctly, two grass models being displayed at the respective coordinates which is exactly what I wanted!
Thanks again for your help Joseph Meadows, this is exactly what I was looking for!
I am not sure if you solved this yet, but you did not need the second constructor. I was unsure of the data type you were using for the coordinates, and I assumed you use doubles because that is what I have grown accustomed to using.
In actuality, all types can be parsed from a string. Look here:
this.makeGrass(new Vector3f(Double.parseDouble(ArgArray[0]),
Double.parseDouble(ArgArray[1]),
Double.parseDouble(ArgArray[2])));
This right now is turning the string into a double. That is what
Double.parseDouble();
does.
It looks like you are using floats though, so you can always just use the float parsing method:
Float.parseFloat("String");
That would result with this:
this.makeGrass(new Vector3f(Float.parseFloat(ArgArray[0]),
Float.parseFloat(ArgArray[1]),
Float.parseFloat(ArgArray[2])));
Sorry for the late response, and you are surely welcome for the help. I just love being useful!

Categories