Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
I need to split a text file into various fields. I can control the way in which the values are divided however, since there are occasionally commas within each value, I can't use CSV. What is the best way to go about importing the file? Would TAB be a better delimiter?
The issue lies within Lippincott, Williams & Wilkins. That's all one field.
Example data
History of Education Quarterly,1748-5959,na,Wiley-Blackwell, History of Political Economy,1527-1919,0018-2702,Duke University Press, History of Political Economy - Annual Supplement,na,missing, History Teacher,0018-2745,0018-2745,The Society for History Education, History Today,na,0018-2753,History Today Limited, Home Healthcare Nurse,na,0884-741X,Lippincott, Williams & Wilkins, Hospitality Law,na,0889-5414,LRP Publications, Hudson Review,na,0018-702X,Hudson Review Incorporated, Humanist - DC,na,0018-7399,American Humanist Associatioin, Idealistic Studies,na,0894-5373,F&W Media,
Instead of hard coding a delimiter, why not just make it a configurable parameter, then if the input should ever change or something like that, you can easily adapt without having to rewrite.
If that's not an option, TAB or | seem like reasonable options without knowing what the input is
You choose any separator and it would break the day one of your data values have them. So, why not embrace a CSV library that would make sure the separators are escaped when required and read them back as well effortlessly.
Here's how you would do it with OpenCSV
String[] values = {"one", "two,three", "four , five"};
CSVWriter writer = new CSVWriter(new FileWriter("yourfile.csv"));
writer.writeNext(values);
writer.close();
CSVReader reader = new CSVReader(new FileReader("yourfile.csv"));
String [] entries;
// reading just the first line
if ((entries = reader.readNext()) != null) {
System.out.println(entries[0] + ", " + entries[1] + ", " + entries[2]);
}
reader.close();
You could actually use any custom separator with OpenCSV like a Tab \t if you want.
CSVWriter writer = new CSVWriter(new FileWriter("yourfile.csv"), '\t');
But using CSVs makes your data files compatible with many other tools as well. So, it entirely depends on your compatibility requirements for your data files if any.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 1 year ago.
Improve this question
I have been preparing for technical interview. So in one thing I am just not sure. If I write for example
try {
...
} catch (InterruptedException ie) {
throw new IllegalStateException(String.format("Player [%s]: failed to reply message [%s]"), ie);
}
Does this prevent already String injection? or Do I have to write like following:
String errorMsg = String.format("Player [%s]: failed to reply message [%s]");
throw new IllegalStateException(errorMsg, ie);
There is no difference between the two snippets. Neither protects against 'string injection'.
There are only 4 non-boneheaded mitigations against string injection attacks:
Ensure that where-ever the strings end up, it is impossible for this to be a security issue in any way or form. For example, if your data is going to a binary file where all system operators are aware the contents are straight from the web, it doesn't matter what's uploaded.
Do not render the string at all. Throwing the baby out with the bathwater.
Use a whitelist. If the string consists solely of these allowed things, allow it. By default, do not allow it.
Use escapers.
Honourable mention for the concept of blacklisting: Have a list of known-malicious stuff, and allow all strings unless they contain something on the blacklist. This is bone headed and should never be used. For example, if you scan incoming data for <script>, you messed up. Don't do this. It doesn't work. Blacklists are trivially bypassed. Whitelists are what you're looking for.
The vastly most common strategy is the 4th: Escapers. For example, when you have a web server that takes in a username and a user's telephone number and a user's full name, and then renders all this information back out on their public website, then:
The phone number should be mitigated using the whitelist strategy. A single +, digits in the range 0-9, spaces, dashes, and nothing else. If that's what the input is like, allow it. Otherwise don't.
The user's real name should be mitigated with escaping: Take the data as provided and inject it verbatim into your database, but treat this data as tainted in all interactions with that data: For example, when rendering that public page, the 'full name' string needs to be washed through an HTML escaper which e.g. replaces all < with <.
Your code doesn't do any of these 4 things (either version of it).
In general it is an extremely bad idea to consider the string returned by an exception's .getMessage() to be already 'safe' (escaped / passed the whitelist verifier). Instead, the code that invokes .getMessage() needs to apply one of the 4 mitigations as explained above.
Hi i know the use of getters is to restrict access. The issue is i have a buffered writer that is using some information in another class. Therefore using getters i have gotten that information and written into a file using the bufferedWriter.
The issue comes when i am trying to use some information in the same class as the bufferedWriter. It doesnt write those details . Shows no error in the code either just doesnt write it. If the data is in the same class as the buffered writer i assume that it doesnt need to be accessed using getters? although the values are stored in another method. Explain this
Thank you alot
bufWrite.write("Your Character Class:" + character_Class + "\n");
bufWrite.write("Your Character Level:" + level + "\n");
The character class and level here are from 1 class called character. To access these information i have used getters (Since the buffered writer is inside another class.Basically i have 3 classes. Character details in one and i am using the bufferedWriter which is in a different class to store these details into a file.)
These details are written into the file completely.
for(Object o:skillInfo){
bufWrite.write("" + o + "- Rank(skill Points) :" + rank);
}
i am trying to use this foreach loop to write stuff inside a linkedlist. This linked list is inside the same class as the buffered writer statements are. But this doesnt get written while the other details(as listed above) get written. The only difference is that those details are using getters since they are not in the same class as the bufferedWriter is while the linked list is in the same as the bufferedWriter thus doesnt use a getter.
I hope this is clear enough
**Update
Also please note that the character.Level and the rank are user entered values.
To answer your question, a BufferedWriter by itself does not 'require' getters.
If your BufferedWriter is not writing data, then the problem might be how you are actually writing the data. You also might want to check if you are calling .close() when you are done writing the data.
Another problem might be how your data is returned from the get-er.
Your question is rather ambiguous, so any answer can only be a guess. I recommend reading up on how to make a good question, so that people can provide better answers.
https://stackoverflow.com/help/how-to-ask
https://stackoverflow.com/help/mcve
*Edit:
Thank you for updating your question.
If you want to write out the details of character_Class, you do need to call the getters of the specific information that you want to get from the class.
The reason why bufWrite.write("Your Character Class:" + character_Class + "\n"); doesn't throw an exception or crash is because of how java coerces classes to String.
The line of code:
"Your Character Class:" + character_Class
is the same as:
"Your Character Class:" + character_Class.toString()
Same happens with objects from skillInfo:
bufWrite.write("" + o.toString() + "- Rank(skill Points) :" + rank);
Basically, if you concatenate '+' a String and an object, it calls toString() on the object. If you concatenate a String and a primitive type, it just converts them to Strings first.
Assume I have a StringBuffer with values "1 \n 2 \n 3 \n...etc" where \n is a line break.
How would I add these values to an existing CSV file as a column using Java? Specifically, this would be the last column.
For example, let's say I have a CSV file that looks like this:
5, 2, 5
2, 3, 1
3, 5, 2
..
etc.
The output should look like this given the StringBuffer after using the method to add the column to the csv file:
5, 2, 5, 1
2, 3, 1, 2
3, 5, 2, 3
..
etc.
I also plan to add columns with 1000s of values so I am looking for something that does not have high memory consumption.
Thanks ahead of time.
Edit: Columns may be different sizes. I see people saying to add it at the end of each line. The problem is, it will add the values to the wrong columns and I cannot have that happen. I thank you all for your suggestions though as they were very good.
Edit 2: I have received critique about my use of StringBuffer and yes, I agree, if this problem were isolated, I would also suggest StringBuilder. The context of this problem is a program that has synchronized threads (acting as scenarios) collecting response times given a range of concurrent threads. The concurrent threads execute concurrent queries to a database and once the query has been executed, the result is appended to a StringBuffer. All the response times for each synchronized thread is appended to a StringBuffer and written to a CSV document. There can be several threads with the same response time. I can use StringBuilder but then I would have to manually synchronize the threads appending the response times and in my case, I do not think it would make too much of a difference in performance and would add an unnecessary amount of code. I hope this helps and I once again, thank you all for your concerns and suggestions. If after reading this, you are still not convinced that I should use StringBuffer, then I ask that we please take this discussion offline.
Edit 3: I have figured out how to go around the issue of adding the columns if the rows are different sizes. I simply add commas for every missing column (also note, that my rows would be growing with each column). It looks like #BorisTheSpider's conceptual solution actually works with this modification. The problem is I am not sure how to add the text at the end of each line. My code so far (I removed code to conserve space):
//Before this code there is a statement to create a test.csv file (this file has no values before this loop occurs).
for (int p = 0; p<(max+1); p = p + inc){
threadThis2(p);
//threadThis2 appends to the StringBuffer with several comma delimited values.
//p represents the number of threads/queries to execute at the same time.
comma = p/inc; //how many commas to put if there is nothing on the line.
for (int i = 0; i < comma; i++) {
commas.append(",");
}
br = new BufferedReader (new FileReader("test.csv"));
List <String> avg = Arrays.asList(sb.toString().split(", "));
for (int i = 0; i < avg.size(); i++) {
if (br.readLine()==null)
{w.write(commas.toString() + avg.get(i).toString() + ", \n");}
else { w.write(avg.get(i).toString() + ", \n");}
}
br.close();
sb.setLength(0);
commas.setLength(0);
}
Please note this code is in its early stages (I will of course declare all the variables outside the for loop later on). So far this code works. The problem is that the columns are not side by side, which is what I want. I understand I may be required to create temporary files but I need to approach this problem very carefully as I might need to have a lot of columns in the future.
Apparently there are two basic requirements:
Append a column to an existing CSV file
Allow concurrent operation
To achieve Requirement #1, the original file has to be read and rewritten as a new file, including the new column, irrespective of its location (i.e., in a StringBuffer or elsewhere).
The best (and only generic) way of reading a CSV file would be via a mature and field-proven library, such as OpenCSV, which is lightweight and commercially-friendly, given its Apache 2.0 license. Otherwise, one has to either do many simplifications (e.g., always assume single-line CSV records), or re-invent the wheel by implementing a new CSV parser.
In either case, a simple algorithm is needed, e.g.:
Initialize a CSV reader or parser object from the library used (or from whatever custom solution is used), supplying the existing CSV file and the necessary parameters (e.g., field separator).
Read the input file record-by-record, via the reader or parser, as a String[] or List<String> structure.
Manipulate the structure returned for every record to add or delete any extra fields (columns), in memory.
Add blank fields (i.e., just extra separators, 1 per field), if desired or needed.
Use a CSV writer from the library (or manually implement a writer) to write the new record to the output file.
Append a newline character at the end of each record written to the output file.
Repeat for all the records in the original CSV file.
This approach is also scalable, as it does not require any significant in-memory processing.
For Requirement #2, there are many ways of supporting concurrency and in this scenario it is more efficient to do it in a tailored manner (i.e., "manually" in the application), as opposed to relying on a thread-safe data structure like StringBuffer.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
im trying to create a restaurant system that will create food items and such from a menu.
ill be learning jdbc soon and im sure that would help but for now i think the simplest way is too create my menu in notepad.
whats the best way to line up and read from a notepad file like a menu.
please try speak clearly, im not exactly sure of all terminologies.
this one looks promising but ive no idea whats goin on.
/////////////////////////////////////////////////////////////////////
im still stuck with this.
ive decided too make a seperate mthod for reading the file.
ive tried every example i can think of. could someone just show me an example of how too define a files classpath.
if i type menu.txt it just doesnt work.
Have a look at Sun's Java Tutorial
Easiest option is to simply use the Apache Commons IO JAR and import the org.apache.commons.io.FileUtils class. There are many possibilities when using this class, but the most obvious would be as follows;
List<String> lines = FileUtils.readLines(new File("untitled.txt"));
It's that easy.
"Don't reinvent the wheel."
Can I ask what sort of content/data you will be reading from this file as there may be other (even simpler) possibilities?
i.e.
Properties
foo="bar"
String Tokens
foo,bar,fu,baz
Let me know if you require more details with any of the processes I've mentioned.
http://java.sun.com/j2se/1.5.0/docs/api/java/io/BufferedReader.html#readLine()
"Notepad" files are just text files, so you just read it in with a a Reader instance. Since Notepad supports windows Unicode, you may need to specify a charset of "UTF-16LE".
String filename = "myfile.txt";
BufferedReader reader = new BufferedReader(new FileReader(filename));
try{
String line;
//as long as there are lines in the file, print them
while((line = reader.readLine()) != null){
System.out.println(line);
}
} catch (IOException e) {
e.printStackTrace();
}
The basic idea is that you create a file reader object
FileReader fr = new FileReader('file.txt');
and then go over the file line by line parsing each line and saving the stuff to some internal data storage (Array, HashMap).
The while loop in the example you have does just this. The FileReader class will take care of the line ending for you, and will return null when there's no more lines to be read. What you need to do inside the while loop is to parse each line and separate the different bits of data (course name, price etc.) from each other.
EDIT: To parse the lines you would do something like the following. What is inside the while loop depends on how you format the menu files. The following works on the assumption that the menu files contains the price and the name of the course (in that order) separated by a comma on each line.
12.95$,Penne ala Arabiata
8.15$,Fish Soup
Notice that you can't use a comma in the price if you do this. You can of course use a semicolon as the separator between the data fields instead of a comma. The number of data fields is of course also up to you.
String line = "";
// read lines from file
while ((line = fr.readLine()) != null) {
// parse each line
tokens = line.split(",");
String price = tokens[0];
String courseName = tokens[1];
// extract all other information
}
In your final code you'll want to save the data fields into some structure instead of just extracting them from the file. Another thing to note is that the price is a String NOT a number because of the dollar sign. Should you wish to do any calculations with the prices you'll of course need the convert it to a number with parseFloat() or parseDouble().
And of course if you do use the csv (comma separated values) format, it's better to go for a csv library to do the parsing for you instead of writing the parser yourself.
http://opencsv.sourceforge.net/
my question is related to my previous question (How to display data from txt file in specific format).
I was wondering if it is possible at the first place, to store data on txt file in specific format rather than store it first and then retreived it again and display it in specific format?
e.g. instead of store data on txt file like this
Jessica
Walking
20 minutes
Matthew
Run
10 minutes
I wanted to store it in txt file in this format
Jessica Walking 20 minutes
Matthew Run 10 minutes
Regarding your comment to adeel's answer:
Thanks adeel825, however I dont know where to put the "\t".. so far I use this method: new PrintStream(fout).println (name); new PrintStream(fout).println (exercise); new PrintStream(fout).println("10 minutes");
First, don't call "new PrintStream(fout)" everytime you print something. Do this:
PrintStream ps = new PrintStream(fout);
ps.print(name);
ps.print('\t');
ps.print(exercise);
ps.print('\t');
ps.print(time);
ps.println();
Or simply:
PrintStream ps = new PrintStream(fout);
ps.println(name + '\t' + exercise + '\t' + time);
Edit
In response to your comment:
one more question...some of the name are too long and it requires more tab..I have put ps.print('\t','\t'); but it seems not working..
If this is a problem, it sounds like you are trying to store them in the manner you want to display them. I had assumed you were trying to store them in a way that would be easy to parse programmatically. If you want to store them displayed in columns, I'd suggest padding with spaces rather than tabs.
If you know that all the columns are going to be less than, say, 30 characters wide, you could do something like this with printf:
ps.printf("%30s%30s%30s%n", name, exercise, time);
That syntax can look quite byzantine if you're not used to it.. basiclly each "%30s" means pad the string argument so that it is at least 30 characters wide. Your result won't look right if any of the values are 30 or more characters wide. If you can't know ahead of time, you'll have to loop through the values in each column to determine how wide the columns need to be.
There is no problem with storing data this way. All you need to do is write out the values and delimit them with a tab character "\t"
You need to "handcode" your formatting. Best way to do this would be to wrap your file-accessing code somewhere, and create something like:
OpenFile()
CreateEntry(name, type, time)
If you want to write records of fixed length you can use the String.[format][1] method
ps.println(String.format("%20s%20s%20s",name,exercise,time))
This will create a table with 20 characters in each field. You can readup on the syntax here
[1]: http://java.sun.com/javase/6/docs/api/java/lang/String.html#format(java.lang.String, java.lang.Object...)