We Keep Coding

How to parse a text file with separated json objects in Java? [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 5 years ago.
Improve this question
I have a text file which gets updated in every 15-16 minute with some json data. These json data are separated by #### lines in between. Snippet of the file is :
[{"accountId":"abc","items":[{"serviceName":"XYZ","dataCenter":"TG","startTimeUtc":"2017-04-05T19:57:33.509+0000","endTimeUtc":"2017-04-05T19:57:33.509+0000","usage":[{"resourceName":"XYZ_EE_PAAS_GATEWAYS","quantity":7,"units":"number"}]}]},{"accountId":"XYZp1cm9mbe","items":[{"serviceName":"XYZ","dataCenter":"TG","startTimeUtc":"2017-04-05T19:57:33.509+0000","endTimeUtc":"2017-04-05T19:57:33.509+0000","usage":[{"resourceName":"XYZ_EE_PAAS_GATEWAYS","quantity":6,"units":"number"}]}]}]
######################
[{"accountId":"abc","items":[{"serviceName":"XYZ","dataCenter":"TG","startTimeUtc":"2017-04-05T19:59:33.523+0000","endTimeUtc":"2017-04-05T19:59:33.523+0000","usage":[{"resourceName":"XYZ_EE_PAAS_GATEWAYS","quantity":7,"units":"number"}]}]},{"accountId":"XYZp1cm9mbe","items":[{"serviceName":"XYZ","dataCenter":"TG","startTimeUtc":"2017-04-05T19:59:33.523+0000","endTimeUtc":"2017-04-05T19:59:33.523+0000","usage":[{"resourceName":"XYZ_EE_PAAS_GATEWAYS","quantity":6,"units":"number"}]}]}]
######################
[{"accountId":"abc","items":[{"serviceName":"XYZ","dataCenter":"TG","startTimeUtc":"2017-04-05T20:01:33.531+0000","endTimeUtc":"2017-04-05T20:01:33.531+0000","usage":[{"resourceName":"XYZ_EE_PAAS_GATEWAYS","quantity":7,"units":"number"}]}]},{"accountId":"XYZp1cm9mbe","items":[{"serviceName":"XYZ","dataCenter":"TG","startTimeUtc":"2017-04-05T20:01:33.531+0000","endTimeUtc":"2017-04-05T20:01:33.531+0000","usage":[{"resourceName":"XYZ_EE_PAAS_GATEWAYS","quantity":6,"units":"number"}]}]}]
######################
This file gets updated every 15-16 minute with new entry. I want to read the file and store the latest entry excluding the #### line in a json object. How to do it in java ? I don't want to use the 15 min interval as it is not constant.
My simple requirement is at any point of time I would read the file and want to retrieve the last json in above the ### line.

With Java 8, you can do it like this:
public JsonObject retrieveLastEntry(Path path) throws IOException {
String[] jsonLines = Files.lines(path)
.filter(line -> !line.equals("######################")
.toArray();
String lastJsonLine = jsonLines[jsonLines.length - 1];
return MyFavoriteJsonParser.parse(lastJsonLine);
}
MyFavoriteJsonParser refers to whatever JSON library you want to use (maybe have a look at this question). There might be few performance considerations here. If your file is very large (considerably more than a few MB), then the .toArray() call maybe not right for you. In fact, if performance is extremely crucial, you might even need to consider parsing the file backwards. But the golden rule for performance optimization is to go with a simple solution first and see if (and where) it might be not performant enough.
If your JSON goes across lines, however, the Stream API is not the best choice. In that case, a regular iteration comes to the rescue:
public JsonObject retrieveLastEntry(File file) throws IOException {
String lastJson = "";
StringBuffer sb = new StringBuffer();
try (BufferedReader reader = new BufferedReader(new InputStreamReader(new FileReader(file), "UTF-8")))) {
String line;
while ((line = reader.readLine()) != null) {
if (line.equals("######################") {
lastJson = sb.toString(); sb.setLength(0);
} else {
sb.append(line).append('\n');
}
}
return MyFavoriteJsonParser.parse(lastJsonLine);
}
The basic idea is to aggregate lines between the ###... and put them into a variable whenever a new separator is reached. You still might want to consider the case of having no entry at all and handle IOExceptions properly.
I think this is pretty much the idiomatic way of doing it.

How to handle importing a CSV file with differing column lengths

Im working on a project for school and am having a really hard time figuring out how to import and format a CSV file into a usable format. The CSV contains a movie name in the first column, and showtimes in the rows, so it would look something like this.
movie1, 7pm, 8pm, 9pm, 10pm
movie2, 5pm, 8pm
movie3, 3pm, 7pm, 10pm
I think I want to split each row into its own array, maybe an arraylist of the arrays? I really dont know where to even start so any pointers would be appreciated.
Preferably dont want to use any external libraries.

I would go with a Map having movie name as key and timings as list like the one below:
Map<String, List<String>> movieTimings = new HashMap<>();
It will read through csv file and put the values into this map. If the key already exists then we just need to add the value into the list. You can use computeIfAbsent method of Map (Java 8) to see whether the entry exists or not, e.g.:
public static void main(String[] args) {
Map<String, List<String>> movieTimings = new HashMap<>();
String timing = "7pm";//It will be read from csv
movieTimings.computeIfAbsent("test", s -> new ArrayList<>()).add(timing);
System.out.println(movieTimings);
}
This will populate your map once the file is read. As far as reading of file is concerned, you can use BuffferedReader or OpenCSV (if your project allows you to use third party libraries).

I have no affiliation with Univocity - but their Java CSV parser is amazing and free. When I had a question, one of the developers got back to me immediately. http://www.univocity.com/pages/about-parsers
You read in a line and then cycle through the fields. Since you know the movie name is always there and at least one movie time, you can set it up any way you like including an arraylist of arraylists (so both are variable length arrays).
It works well with our without quotes around the fields (necessary when there are apostrophes or commas in the movie names). In the problem I solved, all rows had the same number of columns, but I did not know the number of columns before I parsed the file and each file often had a different number of columns and column names and it worked perfectly.

You can use opencsv to read the CSV file and add each String[] to an ArrayList. There are examples in the FAQ section of opencsv's website.
Edit: If you don't want to use external libraries you can read the CSV using a BufferedReader and split the lines by commas.
BufferedReader br = null;
try{
List<String[]> data = new ArrayList<String[]>();
br = new BufferedReader(new FileReader(new File("csvfile")));
String line;
while((line = br.readLine()) != null){
String[] lineData = line.split(",");
data.add(lineData);
}
}catch(Exception e){
e.printStackTrace();
}finally{
if(br != null) try{ br.close(); } catch(Exception e){}
}

Reading parts of a text file

EDIT: I guess this thread can be closed, since all my questions have been answered! Thanks to everyone who helped me!
EDIT: I stumbled upon an error at openFileInput("myfilename.txt");. This is the error: The method openFileInput(String) is undefined for the type Game. I read an answer here: Android File I/O openFileInput() undefined, but must admit that I don't quite understand it...
I'm trying to read parts of a text file, till the token ";". This is the code I wrote:
InputStream instream = openFileInput("myfilename.txt");
String line=null;
InputStreamReader inputreader = new InputStreamReader(instream);
BufferedReader buffreader = new BufferedReader(inputreader);
while((line=buffreader.readLine())!=null){
String[] parts=line.split(";");
int intOne = Integer.parseInt(parts([0]);
int intTwo = Integer.parseInt(parts([1]);
String strLine = parts([3]);
}
public static void Start(){
while(there is text in the file){
// read first line till ';';
// name that variable intOne;
// read first line after ';' till next ';';
// name that variable intTwo;
// read next line, declare as strLine;
// System.out.println(strLine);
}
}
Beneath it is the idea of what it should do. But I have some questions:
Am I right to say that the String[] parts is an array?
I want to have a bigger file, but read only 3 lines per loop. Or could I, when I have a file of 100 lines, read that all at once, then recall them from the parts[]? Or would that take way too much time?
Where should the text file be? When I'm testing in Eclipse, in the project folder? And when I export it inside the jar?
Hope someone can answer my questions!
(My source is: Read specific string from each line of text file using BufferedReader in java, all credits to Gilbert Le Blanc!)
EDIT: When I do this in the file:
Hello,
I am coding;
Will the pars[0] be Hello,, because that's one line, or Hello, I am coding? And will it take the enter with it?
Another EDIT:
I wish to create some sort of textbased RPG engine, where you only have to edit the text file, to change the story. For example (in the text file):
30;60; //Score needed for this piece of the story
Hello!; // The text
Hi!;5; // The first possible answer; the score you'll get when you answer this
Shut up!;-5; // The second possible answer; the score you'll get when you answer this

What you rly want is reading one char after another. I used some nice BufferedSegmentReader from the framework Smooks, which could be interesting for you. Look at the sourcecode here:
http://grepcode.com/file/repo1.maven.org/maven2/org.milyn/milyn-smooks-all/1.5/org/milyn/edisax/BufferedSegmentReader.java
It reads characters one after another from a stream and puts it into a StringBuffer. There are delimiters to indicate when one "segment" is done reading, and after that you can work with this segment till you tell the BufferedSegmentReader to move on.
I think this rly suits your case and is an approach you are looking for.

java program to retreive page source from google search automatically [duplicate]

I have 30000 dictionary words. In that I want to to search each word in Google and want to find hits of each word using Java program. Is it possible?

Look up <estimatedTotalResultsCount> using Google's SOAP search API.
You'll be limited to 1000 queries per day though. This limit is removed if you use their AJAX API.

Since your duplicate post is closed, I'll post my answer here as well:
Whether this is possible or not doesn't really matter: Google doesn't want you to do that. They have a public AJAX-search API developers can use: http://code.google.com/apis/ajaxsearch/web.html

Here is a Sun tutorial on Reading to and Writing from an URLConnection.
The simplest URL I can see to make a Google search is like:
http://www.google.com/#q=wombat

Reading from a url with java is pretty straight forward. A basic working example is as follows
public Set<String> readUrl(String url) {
String line;
Set<String> lines = new HashSet<String>();
try {
URL url = new URL(url);
URLConnection page = url.openConnection();
BufferedReader in = new BufferedReader( new InputStreamReader(page.getInputStream()));
while ((line = in.readLine()) != null) {
lines.add(line);
}
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return lines;
}

I'd recommend separating your problem into pieces. Get each one working, then marry them together for the solution you want.
You have a few things going on here:
Downloading text from a URL
Scanning a stream of characters and breaking it up into words
Iterating through a list of words and tallying up the hits from your dictionary
Computer science is all about taking large problems and decomposing them into smaller ones. I'd recommend that you start learning how to do that now.

Reading an XML File using FileInputStream (for Java)?

For my project I have to serialize and deserialize a random tree using Java and XStream. My teacher made the Tree/RandomTree algorithms, so I don't have to worry about that. What I don't know how to do is this: I am using FileInputStream to read/write the xml file that I serialized and deserialized, but when I deserialize, I do not know the method used to read the file. After I read the file I should be able to convert it from XML and then print it out as a string. Here's what I have so far. (I imported everything correctly, just didn't add it to my code segment).
FileInputStream fin;
try
{
// Open an input stream
fin = new FileInputStream ("/Users/Pat/programs/randomtree.xml");
//I don't know what to put below this, to read FileInpuStream object fin
String dexml = (String)xstream.fromXML(fin);
System.out.println(dexml);
// Close our input stream
fin.close();
System.out.println(dexml);
// Close our input stream
fin.close();
}
// Catches any error conditions
catch (IOException e)
{
System.err.println ("Unable to read from file");
System.exit(-1);
}
Edit: I figured it out; I don't think I have to print it as a string, I just needed to make a benchmarking framework to time it and such, but thanks again!

The xstream.fromXML() method will do the reading from the input stream for you. I think the problem is that you are casting the return value from xstream.fromXML(fin) into a String when it should be cast to the type of object you originally serialized (RandomTree I assume). So the code would look like this:
RandomTree tree = (RandomTree)xstream.fromXML(fin);
EDIT: after clarification in comments, the author's goal is to first read into a String so the XML contents can be printed before deserialization. With that goal in mind, I recommend taking a look at the IOUtils library mentioned in this thread

From what I understand from http://x-stream.github.io/tutorial.html (I've never worked with XStream before), you need to define your types first. Casting to String is definitely wrong, you probably want a customized type (depending on what's inside your random XML), then you need to map the XML tags to your members:
e.g.
xstream.alias("person", Person.class);
xstream.alias("phonenumber", PhoneNumber.class);
meaning that it maps the "person" tag inside your XML to your Person class.
To derserialize, you can do:
RandomTree myRandomTree = (RandomTree)xstream.fromXML( xml );
Also, you are closing your stream twice, and you probably want to do it in a finally block :)
edit: Having read your comment above...
Your task involves two steps:
Deserialization
Serialization
In order to serialize your object, you must deserialize it first from your input file.
To output your Object as String, simply do
String xml = xstream.toXML( myRandomTree );