This might sound a bit complicated, I'll try to simplify what I am asking.
A program I am developing can read and write from/to files using a JTextArea. When files are rather large, it does take a cumbersome amount of time to read the data from that file into the text area. As an example, I have a file that currently has 40,000 lines of text, roughly 50 characters a line; also, some lines wrap. There is quite a lot of text and it takes a lot more time to read from that file then I would like.
Currently, I am using the standard read method utilizing a BufferedReader instance that the JTextArea component includes. What I would like to do is load the JTextArea with a certain amount of text loaded on screen. The rest of the text that is off-screen loaded in a separate thread in the background.
Would using a InputStream and write each character to an array then write characters to the JTextArea be sufficient? Or should there be a different approach at this? I'm trying to accomplish a fast and efficient read method.
There are two, immediate, issues at hand
First, the need to read a file in such away that it can progressively update the UI without causing unacceptable delays
Second, the ability for the JTextArea to actually deal with this amount of data...
The first issues is, relatively, simple to fix. What you need to make sure of is that you are not blocking the Event Dispatching Thread while you read the file and that you are only updating the JTextArea from within the context of the Event Dispatching Thread. To this end a SwingWorker is an excellent choice, for example...
public class FileReaderWorker extends SwingWorker<List<String>, String> {
private File file;
private JTextArea ta;
public FileReaderWorker(File file, JTextArea ta) {
this.file = file;
this.ta = ta;
}
public File getFile() {
return file;
}
public JTextArea getTextArea() {
return ta;
}
#Override
protected List<String> doInBackground() throws Exception {
List<String> contents = new ArrayList<>(256);
try (BufferedReader br = new BufferedReader(new FileReader(getFile()))) {
String text = null;
while ((text = br.readLine()) != null) {
// You will want to deal with adding back in the new line characters
// here if that is important to you...
contents.add(text);
publish(text);
}
}
return contents;
}
#Override
protected void done() {
try {
get();
} catch (InterruptedException | ExecutionException ex) {
ex.printStackTrace();
// Handle exception here...
}
}
#Override
protected void process(List<String> chunks) {
JTextArea ta = getTextArea();
for (String text : chunks) {
ta.append(text);
}
}
}
Take a look at Concurrency in Swing and Worker Threads and SwingWorker for more details
ps- You don't need to use the List to store the contents, I just did it as an example...
The second problem is far more complicated and would need some additional testing to ensure that it is actually a problem, but generally speaking, contents of over about 1mb tends to lead to issues...
To this end, you would need to be able to manage the JScrollPane, be able to request chunks of text from the file both in backwards and forwards direction and try and effectively "fudge" the process (so that you only have the text you need loaded, but can still make it look like you have all the text loaded within the JTextArea)...
You could also take a look at FileChannel, which provides more functionality over the standard java.io classes, including memory mapping, for starters, have a look at Reading, Writing, and Creating Files.
You also might consider using a JList or JTable which are highly optimised for displaying large quantities of data. There are limitations to this, as there is an expectation of fixed row heights, which when changed (to dynamic row heights) can affect the performance, but might be a suitable alternative...
Related
I downloaded my extended listening history from Spotify and I am trying to make a program to turn the data into a list of artists without doubles I can easily make sense of. The file is rather huge because it has data on every stream I have done since 2016 (307790 lines of text in total). This is what 2 lines of the file looks like:
{"ts":"2016-10-30T18:12:51Z","username":"edgymemes69endmylifepls","platform":"Android OS 6.0.1 API 23 (HTC, 2PQ93)","ms_played":0,"conn_country":"US","ip_addr_decrypted":"68.199.250.233","user_agent_decrypted":"unknown","master_metadata_track_name":"Devil's Daughter (Holy War)","master_metadata_album_artist_name":"Ozzy Osbourne","master_metadata_album_album_name":"No Rest for the Wicked (Expanded Edition)","spotify_track_uri":"spotify:track:0pieqCWDpThDCd7gSkzx9w","episode_name":null,"episode_show_name":null,"spotify_episode_uri":null,"reason_start":"fwdbtn","reason_end":"fwdbtn","shuffle":true,"skipped":null,"offline":false,"offline_timestamp":0,"incognito_mode":false},
{"ts":"2021-03-26T18:15:15Z","username":"edgymemes69endmylifepls","platform":"Android OS 11 API 30 (samsung, SM-F700U1)","ms_played":254120,"conn_country":"US","ip_addr_decrypted":"67.82.66.3","user_agent_decrypted":"unknown","master_metadata_track_name":"Opportunist","master_metadata_album_artist_name":"Sworn In","master_metadata_album_album_name":"Start/End","spotify_track_uri":"spotify:track:3tA4jL0JFwFZRK9Q1WcfSZ","episode_name":null,"episode_show_name":null,"spotify_episode_uri":null,"reason_start":"fwdbtn","reason_end":"trackdone","shuffle":true,"skipped":null,"offline":false,"offline_timestamp":1616782259928,"incognito_mode":false},
It is formatted in the actual text file so that each stream is on its own line. NetBeans is telling me the exception is happening at line 19 and it only fails when I am looking for a substring bounded by the indexOf function. My code is below. I have no idea why this isn't working, any ideas?
import java.util.*;
public class MainClass {
public static void main(String args[]){
File dat = new File("SpotifyListeningData.txt");
List<String> list = new ArrayList<String>();
Scanner swag = null;
try {
swag = new Scanner(dat);
}
catch(Exception e) {
System.out.println("pranked");
}
while (swag.hasNextLine())
if (swag.nextLine().length() > 1)
if (list.contains(swag.nextLine().substring(swag.nextLine().indexOf("artist_name"), swag.nextLine().indexOf("master_metadata_album_album"))))
System.out.print("");
else
try {list.add(swag.nextLine().substring(swag.nextLine().indexOf("artist_name"), swag.nextLine().indexOf("master_metadata_album_album")));}
catch(Exception e) {}
System.out.println(list);
}
}
Find a JSON parser you like.
Create a class that with the fields you care about marked up to the parsers specs.
Read the file into a collection of objects. Most parsers will stream the contents so you're not string a massive string.
You can then load the data into objects and store that as you see fit. For your purposes, a TreeSet is probably what you want.
Your code will throw a lot of exceptions only because you don't use braces. Please do use braces in each blocks, whether it is if, else, loops, whatever. It's a good practice and prevent unnecessary bugs.
However, everytime scanner.nextLine() is called, it reads the next line from the file, so you need to avoid using that in this way.
The best way to deal with this is to write a class containing the fields same as the json in each line of the file. And map the json to the class and get desired field value from that.
Your way is too much risky and dependent on structure of the data, even on whitespaces. However, I fixed some lines in your code and this will work for your purpose, although I actually don't prefer operating string in this way.
while (swag.hasNextLine()) {
String swagNextLine = swag.nextLine();
if (swagNextLine.length() > 1) {
String toBeAdded = swagNextLine.substring(swagNextLine.indexOf("artist_name") + "artist_name".length() + 2
, swagNextLine.indexOf("master_metadata_album_album") - 2);
if (list.contains(toBeAdded)) {
System.out.print("Match");
} else {
try {
list.add(toBeAdded);
} catch (Exception e) {
System.out.println("Add to list failed");
}
}
System.out.println(list);
}
}
I am trying to build a calendar app and use a memo section that I labeled notes. I have an add button to add a new note and I want it to add to the current file in the path. I am trying to use a BufferWriter to do this. I have attached the newNote() method that opens a new frame and allows for the new text. I think I am trying to append the new text to the current file but the examples I have seen shows to do it this way. The output of the txt file is not what I expected. I think it is due to calling the textArea object and it is pulling the data of the object and not the input inside the textArea. I am somewhat new to Java and am doing this project for personal use, not a class. Any help and insight would be appreciated. This is also my first time posting in a forum so please let me know if there is a better way of doing this.
The newNote() method.
public static void newNote() {//opens new frame to create a new note
//variables for the new window
JFrame noteFrame = new JFrame("New Note");
JPanel notePanel = new JPanel();
JButton cancelButton = new JButton("Cancel");
JButton addButton = new JButton("Add");
JTextArea textArea = new JTextArea("Add notes here");
//creates and positions buttons
addButton.setBounds(150,330,65,40);
addButton.addActionListener(new ActionListener() {//writes contents to a txt file when Add is clicked
#Override
public void actionPerformed(ActionEvent actionEvent) {
BufferedWriter writer = null;
try {
writer = new BufferedWriter((new FileWriter("/home/skydawg/pCloudDrive/Documents/test/Log.txt", true)));
writer.write(String.valueOf(textArea));
writer.newLine();
writer.flush();
} catch (IOException ioe) {
ioe.printStackTrace();
} finally {
if (writer != null) try {
writer.close();
} catch (IOException ioe2) {
// just ignore it
}
noteFrame.dispose();//closes the frame
}}
});
The output to the txt file
newjavax.swing.JTextArea[,10,10,280x295,layout=javax.swing.plaf.basic.BasicTextUI$UpdateHandler,alignmentX=0.0,alignmentY=0.0,border=javax.swing.plaf.synth.SynthBorder#13e59af,flags=296,maximumSize=,minimumSize=,preferredSize=,caretColor=javax.swing.plaf.ColorUIResource[r=0,g=0,b=0],disabledTextColor=javax.swing.plaf.ColorUIResource[r=218,g=218,b=218],editable=true,margin=javax.swing.plaf.InsetsUIResource[top=0,left=0,bottom=0,right=0],selectedTextColor=javax.swing.plaf.ColorUIResource[r=255,g=255,b=255],selectionColor=javax.swing.plaf.ColorUIResource[r=134,g=108,b=186],colums=0,columWidth=0,rows=0,rowHeight=0,word=true,wrap=true]
Here:
writer.write(String.valueOf(textArea));
the javadoc for that method:
Returns the string representation of the Object argument.
That doesn't do what you expect it to do. That is like calling textArea.toString() ... which again: doesn't give you the string content of your textArea.
Instead, you want to call textArea.getText(), and write that to your file.
Beyond that: you should not throw so many things together. Meaning: write a utility class that just writes some text to a file. Test that separately. And only when that part works, you put your GUI around that. Your approach is: throwing everything together, testing all at once. Which means that you run from one problem to the next. Don't do that. Slice your problem into many small parts, and think hard "how can i solve and test each part in isolation"!
Finally, as you have just seen: GUI programming and GUI components are complicated. Trying to learn these things by trial and error is a really inefficient strategy! It is really better to read a good book or tutorial (see for example). Start with working examples, instead of pulling together 5 different things that are all new to you! And most importantly: each time you use a new class you haven't studied before: take the time to read the corresponding javadoc. Top to bottom! You can invest that time upfront, or you can spend 1 hour not understanding what is going on, to then invest that time.
JTextArea extends JTextComponent. So you have a method called write(java.io.Writer writer).
That means, you can call textArea.write(writer) in your code.
This is just for a simple command-line standalone program in Java.
I'd like to open a file to write to, and keep it open. I need to write formatted floats/doubles to it, in human-readable ASCII, like a CSV file.
I have tried various approaches (1) (2) (3) I have found through my favorite search engine, and they have the form:
try {
// some file handle opening sequence
}
catch ( <some exception> ) {
// do something
}
finally {
// do something else
}
(...or in the case of the third example, the file opening/writing/closing is inside a function that throws an exception.) I realize it's good programming style to make sure that you've opened a file ok, but for my purposes that's really not necessary.
Anyway the problem with the above approach is that outside of the try{} block, the filehandle is closed. I'd like to keep it open, because the kernel of my code consists of a huge loop that I go through a few 100,000 times (say), and each time through I'd like to output a single float (in ASCII) to the file.
With the above form, the only way to do that is to enclose my huge for loop inside the try{} block. Which seems silly. Alternatively, I could re-open the file every time through the loop, but that means additional logic, opening the file as a 'new' file the first time, and appending in all subsequent times.
Is there some way to open the file, keep it open to write to it occasionally, and then close it when I'm done?
Something like:
{
// open file "data.out"
}
for (i=0;i<100000;i++) {
// do a lot of stuff
//
// calculate some quantity "x"
//
// output float "x" in ASCII form, appending it to data.out
}
{
// close data.out
}
Does Java allow that? Thanks.
Of course you can simple store your FileWriter somewhere, as any other variable. You can, for example, encapsulate the whole writing logic in its own class, which offers one write method for your specified format.
But why does it seem silly? Perhaps this approach might help...
public void methodA(File myFile) throws IOException{
try ( FileWriter writer = new FileWriter( myFile ) ) {
writeTo(writer);
}
}
private void writeTo(FileWriter writer) throws IOException {
for (i=0;i<100000;i++) {
// do a lot of stuff
//
// calculate some quantity "x"
//
// output float "x" in ASCII form, appending it to data.out
}
}
This way, one method takes care of the opening/closing/exceptions, while the other method can concentrate on the important writing stuff, using the FileWriter given to it.
as you said the file is closed at the end of the try block. Possibly
the FileWriter object is created inside the try block:
(You did not post a real java code, only a pseudo code.)
Example, hope this helps
public static void main(String[] args)
{
...
BufferedWriter ofs=null; // should by outside the try block
try
{
Path logfile = Paths.set("C:\\temp\\log.log");
ofs = Files.newBufferedWriter(logfile); // new in java 8
YourWorker.doYourJob(ofs);
} catch (Exception e)
{ e.printStackTrace();
} finally
{
if (ofs!=null) { try { ofs.close(); } catch (Exception e) {} }
}
System.exit(1);
} //---------- end of main()
} //---- end of class
I've started drawing plugs in Java, like connectors using bezier curves, but just the visual stuff.
Then I begin wondering about making some kind of modular thing, with inputs and outputs. However, I'm very confused on decisions about how to implement it. Let's say for example, a modular synthesizer, or Pure Data / MaxMSP concepts, in which you have modules, and any module has attributes, inputs and outputs.
I wonder if you know what keywords should I use to search something to read about. I need some basic examples or abstract ideas concerning this kind of interface. Is there any some design pattern that fits this idea?
Since you're asking for a keyword real-time design patterns, overly OOP is often a performance bottleneck to real-time applications, since all the objects (and I guess polymorphism to some extent) add overhead.
Why real-time application? The graph you provided looks very sophisticated,
You process the incoming data multiple times in parallel, split it up, merge it and so on.
Every node in the graph adds different effects and makes different computations, where some computations may take longer than others - this leads to the conclusion, that in order to have uniform data (sound), you have to keep the data in sync. This is no trivial task.
I guess some other keywords would be: sound processing, filter. Or you could ask companies that work in that area for literature.
Leaving the time sensitivity aside, I constructed a little OOP example,
maybe an approach like that is sufficient for less complex scenarios
public class ConnectionCable implements Runnable, Closeable {
private final InputLine in;
private final OutputLine out;
public ConnectionCable(InputLine in, OutputLine out) {
this.in = in;
this.out = out;
// cable connects open lines and closes them upon connection
if (in.isOpen() && out.isOpen()) {
in.close();
out.close();
}
}
#Override
public void run() {
byte[] data = new byte[1024];
// cable connects output line to input line
while (out.read(data) > 0)
in.write(data);
}
#Override
public void close() throws IOException {
in.open();
out.open();
}
}
interface Line {
void open();
void close();
boolean isOpen();
boolean isClosed();
}
interface InputLine extends Line {
int write(byte[] data);
}
interface OutputLine extends Line {
int read(byte[] data);
}
I have 1000 lines in a file which will be served to the user every time he/she loads the application.
My current approach is:
MainActivity: onCreate: Start an AsyncTask
AsyncTask onPreExecute: show progress dialiog
AsyncTask doInBackground: Check if the key/value is present in sharedpreferences, If yes, then do nothing in doInBackground. If no (first time user), read from the raw file and create a stringbuilder. Store the content of StringBuilder as key value pair in sharedpreferences.
AsyncTask onPostExecute: populate textview from sharedpreferences. Dismiss the progress dialog.
The code to read from file in the doInBackground method is:
StringBuilder sb = new StringBuilder();
InputStream textStream = getBaseContext().getResources().openRawResource(R.raw.file);
BufferedReader bReader = new BufferedReader(new InputStreamReader(textStream));
String aJsonLine = null;
try {
while ((aJsonLine = bReader.readLine()) != null) {
sb.append(aJsonLine + System.getProperty("line.separator"));
}
} catch (IOException e) {
e.printStackTrace();
} finally{
try {
bReader.close();
textStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
I am seeing that the user has to wait for around 9-10 seconds for first launch and 4-5 seconds for subsequent launches. Any suggestions to improve the performance in my case.
You don't need to make your user to wait for the whole list to get loaded. Once you have enough data to fill the screen (10-20 items, maybe?), populate the onscreen list or whatever with the data you already have, this will make the delay totally insignificant.
You may check http://developer.android.com/reference/android/content/AsyncTaskLoader.html to see how it's supposed to be done.
As a small sideline to the other comments, as aJsonLine is a String, it's a better idea to store its value along with the newline by using two append() instead of a single one:
sb.append(aJsonLine);
sb.append(System.getProperty("line.separator"));
instead of:
sb.append(aJsonLine + System.getProperty("line.separator"));
With the later, both the aJsonLine and the result of System.getProperty("line.separator")) need to be converted to StringBuilder before the concatenation between them with can take place and the final value be passed as a parameter.
Of course, you should also cache the value of System.getProperty("line.separator")) al
I'd rather read the JSON stream through a JsonReader and extract the name value pairs I'm interested in. String concatenation / garbage collection are expensive operations. The way the code is written now, these operations will slow down the task. Also there are inefficiencies in the code like accessing the line separator on every iteration of the loop System.getProperty("line.separator").
You should see a significant performance boost just by using a JSONReader.