Regular Expressions to Display Messages from a Text Log File - java

I'm trying to figure out how to use regular expressions to condense and sort the information I'm getting from this code. Here's the code and I'll explain as I go:
import java.io.*;
import java.util.*;
public class baseline
{
// Class level variables
static Scanner sc = new Scanner(System.in);
public static void main(String[] args) throws IOException,
FileNotFoundException { // Start of main
// Variables
String filename;
// Connecting to the output file with a buffer
PrintWriter outFile = new PrintWriter(
new BufferedWriter(
new FileWriter("chatOutput.log")));
// Get the input file
System.out.print("Please enter full name of the file: ");
filename = sc.next();
// Assign the name of the input file to a file object
File log = new File(filename);
String textLine = null; // Null
String outLine = ""; // Null
BufferedWriter bw = null;
try
{
// assigns the input file to a filereader object
BufferedReader infile = new BufferedReader(new FileReader(log));
sc = new Scanner(log);
while(sc.hasNext())
{
String line=sc.nextLine();
if(line.contains("LANTALK"))
System.out.println(line);
} // End of while
try
{
// Read data from the input file
while((textLine = infile.readLine()) != null)
{
// Print to output file
outLine = textLine;
sc = new Scanner (outLine);
while(sc.hasNext())
{
String line=sc.nextLine();
if(line.contains("LANTALK"))
outFile.printf("%s\n",outLine);
}// end of while
} // end of while
} // end of try
finally // This gets executed even when an exception is thrown
{
infile.close();
outFile.close();
} // End of finally
} // End of try
catch (FileNotFoundException nf) // Goes with first try
{
System.out.println("The file \""+log+"\" was not found");
} // End of catch
catch (IOException ioex) // Goes with second try
{
System.out.println("Error reading the file");
} // End of catch
} // end of main
} // end of class
So I'm reading an input file, getting only the lines that display "LANTALK", and printing them out to another file. And here is a sample of what the output looks like so far:
14:29:39.731 [D] [T:000FEC] [F:LANTALK2C] <CMD>LANMSG</CMD>
<MBXID>922</MBXID><MBXTO>5608</MBXTO><SUBTEXT>LanTalk</SUBTEXT><MOBILEADDR>
</MOBILEADDR><LAP>0</LAP><SMS>0</SMS><MSGTEXT>It is mailing today right?
</MSGTEXT>
14:41:33.703 [D] [T:000FF4] [F:LANTALK2C] <CMD>LANMSG</CMD>
<MBXID>929</MBXID><MBXTO>5601</MBXTO><SUBTEXT>LanTalk</SUBTEXT><MOBILEADDR>
</MOBILEADDR><LAP>0</LAP><SMS>0</SMS><MSGTEXT>Either today or tomorrow -
still waiting to hear. </MSGTEXT>
And what I need is to get all of the characters between <MSGTEXT> and </MSGTEXT> to be able to display the message cleanly. How should I write this into the code to repeat with every "LANTALK" line and still write out correctly? Thanks!

Try it with Jsoup.
Example:
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
....
while(sc.hasNext())
{
String line=sc.nextLine();
if(line.contains("LANTALK")){
Document doc = Jsoup.parse(line);
Element msg = doc.select("MSGTEXT").first();
System.out.println(msg.text());
}
System.out.println(line);
} // End of while
.....

You can find MSGTEXT using a regex:
<MSGTEXT>(.*?)</MSGTEXT>
However, some of the messages contain newlines, which makes this a bit more difficult.
One way to get past this is to read the entire file into a String, and then look for matches.
try {
String text = new String(Files.readAllBytes(Paths.get(log)));
Matcher m = Pattern.compile("<MSGTEXT>(.*?)</MSGTEXT>", Pattern.DOTALL).matcher(text);
while (m.find()) {
System.out.println("Message: " + m.group(1));
}
} catch (IOException e) {
//Handle exception
}
Console output:
Message: It is mailing today right?
Message: Either today or tomorrow -
still waiting to hear.
Keep in mind that if you are dealing with large log files this approach could use a lot of memory.
Also note that parsing XML with regex is generally considered a bad idea; it works fine for now, but if you plan on doing anything more complicated you should use an XML parser as others have suggested.

Related

File manipulation (changing lines in a File) in java

I'm trying to read in a file and change some lines.
The instruction reads "invoking java Exercise12_11 John filename removes the string John from the specified file."
Here is the code I've written so far
import java.util.Scanner;
import java.io.*;
public class Exercise12_11 {
public static void main(String[] args) throws Exception{
System.out.println("Enter a String and the file name.");
if(args.length != 2) {
System.out.println("Input invalid. Example: John filename");
System.exit(1);
}
//check if file exists, if it doesn't exit program
File file = new File(args[1]);
if(!file.exists()) {
System.out.println("The file " + args[1] + " does not exist");
System.exit(2);
}
/*okay so, I need to remove all instances of the string from the file.
* replacing with "" would technically remove the string
*/
try (//read in the file
Scanner in = new Scanner(file);) {
while(in.hasNext()) {
String newLine = in.nextLine();
newLine = newLine.replaceAll(args[0], "");
}
}
}
}
I don't quite know if I'm headed in the correct direction because I'm having some issue getting the command line to work with me. I only want to know if this is heading in the correct direction.
Is this actually changing the lines in the current file, or will I need different file to make alterations? Can I just wrap this in a PrintWriter to output?
Edit: Took out some unnecessary information to focus the question. Someone commented that the file wouldn't be getting edited. Does that mean I need to use PrintWriter. Can I just create a file to do so? Meaning I don't take a file from user?
Your code is only reading file and save lines into memory. You will need to store all modified contents and then re-write it back to the file.
Also, if you need to keep newline character \n to maintain format when re-write back to the file, make sure to include it.
There are many ways to solve this, and this is one of them. It's not perfect, but it works for your problem. You can get some ideas or directions out of it.
List<String> lines = new ArrayList<>();
try {
Scanner in = new Scanner(file);
while(in.hasNext()) {
String newLine = in.nextLine();
lines.add(newLine.replaceAll(args[0], "") + "\n"); // <-- save new-line character
}
in.close();
// save all new lines to input file
FileWriter fileWriter = new FileWriter(args[1]);
PrintWriter printWriter = new PrintWriter(fileWriter);
lines.forEach(printWriter::print);
printWriter.close();
} catch (IOException ioEx) {
System.err.println("Error: " + ioEx.getMessage());
}

Parsing a log file as XML

I've got a code here that takes the input of a file, and displays only certain lines containing the words "LANTALK" to the console, and then it writes out those lines to an external files. What I need, is to be able to filter the information within the lines to display in a certain way.
Here is the full code:
import java.io.*;
import java.util.*;
public class baseline
{
// Class level variables
static Scanner sc = new Scanner(System.in);
public static void main(String[] args) throws IOException,
FileNotFoundException { // Start of main
// Variables
String filename;
// Connecting to the output file with a buffer
PrintWriter outFile = new PrintWriter(
new BufferedWriter(
new FileWriter("chatOutput.log")));
// Get the input file
System.out.print("Please enter full name of the file: ");
filename = sc.next();
// Assign the name of the input file to a file object
File log = new File(filename);
String textLine = null; // Null
String outLine = ""; // Null
BufferedWriter bw = null;
try
{
// assigns the input file to a filereader object
BufferedReader infile = new BufferedReader(new FileReader(log));
sc = new Scanner(log);
while(sc.hasNext())
{
String line=sc.nextLine();
if(line.contains("LANTALK"))
System.out.println(line);
} // End of while
try
{
// Read data from the input file
while((textLine = infile.readLine()) != null)
{
// Print to output file
outLine = textLine;
sc = new Scanner (outLine);
while(sc.hasNext())
{
String line=sc.nextLine();
if(line.contains("LANTALK"))
outFile.printf("%s\n",outLine);
}// end of while
} // end of while
} // end of try
finally // This gets executed even when an exception is thrown
{
infile.close();
outFile.close();
} // End of finally
} // End of try
catch (FileNotFoundException nf) // Goes with first try
{
System.out.println("The file \""+log+"\" was not found");
} // End of catch
catch (IOException ioex) // Goes with second try
{
System.out.println("Error reading the file");
} // End of catch
} // end of main
} // end of class
Here is sample line of the input file:
08:25:26.668 [D] [T:000FF4] [F:LANTALK2C] <CMD>LANMSG</CMD>
<MBXID>1124</MBXID><MBXTO>5760</MBXTO><SUBTEXT>LanTalk</SUBTEXT><MOBILEADDR>
</MOBILEADDR><LAP>0</LAP><SMS>0</SMS><MSGTEXT>but didn't give me the info I
needed</MSGTEXT>
08:25:26.672 [+] [T:000FF4] [S:1:1:1124:5607:5] LANMSG [0/2 | 0]
And here is what I'm trying to get the output to look like:
8:25:00 AM [Steve Jobs] to [John Smith] but didn't give me the info I needed
Does anyone have any suggestions on the best way to do this? I'm thinking some sort of an XML parser, but the file being read in is a .log, and I'm unsure of how to convert it in this instance since it has already been read. Thanks!
You need an hybrid approach: read from the buffered reader the line that has LANTALK then the store in an stringbuilder from the first < to the last >. After that jsoup can do the xml trick for you
Edit
download jsoup from here
https://mvnrepository.com/artifact/org.jsoup/jsoup/1.8.3
Then in your if block just play with String indexOf and lastIndexOf until you have read the xml inside the log lines (put that into an string builder)

Java - How to remove blank lines from a text file

I want to be able to remove blank lines from a text file, for example:
Average Monthly Disposable Salary
1
Switzerland
$6,301.73
2014
2
Luxembourg
$4,479.80
2014
3
Zambia
$4,330.98
2014
--To This:
Average Monthly Disposable Salary
1
Switzerland
$6,301.73
2014
2
Luxembourg
$4,479.80
2014
3
Zambia
$4,330.98
2014
All of the code I have is below:
public class Driver {
public static void main(String[] args)
throws Exception {
Scanner file = new Scanner(new File("src/data.txt"));
PrintWriter write = new PrintWriter("src/data.txt");
while(file.hasNext()) {
if (file.next().equals("")) {
continue;
} else {
write.write(file.next());
}
}
print.close();
file.close();
}
}
The problem is that the text file is empty once I go back and look at the file again.
Im not sure why this is acting this way since they all seem to be blank characters, \n showing line breaks
Your code was almost correct, but there were a few bugs:
You must use .nextLine() instead of .next()
You must write to a different file while reading the original one
Your print.close(); should be write.close();
You forgot to add a new line after each line written
You don't need the continue; instruction, since it's redundant.
public static void main(String[] args) {
Scanner file;
PrintWriter writer;
try {
file = new Scanner(new File("src/data.txt"));
writer = new PrintWriter("src/data2.txt");
while (file.hasNext()) {
String line = file.nextLine();
if (!line.isEmpty()) {
writer.write(line);
writer.write("\n");
}
}
file.close();
writer.close();
} catch (FileNotFoundException ex) {
Logger.getLogger(Test.class.getName()).log(Level.SEVERE, null, ex);
}
}
If you want to keep the original name, you can do something like:
File file1 = new File("src/data.txt");
File file2 = new File("src/data2.txt");
file1.delete();
file2.renameTo(file1);
Try org.apache.commons.io and Iterator
try
{
String name = "src/data.txt";
List<String> lines = FileUtils.readLines(new File(name));
Iterator<String> i = lines.iterator();
while (i.hasNext())
{
String line = i.next();
if (line.trim().isEmpty())
i.remove();
}
FileUtils.writeLines(new File(name), lines);
}
catch (IOException e)
{
e.printStackTrace();
}
You could copy to a temporary file and rename it.
String name = "src/data.txt";
try(BufferedWriter bw = new BufferedWriter(name+".tmp)) {
Files.lines(Paths.get(name))
.filter(v -> !v.trim().isEmpty())
.forEach(bw::println);
}
new File(name+".tmp").renameTo(new File(name));
This piece of code solved this problem for me
package linedeleter;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileWriter;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.Scanner;
public class LineDeleter {
public static void main(String[] args) throws FileNotFoundException, IOException {
File oldFile = new File("src/data.txt"); //Declares file variable for location of file
Scanner deleter = new Scanner(oldFile); //Delcares scanner to read file
String nonBlankData = ""; //Empty string to store nonblankdata
while (deleter.hasNextLine()) { //while there are still lines to be read
String currentLine = deleter.nextLine(); //Scanner gets the currentline, stories it as a string
if (!currentLine.isBlank()) { //If the line isn't blank
nonBlankData += currentLine + System.lineSeparator(); //adds it to nonblankdata
}
}
PrintWriter writer = new PrintWriter(new FileWriter("src/data.txt"));
//PrintWriter and FileWriter are declared,
//this part of the code is when the updated file is made,
//so it should always be at the end when the other parts of the
//program have finished reading the file
writer.print(nonBlankData); //print the nonBlankData to the file
writer.close(); //Close the writer
}
}
As mentioned in the comments, of the code block, your sample had the print writer declared after your scanner meaning that the program had already overwritten your current file of the same name. Therefore there was no code for your scanner to read and thus, the program gave you a blank file
the
System.lineSeparator()
Just adds an extra space, this doesn't stop the program from continuing to write on that space, however, so it's all good

Reading Strings from text files in java

im studying for my programming final exam. I have to write a program which opens a file which is stored in the string fileName and look in the file for a String called personName and this should print the first string after personName then the program should terminate after printing it,
if the argument personName is not in the file then it should print "this name doen't exsit" then if an IOException occurs it should then print "there is an IO Error" and the program should exsit using system.exit(0)
the program should use the file info.txt and each line should contain two strings
first string name and second age.
everything must be in one method
data.txt contains
Max 60.0
joe 19.0
ali 20.0
my code for this so far is :
public class Files{
public void InfoReader(String fileName, String personName)
{
try{
try{
// Open the file that is the first
// command line parameter
FileInputStream fstream = new FileInputStream("C://rest//data.txt");
// Get the object of DataInputStream
DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
//Read File Line By Line
while ((fileName = br.readLine()) != null) {
// Print the content on the console
(new Files()).infoReader("info.txt","Joe"); //this prints the age
}
//Close the input stream
in.close();
}
catch (IOException e)
{//Catch exception if any
System.out.println(" there is an IO Error");
System.exit(0);
}
}
catch (Exception e)
{//Catch exception if any
System.out.println("that name doesn't exists");
}
}
}
infoReader(info.txt,Joe); should print 19.0
But I am getting a java.lang.StackOverflowError
any help would be much appreciated!!
Thanks in advance!
This is what I think you are trying to do. And if doesn't, at least can work as an example. Just as amit mentions, your current error is because of the recursive call, which I think is not necessary.
import java.io.BufferedReader;
import java.io.DataInputStream;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStreamReader;
public class Files {
public void InfoReader(String fileName, String personName) {
try {
// Open the file that is the first command line parameter
FileInputStream fstream = new FileInputStream(fileName);
// Get the object of DataInputStream
DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String line = null;
//Loop until there are no more lines in the file
while((line = br.readLine()) != null) {
//Split the line to get 'personaName' and 'age'.
String[] lineParts = line.split(" ");
//Compare this line personName with the one provided
if(lineParts[0].equals(personName)) {
//Print age
System.out.println(lineParts[1]);
br.close();
System.exit(0);
}
}
br.close();
//If we got here, it means that personName was not found in the file.
System.out.println("that name doesn't exists");
} catch (IOException e) {
System.out.println(" there is an IO Error");
}
}
}
If you use the Scanner class, it would make your life so much easier.
Scanner fileScanner = new Scanner (new File(fileName));
while(fileScanner.hasNextLine()
{
String line = fileScanner.nextLine();
Scanner lineScanner = new Scanner(line);
String name = lineScanner.next(); // gets the name
double age = Double.parseDouble(lineScanner.next()); // gets the age
// That's all really! Now do the rest!
}
Use commons-io and dont forget the encoding!
List<String> lines = FileUtils.readLines(file, encoding)

How do I make my java code search only for a to z and 0 to 9

My java code takes almost 10-15minutes to run (Input file is 7200+ lines long list of query). How do I make it run in short time to get same results?
How do I make my code to search only for aA to zZ and 0 to 9??
If I don't do #2, some characters in my output are shown as "?". How do I solve this issue?
// no parameters are used in the main method
public static void main(String[] args) {
// assumes a text file named test.txt in a folder under the C:\file\test.txt
Scanner s = null;
BufferedWriter out = null;
try {
// create a scanner to read from the text file test.txt
FileInputStream fstream = new FileInputStream("C:\\user\\query.txt");
DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
// Write to the file
out = new BufferedWriter(new FileWriter("C:\\user\\outputquery.txt"));
// keep getting the next String from the text, separated by white space
// and print each token in a line in the output file
//while (s.hasNext()) {
// String token = s.next();
// System.out.println(token);
// out.write(token + "\r\n");
//}
String strLine="";
String str="";
while ((strLine = br.readLine()) != null) {
str+=strLine;
}
String st=str.replaceAll(" ", "");
char[]third =st.toCharArray();
System.out.println("Character Total");
for(int counter =0;counter<third.length;counter++){
//String ch= "a";
char ch= third[counter];
int count=0;
for ( int i=0; i<third.length; i++){
// if (ch=="a")
if (ch==third[i])
count++;
}
boolean flag=false;
for(int j=counter-1;j>=0;j--){
//if(ch=="b")
if(ch==third[j])
flag=true;
}
if(!flag){
System.out.println(ch+" "+count);
out.write(ch+" "+count);
}
}
// close the output file
out.close();
} catch (IOException e) {
// print any error messages
System.out.println(e.getMessage());
}
// optional to close the scanner here, the close can occur at the end of the code
finally {
if (s != null) {
// close the input file
s.close();
}
}
}
For something like this I would NOT recommend java though it entirely possible it is much easier with GAWK or something similar. GAWK also has java like syntax so its easy to pick up. You should check it out.
SO isn't really the place to ask such a broad how-do-I-do-this-question but I will refer you to the following page on regular expression and text match in Java. Also, check out the Javadocs for regexes.
If you follow that link you should get what you want, else you could post a more specific question back on SO.

Categories