We Keep Coding

Multi word replacement within xml file using java

I have an xml-base .tbx file containing code like this:
<descripGrp>
<descrip type="subjectField">406001</descrip>
</descripGrp>
<langSet xml:lang="en">
<tig>
<term>competence of the Member States</term>
<termNote type="termType">fullForm</termNote>
<descrip type="reliabilityCode">3</descrip>
</tig>
</langSet>
<langSet xml:lang="pl">
<tig>
<term>kompetencje państw członkowskich</term>
<termNote type="termType">fullForm</termNote>
<descrip type="reliabilityCode">3</descrip>
</tig>
</langSet>
</termEntry>
<termEntry id="IATE-290">
<descripGrp>
<descrip type="subjectField">406001</descrip>
</descripGrp>
I want to search and replace within entire (almost 50 MiB) file for codes from the field "subjectField" and replace the with proper text, eg.
406001 is for Political ideology, 406002 for Political institution.
I have a table with codes and corresponding names:
406001 Political ideology
406002 Political institution
406003 Political philosophy
There's five hundred of such codes so doing it by hand would take like forever.
I'm not a programmer (I'm learnig) but I know a little java so I made some little app which, I supposed, would help me, however the result is discouraging (luckily I'm not discouraged :))
That's what I wrote, the result is that it works extremely slow, doesn't replace those codes at all. It processed 1/5 of the file in 15 minutes (!). Additionally there are no new line characters in the output file so the entire xml code is in one line.
Any tips on which way I should go?
File log= new File("D:\\IATE\\export_EN_PL_2017-03-07_All_Langs.tbx"); // TBX file to be processed
File newe = new File("D:\\IATE\\now.txt"); // output file
String search = "D:\\IATE\\org.txt"; // file containing codes "40600" etc
String replace = "D:\\IATE\\rplc.txt"; // file containing names
try {
FileReader fr = new FileReader(log);
String s;
String s1;
String s2;
String totalStr = "";
String tot1 = "";
String tot2 = "";
FileReader fr1 = new FileReader(search);
FileReader fr2 = new FileReader(replace);
try (BufferedReader br = new BufferedReader(fr)) {
try (BufferedReader br1 = new BufferedReader(fr1)) {
try (BufferedReader br2 = new BufferedReader(fr2)) {
while ((s = br.readLine()) != null) {
totalStr += s;
while((s1 = br1.readLine()) != null){
tot1 += s1;
while ((s2 = br2.readLine()) != null){
tot2 += s2;
}
}
totalStr = totalStr.replaceAll(tot1, tot2);
FileWriter fw = new FileWriter(newe);
fw.write(totalStr);
fw.write("\n");
fw.close();
}
} catch (Exception e) {
e.printStackTrace();
}
} catch (Exception e) {
e.printStackTrace();
}
}
} catch (Exception e) {
e.printStackTrace();
}
}

Its going to take a lot of redundant work to traverse 2 files to get matching values. Before you replace values in the .tbx files you should set up a properties file to read from. Here's a function that would do that:
public static Properties getProps(String pathToNames, String pathToNumbers){
Properties prop = new Properties();
try{
File names = new File(pathToNames);
BufferedReader theNames = new BufferedReader( new InputStreamReader (new FileInputStream(names)));
File numbers = new File(pathToNumbers);
BufferedReader theNumbers = new BufferedReader( new InputStreamReader (new FileInputStream(numbers)));
String name;
String number;
while(((name = theNames.readLine())!= null)&&((number = theNumbers.readLine())!= null)){
prop.put(number, name);
}
theNames.close();
theNumbers.close();
}catch(Exception e){
e.printStackTrace();
}
return prop;
}
Assuming you are using Java 8, you can check that the function is working with this:
thePropertiesFile.forEach((Object key, Object value) ->{
System.out.println(key+ " " +value);
});
Now you can write a function that will convert properly. Use a PrintStream to achieve the output functionality you want.
static String workingDir = System.getProperty("user.dir");
public static void main(String[] args){
Properties p = getProps(workingDir+"path/to/names.txt",workingDir+"path/to/numbers.txt");
File output = new File(workingDir+"path/to/output.txt");
try {
PrintStream ps = new PrintStream(output);
BufferedReader tbx = new BufferedReader(new InputStreamReader (new FileInputStream(new File(workingDir+"path/to/the.tbx"))));
String currentLine;
String theNum;
String theName;
int c; //temp index
int start;
int end;
while((currentLine = tbx.readLine()) != null){
if(currentLine.contains("subjectField")){
c = currentLine.indexOf("subjectField");
start = currentLine.indexOf(">", c)+1;
end = currentLine.indexOf("<", c);
theNum = currentLine.substring(start, end);
theName = p.getProperty(theNum);
currentLine = currentLine.substring(0,start)+theName+currentLine.substring(end);
}
ps.println(currentLine);
}
ps.close();
tbx.close();
} catch (IOException e) {
e.printStackTrace();
}
}
For numbers that don't exist, this will replace them with a null string. You can update that for your specific use.
If theNum contains multiple values, split into an array:
theName = "";
if(theNum.contains(","){
int[] theNums = theNum.split(",");
for (int num : theNums) {
theName += p.getProperty(num);
theName += ",";
}
theName = theName.replaceAll(",$", ""); //get rid of trailing comma
}
else
theName = p.getProperty(theNum);

Regular Expression..Splitting a string array twice

I have a text file with state-city values:-
These are the contents in my file:-
Madhya Pradesh-Bhopal
Goa-Bicholim
Andhra Pradesh-Guntur
I want to split the state and the city... Here is my code
FileInputStream fis= new FileInputStream("StateCityDetails.txt");
BufferedInputStream bis = new BufferedInputStream(fis);
int h=0;
String s;
String[] str=null;
byte[] b= new byte[1024];
while((h=bis.read(b))!=-1){
s= new String(b,0,h);
str= s.split("-");
}
for(int i=0; i<str.length;i++){
System.out.println(str[1]); ------> the value at 1 is Bhopal Goa
}
}
Also I have a space between Madhya Pradesh..
So i want to Remove spaces between the states in the file and also split the state and city and obtain this result:-
str[0]----> MadhyaPradesh
str[1]----> Bhopal
str[2]-----> Goa
str[3]----->Bicholim
Please Help..Thank you in advance :)

I would use a BufferedReader here, rather than the way you are doing it. The code snippet below reads each line, split on hyphen (-), and removes all whitespace from each part. Each component is entered into a list, in left to right (and top to bottom) order. The list is converted to an array at the end in case you need this.
List<String> names = new ArrayList<String>();
BufferedReader br = null;
try {
String currLine;
br = new BufferedReader(new FileReader("StateCityDetails.txt"));
while ((currLine = br.readLine()) != null) {
String[] parts = currLine.split("-");
for (int i=0; i < parts.length; ++i) {
names.add(parts[i].replaceAll(" ", ""));
}
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (br != null) br.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
// convert the List to an array of String (if you require it)
String[] nameArr = new String[names.size()];
nameArr = names.toArray(nameArr);
// print out result
for (String val : nameArr) {
System.out.println(val);
}

java read only integers from file

If I have a file that contains for example:
results1: 2, 4, 5, 6, 7, 8, 9
results2: 5, 3, 7, 2, 8, 5, 2
I want to add the integers from each line to a array. One array
for each line. How can I do this with code that does only read the integers?
Here's what I got this far
String data = null;
try {
Scanner in = new Scanner(new File(myFile));
while (in.hasNextLine()) {
data = in.nextLine();
numbers.add(data);
}
in.close();
} catch (Exception e) {
}

Easy.
One line per array, not two as you have it. New line after each one.
Read each line as a String, discard the leading "resultsX:", and split what remains at a delimiter of your choosing (e.g. comma). Parse each into an integer and add it to a List.
I don't think that leading "results1: " is adding any value. Why do you have that?

public static void main(String[] args) throws IOException {
BufferedReader reader=null;
try {
reader = new BufferedReader(new FileReader(new File("PATH TO FILE")));
// Only works if File allways contains at least two lines ... all lines after the second
// will be ignored
System.out.println(String.format("Array 1 : %s", Arrays.toString(stringArray2IntArray(readNextLine(reader)))));
System.out.println(String.format("Array 2 : %s", Arrays.toString(stringArray2IntArray(readNextLine(reader)))));
} finally {
if (reader!=null) {
reader.close();
}
}
}
private static Integer[] stringArray2IntArray(String[] numStrings) {
List<Integer> result = new ArrayList<Integer>();
for (int i = 0; i < numStrings.length; i++) {
result.add(Integer.parseInt(numStrings[i].trim()));
}
return result.toArray(new Integer[numStrings.length]);
}
private static String[] readNextLine(BufferedReader reader) throws IOException {
return reader.readLine().split(":")[1].split(",");
}

Assuming you have an input file, like this:
2,4,5,6,7,8,9
5,3,7,2,8,5,2
here is a code snippet to load it:
String firstLine = "";
String secondLine = "";
File file = new File("path/to/file");
try {
BufferedReader br = new BufferedReader(new FileReader(file));
firstLine = br.readLine();
secondLine = br.readLine();
} catch(Exception e){
e.printStackTrace();
}
String[] firstResult = firstLine.split(",");
String[] secondResult = secondLine.split(",");
int[] firstIntegers = new int[firstResult.length];
for(int i = 0; i <= firstResult.length ; i++){
firstIntegers[i] = Integer.parseInt(firstResult[i]);
}
int[] secondIntegers = new int[secondResult.length];
for(int i = 0; i <= secondResult.length ; i++){
firstIntegers[i] = Integer.parseInt(secondResult[i]);
}

Open the file with a BufferedReader br and then read it line by line.
Store each line in an int array and add all those int arrays to a list. At the end, this list will contain all the int arrays that we wanted, so iterate this list to do whatever you want to do next.
String filePath = "/path/to/file";
BufferedReader br = null;
List<Integer[]> arrays = new ArrayList<>(); //this contains all the arrays that you want
try {
br = new BufferedReader(new FileReader(filePath));
String line = null;
while ((line = br.readLine()) != null) {
line = line.substring(line.indexOf(":")+2); //this starts the line from the first number
String[] stringArray = line.split(", ");
Integer[] array = new Integer[stringArray.length];
for (int i = 0; i < stringArray.length; ++i) {
array[i] = Integer.parseInt(stringArray[i]);
}
arrays.add(array);
}
} catch (FileNotFoundException ex) {
System.err.println(ex);
} catch (IOException ex) {
System.err.println(ex);
} finally {
try {
br.close();
} catch (Exception ex) {
System.err.println(ex);
}
}
Since ArrayLists keep insertion order, then, e.g., arrays.get(3) will give you the array of the fourth line (if there is such a line) and arrays.size() will give you the number of lines (i.e., int arrays) that are stored.

JAVA: read .txt file and store doubles into array

I have the following code already, which goes as far as appending the doubles.
Scanner sc1 = new Scanner(System.in);
try {
System.out.println("Enter filename:");
String name = sc1.nextLine();//determines name of file
File file = new File(name);//creates above file
FileReader fileReader = new FileReader(file);//file is read
BufferedReader bufferedReader = new BufferedReader(fileReader );//BufferReader reads file, line by line
StringBuffer stringBuffer = new StringBuffer();//appended to StringBuffer
String line;//reads each
while ((line = bufferedReader.readLine()) != null)
{
stringBuffer.append(line + "\n");
}
fileReader.close();
System.out.println("Contents of file:");
System.out.println(stringBuffer.toString());
}
catch (IOException e)
{
e.printStackTrace();
}
The code reads a file that has a double on each line
ex:
1
2
3
4
5
...
How do I store each of these doubles into an array?

If you want to stick to using Array only, convert StringBuffer to String[] like this:
String [] stringArray = stringBuffer.split("\n");
Then convert string array to double array:
Double [] doubleArray = new Double[stringArray.size];
for(int i=0 : i < stringArray.size : i++){
doubleArray[i] = Doube.parseDouble(stringArray[i]);
}
If you must use StringBuffer, you can divide into array by StringTokenizer class.
Alternatively, you can use String instead of StringBuffer and then split() method will work with de-limiter.

You could use an ArrayList:
ArrayList<Double> doubles = new ArrayList<Double>();
while ((line = bufferedReader.readLine()) != null)
{
doubles.add(Double.parseDouble(line));
}

Basic solution
ArrayList<Double> doubles = new ArrayList<>();
while ((line = bufferedReader.readLine()) != null)
{
stringBuffer.append(line + "\n");
try{
double value = Double.parseDouble(line );
doubles.add(value);
} catch(NullPointerException e){
//null string
} catch(NumberFormatException e){
//no parsable on the current line
}
}

I don't get the problem...Just make an array that stores the contents of the line as the file is read.

Java read large text file with separator

I'm trying to read a large text file in the form of:
datadfqsjmqfqs+dataqfsdqjsdgjheqf+qsdfklmhvqziolkdsfnqsdfmqdsnfqsdf+qsjfqsdfmsqdjkgfqdsfqdfsqdfqdfssdqdsfqdfsqdsfqdfsqdfs+qsfddkmgqjshfdfhsqdflmlkqsdfqdqdf+
I want to read this string in the text file as one big java String. Is this possible? I know the use of the split method.
It worked to read it line by line, but what I really need is to split this long text-string at the '+' sign. Afterwards I want to store it as an array, arraylist, list,...
Can anyone help me with this? Because every information on the internet is just about reading a file line by line.
Thanks in advance!

String inpStr = "datadfqsjmqfqs+dataqfsdqjsdgjheqf+qsdfklmhvqziolkdsfnqsdfmqdsnfqsdf+qsjfqsdfmsqdjkgfqdsfqdfsqdfqdfssdqdsfqdfsqdsfqdfsqdfs+qsfddkmgqjshfdfhsqdflmlkqsdfqdqdf+";
String[] inpStrArr = inpStr.split("+");
Hope this is what you need.

You can read file using BufferedReader or any IO-classes.suppose you have that String in testing.txt file then by reading each line from file you can split it by separator (+). and iterate over array and print.
BufferedReader br = null;
try {
String sCurrentLine;
br = new BufferedReader(new FileReader("C:\\testing.txt"));//file name with path
while ((sCurrentLine = br.readLine()) != null) {
String[] strArr = sCurrentLine.split("\\+");
for(String str:strArr){
System.out.println(str);
}
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (br != null)br.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}

It seems to me like your problem is that you don't want to read the file line by line. So instead, try reading it in parts (say 20 characters each time and building your string):
char[] c = new char[20]; //best to save 20 as a final static somewhere
ArrayList<String> strings = new ArrayList<String>();
StringBuilder sb = new StringBuilder();
BufferedReader br = new BufferedReader(new FileReader(filename));
while (br.read(c) == 20) {
String str = new String(c);
if (str.contains("+") {
String[] parts = str.split("\\+");
sb.append(parts[0]);
strings.add(sb.toString());
//init new StringBuilder:
sb = new StringBuilder();
sb.add(parts[1]);
} else {
sb.append(str);
}
}

You should be able to get a String of length Integer.MAX_VALUE (always 2147483647 (231 - 1) by the Java specification, the maximum size of an array, which the String class uses for internal storage) or half your maximum heap size (since each character is two bytes), whichever is smaller
How many characters can a Java String have?

Try this one:
private static void readLongString(File file){
ArrayList<String> list = new ArrayList<String>();
StringBuilder builder = new StringBuilder();
int r;
try{
InputStream in = new FileInputStream(file);
Reader reader = new InputStreamReader(in);
while ((r = reader.read()) != -1) {
if(r=='+'){
list.add(builder.toString());
builder = new StringBuilder();
}
builder.append(r);
}
}catch (IOException ex){
ex.printStackTrace();
}
for(String a: list){
System.out.println(a);
}
}

Here is one way, caveat being you can't load more than the max int size (roughly one GB)
FileReader fr=null;
try {
File f=new File("your_file_path");
fr=new FileReader(f);
char[] chars=new char[(int)f.length()];
fr.read(chars);
String s=new String(chars);
//parse your string here
} catch (Exception e) {
e.printStackTrace();
}finally {
if(fr!=null){
try {
fr.close();
} catch (IOException e) {
}
}
}