We Keep Coding

Save huge amount of Strings in file during program execution in Java

In my program the main logic is to construct Strings in different methods and then to save them in a specific order in a file. But my memory consumption is very high, so I wonder how to save less strings in memory. So I will try to simplify the program for the ease of reading here. My small String Generating methods do like:
Then my main logic is something like:
BufferedWriter bw = new BufferedWriter(fw);
for(int i=1;i<1_000_000_000;i++){
bw.write(processMethod(i));
}
bw.close();
where the processMethod is a method that calls methods like generateTag m times, makes String using StringBuilder and then saves to BufferedWriter .
public static String generateTag(final String tagName, final String name, final String value) {
StringBuilder attribute = new StringBuilder();
attribute.append("<tag:");
attribute.append(tagName);
attribute.append(" name=\"");
attribute.append(name);
attribute.append("\">");
attribute.append(value);
attribute.append("</tag:");
attribute.append(tagName);
attribute.append(">");
return attribute.toString();
}
So when I start the processMethod executes 1_000_000_000 times and then m times is called generateTag like methods. I have 1_000_000_000 * m strings in the memory. How can I easily remove their creation? I think something like:
public static String generateTag(final String tagName, final String name, final String value, final BufferedWriter bf) {
....
bf.write(someBuilder.toString());
..
}
But passing BufferedWriter around is not a good I think.
Can you suggest me some less String created description.

If indeed your program merely calls methods one after the other and those methods generate strings and those strings are written to the file in the order they are generated, then it's simpler to write directly to the file using the main BufferedWriter:
try (BufferedWriter bw = new BufferedWriter(fw);) {
for(int i=1;i<1_000_000_000;i++){
processMethod(i, bw);
}
}
(Note that I used try-with-resources to automatically close the buffered writer).
And then when processMethod calls generateTag, it passes the buffered writer to it:
public void processMethod(int i, BufferedWriter bw) {
...
generateTag(...,bw);
...
}
And generateTag is going to be:
public static void generateTag(final String tagName, final String name, final String value, final BufferedWriter bw) {
bw.write("<tag:");
bw.write(tagName);
bw.write(" name=\"");
bw.write(name);
bw.write("\">");
bw.write(value);
bw.write("</tag:");
bw.write(tagName);
bw.write(">");
}
Since BufferedWriter is buffered, it means that there is disk access not every time you call write, but every time the buffer is filled up. So this won't cost you in disk access speed. But it will save you all that memory.
Of course, if you don't write the results serially, or the result of one method depends on the result of the other, then you need to modify this, but still, you should write as soon as you have the next piece of String ready.

One thing you may consider is to not saving them in memory, and instead write them as soon as possible...
many XML api like jdom, dom and dom4j abet the harmful habits of building tree model in memory, whereas in reality, it is far more efficient to dump the byte out into output buffer asap...
You could rewrite your method to include an output stream variable and flush out all the bytes out with the output stream.

Use the Appendable abstraction to ensure the generateTag (and likewise processMethod, if you wish) have no dependency to BufferedWriter. In fact, we can even pass a StringBuilder to do the same as before, a feature, which we can use to provide an appropriate overload:
public static CharSequence generateTag(
final String tagName, final String name, final String value) {
StringBuilder attribute=new StringBuilder(80);
try { generateTag(tagName, name, value, attribute); }
catch (IOException ex) { throw new AssertionError(ex); }
return attribute;
}
public static void generateTag(
final String tagName, final String name, final String value, Appendable attribute)
throws IOException {
attribute.append("<tag:");
attribute.append(tagName);
attribute.append(" name=\"");
attribute.append(name);
attribute.append("\">");
attribute.append(value);
attribute.append("</tag:");
attribute.append(tagName);
attribute.append(">");
}
Note that the overload returns CharSequence rather than String to be able to omit the final StringBuilder.toString() copying step. Unfortunately, calling append(CharSequence)on the BufferedWriter with it would not pay off, given the current implementation. Still there are APIs allowing to use CharSequence directly.

Deleteing a substring buffer is throwing error?

Today is my first day in programming world i am trying to learn java so please excuse me for these sort of question
Deleting a stringbuffer object is throwing error?please help me to fix it my code is below
class Demo
{
public static void main(String[] args)
{
StringBuffer sb=new StringBuffer(); //default 16
sb.append("abc");
sb.append("xyz");
sb.append("123");
System.out.println("before delete():"+sb);
sb.delete(3,6);
System.out.println("after delete():"+sb);
sb.delete(99); //here error please help?
System.out.println("after delete():"+sb);
}
}

There is no method delete(int) defined in StringBuffer class.
It has to be delete(int, int). The other method which is offered by StringBuffer class is deleteCharAt(int)
Also, you string buffer object doesn't have anything at 99th index. Even if you use deleteCharAt(int) it is going to throw StringIndexOutOfBoundsException

I think there's no delete method with a single argument. You're calling a non-existing method.
StringBuffer

Thread Safe String Split

I have ran a split test with StringTokenizer and String's method indexOf() followed by substring(). After result I have to knew, the second one is faster than StringTokenizer. But my question is which is best for Thread safety?
String split using StringTokenizer:
String sample=null;
StringTokenizer tokens = new StringTokenizer("FF#Mukil","#");
if(tokens.hasMoreTokens())
{
sample= tokens.nextToken();
System.out.println(sample);
}
String split using String's method:
String sample= "FF#Mukil";
int pos=sample.indexOf('#');
System.out.println(sample.substring(0,pos));
In these examples, sample is global variable
Thanks in advance
Edited:
In Previous first example is like below
StringTokenizer tokens = new StringTokenizer("FF#Mukil","#");
if(tokens.hasMoreTokens())
{
String sample= tokens.nextToken();
System.out.println(sample);
}
I have changed it like below
String sample=null;
StringTokenizer tokens = new StringTokenizer("FF#Mukil","#");
if(tokens.hasMoreTokens())
{
sample= tokens.nextToken();
System.out.println(sample);
}
Edited: Added java file source.
//FileName=MySplitUtil.java
import javax.servlet.http.HttpServletRequest;
import java.util.StringTokenizer;
public class MySplitUtil
{
public static void setUsername_locale(HttpServletRequest request)
{
String value = (String) request.getAttribute("userLocale"); // userLocale's sample value looks like ffmukil$en_US
splite1(value,request);
splite2(value,request);
}
private static void splite1(String value,HttpServletRequest request)
{
StringTokenizer tokens = new StringTokenizer(value,"$");
String username=null;
String locale=null;
if(tokens.hasMoreTokens())
{
username = tokens.nextToken();
locale = tokens.nextToken();
}
request.setAttribute("usrName",username);
request.setAttribute("usr_Locale",locale);
}
private static void splite2(String value,HttpServletRequest request)
{
int pos = value.indexOf('$');
String username = value.substring(0,pos);
String locale = value.substring(pos+1,value.length());
request.setAttribute("usrName",username);
request.setAttribute("usr_Locale",locale);
}
}

Since you only use local variables, both versions are perfectly safe, whatever you do with them...
To run into thread safety issues, you need to (improperly) share something across threads. Nothing can be shared in your example.
More generally, Strings are immutable and therefore thread safe. So the second version, assuming that the sample variable is shared, would be thread safe.
EDIT
Your latest edit confirms 2 things:
your class is stateless (no instance variables)
you only use local variables
So both methods are thread safe and you should choose the simpler and/or more efficient one.

In java, Strings are immutable, nothing will affect the original string.
That immutability makes the Strings thread safe. It can be accesses by several StringTokenizer or string.substring at the same time without any problems.
However, the problem could come from the class StringTokenizer that you should not use on several strings at the same time. But, as pointed out by others, in your code you are only using local variables, so you won't have thread safety issues.

According to the Javadoc: "A StringTokenizer object internally maintains a current position within the string to be tokenized. Some operations advance this current position past the characters processed."
So it's not threadsafe. Why would you want to share it anyway? Just make it a local variable.

replace a string segment from input stream

I am trying to receive a huge text file as an inputstream and want to convert a string segment with another string. I am strictly confused how to do it, it works well if I convert whole inputstream as a string which I don't want as some of the contents are lost. can anyone please help how to do it??
e.g.
if I have a file which has the contents "This is the test string which needs to be modified". I want to accept this string as input stream and want to modify the contents to "This is the test string which is modified" , ( by replacing 'needs to be' with is).
public static void main(String[] args) {
String string = "This is the test string which needs to be modified";
InputStream inpstr = new ByteArrayInputStream(string.getBytes());
//Code to do
}
In this I want the output as: This is the test string which is modified
Thanking you in advance.

If the text to be changed will always fit in one logical line, as I stated in comment, I'd go with simple Line Reading (if applyable) using something like:
public class InputReader {
public static void main(String[] args) throws IOException
{
String string = "This is the test string which needs to be modified";
InputStream inpstr = new ByteArrayInputStream(string.getBytes());
BufferedReader rdr = new BufferedReader(new InputStreamReader(inpstr));
String buf = null;
while ((buf = rdr.readLine()) != null) {
// Apply regex on buf
// build output
}
}
}
However I've always like to use inheritance so I'd define this somewhere:
class MyReader extends BufferedReader {
public MyReader(Reader in)
{
super(in);
}
#Override
public String readLine() throws IOException {
String lBuf = super.readLine();
// Perform matching & subst on read string
return lBuf;
}
}
And use MyReader in place of standard BufferedReader keeping the substitution hidden inside the readLine method.
Pros: substitution logic is in a specified Reader, code is pretty standard.
Cons: it hides the substitution logic to the caller (sometimes this is also a pro, still it depends on usage case)
HTH

May be I understood you wrong, but I think you should build a stack machine. I mean you can use a small string stack to collect text and check condition of replacement.
If just collected stack already is not matched to your condition, just flush stack to output and collect it again.
If your stack is similar with condition, carry on collecting it.
If your stack is matched your condition, make a modification and flush modified stack to output.

Select object dynamically

Here's the situation :
I have 3 objects all named **List and I have a method with a String parameter;
gameList = new StringBuffer();
appsList = new StringBuffer();
movieList = new StringBuffer();
public void fetchData(String category) {
URL url = null;
BufferedReader input;
gameList.delete(0, gameList.length());
Is there a way to do something like the following :
public void fetchData(String category) {
URL url = null;
BufferedReader input;
"category"List.delete(0, gameList.length());
, so I can choose which of the lists to be used based on the String parameter?

I suggest you create a HashMap<String, StringBuffer> and use that:
map = new HashMap<String, StringBuffer>();
map.put("game", new StringBuffer());
map.put("apps", new StringBuffer());
map.put("movie", new StringBuffer());
...
public void fetchData(String category) {
StringBuffer buffer = map.get(category);
if (buffer == null) {
// No such category. Throw an exception?
} else {
// Do whatever you need to
}
}

If the lists are fields of your object - yes, using reflection:
Field field = getClass().getDeclaredField(category + "List");
List result = field.get();
But generally you should avoid reflection. And if your objects are fixed - i.e. they don't change, simply use an if-clause.

The logically simplest way taking your question as given would just be:
StringBuffer which;
if (category.equals("game"))
which=gameList;
else if (category.equals("apps"))
which=appList;
else if (category.equals("movie"))
which=movieList;
else
... some kind of error handling ...
which.delete();
As Jon Skeet noted, if the list is big or dynamic you probably want to use a map rather than an if/else/if.
That said, I'd encourage you to use integer constant or an enum rather than a String. Like:
enum ListType {GAME, APP, MOVIE};
void deleteList(ListType category)
{
if (category==GAME)
... etc ...
In this simple example, if this is all you'd ever do with it, it wouldn't matter much. But I'm working on a system now that uses String tokens for this sort of thing all over the place, and it creates a lot of problems.
Suppose you call the function and by mistake you pass in "app" instead of "apps", or "Game" instead of "game". Or maybe you're thinking you added handling for "song" yesterday but in fact you went to lunch instead. This will successfully compile, and you won't have any clue that there's a problem until run-time. If the program does not throw an error on an invalid value but instead takes some default action, you could have a bug that's difficult to track down. But with an enum, if you mis-spell the name or try to use one that isn't defined, the compiler will immediately alert you to the error.
Suppose that some functions take special action for some of these options but not others. Like you find yourself writing
if (category.equals("app"))
getSpaceRequirements();
and that sort of thing. Then someone reading the program sees a reference to "app" here, a reference to "game" 20 lines later, etc. It could be difficult to determine what all the possible values are. Any given function might not explicitly reference them all. But with an enum, they're all neatly in one place.

You could use a switch statement
StringBuffer buffer = null;
switch (category) {
case "game": buffer = gameList;
case "apps": buffer = appsList;
case "movie": buffer = movieList;
default: return;
}