Load commented properties to Properties object in java - java

I'm trying to load properties to a Properties object in java using load(new FileReader()) method. All the properties are loaded except the properties start with(#) commented ones. How to load these commented properties to the Properties object using java API. Only manual way?
Thanks in Advance.


I could propose you to extend the java.util.Properties class to override this specificities but it was not designed for it : many things are hardcoded and not overridable. So you should do entire copy-paste of methods with little modification.
For example, at a time, the LineReader used in internal does that when you load a properties file :
if (isNewLine) {
isNewLine = false;
if (c == '#' || c == '!') {
isCommentLine = true;
continue;
}
}
The # is hardcoded.
Edit
Another way could be read line by line the proprties file, remove the first char if it is # and write the read line, modified if needed, in a ByteArrayOutputStream. then you could load the properties with a ByteArrayInputStream from ByteArrayOutputStream.toByteArray().
Here a possible implementation with a unit test :
With as input myProp.properties :
dog=woof
#cat=meow
The unit test :
#Test
public void loadAllPropsIncludingCommented() throws Exception {
// check properties commented not retrieved
Properties properties = new Properties();
properties.load(LoadCommentedProp.class.getResourceAsStream("/myProp.properties"));
Assert.assertEquals("woof", properties.get("dog"));
Assert.assertNull(properties.get("cat"));
// action
BufferedReader bufferedIs = new BufferedReader(new FileReader(LoadCommentedProp.class.getResource("/myProp.properties").getFile()));
ByteArrayOutputStream out = new ByteArrayOutputStream();
String currentLine = null;
while ((currentLine = bufferedIs.readLine()) != null) {
currentLine = currentLine.replaceFirst("^(#)+", "");
out.write((currentLine + "\n").getBytes());
}
bufferedIs.close();
out.close();
// assertion
ByteArrayInputStream in = new ByteArrayInputStream(out.toByteArray());
properties = new Properties();
properties.load(in);
Assert.assertEquals("woof", properties.get("dog"));
Assert.assertEquals("meow", properties.get("cat"));
}

Related

How to store specific Key-Value Pairs from properties file

I want to store the key values pair in java from the config.properties file. Problem is it has some other which in dont want to store in array or hashmap.Below is my config.properties file. One thing the line must start with #usergroup and end of line should be End_TT_Executive as described in the file
#Usergroup
TT_Executive
#Tilename
KPI
#No of Submenu=3
#Submenu_1
OPs_KPI=https://tntanalytics3.sl1430087.sl.dst.ibm.com:8443/CAP-T/res/html/underprogress.html
#Submenu_2
Ontime_OnBudget=https://tntanalytics3.sl1430087.sl.dst.ibm.com:8443/CAP-T/res/html/underprogress.html
#submenu_3
Ops_KPI_Cloud=https://tntanalytics3.sl1430087.sl.dst.ibm.com:8443/CAP-T/res/html/underprogress.html
#Tilename
Alerting Dashboard
#No of submenu=0
Alerting_Dashboard=https://tntanalytics3.sl1430087.sl.dst.ibm.com:8443/CAP-T/res/html/underprogress.html
#Tilename
FTE_Dashboard
#No of submenu=3
#Submenu_1
FTE_Market_Sector_TT_Executive= https://tntanalytics3.sl1430087.sl.dst.ibm.com:8443/CAP-T/res/html/underprogress.html
#submenu_2
FTE_Account_TT_Executive= http://tntanalytics1.sl1430087.sl.dst.ibm.com/ibmcognos/bi/?pathRef=.public_folders%2FP=false
#Submenu_3
FTE_Laborpool_TT_Executive= https://tntanalytics3.sl1430087.sl.dst.ibm.com:8443/CAP-T/res/html/underprogress.html
#Tilename
PCR
#No of Submenu=0
PCR=https://tntanalytics3.sl1430087.sl.dst.ibm.com:8443/CAP-T/res/html/underprogress.html
End_TT_Executive
How can I do this? The key value pair are with URL only rest is some title for understanding.

Suppose your config.properties is like:
p1=abc
p2=def
p3=zxc
p4=eva
and you want to load p1 and p2 to map.
You can load all properties into Properties instance with:
InputStream inputStream = null;
try
{
inputStream = new BufferedInputStream(new FileInputStream("config.properties"));
Properties properties = new Properties();
properties.load(new InputStreamReader(inputStream, "UTF-8")); // load all properties in config.properties file
}
catch ( IOException e )
{
e.printStackTrace();
}
finally
{
inputStream.close();
}
Then you create a map:
Map<String, String> propertyMap = new HashMap<>();
You also need a String[] to store all properties which you want to load to propertyMap.
String[] wantedProperties = new String[]{"p1", "p2"};
Then you write a for loop to load the properties you wanted:
for (String property : wantedProperties) {
propertyMap.put(property, properties.getProperty(property));
}
Now propertyMap is what you want.
If you want to store to List:
List<String> propertyList = new ArrayList<>();
for (String property : wantedProperties) {
propertyList.add(properties.getProperty(property));
}
This is the way to save to list. It'll help you more if you find the solution yourself.

updating the value read using value annotation in spring

I have a properties file say as follows:
apple=1
mango=2
banana=3
pineapple=4
I am using value annotation in the java program to access the values. I have a method in my class that computes a value i want to update the apple attribute in the properties file with the value that the method returns.
public class test {
#Value("${apple}")
private int apple;
public void testMethod() {
int new_val = 0;
if (apple > 0)
new_val = 300;
else
new_val = 200;
// now i want to update the value of apple in the file to new_val,(apple = new_val) other attributes should remain unchanged.
}
}
can someone let me know how to update the value in the properties file. In this example i want my properties file to become
apple=300
mango=2
banana=3
pineapple=4

Usually we defines constant values in properties, so it does not change.
But if it is your requirement to change it.
You can do it like:
1) Using Apache Commons Configuration library
PropertiesConfiguration conf = new PropertiesConfiguration("yourproperty.properties");
props.setProperty("apple", "300");
conf.save();
2) Using Java input and output stream
FileInputStream in = new FileInputStream("yourproperty.properties");
Properties props = new Properties();
props.load(in);
in.close();
FileOutputStream out = new FileOutputStream("yourproperty.properties");
props.setProperty("apple", "300");
props.store(out, null);
out.close();

How to read files with an offset from Hadoop using Java

Problem: I want to read a section of a file from HDFS and return it, such as lines 101-120 from a file of 1000 lines.
I don't want to use seek because I have read that it is expensive.
I have log files which I am using PIG to process down into meaningful sets of data. I've been writing an API to return the data for consumption and display by a front end. Those processed data sets can be large enough that I don't want to read the entire file out of Hadoop in one slurp to save wire time and bandwidth. (Let's say 5 - 10MB)
Currently I am using a BufferedReader to return small summary files which is working fine
ArrayList lines = new ArrayList();
...
for (FileStatus item: items) {
// ignoring files like _SUCCESS
if(item.getPath().getName().startsWith("_")) {
continue;
}
in = fs.open(item.getPath());
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String line;
line = br.readLine();
while (line != null) {
line = line.replaceAll("(\\r|\\n)", "");
lines.add(line.split("\t"));
line = br.readLine();
}
}
I've poked around the interwebs quite a bit as well as Stack but haven't found exactly what I need.
Perhaps this is completely the wrong way to go about doing it and I need a completely separate set of code and different functions to manage this. Open to any suggestions.
Thanks!
As added noted based on research from the below discussions:
How does Hadoop process records records split across block boundaries?
Hadoop FileSplit Reading

I think SEEK is a best option for reading files with huge volumes. It did not cause any problems to me as the volume of data that i was reading was in the range of 2 - 3GB. I did not encounter any issues till today but we did use file splitting to handle the large data set. below is the code which you can use for reading purpose and test the same.
public class HDFSClientTesting {
/**
* #param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stub
try{
//System.loadLibrary("libhadoop.so");
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
conf.addResource(new Path("core-site.xml"));
String Filename = "/dir/00000027";
long ByteOffset = 3185041;
SequenceFile.Reader rdr = new SequenceFile.Reader(fs, new Path(Filename), conf);
Text key = new Text();
Text value = new Text();
rdr.seek(ByteOffset);
rdr.next(key,value);
//Plain text
JSONObject jso = new JSONObject(value.toString());
String content = jso.getString("body");
System.out.println("\n\n\n" + content + "\n\n\n");
File file =new File("test.gz");
file.createNewFile();
}
catch (Exception e ){
throw new RuntimeException(e);
}
finally{
}
}
}

Decoding name of file attachment doesnt work with properties

I read this documentation:
http://docs.oracle.com/javaee/6/api/javax/mail/internet/package-summary.html
so I add some properties to mimeMessage:
Properties props = new Properties();
props.put("mail.mime.decodefilename", true);
Session mailConnection = Session.getInstance(props, null);
source = new FileInputStream(emlFile);
MimeMessage message = new MimeMessage(mailConnection, source);
now I am expectaing that method bodyPart.getFileName() return correct name of file. But with this configuration it still doesn work and I need to call mimeUtils: MimeUtility.decodeText - what I dont want. I also try:
props.put("mail.mime.decodefilename", "true");
but with no success. So what I am doing wrong ?
UPDATE:
after debuging I had this sollution:
this works
Properties props = System.getProperties();
props.put("mail.mime.decodefilename", "true");
this doesnt work:
Properties props = new Properites();
props.put("mail.mime.decodefilename", "true");
so if filename is decoding depends on system property too. Does anyone know which properties ? I dont have pattion to try all system properties and solve which one it is

MimeMessage.getFileName
If the mail.mime.encodefilename System property is set to true, the
MimeUtility.decodeText method will be used to decode the filename.
Now when one looks at the implementation, this is how the MimeUtility.decodeText comes into picture during the invocation of getFileName:
if (decodeFileName && filename != null) {
try {
filename = MimeUtility.decodeText(filename);
} catch (UnsupportedEncodingException ex) {
throw new MessagingException("Can't decode filename", ex);
}
}
Where decodeFileName is initialized like this:
s = System.getProperty("mail.mime.decodefilename");
// default to false
decodeFileName = s != null && !s.equalsIgnoreCase("false");
The javadoc seems to be conflicting with the implementation.
So, try setting mail.mime.decodefilename instead of mail.mime.encodefilename, probably using System.setProperty.

Java: How to I change the configuration file value in Java easily?

I have a config file, named config.txt, look like this.
IP=192.168.1.145
PORT=10022
URL=http://www.stackoverflow.com
I wanna change some value of the config file in Java, say the port to 10045. How can I achieve easily?
IP=192.168.1.145
PORT=10045
URL=http://www.stackoverflow.com
In my trial, i need to write lots of code to read every line, to find the PORT, delete the original 10022, and then rewrite 10045. my code is dummy and hard to read. Is there any convenient way in java?
Thanks a lot !

If you want something short you can use this.
public static void changeProperty(String filename, String key, String value) throws IOException {
Properties prop =new Properties();
prop.load(new FileInputStream(filename));
prop.setProperty(key, value);
prop.store(new FileOutputStream(filename),null);
}
Unfortunately it doesn't preserve the order or fields or any comments.
If you want to preserve order, reading a line at a time isn't so bad.
This untested code would keep comments, blank lines and order. It won't handle multi-line values.
public static void changeProperty(String filename, String key, String value) throws IOException {
final File tmpFile = new File(filename + ".tmp");
final File file = new File(filename);
PrintWriter pw = new PrintWriter(tmpFile);
BufferedReader br = new BufferedReader(new FileReader(file));
boolean found = false;
final String toAdd = key + '=' + value;
for (String line; (line = br.readLine()) != null; ) {
if (line.startsWith(key + '=')) {
line = toAdd;
found = true;
}
pw.println(line);
}
if (!found)
pw.println(toAdd);
br.close();
pw.close();
tmpFile.renameTo(file);
}

My suggestion would be to read the entire config file into memory (maybe into a list of (attribute:value) pair objects), do whatever processing you need to do (and consequently make any changes), then overwrite the original file with all the changes you have made.
For example, you could read the config file you have provided by line, use String.split("=") to separate the attribute:value pairs - making sure to name each pair read accordingly. Then make whatever changes you need, iterate over the pairs you have read in (and possibly modified), writing them back out to the file.
Of course, this approach would work best if you had a relatively small number of lines in your config file, that you can definitely know the format for.

this code work for me.
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.Properties;
public void setProperties( String key, String value) throws IOException {
Properties prop = new Properties();
FileInputStream ip;
try {
ip = new FileInputStream("config.txt");
prop.load(ip);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
prop.setProperty(key, value);
PrintWriter pw = new PrintWriter("config.txt");
prop.store(pw, null);
}

Use the Properties class to load/save configuration. Then simply set the value and save it again.
Properties p = new Properties();
p.load(...);
p.put("key", "value");
p.save(...)
It's easy and straightforward.
As a side, if your application is a single application that does not need to scale to run on multiple computers, do not bother to use a database to save config. It is utter overkill. However, if you application needs real time config changes and needs to scale, Redis works pretty well to distribute config and handle the synchronization for you. I have used it for this purpose with great success.

Consider using java.util.Properties and it's load() and store() methods.
But remember that this would not preserve comments and extra line breaks in the file.
Also certain chars need to be escaped.

If you are open to use third party libraries, explore http://commons.apache.org/configuration/. It supports configurations in multiple format. Comments will be preserved as well. (Except for a minor bug -- apache-commons-config PropertiesConfiguration: comments after last property is lost)

Categories