We Keep Coding

Common configuration 2 - commented properties

i'm developing a java desktop application to use as a tool to manipulate properties files.
So my application need to to load a TXT file containing several keys (or properties), and be able to changes there values.
I use Apache commons configurations 2 project to do this work.
Everything was going well until a face this situation..
there is a txt file contating several properties but, one properties (or key) has a # before his name (look at config.txt file below), so apache commons do not recognize this propertie.
When i set a value to this propertie (look at last two lines on main method), apache commons configurator creates a new line on config.txt
This is not good to me, because i need to preserve the original config.txt format, so if i need to set a value to fileDb propertie, my expected behavior is to remove the # from fileDb name and do not create a new fileDb propertie on config.txt.
the program that uses this configuration file (config.txt) understands that if the property is commented (if it has "#" in its name), it means that it should not be used, but if it is uncommented (without "#" in the name) means that it should be used and that its value should be read.
So what can i do to add or remove "#" from the propertie name using common configuration? if is not possible to do it with commons configurator, how to do it?
On the ConfigTest class i show how the problem ocours.
Before run the code, the file config.txt has a commented propertie named fileDB
#fileDb = C:/test/test.db
the maind method try to change the filedb value to "C://aa//bb//cc"
but after the executions end wil appears a new propetie with the same name "fileDb" but uncomented (without "#"). At the end, config.txt wil have two fileDb propertie, one with "#" and other without "#"
#fileDb = C:/test/test.db
fileDb = C://aa//bb//cc
My expectd behavior after the code execution is to has only one fileDb propetie
fileDb = C://aa//bb//cc
After be able to uncoment this propertie i expect to be able to comment again if necessery (if user wish to do it on my app).
ConfigTest class
import org.apache.commons.configuration2.Configuration;
import org.apache.commons.configuration2.FileBasedConfiguration;
import org.apache.commons.configuration2.PropertiesConfiguration;
import org.apache.commons.configuration2.builder.FileBasedConfigurationBuilder;
import org.apache.commons.configuration2.builder.fluent.FileBasedBuilderParameters;
import org.apache.commons.configuration2.builder.fluent.Parameters;
import org.apache.commons.configuration2.ex.ConfigurationException;
import java.io.File;
public class ConfigTest {
private Configuration localConfiguration;
private FileBasedConfigurationBuilder<FileBasedConfiguration> fileBuilder;
private ConfigTest() throws ConfigurationException {
this.loadPropertiesFile();
}
private void loadPropertiesFile() throws ConfigurationException {
File file = new File("config.txt");
FileBasedBuilderParameters fb = new Parameters().fileBased();
fb.setFile(file);
fileBuilder = new FileBasedConfigurationBuilder<FileBasedConfiguration>(PropertiesConfiguration.class);
fileBuilder.configure(fb);
localConfiguration = fileBuilder.getConfiguration();
}
public boolean saveModifications(){
try {
fileBuilder.save();
return true;
} catch (ConfigurationException e) {
e.printStackTrace();
return false;
}
}
public void setProperty(String key, String value){
localConfiguration.setProperty(key,value);
}
public static void main(String[] args) throws ConfigurationException {
ConfigTest test = new ConfigTest();
//change outputOnSSD propertie value to false
test.setProperty("outputOnSSD", "false");
//here when i set a new value to fileDb propertie, a new line on config.txt with a new fileDb propertie name is created
//the question is: how to remove # from fileDb propetie name on config.txt to commons config recognize it as a existent propertie?
test.setProperty("fileDb", "C://aa//bb//cc");
test.saveModifications();
}
}
File "config.txt" used by ConfigTest class to change his value
########################################################################
# Local environment configuration
########################################################################
# Defines program localization/language
locale = pt-BR
# Temporary directory for processing: "default" uses the system temporary folder.
indexTemp = /home/downloads
# Enable if is on a SSD disk.
indexTempOnSSD = true
# Enable if output/case folder is on SSD. If enabled, index is created directly in case folder,
outputOnSSD = false
# Number of processing threads/workers: "default" uses the number of CPU logical cores.
numThreads = 14
# Full path for hash index database.
#fileDb = C:/test/test.db

Modifying of the Mwe2 workflow generated by Xtext programmatically

I'm running Mwe2 workflow programmatically and I need to add referencedResource for the Terminals.xtext to the GenerateYourDsl.mwe2 file in order to run this workflow successfully. Mentioned GenerateYourDsl.mwe2 file is generated by Xtext when I create new project - I didn't modify it.
Is it somehow possible to get object out of this workflow, access it in Java and add this one attribute to the language attribute of the component attribute of the workflow ?
Here's the workflow:
Workflow {
component = XtextGenerator {
configuration = {
project = StandardProjectConfig {
baseName = "org.example.yourdsl"
rootPath = rootPath
eclipsePlugin = {
enabled = true
}
createEclipseMetaData = true
}
code = {
encoding = "UTF-8"
lineDelimiter = "\n"
fileHeader = "/*\n * generated by Xtext \${version}\n */"
}
}
language = StandardLanguage {
name = "org.example.yourdsl.YourDsl"
fileExtensions = "yourdsl"
//following line needs to be added - in original generated workflow it is not
referencedResource = "platform:/resource/org.eclipse.xtext/org/eclipse/xtext/common/Terminals.xtext"
serializer = {
generateStub = false
}
validator = {
// composedCheck = "org.eclipse.xtext.validation.NamesAreUniqueValidator"
// Generates checks for #Deprecated grammar annotations, an IssueProvider and a corresponding PropertyPage
generateDeprecationValidation = true
}
}
}
}
I'm generating projects programmatically (using CliProjectsCreator and WizardConfiguration) and need to run the workflow when the projects are generated in order to generate src, src-gen files etc. But as I'm trying to run it programmatically (not in Eclipse) and Terminals are used in my grammar, I'm getting this error:
XtextLinkingDiagnostic: null:1 Couldn't resolve reference to Grammar 'org.eclipse.xtext.common.Terminals'.
TransformationDiagnostic: null:36 Cannot create datatype WS. If this is supposed to return EString, make sure you have imported 'http://www.eclipse.org/emf/2002/Ecore' (ErrorCode: NoSuchTypeAvailable)
TransformationDiagnostic: null:39 Cannot create datatype NAME_TERMINAL. If this is supposed to return EString, make sure you have imported 'http://www.eclipse.org/emf/2002/Ecore' (ErrorCode: NoSuchTypeAvailable)
TransformationDiagnostic: null:42 Cannot create datatype VALUE_TERMINAL. If this is supposed to return EString, make sure you have imported 'http://www.eclipse.org/emf/2002/Ecore' (ErrorCode: NoSuchTypeAvailable)

there should be no need to add the terminals grammar explicitely. It should be found on the classpath via org.eclipse.xtext.resource.ClassloaderClasspathUriResolver.resolve(Object, URI) if the workflow is called with a proper classpath.
have a look what the wizard creates when selecting maven: a call to the maven exec plugin calling into Mwe2Launcher main with the proper classpath

Can we get properties using Expression from Properties File

Can we get a property from properties file using some expression. e.g
If I have properties in file like this
user/a/b=active
user/a/c=active
user/a/d=active
Now How can I get all properties which are active. Also Can I get all active using user/a/* or something like that

The java.util.Properties class has stringPropertyNames() method
you can use the method to iterate all the names and check name and value
Properties prop = new Properties();
// add some properties
prop.put("user/a/b", "active");
prop.put("user/a/c", "active");
prop.put("user/a/d", "active");
// save the Property names in the set
Set<String> set = prop.stringPropertyNames();
for (String name: set) {
if (name.startsWIth("user/a/")) {
//check value and do something
}
}

How to pass a file as parameter in mapreduce

I want to search for particular words in a file and display its count. When the word to be searched is a single word, I am able to do it by setting the configuration in the driver like below :
Driver class :
Configuration conf = new Configuration();
conf.set("wordtosearch", "fun");
Mapper class :
public static class SearchMapper extends
Mapper<LongWritable, Text, Text, IntWritable> {
// Map code goes here.
private final static IntWritable one = new IntWritable(1);
private Text word = new Text();
public void map (LongWritable Key, Text value,Context context )throws IOException,InterruptedException{
Configuration conf = context.getConfiguration();
//retrieve the wordToSearch variable
String wordToSearch = conf.get("wordtosearch");
String txt= value.toString();
if(txt.compareTo(wordToSearch)==0){
word = context.getCurrentValue();
context.getCurrentKey();
word.set(txt);
context.write(word, one);
}
But when there is a list of words in a file, I dont know how to pass it. Some posts refers to use distributed cache but while doing that I am getting "distributed cache is deprecated" error. Are there any similar methods in the new api to pass the file ?

Yes, there is also a way in the new API.
First, store the file in HDFS. Then, in the Driver class (in the main method), do the following:
Configuration conf = getConf();
...
Job job = Job.getInstance(conf); ...
job.addCacheFile(new Path(filename).toUri());
Finally, in the mapper class (for instance in the setup() method), do the following:
URI[] localPaths = context.getCacheFiles();
If you have a single file, it should be stored in localPaths[0].

You can try this:judge the parameter wether is a file, then according to the type of parameter execute the operation respectively

If the list of words has a reasonable size, you can still pass it to the configuration:
Driver class: read the file
Driver class: add the list of words in the configuration, doing for instance conf.set("wordListToSearch", "fun:foo:bar"
Mapper class: read the configuration and retrieve your list of words

Java properties containing dollar delimited variables

I'm storing my app settings in properties file that I use in Ant and in the Java app. Maybe it's not good pratice, but I find it very handy to avoid duplications. The file contains variables such as:
usefulstuff.dir = ${user.home}/usefulstuff
So that other people can run the program on *nix systems, provided that they have the usefulstuff folder in their home directory.
Now, the fascinating thing is that this properties file works fine in Ant (the variable gets resolved to /home/username), while when I load the same file directly in the Java app, I get a string containing ${user.home}/usefulstuff, which is not very useful indeed.
I load the props with this code in Ant:
<loadproperties srcFile="myProps.properties"/>
And in the Java app:
FileInputStream ins = new FileInputStream(propFilePath);
myProps.load(ins);
ins.close();
Am I missing anything? Maybe is there a better way to load properties in a Java app than load()?

I don't think it's particularly "fascinating" that this works in Ant - Ant is deliberately written to do so:
Properties are key-value-pairs where Apache Ant tries to expand ${key} to value at runtime.
and
Ant provides access to all system properties as if they had been defined using a <property> task. For example, ${os.name} expands to the name of the operating system.
If you want the same behaviour, you'll need to implement the same sort of logic. It's possible that you could use the classes from Ant directly, if they do what you want - and if you're happy to ship the relevant binaries (and abide by the licence).
Otherwise, you might want to use a regular expression to find all the matches - or (probably simpler) iterate over all of the system properties and do a simple replacement on them.

As Jon said, it should be straighforward to write the property handling yourself. For eg:
import java.util.*;
public class PropertiesTest
{
public static void main(String[] args)
{
Properties props = new Properties();
props.setProperty("foo", "foo/${os.name}/baz/${os.version}");
props.setProperty("bar", "bar/${user.country}/baz/${user.country}");
System.out.println("BEFORE:");
printProperties(props);
resolveSystemProperties(props);
System.out.println("\n\nAFTER:");
printProperties(props);
}
static void resolveSystemProperties(Properties props)
{
Map<String, String> sysProps = readSystemProperties();
Set<String> sysPropRefs = sysProps.keySet();
Enumeration names = props.propertyNames();
while (names.hasMoreElements())
{
String name = (String) names.nextElement();
String value = props.getProperty(name);
for (String ref : sysPropRefs)
{
if (value.contains(ref))
{
value = value.replace(ref, sysProps.get(ref));
}
}
props.setProperty(name, value);
}
}
static Map<String, String> readSystemProperties()
{
Properties props = System.getProperties();
Map<String, String> propsMap =
new HashMap<String, String>(props.size());
Enumeration names = props.propertyNames();
while (names.hasMoreElements())
{
String name = (String) names.nextElement();
propsMap.put("${" + name + "}", props.getProperty(name));
}
return propsMap;
}
static void printProperties(Properties props)
{
Enumeration names = props.propertyNames();
while (names.hasMoreElements())
{
String name = (String) names.nextElement();
String value = props.getProperty(name);
System.out.println(name + " => " + value);
}
}
}