Does spring batch provide any dynamic/generic file writers? For example, if i have multiple requirements of generating a file and i have one view created for each purpose, all i want to do is specify the view name and i want spring-batch to extract the data from the view to a flat file with column headings. This is as simple as if you have used dbviz or sql developer, just export the result of a query to a file.
Recently i had 4 different requirements of extracting data to file, and i have repeated the config file and created a record bean and record mapper to map the bean to the columns of view for each file. Rather than repeating this entire process every time, i am looking to see if Spring batch or any other java frameworks provides a generic approach to extracting a file based on the table and its columns without writing result set mappers or dealing with field extractors.
I can build a generic spring batch file extractor but wanted to check if spring-batch already does that, which seems like a basic thing?
Also if i create a generic extractor then the attributes of the bean will be kinda dynamic based on the columns names of the view. So in that scenario, if i have about 50 columns i dont want to specify each attribute in fieldExtractor, i want all the attributes to be extracted. Currently i am specifying attributes in spring config as given below, but i don't want to spefiy the attribute names. I just want to say extract all attributes. Is that possible?
<bean id="CsvItemWriter" class="some.class.FileWriter" scope="step">
<property name="resource" value="file://#{jobParameters['file.name']}"/>
<property name="shouldDeleteIfExists" value="true" />
<property name="lineAggregator">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<property name="delimiter">
<util:constant static-field="org.springframework.batch.item.file.transform.DelimitedLineTokenizer.DELIMITER_TAB" />
</property>
<property name="fieldExtractor"> <bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
<property name="names" value="name.last, name.first, name.middle, birthDate, gender, homePhone, cellPhone"/>
</bean>
</property>
</bean>
</property>
In case some one needs it, i found a similar question in SO that was difficult to find. I am using columnMapRowMapper as mentioned here
Related
I have built a small app that reads an Excel file and creates all the necessary SQL statements for me. Excel file is manipulated by customers from time to time.
The first line of the Excel file includes a title I need to translate into integers when reading the lines. For example, the title "english" needs to be translated into "30" before I generate the SQL statements. (Just some internal definitions). (You could compare it with DNS - Human readable domain name into IP-Address mapping)
Currently I do the mapping manually, but I want to get rid of this step with a tiny Spring configuration. Because the headlines are not fixed, it is necessary to read the headline information from a properties file.
I found something like this:
<bean class="java.util.HashMap" id="exampleMapping">
<constructor-arg index="0">
<map>
<entry key="theKey" value="theValue"/>
<entry key="otherKey" value="otherValue"/>
</map>
</constructor-arg>
</bean>
Which seems to work quite well. But the Spring configuration is compiled into a jar. So my first choice (if possible) would be to externalize the key value pairs into a properties file.
You're in luck, because the Properties class implements Map!
Just define the properties bean like so:
<util:properties id="myProps" location="classpath:myProps.properties" />
(Don't forget to import the Spring util namespace)
I have created a spring configuration file that works well.
My next step was to separate user configuration properties from system properties.
I have decided to create additional xml file with beans that will be configured by the user.
I had problem to create few such logical beans encapsulating properties that will be used by real class beans:
I have found over the net an option to reference proprieties in such way:
UserConf.xml
<bean id="numberGuess" class="x...">
<property name="randomNumber" value="5"/>
<!-- other properties -->
</bean>
SystemConf.xml
<import resource="UserConf.xml" />
<bean id="shapeGuess" class="y...">
<property name="initialShapeSeed" value="#{ numberGuess.randomNumber }"/>
<!-- other properties -->
</bean>
But my problem is that i need x... class to be something logical that shouldn't be initialized at all, and i don't want it to disclose any info of the class hierarchy of the system since it should be only in use configuration xml file.
Solution1 is to create a Java object representing this proprieties:
public class MyProps(...)
and add a bean parent in the spring system configuration:
<bean id="MyProps" class="path to MyProps"/>
in the user side change the previous bean to be:
<bean id="numberGuess" parent="MyProps">
<property name="randomNumber" value="5"/>
<!-- other properties -->
</bean>
Solution2 is to use flat configuration file just like Database.props, and load it using factory.
Solution3 is to use Spring Property Placeholder configuration to load properties from XML properties file (e.g. example), but here i simply don't know how to get a more complex nested structure of properties (properties need to be separated by different logical names, e.g. minNumber will be defined both under xAlgo and y algo).
I don't like to create new Java class only to deal with this problem or to move my user configuration to a flat props file (i need the xml structure), is their any other solution??
I will answer my own question, since it looks as the best solution for me (and much more simplistic than was suggested)
I will use PropertiesFactoryBean to do the work for me:
e.g.
UserConf.xml
<bean id="numberGuess" class="org.springframework.beans.factory.config.PropertiesFactoryBean">
<property name="properties">
<props>
<prop key="randomNumber">3</prop>
<!-- other properties -->
</bean>
SystemConf.xml
<import resource="UserConf.xml" />
<bean id="shapeGuess" class="y...">
<property name="initialShapeSeed" value="#{ numberGuess.randomNumber }"/>
<!-- other properties -->
</bean>
First if you don't know about the property place holder you should take a look at that. Also #Value("${some.property:defaultvalue}") is something you should look at.
Second the word configuration is ambiguous in Spring. Spring uses this word but they mean developer configuration not user configuration. Despite what people say or think Spring is not a configuration engine.
I'm not sure what your trying to do but you should be aware that your configuration will not be adjusted at runtime which is frequently needed for something like "user" configuration. So most people write their own configuration layer.
Another thing you should take a look at is not using the XML configuration and instead use Java Configuration which will give you way more flexibility.
In my spring batch project I can do something like this:
<bean id="exampleTasklet" class="my.custom.Tasklet">
<property name="message" value="job parameter value: #{jobParameters['arg1']}"/>
</bean>
and the message property will have a value taken from the spring batch job parameters. However, the value that I actually want to assign is very large and I don't want to put it in the xml file. I know this syntax doesn't work, but I would like to do something like:
<bean id="exampleTasklet" class="my.custom.Tasklet">
<property name="message" read-value-from-file="/path/to/file.txt"/>
</bean>
and that file would contain the line "job parameter value: #{jobParameters['arg1']}" which spring will parse as if the file content was in a value="" attribute.
Is there a nice way to do this?
I think what you are looking for is a PropertyPlaceholderConfigurer.
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="location" value="/path/to/file.properties" />
<property name="placeholderPrefix" value="#{" />
<property name="placeholderSuffix" value="}" />
</bean>
This is run by Spring as a bean processor and will attempt to resolve placeholder tokens. There is a default instance that will resolve against system properties, using this notation: ${propertyname}. For your notation, you would need to specify the placeholderPrefix/Suffix. When there are multiple bean processors, the order is determined by the order property. By default, if a processor fails to resolve a placeholder, execution fails, but this can be altered by setting ignoreUnresolvablePlaceholders. Since the mechanism is property driven, you probably want to consider a notation like:
<property name="message" value="job parameter value: #{jobParameters.arg1}"/>
Or, if what you're trying to convey is that arg1 is also a parameter, you might try:
<property name="message" value="job parameter value: #{jobParameters.${arg1}}"/>
Spring loops over the bean processors until no replacements are performed, or an exception is raised. So defining a property as ${something.${orOther}} is valid.
I would suggest you to use a String as file name and in your bean open that file.
I'm not sure if I get your problem right. I'm just suggesting something like Spring MessageBundle
Something like this:
<bean id="exampleTasklet" class="my.custom.Tasklet">
<property name="messagePath" location="/path/to/file.txt"/>
</bean>
And in your exampleTasklet read the file and do your thing (I'm not sure what it is)
If anybody came here to do something like this from a properties-file:
If you want a property from a .properties-file to appear in the JobParameters, you won't find ready-to-use solution. You can do the following:
Wrap a bean around your properties file.
Pass this bean to another one which has access to the JobParameters and can pump the properties from the file into that class.
Then you should be able to access your properties with Spring's Expression Language and do something like:
<bean id="myBean" class="my.custom.Bean">
<property name="prop" value="#{jobParameters['arg1']}"/>
</bean>
Alternatively, I think the solution proposed by Devon_C_Miller is much easier. You don't have the properties in your JobParameters then. But if the replacement in the XML configuration is the only thing you want, you only have to change your placeholders to:
${myPropFromFile}
Happy batching, everyone ;-)
lets say, I have a lot of stuff within my spring application context which looks like that
<bean name="foo.0001" class="com.example.MyClass">
<property name="name" value="foo.name.0001"/>
<property name="zap">
<bean class="com.example.Other">
<property name="name" value="foo.name.0001"/>
</bean>
</property>
<property name="bar">
<bean class="com.example.NextOther">
<property name="name" value="foo.name.0001"/>
</bean>
</property>
</bean>
so the string foo.name.0001 appears within the bean definition several times. Because it is a larger system with several blocks of this configuration, it is quite annoying to modify each of those ids. Ideally I would want to set it only once within a block. Is there a possibility to set some kind of property which exists only in a local scope of a bean definition?
I'm not sure how that would logically work, as you would still have to reference that value somehow to pass it to the nested beans. If you are worried about defining it multiple times, you can have a look at Springs PropertyPlaceholderConfigurer. It will allow you to the following:
<property name="bar">
<bean class="com.example.NextOther">
<property name="name" value="${foo.name.001}"/>
</bean>
</property>
This would allow you to define it once, and reference it from multiple locations.
It depends how much effort you want to put into to this but your requirements could be fufilled by a spring custom namespace. These are ideal when you have lots of identical blocks of beans each configured differently.
Basically you'd define the xml schema then write a bean definition parser that sets up the beans as required.
See here for more details:
http://www.javaworld.com/javaworld/jw-02-2008/jw-02-springcomponents.html
This is how Spring security simplified its xml configuration.
I need to load a specific applicationContext.xml file according to a given system property. This itself loads a file with the actual configuration. Therefore I need two PropertyPlaceHolderConfigurer, one which resolves the system param, and the other one within the actual configuration.
Any ideas how to do this?
Yes you can do more than one. Be sure to set ignoreUnresolvablePlaceholders so that the first will ignore any placeholders that it can't resolve.
<bean id="ppConfig1" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="ignoreUnresolvablePlaceholders" value="true"/>
<property name="locations">
<list>
<value>classpath*:/my.properties</value>
</list>
</property>
</bean>
<bean id="ppConfig2" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="ignoreUnresolvablePlaceholders" value="false"/>
<property name="locations">
<list>
<value>classpath*:/myOther.properties</value>
</list>
</property>
</bean>
Depending on your application, you should investigate systemPropertiesMode, it allows you to load properties from a file, but allow the system properties to override values in the property file if set.
Another solution is to use placeholderPrefix property of PropertyPlaceholderConfigurer. You specify it for the second (third, fourth...) configurer, and then prefix all your corresponding placeholders, thus there will be no conflict.
<bean id="mySecondConfigurer"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"
p:location="classpath:/myprops.properties"
p:placeholderPrefix="myprefix-"/>
<bean class="com.mycompany.MyClass" p:myprop="${myprefix-value.from.myprops}"/>
Beware -- there might be a bug related to multiple configurers. See http://jira.spring.io/browse/SPR-5719 for more details.
I'm unable to get multiple to work locally... but I'm not yet blaming anyone but myself.
On my own side, playing with PropertyPlaceholderConfigurer both properties :
order (should be lower for first accessed/parsed PPC)
ignoreUnresolvablePlaceholders ("false" for first accessed/parsed PPC, "true" for next one)
and also give 2 distinct id(s) to both PPC (to avoid one to be overwritten by the other)
works perfectly
Hope it helps
You can't do this directly, and this JIRA issue from Spring explains why (check the comment from Chris Beams for a detailed explanation):
https://jira.springsource.org/browse/SPR-6428
However, he does provide a workaround using Spring 3.1 or later, which is to use the PropertySourcesPlaceholderConfigurer class instead of PropertyPlaceholderConfigurer class.
You can download a Maven-based project that demonstrates the problem and the solution from the Spring framework issues github:
https://github.com/SpringSource/spring-framework-issues
Look for the issue number, SPR-6428, in the downloaded projects.
We have the following approach working:
<util:properties id="defaultProperties">
<prop key="stand.name">DEV</prop>
<prop key="host">localhost</prop>
</util:properties>
<context:property-placeholder
location="file:${app.properties.path:app.properties}"
properties-ref="defaultProperties"/>
System property app.properties.path can be used to override path to config file.
And application bundles some default values for placeholders that cannot be defined with defaults in common modules.
Just giving 2 distinct ids worked for me. I am using spring 3.0.4.
Hope that helps.
In case, you need to define two PPC's (like in my situation) and use them independently. By setting property placeholderPrefix, you can retrieve values from desired PPC. This will be handy when both set of PPC's properties has same keys, and if you don't use this the property of ppc2 will override ppc1.
Defining your xml:
<bean name="ppc1"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="properties" ref="ref to your props1" />
<property name="placeholderPrefix" value="$prefix1-{" />
</bean>
<bean name="ppc2"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="properties" ref="ref to your props2" />
<property name="placeholderPrefix" value="$prefix2-{" />
</bean>
Retrieving during Run time:
#Value(value = "$prefix1-{name}")
private String myPropValue1;
#Value(value = "$prefix2-{name}")
private String myPropValue2;