We Keep Coding

Beanio stream to Process list of inner elements in xml flat file

I am new to beanio. Can some one please help me with this. I am trying to process the input file, which looks like below, where existence of Item elements inside Data element is numerous:
<?xml version="1.0" encoding="windows-1252"?>
<ROOT xmlns="http://www.post.ch/schemas/dfu/2006/20/Report11">
<Sender>
<Provider>
<Data>
<Item SendingID="314" ItemID="996099999902916713" IdentCode="996099999902916713" AdrZ1="Meier" AdrZ2="" AdrZ3="" AdrZ4="Dorfweg" AdrZ5="6" AdrZ6="4117" AdrZ7="Burg im Leimental" DFUTransferDateTime="2019-07-18T13:24:56" ItemWeight="480" EventDateTime="2019-07-22T04:13:59" EventNumber="9209" EventDescription="Ordre voisin souhaité" EventNumber_Sub="" Event_SubDescription="" EventZipCode="303001" PrZl="SAZU" DeliveredTo="" TrackAndTrace="https://service.post.ch/EasyTrack/secure/GKDirectSearch.do?formattedParcelCodes=996099999902916713;&lang=fr" Status="31" StatusDescription="Prêt pour la distribution" ExpectedDeliveryDate="27.07.19" DeadlinePickup="" />
<Item SendingID="313" ItemID="996099999902916585" IdentCode="996099999902916585" AdrZ1="Müllerer" AdrZ2="" AdrZ3="" AdrZ4="Gärischstrasse" AdrZ5="27" AdrZ6="4512" AdrZ7="Bellach" DFUTransferDateTime="2019-07-17T13:51:23" ItemWeight="1080" EventDateTime="2019-07-22T07:32:59" EventNumber="4000" EventDescription="Distribué" EventNumber_Sub="" Event_SubDescription="" EventZipCode="462070" PrZl="" DeliveredTo=" " TrackAndTrace="https://service.post.ch/EasyTrack/secure/GKDirectSearch.do?formattedParcelCodes=996099999902916585;&lang=fr" Status="61" StatusDescription="Distribué" ExpectedDeliveryDate="22.07.19" DeadlinePickup="" />
</Data>
</Provider>
</Sender>
</ROOT>
My stream in beanio xml looks like below:
<stream name="swisspost" format="xml" xmlName="ROOT" xmlNamespace="http://www.post.ch/schemas/dfu/2006/20/Report11">
<record name="shippingData" xmlName="Sender" class="com.test.eventproc.carrier.data.ShippingDataDO">
<property name="carrierMoniker" value="swisspost"/>
<segment name="Provider">
<segment name="Data">
<segment name="Item">
<field name="trackingNumber" xmlName="IdentCode" xmlType="attribute"></field>
</segment>
<segment name="shippingDetail" xmlName="Item" xmlType="element" class="com.test.eventproc.carrier.data.ShippingDetailDO">
<field name="trackingNumber" xmlName="IdentCode" xmlType="attribute"></field>
<field name="eventCode" xmlName="EventNumber" xmlType="attribute"></field>
<field name="eventDate" xmlName="EventDateTime" xmlType="attribute"></field>
<field name="eventName" xmlName="EventDescription" xmlType="attribute"></field>
</segment>
</segment>
</segment>
</record>
</stream>
As per my above stream only first Item element value is processed and the program exits. Is there a way, i can process all Item elements? Below is the code, i am using to populate BeanReader object:
StreamFactory factory = StreamFactory.newInstance();
factory.loadResource("mapping.xml");//mapping.xml is my bean-io mapping file.
File originalFile = convert(file);//file is the input xml flat file
FileInputStream inputStream = new FileInputStream(originalFile);
BeanReader reader = factory.createReader(streamName, new InputStreamReader(inputStream));
Object record;
List<ShippingDataDO> shippingDataDOs = new ArrayList<ShippingDataDO>();
while ((record = reader.read()) != null) {
ShippingDataDO shippingDataDO = (ShippingDataDO) record;
shippingDataDOs.add(shippingDataDO);
}
I tried mapping xml something like below. But, now no records are retrieved from the input flat xml file.
<stream name="swisspost" format="xml" xmlName="ROOT" xmlNamespace="http://www.post.ch/schemas/dfu/2006/20/Report11">
<record name="Sender">
<segment name="Provider">
<segment name="Data">
<segment name="shippingData" xmlName="Item" maxOccurs="unbounded" collection="list" class="com.narvar.carrier.utilities.pojo.ShippingDataDO">
<property name="carrierMoniker" value="swisspost"/>
<field name="trackingNumber" xmlName="IdentCode" xmlType="attribute"></field>
</segment>
<segment name="shippingDetail" xmlName="Item" xmlType="element" maxOccurs="unbounded" class="com.narvar.carrier.utilities.pojo.ShippingDetailDO" collection="list">
<field name="trackingNumber" xmlName="IdentCode" xmlType="attribute"></field>
<field name="eventCode" xmlName="EventNumber" xmlType="attribute"></field>
<field name="eventDate" xmlName="EventDateTime" xmlType="attribute"></field>
<field name="eventName" xmlName="EventDescription" xmlType="attribute"></field>
</segment>
</segment>
</segment>
</record>
</stream>

You just need to make a small adjustment to your mapping file. You must tell BeanIO that the shippingDetail segment is a repeatable segment, by setting the maxOccurs attribute to unbounded and setting the collection attribute to list.
See the Repeating Segments documentation for more detail, in essence:
Similar to repeating fields, BeanIO supports repeating segments, which may be bound to a collection of bean objects.
In our mapping file, in order to bind a segment to a collection, simply set it's collection attribute to the fully qualified class name of a java.util.Collection or java.util.Map subclass, or to one of the collection type aliases
Repeating segments can declare the number of occurrences using the minOccurs and maxOccurs attributes. If not declared, minOccurs will default to 1, and maxOccurs will default to the minOccurs value or 1, whichever is greater.
Your shippingDetail segment then becomes:
<segment name="shippingDetail" xmlName="Item" xmlType="element" maxOccurs="unbounded"
class="com.test.eventproc.carrier.data.ShippingDetailDO" collection="list">
<field name="trackingNumber" xmlName="IdentCode" xmlType="attribute"></field>
<field name="eventCode" xmlName="EventNumber" xmlType="attribute"></field>
<field name="eventDate" xmlName="EventDateTime" xmlType="attribute"></field>
<field name="eventName" xmlName="EventDescription" xmlType="attribute"></field>
</segment>
Hope this help!

beanIO: identify different records with literal

SITUATION:
I use beanIO 2.1.0 to read a csv-file into different kind of objects.
This is my csv-File. A list of animals (color, type, number of legs).
In my list are also animals without a type (last row).
brown;cat;4
white;dog;4
brown;dog;4
black;;8
I want to read the csv-file into different animal-objects.
If the type is 'cat' it should be a cat-object. The same with dog.
If the type isn't cat or dog, e.g. empty or an unknown animal-type, then it should be an animal-object.
Here the belonging beanIO-mapping:
<beanio xmlns="http://www.beanio.org/2012/03" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.beanio.org/2012/03 http://www.beanio.org/2012/03/mapping.xsd">
<stream name="animalFile" format="csv" >
<parser>
<property name="delimiter" value=";"/>
</parser>
<record name="animal" class="zoo.Cat">
<field name="color" />
<field name="type" rid="true" literal="cat"/>
<field name="legs"/>
</record>
<record name="animal" class="zoo.Dog">
<field name="color" />
<field name="type" rid="true" literal="dog"/>
<field name="legs"/>
</record>
<record name="animal" class="zoo.Animal" >
<field name="color" />
<field name="type"/>
<field name="legs"/>
</record>
</stream>
</beanio>
My program reads the csv-file, parses it with beanIO and calls the toString-method of the parsed objects.
This is the output. It looks fine:
CAT: brown;cat;4
DOG: white;dog;4
DOG: brown;dog;4
ANIMAL: black;;8
PROBLEM:
Now I just change the order of the animals in the csv-file.
In the second row is the unknown animal-type:
brown;cat;4
black;;8
white;dog;4
brown;dog;4
This ist the new output!
When the first unknown animal is found, then all the following rows are also unknown animals.
CAT: brown;cat;4
ANIMAL: black;;8
ANIMAL: white;dog;4
ANIMAL: brown;dog;4
QUESTION:
Is it a bug in beanIO or can I configure it in the beanIO-mapping?

EDIT: Updated answer after comments from OP.
This is not a bug in BeanIO. You have two options to identify a record with. First, you have the literal attribute as you used it so far. Secondly you can also use a regular expression (regex) to identify records with.
You want to match an Animal object when the type field is not cat or dog, or as you stated when it is an empty string/object.
Your type field definition could be one of two for the Animal record.
<field name="type" rid="true" regex="\s*" />
Here it will match whenever the type field contains spaces as defined by the java regular expressions.
OR
<field name="type" rid="true" regex=""^(?:(?!\b(cat|dog)\b).)*$" />
This will match any record where the type field doesn't contain the words cat or dog.
Try it with this Animal record:
<record name="animal" class="zoo.Animal" >
<field name="color" />
<field name="type" rid="true" regex=""^(?:(?!\b(cat|dog)\b).)*$" />
<field name="legs"/>
</record>
Off-topic. Technically you are not reading a CSV file because then your delimiter must be a comma. Instead, you have a delimited format which uses a semi-colon (;) as a delimiter.
I would also suggest that you make the names of your record definitions unique in your xml mapping file. The record name is used in error messages for reporting the location of a problem. If you have the same record name for all records, you will not know where to look for the problem.

Key-value mapping in BeanIO fixed-length record

I have the following specification for a fixed-length data file (refer to record-C type of specification, page 4)
a second part, having a length of 1,800 characters, consisting of a table of 75 elements to be used for the display of the only data present in the communication; each of these elements is constituted by a field-code
of 8 characters and by a field-value of 16 characters
It means that the first 89 characters (omitted in the above summary) are plain old fixed-length and then, for the remaining 1800, I have to take them into groups of key-value pairs each counting up to 24 characters. Blank spaces are trimmed and empty pairs are not considered in the process.
Ideally, my bean may be constructed like
public class RecordC{
private List<Pair<String, String>> table = new ArrayList<>(MAX_TABLE_SIZE); //I don't want to use Map **yet**
}
Something can be e.g. Apache Common's Pair<String,String> or anything suitable for KVP mapping.
I understand that I can create a whole TypeHandler that takes the full 1800 bytes but I wanted to exploit the power of BeanIO.
Here is what I have done so far
<record name="RECORD_C" class="it.csttech.ftt.data.beans.ftt2017.RecordC" order="3" minOccurs="1" maxOccurs="1" maxLength="2000">
<field name="tipoRecord" rid="true" at="0" ignore="true" required="true" length="1" lazy="true" literal="C" />
<field name="cfContribuente" at="1" length="16" align="left" trim="true" lazy="true" />
<field name="progressivoModulo" at="17" length="8" padding="0" align="right" trim="true" lazy="true" />
<field name="spazioDisposizioneUtente" at="25" length="3" align="left" trim="true" lazy="true" />
<field name="spazioUtente" at="53" length="20" align="left" trim="true" lazy="true" />
<field name="cfProduttoreSoftware" at="73" length="16" align="left" trim="true" lazy="true" />
<segment name="table" collection="list" lazy="true" class="org.apache.commons.lang3.tuple.ImmutablePair">
<field name="key" type="java.lang.String" at="0" length="8" trim="true" lazy="true" setter="#1" />
<field name="value" type="java.lang.String" at="8" length="16" trim="true" lazy="true" setter="#2" />
</segment>
<field name="terminatorA" at="1897" length="1" rid="true" literal="A" ignore="true" />
</record>
Unfortunately this does not work in testing. I get only a single record in the list, decoded at positions [0-7] and [8-23] instead of expected [89-113][114-???][....][....]
Question is: how do I in BeanIO declare repeating fixed-length fields?

I have currently resolved my unmarshalling problem by removing all at attributes in the RecordC specifications. As I found out, the "at" attribute is absolute to the record and not relative to the repeating segment. However this forced me to add some ignored fields in the unmarshalling at the sole cost of a few ignores.
I will test the marshalling against the official controller once I have data

Trying to serialize an object compactly using Castor

I'm using Castor to write out a map of user ID's to time intervals. I'm using it to save and resume progress in a lengthy task, and I'm trying to make the XML as compact as possible. My map is from string userID's to a class that contains the interval timestamps, along with additional transient data that I don't need to serialize.
I'm able to use a nested class mapping:
...
<field name="userIntervals" collection="map">
<bind-xml name="u">
<class name="org.exolab.castor.mapping.MapItem">
<field name="key" type="string"><bind-xml name="n" node="attribute"/></field>
<field name="value" type="my.package.TimeInterval"/>
</class>
</bind-xml>
</field>
...
<class name="my.package.TimeInterval">
<map-to xml="ti"/>
<field name="intervalStart" type="long"><bind-xml name="s" node="attribute"/></field>
<field name="intervalEnd" type="long"><bind-xml name="e" node="attribute"/></field>
</class>
...
And get output that looks like:
<u n="36164639"><value s="1292750896000" e="1292750896000"/></u>
What I'd like is the name, start, and end of the user in a single node like this.
<u n="36164639" s="1292750896000" e="1292750896000"/>
But I can't seem to finagle it so the start and end attributes in the "value" go in the same node as the "key". Any ideas would be greatly appreciated.

Nash,
I think to arrange the castor mapping is bit tricky.
If you want to have structure like
<u n="36164639" s="1292750896000" e="1292750896000"/>
Then you need to create a new pojo file where it will be having
all the three fields Key,intervalStart,intervalEnd.
And let the File name as KeyTimeInterval
And map it like the below.
<field name="userIntervals" collection="map">
<class name="org.exolab.castor.mapping.MapItem">
<field name="u" type="my.package.KeyTimeInterval">
<bind-xml name="u" node="element"/>
</field>
</class>
</field>
<class name="my.package.KeyTimeInterval">
<field name="key" type="String">
<bind-xml name="n" node="attribute"/></field>
<field name="intervalStart" type="long">
<bind-xml name="s" node="attribute"/></field>
<field name="intervalEnd" type="long">
<bind-xml name="e" node="attribute"/></field>
</class>

I think you should be able to use location on s and e. Try this:-
...
<class name="my.package.TimeInterval">
<map-to xml="ti"/>
<field name="intervalStart" type="long">
<bind-xml name="s" location="u" node="attribute"/>
</field>
<field name="intervalEnd" type="long">
<bind-xml name="e" location="u" node="attribute"/>
</field>
</class>

Am answering my own question here, since there is a solution that does exactly what I want, and there's actually an error in the explanation at http://www.castor.org/xml-mapping.html#Sample-3:-Using-the-container-attribute - the container attribute is exactly what's needed here.
Changing one line in the mapping:
<field name="value" type="my.package.TimeInterval" container="true"/>
did exactly what I wanted, it didn't create a subelement for the value, just mapped the fields into the existing parent element. Since then, I've used this quite a few times to map multiple-value classes into their parent.
The error of course is the documentation states you do this by setting the container attribute to false. Of course, it should be true.

Mapping collection to XML in Castor

I'm trying to map a POJO to XML using Castor.
Let's say I have a Order that has a collection of Items... is there any way of achieving an xml like the following:
<order>
...order attributes
<items>
<item> ..item attributes </item>
<item> ..other item </item>
</items>
</order>
I could make something similar but without the <items> node. This wouldn't be a problem in other case but my XML must adhere to a strict XSD schema so I need to do it like that.
Thanks!
I though of a kind of "workaround" that would involve creating a new java object (that would be the node) that would only contain the list of items... can anyone think of a better approach? there's a 100 rep bounty open since now!

You can use the location attribute of the bind-xml lement
http://castor.codehaus.org/1.2/xml-mapping.html#6.-Location-attribute
Example from the docs:
<class name="Foo">
<field name="bar" type="Bar">
<bind-xml name="bar" location="abc"/>
</field>
</class>
Produces the following XML:
<foo>;
<abc>
<bar>...</bar>
</abc>
</foo>

The other answer doesn't use the collection attribute which I think is probably what you're ultimately needing.
Something like this might work when included in your mapping for the Order object:
<field name="items" type="item" collection="arraylist" >
<bind-xml name="items" node="element"/>
</field>