I am using BeanIO version 2.1.0. I am currently trying to create an XML file using a template with BeanIO that contains a list that I am repeating it two times. I am not sure how do I make sure second list's element do not appear in the first list.
For example:
{
"abc": {
"def": [
{
"id": 1
"fname": "James"
},
{
"id": 2
"lname": "Washington"
}
]
}
}
So as you can see from the above example, I have a list def that is repeated 2 times. So in the first list there is a field called fname and in the second list there is a field called lname. How do I define the list in the template so that lname does not appear when creating the first list?
I tried to change my template to include both fname and lname, but I am confused on how do I ensure to not include lname in the first list and fname in the second list.
<beanio>
<template name="example"?
<segment name="abc" class="com.abc">
<segment name="def" collection="list" class="com.def">
<field name="id" type="int" />
<field name="fname" type="string" />
<field name="lname" type="string" />
</segment>
</segment>
</template>
</beanio>
Related
I am new to beanio. Can some one please help me with this. I am trying to process the input file, which looks like below, where existence of Item elements inside Data element is numerous:
<?xml version="1.0" encoding="windows-1252"?>
<ROOT xmlns="http://www.post.ch/schemas/dfu/2006/20/Report11">
<Sender>
<Provider>
<Data>
<Item SendingID="314" ItemID="996099999902916713" IdentCode="996099999902916713" AdrZ1="Meier" AdrZ2="" AdrZ3="" AdrZ4="Dorfweg" AdrZ5="6" AdrZ6="4117" AdrZ7="Burg im Leimental" DFUTransferDateTime="2019-07-18T13:24:56" ItemWeight="480" EventDateTime="2019-07-22T04:13:59" EventNumber="9209" EventDescription="Ordre voisin souhaité" EventNumber_Sub="" Event_SubDescription="" EventZipCode="303001" PrZl="SAZU" DeliveredTo="" TrackAndTrace="https://service.post.ch/EasyTrack/secure/GKDirectSearch.do?formattedParcelCodes=996099999902916713;&lang=fr" Status="31" StatusDescription="Prêt pour la distribution" ExpectedDeliveryDate="27.07.19" DeadlinePickup="" />
<Item SendingID="313" ItemID="996099999902916585" IdentCode="996099999902916585" AdrZ1="Müllerer" AdrZ2="" AdrZ3="" AdrZ4="Gärischstrasse" AdrZ5="27" AdrZ6="4512" AdrZ7="Bellach" DFUTransferDateTime="2019-07-17T13:51:23" ItemWeight="1080" EventDateTime="2019-07-22T07:32:59" EventNumber="4000" EventDescription="Distribué" EventNumber_Sub="" Event_SubDescription="" EventZipCode="462070" PrZl="" DeliveredTo=" " TrackAndTrace="https://service.post.ch/EasyTrack/secure/GKDirectSearch.do?formattedParcelCodes=996099999902916585;&lang=fr" Status="61" StatusDescription="Distribué" ExpectedDeliveryDate="22.07.19" DeadlinePickup="" />
</Data>
</Provider>
</Sender>
</ROOT>
My stream in beanio xml looks like below:
<stream name="swisspost" format="xml" xmlName="ROOT" xmlNamespace="http://www.post.ch/schemas/dfu/2006/20/Report11">
<record name="shippingData" xmlName="Sender" class="com.test.eventproc.carrier.data.ShippingDataDO">
<property name="carrierMoniker" value="swisspost"/>
<segment name="Provider">
<segment name="Data">
<segment name="Item">
<field name="trackingNumber" xmlName="IdentCode" xmlType="attribute"></field>
</segment>
<segment name="shippingDetail" xmlName="Item" xmlType="element" class="com.test.eventproc.carrier.data.ShippingDetailDO">
<field name="trackingNumber" xmlName="IdentCode" xmlType="attribute"></field>
<field name="eventCode" xmlName="EventNumber" xmlType="attribute"></field>
<field name="eventDate" xmlName="EventDateTime" xmlType="attribute"></field>
<field name="eventName" xmlName="EventDescription" xmlType="attribute"></field>
</segment>
</segment>
</segment>
</record>
</stream>
As per my above stream only first Item element value is processed and the program exits. Is there a way, i can process all Item elements? Below is the code, i am using to populate BeanReader object:
StreamFactory factory = StreamFactory.newInstance();
factory.loadResource("mapping.xml");//mapping.xml is my bean-io mapping file.
File originalFile = convert(file);//file is the input xml flat file
FileInputStream inputStream = new FileInputStream(originalFile);
BeanReader reader = factory.createReader(streamName, new InputStreamReader(inputStream));
Object record;
List<ShippingDataDO> shippingDataDOs = new ArrayList<ShippingDataDO>();
while ((record = reader.read()) != null) {
ShippingDataDO shippingDataDO = (ShippingDataDO) record;
shippingDataDOs.add(shippingDataDO);
}
I tried mapping xml something like below. But, now no records are retrieved from the input flat xml file.
<stream name="swisspost" format="xml" xmlName="ROOT" xmlNamespace="http://www.post.ch/schemas/dfu/2006/20/Report11">
<record name="Sender">
<segment name="Provider">
<segment name="Data">
<segment name="shippingData" xmlName="Item" maxOccurs="unbounded" collection="list" class="com.narvar.carrier.utilities.pojo.ShippingDataDO">
<property name="carrierMoniker" value="swisspost"/>
<field name="trackingNumber" xmlName="IdentCode" xmlType="attribute"></field>
</segment>
<segment name="shippingDetail" xmlName="Item" xmlType="element" maxOccurs="unbounded" class="com.narvar.carrier.utilities.pojo.ShippingDetailDO" collection="list">
<field name="trackingNumber" xmlName="IdentCode" xmlType="attribute"></field>
<field name="eventCode" xmlName="EventNumber" xmlType="attribute"></field>
<field name="eventDate" xmlName="EventDateTime" xmlType="attribute"></field>
<field name="eventName" xmlName="EventDescription" xmlType="attribute"></field>
</segment>
</segment>
</segment>
</record>
</stream>
You just need to make a small adjustment to your mapping file. You must tell BeanIO that the shippingDetail segment is a repeatable segment, by setting the maxOccurs attribute to unbounded and setting the collection attribute to list.
See the Repeating Segments documentation for more detail, in essence:
Similar to repeating fields, BeanIO supports repeating segments, which may be bound to a collection of bean objects.
In our mapping file, in order to bind a segment to a collection, simply set it's collection attribute to the fully qualified class name of a java.util.Collection or java.util.Map subclass, or to one of the collection type aliases
Repeating segments can declare the number of occurrences using the minOccurs and maxOccurs attributes. If not declared, minOccurs will default to 1, and maxOccurs will default to the minOccurs value or 1, whichever is greater.
Your shippingDetail segment then becomes:
<segment name="shippingDetail" xmlName="Item" xmlType="element" maxOccurs="unbounded"
class="com.test.eventproc.carrier.data.ShippingDetailDO" collection="list">
<field name="trackingNumber" xmlName="IdentCode" xmlType="attribute"></field>
<field name="eventCode" xmlName="EventNumber" xmlType="attribute"></field>
<field name="eventDate" xmlName="EventDateTime" xmlType="attribute"></field>
<field name="eventName" xmlName="EventDescription" xmlType="attribute"></field>
</segment>
Hope this help!
SITUATION:
I use beanIO 2.1.0 to read a csv-file into different kind of objects.
This is my csv-File. A list of animals (color, type, number of legs).
In my list are also animals without a type (last row).
brown;cat;4
white;dog;4
brown;dog;4
black;;8
I want to read the csv-file into different animal-objects.
If the type is 'cat' it should be a cat-object. The same with dog.
If the type isn't cat or dog, e.g. empty or an unknown animal-type, then it should be an animal-object.
Here the belonging beanIO-mapping:
<beanio xmlns="http://www.beanio.org/2012/03" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.beanio.org/2012/03 http://www.beanio.org/2012/03/mapping.xsd">
<stream name="animalFile" format="csv" >
<parser>
<property name="delimiter" value=";"/>
</parser>
<record name="animal" class="zoo.Cat">
<field name="color" />
<field name="type" rid="true" literal="cat"/>
<field name="legs"/>
</record>
<record name="animal" class="zoo.Dog">
<field name="color" />
<field name="type" rid="true" literal="dog"/>
<field name="legs"/>
</record>
<record name="animal" class="zoo.Animal" >
<field name="color" />
<field name="type"/>
<field name="legs"/>
</record>
</stream>
</beanio>
My program reads the csv-file, parses it with beanIO and calls the toString-method of the parsed objects.
This is the output. It looks fine:
CAT: brown;cat;4
DOG: white;dog;4
DOG: brown;dog;4
ANIMAL: black;;8
PROBLEM:
Now I just change the order of the animals in the csv-file.
In the second row is the unknown animal-type:
brown;cat;4
black;;8
white;dog;4
brown;dog;4
This ist the new output!
When the first unknown animal is found, then all the following rows are also unknown animals.
CAT: brown;cat;4
ANIMAL: black;;8
ANIMAL: white;dog;4
ANIMAL: brown;dog;4
QUESTION:
Is it a bug in beanIO or can I configure it in the beanIO-mapping?
EDIT: Updated answer after comments from OP.
This is not a bug in BeanIO. You have two options to identify a record with. First, you have the literal attribute as you used it so far. Secondly you can also use a regular expression (regex) to identify records with.
You want to match an Animal object when the type field is not cat or dog, or as you stated when it is an empty string/object.
Your type field definition could be one of two for the Animal record.
<field name="type" rid="true" regex="\s*" />
Here it will match whenever the type field contains spaces as defined by the java regular expressions.
OR
<field name="type" rid="true" regex=""^(?:(?!\b(cat|dog)\b).)*$" />
This will match any record where the type field doesn't contain the words cat or dog.
Try it with this Animal record:
<record name="animal" class="zoo.Animal" >
<field name="color" />
<field name="type" rid="true" regex=""^(?:(?!\b(cat|dog)\b).)*$" />
<field name="legs"/>
</record>
Off-topic. Technically you are not reading a CSV file because then your delimiter must be a comma. Instead, you have a delimited format which uses a semi-colon (;) as a delimiter.
I would also suggest that you make the names of your record definitions unique in your xml mapping file. The record name is used in error messages for reporting the location of a problem. If you have the same record name for all records, you will not know where to look for the problem.
I have the following specification for a fixed-length data file (refer to record-C type of specification, page 4)
a second part, having a length of 1,800 characters, consisting of a table of 75 elements to be used for the display of the only data present in the communication; each of these elements is constituted by a field-code
of 8 characters and by a field-value of 16 characters
It means that the first 89 characters (omitted in the above summary) are plain old fixed-length and then, for the remaining 1800, I have to take them into groups of key-value pairs each counting up to 24 characters. Blank spaces are trimmed and empty pairs are not considered in the process.
Ideally, my bean may be constructed like
public class RecordC{
private List<Pair<String, String>> table = new ArrayList<>(MAX_TABLE_SIZE); //I don't want to use Map **yet**
}
Something can be e.g. Apache Common's Pair<String,String> or anything suitable for KVP mapping.
I understand that I can create a whole TypeHandler that takes the full 1800 bytes but I wanted to exploit the power of BeanIO.
Here is what I have done so far
<record name="RECORD_C" class="it.csttech.ftt.data.beans.ftt2017.RecordC" order="3" minOccurs="1" maxOccurs="1" maxLength="2000">
<field name="tipoRecord" rid="true" at="0" ignore="true" required="true" length="1" lazy="true" literal="C" />
<field name="cfContribuente" at="1" length="16" align="left" trim="true" lazy="true" />
<field name="progressivoModulo" at="17" length="8" padding="0" align="right" trim="true" lazy="true" />
<field name="spazioDisposizioneUtente" at="25" length="3" align="left" trim="true" lazy="true" />
<field name="spazioUtente" at="53" length="20" align="left" trim="true" lazy="true" />
<field name="cfProduttoreSoftware" at="73" length="16" align="left" trim="true" lazy="true" />
<segment name="table" collection="list" lazy="true" class="org.apache.commons.lang3.tuple.ImmutablePair">
<field name="key" type="java.lang.String" at="0" length="8" trim="true" lazy="true" setter="#1" />
<field name="value" type="java.lang.String" at="8" length="16" trim="true" lazy="true" setter="#2" />
</segment>
<field name="terminatorA" at="1897" length="1" rid="true" literal="A" ignore="true" />
</record>
Unfortunately this does not work in testing. I get only a single record in the list, decoded at positions [0-7] and [8-23] instead of expected [89-113][114-???][....][....]
Question is: how do I in BeanIO declare repeating fixed-length fields?
I have currently resolved my unmarshalling problem by removing all at attributes in the RecordC specifications. As I found out, the "at" attribute is absolute to the record and not relative to the repeating segment. However this forced me to add some ignored fields in the unmarshalling at the sole cost of a few ignores.
I will test the marshalling against the official controller once I have data
I would like to map the totalAmt tag in below xml file, both its value 100 and it's attribute Ccy.
<?xml version="1.0" encoding="UTF-8"?>
<transaction>
<id>
<eId>transactionId001</eId>
</id>
<amount>
<totalAmt Ccy="XXX">100</totalAmt>
</amount>
</transaction>
By reading BeanIO reference guide and posts here I got the impression that only one of them can be mapped.
So my question is: Can BeanIO handle this tag and could you show me how?
What I have tried and didn't work:
<segment name="amount">
<field name="totalAmount" xmlName="totalAmt"></field>
<field name="currency" xmlName="Ccy" xmlType="attribute"></field>
</segment>
Close, but you still need to add the segment element inside the segment tag to tell which field the attribute is belong to.
example.
<segment name="amount">
<field name="totalAmount" xmlName="totalAmt"></field>
<segment name="totalAmt">
<field name="type" xmlName="Ccy" xmlType="attribute"></field>
</segment>
</segment>
I am using bean io 2.1 version
The
<segment name="totalAmt">
<field name="totalAmount" xmlType="text"></field> --->the bean variable "totalAmount" will give say 100
<field name="Cctype" xmlName="Ccy" xmlType="attribute" default="XXX"></field> -->either set default value as XXX or it will take from cctype variable
</segment>
I recently upgraded my BeanIO framework to 2.0.6 version to parse my flat (tab delimited) files to java objects and I noticed so weird behavior.
I can't leave fields null in the last file line at the end because BeanIO throws this error message at me: "Expected minimum 1 occurrences."
I tried to even set the maxLength to 4 on the entire record so that it account for the extra null field at the end but it still throws that exception. What's strange is that it only does it for the last line and not for null fields in the other lines.
Mapping:
<beanio xmlns="http://www.beanio.org/2012/03"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.beanio.org/2012/03 http://www.beanio.org/2012/03/mapping.xsd">
<stream name="Inventory" format="delimited" strict="true" resourceBundle="com.crunchtime.mapping.cdp.Inventory">
<record name="myRecord" minOccurs="1" maxOccurs="unbounded" minLength="0" maxLength="4" class="com.test.Record">
<field name="userName" type="string"/>
<field name="userId" type="string"/>
<field name="type" type="string"/>
<field name="version" type="string"/>
</record>
</stream>
</beanio>
File:
Mark User1 M 1.0
Tom User2 D 1.1
Jim User3 M 2.0
Scott User4 G
Does anybody has any ideas on how to disable that behavior?
I looked at beanio.properties but I can't modify since it's locked.
Using BeanIO 2.0 or later, you must configure fields that may not be present in the input stream with minOccurs="0".