We Keep Coding

Castor JDO / OQL query returning duplicate objects and missing some dependants

I have this issue when querying certain objects (persisting in Postgres database)
using Castor 0.9.6 (can't upgrade)
Problem only occurs on production, not dev machine. Despite using same database.
udOql= db.getOQLQuery( "SELECT DISTINCT ud FROM model.objects.UserDefinition ud WHERE ud.siteId = $1");
udOql.bind(other.getSiteId());
This normally is working fine but for some reason I am getting duplicate objects, even when I include DISTINCT
Mapping XML is as follows:
<class name="model.objects.UserDefinition" identity="userId" key-generator="HIGH-LOW">
<cache-type type="unlimited" capacity="1000"/>
<map-to table="userdefinition" xml="userDefinition" />
<field name="siteId" type="java.lang.Integer">
<sql name="siteid" type="integer" />
</field>
<field name="userAttributeDefinition" type="model.objects.UserAttributeDefinition" collection="vector">
<sql many-key="userid" />
</field>
<field name="roleAssignmentDefinition" type="model.objects.RoleAssignmentDefinition" collection="vector">
<sql many-key="userid" />
</field>
</class>
Not only will it return multiple UserDefinition objects, some of them are missing some linked roleAssignmentDefinition or userAttributeDefinition. Usually after restarting tomcat it fixes the problem but then it comes back. Sometimes the issue the issue is there from the start.
The database holds correct data. Any tips, pointers much appreciated

beanIO: identify different records with literal

SITUATION:
I use beanIO 2.1.0 to read a csv-file into different kind of objects.
This is my csv-File. A list of animals (color, type, number of legs).
In my list are also animals without a type (last row).
brown;cat;4
white;dog;4
brown;dog;4
black;;8
I want to read the csv-file into different animal-objects.
If the type is 'cat' it should be a cat-object. The same with dog.
If the type isn't cat or dog, e.g. empty or an unknown animal-type, then it should be an animal-object.
Here the belonging beanIO-mapping:
<beanio xmlns="http://www.beanio.org/2012/03" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.beanio.org/2012/03 http://www.beanio.org/2012/03/mapping.xsd">
<stream name="animalFile" format="csv" >
<parser>
<property name="delimiter" value=";"/>
</parser>
<record name="animal" class="zoo.Cat">
<field name="color" />
<field name="type" rid="true" literal="cat"/>
<field name="legs"/>
</record>
<record name="animal" class="zoo.Dog">
<field name="color" />
<field name="type" rid="true" literal="dog"/>
<field name="legs"/>
</record>
<record name="animal" class="zoo.Animal" >
<field name="color" />
<field name="type"/>
<field name="legs"/>
</record>
</stream>
</beanio>
My program reads the csv-file, parses it with beanIO and calls the toString-method of the parsed objects.
This is the output. It looks fine:
CAT: brown;cat;4
DOG: white;dog;4
DOG: brown;dog;4
ANIMAL: black;;8
PROBLEM:
Now I just change the order of the animals in the csv-file.
In the second row is the unknown animal-type:
brown;cat;4
black;;8
white;dog;4
brown;dog;4
This ist the new output!
When the first unknown animal is found, then all the following rows are also unknown animals.
CAT: brown;cat;4
ANIMAL: black;;8
ANIMAL: white;dog;4
ANIMAL: brown;dog;4
QUESTION:
Is it a bug in beanIO or can I configure it in the beanIO-mapping?

EDIT: Updated answer after comments from OP.
This is not a bug in BeanIO. You have two options to identify a record with. First, you have the literal attribute as you used it so far. Secondly you can also use a regular expression (regex) to identify records with.
You want to match an Animal object when the type field is not cat or dog, or as you stated when it is an empty string/object.
Your type field definition could be one of two for the Animal record.
<field name="type" rid="true" regex="\s*" />
Here it will match whenever the type field contains spaces as defined by the java regular expressions.
OR
<field name="type" rid="true" regex=""^(?:(?!\b(cat|dog)\b).)*$" />
This will match any record where the type field doesn't contain the words cat or dog.
Try it with this Animal record:
<record name="animal" class="zoo.Animal" >
<field name="color" />
<field name="type" rid="true" regex=""^(?:(?!\b(cat|dog)\b).)*$" />
<field name="legs"/>
</record>
Off-topic. Technically you are not reading a CSV file because then your delimiter must be a comma. Instead, you have a delimited format which uses a semi-colon (;) as a delimiter.
I would also suggest that you make the names of your record definitions unique in your xml mapping file. The record name is used in error messages for reporting the location of a problem. If you have the same record name for all records, you will not know where to look for the problem.

Key-value mapping in BeanIO fixed-length record

I have the following specification for a fixed-length data file (refer to record-C type of specification, page 4)
a second part, having a length of 1,800 characters, consisting of a table of 75 elements to be used for the display of the only data present in the communication; each of these elements is constituted by a field-code
of 8 characters and by a field-value of 16 characters
It means that the first 89 characters (omitted in the above summary) are plain old fixed-length and then, for the remaining 1800, I have to take them into groups of key-value pairs each counting up to 24 characters. Blank spaces are trimmed and empty pairs are not considered in the process.
Ideally, my bean may be constructed like
public class RecordC{
private List<Pair<String, String>> table = new ArrayList<>(MAX_TABLE_SIZE); //I don't want to use Map **yet**
}
Something can be e.g. Apache Common's Pair<String,String> or anything suitable for KVP mapping.
I understand that I can create a whole TypeHandler that takes the full 1800 bytes but I wanted to exploit the power of BeanIO.
Here is what I have done so far
<record name="RECORD_C" class="it.csttech.ftt.data.beans.ftt2017.RecordC" order="3" minOccurs="1" maxOccurs="1" maxLength="2000">
<field name="tipoRecord" rid="true" at="0" ignore="true" required="true" length="1" lazy="true" literal="C" />
<field name="cfContribuente" at="1" length="16" align="left" trim="true" lazy="true" />
<field name="progressivoModulo" at="17" length="8" padding="0" align="right" trim="true" lazy="true" />
<field name="spazioDisposizioneUtente" at="25" length="3" align="left" trim="true" lazy="true" />
<field name="spazioUtente" at="53" length="20" align="left" trim="true" lazy="true" />
<field name="cfProduttoreSoftware" at="73" length="16" align="left" trim="true" lazy="true" />
<segment name="table" collection="list" lazy="true" class="org.apache.commons.lang3.tuple.ImmutablePair">
<field name="key" type="java.lang.String" at="0" length="8" trim="true" lazy="true" setter="#1" />
<field name="value" type="java.lang.String" at="8" length="16" trim="true" lazy="true" setter="#2" />
</segment>
<field name="terminatorA" at="1897" length="1" rid="true" literal="A" ignore="true" />
</record>
Unfortunately this does not work in testing. I get only a single record in the list, decoded at positions [0-7] and [8-23] instead of expected [89-113][114-???][....][....]
Question is: how do I in BeanIO declare repeating fixed-length fields?

I have currently resolved my unmarshalling problem by removing all at attributes in the RecordC specifications. As I found out, the "at" attribute is absolute to the record and not relative to the repeating segment. However this forced me to add some ignored fields in the unmarshalling at the sole cost of a few ignores.
I will test the marshalling against the official controller once I have data

Mapping both xml element and its attribute using BeanIO

I would like to map the totalAmt tag in below xml file, both its value 100 and it's attribute Ccy.
<?xml version="1.0" encoding="UTF-8"?>
<transaction>
<id>
<eId>transactionId001</eId>
</id>
<amount>
<totalAmt Ccy="XXX">100</totalAmt>
</amount>
</transaction>
By reading BeanIO reference guide and posts here I got the impression that only one of them can be mapped.
So my question is: Can BeanIO handle this tag and could you show me how?
What I have tried and didn't work:
<segment name="amount">
<field name="totalAmount" xmlName="totalAmt"></field>
<field name="currency" xmlName="Ccy" xmlType="attribute"></field>
</segment>

Close, but you still need to add the segment element inside the segment tag to tell which field the attribute is belong to.
example.
<segment name="amount">
<field name="totalAmount" xmlName="totalAmt"></field>
<segment name="totalAmt">
<field name="type" xmlName="Ccy" xmlType="attribute"></field>
</segment>
</segment>

I am using bean io 2.1 version
The
<segment name="totalAmt">
<field name="totalAmount" xmlType="text"></field> --->the bean variable "totalAmount" will give say 100
<field name="Cctype" xmlName="Ccy" xmlType="attribute" default="XXX"></field> -->either set default value as XXX or it will take from cctype variable
</segment>

Searching Images in Solr 3.3.0

I am working with Solr 3.3.0 version and I need to index and search Images. I m able to index the image files but it fails to retrieve them as search results.
Can anyone of you help me out with this.
My data-config.xml is :
<dataConfig>
<dataSource type="BinFileDataSource" name="bin"/>
<document>
<entity name="f" processor="FileListEntityProcessor" recursive="true"
rootEntity="false"
dataSource="null" baseDir="C:/Files"
fileName=".*\.(DOC)|(PDF)|(XML)|(xml)|(JPEG)|(jpg)|(ZIP)|(zip)|(pdf)|(doc)" onError="skip">
<entity name="tika-test" processor="TikaEntityProcessor"
url="${f.fileAbsolutePath}" format="text" dataSource="bin" onError="skip">
<field column="Author" name="author" meta="true"/>
<field column="title" name="title" meta="true"/>
<field column="text" name="text"/>
<field column="id" name="id"/>
<field column="Keywords" name="keywords" meta="true"/>
</entity>
<field column="file" name="fileName"/>
<field column="fileAbsolutePath" name="links"/>
</entity>
</document>
</dataConfig>
This works fine for types other than images, I am not able to get images in search result

Ans) you have to search in solr admin page using the query in URL after indexed..
when ever you images are indexed it will display in Overview tab of solr admin ..
suppose it is not displaying the documents and time in solr web pag**e the image is not indexed still iam searching the same .....