Spring saving extra information in couchbase - java

I am saving a Document in the Couchbase using Spring. For some fields extra information is added.
POJO:
#Document public class PlayerTxn implements Serializable {
private static final long serialVersionUID = -2569497126561L;
#Id private String id;
#Field private Date txnDate;
#Field private BigDecimal wagerAmount;
#Field private BigDecimal pointsAwarded;
#Field private String segment;
RequiredResult:
{ "txnDate": 234234234324, "wagerAmount": 234.33, "pointsAwarded":
23.2, "segment": "xxx" }
End result:
{ "_class": "com.app.model.PlayerTxn", "segment":
"xxx", "wagerAmount": {
"intCompact": 24312,
"scale": 2,
"precision": 5,
"stringCache": "243.12" }, "pointsAwarded": {
"intCompact": -9223372036854776000,
"scale": 38,
"precision": 0,
"intVal": {
"signum": 1,
"bitCount": 0,
"mag": [
3800,
-457875904,
-1778440383,
-1805069212,
295579091
],
"lowestSetBit": 0,
"firstNonzeroIntNum": 0,
"bitLength": 0
} }, "txnDate": 1466417747057 }
Had to write a customConverter for BigDecimal.
But still "_class" is being added in the Document. Any idea how can I remove it?

Looks like you need to add custom converters for BigDecimal values, something like:
// extend or add a customized couchbase config like this
public class CustomCouchbaseConfig extends AbstractCouchbaseConfiguration {
#Override
public CustomConversions customConversions() {
return new CustomConversions(Arrays.asList(
BigDecimalToString.INSTANCE,
StringToBigDecimalConverter.INSTANCE));
}
#WritingConverter
public static enum BigDecimalToString implements Converter<BigDecimal, String> {
INSTANCE;
#Override
public String convert(BigDecimal source) {
// or a more appropriate implementation
return source.toString() ;
}
}
#ReadingConverter
public static enum StringToBigDecimalConverter implements Converter<String, BigDecimal> {
INSTANCE;
#Override
public BigDecimal convert(String source) {
return new BigDecimal(source);
}
}
}
Be sure, to make those converter methods null-safe!
See the Spring Data Couchbase Reference for more information:
http://docs.spring.io/spring-data/couchbase/docs/2.1.2.RELEASE/reference/html/#datatypes
Edit:
Regarding the _class attribute, see this SO question and the answer by Oliver Giercke for the reasoning behind _class.
The question is aimed at MongoDB but can easily be translated for Couchbase as well.

Related

Jackson parsing error: exception org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "Results"

There's a few topics like this, however I've read them all and still no luck.
I have a class to which I've made to deserialize some JSON responses from a web service. In short, I've spent too much time looking at this and I'm hoping someone can pick out the error of my ways. As per title, I'm using the Jackson libs.
Snippet of the class below:
final class ContentManagerResponse implements Serializable {
#JsonProperty("Results")
private List<OrgSearchResult> results = null;
#JsonProperty("PropertiesAndFields")
private PropertiesAndFields propertiesAndFields;
#JsonProperty("TotalResults")
private Integer totalResults;
#JsonProperty("CountStringEx")
private String countStringEx;
#JsonProperty("MinimumCount")
private Integer minimumCount;
#JsonProperty("Count")
private Integer count;
#JsonProperty("HasMoreItems")
private Boolean hasMoreItems;
#JsonProperty("SearchTitle")
private String searchTitle;
#JsonProperty("HitHighlightString")
private String hitHighlightString;
#JsonProperty("TrimType")
private String trimType;
#JsonProperty("ResponseStatus")
private ResponseStatus responseStatus;
#JsonIgnore
private Map<String, Object> additionalProperties = new HashMap<String, Object>();
#JsonProperty("Results")
public List<OrgSearchResult> getResults() {
return results;
}
#JsonProperty("Results")
public void setResults(List<OrgSearchResult> results) {
this.results = results;
}
//additional getters and setters.
As said, Results is the property which seems to be having the error.
The JSON response is below.
{
"Results": [
{
"TrimType": "Location",
"Uri": 1684
}
],
"PropertiesAndFields": {},
"TotalResults": 1,
"CountStringEx": "1 Location",
"MinimumCount": 1,
"Count": 0,
"HasMoreItems": false,
"SearchTitle": "Locations - type:Organization and id:24221",
"HitHighlightString": "",
"TrimType": "Location",
"ResponseStatus": {}
}
I'm using the same class to deserialize the following response and it works:
{
"Results": [
{
"LocationIsWithin": {
"Value": true
},
"LocationSortName": {
"Value": "GW_POS_3"
},
"LocationTypeOfLocation": {
"Value": "Position",
"StringValue": "Position"
},
"LocationUserType": {
"Value": "RecordsWorker",
"StringValue": "Records Co-ordinator"
},
"TrimType": "Location",
"Uri": 64092
}
],
"PropertiesAndFields": {},
"TotalResults": 1,
"MinimumCount": 0,
"Count": 0,
"HasMoreItems": false,
"TrimType": "Location",
"ResponseStatus": {}
}
Is the error message just misleading? The structure is identical aside from the second (working) payload not having some of the fields present in the class. I'd expect this one to error if anything.
For what its worth I've also included the OrgSearchResult class below:
final class OrgSearchResult implements Serializable {
#JsonProperty("TrimType") private String trimType;
#JsonProperty("Uri") private String uri;
#JsonIgnore private Map<String, Object> additionalProperties = new HashMap<String, Object>();
//getters and setters
A lot of troubleshooting. I've even tried to use ignore properties can't seem to get them to work.
Full error:
org.codehaus.jackson.map.exc.UnrecognizedPropertyException:
Unrecognized field "Results" (Class
sailpoint.doet.contentmanager.ContentManagerResponse), not marked as
ignorable at [Source: java.io.StringReader#5c6648b0; line: 1, column:
13] (through reference chain:
sailpoint.doet.contentmanager.ContentManagerResponse["Results"])
You can improve readability of POJO class by using PropertyNamingStrategy.UPPER_CAMEL_CASE strategy. Also, you can use JsonAnySetter annotation to read all extra properties. Below example shows how model could look like:
import com.fasterxml.jackson.annotation.JsonAnySetter;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.PropertyNamingStrategy;
import java.io.File;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class JsonApp {
public static void main(String[] args) throws Exception {
File jsonFile = new File("./resource/test.json").getAbsoluteFile();
ObjectMapper mapper = new ObjectMapper();
mapper.setPropertyNamingStrategy(PropertyNamingStrategy.UPPER_CAMEL_CASE);
System.out.println(mapper.readValue(jsonFile, ContentManagerResponse.class));
}
}
class ContentManagerResponse {
private List<OrgSearchResult> results;
private Map<String, Object> propertiesAndFields;
private Integer totalResults;
private String countStringEx;
private Integer minimumCount;
private Integer count;
private Boolean hasMoreItems;
private String searchTitle;
private String hitHighlightString;
private String trimType;
private Map<String, Object> responseStatus;
// getters, setters, toString
}
class OrgSearchResult {
private String trimType;
private String uri;
private Map<String, Object> additionalProperties = new HashMap<>();
#JsonAnySetter
public void additionalProperties(String name, Object value) {
additionalProperties.put(name, value);
}
// getters, setters, toString
}
For first JSON payload above code prints:
ContentManagerResponse{results=[OrgSearchResult{trimType='Location', uri='1684', additionalProperties={}}], propertiesAndFields={}, totalResults=1, countStringEx='1 Location', minimumCount=1, count=0, hasMoreItems=false, searchTitle='Locations - type:Organization and id:24221', hitHighlightString='', trimType='Location', responseStatus='{}'}
For second JSON payload above code prints:
ContentManagerResponse{results=[OrgSearchResult{trimType='Location', uri='64092', additionalProperties={LocationSortName={Value=GW_POS_3}, LocationUserType={Value=RecordsWorker, StringValue=Records Co-ordinator}, LocationIsWithin={Value=true}, LocationTypeOfLocation={Value=Position, StringValue=Position}}}], propertiesAndFields={}, totalResults=1, countStringEx='null', minimumCount=0, count=0, hasMoreItems=false, searchTitle='null', hitHighlightString='null', trimType='Location', responseStatus='{}'}
You do not need to implement Serializable interface.

Deserialize a varying number of objects into a list in Java using Jackson

My Spring Boot app makes a call to a REST API and receives a JSON with a varying number of entities. E.g.
{
"content": {
"guest_1": {
"name": {
"firstName": "a",
"lastName": "b"
},
"vip": false
},
"guest_2": {
"name": {
"firstName": "c",
"lastName": "d"
},
"vip": false
},
...more guests omitted...
}
}
There can be 1 to many guests and I don't know their number upfront. As you can see, they aren't in an array, they are objects instead.
I'd like to avoid deserializing into a class like
public class Content {
#JsonProperty("guest_1")
private Guest guest1;
#JsonProperty("guest_2")
private Guest guest2;
// More Guests here each having their own field
}
What I'd like to use is
public class Content {
private List<Guest> guests;
}
The #JsonAnySetter annotation I read about at https://www.baeldung.com/jackson-annotations looks promising but I couldn't get it to work.
3.2. Convert to an object at https://www.baeldung.com/jackson-json-node-tree-model looks also good but it didn't work out either.
I'm not sure if I can make Jackson do this in a declarative way or I should write a custom JsonDeserializer. Could you please help me?
#JsonAnySetter will work as it allows to specify a POJO type as second parameter. You could recreate the example JSON as, omitting setXXX() and getXXX() methods on POJOs for clarity:
private static class Content {
private Guests content;
}
private static class Guests {
private List<Guest> guests = new ArrayList<>();
#JsonAnySetter
private void addGuest(String name, Guest value) {
guests.add(value);
}
}
private static class Guest {
private Name name;
private boolean vip;
}
private static class Name {
private String firstName;
private String lastName;
}
With your JSON example will produce:
Content root = new ObjectMapper().readValue(json, Content.class);
root.getContent().getGuests().stream()
.map(Guest::getName)
.map(Name::getFirstName)
.forEach(System.out::println); // a, c

Java controller getting only first item from Set via JSON

Trying to save One to Many JPA relationship. I have written a custom controller. I am getting only the first id in giftSet and not all the ids. I have simplified the code.
My Post request-
{
"name": "Project 7",
"giftSet": [
{
"id": "1"
},
{
"id":"33"
}
]
}
class Holiday{
private String name;
private Set<GiftConfig> giftSets;
}
class GiftSet {
private Integer id;
private Holiday holiday;
}
class GiftConfig {
private Integer id;
private String name;
}
#RequestMapping(method = RequestMethod.POST, value="/api/saveholiday")
public ResponseEntity<Map<String, Holiday>> saveHoliday(#RequestBody Holiday holiday) {
System.out.println(holiday);
return null;
}
First, I add multiple GiftConfig. After that, while creating Holiday, I add details for GiftSet as well.
In debug mode, I see only id 1 in giftSet and not both ids 1 and 33.
Note- Changing Set to List is not an option.
Introduction
I see 2 problems and one possible last issue.
You are missing setters/getters in order for de-serialization to work on the JSON.
Your payload doesn't seem to be working for me.
As pcoates mentioned in a comment, you could also use #JsonAutoDetect(fieldVisibility = Visibility.ANY) - but I haven't tested this.
Finally, also be careful about having a circular reference if you convert from java back to JSON. I see that a Holiday has a set of giftSets, but a giftSet points to a holiday.
If the gitset points to the same parent holiday, this is a circular reference and will crash.
Getters and Setters
Your problem is that you are missing getters and setters.
Either use lombok and add a #data annotation or add a getter and setter .
#Data
public static class Holiday{
private String name;
private Set<GiftSet> giftSets;
}
#Data
public static class GiftSet {
private Integer id;
private Holiday holiday;
}
Payload
I used the following payload:
{
"name": "HolidaySet",
"giftSets": [
{
"id": 1111,
"holiday": {
"name": null,
"giftSets": null
}
},
{
"id": 1112,
"holiday": {
"name": null,
"giftSets": null
}
}
]
}
Quick Test
I did a quick test to see what the payload should be like.
#RequestMapping(method = RequestMethod.POST, value="/api/saveholiday")
public ResponseEntity<Map<String, Holiday>> saveHoliday(#RequestBody Holiday holiday) throws JsonProcessingException {
System.out.println(holiday);
fakeItTest();
return null;
}
private void fakeItTest() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper();
Set<GiftSet> giftSets2 = new HashSet<>();
GiftSet gg = new GiftSet();
gg.setId(1111);
gg.setHoliday(new Holiday());
giftSets2.add(gg);
GiftSet gg2 = new GiftSet();
gg2.setId(1112);
gg2.setHoliday(new Holiday());
giftSets2.add(gg2);
Holiday holiday2 = new Holiday();
holiday2.setName("HolidaySet");
holiday2.setGiftSets(giftSets2);
String a = objectMapper.writeValueAsString(holiday2);
System.out.println(a);
}
#Data
public static class Holiday{
private String name;
private Set<GiftSet> giftSets;
}
#Data
public static class GiftSet {
private Integer id;
private Holiday holiday;
}

Spring CrudRepository query with child element?

i have couchbase document like following
{
"contentTimestamp": 1470216079085,
"version": 12,
"content": [
{
"text": "ABC",
"params": {
"TYPE": "TEXT"
}
}
],
"readers": {
"u_id_1": 0,
"u_id_2": 0,
},
"contributors": [
{
"id": "u_id_1"
}
]
}
Document class
#Document
public class ContentDoc implements Serializable{
private static final long serialVersionUID = 1L;
#Id
private String id;
#Field
private Integer version = 12;
#Field
private List<Content> content = new ArrayList<>();
#Field
private Map<String, Object> readers = new HashMap<>();
//etc
//getter setter
}
Service
#Service
public interface ContentDocRepository extends CrudRepository<ContentDoc, String> {
public List<ContentDoc> findByReadersIn(String reader) throws Exception;
}
Testcase
#RunWith(SpringJUnit4ClassRunner.class)
public class Tests {
#Autowired
private ContentDocRepository contentDocRepository;
#Test
public void cotentDocRepoTest(){
List<ContentDoc> contents = contentDocRepository.findByReadersIn("u_id_1");
Assert.assertNotNull(contents);
System.out.println(contents)
}
}
I wrote code as per above but not able to retrieve result always got empty arraylist.
Anyone knows what going wrong with my code and how can i execute query with child element?
Thanks in advances.
After long RND and experiment i got solution,
we dont have way to finding child element with method name so we need
to do as per my following answer
Steps :
Create custom view in couchbase as per following
viewname : findContentByUser
function (doc, meta) {
if(doc._class == "package.model.ContentDoc") {
for(var i=0; i < doc.contributors.length; i++){
emit(doc.contributors[i].id, null);
}
}
}
Repository : binding viewname and designDocument with impl method as per following
#Repository
public interface ContentDocRepository extends CrudRepository<ContentDoc, String> {
#View(viewName = "findContentByUser", designDocument="dev_content")
public List<ContentDoc> findByContributors_id(String id);
}
Finally Got result :)
You don't need to create a view anymore, just use the #N1qlPrimaryIndexed and #ViewIndexed annotations and it should work out-of-the-box:
#N1qlPrimaryIndexed
#ViewIndexed(designDoc = "building")
public interface BuildingRepository extends
CouchbasePagingAndSortingRepository<Building, String> {
List<Building> findByCompanyId(String companyId);
}
I answered a very similar question here Using IN clause in couchbase N1Ql #query or use findAll(keys) from couchbase JPA
And you can follow my tutorial here:
https://blog.couchbase.com/couchbase-spring-boot-spring-data/
#Service
public interface ContentDocRepository extends CrudRepository<ContentDoc, String> {
#View(viewName = "findContentByUser", designDocument="dev_content")
public List<ContentDoc> findByContributors_id(String id) throws Exception;
}

RoboSpice persist JSON array with OrmLite

I'm using RoboSpice with Spring for Android and would like to persist a JSON array of objects with OrmLite. GSON is used for the JSON marshalling. With the default caching everything works as expected. But OrmLite doesn't seem to like the array of objects.
This is a simplified version of the JSON:
[{"id": 1, "title": "Test 1"},{"id": 2, "title": "Test 3"},{"id": 3, "title": "Test 3"}]
I would like to persist this in the following object:
#DatabaseTable
public class Foo {
#DatabaseField(id = true)
private int id;
#DatabaseField
private String title;
// getters and setters
...
}
Based on the RoboSpice OrmLite example I've created the following GsonSpringAndroidSpiceService class to add the OrmLite CacheManager. This is where the problem starts.
public class CustomGsonSpringAndroidSpiceService extends GsonSpringAndroidSpiceService
{
#Override
public CacheManager createCacheManager(Application application)
{
// add persisted classes to class collection
List<Class<?>> classCollection = new ArrayList<Class<?>>();
classCollection.add(Foo.class);
// init
CacheManager cacheManager = new CacheManager();
cacheManager.addPersister(new InDatabaseObjectPersisterFactory(
application, new RoboSpiceDatabaseHelper(
application, "database.db", 1), classCollection));
return cacheManager;
}
}
This results in the following error:
RequestProcessor.java:174(22356): java.lang.RuntimeException: Class [Lcom.example.model.Foo; is not handled by any registered factoryList
When I change classCollection.add(Foo.class); to classCollection.add(Foo[].class);
I get the following error:
RequestProcessor.java:174(22601): 14:42:23.112 pool-5-thread-1 An unexpected error occured when processsing request CachedSpiceRequest [requestCacheKey=foo, cacheDuration=-1, spiceRequest=com.example.app.FooRequest#4055df40]
RequestProcessor.java:174(22601): java.lang.IllegalArgumentException: No fields have a DatabaseField annotation in class [Lcom.example.app.model.Foo;
Anyone an idea how to handle a JSON array with the OrmLite CacheManager ?
I've found a work around to this problem. I added an extra result object which holds the array of objects. Off course this is only possible if you are able to manipulate the JSON. Still not really happy with this because I have introduce an useless class to my model.
So my the JSON looks like:
{
"id": 1,
"result":[{"id": 1, "title": "Test 1"},{"id": 2, "title": "Test 3"},{"id": 3, "title": "Test 3"}]
}
And I added the following class to hold the JSON result:
#DatabaseTable
public class FooResult {
#DatabaseField(id = true)
private int id;
#ForeignCollectionField(eager = false)
private Collection<Foo> result;
// getters and setters
...
}
Also added the foreign relation the the Foo class:
#DatabaseTable
public class Foo {
#DatabaseField(id = true)
private int id;
#DatabaseField
private String title;
#DatabaseField(foreign = true)
private FooResult result;
// getters and setters
...
}
I have found the way which works for me. I did not changed my json.
#DatabaseTable(tableName = SampleContract.Contributor.TABLE)
public class Contributor {
#DatabaseField(generatedId = true, columnName = SampleContract.Contributor._ID)
private int id;
#DatabaseField(columnName = SampleContract.Contributor.LOGIN)
public String login;
#DatabaseField(columnName = SampleContract.Contributor.CONTRIBUTIONS)
public int contributions;
#DatabaseField(foreign = true)
private ContributorsResult result;
#SuppressWarnings("serial")
#DatabaseTable(tableName = "contributor_list")
public static class ContributorsResult extends ArrayList<Contributor> {
#DatabaseField(id = true)
private int id = 0;
#ForeignCollectionField(eager = false)
private Collection<Contributor> result = this;
public Collection<Contributor> getResult() {
return result;
}
public void setResult(Collection<Contributor> result) {
if (result != null) {
this.clear();
this.addAll(result);
}
}
}
}
I struggled with this the whole of today and finally figured how to save a JSON array into a SQLite database using Robospice Spring Android without modifying the JSON.
This is my post JSON array returned from my server:
[
{
"id": "5573547af58cd75df03306cc",
"name": "Simon",
"postheader": "First Post"
},
{
"id": "55735475f58cd75df03306cb",
"name": "Tyron",
"postheader": "Second Post"
}
]
This is similar to the JSON in this question:
[{"id": 1, "title": "Test 1"},{"id": 2, "title": "Test 3"},{"id": 3, "title": "Test 3"}]
Step 1:
You will need to create 2 classes on the android side.
One will be the normal object that you want to save from the array. In my case, I have an object called "Post" and my server returns an array of "Post".
#DatabaseTable(tableName = "post")
#JsonIgnoreProperties(ignoreUnknown = true)
public class Post implements Serializable {
#DatabaseField(id = true)
private String id;
#DatabaseField
private String name;
#DatabaseField
private String postheader;
#DatabaseField(foreign = true,foreignAutoCreate = true,foreignAutoRefresh = true)
private EmbedPost posts;
The other object will be a wrapper object that wraps over the array, I called mine "EmbedPost".
#DatabaseTable
public class EmbedPost implements Serializable {
#DatabaseField(allowGeneratedIdInsert=true, generatedId=true)
private int ID;
#ForeignCollectionField(eager = false)
private Collection<Post> posts;
By defining an int called ID in my EmbedPost class, I'm effectively creating a object that if I converted to JSON would look like this:
{"id": 1, "posts": [
{
"id": "5573547af58cd75df03306cc",
"name": "Simon",
"postheader": "First Post"
},
{
"id": "55735475f58cd75df03306cb",
"name": "Tyron",
"postheader": "Second Post"
}
]}
This is effectively a JSON string that looks very similar to what Uipko used in his solution.
{
"id": 1,
"result":[{"id": 1, "title": "Test 1"},{"id": 2, "title": "Test 3"},{"id": 3, "title": "Test 3"}]
}
Step 2:
You now persist it using SpringAndroidSpiceService.
public class AndroidSpiceService extends SpringAndroidSpiceService {
private static final int WEBSERVICES_TIMEOUT = 10000;
#Override
public CacheManager createCacheManager( Application application ) {
CacheManager cacheManager = new CacheManager();
List< Class< ? >> classCollection = new ArrayList< Class< ? >>();
// add persisted classes to class collection
classCollection.add(EmbedPost.class);
classCollection.add( Post.class );
// init
RoboSpiceDatabaseHelper databaseHelper = new RoboSpiceDatabaseHelper( application, "sample_database.db", 3);
InDatabaseObjectPersisterFactory inDatabaseObjectPersisterFactory = new InDatabaseObjectPersisterFactory( application, databaseHelper, classCollection );
cacheManager.addPersister( inDatabaseObjectPersisterFactory );
return cacheManager;
}
#Override
public RestTemplate createRestTemplate() {
RestTemplate restTemplate = new RestTemplate();
// set timeout for requests
HttpComponentsClientHttpRequestFactory httpRequestFactory = new HttpComponentsClientHttpRequestFactory();
httpRequestFactory.setReadTimeout( WEBSERVICES_TIMEOUT );
httpRequestFactory.setConnectTimeout( WEBSERVICES_TIMEOUT );
restTemplate.setRequestFactory( httpRequestFactory );
MappingJackson2HttpMessageConverter mappingJackson2HttpMessageConverter = new MappingJackson2HttpMessageConverter();
final List<HttpMessageConverter< ? >> listHttpMessageConverters = restTemplate.getMessageConverters();
listHttpMessageConverters.add( mappingJackson2HttpMessageConverter );
restTemplate.setMessageConverters( listHttpMessageConverters );
return restTemplate;
}
}
Be sure to add both the EmbedPost.class and Post.class to your classCollection as ORMLite cannot do its work without you persisting both. You had used ForeignKeys when you wrote up your objects and these foreign keys have to tie to something so therefore both classes must persist.
If you run into trouble, try using the logcat to figure it out. You might have to read all the messages and not just the error ones.
See my post here to see how to read all logcat messages:
Robospice storing object that extends ArrayList in database via Ormlite

Categories