I'd like to convert a flat file with many different keys to a java bean dto, as follows:
Flat file:
mykey=example
anotherkey=test
adress-street-1=downtown street
address-town-1=nyc
address-stree-2=some street
adrerss-town-2=los angeles
Target bean:
public class Content {
private String mykey;
private String anotherkey;
private List<Address> address;
}
public class Address {
private String street,
private String town;
}
Question: how could I achieve the mapping between the flat file and the target bean? Imagine a few hundred property keys. Some may occur multiple times, in whose cases they and with an index number, like address-town-1.
Content content = new Content();
while(true) {
String line = reader.readLine();
String key = line.split("=")[0];
String value = line.split("=")[1];
switch(key) {
case "mykey": content.setMykey(value);
case "anotherkey": content.setAnotherkey(value);
...
}
}
But I would have to code those mappings a few hundred times, which does not feel right.
Question: is there a better way, eg by using some kind of xml configuration/mapping file, and reflection?
BeanIO comes close to what I want to achieve. But it lacks the ability to just convert multiple different fields to the bean field name. And also it cannot group sets like the address-* example into a List.
Is there any known library or concept that I could use? Some framework that could do the formatting, using eg a xml configuration file?
Or would I have to write my own parser and use reflection to write the data to my java objects?
Related
I am working on a project, where users can write other users mails. The mail-class looks currently like this:
public class Mail
{
/*
* Fields
*/
private final String _content;
private final String _sender;
private final String _date;
private final String _topic;
/**
* Constructor of the Mail-class.
*
* #param mail The mail as a string representation.
*/
public Mail(String mail)
{
_content = ...;
_sender = ...;
_date = ...;
_topic = ...;
}
// And all the getters.
#Override
public String toString()
{
// Make the reperentation as a String
}
Now, the problem is, that I need to save all the mails into a .yml userdata file. For me, the simplest way to do this is to firstly convert every mail to a String and then save all mails into a List<String> mails which will be written directly into the .yml where I can access and read the list of String with ease. Now, once I start the server, all mails should be converted back into the object-representation. The question is, how can I save the mail-Object as a String and load it, that I can operate using those mail objects.
My first ideas were something like this sender#date#topic#content and then splitting the String via String.split("#");. But when I do this, I need to ban the # character for the content and the topic. The date and the username will never contain #. I don't want to ban any character. Unfortunately I have nothing found on my researches so far on how to solve this task.
The best thing to do is use XML or JSON to markup the object and convert it to a string for you. There are tons of libraries you can use to do that. They all operate in roughly the same manner: add an annotations to your fields and class.
For the sake of simplicity lets assume I have a Document object with seven fields (but imagine that it can have many more). This object looks something like this:
#Getter
#Setter
public class Document {
private String fileName;
private String fileType;
private String createdBy;
private Date createdAt;
private Date lastModifiedAt;
private List<String> modifiers;
private Long timesModified;
}
I want to create an endpoint which can receive any number of #RequestParam and returns a List<Document> of all the documents which match the given query. For example: return all documents with fileType == doc, which were created between createdAt == 01/01/2021 && createdAt 31/01/2021, modified timesModified == 5 times and modifiers.contains("Alex"). The reason for this is that I want to allow the user to query for documents depending on combination of fields the user wants. Originally to handle this we created the endpoint like so:
#GetMapping(value = {RestApi.LIST})
public ResponseEntity<List<Document>> getDocuments (#RequestParam Map<String, Object> optionalFilters) {
List<Document> documents = documentService.getListOfDocuments(optionalFilters);
if (documents != null) {
return new ResponseEntity<>(documents, HttpStatus.OK);
}
return new ResponseEntity<>(HttpStatus.NOT_FOUND);
}
The problem with this is that because we use optionalFilters as Map<String, Object> this requires us to perform a lot of casting in our code and overall makes our code very tedious and cumbersome because we have to iterate through the whole map and create a custom query depending the fields that were passed. In order to try and improve this I created an OptionalFilters object:
#Getter
#Setter
#NoArgsConstructor
public class OptionalFilters {
private String fileName;
private String fileType;
private String createdBy;
private Date createdAt;
private Date lastModifiedAt;
private List<String> modifiers;
private Long timesModified;
}
And modified the endpoint to this:
#GetMapping(value = {RestApi.LIST})
public ResponseEntity<List<Document>> getDocuments (#Valid OptionalFilters optionalFilters) {
List<Document> documents = documentService.getListOfDocuments(optionalFilters);
if (documents != null) {
return new ResponseEntity<>(documents, HttpStatus.OK);
}
return new ResponseEntity<>(HttpStatus.NOT_FOUND);
}
However, although that this simplifies the way we receive the parameters and extract the values from them, we still need to iterate through all the parameters and create a custom query. Is there some way to elevate and take advantage of Spring-Data (or any other solution) so that I don't have to create a custom query depending on each query param that is passed through? I am using Solr as the repository if this may be any help.
Using Query by Example is one the most simple option but it has its limitations. Excerpt from the above link:
Limitations
Like all things, the Query by Example API has some limitations. For instance:
Nesting and grouping statements are not supported, for example:
(firstName = ?0 and lastName = ?1) or seatNumber = ?2
String matching only includes exact, case-insensitive, starts, ends, contains, and regex
All types other than String are exact-match only
Query by Example is suitable choice if your filtering is never too complicated. But when restirictions like above hit the fan of your CPU cooler the choice is to use Specifications to construct queries.
One big difference is also that Using Query by Example you need to explicitly populate the example by its getters and setters. With specification you can make it in a generic way (with Java generics) using just use field names
In your case you could just pass the map to generic method and create filtering by just looping and adding by and (note that the link's example has static stuff mostly but it has not to be, you just need field name/criterion -pair to loop it in a generic way)
With specifications you can do anything that can be done with Query by Example and almost anything else also. The overhead to get familiar with specifications might be bigger but the advantage using specifications will be rewarding.
In a nutshell:
Spring interface Specification is based on JPA CriteriaQuery and for each you need only to implement one method:
Predicate toPredicate (Root<User> root, CriteriaQuery<?> query, CriteriaBuilder builder);
Repository interfaces needs just to extend JpaSpecificationExecutor<YourClass> When you have a set of predicates, you can - for example -
repository.findAll(Specification.where(spec1).and(spec2));
It might seem complicated or difficult at start but it is not that at all. The greatest advantage with Specification is that you can do almost anything programmatically instead of manipulating JPQL queries or so.
I couldn't find a better title (feel free to edit it if you find a better one), but the use case is the following. I have two lists of constants. One of those contains the constants I use in my application, the other contains the different constants that are sent to me via a CSV file (along with data).
To give a rough exemple : in the CSV file, there is a field called "id of the client". In my application, I want to use a field called "clientId". So I basically need to create a static link between the two constants, so that I can easily switch from one to the other depending on what I need to achieve.
I've thought about creating a static Map(String, String) of values, but I figured there might be better solutions.
Thanks !
EDIT : changed title to "N" constants instead of 2, because Hashmap doesn't seem to be an option any longer in that case.
you can use the double bracket innitializer idiom to keep map initialization close to the map declaration, so it would be not so "ugly" eg:
static Map<String, String> someMap = new HashMap<String, String>() {{
put("one", "two");
put("three", "four");
}};
Beware that without the static modifier each anonymous class (there is one created in this example) holds a refernce to the enclosing object and if you'll give a reference to this map to some other class it will prevent the enclosing class from being garbage collect.
Fortunatelly, there is a hope for us with java update, in java 9 there will be very handy Map.of() to help us do it more safely.
The best way to separate the mapping from your application code is to use a properties file where in which you define your mapping.
For example, you could have a csv-mapping.properties in the root of your resources and load them with the following code:
final Properties properties = new Properties();
properties.load( this.getClass().getResourceAsStream( "/csv-mapping.properties" ) );
This will work just like a Map, with the added separation of code from configuration.
There are many methods that you can use to easily solve these types of problem.
One way is to use a Properties file, or file containing the key value pair.
Here is the code for Properties.
import java.util.ResourceBundle;
public class ReadingPropertiesFile {
public static void main(String[] args) {
ResourceBundle messages;
messages = ResourceBundle.getBundle("msg");
System.out.println(messages.getString("ID"));
}
}
msg.properties file contains values::
ID = ClientID.
PRODUCT_ID = prod_ID
The output of the program is ClientID.
You can also read from a simple text file. Or you could use the map as you are using. But I would suggest you to use the properties file.
One good option would be to use an enum to create such mappings beetween multiple constants to a single common sense value, eg:
import java.util.Arrays;
import java.util.Collections;
import java.util.HashSet;
import java.util.Set;
public enum MappingEnum {
CLIENT_ID("clientId", "id of the client", "clientId", "IdOfTheClient"),
CLIENT_NAME("clientName", "name of the client", "clientName");
private Set<String> aliases;
private String commonSenseName;
private MappingEnum(String commonSenseName, String... aliases) {
this.commonSenseName = commonSenseName;
this.aliases = Collections.unmodifiableSet(new HashSet<String>(Arrays.asList(aliases)));
}
public static MappingEnum fromAlias(String alias) {
for (MappingEnum mappingEnum : values()) {
if (mappingEnum.getAliases().contains(alias)) {
return mappingEnum;
}
}
throw new RuntimeException("No MappingEnum for mapping: " + alias);
}
public String getCommonSenseName() {
return commonSenseName;
}
}
and then you can use it like:
String columnName = "id of the client";
String targetFieldName = MappingEnum.fromAlias(columnName).getCommonSenseName();
I have a property file (a.txt) which has the values (Example values given below) like below
test1=10
test2=20
test33=34
test34=35
By reading this file, I need to produce an output like below
value = 35_20_34_10
which means => I have a pattern like test34_test2_test33_test1
Note, If the 'test33' has any value other than 34 then I need to produce the value like below
value = 35_20_10
which means => I have a pattern like test34_test2_test1
Now my problem is, every time when the customer is making the change in the logic, I am making the change in the code. So what I expect is, I want to keep the logic (pattern) in another property file so I will be sending the two inputs to the util (one input is the property file (A.txt) another input will be the 'pattern.txt'),
My util has to be compare the A.txt and the business logic 'pattern.txt' and produce the output like
value = 35_20_34_10 (or)
value = 35_20_10
If there an example for such pattern based logic as I expect?
Any predefined util / java class does this?
Any help would be Great.
thanks,
Harry
First of all, svasa's answer makes a lot of sense, but covers different level of
abstraction. I recommend you read his answer too, that pattern should
be useful.
You may wanna look at Apache Velocity and FreeMarker libraries to see how they structure their API.
Those are template engines - they usually have some abstraction of pattern or format, and abstraction of variable/value binding (or namespace, or source). You can render a template by binding it with a binding/namespace, which yields the result.
For example, you may wanna have a pattern "<a> + <b>", and binding that looks like a map: {a: "1", b: "2"}. By binding that binding to that pattern you'll get "1 + 2", when interpreting <...> as variables.
You basically load the pattern from your pattern.txt, then load your data file A.txt (for example, by treating it as properties and using Properties class) and construct binding based on these properties. You'll get your output and possibility to customize the pattern all the time.
You may call the sequences like test34_test2_test33_test1 as a pattern, let me call them as constraints when building something.
To me this problem best fits into a
builder pattern.
When building the value you want, you tell the builder that these are my constraints(pattern) and these are my original properties like below:
new MyPropertiesBuilder().setConstraints(constraints).setProperties(original).buildValue();
Details:
Set some constraints in a separate file where you specify the order of the properties and their values like :
test34=desiredvalue-could-be-empty
test2=desiredvalue-could-be-empty
test33=34
test1=desiredvalue-could-be-empty
The builder goes over the constraints in the order specified, but get the values from the original properties and build the desired string.
One way to achieve your requirement through builder pattern is to define classes like below :
Interface:
public interface IMyPropertiesBuilder
{
public void setConstraints( Properties properties );
public void setProperties( Properties properties );
public String buildValue();
}
Builder
public class MyPropertiesBuilder implements IMyPropertiesBuilder
{
private Properties constraints;
private Properties original;
#Override
public void setConstraints( Properties constraints )
{
this.constraints = constraints;
}
#Override
public String buildValue()
{
StringBuilder value = new StringBuilder();
Iterator it = constraints.keySet().iterator();
while ( it.hasNext() )
{
String key = (String) it.next();
if ( original.containsKey( key ) && constraints.getProperty( key ) != null && original.getProperty( key ).equals( constraints.getProperty( key ) ) )
{
value.append( original.getProperty( key ) );
value.append( "_" );
}
}
return value.toString();
}
#Override
public void setProperties( Properties properties )
{
this.original = properties;
}
}
User
public class MyPropertiesBuilderUser
{
private Properties original = new Properties().load(new FileInputStream("original.properties"));;
private Properties constraints = new Properties().load(new FileInputStream("constraints.properties"));
public String getValue()
{
String value = new MyPropertiesBuilder().setConstraints(constraints).setProperties(original).buildValue();
}
}
I'm trying to use gson to convert this returned JSON into some kind of
data structure such that I can extract useful data.
For Example:
http://search.twitter.com/search.json?q=test&rpp=1
Returns:
{
"completed_in":0.028,
"max_id":196386333906837504,
"max_id_str":"196386333906837504",
"next_page":"?page=2&max_id=196386333906837504&q=test&rpp=1",
"page":1,
"query":"test",
"refresh_url":"?since_id=196386333906837504&q=test",
"results":[
{
"created_at":"Sat, 28 Apr 2012 23:52:05 +0000",
"from_user":"della_ky",
"from_user_id":525641596,
"from_user_id_str":"525641596",
"from_user_name":"kydella modeste",
"geo":null,
"id":196386333906837504,
"id_str":"196386333906837504",
"iso_language_code":"en",
"metadata":{
"result_type":"recent"
},
"profile_image_url":"http:\/\/a0.twimg.com\/profile_images\/2159990525\/webcam-toy-photo3_20_2__normal.jpg",
"profile_image_url_https":"https:\/\/si0.twimg.com\/profile_images\/2159990525\/webcam-toy-photo3_20_2__normal.jpg",
"source":"<a href="http:\/\/mobile.twitter.com" rel="nofollow">Mobile Web<\/a>",
"text":"RT #Y__U__NOOO: #SongsIKnowOffByHeart ALL SONGS I LISTEN TO. BRAIN, Y U NO REMEMBER TEST ANSWERS LIKE THAT?!?",
"to_user":null,
"to_user_id":null,
"to_user_id_str":null,
"to_user_name":null
}
],
"results_per_page":1,
"since_id":0,
"since_id_str":"0"
}
Ultimately, I would like to be able to output a list of tweets with the
name of the sender and the date/time of the tweet.
I have read through the gson documentation but it's going over my head
to be honest - lots of new concepts there for me.
Do I need to define a class which maps exactly to the structure of the
JSON in order to then populate an instance of that class? If so this
seems very inflexible/laborious. Ideally I'm looking for something
which will handle JSON in any form and give me a structure I can use
automatically...
Is anyone able to give me some pointers? Being new to this - the more
detailed and in words of the fewest syllables the better!
Update - Thanks to the responses I've already had on this I've had a go at putting a class together to capture the twitter JSON. However, since the JSON has an embedded ArrayList of Objects I'm struggling a bit... So far I have
public class tweetData {
private double completed_in;
private long max_id;
private long max_id_str;
private String next_page;
private int page;
private String query;
private String refresh_url;
private List<tweetDetails> tweets = new ArrayList<tweetDetails>();
}
and
public class tweetDetails {
private String created_at;
private String from_user;
private long from_user_id;
private long from_user_id_str;
private String from_user_name;
private String geo;
private long id;
private long id_str;
private String iso_language_code;
// "metadata":
// {
// "result_type":"recent"
// },
private String profile_image_url;
private String profile_image_url_https;
private String source;
private String text;
private String to_user;
private String to_user_id;
private String to_user_id_str;
private String to_user_name;
}
Which I'm instantiating with
URI uri = new URI("http", "search.twitter.com", "/search.json", "q="+ searchTerms + "&rrp=" + RRP, null);
URL twitterSearch = uri.toURL();
URLConnection yc = twitterSearch.openConnection();
JsonReader reader = new JsonReader(new InputStreamReader(yc.getInputStream()));
Gson gson = new Gson();
tweetData data = gson.fromJson(reader, tweetData.class);
System.out.println(data);
The basic name:values are being populated correctly but the ArrayList is not.
tweetData : 0.17196614959919140865196614959919140865?page=2&max_id=196614959919140865&q=test1test?since_id=196614959919140865&q=testSIZE 0[]
So, I'm still struggling a bit - any more tips hugely appreciated!
Tia,
Tom
Do I need to define a class which maps exactly to the structure of the JSON in order to then populate an instance of that class? If so this seems very inflexible/laborious.
Yes. GSON is a library that can be used to convert Java Objects into their JSON representation. It can also be used to convert a JSON string to an equivalent Java object. This is really powerful because you can automagically instantiate your Java objects from the JSON representation. Assuming your JSON doesn't change its structure, you only have to define the appropriate Java object representation once.
Ideally I'm looking for something which will handle JSON in any form and give me a structure I can use automatically...
However, if you don't want automagical serialisation/deserialisation, then try looking at a simpler library such as java.net/projects/jsonp.
You can extract stuff from it just by querying the keys:
final JSONObject json = new JSONObject(theJsonString);
final String id = json.getString("max_id");
final JSONArray results = json.getJSONArray("results");
final String user = results.getJSONObject(2).getString("from_user");
Gson actually does all the serialization for you. So yes, you would have to write the classes yourself. To you, this seams inflexible and laborious, but that's only because that library isn't made for what you're asking for (it doesn't parse 'arbitrary' JSON).
I would suggest at least considering writing the classes and using gson. The reason I say that is because either way your application's logic will have to expect a very specific format, and writing out that format in a Java class will make things tidier. Here's a nice guide that will help you get started that way.
If you want to simply decode the JSON without serializing it into a Java class (IMHO the only way to use 'arbitrary' JSON), you'll want to use another library. Try this one. It allows you to decode the JSON, and use it by getting values from it (as described in this question: Convert a JSON string to object in Java ME?).
There are some tools that do gson to schema mapping. You give some sample JSON responses, and the java classes to access them are created for you.
http://www.jsonschema2pojo.org/
Gson is a slick beast!
Or at least it became so over the years that have passed since the question had been asked.
You can pass it an Object.class as a second parameter to the fromJson() method and it will parse your Json into a reasonable structure of LinkedTreeMaps and ArrayLists.
Object result = (new Gson()).fromJson(jsonString, Object.class)
More than that, you can really do partial parsing and leave loose ends at any level of your object structure by defining a certain field as Object!
Gson will then parse Json into your structure and your field of type Object will contain the above mentioned structure of LinkedTreeMaps and ArrayLists.
E.g., you may define a class
Person {
String name;
Object details;
}
(Imagine, you care mostly about the person's name but may want the details also somewhere. To log them, for instance.)
Then you can pass the following Json to the fromJson(input, Person.class) method as a first parameter
{
"name": "Carlsson",
"details": {
"address": "Stockholm",
"phones": [
"work": "233-322-233-322",
"home": "none"
]
}
}
The result will have the name field filled with "Carlsson" string and details field will contain a LinkedTreeMap with keys "address" and "phones", etc.