In our applications we are using property files very much. Since a few months I have started to learn Guava and I liked it a lot actually.
What is the best way to create a Map<String, Datasource> ?
The property file format is not strict. It can be changed if It can be expressed better with another format?
Sample property file:
datasource1.url=jdbc:mysql://192.168.11.46/db1
datasource1.password=password
datasource1.user=root
datasource2.url=jdbc:mysql://192.168.11.45/db2
datasource2.password=password
datasource2.user=root
The easiest thing is probably to use JSON rather than a properties file for this:
{
"datasources": [
{
"name": "datasource1",
"url": "jdbc:mysql://192.168.11.46/db1",
"user": "root",
"password": "password"
},
{
"name": "datasource2",
"url": "jdbc:mysql://192.168.11.46/db2",
"user": "root",
"password": "password"
}
]
}
Then you can just use a library such as Gson to convert that into objects:
public class DataSources {
private List<DataSourceInfo> dataSources;
public Map<String, DataSource> getDataSources() {
// create the map
}
}
public class DataSourceInfo {
private String name;
private String url;
private String user;
private String password;
// constructor, getters
}
Then to get the map:
Gson gson = new Gson();
Map<String, DataSource> dataSources = gson.fromJson(/* file or stream here */,
DataSources.class).getDataSources();
Properties class is a subclass of HashTable, which in turn implements Map.
You load it as usual with:
Properties properties = new Properties();
try {
properties.load(new FileInputStream("filename.properties"));
} catch (IOException e) {
}
edit: Ok so you want to transform it to Map<String, Datasource> ;)
//First convert properties to Map<String, String>
Map<String, String> m = Maps.fromProperties(properties);
//Sort them so that password < url < user for each datasource and dataSource1.* < dataSource2.*. In your case default string ordering is ok so we can take a normal treemap
Map<String, String> sorted = Maps.newTreeMap();
sorted.putAll(m);
//Create Multimap<String, List<String>> mapping datasourcename->[password,url, user ]
Function<Map.Entry<String, String>, String> propToList = new Function<String, Integer>() {
#Override
public String apply(Map.Entry<String, String> entry) {
return entry.getKey().split("\\.")[0];
}
};
Multimap<Integer, String> nameToParamMap = Multimaps.index(m.entrySet(), propToList);
//Convert it to map
Map<String, Collection<String>> mm = nameToParamMap.asMap();
//Transform it to Map<String, Datasource>
Map<String, Datasource> mSD = Maps.transformEntries(mm, new EntryTransformer<String, Collection<String>, DataSource>() {
public DataSource transformEntry(String key, Collection<String> value) {
// Create your datasource. You know by now that Collection<String> is actually a list so you can assume elements are in order: [password, url, user]
return new Datasource(.....)
}
};
//Copy transformed map so it's no longer a view
Map<String, Datasource> finalMap = Maps.newHashMap(mSD);
There's probably an easier way, but this should work :)
Still you're better off with json or xml. You can also load properties of different datasources from different files.
edit2: with less guava, more java:
//Sort them so that password < url < user for each datasource and dataSource1.* < dataSource2.*. In your case default string ordering is ok so we can take a normal SortedSet
SortedSet <String> sorted = new SortedSet<String>();
sorted.putAll(m.keySet);
//Divide keys into lists of 3
Iterable<List<String>> keyLists = Iterables.partition(sorted.keySet(), 3);
Map<String, Datasource> m = new HashMap<String, Datasource>();
for (keyList : keyLists) {
//Contains datasourcex.password, datasroucex.url, datasourcex.user
String[] params = keyList.toArray(new String[keyList.size()]);
String password = properties.get(params[0]);
String url = properties.get(params[1]);
String user = properties.get(params[2]);
m.put(params[0].split("\\.")[0], new DataSource(....)
}
If the file that you are using for configuration is not strict, you could use an XML file to store the defintion.
Example definition:
<resources>
<configuration>
<datasrouce>
<connection name="" url="" password="" user=""/>
<connection name="" url="" password="" user=""/>
</datasource>
</configuration>
</resources>
The using a Connection manager class you could just read the XML to obtain connection info and create an instance of connections and mange them.
Related
I want to create the following object for the response but I don't want to create the class for this. Is there any way I can achieve this using map or other util classes?
"conferenceData": {
"createRequest": {
"conferenceSolutionKey": {
"type": "hangoutsMeet"
},
"requestId": "RANDOM_STRING2"
}
}
I think below is the solution you are looking for...
Map<String, Object> conferenceData = new HashMap<>();
Map<String, Object> createRequest = new HashMap<>();
Map<String, Object> keyAndRequestId= new HashMap<>();
Map<String, String> type = new HashMap<>();
type.put("type", "hangoutsMeet");
keyAndRequestId.put("conferenceSolutionKey", type);
keyAndRequestId.put("requestId", "RANDOM_STRING2");
createRequest.put("createRequest", keyAndRequestId);
conferenceData.put("conferenceData", createRequest);
Try to use :
Map<String,Object> result = new ObjectMapper().readValue(JSON_SOURCE, HashMap.class);
Or if you are using org.json, JSONObject has a method toMap(). You can easily do :
Map<String, Object> myMap = myJsonObject.toMap();
So, I have a value type :
class Session {
long createdAt;
List<String> postIds;
}
Using the jedis client(3.0.0-m1 is that matters), I am currently performing an hset to create the entries and hgetAll to retrieve all the key-values:
private redis.clients.jedis.Jedis jedis;
void createSession(String idAsKey, Map<String, String> hashFieldValues) {
jedis.hset(idAsKey, hashFieldValues);
}
Map<String, String> fetchSession(String idAsKey) {
return jedis.hgetAll(idAsKey);
}
The challenge that I am currently looking at is the ease of converting the Map<String, String> into the Session object. Is there an existing way to do this?
Server response for an equivalent command
1) "createdAt"
2) "1556099708307"
3) "postIds"
4) "[a, b, c]"
PS: Starting to learn Redis, hoping this kind of mapping might have already been solved for. Yes, not looking for a client change as an answer at least.
Jedis doesn't offer a way to map objects to hash structures.
If you are using spring, then you can look at HashMappers. A HashMapper converts a POJO to a hash and vice-versa. In your case, the HashMapper will convert a Session to a hash, and the other way round.
You are not using the fields separately, but simultaneously. Because of that, I'd suggest you to use plain and simple Redis Strings instead of using Redis Hashes. So you'd be using set to save entries and get to retrieve them.
Using above suggestions, your code may become as follows:
private redis.clients.jedis.Jedis jedis;
private com.google.gson.Gson gson; // see Note
void createSession(String idAsKey, Session object) {
String serializedValue = gson.toJson(object);
jedis.set(idAsKey, serializedValue);
}
Session fetchSession(String idAsKey) {
String serializedValue = jedis.get(idAsKey);
Session deserializedObject = gson.fromJson(serializedValue, Session.class);
return deserializedObject;
}
Note: I have used Gson for the purpose of serialization/deserialization. Needless to say, you can use any library.
You can convert the map to POJO
Session session = new ObjectMapper().convertValue(map, Session.class);
So you don't need a special handling expect using a mapper library as Jackson-Databind
You can save and fetch the data to and from Redis like below:
public Map<String, Object> saveDataInRedis(String id, Object obj) {
Map<String, Object> result = new HashMap<>();
String jsonObj = "";
try {
jsonObj = objectMapper.writeValueAsString(obj);
System.out.println(jsonObj);
} catch (JsonProcessingException jpe) {
logger.warn("In saveDataInRedis Exception :: "+jpe);
}
try {
valOps.set(id, jsonObj);
result.put(DataConstants.IS_SUCCESS, true);
result.put(DataConstants.MESSAGE, "Data saved succesfully in redis");
}catch(RedisConnectionFailureException e){
result =null;
logger.warn("In saveDataInRedis Exception e :: "+e);
}
System.out.println(valOps.getOperations().getClass());
System.out.println(jedisConnectionFactory.getPoolConfig().getMaxTotal());
return result;
}
Now get data from redis:
public Map<String, Object> getDataFromRedis(String id) {
Map<String, Object> result = new HashMap<>();
String jsonObj = valOps.get(id);
System.out.println("jsonObj :: " + jsonObj);
Session obj = null;
try {
obj = (Session) objectMapper.readValue(jsonObj, Session.class);
} catch (Exception e) {
result.put("data", null);
logger.warn("Data from redis is deleted");
logger.warn("In getDataFromRedis Exception e :: "+e);
}
if (obj != null) {
result.put(DataConstants.IS_SUCCESS, true);
result.put("data", obj);
}
System.out.println("result :: " + result);
return result;
}
I have this JSON response, from a remote server and i really hope i can get help.
{
"data": {
"6111": {
"prereq": "0",
"mast": "The Master Tree"
},
"6112": {
"prereq": "1",
"mast": "Another Master Tree"
}
}
}
I use GSON to parse JSON, using the #SerializedName and #Exposeto obtain the value into a custom Model. But i do not understand how to get past the
"6111"
"6112"
I have checked other questions via the gson tag, to no avail.
try this
Iterator<String> iter = json.keys();
while (iter.hasNext()) {
String key = iter.next();
try {
Object value = json.get(key);
} catch (JSONException e) {
// Something went wrong!
}
}
///////////////////update////////////////////
JSONObject issueObj = new JSONObject(jsonContent);
Iterator iterator = issueObj.keys();
while(iterator.hasNext()){
String key = (String)iterator.next();
JSONObject issue = issueObj.getJSONObject(key);
// get id from issue
String _pubKey = issue.optString("id");
}
If you're using Gson, any time you have an object with keys you don't know ahead of time, you can just use Map instead of a custom object.
In this case, each element of the Map will be some "known" data structure, so you would use Map<String, MyObject>.
Your top-level class:
public class MyResponse {
#SerializedName("data")
#Expose
private Map<String, MyObject> data;
...
}
And your map's value class:
public class MyObject {
#SerializedName("prereq")
#Expose
private String prereq;
#SerializedName("mast")
#Expose
private String mast;
...
}
In the specific case of the json text you posted, you would then be able to use these objects like this:
response.getData().get("6111").getMast();
But you can also do anything you could normally do with a Map:
Map<String, MyObject> data = response.getData();
for (String key: data.keySet() {
...
}
for (MyObject obj : data.values()) {
...
}
I have an enhanced question regarding Flatten a JSON string to Map using Gson or Jackson.
My scenario included duplicated keys, so the solution in the above question will cause some duplicated keys overwritten. So I am thinking to construct keys by combining each level's key together.
So how to achieve that?
For example:
{
"id" : "123",
"name" : "Tom",
"class" : {
"subject" : "Math",
"teacher" : "Jack"
}
}
I want to get the Map:
"id" : "123",
"name" : "Tom",
"class.subject" : "Math",
"class.teacher" : "Jack"
************************Update Solution*************************************
Based on #Manos Nikolaidis's answer, I am able to achieve the following solution by considering ArrayNode.
public void processJsonString(String jsonString) throws Exception {
ObjectMapper mapper = new ObjectMapper();
ArrayNode arrayNode = (ArrayNode) mapper.readTree(jsonString);
processArrayNode(arrayNode);
}
private void processObjectNode(JsonNode jsonNode) {
Map<String, String> result = new HashMap<>();
Iterator<Map.Entry<String, JsonNode>> iterator = jsonNode.fields();
iterator.forEachRemaining(node -> mapAppender(result, node, new ArrayList<String>()));
}
private void processArrayNode(ArrayNode arrayNode) {
for (JsonNode jsonNode : arrayNode) {
processObjectNode(jsonNode);
}
}
private void mapAppender(Map<String, String> result, Map.Entry<String, JsonNode> node, List<String> names) {
names.add(node.getKey());
if (node.getValue().isTextual()) {
String name = names.stream().collect(Collectors.joining("."));
result.put(name, node.getValue().asText());
} else if (node.getValue().isArray()) {
processArrayNode((ArrayNode) node.getValue());
} else if (node.getValue().isNull()) {
String name = names.stream().collect(Collectors.joining("."));
result.put(name, null);
} else {
node.getValue().fields()
.forEachRemaining(nested -> mapAppender(result, nested, new ArrayList<>(names)));
}
}
You can get the JSON as JsonNode and go through all fields recursively and add key and value field to a Map. When a value is an object instead of string you can add the field name to List to be joined with periods when a string is finally encountered. First create (for readability) a separate method that add Json fields to a Map:
void mapAppender(Map<String, String> result, Entry<String, JsonNode> node, List<String> names) {
names.add(node.getKey());
if (node.getValue().isTextual()) {
String name = names.stream().collect(joining("."));
result.put(name, node.getValue().asText());
} else {
node.getValue().fields()
.forEachRemaining(nested -> mapAppender(result, nested, new ArrayList<>(names)));
}
}
and use it like this:
ObjectMapper mapper = new ObjectMapper();
Map<String, String> result = new HashMap<>();
mapper.readTree(json).fields()
.forEachRemaining(node -> mapAppender(result, node, new ArrayList<String>()));
Where fields() returns an Iterator. Beware of StackOverflowErrors and perhaps low performance for deeply nested JSON.
I resolved this using below simple code, Only think is need to download jettison and flattener.JsonFlattener library
import java.util.Map;
import org.codehaus.jettison.json.JSONObject;
import com.github.wnameless.json.flattener.JsonFlattener;
public class test {
public static void main(String[] args) {
String jsonString = "{\"id\" : \"123\",\"name\" : \"Tom\",\"class\" : {\"subject\" : \"Math\",\"teacher\" : \"Jack\"}}";
JSONObject jsonObject = new JSONObject();
String flattenedJson = JsonFlattener.flatten(jsonString);
Map<String, Object> flattenedJsonMap = JsonFlattener.flattenAsMap(jsonString);
System.out.println(flattenedJsonMap);
}
}
Reference link : https://github.com/wnameless/json-flattener
I have an enhanced question regarding Flatten a JSON string to Map using Gson or Jackson.
My scenario included duplicated keys, so the solution in the above question will cause some duplicated keys overwritten. So I am thinking to construct keys by combining each level's key together.
So how to achieve that?
For example:
{
"id" : "123",
"name" : "Tom",
"class" : {
"subject" : "Math",
"teacher" : "Jack"
}
}
I want to get the Map:
"id" : "123",
"name" : "Tom",
"class.subject" : "Math",
"class.teacher" : "Jack"
************************Update Solution*************************************
Based on #Manos Nikolaidis's answer, I am able to achieve the following solution by considering ArrayNode.
public void processJsonString(String jsonString) throws Exception {
ObjectMapper mapper = new ObjectMapper();
ArrayNode arrayNode = (ArrayNode) mapper.readTree(jsonString);
processArrayNode(arrayNode);
}
private void processObjectNode(JsonNode jsonNode) {
Map<String, String> result = new HashMap<>();
Iterator<Map.Entry<String, JsonNode>> iterator = jsonNode.fields();
iterator.forEachRemaining(node -> mapAppender(result, node, new ArrayList<String>()));
}
private void processArrayNode(ArrayNode arrayNode) {
for (JsonNode jsonNode : arrayNode) {
processObjectNode(jsonNode);
}
}
private void mapAppender(Map<String, String> result, Map.Entry<String, JsonNode> node, List<String> names) {
names.add(node.getKey());
if (node.getValue().isTextual()) {
String name = names.stream().collect(Collectors.joining("."));
result.put(name, node.getValue().asText());
} else if (node.getValue().isArray()) {
processArrayNode((ArrayNode) node.getValue());
} else if (node.getValue().isNull()) {
String name = names.stream().collect(Collectors.joining("."));
result.put(name, null);
} else {
node.getValue().fields()
.forEachRemaining(nested -> mapAppender(result, nested, new ArrayList<>(names)));
}
}
You can get the JSON as JsonNode and go through all fields recursively and add key and value field to a Map. When a value is an object instead of string you can add the field name to List to be joined with periods when a string is finally encountered. First create (for readability) a separate method that add Json fields to a Map:
void mapAppender(Map<String, String> result, Entry<String, JsonNode> node, List<String> names) {
names.add(node.getKey());
if (node.getValue().isTextual()) {
String name = names.stream().collect(joining("."));
result.put(name, node.getValue().asText());
} else {
node.getValue().fields()
.forEachRemaining(nested -> mapAppender(result, nested, new ArrayList<>(names)));
}
}
and use it like this:
ObjectMapper mapper = new ObjectMapper();
Map<String, String> result = new HashMap<>();
mapper.readTree(json).fields()
.forEachRemaining(node -> mapAppender(result, node, new ArrayList<String>()));
Where fields() returns an Iterator. Beware of StackOverflowErrors and perhaps low performance for deeply nested JSON.
I resolved this using below simple code, Only think is need to download jettison and flattener.JsonFlattener library
import java.util.Map;
import org.codehaus.jettison.json.JSONObject;
import com.github.wnameless.json.flattener.JsonFlattener;
public class test {
public static void main(String[] args) {
String jsonString = "{\"id\" : \"123\",\"name\" : \"Tom\",\"class\" : {\"subject\" : \"Math\",\"teacher\" : \"Jack\"}}";
JSONObject jsonObject = new JSONObject();
String flattenedJson = JsonFlattener.flatten(jsonString);
Map<String, Object> flattenedJsonMap = JsonFlattener.flattenAsMap(jsonString);
System.out.println(flattenedJsonMap);
}
}
Reference link : https://github.com/wnameless/json-flattener