i'm trying to convert this method to an a reactive method
#GetMapping(RestConstants.BASE_PATH_AUDIENCE + "/link")
public List<String> users () {
List<String> list= new ArrayList<>();
MongoCollection mongoCollection = mongoTemplate.getCollection("collection");
DistinctIterable distinctIterable = mongoCollection.distinct("user_name", String.class);
MongoCursor mongoCursor = distinctIterable.iterator();
while (mongoCursor.hasNext()){
String user = (String)mongoCursor.next();
creatorsList.add(user);
}
return list;
}
I Have something like this but i don't know how to conver the ArrayList to return an Mono<List>
#GetMapping(RestConstants.BASE_PATH_AUDIENCE + "/link")
public Mono<List<String>> usersReactive () {
List<Mono<String>> list= new ArrayList<List>();
MongoCollection mongoCollection = mongoTemplate.getCollection("collection");
DistinctIterable distinctIterable = mongoCollection.distinct("user_name", String.class);
MongoCursor mongoCursor = distinctIterable.iterator();
while (mongoCursor.hasNext()){
String user = (String)mongoCursor.next();
list.add(user);
}
return list;
}
If you really want a Mono, then just wrap the value that you want to transport in it:
return Mono.just(creatorsList);
But I doubt you really want to return a list in a Mono. Usually, reactive endpoints returning a number of items would return a Flux
return Flux.fromIterable(creatorsList);
But since your MongoCursor is already iterable (you use its iterator in an enhanced for-loop), you can stream the cursor directly to the flux. This saves you from collecting all items into a list first.
return Flux.fromIterable(cursor);
And finally, if you are trying to convert your application to be reactive, it is wise to use the Mongo driver with native support for reactive streams: https://docs.mongodb.com/drivers/reactive-streams/
Related
I wrote a code to lookup for some movie names on IMDB, but if for instance I am searching for "Harry Potter", I will find more than one movie. I would like to use multithreading, but I don't have much knowledge on this area.
I am using strategy design pattern to search among more websites, and for instance inside one of the methods I have this code
for (Element element : elements) {
String searchedUrl = element.select("a").attr("href");
String movieName = element.select("h2").text();
if (movieName.matches(patternMatcher)) {
Result result = new Result();
result.setName(movieName);
result.setLink(searchedUrl);
result.setTitleProp(super.imdbConnection(movieName));
System.out.println(movieName + " " + searchedUrl);
resultList.add(result);
}
}
which, for each element (which is the movie name), will create a new connection on IMDB to lookup for ratings and other stuff, on the super.imdbConnection(movieName) line.
The problem is, I would like to have all the connections at the same time, because on 5-6 movies found, the process will take much longer than expected.
I am not asking for code, I want some ideeas. I thought about creating an inner class which implements Runnable, and to use it, but I don't find any meaning on that.
How can I rewrite that loop to use multithreading?
I am using Jsoup for parsing, Element and Elements are from that library.
The most simple way is parallelStream()
List<Result> resultList = elements.parallelStream()
.map(e -> {
String searchedUrl = element.select("a").attr("href");
String movieName = element.select("h2").text();
if(movieName.matches(patternMatcher)){
Result result = new Result();
result.setName(movieName);
result.setLink(searchedUrl);
result.setTitleProp(super.imdbConnection(movieName));
System.out.println(movieName + " " + searchedUrl);
return result;
}else{
return null;
}
}).filter(Objects::nonNull)
.collect(Collectors.toList());
If you don't like parallelStream() and want to use Threads, you can to this:
List<Element> elements = new ArrayList<>();
//create a function which returns an implementation of `Callable`
//input: Element
//output: Callable<Result>
Function<Element, Callable<Result>> scrapFunction = (element) -> new Callable<Result>() {
#Override
public Result call() throws Exception{
String searchedUrl = element.select("a").attr("href");
String movieName = element.select("h2").text();
if(movieName.matches(patternMatcher)){
Result result = new Result();
result.setName(movieName);
result.setLink(searchedUrl);
result.setTitleProp(super.imdbConnection(movieName));
System.out.println(movieName + " " + searchedUrl);
return result;
}else{
return null;
}
}
};
//create a fixed pool of threads
ExecutorService executor = Executors.newFixedThreadPool(elements.size());
//submit a Callable<Result> for every Element
//by using scrapFunction.apply(...)
List<Future<Result>> futures = elements.stream()
.map(e -> executor.submit(scrapFunction.apply(e)))
.collect(Collectors.toList());
//collect all results from Callable<Result>
List<Result> resultList = futures.stream()
.map(e -> {
try{
return e.get();
}catch(Exception ignored){
return null;
}
}).filter(Objects::nonNull)
.collect(Collectors.toList());
My database tables has a few jsonb columns. I am using PostgreSQL e.g.
CREATE TABLE trades (
id serial NOT NULL,
accounts jsonb,
//..
//..
);
I need to map these jsonb columns to my data model using Spring RowMapper mapRow():
public class DataModelRowMapper implements RowMapper<TestModel> {
#Override
public TestModel mapRow(final ResultSet rs,
final int rowNum) throws SQLException {
List<Account> accounts = jsonParser.parse(rs.getString("accounts"), new TypeReference<List<Account>>() {
});
//other jsonb columns
Account account = accounts.stream()
.filter(account -> account.getType() == Type.CLIENT)
.findFirst()
.orElse(new Account());
final TestModel testModel = new TestModel();
testModel.setId(rs.getString("id"));
testModel.setAccountName(account.getName());
return testModel;
}
}
Inside mapRow(), I parse the json to a Java List and then stream through to find the appropriate value as multiple accounts are returned. I have a few additional jsonb columns for which I do similar operations inside mapRow().
Previously, I was returning the exact values from the SQL query itself which proved to be slow and then moved this filtering logic to inside mapRow() in java code as the intention is to increase performance and return the result.
My question is, should I be parsing and filtering logic inside mapRow ? Is there a better faster way of loading jsonb data and mapping to TestModel accountName string property ?
My issue is, the sql query runs quick <1 ms on the db, but the java layer is adding some overhead.
If you look at the whole system, you may not need to parse into the fields at all.
for example, in most cases, it is required to return the JSON without modifications. just return the string. no need to parse back and forth, it's really faster.
example spring mvc:
// a method that returns a list of objects in JSON by search criteria
#RequestMapping(value = "/getAll", method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE, consumes = MediaType.APPLICATION_JSON_VALUE)
public #ResponseBody
void getAll(HttpServletResponse response, #RequestBody final Entity request) throws IOException {
List<String> res = null;
try {
List<Entity> regList = entityDAO.getAll(request); // get from DB list of objects with JSON
res = regList.stream().map(h -> h.getJson_obj()).collect(Collectors.toList()); // just collect JSONs
// res = regList.stream().map(h -> h.getJson_obj().replaceFirst("\\{", "{ \"vehicle_num\":" + h.getVehicle_num() + ",")).collect(Collectors.toList()); //it is also possible to add dynamic data without parsing
} catch (Exception e) {
logger.error("getAll ", e);
}
String resStr = "[" + res.stream().filter(t -> t != null).collect(Collectors.joining(", ")) + "]"; // join in one String
response.setHeader("Content-Type", "application/json;charset=UTF-8");
response.setStatus(200);
response.getWriter().println(resStr);
}
ps sorry, I can't leave a comment, so the answer.
This is an example that I made it, what should I do to get the treated data to send the result list as a WS json response ?
#Produces(MediaType.APPLICATION_JSON)
public List<String> tryFlux(#QueryParam("names") List<String> names) {
String[] array = new String[names.size()];
Flux.fromIterable(asList(names.toArray(array))).
doOnNext(this::executeService).doOnError(ex -> handleError(ex, names)).retry(1).subscribe();
return ??; //Need help here
}
You can wrap already resolved values with Mono to return JSON data.
#Produces(MediaType.APPLICATION_JSON)
public Mono<JSONResponseObject> tryFlux(#QueryParam("names") List<String> names) {
String[] array = new String[names.size()];
Flux.fromIterable(asList(names.toArray(array))).
doOnNext(this::executeService).doOnError(ex -> handleError(ex, names)).retry(1).subscribe();
return Mono.just(jsonResponseObject); //Need help here
}
I have a code snippet in spring controller and DAO classes that fetches data from the database. The end result is to get a json output. When the result is processed, it displays just the array without displaying the name of the value-pair. Below is a snippet
dao class snippet
#SuppressWarnings("unchecked")
public List <Object[]> getResult(String no) {
List <Object[]> dataList = new ArrayList<>();
List <Object[]> results = sessionFactory.getCurrentSession().createQuery("select i.F_NO, i.LAST_NAME, i.FIRST_NAME, i.HOME_PHONE, i.SEX_NO from Data i where f_no = :f_no")
.setParameter("_no", no).list();
if(!CollectionUtils.isEmpty(results)){
Object firstDataOccurence = results.get(0);
dataList.add((Object[]) firstDataOccurence);
return dataList;
}
else{
return dataList;
}
}
controller snippet
#RequestMapping(value = "/getData/{tin}", method = RequestMethod.GET, headers = "Accept=application/json")
#ResponseBody
public List<Object[]> getData(#PathVariable("no") String searchName) {
searchName = searchName.trim();
List <Object[]> dataList= myDao.getResult(searchName);
return dataList;
}
now with the above my output becomes
[["100123","demo12","demo7","demo9","demo19","demo28","demo27","demo4","demo5","demo24","demo20"]]
with above result, u find out that the keyname is not there. Please how can I get a key:value pairs in my json response.
Step 1) Instead of Object list you can create a POJO and set the value
of db Object list to a pojo.
Step 2) You need to convert your normal list to JsonArray .Please
follow following steps to do so.
List <Object[]> results = sessionFactory.getCurrentSession().createQuery("select i.F_NO, i.LAST_NAME, i.FIRST_NAME, i.HOME_PHONE, i.SEX_NO from Data i where f_no = :f_no")
.setParameter("_no", no).list();
JSONObject jObject = new JSONObject();
try
{
JSONArray jArray = new JSONArray();
for (NewPojo obj: results)
{
JSONObject studentJSON = new JSONObject();
studentJSON.put("objfname", obj.getFName());
studentJSON.put("objlname ", obj.getLName());
jArray.put(studentJSON);
}
jObject.put("NewList", jArray);
} catch (JSONException jse) {
jse.printStacktrace();
}
Note : You can also use JSONObject also if you need
Ref links:
http://www.java2novice.com/java-json/javax.json/create-json-array/
How to Create JSONArray for a List<Class name>
I want to convert an array list to json string of a specific format. I get all the users emails in an array list and want to convert it into the following format of JSON.
[
{"email":"abc#gmail.com"},
{"email":"xyz#gmail.com"}
]
My controller action is
public static Result apiCustomers(){
List<Customer> customerList = Model.coll(Customer.class).find().toArray();
List<String> emails = new ArrayList<String>();
for(Customer c : customerList){
emails.add(c.email);
}
//ObjectNode result = Json.newObject();
//result.put("emails", Json.toJson(emails));
return ok();
}
How can I convert the emails list to the above json format?
Thanks in advance
Why use another JSON ser/des lib? Play has one built-in (a wrapper around jackson - which is really fast).
Starting from your code:
public static Result apiCustomers(){
List<Customer> customerList = Model.coll(Customer.class).find().toArray();
List<String> emails = new ArrayList<String>();
for(Customer c : customerList){
emails.add(c.email);
}
return ok(Json.toJson(emails));
}
This uses some defaults but should be enough.
Or manually:
public static Result apiCustomers(){
ArrayNode arrayNode = new ArrayNode(JsonNodeFactory.instance);
List<Customer> customerList = Model.coll(Customer.class).find().toArray();
for(Customer c : customerList){
ObjectNode mail = Json.newObject();
mail.put("email", c.email);
arrayNode.add(mail);
}
return ok(arrayNode);
}
No need for Gson.
You can use this library:
http://code.google.com/p/google-gson/
Pretty easy tutorial about it here:
http://www.mkyong.com/java/how-do-convert-java-object-to-from-json-format-gson-api/
Or you can write a custom toJson method to your classes or an util Json (not a big deal)
in your case it should be something like that (i didn't test it):
public String toJson(List<String> emails) {
StringBuilder result = new StringBuilder();
result.append("[");
result.append("\n");
for (String s : emails) {
result.append("{");
result.append("\"email\":");
result.append("\"");
result.append(s);
result.append("\"");
result.append("}");
result.append(",");
result.append("\n");
}
result.append("]");
return result.toString();
}
With the benefit of Java 8 (and I suspect a newer Jackson version):
private static final ObjectMapper mapper = new ObjectMapper();
...
List<Customer> customerList = Model.coll(Customer.class).find().toArray();
ArrayNode emails = mapper.createArrayNode();
customerList.forEach(c -> emails.add(c.email));