We Keep Coding

When spring boot service starts, insert data into mongodb

I'm currently new to the Spring Boot Java framework and I'm building a simple application. When my service starts, I want to be able to read a raw file from a URL, parse that data, and upload it into my mongodb database of atlas. So far this is what I have:
#Service
public class CoronaVirusDataService {
private List<LocationStats> allConfirmedStats = new ArrayList<>();
MongoOperations mongoOperations;
#PostConstruct // run this method as soon as the application runs
#Scheduled(cron = "* * 1 * * *") // execute this method every day
public void fetchVirusData() {
List<LocationStats> newStats = new ArrayList<>(); // to hold the stats of each state
HttpClient client = HttpClient.newHttpClient();
// creating a new http request
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create(ConstantsUtil.VIRUS_CONFIRMED_DATA_URL))
.build();
// get a response by having the client send the request
try {
HttpResponse<String> httpResponse = client.send(request, HttpResponse.BodyHandlers.ofString());
// parse the body of the request from csv format to readable format
StringReader csvBodyReader = new StringReader(httpResponse.body());
Iterable<CSVRecord> records = CSVFormat.DEFAULT.withFirstRecordAsHeader().parse(csvBodyReader);
for (CSVRecord record: records) {
// create a model with the parsed data
LocationStats stats = new LocationStats();
stats.setState(record.get("Province/State"));
stats.setCountry(record.get("Country/Region"));
// the latest day
int latestCases = Integer.parseInt(record.get(record.size() - 1));
int prevDayCases = Integer.parseInt(record.get(record.size() - 2));
stats.setLatestTotalCases(latestCases);
stats.setDiffFromPreviousDay(prevDayCases);
mongoOperations.save(LocationStats);
// add to new stats
newStats.add(stats);
}
// assign to class array -> we use this array to display the data
this.allConfirmedStats = newStats;
} catch (IOException | InterruptedException e) {
e.printStackTrace();
}
}
}
So the main issue with this is the data is not saving to the mongoDB once I call mongoOperations.save(). Also, I've learned that it is bad practice to maintain some type of state in a Service. What is the best practice for this? Will inserting the data into MongoDB take care of that since we are not managing state.
Here is my model class that I want to save to mongodb
#Document(collection = "LocationStats")
public class LocationStats {
/** Location model to show corona virus statistics in each state*/
#Id
private String state;
private String country;
private int latestTotalCases;
private int diffFromPreviousDay;
public String getState() {
return state;
}
public void setState(String state) {
this.state = state;
}
public String getCountry() {
return country;
}
public void setCountry(String country) {
this.country = country;
}
public int getLatestTotalCases() {
return latestTotalCases;
}
public void setLatestTotalCases(int latestTotalCases) {
this.latestTotalCases = latestTotalCases;
}
public int getDiffFromPreviousDay() {
return diffFromPreviousDay;
}
public void setDiffFromPreviousDay(int diffFromPreviousDay) {
this.diffFromPreviousDay = diffFromPreviousDay;
}
#Override
public String toString() {
return "LocationStats{" +
"state='" + state + '\'' +
", country='" + country + '\'' +
", latestTotalCases=" + latestTotalCases +
'}';
}
}
once I have my models saved into mongoDB, I want to read from the database and get all the data from each collection and display it on the webpage. I'm thinking I'd fetch that data within the controller class and pass it to the frontend, is this good practice? here is my controller class.
#Controller
public class HomeController {
/** Controller class to generate/render the html UI */
#Autowired
CoronaVirusDataService coronaVirusDataService;
#Autowired
MongoOperations mongoOperations;
#GetMapping("/") // map this to the root template
public String home(Model model) {
List<LocationStats> allStats = coronaVirusDataService.getAllConfirmedStats();
// instead of above getter method, have a method call that fetches all data from mongoDB and return it as a List<LocationStats>
// get the total confirmed cases
int totalConfirmedCases = allStats.stream().mapToInt(LocationStats::getLatestTotalCases).sum();
int totalNewCases = allStats.stream().mapToInt(LocationStats::getDiffFromPreviousDay).sum();
// send the models to the view
model.addAttribute("locationStats", allStats);
model.addAttribute("totalReportedCases", totalConfirmedCases);
model.addAttribute("totalNewCases", totalNewCases);
return "home";
}
}

How to make Vertx MongoClient operation synchronous yet not blocking event loop in Java?

I am trying to save a new document to MongoDB using the Vertx MongoClient as follows:
MongoDBConnection.mongoClient.save("booking", query, res -> {
if(res.succeeded()) {
documentID = res.result();
System.out.println("MongoDB inserted successfully. + document ID is : " + documentID);
}
else {
System.out.println("MongoDB insertion failed.");
}
});
if(documentID != null) {
// MongoDB document insertion successful. Reply with a booking ID
String resMsg = "A confirmed booking has been successfully created with booking id as " + documentID +
". An email has also been triggered to the shared email id " + emailID;
documentID = null;
return new JsonObject().put("fulfillmentText", resMsg);
}
else {
// return intent response
documentID = null;
return new JsonObject().put("fulfillmentText",
"There is some issues while booking the shipment. Please start afreash.");
}
The above code successfully writes the query jsonObject to MongoDB collection booking. However, the function which contains this code always returns with There is some issues while booking the shipment. Please start afreash.
This is happening probably because the MongoClient save() handler "res" is asynchronous. But, I want to return conditional responses based on successful save() operation and on failed save operation.
How to achieve it in Vertx Java?

Your assumption is correct, you dont wait for the async response from the database. What you can do, is to wrap it in a Future like this:
public Future<JsonObject> save() {
Future<JsonObject> future = Future.future();
MongoDBConnection.mongoClient.save("booking", query, res -> {
if(res.succeeded()) {
documentID = res.result();
if(documentID != null) {
System.out.println("MongoDB inserted successfully. + document ID is : " + documentID);
String resMsg = "A confirmed booking has been successfully created with booking id as " + documentID +
". An email has also been triggered to the shared email id " + emailID;
future.complete(new JsonObject().put("fulfillmentText", resMsg));
}else{
future.complete(new JsonObject().put("fulfillmentText",
"There is some issues while booking the shipment. Please start afreash."))
}
} else {
System.out.println("MongoDB insertion failed.");
future.fail(res.cause());
}
});
return future;
}
Then i assume you have and endpoint that eventually calls this, eg:
router.route("/book").handler(this::addBooking);
... then you can call the save method and serve a different response based on the result
public void addBooking(RoutingContext ctx){
save().setHandler(h -> {
if(h.succeeded()){
ctx.response().end(h.result());
}else{
ctx.response().setStatusCode(500).end(h.cause());
}
})
}

You can use RxJava 2 and a reactive Mongo Client (io.vertx.reactivex.ext.mongo.MongoClient)
Here is a code snippet:
Deployer
public class Deployer extends AbstractVerticle {
private static final Logger logger = getLogger(Deployer.class);
#Override
public void start(Future<Void> startFuture) {
DeploymentOptions options = new DeploymentOptions().setConfig(config());
JsonObject mongoConfig = new JsonObject()
.put("connection_string",
String.format("mongodb://%s:%s#%s:%d/%s",
config().getString("mongodb.username"),
config().getString("mongodb.password"),
config().getString("mongodb.host"),
config().getInteger("mongodb.port"),
config().getString("mongodb.database.name")));
MongoClient client = MongoClient.createShared(vertx, mongoConfig);
RxHelper.deployVerticle(vertx, new BookingsStorage(client), options)
.subscribe(e -> {
logger.info("Successfully Deployed");
startFuture.complete();
}, error -> {
logger.error("Failed to Deployed", error);
startFuture.fail(error);
});
}
}
BookingsStorage
public class BookingsStorage extends AbstractVerticle {
private MongoClient mongoClient;
public BookingsStorage(MongoClient mongoClient) {
this.mongoClient = mongoClient;
}
#Override
public void start() {
var eventBus = vertx.eventBus();
eventBus.consumer("GET_ALL_BOOKINGS_ADDRESS", this::getAllBookings);
}
private void getAllBookings(Message msg) {
mongoClient.rxFindWithOptions("GET_ALL_BOOKINGS_COLLECTION", new JsonObject(), sortByDate())
.subscribe(bookings -> {
// do something with bookings
msg.reply(bookings);
},
error -> {
fail(msg, error);
}
);
}
private void fail(Message msg, Throwable error) {
msg.fail(500, "An unexpected error occurred: " + error.getMessage());
}
private FindOptions sortByDate() {
return new FindOptions().setSort(new JsonObject().put("date", 1));
}
}
HttpRouterVerticle
// inside a router handler:
vertx.eventBus().rxSend("GET_ALL_BOOKINGS_ADDRESS", new JsonObject())
.subscribe(bookings -> {
// do something with bookings
},
e -> {
// handle error
});

upwork-api return 503 ioexception

I created app for getting info from upwork.com. I use java lib and Upwork OAuth 1.0. The problem is local request to API works fine, but when I do deploy to Google Cloud, my code does not work. I get ({"error":{"code":"503","message":"Exception: IOException"}}).
I create UpworkAuthClient for return OAuthClient and next it is used for requests in JobClient.
run() {
UpworkAuthClient upworkClient = new UpworkAuthClient();
upworkClient.setTokenWithSecret("USER TOKEN", "USER SECRET");
OAuthClient client = upworkClient.getOAuthClient();
//set query
JobQuery jobQuery = new JobQuery();
jobQuery.setQuery("query");
List<JobQuery> jobQueries = new ArrayList<>();
jobQueries.add(jobQuery);
// Get request of job
JobClient jobClient = new JobClient(client, jobQuery);
List<Job> result = jobClient.getJob();
}
public class UpworkAuthClient {
public static final String CONSUMERKEY = "UPWORK KEY";
public static final String CONSUMERSECRET = "UPWORK SECRET";
public static final String OAYTРCALLBACK = "https://my-app.com/main";
OAuthClient client ;
public UpworkAuthClient() {
Properties keys = new Properties();
keys.setProperty("consumerKey", CONSUMERKEY);
keys.setProperty("consumerSecret", CONSUMERSECRET);
Config config = new Config(keys);
client = new OAuthClient(config);
}
public void setTokenWithSecret (String token, String secret){
client.setTokenWithSecret(token, secret);
}
public OAuthClient getOAuthClient() {
return client;
}
public String getAuthorizationUrl() {
return this.client.getAuthorizationUrl(OAYTРCALLBACK);
}
}
public class JobClient {
private JobQuery jobQuery;
private Search jobs;
public JobClient(OAuthClient oAuthClient, JobQuery jobQuery) {
jobs = new Search(oAuthClient);
this.jobQuery = jobQuery;
}
public List<Job> getJob() throws JSONException {
JSONObject job = jobs.find(jobQuery.getQueryParam());
jobList = parseResponse(job);
return jobList;
}
}
Local dev server works fine, I get resilts on local machine, but in Cloud not.
I will be glad to any ideas, thanks!

{"error":{"code":"503","message":"Exception: IOException"}}
doesn't seem like a response return by Upwork API. Could you please provide the full response including the returned headers? So, we will take a more precise look into it.

Android Facebook Unsupported post request error

I am working on a Facebook application and am currently trying to have my app tag one of the user's friends. I almost have it working 100%, except when it is supposed to be tagging the person, I instead get an error message that follows:
{"error":{"message":"Unsupported post request.","type":"GraphMethodException","code":100}}
The user and photo IDs are for sure correct, that is not the issue. Otherwise, I'm not sure what else could be causing this error. Code is below for reference. Thanks much!
public void setTag() {
String relativePath = Constants.photoID + "/tags/" + Constants.userID;
Bundle params = new Bundle();
params.putString("x", "5");
params.putString("y", "5");
Constants.mAsyncRunner.request(relativePath, params, "POST", new TagPhotoRequestListener(),
null);
}
public class TagPhotoRequestListener extends BaseRequestListener {
#Override
public void onComplete(final String response, final Object state) {
if (response.equals("true"))
{
String message = "User tagged in photo at (5, 5)" + "\n";
message += "Api Response: " + response;
Log.i("TagPhotoRequestListener", message);
}
else
{
Log.w("TagPhotoRequestListener", "User could not be tagged.");
}
}
public void onFacebookError(FacebookError e) {
Log.w("TagPhotoRequestListener", "Facebook Error: " + e.getMessage());
}
}
EDIT: Here is my code for posting of a picture and getting the photoID. For testing purposes it's just a single photo from my sdcard.
public void postPhoto() {
byte[] data = null;
Bitmap bi = BitmapFactory.decodeFile("/mnt/sdcard/Download/KathleenSchedule.jpg");
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bi.compress(Bitmap.CompressFormat.JPEG, 100, baos);
data = baos.toByteArray();
Bundle params = new Bundle();
params.putString(Facebook.TOKEN, Constants.mFacebook.getAccessToken());
params.putString("method", "photos.upload");
params.putByteArray("picture", data);
AsyncFacebookRunner mAsyncRunner = new AsyncFacebookRunner(Constants.mFacebook);
mAsyncRunner.request(null, params, "POST", new PhotoUploadListener(), null);
}
public class PhotoUploadListener extends BaseRequestListener {
#Override
public void onComplete(final String response, final Object state) {
try {
// process the response here: (executed in background thread)
Log.d("PhotoUploadListener", "Response: " + response.toString());
JSONObject json = Util.parseJson(response);
System.out.println(response);
final String photo_id = json.getString("pid");
Constants.photoID = photo_id;

Ok, now your problem is clear.
You are using the photos.upload method which is deprecated:
We are in the process of deprecating the REST API, so if you are
building a new application you shouldn't use this function. Instead
use the Graph API and POST to the photos connection of the User object
Because you are using an old method, you're getting and old response type.
Switch to using the graph api way, and you should get the photo id in the response.

How Can I create,send and receive iq packets using smack(java)

I am connected to server(Xmpp)
but unable to send and receive packets at my psi client
Here is snippet of my code
POSClientIQ posclientiq = new POSClientIQ();
posclientiq.connectXMPPServer();
posclientiq.processMessage();
}
public void processMessage()
{ try{
final IQ iq1 = new IQ() {
public String getChildElementXML() {
return "<iq type='get' from ='sam'><query xmlns='jabber:iq:roster'></query></iq>";
}
};
iq1.setType(IQ.Type.GET);
// PacketCollector collector = connection.createPacketCollector(new PacketIDFilter(iq1.getPacketID()));
connection.sendPacket(iq1);
System.out.println("Message send");

The getChildElementXML() returns the tag. If you are using Smack then you don't need to write your own IQ implementation unless it is a custom query. For your case, to query the roster use RosterPacket.

If you have a custom query and you would like to use your IQ implementation then:
final IQ iq = new IQ() {
public String getChildElementXML() {
return "<query xmlns='http://jabber.org/protocol/disco#info'/>"; // here is your query
//this returns "<iq type='get' from='User#YourServer/Resource' id='info1'> <query xmlns='http://jabber.org/protocol/disco#info'/></iq>";
}};
// set the type
iq.setType(IQ.Type.GET);
// send the request
connection.sendPacket(iq);
As you can see you have here your custom query and you use Smack to set the rest of your IQ e.g. setting the type. Please note that Smack fills the "from" for you based on the JID your are logged into.

//To retrieve archive msges from server..
MyCustomIQ iq = new MyCustomIQ();
iq.setType(IQ.Type.set);
mConnection.sendIqWithResponseCallback(iq, new PacketListener() {
#Override
public void processPacket(Packet packet) throws SmackException.NotConnectedException {
Log.i("Send IQ with Response", "****** message " + packet);
}
}, new ExceptionCallback() {
#Override
public void processException(Exception exception) {
exception.printStackTrace();
Log.i("IO archjieve Exception",""+ exception.getMessage());
}
}, 5000);
mConnection.sendPacket(new Presence(Presence.Type.available));
PacketTypeFilter filter=new PacketTypeFilter(org.jivesoftware.smack.packet.Message.class);
PacketListener myListener=new PacketListener(){
public void processPacket(Packet packet){
if(((Message) packet).getType().equals(Message.Type.chat))
{
((Message) packet).getBody();
}
else if(((Message) packet).getType().equals(Message.Type.normal))
{
DefaultPacketExtension pacExten=PacketUtil.packetExtensionfromCollection(packet.getExtensions(), "result", "urn:xmpp:mam:0");
String strMsg=pacExten.getValue("body");
}
}
}
;
mConnection.addPacketListener(myListener, filter);
//My Custom IQ
class MyCustomIQ extends IQ {
String token;
protected MyCustomIQ() {
super("query","urn:xmpp:mam:0");
}
#Override
protected IQChildElementXmlStringBuilder getIQChildElementBuilder(IQChildElementXmlStringBuilder xml) {
// String queryId = prefix + Long.toString(new AtomicLong().incrementAndGet());
xml.attribute("queryid",queryId);
xml.rightAngleBracket();
return xml;
}
}
//You may get the response in PacketListerener sometimes so put debug in that also