We Keep Coding

How do I exstract data from multiple websites using restful API and Spring? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 1 year ago.
Improve this question
I got a task at school in which I have to do the following:
Implement the RESTful endpoint API, which simultaneously makes calls
to the following websites:
https://pizzerijalimbo.si/meni/
https://pizzerijalimbo.si/kontakt/
https://pizzerijalimbo.si/my-account/
https://pizzerijalimbo.si/o-nas/
The input for the endpoint is ‘integer’, which represents the number
of simultaneous calls to the above web pages (min 1 represents all
consecutive calls, max 4 represents all simultaneous calls).
Extracts a short title text from each page and saves this text in a
common global structure (array, folder (). The program should also
count successful calls. Finally, the service should list the number of
successful calls, the number of failed calls and the saved address
texts from all web pages.
With some help I managed to do something, but I still need help with data exstraction using Jsoup or any other method.
Here is the code that I have:
import java.util.Arrays;
import java.util.List;
import java.io.IOException;
import java.net.URL;
import java.util.Scanner;
#RestController
public class APIcontroller {
#Autowired
private RestTemplate restTemplate;
List<String> websites = Arrays.asList("https://pizzerijalimbo.si/meni/",
"https://pizzerijalimbo.si/kontakt/",
"https://pizzerijalimbo.si/my-account/",
"https://pizzerijalimbo.si/o-nas/");
#GetMapping("/podatki")
public List<Object> getData(#RequestParam(required = true) int numberOfWebsites) {
List<String> websitesToScrape = websites.subList(0, numberOfWebsites);
for (String website : websitesToScrape) {
Document doc = Jsoup.connect("https://pizzerijalimbo.si/meni/").get();
log(doc.title());
Elements newsHeadlines = doc.select("#mp-itn b a");
for (Element headline : newsHeadlines) {
log("%s\n\t%s",
headline.attr("title"), headline.absUrl("href"));
}
}
}
}
I also need to do it parallel, so the calls to a secific website go on at the same time.
But the main problem now is with the log funcion which does not work properly.
What I have tried:
I tried to solve the problem using Jsoup library, but I dont seem to
undersand it well, so I got an error in the for loop which says that
the method log is undefined. I also need to do a try catch to count possible failed calls and count the calls that are successfull as you can see in the task description.

WebScrapperController.java
package com.stackovertwo.stackovertwo;
import java.io.IOException;
import java.io.StringReader;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import org.jsoup.Jsoup;
import org.jsoup.select.Elements;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.client.RestTemplate;
//import org.w3c.dom.Document;
//import org.w3c.dom.DocumentFragment;
import org.jsoup.nodes.Document;
import org.w3c.dom.Node;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
#RestController
public class WebScrapperController {
#GetMapping("/")
public String index() {
return "Greetings from Spring Boot!";
}
// #Autowired
// private RestTemplate restTemplate;
#Autowired
WebScrapperService webScrapperService;
List<String> websites = Arrays.asList("https://pizzerijalimbo.si/meni/",
"https://pizzerijalimbo.si/kontakt/",
"https://pizzerijalimbo.si/my-account/",
"https://pizzerijalimbo.si/o-nas/");
#GetMapping("/podatki")
public ResponseEntity<Object> getData(#RequestParam(required = true) int numberOfWebsites) throws InterruptedException, ExecutionException {
List<SiteResponse> webSitesToScrape = new ArrayList<>();
// List<String> websitesToScrape = websites.subList(0, numberOfWebsites);
List<SiteResponse> responseResults = new ArrayList<SiteResponse>();
CompletableFuture<SiteResponse> futureData1 = webScrapperService.getWebScrappedContent(websites.get(0));
CompletableFuture<SiteResponse> futureData2 = webScrapperService.getWebScrappedContent(websites.get(1));
//CompletableFuture.allOf(futureData1, futureData2).join();
webSitesToScrape.add(futureData1.get());
webSitesToScrape.add(futureData2.get());
List<SiteResponse> result = webSitesToScrape.stream().collect(Collectors.toList());
return ResponseEntity.ok().body(result);
}
}
WebScrapperService.java
package com.stackovertwo.stackovertwo;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.select.Elements;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.ResponseEntity;
import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;
import java.util.concurrent.CompletableFuture;
#Service
public class WebScrapperService {
#Autowired
private RestTemplate restTemplate;
Logger logger = LoggerFactory.getLogger(WebScrapperService.class);
#Async
public CompletableFuture<SiteResponse> getWebScrappedContent(String webSiteURL)
//throws InterruptedException
{
logger.info("Starting: getWebScrappedContent for webSiteURL {} with thread {}", webSiteURL, Thread.currentThread().getName());
HttpEntity<String> response = restTemplate.exchange(webSiteURL,
HttpMethod.GET, null, String.class);
//Thread.sleep(1000);
SiteResponse webSiteSummary = null ;
String resultString = response.getBody();
HttpHeaders headers = response.getHeaders();
int statusCode = ((ResponseEntity<String>) response).getStatusCode().value();
System.out.println(statusCode);
System.out.println("HEADERS"+headers);
try
{
Document doc = (Document) Jsoup.parse(resultString);
Elements header = doc.select(".elementor-inner h2.elementor-heading-title.elementor-size-default");
System.out.println(header.get(0).html());
// Return the fragment.
webSiteSummary = new SiteResponse(statusCode, header.get(0).html());
}
catch(Exception e) {
System.out.println("Exception "+e.getMessage());
}
logger.info("Complete: getWebScrappedContent for webSiteURL {} with thread {}", webSiteURL, Thread.currentThread().getName());
return CompletableFuture.completedFuture(webSiteSummary);
}
}
SpringBootApp.java
package com.stackovertwo.stackovertwo;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.web.client.RestTemplate;
import java.security.KeyManagementException;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.X509Certificate;
//import javax.net.ssl.HostnameVerifier;
//import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
//import javax.net.ssl.SSLSession;
//import javax.net.ssl.TrustManager;
//import javax.net.ssl.X509TrustManager;
//import javax.security.cert.X509Certificate;
import org.apache.http.conn.ssl.TrustStrategy;
import org.apache.http.impl.client.*;
import org.apache.http.conn.ssl.*;
#SpringBootApplication
public class SpringBootApp
{
public static void main(String[] args)
{
SpringApplication.run(SpringBootApp.class, args);
}
#Bean
public RestTemplate restTemplate() throws KeyManagementException, NoSuchAlgorithmException, KeyStoreException {
TrustStrategy acceptingTrustStrategy = (X509Certificate[] chain, String authType) -> true;
SSLContext sslContext = org.apache.http.ssl.SSLContexts.custom()
.loadTrustMaterial(null, acceptingTrustStrategy)
.build();
SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(sslContext);
CloseableHttpClient httpClient = HttpClients.custom()
.setSSLSocketFactory(csf)
.build();
HttpComponentsClientHttpRequestFactory requestFactory =
new HttpComponentsClientHttpRequestFactory();
requestFactory.setHttpClient(httpClient);
//return new RestTemplate();
RestTemplate restTemplate = new RestTemplate(requestFactory);
return restTemplate;
}
}
Note: I disabled SSL verification while calling the webulr in resttemplate, but its not recommendd inproduction (For assignment its ok). But you need to import the keys via java keystore in case production : https://myshittycode.com/2015/12/17/java-https-unable-to-find-valid-certification-path-to-requested-target-2/

Having trouble in verifying the PACT from provider side using java junit5 maven spring-boot

I have to do PACT verification in java + spring-boot + maven.
I am currently running this tests with junit5.
My pom.xml looks like
<dependency>
<groupId>au.com.dius</groupId>
<artifactId>pact-jvm-provider-junit5</artifactId>
<version>4.0.10</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>au.com.dius</groupId>
<artifactId>pact-jvm-consumer-junit5</artifactId>
<version>4.0.10</version>
</dependency>
<plugin>
<groupId>au.com.dius.pact.provider</groupId>
<artifactId>maven</artifactId>
<version>4.1.11</version>
<configuration>
<pactDirectory>${basedir}/target/pacts</pactDirectory>
<pactBrokerUrl><BROKERURL></pactBrokerUrl>
<projectVersion><PROJECTVERSION></projectVersion>
<trimSnapshot>true</trimSnapshot>
</configuration>
</plugin>
Consumer side code is runnning perfectly and published on pact broker too, here is the snippet of it.
package com.contract;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.CoreMatchers.notNullValue;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.junit.Assert.assertEquals;
import java.io.IOException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.fluent.Request;
import org.apache.http.entity.StringEntity;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.testng.Assert;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import au.com.dius.pact.consumer.MockServer;
import au.com.dius.pact.consumer.dsl.PactDslWithProvider;
import au.com.dius.pact.consumer.junit5.PactConsumerTestExt;
import au.com.dius.pact.consumer.junit5.PactTestFor;
import au.com.dius.pact.core.model.RequestResponsePact;
import au.com.dius.pact.core.model.annotations.Pact;
#ExtendWith(PactConsumerTestExt.class)
class ContractTest {
public static final String jwt_token = "Bearer eyJraWQiOiIyZjFiNzlmMS0xMDQ2LTQ2NGYtYjM5YS0xOGY4MDg5ZGMyMzIiLCJ0eXAiOiJhdCtqd3QiLCJhbGciOiJSUzI1NiJ9.eyJhdWQiOiJodHRwczovL3RlbmFudDE1LWFkZW1hdHJpeHN0YWNrNC10cmlhbC5kc21sYWIuYm1jLmNvbSIsImNsaWVudF9pZCI6IjVmNDU4ZjBlLWZmMzAtNDQ2ZC04ZTQwLWYzNjBlNjgyZjYyNSIsInRlbmFudCI6InN0YWNrNHRlbmFudDYiLCJqdGkiOiI4NDc3MDBhNC1hYWY0LTQ2NWMtYmNhYi05MDM0MzYxOTY0MDIiLCJzdWIiOiJEZW1vIiwiaXNzIjoiaHR0cHM6Ly9hZGVpdHNtMi1yc3NvLmRzbWxhYi5ibWMuY29tL3Jzc28iLCJpYXQiOjE2MjY3Njg0MjQsImV4cCI6MTYyNjc2OTMyNH0.EoXpqiyo5nvdPeYHfqq0sa15dQKYayD60UEboPVJojuYBKmvvj2yU03e61wy9Fkq2pZdoTNTifAeLDiG0dYKdlYOI5YTx_K6HGMav6ofYeOwIUj4OBVu7NyWxXVQz-3aXoIyu-MifUtvs6oQwf2YZmOEVtbPuCBxa9C9yA4i72g1TD31Rba1-e5cGG5ipiIE7UaunJ2K-mkt-BL2kzmu6OHdIP6vly7iTzxfOccdrjXEmedgX2hpnPcL_2os5wHCLwdHJwuYLPlgqDbSLHgpXdGL43Jg4ASBbFHg3h30y1yXYJazlgOwvVeBoOVcQYnXBh7wHTMik7zVMAo1VL8N_Q";
public static final String API_V1_VIEW_ALGORITHM = "/aif/api/v1.0/algorithm/b82d4617-c39f-448b-bdf8-3fd52e3250ba";
public static final String BODY_GET_VIEW_ALGO_V1 = "{\"tenantId\":\"6881408\",\"id\":\"5bc098db-7acb-4361-86a9-3c439b477142\",\"name\":\"New Job\",\"description\":\"New Job\",\"creationTime\":\"1625054173273\",\"template\":\"\",\"modifiedTime\":\"1625054173273\",\"owner\":\"ppan\",\"enable\":\"true\",\"executionMode\":\"onDemand\",\"definition\":\"\"}";
private final static Map<String, String> headers;
static {
headers = new HashMap<>();
headers.put("Content-Type", "application/json");
headers.put("Authorization", jwt_token);
headers.put("authtype", "rsso-jwt");
}
#BeforeEach
public void setUp(MockServer mockServer) {
assertThat(mockServer, is(notNullValue()));
}
/* ----------------------------------Pact methods ----------------------------------------------------*/
#Pact(provider = <provider name> , consumer = <consumer name>)
public RequestResponsePact createPactViewAlgoV1(PactDslWithProvider builder) {
return builder
.given("algorithm id")
.uponReceiving("a request with details of Algorithm")
.path(API_V1_VIEW_ALGORITHM)
.method("GET")
.headers(headers)
.willRespondWith()
.status(200)
.body(BODY_GET_VIEW_ALGO_V1)
.toPact();
}
/* ----------------------------------Test methods ----------------------------------------------------*/
#Test
#PactTestFor(pactMethod = "createPactViewAlgoV1")
public void testTokenRequest(MockServer mockServer) throws ClientProtocolException, IOException {
HttpResponse httpResponse = Request.Get(mockServer.getUrl() + API_V1_VIEW_ALGORITHM)
.addHeader("Content-Type", headers.get("Content-Type"))
.addHeader("Authorization",headers.get("Authorization"))
.addHeader("authtype", headers.get("authtype"))
.execute()
.returnResponse();
assertEquals(httpResponse.getStatusLine().getStatusCode(), 200);
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.enable(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY);
AIFAlgorithmModelNew actualResult = objectMapper.readValue(httpResponse.getEntity().getContent(), AIFAlgorithmModelNew.class);
assertEquals(actualResult.getTenantId().toString(), "6881408");
assertEquals(actualResult.getExecutionMode().toString(), "onDemand");
assertEquals(actualResult.getEnable().toString(), "true");
assertEquals(actualResult.getName().toString(), "New Job");
}
}
I have written my Pact provider test class with standard pact format.
I am currently trying to run this on port: 8091.
Tried with port 9362 as well.
looks like this
package com.aif.api.contract;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.mock.mockito.MockBean;
import org.springframework.boot.test.mock.mockito.SpyBean;
import org.springframework.context.ApplicationContext;
import org.springframework.test.context.TestPropertySource;
import org.springframework.test.context.junit.jupiter.SpringExtension;
<Code libraries>
import au.com.dius.pact.provider.junit.Provider;
import au.com.dius.pact.provider.junit.State;
import au.com.dius.pact.provider.junit.loader.PactBroker;
import au.com.dius.pact.provider.junit5.HttpTestTarget;
import au.com.dius.pact.provider.junit5.HttpsTestTarget;
import au.com.dius.pact.provider.junit5.PactVerificationContext;
import au.com.dius.pact.provider.junit5.PactVerificationInvocationContextProvider;
import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;
import io.restassured.RestAssured;
import io.restassured.http.Cookie;
import io.restassured.config.SSLConfig;
import static org.mockito.Mockito.when;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.Base64;
import java.util.UUID;
import javax.ws.rs.core.Response;
import org.apache.http.HttpRequest;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.TestTemplate;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mockito;
import org.mockito.MockitoAnnotations;
#ExtendWith(SpringExtension.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT, classes = main.class)
#AutoConfigureMockMvc
#TestPropertySource(locations = "classpath:application-contract-test.properties")
#Provider("Provider")
#PactBroker(
host="${PACTBROKER_HOST:PACTBroker URL}",
port="${PACTBROKER_PORT:9292}"
)
public class BasicTest {
static {
System.setProperty("javax.net.ssl.keyStore", "./conf/server_key.p12");
System.setProperty("javax.net.ssl.keyStorePassword", "Y2hhbmdlaXQ=");
}
#Autowired
private ApplicationContext context;
#MockBean
Algor
#MockBean
AlgorithmResource algorithmResource;
#Value("${server.port}")
private int serverPort;
#Value("${pact.provider.version}")
private String version;
#Value("${pact.verifier.publishResults}")
private String publishResults;
#BeforeEach
void setupTestTarget(PactVerificationContext context) {
System.out.println("Inside setupTestTarget");
MockitoAnnotations.initMocks(this);
context.setTarget(new HttpTestTarget("localhost", serverPort, "/"));
System.setProperty("pact.provider.version", version);
System.setProperty("pact.verifier.publishResults", publishResults);
}
#TestTemplate
#ExtendWith(PactVerificationInvocationContextProvider.class)
void pactVerificationTestTemplate(PactVerificationContext context, HttpRequest request) {
context.verifyInteraction();
}
/* consumer = <Consumer Name> */
#State("algorithm id")
public void getAlgorithmById() throws Exception {
System.out.println("Inside getAlgorithmById");
AlgorithmResponseTOFull responseTO = new AlgorithmResponseTOFull();
responseTO.setTenantId("6881408");
responseTO.setId("5bc098db-7acb-4361-86a9-3c439b477142");
responseTO.setName("New Job");
responseTO.setDefinition("");
responseTO.setDescription("New Job");
responseTO.setCreationTime("1625054173273");
responseTO.setModifiedTime("1625054173273");
responseTO.setOwner("ppans");
responseTO.setEnable("true");
responseTO.setExecutionMode("onDemand");
responseTO.setTemplate(getAlgorithmTemplateResponseTO());
Response response = Response.ok(Util.buildResponse(responseTO, Constant.STATUS_MSG_SUCCESS)).build();
when(algorithmResource.getById(Mockito.any())).thenReturn(response);
}
}
When I run this, I get error like
Verifying a pact between <consumer name> and <provider name>
[Using Pact Broker PACTBROKERURL:9292]
Given algorithm id
a request with details of Algorithm
Inside pactVerificationTestTemplate
returns a response which
has status code 200 (FAILED)
has a matching body (FAILED)
Failures:
0) Verifying a pact between <consumer name> and <provider name> - a request with details of Algorithm returns a response which has statusResult code 200
expected status of 200 but was 401
1) Verifying a pact between <consumer name> and <provider name> - a request with details of Algorithm returns a response which has a matching body
Expected a response type of 'application/json' but the actual type was 'text/html'
Please help me to resolve this, as I am getting above error and not able to resolve it

In the consumer where the test is written , it is expecting a response of 200 and response type of "application/json".
If possible, please share the consumer test also that you have written.
Please check that the pact which is getting generated at the pact broker url, and also what is the response and response type it is expecting.

A 401 indicates the request contained invalid or no credentials.
Given that your consumer test has a JWT in it, my guess is that it's expired by the time the provider test runs.
See https://docs.pact.io/provider/handling_auth/ for strategies in dealing with this and the workshops here to see how to out them into action: https://docs.pact.io/implementation_guides/workshops/

Accessing request payload in V2 DialogFlow fulfillment webhook

I'm trying to parse the request sent to a java based fulfillment in V2 of the API. I can't find any example documentation in Java for doing this in V2 of the API (com.google.cloud:google-cloud-dialogflow:0.38.0-alpha dependency in my project).
So far I've got as far as writing a very basic Spring MVC controller to accept the request.
How can I parse out the payload in the request, e.g. the parameters that dialog flow sent ?
import com.google.cloud.dialogflow.v2beta1.WebhookRequest;
import com.google.cloud.dialogflow.v2beta1.WebhookResponse;
import com.google.protobuf.Descriptors;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import javax.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.util.Map;
import java.util.stream.Collectors;
#RestController
#RequestMapping("test")
public class TestRequestRestController {
private static final Logger log = LoggerFactory.getLogger(TestRequestRestController.class);
#PostMapping("test1t")
public WebhookResponse getTest1(WebhookRequest request) {
System.out.println(request.toString());
return WebhookResponse.newBuilder().setFulfillmentText("Example reply 1 ").build();
}
}

Not sure about WebhookRequest and WebhookResponse.
The code below code might help you.
import org.springframework.http.HttpEntity;
#PostMapping("test1t")
public String getTest1(HttpEntity<String> httpEntity) {
String reqObject = httpEntity.getBody();
System.out.println("request json object = "+reqObject);
//Get the action
JSONObject obj = new JSONObject(reqObject);
String action = obj.getJSONObject("result").getString("action");
//Get the parameters
JSONObject params = obj.getJSONObject("result").getJSONObject("parameters");
String response = "Hello from Java.";
return "{'speech': '"+response+"', 'displayText':'"+response+"'}";
}

invalid hexadecimal representation of an ObjectId in Mongo DB Rest API

I am using REST API to fetch data using #Form param from Mongo DB and got the exception: 'Invalid hexadecimal representation of ObjectId'. The syntax seems to be correct, not sure whats going wrong there. I am passing new ObjectId (id) in the rest parameter. The code is as below:
//Country.java
package com.speed.infoaxon;
import java.io.IOException;
import java.net.UnknownHostException;
import org.bson.types.ObjectId;
import com.mongodb.BasicDBList;
import com.mongodb.BasicDBObject;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.DBCursor;
public class Country {
public BasicDBObject addDemo(long _id ) throws IOException {
DB db=ConnectToDB.getConnection();
DBCollection collection = db.getCollection("demo");
BasicDBObject buildList = null;
BasicDBObject document = new BasicDBObject();
document.put("_id",new ObjectId("id"));
collection.save(document);
return buildList;
}
}
//getResponse.java
package com.speed.infoaxon;
import java.io.IOException;
import java.net.UnknownHostException;
import javax.ws.rs.Consumes;
import javax.ws.rs.FormParam;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import com.mongodb.BasicDBList;
import com.mongodb.BasicDBObject;
#Path("/add")
public class GetResponse {
#POST
#Path("/addDemo")
#Consumes({MediaType.APPLICATION_JSON, MediaType.APPLICATION_FORM_URLENCODED} )
public BasicDBObject addDemo(#FormParam("_id") long _id) throws IOException
{
System.out.println("inside demo");
Country d = new Country();
BasicDBObject basicDBList=d.addDemo(_id);
return basicDBList;
}
}
Please let me know where is the issue. Thanks in advance.

document.put("_id",new ObjectId("id"));
you're using "id" in quotes which means its id in string you need to pass in the actual id

How to do a simple Google Cloud Trace request in Java

I am trying to perform a simple push traces operation to my Google Cloud Trace project and I simply can't seem to send data across.
Here is my build.gradle file:
apply plugin: 'java'
repositories {
mavenCentral()
}
dependencies {
compile 'com.google.oauth-client:google-oauth-client-java6:1.20.0'
compile 'com.google.apis:google-api-services-cloudtrace:v1-rev6-1.22.0'
}
jar {
from configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }
}
And the following Java code with dummy info for the project ID and my secrets file:
package test;
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.cloudtrace.v1.CloudTrace;
import com.google.api.services.cloudtrace.v1.CloudTraceScopes;
import com.google.api.services.cloudtrace.v1.model.Trace;
import com.google.api.services.cloudtrace.v1.model.TraceSpan;
import com.google.api.services.cloudtrace.v1.model.Traces;
import java.io.FileInputStream;
import java.io.IOException;
import java.security.GeneralSecurityException;
import java.time.Instant;
import java.util.Collections;
public class Test {
public static void main(String[] args) throws IOException, GeneralSecurityException {
HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
GoogleCredential cred = GoogleCredential
.fromStream(
new FileInputStream("/path/to/secrets.json"),
httpTransport,
jsonFactory)
.createScoped(Collections.singletonList(CloudTraceScopes.TRACE_APPEND));
CloudTrace gceTrace = new CloudTrace.Builder(httpTransport, jsonFactory, cred)
.setApplicationName("Google Cloud Trace test app")
.build();
TraceSpan span1 = new TraceSpan();
span1.setName("test");
span1.setStartTime(Long.toString(Instant.now().toEpochMilli()*1000000)+"Z");
Trace trace = new Trace();
trace.setSpans(Collections.singletonList(span1));
Traces traces = new Traces();
traces.setTraces(Collections.singletonList(trace));
gceTrace.projects().patchTraces("myprojectid", traces).execute();
}
}
I currently get the following error that contains no helpful indication except something seems to be wrong with my startTime value:
Exception in thread "main" com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Invalid value at 'traces.traces[0].spans[0].start_time' (type.googleapis.com/google.protobuf.Timestamp), Field 'startTime', Invalid time format: Failed to parse input",
"reason" : "badRequest"
} ],
"message" : "Invalid value at 'traces.traces[0].spans[0].start_time' (type.googleapis.com/google.protobuf.Timestamp), Field 'startTime', Invalid time format: Failed to parse input",
"status" : "INVALID_ARGUMENT"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at test.Test.main(Test.java:44)
I have tried to replace the startTime with the following value:
span1.setStartTime("2016-08-04T01:00:00Z");
which gives me:
Exception in thread "main" com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Request contains an invalid argument.",
"reason" : "badRequest"
} ],
"message" : "Request contains an invalid argument.",
"status" : "INVALID_ARGUMENT"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at test.Test.main(Test.java:44)
I also tried adding a endTime with:
span1.setEndTime("2016-08-04T01:00:01Z");
which also gives me the same error.
I'm pretty much at a lost at what needs to be done as I cannot find a single working Java example for this.
Thank you in advance for any pointers for a working solution.

Finally figured it out. Here's a working example with mandatory and optional fields pointed out.
build.gradle
apply plugin: 'java'
repositories {
mavenCentral()
}
dependencies {
compile 'com.google.oauth-client:google-oauth-client-java6:1.20.0'
compile 'com.google.apis:google-api-services-cloudtrace:v1-rev6-1.22.0'
}
jar {
from configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }
}
Test.java
package test;
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.cloudtrace.v1.CloudTrace;
import com.google.api.services.cloudtrace.v1.CloudTraceScopes;
import com.google.api.services.cloudtrace.v1.model.Trace;
import com.google.api.services.cloudtrace.v1.model.TraceSpan;
import com.google.api.services.cloudtrace.v1.model.Traces;
import java.io.FileInputStream;
import java.io.IOException;
import java.math.BigInteger;
import java.security.GeneralSecurityException;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
public class Test {
public static void main(String[] args) throws IOException, GeneralSecurityException {
HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
GoogleCredential cred = GoogleCredential
.fromStream(
new FileInputStream("/path/to/secrets.json"),
httpTransport,
jsonFactory)
.createScoped(Collections.singletonList(CloudTraceScopes.TRACE_APPEND));
CloudTrace gceTrace = new CloudTrace.Builder(httpTransport, jsonFactory, cred)
.setApplicationName("Google Cloud Trace test app")
.build();
// They are optional
Map<String, String> labels = new HashMap<>();
labels.put("key1", "val1");
TraceSpan span = new TraceSpan();
span.setSpanId(new BigInteger("1")); // Mandatory
span.setName("test"); // Optional
span.setKind("RPC_SERVER"); // Optional
span.setStartTime("2016-08-04T01:00:00Z"); // Optional
span.setEndTime("2016-08-04T01:00:01Z"); // Optional
span.setLabels(labels); // Optional
Trace trace = new Trace();
trace.setProjectId("myprojectid"); // Mandatory
trace.setTraceId("A096D4956A424EEB98AE7863505B1E1F"); // Mandatory
trace.setSpans(Collections.singletonList(span)); // Mandatory
Traces traces = new Traces();
traces.setTraces(Collections.singletonList(trace)); // Mandatory
gceTrace.projects().patchTraces("myprojectid", traces).execute();
}
}
While some values are optional, like startTime or endTime, it makes sense to put something there.
I managed to put it together thanks to this question showing the expected values and looking at the REST API doc describing each field, especially for cryptic values like Trace ID:
patchTraces()
Trace and TraceSpan