I'm trying to parse the request sent to a java based fulfillment in V2 of the API. I can't find any example documentation in Java for doing this in V2 of the API (com.google.cloud:google-cloud-dialogflow:0.38.0-alpha dependency in my project).
So far I've got as far as writing a very basic Spring MVC controller to accept the request.
How can I parse out the payload in the request, e.g. the parameters that dialog flow sent ?
import com.google.cloud.dialogflow.v2beta1.WebhookRequest;
import com.google.cloud.dialogflow.v2beta1.WebhookResponse;
import com.google.protobuf.Descriptors;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import javax.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.util.Map;
import java.util.stream.Collectors;
#RestController
#RequestMapping("test")
public class TestRequestRestController {
private static final Logger log = LoggerFactory.getLogger(TestRequestRestController.class);
#PostMapping("test1t")
public WebhookResponse getTest1(WebhookRequest request) {
System.out.println(request.toString());
return WebhookResponse.newBuilder().setFulfillmentText("Example reply 1 ").build();
}
}
Not sure about WebhookRequest and WebhookResponse.
The code below code might help you.
import org.springframework.http.HttpEntity;
#PostMapping("test1t")
public String getTest1(HttpEntity<String> httpEntity) {
String reqObject = httpEntity.getBody();
System.out.println("request json object = "+reqObject);
//Get the action
JSONObject obj = new JSONObject(reqObject);
String action = obj.getJSONObject("result").getString("action");
//Get the parameters
JSONObject params = obj.getJSONObject("result").getJSONObject("parameters");
String response = "Hello from Java.";
return "{'speech': '"+response+"', 'displayText':'"+response+"'}";
}
Related
I have an application running on port 7070 on my local. It exposes and endpoint /metrics and shows all the tags that are available. Prometheus is not able to get these data and it says 'expected label name, got "BCLOSE"'.
I have been trying to figure this out but not sure why this code doesn't work:
import io.micrometer.prometheus.PrometheusMeterRegistry;
import io.prometheus.client.exporter.common.TextFormat;
import metrics.PrometheusRegistry;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.HttpHeaders;
import javax.ws.rs.core.Response;
import java.io.IOException;
import java.io.StringWriter;
import java.io.Writer;
#Path("/metrics")
public class MetricsController {
private final PrometheusMeterRegistry prometheusRegistry = PrometheusRegistry.INSTANCE();
#GET
public Response getMetrics(#Context HttpHeaders headers) {
Writer writer = new StringWriter();
try {
TextFormat.write004(writer, prometheusRegistry.getPrometheusRegistry().metricFamilySamples());
} catch (IOException e) {
e.printStackTrace();
}
return writer.toString();
}
}
Also, the application is neither a sprintboot nor a spring project.
Tried this:
#GET
public Response getMetrics(#Context HttpHeaders headers) {
String accept = headers.getRequestHeader("Accept").get(0);
System.out.println("Accept Header --------------------------> " + accept);
return Response.ok(prometheusRegistry.scrape(), "application/openmetrics-text").build();
}
Even then the same error as above SS.
This worked for me:
#GET
public Response getMetrics() {
return Response.ok(prometheusRegistry.scrape(), TextFormat.CONTENT_TYPE_004).build();
}
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 1 year ago.
Improve this question
I got a task at school in which I have to do the following:
Implement the RESTful endpoint API, which simultaneously makes calls
to the following websites:
https://pizzerijalimbo.si/meni/
https://pizzerijalimbo.si/kontakt/
https://pizzerijalimbo.si/my-account/
https://pizzerijalimbo.si/o-nas/
The input for the endpoint is ‘integer’, which represents the number
of simultaneous calls to the above web pages (min 1 represents all
consecutive calls, max 4 represents all simultaneous calls).
Extracts a short title text from each page and saves this text in a
common global structure (array, folder (). The program should also
count successful calls. Finally, the service should list the number of
successful calls, the number of failed calls and the saved address
texts from all web pages.
With some help I managed to do something, but I still need help with data exstraction using Jsoup or any other method.
Here is the code that I have:
import java.util.Arrays;
import java.util.List;
import java.io.IOException;
import java.net.URL;
import java.util.Scanner;
#RestController
public class APIcontroller {
#Autowired
private RestTemplate restTemplate;
List<String> websites = Arrays.asList("https://pizzerijalimbo.si/meni/",
"https://pizzerijalimbo.si/kontakt/",
"https://pizzerijalimbo.si/my-account/",
"https://pizzerijalimbo.si/o-nas/");
#GetMapping("/podatki")
public List<Object> getData(#RequestParam(required = true) int numberOfWebsites) {
List<String> websitesToScrape = websites.subList(0, numberOfWebsites);
for (String website : websitesToScrape) {
Document doc = Jsoup.connect("https://pizzerijalimbo.si/meni/").get();
log(doc.title());
Elements newsHeadlines = doc.select("#mp-itn b a");
for (Element headline : newsHeadlines) {
log("%s\n\t%s",
headline.attr("title"), headline.absUrl("href"));
}
}
}
}
I also need to do it parallel, so the calls to a secific website go on at the same time.
But the main problem now is with the log funcion which does not work properly.
What I have tried:
I tried to solve the problem using Jsoup library, but I dont seem to
undersand it well, so I got an error in the for loop which says that
the method log is undefined. I also need to do a try catch to count possible failed calls and count the calls that are successfull as you can see in the task description.
WebScrapperController.java
package com.stackovertwo.stackovertwo;
import java.io.IOException;
import java.io.StringReader;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import org.jsoup.Jsoup;
import org.jsoup.select.Elements;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.client.RestTemplate;
//import org.w3c.dom.Document;
//import org.w3c.dom.DocumentFragment;
import org.jsoup.nodes.Document;
import org.w3c.dom.Node;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
#RestController
public class WebScrapperController {
#GetMapping("/")
public String index() {
return "Greetings from Spring Boot!";
}
// #Autowired
// private RestTemplate restTemplate;
#Autowired
WebScrapperService webScrapperService;
List<String> websites = Arrays.asList("https://pizzerijalimbo.si/meni/",
"https://pizzerijalimbo.si/kontakt/",
"https://pizzerijalimbo.si/my-account/",
"https://pizzerijalimbo.si/o-nas/");
#GetMapping("/podatki")
public ResponseEntity<Object> getData(#RequestParam(required = true) int numberOfWebsites) throws InterruptedException, ExecutionException {
List<SiteResponse> webSitesToScrape = new ArrayList<>();
// List<String> websitesToScrape = websites.subList(0, numberOfWebsites);
List<SiteResponse> responseResults = new ArrayList<SiteResponse>();
CompletableFuture<SiteResponse> futureData1 = webScrapperService.getWebScrappedContent(websites.get(0));
CompletableFuture<SiteResponse> futureData2 = webScrapperService.getWebScrappedContent(websites.get(1));
//CompletableFuture.allOf(futureData1, futureData2).join();
webSitesToScrape.add(futureData1.get());
webSitesToScrape.add(futureData2.get());
List<SiteResponse> result = webSitesToScrape.stream().collect(Collectors.toList());
return ResponseEntity.ok().body(result);
}
}
WebScrapperService.java
package com.stackovertwo.stackovertwo;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.select.Elements;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.ResponseEntity;
import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;
import java.util.concurrent.CompletableFuture;
#Service
public class WebScrapperService {
#Autowired
private RestTemplate restTemplate;
Logger logger = LoggerFactory.getLogger(WebScrapperService.class);
#Async
public CompletableFuture<SiteResponse> getWebScrappedContent(String webSiteURL)
//throws InterruptedException
{
logger.info("Starting: getWebScrappedContent for webSiteURL {} with thread {}", webSiteURL, Thread.currentThread().getName());
HttpEntity<String> response = restTemplate.exchange(webSiteURL,
HttpMethod.GET, null, String.class);
//Thread.sleep(1000);
SiteResponse webSiteSummary = null ;
String resultString = response.getBody();
HttpHeaders headers = response.getHeaders();
int statusCode = ((ResponseEntity<String>) response).getStatusCode().value();
System.out.println(statusCode);
System.out.println("HEADERS"+headers);
try
{
Document doc = (Document) Jsoup.parse(resultString);
Elements header = doc.select(".elementor-inner h2.elementor-heading-title.elementor-size-default");
System.out.println(header.get(0).html());
// Return the fragment.
webSiteSummary = new SiteResponse(statusCode, header.get(0).html());
}
catch(Exception e) {
System.out.println("Exception "+e.getMessage());
}
logger.info("Complete: getWebScrappedContent for webSiteURL {} with thread {}", webSiteURL, Thread.currentThread().getName());
return CompletableFuture.completedFuture(webSiteSummary);
}
}
SpringBootApp.java
package com.stackovertwo.stackovertwo;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.web.client.RestTemplate;
import java.security.KeyManagementException;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.X509Certificate;
//import javax.net.ssl.HostnameVerifier;
//import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
//import javax.net.ssl.SSLSession;
//import javax.net.ssl.TrustManager;
//import javax.net.ssl.X509TrustManager;
//import javax.security.cert.X509Certificate;
import org.apache.http.conn.ssl.TrustStrategy;
import org.apache.http.impl.client.*;
import org.apache.http.conn.ssl.*;
#SpringBootApplication
public class SpringBootApp
{
public static void main(String[] args)
{
SpringApplication.run(SpringBootApp.class, args);
}
#Bean
public RestTemplate restTemplate() throws KeyManagementException, NoSuchAlgorithmException, KeyStoreException {
TrustStrategy acceptingTrustStrategy = (X509Certificate[] chain, String authType) -> true;
SSLContext sslContext = org.apache.http.ssl.SSLContexts.custom()
.loadTrustMaterial(null, acceptingTrustStrategy)
.build();
SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(sslContext);
CloseableHttpClient httpClient = HttpClients.custom()
.setSSLSocketFactory(csf)
.build();
HttpComponentsClientHttpRequestFactory requestFactory =
new HttpComponentsClientHttpRequestFactory();
requestFactory.setHttpClient(httpClient);
//return new RestTemplate();
RestTemplate restTemplate = new RestTemplate(requestFactory);
return restTemplate;
}
}
Note: I disabled SSL verification while calling the webulr in resttemplate, but its not recommendd inproduction (For assignment its ok). But you need to import the keys via java keystore in case production : https://myshittycode.com/2015/12/17/java-https-unable-to-find-valid-certification-path-to-requested-target-2/
I try to build API with springboot, when i make controller and using response.response.setService/.setMesage/.setData i get error "The method setService(String) is undefined for the type Response", please help me to solve this problem.
package com.example.restapi.controller;
import com.example.restapi.entity.Hardware;
import com.example.restapi.service.HardwareService;
import com.example.restapi.util.Response;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.validation.annotation.Validated;
import org.springframework.web.bind.annotation.*;
#RestController
#RequestMapping(value = "hardware")
public class HardwareController {
#Autowired
HardwareService hardwareService;
private String serviceString = "Hardware";
ResponseEntity<Response> create (#RequestBody #Validated Hardware hardware)
{
String nameofCurrMethod = new Throwable()
.getStackTrace()[0]
.getMethodName();
Response response = new Response();
response.setService(this.getClass().getName() + nameofCurrMethod); <==This is error code
response.setMessage("Berhasil Membuat Data"); <==This is error code
response.setData(hardwareService.create(hardware)); <==This is error code
return ResponseEntity
.status(HttpStatus.OK)
.contentType(MediaType.APPLICATION_JSON)
.body(response);
}
}
I want to make a rest call to bitbucket api with ssh private key as header.For example the url is (http://bitbucket.com/rest/api/1.0/repos/testProject/pull-requests?state=OPEN).
Is there a way call this url with spring rest template and how to pass ssl access key as header.
Instead of using SSH keys use Personal Access Tokens:
https://confluence.atlassian.com/bitbucketserver/personal-access-tokens-939515499.html
(Introduced in Bitbucket 5.5)
Then you could use code like this:
package com.company.bitbucket.tools.application;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.StringJoiner;
import java.util.stream.Collectors;
import org.springframework.http.client.ClientHttpRequestInterceptor;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.servlet.ModelAndView;
import com.company.bitbucket.tools.HttpRequestInterceptor;
import com.company.bitbucket.tools.ProjectValue;
import com.company.bitbucket.tools.Projects;
import com.company.bitbucket.tools.UserValue;
import com.company.bitbucket.tools.Users;
#Controller
public class ProjectController {
public static String BITBUCKET_URL = "https://bitbucket.company.com/rest/api/latest/";
public static String PROJECTS = "projects";
public static String PERMISSIONS = "permissions/users?permission=PROJECT_ADMIN";
public static String PAT = "<put your generated token in here>";
#RequestMapping(value={"/projects"}, method = RequestMethod.GET)
public ModelAndView listProjects(){
HashMap<String, String> list = getAdmins();
ModelAndView model = new ModelAndView("projects");
model.addObject("adminMap", list);
return model;
}
private HashMap<String, String> getAdmins(){
HashMap<String, String> projectMap = new HashMap<>();
RestTemplate restTemplate = new RestTemplate();
List<ClientHttpRequestInterceptor> interceptors = new ArrayList<ClientHttpRequestInterceptor>();
interceptors.add(new HttpRequestInterceptor("Authorization", "Bearer ".concat(PAT)));
restTemplate.setInterceptors(interceptors);
Projects projects = restTemplate.getForObject(BITBUCKET_URL.concat("projects?limit=100"), Projects.class);
for(ProjectValue projectValue: projects.getValues()) {
String projectUrl = String.format("%s/%s/%s/%s", BITBUCKET_URL, PROJECTS, projectValue.getKey(), PERMISSIONS);
Users users = restTemplate.getForObject(projectUrl, Users.class);
List<String> names = new ArrayList<>();
for (UserValue value: users.getValues()) {
names.add(value.getUser().getDisplayName());
}
String commaSeparatedNames = String.join(", ", names);
projectMap.put(projectValue.getName(), commaSeparatedNames);
}
return projectMap;
}
}
This code gets a list of project admins using the rest api, but you could change to make whatever rest request you wish.
I am retrieving URL from database and it contains special characters like % - = / ? etc. so while I try to populate that in my page JSON is not able to parse that URL and it is giving me some exception when I was debugging I got like this
(Log4JLogger.java:log:449) [SECURITY FAILURE Anonymous:null#unknown -> /ExampleApplication/IntrusionException] INTRUSION - Mixed encoding (2x) detected
I tried from these link here this my offending URL here
but it is not working he mentioned that bug is solved. but for which version version here I am using is 2.1.0 below is my code
package com.ghn.repufact.review.extractor;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.List;
import javax.ws.rs.core.UriBuilder;
import org.apache.http.NameValuePair;
import org.apache.http.client.utils.URLEncodedUtils;
import org.apache.log4j.Logger;
import org.owasp.esapi.ESAPI;
import org.owasp.esapi.Encoder;
import org.owasp.esapi.Validator;
import org.owasp.esapi.errors.EncodingException;
import org.owasp.esapi.reference.DefaultValidator;
import org.springframework.stereotype.Component;
#Component
public class ValidateURL {
private static final Logger logger=Logger.getLogger(ValidateResponse.class);
public String parseOrgConsumerLink(String myLink) throws URISyntaxException {
if(myLink==null || "".equalsIgnoreCase(myLink))
return myLink;
Encoder enc=ESAPI.encoder();
URI mixURI=new URI(myLink);
UriBuilder uriBuider=UriBuilder.fromUri(enc.canonicalize(mixURI.getAuthority()+mixURI.getPath()));
uriBuider.path(enc.canonicalize(mixURI.getAuthority() + mixURI.getPath()));
logger.info("Uri after URIbuilder:"+uriBuider.build().toString());
List<NameValuePair> params = URLEncodedUtils.parse(mixURI, "UTF-8");
for (NameValuePair nameValuePair : params)
uriBuider.queryParam(enc.canonicalize(nameValuePair.getName()), enc.canonicalize(nameValuePair.getValue()));
String canonicalizedUrl = uriBuider.build().toString();
logger.info("canonicaliz URI:"+canonicalizedUrl);
return canonicalizedUrl;
}
public boolean isCanonicalizedURI(String myLink) throws EncodingException {
Validator validator=DefaultValidator.getInstance();
//boolean flag=validator.isValidInput("test", myLink, "URI", 200, false);
myLink = ESAPI.encoder().encodeForURL(myLink);
boolean flag = validator.isValidInput("APPNAME", myLink, "URLSTRING", 600, true, false);
logger.info("checking for URI:"+myLink+" isCanonical:"+flag);
return flag;
}
}
please let me know if any work around here. By the way I am using spring MVC