I'm struggling with invoking GCP cloud functions via REST API using Java.
The steps that I've performed to do it were:
create a service account with role "Cloud Functions Invoker"
download JSON key file for the newly created service account
in my code, obtain an access token using the following method:
private String getAuthToken() {
File credentialsPath = new File(PATH_TO_JSON_KEY_FILE);
GoogleCredentials credentials;
try (FileInputStream serviceAccountStream = new FileInputStream(credentialsPath)) {
credentials = ServiceAccountCredentials.fromStream(serviceAccountStream);
return credentials
.createScoped(Lists.newArrayList("https://www.googleapis.com/auth/cloud-platform"))
.refreshAccessToken()
.getTokenValue();
} catch (IOException e) {
throw new RuntimeException("Action could not be performed");
}
}
perform a REST call, using the created token:
public <Payload, Response> ResponseEntity<Response> callCloudFunction(
String endpoint,
Payload payload,
Class<Response> klazz
) {
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Collections.singletonList(MediaType.APPLICATION_JSON));
headers.setContentType(MediaType.APPLICATION_JSON);
String url = gCloudUrl + endpoint;
String token = getAuthToken();
String payloadString = null;
if (payload != null) {
try {
ObjectMapper objectMapper = new ObjectMapper();
payloadString = objectMapper.writeValueAsString(payload);
} catch (JsonProcessingException e) {
System.out.println(e.getMessage());
throw new RuntimeException("Could not perform action");
}
}
headers.add("Authorization", String.format("Bearer %s", token));
HttpEntity<String> entity = new HttpEntity<>(payloadString, headers);
return restTemplate.exchange(url, HttpMethod.POST, entity, klazz);
}
The implementation looks fine, but in response I'm getting 401 Unauthorized.
Unfortunately, GCP documentation is not really helpful. I think I've searched through all the possible places.
First of all, agree, it's not clear...
Then, you have to know (and it's not clear again) that you need an access token to call Google Cloud API, but and identity token to call IAP (on App Engine for example) or private Cloud Function and Cloud Run. And this identity token need to be signed by Google.
And, as mentioned in the code, you need to have a service account on your computer, but I recommend you to avoid this on GCP, it's not required if you use default authentication (see my code, on your computer set the GOOGLE_APPLICATION_CREDENTIALS env var that points to the service account key file). The best way is to not use service account key file on your computer also, but it's not yet possible (that is a security issue IMO, and I'm discussing with Google on this...)
Anyway, here a code snippet which works in Java (nowhere in the documentation...)
String myUri = "https://path/to/url";
// You can use here your service account key file. But, on GCP you don't require a service account key file.
// However, on your computer, you require one because you need and identity token and you can generate it with your user account (long story... I'm still in discussion with Google about this point...)
Credentials credentials = GoogleCredentials.getApplicationDefault().createScoped("https://www.googleapis.com/auth/cloud-platform");
IdTokenCredentials idTokenCredentials = IdTokenCredentials.newBuilder()
.setIdTokenProvider((IdTokenProvider) credentials)
.setTargetAudience(myUri).build();
HttpRequestFactory factory = new NetHttpTransport().createRequestFactory(new HttpCredentialsAdapter(idTokenCredentials));
HttpRequest request = factory.buildGetRequest(new GenericUrl(myUri));
HttpResponse httpResponse = request.execute();
System.out.println(CharStreams.toString(new InputStreamReader(httpResponse.getContent(), Charsets.UTF_8)));
NOTE If you want to continue to use RestTemplate object and set manually your token, you can generate it like this
String token = ((IdTokenProvider) credentials).idTokenWithAudience(myUri, Collections.EMPTY_LIST).getTokenValue();
System.out.println(token);
Related
I am trying to call 3rd party rest api using rest template in aws lambda but connection timeout is happening. this third party api is working fine in postman.
Below is the code snippet
#Override
public APIGatewayProxyResponseEvent apply(APIGatewayProxyRequestEvent request) {
Gson gson = new Gson();
EmailRequest emailRequest = gson.fromJson(request.getBody(),EmailRequest.class);
System.out.println("Inside function");
APIGatewayProxyResponseEvent responseEvent = new APIGatewayProxyResponseEvent();
if(emailRequest!=null && emailRequest.getUserId() !=null && emailRequest.getPid()!=null) {
Optional<UserDetail> userDetail = userRepository.findById(emailRequest.getUserId());
Optional<Prospect> prospect = prospectRepository.findById(emailRequest.getPid());
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.setAccept(Collections.singletonList(MediaType.APPLICATION_JSON));
HttpEntity<Prospect> entity = new HttpEntity<>(prospect.get(), httpHeaders);
try {
ResponseEntity<String> result = restTemplate.postForEntity(userDetail.get().getCrmUrl(), entity, String.class);
if(result.getStatusCode().is5xxServerError()) {
responseEvent.setStatusCode(HttpStatus.INTERNAL_SERVER_ERROR.value());
responseEvent.withBody("Issue while sending email details");
}
responseEvent.setStatusCode(HttpStatus.OK.value());
} catch (ResourceAccessException e) {
responseEvent.setStatusCode(HttpStatus.GATEWAY_TIMEOUT.value());
responseEvent.withBody("Rest templete time out");
return responseEvent;
}
responseEvent.setStatusCode(HttpStatus.OK.value());
responseEvent.withBody("UserId crm : " + userDetail.get().getCrmUrl() +" and Prospect first name : " + prospect.get().getFirstName());
} else {
responseEvent.setStatusCode(HttpStatus.BAD_REQUEST.value());
responseEvent.withBody("Please check the request data");
}
return responseEvent;
}
Please help me out how can I access third party rest api in aws lambda.
This could be due to the fact that a lambda function in a VPC does not have access to the internet by default. From docs:
When you connect a function to a VPC in your account, the function can't access the internet unless your VPC provides access.
A popular way to overcome this is through NAT gateway and private subnet as described in:
How do I give internet access to a Lambda function that's connected to an Amazon VPC?
I am using a method where it calls another REST API to retrieve an ID from the DB. When I run the veracode scan for the class I am getting Security flaw "Server-side Request Forgery" at below line.
response = resttemplate.getForEntity(resturl, String.class);
Not sure How to fix this issue. Any help is appreciated. Below is my full code for that method.
public static String getIDFromDB(String resturl) {
String id = null;
RestTemplate resttemplate = new RestTemplate();
ResponseEntity<String> response = new ResponseEntity<>(HTTPStatus.OK)
try {
response = resttemplate.getForEntity(resturl, String.class);
if (response.getStatusCode == HTTPStatus.OK && response.getBody.trim() != null) {
id = response.getBody.trim() ;
}
} Catch(Exception e) {
log.error("failed to get msgID: {}", e);
}
}
This is because you are allowing in your code to pass the resturl completely in your code, so it enables the attacker to bypass and route the URL to their intended destination.
To avoid this, so should externalise and refer the URL having domain and the application contexts with operation name in config files or dB
I'm falling into a problem this morning with a custom request between two application, what i need to do is to let application able to talk eachother with two Rest API cause i need to do some actions on the first application by the second. The two applications are developed with springboot.
Suppose to call this two applications admin and superadmin
superadmin send a request with a RestAPI and a customized header -> name = key value = 1234
admin recieve the request and first of all check if the header is present or not, after that the header is finded it can proceed to do all the task.
Here's the code that i've developed :
SUPERADMIN :
#PostMapping(value="/test_api_header")
public ResponseEntity<String> test_API(#RequestParam String url) {
RestTemplate template = new RestTemplate();
URI targetUrl = UriComponentsBuilder.fromUriString(url) // Build the base link
.path("/test_API") // Add path
.build() // Build the URL
.encode() // Encode any URI items that need to be encoded
.toUri(); // Convert to URI
MultiValueMap<String, String> headers = new LinkedMultiValueMap<>();
headers.add("Content-Type", "application/json");
headers.add("superadminKey", "123456abc");
// build the request
ResponseEntity<String> entity = template.exchange(targetUrl, HttpMethod.GET, new HttpEntity<String>(headers), String.class);
return entity;
}
ADMIN :
#Value("123456abc")
private String saKey;
#GetMapping(value = "/superadmin/test_API")
public String test_API(HttpServletRequest request) {
if (request.getHeader("superadminKey") == saKey) {
return "Finally";
} else {
return "Nothing to do, header not present";
}
}
The SUPERADMIN is able to communicate with the RESTApi in the ADMIN application, in fact on postman i received the answer : Nothing to do, header not present, but i really cannot be able to set that customized header in the superadmin request cause i cannot found it also on postman request in the section "headers".
I've seen that i could also create a customized API Key for this special case, but really don't know how it works, if someone could help me I would be very grateful!
I am trying to consume the following HTTPS endpoints from Yahoo Weather Service:
Yahoo Weather Service API
I am doing some special query according to the API to get the current weather at some parametrized location.
#Service("weatherConditionService")
public class WeatherConditionServiceImpl implements WeatherConditionService {
private static final String URL = "http://query.yahooapis.com/v1/public/yql";
public WeatherCondition getCurrentWeatherConditionsFor(Location location) {
RestTemplate restTemplate = new RestTemplate();
StringBuilder stringBuilder = new StringBuilder();
stringBuilder.append(URL);
stringBuilder.append("?q=select%20item.condition%20from%20weather.forecast%20where%20woeid%20in%20(select%20woeid%20from%20geo.places(1)%20where%20text%3D%22");
// TODO: Validate YQL query injection
stringBuilder.append(location.getName());
stringBuilder.append("%22)&format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys");
WeatherQuery weatherQuery = restTemplate.getForObject(stringBuilder.toString(), WeatherQuery.class);
// TODO: Test Json mapping response
Condition condition = weatherQuery.getQuery().getResults().getChannel().getItem().getCondition();
return new WeatherCondition(condition.getDate(), Integer.parseInt(condition.getTemp()), condition.getText());
}
Location is a class that provides the attribute "name" that is a String description of the location, such as "New York" or "Manila".
Condition an other classes just map the returning object.
When executing I get the following HTTP response:
org.springframework.web.client.HttpClientErrorException: 403 Forbidden
So this means I am not authorized to access the resource from what I understand.
The URL works great if I just copy & paste it in a web browser:
Yahoo Weather Query
I think that mapping is not a problem since I am not getting "400" (Bad Request) but "403" (Forbidden)
There must be some error on the way I use the RestTemplate object. I am researching but I can't find an answer.
The docs say you need an api key. But when I make a call like this:
fetch('https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20weather.forecast%20where%20woeid%20in%20(select%20woeid%20from%20geo.places(1)%20where%20text%3D%22nome%2C%20ak%22)&format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys')
.then(resp=> resp.json())
.then((res)=>console.log(res.query.results))
https://repl.it/NeoM
It works fine without one. Perhaps you've been blackisted for hitting the api too often.
Your code seems fine.
I finally found the answer. It finally WAS a Bad Request because I needed to pass the parameters differently (not as part of the URL).
I found the answer here. Here goes the code for my particular Yahoo Weather API call return a String (I still will have to do some work to use the mapping).
private static final String URL = "http://query.yahooapis.com/v1/public/yql";
public String callYahooWeatherApi() {
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.set("Accept", MediaType.APPLICATION_JSON_VALUE);
UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(URL)
.queryParam("q", "select wind from weather.forecast where woeid=2460286")
.queryParam("format", "json");
HttpEntity<?> entity = new HttpEntity<>(headers);
HttpEntity<String> response = restTemplate.exchange(
builder.build().encode().toUri(),
HttpMethod.GET,
entity,
String.class);
return response.getBody();
}
I'm trying to send a post request to github to create a repository. I've got oauth 2.0 working and the request is correctly signed but github is just returning "Problems parsing JSON"
I'm using Scribe for the oauth side of things and as far as I can tell I've added JSON to the URL but I'm not 100% certain I'm doing it correctly, or am I just missing headers or something?
#POST
#Path("create_repo/{userid}")
#Produces(MediaType.APPLICATION_JSON)
public Response createRepo(#PathParam("userid") String userid) {
OAuthService service = createService().build();
User user = collection.findOneById(userid);
final OAuthRequest request = new OAuthRequest(Verb.POST, "https://api.github.com/user/repos", service);
Token token = new Token(user.getGithubToken(), "SECRET");
service.signRequest(token, request);
request.addHeader("Content-type", "application/vnd.github.v3+json");
request.addHeader("X-OAuth-Scopes", "repo");
request.addQuerystringParameter("name", "Test_v1");
LOGGER.info("Built request: " + request.getCompleteUrl());
final com.github.scribejava.core.model.Response response = request.send();
return Response.ok(response.getBody()).build();
}
The built URL looks like: https://api.github.com/user/repos?access_token=XXX_SECRET_XXX&name=Test_v1
I've also tried swapping the access_token after the params but same result.
Appreciate the any help.
Well I solved this by creating a object, serializing it, and adding it as a payload.
#POST
#Path("create_repo/{userId}/{projectId}")
#Produces(MediaType.APPLICATION_JSON)
public Response createRepo(#PathParam("userId") String userId, #PathParam("projectId") String projectId) {
// Setup collections
User user = userCollection.findOneById(userId);
ProjectDescription projectDescription = projectCollection.findOneById(projectId);
// Build repository object from project description
GithubRepository repository = new GithubRepository();
repository.setName(projectDescription.getTitle());
repository.setDescription(projectDescription.getDescription());
// Serialize object
ObjectMapper mapper = new ObjectMapper();
String jsonInString = null;
try {
jsonInString = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(repository);
} catch (JsonProcessingException e) {
e.printStackTrace();
}
// Build request
OAuthService service = createService().build();
final OAuthRequest request = new OAuthRequest(Verb.POST, PROTECTED_RESOURCE_URL + "/user/repos", service);
request.addHeader("content-type", "application/json");
request.addPayload(jsonInString);
// Sign and send request
Token token = new Token(user.getGithubToken(), "secret");
service.signRequest(token, request);
request.send();
return Response.status(201).build();
}
However, I'd still like to know where I went wrong with my first attempt.
Query string parameters are ignored in POST requests. That's why it worked when passing them in the request body.
From GitHub's API Overview docs:
Parameters
Many API methods take optional parameters. For GET requests, any parameters not specified as a segment in the path can be passed as an HTTP query string parameter:
curl -i "https://api.github.com/repos/vmg/redcarpet/issues?state=closed"
In this example, the ‘vmg’ and ‘redcarpet’ values are provided for the :owner and :repo parameters in the path while :state is passed in the query string.
For POST, PATCH, PUT, and DELETE requests, parameters not included in the URL should be encoded as JSON with a Content-Type of ‘application/json’:
$ curl -i -u username -d '{"scopes":["public_repo"]}' https://api.github.com/authorizations