I'm trying to use Github APIs to get a list of the SHAs corresponding to each commit on a given repository.
I've tried using this API:
https://api.github.com/repos/:owner/:repo/commits
And this return a list of commit object each of them containg its SHA.
The problem is that I perform this API request through a java program but the response always gets empty body and 200 as status code.
The program I'm using is this:
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.net.http.HttpResponse.BodyHandlers;
public class IntermediateStructureMaker {
public static void main(String[] args) {
// create client
HttpClient client = HttpClient.newHttpClient();
// create request
HttpRequest request = HttpRequest
.newBuilder().uri(URI.create("https://api.github.com/repos/jithin-renji/Nuke/commits"))
.build();
// use the client to send the request
client.sendAsync(request, BodyHandlers.ofString())
.thenApply(HttpResponse::body)
.thenAccept(System.out::println)
.join();
}
}
If I set the parameter ?per_page=1 in the request, I get the expected response; so I guess it's a problem of size of the response.
As solution I thought to only ask the SHA of the commits through the API request, is it possible?
Related
I am writing java HttpClient code, to query splunk API, and get search id (sid) as output.
I was able to write this in curl and python, with no issues.
But java is proving to be difficult.
Curl: (Working. Got the sid as output)
curl -u user https://url:8089/services/search/jobs -d"search=|tstats count where index=main"
**output:**
<response>
<sid>1352061658.136</sid>
</response>
Python: (Working. Got sid as output)
import json
import requests
baseurl = 'https://url:8089/services/search/jobs'
username = 'my_username'
password = 'my_password'
payload = {
"search": "|tstats count where index=main",
"count": 0,
"output_mode": "json"
}
headers={"Content-Type": "application/x-www-form-urlencoded"}
response = requests.post(url, auth=(userid,password), data=payload, headers=headers, verify=False)
print(response.status_code)
print(response.text)
Java: (Not working. No matter what request payload, we POST, getting list of all SPlunk jobs, instead of sid, like what we see in curl or python)
import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
public class HttpClientPostJSON {
private static final HttpClient httpClient = HttpClient.newBuilder()
.authenticator(new Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(
"user",
"password".toCharArray());
}
})
.build();
public static void main(String[] args) throws IOException, InterruptedException {
// json formatted data
String json = new StringBuilder()
.append("{")
.append("\"search\":\"|tstats count where index=main\"")
.append("}").toString();
// add json header
HttpRequest request = HttpRequest.newBuilder()
.POST(HttpRequest.BodyPublishers.ofString(json))
.header("Content-Type", "application/x-www-form-urlencoded")
.uri(URI.create("https://url:8089/services/search/jobs"))
.build();
HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
// print status code
System.out.println(response.statusCode());
// print response body
System.out.println(response.body());
}
}
What is the issue with java code? is there a better way to pass payload?
Why am i not gettting splunk search id (sid) as output.
I see some 20MB+ output that is listing all the jobs in the splunk.
Your payload is JSON text, but the mime-type indicates it would consist of urlencoded key-value pairs. The python code results in a x-www-form-urlencoded body:
search=%7Ctstats+count+where+index%3Dmain&count=0&output_mode=json
If you assign this value to the json-String (rename it, please) in the main-method like
String json = "search=%7Ctstats+count+where+index%3Dmain&count=0&output_mode=json";
the payload matches the mime-type.
I'm trying to learn a bit of Java and readthe documentation for the httpClient method since I'm trying to make a call to an endpoint created with Soap UI.
I'm just trying to obtain a response the JSON object but for some reason I keep getting an empty response and a 404 status code. I've been tinkering with it a bit but I can't seem to find the problem.
Here's my code
package org.example;
import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
public class App {
private static final String GET_API_URL = "http://localhost:8080/balance?acctnum=1234567";
public static void main( String[] args ) throws IOException, InterruptedException {
HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder()
.GET()
.header( "accept", "application/json")
.uri(URI.create(GET_API_URL))
.build();
HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println("status:" + response.statusCode());
System.out.println("response:" + response.body());
}
}
both the system.out are being hit but as mentioned the first one returns status 404 and the second one returns "response: ".
Any suggestions?
I am using some external API to GET and POST some ressources, locally my code works fine with the call of different endPoints (GET, POST...) and even with Postman, but when i try to run my code in another platerform (where the ressources are), i get the 412 HTTP error due to a POST call : after looking on the internet, i found out that i should generate an ETagd of the entity (that i went to modify) and add it into the header of my POST endPoint.
For that, i used ShallowEtagHeaderFilter and the #Bean annotation(above the filter method) and the #SpringBootApplication annotation above my class, here is my code :
package main.Runners;
import io.testproject.java.annotations.v2.Parameter;
import okhttp3.*;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.configurationprocessor.json.JSONArray;
import org.springframework.boot.configurationprocessor.json.JSONObject;
import org.springframework.context.annotation.Bean;
import org.springframework.web.filter.ShallowEtagHeaderFilter;
import javax.servlet.Filter;
#SpringBootApplication
public class ActionRunner {
#Parameter(description = "the project ID")
public static String projectId = "xxx";
#Parameter(description = "the test ID")
public static String testId = "yyy";
public static void main(String[] args) throws Exception {
try {
OkHttpClient client = new OkHttpClient().newBuilder()
.build();
Request request = new Request.Builder()
.url("https://api.testproject.io/v2/projects/"+projectId+"/tests/"+testId)
.method("GET", null)
.addHeader("Authorization", "nzmo4DI08ykizYgcp9-5cCTArlxq7k7zt9MYhGmTcRk1")
.build();
Response response = client.newCall(request).execute();
System.out.println("================ this is our response headers ::: \n"+ response.headers());
} catch(Exception e) {
System.out.println(e);
}
}
#Bean
public ShallowEtagHeaderFilter shallowEtagHeaderFilter(){
return new ShallowEtagHeaderFilter();
}
}
I really need Your help since i cant generate any ETag parameter on my GET response header(after checking reponse.headers() ).
Thanks in advance!
I need to call Oauth2 ResT API service to fetch the access token and expire_in values from the JSON file by it.
Below is a sample CURL which i need to call using JAVA i am beginner in JAVA so not able to figure out how to do it however i can do it using shell script.
curl -u 'ClientId:Clientaccesskey' https://oauth2.url/oauth/token -X POST -d 'response_type=token&client_id=ClientId&username=user&password=userpassword&scope=process&grant_type=password'
Sample JSON retured by above curl command --
{"access_token":"accessTokentobefetched","token_type":"bearer","refresh_token":"refreshToken","expires_in":7199,"scope":"process","jti":"somehexadecimalvaliu"}
In shell script we can fetch the value of access token and other fields using AWK command and other commands.
So i need to call this CURL command in JAVA and fetch the value of access token and other keys from the JSON file.
Any help which can help me start with this is welcome as i am new to JAVA and learning.
There are quite a few libraries that you can use to help you make a regular HTTP POST request from Java, but since you seem to require to send plain text/plain body content - I suggest that you use okhttp3. This is a fairly lightweight and easy to work with HTTP client.
You will need to add the following dependency to your pom.xml, grabbed from https://mvnrepository.com/artifact/com.squareup.okhttp3/okhttp/4.7.2:
<!-- https://mvnrepository.com/artifact/com.squareup.okhttp3/okhttp -->
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>4.7.2</version>
</dependency>
If you are using gradle, just visit the before mentioned URL, and get the gradle equivalent dependency declaration.
And here's a complete class that illustrates how the okhttp3 client can be used to perform the POST request, and extract the return value. This example expects that you are using the spring-boot-starter-web dependency (this will include the jackson and tomcat libraries that are used in the example).
package com.example.demo;
import com.fasterxml.jackson.databind.ObjectMapper;
import okhttp3.*;
import org.apache.tomcat.util.codec.binary.Base64;
import org.springframework.stereotype.Component;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.Map;
#Component
public class TokenRequester {
public String getAccessToken() throws IOException {
// Create a new HTTP client
OkHttpClient client = new OkHttpClient().newBuilder().build();
// Create the request body
MediaType mediaType = MediaType.parse("text/plain");
RequestBody body = RequestBody.create(mediaType, "response_type=token&client_id=ClientId&username=user&password=userpassword&scope=process&grant_type=password");
// Build the request object, with method, headers
Request request = new Request.Builder()
.url("https://oauth2.url/oauth/token")
.method("POST", body)
.addHeader("Authorization", createAuthHeaderString("ClientId", "Clientaccesskey"))
.addHeader("Content-Type", "text/plain")
.build();
// Perform the request, this potentially throws an IOException
Response response = client.newCall(request).execute();
// Read the body of the response into a hashmap
Map<String,Object> responseMap = new ObjectMapper().
readValue(response.body().byteStream(), HashMap.class);
// Read the value of the "access_token" key from the hashmap
String accessToken = (String)responseMap.get("access_token");
// Return the access_token value
return accessToken;
}
// Just a helper metod to create the basic auth header
private String createAuthHeaderString(String username, String password) {
String auth = username + ":" + password;
byte[] encodedAuth = Base64.encodeBase64(auth.getBytes(StandardCharsets.US_ASCII));
String authHeader = "Basic " + new String(encodedAuth);
return authHeader;
}
}
You may need to tweak a few things here. I could ask you to supply me the verbose output from the curl command, in order to be sure about the encoding - but give this one a try and see what you get?
Here's a solution that involves only Spring, using a RestTemplate for the POST request.
I found that when you use curl -X POST -d 'key=data', curl will add the header content-type: application/x-www-form-urlencoded, so the solution here will do the same.
This solution sets up the RestTemplate with the headers and body you have specified, and captures the response in an object equivalent to the one you have described.
The following solution consists of two files that you can try to introduce into your solution:
RestTemplateTokenRequester.java
package com.example.demo;
import org.apache.tomcat.util.codec.binary.Base64;
import org.springframework.http.*;
import org.springframework.stereotype.Component;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.util.MultiValueMap;
import org.springframework.web.client.RestTemplate;
import java.nio.charset.StandardCharsets;
#Component
public class RestTemplateTokenRequester {
public TokenResponse requestAccessToken() {
// Create a RestTemplate to describe the request
RestTemplate restTemplate = new RestTemplate();
// Specify the http headers that we want to attach to the request
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_FORM_URLENCODED);
headers.add("Authorization", createAuthHeaderString("ClientId", "Clientaccesskey"));
// Create a map of the key/value pairs that we want to supply in the body of the request
MultiValueMap<String, String> map = new LinkedMultiValueMap<>();
map.add("response_type","token");
map.add("client_id","ClientId");
map.add("username","user");
map.add("password","userpassword");
map.add("scope","process");
map.add("grant_type","password");
// Create an HttpEntity object, wrapping the body and headers of the request
HttpEntity<MultiValueMap<String, String>> entity = new HttpEntity<>(map, headers);
// Execute the request, as a POSt, and expecting a TokenResponse object in return
ResponseEntity<TokenResponse> response =
restTemplate.exchange("https://oauth2.url/oauth/token",
HttpMethod.POST,
entity,
TokenResponse.class);
return response.getBody();
}
// Just a helper metod to create the basic auth header
private String createAuthHeaderString(String username, String password) {
String auth = username + ":" + password;
byte[] encodedAuth = Base64.encodeBase64(auth.getBytes(StandardCharsets.US_ASCII));
String authHeader = "Basic " + new String(encodedAuth);
return authHeader;
}
}
TokenResponse.java
This is simply a POJO that is used by the jackson mapper, to capture the response in an object that you can easily read your result from.
package com.example.demo;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonProperty;
#JsonIgnoreProperties(ignoreUnknown = true)
public class TokenResponse {
#JsonProperty("access_token")
private String accessToken;
#JsonProperty("token_type")
private String tokenType;
#JsonProperty("refresh_token")
private String refreshToken;
#JsonProperty("expires_in")
private Integer expiresIn;
#JsonProperty("scope")
private String scope;
#JsonProperty("jti")
private String jti;
}
I hope this solution will help you - I would prefer it over the other solution I have suggested with okhttp3.
curl is a HTTP client.
better solution is using HTTP client APIs for java to call endpoints.
RestTemplate is common HTTP client comes with spring and it is your best choice.
Say that Java application makes requests to http://www.google.com/... and there's no way to configure the inherited library (making such requests internally), so I can not stub or replace this URL.
Please, share some best practices to create a mock like
whenCalling("http://www.google.com/some/path").withMethod("GET").thenExpectResponse("HELLO")
so a request made by any HTTP client to this URL would be redirected to the mock and replaced with this response "HELLO" in the context of current JVM process.
I tried to find a solution using WireMock, Mockito or Hoverfly, but it seems that they do something different from that. Probably I just failed to use them properly.
Could you show a simple set up from the main method like:
create mock
start mock simulation
make a request to the URL by an arbitrary HTTP client (not entangled with the mocking library)
receive mocked response
stop mock simulation
make the same request as on step 3
receive real response from URL
Here's how to achieve what you want with the API Simulator.
The example demonstrates two different ways to configure Embedded API Simulator as HTTP proxy for the Spring's RestTemplate client. Check with the documentation of the (quote from the question) "inherited library" - often times Java-based clients rely on system properties described here or may offer some way to configure HTTP proxy with code.
package others;
import static com.apisimulator.embedded.SuchThat.*;
import static com.apisimulator.embedded.http.HttpApiSimulation.*;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.net.Proxy.Type;
import java.net.URI;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.ClassRule;
import org.junit.Test;
import org.springframework.http.ResponseEntity;
import org.springframework.http.client.SimpleClientHttpRequestFactory;
import org.springframework.web.client.RestTemplate;
import com.apisimulator.embedded.http.JUnitHttpApiSimulation;
public class EmbeddedSimulatorAsProxyTest
{
// Configure an API simulation. This starts an instance of
// Embedded API Simulator on localhost, default port 6090.
// The instance is automatically stopped when the test ends.
#ClassRule
public static final JUnitHttpApiSimulation apiSimulation = JUnitHttpApiSimulation
.as(httpApiSimulation("my-sim"));
#BeforeClass
public static void beforeClass()
{
// Configure simlets for the API simulation
// #formatter:off
apiSimulation.add(simlet("http-proxy")
.when(httpRequest("CONNECT"))
.then(httpResponse(200))
);
apiSimulation.add(simlet("test-google")
.when(httpRequest()
.whereMethod("GET")
.whereUriPath(isEqualTo("/some/path"))
.whereHeader("Host", contains("google.com"))
)
.then(httpResponse()
.withStatus(200)
.withHeader("Content-Type", "application/text")
.withBody("HELLO")
)
);
// #formatter:on
}
#Test
public void test_using_system_properties() throws Exception
{
try
{
// Set these system properties just for this test
System.setProperty("http.proxyHost", "localhost");
System.setProperty("http.proxyPort", "6090");
RestTemplate restTemplate = new RestTemplate();
URI uri = new URI("http://www.google.com/some/path");
ResponseEntity<String> response = restTemplate.getForEntity(uri, String.class);
Assert.assertEquals(200, response.getStatusCode().value());
Assert.assertEquals("HELLO", response.getBody());
}
finally
{
System.clearProperty("http.proxyHost");
System.clearProperty("http.proxyPort");
}
}
#Test
public void test_using_java_net_proxy() throws Exception
{
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
// A way to configure API Simulator as HTTP proxy if the HTTP client supports it
Proxy proxy = new Proxy(Type.HTTP, new InetSocketAddress("localhost", 6090));
requestFactory.setProxy(proxy);
RestTemplate restTemplate = new RestTemplate();
restTemplate.setRequestFactory(requestFactory);
URI uri = new URI("http://www.google.com/some/path");
ResponseEntity<String> response = restTemplate.getForEntity(uri, String.class);
Assert.assertEquals(200, response.getStatusCode().value());
Assert.assertEquals("HELLO", response.getBody());
}
#Test
public void test_direct_call() throws Exception
{
RestTemplate restTemplate = new RestTemplate();
URI uri = new URI("http://www.google.com");
ResponseEntity<String> response = restTemplate.getForEntity(uri, String.class);
Assert.assertEquals(200, response.getStatusCode().value());
Assert.assertTrue(response.getBody().startsWith("<!doctype html>"));
}
}
When using maven, add the following to project's pom.xml to include the Embedded API Simulator as a dependency:
<dependency>
<groupId>com.apisimulator</groupId>
<artifactId>apisimulator-http-embedded</artifactId>
<version>1.6</version>
</dependency>
... and this to point to the repository:
<repositories>
<repository>
<id>apisimulator-github-repo</id>
<url>https://github.com/apimastery/APISimulator/raw/maven-repository</url>
</repository>
</repositories>