I'm trying to download a zip file through a java program using GitHub API.
The program I'm using is the following:
public static void main(String[] args) {
// create client
HttpClient client = HttpClient.newHttpClient();
// create request
HttpRequest request = HttpRequest.newBuilder().uri(URI.create("https://api.github.com/repos/:owner/:repo/zipball/:ref")).build();
// use the client to send the asynchronous request
InputStream is = client.sendAsync(request, BodyHandlers.ofInputStream())
.thenApply(HttpResponse::body).join();
try {
FileOutputStream out = new FileOutputStream("outputZipFile.zip");
copy(is,out,1024);
out.close();
}catch(Exception e) {}
}
private static void copy(InputStream is, FileOutputStream out, int i) {
// TODO Auto-generated method stub
byte[] buf = new byte[i];
try {
int n = is.read(buf);
while(n>=0) {
out.write(buf,0,n);
n=is.read(buf);
}
out.flush();
}catch(IOException ioe) {
System.out.println(ioe.getStackTrace());
}
}
When I try to run this I get empty body so the output file will be empty as well.
I noticed that using HttpURLConnection insted of Java11 HttpClient makes it work but I'd prefer to use this Java11 feature in order to send asynchronous requests.
I can't understand what I'm doing wrong.
EDIT: The HttpURLConnection code I'm using at the moment is the following:
private void downloadVersion(String sha, String outputDestination) {
try {
URL url = new URL( getDownloadQuery(sha) );
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
if(authToken!=null)
connection.setRequestProperty("Authorization", "Bearer " + authToken) ;
connection.setRequestMethod("GET");
InputStream in = connection.getInputStream();
FileOutputStream out = new FileOutputStream(outputDestination);
copy(in, out, 1024);
out.close();
} catch (Exception e) {}
}
Your url (when set to correct github repos) may be returning redirect status 302. To make HTTP client follow redirects replace HttpClient client = HttpClient.newHttpClient() with use of HttpClient.newBuilder(). You can also simplify your code with try with resources and making use of InputStream.transferTo:
HttpClient client = HttpClient.newBuilder().followRedirects(Redirect.ALWAYS).build();
URI uri = URI.create("https://api.github.com/repos/:owner/:repo/zipball/:ref");
HttpRequest request = HttpRequest.newBuilder().uri(uri).build();
// use the client to send the asynchronous request
InputStream is = client.sendAsync(request, BodyHandlers.ofInputStream())
.thenApply(HttpResponse::body).join();
try (FileOutputStream out = new FileOutputStream("outputZipFile.zip")) {
is.transferTo(out);
}
Related
This should be an easy learning task but something went wrong:
Use API request to get image from some link and check it is in Base64 standard.
private static void getRequest() throws IOException {
URL url = new URL("http://apimeme.com/meme?meme=Alarm-Clock&top=Top+text&bottom=Bottom+text");
InputStream is = (InputStream) url.openStream();
FileInputStream imageInFile = new FileInputStream(is.toString());
byte imageData[] = new byte[2048];
imageInFile.read(imageData);
System.out.println(isBase64(imageInFile.toString()));
}
static boolean isBase64(String path) {
try {
Base64.getDecoder().decode(path);
return true;
} catch(IllegalArgumentException e) {
return false;
}
}
Right now problem is on line
InputStream is = (InputStream) url.openStream();
Server sends 403. How to fix this? Method isBase64 should work but the program doesn't reach this line.
Your need 3 header for avoid http 403 error code
private static void getRequest() throws IOException {
URL url = new URL("http://apimeme.com/meme?meme=Alarm-Clock&top=Top+text&bottom=Bottom+text");
URLConnection is = url.openConnection();
is.setRequestProperty("Host", "apimeme.com");
is.setRequestProperty("Cookie", "*");
is.setRequestProperty("User-Agent", "*");
byte[] imageData = new byte[2048];
File downloadFile = new File("download.png");
FileOutputStream outputStream = new FileOutputStream(downloadFile);
int read;
while ((read = is.getInputStream().read(imageData)) != -1) {
outputStream.write(imageData, 0, read);
}
outputStream.close();
System.out.println(isBase64(downloadFile.getAbsolutePath()));
}
403 is means that you do not have permission to view this URL. You cannot do anything to fix this client side, even authorization would not help (if the ressource required authorization, the server would send 401 instead). So to fix the issue, whoever is in charge of the web server delivering the URL you're trying to access will have to fix this - if they think you should have access.
I'm currently working on a project using an IoT platform " Thingsboard " where I've have created multiple devices, and I want to send data to each one of the devices from a JSON File, I'm using Rest Api to perform this request, but I've struggling for a while how to get the access token of my devices and parse each one of them in my request as a header param. I was just doing manually by getting them with Curl, but I want now to do it automatically. I know that Thingsboard has a Rest client Api written in java (https://thingsboard.io/docs/reference/rest-client/) so I've tried to use that in my script but I's not working. I'm new to working with Rest Api so if anybody can gie me a clue it would be so helpful.
here's a part of my code for the requests :
private static String token;
public String getToken() {
return token;
}
String paramValue = "param\\with\\backslash";
String yourURLStr = "http://host.com?param=" + java.net.URLEncoder.encode(paramValue, "UTF-8");
URL url2 = new URL("https://demo.thingsboard.io/api/v1/token/telemetry?token=$JWT_TOKEN");
HttpsURLConnection conn = (HttpsURLConnection) url2.openConnection();
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/json");
conn.setRequestProperty("Accept", "application/json");
conn.setRequestProperty(JWT_TOKEN_HEADER_PARAM, "Bearer" +token);
conn.setDoOutput(true);
OutputStream outStream = conn.getOutputStream();
OutputStreamWriter outStreamWriter = new OutputStreamWriter(outStream, "UTF-8");
outStreamWriter.write(list.toString());
outStreamWriter.flush();
outStreamWriter.close();
outStream.close();
String response = null;
System.out.println(conn.getResponseCode());
System.out.println(conn.getResponseMessage());
DataInputStream input1 = null;
input1 = new DataInputStream (conn.getInputStream());
while (null != ((response = input1.readLine()))) {
System.out.println(response);
input1.close ();
}
}
catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (ParseException e) {
e.printStackTrace();
}
If you are trying to get the JWT-token to authenticate the following java should work:
Request request = Request.Post("http://THINGSBOARD_URL/api/auth/login");
String body = "{\"username\":\"tenant#thingsboard.org\", \"password\":\"tenant\"}";
request.bodyString(body,ContentType.APPLICATION_JSON);
request.setHeader("Content-Type", "application/json");
request.setHeader("Accept", "application/json");
HttpResponse httpResponse = request.execute().returnResponse();
System.out.println(httpResponse.getStatusLine());
if (httpResponse.getEntity() != null) {
String html = EntityUtils.toString(httpResponse.getEntity());
System.out.println(html);
}
Don't get confused with JWT-Token for tenant authentication and Access-Token for Device Authentication.
I use a simple WebServer from http://www.java2s.com/Code/Java/Network-Protocol/AverysimpleWebserverWhenitreceivesaHTTPrequestitsendstherequestbackasthereply.htm
and Android code from Sending json object via http post method in android
In my main Activity:
AsyncT asyncT = new AsyncT();
asyncT.execute();
Class:
class AsyncT extends AsyncTask<Void,Void,Void>{
#Override
protected Void doInBackground(Void... params) {
try {
URL url = new URL(""); //Enter URL here
HttpURLConnection httpURLConnection = (HttpURLConnection)url.openConnection();
httpURLConnection.setDoOutput(true);
httpURLConnection.setRequestMethod("POST"); // here you are telling that it is a POST request, which can be changed into "PUT", "GET", "DELETE" etc.
httpURLConnection.setRequestProperty("Content-Type", "application/json"); // here you are setting the `Content-Type` for the data you are sending which is `application/json`
httpURLConnection.connect();
JSONObject jsonObject = new JSONObject();
jsonObject.put("para_1", "arg_1");
DataOutputStream wr = new DataOutputStream(httpURLConnection.getOutputStream());
wr.writeBytes(jsonObject.toString());
wr.flush();
wr.close();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (JSONException e) {
e.printStackTrace();
}
return null;
}
}
The connection is established without any errors ("HostConnection::get() New Host Connection established"). However, I am not able to get in my Java server any information from the request. When I read from input stream
BufferedReader in = new BufferedReader(new InputStreamReader(client.getInputStream()));
System.out.println(in);
I get java.io.BufferedReader#4d7hge12
And this outputs nothing:
String line;
while ((line = in.readLine()) != null) {
if (line.length() == 0)
break;
System.out.println(line);
}
Don't re-invent the wheel and use a library for this.
For example okhttp:
public static final MediaType JSON = MediaType.parse("application/json; charset=utf-8");
OkHttpClient client = new OkHttpClient();
String post(String url, String json) throws IOException {
RequestBody body = RequestBody.create(JSON, json);
Request request = new Request.Builder()
.url(url)
.post(body)
.build();
Response response = client.newCall(request).execute();
return response.body().string();
}
If you want to call a REST-API you can use retrofit (which is build ontop of okhttp)
Assuming you're doing this as a learning exercise, so using another library isn't what you're looking for, I would suggest a couple of things:
(1) install Wireshark and see what the actual response coming back the server is, does it look sensible?
(2) break that line of code out into separate lines, is the InputStream / InputStreamReader null?
I have created a Django Rest server. And I am trying to POST a JSON content in java using HttpURLConnection. However, when I am registering OutputStream in java, the server get a POST request with no JSON content. Hence the server rejecting the request with HTTP response code 400. I did write JSON data just after registering OutputSream. May be POST has been made before writing to OutputStream. The following is the code in JAVA.
public static void post(HttpURLConnection urlConnection1)
throws IOException, InterruptedException {
// urlConnection.setRequestMethod("POST");
URL url = new URL("http://127.0.0.1:8000/message/");
byte[] authStr=Base64.encodeBase64("sp:password".getBytes());
String enc=new String(authStr);
HttpURLConnection urlConnection = (HttpURLConnection) url
.openConnection();
urlConnection.setRequestProperty("Authorization", "Basic "+ enc);
try {
urlConnection.setDoOutput(true);
urlConnection.setChunkedStreamingMode(0);
urlConnection
.setRequestProperty("Content-Type", "application/json");
urlConnection.setRequestMethod("POST");
urlConnection.setInstanceFollowRedirects(false);
OutputStream out = urlConnection.getOutputStream();
//Thread.sleep(5000);
writeStream(out);
out.flush();
InputStream in = urlConnection.getInputStream();
readStream(in);
} finally {
urlConnection.disconnect();
}
}
private static void writeStream(OutputStream out) {
// TODO Auto-generated method stub
try {
JSONObject grp = new JSONObject();
JSONObject gp = new JSONObject();
gp.put("id", "g3bj25");
gp.put("from", "someone");
out.write(gp.toString().getBytes());
System.out.println(gp.toString());
} catch (IOException | JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Do I have to make any changes in server side so that it waits for content?
The problem is, that your URL is not complete, the name of the script is missing.
URL url = new URL("http://127.0.0.1:8000/message/");
supposes that you are calling http://127.0.0.1/message/index.php but here you're not allowed to let the index.php (or whatever) implicit.
The script gets called, but the POST data is not sent, or it is truncated by the Webserver when handling the request. I had the same problem and spent hours until I found the reason. Then I did not dig further to find out exactly where the POST data gets dropped.
I am writing an Android program that needs to access a URL with GET variables which will be logged into a database. All I need to do is open a URL so the web server will log the data! How should I go about this?
Thanks
// default HTTP Client
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(url);
try {
HttpResponse httpResponse = httpClient.execute(httpGet);
HttpEntity httpEntity = httpResponse.getEntity();
is = httpEntity.getContent();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
Use this.
Intent browserIntent = new Intent(Intent.ACTION_VIEW, Uri.parse("URL HERE"));
startActivity(browserIntent);
Hope this helps.
EDIT:
Ok, use this. It calls the URL without opening a browser
HttpClient httpClient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpGet httpGet = new HttpGet("URL HERE");
HttpResponse response = httpClient.execute(httpGet, localContext);
You can also use HttpUrlConnection. Sample code -
// Given a URL, establishes an HttpUrlConnection and retrieves
// the web page content as a InputStream, which it returns as
// a string.
private String downloadUrl(String myurl) throws IOException {
InputStream is = null;
// Only display the first 500 characters of the retrieved
// web page content.
int len = 500;
try {
URL url = new URL(myurl);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setReadTimeout(10000 /* milliseconds */);
conn.setConnectTimeout(15000 /* milliseconds */);
conn.setRequestMethod("GET");
conn.setDoInput(true);
// Starts the query
conn.connect();
int response = conn.getResponseCode();
Log.d(DEBUG_TAG, "The response is: " + response);
is = conn.getInputStream();
// Convert the InputStream into a string
String contentAsString = readIt(is, len);
return contentAsString;
// Makes sure that the InputStream is closed after the app is
// finished using it.
} finally {
if (is != null) {
is.close();
}
}
}
// Reads an InputStream and converts it to a String.
public String readIt(InputStream stream, int len) throws IOException, UnsupportedEncodingException {
Reader reader = null;
reader = new InputStreamReader(stream, "UTF-8");
char[] buffer = new char[len];
reader.read(buffer);
return new String(buffer);
}
And obviously since it is a netowork call you cannot do it in main/UI thread. So you can do it in async task. More details and Source