Download a file from an s3 bucket with REST - java

I am attempting to download from an s3 bucket with REST calls. I am able to create buckets, delete buckets and list buckets. Downloading always ends with either an object not found or HTTP response code: 403. The code is below. Also does anyone know if amazon was sdk can be used with non amazon sites?
public void getObject() throws Exception
{
String fmt = "EEE, dd MMM yyyy HH:mm:ss ";
SimpleDateFormat df = new SimpleDateFormat(fmt, Locale.US);
df.setTimeZone(TimeZone.getTimeZone("GMT"));
String ob2 = "/bucket/test.txt";
String bucket = "/bucket";
String method = "GET";
String contentMD5 = "";
String contentType = "";
String date = df.format(new Date()) + "GMT";
// Generate signature
StringBuffer buf = new StringBuffer();
buf.append(method).append("\n");
buf.append(contentMD5).append("\n");
buf.append(contentType).append("\n");
buf.append(date).append("\n");
buf.append(ob2);
String signature = sign(buf.toString());
HttpURLConnection httpConn = null;
URL url = new URL("https”,”s3demo.s3demosite.com",443,bucket);
httpConn = (HttpURLConnection) url.openConnection();
httpConn.setDoInput(true);
httpConn.setDoOutput(true);
httpConn.setUseCaches(false);
httpConn.setDefaultUseCaches(false);
httpConn.setAllowUserInteraction(true);
httpConn.setRequestMethod(method);
httpConn.setRequestProperty("Date", date);
httpConn.setRequestProperty("Content-Length", "0");
String AWSAuth = "AWS " + keyId + ":" + signature;
httpConn.setRequestProperty("Authorization", AWSAuth);
// Send the HTTP PUT request.
int statusCode = httpConn.getResponseCode();
InputStream err = httpConn.getErrorStream();
InputStream is = null;
is = httpConn.getInputStream();
int ch;
StringBuffer sb = new StringBuffer();
while ((ch = err.read()) != -1) {
sb.append((char) ch);
}
if ((statusCode/100) != 2)
{
// Deal with S3 error stream.
InputStream in = httpConn.getErrorStream();
System.out.println("Error: "+errorStr);
}
else
{
System.out.println("download worked”);
}
}

Related

Java HttpURLConnection http timeout

In my local it works perfectly, but when I deploy it gives me this error
nested exception is java.net.ConnectException: Connection timed out (Connection timed out)
with https everything works normal, but http does not work and it gives me the timeout error.
I also just did the tests with restTemplate, OkHttpClient and I get the same result
What am I doing wrong or what should I configure to work, I hope your help, I would be too grateful
public String getFile(String baseName, String extensioFile) {
String rpt;
int BUFFER_SIZE = 4096;
String urlDonwload = "http://datos.susalud.gob.pe/node/223/download";
try {
URL url = new URL(urlDonwload);
HttpURLConnection httpConn = (HttpURLConnection) url.openConnection();
httpConn.setRequestMethod("GET");
httpConn.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729)");
httpConn.setConnectTimeout(900000);
httpConn.setReadTimeout(7200000);
int responseCode = httpConn.getResponseCode();
if (responseCode == HttpURLConnection.HTTP_OK) {
String fileName = "";
String disposition = httpConn.getHeaderField("Content-Disposition");
if (disposition != null) {
// extracts file name from header field
int index = disposition.indexOf("filename=");
if (index > 0) {
fileName = disposition.substring(index + 10, disposition.length() - 1);
}
} else {
// extracts file name from URL
// fileName = urlCamaUci.substring(urlCamaUci.lastIndexOf("/") + 1,
// urlCamaUci.length());
LocalDateTime currentDate = LocalDateTime.now(ZoneOffset.of("-05:00"));
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
String formatDateTime = currentDate.format(formatter);
System.out.println();
fileName = baseName + "_" + formatDateTime.replace(" ", "_").replace(":", "-") + "." + extensioFile;
}
InputStream inputStream = httpConn.getInputStream();
// String saveFilePath = PATH + File.separator + fileName;
File pathSave = new File(fileName);
FileOutputStream outputStream = new FileOutputStream(pathSave.getCanonicalPath());
int bytesRead = -1;
byte[] buffer = new byte[BUFFER_SIZE];
while ((bytesRead = inputStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, bytesRead);
}
outputStream.close();
inputStream.close();
rpt = pathSave.getCanonicalPath();
} else {
rpt = "FAILED";
}
httpConn.disconnect();
} catch (Exception e) {
System.out.println("error search path");
System.out.println(e.getMessage());
rpt = "FAILED";
}
return rpt;
}

Microsoft graph search feature Java

I'm trying to use Microsoft Graph to make a file search. I use this entry point : https://graph.microsoft.com/beta/search/query
My application do not use a user account but a daemon with an application key (see auth method).
And i send a built object.
My java code is rather simple :
public static void main(String[] args) throws Exception{
try {
// Authentication result containing token
IAuthenticationResult result = getAccessTokenByClientCredentialGrant();
String token = result.accessToken();
SearchDocumentResponseModel documentQuery = fileGraphs.searchDocument(token, QUERYSTRING, 0, 25);
System.out.println("Find a document" + documentQuery.toString());
} catch(Exception ex){
throw ex;
}
}
private static IAuthenticationResult getAccessTokenByClientCredentialGrant() throws Exception {
ConfidentialClientApplication app = ConfidentialClientApplication.builder(
CONFIDENTIAL_CLIENT_ID,
ClientCredentialFactory.createFromSecret(CONFIDENTIAL_CLIENT_SECRET))
.authority(TENANT_SPECIFIC_AUTHORITY)
.build();
ClientCredentialParameters clientCredentialParam = ClientCredentialParameters.builder(
Collections.singleton(GRAPH_DEFAULT_SCOPE))
.build();
CompletableFuture<IAuthenticationResult> future = app.acquireToken(clientCredentialParam);
return future.get();
}
The SearchDocumentResponseModel is just a set of POJO that build for me the object that i must send as a request body.
{
"requests":
[{
"entityTypes":["microsoft.graph.driveItem"],
"query":{"query_string":{"query":"any query"}},
"from":0,"size":25
}]
}
The method searchDocument is just here to build the object before i send it to the API
public SearchDocumentResponseModel searchDocument(String accessToken, String stringSearch, int from, int size) throws IOException {
SearchDocumentRequestModel searchRequest = new SearchDocumentRequestModel();
// set values here
...
URL url = new URL("https://graph.microsoft.com/beta/search/query");
return requestsBuilder.buildPostRequest(accessToken, searchRequest, url)
}
Now i want to send to server the Json and expect an answer :
public SearchDocumentResponseModel buildPostRequest(String accessToken, SearchDocumentRequestModel searchRequest, URL url) throws IOException {
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestProperty("Authorization", "Bearer " + accessToken);
conn.setRequestProperty("Accept","application/json");
conn.setRequestProperty("Content-Type","application/json; utf-8");
conn.setDoOutput(true);
conn.setRequestMethod("POST");
// write the input json in a string
String jsonInputString = new Gson().toJson(searchRequest);
try(OutputStream os = conn.getOutputStream()) {
byte[] input = jsonInputString.getBytes(StandardCharsets.UTF_8);
os.write(input, 0, input.length);
}
int httpResponseCode = conn.getResponseCode();
String httpResponseMessage = conn.getResponseMessage();
// reading the response
try(BufferedReader br = new BufferedReader(
new InputStreamReader(conn.getInputStream(), StandardCharsets.UTF_8))) {
StringBuilder response = new StringBuilder();
String responseLine = null;
while ((responseLine = br.readLine()) != null) {
response.append(responseLine.trim());
}
String outputResponse = response.toString();
return new Gson().fromJson(outputResponse, SearchDocumentResponseModel.class);
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
I think i set the properties correctly. Is it coming from my code or from Microsoft Graph ? Thanks !
First of all, you should check if the access token is valid, you can send a request using postman.
If the token is valid, I think it should be the problem of your jsonInputString. The following code works fine.
URL url = new URL("https://graph.microsoft.com/beta/search/query");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestProperty("Authorization", "access_token" );
conn.setRequestProperty("Accept","application/json");
conn.setRequestProperty("Content-Type","application/json; utf-8");
conn.setRequestMethod("POST");
conn.setDoOutput(true);
String str = "";
str += "{";
str += " \"requests\": [";
str += " {";
str += " \"entityTypes\": [";
str += " \"microsoft.graph.driveItem\"";
str += " ],";
str += " \"query\": {";
str += " \"query_string\": {";
str += " \"query\": \"contoso\"";
str += " }";
str += " },";
str += " \"from\": 0,";
str += " \"size\": 25";
str += " }";
str += " ]";
str += "}";
OutputStream os = conn.getOutputStream();
byte[] input = str.getBytes("UTF-8");
os.write(input, 0, input.length);
System.out.println(conn.getResponseCode());
Update:
Query api doesn't support client credential flow.

Sending request from Java to Box for uploading files

I want to upload file to Box using Java HTTP RESTFUL requests.
My code is,
File file = new File("C:/Users/Developer/Desktop/cloud.ppt");
Base64.Encoder encoder = Base64.getEncoder();
Base64.Decoder decoder = Base64.getDecoder();
String boundary1 = "----------------------------741e90d31eff";
String header1 = "--"+boundary1+"\nContent-Disposition: form-data; name=\"file\"; filename="+file.getName()+";\nContent-Type: application/octet-stream";
System.out.println("=====header1======="+header1);
String footer1 = "--"+boundary1+"--";
String header2= header1;
header1 = header1+"\r\n\r\n";
byte[] byteConvert = header1.getBytes("UTF-8");
String headerEncoded1 = encoder.encodeToString(byteConvert);
HttpResponse response;
System.out.println("******headerEncoded1*****"+headerEncoded1);
byte[] byteConvert2 = header2.getBytes("UTF-8");
while(headerEncoded1.endsWith("="))
{
header2+=' ';
header2 = header2+"\r\n\r\n";
byteConvert2 = header2.getBytes("UTF-8");
headerEncoded1 = encoder.encodeToString(byteConvert2);
}
byte fileContent[] = new byte[(int)file.length()];
System.out.println("fileContent[]: " +fileContent);
FileInputStream fin = new FileInputStream(file);
fin.read(fileContent);
fin.close();
String fileContentString = new String(fileContent);
String fileBody = fileContentString;
byteConvert2 = fileBody.getBytes("UTF-8");
System.out.println("byteConvert2: " +byteConvert2);
String bodyEncoded1 = encoder.encodeToString(byteConvert2);
System.out.println("***bodyEncoded1***"+bodyEncoded1);
String bodyBlob = null;
String last4Bytes = bodyEncoded1.substring(bodyEncoded1.length()-4,bodyEncoded1.length());
System.out.println("****last4Bytes****"+last4Bytes);
if(last4Bytes.endsWith("=="))
{
System.out.println("=== if ===");
last4Bytes = last4Bytes.substring(0,2) + "0K";
bodyEncoded1 = bodyEncoded1.substring(0,bodyEncoded1.length()-4) + last4Bytes;
byteConvert2 = footer1.getBytes("UTF-8");;
String footerEncoded = encoder.encodeToString(byteConvert2);
String Combine = headerEncoded1+bodyEncoded1+footerEncoded;
byteConvert2 = Combine.getBytes("UTF-8");
byte[] byteConvert3 = decoder.decode(byteConvert2);
bodyBlob = byteConvert3.toString();
System.out.println("\n***if bodyBlob***"+bodyBlob);
}
else if(last4Bytes.endsWith("="))
{
System.out.println("===else if bodyBlob===");
last4Bytes = last4Bytes.substring(0,3) + "N";
bodyEncoded1 = bodyEncoded1.substring(0,bodyEncoded1.length()-4) + last4Bytes;
footer1 = "\n" + footer1;
byteConvert2 = footer1.getBytes("UTF-8");
String footerEncoded = encoder.encodeToString(byteConvert2);
String strCombineNew= headerEncoded1+bodyEncoded1+footerEncoded;
byteConvert2 = strCombineNew.getBytes("UTF-8");
byte[] byteConvert3 = decoder.decode(byteConvert2);
bodyBlob = byteConvert3.toString();
System.out.println("\n****else if bodyBlob***"+bodyBlob);
}
else
{
System.out.println("===else===");
footer1 = "\r\n" + footer1;
byteConvert2 = footer1.getBytes("UTF-8");
String footerEncoded = encoder.encodeToString(byteConvert2);
String strCombine = headerEncoded1+bodyEncoded1+footerEncoded;
byteConvert2 = strCombine.getBytes("UTF-8");
byte[] byteConvert3 = decoder.decode(byteConvert2);
bodyBlob = byteConvert3.toString();
System.out.println("\n***else***"+bodyBlob);
}
//System.out.println("****bodyBlob***"+bodyBlob);
//java.sql.Blob blob = org.hibernate.Hibernate.createBlob(bodyBlob.getBytes());
//System.out.println("=====blob======"+blob);
String strFolderId = "XXXXXXX";
HttpClient httpclient = HttpClientBuilder.create().build();
String sUrl = "https://upload.box.com/api/2.0/files/content?parent_id="+strFolderId;
HttpPost request = new HttpPost(sUrl);
request.setEntity(new StringEntity(bodyBlob));
request.setHeader("Content-Type","multipart/form-data; boundary="+boundary1);
request.setHeader("Authorization", "Bearer" +strAccessTokenOnly);
System.out.println("\n=== Content-Length ===" +String.valueOf(fileBody.length()));
request.setHeader("Content-Length",String.valueOf(fileBody.length()));
response = httpclient.execute(request);
System.out.println("\n=== response ==="+response.getStatusLine());
Here, If I set header with Content-Length, the execution will stop in between.
And If I did not add the setHeader I am getting as Bad Request, Status 400.
If anyone have worked on related to this scenario, please help me..
Thanks in advance

Hindi decoding error while sending message. Output is something like व� हव�� व�हव ��व� हवहব व�ह

static String username = "######";
static String password = "#####";
static String senderid = "###";
static String message = "वउह";
static String mobileNo = "08447475458";
static String mobileNos = "08447475458,08447475458";
static String scheduledTime = "20110701 02:27:00";
static HttpURLConnection connection = null;
public static void main(String[] args) {
try {
URL url = new URL("http://msdgweb.mgov.gov.in/esms/sendsmsrequest");
connection = (HttpURLConnection) url.openConnection();
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setFollowRedirects(true);
connection = sendSingleSMS(username, password, senderid,
mobileNo, message);
System.out.println("Resp Code:" + connection.getResponseCode());
System.out.println("Resp Message:"
+ connection.getResponseMessage());
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
// Method for sending single SMS.
public static HttpURLConnection sendSingleSMS(String username,
String password, String senderId,
String mobileNo, String message) {
try {
byte[] bytes = message.getBytes("UTF-8");
String smsservicetype = "singlemsg"; // For single message.
String query = "username=" + URLEncoder.encode(username)
+ "&password=" + URLEncoder.encode(password)
+ "&smsservicetype=" + URLEncoder.encode(smsservicetype)
+ "&content=" + URLDecoder.decode(message,"utf-8") + "&mobileno="
+ URLEncoder.encode(mobileNo) + "&senderid="
+ URLEncoder.encode(senderId);
connection.setRequestProperty("Content-length", String
.valueOf(query.length()));
connection.setRequestProperty("Content-Type",
"application/x-www-form-urlencoded;charset = utf-8");
connection.setRequestProperty("User-Agent",
"Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)");
DataOutputStream output = new DataOutputStream(connection
.getOutputStream());
int queryLength = query.length();
output.writeBytes(query);
System.out.println(query);
DataInputStream input = new DataInputStream(connection
.getInputStream());
for (int c = input.read(); c != -1; c = input.read())
System.out.print((char) c);
input.close();
} catch (Exception e) {
System.out.println("Something bad just happened.");
System.out.println(e);
e.printStackTrace();
}
return connection;
I am using this code to send sms from gateway but this code is fine if the text is english but if I give some hindi text then its not able to read it and user gets some characters.
the output is something like
व� हव�� व�हव ��व� हवहব व�ह
The DataInputStream is reading bytes, but then you have to properly convert those bytes to characters using the correct encoding. For that, you could construct a String using the constructor as follows:
String hindiCharSequence = new String(bytes, "UTF-8");
The answer from #ChthonicProject is the correct answer to accept, here a bit code. It uses a ByteArrayOutputStream to collect the bytes. A DataInputStream is more useful for reading binary ints and such. Better use just a buffering. And/or read a byte[] buffer of say 160 bytes.
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (BufferedInputStream input = new BufferedInputStream(connection
.getInputStream())) {
for (int c = input.read(); c != -1; c = input.read()) {
baos.write(c);
}
} // Closes input.
String msg = new String(baos.toByteArray(), StandardCharsets.UTF_8);
System.out.print(msg);
What you did, was considering every byte as a char. But with UTF-8 several bytes may make up a char, which is 16 bit.
May be the problem is with the SMS encoding. By default it uses GSM 7-bit alphabet.
Check if you can specify which encoding will be used in SMS encoding.
Applied this belore query string and its done
System.out.println(message);
String finalmessage = "";
String sss = "";
char ch = 0;
for(int i = 0 ; i < message.length();i++){
ch = message.charAt(i);
int j = (int) ch;
// System.out.println("iiii::"+j);
sss = "&#"+j+";";
finalmessage = finalmessage+sss;
}
System.out.println("ddd"+finalmessage);
message=finalmessage;
System.out.println("unicoded message=="+message);

Response :java.net.SocketException: Unexpected end of file from server Java

I got this error from my program when i am call a URL from URLConnector.. the URL is
http://192.168.2.107/cgi-bin/mediaFileFind.cgi?action=findFile&object=27544704&condition.Channel=0&conditon.Dir[0]="/mnt/sd"&condition.StartTime=2014-8-1 00:00:00&condition.EndTime=2014-8-31 23:59:59
but when i capture HTTP using wire-shark then wire-shark the URl is loss
wire-shark capture only
http://192.168.2.107/cgi-bin/mediaFileFind.cgi?action=findFile&object=27544704&condition.Channel=0&conditon.Dir[0]="/mnt/sd"&condition.StartTime=2014-8-1 00:00:00
only this URL
my Java program is
public String intilizeObject(String IP, String user, String pass, String objectID, String dir, String startTime, String endTime) {
String result = "";
try {
String URL = "http://" + IP + "/cgi-bin/mediaFileFind.cgi?action=findFile&object=" + objectID + "&condition.Channel=0&conditon.Dir[0]=\"" + dir + "\"&condition.StartTime=" + startTime + "&condition.EndTime=" + endTime;
String authString = user + ":" + pass;
byte[] authEncBytes = Base64.encodeBase64(authString.getBytes());
String authStringEnc = new String(authEncBytes);
URL url = new URL(URL);
System.out.println(url);
URLConnection urlConnection = url.openConnection();
urlConnection.setRequestProperty("Authorization", "Basic " + authStringEnc);
InputStream is = urlConnection.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
int numCharsRead;
char[] charArray = new char[1024];
StringBuffer sb = new StringBuffer();
while ((numCharsRead = isr.read(charArray)) > 0) {
sb.append(charArray, 0, numCharsRead);
}
result = sb.toString();
} catch (Exception e) {
result = e.toString();
}
return result;
}

Categories