We Keep Coding

setting s3 bucket policy while PutObject

public String getPolicy() throws Exception {
String policy_document = "{\"expiration\": \"2020-01-01T00:00:00Z\",\n" +
" \"conditions\": [ \n" +
" {\"bucket\": \"bucket\"}, \n" +
" [\"starts-with\", \"$Content-Type\", \"image/\"],\n" +
" [\"content-length-range\", 0, 100]\n" +
" ]\n" +
"}";
String aws_secret_key = "xxxxx";
String policy = (new BASE64Encoder()).encode(policy_document.getBytes("UTF-8"))
.replaceAll("\n", "").replaceAll("\r", "");
Mac hmac = Mac.getInstance("HmacSHA1");
hmac.init(new SecretKeySpec(
aws_secret_key.getBytes("UTF-8"), "HmacSHA1"));
String signature = (new BASE64Encoder()).encode(
hmac.doFinal(policy.getBytes("UTF-8")))
.replaceAll("\n", "");
return policy;
}
While uploading -
ObjectMetadata objectMetadata = new ObjectMetadata();
objectMetadata.setHeader("policy", getPolicy());
s3Client.putObject(bucket, key, inputStream, objectMetadata);
Can we pass policy header like above to reject s3 putObject requests that violate policy conditions ?
I think it is possible via s3Client.setBucketPolicy but is there a way to set these policies for each put request ?

Different Android Phone Uses Different Encoding?

I am developing an app plays song and communicates with a server. The methods are addMusic, deleteMusic, and requestMusicList. All of them uses http GET method.
Here's The Case:
deleteMusic, and requestMusicList works fine in all phones. Except for addMusic, which only works in some phone.
addMusic only works in Samsung phone. While in nuu, mi, and meizu. Those are the phones that we got here, so we only tested the app with those phones.
My Problem:
Since the parameters included in the addMusic url string contains Chinese characters, I think this is the cause of the problem. Here is the log.
08-06 15:00:22.102: V/HttpConnectionManager(13211): Get url string is http://115.28.6.88:7100/cgi-bin/v1/music_add?content_url=http://fdfs.xmcdn.com/group12/M07/03/65/wKgDW1VQa-6iJ2e1AB7JjcEKs2E684.mp3&cover_image_url=http://fdfs.xmcdn.com/group11/M02/03/7B/wKgDa1VQbA2j2mOpAAiTLl27BzM875_mobile_small.jpg&dev_id=test101&duration=252&timestamp=1438844422&title=《Couldyoustaywithme》张靓颖（电视剧《洋嫁》主题曲）&type=type_habit&sign=f834a4667004cebdac1516584b4a4930
08-06 15:00:22.272: I/Adreno200-EGLSUB(13211): <ConfigWindowMatch:2252>: Format RGBA_8888.
08-06 15:00:22.342: V/HttpConnectionManager(13211): Response code from GET: 200
08-06 15:00:22.352: V/DetailManager(13211): Handled http response is {"retcode": 100004, "retinfo": "check sign fail, request_sign:[F834A4667004CEBDAC1516584B4A4930], mysign:[EBFA4D2D40F20DF1B124375A0D9FAD38], mysign_src_nokey:[content_url=http://fdfs.xmcdn.com/group12/M07/03/65/wKgDW1VQa-6iJ2e1AB7JjcEKs2E684.mp3&cover_image_url=http://fdfs.xmcdn.com/group11/M02/03/7B/wKgDa1VQbA2j2mOpAAiTLl27BzM875_mobile_small.jpg&dev_id=test101&duration=252&timestamp=1438844422&title=?Couldyoustaywithme????????????????&type=type_habit]"}
08-06 15:00:22.352: V/AudioFragment(13211): Server request failed: check sign fail, request_sign:[F834A4667004CEBDAC1516584B4A4930], mysign:[EBFA4D2D40F20DF1B124375A0D9FAD38], mysign_src_nokey:[content_url=http://fdfs.xmcdn.com/group12/M07/03/65/wKgDW1VQa-6iJ2e1AB7JjcEKs2E684.mp3&cover_image_url=http://fdfs.xmcdn.com/group11/M02/03/7B/wKgDa1VQbA2j2mOpAAiTLl27BzM875_mobile_small.jpg&dev_id=test101&duration=252&timestamp=1438844422&title=?Couldyoustaywithme????????????????&type=type_habit]
As can be seen in the first line, this contains the URL that I sent. The URL contains these "张靓颖（电视剧《洋嫁》主题曲" Chinese characters under the title parameter. When the server returns a response, these characters became "??????????????".
Please take not that this issue only occurs in the said Android phones(mi, meizu, and nuu). For Samsung, this NEVER occured.
Here is the code of my addMusic method:
public void addMusic(final Music music)
{
new Thread(new Runnable()
{
#Override
public void run()
{
String contentURL = KEY_CONTENT_URL + "=" + music.mContentURL;
String coverImageURLData = "&" + KEY_COVER_IMAGE_URL + "=" + music.mCoverImageURL;
String deviceIDData = "&" + KEY_DEVICE_ID + "=" + mDeviceID;
String duration = "&" + KEY_DURATION + "=" + Long.toString(music.mDuration);
String timestamp = "&" + KEY_TIMESTAMP + "=" + Long.toString(System.currentTimeMillis()/1000);
String titleData = "&" + KEY_TITLE + "=" + music.mTitle.replaceAll("\\s", "");
String typeData = "&" + KEY_TYPE + "=" + music.mType;
String dataArrayString = contentURL + coverImageURLData + deviceIDData + duration + timestamp + titleData + typeData;
String key = "&" + KEY_KEY + "=" + KEY;
String sign = "&" + KEY_SIGN + "=" + Utilities.getMD5String(dataArrayString + key);
String parametersURL = dataArrayString + sign;
try
{
handleHttpResponse(mConnectionManager.get(URL_MAIN + URL_ADD_MUSIC + parametersURL), REQUEST_CODE_ADD_MUSIC);
}
catch (IOException e)
{
e.printStackTrace();
mListener.onServerRequestFail(e.getLocalizedMessage());
}
}
}).start();
}
Here is the code of Http GET:
public String get(String urlString) throws IOException
{
Log.v(LOG_TAG, "Get url string is " + urlString);
URL url = new URL(urlString);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("User-Agent", USER_AGENT);
int responseCode = connection.getResponseCode();
Log.v(LOG_TAG, "Response code from GET: " + responseCode);
if(responseCode == RESPONSE_OK)
{
BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
StringBuffer response = new StringBuffer();
String inputLine;
while ((inputLine = reader.readLine()) != null)
{
response.append(inputLine);
}
reader.close();
return response.toString();
}
else
{
return ERROR_NO_RESPONSE;
}
}
I tried checking the defaultCharset, which is "UTF-8" for all phones tested.
How can I solve this? Please help. Many Thanks
------------ Tried URLEncoder.encode(music.mTitle.replaceAll("\s", ""), "UTF-8") ----------------
Here's the Log:
08-06 15:53:08.852: V/HttpConnectionManager(15061): Get url string is http://115.28.6.88:7100/cgi-bin/v1/music_add?content_url=http://fdfs.xmcdn.com/group6/M09/1E/0B/wKgDg1UrI8iw-rGzAAyaxOKiDrc706.mp3&cover_image_url=http://fdfs.xmcdn.com/group6/M00/1E/18/wKgDg1UrJp7TMEMMAAP5a37TPkE290_mobile_small.jpg&dev_id=test101&duration=103&timestamp=1438847588&title=%E3%80%8Ayouaremysunshine%E3%80%8B%EF%BC%8D%E5%BC%A0%E9%9D%93%E9%A2%96%EF%BC%88%E7%94%B5%E5%BD%B1%E3%80%8A%E4%BD%95%E4%BB%A5%E7%AC%99%E7%AE%AB%E9%BB%98%E3%80%8B%E8%8B%B1%E6%96%87%E6%8F%92%E6%9B%B2&type=type_habit&sign=f7b8c5a0cb1fceb32a54b428311744ad
08-06 15:53:08.902: I/Adreno200-EGLSUB(15061): <ConfigWindowMatch:2252>: Format RGBA_8888.
08-06 15:53:19.692: V/HttpConnectionManager(15061): Response code from GET: 200
08-06 15:53:19.692: V/DetailManager(15061): Handled http response is {"retcode": 100004, "retinfo": "check sign fail, request_sign:[F7B8C5A0CB1FCEB32A54B428311744AD], mysign:[537721D351DF6C8B40C62C4F8672EADF], mysign_src_nokey:[content_url=http://fdfs.xmcdn.com/group6/M09/1E/0B/wKgDg1UrI8iw-rGzAAyaxOKiDrc706.mp3&cover_image_url=http://fdfs.xmcdn.com/group6/M00/1E/18/wKgDg1UrJp7TMEMMAAP5a37TPkE290_mobile_small.jpg&dev_id=test101&duration=103&timestamp=1438847588&title=\u300ayouaremysunshine\u300b\uff0d\u5f20\u9753\u9896\uff08\u7535\u5f71\u300a\u4f55\u4ee5\u7b19\u7bab\u9ed8\u300b\u82f1\u6587\u63d2\u66f2&type=type_habit]"}
08-06 15:53:19.702: V/AudioFragment(15061): Server request failed: check sign fail, request_sign:[F7B8C5A0CB1FCEB32A54B428311744AD], mysign:[537721D351DF6C8B40C62C4F8672EADF], mysign_src_nokey:[content_url=http://fdfs.xmcdn.com/group6/M09/1E/0B/wKgDg1UrI8iw-rGzAAyaxOKiDrc706.mp3&cover_image_url=http://fdfs.xmcdn.com/group6/M00/1E/18/wKgDg1UrJp7TMEMMAAP5a37TPkE290_mobile_small.jpg&dev_id=test101&duration=103&timestamp=1438847588&title=《youaremysunshine》－张靓颖（电影《何以笙箫默》英文插曲&type=type_habit]

I solved it. Thanks to #Andy Turner, and #lorenzo-s for the tips in the comment.
My mistake is that I encoded the string to UTF-8 before getting the MD5 checksum. I noticed that when I send the request(URLEncoded) to the server, the response containing the title is correct. The error was only about the MD5 sign check.
The correct way is to get the MD5 checksum first without encoding the title parameter. Then before appending the title parameter to the URL String, convert it to UTF-8.
Thank you #Andy Turner for reminding me that I have to deal with the "&" in the parameter.

Java, URL output different in java output

I have a question about URI and URL
when i pass a url is work good but result is worst need help!!
as my code look like this.
import java.io.*;
import java.net.*;
import java.net.URL;
public class isms {
public static void main(String[] args) throws Exception {
try {
String user = new String ("boo");
String pass = new String ("boo");
String dstno = new String("60164038811"); //You are going compose a message to this destination number.
String msg = new String("你的哈达哈达!"); //Your message over here
int type = 2; //for unicode change to 2, normal will the 1.
String sendid = new String("isms"); //Malaysia does not support sender id yet.
// Send data
URI myUrl = new URI("http://www.isms.com.my/isms_send.php?un=" + user + "&pwd=" + pass
+ "&dstno=" + dstno + "&msg=" + msg + "&type=" + type + "&sendid=" + sendid);
URL url = new URL(myUrl.toASCIIString());
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
// Get the response
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
// Print the response output...
System.out.println(line);
}
rd.close();
System.out.println(url);
} catch (Exception e) {
e.printStackTrace();
}
}
}
the output in web is different..
on my java output is
你的哈达哈达!
but on my the site is
ÄãµÄ¹þ´ï¹þ´ï!
Help!!

String user = new String ("boo");
You don't need to (and shouldn't) do new String in Java—String user = "boo"; is fine.
String msg = new String("你的哈达哈达!");
Writing non-ASCII characters in your source means that you have to get the -encoding flag to javac to match the encoding you have saved your text files with. It is possible you have saved the .java file as UTF-8 but not configured your build environment to use UTF-8 at compile time.
If you are not sure that you've got this right, you can use ASCII-safe \u escapes in the meantime:
String msg = "\u4F60\u7684\u54C8\u8FBE\u54C8\u8FBE!"; // 你的哈达哈达!
Finally:
URI myUrl = new URI("http://www.isms.com.my/isms_send.php?un=" + user + "&pwd=" + pass
+ "&dstno=" + dstno + "&msg=" + msg + "&type=" + type + "&sendid=" + sendid);
When you're putting a URI together you should URL-escape each of the parameters you include in the string. Otherwise any & or other invalid character in the value will break the query. This also allows you to choose what charset is used to create the query string.
String enc = "UTF-8";
URI myUrl = new URI("http://www.isms.com.my/isms_send.php?" +
"un=" + URLEncoder.encode(user, enc) +
"&pwd=" + URLEncoder.encode(pass, enc) +
"&dstno=" + URLEncoder.encode(dstno, enc) +
"&msg=" + URLEncoder.encode(msg, enc) +
"&type=" + URLEncoder.encode(Integer.toString(type), enc) +
"&sendid=" + URLEncoder.encode(sendid, enc)
);
What the right value for enc is depends on the service you are connecting to, but UTF-8 is a good guess.

Sending post attributes and xml data with a post request?

New to java, GWT and interacting with APIs. I have what I hope is a simple question.
I have successfully interacted with a REST API using the following curl command:
curl -d "OPERATION_NAME=ADD_REQUEST&TECHNICIAN_KEY=xxxxxxxxxxx&INPUT_DATA=<?xml version=%221.0%22 encoding=%22utf-8%22?><Operation><Details><requester>Me</requester><subject>Test</subject><description>Testing curl input</description></Details></Operation>" http://xx.xx.xx.xx/sdpapi/request/
Now, from a tutorial, I have the following code that I am hoping will post a request to the remote server just like the curl command above.
What I am trying to figure out (with no love from google) is how I pass the OPERATION_NAME, TECHNICIAN_KEY and INPUT_DATA parameters in when I am sending the URL. Any suggestions, tutorials, etc. will be appreciated.
The following is from my server side implementation interface:
#Override
public String postToRemoteServer(String serviceUrl)
throws HelpDeskTestException {
try {
//dividing url into host: http://some.server
//path: a/path/in/it
//and parameters: this=that&those=others
int hostStart= serviceUrl.indexOf("//");
int pathStart= serviceUrl.substring(hostStart + 2).indexOf("/");
int parameterStart= serviceUrl.substring(hostStart + 2 + pathStart).indexOf("?");
final String serverHost= serviceUrl.substring(0, hostStart + pathStart + 2);
final String serverPath= serviceUrl.substring(hostStart + 3,
hostStart + pathStart + 2 + parameterStart);
final String serverParameters= serviceUrl.substring(hostStart + pathStart + 3 + parameterStart);
final URL url = new URL(serverHost);
final URLConnection connection= url.openConnection();
connection.setDoOutput(true);
final OutputStreamWriter out= new OutputStreamWriter(connection.getOutputStream());
final BufferedReader in= new BufferedReader(new InputStreamReader(
connection.getInputStream()));
out.write("POST " + serverPath + "\r\n");
out.write("Host: " + serverHost + "\r\n");
out.write("Accept-Encoding: identity\r\n");
out.write("Connection: close\r\n");
out.write("Content-Type: application/x-www-form-urlencoded\r\n");
out.write("Content-Length: " + serverParameters.length() + "\r\n\r\n" +
serverParameters + "\r\n");
String result = "";
String inputLine;
while ((inputLine=in.readLine()) != null) {
result+= inputLine;
}
in.close();
out.close();
return result;
} catch (final Exception e) {
throw new HelpDeskTestException();
}

Consider using this library: Apache HttpClient. Here is an example of making a POST request with it.

Twitter API Failed to validate oauth signature and token java

I'm trying to access resources of a twitter account with http protocol, in an Android app. Twitter uses the open authentication standard OAuth for authentication, consequently I'm following the example in https://dev.twitter.com/docs/auth/oauth#Overview.
My problem is :
I whant to acquiring a request token but when I make the request to the endpoint https://api.twitter.com/oauth/request_token the response is "Failed to validate oauth signature and token".
My code :
FOR SIGNATURE STEP
Timestamp and Nonce
oauthTimestamp = String.valueOf(new Date().getTime());
oauthNonce = Base64.encodeToString(oauthTimestamp.getBytes(),Base64.DEFAULT);
//remove /n at the end of the string
oauthNonce = oauthNonce.substring(0, oauthNonce.length() - 1);
Signature base string
String signatureBaseString =
"POST"
+ "&"
+ URLEncoder.encode("https://api.twitter.com/oauth/request_token")
+ "&"
+ URLEncoder.encode("oauth_callback=" + redirectUrl)
+ URLEncoder.encode("&" + "oauth_consumer_key=" + consumerKey)
+ URLEncoder.encode("&" + "oauth_nonce=" + oauthNonce)
+ URLEncoder.encode("&" + "oauth_signature_method=" + "HMAC-SHA1")
+ URLEncoder.encode("&" + "oauth_timestamp=" + oauthTimestamp)
+ URLEncoder.encode("&" + "oauth_version=" + "1.0");
signature call
signature = getSignatureToken(applicationSecret, signatureBaseString, "HmacSHA1");
signature method
private String getSignatureToken(String consumerSecret, String baseString, String algotithm) {
byte[] keyBytes = (consumerSecret+"&").getBytes();
SecretKeySpec secretKeySpec = new SecretKeySpec(keyBytes, algotithm);
Mac mac;
String res = null;
try {
mac = Mac.getInstance(algotithm);
mac.init(secretKeySpec);
byte[] rawHmac = mac.doFinal((baseString).getBytes());
res = android.util.Base64.encodeToString(rawHmac, android.util.Base64.DEFAULT);
res = res.substring(0, res.length() - 1);
System.out.println("MAC : " + res);
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (InvalidKeyException e) {
e.printStackTrace();
}
return res;
}
REQUEST TOKEN STEP
HttpResponse response;
HttpPost authorization = new HttpPost("https://api.twitter.com/oauth/request_token");
final String headerValue =
"OAuth " +
"oauth_nonce=\""+oauthNonce+"\", " +
"oauth_callback=\""+redirectUrl+"\", " +
"oauth_signature_method=\"HMAC-SHA1\", " +
"oauth_timestamp=\""+oauthTimestamp+"\", " +
"oauth_consumer_key=\""+consumerKey+"\", " +
"oauth_signature=\""+URLEncoder.encode(signature)+"\", " +
"oauth_version=\"1.0\"";
authorization.addHeader("Authorization", headerValue);
response = this.httpClient.execute(authorization);
System.out.println(EntityUtils.toString(response.getEntity()));
response = this.httpClient.execute(authorization) give me HTTP/1.1 401 Unauthorized
Any ideas?
Thanks

Well, you'll have to debug this in stages, but one thing I find strange: why are you removing the last character of the HMAC (signature) in getSignatureToken()? That could break the Base64 encoding.