We are executing web services using java code. The web service response comes in gzip format from the web service provider. We are unzipping the response using GZIPInputStream after receiving the response.
Response is converted into byte codes and then passing as input to gzipinputstream. This code is working fine in Eclipse and able to unzip the response string. The same code is not working in Linux and throwing the error "Not in Gzip format" while passing the byte array to gzipinputstream.
We checked the default charset in Windows is windows-1252 and in Linux is UTF-8. So, we tried to get the bytes in UTF-8 and windows-1252. Both are not working.
Can anyone please help me where is it going wrong and how to resolve the issue?
Tried changing the charset while generating the byte codes of the response.
import java.util.List;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.Authenticator;
import java.net.HttpURLConnection;
import java.net.InetSocketAddress;
import java.net.PasswordAuthentication;
import java.net.Proxy;
import java.net.URL;
import java.io. * ;
import java.nio.charset.*;
import java.util.zip.GZIPInputStream;
import java.nio.charset.*;
public class WSConnectTest {
public final static String UserName = null; //User id login for Fusion
public final static String instanceURL = null;
public final static String USER_PWD = null; // API key shared by CSOD
private static final String PROXY_URL = null; //UBS proxy URL
private static final int PROXY_PORT = 8080;
private static final String PROXY_USERNAME = "USER_NAME";
private static final String PROXY_PASSWORD = "PASSWORD";
final static String USER_AGENT = "Mozilla/5.0";
static Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(
PROXY_URL, PROXY_PORT));
static {
Authenticator authenticator = new Authenticator() {
public PasswordAuthentication getPasswordAuthentication() {
return (new PasswordAuthentication(UserName, USER_PWD.toCharArray()));
}
};
Authenticator.setDefault(authenticator);
}
public static void main(String[] args) throws Exception {
FusionConnect fusionconnect = new FusionConnect();
String theURL = instanceURL + "<RESOURCE_NAME>";
System.out.println("The URL to be called is : " + theURL);
String json = "<JSON_STRING>"
String post_param = new String(json.toString());
System.out.println("The json is :" + json);
PostRequestWithFilter(theURL, post_param);
}
private static void PostRequestWithFilter(String url, String json) throws Exception {
try {
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection(proxy);
con.setRequestMethod("POST");
con.setRequestProperty("User-Agent", "Apache-HttpClient/4.1.1 (java 1.5)");
con.setRequestProperty("Content-Type", "application/json");
con.setRequestProperty("Accept-Language", "UTF-8");
con.setRequestProperty("Accept-Encoding", "gzip, deflate");
con.setDoOutput(true);
con.setConnectTimeout(15000);
System.out.println("get content type :"+con.getRequestProperties());
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
wr.writeBytes(json);
wr.flush();
wr.close();
int responseCode = con.getResponseCode();
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("\nResponse Code : " + responseCode);
System.out.println("\nResponse message : " + con.getResponseMessage());
String inputLine;
StringBuffer response = new StringBuffer();
String ResponseStr = null;
byte[] bresponse = new byte[1024];
String deoutput = null;
BufferedReader in =null;
if (responseCode == con.HTTP_CREATED) { in =new BufferedReader(new InputStreamReader(con.getInputStream()));
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
System.out.println("Response received from Fusion string buffer :"+inputLine);
} in .close();
ResponseStr = response.toString();
System.out.println("response string :"+ResponseStr);
bresponse = ResponseStr.getBytes("UTF-8");
System.out.println("Response received from Fusion Bytes :"+bresponse);
deoutput = unzip(bresponse);
System.out.println("Decompressed response :"+deoutput);
} else { in =new BufferedReader(new InputStreamReader(con.getErrorStream()));
System.out.println("Response Content Type :"+con.getContentType());
System.out.println("Response Content Encoding :"+con.getContentEncoding());
while ((inputLine = in.readLine()) != null) {
response.append(inputLine+"\r");
System.out.println("Response received from Fusion string buffer :"+inputLine);
}
in .close();
ResponseStr = response.toString();
System.out.println("response string :"+response);
bresponse = ResponseStr.getBytes();
for (int i=0; i < bresponse.length; i++)
{
System.out.println("byte code :"+i+" "+bresponse[i]);
}
System.out.println("Response received from Fusion Bytes :"+Charset.defaultCharset()+bresponse);
deoutput = unzip(bresponse);
FileOutputStream fos = new FileOutputStream("fileName1.gz");
DataOutputStream outStream = new DataOutputStream(new BufferedOutputStream(fos));
outStream.writeUTF(ResponseStr);
outStream.close();
System.out.println("Decompressed response :"+deoutput);
}
}
catch(Exception e) {
e.printStackTrace();
}
}
public static String unzip(byte[] compressed) {
if ((compressed == null) || (compressed.length == 0)) {
System.out.println("The response is empty");
throw new IllegalArgumentException("Cannot unzip null or empty bytes");
}
if (!isZipped(compressed)) {
System.out.println("The response is not zipped");
return new String(compressed);
}
StringBuilder output = new StringBuilder();
try (ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(compressed)) {
System.out.println("After byte array input stream :");
try (GZIPInputStream gzipInputStream = new GZIPInputStream(byteArrayInputStream)) {
try (InputStreamReader inputStreamReader = new InputStreamReader(byteArrayInputStream, StandardCharsets.UTF_8)){
try (BufferedReader bufferedReader = new BufferedReader(inputStreamReader)) {
String line;
System.out.println("buffer reader :"+bufferedReader.readLine());
while ((line = bufferedReader.readLine()) != null) {
output.append(line);
System.out.println("line :"+output.toString());
}
} catch(IOException e) {
throw new RuntimeException("Failed to read bufferedReader content", e);
}
}
}
} catch(Exception e) {
e.printStackTrace();
}
return output.toString();
}
public static boolean isZipped(final byte[] compressed) {
System.out.println("(byte)(GZIPInputStream.GZIP_MAGIC) is "+(byte)(GZIPInputStream.GZIP_MAGIC));
System.out.println("gzip magic is "+(byte)(GZIPInputStream.GZIP_MAGIC >> 8));
return (compressed[0] == (byte)(GZIPInputStream.GZIP_MAGIC)) && (compressed[1] == (byte)(GZIPInputStream.GZIP_MAGIC >> 8));
}
}
Related
I want to create a ticket on monday.com. I wrote HTTP method which makes POST call on specific monday server and as a parameter I'm passing graphql query. but unfortunately with no success, I think I'm passing query parameters in a wrong way, but I can't figure what exactly I'm doing wrong.
import java.io.DataOutputStream;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class Monday {
static int id = 1249501957;
static String token = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX";
static String query = "mutation {\n"
+ " create_item(item_name:\"heyyyyyyy\", board_id:" + id + "){\n"
+ " id\n"
+ " }\n"
+ "}";
static String targetURL = "https://levank707.monday.com/projects";
public static void main(String[] args) throws Exception {
executePost(targetURL,query);
}
public static String executePost(String targetURL, String query) {
HttpURLConnection connection = null;
try {
//Create connection
URL url = new URL(targetURL);
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type",
"application/graphql");
connection.setRequestProperty("Content-Length",
Integer.toString(query.getBytes().length));
connection.setRequestProperty("Authorization",token );
connection.setUseCaches(false);
connection.setDoOutput(true);
//Send request
DataOutputStream wr = new DataOutputStream (
connection.getOutputStream());
wr.writeBytes(query);
wr.close();
//Get Response
InputStream is = connection.getInputStream();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
StringBuilder response = new StringBuilder(); // or StringBuffer if Java version 5+
String line;
while ((line = rd.readLine()) != null) {
response.append(line);
response.append('\r');
}
rd.close();
return response.toString();
} catch (Exception e) {
e.printStackTrace();
return null;
} finally {
if (connection != null) {
connection.disconnect();
}
}
}}
I am trying to call a REST API using Java client.
The Rest API https://api.gdc.cancer.gov/data has files data.
When I append file name to the URL (https://api.gdc.cancer.gov/data/556e5e3f-0ab9-4b6c-aa62-c42f6a6cf20c) I can download the given file from using browser.
here filename is 556e5e3f-0ab9-4b6c-aa62-c42f6a6cf20c.
can you please let me know,How can i achieve in this JAVA. The code I am using.
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.ArrayList;
import java.util.List;
public class DownloadFilesAPI {
public DownloadFilesAPI() {
super();
}
public static String sendPostRequest(String requestUrl) {
StringBuffer jsonString = new StringBuffer();
try {
URL url = new URL(requestUrl);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
// connection.setRequestMethod("POST");
// connection.connect();
//Get the response status of the Rest API
// int responsecode = connection.getResponseCode();
//System.out.println("Response code is: " +responsecode);
//connection.getResponseMessage();
// System.out.println(connection.getResponseMessage());
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setRequestProperty("Accept", "application/json");
connection.setRequestProperty("Content-Type", "application/json");
// System.out.println(connection.getResponseMessage());
// System.out.println( JsonPath.from(requestUrl));
OutputStreamWriter writer = new
OutputStreamWriter(connection.getOutputStream());
writer.write(requestUrl);
writer.close();
/* BufferedReader br = new BufferedReader(new
InputStreamReader(connection.getInputStream()));
String line;
while ((line = br.readLine()) != null) {
jsonString.append(line);
}
br.close(); */
connection.disconnect();
} catch (Exception e) {
throw new RuntimeException(e.getMessage());
}
return jsonString.toString();
}
public static void main(String[] args) {
List<String> values = new ArrayList<>();
// values.add("556e5e3f-0ab9-4b6c-aa62-c42f6a6cf20c");
String requestUrl = "https://api.gdc.cancer.gov/data/556e5e3f-0ab9-4b6c-aa62-c42f6a6cf20c";
sendPostRequest(requestUrl);
}
private static String preparePayload(List<String> values) {
StringBuilder sb = new StringBuilder();
for (String value : values) {
sb.append("\"" + value + "\",");
}
String Requiredvalue = sb.toString().substring(0, sb.toString().length() - 1);
return "{ \"ids\":[" + Requiredvalue + "] } } }";
}
}
You can't just output a String since you are trying to download a pdf. If you simply want to download the File there is an easier method adapted from this answer:
String requestUrl = "https://api.gdc.cancer.gov/data/556e5e3f-0ab9-4b6c-aa62-c42f6a6cf20c";
URL url = new URL(requestUrl);
InputStream in = url.openStream();
Files.copy(in, Paths.get("your_filename.pdf"), StandardCopyOption.REPLACE_EXISTING);
in.close();
System.out.println("finished!");
I have tested it for the URL you provided and got the pdf File without problems.
This code that parses from the site booking.uz.gov.ua. But for some reason, he did not want to work. Who can show why does not work, or fix it?
Who can advise something?
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 0
at ua.gov.uz.booking.uz.main(uz.java:137)
package ua.gov.uz.booking;
import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
import javax.script.ScriptException;
import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.text.MessageFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.List;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class uz {
static String html = "";
static String cookie = "";
static String token = "";
static String error = "";
static Map<String, List<String>> headers = null;
static void fetchHtml() {
try {
URL url = new URL("http://booking.uz.gov.ua/");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
headers = conn.getHeaderFields();
String line;
while ((line = rd.readLine()) != null) {
html += line;
}
rd.close();
} catch (Exception e) {
error = e.getMessage();
}
}
static void parseCookie() {
List<String> cookies = headers.get("Set-Cookie");
for (String current_cookie : cookies) {
if (current_cookie.startsWith("_gv_sessid")) {
cookie = current_cookie;
break;
}
}
}
static void parseToken() {
String adapter = "var token;localStorage={setItem:function(key, value){if(key==='gv-token')token=value}};";
Pattern pattern = Pattern.compile("\\$\\$_=.*~\\[\\];.*\"\"\\)\\(\\)\\)\\(\\);");
Matcher matcher = pattern.matcher(html);
if (matcher.find()) {
String obfuscated = matcher.group(0);
ScriptEngineManager factory = new ScriptEngineManager();
ScriptEngine engine = factory.getEngineByName("JavaScript");
try {
engine.eval(adapter + obfuscated);
} catch (ScriptException e) {
error = e.getMessage();
}
token = engine.get("token").toString();
}
}
static String getStationId(String name) {
String json = "";
try {
URL url = new URL("http://booking.uz.gov.ua/en/purchase/station/" + name + "/");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
json += line;
}
rd.close();
} catch (Exception e) {
error = e.getMessage();
}
ScriptEngineManager factory = new ScriptEngineManager();
ScriptEngine engine = factory.getEngineByName("JavaScript");
engine.put("json", json);
try {
engine.eval("var station_id = JSON.parse(json).value[0].station_id");
} catch (ScriptException e) {
error = e.getMessage();
}
return engine.get("station_id").toString();
}
static String getData(String fromName, String toName, String date) {
fetchHtml();
parseCookie();
parseToken();
String from = getStationId(fromName);
String to = getStationId(toName);
String json = "";
try {
URL url = new URL("http://booking.uz.gov.ua/en/purchase/search/");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestProperty("Cookie", cookie);
conn.setRequestProperty("GV-Token", token);
conn.setRequestProperty("GV-Ajax", "1");
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.setRequestProperty("Referer", "http://booking.uz.gov.ua/en/");
conn.setRequestMethod("POST");
String urlParameters = MessageFormat.format("station_id_from={0}&station_id_till={1}&date_dep={2}" +
"&time_dep=00:00&time_dep_till=24:00", from, to, date);
conn.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
wr.writeBytes(urlParameters);
wr.flush();
wr.close();
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
json += line;
}
rd.close();
} catch (Exception e) {
error = e.getMessage();
}
return json;
}
static String getData(String fromName, String toName) {
return getData(fromName, toName, new SimpleDateFormat("MM.dd.yyyy").format(new Date()));
}
public static void main(String[] args) {
if (args.length < 2) {
System.out.println("Онлайн резервування та придбання квиткiв - Укрзалізниця");
System.out.println("Введите: <start_station> <end_station> [MM.DD.YYYY]");
System.exit(1);
}
String data;
if (args.length > 2)
data = getData(args[0], args[1], args[2]);
else
data = getData(args[0], args[1]);
System.out.println(data);
}
}
Line 137 is args[0] = "dsa". If 0 is an IndexOutOfBound, this means that args is an array with size 0 (i.e. empty). Therefore, you cannot access its index 0, because that requires an array with size >= 1.
How are you launching the program affects the content of the args array (that contains the parameters passed to the program when launched via console). Have you checked what args is when launching with a breakpoint? How do you launch your program? (in the ide, or via console?)
I guess you are running your program directly from main method then initialize args. If you are calling main method from other program then you don't have to initialize args
public static void main(String[] args) {
args = new String[5];
args[0] = "dsa";
.......
}
I've followed the tutorial here in an attempt to send data to an android device. I've whittled my Java class down to this:
public class App {
public static void main( String[] args ) {
try {
String apiKey = "api key generated in Google Developer Console...";
String deviceId = "Device Id generated, retrieved directly from logcat, as per tutorial...";
Content content = new Content();
content.addRegId(deviceId);
content.createData("Title", "Notification Message");
URL url = new URL("https://android.googleapis.com/gcm/send");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/json");
conn.setRequestProperty("Authorization", "key=" + apiKey);
conn.setDoOutput(true);
ObjectMapper mapper = new ObjectMapper();
mapper.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
mapper.writeValue(wr, content);
wr.flush();
wr.close();
int responseCode = conn.getResponseCode();
System.out.println(responseCode == 200 ? responseCode + ". This is the response we want..." : responseCode + ". This is not the response we want...");
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
System.out.println(response.toString());
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
and this is my Content class:
public class Content implements Serializable{
private List<String> registration_ids;
private Map<String,String> data;
public void addRegId(String regId) {
if(registration_ids == null)
registration_ids = new LinkedList<String>();
registration_ids.add(regId);
}
public void createData(String title, String message) {
if (data == null)
data = new HashMap<String,String>();
data.put("title", title);
data.put("message", message);
}
}
I'm getting a 400 response code
java.io.IOException: Server returned HTTP response code: 400 for URL: https://android.googleapis.com/gcm/send
I'm sure I'm missing something small, but I can't see where it is.
I'm trying to read html code from a URL Connection. In one case the html file I'm trying to read includes 5 line breaks before the actual doc type declaration. In this case the input reader throws an exception for EOF.
URL pageUrl =
new URL(
"http://www.nytimes.com/2011/03/15/sports/basketball/15nbaround.html"
);
URLConnection getConn = pageUrl.openConnection();
getConn.connect();
DataInputStream dis = new DataInputStream(getConn.getInputStream());
//some read method here
Has anyone ran into a problem like this?
URL pageUrl = new URL("http://www.nytimes.com/2011/03/15/sports/basketball/15nbaround.html");
URLConnection getConn = pageUrl.openConnection();
getConn.connect();
DataInputStream dis = new DataInputStream(getConn.getInputStream());
String urlData = "";
while ((urlData = dis.readUTF()) != null)
System.out.println(urlData);
//exception thrown
java.io.EOFException
at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
at java.io.DataInputStream.readUTF(DataInputStream.java:572)
at java.io.DataInputStream.readUTF(DataInputStream.java:547)
in the case of bufferedreader, it just responds null and doesn't continue
pageUrl = new URL("http://www.nytimes.com/2011/03/15/sports/basketball/15nbaround.html");
URLConnection getConn = pageUrl.openConnection();
getConn.connect();
BufferedReader br = new BufferedReader(new InputStreamReader(getConn.getInputStream()));
String urlData = "";
while(true)
urlData = br.readLine();
System.out.println(urlData);
outputs null
You're using DataInputStream to read data that wasn't encoded using DataOutputStream. Examine the documented behavior for your call to DataInputStream#readUtf(); it first reads two bytes to form a 16-bit integer, indicating the number of bytes that follow comprising the UTF-encoded string. The data you're reading from the HTTP server is not encoded in this format.
Instead, the HTTP server is sending headers encoded in ASCII, per RFC 2616 sections 6.1 and 2.2. You need to read the headers as text, and then determine how the message body (the "entity") is encoded.
This works fine:
package url;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.URL;
/**
* UrlReader
* #author Michael
* #since 3/20/11
*/
public class UrlReader
{
public static void main(String[] args)
{
UrlReader urlReader = new UrlReader();
for (String url : args)
{
try
{
String contents = urlReader.readContents(url);
System.out.printf("url: %s contents: %s\n", url, contents);
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
public String readContents(String address) throws IOException
{
StringBuilder contents = new StringBuilder(2048);
BufferedReader br = null;
try
{
URL url = new URL(address);
br = new BufferedReader(new InputStreamReader(url.openStream()));
String line = "";
while (line != null)
{
line = br.readLine();
contents.append(line);
}
}
finally
{
close(br);
}
return contents.toString();
}
private static void close(Reader br)
{
try
{
if (br != null)
{
br.close();
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
This:
public class Main {
public static void main(String[] args)
throws MalformedURLException, IOException
{
URL pageUrl = new URL("http://www.google.com");
URLConnection getConn = pageUrl.openConnection();
getConn.connect();
BufferedReader dis = new BufferedReader(
new InputStreamReader(
getConn.getInputStream()));
String myString;
while ((myString = dis.readLine()) != null)
{
System.out.println(myString);
}
}
}
Works perfectly. The URL you are supplying, however, returns nothing.