I'm encountering a problem while downloading a gzip file and saving it in the file system with Java 5. When the download is finished, the file (which has the correct name and extension) appears to have a different MIME type... I'm unable to unzip it from my Linux server (using gunzip) and if I try to open it with WinZip on my Windows pc I see a "recursive" archive (like a matryoshka doll). If I type the command file *filename*.gz from the server, it recognizes the file as an ascii text.
Instead, if I try to download the archive using the browser, everything goes well and I can correctly open and unzip the file (even with my Linux server) and now it's recognized as a gzip compressed archive.
Here's the code I'm using to download the file and save it.
Main.java:
public class Main {
public static void main(String[] args) {
String filePath = "";
HttpOutgoingCall httpOngoingCall = null;
httpOngoingCall = new HttpOutgoingCall();
String endpointUrl = "https://myurl/myfile.gz";
try {
InputStream inputStream = httpOngoingCall.callHttps(endpointUrl);
//I also tried with ZipInputStream and GZIPInputStream
BufferedReader br = new BufferedReader(new InputStreamReader(inputStream));
filePath = parseAndWriteResponse(br, "myfile.gz", "C:\\");
System.out.println(filePath);
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
private static String parseAndWriteResponse(BufferedReader br, String fileName,
String destPath) {
String outputFileName = null;
outputFileName = destPath + File.separator + fileName;
String line;
File outputFile = new File(outputFileName);
FileWriter fileWriter = null;
BufferedWriter bw = null;
try {
if (!outputFile.exists()) {
outputFile.createNewFile();
}
} catch (IOException e1) {
}
try {
fileWriter = new FileWriter(outputFile);
bw = new BufferedWriter(fileWriter);
while ((line = br.readLine()) != null) {
bw.write(line);
bw.write("\n");
}
} catch (IOException e) {
} finally {
try {
bw.close();
fileWriter.close();
} catch (IOException e) {
}
}
return outputFileName;
}
HttpOutgoingCall.java:
public class HttpOutgoingCall {
private InputStream inStream = null;
private HttpsURLConnection httpsConnection = null;
private final static int CONNECTION_TIMEOUT = 20000;
public InputStream callHttps(String endpointUrl) throws Exception {
String socksServer = "";
String socksPort = "";
Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());
Properties properties = System.getProperties();
System.setProperty("java.protocol.handler.pkgs", "javax.net.ssl");
java.security.Security
.addProvider(new com.sun.net.ssl.internal.ssl.Provider());
if (!socksServer.equals("")) {
if (System.getProperty("socksProxyHost") == null) {
properties.put("socksProxyHost", socksServer);
}
if (!socksPort.equals("")) {
if (System.getProperty("socksProxyPort") == null) {
properties.put("socksProxyPort", socksPort);
}
}
}
System.setProperties(properties);
System.setProperty("java.protocol.handler.pkgs", "javax.net.ssl");
java.security.Security
.addProvider(new com.sun.net.ssl.internal.ssl.Provider());
TrustManager[] trustAllCerts = new TrustManager[] { new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(
java.security.cert.X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(
java.security.cert.X509Certificate[] certs, String authType) {
}
} };
try {
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, new java.security.SecureRandom());
HttpsURLConnection
.setDefaultSSLSocketFactory(sc.getSocketFactory());
HostnameVerifier hv = new HostnameVerifier() {
public boolean verify(String urlHostName, SSLSession session) {
return true;
}
};
HttpsURLConnection.setDefaultHostnameVerifier(hv);
httpsConnection = (HttpsURLConnection) (new URL(endpointUrl)).openConnection();
httpsConnection.setDoOutput(true);
httpsConnection.setUseCaches(false);
httpsConnection.setConnectTimeout(CONNECTION_TIMEOUT);
httpsConnection.setReadTimeout(CONNECTION_TIMEOUT);
inStream = httpsConnection.getInputStream();
} catch (Exception e) {}
return inStream;
}
Could someone help me? Thanks!
When writing your file, you should send it through a java.util.zip.GZIPOutputStream.
Related
I wrtite parser without third-party libraries. Get html code from web site - http://www.cnn.com/ - but some part of code has unicode instead symbols, for example: "\u003cbr/>Sign in to your TV service provider to get access to \u003cbr/>" i think it is problem with encode - how i can fix it? Sorry for my English. Thank you.
public class Main {
public static void main(String[] args) throws IOException {
String commandLine = Scraper.readLineFromConsole();
Reader reader = Scraper.getReader(commandLine);
Scraper.writeInFileFromURL(reader);
}
public static class Scraper {
public static void writeInFileFromURL(Reader out) {
Reader reader = out;
BufferedReader br = new BufferedReader(reader);
try {
PrintWriter writer = new PrintWriter("newFile.txt");
String htmltext;
while (br.ready()) {
htmltext = br.readLine();
writer.write(new String(htmltext));
}
writer.flush();
writer.close();
} catch (IOException e) {
e.printStackTrace();
}
}
public static String readLineFromConsole() {
BufferedReader reader = new BufferedReader(new InputStreamReader(System.in));
String commandLine = null;
try {
commandLine = reader.readLine();
} catch (IOException e) {
e.printStackTrace();
}
return commandLine;
}
public static Reader getReader(String url)
throws IOException {
// Retrieve from Internet.
if (url.startsWith("http:") || url.startsWith("https:")) {
URLConnection conn = new URL(url).openConnection();
return new InputStreamReader(conn.getInputStream());
}
// Retrieve from file.
else {
return new FileReader(url);
}
}
}
}
This is basically how I am reading the content from an URL and get the content in the variable result. The result is a json parsed after.
I was wondering if it was a way to increase the performance of this request as it takes up to 4~5 seconds while I would like it to take, if possible, less than 2.
Thing is trustAllHosts() and BufferedReader br = new BufferedReader(new InputStreamReader(in)); seems to take some time, but I'm not sure about that. Any idea will be appreciated!
try {
URL url;
// get URL content
url = new URL(apiURL);
trustAllHosts();
conn = (HttpsURLConnection) url.openConnection();
conn.setHostnameVerifier(DO_NOT_VERIFY);
conn.setRequestMethod("GET");
conn.setRequestProperty("some values here");
conn.setConnectTimeout(10000);
in=conn.getInputStream();
// open the stream and put it into BufferedReader
BufferedReader br = new BufferedReader(new InputStreamReader(in));
while ((line=br.readLine())!= null) {
builder.append(line);
}
result=builder.toString();
//System.out.print(result);
br.close();
} catch (MalformedURLException e) {
result=null;
} catch (java.net.SocketTimeoutException e) {
result=null;
} catch (IOException e) {
result=null;
}
catch (Exception e) {
result=null;
}
finally {
try {
in.close();
}catch(Exception e){}
try {
conn.disconnect();
}catch(Exception e){}
return result;
}
trustAllHosts() :
/**
* Trust every server - dont check for any certificate
*/
private static void trustAllHosts() {
// Create a trust manager that does not validate certificate chains
TrustManager[] trustAllCerts = new TrustManager[] { new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return new java.security.cert.X509Certificate[] {};
}
public void checkClientTrusted(X509Certificate[] chain,
String authType) throws CertificateException {
}
public void checkServerTrusted(X509Certificate[] chain,
String authType) throws CertificateException {
}
} };
// Install the all-trusting trust manager
try {
SSLContext sc = SSLContext.getInstance("TLS");
sc.init(null, trustAllCerts, new java.security.SecureRandom());
HttpsURLConnection
.setDefaultSSLSocketFactory(sc.getSocketFactory());
} catch (Exception e) {
e.printStackTrace();
}
}
I'm trying to access a web service through REST API post method and end up with FileNotFoundException
code:
public class TestService {
static {
disableSSLVerification();
}
public static void main(String[] args) {
String[] names = {"login","seq","password"};
String[] values = { "admin", "2811", "admin" };
String url = "https://localhost:8844/login";
try {
httpPost(url, names, values);
} catch (Exception e) {
e.printStackTrace();
}
}
public static String httpPost(String urlStr, String[] paramName, String[] paramVal) throws Exception {
URL url = new URL(urlStr);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setDoOutput(true);
conn.setDoInput(true);
conn.setUseCaches(false);
conn.setAllowUserInteraction(false);
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
OutputStream out = conn.getOutputStream();
Writer writer = new OutputStreamWriter(out, "UTF-8");
for (int i = 0; i < paramName.length; i++) {
writer.write(paramName[i]);
writer.write("=");
writer.write(URLEncoder.encode(paramVal[i], "UTF-8"));
writer.write("&");
}
System.out.println("WRITER: " + writer);
writer.close();
out.close();
if (conn.getResponseCode() != 200) {
throw new IOException(conn.getResponseMessage());
}
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
StringBuilder sb = new StringBuilder();
String line;
while ((line = rd.readLine()) != null) {
sb.append(line);
}
rd.close();
conn.disconnect();
return sb.toString();
}
public static void disableSSLVerification() {
TrustManager[] trustAllCerts = new TrustManager[] { new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(X509Certificate[] certs, String authType) {
}
} };
SSLContext sc = null;
try {
sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, new java.security.SecureRandom());
} catch (KeyManagementException e) {
e.printStackTrace();
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
}
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
HostnameVerifier allHostsValid = new HostnameVerifier() {
public boolean verify(String hostname, SSLSession session) {
return true;
}
};
HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);
}
}
Log:
java.io.FileNotFoundException: https://localhost:8844/login
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown S
ource)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown So
urce)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unkn
own Source)
can anyone please help me to resolve this? please try to help me rather marking this one 'duplicate'.
It's actually an HttpsURLConnection (you are opening a https:// url).
That URL does not exist, try opening it in your browser. If the url exists it could be that you are using a self-signed certificate on that https host, that is rejected by java urlconnection classes(but i don't think that's the case, the exception should be different, in that case you'll need to implement a wrapper that accept the certificate anyway).
You may try removing:
conn.setDoOutput(true);
conn.setDoInput(true);
since response was not received because there's the problem with the method signature. it has been updated. I have changed accordingly and now it is working fine.
String[] names = {"username","password"};
String[] values = { "admin", "admin" };
String url = "https://localhost:8844/session";
I am trying to read the content of the URL with the code down below, but get an 403 error:
https://api.kraken.com/0/public/Time
The URL is reachable in the Problem or via Rest-Connection-Test-Pages like https://apigee.com without a problem. The code works for other HTTPS-Urls just fine. I have no clue what could be wrong here:
public class HttpsClient{
public static void main(String[] args)
{
new HttpsClient().testIt();
}
private void testIt(){
// Install the all-trusting trust manager
try {
SSLContext sc = SSLContext.getInstance("TLS");
sc.init(null, trustAllCerts, new SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
} catch (Exception e) {
;
}
String https_url = "https://api.kraken.com/0/public/Time";
URL url;
HttpsURLConnection con = null;
try {
url = new URL(https_url);
con = (HttpsURLConnection)url.openConnection();
//dumpl all cert info
print_https_cert(con);
try{
System.out.println("****** Content of the URL ********");
BufferedReader br =
new BufferedReader(
new InputStreamReader(con.getInputStream()));
String input;
while ((input = br.readLine()) != null){
System.out.println(input);
}
br.close();
} catch (IOException e) {
e.printStackTrace();
}
} catch (Exception e) {
e.printStackTrace();
}
}
// Create a trust manager that does not validate certificate chains
TrustManager[] trustAllCerts = new TrustManager[]{new X509TrustManager(){
#Override
public X509Certificate[] getAcceptedIssuers(){return null;}
#Override
public void checkClientTrusted(X509Certificate[] certs, String authType){}
#Override
public void checkServerTrusted(X509Certificate[] certs, String authType){}
}};
}
I want to get a connection via HTTPS to a Server. The Server uses a unsigned certificate.
I found a solution to download those certificates from sun.
That is my Code:
{
final char[] CertPassphrase = "changeit".toCharArray();
private boolean installCert() throws
KeyStoreException, NoSuchAlgorithmException,
CertificateException, KeyManagementException,
IOException {
boolean isCertInstalled = true;
File file = new File("jcacerts");
if (!file.isFile())
{
char SEP = File.separatorChar;
File dir = new File(System.getProperty("java.home") + SEP + "lib" + SEP + "security");
file = new File(dir, "jssecacerts");
if (!file.isFile())
{
file = new File(dir, "cacerts");
}
}
InputStream in = new FileInputStream(file);
KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType());
ks.load(in, CertPassphrase);
in.close();
SSLContext context = SSLContext.getInstance("TLS");
TrustManagerFactory tmf =
TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmf.init(ks);
X509TrustManager defaultTrustManager = (X509TrustManager) tmf.getTrustManagers()[0];
SavingTrustManager tm = new SavingTrustManager(defaultTrustManager);
context.init(null, new TrustManager[]
{
tm
}, null);
SSLSocketFactory factory = context.getSocketFactory();
SSLSocket socket = (SSLSocket) factory.createSocket(this.getStrUrl(), this.getPort() );
socket.setSoTimeout(10000);
try
{
socket.startHandshake();
socket.close();
} catch (SSLException e)
{
isCertInstalled = false;
}
if(!isCertInstalled){
X509Certificate[] chain = tm.chain;
if (chain == null)
{
return true;
}
BufferedReader reader =
new BufferedReader(new InputStreamReader(System.in));
System.out.println("Certificat from "+this.getStrUrl()+". [Enter] to download, [q] to quit!");
String line = reader.readLine().trim();
if(line.length()==0){
for(int i=0; i<chain.length; i++){
X509Certificate cert = chain[i];
String alias = this.getStrUrl() + "-" + (i + 1);
ks.setCertificateEntry(alias, cert);
OutputStream out = new FileOutputStream("jssecacerts");
ks.store(out, this.CertPassphrase);
out.close();
System.out.println("Added certificate to keystore 'jcacerts'");
}
return true;
}else{
return false;
}
}else
return true;
}
private HttpsURLConnection openHttpsConnection(){
try{
if(this.installCert()){
return (HttpsURLConnection)this.getUrl().openConnection();
}
else{
System.out.println("Zertifikat heruntergeladen");
return null;
}
}catch (KeyStoreException e){
System.out.println(e.getMessage());
}catch (NoSuchAlgorithmException e){
System.out.println(e.getMessage());
}catch (CertificateException e){
System.out.println(e.getMessage());
}catch (KeyManagementException e){
System.out.println(e.getMessage());
}catch (IOException e){
System.out.println(e.getMessage());
}
return null;
}
private void closeHttpsConnection(HttpsURLConnection con){
con.disconnect();
}
public void print_content(){
HttpsURLConnection con = this.openHttpsConnection();
try{
BufferedReader br = new BufferedReader( new InputStreamReader(con.getInputStream()));
String input;
while ((input = br.readLine()) != null){
System.out.println(input);
}
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
public Https(String Url) { this.setUrl(Url); }
private static class SavingTrustManager implements X509TrustManager
{
private final X509TrustManager tm;
private X509Certificate[] chain;
SavingTrustManager(X509TrustManager tm)
{
this.tm = tm;
}
public X509Certificate[] getAcceptedIssuers()
{
throw new UnsupportedOperationException();
}
public void checkClientTrusted(X509Certificate[] chain, String authType)
throws CertificateException
{
throw new UnsupportedOperationException();
}
public void checkServerTrusted(X509Certificate[] chain, String authType)
throws CertificateException
{
this.chain = chain;
tm.checkServerTrusted(chain, authType);
}
}
In my opinion, the certificate is downloaded completely. But if I try to read from the Input-Stream, following Exceptions are thrown:
javax.net.ssl.SSLHandshakeException:
java.security.cert.CertificateException: No name matching url found
Does someone know the correct solution for my problem or can otherwise help me?