We Keep Coding

why java post method connection timeout even it runs on browser?

I look network on developer tab(google chrome) and here is the ss
I want to get data on java app.
My code is here:
String urlParameters = "tabloNo=4&yil=2017&ay=2&paraBirimi=TL&taraf=10001";
byte[] postData = urlParameters.getBytes( StandardCharsets.UTF_8 );
int postDataLength = postData.length;
String request = "http://172.168.12*.15/getreport";
URL url = new URL( request );
HttpURLConnection conn= (HttpURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setInstanceFollowRedirects(false);
conn.setConnectTimeout(10000);
conn.setRequestMethod("POST");
conn.setRequestProperty("Accept", "pplication/json, text/javascript, */*; q=0.01");
// conn.setRequestProperty("Accept-Encoding:", "gzip, deflate" );
conn.setRequestProperty("Accept-Language", "tr,en;q=0.8" );
conn.setRequestProperty("Content-Length", Integer.toString(postDataLength ));
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8");
conn.setRequestProperty("Host", "ebulten.bddk.org.tr" );
conn.setRequestProperty("Origin","http://172.168.12*.15/");
conn.setRequestProperty("Proxy-Connection", "keep-alive" );
conn.setRequestProperty("Referer", "http://172.168.12*.15/" );
conn.setRequestProperty("User-Agent","Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" );
conn.setRequestProperty("X-Requested-With","XMLHttpRequest" );
conn.setUseCaches(true);
try(DataOutputStream wr = new DataOutputStream(conn.getOutputStream())) {
wr.write( postData );
}
it gives Connection timed out: connect
note:
it is public website and everyone read data. I want to read it with programmatically.
thanks in advance

HttpURLConnection with https InputStream Garbled

I use HttpURLConnection to crawler https://translate.google.com/.
InetSocketAddress addr = new InetSocketAddress("127.0.0.1", 1082);
Proxy proxy = new Proxy(Proxy.Type.HTTP, addr);
url = new URL("https://translate.google.com/");
HttpURLConnection conn = (HttpURLConnection) url.openConnection(proxy);
conn.setRequestProperty("Accept-Encoding", "gzip, deflate, sdch");
conn.setRequestProperty("Connection", "keep-alive");
conn.setRequestProperty("User-Agent",
"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.76 Mobile Safari/537.36");
conn.setRequestProperty("Accept", "*/*");
Map<String, List<String>> reqHeaders = conn.getHeaderFields();
List<String> reqTypes = reqHeaders.get("Content-Type");
for (String ss : reqTypes) {
System.out.println(ss);
}
InputStream in = conn.getInputStream();
String s = IOUtils.toString(in, "UTF-8");
System.out.println(s.substring(0, 100));
Map<String, List<String>> resHeader = conn.getHeaderFields();
List<String> resTypes = resHeader.get("Content-Type");
for (String ss : resTypes) {
System.out.println(ss);
}
Console is
But When I change url to http://translate.google.com/.
It works well.
I know actually HttpURLConnection is HttpsURLConnection when i crawler https://translate.google.com/.
I try to use HttpsURLConnection and it still garbled.
Any suggestions?

conn.setRequestProperty("Accept-Encoding", "gzip, deflate, sdch");
The response is compressed, because the above line tells the server that the client is able to understand encodings specified in Accept-Encoding.
Try to comment this line or handle this situation.
There's a more specific implementation for HTTPS i.e. HttpsURLConnection, in case you're interested in https-specific features, e.g.:
import javax.net.ssl.HttpsURLConnection;
....
URL url = new URL("https://www.google.com/");
HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();

I accept Jerry Chin's answer.Solves my problem.
My answer just recording how i resolve this problem.
If this approach is unreasonable.Let me know, I'll remove this answer.
conn.setRequestProperty("Accept-Encoding", "gzip, deflate, sdch");
And then I check response Content-Encoding.It's gzip.
So i use GZIPInputStream to receive.
InputStream in = conn.getInputStream();
GZIPInputStream gzis=new GZIPInputStream(in);
InputStreamReader reader = new InputStreamReader(gzis);
BufferedReader br = new BufferedReader(reader);
The InputStream is normal.
BTW,If you don't need Accept-Encoding,you can remove it.
And do not forget check user-agent. It's very important and different operating systems corresponding to different user-agent.

Java send request encoding

I'm trying to send get request in java, when it's without headers as below, I get redirected to login page.
URL myURL1 = new URL("http://www.eventlister.com");
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setRequestMethod("GET");
But when request is sent with all headers as it is sent from browser when reading response I get it encoded like this
???}isI??g)b?CZ?n?7BH?:m?u??dkzz; $P????C2?????YT??;??B3m????7/?.o~?p,?????..., even if encoding is set manually in reader.
URL myURL1 = new URL("http://www.eventlister.com");
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setRequestMethod("GET");
con.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36");
con.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
con.setRequestProperty("Accept-Encoding", "gzip, deflate, sdch");
con.setRequestProperty("Accept-Language", "en-US,en;q=0.8,zh-CN;q=0.6,zh;q=0.4,ru;q=0.2,uk;q=0.2,ja;q=0.2,fr-FR;q=0.2,fr;q=0.2");
con.setRequestProperty("Connection", "keep-alive");
con.setRequestProperty("Upgrade-Insecure-Requests", "1");
con.setRequestProperty("Host", "www.eventlister.com");
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream(), StandardCharsets.UTF_8));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
Thank you in advance.

It looks like it's gzip'ed data. Try it without "Accept-Encoding: gzip", so that server returns plain data.
I know this is a comment, not an answer. I don't have comment privilege yet :-(

Java. need to read a url as string and timeout option

already asked in another post, but got no solution
see me script bellow, how can i change that, please full demo, so
that i can set a readtimout if the network is down for example
at all i have to connect to an url, read just one line, get that as a string ... done
hope u can give an example. any other working way is welcome.. thx
try {
URL url = new URL("http://mydomain/myfile.php");
//url.setReadTimeout(5000); does not work
InputStreamReader testi= new InputStreamReader(url.openStream());
BufferedReader in = new BufferedReader(testi);
//in.setReadTimeout(5000); does not work
stri = in.readLine();
Log.v ("GotThat: ",stri);
in.close();
} catch (MalformedURLException e) {
} catch (IOException e) {
}

How about using HttpURLConnection
import java.net.HttpURLConnection;
URL url = new URL("http://mydomain/myfile.php");
HttpURLConnection huc = (HttpURLConnection) url.openConnection();
HttpURLConnection.setFollowRedirects(false);
huc.setConnectTimeout(15 * 1000);
huc.setRequestMethod("GET");
huc.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729)");
huc.connect();
InputStream input = huc.getInputStream();
Code picked from here

Timeout:
URL url = new URL("http://www.google.com");
URLConnection conn = url.openConnection();
conn.connect();
conn.setReadTimeout(10000); //10000 milliseconds (10 seconds)

How to check if a URL exists or returns 404 with Java?

String urlString = "http://www.nbc.com/Heroes/novels/downloads/Heroes_novel_001.pdf";
URL url = new URL(urlString);
if(/* Url does not return 404 */) {
System.out.println("exists");
} else {
System.out.println("does not exists");
}
urlString = "http://www.nbc.com/Heroes/novels/downloads/Heroes_novel_190.pdf";
url = new URL(urlString);
if(/* Url does not return 404 */) {
System.out.println("exists");
} else {
System.out.println("does not exists");
}
This should print
exists
does not exists
TEST
public static String URL = "http://www.nbc.com/Heroes/novels/downloads/";
public static int getResponseCode(String urlString) throws MalformedURLException, IOException {
URL u = new URL(urlString);
HttpURLConnection huc = (HttpURLConnection) u.openConnection();
huc.setRequestMethod("GET");
huc.connect();
return huc.getResponseCode();
}
System.out.println(getResponseCode(URL + "Heroes_novel_001.pdf"));
System.out.println(getResponseCode(URL + "Heroes_novel_190.pdf"));
System.out.println(getResponseCode("http://www.example.com"));
System.out.println(getResponseCode("http://www.example.com/junk"));
Output
200
200
200
404
SOLUTION
Add the next line before .connect() and the output would be 200, 404, 200, 404
huc.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729)");

You may want to add
HttpURLConnection.setFollowRedirects(false);
// note : or
// huc.setInstanceFollowRedirects(false)
if you don't want to follow redirection (3XX)
Instead of doing a "GET", a "HEAD" is all you need.
huc.setRequestMethod("HEAD");
return (huc.getResponseCode() == HttpURLConnection.HTTP_OK);

this worked for me:
URL u = new URL ( "http://www.example.com/");
HttpURLConnection huc = ( HttpURLConnection ) u.openConnection ();
huc.setRequestMethod ("GET"); //OR huc.setRequestMethod ("HEAD");
huc.connect () ;
int code = huc.getResponseCode() ;
System.out.println(code);
thanks for the suggestions above.

Use HttpUrlConnection by calling openConnection() on your URL object.
getResponseCode() will give you the HTTP response once you've read from the connection.
e.g.
URL u = new URL("http://www.example.com/");
HttpURLConnection huc = (HttpURLConnection)u.openConnection();
huc.setRequestMethod("GET");
huc.connect() ;
OutputStream os = huc.getOutputStream();
int code = huc.getResponseCode();
(not tested)

Based on the given answers and information in the question, this is the code you should use:
public static boolean doesURLExist(URL url) throws IOException
{
// We want to check the current URL
HttpURLConnection.setFollowRedirects(false);
HttpURLConnection httpURLConnection = (HttpURLConnection) url.openConnection();
// We don't need to get data
httpURLConnection.setRequestMethod("HEAD");
// Some websites don't like programmatic access so pretend to be a browser
httpURLConnection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729)");
int responseCode = httpURLConnection.getResponseCode();
// We only accept response code 200
return responseCode == HttpURLConnection.HTTP_OK;
}
Of course tested and working.

There is nothing wrong with your code. It's the NBC.com doing tricks on you. When NBC.com decides that your browser is not capable of displaying PDF, it simply sends back a webpage regardless what you are requesting, even if it doesn't exist.
You need to trick it back by telling it your browser is capable, something like,
conn.setRequestProperty("User-Agent",
"Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.13) Gecko/2009073021 Firefox/3.0.13");