We Keep Coding

open URL with Java in background

have a problem here with Java. I'm a beginner and I don't know why it's not working.
What I want:
I have an homepage, I want to change some properties of a module. Therefore I have an .php-file. Going to the URL (looks like http://subdomain.domain.de/path_to_module/file.php?property=XY&api=Z) is all I have to do. To make this faster and easier especially also for friends, I want to code a little app.
I looked up google and get so far:
public void onClick(View v) {
try {
URL url = new URL("http://subdomain.domain.de/path_to_module/file.php?
property=XY&api=Z");
HttpURLConnection connection = null ;
try {
connection = (HttpURLConnection) url.openConnection();
}
finally
{
if( connection != null )
connection.disconnect() ;
}
} catch (MalformedURLException e){
} catch (IOException e){}
}
But that isn't working. The App don't crash or anything. It looks like it works, but when you look on the website it isn't changing anything.
Go to the URL by hand is working, so this can't be the problem...
Do you have any ideas, how to solve the problem?
Please excuse bad spelling or something like that, I'm not a native speaker.

Hey #David Conrad thanks for your help.
If I get you right, the new Code should look like this:
public void onClick(View v) {
try {
URL url = new URL("http://subdomain.domain.de/path_to_module/file.php?
property=XY&api=Z");
HttpURLConnection connection = null ;
try {
connection = (HttpURLConnection) url.openConnection();
connection.connect();
}
finally
{
if( connection != null )
connection.disconnect() ;
}
} catch (MalformedURLException e){
} catch (IOException e){}
}
I tried it, but it didn't work. Did I missunderstand what you are talking about?

The URL openConnection() method doesn't make the connection:
It should be noted that a URLConnection instance does not establish the actual network connection on creation. This will happen only when calling URLConnection.connect().
You have to call URLConnection.connect() on the HttpURLConnection object it returns.
Opens a communications link to the resource referenced by this URL, if such a connection has not already been established.
The HttpURLConnection doesn't begin transferring data right away because you may want to set HTTP headers, or you may want to do a POST or a PUT instead of a GET. You may have to get the input stream, and possibly even read from it, to cause it to make a GET request from the server (if that is what you are trying to do).
InputStream is = connection.getInputStream();
is.transferTo(new ByteArrayOutputStream());
This will read all the data from the connection and write it to a buffer, which you could discard if you don't care about its contents.

AWS elasticsearch access Request signing using Java

I want to connect to elastic search from Java. Elastic search domain is configured in AWS. I am using Jest library for this. Currently i have added my system ip in the elastic search configure access section. So i can access ES endpoint. But this is not the right way of doing it. What are the approches to it ? i know about signing the request but could not find any good reference of how to do it in java. Can anyone give some thoughts ?
This is how my code looks like
JestClientFactory factory = new JestClientFactory();
factory.setHttpClientConfig(new HttpClientConfig.Builder(elasticSearchserverUrl).connTimeout(10000).readTimeout(10000)
.multiThreaded(true).build());
JestClient client = factory.getObject();
Search.Builder searchBuilder = new Search.Builder(query).addIndices(indices).addType(type);
try {
SearchResult result = client.execute(searchBuilder.build());
List<Hit<String, Void>> hits = result.getHits(String.class);
for (Hit<String, Void> hit : hits) {
String log = hit.source;
System.out.println(log);
}
} catch (IOException e) {
}

Mocking/Testing HTTP Get Request

I'm trying to write unit tests for my program and use mock data. I'm a little confused on how to intercept an HTTP Get request to a URL.
My program calls a URL to our API and it is returned a simple XML file. I would like the test to instead of getting the XML file from the API online to receive a predetermined XML file from me so that I can compare the output to the expected output and determine if everything is working correctly.
I was pointed to Mockito and have been seeing many different examples such as this SO post, How to use mockito for testing a REST service? but it's not becoming clear to me how to set it all up and how to mock the data (i.e., return my own xml file whenever the call to the URL is made).
The only thing I can think of is having another program made that's running locally on Tomcat and in my test pass a special URL that calls the locally running program on Tomcat and then return the xml file that I want to test with. But that just seems like overkill and I don't think that would be acceptable. Could someone please point me in the right direction.
private static InputStream getContent(String uri) {
HttpURLConnection connection = null;
try {
URL url = new URL(uri);
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Accept", "application/xml");
return connection.getInputStream();
} catch (MalformedURLException e) {
LOGGER.error("internal error", e);
} catch (IOException e) {
LOGGER.error("internal error", e);
} finally {
if (connection != null) {
connection.disconnect();
}
}
return null;
}
I am using Spring Boot and other parts of the Spring Framework if that helps.

Part of the problem is that you're not breaking things down into interfaces. You need to wrap getContent into an interface and provide a concrete class implementing the interface. This concrete class will then
need to be passed into any class that uses the original getContent. (This is essentially dependency inversion.) Your code will end up looking something like this.
public interface IUrlStreamSource {
InputStream getContent(String uri)
}
public class SimpleUrlStreamSource implements IUrlStreamSource {
protected final Logger LOGGER;
public SimpleUrlStreamSource(Logger LOGGER) {
this.LOGGER = LOGGER;
}
// pulled out to allow test classes to provide
// a version that returns mock objects
protected URL stringToUrl(String uri) throws MalformedURLException {
return new URL(uri);
}
public InputStream getContent(String uri) {
HttpURLConnection connection = null;
try {
Url url = stringToUrl(uri);
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Accept", "application/xml");
return connection.getInputStream();
} catch (MalformedURLException e) {
LOGGER.error("internal error", e);
} catch (IOException e) {
LOGGER.error("internal error", e);
} finally {
if (connection != null) {
connection.disconnect();
}
}
return null;
}
}
Now code that was using the static getContent should go through a IUrlStreamSource instances getContent(). You then provide to the object that you want to test a mocked IUrlStreamSource rather than a SimpleUrlStreamSource.
If you want to test SimpleUrlStreamSource (but there's not much to test), then you can create a derived class that provides an implementation of stringToUrl that returns a mock (or throws an exception).

The other answers in here advise you to refactor your code to using a sort of provider which you can replace during your tests - which is the better approach.
If that isn't a possibility for whatever reason you can install a custom URLStreamHandlerFactory that intercepts the URLs you want to "mock" and falls back to the standard implementation for URLs that shouldn't be intercepted.
Note that this is irreversible, so you can't remove the InterceptingUrlStreamHandlerFactory once it's installed - the only way to get rid of it is to restart the JVM. You could implement a flag in it to disable it and return null for all lookups - which would produce the same results.
URLInterceptionDemo.java:
public class URLInterceptionDemo {
private static final String INTERCEPT_HOST = "dummy-host.com";
public static void main(String[] args) throws IOException {
// Install our own stream handler factory
URL.setURLStreamHandlerFactory(new InterceptingUrlStreamHandlerFactory());
// Fetch an intercepted URL
printUrlContents(new URL("http://dummy-host.com/message.txt"));
// Fetch another URL that shouldn't be intercepted
printUrlContents(new URL("http://httpbin.org/user-agent"));
}
private static void printUrlContents(URL url) throws IOException {
try(InputStream stream = url.openStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(stream))) {
String line;
while((line = reader.readLine()) != null) {
System.out.println(line);
}
}
}
private static class InterceptingUrlStreamHandlerFactory implements URLStreamHandlerFactory {
#Override
public URLStreamHandler createURLStreamHandler(final String protocol) {
if("http".equalsIgnoreCase(protocol)) {
// Intercept HTTP requests
return new InterceptingHttpUrlStreamHandler();
}
return null;
}
}
private static class InterceptingHttpUrlStreamHandler extends URLStreamHandler {
#Override
protected URLConnection openConnection(final URL u) throws IOException {
if(INTERCEPT_HOST.equals(u.getHost())) {
// This URL should be intercepted, return the file from the classpath
return URLInterceptionDemo.class.getResource(u.getHost() + "/" + u.getPath()).openConnection();
}
// Fall back to the default handler, by passing the default handler here we won't end up
// in the factory again - which would trigger infinite recursion
return new URL(null, u.toString(), new sun.net.www.protocol.http.Handler()).openConnection();
}
}
}
dummy-host.com/message.txt:
Hello World!
When run, this app will output:
Hello World!
{
"user-agent": "Java/1.8.0_45"
}
It's pretty easy to change the criteria of how you decide which URLs to intercept and what you return instead.

The answer depends on what you are testing.
If you need to test the processing of the InputStream
If getContent() is called by some code that processes the data returned by the InputStream, and you want to test how the processing code handles specific sets of input, then you need to create a seam to enable testing. I would simply move getContent() into a new class, and inject that class into the class that does the processing:
public interface ContentSource {
InputStream getContent(String uri);
}
You could create a HttpContentSource that uses URL.openConnection() (or, better yet, the Apache HttpClientcode).
Then you would inject the ContentSource into the processor:
public class Processor {
private final ContentSource contentSource;
#Inject
public Processor(ContentSource contentSource) {
this.contentSource = contentSource;
}
...
}
The code in Processor could be tested with a mock ContentSource.
If you need to test the fetching of the content
If you want to make sure that getContent() works, you could create a test that starts a lightweight in-memory HTTP server that serves the expected content, and have getContent() talk to that server. That does seem overkill.
If you need to test a large subset of the system with fake data
If you want to make sure things work end to end, write an end to end system test. Since you indicated you use Spring, you can use Spring to wire together parts of the system (or to wire the entire system, but with different properties). You have two choices
Have the system test start a local HTTP server, and when you have your test create your system, configure it to talk to that server. See the answers to this question for ways to start the HTTP server.
Configure spring to use a fake implementation of ContentSource. This gets you slightly less confidence that everything works end-to-end, but it will be faster and less flaky.

Login to https website and download a page using only core Java APIs

I want to login to a https website with username and password, go to one url in that website and download the page at the url (and maybe parse contents of that page). I want to do this using only core Java apis and not htmlunit, jsoup etc. I got the below code to learn how to do this, but it does not show me how to login to a website. Please tell me how I can login, maintain a session and then finally close the connection.
Source - http://www.mkyong.com/java/java-https-client-httpsurlconnection-example/
import java.net.MalformedURLException;
import java.net.URL;
import java.security.cert.Certificate;
import java.io.*;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLPeerUnverifiedException;
public class HttpsClient{
public static void main(String[] args)
{
new HttpsClient().testIt();
}
private void testIt(){
String https_url = "https://www.google.com/";
URL url;
try {
url = new URL(https_url);
HttpsURLConnection con = (HttpsURLConnection)url.openConnection();
//dumpl all cert info
print_https_cert(con);
//dump all the content
print_content(con);
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
private void print_https_cert(HttpsURLConnection con){
if(con!=null){
try {
System.out.println("Response Code : " + con.getResponseCode());
System.out.println("Cipher Suite : " + con.getCipherSuite());
System.out.println("\n");
Certificate[] certs = con.getServerCertificates();
for(Certificate cert : certs){
System.out.println("Cert Type : " + cert.getType());
System.out.println("Cert Hash Code : " + cert.hashCode());
System.out.println("Cert Public Key Algorithm : "
+ cert.getPublicKey().getAlgorithm());
System.out.println("Cert Public Key Format : "
+ cert.getPublicKey().getFormat());
System.out.println("\n");
}
} catch (SSLPeerUnverifiedException e) {
e.printStackTrace();
} catch (IOException e){
e.printStackTrace();
}
}
}
private void print_content(HttpsURLConnection con){
if(con!=null){
try {
System.out.println("****** Content of the URL ********");
BufferedReader br =
new BufferedReader(
new InputStreamReader(con.getInputStream()));
String input;
while ((input = br.readLine()) != null){
System.out.println(input);
}
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}

Every website manages logins differently. You will need to scout the website, find out how the session is maintained, and mimic the functions in such a way that the server can't tell that it is not a browser.
In general, a web server stores a secret hash in the cookie. Here is the process
Post a login and password to said url using HttpsURLConnection to send the form.
The server responds with a hash in a header that it wants stored in the cookie. Usually has session in the name.
Send requests back with the hash in the header in the correct value
All of the above can be done only using URL and HttpsURLConnection, but you will need to mimic a browser exactly to trick the server.
For scouting, I would recommend using a tool like fiddler. It captures all communication from the webserver and back, so that you can see exactly what is going on at the http level to mimic in your java code.
Here is an overview of fiddler. I have never looked at the logs. Fiddler has a sweet interface. The video is really boring, but it gives an overview of the interface. You want to look at the raw text view, and mimic that.
For your other question, owasp is a great resource for best practices. The reality is that there is a lot of insecure and bad code out there that does stuff that you would never expect. I have seen a server put the boolean value inside of a script tag to be stored as a javascript variable. You just have to carefully watch how the server changes the responses after you log in. For a popular website following best practices, they will use the above method.

Limit web service access to only one app

Right now I am working on a android app and I am totally new to this.
I want to make sure my web-service is only accessible via my app.
My background is PHP. In PHP I don't need to worry about anything like that, because everything runs on a server.
In case of Java and especially Android programming things are different. Even with encryption. Everybody can just open an APK and see how the web service gets accessed. So is there a way to hide or to obfuscate the access to a web service, so only my app will be able to use it?
For test purposes I didn't add any security or encryption. This is the basic call to a web server I am doing right now:
String url = "http://thisismyurl.com/a.php?action=get";
String result = Web.executeWeb(url);
public class Web {
public static String executeWeb(final String url) {
final StringBuilder sb = new StringBuilder();
Thread thread = new Thread(new Runnable() {
public void run()
{
try
{
InputStream is = (InputStream) new URL(url).getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
String result, line = reader.readLine();
result = line;
while((line=reader.readLine())!=null){
result+=line;
}
sb.append(result);
//System.out.println(result);
//Log.i("My Response :: ", result);
} catch (Exception e)
{
// TODO: handle exception
}
}
});
thread.start();
try {
thread.join();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return sb.toString();
}
}
How would I hide this from the prying eyes of hackers? ;-) Is that even possible?
Thanks in advance!

Deploy client authentication using (self signed) certificates within TLS.
This kind of configuration can be enabled on most web servers and Java application servers, and you can normally also configure the web or application server in such a way that you can retrieve the certificate of the private key that the client used to authenticate itself.
Note that HTTPS uses SSL (or now TLS) before any web trafic, so you cannot program this in your application, it does require server configuration.
Check this link on how to configure for Apache 2.

Use your Application's ID ( like IMEI ) as parameter in your webservice call. You need to make a table in database at server side which will store all registered device. Now only these registered device can access your webservice. This is my idea, there should be other idea as well.