Trying to build http://IP:4567/foldername/1234?abc=xyz. I don't know much about it but I wrote below code from searching from google:
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URL;
public class MyUrlConstruct {
public static void main(String a[]){
try {
String protocol = "http";
String host = "IP";
int port = 4567;
String path = "foldername/1234";
URL url = new URL (protocol, host, port, path);
System.out.println(url.toString()+"?");
} catch (MalformedURLException ex) {
ex.printStackTrace();
}
}
}
I am able to build URL http://IP:port/foldername/1234?. I am stuck at query part. Please help me to move forward.
You can just pass raw spec
new URL("http://IP:4567/foldername/1234?abc=xyz");
Or you can take something like org.apache.http.client.utils.URIBuilder and build it in safe manner with proper url encoding
URIBuilder builder = new URIBuilder();
builder.setScheme("http");
builder.setHost("IP");
builder.setPath("/foldername/1234");
builder.addParameter("abc", "xyz");
URL url = builder.build().toURL();
Use OkHttp
There is a very popular library named OkHttp which has been starred 20K times on GitHub. With this library, you can build the url like below:
import okhttp3.HttpUrl;
URL url = new HttpUrl.Builder()
.scheme("http")
.host("example.com")
.port(4567)
.addPathSegments("foldername/1234")
.addQueryParameter("abc", "xyz")
.build().url();
Or you can simply parse an URL:
URL url = HttpUrl.parse("http://example.com:4567/foldername/1234?abc=xyz").url();
In general non-Java terms, a URL is a specialized type of URI. You can use the URI class (which is more modern than the venerable URL class, which has been around since Java 1.0) to create a URI more reliably, and you can convert it to a URL with the toURL method of URI:
String protocol = "http";
String host = "example.com";
int port = 4567;
String path = "/foldername/1234";
String auth = null;
String fragment = null;
URI uri = new URI(protocol, auth, host, port, path, query, fragment);
URL url = uri.toURL();
Note that the path needs to start with a slash.
If you happen to be using Spring already, I have found the org.springframework.web.util.UriComponentsBuilder to be quite nifty. Here is how you would use it in your case.
final URL myUrl = UriComponentsBuilder
.fromHttpUrl("http://IP:4567/foldername/1234?abc=xyz")
.build()
.toUri()
.toURL();
If using Spring Framework:
UriComponentsBuilder.newInstance()
.scheme(scheme)
.host(host)
.path(path)
.build()
.toUri()
.toURL();
A new UriComponentsBuilder class helps to create UriComponents
instances by providing fine-grained control over all aspects of
preparing a URI including construction, expansion from template
variables, and encoding.
Know more:
https://www.baeldung.com/spring-uricomponentsbuilder
JavaDoc:
https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/web/util/UriComponentsBuilder.html
If you use Android, you can use the Uri.Builder API. Example:
val uri = Uri.Builder().scheme("https").authority("s3.amazonaws.com").appendEncodedPath(bucketName).appendEncodedPath(fileName).build()
Docs:
https://developer.android.com/reference/android/net/Uri.Builder
Related
I'm trying to make a program that submits a search query to Google and then opens the browser with the results.
I have managed to connect to Google but I'm stuck because I don't know how to insert the search query into the URL and submit it.
I have tried to use HtmlUnit but it doesn't seem to work.
This is the code so far:
URL url = new URL("http://google.com");
HttpURLConnection hr = (HttpURLConnection) url.openConnection();
System.out.println(hr.getResponseCode());
String str = "search from java!";
You can use the Java.net package to browse the internet. I have used an additional method to create the search query for google to replace the spaces with %20 for the URL address
public static void main(String[] args) {
URI uri= null;
String googleUrl = "https://www.google.com/search?q=";
String searchQuery = createQuery("search from Java!");
String query = googleUrl + searchQuery;
try {
uri = new URI(query);
Desktop.getDesktop().browse(uri);
} catch (IOException | URISyntaxException e) {
e.printStackTrace();
}
}
private static String createQuery(String query) {
query = query.replaceAll(" ", "%20");
return query;
}
The packages used are core java:
import java.awt.Desktop;
import java.net.URI;
import java.net.URISyntaxException;
Using Socket I can send http request to server and get the html response. My objective is to get each image may it be png, jpeg, gif, or any other image types.
However, by looking at the responses from different websites, I noticed that some images do not use html's <img> tag, and instead may be in CSS.
How can I extract both <img> images and css images (e.g. background-image)?
Is it good to use regex to get those images urls from <img>?
Please do not refer me to http classes like Apache HttpClient.
My problem is not on http protocol.
To get all images, including images loaded by css and perhaps js, you need more than the html code.
You need code that understands html and css and js.
You need a full browser.
Fortunately, Java comes with a browser. The JavaFX WebEngine.
Give it a url or html and it will load everything.
As WebKit, it knows the latest image loading technology, for example CSS border-image.
We just need a way to get its images.
It does not provide media list, but since it is pure Java, we can hijack Java's URL handler to intercept its requests:
import java.io.IOException; import java.net.URL; import java.net.URLConnection; import javafx.application.Application; import javafx.application.Platform; import javafx.concurrent.Worker; import javafx.scene.Scene; import javafx.scene.web.WebView; import javafx.stage.Stage;
public class NetworkMonitor extends Application {
private final String url = "http://www.google.com/";
public static void main( String[] args ) {
// Override default http/https handler. Must do once only.
URL.setURLStreamHandlerFactory( protocol ->
protocol.equals( "http" ) ? new HttpHandler() :
protocol.equals( "https" ) ? new HttpsHandler() : null );
// Launch as JavaFX app. Required for WebView / WebEngine.
launch( args );
}
#Override public void start(Stage primaryStage) throws Exception {
// Create webview and listen for ondone
WebView v = new WebView();
v.getEngine().getLoadWorker().stateProperty().addListener( ( prop, old, now ) -> {
if ( now == Worker.State.SUCCEEDED || now == Worker.State.FAILED )
Platform.exit(); } );
// Showing GUI is easiest way to make sure ondone will be fired.
primaryStage.setScene( new Scene( v ) );
primaryStage.show();
// Load the target url.
v.getEngine().load( url );
}
// Your IDE should warn you about the sun package.
private static class HttpHandler extends sun.net.www.protocol.http.Handler {
#Override protected URLConnection openConnection(URL url) throws IOException {
System.out.println( url ); // Capture url!
return super.openConnection( url );
}
}
// If there is no warning, you need to switch to a better IDE!
private static class HttpsHandler extends sun.net.www.protocol.https.Handler {
#Override protected URLConnection openConnection(URL url) throws IOException {
System.out.println( url ); // Capture url!
return super.openConnection( url );
}
}
}
Since you only asked how to get the url, this is what the code do.
The code can be expanded depending on your needs.
For example, two decorator objects for the URLConnection should allow you to intercept getInputStream call and query its header (to determine mime type) and fork the stream (to save a copy of the image).
If this answer is useful, don't forget to vote up!
As other answers have already mentioned, ideally you would use a tool that understands how to parse, render and recurse HTTP resources (i.e. .html/css/js/png/gif/jpg/etc).
That being said, if you were feeling particularly masochistic (and I suspect you are), you could do this yourself...
It's not a perfect solution, but if I was going to attack this with a blunt instrument, I'd use regular expressions (I won't go into the specifics of regex, it's already widely documented on the interwebs). My process would be:
HTTP GET my base page.
Strip out all strings that match your definition of a "resource" (using regex).
Optionally recurse those resources, for more strings.
You've already mentioned that you can perform HTTP request/responses (using Sockets), so I won't cover that here.
Voila!
/**
* Regular expression to match file types - .js/.css/.png/.jpg/.gif
*/
public static final Pattern resources = Pattern.compile("([^\"'\n({}]+\\.(js|css|png|jpg|gif))",
Pattern.CASE_INSENSITIVE | Pattern.MULTILINE);
/**
* Pulls out "resources" from the provided text.
*/
public static Set<String> findResources(URL url, String text) {
Matcher matcher = resources.matcher(text);
Set<String> resources = new HashSet<>();
while (matcher.find()) {
String resource = matcher.group(1);
String urlStr = url.toString();
int endIndex = urlStr.lastIndexOf("/") + 1;
String parentPath = endIndex > 0 ? urlStr.substring(0, endIndex) : urlStr;
String fqResource = resource.startsWith("//") ? url.getProtocol() + ":" + resource :
resource.startsWith("http") ? resource
: resource.startsWith("/") ? getBaseUrl(url) + resource : parentPath + resource;
if (fqResource.contains("?")) {
fqResource = fqResource.substring(0, fqResource.indexOf("?"));
}
resources.add(fqResource);
}
return resources;
}
The regular expression: looks for well formed strings ending in css/js/png/gif/jpg
The method: retrieves all matching strings from the given text (aka 'http response'), tries to build a fully qualified URL, and returns a Set of the data.
I've uploaded a full example here (with sample output). Have fun!
You can use JSoup a HTML & XML parser.
Here is an example how to do it,
String responseData = ""; // HTML data
Document doc = Jsoup.parse(responseData);
Elements images = doc.select("img");
// Elements pngImages = doc.select("img[src$=.png]");
// To parse specific image format in this case png
for(Element image : images){
// Do what ever you wanted to do
}
Here is related official documentation.
Creating an android app to receive data in json format from web server
in my app I should have url as string and use it to fetch data like below
private static final String my_url = "http://example.com/folder/showJsonData.php";
jsonObjectRequest = new JsonObjectRequest(Request.Method.POST, my_url ,new Response.Listener<JSONObject>() {
.
.
.
}
as you see my_url is not complicated or encoded so everyone can access it after decompiling apk.
the question is that how can I make it a little more complicated
please explain it with example.
Thanks
URL encoding is done in the same way on android as in Java SE;
try {
String url = "http://www.example.com/?id=123&art=abc";
String encodedurl = URLEncoder.encode(url,"UTF-8");
Log.d("TEST", encodedurl);
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
To use the admin API, the default vhost "/" must be encoded as %2F, per the documentation: 2nd paragraph, here.
I am stuck getting the java.net.URI class - as used by most any http-requestish thing (e.g., org.apache.http.client.methods.HttpRequestBase) - to emit a string of that form.
I.e., I would like new URI("http", null, "localhost", 8080, "/api/exchanges/%2F", "", null).toASCIIString() to be http://localhost:8080/api/exchanges/%2F? but it's not, instead it is http://localhost:8080/api/exchanges/%252F?.
And if the path is /api/exchanges// (double slash, where the second slash is supposed to be the default vhost, which is wrong, but never mind) then the result is http://localhost:8800/api/exchanges//? (which doesn't work with the RabbitMQ admin service, which thinks it is http://localhost:8800/api/exchanges/ (one slash), which then returns all exchanges on all vhosts.
So, what's the secret?
(By the way, this question is not a dupe: it is all about starting from a File, which really should have specialized knowledge of "/" in paths. I'm here just talking about plain-Jane URIs.)
FYI, wall-of-code TestNG tests that show various permutations - these tests all pass which shows that I can't get %2F out this way:
package com.bakins_bits;
import static org.assertj.core.api.Assertions.assertThat;
import java.io.UnsupportedEncodingException;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
import org.testng.annotations.Test;
public class TestSingleSlashURIPaths
{
#Test(enabled = true)
public void does_URI_or_URL_mangle_single_slash_paths_example_1()
throws MalformedURLException, UnsupportedEncodingException, URISyntaxException {
// ARRANGE
String sut = "http://localhost:8800/api/exchanges//";
// ACT
URL url = new URL(sut);
String path = "/api/exchanges//";
URI uri = new URI(url.getProtocol(), url.getUserInfo(), url.getHost(), url.getPort(), path, "", url.getRef());
String result = uri.toURL().toString();
// ASSERT
assertThat(url.toString()).isEqualTo("http://localhost:8800/api/exchanges//");
assertThat(uri.getPath()).isEqualTo("/api/exchanges//");
assertThat(uri.toString()).isEqualTo("http://localhost:8800/api/exchanges//?");
assertThat(uri.toASCIIString()).isEqualTo("http://localhost:8800/api/exchanges//?");
assertThat(result).isEqualTo("http://localhost:8800/api/exchanges//?");
}
#Test(enabled = true)
public void does_URI_or_URL_mangle_single_slash_paths_example_2()
throws MalformedURLException, UnsupportedEncodingException, URISyntaxException {
// ARRANGE
String sut = "http://localhost:8800/api/exchanges/%2F";
// ACT
URL url = new URL(sut);
String path = "/api/exchanges/%2F";
URI uri = new URI(url.getProtocol(), url.getUserInfo(), url.getHost(), url.getPort(), path, "", url.getRef());
String result = uri.toURL().toString();
// ASSERT
assertThat(url.toString()).isEqualTo("http://localhost:8800/api/exchanges/%2F");
assertThat(uri.getPath()).isEqualTo("/api/exchanges/%2F");
assertThat(uri.toString()).isEqualTo("http://localhost:8800/api/exchanges/%252F?");
assertThat(uri.toASCIIString()).isEqualTo("http://localhost:8800/api/exchanges/%252F?");
assertThat(result).isEqualTo("http://localhost:8800/api/exchanges/%252F?");
}
#Test(enabled = true)
public void does_URI_or_URL_mangle_single_slash_paths_example_3()
throws MalformedURLException, UnsupportedEncodingException, URISyntaxException {
// ARRANGE
String sut = "http://localhost:8800/api/exchanges/%252F";
// ACT
URL url = new URL(sut);
String path = "/api/exchanges/%252F"; // try pre-encoding the '%'
URI uri = new URI(url.getProtocol(), url.getUserInfo(), url.getHost(), url.getPort(), path, "", url.getRef());
String result = uri.toURL().toString();
// ASSERT
assertThat(url.toString()).isEqualTo("http://localhost:8800/api/exchanges/%252F");
assertThat(uri.getPath()).isEqualTo("/api/exchanges/%252F");
assertThat(uri.toString()).isEqualTo("http://localhost:8800/api/exchanges/%25252F?");
assertThat(uri.toASCIIString()).isEqualTo("http://localhost:8800/api/exchanges/%25252F?");
assertThat(result).isEqualTo("http://localhost:8800/api/exchanges/%25252F?");
}
}
Fascinating. #dave_thompson_085 and #Suboptimal gave the answer in comments above (not sure how to given them rep credit since those aren't answers). (I was suffering a bit of tunnel vision and didn't try it myself.)
The problem is apparently in the constructor URI(String scheme, String userInfo, String host, int port, String path, String query, String fragment) because the constructor URI(String str) works.
Note though that even in that case (see test below) URI::getPath() will return not %2F you'd expect (because that was your input) but /.
Nevertheless I'm happy because it means the problem can be resolved in the client library I'm using (which is using the 7 argument constructor, not the 1 argument constructor). So I'll file a bug there. (Unfortunately, the method in that library that has this constructor call is private static inside a helper class where it is called (and put in a data structure which is consumed) in another static method so there's no easy/obvious runtime patch I can make (by, e.g., subclassing). I hate utility classes. I'll have to build my own copy of this library to fix it.)
Thanks!
#Test(enabled = true)
public void does_URI_or_URL_mangle_single_slash_paths_example_4()
throws MalformedURLException, UnsupportedEncodingException, URISyntaxException {
// ARRANGE
String sut = "http://localhost:8800/api/exchanges/%2F?";
// ACT
URI uri = new URI(sut);
String result = uri.toURL().toString();
// ASSERT
assertThat(uri.getPath()).isEqualTo("/api/exchanges//"); // <== this seems wrong!
assertThat(uri.toString()).isEqualTo("http://localhost:8800/api/exchanges/%2F?");
assertThat(uri.toASCIIString()).isEqualTo("http://localhost:8800/api/exchanges/%2F?");
}
Good afternoon all!
I use PayPal REST API java sdk and I want to have different configurations for different environments of my application. Here is how I'm trying to do so:
private static boolean IS_PRODUCTION = false;
private static String PAYPAL_ACCESS_TOKEN;
private static void initPayPal() {
InputStream is = null;
try {
is = ApplicationConfig.class.getResourceAsStream(
IS_PRODUCTION? "/my_paypal_sdk_config.properties" : "/my_paypal_sdk_config_test.properties");
PayPalResource.initConfig(is);
String clientID = ConfigManager.getInstance().getConfigurationMap().get("clientID");
String clientSecret = ConfigManager.getInstance().getConfigurationMap().get("clientSecret");
PAYPAL_ACCESS_TOKEN = new OAuthTokenCredential(clientID, clientSecret).getAccessToken();
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
IOUtils.closeQuietly(is);
}
}
and while trying to get the clientID I have
java.io.IOException: Resource 'sdk_config.properties' could not be found
Strange behavior - I thought I've just configured the sdk to use my own properties file.
Please advice how could I set up those settings properly!
So here is the solution I found:
Create an empty sdk_config.properties file in default location
Load your own properties:
private static void initPayPal() {
InputStream is = null;
try {
is = ApplicationConfig.class.getResourceAsStream(
IS_PRODUCTION ? "/my_paypal_sdk_config.properties" : "/my_paypal_sdk_config_test.properties");
Properties props = new Properties();
props.load(is);
PayPalResource.initConfig(props);
ConfigManager.getInstance().load(props);
String clientID = ConfigManager.getInstance().getConfigurationMap().get("clientID");
String clientSecret = ConfigManager.getInstance().getConfigurationMap().get("clientSecret");
PAYPAL_ACCESS_TOKEN = new OAuthTokenCredential(clientID, clientSecret).getAccessToken();
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
IOUtils.closeQuietly(is);
}
}
We have made some good improvements to the PayPal Java SDK on integration steps. We are removing the need for sdk_config.properties file as they do not work as well, specially for multi-configuration settings.
Now, all you do is create an APIContext instance with clientId, clientSecret, and mode. You pass that context object for any API operation from there on.
Here is how the code would look like for different configurations:
APIContext defaultContext = new APIContext(clientId1, clientSecret1, "sandbox");
APIContext sandboxContext = new APIContext(clientId2, clientSecret2, "sandbox");
APIContext someOtherContext = new APIContext(clientId3, clientSecret3, "live");
APIContext liveContext = new APIContext(clientId, clientSecret, "live");
// Now pass any of the above context in these calls, and it would use those configurations.
Payment payment = new Payment();
// Fill in all the details.
payment.create(defaultContext);
// Replace that defaultContext with any of those other contexts.
Here is the wiki page explaining that: https://github.com/paypal/PayPal-Java-SDK/wiki/Making-First-Call
I had the same error with SDK 0.11 version. I use my own properties file, but code still looked for "sdk_config.properties". I put it into root in my CLASSPATH, but still got the same error. Then I made obvious and horrible solution: put empty "sdk_config.properties" into "rest-api-sdk-0.11.0.jar" library. This street magic solved my problem.