Small question regarding how to use java to retain only the base URL of a rest API please.
As input, many strings, all valid rest APIs.
For instance, the inputs:
https://some-host.com/v1/someapi
https://another-host.fr/api/compute
https://somewhere.host.com/public/api/v3/getsomething
I would like to only retain the bold part, basically, the https, the : and the slashes, the host name. Everything that comes after the host, I would like to discard it.
Currently, I am trying some kind of string.split based on the / character, then trying to re-concat the arrays, but I have a feeling I am not going to the right direction.
What would be the most appropriate way please?
Thank you.
You could just try java.net.URL or java.net.URI. They behave pretty similar.
For example:
URL url = new URL("http://example.com/a/b/c");
url.getProtocol();
url.getHost();
url.getPath();
or:
URI uri = new URI("http://example.com/a/b/c");
uri.getScheme();
uri.getHost();
uri.getPath();
There are several methods in both classes to extract lot's of different parts.
Related
I have a request as follows:
localhost:8000/location/:01
My code takes as input an HttpContext request.
func(HttpExchange r) {
String area_path = r.getRequestURI(); // Equals string "/location/"
}
How do I parse an HttpExchange correctly so I can pull out the "01" from this path and store it as a variable?
That (localhost:8000/location/:01) is not a valid URL or URI
A plain colon character is not legal in the path of a URL or URI. If you want to put a colon in the path, it must be percent-encoded. Furthermore, if this was a URL, it would start with a protocol; e.g. http:.
Now ... it is unclear what the HTTP stack you are using will do with a syntactically incorrect URL / URI, but it could simply be ignoring the colon and the characters after it.
Your code looks a bit odd too. You have tagged the question as [java]. But the code looks like JavaScript rather than Java; i.e. func is a Javascript keyword. But it also looks like you are using the (deprecated) com.sun.net.httpserver.HttpExchange Java class. I don't know what to make of that ...
My advice:
Don't use a colon character in the URL path.
If you must do it, then percent-encode the colon it.
If you cannot encode it properly, then you may need to find and use a different framework for your HTTP request handling. One that will accept and handle a malformed URL / URI in the way that you want. (Good luck finding one!)
Unfortunately, the details in your question are too sketchy to give more detailed advice.
I'm trying to URL percent encode my query param value while using URIBuilder to make an HTTP request to Bing API.
The url looks like
"https://api.datamarket.azure.com/Data.ashx/Bing/SearchWeb/v1/Web?$format=json&Query="
Where the Query String must be like
%27Test%20query%27
Using URLEncoder.encode(string, code), a string such as "test query", gets turned into "test+query" which is unacceptable.
URIUtil.encodeQuery()
returns "test%20query" which is almost acceptable, except it needs the %27 at the beginning and end.
When I try to just concatenate the string to make it valid as such, and then load this into URIBuilder, URIBuilder ends up with
https://api.datamarket.azure.com/Data.ashx/Bing/SearchWeb/v1/Web?%24format=json&Query=%2527test%2520query%2527
which is again unacceptable.
How can I remedy this issue? It's driving me insane.
Thanks for any help.
this is encoded URI.
$ is %24
bank is %20
if you want real URI, you need to decode .
I think decode method works well for you.
reference here:
http://hc.apache.org/httpclient-3.x/apidocs/org/apache/commons/httpclient/util/URIUtil.html
I have a url and I am trying to extract the text before the third slash. I am quite new to the concept in Android. I believe the Pattern class is used to achieve this. My problem is how to.
Take for instance: http://name.mywebsite.com/images.... I only require everything before images. Could anyone point me in the right direction?
You can use the uri method getHost in java.This example will help you,
URI uri = new URI("http://name.mywebsite.com/images");
String host = uri.getHost();
/* It returns name.mywebsite.com/*/
If I have a general url (not restricted to twitter or google) like this:
http://t.co/y4o14bI
is there an easy way to check if this url is shortened?
In the above case, I as a human can of course see that it was shortend, but is there an automatic and elegant way?
You could do a request to the URL, look if you get redirected and if so, assume it's a shortening service. For this you'd have to read the HTTP status codes.
On the other hand, you could whitelist some URL shortening services (t.co, bit.ly, and so on) and assume all links to those domains are shortened.
Drawback of the first method is that it isn't certain, some sites use redirects internally. The drawback of the second method is that you'd have to keep adding shortening services, although only a few are used widely.
One signal may be to request the URL and see if it results in a redirect to another domain. However, without a good definition of what "shortened" means, there is no generic way.
if you know all the domains that can be used to shorten your URLs, check if it is contained :
String[] domains = {"bit.ly", "t.co"...};
for(String domain : domains){
if(url.startsWith("http://" + domain)){
return true;
}
}
return false;
You can't: You will have to work by assumption.
Assumption:
Does www exist in url.
Does the server name end with a valid domain (e.g. com, edu, etc.) or does it has co.xx where xx is a valid country or organization code.
And you can add more assumption based on other url shortening links.
You can't.
You can only check if you list a couple of shorteners and check if the url starts with it.
You can also try checking whether the url is shorter than a given length (and contains path/query string), but some shorteners (tinyurl for example) may have longer urls than normal sites (aol.com)
I would prefer the list of known shorteners.
Here's what you could do in Java, groovy and the like.
Get the url you want to test;
Open the url with HttpURLConnection
Check the response code
if it is a valid code, 200 for example, the you can retrieve the url string in long form from the connection object if it was shortened or back in its original form if it wasn't.
We all love to see some code don't we. Its crude, but hey!
String addr = "http://t.co/y4o14bI";
URL url = new URL(addr);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
if (connection.getResponseCode() == 200) {
String longUrl = connection.url;
System.out.println(longUrl);
} else {
// You decide what you want to do here!
}
Actually, you as a human, can't. The only way you know that it's shortened is that it's a t.co domain. The y4o14bI could be an CMS identifier for all you know.
The best way would be to use a list of known shortener urls, and lookup against that.
And even then you would have problems. I use bit.ly with a personal domain, wtn.gd
So http://wtn.gd/random would also be a shortened URL.
You could maybe do a HTTP HEAD-request, and check for a 301/302 ?
If you request an URL like this, your HttpCLient should receive a HTTP Redirect instead of a HTML page. This wouldn't be an evidence but at least a hint.
Evaluate the URL and look for some clues:
the Path meets certain criteria
only has one step (i.e. not multiple slashes)
does not end with filename extensions
not longer than X characters (would need to evaluate various URL shortening services and adjust the upper bounds for the max token length)
HttpUrlConnection returns a redirect responseCode (i.e. 301, 302)
I would suggest using android.util.Patterns.WEB_URL
public static List<String> findUrls(String input) {
List<String> links = new ArrayList<>();
Matcher m = android.util.Patterns.WEB_URL.matcher(input);
while (m.find()) {
String url = m.group();
links.add(url);
}
return links;
}
Use the unshorten URL service like https://unshorten.me
They have an API as well https://unshorten.me/api
If the URL is shortened it will return the original URL.
If not you will get the same one back.
I have a very simple regex question. Suppose I have 2 conditions:
url =http://www.abc.com/cde/def
url =https://www.abc.com/sadfl/dsaf
How can I extract the baseUrl using regex?
Sample output:
http://www.abc.com
https://www.abc.com
Like this:
String baseUrl;
Pattern p = Pattern.compile("^(([a-zA-Z]+://)?[a-zA-Z0-9.-]+\\.[a-zA-Z]+(:\d+)?/");
Matcher m = p.matcher(str);
if (m.matches())
baseUrl = m.group(1);
However, you should use the URI class instead, like this:
URI uri = new URI(str);
A one liner without regexp:
String baseUrl = url.substring(0, url.indexOf('/', url.indexOf("//")+2));
/^(https?\:\/\/[^\/]+).*/$1/
This will capture ANYTHING that starts with http and $1 will contain everything from the beginning to the first / after the //
Except for write-and-throw-away scripts, you should always refrain from parsing complex syntaxes (e-mail addresses, urls, html pages, etc etc) using regexes.
believe me, you will get bitten eventually.
I'm pretty sure that there is a Java class that will allow path manipulations, but if it has to be a regex,
https?://[^/]+
would work. (s? included to also handle https:)
Looks like the simplest solution to your two specific examples would be the pattern:
[^/]_//[^/]+
i.e.: non-slash (0 or more times), two slashes, non-slash (0 or more times). You can be stricter than that if you wish, as the two existing answers are doing in different ways -- one will reject e.g. URLs starting with ftp:, the other will reject domains with underscores (but accept URLs without a leading protocol://, thereby being even broader than mine in that respect). This variety of answers (all correct wrt your scant specs;-) should suggest to you that your specs are too vague and should be tightened.
Here's a regex that should satisfy the problem as given.
https?://[^/]*
I'm assuming you're asking this partly to gain more knowledge of regexes. If, however, you're trying to pull the host from a URL, it's arguably much more correct to use Java's more robust parsing methods:
String urlStr = "https://www.abc.com/stuff";
URL url = new URL(urlStr);
String host = url.getHost();
String protocol = url.getProtocol();
URL baseUrl = new URL (protocol, host);
This is better, as it should catch more cases if your input URL isn't as strict as described above.
Old post.. thought I might as well put a simple answer to a simple regex Q:
(http|https):\/\/(www.)?(\w+)?\.(\w+)?