In java, I'd like to replace the Host part of an url with a new Host, where both the host and url are supplied as a string.
This should take into account the fact that the host could have a port in it, as defined in the RFC
So for example, given the following inputs
url: http://localhost/me/out?it=5
host: myserver:20000
I should get the following output from a function that did this correctly
http://myserver:20000/me/out?it=5
Does anyone know of any libraries or routines that do Host replacement in an url correctly?
EDIT: For my use case, I want my host replacement to match what a java servlet would respond with. I tried this out by running a local java web server, and then tested it using curl -H 'Host:superduper.com:80' 'http://localhost:8000/testurl' and having that endpoint simply return the url from request.getRequestURL().toString(), where request is a HttpServletRequest. It returned http://superduper.com/testurl, so it removed the default port for http, so that's what I'm striving for as well.
The Spring Framework provides the UriComponentsBuilder. You can use it like this:
import org.springframework.web.util.UriComponentsBuilder;
String initialUri = "http://localhost/me/out?it=5";
UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(initialUri);
String modifiedUri = builder.host("myserver").port("20000").toUriString();
System.out.println(modifiedUri);
// ==> http://myserver:20000/me/out?it=5
Here you need to provide hostname and port in separate calls to get right encoding.
You were right to use java.net.URI. The host and port (and user/password, if they exist) are collectively known as the authority component of the URI:
public static String replaceHostInUrl(String originalURL,
String newAuthority)
throws URISyntaxException {
URI uri = new URI(originalURL);
uri = new URI(uri.getScheme().toLowerCase(Locale.US), newAuthority,
uri.getPath(), uri.getQuery(), uri.getFragment());
return uri.toString();
}
(A URI’s scheme is required to be lowercase, so while the above code can be said not to perfectly preserve all of the original URL’s non-authority parts, an uppercase scheme was never actually legal in the first place. And, of course, it won’t affect the functionality of the URL connections.)
Note that some of your tests are in error. For instance:
assertEquals("https://super/me/out?it=5", replaceHostInUrl("https://www.test.com:4300/me/out?it=5","super:443"));
assertEquals("http://super/me/out?it=5", replaceHostInUrl("http://www.test.com:4300/me/out?it=5","super:80"));
Although https://super/me/out?it=5 is functionally identical to https://super:443/me/out?it=5 (since the default port for https is 443), if you specify an explicit port in a URI, then the URI has a port specified in its authority and that’s how it should stay.
Update:
If you want an explicit but unnecessary port number to be stripped, you can use URL.getDefaultPort() to check for it:
public static String replaceHostInUrl(String originalURL,
String newAuthority)
throws URISyntaxException,
MalformedURLException {
URI uri = new URI(originalURL);
uri = new URI(uri.getScheme().toLowerCase(Locale.US), newAuthority,
uri.getPath(), uri.getQuery(), uri.getFragment());
int port = uri.getPort();
if (port > 0 && port == uri.toURL().getDefaultPort()) {
uri = new URI(uri.getScheme(), uri.getUserInfo(),
uri.getHost(), -1, uri.getPath(),
uri.getQuery(), uri.getFragment());
}
return uri.toString();
}
I quickly tried using java.net.URI, javax.ws.rs.core.UriBuilder, and org.apache.http.client.utils.URIBuilder, and none of them seemed to get the idea of a host header possibly including a port, so they all needed some extra logic from what I could see to make it happen correctly, without the port being "doubled up" at times, and not replaced correctly at other times.
Since java.net.URL doesnt require any extra libs, I used it. I do know that if I was using URL.equals somewhere, that could be a problem as it does DNS lookups possibly, but I'm not so I think it's good, as this covers my use cases, as displayed by the pseudo unit test.
I put together this way of doing it, which you can test it out online here at repl.it !
import java.net.URL;
import java.net.MalformedURLException;
class Main
{
public static void main(String[] args)
{
testReplaceHostInUrl();
}
public static void testReplaceHostInUrl()
{
assertEquals("http://myserver:20000/me/out?it=5", replaceHostInUrl("http://localhost/me/out?it=5","myserver:20000"));
assertEquals("http://myserver:20000/me/out?it=5", replaceHostInUrl("http://localhost:19000/me/out?it=5","myserver:20000"));
assertEquals("http://super/me/out?it=5", replaceHostInUrl("http://localhost:19000/me/out?it=5","super"));
assertEquals("http://super/me/out?it=5", replaceHostInUrl("http://www.test.com/me/out?it=5","super"));
assertEquals("https://myserver:20000/me/out?it=5", replaceHostInUrl("https://localhost/me/out?it=5","myserver:20000"));
assertEquals("https://myserver:20000/me/out?it=5", replaceHostInUrl("https://localhost:19000/me/out?it=5","myserver:20000"));
assertEquals("https://super/me/out?it=5", replaceHostInUrl("https://www.test.com/me/out?it=5","super"));
assertEquals("https://super/me/out?it=5", replaceHostInUrl("https://www.test.com:4300/me/out?it=5","super"));
assertEquals("https://super/me/out?it=5", replaceHostInUrl("https://www.test.com:4300/me/out?it=5","super:443"));
assertEquals("http://super/me/out?it=5", replaceHostInUrl("http://www.test.com:4300/me/out?it=5","super:80"));
assertEquals("http://super:8080/me/out?it=5", replaceHostInUrl("http://www.test.com:80/me/out?it=5","super:8080"));
assertEquals("http://super/me/out?it=5&test=5", replaceHostInUrl("http://www.test.com:80/me/out?it=5&test=5","super:80"));
assertEquals("https://super:80/me/out?it=5&test=5", replaceHostInUrl("https://www.test.com:80/me/out?it=5&test=5","super:80"));
assertEquals("https://super/me/out?it=5&test=5", replaceHostInUrl("https://www.test.com:80/me/out?it=5&test=5","super:443"));
assertEquals("http://super:443/me/out?it=5&test=5", replaceHostInUrl("http://www.test.com:443/me/out?it=5&test=5","super:443"));
assertEquals("http://super:443/me/out?it=5&test=5", replaceHostInUrl("HTTP://www.test.com:443/me/out?it=5&test=5","super:443"));
assertEquals("http://SUPERDUPER:443/ME/OUT?IT=5&TEST=5", replaceHostInUrl("HTTP://WWW.TEST.COM:443/ME/OUT?IT=5&TEST=5","SUPERDUPER:443"));
assertEquals("https://SUPERDUPER:23/ME/OUT?IT=5&TEST=5", replaceHostInUrl("HTTPS://WWW.TEST.COM:22/ME/OUT?IT=5&TEST=5","SUPERDUPER:23"));
assertEquals(null, replaceHostInUrl(null, null));
}
public static String replaceHostInUrl(String url, String newHost)
{
if (url == null || newHost == null)
{
return url;
}
try
{
URL originalURL = new URL(url);
boolean hostHasPort = newHost.indexOf(":") != -1;
int newPort = originalURL.getPort();
if (hostHasPort)
{
URL hostURL = new URL("http://" + newHost);
newHost = hostURL.getHost();
newPort = hostURL.getPort();
}
else
{
newPort = -1;
}
// Use implicit port if it's a default port
boolean isHttps = originalURL.getProtocol().equals("https");
boolean useDefaultPort = (newPort == 443 && isHttps) || (newPort == 80 && !isHttps);
newPort = useDefaultPort ? -1 : newPort;
URL newURL = new URL(originalURL.getProtocol(), newHost, newPort, originalURL.getFile());
String result = newURL.toString();
return result;
}
catch (MalformedURLException e)
{
throw new RuntimeException("Couldnt replace host in url, originalUrl=" + url + ", newHost=" + newHost);
}
}
public static void assertEquals(String expected, String actual)
{
if (expected == null && actual == null)
{
System.out.println("TEST PASSED, expected:" + expected + ", actual:" + actual);
return;
}
if (! expected.equals(actual))
throw new RuntimeException("Not equal! expected:" + expected + ", actual:" + actual);
System.out.println("TEST PASSED, expected:" + expected + ", actual:" + actual);
}
}
I realize this is a pretty old question; but posting a simpler solution in case someone else needs it.
String newUrl = new URIBuilder(URI.create(originalURL)).setHost(newHost).build().toString();
I've added a method to do this in the RawHTTP library, so you can simply do this:
URI uri = RawHttp.replaceHost(oldUri, "new-host");
Added in this commit: https://github.com/renatoathaydes/rawhttp/commit/cbe439f2511f7afcb89b5a0338ed9348517b9163#diff-ff0fec3bc023897ae857b07cc3522366
Feeback welcome, will release it soon.
Or using some regex magic:
public static String replaceHostInUrl(String url, String newHost) {
if (url == null || newHost == null) {
return null;
}
String s = url.replaceFirst("(?i)(?<=(https?)://)(www.)?\\w*(.com)?(:\\d*)?", newHost);
if (s.contains("http://")) {
s = s.replaceFirst(":80(?=/)", "");
} else if (s.contains("https://")) {
s = s.replaceFirst(":443(?=/)", "");
}
Matcher m = Pattern.compile("HTTPS?").matcher(s);
if (m.find()) {
s = s.replaceFirst(m.group(), m.group().toLowerCase());
}
return s;
}
Related
Untested code (just thinking out loud), but I'm thinking there must be a more elegant way to do this.
So there's three ways to set a variable:
just assign it
read it from a properties file (might not be there)
read it from the command line (could be more than one argument or none)
Higher number takes precedence. How would I approach this?
public static final String APP_DOWNLOAD_PATH;
[...]
// If download path is not defined in config.properties, set it to the app dir.
String download = properties.getProperty("download", System.getProperty("user.dir"));
// Override if download path is set via command line.
String override = null;
try {
override = System.getProperty("download");
} catch (NullPointerException | IllegalArgumentException ok) {
// property is either not found or empty.
}
String APP_DOWNLOAD_PATH = (override == null || override.isEmpty()) ? download : override;
E: Added restrictions.
The straight forward approach is the best in my opinion. For example:
public static final String APP_DOWNLOAD_PATH = "foo";
public static void main(String... args) {
String dwnPath = null;
if(args.length > 0) {
dwnPath = args[0];
} else if(System.getProperty("download") != null) { // don't need try-catch, "download" is not null and not empty ("")
// not DRY at all
dwnPath = System.getProperty("download");
} else {
dwnPath = APP_DOWNLOAD_PATH;
}
// rest of program
}
The above could also be extracted to it's own function leading to a one liner:
String dwnPath = getDownloadPath(args, System.getProperty("download"), APP_DOWNLOAD_PATH);
The extracted function can also be more elegant than the code above:
/** Documentation */
public static String getDownloadPath(String[] args, String property, String default) {
return (args.length > 0) ? args[0] : (property != null) ? property : default;
}
If it is ok to use a external library, then this apache commons method could be useful -
StringUtils.firstNonEmpty(System.getProperty("download"),
properties.getProperty("download", System.getProperty("user.dir")),
"some value");
I want to pass a long series of request parameters (over 2000 characters in total) from one .jsp to another (via a URL), and make it seem to the receiving HTTPServletRequest as if it received the request parameters normally.
I cannot simply pass the URL normally as IE11 is truncating the URL at about 2000 characters (see What is the maximum length of a URL in different browsers?) so I need to have some kind of workaround.
It is trivial to save the url in the ClientSession with a key in one .jsp
public String addValue(String aString) {
String key=""+UUID.randomUUID();
mapValues.put(key, aString);
return key;
}
and then retrieve it in the other .jsp
public String getValue(String key) {
return mapValues.get(key);
}
However the other .jsp needs a HTTPServletRequest and not a string
I.e. I need to be able to do
public MyPosition(HttpServletRequest request) {
this.id= (String)request.getParameter("ID");
Is there anyway of doing this by converting the retrieved url to a HTTPServletRequest?
I know that I could rewrite MyPosition to take a string and extract the data from there directly, but I would much rather not touch the very lengthy, legacy code.
If I could do setParameter on the request, then this would be a solution. But such an option is not available (see HttpServletRequest - SetParameter)
The only way to modify an HttpServletRequest is to wrap it.
It sounds like you want to make a standard POST request instead of what sounds like a GET request.
I tried both #dimplex's and #Deadron's solutions, which I think should both work, but didn't manage to implement either in the short time frame I had available.
I ended up replacing the HTTPServletRequest request parameter in the MyPosition function with String urlKey and adding the following line inside the function
RequestStr request=new RequestStr(cSession,urlKey);
now my existing code did not need to be changed at all, and request.getParameter("paramName") would call my function below.
public class RequestStr {
String url = "";
public RequestStr(ClientSession cSession, String urlKey) {
super();
this.url = cSession.getValue(urlKey);
}
public String getParameter(String aParam) {
int i = url.indexOf("?" + aParam + "=");
if (i == -1) {
i = url.indexOf("&" + aParam + "=");
}
if (i == -1) {
return null;
} else {
int j = url.indexOf("&", i + 1);
if (j == -1) {
return url.substring(i + aParam.length() + 2);
} else {
return url.substring(i + aParam.length() + 2, j);
}
}
}
}
So all I needed to do was to save the very long URL in the session in a map with key urlKey and then in the request just passed this urlKey, and then I could retrieve the long URL via the urlKey and then decode it via my RequestStr class
In a class (Java8), I have a String representing an HTTP URL, e.g. String str1="http://www.foo.com/bar", and another string containing a request URI e.g. str2="/bar/wonky/wonky.html".
What is the fastest way in terms of code execution to determine if str2 is within the context of str1 (e.g. the context is /bar) and then construct the complete url String result = "http://www.foo.com/bar/wonky/wonky.html"?
Well I don't know if there is a faster way to just use String.indexOf(). Here is an approach that I think covers the example you gave (demo):
public static boolean overlap(String a, String b_context) {
//Assume the a URL starts with http:// or https://, the next / is the start of the a_context
int root_index = a.indexOf("/", 8);
String a_context = a.substring(root_index);
String a_host = a.substring(0, root_index);
return b_context.startsWith(a_context);
}
Here is a function that uses the same logic but to combine the two urls if they overlap or throw an exception if they don't
public static String combine(String a, String b_context) {
//Assume the a URL starts with http:// or https://, the next / is the start of the a_context
int root_index = a.indexOf("/", 8);
String a_context = a.substring(root_index);
String a_host = a.substring(0, root_index);
if(b_context.startsWith(a_context)) {
return a_host + b_context;
} else {
throw new RuntimeException("urls do not overlap");
}
}
And here is an example of using them
public static void main(String ... args) {
System.out.println(combine("http://google.com/search", "/search?query=Java+String+Combine"));
System.out.println(combine("http://google.com/search", "/mail?inbox=Larry+Page"));
}
So I have a tomcat server that I can successfully send a query such as... localhost:8080/AnarchyChatServer/AnarchyChatServlet?message=asdf&sendto=x&sendfrom=y
This will run the doGet() in my tomcat server, I have nothing in doPost()
Which will simply return asdf x y in the response BODY as HTML. Now in my C# I am trying to build the query string and send the message as such...
public string sendMessage(string message)
{
string url = "";
string response = "No server response.";
using (var wb = new WebClient())
{
UriBuilder baseUri = new UriBuilder("localhost:8080/AnarchyChatServer/AnarchyChatServlet");
message = message.Replace("&", "(AMPERSAND)");
message = message.Replace("?", "(QUESTIONMARK)");
string queryToAppend = "message=" + message;
if (baseUri.Query != null && baseUri.Query.Length > 1)
baseUri.Query = baseUri.Query.Substring(1) + "&" + queryToAppend;
else
baseUri.Query = queryToAppend;
url = baseUri.Uri.ToString();
queryToAppend = "sendto=" + sending;
if (baseUri.Query != null && baseUri.Query.Length > 1)
baseUri.Query = baseUri.Query.Substring(1) + "&" + queryToAppend;
else
baseUri.Query = queryToAppend;
url = baseUri.Uri.ToString();
queryToAppend = "sentfrom=" + account[0];
if (baseUri.Query != null && baseUri.Query.Length > 1)
baseUri.Query = baseUri.Query.Substring(1) + "&" + queryToAppend;
else
baseUri.Query = queryToAppend;
url = baseUri.Uri.ToString();
try
{
response = wb.DownloadString(baseUri.Uri);
}
catch (System.Net.WebException)
{
}
}
return response;
}
Ignore the hideous code reuse and whatnot. Basically the query building seems to work correct as if I output "url" it will return the proper url with the query string, but the issue is is that when I print out response after the call has been made to the server it just says "No server response" as it was initialized at the top of the C# code. Basically I'm wondering how I can query the server. Any insight would be helpful.
Your first problem is the lack of protocol in the Uri. Try this:
UriBuilder baseUri = new UriBuilder("http://localhost:8080/AnarchyChatServer/AnarchyChatServlet");
By the way... To debug these issues in the future, use:
catch (Exception ex)
{ <- breakpoint here
}
And examine ex. Or turn on breaking on thrown CLR exceptions in Visual Studio and debug the code (or just don't swallow the exception). Then you would see this:
Ignoring exceptions when debugging code is not a good idea in general...
i need to extract the top domain of an url and i got his http://publicsuffix.org/index.html
and the java implementation is in http://guava-libraries.googlecode.com and i could not find
any example to extract domain name
say example..
example.google.com
returns google.com
and bing.bing.bing.com
returns bing.com
can any one tell me how can i implement using this library with an example....
It looks to me like InternetDomainName.topPrivateDomain() does exactly what you want. Guava maintains a list of public suffixes (based on Mozilla's list at publicsuffix.org) that it uses to determine what the public suffix part of the host is... the top private domain is the public suffix plus its first child.
Here's a quick example:
public class Test {
public static void main(String[] args) throws URISyntaxException {
ImmutableList<String> urls = ImmutableList.of(
"http://example.google.com", "http://google.com",
"http://bing.bing.bing.com", "http://www.amazon.co.jp/");
for (String url : urls) {
System.out.println(url + " -> " + getTopPrivateDomain(url));
}
}
private static String getTopPrivateDomain(String url) throws URISyntaxException {
String host = new URI(url).getHost();
InternetDomainName domainName = InternetDomainName.from(host);
return domainName.topPrivateDomain().name();
}
}
Running this code prints:
http://example.google.com -> google.com
http://google.com -> google.com
http://bing.bing.bing.com -> bing.com
http://www.amazon.co.jp/ -> amazon.co.jp
I recently implemented a Public Suffix List API:
PublicSuffixList suffixList = new PublicSuffixListFactory().build();
assertEquals(
"google.com", suffixList.getRegistrableDomain("example.google.com"));
assertEquals(
"bing.com", suffixList.getRegistrableDomain("bing.bing.bing.com"));
assertEquals(
"amazon.co.jp", suffixList.getRegistrableDomain("www.amazon.co.jp"));
EDIT: Sorry I've been a little too fast. I didn't think of co.jp. co.uk, and so on. You will need to get a list of possible TLDs from somewhere. You could also take a look at http://commons.apache.org/validator/ to validate a TLD.
I think something like this should work: But maybe there exists some Java-Standard Function.
String url = "http://www.foobar.com/someFolder/index.html";
if (url.contains("://")) {
url = url.split("://")[1];
}
if (url.contains("/")) {
url = url.split("/")[0];
}
// You need to get your TLDs from somewhere...
List<String> magicListofTLD = getTLDsFromSomewhere();
int positionOfTLD = -1;
String usedTLD = null;
for (String tld : magicListofTLD) {
positionOfTLD = url.indexOf(tld);
if (positionOfTLD > 0) {
usedTLD = tld;
break;
}
}
if (positionOfTLD > 0) {
url = url.substring(0, positionOfTLD);
} else {
return;
}
String[] strings = url.split("\\.");
String foo = strings[strings.length - 1] + "." + usedTLD;
System.out.println(foo);