The problem I am trying to solve is figuring out all the external dependencies of an "Etl" java jar. The jar can call out to any number of external web services but there is no real insight to what they are calling out to.
We launch these jars from another java application(e.g. "java -jar jarname.jar"), so that's why I think we might be able to do something to intercept all http requests from the launch jar using a java agent or something like that.
In all, is it possible to intercept all outgoing tcp requests when launching a java jar? I say tcp because we are looking for all outgoing connections database, rabbitmq, http.
The usual strategy is to use tools like wireshark which can be configured to tell you all the details (including the data itself), in a nice user interface, and you can trivially filter on 'only all traffic caused by this specific process'.
You CAN also do this with an agent, but you're signing yourself up for a ton of code work for an end result that is going to be vastly inferior to what tools like wireshark can tell you.
If you want to go the agent route, find every entrypoint call that could possibly set up a TCP connection. Hopefully, that's just Socket, but in practice I bet there is more, and I don't think there's a list out there. Use ASM or bytebuddy or some other bytecode rewrite tool to 'load' every class being defined 'in transit', inspect it, and rewrite any calls to e.g. the Socket constructor to your own utility method that logs this, or wraps this so you can keep track of it. This will be quite a sizable project.
You don't really need an agent for this, though; if you want to know when connections happen, just, in your main, set up a SecurityManager - allow everything, but do some logging in the relevant methods. For example:
class MyNetworkInspectingManager extends SecurityManager {
#Override public void checkAccept(String host, int port) {
log.warn("accepting connection on " + host + ":" + port);
}
#Override public void checkConnect(String host, int port, Object context) {
log.warn("opening connection to " + host + ":" + port);
}
#Override public void checkConnect(String host, int port) {
log.warn("opening connection to " + host + ":" + port);
}
}
and possibly, also override checkListen, checkMulticast.
then, in your main:
public static void main(String[] args) throws Exception {
System.setSecurityManager(new MyNetworkInspectingManager());
}
Still far less useful than wireshark, but it's an hour or so to set this up, vs. the weeks+ to make the agent solution work out.
Related
The following initializes as socket with spring-integration on a dedicated port.
Goal is to give the application 2 minutes of time to run on that port, or fail is not successful. But the application start should not fail directly of that port is not free on startup!
#Configuration
#EnableIntegration
public class SocketConfiguration {
#Value("${app.port}")
private String port;
#Bean
public TcpConnectionFactoryFactoryBean tcpFactory() {
CompletableFuture.runAsync(() -> SocketUtils.findAvailableTcpPort(port, port)).get(2, TimeUnit.MINUTES);
}
}
Problem: SocketUtils is #Deprecated since Spring Framework 6.
What is a proper replacement for it?
The docs only state as an alternative:
Instead of using SocketUtils to find an available local port for a server, it is recommended that you rely on a server's ability to start on a random port
Well, that's not what I want, neither can I use if, because I'm historically bound to that specific port!
There is a org.springframework.test.util.TestSocketUtils instead. But as Gary said it is better to rely on operation system choice via 0 for port option. Although why would one select a random port in a production code...
To get a real selected port you can listen for a TcpConnectionServerListeningEvent which will bring for you that actual value.
So I am working on a software that will monitor(and may alter by acting as a Forrowder) all the DNS requests made by my router.
What I did?
So for first I wrote a Java code that can listens to a specific port and prints all the requests to the console[For now I just want to test with the requests].
The code is:
import java.net.*;
import java.io.*;
public class PortLogger{
public static void main(String[] args) {
LoggerServer loggerServer = new LoggerServer(53);
loggerServer.start();
}
}
class LoggerServer extends Thread{
private int port;
public LoggerServer(int port){
this.port = port;
}
#Override
public void run(){
try{
int id = 1;
ServerSocket server = new ServerSocket(port);
System.out.println("Server Listening at port " + port);
Socket client;
while(true){
client = server.accept();
ClientHandler clientHandler = new ClientHandler(client, id++);
clientHandler.start();
}
}catch(Exception ex){
System.out.println("Exception at Server : 1 :: EX = " + ex);
}
}
}
class ClientHandler extends Thread{
private Socket client;
private int id;
public ClientHandler(Socket client, int id){
this.client = client;
this.id = id;
}
#Override
public void run(){
try {
String data = "";
BufferedReader reader = new BufferedReader(new InputStreamReader(client.getInputStream()));
while(true){
data = reader.readLine();
if(data.length() > 0){
System.out.println("Client : " + id + " :: " + data);
}
}
}catch(Exception ex){
System.out.println("Exception at Client : " + id + " :: EX = " + ex);
}
}
}
The sole propose of this code for now is to Show me all the requests made to the server.
I know that I also have to change the DNS Server in my router for this.
So, for that I first tried by going to internet setup and put the local IP of my computer as DNS server.
But it was showing :
DNS IP and LAN IP must be on different networks!
But I found another way to do it.
It is as follows:
I went to the setup wizard of the router and the set the DNS Server to the same IP.
Surprisingly this worked!
[I have no idea whether this is a bug in the D-Link Firmware or not.
I have also added an exception to allow all request both inbound and outbound to port 53.
What is the problem?
So now the problem is that even after successfully changing the DNS to my servers. There seemed to be no requests at all to the console. I tried a lot but nothing.
I checked that the program was working fine by voluntarily sending request to it using telnet?
Now am I doing anything wrong or there is some bug with the router(its is a old one).
NOTE: The black lines on the images are just to hide my public IP address nothing special.
EDIT: I tried a few more times then found that websites were not opening when I changed the DNS in my router but still nothing in the console!
While it is difficult to give you a complete answer why your application doesn't work I can suggest some ways to investigate:
Port 53 is a privileged port. This means on Linux binding to that port requires root privileges and the application will throw an exception due to 'permission denied' if executed as a 'normal' user. As you are using Windows I don't know what it does if you try to bind as a 'normal' user, or you might be executing as an Admin user (or whatever the equivalent of 'root' is in Windows) and you don't know it. It might even just silently fail i.e. appear to bind when in fact it hasn't and no data is passed through you your application. As an aside, defaulting to 'root' as the default execution user in Linux is not the norm because it's insecure and most Linux distributions if not all do not allow this by default i.e. you can have this but you have to tell the distribution this is what you intend during installation. I'll let you come to your own conclusions what stance Windows takes for making users 'admin'...
In a scenario such as this if it were me I would immediately go to some networking tools to see what is happening. On Linux this is tcpdump or Wireshark. You can also get Wireshark for Windows as it's a GUI application. This will let you monitor and filter network traffic and so will be independent of your application. You can filter by source or destination address and/or port number.
I would leave the DNS setting alone in the router and change the DNS settings in one machine first, call it the test client, and set its DNS address to the machine where your application is running. Using tcpdump or Wireshark you can then make requests on your test_client e.g. browser requests and see the resulting network traffic.
You never mentioned if after changing your router's DNS settings all browser requests from clients fail. This is what I would expect to see if your router can no longer get a name resolution. However there maybe some DNS caching going on in your clients so you may appear to get successful DNS requests on your test_client. Again look at network traffic or use a Linux client which will provide you with much better networking tools.
I have 3 .jsp's. The first one asks the user for their username. Once the form is submitted it is taken to a 2nd jsp where a unique passcode is created for the user. How would I go about taking this passcode and passing it to a 3rd jsp using a socket?
You can use java.net.URL and java.net.URLConnection to fire and handle HTTP requests programmatically. They make use of sockets under the covers and this way you don't need to fiddle with low level details about the HTTP protocol. You can pass parameters as query string in the URL.
String url = "http://localhost:8080/context/3rd.jsp?passcode=" + URLEncoder.encode(passcode, "UTF-8");
InputStream input = new URL(url).openStream();
// ... (read it, it contains the response)
This way the passcode request parameter is available in the 3rd JSP by ${param.passcode} or request.getParameter("passcode") the usual way.
Better is however to just include that 3rd JSP in your 2nd JSP.
request.setAttribute("passcode", passcode);
request.getRequestDispatcher("3rd.jsp").include(request, response);
This way the passcode is available as request attribute in the 3rd JSP by ${passcode} or request.getAttribute("passcode") the usual way.
See also:
Using java.net.URLConnection to fire and handle HTTP requests
Unrelated to the concrete question, this is however a terribly nasty hack and the purpose of this is beyond me. There's somewhere a serious design flaw in your application. Most likely those JSPs are tight coupled with business logic which actually belongs in normal and reuseable Java classes like servlets and/or EJBs and/or JAX-WS/RS which you just import and call in your Java class the usual Java way. JSPs are meant to generate and send HTML, not to act as business services, let alone web services. See also How to avoid Java code in JSP files?
So, you want the username to be submitted from the first JSP to the second, by submitting a form to the second, right?
But, for interaction between the second and third, you want to avoid using the communication mechanisms behind the the JSP files and use your own, right?
Well, how you might implement doing this depends on where you're sending your communication from and to. For instance, are they on the same machine, or on different machines?
Generally speaking, you'll need a client-server type of relationship to be set up here. I imagine that you would want your third JSP to act as the server.
What the third JSP will do is will sit and wait for a client to try to communicate with it. But, before you can do that, you'll first need to bind a port to your application. Ports are allocated by the Operating System and are given to requesting processes.
When trying to implement this in Java, you might want to try something like the following:
int port_number = 1080;
ServerSocket server = new ServerSocket(port_number);
In the above example, the ServerSocket is already bound to the specified port 1080. It doesn't have to be 1080 - 1080 is just an example.
Next, you will want to listen and wait for a request to come in. You can implement this step in the following:
Socket request = null;
while((request = server.accept()) == null)
{}
This will cause the server socket to keep looping until it finally receives a request. When the request comes in, it will create a new Socket object to handle that request. So, you could come back to your loop later on and continue to wait and accept requests, while a child thread handles communication using your newly created request Socket.
But, for your project, I would guess that you don't need to communicate with more than one client at a time, so it's okay if we just simply stop listening once we receive a request, I suppose.
So, now onto the client application. Here, it's a little bit different from what we had with the server. First off, instead of listening in on the port and waiting for are request, the client's socket will actively try to connect to a remote host on their port. So, if there is no server listening in on that port, then the connection will fail.
So, two things will need to be know, those are:
What's the IP Address of the server?
What port is the server listening in on?
There are short-cuts to getting the connection using the Java Socket class, but I assume that you're going to test this out on the same machine, right? If so, then you will need two separate ports for both your client and server. That's because the OS won't allow two separate processes to share the same port. Once a process binds to the port, no other process is allowed to access it until that port releases it back to the OS.
So, to make the two separate JSP's communicate on the same physical machine, you'll need both a local port for your client, and you'll need the server's port number that it's listening in on.
So, let's try the following for the client application:
int local_port = 1079;
int remote_port = 1080;
InetSocketAddress localhost = new InetSocketAddress(local_port);
Socket client = new Socket(); //The client socket is not yet bound to any ports.
client.bind(localhost); //The client socket has just requested the specified port number from the OS and should be bound to it.
String remoteHostsName = "[put something here]";
InetSocketAddress remotehost = new InetSocketAddress(InetAddress.getByName(remoteHostsName), remote_port); //Performs a DSN lookup of the specified remote host and returns an IP address with the allocated port number
client.connect(remotehost); //Connection to the remote server is being made.
That should help you along your way.
A final note should be made here. You can't actually run these two applications using the same JVM. You'll need two separate processes for client and server applications to run.
I'm using Clojure, but I can read Java, so this isn't a Clojure specific question. This doesn't even seem to be working from Java.
I'm trying to implement a bit of a 'ping' function using isReachable. The code I'm using is this:
(.isReachable (java.net.InetAddress/getByName "www.microsoft.com") 5000)
Translated to Java by a good friend of mine:
public class NetTest {
public static void main (String[] args) throws Exception{
String host = "acidrayne.net";
InetAddress a = InetAddress.getByName(host);
System.out.println(a.isReachable(10000));
}
}
Both of these return false. I suppose I must be doin' it wrong, but Google research is telling me differently. I'm confuzzled!
Updated in response to comment that this is wrong:
Using Unix/Linux??
http://bordet.blogspot.com/2006/07/icmp-and-inetaddressisreachable.html says:
Linux/Unix, instead, supports an ICMP "ping" system call. So the implementation of java.net.InetAddress.isReachable() first tries to perform the "ping" system call**; if this fails, it falls back trying to open a TCP socket on [sic - to] port 7, as in Windows.
It turns out that in Linux/Unix the ping system call requires root privileges, so most of the times java.net.InetAddress.isReachable() will fail, because many Java programs are not run as root, and because the target address unlikely has the echo service up and running. Too bad.
The comment below from #EJP indicates the part of about the echo service is wrong, wrong wrong:
That's not correct. isReachable returns true if it gets a ConnectException trying to connect to port 7, as that proves that the host is up and able to send RST segments.
In cases like these, I use a packet sniffer like WireShark, tcpdump (WinDump on Windows) or snoop (Solaris) to confirm what is really happening on the wire.
The correct answer is not actually correct I think. Microsoft.com simply ignore ICMP requests, probably to avoid basic ping flood attacks. As for the second host I've no idea what the problem with the ping might be, but I'm using GNU/Linux and isReachable works just fine.
I need to create a thread pool to ping a range of ip addresses simultaneously using java language.
Please help me out.
Edited 1
If there are lots of thread created, then whether we have to call stop() method explicitly to stop the thread? or will it be taken care?
Here's an implementation of the pingig subtask. Creating a thread pool and threads with the referenced payload shouldn't be too complicated.
Edit
If you can modify the client code on these devices, I suggest a custom protocol instead of using the echo port - something like a heartbeat where you frequently send a small message to the client (on the standard or a different port) and expect an answer within a defined time.
Edit 2
For the thread basics, I really suggest looking at a Java tutorial. To start with: implement a class like public class PingThread extends Thread and place the code from the link inside a while(true) {...} loop in the run method. Use Thread.sleep() to add a wait time between to pings in the the same loop.
If you really need a ThreadGroup, override the constructor Thread(ThredGroup group, String name) so that a PingThread can be created in your specified group.
You may want to implement a switch to stop a PingThread (should be covered by almost every Java tutorial),
Java has no implementation of ICMP by default, so it's not possible to ping a host from Java with the standard library. Your other options are to look for a Java ICMP implementation (I don't know if one exists), or to call the 'ping' executable on your system from Java and parse the output.
Edit: Andreas_D's link indicates that InetAddress.isReachable() uses an ICMP echo request to ping a host, so that's how you can implement the ping.
You can take the code for the ReachableTest from that page and change the ReachableTest class into a Runnable which you can then run in its own thread or by using the executor service from java.util.concurrent:
public class ReachableTest implements Runnable {
private String host;
public ReachableTest(String host) {
this.host = host;
}
public void run() {
try {
InetAddress address = InetAddress.getByName(host);
System.out.println("Name: " + address.getHostName());
System.out.println("Addr: " + address.getHostAddress());
System.out.println("Reach: " + address.isReachable(3000));
}
catch (UnknownHostException e) {
System.err.println("Unable to lookup " + host);
}
catch (IOException e) {
System.err.println("Unable to reach " + host);
}
}
}