Say I have DemoServer project that only logs anything and I created another project say LogServer. So here's what I want to do; I will run both project simultaneously, as the DemoServer is running it will just keep on logging anything and in the LogServer project it should be able to access the logs that the DemoServer generates and save it to a new text file every 1 minute. Since this is the first time I'm doing this I find it hard to figure out how will the two project communicate? Can someone explain in detail how can i achieve this? Please also post links that might help me solve my problem.
I'd go with a local socket which is quite straightforward to implement in Java and can be later used also when DemoServer and LogServer are on different machines.
Developing it should be quite easy:
create your own LogMessage class
create a simple client/server infrastructure by using TCP sockets, take a look here
wrap the socket streams by using ObjectInputStream and ObjectOutputStream to be able to use serialization
just send logs by encapsulating them in messages from one process to the other and you are done
I would suggest you dig into oVirt source code (you can git clone the source) and see what we do with the log collector application.
There are many options to solve your problem -
A. Have your application log into a shared storage (i.e - nfs share) - one that both it and the log server can access. A cron job will run a periodic script that will copy the files that were last accessed, let's say - one hour ago, into a folder, that the the log server can access.
B. Use log4j and write your own Appender that will send the stuff you want log server to collect also to log server (via a file as suggested in section A, or via any other mean) - this way you will be able to contol on a category level basis what stuff the Log server can actually read
Related
Every time a ftp connection is established I want a java program to get triggered.
I am not sure how to check the FTP connections. Is there a way to monitor the port so that I can acknowledge when FTP gets the new connection?
I've researched a bit and found the FTPClient class which basically suggests I need to do my own ftpclient, but I want to know if there is another way.
Even an external program could be an option.
it would be running on the server side, basically the program will be reading the files that the client put on it.
Consider using a file watcher. It doesn't seem that you really care that the file gets there using FTP (in fact, in the future it might get there by some other means), but just that the file is present and needs to be processed.
See Watching a Directory for Changes for details on how to write one.
I have a program that does a similar task server-side with java. It checks if files exist in a folder and launches the rest of the code if so. Like this:
public static boolean needToConvert(){
//get list of companies
List<String> companies = getCompanyNames();
//check whether a file exists in folder of company
for(String company: companies){
if (new File(INPUT_LOCATION + company).list().length > 0){
return true;
}
}
return false;
}
This is a rather in-expensive task, so you could just set the server to "check"; say, every minute or so.
There are a variety of approaches:
Have the FTP service itself launch the java app (sounds expensive) or notify a running Java app / service. This will be the simplest approach ...
Use some kind of external monitoring; e.g.
A generic packet packet sniffing tool (like wireshark or tcpdump) might be able to trigger an external application. If not, you may be able to get it to log info to a file that you scan for connection events.
You could possible implement something using jpcap to do your own packet sniffing in Java
There is a Linux utility called tcpspy that logs TCP/IP connection details to syslog. You could configure it log to a specific file and monitor that file, alternatively, you could modify tcpspy to launch or notify a Java app directly.
I have researched a lot but could not find any thing proper on the topic and hence asking question here.
I want to build an application like dropbox using java sockets, with a social media website where the files we upload on our shared folder gets automatically downloaded or synced onto all of our friends system which we added in the social media website.
Till now what I have thought is I will have a server running and every time a client connects(logs in) I will start 2 handlers, 1 will be for uploads and 1 will be for download. The DOWNLOAD handler will check for a new files from all my friends(meaning they added new files in their shared directory) every 5 minutes and will sync it. and UPLOAD handler will upload the files on the server sending it as a byte array when the handler receives it from the client. Client sends the data to the server using directory watcher to track changes in the directory.
Now the question is starting 2 threads per client, is it feasible? I think it will slow down the server badly as I will imagine to have like 100 clients let say and it means 200 threads. Can you guys just point me in the right direction on as to what approach I should take, I read about NIO and IO and got confused. Also is there any particular library which can be helpful? I looked at Netty, apache mina but don't understand how they can be helpful.
Thanks in advance :)
I recommend you to take a look to this article about NIO: https://today.java.net/pub/a/today/2007/02/13/architecture-of-highly-scalable-nio-server.html. Also try to think about scalability, if your server send files... what's the speed of your hard drives? It's more important, I think, than number of threads; but keep an eye over thread lock.
Why you want to implement something that web makes so well? If I were you I try to think about a secure proxy better than all that stuff about dealing with bytes. Even If you want to transfer files in multiple parts, you may use multipart zip file and download each part programatically, and then rebuild the file. With this approach you can reuse your infrastructure for web and client; also you can benefit of high IO throughtput of modern web servers.
When you think you will have a large number of clients, using the standard Socket and ServerSocket isn't going to work. These require 2 threads per client, as you already pointed out. Eventually, this will eat up all your server resources. What you need is the java.nio package. In there you will find the SocketChannel and the ServerSocketChannel. Through these you can set up non-blocking socket communication. This type of communication is event based. Meaning you can have multiple clients using the same 2 threads on the server for reading and writing.
If you want to know more, check out my Socket programming tutorial. The third part goes into setting up a non-blocking variant and should give you everything you need to get started. If you still have questions, let me know, and I'll see if I can help you out further.
I have a program that I've created that is meant to poll an html internal page with different IPs that update and then will run a telnet session to those IPs to see if the device still has a connection... I'm attempting to challenge myself in creating something further with a dynamic webpage instead of my program spitting out console output...
My Issue:
I dont know what technologies / libraries Java has to execute such things
I want:
A Local Server, to upload a page LOCALLY only (no security is needed as this will be strictly intranet)
My program to implement: A database of sorts to save "logs" essentially that a certain IP / device has had successful connections
in the past....maybe stored to an external file is fine i presume (my
program currently has to re-poll everytime i run it.. i want some kind
of "remembering"..
Is it possible this can all be done in one file? so if i want my computer to run this as soon as it starts up... it will run... grab
its current state of the database of IPs... poll them (periodically)
and then persist and save and update the HTML page dynamically....
I hope i'm being as descriptive as possible... Its a bit of an abstract.. I really just want some introduction to different libraries ... a friend recommended stuff like MongoDB or something but I want to stay strictly to Java programming
I am trying to save a whole mailbox onto disk using the JavaMail API (in essence, perform a full backup_. I can successfully read all the relevant folders into memory and then sever the connection to my mail server. What I can't figure out is how to actually store the folders and/or individual messages if need be on my disk.
I've tried searching around and came up with an interesting link (below) but can't figure this out. Does anyone have any advice on where to get started? I appreciate it, thanks
http://www.oracle.com/technetwork/java/javamail/faq/index.html#serialize
quick note: I tried using a class that contained an ArrayList of type Folder(JavaMail class) and make that serializable, but I still had an exception thrown when I tried to write to disk. I've been working on this problem for about two days now and I could use some pointers. Thanks!
You found the FAQ, but you missed this entry:
How do I store mail messages on my local disk?
And, as the FAQ entry you found says, you can't just serialize the Message objects, let alone the Folder objects.
If all this seems too complicated for you, the simplest approach might be to get an IMAP server that you run on your local machine, then copy the messages from your remote IMAP server to your local IMAP server. The JavaMail FAQ has pointers to IMAP servers you can install locally, and a little web searching will turn up more. JavaMail also comes with a demo program (populate.java) for copying folders.
We are working in Java based web application.
We need to create a dashboard for the application admin to monitor the status of the application.
It will basically like whats happening in the application e.g. log of any activity we want to keep track of, any errors, reason for failue to upload some file. Also if there is any planned activity in the system admin should be able to see if it has taken place or not etc stuff.
What should be the database design? Shall I keep only one table for all activites that need to be tracked and shown on dashboard or a single table for all activity?
As well how do I log such activities?
Please help.
You may use log4j (or similar) with a syslog appender. Syslog has to run as a deamon on the machine and you have good dashboards for supervision/monitoring and so on. You may also connect a network of syslog deamons to gather information on multiple machines.
Another complementary solution is to use JMX and monitoring tools like Shinken, Hyperic, Nagios, etc.