I have a client that make requests to different servers. Sometimes this servers rejects requests from my IP so I need to change it (I have a few public IPs). I also need to change my IP to make geolocalizated request. I'm trying to make a balance server to redirect the client traffic through different server and to keep a log of IPs being rejected. This is what I have in mind:
There would be clients in different networks with different instances of the client. This instances request an output server to the balancer and then all the traffic of clients is redirected through this servers. Output servers could make a connection with the balancer with sockets to say something like "Ey, I'm here. You can use me!". Here I have a silly activity diagram (full of mistakes probably):
Is there a simplest way to do this? Maybe I'm reinventing the wheel. If is a good solution, is if possible to do this with Java/C#? How could I redirect the traffic?
I think you are reinventing the wheel a little, what your describing is just a load balancer in sticky session/sticky IP mode.
There's a few open source projects that will do what you are looking for. (Each word is a link there)
Personally I'd suggest the LVS Project
Related
Well I am new to this and I don't know how to do it, so my senior fellows please help!!!!
There is a situation described below:
An HTTP client is sending a request (Request can be of any type, not concerned regarding the request type) that directly hits a Loadbalancer. The Loadbalancer then redirects the traffic, based on the load of the traffic, towards a "Gateway" system running in two V440 Server, GW logic is written in Java, that actually logically routs this request towards another two server nodes which actually process the request.
Now the scene is something like that: there are several parallel connections are established with this Gateway from several HTTP clients. One connection per client. It has been observed that, while making connections to this GW, in case of some clients the CPU utilization is going 98-99%.
Client is creating one connection with the GW on particular port. Opens a socket connection:
ServerSocket _ss = new ServerSocket(_port);
Socket s = _ss.accept();
and then GW waits for the input to come from the client.
Now my question is:
Why this kind of situation is happening, as it seems all fine
for rest of the clients and there connections.
Only few clients who are creating connections with the GW is making the situation?
Is there anyway we can track this client's IP so that we can understand if this
has been occurred by same clients every time?
Is there any resolution for this?
Since it is not happening for all the clients, we are certainly not going to find an immediate answer for this. However, this is what my limited research on the question yields
Firstly, Question 2
Configure your F5 to capture the client's IP. Since it is HTTP, there are multiple ways of tracking the requests. One is to
sniff the header X-FORWARDED-FOR which will give the client's IP
Address
Or try adding this rule in your logging engine
when CLIENT_ACCEPTED { log local0. "clientIP:[IP::client_addr] accessed" }
If you also need other data such as resources you can use one of the
other events such as HTTP_REQUEST:
when HTTP_REQUEST {
log local0. "clientIP:[IP::client_addr] accessed [HTTP::host][HTTP::uri]" }
Refer link for above here
Secondly, Question 1
For this you need to look at your available traffic statistic mechanisms. I read this, this and this. Enable the statistics, monitor them live, test, mock requests and analyze the output. I do not know of any other options other than this, right now.
Another option, if you can modify your Java program is to include some sort of performance logging mechanism for each request. But this means there is a lot of development and that I do not recommend at all.
Thirdly, Question 3
This is primarily opinion based. As far as I think if you figure out the problem, you can resolve it.
I was wondering if someone could give me a clue. The situation is we have one computer that has a server application, the rest have a client application. Now my question is how would i keep track of the server ip address? What I want is have the clients automatically hunt the server application, regardless of the ip address and find it, because using a wireless network computers are using DHCP. So I was wondering is there anything that can be buried in a computer using Java with which other computers can identify the application. Thanks.
The "good" way is to use DNS (imho the ONLY solution in a productive environment), as #Peter Lawrey already mentioned, and i would prefer that solution.
You have two other possibilities:
Let the server broadcast his address on the network.
Let the clients scan the network.
Both this solutions are not ideal, because you create a lot of noise on the network, may be blocked by a firewall, etc. Therefore, i really suggest you to use DNS.
If you can not use the DNS solution i would go with a solution that lets the clients search for the server.
Take a look at this SO question, might get you started.
If they're on the same physical network, your server can broadcast its IP address via UDP to the broadcast address for that sub net.
You could even follow the uPnP standard.
Alternatively, you could host the location of the server from a known other location, like a web server. For example, call an action on the known web server to retrieve the IP address of the server. This allows you to utilize DNS to look up the first host, then go from there.
If you know the DNS-Name of the server you can use that name instead of the IP. Java takes care of the DNS lookup.
Otherwise, I guess the simplest solution is just to scan the IP range of your network and just try to connect to each IP and test if it works.
Other solutions would include broadcasts by the server and would be much more complicated and maybe overkill for a application hosted in a small network.
For the past few weeks, I have been scouring the internet, the minds of computer programmers, and just a few random people over the situation I am looking to overcome. Basically, what I am trying to do it write a AntiJoinBot "plugin" (if you will) for the popular game Minecraft. This would be like all others in respect that it blocks IPs based on if they are using a proxy or not, but this AntiJoinBot is running on a different VPS than the actual server.
This is the best graph I can make of the situation (it's not that good):
(non-minecraft server) Connection -> Proxy check -> Redirect to the
server -> Minecraft
The only problem is, I need to be able to redirect the IP and close the connection so that the player's real IP is the one that would connect to the server. If the connection is not able to be closed, it would cause real problems due to some of the plugins we are running.
If you have a solution or a better way to do this, please help me.
Redirection of connections along the lines that you want requires support from the (application) protocol. TCP/IP does not support it. AFAIK, SOCKS does not support it either. Unless the Minecraft application protocol (and by implication, Minecraft clients and servers) include support for redirection, you are out of luck.
(FWIW - that's how HTTP redirection works. HTTP has a "protocol element" that allows the server to tell the client to redirect, and where to redirect to. The client then resends the original request to a new address.)
But that doesn't mean that you can't deal with the pests. It just means that the redirection approach is not viable. Try a custom proxy or an IP filter / redirector instead.
You are trying to save the server's resources on the cost of increase Traffic.
I am not sure with the answer but may be by looking into the concept of LBS(Load Balancing Server) you may find the answer.
LBS is purely defined and controlled by us so you can manage the resources of two servers using one load balancing server.
The system I am developing potentially has a very large number of clients (lets say one million) that need to periodically update a central server with some information. Clients are written in Java.
The specific use-case is that the server backend needs to have an up to date mapping of IP address to clients. But the client IPs are dynamic and subject to (effectively random) change.
The solution I have in mind requires the clients to ping the server to update their IP. The period ideally should be once every minute, but even 1 ping/10 mins is acceptable.
My questions, in sequence:
1M pings per 1 min is over 10k/sec. So first off I want to know
the approaches can scale to handle such a load. This is to know the options available.
Assuming you have more than one solution in mind, which of these
would be the most economical? The cost effectiveness is critically important. I don't have my own data center or
static and fat end-point on the net, so the server application will
need to run on some sort of provider or ultimately on the cloud.
Notes:
I considered running the server from home using my own ISP provided connection, but I am neither sure of the performance issues, nor what my ISP will think about a constant stream of pings.
I can't see how the server can auto-discover these IP changes.
Erik, your problem is much simpler than it seems to have been made to sound.
This problem been around for a decade maybe two. No need to re-invent the wheel here.
Why Polling/Pinging is a Bad Idea
The dynamic IPs provided by ISPs can have a variable lease time, but will often be at least 24-72 hours. Pinging your server every 1-10m will be a horrible waist of resources potentially making over a 4,320 useless HTTP requests PER CLIENT in a 72 hour period. Each request will be say around 300 bytes * 4,320 wasted http requests equals 1.3mb wasted bandwidth multiplied by your target client count of 1 million clients, you are talking about a monthly wasted bandwidth of ~1.2 TB! And that's just the wasted bandwidth, not the other bandwidth you might need to run your app and provide useful info.
The clients need to be smarter than just pinging frequently. Rather they should be able to check if their IP address matches the DNS on startup, then only when the IP changes, send a notification to the server. This will cut down your bandwidth and server processing requirements by thousands of times.
What you are describing is Dynamic DNS
What you are talking about is "Dynamic DNS" (both a descriptive name for the technology and also the name of one company that provides a SaaS solution).
Dynamic DNS is quite simply a DNS server that allows you to very rapidly change the mapping between a name and an IP address. Normally this is useful for devices using an ISP which only provides dynamic IPs. Whenever the IP changes for the router/server on a dynamic IP it will inform the Dynamic DNS server of the change.
The defacto standard protocol for dynamic DNS is well documented. Start here: DNS Update API, I think the specifics you are looking for are here: DynDNS Perform Update. Most commercial implementations out there are very close to the same protocol due to the fact that router hardware usually has a built in DynDNS client which everyone wants to use.
Most routers (even cheap ones) already have Dynamic DNS clients built into them. (You can write your own soft client, but the router is likely the most efficient location for this as your clients are likely being NAT'd with a private IP - you can still do it but at a cost of more bandwidth for public IP discovery)
A quick google search for "dynamic DNS java client" brings up full source projects like this one: Java DynDNS client (untested, just illustrating the power of search)
Other Considerations for your System Architecture
Lets say the IP-client mapping thing gets resolved. You figured it all out and it works perfectly, you always knows the IP for each client. Would you then have a nice reliable system for transferring files to clients from mobile devices? I would say no.
Both mobiles and home computers can have multiple connection types, Wi-Fi, Cellular Data, maybe wired data. Each of these networks may have different security systems in place. So a connection from a cellular data mobile to a wifi laptop behind a home router is going to look very different than a wifi mobile device connecting to laptop on the same wifi network.
You may have physical router firewalls to contend with. Also home computers may have windows firewall enabled, maybe norton internet security, maybe symantec, maybe AVG, maybe zone alarm, etc... Do you know the firewall considerations for all these potential clients?
Maybe you could use SIP as protocol for that purpose ?
Probably the java SIP libs already solved your problem.
Nice app by the way.
I would suggest better tweak you java program to know the IP change and then only hit the web service.
You can do it like,
on your java program initiation extract the IP of machine and store
it in Global variable or better some property file.
Run a batch process/scheduler which will check your IP every 30sec/1 minute for change.Java Quartz Scheduler will come very handy for you.
Invoke the web service in case of a change of IP.
This way it reduces your server role and thus traffic and connections.
You could create your own protocol on top of UDP, for example XML based. Define 3 messages:
request - client requests a challenge from server
challenge - server replies with challenge (basically a random number)
response - client sends username and hashed password + challenge back to the server
It's lightweight and not too traffic-heavy. You can load-balance it to multiple servers at any layer or using load-balancer.
Any average PC could handle million such hits per minute, provided you do server-side in C/C++ (I don't know about java network performance)
Please have a look at how no-ip works. Your requirement is exactly same as what it does.
Do I have the use case right? A community of users all want to receive pictures from each other? You don't want to host the images on the server but broadcast them directly to all the users?
There are two questions here. The first question is "how to know if my own WAN IP address has changed."
If you are not NATed then:
InetAddress.getLocalHost()
will tell you your IP address.
If you are NATed, then using dynamic DNS and resolving your own host name will work.
The second question is something like "How to share pictures between hosts which come and go on the internet".
The possible solution space includes:
IP Multicast, probably with Forward Error Correction and Carouseling, e.g. FLUTE.
File Swarming - e.g. bittorrent.
A Publish/Subscribe message bus solution using Jabber, AMQP, JMS, STOMP or similar. Suitable implementations include RabbitMQ, ActiveMQ, etc. JMS Topics are a key concept here.
The solution should avoid the massive overheads of doing things at the IP level.
I created a game and I want to put it on online. I want to buy a website (I'll probably use goddaddy to buy a domain name and use them as the web host) to use as the server to handle game play. Because I would need a separate server for each game, I would need each game's server to exists on different ports. So this leads to my question, is is possible to access these ports on my future web server? (I wrote the program in Java, so I would assume that I would access the ports from the server side by choosing a port for a ServerSocket, and from the client side by using the IP address from the website and the chosen port for a Socket)
(note: also, I am aware that it may be easier to simply use one port and run the servers on different threads instead, but I am just curious to have my question answered)
thanks a lot,
Ian
Technically it is possible to use different ports, but I don't think that a webhoster like goddaddy will let you run a java process that binds to a special port.
If you mean that you are going to create your own TCP server you obviously can create as many instances of your server and configure them to listen to different ports. But it is year 2011 now. This solution was OK in early 90s.
I'd suggest you to use Restful API that works over HTTP. In this case you can forward calls to server side of each application using URL, e.g.
http://www.lan.com/foo/login?user=u123&password=123456 - log in into application foo
http://www.lan.com/bar/login?user=u123&password=123456 - log in into application bar
In this case you need only one server (the web server) that is listening to socket (port 80).
Your server side implementation could be done using various web techonlogis (php, java, asp.net etc) on your choice.
Yes, that should work. The security manager permits connections to a different port on the same IP address that the applet was loaded from.
You can run a Java server on whatever port you want. Each server will accept incoming requests on one port.
The correct way is simply run on one port and each connection will instantiate a new servlet instance (which happens to run in its own thread) that can then service that request. You usually don't need to run separate ports or worry about concurrency, especially if all the stuff that's shared between connections (e.g. multiple players in one game) is handled through database read/writes.
Your host (GoDaddy) will have to allow you use of those ports, but if they are providing proper hosting (not virtual hosting) and given you your own IP there's no reason why you shouldn't be able to.
Your solution may work theoritically, and I like AlexR's solution. But providers like godaddy doesnt let you run a java server, on ANY port. You will need to find out somebody who does. What I found is the cost goes up from $5/mo to about $20/mo, but you get a much better (read faster) machine. Good wishes, - MS.