I have been struggling with this for the entire day now, I hope somebody can help me with this.
My problem is fairly simple: I wish to transfer data (mostly simple commands) from one PC to another over the internet.
I have been able to achieve this using sockets in Java when both computers are connected to my home router. I then connected both computers to the internet using two different mobile phones and attempted to transmit the data again. I used the mobile phones as this provides a direct route to the internet and if I use my router I have to set up port forwarding, at least, that is how I understand it.
I think the problem lies in the method that I set up the client socket. I used:
Socket kkSocket = new Socket(ipAddress, 3333);
where ipAddress is the IP address of the computer running the server. I got the IP address by right-clicking on the connection, status, support. Is that the correct IP address to use or where can I obtain the address of the server? Also, is it possible to get a fixed name for my computer that I can use instead of entering the IP address, as this changes every time I connect to the internet using my mobile phone?
Alternatively, are there better methods to solving my problem such as using http, and if so, where can I find more information about this?
EDIT:
Would it be possible to have the server program running on a server on the internet somewhere. My original server would then be a client that send information to this server. This server would then pass this information to my original client and vice versa. That way, the IP address of my computer won't matter, as I only need to know the address of the server hosted somewhere on the web. Does this seem like a better solution? Where do I begin implementing such a solution?
Thanks!
When you connected to the server that serves StackOverflow, did you type in the IP address? It's 64.34.119.12, if that jogs your memory.
You probably didn't. You probably typed "stackoverflow.com". There's a huge, complex, clever, and in some ways, poorly implemented system called DNS that translates sensible and human-readable names into IP addresses.
One problem with DNS, though, is you need a "static IP", which is exactly what it sounds like: an IP address that doesn't change, which is exactly what you don't have.
So, what can you do?
You can buy a static IP account from your ISP (pretty expensive)
You can use some proxy out in the Internet (a machine that does have a static IP and is willing to bounce your packets back and forth -- I'm not aware of any service that does this for you; you could write one and put it up on Amazon Web Services or Google App Engine, both of which would be free at your level of usage, but they'd be slow, since every packet trying to cross your living room would have have to go via some data-center in Virginia).
You can keep doing what you're doing, looking in the net-configuration of your machine.
You could speed (3) up a little by having your server program look up its own IP address and print it out where you could see it and type it into the server by hand.
You can use DynDNS, as Sergey mentioned (this is the "right" solution, in that it's very general, it just might be a little complicated to set up)
You can use multi-casting.
Multi-casting is an interesting solution, and it may work for you. The idea is, when your server starts up, it announces to the net, "Here I am, I'm providing X server, here's my IP address, talk to me." The problem is, a multi-cast won't leave your living room. Obviously, if every multi-cast were spread to every computer on the Internet, the whole thing would collapse, so your router will ignore, and not route, multi-cast packets. That may or may not be a deal-breaker for you. EDIT Re-reading your question, I see it is a deal-breaker for you. I'd go with #5, but be aware there may be routing issues (address translations that prevent a server from knowing the address that other computers can find it at) or fire-wall issues (that is, your ISP may prevent your server from receiving incoming packets even if the address is correct).
using a direct socket connection with a port like 3333 is usually complicated because different network configurations.
firewalls will make a pleasure preventing the connection, or killing it from time to time.
maintaining a 2-way connection can be a nighmare. the SIP protocol is struggling with this kind of problems.
For a simple application, i suggest you look into the comet technology, where your clients can establish an http connection with a shared server. The server can then bridge commands between them.
html5 will also bring the websocket protocol to the table.
I got the IP address by right-clicking
on the connection, status, support.
Not sure about the "support" part, and I'm not on a Windows machine right now, but I think that the most easy and reliable way to figure out the IP address on Windows is to run "ipconfig" from the command line (Win+R, type "cmd", then "ipconfig" in the opened window). This, of course, should be done on the server side.
However, the problem is that depending on the ISP your IP address may be not within the Internet, but within a local ISP network (so-called NAT). In this case, you'll need to use some sort of black magic called TCP hole punching, which is very complicated and not guaranteed to work. You can figure out if your address is local or not by looking at it. For IPv4 local addresses are almost always like 10.x.x.x or 172.16-31.x.x, or 192.168.x.x. Don't know about IPv6.
You can also check your IP by visiting one of the special sites like www.whatismyip.com. If the address they tell you is different from the one you see by running "ipconfig" or looking at the connection properties, then you're almost certainly behind a NAT (or your ISP is using a transparent proxy, but that's rare).
If you are directly connected to Internet (no local addresses and NAT), then you should also check if you have any firewall software and either to configure it to allow connections to the port you use, or make sure it's in "ask the user" (and not "silently reject") mode, or just disable it completely (this may put your computer at risk, especially if there is no anti-virus software or the system isn't up-to-date).
Also, is it possible to get a fixed
name for my computer that I can use
instead of entering the IP address, as
this changes every time I connect to
the internet using my mobile phone?
Yes, it's possible. There is the thing called DynDNS, and there are DynDNS providers like DynDNS.com, where you can get a third-level domain name for free (like mycoolpc.dyndns.org). You'll have to install and configure some DynDNS client on your PC that will tell the DynDNS server its new IP each time each changed. I don't know anything about particular clients to use because I'm using the one built-in in my home router.
No need to write networking code for this, unless it really floats your boat. Take a look at SCP. http://amath.colorado.edu/computing/software/man/scp.html. There is a windows implementation where you can download putty (windows ssh client), and it is on most linux distributions. Alternatively, you could set up an FTP or SSH server on one or both of the machines.
"a fixed name for my computer that I can use instead of entering the IP address" would be a domain name, these are purchasable online for a few bucks.
Related
I made a networked game in Java using Sockets. It works great, except... only through my LAN. :/ What I want to be able to do is set it up so that I can start the server running on my home computer and send the client code, in an executable jar, to someone else, and have them be able to launch it and connect through the internet. But like... I have no idea how to set up my modem & router for that. :( I have a wireless Clear modem, for which the configuration page looks like this:
http://imageshack.us/photo/my-images/254/modem.jpg
And a Netgear router, whose page looks like this:
http://imageshack.us/photo/my-images/443/routerq.jpg
Right now, in the client and server runner classes, I pass my PC's private IP address along with my chosen port number to the ServerSocket and Socket objects. I hear that Sockets are able to be either UDP or TCP, depending on how you set them up, or something...? I don't do anything like that though, I just instantiate them and pass them the 2 values, and off they go...
In the code for the server:
ServerSocket sock = new ServerSocket();
sock.bind(new InetSocketAddress(IP, 9001));
Code for the client:
Socket sock = new Socket(IP, 9001);
So also, I would need to know how to change the code accordingly with the changes to my router/modem settings.
"IP" is a String representing my computer's private IP, 192.168.1.10, and the program only works if that is the string I pass to the Socket & ServerSocket. Like I said, I have a modem which is connected to a router which is connected to my computer. So, can someone please explain to me (in detail, because I'm a noob) what exactly I would have to do to configure both of them, AND the changes I would have to make to my code?
You'll have to manually port forward, or look at a solution like UPnP or NAT-PMP for automatic port forwarding. Unfortunately, NAT is a pain in the ass to configure and most routers don't properly support it. It's also not a simple task, as you have to support a whole range of different implementations and mechanisms.
It shouldn't be too difficult to set up your router to forward one port to your machine, though. See http://portforward.com/ for an awesome list of how to do it on pretty much any router.
It sounds like you're asking a number of questions here... I'm no network/socket programming specialist, but here's some ideas.
For distribution, you may wish to look into Java Web Start. It'll make it easy for your clients to obtain the app as well as automating updates.
If you're using the Socket constructor like that, the actual implementation will be a system default. You could override this by calling this method with a suitable SocketImplFactory implementation.
As for the IP address and port... Using a hard-coded value for the port should be okay. You'll need to decide on a port or at least some default anyway for clients to connect to. Even so, it might be useful to have the value read from some external configuration file. This'll make it easier should you ever decide to distribute the server app to let other people run servers. They might want to use a different port.
Having the server's IP address (yours) hard-coded in code is definitely a no-no, though. Clients should connect using either an IP address or host name that they need to provide or set in some configuration file. A host name would be required if you don't have a static IP address.
Unless this is something for you and some friends to enjoy, where you can always just initiate a game by providing IP and port in a chat session or something, you'd be better off finding some external hosting solution at a company that takes care of the DNS resolution and network setup for you. These days you'll find stuff like that pretty cheap if you don't have requirements like a Java EE server or database.
EDIT: thought of something else. TCP might be okay for your use-case, but if this is a game that requires minimal lag and quick input synchronization (like a shooter or fighting game) then it's not the best choice. In that case UDP would be better. It would induce the requirement of adding some sort of detection for desync or a mechanism that makes up for any dropped packets.
I am doing an android app allowing users to play online.
Currently, I use a TCP server: when two persons are connected, the server takes care of forwarding the packets between the two clients.
I would like to replace my server by a java servlet with google app engine. This new server will just be used to connect the two players.
It would work in that way:
Player A opens a server socket and then post to the server the connection details.
When a player B wants to play against A, he asks to the server the port number of A and he connects directly to A.
The problem is that I am not sure that it will work if player A is behind a NAT. When player A opens a server socket, that opens one port of its 192.168.x.y address, but does it ask to the box a port forwarding? I assume it doesn't...
So two questions:
Is it possible to make a direct connection TCP between two devices even when there is a NAT or a firewall (I don't know how firewalls work on Android...)
If it isn't possible, what is the best solution: Is it possible to make a TCP server to ensure the exchange of the messages with app engine?
Thank you by advance.
game
Creating direct TCP connection between users under different NAT is mostly possible. There are 4 types of NAT. FC, ARC, PRC, Symmetric. If one of player A or B has symmetric NAT then it is impossible to create TCP P2P connection. In this case you will have to use a server in the middle for exchanging data between two players.
For other types of NAT combinations it is very much possible but not guaranteed. The technique that is used to create TCP P2P connection is called TCP hole punching. Read this answer to know in details about this technique.
Also creating TCP P2P connection is not related to any platform.
First, the device itself is probably not going to be the main problem. If they are at home and using WiFi, you will probably have to deal with a cable modem/DSL modem, which typically includes a firewall. Also if they are at work (or a hotel, conference center, etc.), there may be a corporate firewall to deal with.
I believe most home cable/DSL modems support uPnP (Universal Plug and Play), which includes the Internet Gateway Device Protocol (IGD) designed to let devices determine the external IP address and set up port mappings. In general you can look up NAT traversal for ways to handle connections through a home modem/firewall. I will note that corporate firewalls are a different matter and many of these techniques won't work.
So probably I would recommend you be ready for at least the following four scenarios
Direct connection with nothing creating problems. You can test this by having the server do a test connection when the player first contacts the server. If this works, things are simple.
Home NAT device that understands uPnP. If you have a 10.x.x.x, 172.16.x.x-172.31.x.x, or 192.168.x.x number (typical home WiFi), then you can try to set up the NAT traversal and if that works you can send the appropriate information to your server. It probably would be worthwhile for the server to do a test connection just to be sure that things work.
If you have a firewall that you can't get around, then make a note on the server regarding player A, and when B tries to join A's game, look and see if B will accept connections, and if so then arrange for A to connect to B instead.
If none of the above work, then have A and B both connect to the server and have the server relay messages between A and B.
If you don't want to program all those possibilities, then option 4 is the one that is most likely to work, even if it does mean extra traffic going to/from your server. But note that for corporate networks, they may simply have a rule blocking unknown connections, and there may not be much you can do.
Edit: I was able to get a simple TCP server working on Android without anything special regarding Android itself, so removed a comment saying I didn't know about that.
I was wondering if someone could give me a clue. The situation is we have one computer that has a server application, the rest have a client application. Now my question is how would i keep track of the server ip address? What I want is have the clients automatically hunt the server application, regardless of the ip address and find it, because using a wireless network computers are using DHCP. So I was wondering is there anything that can be buried in a computer using Java with which other computers can identify the application. Thanks.
The "good" way is to use DNS (imho the ONLY solution in a productive environment), as #Peter Lawrey already mentioned, and i would prefer that solution.
You have two other possibilities:
Let the server broadcast his address on the network.
Let the clients scan the network.
Both this solutions are not ideal, because you create a lot of noise on the network, may be blocked by a firewall, etc. Therefore, i really suggest you to use DNS.
If you can not use the DNS solution i would go with a solution that lets the clients search for the server.
Take a look at this SO question, might get you started.
If they're on the same physical network, your server can broadcast its IP address via UDP to the broadcast address for that sub net.
You could even follow the uPnP standard.
Alternatively, you could host the location of the server from a known other location, like a web server. For example, call an action on the known web server to retrieve the IP address of the server. This allows you to utilize DNS to look up the first host, then go from there.
If you know the DNS-Name of the server you can use that name instead of the IP. Java takes care of the DNS lookup.
Otherwise, I guess the simplest solution is just to scan the IP range of your network and just try to connect to each IP and test if it works.
Other solutions would include broadcasts by the server and would be much more complicated and maybe overkill for a application hosted in a small network.
The system I am developing potentially has a very large number of clients (lets say one million) that need to periodically update a central server with some information. Clients are written in Java.
The specific use-case is that the server backend needs to have an up to date mapping of IP address to clients. But the client IPs are dynamic and subject to (effectively random) change.
The solution I have in mind requires the clients to ping the server to update their IP. The period ideally should be once every minute, but even 1 ping/10 mins is acceptable.
My questions, in sequence:
1M pings per 1 min is over 10k/sec. So first off I want to know
the approaches can scale to handle such a load. This is to know the options available.
Assuming you have more than one solution in mind, which of these
would be the most economical? The cost effectiveness is critically important. I don't have my own data center or
static and fat end-point on the net, so the server application will
need to run on some sort of provider or ultimately on the cloud.
Notes:
I considered running the server from home using my own ISP provided connection, but I am neither sure of the performance issues, nor what my ISP will think about a constant stream of pings.
I can't see how the server can auto-discover these IP changes.
Erik, your problem is much simpler than it seems to have been made to sound.
This problem been around for a decade maybe two. No need to re-invent the wheel here.
Why Polling/Pinging is a Bad Idea
The dynamic IPs provided by ISPs can have a variable lease time, but will often be at least 24-72 hours. Pinging your server every 1-10m will be a horrible waist of resources potentially making over a 4,320 useless HTTP requests PER CLIENT in a 72 hour period. Each request will be say around 300 bytes * 4,320 wasted http requests equals 1.3mb wasted bandwidth multiplied by your target client count of 1 million clients, you are talking about a monthly wasted bandwidth of ~1.2 TB! And that's just the wasted bandwidth, not the other bandwidth you might need to run your app and provide useful info.
The clients need to be smarter than just pinging frequently. Rather they should be able to check if their IP address matches the DNS on startup, then only when the IP changes, send a notification to the server. This will cut down your bandwidth and server processing requirements by thousands of times.
What you are describing is Dynamic DNS
What you are talking about is "Dynamic DNS" (both a descriptive name for the technology and also the name of one company that provides a SaaS solution).
Dynamic DNS is quite simply a DNS server that allows you to very rapidly change the mapping between a name and an IP address. Normally this is useful for devices using an ISP which only provides dynamic IPs. Whenever the IP changes for the router/server on a dynamic IP it will inform the Dynamic DNS server of the change.
The defacto standard protocol for dynamic DNS is well documented. Start here: DNS Update API, I think the specifics you are looking for are here: DynDNS Perform Update. Most commercial implementations out there are very close to the same protocol due to the fact that router hardware usually has a built in DynDNS client which everyone wants to use.
Most routers (even cheap ones) already have Dynamic DNS clients built into them. (You can write your own soft client, but the router is likely the most efficient location for this as your clients are likely being NAT'd with a private IP - you can still do it but at a cost of more bandwidth for public IP discovery)
A quick google search for "dynamic DNS java client" brings up full source projects like this one: Java DynDNS client (untested, just illustrating the power of search)
Other Considerations for your System Architecture
Lets say the IP-client mapping thing gets resolved. You figured it all out and it works perfectly, you always knows the IP for each client. Would you then have a nice reliable system for transferring files to clients from mobile devices? I would say no.
Both mobiles and home computers can have multiple connection types, Wi-Fi, Cellular Data, maybe wired data. Each of these networks may have different security systems in place. So a connection from a cellular data mobile to a wifi laptop behind a home router is going to look very different than a wifi mobile device connecting to laptop on the same wifi network.
You may have physical router firewalls to contend with. Also home computers may have windows firewall enabled, maybe norton internet security, maybe symantec, maybe AVG, maybe zone alarm, etc... Do you know the firewall considerations for all these potential clients?
Maybe you could use SIP as protocol for that purpose ?
Probably the java SIP libs already solved your problem.
Nice app by the way.
I would suggest better tweak you java program to know the IP change and then only hit the web service.
You can do it like,
on your java program initiation extract the IP of machine and store
it in Global variable or better some property file.
Run a batch process/scheduler which will check your IP every 30sec/1 minute for change.Java Quartz Scheduler will come very handy for you.
Invoke the web service in case of a change of IP.
This way it reduces your server role and thus traffic and connections.
You could create your own protocol on top of UDP, for example XML based. Define 3 messages:
request - client requests a challenge from server
challenge - server replies with challenge (basically a random number)
response - client sends username and hashed password + challenge back to the server
It's lightweight and not too traffic-heavy. You can load-balance it to multiple servers at any layer or using load-balancer.
Any average PC could handle million such hits per minute, provided you do server-side in C/C++ (I don't know about java network performance)
Please have a look at how no-ip works. Your requirement is exactly same as what it does.
Do I have the use case right? A community of users all want to receive pictures from each other? You don't want to host the images on the server but broadcast them directly to all the users?
There are two questions here. The first question is "how to know if my own WAN IP address has changed."
If you are not NATed then:
InetAddress.getLocalHost()
will tell you your IP address.
If you are NATed, then using dynamic DNS and resolving your own host name will work.
The second question is something like "How to share pictures between hosts which come and go on the internet".
The possible solution space includes:
IP Multicast, probably with Forward Error Correction and Carouseling, e.g. FLUTE.
File Swarming - e.g. bittorrent.
A Publish/Subscribe message bus solution using Jabber, AMQP, JMS, STOMP or similar. Suitable implementations include RabbitMQ, ActiveMQ, etc. JMS Topics are a key concept here.
The solution should avoid the massive overheads of doing things at the IP level.
I know that the Socket class can be instantiated in this way (as an example):
new Socket("taranis", 7);
where "taranis" is the name of the server in a local network. In this respect I have two questions:
1. If my computers do not form a local network but are connected to the Internet, can I use the IP addresses of computers to instantiate a socket?
2. How my computers get names in the local network? Do I give names to the computers when I create the network.
P.S. I herd that computers can establish a network by themself (using zeroconf). Who then gives names to the computers and how can I know these names in advance (I need them to write my code).
Yes, you can create a socket using the ip address, you can do so like this: new Socket("192.168.1.00",8888)
When you install an OS on your computer, usually one of the steps that the OS makes you go through is giving your machine a name. Each OS also has a way of changing these names after installation. So, everyone of your computers probably does have a name. However, the tricky part is getting one machine to know the name of the other. This can be done in a few ways. One is by using a DNS server. This kind of like a middle man which will translate the name (i.e Computer1) to its IP address (192.1.168.1.100). You can also keep these mapping locally, you can put this in the hosts file. This is a mapping between names and ip addresses, and if you use this method, you need to make sure that these hosts files line up across the machines.
Zeroconf is an interesting protocol. The way it works is one computer creates a named service and the second computer just looks up the service by name, and once it finds the service it can connect to it. When the service is discovered, the connecting client and can query for the ip and port to connect to.
Before socket programming, you need some background on networking. Unluckily the questions you ask are not simple to answer as it depends on the specific network configuration you have. Here are some short answers, but may be incorrect due to the dependency on the particular configuration. You will do better to read up on TCP/IP for example here.
Yes. But I doubt that that your computers do not form a local network (LAN). In case they do, you can use their LAN IP address. To find the ipaddress, you can use "ipconfig" command on the "Command Prompt" on Windows, and "ifconfig" on unix. The output is the configuration of each network interface in the computer.
Yes, you can configure the name of each computer on the computer.
For programming, usually you would use ipaddress (use name when the name can be dynamically assigned to one of the computers (using a Dynamic DNS)). IP Address may also be (and is quite commonly) dynamically assigned using a DHCP server.