We Keep Coding

Java - Is there a way a client and a server can connect via IPv4 without port forwarding?

So I'm trying to connect two clients in a Java application, but in a way that one client acts as a server and other client acts as a ... client. I managed to connect them locally which works perfect, but I've been researching whether I can connect a client to a server that are not on a same network (via IPv4 or IPv6). I have read that I should do port forwarding on my router server-side. I know how to port forward, but shouldn't it be possible to do without port forwarding? If I understand correctly, only server-side should be port forwarded and the server can respond to the client without the need for the client to port forward their router? So if I'm correct, another solution would be a 'global' third party server(that is port forwarded) that would connect two clients by receiving and passing information from one client to another?
I'm just learning here, so I'm sorry if this has already been answered here but I haven't found answers to all of this in one place and I'm trying to come to a conclusion.

Yes, you can access a computer from outside the network and connect to a server
You must download the (ngrok) tool on the device that contains the server and run the tool
The client will contact the server without the need to forward the ports
ngrok
Explain the use of the tool on the site with a download link

shouldn't it be possible to do without port forwarding
Yes, you can make a connection between two machines without port-forwarding.
Example: Web servers
Take for example, web servers. By default a web server sits there listening on port 80, with 80 being the port assigned by convention for HTTP.
The web client (browser or such) sends a request by trying to connect on port 80. If there are no obstacles in the way, then the connection proceeds.
Restricted port access
However, there may be an obstacle.
One common obstacle: Unix-oriented operating systems (BSD, macOS, Solaris, Linux, AIX, etc.) by convention restrict access to ports numbered under 1,024 for security reasons. The operating system blocks any incoming connections on port 80. With that security blockage in place, the web request never reaches the server.
Port-forwarding with a packet-filter tool
One way to get past this restriction is to have the web server listen on an unrestricted port, a port numbered above 1,024, up to the 64K limit, such as 8080. Then configure the packet filter tool on the server machine’s OS to do port-forwarding. The incoming request for port 80 is altered to go to port 8080 instead.
A connection is then established between the web server and the web client.
The client thinks it is talking to the server on port 80.
The server thinks the client asked for port 8080.
With the packet filter tool in the middle altering packets on-the-fly, both server and client is none the wiser about packets being altered.
You may want to configure your firewall to allow HTTP connections from outside the machine only on 80, including blocking any external requests for 8080. In this case, only packets altered from 80 to 8080 will reach your web server. Common practice is to close as many ports as possible on a server.
FYI: For encrypted HTTP (HTTPS), the conventional port is 443 rather than 80.
Not a programming issue
Notice that there is no programming issue here. As the programmer, your client software should attempt to connect on the port number as documented for the server in which you are interested. On the server-side machine, or server-side router, port-forwarding will be configured as needed. Your client programming does not care about, or even know about, any port-forwarding that may or may not be in place. Port-forwarding is a network-admin issue, and should be transparent to the programmer.
See sister sites for networking issues
As a network-admin issue, look to the sister sites such as Server Fault and Network Engineering rather than Stack Overflow.

How to decide which port to use in socket communcation

I have to write a Socket communication program to implement
Inter Process Communication.
I understand I need to have a client and server implementation.
Server implementation:
MyServerice = new ServerSocket(PortNumber);
Client and server will be on same machine so I don't want to interfere with any other running applications
How to decide which port number to use?

You can look at this table
those are mostly all, you will never know if any other external app is taking the port you have in mind, so in your case is a
Firt to come, first to serve issue

I think in implementing the websocket object, the WebSocket protocol communications typically use TCP port number 80, so environments that block non-standard internet connections by using a firewall will still pass WebSocket packets.

ZeroMQ pattern for a (legacy) TCP to ZeroMQ proxy

We are building a new client along with a caching proxy component for an existing online game. The current setup looks like:
Server
| \
TCP TCP
| \
Client1 Client2
The new setup will be:
Server
| |
TCP TCP
\ \
Proxy <-- (New!)
| \
0MQ 0MQ
| \
Client1 Client2 <-- (New!)
The proxy component will sit close to the existing game server and speak the old line based TCP protocol to it, while speaking a new Google Protocol Buffers-based protocol over 0MQ to the new client. In addition to forwarding and translating messages between the server and its clients, the proxy will also cache (protobuf) messages destined for clients.
The server responds to client requests, but may also send unsolicited messages to specific clients, or to all clients. Since the proxy will maintain a cache of messages destined for a certain client, it needs to keep some state associated with each client.
I've been reading quite a bit in the ZeroMQ Guide, but I'm still not sure which of its "patterns" or socket combinations would suit this setup best. I'm hoping there's some ZeroMQ wizard out there who can give some advice on what the best way to go here is.
The question is not language specific, but the proxy component will be written in Java and the client in C#.
Many thanks in advance!

I think the Asynchronous Client Server pattern will get you close. Each of your clients creates a DEALER socket which connects to the ROUTER socket in your proxy. Review the example Java source. You could base your proxy on the ServerTask class, using the frontend/ROUTER socket and omitting the backend/dealer socket.
Your server will be able to respond to client requests and send unsolicited messages back to any client. You will need to cache the socket id of each client after the client has sent at least one message.
Your TCP connection to the game server is conceptually similar to the backend DEALER sockets between the server and workers - however since your connection is classic TCP and and not ZeroMQ you cannot use ZeroMQ sockets and will need to do something custom. And your custom solution requires that you not violate ZeroMQ's concurrency rules, when you integrate with classic TCP sockets.
Here is one way - use a set of PAIR sockets (lets call them A and B) and a BlockingQueue to act as a bridge between ZeroMQ and your TCP sockets. And create 3 threads - TCPRead, TCPWrite and ZeroMQ.
TCPRead reads from the TCP connection to the game server. It converts them to protobuf messages and forwards them through the A socket.
TCPWrite polls the BlockingQueue for messages. When it receives a message it converts it and sends it to the game server via TCP connection.
Finally, most of the work is in the ZeroMQ thread. It uses ZeroMQ polling and is constantly polling it's router socket (for all messages from all clients) and the B socket (for messages from the game server). Sending messages to a client is straightforward and is handled by sending the message to the router socket, prefixing the message with the socket id of the client. Sending messages to the server is done by adding a message to the BlockingQueue. It will be picked up by TCPWrite and forwarded to the game server.
Once you get this basic design working, you can go nuts and add workers etc as described in the various patterns.
Hope this helps.

You can also use 0MQ ROUTER sockets in raw mode, meaning you could write the proxy with two ROUTER sockets. I'd stay away from PAIR sockets except for parent-to-child thread pipes.

It takes a while to plough through the ZeroMQ documentation, excellent as it is. I did and came to the conclusion that Asynchronous Client-Server was close to what I initially wanted.
But it wasn't quite. So I then proceeded to build.
What I actually wanted is based on what I'd learnt from the various examples. A full description of what I ended up with is in here as is the code which may be of some use, for reference or as an actual library.

Server UDP and port binding

I am writing this game in Java and have problems with networking architecture.
I decided I will UDP packets. I am just at the beginning, but the problem I am facing is that it seems to be that server have to respond from exactly same IP/Port to client (which is behind router which uses NAT) as client connected that server.
For example I have client A behind router. Client A has IP (local) 192.168.8.100 and it connects server B from port 1234. Server is on 11.11.11.11:2345.
When client A connects to server B it uses 192.168.8.100:1234 but router converts that to (for example) 22.22.22.22:6789.
Now, when server wants to send packets to that client it has to be from 11.11.11.11:2345.
I would like to send data from another port like 11.11.11.11:2222, but this does not seem to work, at least not with my router.
I want to use different port because I want to have two threads one for listening and one for sending data, and each thread would have it's own DatagramSocket. But, as i said once client A connects to server on port 2345, I can not send data from port 2222.
Does anyone know how is this handled? I am doing it in Java, but it's not really a language specific problem.
UPDATE
After #Perception commented I have some more questions regarding his comments:
OK, so if I understand this correctly, if I have server which is hosting 1000 games, each with 2 players, all sending/receiving will have to be done through the same DatagramSocket.
As I understand DatagramSocket is thread safe so I guess I can have one thread doing:
datagramSocket.receive();
while at the same time second thread is doing
datagramSocket.send(.....);
Correct?
Also, two threads can send data at the same time through the same DatagramSocket? Is sending in any way serialized, meaning that second send() starts only after previous send() is finished or is data being sent at the same time?

gorann, I'm not sure if I'm understanding you correctly, but it sounds like you're trying to control the port on which the server communicates with the client. There's no way to control this, and for good reasons.

This is one of the trickier differences between TCP and UDP.
When a new TCP session is initiated, the server side call to accept() gives you a new socket and the OS handles multiplexing the various sessions for you. With UDP, you need to handle the multiplexing yourself. But you need to do so in a way that works with NATs and other firewalls.
The way NAT works is that when it sees an outgoing packet, it creates a temporary rule allow packets to return along the same port pair. Data returning from a port that the client has not yet sent to will likely be blocked.
This gives you two choices:
You could do all of your communication through a single port. This is not a bad option, it just means that you need a way to identify client sessions and route them to the appropriate thread.
You could create a separate port and instruct the client to send to that one instead. Have the server listen on a fixed port. The client sends a message to there, the server then sets up a new session port and sends that number back to the client using the server's listen port. The client then sends a message to the session port, which causes the NAT to open up that port and allow return traffic. Now the client and server thread have their own private port pair.
Option 1 is a bit more work because it requires data to be exchanged between threads, but it scales up better. Option 1 is easier and more CPU efficient because each session thread can be independent, but there are a finite number of ports available.
Either way, I recommend that you have the client include a semi-unique session id in each packet so that the server has more than just the client address and port number to verify who belongs to each session.

Redirect a TCP connection

I have something like a proxy server (written in java) running between my clients and the actual video server (made in c++). Everything the clients send goes through this proxy and is then redirected to the server.
It is working fine, but I have some issues and think it would be better if I could make this proxy server only to listen to the clients requests and then somehow tell the server that a request has been made from the client side, and that it is supposed to create a connection with the client directly.
Basically in the TCP level what I want to happen is something like this:
1- whenever a client sends a SYN to my proxy, the proxy just sends a message to the real server telling the ip and port of the client.
2- The server would then send the corresponding SYN-ACK to the specified client creating a direct connection between client and server.
The proxy would then be just relaying the initial requests (but not the later data transfer) to the actual server. I just don't know if that is possible.
Thank you very much
Nelson R. Perez

That's very much the way some games (and Fog Creek CoPilot) do it, but it requires support on both the server and the client. Basically the proxy has to say to the client and server "try communicating with the directly on this ip and this port" and if they can't get through (because one or both is behind a NAT or firewall), they fall back to going through the proxy.
I found this good description of "peer to peer tcp hole punching" at http://www.brynosaurus.com/pub/net/p2pnat/

Does the proxy and server lives on the same machine? If so, you can pass the connection to the server using Socket Transfer or File Descriptor Passing. You can find examples in C here,
http://www.wsinnovations.com/softeng/articles/uds.html
If they are on the different machines, there is no way to pass connection to the server. However, it's possible to proxy the IP packets to server using VIP (Virtual IP). This is below socket so you have to use Link layer interface, like DLPI.

You don't have control of TCP handshake in userland like that. This is what firewalls/routers do but it all happens in the kernel. Take a look at the firewalling software for your platform - you might not even have to code anything.