I am not sure whether this question is Mina-related or more Azure-related but it has to do with the networking. I have also added Netty tag since Mina and Netty share many networking principles.
I hope to get an advice where to dig into.
I have used certain Mina application quite long in local network, now I am trying to migrate it into the cloud. I deploy Linux virtual machines in Azure (each has public IP but does this really matter?).
They connect (using Mina) to a machine outside Azure that also has its
own public IP. Usual thing:
SocketConnector connector = new NioSocketConnector(numberOfConnectors);
ConnectFuture connectFuture = connector.connect(new
InetSocketAddress(remoteHost, remotePort));
connectFuture.awaitUninterruptibly(connectTimeout);
That Mina machine outside the Azure also runs Mina. Let's call it
server machine.
It accepts connections like this:
NioSocketAcceptor acceptor = new NioSocketAcceptor(acceptor_threads);
org.apache.mina.core.buffer.IoBuffer.setUseDirectBuffer(false);
acceptor.getSessionConfig().setTcpNoDelay(true);
acceptor.setReuseAddress(true);
acceptor.getSessionConfig().setSendBufferSize(buffer_size);
acceptor.getSessionConfig().setMinReadBufferSize(64000);
acceptor.getSessionConfig().setReceiveBufferSize(buffer_size);
acceptor.getSessionConfig().setIdleTime(IdleStatus.BOTH_IDLE, iddle_time);
acceptor.getFilterChain().addLast("codec", new
ProtocolCodecFilter(CodecFactory.getInstance()));
acceptor.setDefaultLocalAddress(new InetSocketAddress(port));
When Azure applications connect to server machine, server saves
IoSession session
to asynchronously push messages back in future like this:
session.write(message);
This worked well inside a local network (without Azure), but in current
deployment server sends message
2017-01-17/15:45:19.823/GMT-00:00 [nioEventLoopGroup-3-3] [...] DEBUG
Sending message to /13.94.143.139:41790
and an Azure machine does not receive anything. Moreover, after a
while the following exception arises on server machine:
2017-01-17/16:01:11.419/GMT-00:00 [NioProcessor-4] [...] ERROR
Exception in IOHandlerConnection timed out
java.io.IOException: Connection timed out
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at org.apache.mina.transport.socket.nio.NioProcessor.read(NioProcessor.java:280)
at org.apache.mina.transport.socket.nio.NioProcessor.read(NioProcessor.java:44)
at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:695)
at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:668)
at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:657)
at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:68)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1141)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2017-01-17/16:01:11.424/GMT-00:00 [NioProcessor-3] [...] DEBUG sessionClosed
I use Mina version 2.0.4 (yes, it is old but it works on local network for several years for now).
I setup Azure network with Java Azure SDK 1.0.0-beta3
Network.DefinitionStages.WithCreate creatableNetwork = azure.networks()
.define(networkName)
.withRegion(region)
.withExistingResourceGroup(resourceGroup)
.withAddressSpace("10.0.0.0/20");
And create virtual machines as
VirtualMachine.DefinitionStages.WithCreate creatableVirtualMachine =
azure.virtualMachines()
.define(String.format(...))
.withRegion(region)
.withExistingResourceGroup(resourceGroup)
.withNewPrimaryNetwork(creatableNetwork)
.withPrimaryPrivateIpAddressStatic(inetAddress.getHostAddress())
.withNewPrimaryPublicIpAddress(String.format("chr-vm-%04d", i)) .withPopularLinuxImage(KnownLinuxVirtualMachineImage.UBUNTU_SERVER_16_04_LTS)
.withRootUserName(linuxUserName)
.withPassword(linuxUserPassword)
.withSize(VirtualMachineSizeTypes.STANDARD_D2_V2)
.withNewStorageAccount(creatableStorageAccount);
I wonder what reasons may prevent traveling messages from server to
Azure client machines? Azure network configuration? Mina configuration? (the first messages from client machines to server machine do come after they connect)
I hope that above information may contain a clue.
I have solved my problem thanks to Peter Pan - MSFT noting about NSG - Network Security Group.
NSG controls in/out rules like a Windows Firewall. You should create NSG, add rules to it, and assign NSG to a particular entity:
There are at least two options to assign NSG:
to a network subnet
to a network interface
There is a tutorial 1 and Java code sample 2. In my case, a separate network interface is created for each VM (since each VM has public IP). So, I assigned one NSG to a single subnet.
Fisrt, create NSG:
NetworkSecurityGroup NSG = azure.networkSecurityGroups()
.define(networkSecurityGroup)
.withRegion(region)
.withExistingResourceGroup(resourceGroup)
.defineRule("Inbound")
.allowInbound()
.fromAnyAddress()
.fromAnyPort()
.toAnyAddress()
.toAnyPort()
.withAnyProtocol()
.withDescription("Incoming messsages")
.withPriority(100)
.attach()
.create();
Than modify the code to explicitly define a subnet and assign NSG to it ( subnet1 is automatically created without NSG if none defined explicitly)
Network.DefinitionStages.WithCreate creatableNetwork = azure.networks()
.define(networkName)
.withRegion(region)
.withExistingResourceGroup(resourceGroup)
.withAddressSpace("10.0.0.0/20")
.defineSubnet(subnetName)
.withAddressPrefix("10.0.0.0/20")
.withExistingNetworkSecurityGroup(NSG)
.attach();
So, the rest of the code remains the same as posted in the question above.
Helpful links:
Azure Portal Tutorial
Java Azure SDK NSG Example
Related
Recent Microsoft patch has broken our Java client connectivity to OPC server. Microsoft raised Authentication level to Packet Integrity. we are using Utgard java OPC library to connect to OPC server which internally uses Jinterop java based COM library. with the patch Our client is not able to add subscription groups to listen to the data changes on the OPC server. We couldn't find where in the library the authentication level is configured. if we can configure it ti DEFAULT the issue will be resolved.
Please let me know if you have any pointers on this.
JISession session = JISession.createSession(ipAddress,userName,pasword);
JIComServer comServer = new JIComServer(JIClassId.valueOf(classId), address, session);
IJIComObject instance = comServer.createInstance();
IJIComObject opcServer = new OPCServer(instance);
//after we will create access base groups and add them to the server, during attaching handler it is failing.
ERROR
*JIException: The RPC server is unavailable. Please check if the COM server is up and running and that route to the COM Server is accessible (A simple "Ping" to the Server machine would do). Also please confirm if the Windows Firewall is not blocking DCOM access. [0x800706BA]
at org.jinterop.dcom.core.JIComServer.call(JIComServer.java:1004) at org.jinterop.dcom.core.JIComServer.call(JIComServer.java:951) at org.jinterop.dcom.core.JIComObjectImpl.call(JIComObjectImpl.java:293) at org.jinterop.dcom.core.JIComObjectImpl.call(JIComObjectImpl.java:159) at org.jinterop.dcom.core.JIFrameworkHelper.attachEventHandler(JIFrameworkHelper.java:285) at org.openscada.opc.dcom.da.impl.OPCGroupStateMgt.attach(OPCGroupStateMgt.java:179)
*
The goal is to publish/send message into ActiveMQ through Java code inside a secured company network.
I have configured ActiveMQ in an AWS Cloud EC2 machine (console access: IPAddress:8161). Also I can publish the messages using the AWS IPAddress and port number 61616 (IPAddress:61616) through Java code.
But now I need to publish messages from inside a company network. It is secured and can't access the AWS IPAddress directly.
So we create reverse proxy for
IPAddress:8161 to activemq-ui.testdemo.com
IPAddress:61616 to activemq-api.testdemo.com
Now I can access ActiveMQ console from our company network using activemq-ui.testdemo.com. But couldn't access activemq-api.testdemo.com through Java code.
Getting Below Error:
SEVERE: Error Message: javax.jms.JMSException: Could not connect to broker URL: tcp://activemq-api.demo.com. Reason:
java.lang.IllegalArgumentException: port out of range:-1
Error looks like expecting port number in the URL. But not sure what to pass for this.
Can anyone help me on how to access ActiveMQ API inside corporate network?
You need to provide the port that the client should attempt to connect to on the connection URI as the error is telling you, something like:
tcp://activemq-api.demo.com:80
The client does not attempt to guess or deduce what the port is you want it to use and so that field is mandatory.
I'm trying to connect to a Grakn Server running on a GCP VM instance. I have read the related post in the discussion forum. One of the replies suggests to do the following on the server side:
Edit ./conf/main/grakn.properties:
storage.hostname=EXTERNAL.ip.v4.address
Edit ./conf/cassandra/cassandra.yaml:
seeds: "EXTERNAL.ip.v4.address"
broadcast_address: EXTERNAL.ip.v4.address
listen_address: INTERNAL.ip.v4.address
rpc_address: INTERNAL.ip.v4.address
Access it from the Java API:
GraknSession session = Grakn.session(“EXTERNAL.ip.v4.address”, keySpaceName);
What i have interpreted from this, for my specific case, is the following:
-Replace EXTERNAL.ip.v4.address with the External IP of my VM instance
-Replace INTERNAL.ip.v4.address with the Internal IP of my VM instance
When i start the server i get this exception cause (I think this is the root cause):
Caused by:
com.netflix.astyanax.connectionpool.exceptions.PoolTimeoutException:
PoolTimeoutException: [host=104.197.131.17(104.197.131.17):9160,
latency=10000(10000), attempts=1]Timed out waiting for connection
at com.netflix.astyanax.connectionpool.impl.SimpleHostConnectionPool.waitForConnection(SimpleHostConnectionPool.java:231)
What am i missing?
Do i have to add firewall rules?
Thanks for the help.
To access Grakn remotely using the Java API, you need to open the following ports on the GCP and Linux VM firewall (eg. iptables):
4567
9160
Make sure you are able to access those from the client machine.
I am trying to test Amazon's new Memcached client with AutoDiscovery. I have one memcached node which I am able to connect to using XMemcached 1.3.5 as well as a standard SpyMemcached library.
I am following the instructions here: http://docs.amazonwebservices.com/AmazonElastiCache/latest/UserGuide/AutoDiscovery.html
The code is almost identical to the example and is:
String configEndpoint = "<server name>.rgcl8z.cfg.use1.cache.amazonaws.com";
Integer clusterPort = 11211;
MemcachedClient client = new MemcachedClient(new InetSocketAddress(configEndpoint, clusterPort));
client.set("theKey", 3600, "This is the data value");
I see the following in the logs when I create the connection. The error happens when I try to set a value:
2013-01-04 22:05:30.445 INFO net.spy.memcached.MemcachedConnection: Added {QA sa=/<ip>:11211, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-01-04 22:05:32.861 INFO net.spy.memcached.ConfigurationPoller: Starting configuration poller.
2013-01-04 22:05:32.861 INFO net.spy.memcached.ConfigurationPoller: Endpoint to use for configuration access in this poll NodeEndPoint - HostName:<our-server>.rgcl8z.cfg.use1.cache.amazonaws.com IpAddress:<ip> Port:11211
2013-01-04 22:05:32.950 WARN net.spy.memcached.MemcachedClient: Configuration endpoint timed out for config call. Leaving the initialization work to configuration poller.
Exception in thread "main" java.lang.IllegalStateException: Client is not initialized
at net.spy.memcached.MemcachedClient.checkState(MemcachedClient.java:1623)
at net.spy.memcached.MemcachedClient.enqueueOperation(MemcachedClient.java:1617)
at net.spy.memcached.MemcachedClient.asyncStore(MemcachedClient.java:474)
at net.spy.memcached.MemcachedClient.set(MemcachedClient.java:905)
at com.thinknear.venice.initializers.VeniceAssets.main(VeniceAssets.java:227)
I've tried this both locally and on a EC2 instance (I can connect using other libraries to the nodes)
I've tried using both 1.4.5 and 1.4.14 Memcached engines
I relaxed the security group constraints as well just in case
Any thoughts on why the config endpoint would be timing out?
Client is not initialised:
You can not directly connect to amazon elastic cache node through your local machine you can only access it through your ec2 machiene.If you want to check you can telnet from your local machine it will not connect I also suufered from the same problem .You can telnet it from your Ec2 machine.so try your code at ec2 machine it will work.
Do telnet on memcache server to check connectivity ,in mine case it was not listed so was not able to made connection ,
problem solved by listing my server to memcache.
i'm coding a command line tool to manage the S3 service. on my local machine, everything works but on the server where it should be executed, fails with the following message:
Error Message: Unable to execute HTTP request: Connection to http://s3.amazonaws.com refused
i make the connection with the following code:
s3 = new AmazonS3Client(credentials,clientConf);
clientConf only sets the protocol to HTTP, as i suspected that maybe could be a problem to connect to HTTPS but i'm having the same result.
now, the server have the following configuration:
debian 6 64 bits
LAMP installed from source
openssl installed from source
java installed from distribution packages packages
this is the network configuration:
auto eth0
iface eth0 inet static
address XX.XX.XX.XX
netmask 255.255.255.255
broadcast XX.XX.XX.XX (same as address)
auto eth0:0
iface eth0:0 inet static
address XX.XX.XX.XX
netmask 255.255.255.255
broadcast XX.XX.XX.XX (same as address)
auto eth0:1
iface eth0:1 inet static
address XX.XX.XX.XX
netmask 255.255.255.255
broadcast XX.XX.XX.XX (same as address)
post-up route add 10.255.255.1 dev eth0
post-up route add default gw 10.255.255.1
wget, telnet, curl, everything works, except this, i have 3 network interfaces as i have 2 SSL and another ip for the other sites.
how i should configure the clientConf to make this work? is a java problem? a network problem? at least, how i can get more debug info? i tried to catch the AmazonClientException exception but doesn't work.
Thanks in advance :)
Regards.
This has been reported as a bug in the Amazon S3 API. Quoth ZachM#AWS:
This appears to be a bug in the SDK. The problem is that the client
configuration object is shared with the Security Token Service client
that DynamoDB uses to establish a session, and it (unlike Dynamo)
doesn't accept the HTTP protocol. There are a couple workarounds:
1) Create your own instance of STSSessionCredentialsProvider and
provide it to your DynamoDB client, or
2) Instead of specifying the protocol in the ClientConfiguration,
specify it with a call to setEndpoint("http://...")
We'll discuss solutions for this bug.
I would recommend using one of the workarounds for now. Good luck getting your connection to work successfully.
(Additional documentation and workarounds)