I'm using Jedis to connect to my Redis instance/cluster in AWS, but I kept getting this error, here's the code, I searched extensively on SO, found the closest one is: String hostname from properties file: Java
I tried both ways, neither worked for me.
So please help.
Here's my Java code:
public static void main(String[] args) {
AWSCredentials credentials = null;
try {
credentials = new ProfileCredentialsProvider("default").getCredentials();
} catch (Exception e) {
throw new AmazonClientException("Cannot load the credentials from the credential profiles file. "
+ "Please make sure that your credentials file is at the correct "
+ "location (/Users/USERNAME/.aws/credentials), and is in valid format.", e);
}
AmazonElastiCacheClient client = new AmazonElastiCacheClient(credentials);
client.setRegion(Region.getRegion(Regions.AP_NORTHEAST_2));
DescribeCacheClustersRequest dccRequest = new DescribeCacheClustersRequest();
dccRequest.setShowCacheNodeInfo(true);
DescribeCacheClustersResult clusterResult = client.describeCacheClusters(dccRequest);
List<CacheCluster> cacheClusters = clusterResult.getCacheClusters();
for (CacheCluster cacheCluster : cacheClusters) {
for (CacheNode cacheNode : cacheCluster.getCacheNodes()) {
String addr = cacheNode.getEndpoint().getAddress();
int port = cacheNode.getEndpoint().getPort();
String url = addr + ":" + port;
System.out.println("formed url is: " + url);
Jedis jedis = new Jedis(url);
System.out.println("Connection to server sucessfully");
// check whether server is running or not
System.out.println("Server is running: " + jedis.ping());
}
}
The last line in the above code keeps throwing this error, here's the stacktrace:
Exception in thread "main" redis.clients.jedis.exceptions.JedisConnectionException: java.net.UnknownHostException: REDISNAME.nquffl.0001.apn2.cache.amazonaws.com:6379
at redis.clients.jedis.Connection.connect(Connection.java:207)
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:93)
at redis.clients.jedis.Connection.sendCommand(Connection.java:126)
at redis.clients.jedis.Connection.sendCommand(Connection.java:121)
at redis.clients.jedis.BinaryClient.ping(BinaryClient.java:106)
at redis.clients.jedis.BinaryJedis.ping(BinaryJedis.java:195)
at sporadic.AmazonElastiCacheClientExample.main(AmazonElastiCacheClientExample.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)Caused by: java.net.UnknownHostException: REDISNAME.nquffl.0001.apn2.cache.amazonaws.com:6379
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at redis.clients.jedis.Connection.connect(Connection.java:184)
... 11 more
What am I doing wrong?
Please point out.
Your setting shoud be this way :
Jedis jedis = new Jedis("REDISNAME.nquffl.0001.apn2.cache.amazonaws.com",6379);
NOT this way :
Jedis jedis = new Jedis("REDISNAME.nquffl.0001.apn2.cache.amazonaws.com:6379");
According to AWS Documentation http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/Access.Outside.html
Amazon ElastiCache is an AWS service that provides cloud-based
in-memory key-value store. On the back end it uses either the
Memcached or Redis engine. The service is designed to be accessed
exclusively from within AWS. However, if the ElastiCache cluster is
hosted inside a VPC, you can use a Network Address Translation (NAT)
instance to provide outside access.
So you have below two options :-
Either you host your app inside the AWS and have proper security group setting to allow access to your elastic-cache cluster from your ec2-instance where your app is deployed.
If you want to run your app outside of AWS then you have to modify the Network Address Translation (NAT) to provide outside access.
IMO, its easy to deploy the code in AWS-Ec2 instance and test it if you are not very familiar with the networking and NAT.
I used to have locally memcache and redis instance where i used to connect for local developement and for other environment like qa,stg,prod used to deploy it in AWS ec2 instance.
Let me know if you any issues.
In my case the 6379 port was not accepting connections, so I changed I configured Redis with different port, and it worked.
Related
In a java service, I'm trying to upload a file in an azure storage directory; therefore I've written a code like this :
import com.azure.core.util.*;
import com.azure.storage.file.share.*;
import com.azure.storage.file.share.models.*;
//Create connexion string
String connectStr ="DefaultEndpointsProtocol=https;AccountName=" + accountName + ";AccountKey=" + accountKey + ";EndpointSuffix=" + endpoint;
//ShareDirectoryClient
ShareDirectoryClient dirClient = new ShareFileClientBuilder().connectionString(connectStr).shareName(shareName).resourcePath(directoryName).configuration(proxyOptions).buildDirectoryClient();
// Create empty file
dirClient.createFile(fileName, body.length());
The HTTPS request must goes through a proxy server, so, I get this error :
"Could not run 'sendFileInDirectoryProxyTest'
reactor.core.Exceptions$ReactiveException: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection timed out: no further information: "
I can't set/use a global setting.
To set a proxy in the Java code, I've tried several things, like using the Configuration Class :
Configuration configuration = new Configuration();
configuration.put("java.net.useSystemProxies", "true");
configuration.put("https.proxyHost", "xxxxxxxxx");
configuration.put("https.proxyPort", "xxxx");
ShareDirectoryClient dirClient = new ShareFileClientBuilder().connectionString(connectStr).shareName(shareName).resourcePath(directoryName).configuration(configuration).buildDirectoryClient();
But it did not solve the issue.
I'm sure it is pretty simple, any help would be appreciated.
Thanks. Charles de Saint Andre.
You need to configure ProxyOptions and set them on the httpClientBuilder. All our Storage client builders have a .httpClient() method that accepts a client, and you can build a client with all defaults + the proxy options using a NettyAsyncClientBuilder(), which has a .proxyOptions() method. Please give that a try and let me know if you have any more issues.
Sample : azure-sdk-for-java/sdk/storage/azure-storage-blob at main · Azure/azure-sdk-for-java (github.com)
I tried Connecting the AWS Neptune with this Java code and got the error , NoHostAvailable Exception
approach 1:
public static void main(String[] args) throws Exception {
Cluster.Builder builder = Cluster.build();
builder.addContactPoint("endpoint");
builder.port(8182);
builder.enableSsl(true);
builder.keyStore("pem-file");
Cluster cluster = builder.create();
GraphTraversalSource g = traversal().withRemote(DriverRemoteConnection.using(cluster));
System.out.println(g.V().limit(10).toList());
cluster.close();
}}
approach 2:
Cluster cluster = Cluster.build("endpoint").
enableSsl(true).keyStore("pem").
handshakeInterceptor( r -> {
NeptuneNettyHttpSigV4Signer sigV4Signer = null;
try {
sigV4Signer = new NeptuneNettyHttpSigV4Signer("us-east-2", new
DefaultAWSCredentialsProviderChain());
} catch (NeptuneSigV4SignerException e) {
e.printStackTrace();
}
try {
sigV4Signer.signRequest(r);
} catch (NeptuneSigV4SignerException e) {
e.printStackTrace();
}
return r;
}).create();
Client client=Cluster.open("src\\conf\\remote-objects.yaml").connect();
client.submit("g.V().limit(10).toList()").all().get();
what ever I do, I am getting this error:
Sep 02, 2021 3:18:34 PM io.netty.channel.ChannelInitializer exceptionCaught
WARNING: Failed to initialize a channel. Closing:
java.lang.RuntimeException: java.lang.NullPointerException
org.apache.tinkerpop.gremlin.driver.Channelizer$AbstractChannelizer.initChannel(Channelizer.java:117)
Caused by: org.apache.tinkerpop.gremlin.driver.exception.NoHostAvailableException: All hosts
are considered unavailable due to previous exceptions. Check the error log to find the actual
reason.
I need the code or the document to connect my Gremlin code in .java file to AWS neptune. I am struggling and tried various number of ways,
1.created EC2 instance and did installed maven and apache still got error and code is running in Server(EC2), i want code to present in IntelliJ
it would be more helpful, if I get the Exact Code any way. what should be added in remote-objects.yaml.
if we require Pem-file to access Amazon Neptune, please help with the creation of it.
Assuming SSL is enabled but IAM is not, in terms of Java code, this is all you need to create the connection.
Cluster.Builder builder = Cluster.build();
builder.addContactPoint("localhost");
builder.port(8182);
builder.enableSsl(true);
builder.serializer(Serializers.GRAPHBINARY_V1D0);
cluster = builder.create();
drc = DriverRemoteConnection.using(cluster);
g = traversal().withRemote(drc);
You may need to add an entry to your /etc/hosts file to get the SSL certs to resolve correctly such as:
127.0.0.1 localhost my-neptune-cluster.us-east-1.neptune.amazonaws.com
If you find that using localhost with SSL enabled does not work then use the actual Neptune cluster DNS name and make the edit to your /etc/hosts file.
The last thing you will need to do is create access to the Neptune VPC from your local machine. One way is using an SSH tunnel as explained in this post
How do I connect to the localhost using UnboundID LDAP SDK? I would think it is pretty straight forward, but maybe not. I connect just fine using the following code, but I would like to have the option to just use the locahost connection and not have to authenticate.
With the connection, I perform a series of add/remove/modify, which works fine with the below connection.
public LDAPConnection connect(LdapConnectionModel connectionModel)
{
this.connectionModel = connectionModel;
try
{
// Determine is SSL port was specified
int port = connectionModel.isSslEnabled() ? SSL_PORT : PORT;
// Determined bind DN
String bindDN = connectionModel.getUsername() + "#" + connectionModel.getDomain();
// Connect
connection = new LDAPConnection(connectionModel.getHost(), port, bindDN, String.valueOf(connectionModel.getPassword()));
// Clear out our password
connectionModel.setPassword(new char[] {});
}
catch (LDAPException e)
{
LOG.warning("CONNECTION FAILED: " + e.getMessage());
LOG.warning(e.getMessage());
}
return connection;
}
For example, getting a connection like this is fine, but then I get this error:
"In order to perform this operation a successful bing must be completed on the connection."
// Connect
connection = new LDAPConnection("localhost",389);
It makes no difference where, or on which host, the directory server is running. When an LDAP client connects to a server, that connection is unauthenticated. LDAP clients must use the BIND request to request the server change the authorization state of the connection to a state that permits the operations that the LDAP client desires.
see also
LDAP: Authentication Best Practices
LDAP: Programming Practices
I'm trying to make a java application which uses redis as a backend. Since Redis is a really fast key-value store which I'd like to use, but redis is made to be used with 1 client so it doesn't have an option for user:pass authentication. I'd like to find a way to implement some kind of authentication, so I tried nginx with the redis2 extension. I did this because I could use client-side certficates and HTTPS. But it's making my application really slow.
I'm thinking about using some kind of tunnel which connects to redis via nginx proxy. For this redis would be listen on localhost and there would be an address which I'd like to use to reach redis, but with https authentication. So basically my current method
JAVA - Jedis - LAN - REDIS ,would be
JAVA - Jedis(with localhost as the tunnel entrance?)-
-SSL LAN - Nginx(tunnel exit) - Redis
Any tip for achieving this? I've been googled the web for the last days but i couldn't come up anything that adds only a little overhead to the native connection.
Redis is designed to work on a secure network, behind a backend application. Client applications are not supposed to connect directly to Redis. It makes Redis a poor choice for a 2-tier application.
Now if you still want to use Redis for this, you have several options. You can encapsulate the Redis server in a HTTP interface. This is what the nginx redis2 module provide. You might also want to have a look at webdis, which is similar (and does not depend on nginx). Webdis offers some access control mechanisms. See the documentation.
Another solution is to establish a tunnel, as you proposed. I would not use nginx for this, but just plain old SSH. Let's suppose Redis server runs on machine B (port 6379) and client runs on machine A.
On machine A, I can run:
ssh user#host_B -L 7008:host_B:6379 -N
It will open a tunnel from A to B from local port 7008 (arbitrary choice), and waits. The user should be declared on host B, and its password known. In another session, still on host A, we can now run:
redis-cli -p 7008 ping
Please note a standard Redis client is used. The tunnel handles authentication, encryption and optionally compression in a transparent way for the client.
Now, your client is a Java application, and you probably do not want to run SSH commands to setup the tunnel. Hopefully, you can use the Jsch package to open the tunnel directly from Java. Here is an example with Jedis:
import redis.clients.jedis.*;
import java.util.*;
import com.jcraft.jsch.*;
public class TestTunnel {
Jedis jedis;
Session session;
JSch jsch = new JSch();
int port;
// None of the following should be hardcoded
static String USER = "user"; // SSH user on the redis server host
static String PASSWD = "XXXXXXXX"; // SSH user password
static String HOST = "192.168.1.62"; // Redis server host
static int PORT = 6379; // Redis server port
public TestTunnel() {
try {
// Open the SSH session
session = jsch.getSession( USER, HOST, 22 );
session.setPassword( PASSWD );
java.util.Properties config = new java.util.Properties();
config.put("StrictHostKeyChecking", "no");
config.put("Compression", "yes");
config.put("ConnectionAttempts","3");
session.setConfig(config);
session.connect();
// Setup port forwarding from localhost to the Redis server
// Local port is ephemeral (given by the OS)
// Jedis connects to localhost using the local port
port = session.setPortForwardingL( 0, HOST, PORT );
jedis = new Jedis( "127.0.0.1", port );
} catch ( JSchException e ) {
// Proper error handling omitted
System.out.println(e);
}
}
public void disconnect() {
jedis.disconnect();
try {
session.delPortForwardingL( port );
session.disconnect();
} catch ( JSchException e ) {
// Proper error handling omitted
System.out.println(e);
}
}
public void mytest( int n ) {
for ( int k = 0; k < n; k++) {
jedis.set("k" + k, "value"+k);
}
System.out.println("Read: "+jedis.get("k0") );
}
public static void main(String[] args) {
TestTunnel obj = new TestTunnel();
obj.mytest(10);
obj.disconnect();
}
}
It works fine, but please note there is an overhead due to the tunnel. The overhead is very low when the network is slow (the Internet for instance). On a fast LAN (1 GbE), it is much more noticeable: the latency can be multiplied by up to 3 when the tunnel is used. The maximum throughput the Redis server can sustain is also impacted. On server-side, the sshd daemon takes some CPU (more than Redis itself).
That said, I don't think raw performance matters much for a 2-tier application.
Note: There's an SSL version of redis called SSL-REDIS which can be found on github:
https://github.com/bbroerman30/ssl-redis 2.6ish
https://github.com/tritondigital/ssl-redis 2.4ish
With this and modifying the Jedis Java client, SSL authentication could be achieved.
I'm using jedis for simple key-value data store... My code is as follows
private static final String HOST = "50.30.35.9";
private static final int PORT = 2863;
Jedis jedis = new Jedis(HOST, PORT);
try {
jedis.set("foo", "bar");
jedis.get("foo");
} catch (Exception e) {
System.err.println("Unable to connect to Redis");
e.printStackTrace();
}
When i test redis server with jedis client, i got the following
exception.
redis.clients.jedis.exceptions.JedisConnectionException:
java.net.SocketTimeoutException: connect timed out
at redis.clients.jedis.Connection.connect(Connection.java:124)
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java: 54)
at redis.clients.jedis.Connection.sendCommand(Connection.java: 77)
at redis.clients.jedis.BinaryClient.set(BinaryClient.java:71)
at redis.clients.jedis.Client.set(Client.java:21)
at redis.clients.jedis.Jedis.set(Jedis.java:48)
Can someone please help?
The question is:
Why you are getting a SocketTimeoutException
The answer is:
Because you cannot connect to the host and port you have in your code, since you cannot telnet 50.30.35.9 2863 to it.
The follow up question is:
I am unable to reach the Redis instance hosted by Redis4you even via telnet, even though the Redis4you portal claims it is running. How do I debug the root cause of this issue?
The answer is:
Contact Redis4You, and find out. From Redis4You contact page:
=> You can contact us with questions or for support issues at following email: redis (#) e-nick . org In case of problems with status of the servers, please follow our Twitter account: redis4you. You can also post comments and questions there, but email will be easier and faster way.
Also looking at their Twitter feed on October 17th they "have upgraded the system to newly released Redis 2.4.0. You need to stop and then start your instance so the upgrade to happen for you."
if ubuntu：
sudo ufw allow 2863/tcp
or close your firewall