Show my code
conf.set( "mongo.input.uri" , "mongodb://127.0.0.1/stackoverflow.mrtest" );
conf.set( "mongo.output.uri" , "mongodb://127.0.0.1/stackoverflow.mrtest_out2" );
the code runs without error when the host is localhost or 127.0.0.1. But when the host changed to my ip wlan0 192.168.1.102, it returned the following error
Cluster created with settings {hosts=[192.168.1.102:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=500}
Exception in monitor thread while connecting to server 192.168.1.102:27017
com.mongodb.MongoSocketOpenException: Exception opening socket
at com.mongodb.connection.SocketStream.open(SocketStream.java:63)
at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:114)
at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:127)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:50)
at com.mongodb.connection.SocketStream.open(SocketStream.java:58)
... 3 more
I have open the port 27017.
sudo iptables -A INPUT -ptcp --dport 27017 -j ACCEPT
My OS is Ubuntu 14.04.
How should I fix it? Thank you!
By default MongoDB only binds to the loopback interface which makes it only accessible from localhost. To change that you need to edit this line in mongod.conf file;
# /etc/mongod.conf
# Listen to local interface only. Comment out to listen on all interfaces.
bind_ip = 127.0.0.1
you can change it to bind_ip = 127.0.0.1,192.168.1.102 to allow LAN and local connections or you can remove or comment out that line to allow all connections.
For more info : MongoDB – Allow remote access
There could be several reasons of it , which in short can be concluded as Your Application is unable to Communicate mongoDB service
1.Check your MongoDB using the same IP configured in your application.yml file,
If not then configure the same used by MongoDB:
spring:
profiles:
active: dev
---
spring:
profiles: dev
data:
mongodb:
host: localhost
port: 27017
Here i assumed my mongo running on localhost, and port 27017, so i configured accordingly.
Check whether your MongoDB service up and running , How to check ?
Execute following command in your terminal
sudo service mongodb status
<pre><code>
● mongodb.service - An object/document-oriented database
Loaded: loaded (/lib/systemd/system/mongodb.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2020-07-03 20:10:15 IST; 1min 54s ago
Docs: man:mongod(1)
Main PID: 14305 (mongod)
Tasks: 23 (limit: 4915)
CGroup: /system.slice/mongodb.service
└─14305 /usr/bin/mongod --unixSocketPrefix=/run/mongodb --config /etc/mongodb.conf</pre></code>
If Status not visible as active and running, you need to start/restart the service
sudo service mongodb restart
I was able to determine that it was an issue with the bind parameter in the /etc/mongod.conf. Instead of commenting it out I set it to 0.0.0.0 to all for remote access.
# network interfaces
net:
port: 27017
bindIp: 0.0.0.0
Exception: com.mongodb.MongoSocketOpenException: Exception opening socket
Solution:
Verify whether you have started "mongo daemon" or not.
Windows Terminal: mongod.exe
Linux Termina: mongod
If you're using Spring Boot and are following the Quick Start, make sure you put this configuration in your application.properties
spring.data.mongodb.uri=[YOUR_URI]
Related
I have a local spring boot application that connects to local MySQL and it works fine.
For connection I use the following property:
spring.datasource.url=jdbc:mysql://localhost:3306/pitstop?useSSL=false&useUnicode=true&characterEncoding=utf8&serverTimezone=UTC
I would like to put my app in docker and try to connect to local DB.
So I need to modify MySQL url.
I used this command to obtain local IP
ip route show | grep "default" | awk '{print $3}' the result is 192.168.1.1. I modify my url like this
spring.datasource.url=jdbc:mysql://192.168.1.1:3306/pitstop?useSSL=false&useUnicode=true&characterEncoding=utf8&serverTimezone=UTC
and try to start docker container with my app by command docker run -p 9001:9001 --network=bizon4ik --rm bizon4ik/mycontainer the result is the exception:
Caused by: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
I had tried to find another IP. I used ip addr show command and found the next record:
3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 30:52:cb:db:8d:e0 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.102/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp3s0
So based on it I took 192.168.1.102 and modify my url but the result is the same exception.
To be sure that founded IPs are correct I run new one container with pure ubuntu docker run -it --rm --network=bizon4ik ubuntu and checked the mentioned above IP:
root#268c4d544328:/# nmap -p 3306 192.168.1.1
Starting Nmap 7.60 ( https://nmap.org ) at 2019-07-28 17:28 UTC
Nmap scan report for 192.168.1.1
Host is up (0.053s latency).
PORT STATE SERVICE
3306/tcp filtered mysql
root#268c4d544328:/# nmap -p 3306 192.168.1.102
Starting Nmap 7.60 ( https://nmap.org ) at 2019-07-28 15:56 UTC
Nmap scan report for my-host (192.168.1.102)
Host is up (0.000088s latency).
PORT STATE SERVICE
3306/tcp closed mysql
So looks fine, the ubuntu container can ping my DB. Do you have any ideas on why I cannot connect through the app to DB?
NB:
I checked in DB SHOW GRANTS; the result is GRANT ALL PRIVILEGES ON *.* TO 'root'#'%' WITH GRANT OPTION
I also checked /etc/mysql/my.cnf file. It has only these records:
!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/mysql.conf.d/
I don't have ~/.my.cnf
The problem was in bind-address for MySQL. I looked it in the /etc/mysql/my.cnf however, the right place is /etc/mysql/mysql.conf.d/mysqld.cnf
I want to connect to HBase running in standalone in a docker, using Java and the HBase API
I use this code to connect :
Configuration config = HBaseConfiguration.create();
config.set("hbase.zookeeper.quorum", "163.172.142.199");
config.set("hbase.zookeeper.property.clientPort","2181");
HBaseAdmin.checkHBaseAvailable(config);
Here is my /etc/hosts file
127.0.0.1 localhost
XXX.XXX.XXX.XXX hbase-srv
Here is the /etc/hosts file from my docker (named hbase-srv)
XXX.XXX.XXX.XXX hbase-srv
With this configuration, I get a connection refused error :
INFO | Initiating client connection, connectString=163.172.142.199:2181 sessionTimeout=90000 watcher=hconnection-0x6aba2b860x0, quorum=163.172.142.199:2181, baseZNode=/hbase
INFO | Opening socket connection to server 163.172.142.199/163.172.142.199:2181. Will not attempt to authenticate using SASL (unknown error)
INFO | Socket connection established to 163.172.142.199/163.172.142.199:2181, initiating session
INFO | Session establishment complete on server 163.172.142.199/163.172.142.199:2181, sessionid = 0x15602f8d8dc0002, negotiated timeout = 40000
INFO | Closing zookeeper sessionid=0x15602f8d8dc0002
INFO | Session: 0x15602f8d8dc0002 closed
INFO | EventThread shut down
org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.net.ConnectException: Connection refused
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1560)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1580)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1737)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.isMasterRunning(ConnectionManager.java:948)
at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:3159)
at hbase.Benchmark.main(Benchmark.java:26)
However, if I remove the lines XXX.XXX.XXX.XXX hbase-srv from both /etc/hosts files I get the error unknown host : hbase-srv
I have also checked, I can successfully telnet to my hbase docker on the client port.
On the docker, all the ports used by HBase are opened and binded to the same number (60000 on 60000, 2181 on 2181, etc).
I also wanted to add that all was fine when I used this configuration on localhost.
If you can't give me an answer to my problem, could you at least give me a procedure to deploy a standalone hbase on a docker.
UPDATE : Here is my Docker file
FROM java:openjdk-8
ADD hbase-1.2.1 /hbase-1.2.1
WORKDIR /hbase-1.2.1
# ZooKeeper
EXPOSE 2181
# HMaster
EXPOSE 60000
# HMaster Web
EXPOSE 60010
# RegionServer
EXPOSE 60020
# RegionServer Web
EXPOSE 60030
EXPOSE 16010
RUN chmod 755 /hbase-1.2.1/bin/start-hbase.sh
CMD ["/hbase-1.2.1/bin/start-hbase.sh"]
My HBase shell is working, I also tried to open the port using iptables for tcp and udp but still the same problem
There are two problems with your Dockerfile:
use hbase master start instead of start-hbase.sh
regionserver is actually not running on 60020
The 2nd problem is not so easy to solve. If run hbase standalone with version >= 1.2.0 (not sure, I'm running 1.2.0), hbase will use ephemeral port instead of the default port or the port you provide in hbase-site.xml which makes it very hard to provide hbase service in docker using the original version.
I add a property named hbase.localcluster.port.ephemeral and managed to build a standalone hbase in docker, which you can reference here.
i have the bin for spark-1.6.0-bin-hadoop2.6 im having issue trying to connect the slave to the master
so far i have tried(on ubuntu 14.04 live usb):
apt-get purge and install openssh-client and server on both systems
I have stated explicitly the ip address of the master in the spark url for the worker
spark://< master ip>:7077 and also tried changing the SPARK_MASTER_IP in /conf/spark-env.sh the worker executes but the log has the following error
im assuming there must be some ssh setup involved but i have tried ssh-keygen and ssh-copy-id # it also doesnt give any reuslts
16/02/22 07:49:16 INFO Worker: Connecting to master 192.168.0.208:7077...
16/02/22 07:49:16 WARN Worker: Failed to connect to master 192.168.0.208:7077
java.io.IOException: Failed to connect to /192.168.0.208:7077
at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:216)
at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:167)
at org.apache.spark.rpc.netty.NettyRpcEnv.createClient(NettyRpcEnv.scala:200)
at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:187)
at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:183)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection refused: /192.168.0.208:7077
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:740)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
... 1 more
16/02/22 07:49:27 INFO Worker: Retrying connection to master (attempt # 2)
i am however able to open the master webUI by typing :8080 on my browser . i am also able to access the webUI of the slave from the master . im almost at the point of giving in so please helllppp.
Make sure that each master and worker has a firewall exception to allow connections for all other workers and masters.
Here's a simplified example from one of our master machines (master0):
$iptables -L
...
ACCEPT all -- worker0.company.com master0.company.com
ACCEPT all -- worker1.company.com master0.company.com
ACCEPT all -- master1.company.com master0.company.com
...
Of course you can also use IPs instead of hostnames.
I recently installed the H2 database in Windows7. To start the H2 command line, I faced the following problem. There is already a process running on 8082 port. To resolve this I need to stop this process.
The Web Console server could not be started. Possible cause: another server is a
lready running at http://169.254.216.99:8082
Root cause: Exception opening port "8082" (port may be in use), cause: "java.net
.BindException: Address already in use: JVM_Bind" [90061-184]
Exception in thread "main" org.h2.jdbc.JdbcSQLException: Exception opening port
"8082" (port may be in use), cause: "java.net.BindException: Address already in
use: JVM_Bind" [90061-184]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
at org.h2.message.DbException.get(DbException.java:168)
at org.h2.util.NetUtils.createServerSocketTry(NetUtils.java:194)
at org.h2.util.NetUtils.createServerSocket(NetUtils.java:160)
at org.h2.server.web.WebServer.start(WebServer.java:357)
at org.h2.tools.Server.start(Server.java:474)
at org.h2.tools.Console.runTool(Console.java:231)
at org.h2.tools.Console.main(Console.java:99)
Caused by: java.net.BindException: Address already in use: JVM_Bind
at java.net.DualStackPlainSocketImpl.bind0(Native Method)
at java.net.DualStackPlainSocketImpl.socketBind(Unknown Source)
at java.net.AbstractPlainSocketImpl.bind(Unknown Source)
at java.net.PlainSocketImpl.bind(Unknown Source)
at java.net.ServerSocket.bind(Unknown Source)
at java.net.ServerSocket.<init>(Unknown Source)
at java.net.ServerSocket.<init>(Unknown Source)
at org.h2.util.NetUtils.createServerSocketTry(NetUtils.java:190)
... 5 more
Press any key to continue . . .
Now I checked which process is running on 8082 port.
netstat -a -n -o | find "8082"
TCP 0.0.0.0:8082 0.0.0.0:0 LISTENING 4472
TCP [::]:8082 [::]:0 LISTENING 4472
I find it that the javaw.exe process is running on port 8082.
tasklist
javaw.exe 4472
I found out few info about java/javaw/javaws from this link Java/Javaw/Javaws.
Now my question is if I closed the 'javaw.exe' process, will I face any further problem to run other applications/program ?
Or can I run the javaw.exe on different port or change H2 port (vice versa) ? How?
EDIT
I know to kill the process -(in command line)
Taskkill /PID 4472 /F
javaw is a similar to java, but it does not shown the black ("DOS") window. You can use the process monitor to find out which process is it, but it can be just another H2 instance running...
In any case, if you want to run your H2 you should either
Kill the other process
Choose another port
I assumed that you had H2 server start already (somehow), and the first server used the 8082 port. So when you try to start another H2 server, it noticed the port was already taken by the first H2 server.
You can check if H2 server already started or not by accessing http://localhost:8082/.
If you would like to change H2 port anyway, you can change it by editing a file called .h2.server.properties which is usually placed at C:\Users\[username] or C:\Documents and Settings\[username].
If you can't find the file .h2.server.properties there, you can create one at C:\Users\[username] by simply copying the code below.
#H2 Server Properties
#Wed Aug 01 16:26:15 JST 2018
0=Generic JNDI Data Source|javax.naming.InitialContext|java\:comp/env/jdbc/Test|sa
1=Generic Firebird Server|org.firebirdsql.jdbc.FBDriver|jdbc\:firebirdsql\:localhost\:c\:/temp/firebird/test|sysdba
10=Generic Derby (Server)|org.apache.derby.jdbc.ClientDriver|jdbc\:derby\://localhost\:1527/test;create\=true|sa
11=Generic Derby (Embedded)|org.apache.derby.jdbc.EmbeddedDriver|jdbc\:derby\:test;create\=true|sa
12=Generic H2 (Server)|org.h2.Driver|jdbc\:h2\:tcp\://localhost/~/test|sa
13=Generic H2 (Embedded)|org.h2.Driver|jdbc\:h2\:~/test|sa
2=Generic SQLite|org.sqlite.JDBC|jdbc\:sqlite\:test|sa
3=Generic DB2|COM.ibm.db2.jdbc.net.DB2Driver|jdbc\:db2\://localhost/test|
4=Generic Oracle|oracle.jdbc.driver.OracleDriver|jdbc\:oracle\:thin\:#localhost\:1521\:XE|sa
5=Generic MS SQL Server 2000|com.microsoft.jdbc.sqlserver.SQLServerDriver|jdbc\:microsoft\:sqlserver\://localhost\:1433;DatabaseName\=sqlexpress|sa
6=Generic MS SQL Server 2005|com.microsoft.sqlserver.jdbc.SQLServerDriver|jdbc\:sqlserver\://localhost;DatabaseName\=test|sa
7=Generic PostgreSQL|org.postgresql.Driver|jdbc\:postgresql\:test|
8=Generic MySQL|com.mysql.jdbc.Driver|jdbc\:mysql\://localhost\:3306/test|
9=Generic HSQLDB|org.hsqldb.jdbcDriver|jdbc\:hsqldb\:test;hsqldb.default_table_type\=cached|sa
webAllowOthers=true
webPort=8082
webSSL=false
If you would like to change H2 port, edit webPort=8082.
I am using the following command to create a database using windows command and connect to it but I am getting java.net.SocketException: Unrecognized Windows Sockets error: 0: JVM_Bind error.
Command used to create a database named xdb and connect to it:
java -cp ./lib/hsqldb.jar org.hsqldb.Server -database.0 file:mydb -dbname.0 xdb
Complete error:
[Server#83cc67]: [Thread[main,5,main]]: checkRunning(false) entered
[Server#83cc67]: [Thread[main,5,main]]: checkRunning(false) exited
[Server#83cc67]: Startup sequence initiated from main() method
[Server#83cc67]: Loaded properties from [C:\Home\hsqldb\server.properties]
[Server#83cc67]: Initiating startup sequence...
[Server#83cc67]: [Thread[HSQLDB Server #83cc67,5,main]]: run()/openServerSocket(
):
java.net.SocketException: Unrecognized Windows Sockets error: 0: JVM_Bind
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:365)
at java.net.ServerSocket.bind(ServerSocket.java:319)
at java.net.ServerSocket.<init>(ServerSocket.java:185)
at java.net.ServerSocket.<init>(ServerSocket.java:97)
at org.hsqldb.HsqlSocketFactory.createServerSocket(Unknown Source)
at org.hsqldb.Server.openServerSocket(Unknown Source)
at org.hsqldb.Server.run(Unknown Source)
at org.hsqldb.Server.access$000(Unknown Source)
at org.hsqldb.Server$ServerThread.run(Unknown Source)
[Server#83cc67]: Initiating shutdown sequence...
[Server#83cc67]: Shutdown sequence completed in 6 ms.
[Server#83cc67]: 2012-05-18 01:31:59.184 SHUTDOWN : System.exit() is called next
Could someone help me understand why am I getting this error and how to solve it?
Thanks
The default port for hsqldb is 9001
Run netstat -an check to see if there is something is LISTENING on port 9001
netstat -an | grep LISTENING to check for all servers listening for incoming connections
netstat -an | grep 9001 to check for a specific port number.
If there is something already there then the new of hsqldb that you are trying to start will fail to bind a socket to the 9001 port.
On Windows 7 you can run TCPView to see what process is currently listening on the "overcrowded" port. Then it's a matter of deciding to terminate that process which is using 9001 or reconfiguring hsqldb and your client application to use a different (unused) port.
It is possible to change the port that hsqldb listens on using the --port XXXX, where XXXX is the new port number.
Also from the java -cp ./lib/hsqldb.jar org.hsqldb.Server --help output...
The server looks for a 'server.properties' file in the current directory and loads properties from it if it exists. Command line options override those loaded from the 'server.properties' file.
There are other possible causes of this error so it would be useful to know what operating system the hsqldb is running on.
Failure to bind to a socket is a problem that can afflict any server application so you can review the answers provided for other server software that return this error such as the question asked about JBOSS here ...
java.net.SocketException: Unrecognized Windows Sockets error: 0: JVM_Bind (JBOSS)
It looks like you try to bind to port 0 and it doesn't exist. Try to config a different port