We Keep Coding

How to get the implicit port from an embedded Neo4j database?

Given one starts an embedded database:
BoltConnector boltConnector = new BoltConnector("bolt");
new GraphDatabaseFactory()
.newEmbeddedDatabaseBuilder(Files.createTempDirectory("part-relasjon").toFile())
.setConfig(boltConnector.enabled, "true")
.setConfig(boltConnector.type, "BOLT")
.setConfig(boltConnector.listen_address, "localhost:0")
.newGraphDatabase();
how do I get hold of the port to which the database is bound? I want to create a Driver to connect to the database:
GraphDatabase.driver("bolt://localhost:?")
I am doing this to run integration tests in a specific application profile. Right now, I find a free port manually and bind the server and driver to it, but I am looking for a way of extracting it from the randomly selected port as it seems like the cleaner solution. I had a look into the Neo4jRule from the test package to see how it is done there but the latter bootstraps a server what is much more complex than the above code which is why I want to avoid it.

This is an example from the documentation here : https://neo4j.com/docs/java-reference/3.3/#tutorials-java-embedded-bolt
GraphDatabaseSettings.BoltConnector bolt = GraphDatabaseSettings.boltConnector( "0" );
GraphDatabaseService graphDb = new GraphDatabaseFactory()
.newEmbeddedDatabaseBuilder( DB_PATH )
.setConfig( bolt.type, "BOLT" )
.setConfig( bolt.enabled, "true" )
.setConfig( bolt.address, "localhost:7687" )
.newGraphDatabase();
As you can see, here, you define the port (here 7687, the default one).
You can find a free port by yourself in java via something like that :
InetSocketAddress address = new InetSocketAddress( 'localhost', 7687 );
new ServerSocket( addres.getPort(), 100, address.getAddress() ).close();
If you got an exception, port is not free.
It seems that since the version 3.3, you can define to 0 the port of any connector, and Neo4j will find a free port.
To know the port, you can try this java code (comes from here : https://github.com/neo4j/neo4j/blob/3.4/community/neo4j-harness/src/main/java/org/neo4j/harness/internal/InProcessServerControls.java):
server.getDependencyResolver().resolveDependency( ConnectorPortRegister.class ).getLocalAddress( "bolt" ).getPort();
Cheers

JUnit test with Embedded tomcat server , how to specify automatic ports for both http and https connectors?

Description
I have made a JUnit test that focus on trying to test a call to a SOAP web service.
I am using an embedded tomcat server for my test in order to run my test with a mock server.
I am also using both http and https connectors.
I need to use automatic ports for both these connectors because the test is running on a Jenkins server and i can't just use port 443 or 8443 as they are already taken.
I understand that using the port 0 as standard port will result in tomcat using automatic port allocation but I can't manage to use it with both connectors.
Expected behavior
I'd like to use automatic port allocation also for my custom ssl connector.
Is it possible to do so in some way ?
Sample code
Here is the code for my tomcat instance :
#Before
public void setup() throws Throwable {
File tomcatWorkingDir = new File(mWorkingDir);
//Empty the target/tomcat-working-dir directory if it exist
//Create the directory otherwise
if(tomcatWorkingDir.exists() && tomcatWorkingDir.isDirectory()){
LOGGER.info("cleaning tomcat-working-dir directory");
FileUtils.cleanDirectory(new File(mWorkingDir));
} else {
LOGGER.info("create tomcat-working-dir directory");
tomcatWorkingDir.mkdir();
}
LOGGER.info("disabling ssl certification validation");
//Disable JVM ssl sockets connection
disableJVMCertificate();
//Add server certificate
createServerCertificate();
//Custom SSL Connector
Connector SSLConnector = getSSLConnector();
mTomcat = new Tomcat();
//Standard http startup port
mTomcat.setPort(0);
//Set up base directory
//Otherwise, tomcat would use the current directory
mTomcat.setBaseDir(mWorkingDir);
LOGGER.info("setting the ssl connector in TOMCAT");
Service service = mTomcat.getService();
service.addConnector(SSLConnector);
//Redirect current port
Connector defaultConnector = mTomcat.getConnector();
defaultConnector.setRedirectPort(SERVER_HTTPS_PORT);
//Configure the way WAR are managed by the engine
mTomcat.getHost().setAutoDeploy(true);
mTomcat.getHost().setDeployOnStartup(true);
//Add mock server into our webApp
String servletName = "/server";
File webApp = new File(mWorkingDir,"../../../ws-mock-server/src/main/webapp");
mTomcat.addWebapp(mTomcat.getHost(), servletName, webApp.getAbsolutePath());
//start tomcat
LOGGER.info("starting TOMCAT");
mTomcat.start();
}
and here for my custom ssl connector.
private static Connector getSSLConnector(){
Connector connector = new Connector();
connector.setPort(SERVER_HTTPS_PORT);
connector.setSecure(true);
//Http protocol Http11AprProtocol
connector.setAttribute("protocol", "org.apache.coyote.http11.Http11AprProtocol");
//Maximum threads allowedd on this instance of tomcat
connector.setAttribute("maxThreads","200");
connector.setAttribute("SSLEnabled", true);
//No client Authentification is required in order to connect
connector.setAttribute("clientAuth", false);
//SSL TLSv1 protocol
connector.setAttribute("sslProtocol","TLS");
//Ciphers configuration describing how server will encrypt his messages
//A common cipher suite need to exist between server and client in an ssl
//communication in order for the handshake to succeed
connector.setAttribute("ciphers","TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA");
LOGGER.info("setting keystore file");
//Here an absolute file path is needed in order to properly set up the keystore attribute
connector.setAttribute("keystoreFile",new File(".").getAbsolutePath().replace("\\", "/")+"/"+mWorkingDir+"/server.jks");
LOGGER.info("setting keystore pass");
connector.setAttribute("keystorePass","changeit");
return connector;
}

I have two solutions for this problem:
Select SSL port manually
The ServerSocket(0) constructor automatically selects a free port. The Tomcat uses this method also.
try (ServerSocket testSocket = new ServerSocket(0)) {
int randomFreePort = testSocket.getLocalPort();
sslConnector.setPort(randomFreePort);
defaultConnector.setRedirectPort( randomFreePort);
} // At this point the testSocket.close() called
tomcat.start();
I know, there is a probability, that an another process allocates the same port between the testSocket.close() and tomcat.start(), but you can detect this situation, with LifecycleState.FAILED.equals(sslConnector.getState()) test.
Use lifecycle listeners
Tomcat connectors are lifecycle aware, so you will be notified on 'before_init' and 'after_init' events. Tomcat initializes the connectors in the order as you added them to the Service.
Add the ssl connector.
Add an http connector. (That will be the 'default' connector. Don't call the mTomcat.getConnector() because it gets the first or creates a new connector. )
When the ssl connector initialization complete, you can get the chosen port with getLocalPort() call.
Before the http connector initialization, call the setRedirectPort
Full example:
Tomcat mTomcat = new Tomcat();
Connector sslConnector = getSSLConnector();
mTomcat.getService().addConnector(sslConnector);
Connector defaultConnector = new Connector();
defaultConnector.setPort(0);
mTomcat.getService().addConnector(defaultConnector);
// Do the rest of the Tomcat setup
AtomicInteger sslPort = new AtomicInteger();
sslConnector.addLifecycleListener(event->{
if( "after_init".equals(event.getType()) )
sslPort.set(sslConnector.getLocalPort());
});
defaultConnector.addLifecycleListener(event->{
if( "before_init".equals(event.getType()) )
defaultConnector.setRedirectPort(sslPort.get());
});
mTomcat.start();

I haven't tried it but from the code it looks like
You can setRedirectPort after server was started
You can use Connector.getLocalPort to get actual port
So I think you could try to add something like
mTomcat.start(); // <-- your existing code
defaultConnector.setRedirectPort(SSLConnector.getLocalPort())

How does jconsole discover the JMX enabled processes on local system?

When I run jconsole it shows me a list of Java processes:
I could then connect to one of these and see its MBeans. How does it discover the JMX processes? How can I do this in a program?

This article shows how to do it using Attach API from JDK tools.jar

Replying since I had this question too and got an answer.
There is a JPS program in JDK which shows java processes.
I am not 100% sure (don't want to dive deep into jconsole code) but 99% sure jconsole uses the same mechanism as jps:
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8u40-b25/sun/tools/jps/Jps.java?av=f
HostIdentifier hostId = arguments.hostId();
MonitoredHost monitoredHost = MonitoredHost.getMonitoredHost(hostId);
// get the set active JVMs on the specified host.
Set<Integer> jvms = monitoredHost.activeVms();
These class are part of tools.jar, you need to include it in the project's classpath.
If we go deeper (I do not expose all the intermediate steps) - finally we'll know that active VMs list is populated from hsperfdata files in temporary directories:
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8u40-b25/sun/jvmstat/perfdata/monitor/protocol/local/PerfDataFile.java
Here is a link to know more:
java hsperfdata directory
At last, here is a code snippet allowing you to get the java processes ids:
sun.jvmstat.monitor.MonitoredHost host = sun.jvmstat.monitor.MonitoredHost.getMonitoredHost(new sun.jvmstat.monitor.HostIdentifier((String) null));
System.out.println(host.activeVms());
P.S.
Then you can use Attach API (as kostya mentioned) to discover the rest of needed things.

After you register the MBean in your application, like this:
MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
//register the MBean
ObjectMbean mBean = new ObjectMbean();
ObjectName name = new ObjectName("com.gd.eventfiltering.jmx:type=SystemConfig");
mbs.registerMBean(mBean, name);
Then you can call your MBean like this:
JMXServiceURL url = new JMXServiceURL(JMX_PATH);
JMXConnector jmxc = JMXConnectorFactory.connect(url, null);
MBeanServerConnection mbsc = jmxc.getMBeanServerConnection();
ObjectName mbeanName = new ObjectName(JMX_OBJECT);
IObjectMbean mbeanProxy = JMX.newMBeanProxy(mbsc, mbeanName,IObjectMbean.class, false);
//call the method
List<EventType> filters = mbeanProxy.methodFromYourMBean();
jmxc.close();

JMX service url cannot connect remote jvm

I wrote JVM monitoring with JMX. I connect MBean sever connection with ths following url
service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmi
It worked using localhost but I would to connect other machine JVM and I also used this;
service:jmx:rmi:///jndi/rmi://OTHER_SERVER_PORT:9999/jmxrmi
It doesn't work.I goggling about this and some example tell to add the following JVM options to target machine JVM.
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9999
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
So where to add the above JVM options? And how to connect other machine JVM using JMX service url? This is my sample class
public class JMXRemote {
public static void main(String[] args) throws Exception {
JMXServiceURL url = new JMXServiceURL(
"service:jmx:rmi:///jndi/rmi://OTHER_PC_PORT:9999/jmxrmi");
JMXConnector c = JMXConnectorFactory.connect(url);
MBeanServerConnection mbsc = c.getMBeanServerConnection();
Object o = mbsc.getAttribute(new ObjectName("java.lang:type=Memory"),
"HeapMemoryUsage");
CompositeData cd = (CompositeData) o;
System.out.println(cd.get("committed"));
}
}

How do I programmatically dump JMX data?

I want to be able to log all JMX data accessible via jconsole. Is there a way to do this programmatically? I'm building a form of logging of the system, and I want to create intervaled data viewable with some tool similar to jconsole.
How would I go about doing this?

java.lang.management.ManagementFactory gives you access to JMX data.
i.g.
List<MemoryPoolMXBean> memPoolBeans = ManagementFactory.getMemoryPoolMXBeans();
for (MemoryPoolMXBean mpb : memPoolBeans) {
System.out.println("Memory Pool: " + mpb.getName());
}
Some samples are available at SO query: [java]+managementfactory
A good read: https://www.ibm.com/developerworks/library/j-jtp09196/index.html
For full implementation connecting to a remote VM:
Map<String,String[]> env = new HashMap<String, String[]>();
env.put( JMXConnector.CREDENTIALS, new String[]{"user","pass"} );
JMXServiceURL address = new JMXServiceURL("service:rmi:///jndi/rmi://host:port/jmxrmi");
JMXConnector connector = JMXConnectorFactory.connect(address,env);
MBeanServerConnection mbs = connector.getMBeanServerConnection();
//get all mbeans
Set<ObjectInstance> beans = mbs.queryMBeans(null,null);
for( ObjectInstance instance : beans )
{
MBeanInfo info = mbs.getMBeanInfo( instance.getObjectName() );
}
From the info, you can query object names and attributes as desired.

I used this command line JMX client as a starting point when I wanted to do the same thing.