Using Blazegraph's `NanoSparqlServer`, how to clear the graph between tests?

Using Blazegraph's `NanoSparqlServer`, how to clear the graph between tests? - java

I am using Blazegraph's 1.5.3 version of their Bigdata DB (now rebranded as Blazegraph). I have a service that acts as a Gateway, implementing a bunch of persistence-layer methods. Now I'm writing unit tests for those methods. I'm using the embedded setup with Jetty. My setup code is below:
int port = 0; // random port
String namespace = "kb";
int queryThreadPoolSize = ConfigParams.DEFAULT_QUERY_THREAD_POOL_SIZE;
boolean forceOverflow = false;
String servletContextListenerClass = ConfigParams.DEFAULT_SERVLET_CONTEXT_LISTENER_CLASS;
System.setProperty(SystemProperties.JETTY_XML, "jetty.xml");
String propertyFile = "RWStore.properties";
System.setProperty(SystemProperties.BIGDATA_PROPERTY_FILE, propertyFile);
final Map<String, String> initParams = new LinkedHashMap<>();
initParams.put("propertyFile", propertyFile);
initParams.put("namespace", namespace);
initParams.put("queryThreadPoolSize", Integer.toString(queryThreadPoolSize));
initParams.put("forceOverflow", Boolean.toString(forceOverflow));
initParams.put("servletContextListenerClass", servletContextListenerClass);
sparqlServer = NanoSparqlServer.newInstance(port, journal, initParams);
LOGGER.info("Waiting for NanoSparqlServer to start...");
NanoSparqlServer.awaitServerStart(sparqlServer);
serverUrl = sparqlServer.getURI().toString();
LOGGER.info("NanoSparqlServer started on: " + serverUrl + '\n');
I am using the default jetty.xml configuration from the com.bigdata 1.5.3 jar:
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure.dtd">
<!-- See http://www.eclipse.org/jetty/documentation/current/ -->
<!-- See http://wiki.eclipse.org/Jetty/Reference/jetty.xml_syntax -->
<Configure id="Server" class="org.eclipse.jetty.server.Server">
<!-- =========================================================== -->
<!-- Configure the Server Thread Pool. -->
<!-- The server holds a common thread pool which is used by -->
<!-- default as the executor used by all connectors and servlet -->
<!-- dispatches. -->
<!-- -->
<!-- Configuring a fixed thread pool is vital to controlling the -->
<!-- maximal memory footprint of the server and is a key tuning -->
<!-- parameter for tuning. In an application that rarely blocks -->
<!-- then maximal threads may be close to the number of 5*CPUs. -->
<!-- In an application that frequently blocks, then maximal -->
<!-- threads should be set as high as possible given the memory -->
<!-- available. -->
<!-- -->
<!-- Consult the javadoc of o.e.j.util.thread.QueuedThreadPool -->
<!-- for all configuration that may be set here. -->
<!-- =========================================================== -->
<Arg name="threadpool"><New id="threadpool" class="org.eclipse.jetty.util.thread.QueuedThreadPool"/></Arg>
<Get name="ThreadPool">
<Set name="minThreads" type="int"><Property name="jetty.threads.min" default="10"/></Set>
<Set name="maxThreads" type="int"><Property name="jetty.threads.max" default="64"/></Set>
<Set name="idleTimeout" type="int"><Property name="jetty.threads.timeout" default="60000"/></Set>
<Set name="detailedDump">false</Set>
</Get>
<!-- =========================================================== -->
<!-- Get the platform mbean server -->
<!-- =========================================================== -->
<Call id="MBeanServer" class="java.lang.management.ManagementFactory"
name="getPlatformMBeanServer" />
<!-- =========================================================== -->
<!-- Initialize the Jetty MBean container -->
<!-- =========================================================== -->
<!-- Note: This breaks CI if it is enabled
<Call name="addBean">
<Arg>
<New id="MBeanContainer" class="org.eclipse.jetty.jmx.MBeanContainer">
<Arg>
<Ref refid="MBeanServer" />
</Arg>
</New>
</Arg>
</Call>-->
<!-- Add the static log to the MBean server.
<Call name="addBean">
<Arg>
<New class="org.eclipse.jetty.util.log.Log" />
</Arg>
</Call>-->
<!-- For remote MBean access (optional)
<New id="ConnectorServer" class="org.eclipse.jetty.jmx.ConnectorServer">
<Arg>
<New class="javax.management.remote.JMXServiceURL">
<Arg type="java.lang.String">rmi</Arg>
<Arg type="java.lang.String" />
<Arg type="java.lang.Integer"><SystemProperty name="jetty.jmxrmiport" default="1090"/></Arg>
<Arg type="java.lang.String">/jndi/rmi://<SystemProperty name="jetty.jmxrmihost" default="localhost"/>:<SystemProperty name="jetty.jmxrmiport" default="1099"/>/jmxrmi</Arg>
</New>
</Arg>
<Arg>org.eclipse.jetty.jmx:name=rmiconnectorserver</Arg>
<Call name="start" />
</New>-->
<!-- =========================================================== -->
<!-- Http Configuration. -->
<!-- This is a common configuration instance used by all -->
<!-- connectors that can carry HTTP semantics (HTTP, HTTPS, SPDY)-->
<!-- It configures the non wire protocol aspects of the HTTP -->
<!-- semantic. -->
<!-- -->
<!-- Consult the javadoc of o.e.j.server.HttpConfiguration -->
<!-- for all configuration that may be set here. -->
<!-- =========================================================== -->
<New id="httpConfig" class="org.eclipse.jetty.server.HttpConfiguration">
<Set name="secureScheme">https</Set>
<Set name="securePort"><Property name="jetty.secure.port" default="8443" /></Set>
<Set name="outputBufferSize"><Property name="jetty.output.buffer.size" default="32768" /></Set>
<Set name="requestHeaderSize"><Property name="jetty.request.header.size" default="8192" /></Set>
<Set name="responseHeaderSize"><Property name="jetty.response.header.size" default="8192" /></Set>
<Set name="sendServerVersion"><Property name="jetty.send.server.version" default="true" /></Set>
<Set name="sendDateHeader"><Property name="jetty.send.date.header" default="false" /></Set>
<Set name="headerCacheSize">512</Set>
<!-- Uncomment to enable handling of X-Forwarded- style headers
<Call name="addCustomizer">
<Arg><New class="org.eclipse.jetty.server.ForwardedRequestCustomizer"/></Arg>
</Call>
-->
</New>
<!-- Configure the HTTP endpoint. -->
<Call name="addConnector">
<Arg>
<New class="org.eclipse.jetty.server.ServerConnector">
<Arg name="server"><Ref refid="Server" /></Arg>
<Arg name="factories">
<Array type="org.eclipse.jetty.server.ConnectionFactory">
<Item>
<New class="org.eclipse.jetty.server.HttpConnectionFactory">
<Arg name="config"><Ref refid="httpConfig" /></Arg>
</New>
</Item>
</Array>
</Arg>
<Set name="host"><SystemProperty name="jetty.host" /></Set>
<Set name="port"><SystemProperty name="jetty.port" default="9999" /></Set>
<Set name="idleTimeout"><SystemProperty name="http.timeout" default="30000"/></Set>
</New>
</Arg>
</Call>
<!-- =========================================================== -->
<!-- Set handler Collection Structure -->
<!-- =========================================================== -->
<Set name="handler">
<New id="Handlers" class="org.eclipse.jetty.server.handler.HandlerCollection">
<Set name="handlers">
<Array type="org.eclipse.jetty.server.Handler">
<Item>
<New id="Contexts" class="org.eclipse.jetty.server.handler.ContextHandlerCollection">
<Call name="addHandler">
<Arg>
<!-- This is the redirect from root to /bigdata -->
<New id="moved" class="org.eclipse.jetty.server.handler.MovedContextHandler">
<Set name="contextPath">/</Set>
<Set name="newContextURL">/bigdata</Set>
<Set name="permanent">true</Set>
<Set name="discardPathInfo">false</Set>
<Set name="discardQuery">false</Set>
</New>
</Arg>
</Call>
<Call name="addHandler">
<Arg>
<!-- This is the bigdata web application. -->
<New id="WebAppContext" class="org.eclipse.jetty.webapp.WebAppContext">
<Set name="war"><SystemProperty name="jetty.resourceBase" default="bigdata-war/src"/></Set>
<Set name="contextPath">/bigdata</Set>
<Set name="descriptor">WEB-INF/web.xml</Set>
<Set name="parentLoaderPriority">true</Set>
<Set name="extractWAR">false</Set>
<Set name="overrideDescriptor"><SystemProperty name="jetty.overrideWebXml" default="bigdata-war/src/WEB-INF/override-web.xml"/></Set>
<Set name="maxFormContentSize">10485760</Set>
</New>
</Arg>
</Call>
</New>
</Item>
</Array>
</Set>
</New>
</Set>
<!-- =========================================================== -->
<!-- extra server options -->
<!-- =========================================================== -->
<Set name="stopAtShutdown">true</Set>
<Set name="stopTimeout">5000</Set>
<Set name="dumpAfterStart"><Property name="jetty.dump.start" default="false"/></Set>
<Set name="dumpBeforeStop"><Property name="jetty.dump.stop" default="false"/></Set>
</Configure>
... and I am using the default RWStore.properties from the same jar:
#
# Note: These options are applied when the journal and the triple store are
# first created.
##
## Journal options.
##
# The backing file. This contains all your data. You want to put this someplace
# safe. The default locator will wind up in the directory from which you start
# your servlet container.
com.bigdata.journal.AbstractJournal.createTempFile=true
# The persistence engine. Use 'Disk' for the WORM or 'DiskRW' for the RWStore.
com.bigdata.journal.AbstractJournal.bufferMode=DiskRW
# Setup for the RWStore recycler rather than session protection.
com.bigdata.service.AbstractTransactionService.minReleaseAge=1
# Enable group commit. See http://wiki.blazegraph.com/wiki/index.php/GroupCommit and BLZG-192.
#com.bigdata.journal.Journal.groupCommit=false
com.bigdata.btree.writeRetentionQueue.capacity=4000
com.bigdata.btree.BTree.branchingFactor=128
# 200M initial extent.
com.bigdata.journal.AbstractJournal.initialExtent=209715200
com.bigdata.journal.AbstractJournal.maximumExtent=209715200
##
## Setup for QUADS mode without the full text index.
##
com.bigdata.rdf.sail.truthMaintenance=false
com.bigdata.rdf.store.AbstractTripleStore.quads=true
com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false
com.bigdata.rdf.store.AbstractTripleStore.textIndex=false
com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms
# Bump up the branching factor for the lexicon indices on the default kb.
com.bigdata.namespace.kb.lex.com.bigdata.btree.BTree.branchingFactor=400
# Bump up the branching factor for the statement indices on the default kb.
com.bigdata.namespace.kb.spo.com.bigdata.btree.BTree.branchingFactor=1024
# Uncomment to enable collection of OS level performance counters. When
# collected they will be self-reported through the /counters servlet and
# the workbench "Performance" tab.
#
# com.bigdata.journal.Journal.collectPlatformStatistics=true
Using these configurations, the server starts up fine, and I can access the Web console, make queries, and interact via the BigdataGraphClient API in Java. Now I'm just trying to figure out how to clear out the graph to avoid data leakage between unit tests. I've tried the following:
Use the BigdataGraphClient Java API to remove all edges and vertices. Leaves some of these edges and vertices in place, for reasons unknown to me.
graph.getEdges.forEach(Edge::remove)
graph.getVertices.forEach(Vertex::remove)
Stop and destroy server. Leaves the journal file in place.
sparqlServer.stop(); sparqlServer.destroy();
Use a temporary journal file by setting com.bigdata.journal.AbstractJournal.createTempFile=true and commenting out com.bigdata.journal.AbstractJournal.file=bigdata.jnl. This clears the journal file, but it throws a DatasetNotFoundException after the first test.
Put the journal file in a temporary directory in /tmp/bigdata-test/bigdata.jnl and delete/recreate that directory between tests. This has the same problem as #2.
Tried to create my own Journal object and pass that in as the IndexManager parameter of the NanoSparqlServer.newInstance method. This fails due to a known issue with old Lucene dependencies. I cannot include these in my project because I am relying on the newer version of Lucene that conflicts with this one. The error thrown is the same as documented in the referenced Jira ticket.
Anybody know of a clean, reliable way to clear the graph between tests (in a tearDown method run after every test)?

Turns out I was running into another issue that made me think my first approach wasn't working. That approach works just fine. I'm leaving the question up in case someone else is wondering how to do this. I'm also open to cleaner/faster ways. If the tests insert a lot of data, iterating through all of the triples/quads and deleting them one-by-one can be slow. I'd prefer something like unlinking the files in the Journal.

Related

Enable programmatic concurrency on Jetty with WorkManager

I managed to configure a custom implementation of the CommonJ − JSR 237 Timer & WorkManager API (http://commonj.myfoo.de) as a JNDI resource on Jetty 6 and 8, but it only works in a global scope.
With this solution JNDI name of the resource is wm/WorkManager, I need it to be java:comp/env/wm/WorkManager, but due to restrictions, I cannot use java:comp/env in a global JNDI name because it's reserved to application scoped resources.
I've created a new configuration file called {jetty.home}/etc/jetty-wtm.xml and added to {jetty.home}/start.ini.
Here is jetty-wtm.xmlcontent for Jetty 6, for greater versions it's a bit different, but works too:
<!-- =============================================================== -->
<!-- Configure Server Time and Work Managers -->
<!-- =============================================================== -->
<Configure id="Server" class="org.mortbay.jetty.Server">
<New id="WorkManager" class="org.mortbay.jetty.plus.naming.Resource">
<Arg>wm/WorkManager</Arg>
<Arg>
<New class="de.myfoo.commonj.work.FooWorkManager">
<Arg>
<New id="threadPool" class="de.myfoo.commonj.util.ThreadPool">
<Arg type="int">0</Arg>
<Arg type="int">10</Arg>
<Arg type="int">2</Arg>
</New>
</Arg>
</New>
</Arg>
</New>
<New id="TimeManager" class="org.mortbay.jetty.plus.naming.Resource">
<Arg>tm/TimeManager</Arg>
<Arg>
<New class="de.myfoo.commonj.timers.FooTimerManager">
<Arg>
<New id="threadPool" class="de.myfoo.commonj.util.ThreadPool">
<Arg type="int">0</Arg>
<Arg type="int">10</Arg>
<Arg type="int">2</Arg>
</New>
</Arg>
</New>
</Arg>
</New>
</Configure>
I need to mantain standard JNDI naming java:comp/env/{RESOURCE} specifically java:comp/env/wm/MyWorkManageracross servers, but standard WEB-INF\jetty-env.xml configuration file doesn't work.
Any ideas?
UPDATE:
I've tested the jetty-env.xml local configuration file in Jetty 9 and it works as expected. It seems in versions under 9 JNDI it's not fully supported. Here is the configuration file contents:
<Configure id="wac" class="org.eclipse.jetty.webapp.WebAppContext">
<New id="WorkManager" class="org.eclipse.jetty.plus.jndi.Resource">
<Arg><Ref id="wac"/></Arg>
<Arg>wm/MyWorkManager</Arg>
<Arg>
<New class="de.myfoo.commonj.work.FooWorkManager">
<Arg>
<New id="threadPool" class="de.myfoo.commonj.util.ThreadPool">
<Arg type="int">0</Arg>
<Arg type="int">10</Arg>
<Arg type="int">2</Arg>
</New>
</Arg>
</New>
</Arg>
</New>
</Configure>

Did you check this,
Sometimes it is useful to pass configuration information to a web app at runtime that you either cannot or cannot conveniently code into
a web.xml . In such cases, you can use
org.eclipse.jetty.plus.jndi.EnvEntry, and even override an entry of
the same name in web.xml.
<New class="org.eclipse.jetty.plus.jndi.EnvEntry">
<Arg></Arg>
<Arg>mySpecialValue</Arg>
<Arg type="java.lang.Integer">4000</Arg>
<Arg type="boolean">true</Arg>
</New>
This example defines a virtual env-entry called mySpecialValue with
value 4000 that is unique within the whole JVM. It is put into JNDI at
java:comp/env/mySpecialValue for every web app deployed. Moreover,
the boolean argument indicates that this value overrides an env-entry
of the same name in web.xml. If you don't want to override, then omit
this argument, or set it to false.
For Resource in custom files inside etc, the documentation states that
Assume the following naming entry is in
$JETTY_HOME/etc/jetty-myjndi.xml:
<New id="jdbc/myds" class="org.eclipse.jetty.plus.jndi.Resource">
<Arg></Arg>
<Arg>jdbc/myds</Arg>
<Arg>
<New class="com.mysql.jdbc.jdbc2.optional.MysqlConnectionPoolDataSource">
<Set name="Url">jdbc:mysql://localhost:3306/chat</Set>
<Set name="User">root</Set>
<Set name="Password">sillyness</Set>
</New>
</Arg>
</New>
Then you can link jdbc/myds into your webapp's namespace as
java:comp/env/jdbc/myfoo by using a WEB-INF/jetty-env.xml file:
<Call class="org.eclipse.jetty.plus.jndi.NamingEntryUtil" name="bindToENC">
<Arg></Arg> <!-- scope of naming entry, ie same as first argument to your naming entry definition, in this case, null -->
<Arg>jdbc/myfoo</Arg>
<Arg>jdbc/myds</Arg>
</Call>
Note that you must use a WEB-INF/jetty-env.xml file to call the
"bindToENC" method and not a context xml file, as the latter is not
interpreted at the correct phase of the webapp's deployment to have
the java:comp/env namespace created.

Java Based Spring WebApp not Running Jetty9

I have a war project with Java Based Spring Configuration (without web.xml) that runs perfectly on mvn jetty:run but when I put inside the machines with Jetty 9 the project is not running.
The jetty 9 server runs but the application doesn`t initialize at all.
The application consists in a Queue Subscriber.
It seems like I need to have a web.xml file.
Does anyone have an ideia what I`m missing?
The xml configuration file for the App on Jetty 9 server is (edit: with the correct conf file)
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_0.dtd">
<!--
JETTY_HOME="<%=node['api']['worker']['jetty']['jetty_home']%>"
JAVA_HOME="<%=node['api']['worker']['jetty']['java_home']%>"
JETTY_CONSOLE="<%=node['api']['worker']['jetty']['jetty_console']%>"
JETTY_USER="<%=node['api']['worker']['jetty']['jetty_user']%>"
JETTY_ARGS="<%=node['api']['worker']['jetty']['jetty_args']%>"
JETTY_RUN="<%=node['api']['worker']['jetty']['jetty_run']%>"
JAVA_OPTIONS="<%=node['api']['worker']['jetty']['java_options']%>"
-->
<Configure id="Server" class="org.eclipse.jetty.server.Server">
<Get name="ThreadPool">
<Set name="minThreads" type="int"><Property name="threads.min" default="10"/></Set>
<Set name="maxThreads" type="int"><Property name="threads.max" default="200"/></Set>
<Set name="idleTimeout" type="int"><Property name="threads.timeout" default="60000"/></Set>
<Set name="detailedDump">false</Set>
</Get>
<Call name="addBean">
<Arg>
<New class="org.eclipse.jetty.util.thread.ScheduledExecutorScheduler"/>
</Arg>
</Call>
<New id="httpConfig" class="org.eclipse.jetty.server.HttpConfiguration">
<Set name="secureScheme">https</Set>
<Set name="securePort"><Property name="jetty.secure.port" default="8443" /></Set>
<Set name="outputBufferSize"><Property name="jetty.output.buffer.size" default="32768" /></Set>
<Set name="requestHeaderSize"><Property name="jetty.request.header.size" default="8192" /></Set>
<Set name="responseHeaderSize"><Property name="jetty.response.header.size" default="8192" /></Set>
<Set name="sendServerVersion"><Property name="jetty.send.server.version" default="true" /></Set>
<Set name="sendDateHeader"><Property name="jetty.send.date.header" default="false" /></Set>
<Set name="headerCacheSize">512</Set>
</New>
<New class="org.eclipse.jetty.server.ServerConnector">
<Arg name="server"><Ref refid="Server" /></Arg>
<Arg name="factories">
<Array type="org.eclipse.jetty.server.ConnectionFactory">
<Item>
<New class="org.eclipse.jetty.server.HttpConnectionFactory">
<Arg name="config"><Ref refid="httpConfig" /></Arg>
</New>
</Item>
</Array>
</Arg>
<Set name="host"><Property name="jetty.host" /></Set>
<Set name="port"><Property name="jetty.port" default="8080"/></Set>
<Set name="idleTimeout"><Property name="http.timeout" default="30000"/></Set>
<Set name="soLingerTime"><Property name="http.soLingerTime" default="-1"/></Set>
<Set name="acceptQueueSize"><Property name="http.acceptQueueSize" default="0"/></Set>
</New>
<Call class="java.lang.System" name="setProperty">
<Arg>logback.configurationFile</Arg>
<Arg>file:///conf/worker/logback.xml</Arg>
</Call>
<Call class="java.lang.System" name="setProperty" >
<Arg>service.config.path</Arg>
<Arg>/opt/webapps/worker</Arg>
</Call>
<Set name="handler">
<New id="Handlers" class="org.eclipse.jetty.server.handler.HandlerCollection">
<Set name="handlers">
<Array type="org.eclipse.jetty.server.Handler">
<Item>
<New id="Contexts" class="org.eclipse.jetty.server.handler.ContextHandlerCollection"/>
</Item>
<Item>
<New id="DefaultHandler" class="org.eclipse.jetty.server.handler.DefaultHandler">
<Set name="showContexts">false</Set>
</New>
</Item>
</Array>
</Set>
</New>
</Set>
<New class="org.eclipse.jetty.webapp.WebAppContext">
<Set name="contextPath">/</Set>
<Set name="war"><Property name="jetty.webapps" default="." />/opt/webapps/worker.war</Set>
<Set name="extractWAR">true</Set>
</New>
<Call name="addBean">
<Arg>
<New id="DeploymentManager" class="org.eclipse.jetty.deploy.DeploymentManager">
<Set name="contexts">
<Ref refid="Contexts" />
</Set>
<Call id="webappprovider" name="addAppProvider">
<Arg>
<New class="org.eclipse.jetty.deploy.providers.WebAppProvider">
<Set name="monitoredDirName"><Property name="jetty.home" default="." />/webapps</Set>
<Set name="scanInterval">1</Set>
<Set name="extractWars">true</Set>
<Set name="configurationClasses">
<Array type="java.lang.String">
<Item>org.eclipse.jetty.webapp.WebInfConfiguration</Item>
<Item>org.eclipse.jetty.webapp.WebXmlConfiguration</Item>
<Item>org.eclipse.jetty.webapp.MetaInfConfiguration</Item>
<Item>org.eclipse.jetty.webapp.FragmentConfiguration</Item>
<Item>org.eclipse.jetty.annotations.AnnotationConfiguration</Item>
<Item>org.eclipse.jetty.webapp.JettyWebXmlConfiguration</Item>
</Array>
</Set>
</New>
</Arg>
</Call>
</New>
</Arg>
</Call>
<Set name="stopAtShutdown">true</Set>
<Set name="stopTimeout">5000</Set>
<Set name="dumpAfterStart"><Property name="jetty.dump.start" default="false"/></Set>
<Set name="dumpBeforeStop"><Property name="jetty.dump.stop" default="false"/></Set>
</Configure>
The jetty-console log is
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
WARNING: System properties and/or JVM args set. Consider using --dry-run or --exec
2014-12-19 19:32:49.749:INFO:oejs.Server:main: jetty-9.0.3.v20130506
2014-12-19 19:32:49.800:INFO:oejdp.ScanningAppProvider:main: Deployment monitor [file:/opt/jetty- distribution-9.0.3.v20130506/webapps/] at interval 1
2014-12-19 19:32:51.554:INFO:oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext#45afc369{/,file:/tmp/jetty-0.0.0.0-8080-worker.war-_-any-/webapp/,AVAILABLE}{/worker/worker.war}
2014-12-19 19:32:51.560:INFO:oejdp.ScanningAppProvider:main: Deployment monitor [file:/opt/jetty-distribution-9.0.3.v20130506/webapps/] at interval 1
2014-12-19 19:32:51.587:INFO:oejs.ServerConnector:main: Started ServerConnector#34cd072c{HTTP/1.1}{0.0.0.0:8080}
Edit: The xml configuration file is now correct and I had to uncomment the above options at ${jetty.home}/start.ini
# ===========================================================
# Enable additional webapp environment configurators
# -----------------------------------------------------------
OPTIONS=plus
etc/jetty-plus.xml
# ===========================================================
# Enable servlet 3.1 annotations
# -----------------------------------------------------------
OPTIONS=annotations
etc/jetty-annotations.xml

Some obvious things to fix first.
You are using Java 8, upgrade to Jetty 9.2.6 (Jetty 9.0.3 is too old and does not support Java 8 well)
Your DTD reference is bad, it is the one from Jetty 7, it has been known to cause problems in Jetty 9.
Use <!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_0.dtd"> instead.
Your class reference is bad, and should be fixed. "org.eclipse.jetty_runˆy.server.Server" doesn't exist

Change Jetty default port

Jetty default port is 8080, but I want to change to default port to some other port (9999).
I read a few tutorials and they said almost all of configuration information is by default maintained in file jetty.xml, this file is located under $JETTY_HOME/etc/. Then, change property jetty.port to 9999. However, when I opened up that file, I couldn't find jetty.port property inside the jetty.xml. I'm currently using Jetty-9.2.1 and the port is at 8080.
I found jetty.port property under jetty-http.xml file. Even though I changed the port to 8090 in the jetty-http.xml file, jetty is still running at port 8080.
jetty.xml
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_0.dtd">
<!-- =============================================================== -->
<!-- Documentation of this file format can be found at: -->
<!-- http://wiki.eclipse.org/Jetty/Reference/jetty.xml_syntax -->
<!-- -->
<!-- Additional configuration files are available in $JETTY_HOME/etc -->
<!-- and can be mixed in. See start.ini file for the default -->
<!-- configuration files. -->
<!-- -->
<!-- For a description of the configuration mechanism, see the -->
<!-- output of: -->
<!-- java -jar start.jar -? -->
<!-- =============================================================== -->
<!-- =============================================================== -->
<!-- Configure a Jetty Server instance with an ID "Server" -->
<!-- Other configuration files may also configure the "Server" -->
<!-- ID, in which case they are adding configuration to the same -->
<!-- instance. If other configuration have a different ID, they -->
<!-- will create and configure another instance of Jetty. -->
<!-- Consult the javadoc of o.e.j.server.Server for all -->
<!-- configuration that may be set here. -->
<!-- =============================================================== -->
<Configure id="Server" class="org.eclipse.jetty.server.Server">
<!-- =========================================================== -->
<!-- Configure the Server Thread Pool. -->
<!-- The server holds a common thread pool which is used by -->
<!-- default as the executor used by all connectors and servlet -->
<!-- dispatches. -->
<!-- -->
<!-- Configuring a fixed thread pool is vital to controlling the -->
<!-- maximal memory footprint of the server and is a key tuning -->
<!-- parameter for tuning. In an application that rarely blocks -->
<!-- then maximal threads may be close to the number of 5*CPUs. -->
<!-- In an application that frequently blocks, then maximal -->
<!-- threads should be set as high as possible given the memory -->
<!-- available. -->
<!-- -->
<!-- Consult the javadoc of o.e.j.util.thread.QueuedThreadPool -->
<!-- for all configuration that may be set here. -->
<!-- =========================================================== -->
<!-- uncomment to change type of threadpool
<Arg name="threadpool"><New id="threadpool" class="org.eclipse.jetty.util.thread.QueuedThreadPool"/></Arg>
-->
<Get name="ThreadPool">
<Set name="minThreads" type="int"><Property name="threads.min" default="10"/></Set>
<Set name="maxThreads" type="int"><Property name="threads.max" default="200"/></Set>
<Set name="idleTimeout" type="int"><Property name="threads.timeout" default="60000"/></Set>
<Set name="detailedDump">false</Set>
</Get>
<!-- =========================================================== -->
<!-- Add shared Scheduler instance -->
<!-- =========================================================== -->
<Call name="addBean">
<Arg>
<New class="org.eclipse.jetty.util.thread.ScheduledExecutorScheduler"/>
</Arg>
</Call>
<!-- =========================================================== -->
<!-- Http Configuration. -->
<!-- This is a common configuration instance used by all -->
<!-- connectors that can carry HTTP semantics (HTTP, HTTPS, SPDY)-->
<!-- It configures the non wire protocol aspects of the HTTP -->
<!-- semantic. -->
<!-- -->
<!-- This configuration is only defined here and is used by -->
<!-- reference from the jetty-http.xml, jetty-https.xml and -->
<!-- jetty-spdy.xml configuration files which instantiate the -->
<!-- connectors. -->
<!-- -->
<!-- Consult the javadoc of o.e.j.server.HttpConfiguration -->
<!-- for all configuration that may be set here. -->
<!-- =========================================================== -->
<New id="httpConfig" class="org.eclipse.jetty.server.HttpConfiguration">
<Set name="secureScheme">https</Set>
<Set name="securePort"><Property name="jetty.secure.port" default="8443" /></Set>
<Set name="outputBufferSize"><Property name="jetty.output.buffer.size" default="32768" /></Set>
<Set name="requestHeaderSize"><Property name="jetty.request.header.size" default="8192" /></Set>
<Set name="responseHeaderSize"><Property name="jetty.response.header.size" default="8192" /></Set>
<Set name="sendServerVersion"><Property name="jetty.send.server.version" default="true" /></Set>
<Set name="sendDateHeader"><Property name="jetty.send.date.header" default="false" /></Set>
<Set name="headerCacheSize">512</Set>
<!-- Uncomment to enable handling of X-Forwarded- style headers
<Call name="addCustomizer">
<Arg><New class="org.eclipse.jetty.server.ForwardedRequestCustomizer"/></Arg>
</Call>
-->
</New>
<!-- =========================================================== -->
<!-- Set the default handler structure for the Server -->
<!-- A handler collection is used to pass received requests to -->
<!-- both the ContextHandlerCollection, which selects the next -->
<!-- handler by context path and virtual host, and the -->
<!-- DefaultHandler, which handles any requests not handled by -->
<!-- the context handlers. -->
<!-- Other handlers may be added to the "Handlers" collection, -->
<!-- for example the jetty-requestlog.xml file adds the -->
<!-- RequestLogHandler after the default handler -->
<!-- =========================================================== -->
<Set name="handler">
<New id="Handlers" class="org.eclipse.jetty.server.handler.HandlerCollection">
<Set name="handlers">
<Array type="org.eclipse.jetty.server.Handler">
<Item>
<New id="Contexts" class="org.eclipse.jetty.server.handler.ContextHandlerCollection"/>
</Item>
<Item>
<New id="DefaultHandler" class="org.eclipse.jetty.server.handler.DefaultHandler"/>
</Item>
</Array>
</Set>
</New>
</Set>
<!-- =========================================================== -->
<!-- extra server options -->
<!-- =========================================================== -->
<Set name="stopAtShutdown">true</Set>
<Set name="stopTimeout">5000</Set>
<Set name="dumpAfterStart"><Property name="jetty.dump.start" default="false"/></Set>
<Set name="dumpBeforeStop"><Property name="jetty.dump.stop" default="false"/></Set>
</Configure>
jetty-http.xml
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_0.dtd">
<!-- ============================================================= -->
<!-- Configure the Jetty Server instance with an ID "Server" -->
<!-- by adding a HTTP connector. -->
<!-- This configuration must be used in conjunction with jetty.xml -->
<!-- ============================================================= -->
<Configure id="Server" class="org.eclipse.jetty.server.Server">
<!-- =========================================================== -->
<!-- Add a HTTP Connector. -->
<!-- Configure an o.e.j.server.ServerConnector with a single -->
<!-- HttpConnectionFactory instance using the common httpConfig -->
<!-- instance defined in jetty.xml -->
<!-- -->
<!-- Consult the javadoc of o.e.j.server.ServerConnector and -->
<!-- o.e.j.server.HttpConnectionFactory for all configuration -->
<!-- that may be set here. -->
<!-- =========================================================== -->
<Call name="addConnector">
<Arg>
<New class="org.eclipse.jetty.server.ServerConnector">
<Arg name="server"><Ref refid="Server" /></Arg>
<Arg name="factories">
<Array type="org.eclipse.jetty.server.ConnectionFactory">
<Item>
<New class="org.eclipse.jetty.server.HttpConnectionFactory">
<Arg name="config"><Ref refid="httpConfig" /></Arg>
</New>
</Item>
</Array>
</Arg>
<Set name="host"><Property name="jetty.host" /></Set>
<Set name="port"><Property name="jetty.port" default="8090" /></Set>
<Set name="idleTimeout"><Property name="http.timeout" default="30000"/></Set>
<Set name="soLingerTime"><Property name="http.soLingerTime" default="-1"/></Set>
</New>
</Arg>
</Call>
</Configure>
I was also advised to use an integration test to configure Jetty to use other port. There's a integration-tests.properties file inside the project. Maybe a solution is to set jetty.port to 9999 inside this file?
integration-tests.properties:
host = localhost
port = 9999

does it work if you set the port when you start it from the command line like this:
java -jar start.jar -Djetty.port=9999

I did this in Jetty 9.x version.
You need to go to $JETTY_HOME/start.ini file and edit this setting jetty.port.
Lets say that you want to run jetty at 9090 port:
Please change jetty.port setting in $JETTY_HOME/start.ini from
jetty.port=8080
to
jetty.port=9090
Then start jetty using java -jar start.jar option.
Then jetty will be running at 9090 port than default 8080 port.
Then do curl -i -XGET "http://localhost:9090". That should give you 200 http status.
Thats it.

Update:
On Jetty 9.x, jetty.port has been deprecated and you can use jetty.http.port instead, as shown below:
$> cd $JETTY_HOME && java -jar start.jar -Djetty.http.port=8080

On jetty 9.2.3.v20140905 it`s need to write in /etc/default/jetty
# JETTY_ARGS
# The default arguments to pass to jetty.
# For example
JETTY_ARGS="jetty.port=8080 jetty.spdy.port=8443 jetty.secure.port=443"
but this change only http port. To change https port in jetty 9.2 create ini file $JETTY_HOME/start.d/https.ini
# Initialize module https
#
--module=https
## HTTPS Configuration
# HTTP port to listen on
https.port=8443
# HTTPS idle timeout in milliseconds
https.timeout=30000
# HTTPS Socket.soLingerTime in seconds. (-1 to disable)
# https.soLingerTime=-1
jetty 9.3 in /etc/default/jetty
# JETTY_ARGS
# The default arguments to pass to jetty.
# For example
JETTY_ARGS="jetty.http.port=8080 jetty.ssl.port=443"
or command line parameters -Djetty.http.port=8080 -Djetty.ssl.port=443

I changed the port successfully, you can try to edit jetty.port in the file located at $Jetty_home/start.d/http.ini.

You need to change the http port in the start.ini file because it will over-right jetty-http.xml config.Or just comment the line in start.ini and keep your config from jetty-http.xml.
In [jetty home]/start.ini
## HTTP port to listen on
#jetty.port=8080

Just for completeness on jetty 7 you can use this:
java -jar start.jar --module=http jetty.port=9080

If you are using eclipse you need to set the run configurations. When you install jetty in eclipse, the default port for jetty is 8080.
So you need to change it into an XML file. If the problem remains, you need to change it in eclipse run configurations.
I hope it works, as it worked for me.

For IntelliJ, this can be done similar to Magnus Lassi's command-line answer.
Run --> Edit Configurations --> Add "-Djetty.port=XXXX", e.g.:
jetty:run-war -Djetty.port=9999

For SSL port you can pass the param:
-Dssl.port=8445
It worked for me.

For a spring boot application, inside your application.properties file
add
server.port = preferred port number eg. 8082
IMAGE OF PORT CONFIGURATION

I changed the jetty.http.port property in jetty/start.ini and it worked for me!

For changing SSL port, you need to change jetty.ssl.port property in below directory:
📁$JETTY_HOME/etc
📄 jetty-ssl.xml
<Configure id="Server" class="org.eclipse.jetty.server.Server">
...
<Call name="addConnector">
<Arg>
<New id="sslConnector" class="org.eclipse.jetty.server.ServerConnector">
...
<Set name="port"><Property name="jetty.ssl.port" deprecated="ssl.port" default="8443" /></Set>
...
</New>
</Arg>
</Call>
...
</Configure>

Change thread pool size in Jetty 9

How can I change thread pool size in embedded Jetty 9? Do we need any specific component for this?

From docs:
The Server instance provides a ThreadPool instance that is the default
Executor service other Jetty server components use. The prime
configuration of the thread pool is the maximum and minimum size and
is set in etc/jetty.xml.
<Configure id="server" class="org.eclipse.jetty.server.Server">
<Set name="threadPool">
<New class="org.eclipse.jetty.util.thread.QueuedThreadPool">
<Set name="minThreads">10</Set>
<Set name="maxThreads">1000</Set>
</New>
</Set>
</Configure>
Or
QueuedThreadPool threadPool = new QueuedThreadPool(100, 10);
Server server = new Server(threadPool);

As noted, and corrected in the Java code example above, the threadpool is now provided as a constructor argument in Jetty 9 (and later).
The corrected XML example:
<Configure id="Server" class="org.eclipse.jetty.server.Server">
<!-- =========================================================== -->
<!-- Configure the Server Thread Pool. -->
<!-- -->
<!-- Consult the javadoc of o.e.j.util.thread.QueuedThreadPool -->
<!-- for all configuration that may be set here. -->
<!-- =========================================================== -->
<Get name="ThreadPool">
<Set name="minThreads" type="int">10</Set>
<Set name="maxThreads" type="int">200</Set>
<Set name="idleTimeout" type="int">60000</Set>
<Set name="detailedDump">false</Set>
</Get>
...

Want to set Jetty to hot deploy, but there is no deployer

I'm running a SmartFox 1.6 server with included Jetty servlet container. I want to configure Jetty to automatically deploy new web apps. Normally this is as simple as adding a scanInterval to the deployer:
<Call name="addLifeCycle">
<Arg>
<New class="org.mortbay.jetty.deployer.ContextDeployer">
<Set name="contexts"><Ref id="Contexts"/></Set>
<Set name="configurationDir"><SystemProperty name="jetty.home" default="."/>/contexts</Set>
<Set name="scanInterval">1</Set>
</New>
</Arg>
</Call>
However, I can't find a deployer at all in the Jetty config file, pasted below. The config is the default config included with Smartfox.
Sorry for needing to include the entire config file, but I wouldn't know which blocks are relevant. Where can I set the scanInterval in the settings below?
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure//EN" "http://jetty.mortbay.org/configure.dtd">
<!-- =============================================================== -->
<!-- Configure the Jetty Server -->
<!-- =============================================================== -->
<Configure id="Server" class="org.mortbay.jetty.Server">
<!-- =========================================================== -->
<!-- Server Thread Pool -->
<!-- =========================================================== -->
<Set name="ThreadPool">
<New class="org.mortbay.thread.BoundedThreadPool">
<Set name="minThreads">10</Set>
<Set name="lowThreads">50</Set>
<Set name="maxThreads">250</Set>
</New>
</Set>
<!-- =========================================================== -->
<!-- Set connectors -->
<!-- =========================================================== -->
<!-- One of each type! -->
<!-- =========================================================== -->
<!-- Use this connector for many frequently idle connections
and for threadless continuations.
-->
<Call name="addConnector">
<Arg>
<New class="org.mortbay.jetty.nio.SelectChannelConnector">
<Set name="port"><SystemProperty name="jetty.port" default="8080"/></Set>
<Set name="maxIdleTime">30000</Set>
<Set name="Acceptors">2</Set>
<Set name="confidentialPort">8443</Set>
</New>
</Arg>
</Call>
<!-- Use this connector if NIO is not available.
<Call name="addConnector">
<Arg>
<New class="org.mortbay.jetty.bio.SocketConnector">
<Set name="port">8081</Set>
<Set name="maxIdleTime">50000</Set>
<Set name="lowResourceMaxIdleTime">1500</Set>
</New>
</Arg>
</Call>
-->
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<!-- To add a HTTPS SSL listener -->
<!-- see jetty-ssl.xml to add an ssl connector. use -->
<!-- java -jar start.jar etc/jetty.xml etc/jetty-ssl.xml -->
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<!-- =========================================================== -->
<!-- Set handler Collection Structure -->
<!-- =========================================================== -->
<Set name="handler">
<New id="handlers" class="org.mortbay.jetty.handler.HandlerCollection">
<Set name="handlers">
<Array type="org.mortbay.jetty.Handler">
<Item>
<New id="contexts" class="org.mortbay.jetty.handler.ContextHandlerCollection"/>
</Item>
<Item>
<New id="defaultHandler" class="org.mortbay.jetty.handler.DefaultHandler"/>
</Item>
<Item>
<New id="requestLog" class="org.mortbay.jetty.handler.RequestLogHandler"/>
</Item>
</Array>
</Set>
</New>
</Set>
<!-- ======================================================= -->
<!-- Configure a WebApp -->
<!-- ======================================================= -->
<!--
<New id="TestContext" class="org.mortbay.jetty.webapp.WebAppContext">
<Arg><Ref id="contexts"/></Arg>
<Arg><SystemProperty name="jetty.home" default="."/>/webapps/test</Arg>
<Arg>/</Arg>
<Set name="classLoader">
<New class="org.mortbay.jetty.webapp.TransformingWebAppClassLoader">
<Arg><Ref id="TestContext"/></Arg>
</New>
</Set>
<Set name="defaultsDescriptor"><SystemProperty name="jetty.home" default="."/>/etc/webdefault.xml</Set>
<Set name="virtualHosts">
<Array type="java.lang.String">
<Item>localhost</Item>
</Array>
</Set>
<Get name="SessionHandler">
<Set name="SessionManager">
<New class="org.mortbay.jetty.servlet.HashSessionManager">
<Set name="maxInactiveInterval">600</Set>
</New>
</Set>
</Get>
</New>
-->
<!-- ======================================================= -->
<!-- Configure a Context -->
<!-- ======================================================= -->
<New class="org.mortbay.jetty.servlet.Context">
<Arg><Ref id="contexts"/></Arg>
<Arg>/javadoc</Arg>
<Set name="resourceBase"><SystemProperty name="jetty.home" default="."/>/javadoc/</Set>
<Call name="addServlet">
<Arg>org.mortbay.jetty.servlet.DefaultServlet</Arg>
<Arg>/</Arg>
</Call>
</New>
<!-- =========================================================== -->
<!-- Discover contexts from webapps directory -->
<!-- =========================================================== -->
<Call class="org.mortbay.jetty.webapp.WebAppContext" name="addWebApplications">
<Arg><Ref id="contexts"/></Arg>
<Arg><SystemProperty name="jetty.home" default="."/>/webserver/webapps</Arg>
<Arg><SystemProperty name="jetty.home" default="."/>/webserver/cfg/webdefault.xml</Arg>
<Arg type="boolean">True</Arg> <!-- extract -->
<Arg type="boolean">False</Arg> <!-- parent priority class loading -->
</Call>
<!-- =========================================================== -->
<!-- Configure Realms -->
<!-- =========================================================== -->
<Set name="UserRealms">
<Array type="org.mortbay.jetty.security.UserRealm">
<Item>
<New class="org.mortbay.jetty.security.HashUserRealm">
<Set name="name">Test Realm</Set>
<Set name="config"><SystemProperty name="jetty.home" default="."/>/webserver/cfg/realm.properties</Set>
</New>
</Item>
</Array>
</Set>
<!-- =========================================================== -->
<!-- Configure Request Log -->
<!-- =========================================================== -->
<Ref id="requestLog">
<Set name="requestLog">
<New id="requestLogImpl" class="org.mortbay.jetty.NCSARequestLog">
<Arg><SystemProperty name="jetty.logs" default="./logs"/>/yyyy_mm_dd.request.log</Arg>
<Set name="retainDays">90</Set>
<Set name="append">true</Set>
<Set name="extended">false</Set>
<Set name="LogTimeZone">GMT</Set>
</New>
</Set>
</Ref>
<!-- =========================================================== -->
<!-- extra options -->
<!-- =========================================================== -->
<Set name="stopAtShutdown">true</Set>
<!-- ensure/prevent Server: header being sent to browsers -->
<Set name="sendServerVersion">true</Set>
</Configure>

you just need to add this Call to your config file.

Answering it here, in case anyone come across this thread. Hot deployment is possible in jetty, you need to specify interval, location etc. in jetty ini files, not in jetty-xmls. See below official jetty documentations
https://www.eclipse.org/jetty/documentation/current/hot-deployment.html

We Keep Coding

Java is a programming language and computing platform first released by Sun Microsystems in 1995.

Using Blazegraph's `NanoSparqlServer`, how to clear the graph between tests? - java

Related

Enable programmatic concurrency on Jetty with WorkManager

Java Based Spring WebApp not Running Jetty9

Change Jetty default port

Change thread pool size in Jetty 9

Want to set Jetty to hot deploy, but there is no deployer

Categories

Resources