I have an application that uses Java, mybatis, and SQL Anywhere. Because of some circumstances, I have to manually load in the SQLA driver jar at runtime. I got this working with this code:
public static URLClassLoader ucl;
public static Driver saDriver;
public static final void loadSqlAnywhere(SqlAnywhereVersions version)
{
if (saDriver == null)
{
try
{
File jarPath = "path\\to\\sajdbc4.jar";
URL jarUrl = new URL("jar:file:" + jarPath.getAbsolutePath() + "!/");
String driverClass = "sap.jdbc4.sqlanywhere.IDriver";
ucl = URLClassLoader.newInstance(new URL[]{ jarUrl });
saDriver = new DriverShim((Driver) Class.forName(driverClass, true, ucl).newInstance());
DriverManager.registerDriver(saDriver);
}
catch (Exception ex)
{
// Could not load SQL Anywhere driver
}
}
}
While the application is running, it's possible for SQL Anywhere to be upgraded. So, I need to completely unload the old driver, let the update happen, then init with the latest jar. I tried this:
private static List<SqlSession> sessions = new ArrayList<>();
private static SqlSessionFactory sessionFactory = new SqlSessionFactoryBuilder();
public static final void unloadSqlAnywhere()
{
if (saDriver != null)
{
// Close out any existing sessions
for (SqlSession session : sessions)
{
session.close();
}
sessions.clear();
// Kill the session factory
sessionFactory = null;
try
{
DriverManager.deregisterDriver(saDriver);
saDriver = null;
ucl.close();
ucl = null;
}
catch (Exception ex)
{
// Failed to unregister SQL Anywhere driver
}
}
}
But using jprofiler, I can see that DriverManager is still referencing sap.jdbc4.sqlanywhere.IDriver, which is causing the SQL Anywhere upgrade to kill my application.
Is there something more I can do to ensure all driver references are removed? Or is there a better way to go about this?
Most JDBC drivers auto-register themselves, e.g. you used to just have to call Class.forName() to load the driver class, and that would auto-register it.
Nowadays you don't even have to do that anymore, since they use the service framework to auto-register by simply being on the classpath, but since your JDBC jar file is not on the classpath, that doesn't apply here.
Since you also register the driver, it is registered twice, and when you deregister, you only remove the registration you created, not the one that was auto-registered.
To remove the auto-registered instance, you have to enumerate all registered drivers to find the instance to deregister:
// Any Java version
for (Enumeration<Driver> e = DriverManager.getDrivers(); e.hasMoreElements(); ) {
Driver driver = e.nextElement();
if (driver.getClass().getName().equals("sap.jdbc4.sqlanywhere.IDriver"))
DriverManager.deregisterDriver(driver);
}
// Java 9+
Optional<Driver> driver = DriverManager.drivers()
.filter(d -> d.getClass().getName().equals("sap.jdbc4.sqlanywhere.IDriver"))
.findAny();
if (driver.isPresent())
DriverManager.deregisterDriver(driver.get());
Related
I've come across many posts about these two topics: Auto-Updating and URLClassloaders. I'll start with the auto updating goal. I found this post here that talks about a 2 jar system. One jar that launches the main app jar: From Stephen C:
The launcher could be a Java application that creates a classloader for the new JAR, loads an entrypoint class and calls some method on it. If you do it this way, you have to watch for classloader storage leaks, but that's not difficult. (You just need to make sure that no objects with classes loaded from the JAR are reachable after you relaunch.)
This is the approach I'm taking, but I'm open to other ideas if they prove easier and/or more reliable. The Coordinator has posted some pretty cool launcher code to which I plan on incorporating some of this reload type code in my launcher, but first I need to get it to work.
My issue is that my main app jar has many other dependencies, and I cannot get some of those classes to load despite the fact that all the jars have been added to the URL's array. This brings up the second topic URLClassloader.
Side Note for future readers: When passing a URL to the URLClassloader that is a directory, a helpful note that would have saved me (an embarrassingly large) amount of time is that the contents of the directory must be .class files! I was originally pointing to my dependent jar directory, no good.
Context for the code below, my launcher jar resides in the same directory as my app jar, which is why I'm using user.dir. I will probably change this, but for now the code works and gets far enough into my app's code to request a connection to a sqlite database before failing.
Launcher:
public class Launcher {
public static void main(String[] args) {
try {
String userdir = System.getProperty("user.dir");
File parentDir = new File(userdir);
ArrayList<URL> urls = getJarURLs(parentDir);
URL[] jarURLs = new URL[urls.size()];
int index = 0;
for (URL u : urls) {
System.out.println(u.toString());
jarURLs[index] = u;
index ++;
}
URLClassLoader urlCL = new URLClassLoader(jarURLs);
Class<?> c = urlCL.loadClass("main.AppStart");
Object [] args2 = new Object[] {new String[] {}};
c.getMethod("main", String[].class).invoke(null, args2);
urlCL.close();
} catch (Exception e1) {
e1.printStackTrace();
}
}
public static ArrayList<URL> getJarURLs(File parentDir) throws MalformedURLException {
ArrayList<URL> list = new ArrayList<>();
for (File f : parentDir.listFiles()) {
if (f.isDirectory()) {
list.addAll(getJarURLs(f));
} else {
String name = f.getName();
if (name.endsWith(".jar")) {
list.add(f.toURI().toURL());
}
}
}
return list;
}
}
Here's an example of the URL output added to the array:
file:/C:/my/path/to/dependent/jars/sqlite-jdbc-3.32.3.2.jar
file:/C:/my/path/to/main/app.jar
file: ... [10 more]
The URLClassloader seems to work well enough to load my main method in app.jar. The main executes a some startup type stuff, before attempting to load a login screen. When the request is made to get the user info database, my message screen loads and displays (<-this is important for later)
the stacktrace containing:
java.sql.SQLException: No suitable driver found for jdbc:sqlite:C:\...\users.db
I understand that this is because that jar is not on the class path, but it's loaded via the class loader, so why can't it find the classes from the jar? From this post JamesB suggested adding Class.forName("org.sqlite.JDBC"); before the connection request. I rebuilt the app jar with this line of code and it worked!
The weird thing that happened next, is that my message screen class can no longer be found even though earlier it loaded and displayed correctly. The message screen is a class inside my main app.jar and not in a dependent jar, which is why I'm baffled. Am I going to have to add Class.forName before every instance of any of my classes? That seems rude..
So what could I be doing wrong with the class loader? Why does it load some classes and not others despite that fact that all the jars have been added to the URL array?
Some other relative info: My app works perfectly as intended when launched from windows command line when the classpath is specified: java -cp "main-app.jar;my/dependent/jar/directory/*" main.AppStart. It's only when I try launching the app via this classloader that I have these issues.
By the way, is this java command universal? Will it work on all operating systems with java installed? If so, could I not just scrap this launcher, and use a process builder to execute the above command? Bonus points for someone who can tell me how to execute the command from a jre packaged with my app, as that's what I plan on doing so the user does not have to download Java.
EDIT
I figured out one of the answers to one of the questions below. Turns out, I didn't need to do any of the code below. My main method loads a login screen but after it's loaded it returns back to the AppLauncher code, thus closing the URLClassLoader! Of course, at that point any requested class will not be found as the loader has been closed! What an oof! Hopefully I will save someone a headache in the future...
Original
Well, after more time, effort, research, and effective use of Eclipse's debugging tool, I was able to figure out what I needed to do to resolve my issues.
So the first issue was my JDBC driver was never registered when passing the jars to the URLClassloader. This is the part I sorta don't understand, so advisement would be welcomed, but there is a static block in the JDBC class that registers the driver so it can be used by DriverManager see code below. Loading the class is what executes that static block, hence why calling Class.forName works.
static {
try {
DriverManager.registerDriver(new JDBC());
} catch (SQLException e) {
e.printStackTrace();
}
}
What I don't understand, is how class loading works if jars are specified via the class path. The URLClassLoader doesn't load any of those classes until they are called, and I never directly work with the JDBC class, thus no suitable driver exception, but are all the classes specified via the classpath loaded initially? Seems that way for static blocks to execute.
Anyhow, to resolve my other issue with some of my app's classes not being found I had to implement my own classloader. I get what I did and how it works well, but still don't understand why I had to do it. All of my jars were loaded to the original URLClassloader so if I could find them and the files within, why couldn't it do it?
Basically, I had to override the findClass and findResource methods to return jarEntry information that I had to store. I hope this code helps someone!
public class SBURLClassLoader extends URLClassLoader {
private HashMap<String, Storage> map;
public SBURLClassLoader(URL[] urls) {
super(urls);
map = new HashMap<>();
try {
storeClasses(urls);
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
private void storeClasses(URL[] urls) throws ClassNotFoundException {
for (URL u : urls) {
try {
JarFile jarFile = new JarFile(new File(u.getFile()));
Enumeration<JarEntry> e = jarFile.entries();
while (e.hasMoreElements()) {
JarEntry jar = e.nextElement();
String entryName = jar.getName();
if (jar.isDirectory()) continue;
if (!entryName.endsWith(".class")) {
//still need to store these non-class files as resources
//let code continue to store entry un-altered
} else {
entryName = entryName.replace(".class", "");
entryName = entryName.replace("/", ".");
}
map.put(entryName, new Storage(jarFile, jar));
System.out.println(entryName);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
#Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
Class<?> c = null;
try {
c = super.findClass(name);
} catch (ClassNotFoundException e) {
Storage s = map.get(name);
try {
InputStream in = s.jf.getInputStream(s.je);
int len = in.available();
c = defineClass(name, in.readAllBytes(), 0, len);
resolveClass(c);
} catch (IOException e1) {
e1.printStackTrace();
}
if (c == null) throw e;
}
return c;
}
#Override
public URL findResource(String name) {
URL url = super.findResource(name);
if (url == null) {
Storage s = map.get(name);
if (s != null) {
try {
url = new URL("jar:"+s.base.toString() + "!/" + name);
} catch (Exception e) {
e.printStackTrace();
}
}
}
return url;
}
private class Storage {
public JarFile jf;
public JarEntry je;
public URL base;
public Storage(JarFile jf, JarEntry je) {
this.jf = jf;
this.je = je;
try {
base = Path.of(jf.getName()).toUri().toURL();
} catch (MalformedURLException e) {
e.printStackTrace();
}
}
}
}
I've written a Spring MVC (Spring framework 4.1.1) java 1.8 application that successfully connects to SAP using the sapjco3.jar driver, and I've accomplished this using the CustomDestinationDataProvider technique. I then use this drive to call RFCs in my SAP R/3 system. The java code is executed via api call from an AngularJS front end application.
Something that I've discovered occuring about 5% of the time that the call to SAP happens is the following error occurs:
NestedServletException: Handler processing failed; nested exception is
java.lang.Error: java.lang.IllegalStateException: DestinationDataProvider
already registered
Here's the contents of my CustomDestinationDataProvider.java file:
public class CustomDestinationDataProvider {
public class MyDestinationDataProvider implements DestinationDataProvider {
private DestinationDataEventListener eL;
private HashMap<String, Properties> secureDBStorage = new HashMap<String, Properties>();
public Properties getDestinationProperties(String destinationName) {
try {
Properties p = secureDBStorage.get(destinationName);
if(p!=null) {
if(p.isEmpty())
throw new DataProviderException(DataProviderException.Reason.INVALID_CONFIGURATION, "destination configuration is incorrect", null);
return p;
}
return null;
} catch(RuntimeException re) {
throw new DataProviderException(DataProviderException.Reason.INTERNAL_ERROR, re);
}
}
public void setDestinationDataEventListener(DestinationDataEventListener eventListener) {
this.eL = eventListener;
}
public boolean supportsEvents() {
return true;
}
public void changeProperties(String destName, Properties properties) {
synchronized(secureDBStorage) {
if(properties==null) {
if(secureDBStorage.remove(destName)!=null)
eL.deleted(destName);
} else {
secureDBStorage.put(destName, properties);
eL.updated(destName); // create or updated
}
}
}
}
public ArrayList<MaterialBean> executeAvailabilityCall(Properties connectProperties, String searchString) {
String destName = "ABAP_AS";
SAPDAO sapDAO = new SAPDAO();
ArrayList<MaterialBean> searchResults = new ArrayList<MaterialBean>();
MyDestinationDataProvider myProvider = new MyDestinationDataProvider();
JCoDestination dest;
try {
com.sap.conn.jco.ext.Environment.registerDestinationDataProvider(myProvider);
} catch(IllegalStateException providerAlreadyRegisteredException) {
}
myProvider.changeProperties(destName, connectProperties);
try {
dest = JCoDestinationManager.getDestination(destName);
searchResults = sapDAO.searchAvailability(dest, searchString);
} catch(JCoException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
myProvider.changeProperties(destName, null);
try {
com.sap.conn.jco.ext.Environment.unregisterDestinationDataProvider(myProvider);
} catch(IllegalStateException providerAlreadyRegisteredException) {
throw new Error(providerAlreadyRegisteredException);
}
return searchResults;
} // end method executeAvailabilityCall()
} // end class CustomDestinationProvider()
My guess is that multiple api calls are occuring at the same time, and once the first query registers the destination data provider, the subsequent queries, which try to also register the destination data provider, fail because they are using the same value for 'destName' in the executeAvailabilityCall method.
Upon first glace, it seems to me like I should use a dynamic value for the destName variable instead of just using "ABAP_AS" for all queries. In other words, I should change the following line:
String destName = "ABAP_AS";
to something like this:
String destName = "ABAP_AS_" + LocalDateTime.now();
This would guarantee a unique value for the destName variable, thus a unique destination provider name.
Any thoughts on the wisdom of trying this? If this is not a good idea, what other solution would be worth exploring?
Yes, you should use multiple unique destination names for your various logon Properties configuration sets. Your class MyDestinationDataProvider is already implemented that way. But why putting a timestamp into the destination name? Why not simply using a destination name schema like "TargetSystem_<SID>_with_<username>"?
Regarding your exception, simply register MyDestinationDataProvider only once and do not permanently register and unregister it. This is not how JCo expects this to be implemented. Quote from the JCo JavaDoc at com.sap.conn.jco.ext.DestinationDataProvider:
Only one implementation of DestinationDataProvider can be registered.
For registering another implementation the infrastructure has first to
unregister the implementation that is currently registered. It is not
recommended to permanently exchange DestinationDataProvider
registrations. The one registered instance should globally manage all
destination configurations for the whole infrastructure environment.
i've loaded jar dynamically from other jar file, then at some point, i want to delete this jar and replace it by newer version, on linux it works fine, while on windows when i try to move the file to backup directory i get file being used by another process exception.
public void loadJarAndClass() {
URL[] jarUrl = new URL[1];
jarUrl[0] = jarFile.toURI().toURL();
classLoader = new URLClassLoader (jarUrl, this.getClass().getClassLoader());
classToLoad = Class.forName ("Burner.MainWindow", true, classLoader);
}
public void unloadJarAndClass() {
/* all object must be collected in order to reload jar */
jarFile = null;
dukeClassLoader = null;
classToLoad = null;
System.gc();
}
my main:
jarPath = currentPath.getAbsolutePath() + File.separator + JAR_NAME;
jarFile = new File(jarPath);
loadJarAndClass();
unloadJarAndClass();
Files.delete(FileSystems.getDefault().getPath(jarPath));
my problem is with the delete which throws exception " the process cannot access the file because it is being used by another process"
how can i bypass this exception and close any handler opened ?
Try using Classloader.close() method,
Here is the extract from the Oracle link,
In Java SE 7, the URLClassLoader close() method effectively invalidates the loader, so that no new classes can be loaded from it. It also closes any JAR files that were opened by the loader. This allows the application to delete or replace these files and, if necessary, create new loaders using new implementations.
Following is the simplified version of the code that might serve the purpose,
public class Example {
public static void main(String[] args) throws IOException, ClassNotFoundException {
String jarPath = "C:\\example.jar";
File jarFile = new File(jarPath);
URL[] jarUrl = new URL[1];
jarUrl[0] = jarFile.toURI().toURL();
URLClassLoader classLoader = new URLClassLoader (jarUrl, Example.class.getClassLoader());
Class classToLoad = Class.forName ("DataGenerator", true, classLoader);
classLoader.close();
Files.delete(FileSystems.getDefault().getPath(jarPath));
}
}
please try closing the classloader before you 'unload' it.
public void unloadJarAndClass() throws IOException {
dukeClassLoader.close();
/* all object must be collected in order to reload jar */
jarFile = null;
dukeClassLoader = null;
classToLoad = null;
// System.gc(); don't do this unless you're really
// sure you have to !
}
I would recommend not calling System.gc() explicitly !
(see Why is it bad practice to call System.gc()? for example)
We have a legacy system that has a admim module that allows users to upload jar files. After the upload, the jar file is validated and if not compliant to internal rules, it is deleted.
The problem is that windows is throwing an exception telling that the file "is already being used by another process." (when I call Files.delete(tmpJar);). I'm not able to identify why the file is open. Seems to me that I have closed everything.
First, we are using primefaces (4.0) to upload the file. Primefaces relies on commons-fileupload (1.3.1). It call the following method:
public void handleFileUpload(FileUploadEvent event) {
Path tmpJar = null;
try {
tmpJar = Files.createFile(Paths.get(event.getFile().getFileName()));
Files.write(tmpJar, event.getFile().getContents());
} catch (IOException e) {
LOGGER.error(e.getMessage(), e);
}
if (tmpJar != null) {
try {
this.validateJar(tmpJar.toString());
Files.delete(tmpJar);
} catch (IOException e) {
LOGGER.error(e.getMessage(), e);
}
}
}
Before NIO Files.write, I was using "standard" java IO classes. The problem isn't related to the above code, because if I comment the call to validateJar, Files.delete(tmpJar) is executed without problems and the file is removed. So, the problem is related with the code below, but I can't find where...
Job is an internal class, basically a simple POJO. "jobAnnotation" is a custom annotation to identify Jobs. I have shortened the code, but the essencial parts are preserved.
private List<Job> validateJar(final String jarPath) throws IOException {
List<Job> jobs = new ArrayList<Job>();
try (JarFile jarFile = new JarFile(jarPath)) {
URL[] jars = { new URL("file:" + jarPath) };
ClassLoader jobClassLoader = URLClassLoader.newInstance(jars, this.getClass().getClassLoader());
Enumeration<JarEntry> jarEntries = jarFile.entries();
while (jarEntries.hasMoreElements()) {
JarEntry jarEntry = jarEntries.nextElement();
String className = jarEntry.getName();
Class<?> classToLoad;
try {
classToLoad = Class.forName(className, true, jobClassLoader);
} catch (Exception e1) {
LOGGER.error(e1.getMessage(), e1);
continue;
}
if (classToLoad.isAnnotationPresent(jobAnnotation)) {
String vlr = null;
try {
Class<?> jobClass = (Class<?>) Class.forName(classToLoad.getCanonicalName(), true, jobClassLoader);
Annotation annotation = jobClass.getAnnotation(jobAnnotation);
Method method = annotation.getClass().getMethod("getValue");
vlr = ((String) method.invoke(annotation, new Object[0]));
} catch (Exception e1) {
LOGGER.error(e1.getMessage(), e1);
}
Job job = new Job();
job.setEnabled(true);
job.setJarfile(jarPath);
job.setClassName(classToLoad.getName());
Parameter parameter = new Parameter();
parameter.setRequired(true);
parameter.setName("name");
parameter.setValue(vlr);
job.addParameter(parameter);
jobs.add(job);
}
}
} catch (IOException e) {
throw e;
}
return jobs;
}
Before using try-with-resources, I was using regular try-catch-finally to close the JarFile, thats the only thing that has a explicit close method. Probably is the classloading that is holding the file open, but I don't know how to close it.
I did some searches, and I found that I can't unload classes (Unloading classes in java?).
So, the problem is, how do I release it? Or how can I remove the file?
BTW, I'm using java 1.7.0_71, jboss 7.1.1, windows 7 (64).
The URLClassLoader class already has a close() method. The close() method will close any Jar file that are opened with the URLClassLoader. This should prevent the "file already in use" exception.
File is already being used by another process. says that it could be not your fault, maybe just another application is used that file. You can check this question to find a process which is used your file.
Some Virus scanner software take a long time in checking JARs. Try to disable the Virusscanner. Other candidates can be the Windows indexer process, or the explorer.exe itself. When you don't find any reason for the file lock, try a delay between the validation and the deletion. Maybe you need a loop with multiple tries.
I am using cucumber-jvm to test the behaviour of the legacy system I'm working on at work. I have to use Java 1.5 and Hibernate 3.3, upgrading is not an option. Since, during my tests, it stores some objects in the database, I have created a new development database.
What bothers me is that I have to manually drop the records (using a sql script) everytime I'll rerun my tests, or they'll fail. And anyone else wanting to run them would have to do the same. I want to quickly and automatically clean my test database, by either:
Creating an empty database and populating it with what I need, or
Using an already existing database, droping the records before starting the tests.
What I have so far: I'm using the cucumber-junit plugin, and the RunTests class redirects to my test database:
#RunWith(Cucumber.class)
#Cucumber.Options(
features = "test/resources/cucumber",
format = "html:target/cucumber"
)
public class RunTests {
private static Configuration configuration;
#BeforeClass
public static void preparaBase() {
// gets the mapped configuration to the db
configuration = HibernateUtil.getConfiguration();
configuration.setProperty("hibernate.connection.url", "test-db-url");
configuration.setProperty("hibernate.connection.username", "user");
configuration.setProperty("hibernate.connection.password", "pass");
// configuration.setProperty("hibernate.hbm2ddl.auto", "create-drop");
// rebuilds the configuration using my test database
HibernateUtil.rebuildSessionFactory(configuration);
}
}
I have tried using the hibernate.hbm2ddl.auto property with the create-drop value and using the import.sql file to prepare the database, but it takes ages to start the tests and it seems that it's not detecting my import.sql file.
Sadly, using Maven and its excellent maven-sql-plugin is not an option (I have suggested a switch to Maven, to no avail). Is there an alternative?
I did it!
I used this ScriptRunner class as such:
#RunWith(Cucumber.class)
#Cucumber.Options(
features = "test/resources/cucumber",
format = "html:target/cucumber"
)
public class RunTests {
private static Configuration configuration;
String url = "test-db-url";
String user = "user";
String pass = "pass";
#BeforeClass
public static void preparaBase() {
// gets the mapped configuration to the db
configuration = HibernateUtil.getConfiguration();
configuration.setProperty("hibernate.connection.url", url);
configuration.setProperty("hibernate.connection.username", user);
configuration.setProperty("hibernate.connection.password", pass);
// rebuilds the configuration using my test database
HibernateUtil.rebuildSessionFactory(configuration);
// executes a script stored in test/resources/cucumber
try {
Class.forName("com.mysql.jdbc.Driver");
Connection conn = DriverManager.getConnection(url, user, pass);
ScriptRunner runner = new ScriptRunner(conn, false, true);
runner.runScript(new BufferedReader(new FileReader("test/resources/cucumber/db.sql")));
} catch (Exception e) {
throw new RuntimeException(e.getMessage(), e);
}
}
}