Good morning,
We are executing the following code and we are reaching an error message when loading a certain number of dlls:
File file = new File("C:\\Users\\jevora\\Downloads\\dng_tests\\dllsCopies");
file.mkdirs();
for (int i = 1; i < 10000; i++) {
String filename = "heatedTankCvode" + i + ".dll";
Files.copy(new File("C:\\Users\\jevora\\Downloads\\dng_tests\\heatedTankCvode.dll").toPath(),
new File(file, filename).toPath(), StandardCopyOption.REPLACE_EXISTING);
NativeLibrary.getInstance(new File(file, filename).getAbsolutePath());
System.out.println("Loaded: " + filename);
}
As you can see here, we want to load 10,000 dlls using JNA. However,in the following log, the process stops at loading the instance 1,051:
Loaded: heatedTankCvode1048.dll
Loaded: heatedTankCvode1049.dll
Loaded: heatedTankCvode1050.dll
Exception in thread "main" java.lang.UnsatisfiedLinkError: Unable to load library 'C:\Users\jevora\Downloads\dng_tests\dllsCopies\heatedTankCvode1051.dll': Native library (win32-x86-64/C:\Users\jevora\Downloads\dng_tests\dllsCopies\heatedTankCvode1051.dll)
About the code, first we copy the dll in a new location with a different name and, then, we try to load it. We wonder if there is a limitation to the amount of dlls that can be loaded. Is there a limitation? can we overcome it?
Thanks in advance
EDIT: I've tried with several memory configurations and it always stop in the 1051 instance
I think that the cause might be explained by this old Microsoft Forum post:
DLL Limit?
It appears that each DLL that you are loading is consuming a TLS (thread local storage) slot. There is a per process limit of 1088 on the number of TLS slots. From all that I have read, the limit is hard ... and there is no way to increase it.
From what I have read, a DLL doesn't have to use TLS, so you should investigate if you can change the way that your DLLs are created so that they don't do this.
Related
I have a temporary file which I want to send the client from the controller in the Play Framework. Can I delete the file after opening a connection using FileInputStream? For example can I do something like this -
File file = getFile();
InputStream is = new FileInputStream(file);
file.delete();
renderBinary(is, "name.txt");
What if file is a large file? If I delete the file, will subsequent reads() on InputStream give an error? I have tried with files of around 1MB I don't get an error.
Sorry if this is a very naive question, but I could not find anything related to this and I am pretty new to Java
I just encountered this exact same scenario in some code I was asked to work on. The programmer was creating a temp file, getting an input stream on it, deleting the temp file and then calling renderBinary. It seems to work fine even for very large files, even into the gigabytes.
I was surprised by this and am still looking for some documentation that indicates why this works.
UPDATE: We did finally encounter a file that caused this thing to bomb. I think it was over 3 Gb. At that point, it became necessary to NOT delete the file while the rendering was in process. I actually ended up using the Amazon Queue service to queue up messages for these files. The messages are then retrieved by a scheduled deletion job. Works out nicely, even with clustered servers on a load balancer.
It seems counter-intuitive that the FileInputStream can still read after the file is removed.
DiskLruCache, a popular library in the Android world originating from the libcore of the Android platform, even relies on this "feature", as follows:
// Open all streams eagerly to guarantee that we see a single published
// snapshot. If we opened streams lazily then the streams could come
// from different edits.
InputStream[] ins = new InputStream[valueCount];
try {
for (int i = 0; i < valueCount; i++) {
ins[i] = new FileInputStream(entry.getCleanFile(i));
}
} catch (FileNotFoundException e) {
....
As #EJP pointed out in his comment on a similar question, "That's how Unix and Linux behave. Deleting a file is really deleting its name from the directory: the inode and the data persist while any processes have it open."
But I don't think it is a good idea to rely on it.
I have been tearing my hair out on this and thus I am looks for some help .
I have a loop of code that performs the following
//imports ommitted
public void afterPropertiesSet() throws Exception{
//building of URL list ommitted
// urlMap is a HashMap <String,String> created and populated just prior
for ( Object urlVar : urlMap.keySet() ){
String myURLvar = urlMap.get(urlVar.toString);
System.out.println ("URL is "+myURLvar );
BufferedImage imageVar = ImageIO.read(myURLvar);//URL confirmed to be valid even for executions that fail
String fileName2Save = "filepath"// a valid file path
System.out.println ("Target path is "+fileName2Save );
File file2Save = new File (fileName2Save);
fileName2Save.SetWriteable(true);//set these just to be sure
fileName2Save.SetReadable(true);
try{
ImageIO.write (imageVar,"png",file2save)//error thrown here
}catch (Exception e){
System.out.println("R: "+file2Save.canRead()+" W: "+file2Save.canWrite()+" E:"+file2Save.canExecute()+" Exists: "+file2Save.exists+" is a file"+file2Save.isFile() );
System.out.println("parent Directory perms");// same as above except on parent directory of destination
}//end try
}//end for
}
This all runs on Windows 7 and JDK 1.6.26 and Netbeans,Tomcat 7.0.14 . The target directory is actually inside my netbeans project directory in a folder for a normal web app ( outside WEB-INF) where I would expect normally to have permission to write files.
When the error occurs I get one of two results for the file a.) All false b.)all true. The Parent directory permission never change all true except for isFile.
The error thrown ( java.IO.error with "access denied" ") does not occur every time ... in fact 60% of the time the loop runs it throws no error. The remaining 40% of the time I get the error on 1 of the 60+ files it writes. Infrequently the same one. The order in which the URLs it starts from changes everytime so the order in which the files are written is variable. The file names have short concise names like "1.png". The images are small..less then 8k.
In order to make sure the permissions are correct I have :
Given "full control" to EVERYONE from the net beans project directory down
Run the JDK,JRE and Netbeans as Administrator
Disabled UAC
Yet the error persists. Google searches for this seem to run the gamut and often read like vodoo. Clearly I ( and Java and Netbeans etc ) should have permission to write a file to the directory .
Anyone have any insight ? This is all ( code and the web server hosting the URL) on a closed system so I can't cut and paste code or stacktrace.
Update: I confirmed the imageURL is valid by doing a println & toString prior to each read. I then confirmed that a.) the web server hosting the target URL returned the image with a http 200 code b.) that the URL returned the image when tested in a web browser. In testing I also put a if () in after the read to confirm that the values was not NULL or empty. I also put in tests for NULL on all the other values . They are always as expected even for a failure .The error always occurs inside the try block. The destination directory is the same every execution. Prior to every execution the directory is empty.
Update 2: Here is one of the stack traces ( in this case perms for file2Save are R: True W:True E: True isFile:True exists:True )
java.io.FileNotFoundException <fullFilepathhere> (Access is denied)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
at javax.imageio.stream.FileImageOutputStream.<init>(FileImageOutputStream.java:53)
at com.sun.imageio.spi.FileImageOutputStreamSpi.createOutputStreamInstance(FileImageOutputStreamSpi.java:37)
at javax.imageio.ImageIO.createImageOutputStream(ImageIO.java:393)
at javax.imageio.ImageIO.write(ImageIO.java:1514)
at myPackage.myClass.afterPropertiesSet(thisClassexample.java:204)// 204 is the line number of the ImageIO write
This may not answer your problem since there can be many other possibilties to your limited information.
One common possibilty for not being able to write a file in web application is the file locking issue on Windows if the following four conditions are met simultaneously:
the target file exists under web root, e.g. WEB-INF folder and
the target file is served by the default servlet and
the target file has been requested at least once by client and
you are running under Windows
If you are trying to replace such a file that meets all of the four conditions, you will not be able to because some servlet containers such as tomcat and jetty will buffer the static contents and lock the files so you are unable to replace or change them.
If your web application has exactly this problem, you should not use the default servlet to serve the file contents. The default servlet is desigend to serve the static content which you do not want to change, e.g. css files, javascript files, background images, etc.
There is a trick to solve the file locking issue on Windows for jetty by disabling the NIO http://docs.codehaus.org/display/JETTY/Files+locked+on+Windows
The trick is useful for development process, e.g. you want to edit the css file and see the change without restarting your web application, but it is not recommended for production mode. If your web application relies on this trick in the production process, then you should seriously consider redesign your codes.
I cannot tell you what's going on or why... I have a feeling that it's something dependent on the way ImageIO tries to save the image. What you could do is saving the BufferedImage by leveraging the ByteArrayOutputStream as described below:
BufferedImage bufferedImage = ImageIO.read(new File("sample_image.gif"));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ImageIO.write( bufferedImage, "gif", baos );
baos.flush(); //Is this necessary??
byte[] resultImageAsRawBytes = baos.toByteArray();
baos.close(); //Not sure how important this is...
OutputStream out = new FileOutputStream("myImageFile.gif");
out.write(resultImageAsRawBytes);
out.close();
I'm not really familiar with the ByteArrayOutputStream, but I guess its reset() function could be handy when dealing with saving multiple files. You could also try using its writeTo(OutputStream out) if you prefer. Documentation here.
Let me know how it goes...
Hello I have following Problem:
Within an uninstall-process I load a JAR (jdbc-driver).
URL pDriverJar = jarToDelete.toURI().toURL();
URL[] lURLList = new URL[]{pDriverJar};
URLClassLoader lLoader = new URLClassLoader(lURLList, Thread.currentThread().getContextClassLoader());
Thread.currentThread().setContextClassLoader(lLoader);
Class<?> aClass = Class.forName("jdbc.Driver"); // was Oracle: oracle.jdbc.OracleDriver but should not be important
if(jarToDelete.delete()){
System.out.println("deleted");
}else {
jarToDelete.deleteOnExit();
}
After terminiation of the JVM, the jar is still existant.
As a workarround, I've created a tempfile, and copied the Jar to that tempfile. But now the Tempfile will not be deleted.
I've read, that if the ClassLoad is GC, the loaded jars can be removed.
Does anyone have an Idea, how to delete this File?
It depends on the operating system. Windows will not let you delete files that are in-use, but Linux will.
One solution would be to start a second process that waits for your JVM to die and then deletes the file, as even if you clear all references to classloaders using it, there is no guarantee that they will release the file. There is no way to force garbage collection (or even finalization) of an object.
Another solution would be to write the classloader that loads the Jar. That way, when you want to get rid of it, you can be certain that the Jar is closed. If the only object that opened it was your classloader, then you can be certain it is free and should be deletable.
This issue was fixed in Java 7; use the close() method in the ClassLoader class. For older versions, there are several options:
Write a custom classloader, e.g. like this
Use reflection and close all JarFile instances; they reside in sun.misc.URLClassPath
As I wanted to kill off a jar-file on the classpath that I needed for java11 but led to problems in java 1.8 or below; A solution that I used, without having to start another process to kill the jar file, was to just make the file 0 bytes by
FileOutputStream out = new FileOutputStream(f);
out.write(new byte[0]);
out.flush(); out.close();
And next time the application starts this code then manages to delete it, even though the 0-byte-file is still in the manifest:
if (f.exists()){
if (f.delete() == false) f.deleteOnExit();
}
I'm trying to use the following code to access one byte with offset of 50 bytes in a raw disk.
randomAccessFile = new RandomAccessFile("C:", "r");
randomAccessFile.seek(50);
byte[] buffer = new byte[1];
randomAccessFile.read(buffer);
But all what I get is the following error:
java.io.FileNotFoundException: C: (Acceso denegado)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:98)
at pru.lseek.main(lseek.java:26)
Is there any way to access a precise byte in a drive from java?
I was looking by myself for a possibility to access raw data form a physical drive. And now as I got it to work, I just want to tell you how. You can access raw disk data directly from within java ... just run the following code with administrator priviliges:
File diskRoot = new File ("\\\\.\\PhysicalDrive0");
RandomAccessFile diskAccess = new RandomAccessFile (diskRoot, "r");
byte[] content = new byte[1024];
diskAccess.readFully (content);
So you will get the first kB of your first physical drive on the system. To access logical drives - as mentioned above - just replace 'PhysicalDrive0' with the drive letter e.g. 'D:'
oh yes ... I tried with Java 1.7 on a Win 7 system ...
Just have a look at the naming of physical drives at http://support.microsoft.com/kb/100027/en-us
If you are interested in writing to a raw volume under Windows, try this (needs Java 7).
String pathname;
// Full drive:
// pathname = "\\\\.\\PhysicalDrive0";
// A partition (also works if windows doesn't recognize it):
pathname = "\\\\.\\GLOBALROOT\\ArcName\\multi(0)disk(0)rdisk(0)partition(5)";
Path diskRoot = ( new File( pathname ) ).toPath();
FileChannel fc = FileChannel.open( diskRoot, StandardOpenOption.READ,
StandardOpenOption.WRITE );
ByteBuffer bb = ByteBuffer.allocate( 4096 );
fc.position( 4096 );
fc.read( bb );
fc.position( 4096 );
//fc.write( bb ); // careful!
fc.close();
Of course, you have to make sure the device is writable and not accessed/locked by the system. Also make sure your application runs with the necessary privileges (elevated privileges).
Btw: Using new RandomAccessFile(drive, "rw") doesn't seem to work because Java doesn't open the file handle in a mode which is compatible to raw devices (exception is java.io.FileNotFoundException (The parameter is incorrect)). But reading works fine also with RandomAccessFile.
RandomAccessFile is not meant to open directories to manipulate entries, you need to create or remove files. "Acceso denegado" probably mean access denied.
To do this anyway you need JNI.
EDIT: What you are trying to do, is really complicated, there is no common way to do that. You can access the harddisc sector by sector, but then you would have to interpret it's structure, which obviously depends on the file system, FAT,NTFS,HPFS etc.
Under Linux you can try to open /dev/<device>, e.g. /dev/hda, or /dev/sdb2. This will give you access to a raw disk (or a partition only) but requires that you have appropriate rights—a “normal” user does not have them, though.
Java can only access files. Unix has the concept of "raw devices" as files in the /dev directory, so what you want is possible there. But not on windows, because it has no such file representation of the raw HD data.
In windows you need to access the raw device identifier as a file. It should work if you pass in the file "\\.\c:", you are using the device UNC name \.\c: (\. means this machine).
For Vista and later I don't think it will work correctly as there are mechanisms in place to prevent raw access to the disk for anything other than device drivers (don't quote me on that)
#hunsricker : note that accessing raw devices require some privileges (depends on drive : removable or not / depends on file system for WinXP : iso9660 is allowed, FAT is not).
Note also that size of read does matter (depending on OS) :
On an iso9660 filesystem, read(1024 bytes) works on XP but fails on Seven.
On Seven it look like the reads must be block aligned : read(2048 bytes) works.
In unix, you may read/write from /dev files. (I'm not sure)
In Windows, I think you need to read/write disk sectors via JNI(Java Native Interface). Calls some C library to talk to the OS.
update: In C library, you may need to use Win32API to get the file handle for example CreateFile(..) function.
https://metacpan.org/pod/Win32API::File
http://jan.newmarch.name/ssw/files/win32.html
What I am doing is I am reading in a html file and I am looking for a specific location in the html for me to enter some text.
So I am using a bufferedreader to read in the html file and split it by the tag . I want to enter some text before this but I am not sure how to do this. The html would then be along the lines of ...(newText)(/HEAD) (The brackets round head are meant to be angled brackets. Don't know how to insert them)
Would I need a PrintWriter to the same file and if so, how would I tell that to write it in the correct location.
I am not sure which way would be most efficient to do something like this.
Please Help.
Thanks in advance.
Here is part of my java code:
File f = new File("newFile.html");
FileOutputStream fos = new FileOutputStream(f);
PrintWriter pw = new PrintWriter(fos);
BufferedReader read = new BufferedReader(new FileReader("file.html"));
String str;
int i=0;
boolean found = false;
while((str= read.readLine()) != null)
{
String[] data = str.split("</HEAD>");
if(found == false)
{
pw.write(data[0]);
System.out.println(data[0]);
pw.write("</script>");
found = true;
}
if(i < 1)
{
pw.write(data[1]);
System.out.println(data[1]);
i++;
}
pw.write(str);
System.out.println(str);
}
}
catch (Exception e) {
e.printStackTrace( );
}
When I do this it gets to a point in the file and I get these errors:
FATAL ERROR: MERLIN: Unable to connect to EDG API,
Cannot find .edg_properties file.,
java.lang.OutOfMemoryError: unable to create new native thread,
Cannot truncate table,
EXCEPTION:Cannot open connection to server: SQLExceptio,
Caught IOException: java.io.IOException: JZ0C0: Connection is already closed, ...
I'm not sure why I get these or what all of these mean?
please Help.
Should be pretty easy:
Read file into a String
Split into before/after chunks
Open a temp file for writing
Write before chunk, your text, after chunk
Close up, and move temp file to original
Sounds like you are wondering about the last couple steps in particular. Here is the essential code:
File htmlFile = ...;
...
File tempFile = File.createTempFile("foo", ".html");
FileWriter writer = new FileWriter(tempFile);
writer.write(before);
writer.write(yourText);
writer.write(after);
writer.close();
tempFile.renameTo(htmlFile);
Most people suggest writing to a temporary file and then copying the temporary file over the original on successful completion.
The forum thread has some ideas of how to do it.
GL.
For reading and writing you can use FileReaders/FileWriters or the corresponding IO stream classes.
For the editing, I'd suggest to use an HTML parser to handle the document. It can read the HTML document into an internal datastructure which simplifies your effort to search for content and apply modification. (Most?) Parsers can serialize the document to HTML again.
At least you're sure to not corrupt the HTML document structure.
Following up on the list of errors in your edit, a lot of that possibly stems from the OutOfMemoryError. That means you simply ran out of memory in the JVM, so Java was unable to allocate objects. This may be caused by a memory leak in your application, or it could simply be that the work you're trying to do does need more memory transiently than you have allocated it.
You can increase the amount of memory that the JVM starts up with by providing the Xmx argument to the java executable, e.g.:
-Xmx1024m
would set the maximum heap size to 1024 megabytes.
The other issues might possibly caused by this; when objects can't reliably be created or modified, lots of weird things tend to happen. That said, there's a few things that look like you can take action. In particular, whatever MERLIN is it looks like it can't do it's work because it needs a property file for EDG, which it's unable to find in the location it's looking. You'll probably need to either put a config file there, or tell it to look at another location.
The other IOExceptions are fairly self-explanatory. Your program could not establish a connection to the server because of a SQLException (the underlying exception itself will probably be found in the logs); and some other part of the program tried to communicate to a remote machine using a closed connection.
I'd look at fixing the properties file (if it's not a benign error) and the memory issues first, and then seeing if any of the remaining problems still manifest.