libusb cannot deinitialize context - java

I need some help with a IllegalStateException in Java.
I got a sourcecode that should read out Data from a USB Device.
That code is not finished yet, but I already got the following error report
Exception in thread "main" java.lang.IllegalStateException: default context is not initialized at org.usb4java.Libusb.exit(Native Method) at testnew.main(testnew.java:122)
Line 122 is LibUsb.exit(null)
Code is below
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.IntBuffer;
import javax.usb.UsbConfiguration;
import javax.usb.UsbDevice;
import javax.usb.UsbDeviceDescriptor;
import javax.usb.UsbDisconnectedException;
import javax.usb.UsbException;
import javax.usb.UsbHostManager;
import javax.usb.UsbHub;
import javax.usb.UsbInterface;
import javax.usb.UsbInterfacePolicy;
import javax.usb.UsbNotActiveException;
import javax.usb.event.UsbPipeDataEvent;
import javax.usb.event.UsbPipeErrorEvent;
import javax.usb.event.UsbPipeListener;
import org.usb4java.BufferUtils;
import org.usb4java.DeviceHandle;
import org.usb4java.LibUsb;
import org.usb4java.LibUsbException;
public class testnew {
private final static short VENDOR_ID = 0x0403;
private final static short PRODUCT_ID = 0x6001;
private static byte IN_ENDPOINT = (byte) 0x81;
private static long TIMEOUT = 5000;
private final static int INTERFACE = 0;
private final static Object CONNECT_HEADER = 000;
private final static Object CONNECT_BODY = 000;
public static UsbDevice getHygrometerDevice(UsbHub hub) {
UsbDevice launcher = null;
for (Object object : hub.getAttachedUsbDevices()) {
UsbDevice device = (UsbDevice) object;
if (device.isUsbHub()) {
launcher = getHygrometerDevice((UsbHub) device);
if (launcher != null)
return launcher;
} else {
UsbDeviceDescriptor desc = device.getUsbDeviceDescriptor();
if (desc.idVendor() == VENDOR_ID && desc.idProduct() == PRODUCT_ID)
return device;
}
}
return null;
}
public static char readKey() {
try {
String line = new BufferedReader(new InputStreamReader(System.in)).readLine();
if (line.length() > 0)
return line.charAt(0);
return 0;
} catch (IOException e) {
throw new RuntimeException("Unable to read key", e);
}
}
public static ByteBuffer read(DeviceHandle handle, int size) {
ByteBuffer buffer = BufferUtils.allocateByteBuffer(size).order(ByteOrder.LITTLE_ENDIAN);
IntBuffer transferred = BufferUtils.allocateIntBuffer();
int result = LibUsb.bulkTransfer(handle, IN_ENDPOINT, buffer, transferred, TIMEOUT);
if (result != LibUsb.SUCCESS) {
throw new LibUsbException("Unable to read data", result);
}
System.out.println(transferred.get() + " bytes read from device");
return buffer;
}
public static void main(String[] args) {
// Search for the missile launcher USB device and stop when not found
UsbDevice device;
try {
device = getHygrometerDevice(UsbHostManager.getUsbServices().getRootUsbHub());
if (device == null) {
System.err.println("Missile launcher not found.");
System.exit(1);
return;
}
// Claim the interface
UsbConfiguration configuration = device.getUsbConfiguration((byte) 1);
UsbInterface iface = configuration.getUsbInterface((byte) INTERFACE);
iface.claim(new UsbInterfacePolicy() {
#Override
public boolean forceClaim(UsbInterface usbInterface) {
return true;
}
});
iface.getUsbEndpoint(IN_ENDPOINT).getUsbPipe().addUsbPipeListener(new UsbPipeListener() {
#Override
public void errorEventOccurred(UsbPipeErrorEvent arg0) {
// TODO Auto-generated method stub
}
#Override
public void dataEventOccurred(UsbPipeDataEvent arg0) {
for (byte b : arg0.getData())
System.out.print(b);
System.out.print("\n");
}
});
;
} catch (UsbNotActiveException | UsbDisconnectedException | UsbException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// Deinitialize the libusb context
LibUsb.exit(null);
}
}
Any suggestions?

you did not initialize the context this is why you get the error when you try to deinitialize it, see http://www.programcreek.com/java-api-examples/index.php?api=org.usb4javaLibUsb example 17
public static void main(String[] args)
{
//Initialize the libusb context
int result = LibUsb.Init(null);
if (result != LibUsb.SUCCESS){
throw new LibUsbException("Unable to initialze libusb",result);
}
[...]
how to open the pipe depends on what transfer type you want to choose (bulk transfer, sychronous transfer, asynchronous transfer), see http://usb4java.org/quickstart/javax-usb.html
for synchronous transfer you can use (copied from http://usb4java.org/quickstart/javax-usb.html)
UsbEndpoint endpoint = iface.getUsbEndpoint((byte) 0x83);
UsbPipe pipe = endpoint.getUsbPipe();
pipe.open();
try
{
byte[] data = new byte[8];
int received = pipe.syncSubmit(data);
System.out.println(received + " bytes received");
}
finally
{
pipe.close();
}
There are IN endpoints and OUT endpoints, you write to OUT and read from IN. Control transfers go to EP0. All USB communication is initiated by the host device, meaning the USB device can not even initiate a communication.
for deeper information on USB protocol see http://www.beyondlogic.org/usbnutshell/usb1.shtml

Related

How Do I play locally stored encrypted video files in android studio

I have a video file I have encrypted using AES algorithm but I need to decrypt the video before playing with video view. The problem is that the time it takes to decrypt the video is too much. Is there a way I can do it.
UPDATE:
E/ExoPlayerImplInternal: Playback error
com.google.android.exoplayer2.ExoPlaybackException: Source error
at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:579)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:164)
at android.os.HandlerThread.run(HandlerThread.java:65)
Caused by: com.google.android.exoplayer2.source.UnrecognizedInputFormatException: None of the available extractors (FragmentedMp4Extractor, Mp4Extractor, FlvExtractor, FlacExtractor, WavExtractor, AmrExtractor, PsExtractor, OggExtractor, TsExtractor, MatroskaExtractor, AdtsExtractor, Ac3Extractor, Ac4Extractor, Mp3Extractor, JpegExtractor) could read the stream.
at com.google.android.exoplayer2.source.BundledExtractorsAdapter.init(BundledExtractorsAdapter.java:92)
at com.google.android.exoplayer2.source.ProgressiveMediaPeriod$ExtractingLoadable.load(ProgressiveMediaPeriod.java:1026)
at com.google.android.exoplayer2.upstream.Loader$LoadTask.run(Loader.java:415)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
at java.lang.Thread.run(Thread.java:764)
When the file is stored locally you probably don't need to encrypt it at all. The local file storage in andorid is not visible for the user or any other apps. Only your app has acces to the storage so you don't need to worry about the user reading confidential data.
Edit: As i just saw your recent comment with local you mean on the device. You can save the video in the internal storage where nothing other than your app has acces to. Take a look at the documentary: https://developer.android.com/training/data-storage/app-specific
Here you go. I don't know if it works, but it should:
import android.net.Uri;
import android.text.TextUtils;
import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
import com.google.android.exoplayer2.C;
import com.google.android.exoplayer2.upstream.BaseDataSource;
import com.google.android.exoplayer2.upstream.DataSource;
import com.google.android.exoplayer2.upstream.DataSpec;
import com.google.android.exoplayer2.upstream.FileDataSource;
import com.google.android.exoplayer2.upstream.TransferListener;
import com.google.android.exoplayer2.util.Assertions;
import java.io.EOFException;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import javax.crypto.BadPaddingException;
import javax.crypto.Cipher;
import javax.crypto.IllegalBlockSizeException;
import javax.crypto.NoSuchPaddingException;
import javax.crypto.ShortBufferException;
import javax.crypto.spec.SecretKeySpec;
import static com.google.android.exoplayer2.util.Util.castNonNull;
import static java.lang.Math.min;
public class EncryptedFileDataSource extends BaseDataSource {
public static class EncryptedFileDataSourceException extends IOException {
public EncryptedFileDataSourceException(Throwable cause) {
super(cause);
}
public EncryptedFileDataSourceException(String message, IOException cause) {
super(message, cause);
}
}
/**
* {#link DataSource.Factory} for {#link EncryptedFileDataSource} instances.
*/
public static final class Factory implements DataSource.Factory {
#Nullable
private TransferListener listener;
private byte[] key;
private String transformation;
public Factory() {
this(null, null);
}
public Factory(byte[] key) {
this(key, null);
}
public Factory(byte[] key, String transformation) {
setKey(key);
setTransformation(transformation);
}
public void setKey(byte[] key) {
if (key == null) {
return;
}
this.key = key;
}
public void setTransformation(String transformation) {
if (transformation == null) {
transformation = "AES/ECB/PKCS5Padding";
}
this.transformation = transformation;
}
/**
* Sets a {#link TransferListener} for {#link EncryptedFileDataSource} instances created by this factory.
*
* #param listener The {#link TransferListener}.
* #return This factory.
*/
public Factory setListener(#Nullable TransferListener listener) {
this.listener = listener;
return this;
}
#Override
public EncryptedFileDataSource createDataSource() {
EncryptedFileDataSource dataSource = new EncryptedFileDataSource(key, transformation);
if (listener != null) {
dataSource.addTransferListener(listener);
}
return dataSource;
}
}
#Nullable
private RandomAccessFile file;
#Nullable
private Uri uri;
private long bytesRemaining;
private boolean opened;
private byte[] key;
private String transformation;
private Cipher cipher;
private byte[] tmpBuffer;
private EncryptedFileDataSource(byte[] key, String transformation) {
super(false);
setKey(key);
setTransformation(transformation);
}
public void setKey(byte[] key) {
this.key = key;
}
public void setTransformation(String transformation) {
this.transformation = transformation;
}
#Override
public long open(#NonNull DataSpec dataSpec) throws EncryptedFileDataSourceException {
try {
byte[] key = this.key;
String transformation = this.transformation;
Uri uri = dataSpec.uri;
this.uri = uri;
transferInitializing(dataSpec);
if (key != null && transformation != null) {
SecretKeySpec keySpec = new SecretKeySpec(key, "AES");
cipher = Cipher.getInstance(transformation);
cipher.init(Cipher.DECRYPT_MODE, keySpec);
}
this.file = openLocalFile(uri);
file.seek(dataSpec.position);
bytesRemaining = dataSpec.length == C.LENGTH_UNSET ? file.length() - dataSpec.position
: dataSpec.length;
if (bytesRemaining < 0) {
throw new EOFException();
}
} catch (IOException
| NoSuchPaddingException
| NoSuchAlgorithmException
| InvalidKeyException e) {
throw new EncryptedFileDataSourceException(e);
}
opened = true;
transferStarted(dataSpec);
return bytesRemaining;
}
#Override
public int read(#NonNull byte[] buffer, int offset, int readLength) throws EncryptedFileDataSourceException {
if (readLength == 0) {
return 0;
} else if (bytesRemaining == 0) {
return C.RESULT_END_OF_INPUT;
} else {
int bytesRead, bytesStored;
try {
int length = (int) min(bytesRemaining, readLength);
Cipher cipher = this.cipher;
if (cipher != null) {
tmpBuffer = resize(tmpBuffer, length);
bytesRead = castNonNull(file).read(tmpBuffer, 0, length);
bytesStored = cipher.doFinal(tmpBuffer, 0, length, buffer, offset);
} else {
bytesStored = bytesRead = castNonNull(file).read(buffer, offset, length);
}
} catch (IOException
| BadPaddingException
| IllegalBlockSizeException
| ShortBufferException e) {
throw new EncryptedFileDataSourceException(e);
}
if (bytesRead > 0) {
bytesRemaining -= bytesRead;
bytesTransferred(bytesRead);
}
return bytesStored;
}
}
#Nullable
#Override
public Uri getUri() {
return uri;
}
#Override
public void close() throws EncryptedFileDataSourceException {
uri = null;
try {
if (file != null) {
file.close();
}
} catch (IOException e) {
throw new EncryptedFileDataSourceException(e);
} finally {
file = null;
key = null;
transformation = null;
cipher = null;
if (opened) {
opened = false;
transferEnded();
}
byte[] tmpBuffer = this.tmpBuffer;
this.tmpBuffer = null;
if (tmpBuffer != null) {
for (int i = tmpBuffer.length - 1; i >= 0; i -= 2) {
tmpBuffer[i] = (byte) 0;
}
}
}
}
static byte[] resize(byte[] b, int newLen) {
if (newLen < 0) return b;
if (b == null || b.length < newLen) {
return new byte[newLen];
} else return b;
}
private static RandomAccessFile openLocalFile(Uri uri) throws FileDataSource.FileDataSourceException {
try {
return new RandomAccessFile(Assertions.checkNotNull(uri.getPath()), "r");
} catch (FileNotFoundException e) {
if (!TextUtils.isEmpty(uri.getQuery()) || !TextUtils.isEmpty(uri.getFragment())) {
throw new FileDataSource.FileDataSourceException(
String.format(
"uri has query and/or fragment, which are not supported. Did you call Uri.parse()"
+ " on a string containing '?' or '#'? Use Uri.fromFile(new File(path)) to"
+ " avoid this. path=%s,query=%s,fragment=%s",
uri.getPath(), uri.getQuery(), uri.getFragment()),
e);
}
throw new FileDataSource.FileDataSourceException(e);
}
}
}
Use it like this:
public void playVideo(Uri uri, byte[] key, #Nullable String transformation) {
playVideo(MediaItem.fromUri(uri), key, transformation);
}
public void playVideo(MediaItem media, byte[] key, #Nullable String transformation) {
ExoPlayer player = new SimpleExoPlayer.Builder(this).build();
playerView.setPlayer(player);
try {
DataSource.Factory dataSourceFactory = new EncryptedFileDataSource.Factory(key, transformation);
ExtractorsFactory extractorsFactory = new DefaultExtractorsFactory();
ProgressiveMediaSource.Factory progressiveFactory = new ProgressiveMediaSource.Factory(
dataSourceFactory,
extractorsFactory
);
ProgressiveMediaSource mediaSource = progressiveFactory.createMediaSource(media);
player.setMediaSource(mediaSource);
//player.setMediaSource(mediaSource, startPositionMs: long);
//player.setMediaSource(mediaSource, resetPosition: boolean);
player.setPlayWhenReady(true);
} catch (Exception e) {
e.printStackTrace();
}
}
I used the following sources to make this:
https://github.com/google/ExoPlayer/blob/4364b9150f236813a2070fc79d3bcb76e11969ac/library/core/src/main/java/com/google/android/exoplayer2/upstream/FileDataSource.java
Android + exoplayer: play AES encrypted videos, locally

TFTP Client using Java NIO

I am trying to create a TFTP client using java NIO. I am able to receive first 512 bytes of data from server, but not able to send acknowledgement to server for getting next block of packet. I am new to java NIO and networking. Not able to find solution for it. So can anyone help me on this to find fix for it? Thanks in advance
package app.sdc.business;
import java.io.ByteArrayOutputStream;
import java.io.DataOutputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.DatagramChannel;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.util.Iterator;
import app.sdc.business.NetworkElementPool.NetworkElement;
public class TftpNioClient {
static byte OP_ERROR = 5, OP_DATAPACKET = 3, OP_ACK = 4, OP_RRQ = 1, OP_WRQ = 2;
static final String LOCALHOST = "localhost";
static InetSocketAddress server = new InetSocketAddress(LOCALHOST, 69);
// main method
public static void main(String[] args) throws IOException {
processDownload();
}
// Will start downloading of all files
public static void processDownload() throws IOException {
Selector sel = Selector.open();
for (NetworkElement ne : NetworkElementPool.getNetworkElements()) {
DatagramChannel channel = DatagramChannel.open();
channel.configureBlocking(false);
channel.connect(server);
channel.register(sel, SelectionKey.OP_READ | SelectionKey.OP_WRITE, ne);
}
int counter = 0;
while (true) {
int n = sel.select(3000);
if (n < 1) {
continue;
}
Iterator<SelectionKey> itr = sel.selectedKeys().iterator();
while (itr.hasNext()) {
SelectionKey key = itr.next();
itr.remove();
if (key.isWritable()) {
counter++;
System.out.println("channel Write...");
downloadUsingTFTPProtocol(key);
} else if (key.isReadable()) {
System.out.println("Channel Read");
}
}
if (counter >= NetworkElementPool.getNetworkElements().size()) {
break;
}
}
}
// method for downloading file
private static void downloadUsingTFTPProtocol(SelectionKey keyy) throws IOException {
ByteArrayOutputStream byteOutOS = new ByteArrayOutputStream();
Selector sel = keyy.selector();
NetworkElement ne = (NetworkElement) keyy.attachment();
String fileName = ne.getFilename();
DatagramChannel channel = DatagramChannel.open();
channel.configureBlocking(false);
channel.register(sel, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
boolean reqSent = false;
ByteBuffer sendBuffer = null;
ByteBuffer receivedBuffer = ByteBuffer.allocate(516);
boolean stop = false;
byte[] dataByte = null;
boolean received = false;
outer: while (true) {
int n = sel.select();
if (n < 1) {
continue;
}
Iterator<SelectionKey> itr = sel.selectedKeys().iterator();
while (itr.hasNext()) {
SelectionKey key = itr.next();
itr.remove();
DatagramChannel dc = (DatagramChannel) key.channel();
if (!received && key.isReadable()) {
System.out.println("receive packet...");
receivedBuffer.clear();
dc.receive(receivedBuffer);
stop = receivedBuffer.position() < 512;
receivedBuffer.flip();
while (receivedBuffer.hasRemaining()) {
System.out.print(receivedBuffer.get());
}
System.out.println();
dataByte = receivedBuffer.array();
received = true;
}
if (key.isWritable()) {
if (!reqSent) {
System.out.println("Sending First Request....");
sendBuffer = createInitialReadRequest("SDCSource/" + fileName);
sendBuffer.flip();
dc.send(sendBuffer, server);
reqSent = true;
} else if (received) {
System.out.println("Send Acknowledgement");
byte[] opCode = new byte[] { dataByte[0], dataByte[1] };
if (opCode[1] == OP_ERROR) {
System.out.println("Error Occured...");
break outer;
} else if (opCode[1] == OP_DATAPACKET) {
byte[] blockNumber = { dataByte[2], dataByte[3] };
sendBuffer = getAcknowledgment(blockNumber, dc, server);
sendBuffer.flip();
dc.send(sendBuffer, server);
DataOutputStream dos = new DataOutputStream(byteOutOS);
dos.write(dataByte, 4, dataByte.length - 4);
}
}
received = false;
}
if (stop) {
break outer;
}
}
}
writeFile(byteOutOS, fileName);
}
// Creates request packet to send request at the beginning
private static ByteBuffer createInitialReadRequest(final String fileName) {
String mode = "octet";
int rrqByteLength = 2 + fileName.getBytes().length + 1 + mode.getBytes().length + 1;
byte[] rrqByteArray = new byte[rrqByteLength];
ByteBuffer reqBuf = ByteBuffer.allocate(rrqByteArray.length);
reqBuf.put((byte) 0).put((byte) OP_RRQ);
reqBuf.put(fileName.getBytes());
reqBuf.put((byte) 0);
reqBuf.put(mode.getBytes());
reqBuf.put((byte) 0);
return reqBuf;
}
// Creating acknowledgement code
private static ByteBuffer getAcknowledgment(byte[] blockNumber, DatagramChannel channel, InetSocketAddress server)
throws IOException {
byte[] acknowledge = { 0, OP_ACK, blockNumber[0], blockNumber[1] };
ByteBuffer buffer = ByteBuffer.allocate(acknowledge.length);
buffer.put(acknowledge);
return buffer;
}
// Create file after all packets have been received
private static void writeFile(ByteArrayOutputStream baoStream, String fileName) {
try {
OutputStream outputStream = new FileOutputStream("SDCTarget/" + fileName);
baoStream.writeTo(outputStream);
} catch (IOException e) {
e.printStackTrace();
}
}
}
package app.sdc.business;
import java.io.File;
import java.util.ArrayList;
import java.util.List;
public class NetworkElementPool {
private static List<NetworkElement> networkElements;
// create a list of network elements by reading files downloading. It is just a hard-coded way to create network elements
static {
networkElements = new ArrayList<NetworkElement>();
File sourceDir = new File("C:/OpenTFTPServer/SDCSource");
if (sourceDir.exists()) {
for (String filename : sourceDir.list()) {
networkElements.add(new NetworkElement("localhost", 8080, filename));
}
} else {
System.err.println("Network Elements couldn't found...");
}
}
public static List<NetworkElement> getNetworkElements() {
return networkElements;
}
// Represents a network element
public static class NetworkElement {
private String host;
private int port;
private String filename;
public String getHost() {
return host;
}
public int getPort() {
return port;
}
public String getFilename() {
return filename;
}
public NetworkElement() {
super();
}
public NetworkElement(String host, int port, String filename) {
super();
this.host = host;
this.port = port;
this.filename = filename;
}
#Override
public String toString() {
return "NetworkElement [host=" + host + ", port=" + port + ", filename=" + filename + "]";
}
}
}
Note: TftpClient class contain one foreach loop to start downloading file from more than one network element.
You're not sending the acknowledgements at the correct time, after a read: you're sending them every time around the loop. You don't need to wait for OP_WRITE to do a write. Just write whenever you need to. You shouldn't even register for OP_WRITE most of the time. See this answer for the correct technique. But in the case of TFTP it's dubious whether you need that at all. Or NIO either.
In your code, if the key.isReadable, then you can read the packet. Then immediately followed by that you can send the ACK back to TFTP server.
} else if (received) {
This block of code can go to the line immediately next to dc.receive(receivedBuffer);
You can refer this example java nio tftp client

How to control universal keyboard application on Windows by java?

i'm trying to develop an application for a universal keyboard which is for camera controlling. I'm able to communicate with device in some basic way on Linux over /dev/ttyACM0 pseudo terminal device. But as far as i know, this doesn't exist in Windows and i'm little bit confused when i examine the samples for comm API and rxtx library.
How can i communicate with this device like i can simply do on Linux? Here is the instractions manual for keyboard: http://www.videotec.com/dep/DCZ_1051.pdf
And here is the code:
Class1.java
package customkeyboard;
import java.io.FileInputStream;
import org.apache.commons.codec.binary.Hex;
public class CustomKeyboard {
public static void main(String[] args) {
// TODO code application logic here
FileInputStream in = null;
try {
in = new FileInputStream("/dev/ttyACM0");
int c = 0;
int streamCounter = 0;
String command = "";
CustomKeyboard2 key2 = new CustomKeyboard2();
key2.main(null);
while ((c = in.read()) != -1) {
if (streamCounter < 6) {
//System.out.print( + " - ");
command += Integer.toHexString(c);
streamCounter++;
if (streamCounter == 6) {
streamCounter = 0;
System.out.println(command);
byte[] bytes = Hex.decodeHex(command.toCharArray());
String deastranza = new String ( bytes, "UTF-8");
System.out.println(deastranza);
command = "";
}
} else {
}
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
Class2.java
package customkeyboard;
import java.io.FileOutputStream;
import org.apache.commons.codec.binary.Hex;
public class CustomKeyboard2 {
public static void main(String[] args) {
try {
FileOutputStream out = null;
out = new FileOutputStream("/dev/ttyACM0");
String command = "[LedImmediate]";
byte[] message = command.getBytes("UTF-8");
out.write(message);
command = "[Led+45]";
message = command.getBytes();
out.write(message);
} catch (Exception ex) {
ex.printStackTrace();
}
}
}

Is there a way to have FileChannels close automatically?

I am currently developing an application that requires random access to many (60k-100k) relatively large files.
Since opening and closing streams is a rather costly operation, I'd prefer to keep the FileChannels for the largest files open until they are no longer needed.
The problem is that since this kind of behaviour is not covered by Java 7's try-with statement, I'm required to close all the FileChannels manually.
But that is becoming increasingly too complicated since the same files could be accessed concurrently throughout the software.
I have implemented a ChannelPool class that can keep track of opened FileChannel instances for each registered Path. The ChannelPool can then be issued to close those channels whose Path is only weakly referenced by the pool itself in certain intervals.
I would prefer an event-listener approach, but I'd also rather not have to listen to the GC.
The FileChannelPool from Apache Commons doesn't address my problem, because channels still need to be closed manually.
Is there a more elegant solution to this problem? And if not, how can my implementation be improved?
import java.io.IOException;
import java.lang.ref.WeakReference;
import java.nio.channels.FileChannel;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;
import java.util.Map;
import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.ConcurrentHashMap;
public class ChannelPool {
private static final ChannelPool defaultInstance = new ChannelPool();
private final ConcurrentHashMap<String, ChannelRef> channels;
private final Timer timer;
private ChannelPool(){
channels = new ConcurrentHashMap<>();
timer = new Timer();
}
public static ChannelPool getDefault(){
return defaultInstance;
}
public void initCleanUp(){
// wait 2 seconds then repeat clean-up every 10 seconds.
timer.schedule(new CleanUpTask(this), 2000, 10000);
}
public void shutDown(){
// must be called manually.
timer.cancel();
closeAll();
}
public FileChannel getChannel(Path path){
ChannelRef cref = channels.get(path.toString());
System.out.println("getChannel called " + channels.size());
if (cref == null){
cref = ChannelRef.newInstance(path);
if (cref == null){
// failed to open channel
return null;
}
ChannelRef oldRef = channels.putIfAbsent(path.toString(), cref);
if (oldRef != null){
try{
// close new channel and let GC dispose of it
cref.channel().close();
System.out.println("redundant channel closed");
}
catch (IOException ex) {}
cref = oldRef;
}
}
return cref.channel();
}
private void remove(String str) {
ChannelRef ref = channels.remove(str);
if (ref != null){
try {
ref.channel().close();
System.out.println("old channel closed");
}
catch (IOException ex) {}
}
}
private void closeAll() {
for (Map.Entry<String, ChannelRef> e : channels.entrySet()){
remove(e.getKey());
}
}
private void maintain() {
// close channels for derefenced paths
for (Map.Entry<String, ChannelRef> e : channels.entrySet()){
ChannelRef ref = e.getValue();
if (ref != null){
Path p = ref.pathRef().get();
if (p == null){
// gc'd
remove(e.getKey());
}
}
}
}
private static class ChannelRef{
private FileChannel channel;
private WeakReference<Path> ref;
private ChannelRef(FileChannel channel, WeakReference<Path> ref) {
this.channel = channel;
this.ref = ref;
}
private static ChannelRef newInstance(Path path) {
FileChannel fc;
try {
fc = FileChannel.open(path, StandardOpenOption.READ);
}
catch (IOException ex) {
return null;
}
return new ChannelRef(fc, new WeakReference<>(path));
}
private FileChannel channel() {
return channel;
}
private WeakReference<Path> pathRef() {
return ref;
}
}
private static class CleanUpTask extends TimerTask {
private ChannelPool pool;
private CleanUpTask(ChannelPool pool){
super();
this.pool = pool;
}
#Override
public void run() {
pool.maintain();
pool.printState();
}
}
private void printState(){
System.out.println("Clean up performed. " + channels.size() + " channels remain. -- " + System.currentTimeMillis());
for (Map.Entry<String, ChannelRef> e : channels.entrySet()){
ChannelRef cref = e.getValue();
String out = "open: " + cref.channel().isOpen() + " - " + cref.channel().toString();
System.out.println(out);
}
}
}
EDIT:
Thanks to fge's answer I have now exactly what I needed. Thanks!
import com.google.common.cache.CacheBuilder;
import com.google.common.cache.CacheLoader;
import com.google.common.cache.LoadingCache;
import com.google.common.cache.RemovalListener;
import com.google.common.cache.RemovalNotification;
import java.io.IOException;
import java.nio.channels.FileChannel;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;
import java.util.concurrent.ExecutionException;
public class Channels {
private static final LoadingCache<Path, FileChannel> channelCache =
CacheBuilder.newBuilder()
.weakKeys()
.removalListener(
new RemovalListener<Path, FileChannel>(){
#Override
public void onRemoval(RemovalNotification<Path, FileChannel> removal) {
FileChannel fc = removal.getValue();
try {
fc.close();
}
catch (IOException ex) {}
}
}
)
.build(
new CacheLoader<Path, FileChannel>() {
#Override
public FileChannel load(Path path) throws IOException {
return FileChannel.open(path, StandardOpenOption.READ);
}
}
);
public static FileChannel get(Path path){
try {
return channelCache.get(path);
}
catch (ExecutionException ex){}
return null;
}
}
Have a look here:
http://code.google.com/p/guava-libraries/wiki/CachesExplained
You can use a LoadingCache with a removal listener which would close the channel for you when it expires, and you can specify expiry after access or write.

Java Server Non Blocking Query

I am using the following code to read some data from Android client. All is going fine. But now i am asked to make this server code non blocking. Is there any suggestions for this ? I was trying to use threads but dont know how ? I am beginner in Java :)
Thanks
import java.awt.image.BufferedImage;
import java.io.BufferedOutputStream;
import java.io.ByteArrayInputStream;
import java.io.DataInputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.Calendar;
import java.util.Date;
import javax.imageio.ImageIO;
public class Server {
//Server Constructor
public Server()
{}
//Variables Initialization
private static ServerSocket server;
byte[] imagetemp;
private static Socket socket1;
private static boolean newImage;
private static Sdfdata data;
private static boolean cond;
public static int port;
private static int number = 0;
//Image Availability return method
public boolean imageAvailable()
{
return newImage;
}
public boolean clientchk()
{
return socket1.isClosed();
}
//Image Flag set by Vis group when image read.
public void setImageFlag(boolean set)
{
newImage = set;
}
// Send the data to the Vis Group
public Sdfdata getData()
{
return data;
}
//Starts the Server
public static boolean start(int port1)
{
try {
port=port1;
server = new ServerSocket(port1);
System.out.println("Waiting for Client to Connect");
//New thread here
socket1=server.accept();
} catch (IOException e) {
System.out.println("Cannot Connect");
e.printStackTrace();
return false;
}
return true;
}
//Stops the Server
public boolean stop()
{
try {
socket1.close();
}
catch (IOException e)
{
e.printStackTrace();
return false;
}
return true;
}
/**
* #param args
* #throws IOException
*/
public static void main(String[] args) throws IOException {
// Starts the server
start(4444);
// DataInput Stream for reading the data
DataInputStream in = null;
try {
in = new DataInputStream(socket1.getInputStream());
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
cond=true;
do {
try
{
//Read Image Data
int length = in.readInt();
//Create an ByteArray of length read from Client for Image transfer
Sdfdata data = new Sdfdata(length);
//for (int i=0; i<length; i++)
//{ data.image[i] = in.readbyte(); }
if (length > 0) {
in.readFully(data.image);
}
//Read Orientation
data.orientation[0] = in.readFloat(); //Orientation x
data.orientation[1] = in.readFloat(); //Orientation y
data.orientation[2] = in.readFloat(); //Orientation z
//Read GPS
data.longitude = in.readDouble();
data.latitude = in.readDouble();
data.altitude = in.readDouble();
//Display orientation and GPS data
System.out.println(data.orientation[0] + " " + data.orientation[1] + " " + data.orientation[2]);
System.out.println(data.longitude + " " + data.latitude + " " + data.altitude);
String fileName = "IMG_" + Integer.toString(++number) + ".JPG";
System.out.println("FileName: " + fileName);
FileOutputStream fos = new FileOutputStream(fileName);
fos.write(data.image);
fos.close();
/*InputStream ins = new ByteArrayInputStream(data.image);
BufferedImage image = ImageIO.read(ins);
ImageIO.write(image, "JPG", new File (fileName));
*/
//set image flag
newImage = true;
} catch (Exception e) {
//System.out.println("EOF Or ? " + e);
cond =false;
socket1.close();
server.close();
start(port);
}
}while (cond);
}
}
Your code starts a server, waits for a connection, reads some data from the first connected client, and then exits after writing this data to a file.
Being asked to make your server "non-blocking" could mean that you are being asked to change it to use asynchronous IO (probably unlikely), or it could mean that you're being asked to handle more than one client at a time - because currently you can only serve one client and then your program exits.
This question is hard to answer because your current code is very far away from where you need it to be and it seems like some reading up on networking, sockets, and Java programming in general would be a good way to start.
I'd recommend Netty for doing anything network-related in Java and their samples and documentation are good and easy to follow. Good luck!

Categories