We Keep Coding

File Downloader with Java Commons IO stuck at 1MB/s+

Lately I've been experimenting with Java and Commons IO trying to create a web file downloader, however I've encountered a problem, in fact it seems that the file download speed does not exceed 1MB/s while the same download from the browser runs smoothly at 3MB /s. Could you help me? I would be really grateful.
This is my downloader code:
package com.application.steammachine;
import com.github.junrar.Archive;
import com.github.junrar.Junrar;
import com.github.junrar.exception.RarException;
import com.github.junrar.rarfile.FileHeader;
import com.github.junrar.volume.FileVolumeManager;
import javafx.beans.property.SimpleStringProperty;
import javafx.concurrent.Task;
import org.apache.commons.io.IOUtils;
import org.ini4j.Wini;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.io.*;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.net.URL;
public class Downloader extends Task<Void> {
private URL url;
private String fileName;
private Game game;
public Downloader(URL url, String fileName, Game game) {
this.url = url;
this.fileName = fileName;
this.game = game;
}
public class ProgressListener implements ActionListener {
private double bytes = 0;
private double mbDownloaded = 0;
private double fileSize = 0;
private double lastMB = 0;
private long initialTime;
private double speed = 0;
private String downloadedText = "";
private String sizeText;
public ProgressListener(double fileSize){
this.fileSize = fileSize;
initialTime = System.nanoTime();
}
#Override
public void actionPerformed(ActionEvent e) {
bytes = ((DownloadCountingOutputStream) e.getSource()).getByteCount();
updateProgress(bytes, fileSize);
mbDownloaded = round(bytes/1e+6, 2);
if(fileSize >= 1073741824){ //>= 1GB
double temp = ((fileSize/1e+6)/1024);
sizeText = round(temp,2) + " GB";
}else {
double temp = (fileSize/1e+6);
sizeText = round(temp,2) + " MB";
}
if(mbDownloaded >= 1024){
downloadedText = String.valueOf(round(mbDownloaded/1024,2));
}else{
downloadedText = String.valueOf(mbDownloaded);
}
if((System.nanoTime() - initialTime) >= (Math.pow(10, 9))){
speed = round((mbDownloaded - lastMB), 3);
initialTime = System.nanoTime();
lastMB = mbDownloaded;
}
updateMessage(String.valueOf(speed)+"MB/s,"+String.valueOf(downloadedText + "/" + sizeText));
}
}
#Override
protected Void call() throws Exception {
URL dl = this.url;
File fl = null;
String x = null;
OutputStream os = null;
InputStream is = null;
try {
updateMessage("Searching files...,---/---");
fl = new File(Settings.getInstallPath() +"/"+ this.fileName);
os = new FileOutputStream(fl);
is = dl.openStream();
DownloadCountingOutputStream dcount = new DownloadCountingOutputStream(os);
double fileSize = Double.valueOf(dl.openConnection().getHeaderField("Content-Length"));
ProgressListener progressListener = new ProgressListener(fileSize);
dcount.setListener(progressListener);
IOUtils.copy(is, dcount, 512000 );
updateMessage("Concluding...,Almost finished");
} catch (Exception e) {
System.out.println(e);
IOUtils.closeQuietly(os);
IOUtils.closeQuietly(is);
updateMessage(",");
this.cancel(true);
return null;
} finally {
IOUtils.closeQuietly(os);
IOUtils.closeQuietly(is);
updateMessage(",");
updateProgress(0, 0);
this.cancel(true);
return null;
}
}
protected static double round(double value, int places) {
if (places < 0) throw new IllegalArgumentException();
BigDecimal bd = new BigDecimal(Double.toString(value));
bd = bd.setScale(places, RoundingMode.HALF_UP);
return bd.doubleValue();
}
}
This is the DownloadCountingOutputStream class, which i use for keeping track of the download status:
import java.awt.event.ActionListener;
import java.io.IOException;
import java.io.OutputStream;
public class DownloadCountingOutputStream extends CountingOutputStream {
private ActionListener listener = null;
public DownloadCountingOutputStream(OutputStream out) {
super(out);
}
public void setListener(ActionListener listener) {
this.listener = listener;
}
#Override
protected void afterWrite(int n) throws IOException {
super.afterWrite(n);
if (listener != null) {
listener.actionPerformed(new ActionEvent(this, 0, null));
}
}
}
Thanks in adavance

First, there are a couple of simple things you could try to speed up the transfers:
Try using a larger transfer buffer size. Change the 8K buffer size to 64K or 512K.
Get rid of the DownloadCountingOutputStream and transfer directly to the FileOutputStream.
These should be simple to try ... and they may help a bit.
On a Linux system, it may also be worthwhile to replace Apache IOUtils.copy call with code that uses the kernel's zero-copy transfer support. See Efficient data transfer through zero copy for an explanation.
The example in the article is for uploading using transferTo, but downloading using transferFrom should be analogous. The article claims a 65% speedup for large file transfers compared with conventional Java I/O. But that is likely to depend on the characteristics of your network connection.

Getting java.io.EOFException when calling readObject() on ObjectInputStream

I am trying to implement simple UDP communication between two sockets. In order to be sent in UDP packet, data needs to be converted to byte array. I created simple class named Packet which is used to store various data types. This class implements Serializable interface so it can be converted to byte array and sent over UDP protocol. Program worked fine until I put HashMap object inside Packet class. Error occurs on server side when recived data is being converted back to Packet object.
Here is client code:
package hr.fer.tel.rassus.udp.server;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.ObjectOutput;
import java.io.ObjectOutputStream;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.util.HashMap;
public class UDPClient {
final static int PORT = 10001; // server port
/**
* #param args the command line arguments
*/
public static void main(String args[]) throws IOException {
HashMap<Integer, Integer> vector = new HashMap<Integer, Integer>();
vector.put(1, 2);
Packet p = new Packet(10000,10001,"co2",0,vector);
byte[] serializedMessage = null;
try {
ByteArrayOutputStream bStream = new ByteArrayOutputStream();
ObjectOutput oo = new ObjectOutputStream(bStream);
oo.writeObject(p);
serializedMessage = bStream.toByteArray();
oo.close();
} catch (Exception ex) {
System.out.println("error "+ex.toString());
}
// determine the IP address of a host, given the host's name
InetAddress address = InetAddress.getByName("localhost");
// create a datagram socket and bind it to any available
// port on the local host
DatagramSocket socket = new DatagramSocket(); //SOCKET
// create a datagram packet for sending data
DatagramPacket packet = new DatagramPacket(serializedMessage, serializedMessage.length,
address, PORT);
// send a datagram packet from this socket
socket.send(packet); //SENDTO
socket.close(); //CLOSE
}
}
Server code:
package hr.fer.tel.rassus.udp.server;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.util.logging.Level;
import java.util.logging.Logger;
public class UDPServer {
final static int PORT = 10001; // server port
/**
* #param args the command line arguments
*/
public static void main(String args[]) throws IOException {
byte[] rcvBuf = new byte[256]; // received bytes
// create a UDP socket and bind it to the specified port on the local
// host
DatagramSocket socket = new DatagramSocket(PORT); //SOCKET -> BIND
while (true) {
// create a DatagramPacket for receiving packets
DatagramPacket packet = new DatagramPacket(rcvBuf, rcvBuf.length);
// receive packet
socket.receive(packet); //RECVFROM
// construct a new String by decoding the specified subarray of
// bytes
// using the platform's default charset
ObjectInputStream iStream;
Packet p = null;
try {
iStream = new ObjectInputStream(new ByteArrayInputStream(packet.getData()));
p = (Packet) iStream.readObject();
iStream.close();
} catch (ClassNotFoundException ex) {
Logger.getLogger(UDPServer.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
Packet class:
package hr.fer.tel.rassus.udp.server;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Set;
/**
*
* #author adrianzgaljic
*/
public class Packet implements Serializable{
private int fromPort;
private int toPort;
private String co2;
private boolean flag;
private long timeStamp;
private HashMap<Integer, Integer> vector;
Set<Integer> keys;
public Packet(int from, int to, String co2, long time, HashMap<Integer, Integer> vector){
this.fromPort = from;
this.toPort = to;
this.co2= co2;
this.timeStamp = time;
this.vector = vector;
flag = false;
}
public Packet(int from){
this.fromPort = from;
this.flag = true;
}
public int getFromPort() {
return fromPort;
}
public int getToPort() {
return toPort;
}
public String getCo2() {
return co2;
}
public long getTimeStamp() {
return timeStamp;
}
public boolean isFlag() {
return flag;
}
public HashMap<Integer, Integer> getVector() {
return vector;
}
}
Stack trace:
Exception in thread "main" java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2325)
at java.io.ObjectInputStream$BlockDataInputStream.readUTFBody(ObjectInputStream.java:3063)
at java.io.ObjectInputStream$BlockDataInputStream.readUTF(ObjectInputStream.java:2864)
at java.io.ObjectInputStream.readUTF(ObjectInputStream.java:1072)
at java.io.ObjectStreamClass.readNonProxy(ObjectStreamClass.java:704)
at java.io.ObjectInputStream.readClassDescriptor(ObjectInputStream.java:830)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at hr.fer.tel.rassus.udp.server.UDPServer.main(UDPServer.java:48)
Java Result: 1

After running your code I realized that my previous answer was not solving the issue, however I will leave it along with the new one, because it describes an issue that may cause similar effect.
New answer:
Your server's datagram packet size is too small to receive all the data you are sending, increase its size by increasing the buffer size:
byte[] rcvBuf = new byte[256]; // received bytes
e.g. new byte[2048]
Old answer:
Swap these two lines in the client:
serializedMessage = bStream.toByteArray();
oo.close();
So you should have this:
oo.close();
serializedMessage = bStream.toByteArray();
The issue with your code is that you are getting the bytes, while ObjectOutputStream didn't actually flush all the data to your ByteArrayOutputStream.

In Java playing back audio using a clip isn't working with no error messages

I've been trying to use the "Trail: Sound" from Oracle and I've gotten to this part. https://docs.oracle.com/javase/tutorial/sound/playing.html#113609
"Using a Clip"
Well I tried to follow the directions which aren't exactly specific and I made some code that seems like it should work. It basically matches examples I found online that work for other people. On my machine nothing happens the program ends without playing any sound right after I start it. It doesn't say any errors.
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.Line;
import javax.sound.sampled.LineUnavailableException;
import javax.sound.sampled.Clip;
import javax.sound.sampled.AudioFormat;
import java.io.File;
import javax.sound.sampled.AudioFileFormat;
import javax.sound.sampled.UnsupportedAudioFileException;
import java.io.IOException;
import javax.sound.sampled.AudioInputStream;
class Demo{
public static void main(String[] args) throws LineUnavailableException, UnsupportedAudioFileException, IOException{
File file = new File("song1.wav");
AudioFileFormat audioFileFormat = AudioSystem.getAudioFileFormat(file);
AudioFormat audioFormat = audioFileFormat.getFormat();
javax.sound.sampled.DataLine.Info dataLineInfo = new javax.sound.sampled.DataLine.Info(Clip.class,audioFormat);
Line theLine = AudioSystem.getLine(dataLineInfo);
Clip clip = (Clip)theLine;
AudioInputStream audioInputStream = AudioSystem.getAudioInputStream(file);
clip.open(audioInputStream);
clip.start();
}
}
EDIT: I figured out after clip.start() I needed to keep the program open.

I use that class to play MP# sounds and works for me
import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import javazoom.jl.player.Player;
import monitorbixao.Main;
/**
*
* #author C007329
*/
public class MusicPlayer extends Thread {
private File music;
private Player player;
public MusicPlayer(File music) {
this.music = music;
}
#Override
public void run() {
play();
}
public void play() {
try {
FileInputStream stream = new FileInputStream(music);
BufferedInputStream buffer = new BufferedInputStream(stream);
this.player = new Player(buffer);
//System.out.println("Executando...");
this.player.play();
//System.out.println("Terminado");
} catch (Exception e) {
//System.out.println("Erro!");
Main.logApp.addMsgLog(MusicPlayer.class.getCanonicalName(), e.getMessage());
e.printStackTrace();
}
}
public void close() {
this.player.close();
//System.out.println("Interrompido...");
}
}

I figured it out. I needed to keep the program open so after "clip.start();" I added this code
java.io.Console c = System.console();
if (c == null) {
System.err.println("No console.");
System.exit(1);
}
String nothing = c.readLine("Wait... ");
:D

Java Server Non Blocking Query

I am using the following code to read some data from Android client. All is going fine. But now i am asked to make this server code non blocking. Is there any suggestions for this ? I was trying to use threads but dont know how ? I am beginner in Java :)
Thanks
import java.awt.image.BufferedImage;
import java.io.BufferedOutputStream;
import java.io.ByteArrayInputStream;
import java.io.DataInputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.Calendar;
import java.util.Date;
import javax.imageio.ImageIO;
public class Server {
//Server Constructor
public Server()
{}
//Variables Initialization
private static ServerSocket server;
byte[] imagetemp;
private static Socket socket1;
private static boolean newImage;
private static Sdfdata data;
private static boolean cond;
public static int port;
private static int number = 0;
//Image Availability return method
public boolean imageAvailable()
{
return newImage;
}
public boolean clientchk()
{
return socket1.isClosed();
}
//Image Flag set by Vis group when image read.
public void setImageFlag(boolean set)
{
newImage = set;
}
// Send the data to the Vis Group
public Sdfdata getData()
{
return data;
}
//Starts the Server
public static boolean start(int port1)
{
try {
port=port1;
server = new ServerSocket(port1);
System.out.println("Waiting for Client to Connect");
//New thread here
socket1=server.accept();
} catch (IOException e) {
System.out.println("Cannot Connect");
e.printStackTrace();
return false;
}
return true;
}
//Stops the Server
public boolean stop()
{
try {
socket1.close();
}
catch (IOException e)
{
e.printStackTrace();
return false;
}
return true;
}
/**
* #param args
* #throws IOException
*/
public static void main(String[] args) throws IOException {
// Starts the server
start(4444);
// DataInput Stream for reading the data
DataInputStream in = null;
try {
in = new DataInputStream(socket1.getInputStream());
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
cond=true;
do {
try
{
//Read Image Data
int length = in.readInt();
//Create an ByteArray of length read from Client for Image transfer
Sdfdata data = new Sdfdata(length);
//for (int i=0; i<length; i++)
//{ data.image[i] = in.readbyte(); }
if (length > 0) {
in.readFully(data.image);
}
//Read Orientation
data.orientation[0] = in.readFloat(); //Orientation x
data.orientation[1] = in.readFloat(); //Orientation y
data.orientation[2] = in.readFloat(); //Orientation z
//Read GPS
data.longitude = in.readDouble();
data.latitude = in.readDouble();
data.altitude = in.readDouble();
//Display orientation and GPS data
System.out.println(data.orientation[0] + " " + data.orientation[1] + " " + data.orientation[2]);
System.out.println(data.longitude + " " + data.latitude + " " + data.altitude);
String fileName = "IMG_" + Integer.toString(++number) + ".JPG";
System.out.println("FileName: " + fileName);
FileOutputStream fos = new FileOutputStream(fileName);
fos.write(data.image);
fos.close();
/*InputStream ins = new ByteArrayInputStream(data.image);
BufferedImage image = ImageIO.read(ins);
ImageIO.write(image, "JPG", new File (fileName));
*/
//set image flag
newImage = true;
} catch (Exception e) {
//System.out.println("EOF Or ? " + e);
cond =false;
socket1.close();
server.close();
start(port);
}
}while (cond);
}
}

Your code starts a server, waits for a connection, reads some data from the first connected client, and then exits after writing this data to a file.
Being asked to make your server "non-blocking" could mean that you are being asked to change it to use asynchronous IO (probably unlikely), or it could mean that you're being asked to handle more than one client at a time - because currently you can only serve one client and then your program exits.
This question is hard to answer because your current code is very far away from where you need it to be and it seems like some reading up on networking, sockets, and Java programming in general would be a good way to start.
I'd recommend Netty for doing anything network-related in Java and their samples and documentation are good and easy to follow. Good luck!

Audio recorder problem in java

I have a problem while recording the audio. I created a servlet and I modified the java sound API demo code to some extent and finally I can record the audio. The problem is that when I play the audio I can see the total time of the audio stored as 645.45 or something like that, but I have been recording the audio only for a couple of mins. One more problem is the audio is getting saved in the Eclipse directory instead of the project directory.
This is the servlet code.
package com;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.sound.sampled.AudioFileFormat;
import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.DataLine;
import javax.sound.sampled.Clip;
import javax.sound.sampled.LineUnavailableException;
import javax.sound.sampled.SourceDataLine;
import javax.sound.sampled.TargetDataLine;
public class SoundRecorder extends HttpServlet {
private static final long serialVersionUID = 1L;
static protected boolean running;
static ByteArrayOutputStream out;
double fileName = Math.random();
//strFilename = nowLong.toString();
public SoundRecorder() {
System.out.println("Filename will be..." + fileName + ".wav");
}
public void init() {
}
public void destroy() {
}
public void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
System.out.println("call received..");
String method = request.getParameter("method");
System.out.println(method);
if("record".equalsIgnoreCase(method)) {
captureAudio(true);
}
else if("stop".equalsIgnoreCase(method)) {
captureAudio(false);
}
else if("play".equalsIgnoreCase(method)) {
System.out.println("yet to write");
playAudio();
}
}
public void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
System.out.println("call received..");
String method = request.getParameter("method");
System.out.println(method);
doGet(request, response);
}
private void captureAudio(boolean capturing) {
File outputFile = new File(fileName + ".wav");
AudioFormat audioFormat = new AudioFormat(AudioFormat.Encoding.PCM_SIGNED,44100.0F, 16, 2, 4, 44100.0F, false);
DataLine.Info info = new DataLine.Info(TargetDataLine.class, audioFormat);
TargetDataLine targetDataLine = null;
try
{
targetDataLine = (TargetDataLine) AudioSystem.getLine(info);
targetDataLine.open(audioFormat);
}
catch (LineUnavailableException e)
{
System.out.println("unable to get a recording line");
e.printStackTrace();
System.exit(1);
}
AudioFileFormat.Type targetType = AudioFileFormat.Type.WAVE;
final Recorder recorder = new Recorder(targetDataLine,targetType,outputFile);
System.out.println("Recording...");
if(capturing){
recorder.start();
}
else {
recorder.stopRecording();
}
}
private void playAudio() {
try {
File file = new File(fileName + ".wav");
AudioInputStream stream = AudioSystem.getAudioInputStream(file);
AudioFormat format = stream.getFormat();
DataLine.Info info = new DataLine.Info(Clip.class, stream.getFormat());
Clip clip = (Clip) AudioSystem.getLine(info);
clip.open(stream);
clip.start();
} catch (Exception e) {
System.err.println("Line unavailable: " + e);
System.exit(-4);
}
}
}
And this is the recorder class
public class Recorder extends Thread {
private TargetDataLine m_line;
private AudioFileFormat.Type m_targetType;
private AudioInputStream m_audioInputStream;
private File m_outputFile;
public Recorder(TargetDataLine line,
AudioFileFormat.Type targetType,
File file)
{
m_line = line;
m_audioInputStream = new AudioInputStream(line);
m_targetType = targetType;
m_outputFile = file;
}
/** Starts the recording.
To accomplish this, (i) the line is started and (ii) the
thread is started.
*/
public void start()
{
m_line.start();
super.start();
}
/** Stops the recording.
*/
public void stopRecording()
{
m_line.stop();
m_line.close();
}
/** Main working method.
*/
public void run()
{
try
{
AudioSystem.write(
m_audioInputStream,
m_targetType,
m_outputFile);
}
catch (IOException e)
{
e.printStackTrace();
}
}
private static void closeProgram()
{
System.out.println("Program closing.....");
System.exit(1);
}
private static void out(String strMessage)
{
System.out.println(strMessage);
}
}

When developing with servlets, you need to realize that there's only one servlet instance throughout the whole webapp's lifetime, from startup until shutdown. So, the HTTP requests from all visitors, all sessions, all browser windows/tabs, etc will all share the same servlet instance. Also, when you make a variable static, it will be shared among all instances of the same class (which is not really relevant here since there's only one servlet instance anyway).
In other words, those variables which you've declared in the servlet are not threadsafe:
static protected boolean running;
static ByteArrayOutputStream out;
double fileName = Math.random();
There's only one of them and they are used by all visitors simultaneously. For the first two variables, which are continuously modified, this will lead to major threadsafety problems and for the third variable this means that all visitors record to the very same file. You need to declare them inside the doGet() block. You'd like to store the recording in the session by an unique request based token as key and then pass that key to the subsequent requests.
As to the problem of the file being saved at the unexpected location; when you use relative paths in java.io.File in a servlet, then it will be relative to the directory from where the webserver is started. If you start it from inside Eclipse, then it's saved in Eclipse directory. You'd like to use absolute path in java.io.File instead. If your intent is to save it in public webcontent (there where your JSP's and the /WEB-INF folder is located), then you need ServletContext#getRealPath() to convert a web path to an absolute disk path.
String relativeWebPath = "filename.ext";
String absoluteDiskPath = getServletContext().getRealPath(relativeWebPath);
File file = new File(absoluteDiskPath);
There's however another problem with this: all files will get erased whenever you redeploy the webapp. If you want a bit more permanent storage, then you'd like to store it outside the web project. E.g. C:/path/to/recordings.
File file = new File("C:/path/to/recordings/filename.ext");