We Keep Coding

Converting ANSI to UTF-8 & java.lang.OutOfMemoryError: Java heap space

My final goal is to convert a file from ANSI to UTF-8. To do so, I use some code with Java :
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
public class ConvertFromAnsiToUtf8 {
public static void main(String[] args) throws IOException {
try {
Path p = Paths.get("C:\\shared_to_vm\\test_encode\\test.csv");
ByteBuffer bb = ByteBuffer.wrap(Files.readAllBytes(p));
CharBuffer cb = Charset.forName("windows-1252").decode(bb);
bb = Charset.forName("UTF-8").encode(cb);
Files.write(p, bb.array());
} catch (Exception e) {
System.out.println(e);
}
}
}
The code works perfectly when I test it on small files. My file is convert from ANSI to UTF-8 and all characters are recognize and well encoded. But as soon as I try to use it on the file I need to convert, I get the error java.lang.OutOfMemoryError: Java heap space.
So far as my understanding goes, I got like 1.5 million lines in my file so I am pretty sure I create too many objects with my application.
Of course, I have checked what this error means and how I could solve it (like here or here for example) but is improving the memory capacity of my JVM the only way to solve it ? And if it is, how much more should i use ?
Any kind of help (tip, advice, link or else) would be greatly appreciated !

Don't read the whole file at once:
ByteBuffer bb = ByteBuffer.wrap(Files.readAllBytes(p));
Instead, try to read line-by-line:
Files.lines(p, Charset.forName("windows-1252")).forEach(line -> {
// Convert your line, write to file
});

Stream the input, convert the character encoding, and write the output as you go. This way, you don't need to read the entire file into memory, but only as much as you want.
If you want to minimize the number of (slowish) system calls, you could use a similar approach, but explicitly create a BufferedInputStream with a larger internal buffer, and then wrap that in an InputStreamReader. But the simple approach shown here is unlikely to be a critical point in many applications.
private static final Charset WINDOWS1252 = Charset.forName("windows-1252");
private static final int DEFAULT_BUF_SIZE = 8192;
public static void transcode(Path input, Path output) throws IOException {
try (Reader r = Files.newBufferedReader(input, WINDOWS1252);
Writer w = Files.newBufferedWriter(output, StandardCharsets.UTF_8, StandardOpenOption.CREATE_NEW)) {
char[] buf = new char[DEFAULT_BUF_SIZE];
while (true) {
int n = r.read(buf);
if (n < 0) break;
w.write(buf, 0, n);
}
}
}

If you have a large file, which is larger then available random access memory you should convert characters chunk-by-chunk.
Following you can found the example:
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.channels.ReadableByteChannel;
import java.nio.channels.WritableByteChannel;
import java.nio.charset.Charset;
import java.nio.charset.CharsetDecoder;
import java.nio.charset.CharsetEncoder;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardOpenOption;
public class Iconv {
private static void iconv(Charset toCode, Charset fromCode, Path src, Path dst) throws IOException {
CharsetDecoder decoder = fromCode.newDecoder();
CharsetEncoder encoder = toCode.newEncoder();
try (ReadableByteChannel source = FileChannel.open(src, StandardOpenOption.READ);
WritableByteChannel destination = FileChannel.open(dst, StandardOpenOption.CREATE, StandardOpenOption.TRUNCATE_EXISTING,
StandardOpenOption.WRITE);) {
ByteBuffer readBytes = ByteBuffer.allocate(4096);
while (source.read(readBytes) > 0) {
readBytes.flip();
destination.write(encoder.encode(decoder.decode(readBytes)));
readBytes.clear();
}
}
}
public static void main(String[] args) throws Exception {
iconv(Charset.forName("UTF-8"), Charset.forName("Windows-1252"), Paths.get("test.csv") , Paths.get("test-utf8.csv") );
}
}

How to get this property value in my java code?

I'm learning Java and sometimes I have some problem to retrieve the information I need from objects...
When I debug my code I can see in targetFile, a path property but I don't know how to get it in my code.
This is a screenshot:
(source: toile-libre.org)
This is my complete code:
package com.example.helloworld;
import com.github.axet.wget.WGet;
import com.github.axet.wget.info.DownloadInfo;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import java.io.File;
import java.io.IOException;
import java.net.URL;
public class HelloWorld {
public static void main(String[] args) throws IOException {
nodejs();
}
public static void nodejs() throws IOException {
// Scrap the download url.
Document doc = Jsoup.connect("http://nodejs.org/download").get();
Element link = doc.select("div.interior:nth-child(2) > table:nth-child(2) > tbody:nth-child(1) > tr:nth-child(1) > td:nth-child(3) > a:nth-child(1)").first();
String url = link.attr("abs:href");
// Print the download url.
System.out.println(url);
// Download file via the scraped url.
URL download = new URL(url);
File target = new File("/home/lan/Desktop/");
WGet w = new WGet(download, target);
w.download();
// Get the targetFile property
// ???
}
}
How can I get this value?

I do not know your code but the field you are interested in may be encapsulated and thus not accessible in your code, but the debugger can see it at runtime :)
Update:
https://github.com/axet/wget/blob/master/src/main/java/com/github/axet/wget/WGet.java
The field is default package, you can only access it from within the package.
This can be frustrating at times, but you should ask yourself why the designers of this class decided to hide this field.

File Not Found Exception - Can't see the issue

I've tried directly linking using the entire path but that hasn't solved it either.
package eliza;
import java.io.*;
public class Eliza {
public static void main(String[] args) throws IOException {
String inputDatabase = "src/eliza/inputDataBase.txt";
String outputDatabase = "src/eliza/outputDataBase.txt";
Reader database = new Reader();
String[][] inputDB = database.Reader(inputDatabase);
String[][] outputDB = database.Reader(outputDatabase);
}
}
Here is the reader class:
package eliza;
import java.io.FileReader;
import java.io.BufferedReader;
import java.io.IOException;
public class Reader {
public String[][] Reader(String name) throws IOException {
int length = 0;
String sizeLine;
FileReader sizeReader = new FileReader(name);
BufferedReader sizeBuffer = new BufferedReader(sizeReader);
while((sizeLine = sizeBuffer.readLine()) != null) {
length++;
}
String[][] database = new String[length][1];
return (database);
}
}
Here's a photo of my directory. I even put these text files in the "eliza" root folder: here
Any ideas?

Since you are using an IDE, you need to give the complete canonical path. It should be
String inputDatabase = "C:\\Users\\Tommy\\Desktop\\Eliza\\src\\eliza\\inputDataBase.txt";
String outputDatabase = "C:\\Users\\Tommy\\Desktop\\Eliza\\src\\eliza\\outputDataBase.txt";
The IDE is probably executing the bytecode from its bin folder and cannot find the relative reference.

give the exact path like
String inputDatabase = "c:/java/src/eliza/inputDataBase.txt";

you have not given the correct path, Please re check
try
{BASE_PATH}+ "Eliza/src/inputDataBase.txt"

The source directory tree isn't generally present during execution, so files that are required at runtime shouldn't be put there ... unless you're going to use them as resources, in which case their pathname is relative to the package root, and does not begin with 'src', and the data is accessed by a getResourceXXX() method, not via a FileInputStream.

How to play an mp3 file in java

I am trying to play a song (mp3 file) in java. I have been looking around for a few hours now and none of the ways I found worked properly.
public void play()
{
String song = "song.mp3";
Media track = new Media(song);
MediaPlayer mediaPlayer = new MediaPlayer(track);
mediaPlayer.play();
}
I have tried doing that but it gives me errors.
I have imported JMF and JLayer.
I have also read other questions that are like this one on this forum and none of them have helped me.
I just need a hand to help play an mp3 file.

The easiest way I found was to download the JLayer jar file from http://www.javazoom.net/javalayer/sources.html and to add it to the Jar library http://www.wikihow.com/Add-JARs-to-Project-Build-Paths-in-Eclipse-%28Java%29
Here is the code for the class
public class SimplePlayer {
public SimplePlayer(){
try{
FileInputStream fis = new FileInputStream("File location.");
Player playMP3 = new Player(fis);
playMP3.play();
}catch(Exception e){System.out.println(e);}
}
}
and here are the imports
import javazoom.jl.player.*;
import java.io.FileInputStream;

For this you'll need to install Java Media Framework (JMF) in your PC. One you have it installed,then try this piece of code:
import javax.media.*;
import java.net.*;
import java.io.*;
import java.util.*;
class AudioPlay
{
public static void main(String args[]) throws Exception
{
// Take the path of the audio file from command line
File f=new File("song.mp3");
// Create a Player object that realizes the audio
final Player p=Manager.createRealizedPlayer(f.toURI().toURL());
// Start the music
p.start();
// Create a Scanner object for taking input from cmd
Scanner s=new Scanner(System.in);
// Read a line and store it in st
String st=s.nextLine();
// If user types 's', stop the audio
if(st.equals("s"))
{
p.stop();
}
}
}
You may run into unable to handle formaterror, that is because Java took out the MP3 support by default (pirate copyright issue), you are required to install a “JMF MP3 plugin” in order to play MP3 file.
Go Java’s JMF website to download it
http://java.sun.com/javase/technologies/desktop/media/jmf/mp3/download.html
To be sure that you are using a supported format file, check here:
http://www.oracle.com/technetwork/java/javase/formats-138492.html
If you are using windows7, you may have to read this as well:
https://forums.oracle.com/forums/thread.jspa?threadID=2132405&tstart=45

How about JavaFX application-
import java.net.URL;
import javafx.application.Application;
import javafx.scene.media.Media;
import javafx.scene.media.MediaPlayer;
import javafx.stage.Stage;
public class VLC extends Application {
void playMedia() {
String mp3 = "00- Tu Hi Mera.mp3";
URL resource = getClass().getResource(mp3);
System.out.println(resource.toString());
Media media = new Media(resource.toString());
MediaPlayer mediaPlayer = new MediaPlayer(media);
mediaPlayer.play();
}
public static void main(String args[]) {
new VLC().playMedia();
}
#Override
public void start(Stage stage) throws Exception {
}
}

How to get the single images of an mp4-Movie in Java

I want to do some image analysis on a video that's stored in .mp4 format. Therefore I need a way to just get the images of this movie in Java.
I goolged a lot and found some libraries like jcodec and jaad. BUT I wasn't able to get the things running with these libraries. And as I found out, there were examples (at least I found none) that showed my usecase.
Can you help me? Do you know any library that can do what I need and is running at least on Win7 64 bit.
Or do you know how to accomplish this with jcodec?
edit:
As I wrote, I tried it with jcodec. I found out how to get the data of a frame, but not how I can get it into something like a BufferedImage or so. I expect that these data isn't in a simple RGB format but in any compressed format or so. (Am I right with that?) I don't know to to decode this data.
You can get the data of a frame with jcodec as follows (at least as far as I understand this):
public static void main(String[] args) throws IOException, MP4DemuxerException {
String path = "videos/video-2011-09-21-20-07-21.mp4";
MP4Demuxer demuxer1 = new MP4Demuxer(new FileInput(new File(path)));
DemuxerTrack videoTrack = demuxer1.getVideoTrack();
Packet firstFrame = videoTrack.getFrames(1);
byte[] data = firstFrame.getData();
}
I also found the following:
http://code.google.com/p/jcodec/source/browse/trunk/src/test/java/org/jcodec/containers/mp4/DitherTest.java?r=70
But this isn't working (has compile errors) with the downloadable jar-package.

you could use jcodec(https://github.com/jcodec/jcodec) in the followinf program i am extracting frames from a video.
/*
* To extract frames from a mp4(avc) video
*
*/
package avc_frame;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import javax.imageio.ImageIO;
import org.jcodec.api.FrameGrab;
import org.jcodec.api.JCodecException;
public class Avc_frame {
/**
* #param args the command line arguments
*/
public static void main(String[] args) throws IOException, JCodecException {
long time = System.currentTimeMillis();
for (int i = 50; i < 57; i++) {
BufferedImage frame = FrameGrab.getFrame(new File("/Users/jovi/Movies/test.mp4"), i);
ImageIO.write(frame, "bmp", new File("/Users/jovi/Desktop/frames/frame_"+i+".bmp"));
}
System.out.println("Time Used:" + (System.currentTimeMillis() - time)+" Milliseconds");
}
}

import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import javax.imageio.ImageIO;
import org.bytedeco.javacpp.opencv_core.IplImage;
import org.bytedeco.javacv.FFmpegFrameGrabber;
import org.bytedeco.javacv.FrameGrabber.Exception;
public class Read{
public static void main(String []args) throws IOException, Exception
{
FFmpegFrameGrabber frameGrabber = new FFmpegFrameGrabber("C:/Users/Digilog/Downloads/Test.mp4");
frameGrabber.start();
IplImage i;
try {
i = frameGrabber.grab();
BufferedImage bi = i.getBufferedImage();
ImageIO.write(bi,"png", new File("D:/Img.png"));
frameGrabber.stop();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}