LWJGL with Slick-Util use a lot of RAM to load texture - java

If I use the Slick-Util to load textures into LWJGL it uses a lot of RAM. Here an example with the console-output:
Message.info ((Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()) / 1000000 + "MB");
TextureManager.loadTexture("gui.wallpaper", "resources/wallpaper.jpg");
Message.info ((Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()) / 1000000 + "MB");
The "TextureManager.loadTexture(...)" method is the following:
String textureFileFormat;
if (thePath.contains("png")) {
textureFileFormat = "PNG";
} else if (thePath.contains("jpg")) {
textureFileFormat = "JPG";
} else if (thePath.contains("jpeg")) {
textureFileFormat = "JPG";
} else {
textureFileFormat = "PNG";
}
theTexture = TextureLoader.getTexture(textureFileFormat, new FileInputStream(thePath), true);
// === Store it onto the HashTable === //
textures.put(theName, theTexture);
This uses (like viewable in the console output) quit 80 MB. Why? The "wallpaper.jpg" is just 290KB lage!
What is wrong?
This is the console output:
2013-09-24 21:32:40 [INFO] [CLIENT] 4MB
Tue Sep 24 21:32:40 CEST 2013 INFO:Use Java PNG Loader = true
2013-09-24 21:32:41 [INFO] [CLIENT] 89MB
I hope, that somebody of you know a solution.

Related

Apache Commons Imaging (EXIF): Setting tags/tags missing

I'm using the Apache Commons Imaging library (Java 8, you can find my code here) and I've come across a few problems with tags:
If I open the image info of e.g. this .jpg file with Win 10, there are "Origin" tags, e.g. "Authors" and "Date acquired":
You can find a list of EXIF tags here and it includes the "Authors" one and also an additional "Artist" one. The "Artist" tag seemed to have existed in the library at one point (source) but the apidocs don't list it anymore and ExifTagConstants.TIFF_TAG_ARTIST doesn't exist. Same thing with the "GPSAltitude" tag: It should exist according to the EXIF list but I can't seem to find it in the library.
I tried to use the "Maker" tag instead:
final TiffOutputDirectory exifDir = outputSet.getOrCreateExifDirectory();
exifDir.removeField(ExifTagConstants.EXIF_TAG_MAKER_NOTE);
exifDir.add(ExifTagConstants.EXIF_TAG_MAKER_NOTE, "Test Maker");
But ExifTagConstants.EXIF_TAG_MAKER_NOTE's type of TagInfoUndefineds doesn't seem to be valid for exifDir.add.
I also tried to add the date:
exifDir.removeField(ExifTagConstants.EXIF_TAG_DATE_TIME_DIGITIZED);
exifDir.add(ExifTagConstants.EXIF_TAG_DATE_TIME_DIGITIZED, "1970/01/01");
This just throws an exception:
org.apache.commons.imaging.ImageWriteException: Tag expects 20 byte(s), not 1
The only tag I've managed to successfully write so far is ExifTagConstants.EXIF_TAG_USER_COMMENT.
How do I use/write these tags (author/artist, date, altitude,...)?
Edit:
I managed to find two of the tags:
exifDir.add(MicrosoftTagConstants.EXIF_TAG_XPAUTHOR, "Test Author");//Author
exifDir.add(GpsTagConstants.GPS_TAG_GPS_ALTITUDE, new RationalNumber(3, 1));//Altitude
... but they aren't written into the file.
exifDir.add(ExifTagConstants.EXIF_TAG_USER_COMMENT, "my comment");
works, so I know it's actually writing tags, it just doesn't work for the two above. Any idea what's wrong?
As #haraldK already mentioned: GPS data isn't part of the actual EFIX directory, which is also the case in the "Apache Commons Imaging" library.
So instead of writing the altitude with
double someDouble = 123.123456789;
int alt = (int) Math.round(someDouble*1000); //round to 3 decimal places
final TiffOutputDirectory exifDir = outputSet.getOrCreateExifDirectory();
exifDir.removeField(GpsTagConstants.GPS_TAG_GPS_ALTITUDE);
exifDir.add(GpsTagConstants.GPS_TAG_GPS_ALTITUDE, new RationalNumber(alt, 1000));
use:
final TiffOutputDirectory gpsDir = outputSet.getOrCreateGPSDirectory();
gpsDir.removeField(GpsTagConstants.GPS_TAG_GPS_ALTITUDE);
gpsDir.add(GpsTagConstants.GPS_TAG_GPS_ALTITUDE, new RationalNumber(alt, 1000));
This will write "123.123" into the "Altitude" field that can be viewed through the Windows Explorer (right-click on image -> Properties -> Details).
As for the other tags:
final TiffOutputDirectory exifDir = outputSet.getOrCreateExifDirectory();
final TiffOutputDirectory gpsDir = outputSet.getOrCreateGPSDirectory();
final TiffOutputDirectory rootDir = outputSet.getOrCreateRootDirectory();
final TiffOutputDirectory intDir = outputSet.getInteroperabilityDirectory(); //Not sure what this one is used for
//Writing into the "Authors" field
rootDir.removeField(MicrosoftTagConstants.EXIF_TAG_XPAUTHOR);
rootDir.add(MicrosoftTagConstants.EXIF_TAG_XPAUTHOR, "Me");
//Writing into the "Program Name" field
rootDir.removeField(ExifTagConstants.EXIF_TAG_SOFTWARE);
rootDir.add(ExifTagConstants.EXIF_TAG_SOFTWARE, "My App");
//Writing into the "Date taken" field
exifDir.removeField(ExifTagConstants.EXIF_TAG_DATE_TIME_ORIGINAL);
exifDir.add(ExifTagConstants.EXIF_TAG_DATE_TIME_ORIGINAL, "1970:01:01 12:34:56");
//Writing into the "Digitized Date"
exifDir.removeField(ExifTagConstants.EXIF_TAG_DATE_TIME_DIGITIZED);
exifDir.add(ExifTagConstants.EXIF_TAG_DATE_TIME_DIGITIZED, "1970:01:01 12:34:56");
//Not visible in "Properties" (use image editing software to see it)
//Writing the GPS time stamp
gpsDir.removeField(GpsTagConstants.GPS_TAG_GPS_DATE_STAMP);
gpsDir.add(GpsTagConstants.GPS_TAG_GPS_DATE_STAMP, "1970:01:01");
//Apparently only writes year & not visible in "Properties" (use image editing software to see it)
I haven't found the right tag for "Date Acquired" yet, so if someone knows the exact one, please comment on this answer.
Old way to go :
You can looking at a specification of EXIF metadatas and you can implement some custom parsing to get them with How to extract EXIF metadatas from JPEG files.
Recent way to go :
You can read EXIF datas with that ImageData class which is working very well
Now if you want to modify/update the images metadatas, you can use the documentation in which you will find the following method :
/**
48 * This example illustrates how to add/update EXIF metadata in a JPEG file.
49 *
50 * #param jpegImageFile
51 * A source image file.
52 * #param dst
53 * The output file.
54 * #throws IOException
55 * #throws ImageReadException
56 * #throws ImageWriteException
57 */
58 public void changeExifMetadata(final File jpegImageFile, final File dst)
59 throws IOException, ImageReadException, ImageWriteException {
60
61 try (FileOutputStream fos = new FileOutputStream(dst);
62 OutputStream os = new BufferedOutputStream(fos);) {
63
64 TiffOutputSet outputSet = null;
65
66 // note that metadata might be null if no metadata is found.
67 final ImageMetadata metadata = Imaging.getMetadata(jpegImageFile);
68 final JpegImageMetadata jpegMetadata = (JpegImageMetadata) metadata;
69 if (null != jpegMetadata) {
70 // note that exif might be null if no Exif metadata is found.
71 final TiffImageMetadata exif = jpegMetadata.getExif();
72
73 if (null != exif) {
74 // TiffImageMetadata class is immutable (read-only).
75 // TiffOutputSet class represents the Exif data to write.
76 //
77 // Usually, we want to update existing Exif metadata by
78 // changing
79 // the values of a few fields, or adding a field.
80 // In these cases, it is easiest to use getOutputSet() to
81 // start with a "copy" of the fields read from the image.
82 outputSet = exif.getOutputSet();
83 }
84 }
85
86 // if file does not contain any exif metadata, we create an empty
87 // set of exif metadata. Otherwise, we keep all of the other
88 // existing tags.
89 if (null == outputSet) {
90 outputSet = new TiffOutputSet();
91 }
92
93 {
94 // Example of how to add a field/tag to the output set.
95 //
96 // Note that you should first remove the field/tag if it already
97 // exists in this directory, or you may end up with duplicate
98 // tags. See above.
99 //
100 // Certain fields/tags are expected in certain Exif directories;
101 // Others can occur in more than one directory (and often have a
102 // different meaning in different directories).
103 //
104 // TagInfo constants often contain a description of what
105 // directories are associated with a given tag.
106 //
107 final TiffOutputDirectory exifDirectory = outputSet.getOrCreateExifDirectory();
108 // make sure to remove old value if present (this method will
109 // not fail if the tag does not exist).
110 exifDirectory.removeField(ExifTagConstants.EXIF_TAG_APERTURE_VALUE);
111 exifDirectory.add(ExifTagConstants.EXIF_TAG_APERTURE_VALUE,
112 new RationalNumber(3, 10));
113 }
114
115 {
116 // Example of how to add/update GPS info to output set.
117
118 // New York City
119 final double longitude = -74.0; // 74 degrees W (in Degrees East)
120 final double latitude = 40 + 43 / 60.0; // 40 degrees N (in Degrees
121 // North)
122
123 outputSet.setGPSInDegrees(longitude, latitude);
124 }
125
126 // printTagValue(jpegMetadata, TiffConstants.TIFF_TAG_DATE_TIME);
127
128 new ExifRewriter().updateExifMetadataLossless(jpegImageFile, os,
129 outputSet);
130 }
131 }
I have just found a usefull example for your use case I think. You will know now how to find them, I am sure it will be easier to modify them :
#Override
public Exif getExif(Photo photo) throws ServiceException {
File file = new File(photo.getPath());
String exposure = "not available";
double aperture = 0.0;
double focalLength = 0.0;
int iso = 0;
boolean flash = false;
String make = "not available";
String model = "not available";
double altitude = 0.0;
try {
final ImageMetadata metadata = Imaging.getMetadata(file);
final JpegImageMetadata jpegMetadata = (JpegImageMetadata) metadata;
if (jpegMetadata.findEXIFValueWithExactMatch(ExifTagConstants.EXIF_TAG_EXPOSURE_TIME)
!= null) {
exposure = jpegMetadata
.findEXIFValueWithExactMatch(ExifTagConstants.EXIF_TAG_EXPOSURE_TIME)
.getValueDescription().split(" ")[0];
}
if (jpegMetadata.findEXIFValueWithExactMatch(ExifTagConstants.EXIF_TAG_APERTURE_VALUE)
!= null) {
aperture = jpegMetadata
.findEXIFValueWithExactMatch(ExifTagConstants.EXIF_TAG_APERTURE_VALUE)
.getDoubleValue();
}
if (jpegMetadata.findEXIFValueWithExactMatch(ExifTagConstants.EXIF_TAG_FOCAL_LENGTH)
!= null) {
focalLength = jpegMetadata
.findEXIFValueWithExactMatch(ExifTagConstants.EXIF_TAG_FOCAL_LENGTH)
.getDoubleValue();
}
if (jpegMetadata.findEXIFValueWithExactMatch(ExifTagConstants.EXIF_TAG_ISO) != null) {
iso = jpegMetadata.findEXIFValueWithExactMatch(ExifTagConstants.EXIF_TAG_ISO)
.getIntValue();
}
if (jpegMetadata.findEXIFValueWithExactMatch(ExifTagConstants.EXIF_TAG_FLASH) != null) {
flash = jpegMetadata.findEXIFValueWithExactMatch(ExifTagConstants.EXIF_TAG_FLASH)
.getIntValue() != 0;
}
if (jpegMetadata.findEXIFValueWithExactMatch(TiffTagConstants.TIFF_TAG_MAKE) != null) {
make = jpegMetadata.findEXIFValueWithExactMatch(TiffTagConstants.TIFF_TAG_MAKE)
.getValueDescription();
}
if (jpegMetadata.findEXIFValueWithExactMatch(TiffTagConstants.TIFF_TAG_MODEL) != null) {
model = jpegMetadata.findEXIFValueWithExactMatch(TiffTagConstants.TIFF_TAG_MODEL)
.getValueDescription();
}
if (jpegMetadata.findEXIFValueWithExactMatch(GpsTagConstants.GPS_TAG_GPS_ALTITUDE)
!= null) {
altitude = jpegMetadata
.findEXIFValueWithExactMatch(GpsTagConstants.GPS_TAG_GPS_ALTITUDE)
.getDoubleValue();
}
return new Exif(photo.getId(), exposure, aperture, focalLength, iso, flash, make, model,
altitude);
} catch (IOException | ImageReadException e) {
throw new ServiceException(e.getMessage(), e);
}
}
I found the Artist Tag:
exifDirectory.add(TiffTagConstants.TIFF_TAG_ARTIST, "JANE DOUGH");
Haven't tested whether this will work though.
EDIT:
It works well.
Online Exif Viewer Image
The image shows that the Artist Tag works.

Looking for better approach to recognize text and symbols from table in image and export to CSV

I am working on OCR as part of my curiocity and interest where I am trying to extract text, numeric values, and symbols from an image of tabular data and then that extracted data I want to export into CSV file.
Sample input image looks like this
Images are generated from pdf files.
I tried using Tesseract OCR with Java and also with Python but after conversion of image to text, the extracted text missing the symbols.
Below is my sample code in Java
package com.sk.ocr;
import org.bytedeco.javacpp.*;
import org.bytedeco.leptonica.*;
import org.bytedeco.tesseract.*;
import static org.bytedeco.leptonica.global.lept.*;
import static org.bytedeco.tesseract.global.tesseract.*;
public class PresetsOCR {
public static void main(String[] args) {
BytePointer outText;
TessBaseAPI api = new TessBaseAPI();
if (api.Init("C:/Program Files/Tesseract-OCR/tessdata", "eng") != 0) {
System.err.println("Could not initialize tesseract.");
System.exit(1);
}
PIX image = pixRead(args.length > 0 ? args[0] : "C:/tmp/images/sample-2.jpeg");
api.SetImage(image);
outText = api.GetUTF8Text();
System.out.println("OCR output:\n" + outText.getString());
api.End();
outText.deallocate();
pixDestroy(image);
}
}
Expected Output:
OCR output:
Location Requirement Type Mult Value Upper Tol Lower Tol
S1-D6 R4 ± R5 R 4 +5.0 -5.0
S1-D7 Ꚛ ø0.5 A B C Ꚛ 0 +5.0 -0.0
S1-D7 ֍ ø0.1 G ֍ 0 +1.0 -0.0
Actual Output:
OCR output:
ea eee) ine | SE eave ees
S16 Res 05 R [4 [ +05 [ 05
|st-p7_ | a0 +05 | 00
Teil oe ser fa]
Above program generates approximately correct output when input image of English text, but fails when the image holds tabular data or special symbols.
Any ideas or pointers for further proceeding is appreciated!

Read tab delimited file and ignore empty space

I am working on a simple project in which a tab delimited text file is read into a program.
My problem:
When reading the text file there are regularly empty data spaces. This lack of data is causing an unexpected output. For lines that do not have data in the token[4] position all data read is ignored and "4" is displayed when I run a System.out.println(Just a test that the data is being read properly). When I incorporate a value in the token[4] position the data reads fine. It is not acceptable that I input a value in the token[4] position. See below for file and code.
2014 Employee Edward Rodrigo 6500
2014 Salesman Patricia Capola 5600 5000000
2014 Executive Suzy Allen 10000 55
2015 Executive James McHale 12500 49
2015 Employee Bernie Johnson 5500
2014 Salesman David Branch 6700 2000000
2015 Salesman Jonathan Stein 4600 300000
2014 Executive Michael Largo 17000 50
2015 Employee Kevin Bolden 9200
2015 Employee Thomas Sullivan 6250
My code is:
// Imports are here
import java.io.*;
import java.util.*;
public class EmployeeData {
public static void main(String[] args) throws IOException {
// Initialize variables
String FILE = "employees.txt"; // Constant for file name to be read
ArrayList<Employee> emp2014; // Array list for 2014 employees
ArrayList<Employee> emp2015; // Array list for 2015 employees
Scanner scan;
// Try statement for error handling
try {
scan = new Scanner(new BufferedReader(new FileReader(FILE)));
emp2014 = new ArrayList();
emp2015 = new ArrayList();
// While loop to read FILE
while (scan.hasNextLine()) {
String l = scan.nextLine();
String[] token = l.split("\t");
try {
String year = token[0];
String type = token[1];
String name = token[2];
String monthly = token[3];
String bonus = token[4];
System.out.println(year + " " + type + " " + name + " " + monthly + " " + bonus);
} catch (Exception a) {
System.out.println(a.getMessage());
}
}
} catch(Exception b) {
System.out.println(b.getMessage());
}
}
}
The output I receive for lines with "Employee" returns in an unexpected way.
Output:
run:
4
2014 Salesman Patricia Capola 5600 5000000
2014 Executive Suzy Allen 10000 55
2015 Executive James McHale 12500 49
4
2014 Salesman David Branch 6700 2000000
2015 Salesman Jonathan Stein 4600 300000
2014 Executive Michael Largo 17000 50
4
4
BUILD SUCCESSFUL (total time: 0 seconds)
I tried to use an if-then to test for null value in token[4] position but that didn't really help me. I've done quite a bit of searching with no success.
I am still very new to the programming world, so please pardon my coding inefficiencies. Any support and general feedback to improve my skills is greatly appreciated!
Thank you,
Bryan
Java Devil is right that the underlying issue because of an ArrayOutOfBoundsException. But it's also worth exploring why you didn't see that. As we discussed in the comments your "Try statement for error handling" is in fact not handling your errors at all, instead it is suppressing them, which is generally a poor plan as it allows your program to continue running even after your assumption (that it works correctly) has been violated.
Here's a slightly cleaned up version of your code. The underlying problem that causes the ArrayOutOfBoundsException is still there, but the issue would be immediately apparent if you'd structured your code this way instead. There's a few comments calling out issues inline.
public class EmployeeData {
// constants should be declared static and final, and not inside main
private static final String FILE = "employees.txt";
// If you have an exception and you don't know how to handle it the best thing
// to do is throw it higher and let the caller of your method decide what to do.
// If there's *nothing* you want to do with an exception allow main() to throw
// it as you do here; your program will crash, but that's a good thing!
public static void main(String[] args) throws IOException {
// Notice the <> after ArrayList - without it you're defining a "raw type"
// which is bad - https://stackoverflow.com/q/2770321/113632
ArrayList<Employee> emp2014 = new ArrayList<>();
ArrayList<Employee> emp2015 = new ArrayList<>();
// A try-with-resources block automatically closes the file once you exit the block
// https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html
try (Scanner scan = new Scanner(new BufferedReader(new FileReader(FILE)))) {
while (scan.hasNextLine()) {
String l = scan.nextLine();
String[] token = l.split("\t");
// The code below this line assumes that token has at least five indicies;
// since that isn't always true you need to handle that edge case before
// accessing the array indicies directly.
String year = token[0];
String type = token[1];
String name = token[2];
String monthly = token[3];
String bonus = token[4];
System.out.println(year + " " + type + " " + name + " " + monthly + " " + bonus);
}
}
}
}
This is happening because you are actually getting an ArrayOutOfBoundsException and the message for that is '4'. Because the index of 4 is greater than the length of the array. You should put in your catch statement b.printStackTrace() as this will give you greater details when ever the caught exception occurs.
You can get around this by adding the following:
String bonus = "";
if(token.length > 4)
bonus = token[4];

Halting program execution between certain hours of the week

I am writing a Java program that is required to copy files and folders between the following hours:
Mon - 18:00 to 06:30
Tue - 18:00 to 06:30
Wed - 18:00 to 06:30
Thu - 18:00 to 06:30
Fri - 18:00 to 06:30
Sat - all day
Sun - all day
The program will run continuously until it has finished copying all files and folders. However, outside of the above hours the program should just sleep.
I am using a properties file to store the above settings.
UPDATE
I am looking for the simplest possible implementation including the format of the properties in the properties file as well as the code that will make the checks.
I would do it like this
final Map<Integer, String> schedule = new HashMap<>();
// parse your settings and fill schedule
schedule.put(Calendar.MONDAY, "18:00 to 06:30");
// ...
// create timer to fire e.g. every hour
new Timer().scheduleAtFixedRate(new TimerTask() {
public void run() {
Calendar c = Calendar.getInstance();
String s = schedule.get(c.get(Calendar.DAY_OF_WEEK));
if (withinTimeRange(c, s)) { // implement withinTimeRange func
// copy files
}
}}, 0, 1000 * 3600);
Since your program is going to run continuously, the simplest solution is to check the day and time before copying a file. If the time is during off hours, go ahead and copy the next file, otherwise Thread.sleep.
If this is an internal, one-off kind of program, I would go ahead and hard-code the business hours instead of reading the properties file. No need to add complexity.
whenever your program is launched, get the current time, and check day today's day.
check whether it lies in permissible time if yes let it continue. If not, find the time at 00:00am of that 'day'. and find the time at xx:yyZZ (start of permissible time). calculate the difference, and let the program sleep for that much of time.
Thank you for your suggestions.
I came up with a working solution in the end which if it gets enough points I will mark as the answer. The way I attempted to solve this problem was by thinking about non-working hours rather than working hours. This code is just for illustration
# Properties
Mon = 06:30-18:00
Tue = 06:30-18:00
Wed = 06:30-18:00
Thu = 06:30-18:00
Fri = 06:30-18:00
Loop over the properties to get their values
String[] days = { "Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat" };
Map<Integer, Integer[]> nonWorkingHours = new HashMap<Integer, Integer[]>();
for( int i = 0; i < days.length; i++ ) // for each property in file
{
// excluded implementation of getConfig
String prop = getConfig( days[ i ] ); // e.g. "06:00-19:00"
// +1 to match CALENDAR.DAY_OF_WEEK
nonWorkingHours.put( i + 1, getHours( prop );
}
My function to parse property excluding error handling
// e.g. if prop = "06:00-19:00" then { 6, 0, 19, 0 } is returned
public Integer[] getHours( String prop )
{
String times = prop.split( "(:|-)" );
Integer[] t = new Integer[4];
for( int i = 0; i < times.length; i++ )
{
t[i] = Integer.parseInt( times[i] );
}
return t;
}
And finally the function that implements the halt
private void sleepIfOutsideWorkingHours()
{
Integer[] data = nonWorkingHours.get( currentDay );
if( data != null )
{
Calendar c = Calendar.getInstance();
Integer currentSeconds = ( Calendar.HOUR_OF_DAY * 3600 ) + ( Calendar.MINUTE * 60 );
Integer stopFrom = ( data[ 0 ] * 3600 ) + ( data[ 1 ] * 60 );
Integer stopTill = ( data[ 2 ] * 3600 ) + ( data[ 3 ] * 60 );
if( currentSeconds > stopFrom && currentSeconds < stopTill )
{
Integer secondsDiff = stopTill - currentSeconds;
if( secondsDiff > 0 )
{
try
{
Thread.sleep( secondsDiff * 1000 ); // turn seconds to milliseconds
}
catch( InterruptedException e )
{
// error handling
}
}
}
}
}
And finally just call the function below just before copying each file and if it is being run outside working hours it will stop the program.
sleepIfOutsideWorkingHours();
I am sure there is a simpler way of doing it :-) but there it is.
You should try using a Continuous Integration system. That scenario can be easily set up using Jenkins CI for example.
The reason i advise doing it so in a ambient like that is that you can keep a better control on the history of your program runs.

How to collect directory listing along with each file CRC checksum?

I use the following command to get dir listing in nix(Linux, AIX, Sunos, HPUX) platforms
Command
ls -latr
Ouput
drwxr-xr-x 2 ricky support 4096 Aug 29 11:59 lib
-rwxrwxrwx 1 ricky support 924 Aug 29 12:00 initservice.sh
cksum command is used for getting CRC checksum.
How can the CRC Checksum be appended after each file something (including directory listing too) like below, maintaining the below format in these nix(Linux, AIX, Sunos, HPUX) platforms?
drwxr-xr-x 2 ricky support 4096 Aug 29 11:59 lib
-rwxrwxrwx 1 ricky support 924 Aug 29 12:00 initservice.sh 4287252281
Update Note : No third party application, I am using java/Groovy to parse the output ultimately into a given format which forms a xml using groovy XmlSlurper (XML's get generated around 5MB sized)
"permission","hardlink","owner","group","fsize","month","date","time","filename","checksum"
All Suggestions are welcome! :)
Update with my code
But here I am calculating md5hex which gives a similar output as md5sum command from linux. So it's no longer cksum as I cannot use jacksum bcz of some licensing issue :(
class CheckSumCRC32 {
public def getFileListing(String file){
def dir = new File(file)
def filename = null
def md5sum = null
def filesize = null
def lastmodified = null
def lastmodifiedDate = null
def lastmodifiedTime = null
def permission = null
Format formatter = null
def list=[]
if(dir.exists()){
dir.eachFileRecurse (FileType.FILES) { fname ->
list << fname
}
list.each{fileob->
try{
md5sum=getMD5CheckSum(fileob.toString())
filesize=fileob.length()+"b"
lastmodified=new Date(fileob.lastModified())
lastmodifiedDate=lastmodified.format('dd/MM/yyyy')
formatter=new SimpleDateFormat("hh:mm:ss a")
lastmodifiedTime=formatter.format(lastmodified)
permission=getReadPermissions(fileob)+getWritePermissions(fileob)+getExecutePermissions(fileob)
filename=getRelativePath("E:\\\\temp\\\\recurssive\\\\",fileob.toString())
println "$filename, $md5sum, $lastmodifiedDate, $filesize, $permission, $lastmodifiedDate, $lastmodifiedTime "
}
catch(IOException io){
println io
}
catch(FileNotFoundException fne){
println fne
}
catch(Exception e){
println e
}
}
}
}
public def getReadPermissions(def file){
String temp="-"
if(file.canRead())temp="r"
return temp
}
public def getWritePermissions(def file){
String temp="-"
if(file.canWrite())temp="w"
return temp
}
public def getExecutePermissions(def file){
String temp="-"
if(file.canExecute())temp="x"
return temp
}
public def getRelativePath(def main, def file){""
return file.toString().replaceAll(main, "")
}
public static void main(String[] args) {
CheckSumCRC32 crc = new CheckSumCRC32();
crc.getFileListing("E:\\temp\\recurssive")
}
}
Output
release.zip, 25f995583144bebff729086ae6ec0eb2, 04/06/2012, 6301510b, rwx, 04/06/2012, 02:46:32 PM
file\check\release-1.0.zip, 3cc0f2b13778129c0cc41fb2fdc7a85f, 18/07/2012, 11786307b, rwx, 18/07/2012, 04:13:47 PM
file\Dedicated.mp3, 238f793f0b80e7eacf5fac31d23c65d4, 04/05/2010, 4650908b, rwx, 04/05/2010, 10:45:32 AM
but still I need a way to calculate hardlink, owner & group. I searched on the net it looks like java7 has this capability & I am stuck with java6. Any help?
Take a look at http://www.jonelo.de/java/jacksum/index.html - it is reported to provide cksum - compatible CRC32 checksums.
BTW, I tried using java.util.zip.CRC32 to calculate checksums, and it gives a different value than cksum does, so must use a slightly different algorithm.
EDIT: I tried jacksum, and it works, but you have to tell it to use the 'cksum' algorithm - apparently that is different from crc32, which jacksum also supports.
Well, you could run the command, then, for each line, run the cksum and append it to the line.
I did the following:
dir = "/home/will"
"ls -latr $dir".execute().in.eachLine { line ->
// let's omit the first line, which starts with "total"
if (line =~ /^total/) return
// for directories, we just print the line
if (line =~ /^d/)
{
println line
}
else
{
// for files, we split the line by one or more spaces and join
// the last pieces to form the filename (there must be a better
// way to do this)
def fileName = line.split(/ {1,}/)[8..-1].join("")
// now we get the first part of the cksum
def cksum = "cksum $dir/$fileName".execute().in.text.split(/ {1,}/)[0]
// concat the result to the original line and print it
println "$line $cksum"
}
}
Special attention to my "there must be a better way to do this".

Categories