**
i am trying to run objective c program from java..
every time geting this error..
Main error is java.io.IOException: Cannot run program "C:\store\1436014813635\panna.exe": CreateProcess error=2, The system cannot find the file specified
help me..
thanks in advance..
**
#Override
public Map<String, Object> compileAndExecute(Code code, String folderName) {
Map<String, Object> finalResult = new HashMap<String, Object>();
try {
String folderPath = "C:\\store\\" + folderName + "\\";
String sourceFilePath = "C:\\store\\" + folderName + "\\panna.m";
String outputFilePath = "C:\\store\\" + folderName + "\\panna.exe";
File fileSaveDir = new File(folderPath);
if (!fileSaveDir.exists()) {
fileSaveDir.mkdirs();
}
File file = new File(sourceFilePath);
log.info("code :"+code.getCode());
ByteBuffer buf = ByteBuffer.wrap(code.getCode().getBytes());
#SuppressWarnings("resource")
FileChannel channel = new FileOutputStream(file, true)
.getChannel();
channel.write(buf);
channel.close();
String command = "gcc -o " + outputFilePath +" "
+sourceFilePath+" -I C:/GNUstep/GNUstep/System/Library/Headers -L C:/GNUstep/GNUstep/System/Library/Libraries -std=c99 -lobjc -lgnustep-base -fconstant-string-class=NSConstantString";
System.out.print("Command :"+command);
finalResult = PannaUtils.runProcessObjectiveC(command, folderName);
log.info("finalResult "+finalResult.get("success")+folderName);
log.info("class Name:" + outputFilePath);
finalResult = PannaUtils.runProcessObjectiveC(outputFilePath, folderName);
log.info("finalResult "+finalResult.get("success")+folderName);
//PannaUtils.getTestCaseResults(folderName, finalResult, code, outputFilePath);
} catch (Exception e) {
e.printStackTrace();
}
return finalResult;
}
//runProcessObjectiveC from here.
public static Map<String, Object> runProcessObjectiveC(String command,
String folderName) {
_log.info(command);
Map<String, Object> afterExecutionOutput = new HashMap<String, Object>();
try{
Process pro = Runtime.getRuntime().exec(command);
ProcessTimeoutThread.addProcess(pro);
long afterExecTime = System.currentTimeMillis();
_log.info(""+afterExecTime);
String error = printLines(command + " error:", pro.getErrorStream());
pro.waitFor();
_log.info("error sop"+error);
_log.info("got exit vlue :"
+ pro.exitValue() );
if (pro.exitValue() > 0) {
afterExecutionOutput.put("success", error);
} else {
afterExecutionOutput.put("success", error);
}
}
catch (Exception e) {
// TODO Auto-generated catch block
_log.info("Main error is "+e.toString());
}
return afterExecutionOutput;
}
Related
Hi am trying to write parquets to an amazon s3 bucket/key using JAVA but getting an [org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "file"].
Cannot write
[s3a://nprd-pr-snd-rtgrev-edp/sndrtgrev/out/jcl-0.snappy.parquet]:
org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for
scheme "file"
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3443)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3466)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3574)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3521)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:540)
at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:496)
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:316)
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:393)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:165)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:146)
at org.apache.hadoop.fs.s3a.S3AFileSystem.createTmpFileForWrite(S3AFileSystem.java:1019)
at org.apache.hadoop.fs.s3a.S3ADataBlocks$DiskBlockFactory.create(S3ADataBlocks.java:816)
at org.apache.hadoop.fs.s3a.S3ABlockOutputStream.createBlockIfNeeded(S3ABlockOutputStream.java:204)
at org.apache.hadoop.fs.s3a.S3ABlockOutputStream.(S3ABlockOutputStream.java:182)
at org.apache.hadoop.fs.s3a.S3AFileSystem.create(S3AFileSystem.java:1369)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1195)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1175)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1064)
at org.apache.parquet.hadoop.ParquetFileWriter.(ParquetFileWriter.java:244)
at org.apache.parquet.hadoop.ParquetWriter.(ParquetWriter.java:273)
at org.apache.parquet.hadoop.ParquetWriter$Builder.build(ParquetWriter.java:494)
at com.jclconsultants.parquet.tool.AwsParquetProcessor.writeParquetWithAvroToAwsBucket(AwsParquetProcessor.java:210)
at com.jclconsultants.parquet.tool.AwsParquetProcessor.run(AwsParquetProcessor.java:81)
at com.jclconsultants.parquet.tool.ToolProcessParquetsFromAwsBucket.launchAwsParquetProcessor(ToolProcessParquetsFromAwsBucket.java:269)
at com.jclconsultants.parquet.tool.ToolProcessParquetsFromAwsBucket.launchThreads(ToolProcessParquetsFromAwsBucket.java:251)
at com.jclconsultants.parquet.tool.ToolProcessParquetsFromAwsBucket.processParquetsFromBucket(ToolProcessParquetsFromAwsBucket.java:160)
at com.jclconsultants.parquet.tool.ToolProcessParquetsFromAwsBucket.main(ToolProcessParquetsFromAwsBucket.java:127)
Here is how I am writing (code was gathered from different post):
private void writeParquetWithAvroToAwsBucket(String outputParquetName, List<GenericData.Record> records) {
URI awsURI = null;
try {
awsURI = new URI("s3a://" + bucketName + "/" + outputParquetName );
} catch (URISyntaxException e1) {
e1.printStackTrace();
return;
}
Path dataFile = new Path(awsURI);
Configuration config = new Configuration();
config.set("fs.s3a.access.key", accesskey);
config.set("fs.s3a.secret.key", secretAccessKey);
config.set("fs.s3a.endpoint", "s3." + Regions.CA_CENTRAL_1.getName() + ".amazonaws.com");
config.set("fs.s3a.impl", "org.apache.hadoop.fs.s3a.S3AFileSystem");
config.set("fs.s3a.aws.credentials.provider", "org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider");
config.set("fs.s3a.server-side-encryption-algorithm", S3AEncryptionMethods.SSE_KMS.getMethod());
config.set("fs.s3a.connection.ssl.enabled", "true");
config.set("fs.s3a.impl.disable.cache", "true");
config.set("fs.s3a.path.style.access", "true");
try (ParquetWriter<GenericData.Record> writer = AvroParquetWriter.<GenericData.Record>builder(dataFile)//
.withSchema(getSchema())//
.withConf(config)//
.withCompressionCodec(SNAPPY)//
.withWriteMode(OVERWRITE)//
.build()) {
for (GenericData.Record record : records) {
writer.write(record);
}
} catch (Exception e) {
e.printStackTrace();
}
}
and the schema si as follow:
private Schema getSchema() {
String json = "{\"type\":\"record\",\r\n" + " \"name\":\"spark_schema\",\r\n" + " \"fields\":[\r\n"
+ " {\"name\":\"PolicyVersion_uniqueId\",\"type\":[\"null\",\"string\"],\"default\":null},\r\n"
+ " {\"name\":\"agreementNumber\",\"type\":[\"null\",\"string\"],\"default\":null},\r\n"
+ " {\"name\":\"epoch_time\",\"type\":[\"null\",\"string\"],\"default\":null},\r\n"
+ " {\"name\":\"xml\",\"type\":[\"null\",\"string\"],\"default\":null},\r\n"
+ " {\"name\":\"filename\",\"type\":[\"null\",\"string\"],\"default\":null},\r\n"
+ " {\"name\":\"message_header\",\"type\":[\"null\",{\"type\":\"map\",\"values\":[\"null\",\"string\"]}],\"default\":null},\r\n"
+ " {\"name\":\"tracking_number\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"fullTermPremium\",\"type\":[\"null\",\"string\"],\"default\":null},\r\n"
+ " {\"name\":\"epoch_date\",\"type\":[\"null\",{\"type\":\"int\",\"logicalType\":\"date\"}],\"default\":null},\r\n"
+ " {\"name\":\"collection_date\",\"type\":[\"null\",{\"type\":\"int\",\"logicalType\":\"date\"}],\"default\":null}\r\n"
+ "]}";
return new Schema.Parser().parse(json);
}
I've also tried with a different credential provider (AnonymousAWSCredentialsProvider and TemporaryAWSCredentialsProvider) but still did not get it to work. I have a feeling that my config is missing something as it fails during the AvroParquetWriter build().
What am I doing wrong?
Notice that I can read parquets from S3 with a similar configuration.
I can also write a parquet file to my local drive and then upload the parquet file to s3 as follow:
private Path writeGenericRecordsToLocalDrive() {
Path dataFile = new Path(OUTPUT_DIR + "/" + parquetName);
Configuration config = new Configuration();
config.set("fs.hdfs.impl", org.apache.hadoop.hdfs.DistributedFileSystem.class.getName());
config.set("fs.file.impl", org.apache.hadoop.fs.LocalFileSystem.class.getName());
try (ParquetWriter<GenericData.Record> writer = AvroParquetWriter.<GenericData.Record>builder(dataFile)//
.withSchema(getSchema())//
.withConf(config)//
.withCompressionCodec(SNAPPY)//
.withWriteMode(OVERWRITE)//
.build()) {
for (GenericData.Record parquet : parquetRecords) {
writer.write(parquet);
nbRecordsInParquet++;
}
} catch (IOException e) {
e.printStackTrace();
}
return dataFile;
}
private void uploadLocalParquetToAwsBucket(Path dataFile) {
AmazonS3 s3Client = null;
try {
File fileToUpload = new File(OUTPUT_DIR + "/" + dataFile.getName());
ObjectMetadata objectMetadata = new ObjectMetadata();
objectMetadata.setHeader(Headers.SERVER_SIDE_ENCRYPTION, "aws:kms");
objectMetadata.setContentLength(fileToUpload.length());
BasicAWSCredentials creds = new BasicAWSCredentials(accessKey, secretAccessKey);
s3Client = AmazonS3ClientBuilder.standard().withRegion(Regions.CA_CENTRAL_1)
.withCredentials(new AWSStaticCredentialsProvider(creds)).build();
String bucketDestination = fileToUpload.getName();
com.amazonaws.services.s3.model.PutObjectRequest putRequest = new com.amazonaws.services.s3.model.PutObjectRequest(
bucketName + "/" + bucketFolder, bucketDestination, new FileInputStream(fileToUpload),
objectMetadata);
putRequest.putCustomRequestHeader(Headers.SERVER_SIDE_ENCRYPTION, "aws:kms");
PutObjectResult putResult = s3Client.putObject(putRequest);
} catch (Exception e) {
e.printStackTrace();
} finally {
if (s3Client != null) {
s3Client.shutdown();
}
}
}
I'm writing a java program whose operation is summarized as follows:
The subfolder with the specified name is selected in the source
folder.
All the pdfs are taken and converted into txt files in
the target folder.
The name of an item is searched inside the
txt.
If the name is found then the pdf from source to target is
copied.
All the txt from the target folder are deleted, in this
way all the pdfs containing the searched word remain.
"It all works", the problem is that the txt files are not deleted from target and I can not understand why. Thanks in advance, best regards.
Launcher.java:
import java.lang.String;
import java.util.Scanner;
import java.io.File;
public class Launcher {
// campi
int status = 1;
static String source = "C:\\Users\\MyUser\\Desktop\\source\\";
static String target = "C:\\Users\\MyUser\\Desktop\\target\\";
public static void main(String [] args) {
// strings
String src = source;
String item = "";
// init
Scanner scan = new Scanner(System.in);
System.out.print("\nFornitore: ");
src += scan.nextLine().toUpperCase() + "\\";
System.out.print("Item: ");
item = scan.nextLine();
// notifier
Launcher launcher = new Launcher();
// threads
MakerThread maker = new MakerThread(launcher, src, target);
SearcherThread searcher = new SearcherThread(launcher, src, target, item);
CleanupThread cleaner = new CleanupThread(launcher, target);
// run
maker.start();
searcher.start();
cleaner.start();
}
}
MakerThread.java:
import java.io.File;
public class MakerThread extends Thread {
// campi
String src = "";
String target = "";
Launcher launcher;
// costruttore
public MakerThread(Launcher launcher, String src, String target) {
this.src = src;
this.target = target;
this.launcher = launcher;
}
// metodi
#Override
public void run() {
try{
synchronized (launcher) {
File folder = new File(this.src);
for (File f : folder.listFiles()) {
// spin-lock
while (launcher.status != 1){
launcher.wait();
}
if(f.isFile() && f.getName().endsWith("pdf")) {
System.out.println("PDF TROVATO: " + f.getName());
// input
String input = win_str(src + f.getName());
// output
String output = target;
output += f.getName().replaceAll(".pdf", ".txt");
output = win_str(output);
// command
String command = "cmd /C java -jar ./pdfbox-app-2.0.11.jar ExtractText ";
command += input + " " + output;
Process p1 = Runtime.getRuntime().exec(command);
p1.waitFor();
}
}
// set status, awake other threads
launcher.status = 2;
launcher.notifyAll();
}
}catch (Exception e) {
System.out.println("Exception: "+e.getMessage());
}
}
private String win_str(String str) {
return "\"" + str + "\"";
}
}
SearcherThread.java:
import java.io.File;
import java.io.BufferedReader;
import java.io.FileReader;
public class SearcherThread extends Thread {
// campi
String src = "";
String target = "";
String item = "";
Launcher launcher;
// costruttore
public SearcherThread(Launcher launcher, String src, String target, String item) {
this.target = target;
this.src = src;
this.item = item;
this.launcher = launcher;
}
// metodi
#Override
public void run() {
try{
synchronized (launcher) {
File folder = new File(this.target);
for(File f : folder.listFiles()) {
// spin-lock
while (launcher.status != 2){
launcher.wait();
}
if(f.isFile() && f.getName().endsWith("txt")) {
System.out.println("SEARCHING IN: " + f.getName());
// init
String line;
BufferedReader br = new BufferedReader(new FileReader(new File(target+f.getName())));
// txt search
while((line = br.readLine()) != null) {
if(line.contains(item)) {
System.out.println(" [" + item + "]" + " trovato in " + f.getName() + "\n");
// input
String pdf = f.getName().replaceAll(".txt", ".pdf");
String input = win_str(src+pdf);
// output
String output = win_str(target);
// command
String command = "cmd /C copy " + input + " " + output;
Process p1 = Runtime.getRuntime().exec(command);
p1.waitFor();
break;
}
}
}
}
// set status, awake other threads
launcher.status = 3;
launcher.notifyAll();
}
}catch (Exception e) {
System.out.println("Exception: "+e.getMessage());
}
}
private String win_str(String str) {
return "\"" + str + "\"";
}
}
CleanupThread.java:
import java.io.File;
public class CleanupThread extends Thread {
// campi
String target = "";
Launcher launcher;
// costruttore
public CleanupThread(Launcher launcher, String target) {
this.target = target;
this.launcher = launcher;
}
// metodi
#Override
public void run() {
try{
synchronized (launcher) {
File folder = new File(this.target);
for (File f : folder.listFiles()) {
// spin-lock
while (launcher.status != 3){
launcher.wait();
}
if(f.isFile() && f.getName().endsWith("txt")) {
System.out.println("CLEANING UP: " + f.getName());
// input
String input = win_str(target + f.getName());
// command
String command = "cmd /C del " + input;
Process p1 = Runtime.getRuntime().exec(command);
p1.waitFor();
}
}
// set status, awake other threads
launcher.status = 1;
launcher.notifyAll();
}
}catch (Exception e) {
System.out.println("Exception: "+e.getMessage());
}
}
private String win_str(String str) {
return "\"" + str + "\"";
}
}
I have a huge CSV file ~800 K records and using that data I have to form a POST request and make a rest call.
Initially I tried WITHOUT using threads but it is taking very long time to process it.
So I thought of using threads to speed up the process and followed below approach.
divided file into relatively smaller files (Tried with 3 files with approx ~5K data each file). (I Did this Manually Before passing to application)
Created 3 threads (traditional way by extending Thread class)
Reads each file with each thread and form HashMap with details required to form request and added it to ArrayList of POST Request
Send Post request to in loop
List item
Get the Response and add it to Response List
Both above approach takes too long to even complete half (>3 hrs). Might be one of them would not be good approach.
Could anyone please provide suggestion which helps me speeding up the process ? It is NOT mandatory to use threading.
Just to add that my code is at the consumer end and does not have much control on producer end.
MultiThreadedFileRead (Main class that Extends Thread)
class MultiThreadedFileRead extends Thread
{
public static final String EMPTY = "";
public static Logger logger = Logger.getLogger(MultiThreadedFileRead.class);
BufferedReader bReader = null;
String filename = null;
int Request_Num = 0;
HashMap<Integer, String> filteredSrcDataMap = null;
List<BrandRQ> brandRQList = null;
List<BrandRS> brandRSList = null;
ReadDLShipFile readDLShipFile = new ReadDLShipFile();
GenerateAndProcessPostBrandAPI generateAndProcessPostBrandAPI = new GenerateAndProcessPostBrandAPI();
MultiThreadedFileRead()
{
}
MultiThreadedFileRead(String fname) throws Exception
{
filename = fname;
Request_Num = 0;
List<BrandRQ> brandRQList = new ArrayList<BrandRQ>();
List<BrandRS> brandRSList = new ArrayList<BrandRS>();
this.start();
}
public void run()
{
logger.debug("File *** : " + this.filename);
//Generate Source Data Map
HashMap<Integer, String> filteredSrcDataMap = (HashMap<Integer, String>) readDLShipFile.readSourceFileData(this.filename);
generateAndProcessPostBrandAPI.generateAndProcessPostRequest(filteredSrcDataMap, this);
}
public static void main(String args[]) throws Exception
{
String environment = ""; // TO BE UNCOMMENTED
String outPutfileName = ""; // TO BE UNCOMMENTED
BrandRetrivalProperties brProps = BrandRetrivalProperties.getInstance();
if(args != null && args.length == 2 && DLPremiumUtil.isNotEmpty(args[0]) && DLPremiumUtil.isNotEmpty(args[1])) // TO BE UNCOMMENTED
{
environment = args[0].trim();
outPutfileName = args[1].trim();
ApplicationInitializer applicationInitializer = new ApplicationInitializer(environment);
applicationInitializer.initializeParams();
logger.debug("Environment : " + environment);
logger.debug("Filename : " + outPutfileName);
// TO BE UNCOMMENTED - START
}else{
System.out.println("Invalid parameters passed. Expected paramenters: Environment");
System.out.println("Sample Request: java -jar BrandRetrival.jar [prd|int] brand.dat");
System.exit(1);
}
MultiThreadedFileRead multiThreadedFileRead = new MultiThreadedFileRead();
List<String> listOfFileNames = multiThreadedFileRead.getFileNames(brProps);
logger.debug("List : " + listOfFileNames);
int totalFileSize = listOfFileNames.size();
logger.debug("totalFileSize : " + totalFileSize);
MultiThreadedFileRead fr[]=new MultiThreadedFileRead[totalFileSize];
logger.debug("Size of Array : " + fr.length);
for(int i = 0; i < totalFileSize; i++)
{
logger.debug("\nFile Getting Added to Array : " + brProps.getCountFilePath()+listOfFileNames.get(i));
fr[i]=new MultiThreadedFileRead(brProps.getCountFilePath()+listOfFileNames.get(i));
}
}
public List<String> getFileNames(BrandRetrivalProperties brProps)
{
MultiThreadedFileRead multiThreadedFileRead1 = new MultiThreadedFileRead();
BufferedReader br = null;
String line = "";
String filename = multiThreadedFileRead1.getCounterFileName(brProps.getCountFilePath(), brProps.getCountFilePathStartsWith());
List<String> fileNameList = null;
logger.debug("filename : " + filename);
if(filename != null)
{
fileNameList = new ArrayList<String>();
try{
br = new BufferedReader(new FileReader(brProps.getCountFilePath()+filename));
while ((line = br.readLine()) != null)
{
fileNameList.add(line);
}
} catch (FileNotFoundException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} catch (IOException e2) {
// TODO Auto-generated catch block ;
e2.printStackTrace();
} catch (Exception e3) {
// TODO Auto-generated catch block
e3.printStackTrace();
} finally
{
if (br != null)
{
try
{
br.close();
} catch (IOException e4)
{
e4.printStackTrace();
} catch (Exception e5)
{
e5.printStackTrace();
}
}
}
}
return fileNameList;
}
// Get the Name of the file which has name of individual CSV files to read to form the request
public String getCounterFileName(String sourceFilePath, String sourceFileNameStartsWith)
{
String fileName = null;
if(sourceFilePath != null && !sourceFilePath.equals(EMPTY) &&
sourceFileNameStartsWith != null && !sourceFileNameStartsWith.equals(EMPTY))
{
File filePath = new File(sourceFilePath);
File[] listOfFiles = filePath.listFiles();
for (int i = 0; i < listOfFiles.length; i++) {
if (listOfFiles[i].isFile() && listOfFiles[i].getName().startsWith(sourceFileNameStartsWith)) {
fileName = listOfFiles[i].getName();
} else if (listOfFiles[i].isDirectory()) {
fileName = null;
}
}
}
return fileName;
}
}
GenerateAndProcessPostBrandAPI
public class GenerateAndProcessPostBrandAPI {
public static Logger logger = Logger.getLogger(GenerateAndProcessPostBrandAPI.class);
List<BrandRQ> brandRQList = new ArrayList<BrandRQ>();
List<BrandRS> brandRSList = new ArrayList<BrandRS>();
DLPremiumClientPost dlPremiumclientPost = new DLPremiumClientPost();
JSONRequestGenerator jsonRequestGenerator = new JSONRequestGenerator();
public List<BrandRQ> getBrandRequstList(Map<Integer, String> filteredSrcDataMap)
{
if(filteredSrcDataMap != null)
{
brandRQList = jsonRequestGenerator.getBrandRQList(filteredSrcDataMap, brandRQList);
}
return brandRQList;
}
public void generateAndProcessPostRequest(Map<Integer, String> filteredSrcDataMap, MultiThreadedFileRead multiThreadedFileRead)
{
if(filteredSrcDataMap != null)
{
brandRQList = jsonRequestGenerator.getBrandRQList(filteredSrcDataMap, brandRQList);
if(brandRQList != null && brandRQList.size() > 0)
{
BrandRetrivalProperties brProps = BrandRetrivalProperties.getInstance();
String postBrandsEndpoint = brProps.getPostBrandsEndPoint();
for (BrandRQ brandRQ : brandRQList)
{
multiThreadedFileRead.Request_Num = multiThreadedFileRead.Request_Num + 1;
logger.debug("Brand Request - " + multiThreadedFileRead.Request_Num + " : " + ObjectToJSONCovertor.converToJSON(brandRQ));
BrandRS brandResponse = dlPremiumclientPost.loadBrandApiResponseJSON(brandRQ, postBrandsEndpoint, DLPremiumUtil.getTransactionID(), BrandConstants.AUTH_TOKEN);
logger.debug("Response - " + multiThreadedFileRead.Request_Num + " : " + ObjectToJSONCovertor.converToJSON(brandResponse));
brandRSList.add(brandResponse);
}
}
}
}
}
I'm trying to fix a bug in the code I wrote which convert a srt file to dxfp.xml. it works fine but when there is a special character such as an ampersand it throws an java.lang.NumberFormatException error. I tried to use the StringEscapeUtils function from apache commons to solve it. Can someone explain to me what it is I am missing here? thanks in advance!
public class SRT_TO_DFXP_Converter {
File input_file;
File output_file;
ArrayList<CaptionLine> node_list;
public SRT_TO_DFXP_Converter(File input_file, File output_file) {
this.input_file = input_file;
this.output_file = output_file;
this.node_list = new ArrayList<CaptionLine>();
}
class CaptionLine {
int line_num;
String begin_time;
String end_time;
ArrayList<String> content;
public CaptionLine(int line_num, String begin_time, String end_time,
ArrayList<String> content) {
this.line_num = line_num;
this.end_time = end_time;
this.begin_time = begin_time;
this.content = content;
}
public String toString() {
return (line_num + ": " + begin_time + " --> " + end_time + "\n" + content);
}
}
private void readSRT() {
BufferedReader bis = null;
FileReader fis = null;
String line = null;
CaptionLine node;
Integer line_num;
String[] time_split;
String begin_time;
String end_time;
try {
fis = new FileReader(input_file);
bis = new BufferedReader(fis);
do {
line = bis.readLine();
line_num = Integer.valueOf(line);
line = bis.readLine();
time_split = line.split(" --> ");
begin_time = time_split[0];
begin_time = begin_time.replace(',', '.');
end_time = time_split[1];
end_time.replace(',', '.');
ArrayList<String> content = new ArrayList<String>();
while (((line = bis.readLine()) != null)
&& (!(line.trim().equals("")))) {
content.add(StringEscapeUtils.escapeJava(line));
//if (StringUtils.isEmpty(line)) break;
}
node = new CaptionLine(line_num, begin_time, end_time, content);
node_list.add(node);
} while (line != null);
} catch (Exception e) {
System.out.println(e);
}
finally {
if (bis != null) {
try {
bis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (fis != null) {
try {
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
private String convertToXML() {
StringBuffer dfxp = new StringBuffer();
dfxp.append("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<tt xml:lang=\"en\" xmlns=\"http://www.w3.org/2006/04/ttaf1\" xmlns:tts=\"http://www.w3.org/2006/04/ttaf1#styling\">\n\t<head>\n\t\t<styling>\n\t\t\t<style id=\"1\" tts:backgroundColor=\"black\" tts:fontFamily=\"Arial\" tts:fontSize=\"14\" tts:color=\"white\" tts:textAlign=\"center\" tts:fontStyle=\"Plain\" />\n\t\t</styling>\n\t</head>\n\t<body>\n\t<div xml:lang=\"en\" style=\"default\">\n\t\t<div xml:lang=\"en\">\n");
for (int i = 0; i < node_list.size(); i++) {
dfxp.append("\t\t\t<p begin=\"" + node_list.get(i).begin_time + "\" ")
.append("end=\"" + node_list.get(i).end_time
+ "\" style=\"1\">");
for (int k = 0; k < node_list.get(i).content.size(); k++) {
dfxp.append("" + node_list.get(i).content.get(k));
}
dfxp.append("</p>\n");
}
dfxp.append("\t\t</div>\n\t</body>\n</tt>\n");
return dfxp.toString();
}
private void writeXML(String dfxp) {
BufferedWriter out = null;
try {
out = new BufferedWriter(new FileWriter(output_file));
out.write(dfxp);
out.close();
} catch (IOException e) {
System.out.println("Error Writing To File:"+ input_file +'\n');
e.printStackTrace();
} finally {
if (out != null) {
try {
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
public static void main(String[] args) {
if ((args.length < 2) || (args[1].equals("-h"))) {
System.out.println("\n<--- SRT to DFXP Converter Usage --->");
System.out
.println("Conversion: java -jar SRT_TO_DFXP.jar <input_file> <output_file> [-d]");
System.out
.println("Conversion REQUIRES a input file and output file");
System.out.println("[-d] Will Display XML Generated In Console");
System.out.println("Help: java -jar SRT_TO_DFXP.jar -h");
} else if (!(new File(args[0]).exists())) {
System.out.println("Error: Input SubScript File Does Not Exist\n");
} else {
SRT_TO_DFXP_Converter converter = new SRT_TO_DFXP_Converter(
new File(args[0]), new File(args[1]));
converter.readSRT();
String dfxp = converter.convertToXML();
if ((args.length == 3) && (args[2].equals("-d")))
System.out.println("\n" + dfxp + "\n");
converter.writeXML(dfxp);
System.out.println("Conversion Complete");
}
}
here's part of the srt file that it is throwing an error when exported and run as a jar file.
1
00:20:43,133 --> 00:20:50,599
literature and paper by Liversmith & Newman, and I think the point is well made that a host of factors
I have made a JavaFX application which includes uploading large files (> 1GB) to a server. Every time I got the same error on same place. Any Suggestions what am I doing wrong here.
ftpclient.connect(server, port);
ftpclient.login(ftpuser, ftppass);
ftpclient.enterLocalPassiveMode();
ftpclient.setKeepAlive(true);
ftpclient.setControlKeepAliveTimeout(3000);
Task<Void> copyMnt = new Task<Void>() {
#Override
protected Void call(){
try {
new Thread(new FTPHandler(ftpclient, source , dest)).run();
} catch (IOException ex) {
Logger.getLogger(MyAppController.class.getName()).log(Level.SEVERE, null, ex);
}
return null;
}
};
new Thread(copyMnt).start();
Now on the FTPHandler Class
// The constructor will set the ftpclient, source and destinations.
#Override
public void run() {
try {
uploadDirectory(this.getClient(), this.getDest(), this.getSrc(), "");
} catch (IOException ex) {
Logger.getLogger(FTPHandler.class.getName()).log(Level.SEVERE, null, ex);
}
}
public static void uploadDirectory(FTPClient ftpClient,
String remoteDirPath, String localParentDir, String remoteParentDir)
throws IOException {
File localDir = new File(localParentDir);
File[] subFiles = localDir.listFiles();
if (subFiles != null && subFiles.length > 0) {
for (File item : subFiles) {
String remoteFilePath = remoteDirPath + "/" + remoteParentDir
+ "/" + item.getName();
if (remoteParentDir.equals("")) {
remoteFilePath = remoteDirPath + "/" + item.getName();
}
if (item.isFile()) {
// upload the file
String localFilePath = item.getAbsolutePath();
java.util.Date date= new java.util.Date();
System.out.println(new Timestamp(date.getTime()) + " : Uploading :: " + localFilePath + " to " + remoteFilePath);
boolean uploaded = uploadSingleFile(ftpClient,
localFilePath, remoteFilePath);
if (uploaded) {
System.out.println("Success : "
+ remoteFilePath);
} else {
System.out.println("Failed : "
+ localFilePath);
}
} else {
// create directory on the server
boolean created = ftpClient.makeDirectory(remoteFilePath);
if (created) {
System.out.println("CREATED the directory: "
+ remoteFilePath);
} else {
System.out.println("COULD NOT create the directory: "
+ remoteFilePath);
}
// upload the sub directory
String parent = remoteParentDir + "/" + item.getName();
if (remoteParentDir.equals("")) {
parent = item.getName();
}
localParentDir = item.getAbsolutePath();
uploadDirectory(ftpClient, remoteDirPath, localParentDir,
parent);
}
}
}
}
Every time I am uploading the files (files are of different types like .iso, .dat etc), The first few files (upload sequence is like first few hundred files are smaller i.e less than few MBs then Last 10 files are more than 1 GB big)will be successfully uploaded (i.e all smaller files and 2 of the last 10 files) but when it starts uploading the third big file I get following exception.
SEVERE: null
org.apache.commons.net.io.CopyStreamException: IOException caught while copying.
at org.apache.commons.net.io.Util.copyStream(Util.java:134)
at org.apache.commons.net.ftp.FTPClient._storeFile(FTPClient.java:653)
at org.apache.commons.net.ftp.FTPClient.__storeFile(FTPClient.java:624)
at org.apache.commons.net.ftp.FTPClient.storeFile(FTPClient.java:1976)
The CopyStreamException has a "cause" exception. Check that using the .getCause(), to see what went wrong.
See the Util.copyStream method:
public static final long copyStream(InputStream source, OutputStream dest,
int bufferSize, long streamSize,
CopyStreamListener listener,
boolean flush)
throws CopyStreamException
{
int bytes;
long total = 0;
byte[] buffer = new byte[bufferSize >= 0 ? bufferSize : DEFAULT_COPY_BUFFER_SIZE];
try
{
while ((bytes = source.read(buffer)) != -1)
{
....
}
}
catch (IOException e)
{
throw new CopyStreamException("IOException caught while copying.",
total, e);
}
return total;
}
Somewhere in your uploadSingleFile function, do
try
{
ftpClient.storeFile(...)
}
catch (Exception e)
{
e.printStackTrace();
if (e.getCause() != null)
{
e.getCause().printStackTrace();
}
}
I do not know Java, so the code may not be 100% correct.
See also Getting full string stack trace including inner exception.