We Keep Coding

Character encoding of parsed strings is wrong only after building jar

I am writing a program that generates PDF files of printable exams. I have all the exam questions stored in a JSON file. The catch is that the exam is in Czech, so there are many special characters (specifically ěščřžýáíé). When I run the program in Idea, it works perfectly - the output is exactly as it is supposed to be.
But when I build the jar executable, the generated files have chunks of wrong encoded text. Specifically anything that went through the JSON parser. Everything hard coded like headers etc. is encoded properly, so the mistake must be in the parser.
The JSON input file is encoded in UTF-8.
I use these two methods to parse the JSON file.
private static Category[] parseJSON(){
JSONParser jsonParser = new JSONParser();
Category[] categories = new Category[0];
try (FileReader reader = new FileReader("otazky.json")){
// Read JSON file
Object obj = jsonParser.parse(reader);
JSONArray categoryJSONList = (JSONArray) obj;
java.util.List<JSONObject> categoryList = new ArrayList<>(categoryJSONList);
categories = new Category[categoryJSONList.size()];
int i = 0;
for (JSONObject category : categoryList) {
categories[i] = parseCategoryObject(category);
i++;
}
} catch (ParseException | IOException e) {
e.printStackTrace();
}
return categories;
}
private static Category parseCategoryObject(JSONObject category) {
String categoryName = (String) category.get("name");
int generateCount = (int) (long) category.get("generateCount");
JSONArray questionsJSONArray = (JSONArray) category.get("questions");
java.util.List<JSONObject> questionJSONList = new ArrayList<>(questionsJSONArray);
Question[] questions = new Question[questionJSONList.size()];
int j = 0;
for (JSONObject question : questionJSONList) {
JSONArray answers = (JSONArray) question.get("answers");
String s = (String) question.get("question");
String[] a = new String[answers.size()];
for (int i = 0; i < answers.size(); i++) {
a[i] = answers.get(i).toString();
}
int c = (int) (long) question.get("correct");
Question q = new Question(s, a, c);
questions[j] = q;
j++;
}
return new Category(categoryName, questions, generateCount);
}
The output looks like this:
...
Právnà norma:
a) je obecnÄ› závaznĂ© pravidlo chovánĂ, kterĂ© nemusĂ mĂt urÄŤitou formu,
b) nemĹŻĹľe bĂ˝t součástĂ právnĂho pĹ™edpisu,
...
While I would need it to look like this:
...
Právní norma:
a) je obecně závazné pravidlo chování, které nemusí mít určitou formu,
b) nemůže být součástí právního předpisu,
...

Benjamin Urquhart suggested that I try using InputStringReader and FileInputStream instead of FileReader to read the file, because with FileReader you cannot specify the encoding (system default is used). I find those two methods hard to use, but I found an alternative - Files.readAllLines, which is fairly easy to use, and it worked.

In itext7,how to change attach files display order by added time

I want to change my attach file order in created pdf,attachment are displayed default by name,
how to change them displayed by add time?
this is my implement method:
#Override
public boolean attachFile(String src, String dest, List<SysItemfile> attachmentpaths) {
try {
PdfName name = new PdfName(src);
PdfDocument pdfDoc = new PdfDocument(new PdfReader(src), new PdfWriter(dest));
List<String> descs = new ArrayList<String>();
int i = 0;
int j = 1;
for (SysItemfile attachmentpath : attachmentpaths) {
String filename = attachmentpath.getFilename();
//test for the file name
System.out.println("filename:"+filename);
if (descs.contains(attachmentpath.getFilename())) {
//get the file suffix
String suffix = filename.substring(filename.lastIndexOf(".") + 1);
String realname = filename.substring(0,filename.lastIndexOf("."));
filename = realname+i+"."+suffix;
i++;
} else {
descs.add(attachmentpath.getFilename());
}
PdfFileSpec spec = PdfFileSpec.createEmbeddedFileSpec(pdfDoc, attachmentpath.getFileurl(),
filename, filename, name, name);
// the first parameter is discription
pdfDoc.addFileAttachment(filename, spec);
}
pdfDoc.close();
} catch (IOException e) {
logger.error("attachFile unsuccess!");
logger.error(e.getLocalizedMessage());
return false;
}
return true;
}
After that , when i add attachment to my pdf,the cann't change the order of attachment display.
what should I do?

As long as you only add attachments, the PDF standard does not allow for prescribing the sort order a PDF viewer uses when displaying the attachments.
If, on the other hand, you make the PDF a portable collection (aka a Portfolio), you can prescribe a schema (i.e. the fields in the detail list) and the sort order (by one or a combination of those fields).
You can quite easily make your PDF with attachments a portable collection with the name and modification date sorted by the latter like this:
try ( PdfReader reader = new PdfReader(...);
PdfWriter writer = new PdfWriter(...);
PdfDocument document = new PdfDocument(reader, writer)) {
PdfCollection collection = new PdfCollection();
document.getCatalog().setCollection(collection);
PdfCollectionSchema schema = new PdfCollectionSchema();
PdfCollectionField field = new PdfCollectionField("File Name", PdfCollectionField.FILENAME);
field.setOrder(0);
schema.addField("Name", field);
field = new PdfCollectionField("Modification Date", PdfCollectionField.MODDATE);
field.setOrder(1);
schema.addField("Modified", field);
collection.setSchema(schema);
PdfCollectionSort sort = new PdfCollectionSort("Modified");
collection.setSort(sort);
}
(SortAttachments test testAttachLikeGeologistedWithCollection)
You actually even can define a custom field using a type PdfCollectionField.TEXT, PdfCollectionField.DATE, or PdfCollectionField.NUMBER by which to sort. You can set the value of such a custom field on a PdfFileSpec via its setCollectionItem method.

java.net.malformedURL exception

URL stringfile = getXsl("test.xml");
File originFile = new File(stringfile.getFile());
String xml = null;
ByteArrayOutputStream pdfStream = null;
try {
FileInputStream fis = new FileInputStream(originFile);
int length = fis.available();
byte[] readData = new byte[length];
fis.read(readData);
xml = (new String(readData)).trim();
fis.close();
xml = xml.substring(xml.lastIndexOf("<HttpCommandList>")+17, xml.lastIndexOf("</HttpCommandList>"));
String[] splitxml = xml.split("</HttpCommand>");
for (int i = 0; i < splitxml.length; i++) {
tmpxml = splitxml[i].trim() + "</HttpCommand>";
System.out.println("splitxml:" +tmpxml);
pdfStream = new ByteArrayOutputStream();
pdf = new com.lowagie.text.Document();
PdfWriter.getInstance(pdf, pdfStream);
pdf.open();
URL xslToUse = getXsl("test.xsl");
// Here am using some utility class to transform
// generate the XML needed by iText to generate the PDF using MessageBuffer contents
String iTextXml = XmlUtil.transformXml(tmpxml.toString(), xslToUse).trim();
// generate the PDF document by parsing the specified XML file
XmlParser.parse(pdf, new ByteArrayInputStream(iTextXml.getBytes()));
}
For the above code, during the XmlParser am getting java.net.malformedURL exception : no protocol
Am trying to generate the pdf document by parsing the specified xml file.

We could need the actual xml-file to decide what is missing. I expect, that there is no protocol defined, just like this:
192.168.1.2/ (no protocol)
file://192.168.1.2/ (there is one)
And URL seems to need one.
Also try:
new File("somexsl.xlt").toURI().toURL();
See here and here.
It always helps spoilering the complete stacktrace. No one knows, where the exception actually occured, if you dont post the line numbers.

Logic breakdown - unexpected behavior testing value

I've got a function which looks in android assets and loads if asset exists.
If Not then looks on file system. If it finds it, it loads it - if file not zero bytes.
If not found on file system it makes 2 attempts to get it from a server.
All of this works except if the the file is already there and the size is 0 - and its failing on some "primitive" java logic test. I can't see what is wrong here and exploring the possibility of it being a bug outside of my own code.
The problem is occurring at
if (lenoffile>1){ // always fails
where any check of this int against 0 causes the function to return and not execute the expected flow.
private String readQuestions(String xml_file){
Log.i(TAG2,"readQuestions");
List<String> mapList = null;
String xmlString = null;
int lenoffile =0;
AssetManager am = this.getAssets();
try {
mapList = Arrays.asList(am.list(""));
if (mapList.contains(xml_file)){
InputStream is = am.open(xml_file);
int length = is.available();
byte[] data = new byte[length];
is.read(data);
xmlString = new String(data);
return xmlString;
}
} catch (IOException e1) {
Log.i(TAG,"Failed in ReadQuestions() from assets");
// e1.printStackTrace();
//return null;
}
Log.i(TAG2,"File not an asset. Must be on FS");
if (me.FileExists(Environment.getExternalStorageDirectory().getAbsolutePath() + APPDIR +"/"+xml_file))
xmlString = m.fileLoader(xml_file);
if (xmlString != null){
lenoffile = xmlString.length();
Log.i(TAG2,"Length of file found=" + lenoffile);
if (lenoffile>1){ // below never gets executed even with lenoffile==0
Log.i(TAG2,"Returning here with a file of length "+ lenoffile);
return xmlString; // assume the full file exists and validate
}
} // if else then there is a null situation
else{ // look on FS and load accordingly
DownloadFileAsync Z = new DownloadFileAsync();
trace:
06-09 12:26:58.197: I/SRCFILE(11792): readQuestions
06-09 12:27:08.822: I/SRCFILE(11792): File not an asset. Must be on FS
06-09 12:27:15.197: I/SRCFILE(11792): Length of file found=0
Thanks in advance.
Kevin

Open Microsoft Word in Java

I'm trying to open MS Word 2003 document in java, search for a specified String and replace it with a new String. I use APACHE POI to do that. My code is like the following one:
public void searchAndReplace(String inputFilename, String outputFilename,
HashMap<String, String> replacements) {
File outputFile = null;
File inputFile = null;
FileInputStream fileIStream = null;
FileOutputStream fileOStream = null;
BufferedInputStream bufIStream = null;
BufferedOutputStream bufOStream = null;
POIFSFileSystem fileSystem = null;
HWPFDocument document = null;
Range docRange = null;
Paragraph paragraph = null;
CharacterRun charRun = null;
Set<String> keySet = null;
Iterator<String> keySetIterator = null;
int numParagraphs = 0;
int numCharRuns = 0;
String text = null;
String key = null;
String value = null;
try {
// Create an instance of the POIFSFileSystem class and
// attach it to the Word document using an InputStream.
inputFile = new File(inputFilename);
fileIStream = new FileInputStream(inputFile);
bufIStream = new BufferedInputStream(fileIStream);
fileSystem = new POIFSFileSystem(bufIStream);
document = new HWPFDocument(fileSystem);
docRange = document.getRange();
numParagraphs = docRange.numParagraphs();
keySet = replacements.keySet();
for (int i = 0; i < numParagraphs; i++) {
paragraph = docRange.getParagraph(i);
text = paragraph.text();
numCharRuns = paragraph.numCharacterRuns();
for (int j = 0; j < numCharRuns; j++) {
charRun = paragraph.getCharacterRun(j);
text = charRun.text();
System.out.println("Character Run text: " + text);
keySetIterator = keySet.iterator();
while (keySetIterator.hasNext()) {
key = keySetIterator.next();
if (text.contains(key)) {
value = replacements.get(key);
charRun.replaceText(key, value);
docRange = document.getRange();
paragraph = docRange.getParagraph(i);
charRun = paragraph.getCharacterRun(j);
text = charRun.text();
}
}
}
}
bufIStream.close();
bufIStream = null;
outputFile = new File(outputFilename);
fileOStream = new FileOutputStream(outputFile);
bufOStream = new BufferedOutputStream(fileOStream);
document.write(bufOStream);
} catch (Exception ex) {
System.out.println("Caught an: " + ex.getClass().getName());
System.out.println("Message: " + ex.getMessage());
System.out.println("Stacktrace follows.............");
ex.printStackTrace(System.out);
}
}
I call this function with following arguments:
HashMap<String, String> replacements = new HashMap<String, String>();
replacements.put("AAA", "BBB");
searchAndReplace("C:/Test.doc", "C:/Test1.doc", replacements);
When the Test.doc file contains a simple line like this : "AAA EEE", it works successfully, but when i use a complicated file it will read the content successfully and generate the Test1.doc file but when I try to open it, it will give me the following error:
Word unable to read this document. It may be corrupt.
Try one or more of the following:
* Open and repair the file.
* Open the file with Text Recovery converter.
(C:\Test1.doc)
Please tell me what to do, because I'm a beginner in POI and I have not found a good tutorial for it.

First of all you should be closing your document.
Besides that, what I suggest doing is resaving your original Word document as a Word XML document, then changing the extension manually from .XML to .doc . Then look at the XML of the actual document you're working with and trace the content to make sure you're not accidentally editing hexadecimal values (AAA and EEE could be hex values in other fields).
Without seeing the actual Word document it's hard to say what's going on.
There is not much documentation about POI at all, especially for Word document unfortunately.

I don't know : is its OK to answer myself, but Just to share the knowledge, I'll answer myself.
After navigating the web, the final solution i found is :
The Library called docx4j is very good for dealing with MS docx file, although its documentation is not enough till now and its forum is still in a beginning steps, but overall it help me to do what i need..
Thanks 4 all who help me..

You could try OpenOffice API, but there arent many resources out there to tell you how to use it.

You can also try this one: http://www.dancrintea.ro/doc-to-pdf/

Looks like this could be the issue.