We Keep Coding

What is the best way of parsing an Input String from a Socket Connection when you don't own the Socket server and can't see what it is doing?

I have posted this before. I am an absolutely new to this type of situation. I am well versed in the M3/MOVEX proprietary software, but I have never used sockets to connect to a server before. I have gotten to this point: I can connect - I can send an XML String to the server - I can get a response back and
put it in a string. I have tried document builder to parse it, but when I do, the document comes back null.
try {
//TCPSocket = new Socket("10.33.106.29", 10101);
//Open the socket connection
TCPSocket = new Socket("idc-v-pobap02t.ebsco.com", 10101);
//TCPSocket = new Socket("idc-v-pobap02t", 10101);
//Initialize the reader and writer for the socket
is = new BufferedReader(new
InputStreamReader(TCPSocket.getInputStream()));
pw = new PrintWriter(TCPSocket.getOutputStream(), true);
//Send Data to the socket server
if (TCPSocket != null && is != null) {
pw.println(XMLString + "\n");
pw.flush();
}
///The
while((responseLine = is.readLine()) != null){
//Do Something
}
//Hangs so I took it out
//Read the Data from the socket server.
responseLine = is.readLine(); //Has my input XML String
is.close();
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder;
builder = factory.newDocumentBuilder();
doc = builder.newDocument();
doc.setXmlVersion("1.0");
doc = builder.parse(new InputSource(new
StringReader(responseLine)));
I tried this as well earlier:
XMLInputFactory f = XMLInputFactory.newInstance();
XMLStreamReader r;
try {
r = f.createXMLStreamReader(new StringReader(responseLine));
int eventType = r.next();
while(r.hasNext()) {
r.require(XMLStreamReader.START_ELEMENT, null,
"TagName");
if(eventType == XMLStreamConstants.END_ELEMENT){
break;
}
//EventName = r.getName();
if(eventType == XMLStreamConstants.CHARACTERS
|| eventType == XMLStreamConstants.CDATA
|| eventType == XMLStreamConstants.SPACE
|| eventType == XMLStreamConstants.ENTITY_REFERENCE) {
serverResponse.append(r.getText());
}
r.nextTag();
eventType = r.nextTag();
}
} catch (XMLStreamException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Is there something I am missing here? As I said, I am totally new at this, so please don't assume that I know anything if you respond.

I have it figured out. This is what I did - it works, but if there is a better way - let me know.
XML Stream looks like this:
<loads>
<load date="" groupsplitting="allowed" sequencemaxlookahead="0"
usesequenceovercontainers="true" containerselectionrule="7"
numstagingpositions="0" loadproportionally="false"
sequenceiscontainertype="false" customerid="" stage="1" userint1="0"
userstring2="" userstring1="" shipdate="" notes=""
loadid="2000151290:01001072269" id="2000151290:01001072269"/>
</loads>
serverResponse.append(responseLine);
DocumentBuilder builder =
DocumentBuilderFactory.newInstance().newDocumentBuilder();
doc = builder.parse(new InputSource(new StringReader(responseLine)));
doc.getDocumentElement().normalize();
doc.getDocumentElement().getNodeName();
Element rootElement = doc.getDocumentElement();
NodeList sElement = doc.getElementsByTagName("load");
for (int i = 0; i < sElement.getLength(); i++) {
Element node = (Element)sElement.item(i);
NamedNodeMap attributes = node.getAttributes();
for (int i2 = 0; i2 < attributes.getLength(); i2++) {
Attr attr = (Attr) attributes.item(i2);
if(attr.getNodeName() == "loadid"){
Delivery = attr.getNodeValue();
}
}
}

Save content from XML into CSV file

I have below code which is going by all XML content.
Now, I have open stream writer on beginig, but I don't know how to add to the method:
bw.write
ReadXML.java
public class ReadXML {
public static void main(String[] args) {
try {
File file = new File("C:\\test.xml");
File outputFile = new File("C:\\test.csv");
DocumentBuilder dBuilder = DocumentBuilderFactory.newInstance().newDocumentBuilder();
Document doc = dBuilder.parse(file);
BufferedWriter bw = null;
FileWriter fw = null;
System.out.println("Root element :" + doc.getDocumentElement().getNodeName());
if (doc.hasChildNodes()) {
printNote(doc.getChildNodes());
}
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
private static void printNote(NodeList nodeList) {
for (int count = 0; count < nodeList.getLength(); count++) {
Node tempNode = nodeList.item(count);
if (tempNode.getNodeType() == Node.ELEMENT_NODE) {
System.out.println("\nNode Name =" + tempNode.getNodeName() + " [OPEN]");
System.out.println("Node Value =" + tempNode.getTextContent());
if (tempNode.hasAttributes()) {
// get attributes names and values
NamedNodeMap nodeMap = tempNode.getAttributes();
for (int i = 0; i < nodeMap.getLength(); i++) {
Node node = nodeMap.item(i);
System.out.println("attr name : " + node.getNodeName());
System.out.println("attr value : " + node.getNodeValue());
}
}
if (tempNode.hasChildNodes()) {
// loop again if has child nodes
printNote(tempNode.getChildNodes());
}
System.out.println("Node Name =" + tempNode.getNodeName() + " [CLOSE]");
} } }}
can you please help me with it? if you know how to resolve the issue will be great.
thanks!

Okay, still not sure what's exactly your problem but maybe this helps.
First, open the writer:
final BufferedWriter w = new BufferedWriter(new FileWriter(outputFile));
Then pass it to printNote:
printNote(doc.getChildNodes(), w);
Modify the method accordingly:
private static void printNote(final NodeList nodeList, final BufferedWriter w) throws IOException {
// ...
}
When you have the node you want to write to the file do:
w.write(node.getTextContent());
w.newLine();
Don't forget to close your writer after you're done!
Edit
Examples for closing the writer:
Old school
public static void mainv1(String[] args) {
File file = new File("C:\\test.xml");
File outputFile = new File("C:\\test.csv");
BufferedWriter bw = null;
try {
DocumentBuilder dBuilder = DocumentBuilderFactory.newInstance().newDocumentBuilder();
Document doc = dBuilder.parse(file);
System.out.println("Root element :" + doc.getDocumentElement().getNodeName());
// Open in try because FileWriter constructor throws IOException
bw = new BufferedWriter(new FileWriter(outputFile));
if (doc.hasChildNodes()) {
printNote(doc.getChildNodes(), bw);
}
} catch (Exception e) {
System.out.println(e.getMessage());
} finally {
// Check for null because bw won't be initialized if document parsing failed
if (bw != null) {
try {
bw.close();
} catch (final IOException e) {
// Log error
}
}
}
}
Java7 and higher
public static void main(String[] args) {
File file = new File("C:\\test.xml");
File outputFile = new File("C:\\test.csv");
// Since Java7 you can use try-with-resources
// The finally block closing the writer will be created automatically
try (BufferedWriter bw = new BufferedWriter(new FileWriter(outputFile))) {
DocumentBuilder dBuilder = DocumentBuilderFactory.newInstance().newDocumentBuilder();
Document doc = dBuilder.parse(file);
System.out.println("Root element :" + doc.getDocumentElement().getNodeName());
if (doc.hasChildNodes()) {
printNote(doc.getChildNodes(), bw);
}
} catch (Exception e) {
System.out.println(e.getMessage());
}
}

How to replace a part of a string in an xml file?

I have an xml file with something like this:
<Verbiage>
The whiskers plots are based on the responses of incarcerated
<Choice>
<Juvenile> juveniles who have committed sexual offenses. </Juvenile>
<Adult> adult sexual offenders. </Adult>
</Choice>
If the respondent is a
<Choice>
<Adult>convicted sexual offender, </Adult>
<Juvenile>juvenile who has sexually offended, </Juvenile>
</Choice>
#his/her_lc# percentile score, which defines #his/her_lc# position
relative to other such offenders, should be taken into account as well as #his/her_lc# T score. Percentile
scores in the top decile (> 90 %ile) of such offenders suggest that the respondent
may be defensive and #his/her_lc# report should be interpreted with this in mind.
</Verbiage>
I am trying to find a way to parse the xml file (I've been using DOM), search for #his/her_lc# and replace that with "her". I've tried using FileReader,BufferedReader, string.replaceAll, FileWriter, but those didn't work.
Is there a way I could do this using XPath?
Ultimately I want to search this xml file for this string and replace it with another string.
do I have to add a tag around the string I want it parse it that way?
Code I tried:
protected void parse() throws ElementNotValidException {
try {
//Parse xml File
File inputXML = new File("template.xml");
DocumentBuilderFactory parser = DocumentBuilderFactory.newInstance(); // new instance of doc builder
DocumentBuilder dParser = parser.newDocumentBuilder(); // calls it
Document doc = dParser.parse(inputXML); // parses file
FileReader reader = new FileReader(inputXML);
String search = "#his/her_lc#";
String newString;
BufferedReader br = new BufferedReader(reader);
while ((newString = br.readLine()) != null){
newString.replaceAll(search, "her");
}
FileWriter writer = new FileWriter(inputXML);
writer.write(newString);
writer.close();
} catch (ParserConfigurationException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (SAXException e) {
e.printStackTrace();
}
Code I was given to fix:
try {
File inputXML = new File("template.xml"); // creates new input file
DocumentBuilderFactory parser = DocumentBuilderFactory.newInstance(); // new instance of doc builder
DocumentBuilder dParser = parser.newDocumentBuilder(); // calls it
Document doc = dParser.parse(inputXML); // parses file
doc.getDocumentElement().normalize();
NodeList pList = doc.getElementsByTagName("Verbiage"); // gets element by tag name and places into list to begin parsing
int gender = 1; // gender has to be taken from the response file, it is hard coded for testing purposes
System.out.println("----------------------------"); // new line
// loops through the list of Verbiage tags
for (int temp = 0; temp < pList.getLength(); temp++) {
Node pNode = pList.item(0); // sets node to temp
if (pNode.getNodeType() == Node.ELEMENT_NODE) { // if the node type = the element node
Element eElement = (Element) pNode;
NodeList pronounList = doc.getElementsByTagName("pronoun"); // gets a list of pronoun element tags
if (gender == 0) { // if the gender is male
int count1 = 0;
while (count1 < pronounList.getLength()) {
if ("#he/she_lc#".equals(pronounList.item(count1).getTextContent())) {
pronounList.item(count1).setTextContent("he");
}
if ("#he/she_caps#".equals(pronounList.item(count1).getTextContent())) {
pronounList.item(count1).setTextContent("He");
}
if ("#his/her_lc#".equals(pronounList.item(count1).getTextContent())) {
pronounList.item(count1).setTextContent("his");
}
if ("#his/her_caps#".equals(pronounList.item(count1).getTextContent())) {
pronounList.item(count1).setTextContent("His");
}
if ("#him/her_lc#".equals(pronounList.item(count1).getTextContent())) {
pronounList.item(count1).setTextContent("him");
}
count1++;
}
pNode.getNextSibling();
} else if (gender == 1) { // female
int count = 0;
while (count < pronounList.getLength()) {
if ("#he/she_lc#".equals(pronounList.item(count).getTextContent())) {
pronounList.item(count).setTextContent("she");
}
if ("#he/she_caps3".equals(pronounList.item(count).getTextContent())) {
pronounList.item(count).setTextContent("She");
}
if ("#his/her_lc#".equals(pronounList.item(count).getTextContent())) {
pronounList.item(count).setTextContent("her");
}
if ("#his/her_caps#".equals(pronounList.item(count).getTextContent())) {
pronounList.item(count).setTextContent("Her");
}
if ("#him/her_lc#".equals(pronounList.item(count).getTextContent())) {
pronounList.item(count).setTextContent("her");
}
count++;
}
pNode.getNextSibling();
}
}
}
// write the content to file
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = transformerFactory.newTransformer();
DOMSource source = new DOMSource(doc);
System.out.println("-----------Modified File-----------");
StreamResult consoleResult = new StreamResult(System.out);
transformer.transform(source, new StreamResult(new FileOutputStream("template.xml"))); // writes changes to file
} catch (Exception e) {
e.printStackTrace();
}
}
This code I think would work if I could figure out how to associate the tag Pronoun with the pronounParser that this code is in.

I used this example and your template.xml, and I think it works.
public static void main(String[] args) {
File inputXML = new File("template.xml");
BufferedReader br = null;
String newString = "";
StringBuilder strTotale = new StringBuilder();
try {
FileReader reader = new FileReader(inputXML);
String search = "#his/her_lc#";
br = new BufferedReader(reader);
while ((newString = br.readLine()) != null){
newString = newString.replaceAll(search, "her");
strTotale.append(newString);
}
} catch ( IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} // calls it
finally
{
try {
br.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
System.out.println(strTotale.toString());
}
First you must reassign the result of replaceAll:
newString = newString.replaceAll(search, "her");
Second I used a StringBuffer to collect all lines.
I hope this help.

Since strings are immutable you can not modify them, use
StringBuilder/StringBuffer
instead of String.
FileReader reader = new FileReader(inputXML);
String search = "#his/her_lc#";
String newString;
StringBuffer str;
BufferedReader br = new BufferedReader(reader);
while ((newString = br.readLine()) != null){
str.append(newString.replaceAll(search, "her"));
}
FileWriter writer = new FileWriter(inputXML);
writer.write(str);
writer.close();

Split XML file with the new files with encoding="ISO-8859-1"

i have a program to take a big xml file and validate it and then split it in smaller files. the issue am getting is that the encoding of the new files is UTF-8 .I need them in ISO-8859-1
herez the code
public class SplitMain {
public static void main(String [] args) throws Exception {
validateInputFile("D:/sanket/cms_dev/XmlSplitSample/src/inputFile/");
File input = new File("D:/sanket/cms_dev/XmlSplitSample/src/inputFile/sample.xml");
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
Document doc = dbf.newDocumentBuilder().parse(input);
XPath xpath = XPathFactory.newInstance().newXPath();
NodeList nodes = (NodeList) xpath.evaluate("//DataFile/Contact", doc, XPathConstants.NODESET);
int itemsPerFile = 5;
int fileNumber = 0;
Document currentDoc = dbf.newDocumentBuilder().newDocument();
Node rootNode = currentDoc.createElement("DataFile");
File currentFile = new File("nufile"+fileNumber+".xml");
for (int i=1; i <= nodes.getLength(); i++) {
Node imported = currentDoc.importNode(nodes.item(i-1), true);
rootNode.appendChild(imported);
if (i % itemsPerFile == 0) {
writeToFile(rootNode, currentFile);
rootNode = currentDoc.createElement("DataFile");
currentFile = new File("nufile"+(++fileNumber)+".xml");
System.out.println(currentFile);
}
}
writeToFile(rootNode, currentFile);
}
private static void writeToFile(Node node, File file) throws Exception {
Transformer transformer = TransformerFactory.newInstance().newTransformer();
transformer.transform(new DOMSource(node), new StreamResult(new FileWriter(file)));
}
private static void validateInputFile(String WORK_DIRECTORY)
{
//String workingDir=config.getProperty(WORK_DIRECTORY);//comment for automating the process
String workingDir=WORK_DIRECTORY;//added for automating the process
String finalString = null;
File folder = new File(workingDir);
if (folder.isFile())
{
System.out.println("watever");
return ;
}
String[] fileNameArray = folder.list();
String xmlExtension=".xml";
for (String fileName : fileNameArray) {
try{
//XMLtoString
BufferedReader br = new BufferedReader(new FileReader(new File(workingDir + "/" +fileName)));
String line;
StringBuilder stringBuilder = new StringBuilder();
while((line=br.readLine())!= null)
{
stringBuilder.append(line.trim());
}
finalString = stringBuilder.toString();
StringBuilder sb = new StringBuilder();
if(finalString == null)
return;
System.out.println(finalString);
for(int i=0;i<finalString.length();i++)
{
if (finalString.charAt(i) == '&')
{
sb.append("&");
}
else
{
sb.append(finalString.charAt(i));
}
}
finalString=sb.toString();
//StringToXML
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
System.out.println(finalString);
DocumentBuilder builder=factory.newDocumentBuilder();
Document document = builder.parse( new InputSource(new StringReader( finalString ) ) );
TransformerFactory tranFactory = TransformerFactory.newInstance();
Transformer aTransformer = tranFactory.newTransformer();
Source src = new DOMSource(document);
Result dest = new StreamResult( new File( workingDir + "/" +fileName) );
aTransformer.transform( src, dest );
}
catch (Exception e) {
e.printStackTrace();
}
}
}
}

You need to specify the Transformer encoding e.g.:
transformer.setOutputProperty(OutputKeys.ENCODING, "ISO-8859-1");
and then write out using a stream (if you use a writer, it'll go through another unwanted layer of encoding)

Strip whitespace and newlines from XML in Java

Using Java, I would like to take a document in the following format:
<tag1>
<tag2>
<![CDATA[ Some data ]]>
</tag2>
</tag1>
and convert it to:
<tag1><tag2><![CDATA[ Some data ]]></tag2></tag1>
I tried the following, but it isn't giving me the result I am expecting:
DocumentBuilderFactory dbfac = DocumentBuilderFactory.newInstance();
dbfac.setIgnoringElementContentWhitespace(true);
DocumentBuilder docBuilder = dbfac.newDocumentBuilder();
Document doc = docBuilder.parse(new FileInputStream("/tmp/test.xml"));
Writer out = new StringWriter();
Transformer tf = TransformerFactory.newInstance().newTransformer();
tf.setOutputProperty(OutputKeys.INDENT, "no");
tf.transform(new DOMSource(doc), new StreamResult(out));
System.out.println(out.toString());

Working solution following instructions in the question's comments by #Luiggi Mendoza.
public static String trim(String input) {
BufferedReader reader = new BufferedReader(new StringReader(input));
StringBuffer result = new StringBuffer();
try {
String line;
while ( (line = reader.readLine() ) != null)
result.append(line.trim());
return result.toString();
} catch (IOException e) {
throw new RuntimeException(e);
}
}

recursively traverse the document. remove any text nodes with blank content. trim any text nodes with non-blank content.
public static void trimWhitespace(Node node)
{
NodeList children = node.getChildNodes();
for(int i = 0; i < children.getLength(); ++i) {
Node child = children.item(i);
if(child.getNodeType() == Node.TEXT_NODE) {
child.setTextContent(child.getTextContent().trim());
}
trimWhitespace(child);
}
}

As documented in an answer to another question, the relevant function would be DocumentBuilderFactory.setIgnoringElementContentWhitespace(), but - as pointed out here already - that function requires the use of a validating parser, which requires an XML schema, or some such.
Therefore, your best bet is to iterate through the Document you get from the parser, and remove all nodes of type TEXT_NODE (or those TEXT_NODEs which contain only whitespace).

I support #jtahlborn's answer. Just for completeness, I adapted his solution to completely remove the whitespace-only elements instead of just clearing them.
public static void stripEmptyElements(Node node)
{
NodeList children = node.getChildNodes();
for(int i = 0; i < children.getLength(); ++i) {
Node child = children.item(i);
if(child.getNodeType() == Node.TEXT_NODE) {
if (child.getTextContent().trim().length() == 0) {
child.getParentNode().removeChild(child);
i--;
}
}
stripEmptyElements(child);
}
}

Java8+transformer does not create any but Java10+transformer puts everywhere empty lines. I still want to keep a pretty indents. This is my helper function to create xml string from any DOMElement instance such as doc.getDocumentElement() root node.
public static String createXML(Element elem) throws Exception {
DOMSource source = new DOMSource(elem);
StringWriter writer = new StringWriter();
StreamResult result = new StreamResult(writer);
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = transformerFactory.newTransformer();
transformer.setOutputProperty(OutputKeys.INDENT, "yes");
//transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", "2");
//transformer.setOutputProperty("http://www.oracle.com/xml/is-standalone", "yes");
transformer.setOutputProperty(OutputKeys.DOCTYPE_PUBLIC,"yes");
transformer.setOutputProperty("http://www.oracle.com/xml/is-standalone", "yes");
transformer.transform(source, result);
// Java10-transformer adds unecessary empty lines, remove empty lines
BufferedReader reader = new BufferedReader(new StringReader(writer.toString()));
StringBuilder buf = new StringBuilder();
try {
final String NL = System.getProperty("line.separator", "\r\n");
String line;
while( (line=reader.readLine())!=null ) {
if (!line.trim().isEmpty()) {
buf.append(line);
buf.append(NL);
}
}
} finally {
reader.close();
}
return buf.toString(); //writer.toString();
}

In order to strip whitespace and newlines from XML in Java try the following solution which uses StringBuffer() and conditional logic:
public static String LimpaXML(String xml) {
StringBuffer result = new StringBuffer();
char c_prev = '\0';
xml = xml.trim();
int len = xml.length();
for (int i=0; i<len; i++) {
char c = xml.charAt(i);
char c_next = (i+1 < len) ? xml.charAt(i+1) : '\0';
if (c == '\n') continue;
if (c == '\r') continue;
if (c == '\t') c = ' ';
if (c == ' ') {
if (c_prev == ' ') continue;
if (c_next == '\0') continue;
if (c_prev == '>') continue;
if (c_next == '>') continue;
}
result.append(c);
c_prev = c;
}
return result.toString();
}

Try this code. read and write methods in FileStream ignore whitespace and indents.
try {
File f1 = new File("source.xml");
File f2 = new File("destination.xml");
InputStream in = new FileInputStream(f1);
OutputStream out = new FileOutputStream(f2);
byte[] buf = new byte[1024];
int len;
while ((len = in.read(buf)) > 0){
out.write(buf, 0, len);
}
in.close();
out.close();
System.out.println("File copied.");
} catch(FileNotFoundException ex){
System.out.println(ex.getMessage() + " in the specified directory.");
System.exit(0);
} catch(IOException e7){
System.out.println(e7.getMessage());
}