As an upshot (and I can expound by code examples if necessary), I have just realized that my REST API, written in Java, provided by CXF and served by Tomcat 7 is case sensitive when it comes to posting XML content.
Is there any way to make the XML, which usually is a marshalled representation of the entity a service creates, can be case insensitive?
I can certainly post examples of the entity class, service, and their annotations if necessary but as bare minimum, if an instance variable in the entity is private String firstName, the XML tag must be <firstName>...</firstName> and not <firstname>...</firstname> but I would like to make the latter marshall-able.
A complete solution involves a lot of work but it is perfectly possible. Following the link posted by #matiaselgart, the general solution would be
1 - Add a CXF Interceptor to manipulate the Message
2 - Read the incoming content, extract the XML, and process it with a StreamReaderDelegate to convert to lowercase
3 - Replace the content in Message with the output
The JAXB tags should be in lowercase, so the streamer could convert them easily, and be processed by JAXB unmarshaller. In your example private String firstName, the XML tag must be <firstname>...</firstname> and not <firstName>...</firstName>.
CXF Interceptor
public class CaseInsensitiveInterceptor extends AbstractPhaseInterceptor<Message> {
public CaseInsensitiveInterceptor () {
super(Phase.RECEIVE);
}
public void handleMessage(Message message) {
//Get the message body as input stream, process the xml, and set a new non-consumed inputStream into Message
InputStream in = message.getContent(InputStream.class);
InputStream bin = xmlToLowerCase (in);
message.setContent(InputStream.class, bin);
}
public void handleFault(Message messageParam) {
//Invoked when interceptor fails
}
}
Configuration
Add the interceptor in the bus or in the provider
<bean id="caseInsensitiveInterceptor" class="CaseInsensitiveInterceptor " />
<cxf:bus>
<cxf:inInterceptors>
<ref bean="caseInsensitiveInterceptor"/>
</cxf:inInterceptors>
</cxf:bus>
Case Insensitive StreamReaderDelegate
I think you can use the StreamReaderDelegate from here and convert the XMLStreamReader to InputStream using this link . The method xmlToLowerCase is called from interceptor
WARNING: I have not tested this part of the code.
private static class MyStreamReaderDelegate extends StreamReaderDelegate {
public MyStreamReaderDelegate(XMLStreamReader xsr) {
super(xsr);
}
#Override
public String getAttributeLocalName(int index) {
return super.getAttributeLocalName(index).toLowerCase();
}
#Override
public String getLocalName() {
return super.getLocalName().toLowerCase();
}
}
public InputStream xmlToLowerCase (InputStream in){
XMLInputFactory xif = XMLInputFactory.newInstance();
XMLStreamReader xsr = xif.createXMLStreamReader(in);
xsr = new MyStreamReaderDelegate(xsr);
String xml = getOuterXml(xsr);
return new ByteArrayInputStream (xml.getBytes());
}
//https://coderanch.com/t/478588/XMLStreamReader-InputStream
private String getOuterXml(XMLStreamReader xmlr) throws TransformerConfigurationException,
TransformerFactoryConfigurationError, TransformerException
{
Transformer transformer = TransformerFactory.newInstance().newTransformer();
StringWriter stringWriter = new StringWriter();
transformer.transform(new StAXSource(xmlr), new StreamResult(stringWriter));
return stringWriter.toString();
}
I have a data source that has switched from base64 embedded image data to XOP image data. I'm using Java/JAXB to unmarshal the data and can't find any good sources describing how it's done. All references seem to describe doing this with SOAP messages which seem to take care of some of the heavy lifting for you.
In my case, the data is coming in effectively as a string that needs to be unmarshalled into objects created by JAXB. Since the new messages start with
Mime-Version: 1.0
Content-Type: multipart/related; start-info="text/xml"; type="application/xop+xml";
start="<-963165769043289641.1400077877224#xxxxxyyyyyy.ca>";
boundary="----=_Part_0_-338193320.1400077877317"
obviously it won't unmarshal the data as XML as it doesn't look like XML (no data allowed before prolog).
Can this be done with JAXB? The creation of the object from the schema seems to work fine and creates what looks like a proper object (it creates an Include element). I've tried creating an AttachmentUnmarshaller manually but it hasn't helped ... the incoming data is still not recognized as XML. I feel like I'm missing a fundamental step but can't find any good references or examples.
Any help would be greatly appreciated.
Below is an approach that might help.
Message
Below is roughly what you are going to be processing:
MIME-Version: 1.0
Content-Type: Multipart/Related;boundary=MIME_boundary;
...
--MIME_boundary
Content-Type: application/xop+xml;
...
<soap:Envelope ...
<soap:Body>...
<foo>
<photo xmlmime:contentType='image/png'>
<xop:Include xmlns:xop='http://www.w3.org/2004/08/xop/include'
href='cid:http://example.org/me.jpeg'/></m:photo>
...
--MIME_boundary
Content-Type: image/png
Content-Transfer-Encoding: binary
Content-ID: <http://example.org/me.png>
// binary octets for png
Demo Code
Demo
The code below assumes you have processed the message to do the following:
Extract the XML fragment
Extract the attachments and are able to key them on the cid.
import java.io.File;
import javax.xml.bind.*;
public class Demo {
private static String base64 = "/9j/4AAQSkZJRgABAgAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCAABAAEDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD5/ooooA//2Q==";
public static void main(String[] args) throws Exception {
// Create the JAXBContext & Unmarshaller
JAXBContext jc = JAXBContext.newInstance(Foo.class);
Unmarshaller unmarshaller = jc.createUnmarshaller();
// Create a custom AttachementUnmarshaller. Populate the Map
// AttachmentUnmarshaller with the attachments keyed on the cid.
MyAttachmentUnmarshaller attachmentUnmarshaller = new MyAttachmentUnmarshaller();
attachmentUnmarshaller.getAttachments().put("cid:http://example.org/me.jpeg", DatatypeConverter.parseBase64Binary(base64));
// Set the AttachmentUnmarshaller on the Unmarshaller
unmarshaller.setAttachmentUnmarshaller(attachmentUnmarshaller);
// Unmarshal the XML piece
File xml = new File("src/forum24407360/input.xml");
Foo foo = (Foo) unmarshaller.unmarshal(xml);
}
}
MyAttachmentUnmarshaller
Below is what the AttachmentUnmarshaller looks like. An AttachmentUnmarshaller is passed in the cid and is responsible for returning the corresponding binary data. In a JAX-WS environment this is handled automatically for you, but there is nothing preventing you from doing it manually. You can find more information about it here: What's the most standard Java way to store raw binary data along with XML?.
import java.io.*;
import java.util.*;
import javax.activation.*;
import javax.xml.bind.attachment.AttachmentUnmarshaller;
public class MyAttachmentUnmarshaller extends AttachmentUnmarshaller {
private Map<String, byte[]> attachments = new HashMap<String, byte[]>();
public Map<String, byte[]> getAttachments() {
return attachments;
}
#Override
public DataHandler getAttachmentAsDataHandler(String cid) {
byte[] bytes = attachments.get(cid);
return new DataHandler(new ByteArrayDataSource(bytes));
}
#Override
public byte[] getAttachmentAsByteArray(String cid) {
return attachments.get(cid);
}
#Override
public boolean isXOPPackage() {
return true;
}
private static class ByteArrayDataSource implements DataSource {
private byte[] bytes;
public ByteArrayDataSource(byte[] bytes) {
this.bytes = bytes;
}
public String getContentType() {
return "application/octet-stream";
}
public InputStream getInputStream() throws IOException {
return new ByteArrayInputStream(bytes);
}
public String getName() {
return null;
}
public OutputStream getOutputStream() throws IOException {
return null;
}
}
}
XML Piece
<foo>
<image>
<xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:http://example.org/me.jpeg"/>
</image>
</foo>
We are parsing an XML file with the SAX parser. Is it possible to get the schema location from the XML?
<view id="..." title="..."
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="{schema}">
I want to retrieve the {schema} value from the XML. Is this possible? And how to I access this value of noNamespaceSchemaLocation? I'm using the default SAX Parser.
#Override
public void startElement(String uri, String localName,
String name, Attributes attributes)
{ .... }
Thank you.
It all depends with what kind of tool/library you are working (a basic SAXParser? Xerces? JDom? ...) But what you want is the value of the attribute "noNamespaceSchemaLocation" in the namspace defined by the URI "http://www.w3.org/2001/XMLSchema-instance"
in JDom, it would be something like:
Element view = ...; // get the view element
String value = view.getAttributeValue("noNamespaceSchemaLocation", Namespace.getNamespace("http://www.w3.org/2001/XMLSchema-instance"));
Here is how I get the XSD's name using XMLStreamReader:
public static String extractXsdValueOrNull(#NonNull final InputStream xmlInput)
{
final XMLInputFactory f = XMLInputFactory.newInstance();
try
{
final XMLStreamReader r = f.createXMLStreamReader(xmlInput);
while (r.hasNext())
{
final int eventType = r.next();
if (XMLStreamReader.START_ELEMENT == eventType)
{
for (int i = 0; i <= r.getAttributeCount(); i++)
{
final boolean foundSchemaNameSpace = XMLConstants.W3C_XML_SCHEMA_INSTANCE_NS_URI.equals(r.getAttributeNamespace(i));
final boolean foundLocationAttributeName = SCHEMA_LOCATION.equals(r.getAttributeLocalName(i));
if (foundSchemaNameSpace && foundLocationAttributeName)
{
return r.getAttributeValue(i);
}
}
return null; // only checked the first element
}
}
return null;
}
catch (final XMLStreamException e)
{
throw new RuntimeException(e);
}
}
Actually XMLStreamReader does all the magic, namely:
only parses the XML's beginning (not the whole XML)
does not assume a particular namespace alias (i.e. xsi)
I have to convert docx file format (which is in openXML format) into JSON format. I need some guidelines to do it. Thanks in advance.
You may take a look at the Json-lib Java library, that provides XML-to-JSON conversion.
String xml = "<hello><test>1.2</test><test2>123</test2></hello>";
XMLSerializer xmlSerializer = new XMLSerializer();
JSON json = xmlSerializer.read( xml );
If you need the root tag too, simply add an outer dummy tag:
String xml = "<hello><test>1.2</test><test2>123</test2></hello>";
XMLSerializer xmlSerializer = new XMLSerializer();
JSON json = xmlSerializer.read("<x>" + xml + "</x>");
There is no direct mapping between XML and JSON; XML carries with it type information (each element has a name) as well as namespacing. Therefore, unless each JSON object has type information embedded, the conversion is going to be lossy.
But that doesn't necessarily matter. What does matter is that the consumer of the JSON knows the data contract. For example, given this XML:
<books>
<book author="Jimbo Jones" title="Bar Baz">
<summary>Foo</summary>
</book>
<book title="Don't Care" author="Fake Person">
<summary>Dummy Data</summary>
</book>
</books>
You could convert it to this:
{
"books": [
{ "author": "Jimbo Jones", "title": "Bar Baz", "summary": "Foo" },
{ "author": "Fake Person", "title": "Don't Care", "summary": "Dummy Data" },
]
}
And the consumer wouldn't need to know that each object in the books collection was a book object.
Edit:
If you have an XML Schema for the XML and are using .NET, you can generate classes from the schema using xsd.exe. Then, you could parse the source XML into objects of these classes, then use a DataContractJsonSerializer to serialize the classes as JSON.
If you don't have a schema, it will be hard getting around manually defining your JSON format yourself.
The XML class in the org.json namespace provides you with this functionality.
You have to call the static toJSONObject method
Converts a well-formed (but not necessarily valid) XML string into a JSONObject. Some information may be lost in this transformation because JSON is a data format and XML is a document format. XML uses elements, attributes, and content text, while JSON uses unordered collections of name/value pairs and arrays of values. JSON does not does not like to distinguish between elements and attributes. Sequences of similar elements are represented as JSONArrays. Content text may be placed in a "content" member. Comments, prologs, DTDs, and <[ [ ]]> are ignored.
If you are dissatisfied with the various implementations, try rolling your own. Here is some code I wrote this afternoon to get you started. It works with net.sf.json and apache common-lang:
static public JSONObject readToJSON(InputStream stream) throws Exception {
SAXParserFactory factory = SAXParserFactory.newInstance();
factory.setNamespaceAware(true);
SAXParser parser = factory.newSAXParser();
SAXJsonParser handler = new SAXJsonParser();
parser.parse(stream, handler);
return handler.getJson();
}
And the SAXJsonParser implementation:
package xml2json;
import net.sf.json.*;
import org.apache.commons.lang.StringUtils;
import org.xml.sax.*;
import org.xml.sax.helpers.DefaultHandler;
import java.util.ArrayList;
import java.util.List;
public class SAXJsonParser extends DefaultHandler {
static final String TEXTKEY = "_text";
JSONObject result;
List<JSONObject> stack;
public SAXJsonParser(){}
public JSONObject getJson(){return result;}
public String attributeName(String name){return "#"+name;}
public void startDocument () throws SAXException {
stack = new ArrayList<JSONObject>();
stack.add(0,new JSONObject());
}
public void endDocument () throws SAXException {result = stack.remove(0);}
public void startElement (String uri, String localName,String qName, Attributes attributes) throws SAXException {
JSONObject work = new JSONObject();
for (int ix=0;ix<attributes.getLength();ix++)
work.put( attributeName( attributes.getLocalName(ix) ), attributes.getValue(ix) );
stack.add(0,work);
}
public void endElement (String uri, String localName, String qName) throws SAXException {
JSONObject pop = stack.remove(0); // examine stack
Object stashable = pop;
if (pop.containsKey(TEXTKEY)) {
String value = pop.getString(TEXTKEY).trim();
if (pop.keySet().size()==1) stashable = value; // single value
else if (StringUtils.isBlank(value)) pop.remove(TEXTKEY);
}
JSONObject parent = stack.get(0);
if (!parent.containsKey(localName)) { // add new object
parent.put( localName, stashable );
}
else { // aggregate into arrays
Object work = parent.get(localName);
if (work instanceof JSONArray) {
((JSONArray)work).add(stashable);
}
else {
parent.put(localName,new JSONArray());
parent.getJSONArray(localName).add(work);
parent.getJSONArray(localName).add(stashable);
}
}
}
public void characters (char ch[], int start, int length) throws SAXException {
JSONObject work = stack.get(0); // aggregate characters
String value = (work.containsKey(TEXTKEY) ? work.getString(TEXTKEY) : "" );
work.put(TEXTKEY, value+new String(ch,start,length) );
}
public void warning (SAXParseException e) throws SAXException {
System.out.println("warning e=" + e.getMessage());
}
public void error (SAXParseException e) throws SAXException {
System.err.println("error e=" + e.getMessage());
}
public void fatalError (SAXParseException e) throws SAXException {
System.err.println("fatalError e=" + e.getMessage());
throw e;
}
}
Converting complete docx files into JSON does not look like a good idea, because docx is a document centric XML format and JSON is a data centric format. XML in general is designed to be both, document and data centric. Though it is technical possible to convert document centric XML into JSON, handling the generated data might be overly complex. Try to focus on the actual needed data and convert only that part.
If you need to be able to manipulate your XML before it gets converted to JSON, or want fine-grained control of your representation, go with XStream. It's really easy to convert between: xml-to-object, json-to-object, object-to-xml, and object-to-json. Here's an example from XStream's docs:
XML
<person>
<firstname>Joe</firstname>
<lastname>Walnes</lastname>
<phone>
<code>123</code>
<number>1234-456</number>
</phone>
<fax>
<code>123</code>
<number>9999-999</number>
</fax>
</person>
POJO (DTO)
public class Person {
private String firstname;
private String lastname;
private PhoneNumber phone;
private PhoneNumber fax;
// ... constructors and methods
}
Convert from XML to POJO:
String xml = "<person>...</person>";
XStream xstream = new XStream();
Person person = (Person)xstream.fromXML(xml);
And then from POJO to JSON:
XStream xstream = new XStream(new JettisonMappedXmlDriver());
String json = xstream.toXML(person);
Note: although the method reads toXML() XStream will produce JSON, since the Jettison driver is used.
If you have a valid dtd file for the xml snippet, then you can easily convert xml to json and json to xml using the open source eclipse link jar. Detailed sample JAVA project can be found here: http://www.cubicrace.com/2015/06/How-to-convert-XML-to-JSON-format.html
I have come across a tutorial, hope it helps you.
http://www.techrecite.com/xml-to-json-data-parser-converter
Use
xmlSerializer.setForceTopLevelObject(true)
to include root element in resulting JSON.
Your code would be like this
String xml = "<hello><test>1.2</test><test2>123</test2></hello>";
XMLSerializer xmlSerializer = new XMLSerializer();
xmlSerializer.setForceTopLevelObject(true);
JSON json = xmlSerializer.read(xml);
Docx4j
I've used docx4j before, and it's worth taking a look at.
unXml
You could also check out my open source unXml-library that is available on Maven Central.
It is lightweight, and has a simple syntax to pick out XPaths from your xml, and get them returned as Json attributes in a Jackson ObjectNode.
I am using jaxb for my application configurations
I feel like I am doing something really crooked and I am looking for a way to not need an actual file or this transaction.
As you can see in code I:
1.create a schema into a file from my JaxbContext (from my class annotation actually)
2.set this schema file in order to allow true validation when I unmarshal
JAXBContext context = JAXBContext.newInstance(clazz);
Schema mySchema = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI).newSchema(schemaFile);
jaxbContext.generateSchema(new MySchemaOutputResolver()); // ultimately creates schemaFile
Unmarshaller u = m_context.createUnmarshaller();
u.setSchema(mySchema);
u.unmarshal(...);
do any of you know how I can validate jaxb without needing to create a schema file that sits in my computer?
Do I need to create a schema for validation, it looks redundant when I get it by JaxbContect.generateSchema ?
How do you do this?
Regarding ekeren's solution above, it's not a good idea to use PipedOutputStream/PipedInputStream in a single thread, lest you overflow the buffer and cause a deadlock. ByteArrayOutputStream/ByteArrayInputStream works, but if your JAXB classes generate multiple schemas (in different namespaces) you need multiple StreamSources.
I ended up with this:
JAXBContext jc = JAXBContext.newInstance(Something.class);
final List<ByteArrayOutputStream> outs = new ArrayList<ByteArrayOutputStream>();
jc.generateSchema(new SchemaOutputResolver(){
#Override
public Result createOutput(String namespaceUri, String suggestedFileName) throws IOException {
ByteArrayOutputStream out = new ByteArrayOutputStream();
outs.add(out);
StreamResult streamResult = new StreamResult(out);
streamResult.setSystemId("");
return streamResult;
}});
StreamSource[] sources = new StreamSource[outs.size()];
for (int i=0; i<outs.size(); i++) {
ByteArrayOutputStream out = outs.get(i);
// to examine schema: System.out.append(new String(out.toByteArray()));
sources[i] = new StreamSource(new ByteArrayInputStream(out.toByteArray()),"");
}
SchemaFactory sf = SchemaFactory.newInstance( XMLConstants.W3C_XML_SCHEMA_NS_URI );
m.setSchema(sf.newSchema(sources));
m.marshal(docs, new DefaultHandler()); // performs the schema validation
I had the exact issue and found a solution in the Apache Axis 2 source code:
protected List<DOMResult> generateJaxbSchemas(JAXBContext context) throws IOException {
final List<DOMResult> results = new ArrayList<DOMResult>();
context.generateSchema(new SchemaOutputResolver() {
#Override
public Result createOutput(String ns, String file) throws IOException {
DOMResult result = new DOMResult();
result.setSystemId(file);
results.add(result);
return result;
}
});
return results;
}
and after you've acquired your list of DOMResults that represent the schemas, you will need to transform them into DOMSource objects before you can feed them into a schema generator. This second step might look something like this:
Unmarshaller u = myJAXBContext.createUnmarshaller();
List<DOMSource> dsList = new ArrayList<DOMSource>();
for(DOMResult domresult : myDomList){
dsList.add(new DOMSource(domresult.getNode()));
}
String schemaLang = "http://www.w3.org/2001/XMLSchema";
SchemaFactory sFactory = SchemaFactory.newInstance(schemaLang);
Schema schema = sFactory.newSchema((DOMSource[]) dsList.toArray(new DOMSource[0]));
u.setSchema(schema);
I believe you just need to set a ValidationEventHandler on your unmarshaller. Something like this:
public class JAXBValidator extends ValidationEventCollector {
#Override
public boolean handleEvent(ValidationEvent event) {
if (event.getSeverity() == event.ERROR ||
event.getSeverity() == event.FATAL_ERROR)
{
ValidationEventLocator locator = event.getLocator();
// change RuntimeException to something more appropriate
throw new RuntimeException("XML Validation Exception: " +
event.getMessage() + " at row: " + locator.getLineNumber() +
" column: " + locator.getColumnNumber());
}
return true;
}
}
And in your code:
Unmarshaller u = m_context.createUnmarshaller();
u.setEventHandler(new JAXBValidator());
u.unmarshal(...);
If you use maven using jaxb2-maven-plugin can help you. It generates schemas in generate-resources phase.