I have XML file where some symbols were incorrectly encoded because of mixing UTF-16 and UTF-8.
For example, š symbol is encoded as ļæ½ļæ½ (š) instead of š (š).
I want to unmarshal this XML file but it fails when Unmarshaller meet these incorrect symbols. If I decode only them with StringEscapeUtils#unescapeHtml4 (or StringEscapeUtils#unescapeXml) everything works correctly.
But I don't want to read XML to string then decode it and then unmarshal.
How I could do the same inside the unmarshalling process (not to read XML file to string before)?
I created a simple test to reproduce this:
public class XmlReaderTest {
private static final Pattern HTML_UNICODE_REGEX = Pattern.compile("&#[a-zA-Z0-9]+;&#[a-zA-Z0-9]+;");
#Test
public void test() throws Exception {
final Unmarshaller unmarshaller = JAXBContext.newInstance(Value.class).createUnmarshaller();
final XMLInputFactory factory = createXmlInputFactory();
String xml = "<?xml version=\"1.0\" encoding=\"UTF-8\"?><value><name>š & š OĢ</name></value>";
XMLEventReader xmlReader = factory.createXMLEventReader(new StringReader(decodeHtmlEntities(xml)));
Value result = (Value)unmarshaller.unmarshal(xmlReader);
assert result.name.equals("\uD83D\uDCDE & \uD83D\uDCDE OĢ");
XMLEventReader xmlReader2 = factory.createXMLEventReader(new StringReader(xml));
Value result2 = (Value)unmarshaller.unmarshal(xmlReader2); // ! exception
assert result2.name.equals("\uD83D\uDCDE & \uD83D\uDCDE OĢ");
}
#XmlRootElement(name = "value")
private static class Value {
#XmlElement
public String name;
}
private String decodeHtmlEntities(String readerString) {
StringBuffer unescapedString = new StringBuffer();
Matcher regexMatcher = HTML_UNICODE_REGEX.matcher(readerString);
while (regexMatcher.find()) {
regexMatcher.appendReplacement(unescapedString, StringEscapeUtils.unescapeHtml4(regexMatcher.group()));
}
regexMatcher.appendTail(unescapedString);
return unescapedString.toString();
}
private XMLInputFactory createXmlInputFactory() {
XMLInputFactory factory = XMLInputFactory.newFactory();
factory.setProperty(XMLInputFactory.IS_SUPPORTING_EXTERNAL_ENTITIES, false);
factory.setProperty(XMLInputFactory.SUPPORT_DTD, false);
return factory;
}
}
I have azure functions with http trigger who get data from external source.
inside the functions I created RSS file with these external data.
I want update this file whenever new data is received (in this state it deletes previous data).
in addition I want to know how I can get the url of this RSS file for used it in logic app.
any idea will be approciated.
the Function class look like this:
public class Function {
private HashMap<String, String> crntNewItemList = new HashMap<>();
#FunctionName("HttpAddFeedItem")
public HttpResponseMessage run(
#HttpTrigger(name = "req", methods = {HttpMethod.GET, HttpMethod.POST}, authLevel = AuthorizationLevel.ANONYMOUS) HttpRequestMessage<Optional<String>> request,
final ExecutionContext context) {
context.getLogger().info("Java HTTP trigger processed a request.");
// Parse query parameter
String body = request.getBody().get(); // Get request body
if (body == null) {
return request.createResponseBuilder(HttpStatus.BAD_REQUEST).body("Please pass a name on the query string or in the request body").build();
} else {
String contentType = request.getHeaders().get("content-type");
initNewsItemList(contentType, body);
String copyright = "Copyright hold by XXX";
String title = "Eclipse and Java Information";
String description = "Eclipse and Java Information";
String language = "en";
String link = "httXX://XXw.XXX.com/";
Calendar cal = new GregorianCalendar();
Date creationDate = cal.getTime();
SimpleDateFormat date_format = new SimpleDateFormat(
"EEE', 'dd' 'MMM' 'yyyy' 'HH:mm:ss' 'Z", Locale.US);
String pubdate = date_format.format(creationDate);
Feed rssFeeder = new Feed(title, link, description, language,
copyright, pubdate);
// now add one example entry
FeedMessage feed = new FeedMessage();
feed.setTitle(crntNewItemList.get("title"));
feed.setDescription(crntNewItemList.get("description"));
feed.setAuthor("nonsense#somewhere.de (XXX)");
feed.setGuid("htXXs://XXw.XXX.com/tutorials/RSSFeed/article.html");
feed.setLink(crntNewItemList.get("link"));
rssFeeder.getMessages().add(feed);
// now write the file
RSSFeedWriter writer = new RSSFeedWriter(rssFeeder, "articles.rss");
try {
writer.write();
} catch (Exception e) {
e.printStackTrace();
}
return request.createResponseBuilder(HttpStatus.OK).body("success").build();
}
}
****Update****
the RSSFeedWriter class is:
public class RSSFeedWriter {
private String outputFile;
private Feed rssfeed;
public RSSFeedWriter(Feed rssfeed, String outputFile) {
this.rssfeed = rssfeed;
this.outputFile = outputFile;
}
public void write() throws Exception {
// create a XMLOutputFactory
XMLOutputFactory outputFactory = XMLOutputFactory.newInstance();
// create XMLEventWriter
XMLEventWriter eventWriter = outputFactory
.createXMLEventWriter(new FileOutputStream(outputFile));
// create a EventFactory
XMLEventFactory eventFactory = XMLEventFactory.newInstance();
XMLEvent end = eventFactory.createDTD("\n");
// create and write Start Tag
StartDocument startDocument = eventFactory.createStartDocument();
eventWriter.add(startDocument);
// create open tag
eventWriter.add(end);
StartElement rssStart = eventFactory.createStartElement("", "", "rss");
eventWriter.add(rssStart);
eventWriter.add(eventFactory.createAttribute("version", "2.0"));
eventWriter.add(end);
eventWriter.add(eventFactory.createStartElement("", "", "channel"));
eventWriter.add(end);
// Write the different nodes
createNode(eventWriter, "title", rssfeed.getTitle());
createNode(eventWriter, "link", rssfeed.getLink());
createNode(eventWriter, "description", rssfeed.getDescription());
createNode(eventWriter, "language", rssfeed.getLanguage());
createNode(eventWriter, "copyright", rssfeed.getCopyright());
createNode(eventWriter, "pubdate", rssfeed.getPubDate());
for (FeedMessage entry : rssfeed.getMessages()) {
eventWriter.add(eventFactory.createStartElement("", "", "item"));
eventWriter.add(end);
createNode(eventWriter, "title", entry.getTitle());
createNode(eventWriter, "description", entry.getDescription());
createNode(eventWriter, "link", entry.getLink());
createNode(eventWriter, "author", entry.getAuthor());
createNode(eventWriter, "guid", entry.getGuid());
eventWriter.add(end);
eventWriter.add(eventFactory.createEndElement("", "", "item"));
eventWriter.add(end);
}
eventWriter.add(end);
eventWriter.add(eventFactory.createEndElement("", "", "channel"));
eventWriter.add(end);
eventWriter.add(eventFactory.createEndElement("", "", "rss"));
eventWriter.add(end);
eventWriter.add(eventFactory.createEndDocument());
eventWriter.close();
}
private void createNode(XMLEventWriter eventWriter, String name,
String value) throws XMLStreamException {
XMLEventFactory eventFactory = XMLEventFactory.newInstance();
XMLEvent end = eventFactory.createDTD("\n");
XMLEvent tab = eventFactory.createDTD("\t");
// create Start node
StartElement sElement = eventFactory.createStartElement("", "", name);
eventWriter.add(tab);
eventWriter.add(sElement);
// create Content
Characters characters = eventFactory.createCharacters(value);
eventWriter.add(characters);
// create End node
EndElement eElement = eventFactory.createEndElement("", "", name);
eventWriter.add(eElement);
eventWriter.add(end);
}
}
Regarding the issue, I suggest you create the rss file and upload the file to Azure blob. In the Azure logic app, we can use the Azure Blob Storage Connector to process the rss file.
For more details, please refer to the following steps
Install Azure blob storage SDK
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>azure-storage</artifactId>
<version>8.4.0</version>
</dependency>
Update RSSFeedWriter class
public class RSSFeedWriter {
private String outputFile; // the file name
private Feed rssfeed;
private String connectionString; // the storage account connection string
public RSSFeedWriter(Feed rssfeed, String outputFile) {
this.rssfeed = rssfeed;
this.outputFile = outputFile;
}
public RSSFeedWriter(Feed rssfeed, String outputFile,String connectionString) {
this.rssfeed = rssfeed;
this.outputFile = outputFile;
this.connectionString=connectionString;
}
public void write() throws Exception {
CloudStorageAccount storageAccount = CloudStorageAccount.parse(connectionString);
CloudBlobClient blobClient = storageAccount.createCloudBlobClient();
CloudBlobContainer container = blobClient.getContainerReference("test");
CloudBlockBlob blob =container.getBlockBlobReference("test.rss");
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
// create a XMLOutputFactory
XMLOutputFactory outputFactory = XMLOutputFactory.newInstance();
// create XMLEventWriter
XMLEventWriter eventWriter = outputFactory
.createXMLEventWriter(outputStream);
// create rss
...
// upload rss to Azure blob
blob.upload(new ByteArrayInputStream(outputStream.toByteArray()),outputStream.toByteArray().length);
outputStream.close();
}
private void createNode(XMLEventWriter eventWriter, String name,
String value) throws XMLStreamException {
XMLEventFactory eventFactory = XMLEventFactory.newInstance();
XMLEvent end = eventFactory.createDTD("\n");
XMLEvent tab = eventFactory.createDTD("\t");
// create Start node
StartElement sElement = eventFactory.createStartElement("", "", name);
eventWriter.add(tab);
eventWriter.add(sElement);
// create Content
Characters characters = eventFactory.createCharacters(value);
eventWriter.add(characters);
// create End node
EndElement eElement = eventFactory.createEndElement("", "", name);
eventWriter.add(eElement);
eventWriter.add(end);
}
}
update Function code
#FunctionName("HttpAddFeedItem")
public HttpResponseMessage run(
#HttpTrigger(name = "req", methods = {HttpMethod.GET, HttpMethod.POST}, authLevel = AuthorizationLevel.ANONYMOUS) HttpRequestMessage<Optional<String>> request,
final ExecutionContext context) {
context.getLogger().info("Java HTTP trigger processed a request.");
...
// get the storage account connection string you store in app settings or local.settings.json with the key name
String connectionString = System.getenv("AzureWebJobsStorage");
String outputFile=""; // the file name
RSSFeedWriter writer = new RSSFeedWriter(rssFeeder, outputFile,connectionString);
try {
writer.write();
} catch (Exception e) {
e.printStackTrace();
}
return request.createResponseBuilder(HttpStatus.OK).body("success").build();
}
}
update
If you want to directly access the rss file via the blob URL in the browser, please refer to the following steps
Please set the container public access level as Public read access for blobs only. For more details, please refer to the document
Please ensure the blob's content type is right. The rss file's content type can be application/rss+xml. Regarding how to change the content type, please refer to the following code
CloudStorageAccount storageAccount = CloudStorageAccount.parse(Constr)
CloudBlobClient blobClient = storageAccount.createCloudBlobClient();
CloudBlobContainer container = blobClient.getContainerReference("test");
if(!container.exists()){
container.create();
}
CloudBlockBlob blob =container.getBlockBlobReference("test.rss");
blob.getProperties().setContentType("application/rss+xml");
blob.uploadProperties();
After that, we can access the file in the browser
Apache CXF/JAXB is not unmarshaling Japanese characters. If we are printing the xml using System.out.println output is coming properly like below.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<employee id="1470">
<designation>Eng</designation>
<name>ććć„</name>
<salary>20000.0</salary>
</employee>
If we are passing the same XML to CXF layer it is converting like below.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<employee id="1470">
<designation>Eng</designation>
<name>???</name>
<salary>20000.0</salary>
</employee>
How to solve this issue. Thanks in advance.
Can you try use PrintWriter on top of StringWriter?
StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw);
Can you try with XMLEventWriter to System.out?
XMLEventFactory events = XMLEventFactory.newInstance();
QName bar = new QName("urn:bar", "bar");
XMLOutputFactory factory = XMLOutputFactory.newInstance();
factory.setProperty(XMLOutputFactory.IS_REPAIRING_NAMESPACES, true);
XMLEventWriter writer = factory.createXMLEventWriter(System.out);
JAXBContext pContext = JAXBContext.newInstance(target);
Marshaller marshaller = pContext.createMarshaller();
marshaller.setProperty(Marshaller.JAXB_ENCODING, "UTF-8");
marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
marshaller.marshal(pObject, writer);
writer.add(events.createStartDocument());
writer.setDefaultNamespace("urn:bar");
writer.add(events.createStartElement(bar, null, null));
writer.add(events.createEndDocument());
writer.flush();
My program JAVA:
public static String toXml() {
KtpMessage ktpMessage =new KtpMessage();
ktpMessage.setdetails("test");
XStream xstream = new XStream(new StaxDriver());
String objectXml = xstream.toXML(ktpMessage);
return objectXml;
The result is :
<?xml version='1.0' encoding='utf-8'?><myclasses.Message><details>test</details></myclasses.Message>
my problem:
I want to generate the "objectXml" but without <?xml version='1.0' encoding='utf-8'?>
how can I do that ?
i want have this result:
<myclasses.Message><details>test</details></myclasses.Message>
thanks for your help
If you create your own StaxWriter you can use the constructor that tells it not to write the startDocument StAX event (which is what creates the XML declaration). Something like this (exception handling omitted):
StaxDriver drv = new StaxDriver();
XStream xstream = new XStream(drv);
StringWriter strWriter = new StringWriter();
StaxWriter sw = new StaxWriter(drv.getQnameMap(),
drv.getOutputFactory().createXMLStreamWriter(strWriter),
false, // don't do startDocument
true); // do repair namespaces
xstream.marshal(ktpMessage, sw);
sw.close();
String objectXml = strWriter.toString();
I am working on a android(2.2) project which needs xsl transformation. The below code works perfectly in a regular non-android java project
public static String transform() throws TransformerException {
Source xmlInput = new StreamSource(new File("samplexml.xml"));
Source xslInput = new StreamSource(new File("samplexslt.xslt"));
TransformerFactory factory = TransformerFactory.newInstance();
Transformer transformer = factory.newTransformer(xslInput);
OutputStream baos = new ByteArrayOutputStream();
Result result = new StreamResult(baos);
transformer.transform(xmlInput, result);
return baos.toString();
}
I need similar functionality on android. For this I created 2 files under resources/raw:
samplexml.xml
samplexslt.xslt
(contents of these files come from here.
I tried the below code & it does not work (note the StreamSource constructor arg):
public static String transform() throws TransformerException {
TransformerFactory factory = TransformerFactory.newInstance();
Source xmlInput = new StreamSource(this.getResources().openRawResource(R.raw.samplexml));
Source xslInput = new StreamSource(this.getResources().openRawResource(R.raw.samplexslt));
Transformer transformer = factory.newTransformer(xslInput);//NullPointerException here
OutputStream baos = new ByteArrayOutputStream();
Result result = new StreamResult(baos);
transformer.transform(xmlInput, result);
}
I saw the spec & believe I need to set a systemId. But I couldn't get the above code to work.
So, in an android project, how to handle xslt transformations? Please provide your thoughts.
As we know that we Cannot usethisin a static context and you are doing this in your static method transform(). You can do it like this_
public class YourLoadXSLClass extends Activity {
static Resources res;
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
res = getResources();
String strHTML = transform();
// Other code.....
}
/*
* Your method that Transform CSLT.
*/
public static String transform() throws TransformerException {
TransformerFactory factory = TransformerFactory.newInstance();
// Now your raw files are accessible here.
Source xmlInput = new StreamSource(
LoadXSLTinWebview.res.openRawResource(R.raw.samplexml));
Source xslInput = new StreamSource(
LoadXSLTinWebview.res.openRawResource(R.raw.samplexslt));
Transformer transformer = factory.newTransformer(xslInput);
OutputStream baos = new ByteArrayOutputStream();
Result result = new StreamResult(baos);
transformer.transform(xmlInput, result);
return baos.toString();
}
}
Here is the complete class code that do the needful. I hope this will help you & all!
I've never done anything with XSLT but, looking at your code, logically there are only two things that could cause an NPE on that line. The first would be that factory might be null but that doesn't make sense.
That leaves xslInput as being the culprit which suggests openRawResource(R.raw.samplexslt) is failing to return a valid InputStream for the StreamSource constructor to use. Try putting a log statement in such as...
if (xslInput != null {
Transformer transformer = factory.newTransformer(xslInput);
...
}
else
Log.d("SomeTAG", "xslInput is null!!!");
If it turns out that xslInput is actually null then it suggests openRawResource(...) can't find/process the .xslt file properly. In that case I'd suggest using AssetManagerto open the .xslt file by name...
AssetManager am = this.getAssets();
Source xslInput = new StreamSource(am.open("samplexslt.xslt"));