Raised a bounty as the only answer doesn't provide a good implementation for Android. Is there a speedier implementation compatible with Android? Or is SimpleXML the best performance I'll get?
I'm fairly novice to Java and Android development so don't know the proper procedure for deserializing an xml string to an object. I found a method that works in:
public static Object deserializeXMLToObject(String xmlFile,Object objClass) throws Exception
{
try
{
InputStream stream = new ByteArrayInputStream(xmlFile.getBytes("UTF-8"));
Serializer serializer = new Persister();
objClass = serializer.read(objClass, stream);
return objClass;
}
catch (Exception e)
{
return e;
}
}
Where xmlFile is the (misnamed) xml string, and objClass is an empty class of the class I want to deserialize to. This is generally a list of other objects.
Example class:
#Root(name="DepartmentList")
public class DepartmentList {
#ElementList(entry="Department", inline=true)
public List<Department> DepartmentList =new ArrayList<Department>();
public boolean FinishedPopulating = false;
}
Department class:
public class Department {
#Element(name="DeptID")
private String _DeptID ="";
public String DeptID()
{
return _DeptID;
}
public void DeptID(String Value)
{
_DeptID = Value;
}
#Element(name="DeptDescription")
private String _DeptDescription ="";
public String DeptDescription()
{
return _DeptDescription;
}
public void DeptDescription(String Value)
{
_DeptDescription = Value;
}
}
Example XML:
<DepartmentList>
<Department>
<DeptID>525</DeptID>
<DeptDescription>Dept 1</DeptDescription>
</Department>
<Department>
<DeptID>382</DeptID>
<DeptDescription>Dept 2</DeptDescription>
</Department>
</DepartmentList>
This has been working fine throughout the app, but I have come to a point where it needs to deserialise >300 objects in the list. This only takes approximately 5 secs, or close to a minute when debugging, but users are not happy with that performance and time wasted when debugging isn't desirable. Is there any way to speed this up? Or is there another way I should be doing this? Preferably only by changing the deserializeXMLToObject method.
I am sure someone will point to a better library that's out there, but according to one detailed answer, they are all slow on Android.
So here is my quick hack (yes I know its not very maintainable and is brittle to the XML not being formed exactly as specified) and some results:
private void doTest()
{
Thread t = new Thread()
{
public void run()
{
runOne(2000);
runOne(300);
runOne(20000);
}
private void runOne(int num)
{
String start = "<DepartmentList>";
String mid1 = "<Department>\n" +
"<DeptID>";
String mid2 = "</DeptID>\n" +
"<DeptDescription>Dept ";
String mid3 = "</DeptDescription></Department>";
String fin = "</DepartmentList>";
StringBuffer sb = new StringBuffer();
sb.append(start);
for (int i=0; i< num; i++)
{
sb.append(mid1);
sb.append(""+i);
sb.append(mid2);
sb.append(""+i);
sb.append(mid3);
}
sb.append(fin);
Pattern p = Pattern.compile(
"<Department\\s*>\\s*<DeptID\\s*>([^<]*)</DeptID>\\s*<DeptDescription\\s*>([^<]*)</DeptDescription>\\s*</Department>");
long startN = System.currentTimeMillis();
DepartmentList d = new DepartmentList();
List<Department> departments = d.DepartmentList;
Matcher m = p.matcher(sb);
while (m.find())
{
Department department = new Department();
department.DeptID(m.group(1));
department.DeptDescription(m.group(2));
departments.add(department);
}
long endN = System.currentTimeMillis();
Log.d("Departments", "parsed: " + departments.size() + " in " + (endN-startN) + " millis");
Log.d("Departments", "lastone: " + departments.get(departments.size() -1)._DeptID + " desc: " + departments.get(departments.size() -1)._DeptDescription);
}
};
t.start();
}
public class DepartmentList {
public List<Department> DepartmentList =new ArrayList<Department>();
public boolean FinishedPopulating = false;
}
public class Department {
private String _DeptID ="";
public String DeptID()
{
return _DeptID;
}
public void DeptID(String Value)
{
_DeptID = Value;
}
private String _DeptDescription ="";
public String DeptDescription()
{
return _DeptDescription;
}
public void DeptDescription(String Value)
{
_DeptDescription = Value;
}
}
I pasted this into an Android project and called it from the onCreate() method. Here is the results:
Platform num=300 num=2000 num=20000
=================================================
Nexus 7 5 38 355
Galaxy Y 29 430 1173
HTC Desire HD 19 189 539
Galaxy Nexus 14 75 379
All times are in milliseconds.
For my research this is the best way to optimize:
"Simple will dynamically build your object graph, this means that it will need to load classes that have not already been loaded, and build a schema for each class based on its annotations using reflection. So first use will always be by far the most expensive. Repeated use of the same persister instance will be many times faster. So try to avoid multiple persister instances, just use one if possible."
So refactoring your code to use the same Persister should improve you performance.
This and other tips I got from this question. In that case, this refactoring improved the performance, as stated by the author (from 15s to 3-5s).
Hope it helps
You can eliminate the intermediate (de)serialization steps by serializing directly to XML and deserializing directly from XML, using e.g. JAXB or XStream.
You may also be able to speed things up via multithreading. I'll assume that all of the XML strings you want to deserialize are in a ConcurrentLinkedQueue; alternatively, you can synchronize access to whatever non-threadsafe collection you're using. Use something like a ThreadPoolExecutor to minimize thread creation overhead.
public class DeserializeXML implements Runnable {
private final String xml;
private final ConcurrentLinkedQueue<Object> deserializedObjects;
public DeserializeXML(String xml, ConcurrentLinkedQueue deserializedObjects) {
this.xml = xml;
this.deserializedObjects = deserializedObjects;
}
public void run() {
deserializedObjects.offer(deserializeXMLToObject(xml, Object.class));
}
}
// ***
ConcurrentLinkedQueue<String> serializedObjects;
ConcurrentLinkedQueue<Object> deserializedObjects;
ThreadPoolExecutor executor = new ThreadPoolExecutor;
while(!serializedObjects.isEmpty()) {
executor.execute(new DeserializeXML(serializedObjects.poll(), deserializedObjects));
}
Got a similar issue with a SOAP Web Service times ago.
In the end I've changed the format of the XML, transforming the nodes in attributes.
example:
<node>
<attr1>value1</attr1>
<attr2>value2</attr2>
<attr3>value3</attr3>
<attr4>value4</attr4>
</node>
was transformed in
<node attr1='value1'attr2='value2' attr3='value3' attr4='value4' />
Maybe not the best solution theorically wise, but performance improvement was really good. Ofc if your XML is a bit more complex (repeteable nodes at various levels or multi level lists) things may be a bit complex.
Use a server proxy and transform your XML to JSON
Related
I am working on a maven managed Java project. The goal is to read tasks from a JSON file and process them. We also try to keep the whole project modular. Therefore we have a core-project and many modules.
The basic idea is, that the core defines the tasks, how to retrieve them ... and the module is performing the task.
I've read a lot about circular dependencies or Dependency Inversion and how to avoid them in theory. However I am unable to put those examples and methods into practice and I am not quite sure if this is my problem. Therefore I was hoping someone kind point me into the right direction. Here's my issue (simplified):
I read "tasks" from a JSON file and convert them into an array using GSON.
[
{
"name": "Do_SOMETHING_1",
"arguments": "myArgs, myArgs_devided_by_comma",
"network": [
{
"start_ip": "192.168.1.1",
"end_ip": "192.168.1.255"
}
]
},
{
"name": "Do_SOMETHING_2",
"arguments": "myArgs, myArgs_devided_by_comma"
}
]
My Task.class looks like this:
public class Task {
private String name;
private String arguments;
private List<network> network;
public String getType() {
return command.toLowerCase();
}
public String[] getArgs() {
String[] args = arguments.split("\\,");
return args;
}
#Override
public String toString() {
return name + " - " + arguments;
}
}
class network {
private String start_ip;
private String end_ip;
#Override
public String toString() {
return start_ip + " - " + end_ip;
}
}
To handle the Tasks I wrote a Jobs class which checks for jobs, builds the array of tasks and then tries to instantiate a new class to process the task depending on what the JSON file specified. It looks like this:
public class Job {
public static Task[] task;
ExecutorService executor = Executors.newFixedThreadPool(25);
public void run() {
try {
fetch_jobs();
} catch (IOException e) {
//Handle e
}
process_jobs();
executor.shutdown();
while (!executor.isTerminated()) {
//Wait until exit
}
}
public void fetch_jobs() throws UnsupportedEncodingException, IOException {
Reader reader = new InputStreamReader(Job.class.getResourceAsStream("/tasks.json"), "UTF-8");
Gson gson = new GsonBuilder().create();
task = gson.fromJson(reader, Task[].class);
}
private void process_jobs() {
for (int i = 0; i < task.length; i++) {
switch(task[i].getType()) {
case "Do_SOMETHING_1":
Runnable worker1 = new Do_SOMETHING_1(task[i]);
executor.execute(worker1);
break;
case "Do_SOMETHING_2":
Runnable worker2 = new Do_SOMETHING_2(task[i]);
executor.execute(worker2);
break;
default:
//Do Nothing
}} }
}
Do_SOMETHING_1 and Do_SOMETHING_2 are maven modules which currently depend on the core so they know what a task is and so I can pass along the task-type-variable (task[i]).
The Problem is that in the core application I need to call Do-SOMETHING_1 and Do_SOMETHING_2. Therefore the core needs to depend on the modules, which is not good I guess. However how could I pass the task along so that Do_SOMETHING_1 can access those information. E.g. Network and arguments... ?
Any help is appreciated since I am really stressed out here.
Hierarchy:
Module(Do_SOMETHING_1) depends on CORE
Module(Do_SOMETHING_2) depends on CORE
CORE depends on Module(Do_SOMETHING_1) <-- WRONG
CORE depends on Module(Do_SOMETHING_2) <-- WRONG
Cheers,
Stephanie.
You shouldn't hard-code the implementation classes into the switch like that; it essentially eliminates all the advantages of polymorphism. Instead, use an Abstract Factory to instantiate the Runnable for the operation.
This is a case where Java's Service Provider Interface is perfect; it essentially allows you to scan for implementations (plugins) of some interface at runtime, and isn't difficult to use. See the javax.crypto API for an example of how this works in practice.
My basic question: is there anything built that already does this automatically (doesn't have to be part of a popular library/package)? The main things I'm working with are Spring (MVC) and Jackson2.
I understand there are a few manual ways to do this:
Create a method in each class that serializes its specific properties into property=value& form (kind of stinks because it's a bunch of logic duplication, I feel).
Create a function that accepts an object, and uses reflection to dynamically read all the properties (I guess the getters), and build the string by getting each. I'm assuming this is how Jackson works for serialization/deserialization in general, but I really don't know.
Use some feature of Jackson to customly serialize the object. I've researched custom serializers, but it seems they are specific to a class (so I'd have to create one for each Class I'm trying to serialize), while I was hoping for a generic way. I'm just having trouble understanding how to apply one universally to objects. A few of the links:
http://techtraits.com/Programming/2011/11/20/using-custom-serializers-with-jackson/
http://wiki.fasterxml.com/JacksonHowToCustomSerializers
Use ObjectMapper.convertValue(object, HashMap.class);, iterate over the HashMap's key/value pairs, and build the string (which is what I'm using now, but I feel the conversions are excessive?).
I'm guessing there's others I'm not thinking of.
The main post I've looked into is Java: Getting the properties of a class to construct a string representation
My point is that I have several classes that I want to be able to serialize without having to specify something specific for each. That's why I'm thinking a function using reflection (#2 above) is the only way to handle this (if I have to do it manually).
If it helps, an example of what I mean is with, say, these two classes:
public class C1 {
private String C1prop1;
private String C1prop2;
private String C1prop3;
// Getters and setters for the 3 properties
}
public class C2 {
private String C2prop1;
private String C2prop2;
private String C2prop3;
// Getters and setters for the 3 properties
}
(no, the properties names and conventions are not what my actual app is using, it's just an example)
The results of serializing would be C1prop1=value&C1prop2=value&C1prop3=value and C2prop1=value&C2prop2=value&C2prop3=value, but there's only one place that defines how the serialization happens (already defined somewhere, or created manually by me).
So my idea is that I will have to end up using a form of the following (taken from the post I linked above):
public String toString() {
StringBuilder sb = new StringBuilder();
try {
Class c = Class.forName(this.getClass().getName());
Method m[] = c.getDeclaredMethods();
Object oo;
for (int i = 0; i < m.length; i++)
if (m[i].getName().startsWith("get")) {
oo = m[i].invoke(this, null);
sb.append(m[i].getName().substring(3) + ":"
+ String.valueOf(oo) + "\n");
}
} catch (Throwable e) {
System.err.println(e);
}
return sb.toString();
}
And modify it to accept an object, and change the format of the items appended to the StringBuilder. That works for me, I don't need help modifying this now.
So again, my main question is if there's something that already handles this (potentially simple) serialization instead of me having to (quickly) modify the function above, even if I have to specify how to deal with each property and value and how to combine each?
If it helps, the background of this is that I'm using a RestTemplate (Spring) to make a GET request to a different server, and I want to pass a specific object's properties/values in the URL. I understand I can use something like:
restTemplate.getForObject("URL?C1prop1={C1Prop1}&...", String.class, C1Object);
I believe the properties will be automatically mapped. But like I said, I don't want to have to make a different URL template and method for each object type. I'm hoping to have something like the following:
public String getRequest(String url, Object obj) {
String serializedUri = SERIALIZE_URI(obj);
String response = restTemplate.getForObject("URL?" + serializedUri, String.class);
return response;
}
where SERIALIZE_URI is where I'd handle it. And I could call it like getRequest("whatever", C1Object); and getRequest("whateverElse", C2Object);.
I think, solution number 4 is OK. It is simple to understand and clear.
I propose similar solution in which we can use #JsonAnySetter annotation. Please, see below example:
import com.fasterxml.jackson.annotation.JsonAnySetter;
import com.fasterxml.jackson.databind.ObjectMapper;
public class JacksonProgram {
public static void main(String[] args) throws Exception {
C1 c1 = new C1();
c1.setProp1("a");
c1.setProp3("c");
User user = new User();
user.setName("Tom");
user.setSurname("Irg");
ObjectMapper mapper = new ObjectMapper();
System.out.println(mapper.convertValue(c1, UriFormat.class));
System.out.println(mapper.convertValue(user, UriFormat.class));
}
}
class UriFormat {
private StringBuilder builder = new StringBuilder();
#JsonAnySetter
public void addToUri(String name, Object property) {
if (builder.length() > 0) {
builder.append("&");
}
builder.append(name).append("=").append(property);
}
#Override
public String toString() {
return builder.toString();
}
}
Above program prints:
prop1=a&prop2=null&prop3=c
name=Tom&surname=Irg
And your getRequest method could look like this:
public String getRequest(String url, Object obj) {
String serializedUri = mapper.convertValue(obj, UriFormat.class).toString();
String response = restTemplate.getForObject(url + "?" + serializedUri, String.class);
return response;
}
Lets we have c1.
c1.setC1prop1("C1prop1");
c1.setC1prop2("C1prop2");
c1.setC1prop3("C1prop3");
Converts c1 into URI
UriComponentsBuilder.fromHttpUrl("http://test.com")
.queryParams(new ObjectMapper().convertValue(c1, LinkedMultiValueMap.class))
.build()
.toUri());
After we will have
http://test.com?c1prop1=C1prop1&c1prop2=C1prop2&c1prop3=C1prop3
I'm trying to implement the Externalizable interface to store the data using the LWUIT-IO's storage. This worked great for simple objects that are composed of Strings, booleans and ints.
However, I have an object that is composed of these types, but also of a Vector of the above mentioned Externalizable object. This seem to mess up the process and I get nothing when I try to retrieve the object from storage.
I assumed it was like the Serializable interface and that the Externalizable objects inside the main object are automatically handled. I'm not sure if this is true, or why it's failing.
The object inside the object is:
public class Song implements Externalizable{
String name = "examplesongname";
public void externalize(DataOutputStream out) throws IOException {
out.writeUTF(name);
}
public void internalize(int version, DataInputStream in) throws IOException {
name = in.readUTF();
}
public String getObjectId() {
return "pat.objects.Song";
}
public int getVersion() {
return 1;
}
}
The containing object is as follows:
public class Playlist implements Externalizable{
String name = "exampleplaylistname";
Vector songs = new Vector();
public void externalize(DataOutputStream out) throws IOException {
out.writeUTF(name);
out.write(songs.size());
Enumeration allItems = songs.elements();
while(allItems.hasMoreElements()){
Externalizable nextItem = (Externalizable) allItems.nextElement();
nextItem.externalize(out);
}
}
public void internalize(int version, DataInputStream in) throws IOException {
name = in.readUTF();
int size = in.readInt();
songs= new Vector();
for(int currentIndex = 0; currentIndex < size; currentIndex++){
Object nextItem = new Object();
((Externalizable)nextItem).internalize(version, in);
songs.addElement(nextItem);
}
}
}
public String getObjectId() {
return "pat.objects.Playlist";
}
public int getVersion() {
return 1;
}
}
What am I doing wrong or missing that is making the Playlist (containing object) fail to be stored while if I try to store the first one by itself it works?
Please note that the overriding methods are different that normal Java since this is the LWUIT version of Externalizable interface.
You need to use Util.register(...) to register these classes as externalizable when your app starts up.
Also the call directly to externalize isn't correct. You should use Util.writeObject/readObject to write another externalizable object (with its own version number). You can then avoid the loop over the vector which would be redundant and just write the whole vector.
I would also suggest using Util.readUTF/writeUTF which support null strings as well.
As a sidenote, I'd suggest migrating to Codename one since LWUIT is no longer maintained by anyone. Also Steve Hannah has a nice writeup on externalization is Codename One which is pretty similar to LWUIT (although it now supports Lists and Maps): http://www.shannah.ca/blog/?p=234
Which ORM supports a domain model of immutable types?
I would like to write classes like the following (or the Scala equivalent):
class A {
private final C c; //not mutable
A(B b) {
//init c
}
A doSomething(B b) {
// build a new A
}
}
The ORM has to initialized the object with the constructor. So it is possible to check invariants in the constructor. Default constructor and field/setter access to intialize is not sufficient and complicates the class' implementation.
Working with collections should be supported. If a collection is changed it should create a copy from the user perspective. (Rendering the old collection state stale. But user code can still work on (or at least read) it.) Much like the persistent data structures work.
Some words about the motivation. Suppose you have a FP-style domain object model. Now you want to persist this to a database. Who do you do that? You want to do as much as you can in a pure functional style until the evil sides effect come in. If your domain object model is not immutable you can for example not share the objects between threads. You have to copy, cache or use locks. So unless your ORM supports immutable types your constrainted in your choice of solution.
UPDATE: I created a project focused on solving this problem called JIRM:
https://github.com/agentgt/jirm
I just found this question after implementing my own using Spring JDBC and Jackson Object Mapper. Basically I just needed some bare minimum SQL <-> immutable object mapping.
In short I just use Springs RowMapper and Jackson's ObjectMapper to map Objects back and forth from the database. I use JPA annotations just for metadata (like column name etc...). If people are interested I will clean it up and put it on github (right now its only in my startup's private repo).
Here is a rough idea how it works here is an example bean (notice how all the fields are final):
//skip imports for brevity
public class TestBean {
#Id
private final String stringProp;
private final long longProp;
#Column(name="timets")
private final Calendar timeTS;
#JsonCreator
public TestBean(
#JsonProperty("stringProp") String stringProp,
#JsonProperty("longProp") long longProp,
#JsonProperty("timeTS") Calendar timeTS ) {
super();
this.stringProp = stringProp;
this.longProp = longProp;
this.timeTS = timeTS;
}
public String getStringProp() {
return stringProp;
}
public long getLongProp() {
return longProp;
}
public Calendar getTimeTS() {
return timeTS;
}
}
Here what the RowMapper looks like (notice it mainly delegats to Springs ColumnMapRowMapper and then uses Jackson's objectmapper):
public class SqlObjectRowMapper<T> implements RowMapper<T> {
private final SqlObjectDefinition<T> definition;
private final ColumnMapRowMapper mapRowMapper;
private final ObjectMapper objectMapper;
public SqlObjectRowMapper(SqlObjectDefinition<T> definition, ObjectMapper objectMapper) {
super();
this.definition = definition;
this.mapRowMapper = new SqlObjectMapRowMapper(definition);
this.objectMapper = objectMapper;
}
public SqlObjectRowMapper(Class<T> k) {
this(SqlObjectDefinition.fromClass(k), new ObjectMapper());
}
#Override
public T mapRow(ResultSet rs, int rowNum) throws SQLException {
Map<String, Object> m = mapRowMapper.mapRow(rs, rowNum);
return objectMapper.convertValue(m, definition.getObjectType());
}
}
Now I just took Spring JDBCTemplate and gave it a fluent wrapper. Here are some examples:
#Before
public void setUp() throws Exception {
dao = new SqlObjectDao<TestBean>(new JdbcTemplate(ds), TestBean.class);
}
#Test
public void testAll() throws Exception {
TestBean t = new TestBean(IdUtils.generateRandomUUIDString(), 2L, Calendar.getInstance());
dao.insert(t);
List<TestBean> list = dao.queryForListByFilter("stringProp", "hello");
List<TestBean> otherList = dao.select().where("stringProp", "hello").forList();
assertEquals(list, otherList);
long count = dao.select().forCount();
assertTrue(count > 0);
TestBean newT = new TestBean(t.getStringProp(), 50, Calendar.getInstance());
dao.update(newT);
TestBean reloaded = dao.reload(newT);
assertTrue(reloaded != newT);
assertTrue(reloaded.getStringProp().equals(newT.getStringProp()));
assertNotNull(list);
}
#Test
public void testAdding() throws Exception {
//This will do a UPDATE test_bean SET longProp = longProp + 100
int i = dao.update().add("longProp", 100).update();
assertTrue(i > 0);
}
#Test
public void testRowMapper() throws Exception {
List<Crap> craps = dao.query("select string_prop as name from test_bean limit ?", Crap.class, 2);
System.out.println(craps.get(0).getName());
craps = dao.query("select string_prop as name from test_bean limit ?")
.with(2)
.forList(Crap.class);
Crap c = dao.query("select string_prop as name from test_bean limit ?")
.with(1)
.forObject(Crap.class);
Optional<Crap> absent
= dao.query("select string_prop as name from test_bean where string_prop = ? limit ?")
.with("never")
.with(1)
.forOptional(Crap.class);
assertTrue(! absent.isPresent());
}
public static class Crap {
private final String name;
#JsonCreator
public Crap(#JsonProperty ("name") String name) {
super();
this.name = name;
}
public String getName() {
return name;
}
}
Notice in the above how easy it is to map any query into immutable POJO's. That is you don't need it 1-to-1 of entity to table. Also notice the use of Guava's optionals (last query.. scroll down). I really hate how ORM's either throw exceptions or return null.
Let me know if you like it and I'll spend the time putting it on github (only teste with postgresql). Otherwise with the info above you can easily implement your own using Spring JDBC. I'm starting to really dig it because immutable objects are easier to understand and think about.
Hibernate has the #Immutable annotation.
And here is a guide.
Though not a real ORM, MyBatis may able to do this. I didn't try it though.
http://mybatis.org/java.html
AFAIK, there are no ORMs for .NET supporting this feature exactly as you wish. But you can take a look at BLTookit and LINQ to SQL - both provide update-by-comparison semantics and always return new objects on materialization. That's nearly what you need, but I'm not sure about collections there.
Btw, why you need this feature? I'm aware about pure functional languages & benefits of purely imutable objects (e.g. complete thread safety). But in case with ORM all the things you do with such objects are finally transformed to a sequence of SQL commands anyway. So I admit the benefits of using such objects are vaporous here.
You can do this with Ebean and OpenJPA (and I think you can do this with Hibernate but not sure). The ORM (Ebean/OpenJPA) will generate a default constructor (assuming the bean doesn't have one) and actually set the values of the 'final' fields. This sounds a bit odd but final fields are not always strictly final per say.
SORM is a new Scala ORM which does exactly what you want. The code below will explain it better than any words:
// Declare a model:
case class Artist ( name : String, genres : Set[Genre] )
case class Genre ( name : String )
// Initialize SORM, automatically generating schema:
import sorm._
object Db extends Instance (
entities = Set() + Entity[Artist]() + Entity[Genre](),
url = "jdbc:h2:mem:test"
)
// Store values in the db:
val metal = Db.save( Genre("Metal") )
val rock = Db.save( Genre("Rock") )
Db.save( Artist("Metallica", Set() + metal + rock) )
Db.save( Artist("Dire Straits", Set() + rock) )
// Retrieve values from the db:
val metallica = Db.query[Artist].whereEqual("name", "Metallica").fetchOne() // Option[Artist]
val rockArtists = Db.query[Artist].whereEqual("genres.name", "Rock").fetch() // Stream[Artist]
Help! I have an Axis web service that is being consumed by a C# application. Everything works great, except that arrays of long values always come across as [0,0,0,0] - the right length, but the values aren't deserialized. I have tried with other primitives (ints, doubles) and the same thing happens. What do I do? I don't want to change the semantics of my service.
Here's what I ended up with. I have never found another solution out there for this, so if you have something better, by all means, contribute.
First, the long array definition in the wsdl:types area:
<xsd:complexType name="ArrayOf_xsd_long">
<xsd:complexContent mixed="false">
<xsd:restriction base="soapenc:Array">
<xsd:attribute wsdl:arrayType="soapenc:long[]" ref="soapenc:arrayType" />
</xsd:restriction>
</xsd:complexContent>
</xsd:complexType>
Next, we create a SoapExtensionAttribute that will perform the fix. It seems that the problem was that .NET wasn't following the multiref id to the element containing the double value. So, we process the array item, go find the value, and then insert it the value into the element:
[AttributeUsage(AttributeTargets.Method)]
public class LongArrayHelperAttribute : SoapExtensionAttribute
{
private int priority = 0;
public override Type ExtensionType
{
get { return typeof (LongArrayHelper); }
}
public override int Priority
{
get { return priority; }
set { priority = value; }
}
}
public class LongArrayHelper : SoapExtension
{
private static ILog log = LogManager.GetLogger(typeof (LongArrayHelper));
public override object GetInitializer(LogicalMethodInfo methodInfo, SoapExtensionAttribute attribute)
{
return null;
}
public override object GetInitializer(Type serviceType)
{
return null;
}
public override void Initialize(object initializer)
{
}
private Stream originalStream;
private Stream newStream;
public override void ProcessMessage(SoapMessage m)
{
switch (m.Stage)
{
case SoapMessageStage.AfterSerialize:
newStream.Position = 0; //need to reset stream
CopyStream(newStream, originalStream);
break;
case SoapMessageStage.BeforeDeserialize:
XmlWriterSettings settings = new XmlWriterSettings();
settings.Indent = false;
settings.NewLineOnAttributes = false;
settings.NewLineHandling = NewLineHandling.None;
settings.NewLineChars = "";
XmlWriter writer = XmlWriter.Create(newStream, settings);
XmlDocument xmlDocument = new XmlDocument();
xmlDocument.Load(originalStream);
List<XmlElement> longArrayItems = new List<XmlElement>();
Dictionary<string, XmlElement> multiRefs = new Dictionary<string, XmlElement>();
FindImportantNodes(xmlDocument.DocumentElement, longArrayItems, multiRefs);
FixLongArrays(longArrayItems, multiRefs);
xmlDocument.Save(writer);
newStream.Position = 0;
break;
}
}
private static void FindImportantNodes(XmlElement element, List<XmlElement> longArrayItems,
Dictionary<string, XmlElement> multiRefs)
{
string val = element.GetAttribute("soapenc:arrayType");
if (val != null && val.Contains(":long["))
{
longArrayItems.Add(element);
}
if (element.Name == "multiRef")
{
multiRefs[element.GetAttribute("id")] = element;
}
foreach (XmlNode node in element.ChildNodes)
{
XmlElement child = node as XmlElement;
if (child != null)
{
FindImportantNodes(child, longArrayItems, multiRefs);
}
}
}
private static void FixLongArrays(List<XmlElement> longArrayItems, Dictionary<string, XmlElement> multiRefs)
{
foreach (XmlElement element in longArrayItems)
{
foreach (XmlNode node in element.ChildNodes)
{
XmlElement child = node as XmlElement;
if (child != null)
{
string href = child.GetAttribute("href");
if (href == null || href.Length == 0)
{
continue;
}
if (href.StartsWith("#"))
{
href = href.Remove(0, 1);
}
XmlElement multiRef = multiRefs[href];
if (multiRef == null)
{
continue;
}
child.RemoveAttribute("href");
child.InnerXml = multiRef.InnerXml;
if (log.IsDebugEnabled)
{
log.Debug("Replaced multiRef id '" + href + "' with value: " + multiRef.InnerXml);
}
}
}
}
}
public override Stream ChainStream(Stream s)
{
originalStream = s;
newStream = new MemoryStream();
return newStream;
}
private static void CopyStream(Stream from, Stream to)
{
TextReader reader = new StreamReader(from);
TextWriter writer = new StreamWriter(to);
writer.WriteLine(reader.ReadToEnd());
writer.Flush();
}
}
Finally, we tag all methods in the Reference.cs file that will be deserializing a long array with our attribute:
[SoapRpcMethod("", RequestNamespace="http://some.service.provider",
ResponseNamespace="http://some.service.provider")]
[return : SoapElement("getFooReturn")]
[LongArrayHelper]
public Foo getFoo()
{
object[] results = Invoke("getFoo", new object[0]);
return ((Foo) (results[0]));
}
This fix is long-specific, but it could probably be generalized to handle any primitive type having this problem.
Here's a more or less copy-pasted version of a blog post I wrote on the subject.
Executive summary: You can either change the way .NET deserializes the result set (see Chris's solution above), or you can reconfigure Axis to serialize its results in a way that's compatible with the .NET SOAP implementation.
If you go the latter route, here's how:
... the generated
classes look and appear to function
normally, but if you'll look at the
deserialized array on the client
(.NET/WCF) side you'll find that the
array has been deserialized
incorrectly, and all values in the
array are 0. You'll have to manually
look at the SOAP response returned by
Axis to figure out what's wrong;
here's a sample response (again,
edited for clarity):
<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv=http://schemas.xmlsoap.org/soap/envelope/>
<soapenv:Body>
<doSomethingResponse>
<doSomethingReturn>
<doSomethingReturn href="#id0"/>
<doSomethingReturn href="#id1"/>
<doSomethingReturn href="#id2"/>
<doSomethingReturn href="#id3"/>
<doSomethingReturn href="#id4"/>
</doSomethingReturn>
</doSomethingResponse>
<multiRef id="id4">5</multiRef>
<multiRef id="id3">4</multiRef>
<multiRef id="id2">3</multiRef>
<multiRef id="id1">2</multiRef>
<multiRef id="id0">1</multiRef>
</soapenv:Body>
</soapenv:Envelope>
You'll notice that Axis does not
generate values directly in the
returned element, but instead
references external elements for
values. This might make sense when
there are many references to
relatively few discrete values, but
whatever the case this is not properly
handled by the WCF basicHttpBinding
provider (and reportedly by gSOAP and
classic .NET web references as well).
It took me a while to find a solution:
edit your Axis deployment's
server-config.wsdd file and find the
following parameter:
<parameter name="sendMultiRefs" value="true"/>
Change it to false,
then redeploy via the command line,
which looks (under Windows) something
like this:
java -cp %AXISCLASSPATH% org.apache.axis.client.AdminClient server-config.wsdl
The web service's
response should now be deserializable
by your .NET client.
Found this link that may offer a better alternative: http://www.tomergabel.com/GettingWCFAndApacheAxisToBeFriendly.aspx