I am pretty far into my first Android application, and I have the sneaking suspicion that I'm "Doing It Wrong". My app talks to a Ruby on Rails server and serializes objects back and forth via XML. Before I knew what was happening, I found myself knee deep in writing my own crappy ORM, a problem which is compounded by the fact that I haven't written any Java since high school.
My conflict here is that I want my client-side (android) app to be capable of serializing via a variety of methods, such as HTTP/XML, to a local database, or out to the local filesystem. I started out with the Strategy pattern, but I feel like my solution is badly lacking.
For one, should I re-implement all of Rails model validation on the client side, because I don't know if I'm always going to be working with Rails on the other side? The even bigger issue is that right now I can only represent flat objects as key-values, as my code can't handle nested objects like a true ORM.
I'm sure Android devs deal with this all the time, so I'm interested to hear what other people do to cope with these issues.
I wouldn't approach your Android application as an extension of a Ruby app - rather a consumer of an API. If you can try to expose your server application as JSON (or other format, but JSON is the most lightweight) and consume these APIs from your Android application you would most likely have less problems as JSON is already in K/V format.
I have not yet written Android objects to SQLite yet, but I have written them as both Parcelable objects and to the SharedPreferences. Both of these strategies are sufficient for small to mid-range apps. For data intensive apps, obviously you will have to take it a step further to SQLite.
There are some great articles for these approaches: Managing State.
It boils down to designing your objects in a way that can be serialized easily. That means no circular references or extremely complex objects. This shouldn't be a very large problem, especially if your data is in JSON format already. You simply need to extend some classes and add functions that return a Parcelable object representation or a string representation so your objects can be saved thus.
I would avoid cloning your server-side objects and validation in Android as it then requires modifying both sources if you make small changes. The server should handle all data and validation and you should simply be requesting, caching, and sending data from Android.
I'd be interested to hear if there any challenges to writing objects to SQLite, but I imagine it's not that much more of step from the details I've outlined above. Hope this helps in some capacity!
Hessian is great for RPC. You don't have to do any serialization yourself. It doesn't use XML, so it's more efficient and more appropriate for a mobile platform.
I haven't done much of persistence storage on Android but I think you need to use SQLiteDatabase and make your own Cusor that De/Serializes your object so that it can be added to the database. A possible solution would be to extend a SQLiteCursor or an AbstractCursor.
Otherwise I don't think there is other solution apart from, possibly, "hardcore" Serializabled (Which I suspect it may be too much for a phone, I may be wrong)
I think you might be going too heavy for a smart phone application. I would look at using RESTful style web services with JSON content.
Looking to your question I got the feeling that maybe you just over-complicating your requirements? Why can't you just use JSON format to represent your objects data in portable way? Then you will be able just to store it either on file system or in database in simple text field. You can leverage android-active-record library for transparent DB persistence (http://code.google.com/p/android-active-record)
Related
I have a client-server system implemented in C#, and the client and server exchange .Net objects via serialization / deserialization and communicating via TCP/IP. This runs on a local network, it is not web-based or Internet-based.
Now I want to include Android clients connected by wifi. Again, this is local network only, not via the Internet and not web-based. The Android programming will be in Java. (I am aware of Mono for Android, but prefer not to get into that now.)
Is there some fairly simple way to implement object to object interchange between Java and .Net objects, provided, of course, that they are compatible?
I've looked a bit at JSON (Jackson on the Java end and Json.Net on the .Net end), and I'm guessing it can probably be done, but only with major efforts on remapping things at each end as soon as the objects become fairly complicated.
Any other suggestions? JSON-based or otherwise?
PS. My question is somewhat related to this one Mapping tool for converting Java's JSON to/from C#, but it never got a suitable answer, perhaps due to insufficient info in the question. Also, I don't care whether I end up using a JSON-based transport or XML or something else.
I would suggest either JSON or XML (which is based on a .xsd file) because these are independent of their respective implementations (instead of something like an ObjectOutputStream in java).
The problem of having this format between the two components (client and server) is that they need to be at the same version. My best practice is to have one underlying definition of the format (i use xml with an xsd file which specifies how the xml has to look like), then use jaxb to generated java classes. That way you can (un)marshal from/to xml in the java part.
I am very sure a similar thing exists in the world of .NET.
JSON is smaller than xml in size, i find xml to be more readable.
SO user "default locale" should get the honor for this, but he/she has only answered via a comment. So just to make it very clear what my choice was I'll answer my own question.
I've decided to go with Google Protocol Buffers, which in my opinion has much better support for moving objects back and forth between Java and .Net than JSON. Because I have a lot of experience with C#, and a lot of existing C#-defined classes, I've selected Marc Gravell's protobuf-net program for the .Net end, and Google's own support for the Android end (no - see edit). This implies that I'm defining the objects in C#, not in .proto files - protobuf-net generates the .proto files from which I then generate the Java code.
Incidentally, as the transport mechanism I'm using a little-known program called naga on the Android end. http://code.google.com/p/naga/ Naga seems to work fine, and is well-documented and has sample programs, and should be better known in my opinion.
EDIT:
OK, I've got it working now to my satisfaction. Here's what I'm using:
Google Protocol buffers as the interchange format: https://developers.google.com/protocol-buffers/
Marc Gravell's protobuf-net at the C# end: http://code.google.com/p/protobuf-net/
A program called called protostuff at the Java end: http://code.google.com/p/protostuff/
(I prefer protostuff to the official Google Java implementation of protocol buffers due to Google's implementation being based on the Java objects being immutable.)
Actually, I'm not using pure protocol buffers as the interchange format - I prefix the data with the name of the (outermost) class being transmitted. This makes the data self-identifying for deserializing at the other end.
You can also try wox (https://github.com/codelion/wox), it is a cross platform serialization library for Java and C# based on XML.
Shelve is an ultra-simple No-SQL persistence layer which allows you to trivially persist a mapping of objects. It's a commonly used package in Python because it allows you to trivially add persistence to any application.
It's simplistic nature means it's somewhat limited - but it's surprisingly useful. You can map any arbitrary hashable key onto any serializable object.
Does something like this exist for Android? I'm writing a very simple app, and I've noticed that I'm spending a lot of time faffing around with table structures, select & insert statements. That's the sort of thing I almost never do in Python since I'd usually have some kind of NoSQL alternative.
I'm not expecting to to work exactly the same way - clearly Python and Java are languages with very different characteristics. I just want something that nearly as simple to use and requires less manual SQL faffing.
One more thing - this is a fairly trivial app. I'd prefer to introduce the bare minimum of additional project dependencies. Preference will be given to solutions which require nothing more than the Android APIs.
You said preference to Java API answers so I probably won't get preference, but Couchbase Mobile is the best Android No-SQL I have ever come across.
http://www.couchbase.org/get/couchbase-mobile-for-android/current
You can use SharedPrefences from DataStore. It's pretty much what you need. You don't need full SQLite power for this.
Dear Stack Overflow Community,
I am a Java programmer in front of a task of building a complex, data-driven, web application (SaaS) and I'm searching for technologies to use. I have already made some decisions and I believe I'm proficient enough to build the appliaction using just the technologies I have decided for (I'm definitely not saying it would be perfect, just that it would be working). However, I'd like to make my task easier and that's why I need your help.
Brief description of the project
Back-end
The application will be heavily data-driven, meaning that everything will be stored in a self-descripting database. This means the database itself will be entirely described with metadata and the application will not know what data it reads and writes. There won't probably be any regular entities (in terms of JPA #Entity) because the application won't know the structure of the data; it will obtain it from the metadata. Only the metadata will have a pre-determined structure. To put it simply, the metadata is the alpha-omega of the application because it will tell the application WHEN and WHAT to display and HOW to display it.
The application will probably utilize stored procedures to perform some low-level tasks on the data, such as automatical auditing, logging and translating to user's language, thus most likely eliminating any possibility to use ORM frameworks because there won't be just simple CRUD operations. Therefore, JDBC seems like my only option (doesn't it?).
Front-end
The UI will be "dumb" in terms that it will not know what data it is displaying (to some extent, of course). It will just know how to display it based on the metadata which it will obtain from the database. All UI controls (like menu items, buttons, etc) will be created based on current application's state and the UI will NOT know what the controls do. This means that clicking a menu item or a button will just send an identifier of associated action to the back-end and the server will decide what to do.
My goals
My main goal is to have the application as lightweight as possible with as least dependencies as possible. Because the application will be very complex, I'd like to avoid any heavy framework(s) because there is a very high probability that I'd need to customize a lot of its functionality.
What I have already decided for
Please object to the following decisions only if you think they're absolutely non-viable for my application, as I have already implemented some core functionality using these technologies:
Servlets on Tomcat, Guice DI, AOP (AspectJ)
I believe all of these technologies are lightweight enough and I don't need to learn J2EE.
GWT with GIN-jection on the front-end
Seems like the best option for me because I'm very familiar with Java and Swing and don't want to write any Javascript, PHP or learn a new language. GIN is a little brother of Guice and I will be using the same syntax and principles on both the client and server.
MSSQL RDBMS
This is actually a requirement from company management as I'd much rather like to go with an open-source solution. Too bad for me..
Maven 2
I think no-one can object to this :)
What I need help with
DB communication
I think that ORM is ruled out (is it?) so I need to use JDBC. Do you think Spring JDBC is lightweight and flexible enough for my use? I would often need to "blindly" read data from database, mapping it to some generic entity (because I won't assume any pre-determined structure), and then send the data using some generic DTO to the client along with the metadata telling it what data it is and how to display it. Or do you know any alternatives? Or should I do this myself?
Client/Server communication
GWT and its GWT-RPC mechanism seems not very suited for sending the generic data I need. Although I'm convinced that it's doable using GWT-RPC, are there any alternatives? But I definitely want to use GWT.
Security
Do you know any security libraries / frameworks that would help me? I'm aware of the existence of Spring-security; do you think it's flexible enough for my use or I'd be better off implementing that myself? Also, is Spring's IoC an integral part of the Spring framework, or would I be able to continue to use Guice?
Anything else that you think might be useful?
I really appreciate any advice and suggestions because I wouldn't dare to try to make such decisions myself. Please ask me if you need more information.
Thank you in advance!
eQui
I think you are over-engineering the solution. Take a look at
http://thedailywtf.com/Articles/Programming-Sucks!-Or-At-Least,-It-Ought-To-.aspx
If everything is driven by the DB, you are going to have immense difficulty making things happen in the UI, and you aren't going to be able to use many of the tools that make UI development easier.
I also suggest you take a look at Spring Roo, if your application is mainly just updating data in a database.
UI framework and implications for client/server communication
You say that any UI action will triger the backend (and potentially the DB). This mean that UI interraction will be somewhat slow anyway, and more than that will require a round trip to the server.
GWT is especially suited to avoid as much as possible round trips to the server and do all UI work on client side. In this model, only information that will transit from client to server is real data, and not UI metadata. GWT will do the job, but you'll be using a somewhat low level tool, needed for advenced optimisation you'll be unable to perform anyway...
Framework like ZK or Vaadin seems more suited to what you want to do. The client side has nice widgets with a rich UI, but you manipulate the UI from the server side. The framework manage client/server communication for you (no need of REST, RPC or javascript). The main limitation of theses framework is scalability, with all theses chatty round trip. But because your requirement impose that chatty behaviour anyway, you could really benefit from the abstraction they provide, are they are at not cost in your case.
I have tried both GWT and Zk to do some proof of concept for my company. We ended choosing GWT, because of it's hability to be embedded nicelly into any existing UI and to fine tune what you do... In particular avoid as much as possible rountrip to server. But ZK is really easier and faster in term of developmeent hours.
The side effect is that would totally solve your client/server communication concern, leting the framework performing it in an optimized way (Zk is able to intelligently regroup several UI event before sending them to server).
DB and ORM
For DB design, i tend to think that using fine granularity things in DB will make it very very slow. If each widget is one or several rows in the database you'll have to perform many lookup to perform the simpliest thing.
Problem is if your UI is just a little complex with a few dozen of elements (a few button, checkboxes, labels and widgets), compositing a screen will require lot of requests to the DB. Rendering just one page might be very slow and scalability would be very very bad.
I know this because i worked on somewhat generic bug tracking system with similar (but simpler) requirements than yours and we had exactly this problem.
So i would try to describe UI in some templating or XML format. Maybe you'll not show this data to the user, providing it with a nice abstraction, but instead of performning many queries for just one screen, you'll save the whole screen as one blob.
A really dumb and basic implementation of this would be to store HTML/CSS/PNG file in your DB and load it as needed, with user being responsible for making theses HTML file by hand. Of course this would be terrible for the user. That's why you need a nice and fancy editor UI editor that would work on an intermediary format of your own. Another dumb implementation would be some sort of wiki templating. This is not what you need, you need more. But you have the idea, I would seek in that direction...
For the maintenance and debugging too, this would be far easier to the whole UI description to a few file, to understand what is really implemented than to read lot of tabuled data in your prefered SQL editor. Users would have they export/import format to easily version, backup or experiment.
Security
I would say by hand... Because you have a generic UI generated by user it seem likely that the security will be generic too and dependant of database content.
Hope it help...
For the backend, i implemented a program which had a similar interaction with the database. the code was database structure oblivious, instead, it read a config file describing the db and could construct complex sql queries based on this information. most of the code is proprietary, but one bit of it got pushed into an open source project called sqlbuilder. may be useful to you on the backend.
I think you're on the right track, with your tool seclection. Your 100% data driven model is going to be hard to maintain. But I understand that's a requirement not an option. Normal source control is going to fail you becuase of the ui application logic all being in the meta-data. You'll need some good test databases and some way to maintain them, such as regularily mysqldump them out and check them in to souce code control to handle all the differences, etc..
You're wise to stay away from various ORM solutions and just use JDBC for this type of app.
Let me give you some warnings about GWT. On the surface it will abstract all the uglyness of html, javascript away and give you clean heirarchy's BUT...
1) If the abstraction fails you how do you easily debug?
2) Do you want any of your site to be visible to Google or other search engines, if yes GWT is not for you
3) Do you want to use any HTML5 technogies or do you want to be stuck in IE 5 compatability mode?
So...
I think you'll be much better off Implementing the UI as simple HTML controls with a small set of jQuery ajax interactions with the server. You can define an input type in your database, your serverlet can generate an input tag and then you have two options you can have some standard event bindings in jquery to tell your server that button1. is clicked, or that select2 has changed, etc.
Your server can send back javascript to change the state of the ui - simply load the javascript inside a div so it runs on the client. or 2) You let the input submit the data to the server and do an old school page refesh and the servlet build the next ui screen based on the database.
Building an interface dynamically in HTML from a database is easy and straight foward compared to doing the same in SWING or Windows Forms. You just have to write out a big text string, been doing that since 1999.
That approach is going to be much more lightweight - simpler to debug, understand and modify in the long run than going with the "GWT automatically compiles to unreadable javascript that doesn't run in my browser for some unknown reason" solution.
I have done some searching but haven't come up with anything on this topic. I was wondering if anyone has ever compared (to some degree) the performance difference between an RPC over a socket and a REST web service. If both do the same thing, which would have a tendency to be the better performer? I've already started building some socket code and would like to know if REST would give better performance before I progress much further. Any input would be really appreciated. Thanks indeed
RMI
Feels like a local API, much like
XMLRPC
Can provide some fairly nice remote
exception data
Java specific means this causes lock
in and limits your options
Has horrible versioning problems
between different versions of clients
Skeleton files must be compiled in
like CORBA, which is not very flexible
REST:
easy to route around firewalls
useful for uploading files as it can
be rather lightweight
very simple if you just want to shove
simple things at something and get
back an integer (like for uploaders)
easy to proxy security behind Apache
and let it take the heat
does not define any standard format
for the way the data is being
exchanged (could be JSON, YAML 1.0,
YAML 2.0, arbitrary XML format, etc)
does not define any convention about
having remote faults sent back to the
caller, integer codes are frequently
used, but method of sending back data
is not defined. Ideally this would be
standardized.
may require a lot of work on the
client side caller of the library to
make use of data (custom serialization
and so forth)
In short from here
web services do allow a loosely
coupled architecture. With RMI, you
have to make sure that the objects
stay in sync in all applications
RMI works best for smaller
applications, that are not
internet-related and thus not scalable
Its hard to imagine that REST is faster than a simple socket connection given it also goes over a Socket.
However REST may be performant enough, standard and easier to use. I would test whether REST is fast enough and meets your requirements first (or one of the many other existing solutions) before attempting your own Socket solution.
I'm coming from a web-development background and I am wondering how I would make a learning algorithm in Java/C++. Not so much the algorithm part, but making the program "remember" what it learned from the previous day. I would think something like saving a file, but I suspect their might be an easier way. Apologies if this question is just over the top stupid. Thanks.
I think that would depend a bit on the problem domain. You might want to store learned "facts" or "relationships" in a DB so that they can be easily searched. If you are training a neural network, then you'd probably just dump the network state to a file. In general, I think once you have a mechanism that does the learning, the appropriate storage representation will be relatively apparent.
Maybe if you can flesh out your plan on what kind of learning you'd like to implement, people can provide more guidance on what the implementation should look like, including the state storage.
Not stupid, but a little ill-formed maybe.
What you're going to do as your program "learns" something is update the state of some data structure. If you want to retain that state, you need to persist the data structure to some external store. That means translating the data structure to some external formal that you can read back in without loss.
Java provides a straightforward way to do this via the Serializable interface; you Serialize the data by sending Serializable ojects out through an ObjectStream; the same ObjectStream will reload them later.
If you want to access and save large amounts of data maybe a database would work well. This would allow you to structure the data and look it up in an easier manner. I'm not too well versed on the subject but I think for remembering and recalling things a database would be vastly superior to a file system.
A robust/flexible solution in Java (C++ too, but I wouldn't know how) would be using a database. Java 1.6 comes with the Apache derby database which can be embedded in your apps. Using JPA (Java Persistence API) makes it easy to interface with any database you can find drivers for.
You should look into Neural Network software development. Here's a collection of nice Neural Network libraries for different languages. I am not sure if this is the easy way but once accomplished would be very handy.