Java; Runtime Interpretation; Strategies To Add Plugins - java

I'm beginning to start on my first large project. It will be a program very similar to Rosetta Stone. It will be a program, used for learning a foreign language, written in Java using Swing. In my program I plan on the user being able to select downloaded courses to learn from. I will be able to create an English course since I am a native English speaker. However, I want people who speak other languages to be able to write courses for users to use as well (this is an essential part for my program to work).
Since I want the users to be able to download courses of languages they want, having it hard-coded into the program is out of the question. The courses needed to be interpreted during the runtime. Also since I want others to collaborate with my work (ie make courses), I need to make it easy for them to do so.
What would be the best way to go about doing this?
The idea I have come up with is having a strict empty course outline (hard-coded) with a simple xml file which details the text and sounds to be used. The drawback to this is that it extremely limits the author. Different languages may need to start out with learning different parts.
Any advice on the problem at hand as well as the project as a whole will be greatly appreciated. Any links to any relevant resources or information would also be greatly appreciated.
Think you for your time and effort,
Joseph Pond

Simply, you should base your program on a system such as Eclipse RCP, or the Netbeans Platform. Both of these systems already deal with exactly this problem, and both are perfectly adequate for this task. They're not just for IDEs.
It's a larger first step as you will need to learn one of these platforms beyond simply just Swing.
But, they solve the problem, and their overall organization and technique will serve your program well anyway.
Don't reinvent this wheel, just learn one of these instead.

If you are set on doing this from scratch (Will's idea isn't bad), What I would do is first lay down the file format that would be easiest to create your language course in. It could be XML, plaintext or some other format you come up with yourself.
You will probably need some flexibility in the language format because you will want to actually be able to specify things like questions and answers. XML is a pain because of all the extra terminators, but it gives a good amount of meta-data. If you like XML for that, you may consider defining your language file in YML, it gives you the data of XML but uses whitespace delineators instead of angle brackets.
You probably also want to define your file in the language it's created for, so you might or might not want to require english words as keys. If you don't want any english, you may have to skip both XML and YML and come up with your own file format--possibly where the layout and/or special symbols define the flow and "functionality".
Once you have defined the file format, you won't have to worry about hard-coding anything... you won't be able to because it will already be in the file.
Plug-in functionality would be nice as well... This is where your definition file also contains information that tells you what class to instantiate (reflectively) and use to parse/display the data. In that way you could add new types of questions just by delivering a new jar file.
If this is confusing, sorry, this is difficult in a one-way forum because I can't look at your face and see if you're following me or if I'm even going in the right direction. If you think I'm on the right track and want more details (I've done a bit of this stuff before) feel free to leave a follow-up question (or an email address) in a comment and I'd be glad to discuss it with you further.

If I was doing this, I'd seriously consider using Eclipse EMF to model the "language" for defining courses. EMF is rather daunting to start with, but it gives you:
A high-level model that can be entered/edited in a variety of ways.
An automatic mechanism for serializing "instances" (i.e. courses) to XML. (And you can tinker with the serialization if you choose.)
Automatically generated Java classes for in-memory representations of your instances. These provide APIs that are tuned to your model, an generic ones that are the EMF equivalent of Java reflection ... but based on EMF model classes rather than Java classes.
An automatically generated tree editor for your "instances".
Hooks for implementing your own constraints / validation rules to say what is a valid "course".
Related Eclipse plugins offer:
Mappings to text-based languages with generation of parsers/unparsers
Mappings to graphical languages; e.g. notations using boxes / arrows / etc
Various more advanced persistence mechanisms
Comparisons/differencing, model-to-model transformations, constraints in OCL, etc
I've used EMF in a couple of largish projects, and the main point that keeps me coming back for more is ease of model evolution ... compared with building everything at a lower level of abstraction. If my model (language) needs to be extended / changed, I can make the necessary changes using the EMF Model editor, regenerate the code, extend my custom code to do the right stuff with the extensions, and I'm pretty much done (modulo conversion of stored instances).

Related

Niche Templating Engine for Batch Jobs

First of all start off by saying this is more of an exploratory question more than a technical problem. I feel it doesn't belong to Code Review because there's nothing to review. I'm just trying to figure out the best approach to take.
My requirement is to build a batch process that can process user-defined files. These files usually come from external sources, so the filenames are not standard. One requirement that's causing me some headaches is supporting arbitrary dates in the filenames. And since these are batch job definitions that run on particular intervals, the definition has to be flexible enough to support it.
For example, one definition might be
File1_Type1_{CurrentDate in YYMMDD}
File1_Type2_{CurrentDate in YYYYMMDD}
File1_Type3_Static_Text
So basically, I feel like I need a full-fledged template engine in order to support these cases. However, that sounds like huge overkill, so I'm interested to hear people's thoughts on this.
Since I'm focusing on Java/Scala, I've found this library
https://scalate.github.io/scalate/documentation/ssp-reference.html
If we let users create ssp files like so:
#import(java.util.Date)
File1_Type1_${new Date}
then it gives the user full control over the entire formatting. But feels overkill to me? Or not? Welcome any feedback.
There are a huge number in the Java space. I've used Apache velocity and https://freemarker.apache.org/.
I'm not aware of anything in the Scala space, but it would be an intriguing idea.

Drools workbench goals

I'm really new to the drools environnement, and try to understand it.
I've done a first java project which basicaly parse a drl rules files, apply them to a java object and output a result according the object properties and the rules.
Stop me if I'm wrong, but I think it's the ultra-basics of drools.
My second step was to use a decision table (xls) : I compiled it to a drl file and used it. It works.
Now, I'm looking for a tool allowing to manage xls, compiling them live to drl. I was thinking it was the aim of the drools Workbench (old guvnor) so I installed it(yes I succeeded ;-) ).
With the help of http://docs.jboss.org/drools/release/latest/drools-docs/html/wb.Workbench.html#wb.Installation , I created a new project containing my model, my drl, and I deployed it...
->But now, I'm not really clear about the next steps : there is a maven repository, but what does it contain?
->Was I'm thinking right that the workbench could allow me to dynamicaly compile my xls to drl?
->Do I missed a point about the workbench?
(The question invites being closed as its answers will be "primarily opinion-based". But it has been around for 10 hours no without a single vote for close, so I'll just answer it, based on my opinion and my experience. YMMV.)
Regarding maven, it is of course the prime choice as a build tool for Java applications in general, and Drools supports this very well. OTOH, you may write a non-trivial application in, say, Eclipse, compile your rules offline, use simple jar building step for .class and serialized packages or rule base and deploy that. The development process of your institution will have its say, or you take your pick.
The foremost intent of Drools Workbench is to make rule programming viable for, well, hum, non-programmers. I have already unveiled myself as an orthodox believer in the dogma that, except for the mechanic parts, programming remains an "art" (D.E.Knuth), a "discipline" (E.W.Dijkstra) and a "science" (D.Gries). There is nothing wrong with entering the ingredients for a Java Bean via some GUI (provided you have learned the basics about data types including collections), although doing the equivalent in a modern IDE isn't really difficult. Composing rules, conditions and consequences, in a GUI is where I begin to have my doubts. You'll need a full understanding of the underlying principles anyway, and switching between code items in a GUI is for me highly inconvenient in comparison to seeing them (with syntax highlighting and all) all in a modern code editor, with its search and replace and refactoring possibilities.
A last word on the XLS decision table. While many see this as a strong point in favour of Drools, I always add the warning that its unreflected use leads to an abuse of rules as a collection for sets of literals, with uniform logic throughout, something which is much more efficiently represented by a collection of facts and a single rule.
No doubt, others may have to say more in favour of the workbench - not that I'm completely blind to its advantages either ;-)

Automatically generating Java source code

I'm looking for a way to automatically generate source code for new methods within an existing Java source code file, based on the fields defined within the class.
In essence, I'm looking to execute the following steps:
Read and parse SomeClass.java
Iterate through all fields defined in the source code
Add source code method someMethod()
Save SomeClass.java (Ideally, preserving the formatting of the existing code)
What tools and techniques are best suited to accomplish this?
EDIT
I don't want to generate code at runtime; I want to augment existing Java source code
What you want is a Program Transformation system.
Good ones have parsers for the language you care about, build ASTs representing the program for the parsed code, provide you with access to the AST for analaysis and modification, and can regenerate source text from the AST. Your remark about "scanning the fields" is just a kind of traversal of the AST representing the program. For each interesting analysis result you produce, you want to make a change to the AST, perhaps somewhere else, but nonetheless in the AST.
And after all the chagnes are made, you want to regenerate text with comments (as originally entered, or as you have constructed in your new code).
There are several tools that do this specifically for Java.
Jackpot provides a parser, builds ASTs, and lets you code Java procedures to do what you want with the trees. Upside: easy conceptually. Downside: you write a lot more Java code to climb around/hack at trees than you'd expect. Jackpot only works with Java.
Stratego and TXL parse your code, build ASTs, and let you write "surce-to-source" transformations (using the syntax of the target language, e.g., Java in this case) to express patterns and fixes. Additional good news: you can define any programming language you like, as the target language to be processed, and both of these have Java definitions.
But they are weak on analysis: often you need symbol tables, and data flow analysis, to really make analyses and changes you need. And they insist that everything is a rewrite rule, whether that helps you or not; this is a little like insisting you only need a hammer in toolbox; after all, everything can be treated like a nail, right?
Our DMS Software Reengineering Toolkit allows the definition of an abitrary target language (and has many predefined langauges including Java), includes all the source-to-source transformation capabilities of Stratego, TXL, the procedural capability of Jackpot,
and additionally provides symbol tables, control and data flow analysis information. The compiler guys taught us these things were necessary to build strong compilers (= "analysis + optimizations + refinement") and it is true of code generation systems too, for exactly the same reasons. Using this approach you can generate code and optimize it to the extent you have the knowledge to do so. One example, similar to your serialization ideas, is to generate fast XML readers and writers for specified XML DTDs; we've done that with DMS for Java and COBOL.
DMS has been used to read/modify/write many kinds of source files. A nice example that will make the ideas clear can be found in this technical paper, which shows how to modify code to insert instrumentation probes: Branch Coverage Made Easy.
A simpler, but more complete example of defining an arbitrary lanauges and transformations to apply to it can be found at How to transform Algebra using the same ideas.
Have a look at Java Emitter Templates. They allow you to create java source files by using a mark up language. It is similar to how you can use a scripting language to spit out HTML except you spit out compilable source code. The syntax for JET is very similar to JSP and so isn't too tricky to pick up. However this may be an overkill for what you're trying to accomplish. Here are some resources if you decide to go down that path:
http://www.eclipse.org/articles/Article-JET/jet_tutorial1.html
http://www.ibm.com/developerworks/library/os-ecemf2
http://www.vogella.de/articles/EclipseJET/article.html
Modifying the same java source file with auto-generated code is maintenance nightmare. Consider generating a new class that extends you current class and adds the desired method. Use reflection to read from user-defined class and create velocity templates for the auto-generating classes. Then for each user-defined class generate its extending class. Integrate the code generation phase in your build lifecycle.
Or you may use 'bytecode enhancement' techniques to enhance the classes without having to modify the source code.
Updates:
mixing auto-generated code always pose a risk of someone modifying it in future to just to tweak a small behavior. It's just the matter of next build, when this changes will be lost.
you will have to solely rely on the comments on top of auto-generated source to prevent developers from doing so.
version-controlling - Lets say you update the template of someMethod(), now all of your source file's version will be updated, even if the source updates is auto-generated. you will see redundant history.
You can use cglib to generate code at runtime.
Iterating through the fields and defining someMethod is a pretty vague problem statement, so it's hard to give you a very useful answer, but Eclipse's refactoring support provides some excellent tools. It'll give you constructors which initialize a selected set of the defined members, and it'll also define a toString method for you.
I don't know what other someMethod()'s you'd want to consider, but there's a start for you.
I'd be very wary of injecting generated code into files containing hand-written code. Hand-written code should be checked into revision control, but generated code should not be; the code generation should be done as part of the build process. You'd have to structure your build process so that for each file you make a temporary copy, inject the generated source code into it, and compile the result, without touching the original source file that the developers work on.
Antlr is really a great tool that can be used very easily for transforming Java source code to Java source code.

Why choose an XSL-transformation?

For a current project the decision has to be made whether to use XML and an XSL-transformation to produce HTML or to directly use HTML-templates.
I'd be interested in arguments for or against the XSL-approach. I understand that in cases where you have to support many different layouts, an XSL-solution has a lot of advantages, but why would you choose it in those cases where you only have to support one target layout?
Edit: We're talking about Java here.
XSLT is a functional programming language and you can use it to create frontends as rich as any templating system. However, you shouldn't — you and your team will go insane.
Both options present the opportunity of transforming objects into a presentation form in a logical sort of way. XSLT is best suited for creating more XML, which might lead you to believe that it's a perfect candidate to use to create XHTML. However, creating XHTML shouldn't be the primary goal — Creating a user experience is. Don't concern yourself with the medium.
Two significant drawbacks to XSLT concern the syntax: Your templates, and the templates that they include, and the templates that those templates include will all be gigantic and verbose. Second, you'll have to do a lot of functional programming, and less-experienced engineers may be confused and terrified when they encounter a recursive template with an accumulating function parameter instead of a simple for loop.
If you're attracted by the beauty of transforming logically-constructed, valid XML entities, consider instead a type-safe templating system that transforms beans instead. Check out Google XML Pages, and create logically-organized, type-safe templates that will be easy for future engineers to pick up and extend.
I created an XML/XSLT-driven UI for an enterprise product about 5 years ago. We're still using it, and I can now look back on my experience and see many pros and cons:
Pros:
XSL is a powerful declarative language, useful & fun for experienced developers, and transforms can do pretty amazing things in a few lines of code
XSL is designed for use with XML, so if your data is already XML then it makes a lot of sense
Separation of concerns (rendering vs. data) is better than many template languages
XSL-based rendering can be easily "subclassed". By that I mean: let's say you have data class A with associated template A.xslt. For class B derived from A, you can easily create B.xslt with only the small differences, and include A.xslt for inherited behaviors. This makes it less succeptible to breaking due to changes in A.xslt.
The above point also gives you the power to do overrides. For class A with associated A.xslt, we can easily switch the associated template to A-custom.xslt, which is a few small changes plus inheritance of A.xslt. We can do this on the fly in the field and again, the benefit is that A-custom.xslt is only a few lines, not an entire modified copy of the original A.xslt. The small footprint means it's more likely to work with multiple versions of A.xslt.
In .NET 2.0, XSLT is compiled and becomes very fast. There may be similar tech for Java. (Most template languages do this now too.)
In .NET, it's possible to create an "Object XPath Navigator", which lets you transform your data objects without having to convert them to an XML object. Again there may be similar tech in Java
XSLT is smart about HTML & handles escaping, white space issues, etc. well
Cons:
XSL is a powerful declarative language, confusing to newer programmers - and fewer people know XSLT well
XSL is verbose. XML is often verbose too.
XSL transforms are probably slower than "native" templates. Even when compiled there's still more state overhead to XSL than most template languages
It's hard to pass parameters to XSLs, you have to either send them in line with your data (forcing you to create extra XML) or via system-specific methods (which may also involve constructing XML data)
If you don't have an ObjectXPathNavigator or equivalent, you'll incur significant overhead when turning your data objects into XML for transformation
Depending on the capabilities of your transformer, you may also incur buffering overhead as you transform into a string buffer and then send that string to the output device
The more advanced your XSLT usage, the less likely it is that your tools will support you (specifically as you start to use includes or faster ways to pass XML data in)
I'll try to update as I think of more issues. I think that looking back now, my verdict would be to stick with a common template language. What were once big issues when I selected XML/XSLT have now been addressed by newer and more mature revisions of the major template engines. We do still benefit greatly from the ability to inherit .xslt files, which is something most template engines don't do well. But in the end the value of having lots of developers providing examples is far greater (compare ASP.NET answers vs XSLT answers on StackOverflow, for instance.)
Hope that helps!
I've done significant development using XSLT and it has been both tremendously successful and a complete failure at two different sites.
A few thoughts before a conclusion:
I don't think anyone would argue that XSLT is far more powerful than a template parsing engine, it's a functional language.
Although it's not as widely adopted as most procedural languages, it's still a real language that's being used out there for actual projects, people can be hired already with knowledge of XSLT and it's a transferable skill for your current staff.
XSLT has also been around for a while now, the implementations are mature, I'm sure this is the case for long running templating engines (like Velocity) but newer engines may be less robust.
Whatever template language you decide on it's unlikely to be as well documented as XSLT. Check out any of the Michael Kay Programmer's reference series for an example on how to do a great reference book.
Tool support is generally very good ... if you have a budget. XMLSpy and Stylus Studio have both been very useful for me in the past.
XSLT is not only hard but, more importantly, different. Most people are not Computer Science graduates formally trained in functional programming. The majority of programmers will write XSLT in a procedural style which will not harness any power of the language and give you a maintenance headache.
XSLT transforms can be slow and can take a lot of memory. You may have problems if you have a stylesheet with a large XML input.
I love XSLT but whether you should use it or not comes down to a few points:
Are you committed to XSLT? Do you have serious in-house expertise in XSLT? Are you prepared to get some?
Is your data in XML? Does it make sense in XML? Do you have someone in-house who loves your data enough to make sure it's well structured and there's always an appropriate schema?
Unless the answer to those questions is yes and you have complex data that requires a complex rendering process, I wouldn't consider using XSLT ... especially if there's no experience in the team. Bad XSLT is much, much, much worse than a bad template.
However, it can render complex data in a maintainable fashion which would be impossible using many of today's templating engines.
Going the XSL way will future-proof your application. Meaning, if you decide in the future to add more templates with different layouts you will be able capitalize on those advantages. In my current project we save off the XML used (in an XMLType or CLOB) and allow other applications to access the data and XSL templates to generate documents via a web service. This was an after thought of the original design that was super easy to implement due to our decision to use XML/XSL.
XSLT has the advantage of being able to also produce output in other document types (i.e. pdf) and pdf output is very likely nowadays. XML/XSLT does also separate data from the view.
When we have done XSLT in the past, it was to allow the ability to extend our product. The output remained the same, only the presentation layer needed to change. This allowed us a lot of flexibility when we had clients that wanted to "customize" their UI, since all we needed to do was replace the XSLT file. If you foresee needing to make a lot of those kinds of changes, XSLT might be your answer.
However, as stated above, the XSLT syntax and functional programming mentality can make it difficult to effectively produce templates. We found that we liked to stick to the tricks that we learned and when we had client requests that fell outside of what we already knew, no one wanted to volunteer for the ticket. Usually someone eventually figured out how to do the task and our "bag of tricks" got larger, but it was often very cumbersome to figure out new things.
If you don't foresee change the UI ever, or at least not much, XSLT may not be worth the extra effort.
Please don't use XML/XSLT for web front-ends. I was in projects like this and it's horrible. Often you have to first produce the XML from objects or something similar, which doesn't make sense. A second point is, that there are so many good HTML editors out there for free, but I've found none for XSLT. So editing complex XSLT is no fun. I would recommend to go with HTML templates and a common template engine.
Depending on your application, having an XML layer that is then transformed to XHTML via XSLT also meens, that you can write easy WebServices to the XML layer - allowing your customers to consume your sites data...
Having the XML sent to the browser with a transformation link (forgot the exact syntax...) also meens less bandwith needed, as the XSLT file will stay the same and you only need to pass the raw XML it is built from - sort of like using an external CSS style sheet instead of adding the style attributes to your markup ;)
I think you need to examine what the source of your data will be. As mentioned by boris callens earlier, if the you are pulling from a database you will have to transform first to XML, then apply your transformations. Should the data source be RSS or the like, then XSLT is a natural choice.
XPATH and XSLT has a high learning curve and functional programming can be daunting to get your arms around. In time crunch this may not be the right choice.
For front end work JSON has a lighter payload, and is readily supported by jQuery and other Javascript libraries. You may want to consider JSON as the data protocol as the jQuery library is far more accessible to developers and the time to productivity with the framework is far less than with XSLT, embedded Javascript in tags, awful syntax and all the other minutia that come with XML/XPATH/XSLT on the front end.
Keep it simple. That's a principle that one gets to appreciate more and more.
Velocity or Freemarker are incredibly flexible and versatile. Your code base will be clear, easily understandable, and it will run much (much) faster than the X monstrosities.
http://fishbowl.pastiche.org/2002/02/12/xslt_is_the_spawn_of_satan/
I see how the XSL approach can be handy if your data is already XML.
But usually it isn't. It's somewhere in a database, needs to be generated on the spot or comes from some service.
Creating XML from this source to then be able to create HTML from that XML is useless in my opinion. I would stick with (X)Html templates.
In contrast to HTML, there are a lot of XML tools available if you need to do parsing and processing of the templates in any way. So you should choose XML to get the benefits of using tools and libraries for XML.
However, that said, it may just be that XHTML fits your needs, since this gives you full support of XML tools and libraries while still being normal HTML which is correctly processed by modern web browsers. If you need to do post-processing of those later on, you can still apply XSLT to the XHTML data.
I've used XML & XSLT in a previous project, financial web sites, and it worked well for us, but:
We had multiple customers, which
varied the number of outputs we had.
We could replace the XSLT stylesheet
and this made changes to the site
easier to manage for the developers
We had a specialist web editor on the team. We gave them example XML & they could edit the stylesheets directly
If there were ever any wording changes that needed to go onto the website yesterday ( it was a bank, this happened surprisingly often), we could just deploy the new XSLT without redeploying the entire site.
Multiple different output formats were needed. We used FOP for transformation to PDF, which is based upon the same sort of technology, so wasn't too hard for us to understand :-)
The main reason I see for using XSLT is if you have multiple sites all based upon the same XML, but requiring different HTML output.
XML + XSLT are really cool. You have the ability to output many types of target formats in the future.
But ne aware of embedded HTML in the XML. Firefox XSLT doesn't support "disable-output-escaping". See Bugzilla.
We use XSLT to generate html in our content management system and it works just fine.
Some hints: Don't try to generate all the page at once from one big hairy XML, you'll go insane. Use the HTML template (plain text/html file with styles, decorations and basic markup) with embedded markers (like, <!--MENU-->, <!--CONTENT-->), and replace markers with xslt-transformation of appropriate data.
Having said that, I doubt you really need xslt if you only going to have one layout, forever.

Application configuration files [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
OK, so I don't want to start a holy-war here, but we're in the process of trying to consolidate the way we handle our application configuration files and we're struggling to make a decision on the best approach to take. At the moment, every application we distribute is using it's own ad-hoc configuration files, whether it's property files (ini style), XML or JSON (internal use only at the moment!).
Most of our code is Java at the moment, so we've been looking at Apache Commons Config, but we've found it to be quite verbose. We've also looked at XMLBeans, but it seems like a lot of faffing around. I also feel as though I'm being pushed towards XML as a format, but my clients and colleagues are apprehensive about trying something else. I can understand it from the client's perspective, everybody's heard of XML, but at the end of the day, shouldn't be using the right tool for the job?
What formats and libraries are people using in production systems these days, is anyone else trying to avoid the angle bracket tax?
Edit: really needs to be a cross platform solution: Linux, Windows, Solaris etc. and the choice of library used to interface with configuration files is just as important as the choice of format.
YAML, for the simple reason that it makes for very readable configuration files compared to XML.
XML:
<user id="babooey" on="cpu1">
<firstname>Bob</firstname>
<lastname>Abooey</lastname>
<department>adv</department>
<cell>555-1212</cell>
<address password="xxxx">ahunter#example1.com</address>
<address password="xxxx">babooey#example2.com</address>
</user>
YAML:
babooey:
computer : cpu1
firstname: Bob
lastname: Abooey
cell: 555-1212
addresses:
- address: babooey#example1.com
password: xxxx
- address: babooey#example2.com
password: xxxx
The examples were taken from this page: http://www.kuro5hin.org/story/2004/10/29/14225/062
First: This is a really big debate issue, not a quick Q+A.
My favourite right now is to simply include Lua, because
I can permit things like width=height*(1+1/3)
I can make custom functions available
I can forbid anything else. (impossible in, for instance, Python (including pickles.))
I'll probably want a scripting language somewhere else in the project anyway.
Another option, if there's a lot of data is to use sqlite3, because they're right to claim
Small.
Fast.
Reliable.
Choose any three.
To which I would like to add:
backups are a snap. (just copy the db file.)
easier to switch to another db, ODBC, whatever. (than it is from fugly-file)
But again, this is a bigger issue. A "big" answer to this probably involves some kind of feature matrix or list of situations like:
Amount of data, or short runtime
For large amounts of data, you might want efficient storage, like a db.
For short runs (often), you might want something that you don't need to do a lot of parsing for, consider something that can be mmap:ed in directly.
What does the configuration relate to?
Host:
I like YAML in /etc. Is that reimplemented in windows?
User:
Do you permit users to edit config with text editor?
Should it be centrally manageable? Registry / gconf / remote db?
May the user have several different profiles?
Project:
File(s) in project directory? (Version control usually follows this model...)
Complexity
Are there only a few flat values? Consider YAML.
Is the data nested, or dependent in some way? (This is where it gets interesting.)
Might it be a desirable feature to permit some form of scripting?
Templates can be viewed as a kind of configuration files..
XML XML XML XML. We're talking config files here. There is no "angle bracket tax" if you're not serializing objects in a performance-intense situation.
Config files must be human readable and human understandable, in addition to machine readable. XML is a good compromise between the two.
If your shop has people that are afraid of that new-fangled XML technology, I feel bad for you.
Without starting a new holy war, the sentiments of the 'angle bracket tax' post is one area where I majorly disagree with Jeff. There's nothing wrong with XML, it's reasonably human readable (as much as YAML or JSON or INI files are) but remember its intent is to be read by machines. Most language/framework combos come with an XML parser of some sort for free which makes XML a pretty good choice.
Also, if you're using a good IDE like Visual Studio, and if the XML comes with a schema, you can give the schema to VS and magically you get intellisense (you can get one for NHibernate for example).
Ulimately you need to think about how often you're going to be touching these files once in production, probably not that often.
This still says it all for me about XML and why it's still a valid choice for config files (from Tim Bray):
"If you want to provide general-purpose data that the receiver might want to do unforeseen weird and crazy things with, or if you want to be really paranoid and picky about i18n, or if what you’re sending is more like a document than a struct, or if the order of the data matters, or if the data is potentially long-lived (as in, more than seconds) XML is the way to go.
It also seems to me that the combination of XML and XPath hits a sweet spot for data formats that need to be extensible; that is to say, it’s pretty easy to write XML-processing code that won’t fail in the presence of changes to the message format that don’t touch the piece you care about."
#Guy
But application config isn't always just key/value pairs. Look at something like the tomcat configuration for what ports it listens on. Here's an example:
<Connector port="80" maxHttpHeaderSize="8192"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" redirectPort="8443" acceptCount="100"
connectionTimeout="20000" disableUploadTimeout="true" />
<Connector port="8009"
enableLookups="false" redirectPort="8443" protocol="AJP/1.3" />
You can have any number of connectors. Define more in the file and more connectors exist. Don't define any more and no more exist. There's no good way (imho) to do that with plain old key/value pairs.
If your app's config is simple, then something simple like an INI file that's read into a dictionary is probably fine. But for something more complex like server configuration, an INI file would be a huge pain to maintain, and something more structural like XML or YAML would be better. It all depends on the problem set.
We are using ini style config files. We use the Nini library to manage them. Nini makes it very easy to use. Nini was orignally for .NET but it has been ported to other platforms using Mono.
XML, JSON, INI.
They all have their strengths and weaknesses.
In an application context, I feel that the abstraction layer is the important thing.
If you can choose a way to structure the data that is a good middle ground between human readability and how you want to access/abstract the data in code, you're golden.
We mostly use XML where I work, and I cant really believe that a configuration file loaded into a cache as objects when first read or after it has been written to, and then abstracted away from the rest of the program, really is that much of a hit on neither CPU nor disk space.
And it is pretty readable too, as long as you structure the file right.
And all languages on all platforms supports XML through some pretty common libraries.
#Herms
What I really meant was to stick to the recommended way software should store configuration values for any given platform.
What you often get then is also the recommended ways these should/can be modified. Like a configuration menu in a program or a configuration panel in a "system prefs" application (for system services softwares ie). Not letting the end users modify them directly via RegEdit or NotePad...
Why?
The end users (=customers) are used to their platforms
System for backups can better save "safe setups" etc
#ninesided
About " choice of library ", try to link in (static link) any selected library to lower the risk of getting into a version-conflict-war on end users machines.
If your configuration file is write-once, read-only-at-bootup, and your data is a bunch of name value pairs, your best choice is the one your developer can get working first.
If your data is a bit more complicated, with nesting etc, you are probably better off with YAML, XML, or SQLite.
If you need nested data and/or the ability to query the configuration data after bootup, use XML or SQLite. Both have pretty good query languages (XPATH and SQL) for structured/nested data.
If your configuration data is highly normalized (e.g. 5th normal form) you are better off with SQLite because SQL is better for dealing with highly normalized data.
If you are planning to write to the configuration data set during program operation, then you are better off going with SQLite. For example, if you are downloading configuration data from another computer, or if you are basing future program execution decisions on data collected in previous program execution. SQLite implements a very robust data storage engine that is extremely difficult to corrupt when you have power outages or programs that are hung in an inconsistent state due to errors. Corruptible data leads to high field support costs, and SQLite will do much better than any home-grown solution or even popular libraries around XML or YAML.
Check out my page for more information on SQLite.
As far as I know, the Windows registry is no longer the preferred way of storing configuration if you are using .NET - most applications now make use of System.Configuration [1, 2]. Since this is also XML based it seems to be that everything is moving in the direction of using XML for configuration.
If you want to stay cross-platform I would say that using some sort of a text file would be the best route to go. As for the formatting of said file, you might want to take into account if a human is going to be manipulating it or not. XML seems to be a bit more friendly to manual manipulation than INI files due to the visible structure of the file.
As for the angle bracket tax - I don't worry about it too often as the XML libraries take care of abstracting it. The only time it might be a consideration is if you have very little storage space to work with and every byte counts.
[1] System.Configuration Namespace - http://msdn.microsoft.com/en-us/library/system.configuration.aspx
[2] Using Application Configuration Files in .NET - http://www.developer.com/net/net/article.php/3396111
We are using properties files, simply because Java supports them natively. A couple of months ago I saw that SpringSource Application Platform uses JSON to configure their server and it looks very interesting. I compared various configuration notations and came to the conclusion that XML seems to be the best fit at the moment. It has nice tools support and is rather platform independent.
Re: epatel's comment
I think the original question was asking about application configuration that an admin would be doing, not just storing user preferences. The suggestions you gave seem more for user prefs than application config, and aren't usually something that the user would ever deal with directly (the app should provide the configuration options in the UI, and then update the files). I really hope you'd never make the user have to view/edit the Registry. :)
As for the actual question, I'd say XML is probably OK, as plenty of people will be used to using that for configuration. As long as you organize the configuration values in an easy to use manner then the "angle bracket tax" shouldn't be too bad.
Maybe a bit of a tangent here but my opinion is that the config file should be read into a key value dictionary/hash table when the app first starts up and always accessed via this object from then on for speed. Typically the key/value table starts off as string to string but helper functions in the object do things such DateTime GetConfigDate(string key) etc...
I think the only important thing is to choose a format that you prefer and can navigate quickly. XML and JSON are both fine formats for configs and are widely supported--technical implementation isn't at the crux of the issue, methinks. It's 100% about what makes the task of config files easier for you.
I have started using JSON, because I work quite a bit with it as a data transport format, and the serializers make it easy to load into any development framework. I find JSON easier to read than XML, which makes handling multiple services, each using a config file that is modified quite frequently, that much easer for me!
What platform are you working on? I'd recommend trying to use the preferred/common method for it.
MacOSX - plists
Win32 - Registry (or are there a new one here, long since I developed on it)
Linux/Unix - ~/.apprc (name-value perhaps)

Categories