Unfortunately I couldn't find anything specific to this topic / to my problem. Here we go:
I'm building a JavaFX Business Application for a friend of mine. Unfortunately I do not have any possibility to connect to a Database. I want the Application to load a savestate from a file. The application contains a list with clients and the clients got some specific properties. I do not want to hardcode this to a .prop or .txt file, because I'm sure that there's a different way of doing this, isn't there?
Thanks in advance, appreciate it!
Lots of choices for persisting data to local storage. The exact choice depends on your needs. You do not describe enough details to make a specific recommendation.
Here is a list of possibilities, roughly in increasing order of complexity of your data.
Text file
If you have small amounts of simple data, save to a text file. You can store each piece in a separate file, or combine into a single file. Recent versions of Java have new classes to make this easier than ever. See Oracle Tutorial.
Comma-separate & Tab-delimited
For sets of structured data, write to text files in comma-separated values (CSV) or tab-delimited values. For example a list of people with rows for each person, and columns for name, phone number, and email address.
While reading/writing such files is easy enough to program yourself, I suggest using an established library to eliminate the drudgery, avoid bugs, and save yourself some time. There are a few such libraries written in Java.
My favorite is the Apache Commons CSV project. This library makes easy work of the chore of reading/writing such files. Despite the name, this library supports tab-delimited as well as comma-separated formats. I've written a few Answers here on Stack Overflow showing how to use this library, as you can see here, here, and here.
By the way, plain old ASCII defines a few character positions explicitly for delimiting in data files, with four levels of grouping (document, group, record/row, and field). Unicode, of course, inherits these from ASCII as code points. I am puzzled why these have remained so obscure and so infrequently used. Seems much more logical to me than using commas and tabs which may well exist inside the data payload.
Serialization
You can write out the data values stored within an object. This is called serialization. Java has a serialization facility built-in, but be sure to study up on the details.
To more simply write out an object’s values and later read them back in to reconstitute an object, I have enjoyed using the Simple XML Serialization project. This works well for relatively simple needs, and is aimed at the situation where you want the structure of a class to drive the process of determining what to write.
Java has other XML binding facilities both built-in and third-party. These are much more powerful in their flexibility. They are especially good for when you want to define and verify the XML structure in a rigid fashion such as defining a XML DTD or XML Schema against which to validate the data and perhaps even generate the Java class in which to represent the data.
Embedded database
For more complicated data, use an embedded relational database.
The SQLite database is bundled with many platforms. This is a C-based library, not pure Java. As the name indicates, SQLite is indeed quite “lite“, lacking rigid data types and many other common database features. SQLite is meant to be an alternative to writing text files than as a competitor to more serious databases. It is a great product if your needs fit the sweet-spot of its capabilities.
My first choice for an embedded database would be H2 Database Engine. Built in pure Java. Can be run inside your app, or separately as a server (you choice). Has sophisticated relational database features. Has been around for years, often updated, and is well-worn. The principal author has much experience in the field.
Related
I'm designing a new application in JSE which I want to internationalize.
I've never done such an application. I'm looking for the best practices about the internationalization. The application while be writing the translated data in files or DB. I've searched about best practices but I didn't found anything about my main question(the first one).
Should I put all the internationalization data in some layer or next to the object they are about ?
Could I directly use the properties files as a kind of enum to do a switch case ?
Or can I reverse engineer the data catched and know the default internationalize value and work with it?
I did encounter several strategies. I would start with a properties file.
One factor is that the data must be professionally maintained:
keep it in version control.
keep a version number for us humans, "1.0.23"
keep the texts ordered and nice, to help translation.
keep a second properties file with a glossary for consistent translation.
Undermore I did see generating properties or java ListResourceBundles from DocBook XML, Excel, translation memories. And yes, database.
Maintenance of data must be done careful, as several different parties will use the text at different times.
Programming tools, consistency checks and preparing data, communicating are tasks not to neglect.
Properties files are not entirely ideal, but IDEs have generally some support for them.
Set up everything for UTF-8, though take notice that properties files use ISO-8859-1, but you can use \uXXXX escaping or do a encoding conversion in your build process. ListResourceBundle java sources, generated than, would be an alternative.
Summary:
I am trying to write a utility program that is based on the information contained in a separate file. The object has to be such that any information on the physical file can be retrieved quickly and can be updated quickly as well.
Details:
The file is a normal ANSI encoded file that is supposed to store definitions of the physical quantities stated in the SI system. What I really want is that I should be able to read and write changes to the definitions whenever required. I'll be using markers(like ":") to get the headings and definitions like:
Length:metre:m:"..length of path traveled by light in vacuum in
1/299792458th of a second"
and so on.
So in this case is extending RandomAccessFile an option? Will it help me in quick retrieval and syncing of data? Should I use another approach?
If you want these things, then I'd advise you to use an embedded ACID database like H2:
Guarantee that you don't lose changes that you made
Have more than one program access the info
This is because coding up something that correctly does this using low level facilities like RandomAccessFile is quite hard. Storing persistent application state in embedded DBs is commonly done. H2 is probably the most popular among DBs implemented in pure Java.
On how to actually do this, see this: Embedding the Java h2 database programmatically
You prob. want to look at introduction on relational DBs & SQL if you aren't familiar with them.
I need to generate firmware file for embedded device, and I'm wondering what are the best practices of the format of data structures like that.
There should be several items in that file: firmware for MCU, some data parts to be written to the device flash memory.
The obvious way is to implement that format "by hand": some header at the start of the file (header should contain version of data structure), then address table, and then data. That's the way I used in previous projects, and I'm really tired of maintaining this structure when I need to change something.
Is there some existing container to store data collections in the binary file? There would be nice to have library for many platforms (at least, C++ and Java) that can handle this data collection. What comes to mind is SQLite database file with one or several tables, but it seems to be overhead for my needs.
Is there some best practices to achieve what I need?
IMHO, The most common format is ZIP or in Java terms JAR. This can be browsed in Windows Explorer and read on a wide variety of platforms.
I'm working on a system which predicts soccer matches at work. I have several pre-existing databases which each contain broadly the same data, although some vendors provide more data than others. I have a core set of fields that my application uses and which all vendors provide:
homeTeamId, awayTeamId, fullTimeHomeGoals, fullTimeAwayGoals, homeShotsOnTarget, awayShotsOnTarget, etc...
Because these databases have come from different sources the field-names vary. Also some of this data is subjective (the definition of a shot on target varies). This means that I need to know which vendor a match came from. There is also overlap because several vendors will have the data for a particular match.
At the moment we are using one source of data at a time, but we will use two or more vendors at once based on the competition covered by that vendor in future (by selecting based on competition we remove the issue of duplicate matches).
My solution was to use XML to store a mapping of the fieldName. E.g
<Schemas>
<Schema>
<SchemaName>VendorA</SchemaName>
<TableName>VendorA_MatchResults</TableName>
<FullTimeHomeGoals>homeFullTimeScore</FullTimeHomeGoals>
Etc...
</Schema>
</Schemas>
Then whenever I need a field for a sql query, look at the vendor the user has specified in the job configuration XML and lookup the fields relevant to that data vendor. When we come to uses results from two vendors I was planning to use a view and treat this as a new vendor in the XML.
This must be a reasonably common problem but I couldn't find anything online discussing how to tackle it. My gut instinct says the DB should be able to handle this internally instead, perhaps with a view?
I'd be grateful for any advice or ideas.
For background, I'm using MySql and Java to develop this application.
I think what you are doing is good.
You should create a class which stores these configurations for each schema. Store these configs in a Map. Make sure that these entries and their configurations come from a config file, just like you did.
I would recommend Spring here, it makes your life easier. Each time when you want to add a new vendor, you just need to edit this config file, and restart your app.
I am creating a few JAX-WS endpoints, for which I want to save the received and sent messages for later inspection. To do this, I am planning to save the messages (XML files) into filesystem, in some sensible hierarchy. There will be hundreds, even thousands of files per day. I also need to store metadata for each file.
I am considering to put the metadata (just a couple of fields) into database table, but the XML file content itself into files in a filesystem in order not to bloat the database with content data (that is seldomly read).
Is there some simple library that helps me in saving, loading, deleting etc. the files? It's not that tricky to implement it myself, but I wonder if there are existing solutions? Just a simple library that already provides easy access to filesystem (preferrably over different operating systems).
Or do I even need that, should I just go with raw/custom Java?
Is there some simple library that
helps me in saving, loading, deleting
etc. the files? It's not that tricky
to implement it myself, but I wonder
if there are existing solutions? Just
a simple library that already provides
easy access to filesystem (preferrably
over different operating systems).
Java API
Well, if what you need to do is really simple, you should be able to achieve your goal with java.io.File (delete, check existence, read, write, etc.) and a few stream manipulations with FileInputStream and FileOutputStream.
You can also throw in Apache commons-io and its handy FileUtils for a few more utility functions.
Java is independent of the OS. You just need to make sure you use File.pathSeparator, or use the constructor File(File parent, String child) so that you don't need to explicitly mention the separator.
The Java file API is relatively high-level to abstract the differences of the many OS. Most of the time it's sufficient. It has some shortcomings only if you need some relatively OS-specific feature which is not in the API, e.g. check the physical size of a file on the disk (not the the logical size), security rights on *nix, free space/quota of the hard drive, etc.
Most OS have an internal buffer for file writing/reading. Using FileOutputStream.write and FileOutputStream.flush ensure the data have been sent to the OS, but not necessary written on the disk. The Java API support also this low-level integration to manage these buffering issue (example here) for system such as database.
Also both file and directory are abstracted with File and you need to check with isDirectory. This can be confusing, for instance if you have one file x, and one directory /x (I don't remember exactly how to handle this issue, but there is a way).
Web service
The web service can use either xs:base64Binary to pass the data, or use MTOM (Message Transmission Optimization Mechanism) if files are large.
Transactions
Note that the database is transactional and the file system not. So you might have to add a few checks if operations fails and are re-tried.
You could go with a complicated design involving some form of distributed transaction (see this answer), or try to go with a simpler design that provides the level of robustness that you need. A possible design could be:
Update. If the user wants to overwrite a file, you actually create a new one. The level of indirection between the logical file name and the physical file is stored in database. This way you never overwrite a physical file once written, to ensure rollback is consistent.
Create. Same story when user want to create a file
Delete. If the user want to delete a file, you do it only in database first. A periodic job polls the file system to identify files which are not listed in database, and removes them. This two-phase deletes ensures that the delete operation can be rolled back.
This is not as robust as writting BLOB in real transactional database, but provide some robustness. You could otherwise have a look at commons-transaction, but I feel like the project is dead (2007).
There is DataNucleus, a Java persistence provider. It is little too heavy for this case, but it supports JPA and JDO java standards with different datastores (RDBMS, object storage, XML, JSON, Excel, etc.). If the product is already using JPA or JDO, it might be worth considering using NataNucleus, as saving data into different datastores should be transparent. I suppose DataNucleus supports splitting the data into several files, creating the sensible directory/file structure I wanted (in my question), but this is just a guess.
Support for XML and JSON seems to be experimental.