I have been tasked with an assignement at work, that needs persistance of client side settings. These settings are represented by a XML-file in the C# version of the framework I am implementing, and it represents objects with an hierarchical structure of up to six levels. Also, the number of objects in the different levels are sometimes arbitrary, like an ArrayList. Object serialization is unfortunately not an option, due to a requirement of being able to edit the file directly. It has been more than implied that I have to implement the persistance of these settings as a .properties-file, but I am having some concerns about if that is a good idea.
As far as I know, a .properties-file can only be used in the same way as a HashMap, with a key-value pairing of only Strings. The representation of complex objects seems only possible with long, complex key strings, representing the whole "path" of every single element, including it's individual index in possibly several lists.
filters.filter3.traces.rule4.add1.value="8"
I have not been able to find any decent ways to persist hierarchical objects in a .properties-file, but I have not been able to find any definite proof that it is not possible, either.
So, is a .properties-file suitable for storing hierarchical settings-objects? Or should I implement my properties as an XML-file or JSON instead?
You have a perfect example in the form of log4j: here is property sample from here.
log4j.appender.R = org.apache.log4j.DailyRollingFileAppender
log4j.appender.R.File = logs/bensApps.log
log4j.appender.R.Append = true
log4j.appender.R.DatePattern = '.'yyy-MM-dd
log4j.appender.R.layout = org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern = %d{yyyy-MM-dd HH:mm:ss} %c{1} [%p] %m%n
If you have a hierarchy in those keys, like:
Map<String,Object> root = new HashMap<>();
root.put("level1", "foobar");
root.put("level2", Collections.singletonMap("p", "foobar");
It is up to you to translate it into:
level1=foobar
level2.p=foobar
And when reading the file, to split . into submap:
Map<String, Object> root = new HashMap<>();
Properties propz = new Properties();
// load from your file
for (Map.Entry<String,Object> entry : propz.entrySet()) {
String[] path = entry.getKey().split('\\.');
Map<String,Object> parent = root;
int n = path.length - 1;
for (int i = 0; i < n; ++i) {
String p = path[i];
Object child = parent.get(p);
if (null == child) {
Map<String,Object> _p = new HashMap<>();
parent.put(p, _p);
parent = _p;
} else if (child instanceof Map) {
parent = (Map<String,Objext>) child;
} else {
// it is up to you to do something when the path extends beyond a "final" key
}
}
parent.put(path[n], entry.getValue());
}
However, this is a "reinvent the wheel" pattern, the other answers point you probably better solution than doing it by yourself. This example also show you one kind of problem:
p1=foobar
p1.p2=foobar
In the case of a machine generated properties, this won't happen, and an exception is perhaps the best answer. But in the case of human manipulated properties, this may have some meaning.
On a side note, since Java 7 come with JAXB, you can also do it in full XML, without requiring additional libraries. And using a hierarchy of some sort.
I would stick with an XML file. Nothing irritates users more than having different formats with different variants of an API framework.
You can use libraries such as Apache XMLBeans to simplify the process of reading and writing to XML files, using schema files to auto-generate appropriate Java classes.
Properties files can be used to store hierarchical data, but it's a real pain to edit that stuff by hand and it's easy to introduce errors. Some would argue that XML editing isn't a breeze either, but at least there exists editors that can make this easier when linked with an XSD.
I have always found this way of storing hierarchical configuration data quite cumbersome, but it is very well possible. For example, it is (used to be?) the most common way of configuring Log4j, but from my experience, more and more developers are switching to XML here as well.
I really fail to see the point of using properties files for this purpose.
I think you've already got the gist of it - Java properties files can express hierarchies, but awkwardly.
Take a look at the Typesafe Config library, which is what Play Framework uses. Its primary language is HOCON (human-optimized config notation) which is great for expressing hierarchical properties. The library also plays well with both properties files and JSON, so you can easily support any and all of these languages simultaneously.
In a Java properties file, as you've said, you're restricted to only using dotted keys
a.b.c = 42
In HOCON, you can choose to either do that or to nest sections with curly braces. So that example could be expressed in any of the following ways:
a.b.c : 42
a {
b.c : 42
}
a.b {
c : 42
}
a {
b {
c : 42
}
}
Related
Our C++ software use ITK to write DICOM files. In it we have a Private Tag with LO (Long String) as VR and 2 decimal values as value like 0.3234\0.34223.
The LO choice is inherent to ITK.
In other java application, I use dcm4che3 to read/write them. Since it respects the DICOM protocol, backslash are forbidden, and dcm4che interpret the value as "0.3234" and never reach the second value.
All DICOM viewer applications I use can display this value.
So my question is: Is there a trick in dcm4che to read this complete value as a string "0.3234\0.34223" despite the presence of a backslash?
Below, the code I use:
public DicomInfo uploadFile(MultipartFile file) throws IOException, ParseException {
DicomInfo infos = new DicomInfo();
Attributes attrs = readDicomAttributes(file);
infos.setTags(toAttributesObject(attrs).toString());
}
private static JsonObject toAttributesObject(Attributes targetSeriesAttrs)
{
StringWriter strWriter = new StringWriter();
JsonGenerator gen = Json.createGenerator(strWriter);
JSONWriter writer = new JSONWriter(gen);
writer.write(targetSeriesAttrs);
gen.flush();
gen.close();
return Json.createReader(new
StringReader(strWriter.toString())).readObject();
}
public Attributes readDicomAttributes(MultipartFile file) throws IOException
{
DicomInputStream dis = new DicomInputStream(file.getInputStream());
Attributes dataSet = dis.readDataset(-1, Tag.PixelData);
Attributes fmi = dis.readFileMetaInformation();
dis.close();
fmi.addAll(dataSet);
return fmi;
}
In the JSON I get for this tag:
\"00110013\":{\"vr\":\"LO\",\"Value\":[\"0.4323\"]},
As you can see it is LO and the second part is already lost.
The method I use to get the specific attribute:
attr.getStrings(0x00110013)
send back a table with only one value, 0.4323.
The problem happens during the readDataSet function.
When I open tags with software like free dicom viewer, I have the complete data, so data is here.
Ok I found the source of the problem... It is the addAll fmi.addAll(dataSet);
In dataSet, getStrings works perfectly. In fmi after addAll, the attributes lost the second value.
So my problem is to solve this addAll issue now: dcm4che3 java lib: Attributes.addAll method seems to lost multiple LO values
See answer from Paolo, and please believe us that the backslash is not a violation of the VR. Like he said, the attribute is 2-dimensional, i.e. it has two values of VR LO which are separated by the backslash.
I know a bit about the dcm4che project and the people behind it, and it is nearly unthinkable to me that it is generally incapable of handling this.
I strongly suspect that your problem is related to the fact that your attribute is private. That is, without any additional information to the tag and its value, dcm4che (and any other product) can never know that the attribute's value is encoded as VR LO (Long String).
The default transfer syntax in DICOM is Implicit Little Endian. This means, that the dataset does not convey an explicit information about the VR of the attributes in the dataset. This information is implicitly encoded by the Tag of the attribute, and the data dictionary (DICOM Part 6) must be used to look up the tag and obtain the corresponding VR. Obvioulsy this only works for well-known DICOM tags defined in the standard and fails for private ones.
So one option is to try encoding the dataset in Explicit Little Endian, which means that the VR is part of the attribute's encoding in the dataset. I would expect this to be sufficient.
Mature toolkits (like dcm4che) allow for extending the data dictionary by configuration, that is, you can extend the "official" data dictionary used by the tookit with your custom tag definitions - including the VR. So the tag can be looked up in the case that the VR is not explicitly given in the dataset itself.
Again, I am not an expert for dcm4che, but a quick search at google for "dcm4che private dictionary" yields this promising link.
I am more than confident that you can solve the problem in dcm4che and that you do not have to migrate to a different toolkit.
The solution of this problem is to write
dataSet.addAll(fmi);
return dataSet;
instead of
fmi.AddAll(dataSet);
return fmi;
since the addAll methods lost multiple values of private LO
LO can have multiple values separated by a backslash.
The DICOM standard says that in the VR "LO" the backslash cannot be used in values because it is used to separate the different elements.
In VRs that don't allow multiple elements then the backslash can be used in values.
So dcm4che is wrong here.
EDIT: I changed my mind. I would find a way to generate the Java class and load the JSON as an object of that class.
I just discovered that exists a variant of JSON called JSON-LD.
It seems to me a more structured way of defining JSON, that reminds me XML with an associated schema, like XSD.
Can I create a Java class from JSON-LD, load it at runtime and use it to convert JSON-LD to an instantiation of that class?
I read the documentation of both the implementations but I found nothing about it. Maybe I read them bad?
Doing a Google search brought me to a library that will decode the JSON-LD into an "undefined" Object.
// Open a valid json(-ld) input file
InputStream inputStream = new FileInputStream("input.json");
// Read the file into an Object (The type of this object will be a List, Map, String, Boolean,
// Number or null depending on the root object in the file).
Object jsonObject = JsonUtils.fromInputStream(inputStream);
// Create a context JSON map containing prefixes and definitions
Map context = new HashMap();
// Customise context...
// Create an instance of JsonLdOptions with the standard JSON-LD options
JsonLdOptions options = new JsonLdOptions();
// Customise options...
// Call whichever JSONLD function you want! (e.g. compact)
Object compact = JsonLdProcessor.compact(jsonObject, context, options);
// Print out the result (or don't, it's your call!)
System.out.println(JsonUtils.toPrettyString(compact));
https://github.com/jsonld-java/jsonld-java
Apparently, it can take it from just a string as well, as if reading it from a file or some other source. How you access the contents of the object, I can't tell. The documentation seems to be moderately decent, though.
It seems to be an active project, as the last commit was only 4 days ago and has 30 contributors. The license is BSD 3-Clause, if that makes any difference to you.
I'm not in any way associate with this project. I'm not an author nor have I made any pull requests. It's just something I found.
Good luck and I hope this helped!
see this page: JSON-LD Module for Jackson
I'm relatively new to Protobufs and I was wondering how I would use it to process a list of structures.
Lets say I have a configuration file that looks like this:
Bucket{
name = "A";
path = "~/Document/...";
traffic = 5;
}
Bucket{
name = "B";
path = "~/Document/...";
traffic = 6;
}
Bucket{
name = "C";
path = "~/Document/...";
traffic = 7;
}
etc etc.
So I am using a protobuf to essentially structure this for ease of use later in Java. I'm essentially trying to make a map in a map (a protobuf to help find the correct bucket, and then another map to obtain member attributes inside the bucket).
option java_outer_classname = "Buckets";
message Bucket {
required string name = 1;
required string path = 2;
optional int32 traffic = 3;
}
message BucketList {
required Bucket bucket = 1;
}
I'm confused on how I would link the two, as in, how would I pass the configuration file into the protobuf methods (after it compiles into a java class) and use to to access the bucket member to do stuff like say get the path from the bucket with the name A?
Thank you!
It's perfectly acceptable to use Protobuf as a mechanism to declare and parse a text configuration file. However, one must keep in mind that Protobuf's purpose is the declare the format of the file, be it plain text or its binary wire format. Higher level semantic constraints need to be enforced in custom application code, once the configuration has been read.
You've got the wrong idea of what protobuf is used for, it is a data interchange library which means its used to encode and exchange data between programs. It was never meant to be used for configuration and doesn't have a way to read a text-based definition of the data since it deals with binary files.
Looking at your config format you have two options:
The format you've chosen looks a lot like HOCON, so https://github.com/typesafehub/config should be able to read it and provide a readable config object with a small amount of editing.
If you want a type-safe config (defining the structure of the config as an actual java object) you will need to use some other format which supports deserialization to object. A JSON configuration can be read into objects using libraries like https://github.com/google/gson or https://github.com/FasterXML/jackson
Trying to load a model from a CIM/XML file acording to IEC 61970 (Common Information Model, for power systems models), I found a problem;
According JAXB´s graphs between elements are provided by #XmlREF #XmlID and these both should be equals to match. But in CIM/RDF the references to a resource through an ID, i.e. rdf:resource="#_37C0E103000D40CD812C47572C31C0AD" contain the "#" character, consequently JAXB is unable to match "GeographicalRegion" vs. "SubGeographicalRegion.Region" when in the rdf:resource atribute the "#" character is present.
Here an example:
<cim:GeographicalRegion rdf:ID="_37C0E103000D40CD812C47572C31C0AD">
<cim:IdentifiedObject.name>GeoRegion</cim:IdentifiedObject.name>
<cim:IdentifiedObject.localName>OpenCIM3bus</cim:IdentifiedObject.localName>
</cim:GeographicalRegion>
<cim:SubGeographicalRegion rdf:ID="_ID_SubGeographicalRegion">
<cim:IdentifiedObject.name>SubRegion</cim:IdentifiedObject.name>
<cim:IdentifiedObject.localName>SubRegion</cim:IdentifiedObject.localName>
<cim:SubGeographicalRegion.Region rdf:resource="#_37C0E103000D40CD812C47572C31C0AD"/>
</cim:SubGeographicalRegion>
I realize you're asking for a solution using JAXB, but I would urge you to consider an RDF-based solution as it is more flexible and robust. You're basically trying to reinvent what RDF parsers already have built in. RDF/XML is a difficult format to parse, it doesn't make much sense to try and hack your own parsing together - especially since files that have very different XML structures can express exactly the same information: this only becomes apparent when looking at the level of the RDF. You may find that your JAXB parser workaround works on one CIM/RDF file but completely fails on another.
So, here's an example of how to process your file using the Sesame RDF API. No inferencing is involved, this just parses the file and puts it in an in-memory RDF model, which you can then manipulate and query from any angle.
Assuming the root element of your CIM file looks something like this:
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
xmlns:cim="http://example.org/cim/">
(only a guess of course, but I need prefixes for a proper example)
Then you can do the following, using Sesame's Rio RDF/XML parser:
String baseURI = "http://example.org/my/file";
FileInputStream in = new FileInputStream("/path/to/my/cim.rdf");
Model model = Rio.parse(in, baseURI, RDFFormat.RDFXML);
This creates an in-memory RDF model of your document. You can then simply filter-query over that. For example, to print out the properties of all resources that have _37C0E103000D40CD812C47572C31C0AD as their SubGeographicalRegion.Region:
String CIM_NS = "http://example.org/cim/";
ValueFactory vf = ValueFactoryImpl.getInstance();
URI subRegion = vf.createURI(CIM_NS, "SubGeographicalRegion.Region");
URI res = vf.createURI("http://example.org/my/file#_37C0E103000D40CD812C47572C31C0AD");
Set<Resource> subs = model.filter(null, subRegion, res).subjects();
for (Resource sub: subs) {
System.out.println("resource: " + sub + " has the following properties: ");
for (URI prop: model.filter(sub, null, null).predicates()) {
System.out.println(prop + ": " + model.filter(sub, prop, null).objectValue());
}
}
Of course at this point you can also choose to convert the model to some other syntax format for further handling by your application - as you see fit. The point is that the difference between the identifiers with the leading # and without has been resolved for you by the RDF/XML parser.
This is of course personal opinion only, since I don't know the details of your use case, but I think you'll find that this is quite quick and flexible. I should also point out that although the above solution keeps the entire model in memory, you can easily adapt this to a more streaming (and therefore less memory-intensive) approach if you find your files are too big.
I have a RDF file that I am able to read using
Model model = ModelFactory.createDefaultModel();
// use the FileManager to find the input file
InputStream in = FileManager.get().open(args[0]);
if (in == null) {
throw new IllegalArgumentException(
"File: " + args[0] + " not found");
}
// read the RDF/XML file
model.read(in, null);
I also have OWL file which contains the description of the ontology which is used for creating my models. My question is: do I need to read this file (and how?) in order to work with my RDF model correctly?
To make myself clear, I will give ou an example:
I need to know whether one resource has some relationship with other resource (for example Station1 has predicate "isResponsibleFor" Workorder1). How can I do this with Jena?
If I try to use something like resource.hasProperty(ResourceFactory.createProperty("isResponsibleFor")), it returns false (but the property is there!).
Can you direct me to some advanced tutorial on this topic perhaps? I found many tutorials on Papache site etc. but they do not provide me with the information I am looking for. Sorry if the question is not clear, I am quite new to Jena
EDIT: currently, I am searching whether my model contains given statement using this:
public static boolean containsStatement(Model model, String sub,
String pred, String obj) {
// list the statements in the Model
StmtIterator iter = model.listStatements();
// print out the predicate, subject and object of each statement
while (iter.hasNext()) {
Statement stmt = iter.nextStatement(); // get next statement
Resource subject = stmt.getSubject(); // get the subject
Property predicate = stmt.getPredicate(); // get the predicate
RDFNode object = stmt.getObject(); // get the object
if (subject.toString().contains(sub)
&& predicate.toString().contains(pred)
&& object.toString().contains(obj)) {
return true;
}
}
return false;
}
but I am pretty sure that this is highly ineffective approach.. could you suggest me something more elegant and fast? Thanks!
Short answer: no, you don't need the ontology to work with your RDF file, but in many cases it can help your application.
First, you can shorten loading your file:
Model model = FileManager.get().loadModel( args[0] );
Now, in order to work with the relationship between resources, as given by the URI of the property connecting the subject resource to the object, you need the full URI of the predicate. Typically, this will be something like http://example.com/foo#isResponsibleFor. If you just use the short-name of predicate, it won't work - which is what you are finding.
You don't show any examples of your actual RDF data, so I'm going to use a fake namespace. Use your actual namespace in your code. In the meantime:
String NS = "http://example.com/example#";
Property isResponsibleFor = model.getProperty( NS + "isResponsibleFor" );
Resource station = model.getResource( NS + "station1" );
for (StmtIterator i = station.listProperties( isResponsibleFor ); i.hasNext(); ) {
Statement s = i.next();
Resource workorder = s.getResource();
// now you can do something with the work-order resource
}
In your code, you had:
public static boolean containsStatement(Model model, String sub, String pred, String obj)
There are a number of things wrong here. First, it's better if you can write your code in a more object-oriented style, which tends not to use static methods if that can be avoided. Second, don't use strings when you refer to things in a model. Jena has the Resource class to denote resources in a model, and many other RDF-specific classes as well. Use strings for handling input from your user, otherwise convert strings to resources or other RDF objects as soon as you can. Thirdly, I'd advise against exposing the details of your representation via your object's API. containsStatement makes it clear in the API that you are using RDF triples; that's not a detail that callers of the API need to know and it breaks encapsulation. A better API would have methods such as listWorkItems - that relates to the domain, and hides details of the implementation.
Regarding the use of your ontology, there are two specific ways your application can benefit from using your ontology. First, you can automatically generate statements such as:
Property isResponsibleFor = model.getProperty( NS + "isResponsibleFor" );
by using Jena's schemagen vocabulary generator tool. You can use schemagen as part of your build process to ensure that your vocabulary class automatically stays up-to-date as your ontology changes.
Secondly, by using Jena's inference engines, you can use the ontology to infer additional statements about your domain. For example, suppose you have class WidgetOrder, which is a type of WorkItem. Without inference, and without your ontology, if you ask the model object to list all of the WorkItems, it won't list the WidgetOrder resources. However, with the ontology and the reasoner, listing resources of type WorkItem will also return the resources that only have a declared type of WidgetOrder, because the other types can be inferred.