Protobuffers to XML - java

I want to convert my google protocol object to XML format. In that I would like to keep some fields as attribute.
Instead of :
<field>
<name>ApiFieldHeaderName</name>
<maxLength>50</maxLength>
</field>
I want following :
<field name="ApiFieldHeaderName" maxLength="50" ></field>
My protocol is
string name = 1;
int32 maxLength = 2;
And then I have gone through some forum and used xml_disposition
[(xml_disposition) = ATTRIBUTE]
However, I am getting the error:
Option "(xml_disposition)" unknown.
I am using proto3 and language is Java.
syntax = "proto3";
option optimize_for = SPEED;

I believe you've found xml_disposition from a side discussion on the protobuf newsgroup from 2009. The option mentioned, however, was purely hypothetical. As far as I know: no such xml_disposition custom option exists - and least, not as a standard option - and no code-generator looks for it. No mention of xml_disposition exists in the Google protobuf source, and the current version of protoc (3.5.1) does not recognise it and an inbuilt option.
So:
yes, it is possible to add custom extensions in .proto that you can annotate fields (etc) with
yes, you can write your own codegen tools to check for those custom options and emit additional generated code to implement what you want
but none of this ships by default
Side note: custom options must be defined in "proto2" syntax files, but a "proto3" file can still import and use those custom options from the "proto2" file.

Related

Tapestry 5.4 core.properties override

I have a tapestry 5.4 project, and I want to override one element of the default core.properties file.
I tried to add to login_en.properties and login_hu.properties a new line (core-default-error-banner=...) but it do not override it.
Is there any way to overwrite it?
Thanks for the answers in advance.
If you intend to overwrite the header line of the Errors component, just specify your own message id in the page/component template like this:
<t:errors banner="message:your-translated-error-msg-id" />
To support other locales, just translate the built-in message catalog (property) files yourself: To have Tapestry use these new files, just put them in the corresponding package-named directory within your own app (for example, src/main/resources/org/apache/tapestry5/core.properties). More informtation can be found from Tapestry site.

How to create a tensorflow serving client for the 'wide and deep' model?

I've created a model based on the 'wide and deep' example (https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/learn/wide_n_deep_tutorial.py).
I've exported the model as follows:
m = build_estimator(model_dir)
m.fit(input_fn=lambda: input_fn(df_train, True), steps=FLAGS.train_steps)
results = m.evaluate(input_fn=lambda: input_fn(df_test, True), steps=1)
print('Model statistics:')
for key in sorted(results):
print("%s: %s" % (key, results[key]))
print('Done training!!!')
# Export model
export_path = sys.argv[-1]
print('Exporting trained model to %s' % export_path)
m.export(
export_path,
input_fn=serving_input_fn,
use_deprecated_input_fn=False,
input_feature_key=INPUT_FEATURE_KEY
My question is, how do I create a client to make predictions from this exported model? Also, have I exported the model correctly?
Ultimately I need to be able do this in Java too. I suspect I can do this by creating Java classes from proto files using gRPC.
Documentation is very sketchy, hence why I am asking on here.
Many thanks!
I wrote a simple tutorial Exporting and Serving a TensorFlow Wide & Deep Model.
TL;DR
To export an estimator there are four steps:
Define features for export as a list of all features used during estimator initialization.
Create a feature config using create_feature_spec_for_parsing.
Build a serving_input_fn suitable for use in serving using input_fn_utils.build_parsing_serving_input_fn.
Export the model using export_savedmodel().
To run a client script properly you need to do three following steps:
Create and place your script somewhere in the /serving/ folder, e.g. /serving/tensorflow_serving/example/
Create or modify corresponding BUILD file by adding a py_binary.
Build and run a model server, e.g. tensorflow_model_server.
Create, build and run a client that sends a tf.Example to our tensorflow_model_server for the inference.
For more details look at the tutorial itself.
Just spent a solid week figuring this out. First off, m.export is going to deprecated in a couple weeks, so instead of that block, use: m.export_savedmodel(export_path, input_fn=serving_input_fn).
Which means you then have to define serving_input_fn(), which of course is supposed to have a different signature than the input_fn() defined in the wide and deep tutorial. Namely, moving forward, I guess it's recommended that input_fn()-type things are supposed to return an InputFnOps object, defined here.
Here's how I figured out how to make that work:
from tensorflow.contrib.learn.python.learn.utils import input_fn_utils
from tensorflow.python.ops import array_ops
from tensorflow.python.framework import dtypes
def serving_input_fn():
features, labels = input_fn()
features["examples"] = tf.placeholder(tf.string)
serialized_tf_example = array_ops.placeholder(dtype=dtypes.string,
shape=[None],
name='input_example_tensor')
inputs = {'examples': serialized_tf_example}
labels = None # these are not known in serving!
return input_fn_utils.InputFnOps(features, labels, inputs)
This is probably not 100% idiomatic, but I'm pretty sure it works. For now.

Add extension to CertificateRequestMessage

I'm trying to put a custom element (a template name to use for the PKI) in my Certificate request that I generate using bouncycastle library.
The RFC says that you can use an extension that follows :
id-regInfo-utf8Pairs OBJECT IDENTIFIER ::= { id-regInfo 1 }
--with syntax UTF8Pairs ::= UTF8String
The few lines of code concerning this are
CertificateRequestMessageBuilder msgbuilder = new
CertificateRequestMessageBuilder(BigInteger.valueOf(reqId));
msgbuilder.addExtension(new ASN1ObjectIdentifier("1.3.6.1.5.5.7.7.5.2.1"), false, ???);
I can't find any information on what to put in that addExtension function, nor any example to adapt from.
I guess the new ASN1ObjectIdentifier("1.3.6.1.5.5.7.7.5.2.1") is close to what is expected, but the following ASN1Encodable that is supposed to be the value of that field is a mystery to me.
I just want to get something along template:encipherment in that field.
This seems like a pretty simple thing to do but the lack of documentation coupled to my lack of experience with BouncyCastle causes me a lot of problems.
The questions are :
Am I doing what I want the correct way ? (Using the extensions of the CRMF to convey my custom field)
If I do, how do I make this work ?

Doxygen annotation/attribute are ignored

I use doxygen to generate xml which then i transform to a custom documentation.
Is there a possibility that doxygen includes the annotation of a field / class / function.
The annotation are ignored in both java and c#.
ex:
class User
{
[Required]
string UserName {get;set;}
}
the "Required" annotation is not parsed/displayed in doxygen.
What I would like to have in the xml / html output of doxygen is all the annotated Annotations of a property / field / class (in the ex. "[Required]").
EXTRACT_ALL=YES is useless in this case. Look at this answer, I think it is good idea:
Doxygen and add a value of an attribute to the output documentation
So you have to create filter (for example in phyton) which will be used by Doxygen to convert annotation to comment. Don’t forget to inform Doxygen about your filter: INPUT_FILTER = doxygenFilter.py
I have the same problem so I modified that example in this way:
#!/usr/bin/env python
import sys
import re
if (len(sys.argv) < 2):
print "No input file"
else:
f = open(sys.argv[1])
line = f.readline()
while line:
re1 = re.compile("\s*\[(.*)]\s*")
re1.search(line)
sys.stdout.write(re1.sub(r"/// <para>Annotation: [\1]</para>\n", line))
#sys.stdout.write(line)
line = f.readline()
f.close()
So code like
[AnyAnnotation()]
Will be converted to:
/// <param> Annotation [AnyAnnotation()] </param>`
So I got very nice result. Tag <param> is to avoid Doxygen put this annotation description to main description. Instead it will put it to remarks section.
I'm not sure what you're asking but I will say a few things that might help you.
Doxygen must be configured to produce documentation for code elements that have no Doxygen comments. In other words, you can tell Doxygen to produce documentation for all functions, variables, macros, etc even if they aren't documented in the code. Set EXTRACT_ALL=YES in your config file.
If you run DoxyWizard you will get a better feel for all of the available options and the effect of each option. DoxyWizard is the GUI front end to Doxygen.
And by the way, bravo for documenting your code!

How to validate an XML document using a RELAX NG schema and JAXP?

I would like to validate XML documents using RELAX NG schemata, and I would like to use the JAXP validation API.
From Googling around, it appeared that I could use Jing and the ISO RELAX JARV to JAXP Bridge. Unfortunately, after adding both to my classpath, I can't get it to work. SchemaFactory is just throwing an IllegalArgumentException as soon as it tries to instantiate a factory — I looked inside SchemaFactory, apparently SchemaFactoryFinder is returning a null result.
So I'd appreciate answers to either question:
How can I make this work with Jing and this bridge?
Is there a better/different set of libraries I should try?
I need this to work with Java 5 and Java 6.
Thanks!
I resolved this very error on Java 1.6 with the following line:
// Specify you want a factory for RELAX NG "compact"
System.setProperty(SchemaFactory.class.getName() + ":" + XMLConstants.RELAXNG_NS_URI, "com.thaiopensource.relaxng.jaxp.CompactSyntaxSchemaFactory");
SchemaFactory factory = SchemaFactory.newInstance(XMLConstants.RELAXNG_NS_URI);
This allows me to use Jing to validate an XML document against a Compact RELAX NG schema. Full example below. I didn't use the bridge or anything else. The runtime classpath only has jing.jar (20091111) and my own Validator class.
import java.io.File;
import java.io.IOException;
import javax.xml.XMLConstants;
import javax.xml.transform.stream.StreamSource;
import javax.xml.validation.Schema;
import javax.xml.validation.SchemaFactory;
import javax.xml.validation.Validator;
import org.xml.sax.SAXException;
public class Validate
{
public static void main(String[] args) throws SAXException, IOException
{
// Specify you want a factory for RELAX NG
System.setProperty(SchemaFactory.class.getName() + ":" + XMLConstants.RELAXNG_NS_URI, "com.thaiopensource.relaxng.jaxp.CompactSyntaxSchemaFactory");
SchemaFactory factory = SchemaFactory.newInstance(XMLConstants.RELAXNG_NS_URI);
// Load the specific schema you want.
// Here I load it from a java.io.File, but we could also use a
// java.net.URL or a javax.xml.transform.Source
File schemaLocation = new File(args[0]);
// Compile the schema.
Schema schema = factory.newSchema(schemaLocation);
// Get a validator from the schema.
Validator validator = schema.newValidator();
for (int i = 1; i < args.length; i++)
{
String file = args[i];
// Check the document
try
{
validator.validate(new StreamSource(new File(file)));
System.out.println(file + " is valid.");
}
catch (SAXException ex)
{
System.out.print(file + " is not valid because: " + ex.getMessage());
}
}
}
}
Once again, I've only tested this ion Java 1.6.
$ java -version
java version "1.6.0_01"
Java(TM) SE Runtime Environment (build 1.6.0_01-b06)
Java HotSpot(TM) Client VM (build 1.6.0_01-b06, mixed mode, sharing)
See Stefan Bodewig's Weblog written on March 7, 2008 titled RELAX NG Validation in XMLUnit:
Since last night XMLUnit's trunk contains a new Validator class that is based on javax.xml.validation which is part of JAXP 1.3 (i.e. Java5+).
...
To the best of my knowledge there is no JAXP implementation that supported RELAX NG out of the box. Sun's own JAXP 1.4 (Java6+) certainly doesn't. Some searching around brought me to Kohsuke Kawaguchi's Blog who should know, given his work on JAXP, Sun's Multi Schema Validator, isorelax and other stuff.
Using his isorelax-bridge and Jing didn't get me anywhere on Java6. I went back to Kohsuke Kawaguchi's article and read the comments: the bridge doesn't work with Java6 since they changed the SchemaFactory lookup algorithm. OK, tried Java5 instead - progress, I now get a NullPointerException somewhere inside of Jing, so at least it is loading the factory. Next I replaced Jing with MSV (which is here now, no matter how many links out there lead you to the WebServices stack page at Sun, so much for "good URLs never change") and really, my simplistic tests pass.
So you may have to jump through some hoops to get RELAX NG support into your JAXP setup - in my case Java5, MSV and Kawaguchi's bridge worked, but the comments indicate it should be doable with Java6 as well - but once you manage to configure everything correctly, XMLUnit will now be there to let you assert your document's validity in Unit tests. It seems that it doesn't work for compact syntax, though.
To read the comments on Kohsuke Kawaguchi's blog, you have to use archive.org because somehow they are all gone now:
Java 5 interprets the Service Provider
file as a list of key/value pairs,
which is a violation to the Java 5 & 6
JAR file specification but happens to
match your example.
Java 6 parses the Service Provider
file as specified, ie. as a list of
fully qualified class names, but thus
fails to instantiate your adapter's
SchemaFactory as the Service Provider
file's contents are invalid.
To be compatible with both Java 5 and
Java 6 without having to change the
JAXP-JARV-adapter JAR file, one can
simply add another JAR file containing
a correct
javax.xml.validation.SchemaFactory
Service Provider file.
I can't help you with the JAXP validation API, but Nux provides a class that can validate virtually every type of schema known to man. Regarding RELAX NG schemata, use this factory method to create the relevant validator object.
Another option is Trang, which is a RelaxNG-to-XMLSchema translator. I believe it's designed to be used as a build tool rather than a runtime library, but your best option might be to convert your schema to XMLSchema using Trang at build time, and then validate against that instead. That way, you can see exactly what the translation looks like, whilst getting full advantage of the XML Schema support of JAXP.
... IllegalArgumentException as soon as it tries to instantiate a factory
Means the schema language isn't recognized, there could be a few causes.
Since the Sun JDK doesn't include a RELAX NG validator by default, it could be that it's not being found.
It could be that you've made an error in the schema language identifier.

Categories