We Keep Coding

Using M2Doc programmatically : Error in the generated .docx document

I'm trying to use M2Doc programmatically, I managed to generate my .docx file without getting errors in the validation part but I'm getting the following Error in the generated document:
{m:self.Name} Couldn't find the 'aqlFeatureAccess(org.eclipse.emf.common.util.URI.Hierarchical,java.lang.String)' service
The "self.Name" part is what I wrote in my template.
I think I'm lacking some kind of reference to a service but I don't know how to fix it.
The self variable is a reference to a model based on a meta-model I created. But I'm not sure I imported it correctly in my code.
I based my code on the code I found on the M2Doc website + some code I found on their GitHub, especially concerning how to add a service in the queryEnvironment.
I searched in the source code of acceleo and M2Doc to see which services they add but it seems that they already import all the services I'm using.
As I said, the validation part is going well and doesn't generate a validation file.
public static void parseDocument(String templateName) throws Exception{
final URI templateURI = URI.createFileURI("Template/"+templateName+"."+M2DocUtils.DOCX_EXTENSION_FILE);
final IQueryEnvironment queryEnvironment =
org.eclipse.acceleo.query.runtime.Query.newEnvironmentWithDefaultServices(null);
final Map<String, String> options = new HashMap<>(); // can be empty
M2DocUtils.prepareEnvironmentServices(queryEnvironment, templateURI, options); // delegate to IServicesConfigurator
prepareEnvironmentServicesCustom(queryEnvironment, options);
final IClassProvider classProvider = new ClassProvider(ClassLoader.getSystemClassLoader()); // use M2DocPlugin.getClassProvider() when running inside Eclipse
try (DocumentTemplate template = M2DocUtils.parse(templateURI, queryEnvironment, classProvider)) {
ValidationMessageLevel validationLevel = validateDocument(template, queryEnvironment, templateName);
if(validationLevel == ValidationMessageLevel.OK){
generateDocument(template, queryEnvironment, templateName, "Model/ComplexKaosModel.kaos");
}
}
}
public static void prepareEnvironmentServicesCustom(IQueryEnvironment queryEnvironment, Map<String, String> options){
Set<IService> services = ServiceUtils.getServices(queryEnvironment, FilterService.class);
ServiceUtils.registerServices(queryEnvironment, services);
M2DocUtils.getConfigurators().forEach((configurator) -> {
ServiceUtils.registerServices(queryEnvironment, configurator.getServices(queryEnvironment, options));
});
}
public static void generateDocument(DocumentTemplate template, IQueryEnvironment queryEnvironment,
String templateName, String modelPath)throws Exception{
final Map<String, Object> variable = new HashMap<>();
variable.put("self", URI.createFileURI(modelPath));
final Monitor monitor = new BasicMonitor.Printing(System.out);
final URI outputURI = URI.createFileURI("Generated/"+templateName+".generated."+M2DocUtils.DOCX_EXTENSION_FILE);
M2DocUtils.generate(template, queryEnvironment, variable, outputURI, monitor);
}

The variable "self" contains an URI:
variable.put("self", URI.createFileURI(modelPath));
You have to load your model and set the value of self to an element from your model using something like:
final ResourceSet rs = new ResourceSetImpl();
final Resource r = rs.getResource(uri, true);
final EObject value = r.getContents()...;
variable.put("self", value);
You can get more details on resource loading in the EMF documentation.

How to pass data from vaadin webapp to C# GUI app

I have webapp in Vaadin Framework 8. I have Windows GUI app in C#.
The gui app is using WebBrowser component to display webapp. WebBrowser component is internally using IE11 core through ActiveX. I can successfully load and display the webapp in the gui app browser component.
I need to pass data from webapp to the gui app.
The webapp has many rows loaded on server side, only few are displayed in grid. I want to pass all data from webapp to gui app in some format (csv or json).
I have tryed some approaches, but I wasn't successfull.
[Approach 1]
Webapp: attach downloadable resource (csv) to Link with predefined id using FileDownloader. Download by user mouse click works fine, file save dialog pops up and data are downloaded successfully.
Link link = new Link("Data");
link.setId("myId");
StreamResource resource = getMyResource(data);
FileDownloader downloader = new FileDownloader(resource);
downloader.extend(link);
Page.getCurrent().getJavaScript().addFunction("test", new JavaScriptFunction() {
#Override
public void call(JsonArray arguments) {
Page.getCurrent().getJavaScript()
.execute("document.getElementById('myId').click()");
}
});
Gui app: raise onClick event on link and capture WebBrowser.FileDownload event, capture WebBrowser.Navigate event.
I have failed to raise onClick event from C# using:
HtmlElement el = webBrowser.Document.GetElementById("myId");
el.RaiseEvent("onClick");
el.InvokeMember("click");
webBrowser.Document.InvokeScript("document.getElementById('myId').click();", null);
webBrowser.Document.InvokeScript("test", null);
Result:
WebBrowser.FileDownload event doesn't work (is fired but can't capture url nor data), capture WebBrowser.Navigate event works partialy (can see resource url, but can't download data using byte[] b = new WebClient().DownloadData(e.Url);).
[Approach 2]
Similar to approach 1. I tryed to get resource url, put the direct url to Link and download the resource in c# using direct link. I can construct the same resource url as is used by browser to download data when user clicks the link.
Extended file downloader that keeps resource, key and connector:
public class ExtendedFileDownloader extends FileDownloader {
private String myKey;
private Resource myResource;
private ClientConnector myConnector;
public ExtendedFileDownloader(StreamResource resource, ClientConnector connector) {
super(resource);
myConnector = connector;
}
#Override
protected void setResource(String key, Resource resource) {
super.setResource(key, resource);
myKey = key;
myResource = resource;
}
public String getResourceUrl() {
ResourceReference ref =
ResourceReference.create(
myResource,
(myConnector != null) ? myConnector : this,
myKey);
String url = ref.getURL();
return url;
}
}
In view:
// fix app://path... urls to /<base-path>/path urls
private String fixResourceReferenceUrl(String resourceReferenceUrl) {
String resourceReferencePath = resourceReferenceUrl.replace("app://", "");
String uiBaseUrl = ui.getUiRootPath();
String fixedUrl = uiBaseUrl + "/" + resourceReferencePath;
return fixedUrl;
}
Link link2 = new Link("Data2");
link2.setId("myId2");
StreamResource resource = getMyResource(data);
ExtendedFileDownloader downloader = new ExtendedFileDownloader(resource, this);
String fixedResourceUrl = fixResourceReferenceUrl(downloader.getResourceUrl());
link2.setResource(new ExternalResource(fixedResourceUrl));
Result:
The data cannot be downloaded using this link, server error 410 or NotFound errors.
Any Ideas ? Any other approaches to try ?

I have finally solved the problem. The solution is very close to approach 2.
The resource url is passed in element with custom attribute. C# WebClient needs to set cookies from WebBrowser and Referer HTTP headers. The data can be successfully downloaded by C# app.
Element attribute in vaadin webapp can be set using Vaadin-addon Attributes.
Cookies in C# app can be retrieved using this solution.
// Fix resource urls begining with app://
public String fixResourceReferenceUrl(String resourceReferenceUrl) {
try {
String uiRootPath = UI.getCurrent().getUiRootPath();
URI location = Page.getCurrent().getLocation();
String appLocation = new URIBuilder()
.setScheme(location.getScheme())
.setHost(location.getHost())
.setPort(location.getPort())
.setPath(uiRootPath)
.build()
.toString();
String resourceReferencePath = resourceReferenceUrl.replace("app://", "");
String fixedUrl = appLocation + "/" + resourceReferencePath;
return fixedUrl;
}
catch (Exception e) {
return null;
}
}
In view (using ExtendedFileDownloader from above):
Link link = new Link("Data");
link.setId("myId");
StreamResource resource = getMyResource(data);
ExtendedFileDownloader downloader = new ExtendedFileDownloader(resource);
downloader.extend(link);
Attribute attr = new Attribute("x-my-data", fixResourceReferenceUrl(downloader.getResourceUrl()));
attr.extend(link);
link.setVisible(true);
In C# app:
[DllImport("wininet.dll", SetLastError = true)]
public static extern bool InternetGetCookieEx(
string url,
string cookieName,
StringBuilder cookieData,
ref int size,
Int32 dwFlags,
IntPtr lpReserved);
private const Int32 InternetCookieHttponly = 0x2000;
public static String GetUriCookies(String uri)
{
// Determine the size of the cookie
int datasize = 8192 * 16;
StringBuilder cookieData = new StringBuilder(datasize);
if (!InternetGetCookieEx(uri, null, cookieData, ref datasize, InternetCookieHttponly, IntPtr.Zero))
{
if (datasize < 0)
return null;
// Allocate stringbuilder large enough to hold the cookie
cookieData = new StringBuilder(datasize);
if (!InternetGetCookieEx(
uri,
null, cookieData,
ref datasize,
InternetCookieHttponly,
IntPtr.Zero))
return null;
}
return cookieData.ToString();
}
private void button_Click(object sender, EventArgs e)
{
HtmlElement el = webBrowser.Document.GetElementById("myId");
String url = el.GetAttribute("x-my-data");
String cookies = GetUriCookies(url);
WebClient wc = new WebClient();
wc.Headers.Add("Cookie", cookies);
wc.Headers.Add("Referer", WEB_APP_URL); // url of webapp base path, http://myhost/MyUI
byte[] data = wc.DownloadData(url);
}

spring boot 2 properties configuration

I have some code that works properly on spring boot prior to 2 and I find it hard to convert it to work with spring boot 2.
Can somebody assist?
public static MutablePropertySources buildPropertySources(String propertyFile, String profile)
{
try
{
Properties properties = new Properties();
YamlPropertySourceLoader loader = new YamlPropertySourceLoader();
// load common properties
PropertySource<?> applicationYamlPropertySource = loader.load("properties", new ClassPathResource(propertyFile), null);
Map<String, Object> source = ((MapPropertySource) applicationYamlPropertySource).getSource();
properties.putAll(source);
// load profile properties
if (null != profile)
{
applicationYamlPropertySource = loader.load("properties", new ClassPathResource(propertyFile), profile);
if (null != applicationYamlPropertySource)
{
source = ((MapPropertySource) applicationYamlPropertySource).getSource();
properties.putAll(source);
}
}
propertySources = new MutablePropertySources();
propertySources.addLast(new PropertiesPropertySource("apis", properties));
}
catch (Exception e)
{
log.error("{} file cannot be found.", propertyFile);
return null;
}
}
public static <T> void handleConfigurationProperties(T bean, MutablePropertySources propertySources) throws BindException
{
ConfigurationProperties configurationProperties = bean.getClass().getAnnotation(ConfigurationProperties.class);
if (null != configurationProperties && null != propertySources)
{
String prefix = configurationProperties.prefix();
String value = configurationProperties.value();
if (null == value || value.isEmpty())
{
value = prefix;
}
PropertiesConfigurationFactory<?> configurationFactory = new PropertiesConfigurationFactory<>(bean);
configurationFactory.setPropertySources(propertySources);
configurationFactory.setTargetName(value);
configurationFactory.bindPropertiesToTarget();
}
}
PropertiesConfigurationFactory doesnt exist anymore and the YamlPropertySourceLoader load method no longer accepts 3 parameters.
(the response is not the same either, when I have tried invoking the new method the response objects were wrapped instead of giving me the direct strings/integers etc...)

The PropertiesConfigurationFactory should be replaced with Binder class.
Binder class
Sample code:-
ConfigurationPropertySource source = new MapConfigurationPropertySource(
loadProperties(resource));
Binder binder = new Binder(source);
return binder.bind("initializr", InitializrProperties.class).get();
We were also using PropertiesConfigurationFactory to bind a POJO to a
prefix of the Environment. In 2.0, a brand new Binder API was
introduced that is more flexible and easier to use. Our binding that
took 10 lines of code could be reduced to 3 simple lines.
YamlPropertySourceLoader:-
Yes, this class has been changed in version 2. It doesn't accept the third parameter profile anymore. The method signature has been changed to return List<PropertySource<?>> rather than PropertySource<?>. If you are expecting single source, please get the first occurrence from the list.
Load the resource into one or more property sources. Implementations
may either return a list containing a single source, or in the case of
a multi-document format such as yaml a source for each document in the
resource.

Since there is no accepted answer yet, i post my full solution, which builds upon the answer from #nationquest:
private ConfigClass loadConfiguration(String path){
MutablePropertySources sources = new MutablePropertySources();
Resource res = new FileSystemResource(path);
PropertiesFactoryBean propFactory = new PropertiesFactoryBean();
propFactory.setLocation(res);
propFactory.setSingleton(false);
// resolve potential references to local environment variables
Properties properties = null;
try {
properties = propFactory.getObject();
for(String p : properties.stringPropertyNames()){
properties.setProperty(p, env.resolvePlaceholders(properties.getProperty(p)));
}
} catch (IOException e) {
e.printStackTrace();
}
sources.addLast(new PropertiesPropertySource("prefix", properties));
ConfigurationPropertySource propertySource = new MapConfigurationPropertySource(properties);
return new Binder(propertySource).bind("prefix", ConfigClass.class).get();
}
The last three lines are the relevant part here.

dirty code but this is how i solved it
https://github.com/mamaorha/easy-wire/tree/master/src/main/java/co/il/nmh/easy/wire/core/utils/properties

Calling Existing PipeLine in GATE

I am new to Java and I want to call my saved pipeline using GATE JAVA API through Eclipse
I am not sure how I could do this although I know how to create new documents etc
FeatureMap params = Factory.newFeatureMap();
params.put(Document.DOCUMENT_URL_PARAMETER_NAME, new URL("http://www.gate.ac.uk"));
params.put(Document.DOCUMENT_ENCODING_PARAMETER_NAME, "UTF-8");
// document features
FeatureMap feats = Factory.newFeatureMap();
feats.put("date", new Date());
Factory.createResource("gate.corpora.DocumentImpl", params, feats, "This is home");
//End Solution 2
// obtain a map of all named annotation sets
Document doc = Factory.newDocument("Document text");
Map <String, AnnotationSet> namedASes = doc.getNamedAnnotationSets();
System.out.println("No. of named Annotation Sets:" + namedASes.size());
// no of annotations each set contains
for (String setName : namedASes.keySet()) {
// annotation set
AnnotationSet aSet = namedASes.get(setName);
// no of annotations
System.out.println("No. of Annotations for " +setName + ":" + aSet.size());

There is a good example of GATE usage from java. Probably it does exactly what you want. BatchProcessApp.java.
In particular:
loading pipeline is done with lines
// load the saved application
CorpusController application =
(CorpusController)PersistenceManager.loadObjectFromFile(gappFile);
pipeli executed with
// run the application
application.execute();
Code is informative, clear and could be easy changed for your particular needs. The oxygen of open source project :)

Something like this could be used(do not forget to init GATE: set GATE home and etc):
private void getProcessedText(String textToProcess) {
Document gateDocument = null;
try {
// you can use your method from above to build document
gateDocument = createGATEDocument(textToProcess);
corpusController.getCorpus().add(gateDocument);
corpusController.execute();
// put here your annotations processing
} catch (Throwable ex) {
ex.printStackTrace();
} finally {
if (corpusController.getCorpus() != null) {
corpusController.getCorpus().remove(gateDocument);
}
if (gateDocument != null) {
Factory.deleteResource(gateDocument);
}
}
}
private CorpusController initPersistentGateResources() {
try {
Corpus corpus = Factory.newCorpus("New Corpus");
corpusController = (CorpusController) PersistenceManager.loadObjectFromFile(new File("PATH-TO-YOUR-GAPP-FILE"));
corpusController.setCorpus(corpus);
} catch (Exception ex) {
ex.printStackTrace();
}
return corpusController;
}

Convenient way to extract data from the MtGox/PubNub JSON API?

I'm using the PubNub API with Java for pulling data from MtGox.
When retrieving data, the API delivers it in form of a JSONObject, which represents a tree structure of JSON data. Trying to extract bits of information directly from a JSONObject produces ugly code with lots of calls to getJSONObject(String), for which again exceptions might need to be handled.
Therefor, I'm looking for a convenient way to extract information from the JSONObject responses. So far, I've come across the possibility to convert the JSONObject into a POJO and then access the POJO. For conversion, I found the ObjectMapper from the Jackson library, which does a nice job here:
public void successCallback(String channel, Object message) {
JSONObject messageJson = (JSONObject) message;
ObjectMapper mapper = new ObjectMapper();
Message myMessage = mapper.readValue(messageJson.toString(), Message.class);
// do stuff with myMessage here
}
This approach has the disadvantage that I have to write my own POJO classes, e.g. the Message class in the above example, because I could not find these classes ready to use anywhere.
How to conveniently access the information stored in the JSONObject?

PubNub Java Class for MtGox JSON API
It's easy to create a ready made Java Class for ingesting the live feed provided by Mt.Gox This is a work-in-progress post to show you how to access the PubNub Data Feed from Mt.Gox as shown in the Dev Console live feed!
Official Bitcoin Wiki JSON Streaming API
We will be working from the Bitcoin wiki feed instructions provided by Bitcoin official Wiki: https://en.bitcoin.it/wiki/MtGox/API/Pubnub - continue reading below the screenshot to continue.
To see the live real-time data feed we will be using, please checkout the following two links:
Live Feed Trade Events (Buy/Sell Feed): https://www.pubnub.com/console?sub=sub-c-50d56e1e-2fd9-11e3-a041-02ee2ddab7fe&pub=demo&channel=dbf1dee9-4f2e-4a08-8cb7-748919a71b21&origin=pubsub.pubnub.com&ssl=true
Live Feed Ticker Updates (Price Changes): https://www.pubnub.com/console?sub=sub-c-50d56e1e-2fd9-11e3-a041-02ee2ddab7fe&pub=demo&channel=d5f06780-30a8-4a48-a2f8-7ed181b4a13f&origin=pubsub.pubnub.com&ssl=true
Trade Lag Example: https://www.mtgox.com/lag.html
PubNub Java SDK Docs
We will be using the PubNub Java SDK Docs
http://www.pubnub.com/docs/java/javase/overview/data-push.html
Specifically we'll be using the mtgox.subcribe(...) instance method to focus our efforts which looks like the following:
Download JAR or Source: https://github.com/pubnub/mtgox
import org.json.JSONObject;
import com.pubnub.mtgox.MtGox;
import com.pubnub.mtgox.MtGoxCallback;
public class PubnubMtGoxSample {
public static void main(String[] args) {
MtGox mtgx = new MtGox();
mtgx.subscribe("ticker.BTCUSD", new MtGoxCallback(){
#Override
public void callback(JSONObject data) {
try {
String channel_name = data.getString("channel_name");
String avg_value = data.getJSONObject("ticker").getJSONObject("avg").getString("value");
System.out.println(channel_name + " : " + avg_value);
} catch (Exception e) {}
}});
}
}
See Full MtGox Example with Java Source Code - https://github.com/pubnub/mtgox/blob/master/java/examples/PubnubMtGoxSample.java
To compile the example got to https://github.com/pubnub/mtgox/tree/master/java and run
javac -cp Pubnub-MtGox.jar:libs/json-20090211.jar examples/PubnubMtGoxSample.java
And then to RUN:
java -cp .:examples/:Pubnub-MtGox.jar:Pubnub-StandardEdition-3.5.6.jar:libs/json-20090211.jar:libs/bcprov-jdk15on-1.47.jar:libs/slf4j-api-1.7.5.jar:libs/slf4j-nop-1.7.5.jar PubnubMtGoxSample

The Concept
For me, the best solution was to convert the JSONObjects from the PubNub API to bean classes which I found in the MtGox module of the XChange library.
Admitted, this approach adds quite a bit of glue code as you can see at the end of this answer, but I think it's worth the trouble, because after the conversion, the code gets much simpler. E.g. for getting the rate and currency from the ticker at which BTC was last traded, you can simply write
ticker.getLast().getValue()
and
ticker.getLast().getCurrency()
How To Do It
The mtgox module of the XChange library is available as a maven artifact, which is very convenient. You only need to add this module as a dependency to your project and the project setup is done.
In the xchange-mtgox module, you will find the package com.xeiam.xchange.mtgox.v2.dto.marketdata with the two calsses MtGoxTrade and MtGoxTicker.
Converting from JSONObject to one of these classes is easy with the Jackson ObjectMapper. As an advantage, the Jackson library is automatically imported as a transitive dependency of the xchange-mtgox maven artifact. That means that if you're using maven, you don't even have to write a single line of code to add it to your project.
Below is a complete runnable Example. Most is standard code for using PubNub. The important bits are between the marks // BEGIN OF IMPORTANT PART and // END OF IMPORTANT PART.
public class PubNubMtGoxBeanExample {
private static final String MTGOXPUBNUB_SUBSCRIBE_KEY = "sub-c-50d56e1e-2fd9-11e3-a041-02ee2ddab7fe";
private static final String MTGOXPUBNUB_BTCEUR_CHANNEL = "0bb6da8b-f6c6-4ecf-8f0d-a544ad948c15";
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
static {
OBJECT_MAPPER.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
}
public static void main(String[] args) throws PubnubException {
Pubnub pubnub = new Pubnub("demo", MTGOXPUBNUB_SUBSCRIBE_KEY);
pubnub.subscribe(MTGOXPUBNUB_BTCEUR_CHANNEL, new Callback() {
#Override
public void successCallback(String channel, Object message) {
// BEGIN OF IMPORTANT PART
JSONObject messageJson = (JSONObject) message;
JSONObject tickerJson;
try {
tickerJson = messageJson.getJSONObject("ticker");
} catch (JSONException e) {
throw new RuntimeException(e);
}
MtGoxTicker ticker;
try {
// the following line is the most important, because it maps from the JSONObject to the MtGoxTicker class
ticker = OBJECT_MAPPER.readValue(tickerJson.toString(), MtGoxTicker.class);
} catch (JsonParseException e) {
throw new RuntimeException(e);
} catch (JsonMappingException e) {
throw new RuntimeException(e);
} catch (IOException e) {
throw new RuntimeException(e);
}
String currency = ticker.getLast().getCurrency();
BigDecimal value = ticker.getLast().getValue();
System.out.println(currency + " " + value);
// END OF IMPORTANT PART
}
#Override
public void connectCallback(String channel, Object message) {
System.out.println("connectCallback on channel:" + channel + " : " + message.getClass() + " : " + message.toString());
}
#Override
public void disconnectCallback(String channel, Object message) {
System.out.println("disconnectCallback on channel:" + channel + " : " + message.getClass() + " : " + message.toString());
}
#Override
public void reconnectCallback(String channel, Object message) {
System.out.println("reconnectCallback on channel:" + channel + " : " + message.getClass() + " : " + message.toString());
}
#Override
public void errorCallback(String channel, PubnubError error) {
System.out.println("errorCallback on channel " + channel + " : " + error.toString());
}
});
}
}
For clarity, I've removed the imports, which you can add back in with the appropriate hotkeys in most IDEs (it's Ctrl+Shift+O in Eclipse).
Morevoer, note that there is a performance penalty in this code, which can be avoided by following the answer to the question
How to efficiently map a org.json.JSONObject to a POJO?