We Keep Coding

What does the spring.cloud.stream.source really do?

I am trying to understand how the new functional model of Spring Cloud Streams works and how the configuration actually works under the hood.
One of the properties I am unable to figure out is spring.cloud.stream.source.
What does this property actually signify ?
I could not understand the documentation :
Note that preceding example does not have any source functions defined
(e.g., Supplier bean) leaving the framework with no trigger to create
source bindings, which would be typical for cases where configuration
contains function beans. So to trigger the creation of source binding
we use spring.cloud.stream.source property where you can declare the
name of your sources. The provided name will be used as a trigger to
create a source binding.
What if I did not need a Supplier ?
What exactly is a source binding and why is it important ?
What if I only wanted to produce to a messaging topic ? Would I still need this property ?
I also could not understand how it is used in the sample here.

Spring cloud stream looks for java.util Function<?, ?, Consumer<?>, Supplier<?> beans and creates bindings for them.
In the supplier case, the framework polls the supplier (each second by default) and sends the resulting data.
For example
#Bean
public Supplier<String> output() {
return () -> "foo";
}
spring.cloud.stream.bindings.output-out-0.destination=bar
will send foo to destination bar each second.
But, what if you don't need a polled source, but you want to configure a binding to which you can send arbitrary data. Enter spring.cloud.stream.source.
spring.cloud.stream.source=output
spring.cloud.stream.bindings.output-out-0.destination=bar
allows you to send arbitrary data to the stream bridge
bridge.send("output-out-0", "test");
In other words, it allows you to configure one or more ouput bindings that you can use in the StreamBridge; otherwise, when you send to the bridge, the binding is created dynamically.

What replaces FileResolver in the JasperReports API?

One of our software utilities uses a class that implements net.sf.jasperreports.engine.util.FileResolver to load report elements (like images for example) that reside at a path relative to the report or that are to be loaded via a proprietary file server protocol. As of the latest version, 6.6.0, I see that the plan is to remove the FileResolver class entirely. However, in the Javadocs, it only notes that the class will be removed. No details about a replacement are specified.
I am not expecting to be able to trade out the FileResolver with another class with a 1:1 substitution, but would really like to know what the report filler is now using to locate external report elements.

FileResolver was deprecated in favor of net.sf.jasperreports.repo.RepositoryService implementations.
There's a builtin implementation named net.sf.jasperreports.repo.FileRepositoryService which is roughly the equivalent of the deprecated net.sf.jasperreports.engine.util.SimpleFileResolver.
Repository services are registered as JasperReportsContext extensions.
That can be done either in a jasperreports_extension.properties file like this:
net.sf.jasperreports.extension.registry.factory.file.repository=net.sf.jasperreports.repo.FileRepositoryServiceExtensionsRegistryFactory
net.sf.jasperreports.extension.file.repository.root=/path/to/repository
net.sf.jasperreports.extension.registry.factory.persistence=net.sf.jasperreports.repo.FileRepositoryExtensionsRegistryFactory
Registering the extensions can also be done by programmatically creating a JasperReportsContext object and then using it to fill the reports:
SimpleJasperReportsContext context = new SimpleJasperReportsContext();
FileRepositoryService fileRepository = new FileRepositoryService(context, "/path/to/repository", false);
context.setExtensions(RepositoryService.class, Collections.singletonList(fileRepository));
context.setExtensions(PersistenceServiceFactory.class, Collections.singletonList(FileRepositoryPersistenceServiceFactory.getInstance()));
JasperPrint jasperPrint = JasperFillManager.getInstance(context).fill(jasperReport, params);
If you need to implement a custom repository service, you can take FileRepositoryService as a reference. You'll probably want to implement StreamRepositoryService and register PersistenceServices (as in FileRepositoryPersistenceServiceFactory).
If what you need to do is about resource paths relative to the report, you can also take a look at the JRFiller methods that take a JasperReportSource argument. Passing such an object is meant to automatically resolve report resource references as relative to the report (provided that the repository service implements resource lookup based on RepositoryContext).

Apache Spark - passing configuration to nodes

I have an application where I am using Apache Spark 1.4.1 - Standalone cluster. Code from this application has evolved and it's quite complicated (more than a few lines of code as we see in most Apache Spark examples), with lots of method calls from one class to another.
I am trying to add code that when encounters a problem with data (while processing it on the cluster nodes) it notifies an external application. For contacting the external application we have connection details setup in a config file. I want to pass somehow the connection details to the cluster nodes but passing them as parameters to each method that runs on nodes (as parameters or broadcast variable) is not ok for my application because it means that each and every method has to pass them and we've got lots of "chained method calls" (method A calls B, B calls C.....Y calls Z) which is different from most Apache Spark example where we see only one or two method calls.
I am trying to workaround this problem - is there a way to pass data to nodes besides method parameters and broadcast variables ? For example I was looking to setup a env property that would point to the config file (using System.setProperty) and to set it on all nodes, so that I can read connection details on the fly and the code would isolated in one block of code only, but I've got no luck so far.

Actually after some hours of investigation I found a way that really suits my needs. There are two spark properties (one for driver, one for executors) that can be used for passing parameters that can be then read using System.getProperty() :
spark.executor.extraJavaOptions
spark.driver.extraJavaOptions
Using them is more simpler than the approach suggested in above post and you could easily make your application to switch configuration from one environment to another (e.g QA/DEV vs PROD) when you've got all environment setup in your project.
They can be set in the SparkConf object when you're initializing the SparkContext.
The post that helped me a lot in figuring the solution is : http://progexc.blogspot.co.uk/2014/12/spark-configuration-mess-solved.html

The properties you provide as part of --properties-file will be loaded at runtime and will be available only as part of driver but not on any of the executors. But you can always make it available to the executors.
Simple hack:
private static String getPropertyString(String key, Boolean mandatory){
String value=sparkConf.get(key,null);
if(mandatory && value == null ){
value = sparkConf.getenv(key);
if(value == null)
shutDown(key); // Or whatever action you would like to take
}
if(value !=null && sparkConf.getenv(key)==null )
sparkConf.setExecutorEnv(key,value);
return value;
}
First time when your driver kicks, it will find all the properties provided from properties file from sparkconf. As soon as it finds, check whether that key already present in environment if not set those values to executors using setExecutorEnv in your program.
Its tough to distinguish whether your program is in driver or in executor so check whether the property exists in sparkconf if not then check it against environment using getenv(key).

I suggest the following solution:
Put the configuration in a database.
Put the database connection details in a JOCL (Java Object configuration Language) file and have this file available on the class path of each executors.
Make a singleton class that reads the DB connection details from the JOCL, connects to the database, extracts the configuration info and exposes it as getter methods.
Import the class into the context where you have your Spark calls and use it to access the configuration from within them.

Open a File within an EJB

I know that it is in general not allowed to access local Files (with java.io) within an EJB: nevertheless, I have an EJB which has to open an Excel File and edit it using the apache.poi library. If I do something like this:
#Stateless
public class MyEJB {
public void editExcel(){
...
InputStream in = MyEJB.class.getClassLoader().getResourceAsStream("/xls/ExcelFile.xls");
final Workbook generatedExcel = new XLSTransformer().transformXLS(in, beans);
...
} }
The filesystem structure:
MyEar.ear
--my-ejb.jar
---com
-----company
-------ejbs
----------MyEJB.class
---xls
-----ExcelFile.xls
In this case I will get an Instance of ZipFile$ZipFileInputStream (private inner class of ZipFileInputStream) and XLSTransformer will throw an IllegalArgumentException("Your InputStream was neither an OLE2 stream, nor an OOXML stream"), as it expects an InputStream representing ExcelFile.xls and gets instead a stream representing the whole my-ejb.jar.
My questions: do you know how to solve this situation? What is the best practice for accessing file within EJBs?
Thanks a lot!

I also had to return a modified XLS template out of a EE container once. Though I always look at such resources not as a part of the WAR/EAR deployment but as a configurable entity managed by customers.
So, one simple solution might be to save your template in a configuration directory, and provide it's path & name trough JNDI to your application.
BUT
You might also use JNDI for representing a more complex datatype representing an excel file, so your EJB code stays free of direct file operations.
You may find some pointers (for JBoss JNDI AS) here: http://middlewaremagic.com/jboss/?p=1690

Java app architecture - need a name for logging, want to keep classes uncoupled

I'm using a third party software library with a log prototype like this:
runtime.getInstance().log(int logtype, String moduleName, String logtext);
I have a utility library that I want to be library independent, but I also want to be able to log things to the software package from my own classes. This is fine and good, as the text messages are pretty universal, things like "you've passed bad data!" and "blah blah was successful!" Additionally, I've already wrapped the software vendor's logging functionality, so I'm not even worried about conforming to some random API.
What I am worried about (why I'm writing this post) is that there are going to be various different modules throughout my system. So the problem is like:
ModuleFoo extends com.thirdpartyvendor.BaseModule
ModuleBar extends com.thirdpartyvendor.BaseModule
ModuleFoo ---contains instance of---> IndependentDataStructure ---tries to write a log entry to my WrappedLogger ---> but data structure doesn't have a reference to ModuleFoo.
ModuleBar ---contains instance of---> IndependentDataStructure ---tries to write a log entry to my WrappedLogger ---> but data structure doesn't have a reference to ModuleBar.
Currently my system passes a field String moduleName around which quite frankly makes me sick... but I want the log entries to tell me what my module is! How can the logger know whether the IndependentDataStructure instance is working with ModuleFoo and not ModuleBar (or some other module) without IndependentDataStructure containing a reference to a BaseModule (or a String moduleName)?

Logging APIs such as Log4J and SLF4J have the concept of a diagnostic context, a way to store various bits of contextual information in a ThreadLocal map which the log message formatters can access to decorate the messages. Typical uses for this are things like putting the name of the currently authenticated user into log messages in a web application (using a servlet filter to store the username in the MDC for each request), would you be able to use a similar concept in your system?

runtime.getInstance().log(int logtype,
this.getClass().getSimpleName(),
String logtext);
If i've got it right...
EDIT: and for automated (but a bit slowly) auto-method name:
Thread.currentThread().getStackTrace()[level].getMethodName();
(where level is an integer specifies the number of classes through the log request is passed)