Adding new custom properties in Kafka Connect - java

I am trying to add new custom Kafka Converter which is a modification of JsonConverterConfig in connect-json. I am trying to add some new custom property say "schemas.modifications.enable" in the converter which extends JsonConverterConfig. But Kafka connect is not able to find details about the converter.
My code Snippet :
public class ModifiedJsonConfig extends JsonConverterConfig {
public static final String SCHEMAS_MODIFY_CONFIG = "schemas.modifications.enable";
public static final boolean SCHEMAS_MODIFY_CONFIG_DEFAULT = true;
private static final String SCHEMAS_MODIFY_CONFIG_DOC = "The maximum number of schemas that can be cached in this converter instance.";
private static final String SCHEMAS_MODIFY_CONFIG_DISPLAY = "Schema Cache Size";
private final static ConfigDef CONFIG;
static {
String group = "Schemas-modification";
int orderInGroup = 0;
CONFIG = ConverterConfig.newConfigDef();
CONFIG.define(SCHEMAS_MODIFY_CONFIG, Type.BOOLEAN, SCHEMAS_MODIFY_CONFIG_DEFAULT, Importance.HIGH, SCHEMAS_MODIFY_CONFIG_DOC, group,
orderInGroup++, Width.MEDIUM, SCHEMAS_MODIFY_CONFIG_DISPLAY);
}
public static ConfigDef configDef() {
return CONFIG;
}
public ModifiedJsonConfig(Map<String, ?> props) {
super(props);
}
public boolean schemasModified() {
return getBoolean(SCHEMAS_MODIFY_CONFIG);
}
}
But I am getting the error here :
ERROR Stopping due to error (org.apache.kafka.connect.cli.ConnectDistributed:83)
org.apache.kafka.common.config.ConfigException: Unknown configuration 'schemas.modifications.enable'
But I have defined this configuration. It would be really helpful if you can help me set a custom converter property here.
Thanks in advance.

Related

Inject value into a static field

Here is my class:
public class DeduplicationErrorMetric extends AbstractErrorMetric {
public static final String DEDUPLICATIONS_METRIC = "deduplications";
public static final String KAFKA_MESSAGES_METRIC = "kafka.messages";
private static String DEDUPLICATION_TOPIC_NAME;
private static final List<Tag> DEDUPLICATION_ERROR_TAGS = List.of(Tag.of("type", "failed"));
private static final List<Tag> KAFKA_ERROR_TAGS = List.of(Tag.of("topic", DEDUPLICATION_TOPIC_NAME),
Tag.of("action", "send"), Tag.of("result", "failure"));
public DeduplicationErrorMetric() {
super(Map.of(
DEDUPLICATIONS_METRIC, DEDUPLICATION_ERROR_TAGS,
KAFKA_MESSAGES_METRIC, KAFKA_ERROR_TAGS
));
}
#Override
public void incrementMetric(String key) {
errorCounters.get(key).increment();
}
}
I have #Value("${kafka.topic.deduplication}") in my application.yml, and I need to insert the value into DEDUPLICATION_TOPIC_NAME before the bean will be created. How can I do it?
You can use the setter to do this but I'd advocate against this practice !
This means your field will be null before a first instance comes and invokes this injection point
Your static field is not final so can lead to modification, thus lead to hard to debug bugs
It will not solve your current problem as the null value will be used in this case for KAFKA_ERROR_TAGS
#Value("${kafka.topic.deduplication}")
private void setDeduplicationTopicName(String deduplicationTopicName) {
this.DEDUPLICATION_TOPIC_NAME = deducplicationTopicName;
}
Instead, maybe try to create a #Singleton bean and use #Value on its fields, then you're sure you have only one instance.
For your list, you can then use #PostConstruct to make sure it's instantiated once
What you could do here is to directly use injection from a properties file.
If it is a SpringBoot app, in you application properties set your kafka.topic.deduplication property (you can have different values for different environments).
This way, Spring will get the value while constructing the bean.
Your code could look something like this:
public class DeduplicationErrorMetric extends AbstractErrorMetric {
public static final String DEDUPLICATIONS_METRIC = "deduplications";
public static final String KAFKA_MESSAGES_METRIC = "kafka.messages";
private static final List<Tag> DEDUPLICATION_ERROR_TAGS = List.of(Tag.of("type", "failed"));
private static final List<Tag> KAFKA_ERROR_TAGS = List.of(Tag.of("topic", deduplicationTopicName),
Tag.of("action", "send"), Tag.of("result", "failure"));
#Value("${kafka.topic.deduplication}")
private String deduplicationTopicName;
public DeduplicationErrorMetric() {
super(Map.of(
DEDUPLICATIONS_METRIC, DEDUPLICATION_ERROR_TAGS,
KAFKA_MESSAGES_METRIC, KAFKA_ERROR_TAGS
));
}
#Override
public void incrementMetric(String key) {
errorCounters.get(key).increment();
}
}
Remove the keyword "static" and then you will be able to change it in the instance.
Static means that the field is locked to the class.
public class DeduplicationErrorMetric extends AbstractErrorMetric {
public static final String DEDUPLICATIONS_METRIC = "deduplications";
public static final String KAFKA_MESSAGES_METRIC = "kafka.messages";
private String DEDUPLICATION_TOPIC_NAME;
private static final List<Tag> DEDUPLICATION_ERROR_TAGS = List.of(Tag.of("type", "failed"));
private List<Tag> KAFKA_ERROR_TAGS = List.of(Tag.of("topic", DEDUPLICATION_TOPIC_NAME),
Tag.of("action", "send"), Tag.of("result", "failure"));
public DeduplicationErrorMetric() {
super(Map.of(
DEDUPLICATIONS_METRIC, DEDUPLICATION_ERROR_TAGS,
KAFKA_MESSAGES_METRIC, KAFKA_ERROR_TAGS
));
}
#Override
public void incrementMetric(String key) {
errorCounters.get(key).increment();
}
public void setTopic(String value){
DEDUPLICATION_TOPIC_NAME = value;
}
}
private void example(){
DeduplicationErrorMetric dem = new DeduplicationErrorMetric();
//Set the instance value directly
dem.DEDUPLICATION_TOPIC_NAME = "Test";
//Set via a function, potentially with other variables.
demo.setTopic("Test");
}
I would also recommend making the variable name lowercase now that it is not static, as good coding practice.

Initialisation of Spring ConfigurationProperties

I have the following class mapping parts of the properties from the application.properties:
#Component
#ConfigurationProperties(prefix = "city")
#Getter
#Setter
public class CityProperties {
private int populationAmountWorkshop;
private double productionInefficientFactor;
private Loaner loaner = new Loaner();
private Tax tax = new Tax();
private Guard pikeman = new Guard();
private Guard bowman = new Guard();
private Guard crossbowman = new Guard();
private Guard musketeer = new Guard();
#Getter
#Setter
public static class Loaner {
private int maxRequest;
private int maxAgeRequest;
private int maxNbLoans;
}
#Getter
#Setter
public static class Tax {
private double poor;
private double middle;
private double rich;
private int baseHeadTax;
private int basePropertyTax;
}
#Getter
#Setter
public static class Guard {
private int weeklySalary;
}
}
A portion of application.properties:
#City
# Amount of inhabitants to warrant the city to have one workshop
city.populationAmountWorkshop=2500
# Factor that is applied on the efficient production to get the inefficient production
city.productionInefficientFactor=0.6
# Maximum requests per loaner
city.loaner.maxRequest=6
# Maximum age of loan request in weeks
city.loaner.maxAgeRequest=4
# Maximum loan offers per loaner
city.loaner.maxNbLoans=3
# Weekly tax value factor for the various population per 100 citizens
city.tax.poor=0
city.tax.middle=0.6
city.tax.rich=2.0
city.tax.baseHeadTax=4
city.tax.basePropertyTax=280
city.pikeman.weeklySalary=3
city.bowman.weeklySalary=3
city.crossbowman.weeklySalary=4
city.musketeer.weeklySalary=6
Then this is the application for the test setup:
#SpringBootApplication
#Import({ServerTestConfiguration.class})
#ActiveProfiles("server")
#EnableConfigurationProperties
#PropertySource(value = {"application.properties", "server.properties", "bean-test.properties"})
public class SavegameTestApplication {
}
These are annotations on the ServerTestConfiguration class all other imported confugrations are the same as I use in the production case as well:
#Configuration
#EnableAutoConfiguration
#Import(value = {ClientServerInterfaceServerConfiguration.class, ServerConfiguration.class, ImageConfiguration.class})
public class ServerTestConfiguration {
...
}
And finally the constructor of my test class that initializes the Spring-Boot application:
public CityWallSerializationTest() {
SpringApplicationBuilder builder = new SpringApplicationBuilder(SavegameTestApplication.class);
DependentAnnotationConfigApplicationContext context = (DependentAnnotationConfigApplicationContext)
builder.contextClass(DependentAnnotationConfigApplicationContext.class).profiles("server").run();
setContext(context);
setClientServerEventBus((AsyncEventBus) context.getBean("clientServerEventBus"));
IConverterProvider converterProvider = context.getBean(IConverterProvider.class);
BuildProperties buildProperties = context.getBean(BuildProperties.class);
Archiver archiver = context.getBean(Archiver.class);
IDatabaseDumpAndRestore databaseService = context.getBean(IDatabaseDumpAndRestore.class);
TestableLoadAndSaveService loadAndSaveService = new TestableLoadAndSaveService(context, converterProvider,
buildProperties, archiver, databaseService);
setLoadAndSaveService(loadAndSaveService);
}
This works fine in my production code, however when I want to write some tests using a spring boot application the values are not initialized.
Printing out the CityProperties at the end of the constructor results in this output:
CityProperties(populationAmountWorkshop=0, productionInefficientFactor=0.0, loaner=CityProperties.Loaner(maxRequest=0, maxAgeRequest=0, maxNbLoans=0), tax=CityProperties.Tax(poor=0.0, middle=0.0, rich=0.0, baseHeadTax=0, basePropertyTax=0), pikeman=CityProperties.Guard(weeklySalary=0), bowman=CityProperties.Guard(weeklySalary=0), crossbowman=CityProperties.Guard(weeklySalary=0), musketeer=CityProperties.Guard(weeklySalary=0))
I would like to understand how Spring handles the initialization of these ConfigurationProperties annotated classes, how the magic happens so to speak. I want to know this in order to properly debug the application to figure out where it goes wrong.
The productive code is a JavaFX application, that makes the whole initialization a bit more complicated:
#Slf4j
#SpringBootApplication
#Import(StandaloneConfiguration.class)
#PropertySource(value = {"application.properties", "server.properties"})
public class OpenPatricianApplication extends Application implements IOpenPatricianApplicationWindow {
private StartupService startupService;
private GamePropertyUtility gamePropertyUtility;
private int width;
private int height;
private boolean fullscreen;
private Stage primaryStage;
private final AggregateEventHandler<KeyEvent> keyEventHandlerAggregate;
private final MouseClickLocationEventHandler mouseClickEventHandler;
private ApplicationContext context;
public OpenPatricianApplication() {
width = MIN_WIDTH;
height = MIN_HEIGHT;
this.fullscreen = false;
keyEventHandlerAggregate = new AggregateEventHandler<>();
CloseApplicationEventHandler closeEventHandler = new CloseApplicationEventHandler();
mouseClickEventHandler = new MouseClickLocationEventHandler();
EventHandler<KeyEvent> fullScreenEventHandler = event -> {
try {
if (event.getCode().equals(KeyCode.F) && event.isControlDown()) {
updateFullscreenMode();
}
} catch (RuntimeException e) {
log.error("Failed to switch to/from fullscreen mode", e);
}
};
EventHandler<KeyEvent> closeEventWindowKeyHandler = event -> {
if (event.getCode().equals(KeyCode.ESCAPE)) {
log.info("Pressed ESC");
context.getBean(MainGameView.class).closeEventView();
}
};
addKeyEventHandler(closeEventHandler);
addKeyEventHandler(fullScreenEventHandler);
addKeyEventHandler(closeEventWindowKeyHandler);
}
/**
* Add a key event handler to the application.
* #param eventHandler to be added.
*/
private void addKeyEventHandler(EventHandler<KeyEvent> eventHandler) {
keyEventHandlerAggregate.addEventHandler(eventHandler);
}
public static void main(String[] args) {
launch(args);
}
#Override
public void init() {
SpringApplicationBuilder builder = new SpringApplicationBuilder(OpenPatricianApplication.class);
context = builder.contextClass(DependentAnnotationConfigApplicationContext.class).profiles("standalone")
.run(getParameters().getRaw().toArray(new String[0]));
this.startupService = context.getBean(StartupService.class);
this.gamePropertyUtility = context.getBean(GamePropertyUtility.class);
if (startupService.checkVersion()) {
startupService.logEnvironment();
CommandLineArguments cmdHelper = new CommandLineArguments();
Options opts = cmdHelper.createCommandLineOptions();
CommandLine cmdLine = cmdHelper.parseCommandLine(opts, getParameters().getRaw().toArray(new String[getParameters().getRaw().size()]));
if (cmdLine.hasOption(CommandLineArguments.HELP_OPTION)){
cmdHelper.printHelp(opts);
System.exit(0);
}
if (cmdLine.hasOption(CommandLineArguments.VERSION_OPTION)) {
System.out.println("OpenPatrician version: "+OpenPatricianApplication.class.getPackage().getImplementationVersion());
System.exit(0);
}
cmdHelper.persistAsPropertyFile(cmdLine);
}
}
#Override
public void start(Stage primaryStage) {
this.primaryStage = primaryStage;
this.primaryStage.setMinWidth(MIN_WIDTH);
this.primaryStage.setMinHeight(MIN_HEIGHT);
primaryStage.getIcons().add(new Image(getClass().getResourceAsStream("/icons/trade-icon.png")));
UIFactory uiFactory = context.getBean(UIFactory.class);
uiFactory.setApplicationWindow(this);
BaseStartupScene startupS = uiFactory.getStartupScene();
Scene defaultScene = new Scene(startupS.getRoot(), width, height);
defaultScene.getStylesheets().add("/styles/font.css");
this.fullscreen = Boolean.valueOf((String) gamePropertyUtility.getProperties().get("window.fullscreen"));
startupS.setSceneChangeable(this);
defaultScene.setOnMousePressed(mouseClickEventHandler);
defaultScene.setOnKeyPressed(keyEventHandlerAggregate);
try {
CheatKeyEventListener cheatListener = context.getBean(CheatKeyEventListener.class);
if (cheatListener != null) {
addKeyEventHandler(cheatListener);
}
} catch (Exception e) {
// the cheat listener is no defined for the context.
e.printStackTrace();
}
setCursor(defaultScene);
primaryStage.setFullScreen(fullscreen);
primaryStage.setFullScreenExitHint("");
primaryStage.setTitle("OpenPatrician");
primaryStage.setScene(defaultScene);
primaryStage.show();
}
private void setCursor(Scene scene) {
URL url = getClass().getResource("/icons/64/cursor.png");
try {
Image img = new Image(url.openStream());
scene.setCursor(new ImageCursor(img));
} catch (IOException e) {
log.warn("Failed to load cursor icon from {}", url);
}
}
/**
* #see SceneChangeable#changeScene(OpenPatricianScene)
*/
#Override
public void changeScene(final OpenPatricianScene scene) {
primaryStage.getScene().setOnMousePressed(mouseClickEventHandler);
primaryStage.getScene().setOnKeyPressed(keyEventHandlerAggregate);
primaryStage.getScene().setRoot(scene.getRoot());
}
/**
* Toggle between full screen and non full screen mode.
*/
public void updateFullscreenMode() {
fullscreen = !fullscreen;
primaryStage.setFullScreen(fullscreen);
}
#Override
public double getSceneWidth() {
return primaryStage.getScene().getWidth();
}
#Override
public double getSceneHeight() {
return primaryStage.getScene().getHeight();
}
#Override
public void stop() throws Exception {
System.out.println("Stopping the UI Application");
stopUIApplicationContext();
super.stop();
}
/**
* Closing the application context for the user interface.
*/
private void stopUIApplicationContext() {
AsyncEventBus eventBus = (AsyncEventBus) context.getBean("clientServerEventBus");
eventBus.post(new GameStateChange(EGameStatusChange.SHUTDOWN));
((AbstractApplicationContext)context).close();
}
}
It seems like you are mismatching test code and production code. Let me explain:
#SpringBootApplication is intented to run your class with the main. Usually, this looks like:
#SpringBootApplication
public class MyApplication {
public static void main(String[] args] {
SpringApplication.run(MyApplication .class, args);
}
}
If you want to test an application, then your test class has to be annotated with #SpringBootTest. This annotation will automatically detect a class annotated by #SpringBootConfiguration (which is included by #SpringBootApplication).
Within your #SpringBootTest class, you can also use #Import to load other configuration classes.
#ActiveProfiles can only be used in a test context... You are not using any profile when running your "test" class, as this annotation is test purpose only. It will be if you switch to #SpringBootTest.
Still in a Test context, if you want to load properties files (other than application.properties), you have to use #TestPropertySource instead of #PropertySource.
Your "Main" test class is supposed to look something like that:
#SpringBootTest
#ActiveProfiles("server")
#Import({ServerTestConfiguration.class})
#TestPropertySource(locations = {"classpath:/server.properties, "classpath:/bean-test.properties"}})
public class SavegameTestApplication {
}
Fore more information on Spring Boot Application Test, check this: https://www.baeldung.com/spring-boot-testing#integration-testing-with-springboottest
Once you are done with test class, you should be removing your CityWallSerializationTest which is basically a re-implementation of #SpringBootTest. Please note that you can use #Autowired in your test classes as well.
The class that handles the binding of ConfigurationProperties is ConfigurationPropertiesBindingPostProcessor.
In this particular case it turned out that the only properties file loaded was the application.properties that was present on the class path from the test project instead of the application.properties that actually contains the proper key value pairs.
This can be seen at startup when running the application with -Dlogging.level.org.springframework=DEBUG and then look in the output for:
2019-12-31 10:54:49,884 [main] DEBUG o.s.b.SpringApplication : Loading source class ch.sahits.game.savegame.SavegameTestApplication
2019-12-31 10:54:49,908 [main] DEBUG o.s.b.c.c.ConfigFileApplicationListener : Loaded config file 'file:<path>/OpenPatrician/OpenPatricianModel/target/classes/application.properties' (classpath:/application.properties)
2
The second line specifies the location of the application.properties that is loaded.

Use Dropwizard configuration in a method that establishes a connection to a MongoDB database

I am coding Dropwizard micro-services that fetch data in a MongoDB database. The micro-services work fine but I'm struggling to use in my DAO the configuration coming from my Dropwizard configuration Java class. Currently I have
public class XDAO implements IXDAO {
protected DB db;
protected DBCollection collection;
/* singleton */
private static XDAO instance;
/* Get singleton */
public static synchronized XDAO getSingleton(){
if (instance == null){
instance = new XDAO();
}
return instance;
}
/* constructor */
public XDAO(){
initDatabase();
initDatabaseIndexes();
}
private void initDatabase(){
MongoClient client = null;
try {
client = new Mongo("10.126.80.192",27017);
db = client.getDB("terre");
//then some other code
}
catch (final MongoException e){
...
}
catch (UnknownHostException e){
...
}
}
}
I want to unhard-code the three arguments in these two lines :
client = new Mongo("10.126.80.192", 27017);
db = client.getDB("terre");
My MongoConfiguration Java class is :
public class MongoConfiguration extends Configuration {
#JsonProperty
#NotEmpty
public String host;
#JsonProperty
public int port = 27017;
#JsonProperty
#NotEmpty
public String db_name;
public String getMongohost() {
return host;
}
public void setMongohost(String host) {
this.host = host;
}
public int getMongoport() {
return port;
}
public void setMongoport(int port) {
this.port = port;
}
public String getDb_name() {
return db_name;
}
public void setDb_name(String db_name) {
this.db_name = db_name;
}
}
My Resource class that uses the DAO is :
#Path("/mongo")
#Produces(MediaType.APPLICATION_JSON)
public class MyResource {
private XDAO xDAO = XDAO.getSingleton();
private String mongohost;
private String db_name;
private int mongoport;
public MyResource(String db_name, String mongohost, int mongoport) {
this.db_name = db_name;
this.mongohost = mongohost;
this.mongoport = mongoport;
}
public MyResource() {
}
#GET
#Path("/findByUUID")
#Produces(value = MediaType.APPLICATION_JSON)
#Timed
public Entity findByUUID(#QueryParam("uuid") String uuid) {
return xDAO.findByUUid(uuid);
}
}
And in my application class there is
#Override
public void run(final MongoConfiguration configuration, final Environment environment) {
final MyResource resource = new MyResource(configuration.getDb_name(), configuration.getMongohost(), configuration.getMongoport());
environment.jersey().register(resource);
}
To solve my problem I tried many things. The last thing I tried was to add these four fields in my XDAO
private String mongohost;
private String db_name;
private int mongoport;
private static final MongoConfiguration configuration = new MongoConfiguration();
Coming with this piece of code in the constructor of the XDAO:
public XDAO(){
instance.mongohost = configuration.getMongohost();
instance.mongoport = configuration.getMongoport();
instance.db_name = configuration.getDb_name();
/* then like before */
initDatabase();
initDatabaseIndexes();
}
When I try this I have a null pointer exception when my initDatabase method is invoked : mongoHost and db_name are null
The problem is that you are creating a new configuration in your XDAO with private static final MongoConfiguration configuration = new MongoConfiguration(); instead of using the config from Dropwizard's run method.
When you do this, the fields host and db_name in the new configuration are null, which is why you are getting the NPE when instantiating XDAO
You need to pass the instance of MongoConfiguration that you get from Dropwizard in your application class to your XDAO, ideally when the singleton XDAO is created so it has non-null values for db_name and host
This code below part of the problem - you are creating the singleton without giving XDAO the MongoConfiguration configuration instance.
public class XDAO implements IXDAO {
//... snip
/* Get singleton */
public static synchronized XDAO getSingleton(){
if (instance == null){
instance = new XDAO(); // no configuration information is included!
}
return instance;
}
/* constructor */
public XDAO(){
initDatabase(); // this call needs db_name & host but you haven't set those yet!!
initDatabaseIndexes();
}
I recommend you modify your application class to create XDAO along the lines of this:
#Override
public void run(final MongoConfiguration configuration, final Environment environment) {
XDAO XDAOsingleton = new XDAO(configuration);
XDAO.setSingletonInstance(XDAOsingleton); // You need to create this static method.
final MyResource resource = new MyResource(configuration.getDb_name(), configuration.getMongohost(), configuration.getMongoport()); // MyResource depends on XDAO so must be created after XAO's singleton is set
environment.jersey().register(resource);
}
You may also need to take initDatabase() etc out of XDAO's constructor depending on if you keep public static synchronized XDAO getSingleton()
I also recommend you change the constructor of MyResource to public MyResource(XDAO xdao). The resource class doesn't appear to need the configuration information, and it is better to make the dependency on an XDAO explicit (you then also don't need to keep the XDAO singleton in a static field inside XDAO's class).
To get MongoDB integrated in a simple way to Dropwizard, please try and use MongoDB Managed Object. I will explain this in 3 simple steps:
Step 1: Create a simple MongoManged class:
import com.mongodb.Mongo;
import io.dropwizard.lifecycle.Managed;
public class MongoManaged implements Managed {
private Mongo mongo;
public MongoManaged(Mongo mongo) {
this.mongo = mongo;
}
#Override
public void start() throws Exception {
}
#Override
public void stop() throws Exception {
mongo.close();
}
}
Step 2: Mention MongoDB Host, Port, DB Name in a config yml file:
mongoHost : localhost
mongoPort : 27017
mongoDB : softwaredevelopercentral
Step 3: Bind everything together in the Application Class:
public class DropwizardMongoDBApplication extends Application<DropwizardMongoDBConfiguration> {
private static final Logger logger = LoggerFactory.getLogger(DropwizardMongoDBApplication.class);
public static void main(String[] args) throws Exception {
new DropwizardMongoDBApplication().run("server", args[0]);
}
#Override
public void initialize(Bootstrap<DropwizardMongoDBConfiguration> b) {
}
#Override
public void run(DropwizardMongoDBConfiguration config, Environment env)
throws Exception {
MongoClient mongoClient = new MongoClient(config.getMongoHost(), config.getMongoPort());
MongoManaged mongoManaged = new MongoManaged(mongoClient);
env.lifecycle().manage(mongoManaged);
MongoDatabase db = mongoClient.getDatabase(config.getMongoDB());
MongoCollection<Document> collection = db.getCollection(config.getCollectionName());
logger.info("Registering RESTful API resources");
env.jersey().register(new PingResource());
env.jersey().register(new EmployeeResource(collection, new MongoService()));
env.healthChecks().register("DropwizardMongoDBHealthCheck",
new DropwizardMongoDBHealthCheckResource(mongoClient));
}
}
I have used these steps and written a blog post and a sample working application code is available on GitHub. Please check: http://softwaredevelopercentral.blogspot.com/2017/09/dropwizard-mongodb-tutorial.html

OSGi reading configurations

I am trying to code a OSGi bundle which can be initiated using multiple configurations. Purpose of my bundle is to rewrite static links in html and redirect it to a CDN URL. I am using org.apache.sling.rewriter.Transformer to achieve this.
#Component(metatype = true, label = "CDN Link Rewriter", configurationFactory = true, immediate = true)
#Service(value = TransformerFactory.class)
public class LinkTransformer implements Transformer,
TransformerFactory {
#Property(label = "Static URL Extensions", value = "js,jpg,png,css,gif")
private static final String STATIC_FILES_EXTNS = "static_file_extn";
#Property(label = "Domain Path", value = "")
private static final String DOMAIN_PATH = "domain_path";
#Property(label = "CDN Url prefix", value = "")
private static final String CDN_URL_PREFIX = "cdn_url_prefix";
#Property(label = "Tags to check", value = "a,img,link,script")
private static final String TAGS_TO_CHECK = "tags_to_check";
#Property(label = "Attributes to check", d value = "src,href")
private static final String ATTRS_TO_CHECK = "attrs_to_check";
#Property(value = "append-version", propertyPrivate = true)
private static final String PIPELINE_TYPE = "pipeline.type";
#Property(value = "global", propertyPrivate = true)
private static final String PIPELINE_MODE = "pipeline.mode";
#Activate
protected void activate(final Map<String, Object> props) {
this.update(props);
}
#Modified
protected void update(final Map<String, Object> props) {
}
public LinkTransformer() {
}
#Override
public void init(org.apache.sling.rewriter.ProcessingContext context,
org.apache.sling.rewriter.ProcessingComponentConfiguration config)
throws IOException {
}
#Override
public final Transformer createTransformer() {
return new LinkTransformer();
}
//some other methods
}
Problem: I am unable to access my configurations in my bundle. I am able to create multiple sets of configurations in Felix console. But #Activate method is called only at the time of bundle installation. During Link transformation activty only init() method is being called. Hence I am unable to get hold of configurations. Can anyone tell me how to get configurations ?
The problem with above approach is implementing to different interfaces in same class. Thanks to #Balazs Zsoldos you can check the answer here
Here, All I had to do was seperately implement Transformer and TransformerFactory.
#Component(configurationFactory = true, metatype = true, policy = ConfigurationPolicy.REQUIRE, label = "CDN Link Rewriter", description = "Rewrites links to all static files to use configurable CDN")
#Service(value = TransformerFactory.class)
public class StaticLinkTransformerFactory implements TransformerFactory {
//all property declarations as in question
private Map<String, Object> map;
#Activate
void activate(Map<String, Object> map) {
this.map = map;
}
#Override
public Transformer createTransformer() {
return new StaticLinkTransformer(map);
}
}
StaticLinkTransformer can be implemented as plain java class without any component or service annotations.

Custom CacheResolver not working

I have a Spring Boot project with a custom CacheResolver as I need to decide on runtime which cache I want to use, I don't have any compilation errors but, when I do some tests and place a break point at my custom CacheResolver it never steps into it.
This is my Configuration class for the Cache:
#Configuration
#EnableCaching(proxyTargetClass = true)
#PropertySource(CacheConfig.CLASSPATH_DEPLOY_CACHE_PROPERTIES_PROPERTIES)
public class CacheConfig extends CachingConfigurerSupport{
public static final String CLASSPATH_DEPLOY_CACHE_PROPERTIES_PROPERTIES = "classpath:/deploy/cache-properties.properties";
public static final String CACHEABLE_DOCUMENTS_PROPERTY = "cacheable.documents";
public static final String TTL_CACHEABLE_DOCUMENTS_PROPERTY = "ttl.cacheable.documents";
public static final String SIZED_CACHEABLE_DOCUMENTS_PROPERTY = "sized.cacheable.documents";
public static final String CACHE_NAME = "permanentCache";
public static final String TTL_CACHE = "ttlCache";
public static final String SIZED_CACHE = "sizedCache";
public static final String CACHEABLE_DOCUMENTS = "cacheableDocuments";
public static final String SIZED_CACHEABLE_DOCUMENTS = "sizedCacheableDocuments";
public static final int WEIGHT = 1000000;
public static final int TO_KBYTES = 1000;
#Inject
protected Environment environment;
//#Bean
#Override
public CacheManager cacheManager() {
SimpleCacheManager cacheManager = new SimpleCacheManager();
GuavaCache sizedCache = new GuavaCache(SIZED_CACHE, CacheBuilder.newBuilder().maximumWeight(WEIGHT).weigher(
(key, storable) -> {
String json = ((Storable) storable).toJson();
return json.getBytes().length / TO_KBYTES;
}
).build());
GuavaCache permanentCache = new GuavaCache(CACHE_NAME,CacheBuilder.newBuilder().build());
//GuavaCache ttlCache = new GuavaCache(TTL_CACHE, CacheBuilder.newBuilder().expireAfterWrite(30, TimeUnit.MINUTES).build());
cacheManager.setCaches(Arrays.asList(permanentCache,sizedCache));
return cacheManager;
}
#Bean(name = "wgstCacheResolver")
#Override
public CacheResolver cacheResolver(){
CacheResolver cacheResolver = new WgstCacheResolver(cacheManager(),cacheableDocuments(),sizedCacheableDocuments());
return cacheResolver;
}
#Bean(name = CACHEABLE_DOCUMENTS)
public List<String> cacheableDocuments(){
String[] cacheableDocuments = StringUtils.commaDelimitedListToStringArray(environment.getProperty(CACHEABLE_DOCUMENTS_PROPERTY));
return Arrays.asList(cacheableDocuments);
}
#Bean(name = SIZED_CACHEABLE_DOCUMENTS)
public List<String> sizedCacheableDocuments(){
String[] sizedCacheableDocuments = StringUtils.commaDelimitedListToStringArray(environment.getProperty(SIZED_CACHEABLE_DOCUMENTS_PROPERTY));
return Arrays.asList(sizedCacheableDocuments);
}
}
Here is my CacheResolver
public class WgstCacheResolver extends AbstractCacheResolver {
private final List<String> cacheableDocuments;
private final List<String> sizedCacheableDocuments;
public WgstCacheResolver(final CacheManager cacheManager,final List<String> cacheableDocuments, final List<String> sizedCacheableDocuments) {
super(cacheManager);
this.cacheableDocuments = cacheableDocuments;
this.sizedCacheableDocuments = sizedCacheableDocuments;
}
/**
* Resolves the cache(s) to be updated on runtime
* #param context
* #return*/
#Override
protected Collection<String> getCacheNames(final CacheOperationInvocationContext<?> context) {
final Collection<String> cacheNames = new ArrayList<>();
final AbstractDao dao = (AbstractDao)context.getTarget();
final String documentType = dao.getDocumentType().toString();
if (cacheableDocuments.contains(documentType)){
cacheNames.add("permanentCache");
}
if (sizedCacheableDocuments.contains(documentType)){
cacheNames.add("sizedCache");
}
return cacheNames;
}
}
And here my DAO where I use the cache:
#Component
#Scope(value = ConfigurableBeanFactory.SCOPE_PROTOTYPE, proxyMode = ScopedProxyMode.DEFAULT)
#CacheConfig(cacheResolver = "wgstCacheResolver")
public class CacheableDao<T extends Storable> extends AbstractDao<T> {
private final Logger logger = LoggerFactory.getLogger(CacheableDao.class);
public CacheableDao(final Bucket bucket, final Class<T> typeParameterClass, final DocumentType documentType) {
super(bucket, typeParameterClass, documentType);
}
#Cacheable(key = "{#root.methodName, #root.target.generateFullKey(#key)}")
public T get(final String key) throws DatastoreAccessException, ObjectMappingException {
//do something
}
.
.
.
}
I have tried implementing CacheResolver instead of extending AbstractCacheResolver but it didn't make any difference.
Thank you.
Cache names need to be included at some point, just specifying the CacheResolver to use is not enough, the #Cacheable class needs to be aware of the available cache names, so I included them with the #CacheConfig annotation:
#CacheConfig(cacheNames = {WgstCacheConfig.PERMANENT_CACHE, WgstCacheConfig.SIZED_CACHE},
cacheResolver = WgstCacheConfig.WGST_CACHE_RESOLVER)
public class CacheableDao<T extends Storable> extends AbstractDao<T> {
One thing that I don't like is that I need to provide a null CacheManager, even if I'm not using it, otherwise I get the following error:
Caused by: java.lang.IllegalStateException: No CacheResolver specified, and no bean of type CacheManager found. Register a CacheManager bean or remove the #EnableCaching annotation from your configuration.
So I left it like this, and it works:
#Bean
public CacheManager cacheManager() {
return null;
}
#Bean(name = WGST_CACHE_RESOLVER)
public CacheResolver cacheResolver(){
CacheResolver cacheResolver = new WgstCacheResolver(cacheableDocuments(),sizedCacheableDocuments(),getPermanentCache(),
getSizedCache());
return cacheResolver;
}
Reran my tests, stepping through my custom CacheResolver and it is behaving as expected resolving to the correct cache(s)
My configuration class is not extending CachingConfigurerSupport anymore.
After a bit of back and forth (Sorry about that!) it turns out this is indeed a bug in Spring Framework.
I've created SPR-13081. Expect a fix for the next maintenance release (4.1.7.RELEASE). Thanks for the sample project!

Categories