We Keep Coding

Why does Hazelcast not show entry in the Map when added via CachePut/Cacheable

I have a use case where by the #CachePut annotation adds an entry to the cache, and I have to retrieve it manually (via code).
I can see that the total backup count gives me 1 as the number of entries, but all the maps give me the size as 0. So, I am not sure what I am doing wrong.
Here's my code
HazelcastConfig.java
#Configuration
public class HazelcastConfig {
#Bean
public Config hazelcastConf() {
Config c = new Config()
.setInstanceName("hazelcast-instance")
.addMapConfig(
new MapConfig()
.setName("testmap")
.setEvictionConfig(
new EvictionConfig()
.setEvictionPolicy(EvictionPolicy.LRU)
.setMaxSizePolicy(MaxSizePolicy.PER_NODE)
.setSize(1000)
)
.setTimeToLiveSeconds(500000)
);
c.getNetworkConfig().getRestApiConfig().setEnabled(true);
c.getNetworkConfig().getRestApiConfig().enableGroups(RestEndpointGroup.DATA);
return c;
}
}
TestServiceImpl.java
#Service
public class TestServiceImpl implements TestService {
#Autowired
#Lazy
private RestTemplate restTemplate;
#Override
#CachePut(value = "testmap", key="1")
public String getId() {
System.out.println("--------------------------");
System.out.println("-------INSIDE getId-------");
String id = null;
CBObject obj = restTemplate.getForObject("http://localhost:3000/testCB", CBObject.class);
if (null != obj && null != obj.getId()) {
id = String.valueOf(obj.getId());
}
System.out.println("-------- EXIT getId-------");
System.out.println("--------------------------");
return id;
}
#Bean
public RestTemplate restTemplate() {
return new RestTemplate();
}
}
TestController.java
#RestController
#RequestMapping("/v1")
public class TestController {
#Autowired
private TestService testService;
#GetMapping("/testCB")
public ResponseEntity<?> doCB() {
Map<String, String> resp = new HashMap<>();
String id = testService.getId();
if (null != id) {
resp.put("id", id);
}
Config config = new HazelcastConfig().hazelcastConf();
System.out.println(config.getMapConfig("testmap").getTotalBackupCount()); // 1
HazelcastInstance hz = Hazelcast.getHazelcastInstanceByName(config.getInstanceName());
System.out.println(hz.getReplicatedMap("testmap").size()); // 0
System.out.println(hz.getMap("testmap").size()); // 0
System.out.println(hz.getMultiMap("testmap").size()); // 0
return ResponseEntity.status(HttpStatus.ACCEPTED).body(resp);
}
}

Have you explicitly enabled caching using the #EnableCaching annotation (Ref Doc, Javadoc)?
Also, see the guidance from Spring Boot in the Ref Doc on Caching if you are using Spring Boot.
Furthermore, when using Spring Boot, you can either add the command-line switch --debug to your launch command or set the debug property to true in Spring Boot application.properties to get output from the Auto-configuration that has been applied. In particular you will want to see that the CacheAutoConfiguration class has been processed.
If you are NOT using Spring Boot, then in addition to the #EnableCaching annotation, you will also need to explicitly declare a CacheManager bean, such as:
#Bean
HazelcastCacheManager cacheManager(HazelcastInstance hazelcaseInstance) {
return new HazelcastCacheManager(hazelcastInstance);
}
This will require the com.hazelcast:hazelcast-spring JAR dependency on your runtime classpath.
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast-spring</artifactId>
<version>${hazelcast.version}</version>
<scope>runtime</scope>
</dependency>
For example.
NOTE: Do you not confuse the HazelcastCacheManager Spring Cache Abstraction CacheManager implementation, which requires the hazelcast-spring JAR and is required by Spring's Cache Abstraction either with or without Spring Boot, with Hazelcast's standard HazelcastCacheManager. These 2 classes are not the same thing.
Alternatively, you could also use Hazelcast as a JCache caching provider implementation in either Spring Framework or Spring Boot. The core Spring Framework offers support for using JCache as well. When using Spring Boot, you will need to specify the JCache cache provider type for Hazelcast (i.e. Embedded or Client/Server). I will leave this as an exercise for you to figure out.
Lastly, I recently built an example for my own testing purposes using Hazelcast as a caching provider in Spring Framework's Cache Abstraction using Spring Boot, if you would like to take a look.
Hope this helps!
Cheers!

KafkaController required a bean of type 'org.springframework.kafka.requestreply.ReplyingKafkaTemplate' that could not be found?

I am trying to use the RepliyngKafkaTemplate like I managed to use the KafkaTemplate in a REST controller.
#RestController
public class TestController {
#Autowired
private ReplyingKafkaTemplate<Object, KafkaExampleRecord, KafkaExampleRecord> replyingTemplate;
#PostMapping("/test/request")
public void requestReply(#RequestBody KafkaExampleRecord record) throws ExecutionException, InterruptedException, TimeoutException {
ProducerRecord<Object, KafkaExampleRecord> producerRecord = new ProducerRecord<>("mytopic", record);
RequestReplyFuture<Object, KafkaExampleRecord, KafkaExampleRecord> replyFuture = replyingTemplate.sendAndReceive(producerRecord);
SendResult<Object, KafkaExampleRecord> sendResult = replyFuture.getSendFuture().get(10, TimeUnit.SECONDS);
ConsumerRecord<Object, KafkaExampleRecord> consumerRecord = replyFuture.get(10, TimeUnit.SECONDS);
}
}
However I am getting the following exception.
Field replyingTemplate in com.blah.KafkaController required a bean of type 'org.springframework.kafka.requestreply.ReplyingKafkaTemplate' that could not be found.
I enabled auto configuration like this.
#Configuration
#EnableKafka
public class KafkaConfig {
}
All Kafka settings are in my application.yml.
What else do I need? Do I really have to define beans? Seems unnecessary.

Do I really have to define beans? Seems unnecessary.
Yes, you have to declare a beans for the replying template (including the reply container); Spring Boot only auto configures a simple KafkaTemplate.

Can you check, whether you are scanning the basePackages correctly. Sometimes, you may end-up with this issue, if you not scanning the packages correctly, and I have experienced this many times in the Spring Boot application.
#ComponentScan(
basePackages = {
"x.x.x.x"
}
)

How to set tableName dynamically using environment variable in spring boot?

I am using AWS ECS to host my application and using DynamoDB for all database operations. So I'll have same database with different table names for different environments. Such as "dev_users" (for Dev env), "test_users" (for Test env), etc.. (This is how our company uses same Dynamo account for different environments)
So I would like to change the "tableName" of the model class using the environment variable passed through "AWS ECS task definition" environment parameters.
For Example.
My Model Class is:
#DynamoDBTable(tableName = "dev_users")
public class User {
Now I need to replace the "dev" with "test" when I deploy my container in test environment. I know I can use
#Value("${DOCKER_ENV:dev}")
to access environment variables. But I'm not sure how to use variables outside the class. Is there any way that I can use the docker env variable to select my table prefix?
My Intent is to use like this:
I know this not possible like this. But is there any other way or work around for this?
Edit 1:
I am working on the Rahul's answer and facing some issues. Before writing the issues, I'll explain the process I followed.
Process:
I have created the beans in my config class (com.myapp.users.config).
As I don't have repositories, I have given my Model class package name as "basePackage" path. (Please check the image)
For 1) I have replaced the "table name over-rider bean injection" to avoid the error.
For 2) I printed the name that is passing on to this method. But it is Null. So checking all the possible ways to pass the value here.
Check the image for error:
I haven't changed anything in my user model class as beans will replace the name of the DynamoDBTable when the beans got executed. But the table name over riding is happening. Data is pulling from the table name given at the Model Class level only.
What I am missing here?

The table names can be altered via an altered DynamoDBMapperConfig bean.
For your case where you have to Prefix each table with a literal, you can add the bean as such. Here the prefix can be the environment name in your case.
#Bean
public TableNameOverride tableNameOverrider() {
String prefix = ... // Use #Value to inject values via Spring or use any logic to define the table prefix
return TableNameOverride.withTableNamePrefix(prefix);
}
For more details check out the complete details here:
https://github.com/derjust/spring-data-dynamodb/wiki/Alter-table-name-during-runtime

I am able to achieve table names prefixed with active profile name.
First added TableNameResolver class as below,
#Component
public class TableNameResolver extends DynamoDBMapperConfig.DefaultTableNameResolver {
private String envProfile;
public TableNameResolver() {}
public TableNameResolver(String envProfile) {
this.envProfile=envProfile;
}
#Override
public String getTableName(Class<?> clazz, DynamoDBMapperConfig config) {
String stageName = envProfile.concat("_");
String rawTableName = super.getTableName(clazz, config);
return stageName.concat(rawTableName);
}
}
Then i setup DynamoDBMapper bean as below,
#Bean
#Primary
public DynamoDBMapper dynamoDBMapper(AmazonDynamoDB amazonDynamoDB) {
DynamoDBMapper mapper = new DynamoDBMapper(amazonDynamoDB,new DynamoDBMapperConfig.Builder().withTableNameResolver(new TableNameResolver(envProfile)).build());
return mapper;
}
Added variable envProfile which is an active profile property value accessed from application.properties file.
#Value("${spring.profiles.active}")
private String envProfile;

We have the same issue with regards to the need to change table names during runtime. We are using Spring-data-dynamodb 5.0.2 and the following configuration seems to provide the solutions that we need.
First I annotated my bean accessor
#EnableDynamoDBRepositories(dynamoDBMapperConfigRef = "getDynamoDBMapperConfig", basePackages = "my.company.base.package")
I also setup an environment variable called ENV_PREFIX which is Spring wired via SpEL.
#Value("#{systemProperties['ENV_PREFIX']}")
private String envPrefix;
Then I setup a TableNameOverride bean:
#Bean
public DynamoDBMapperConfig.TableNameOverride getTableNameOverride() {
return DynamoDBMapperConfig.TableNameOverride.withTableNamePrefix(envPrefix);
}
Finally, I setup the DynamoDBMapperConfig bean using TableNameOverride injection. In 5.0.2, we had to setup a standard DynamoDBTypeConverterFactory in the DynamoDBMapperConfig builder to avoid NPE.:
#Bean
public DynamoDBMapperConfig getDynamoDBMapperConfig(DynamoDBMapperConfig.TableNameOverride tableNameOverride) {
DynamoDBMapperConfig.Builder builder = new DynamoDBMapperConfig.Builder();
builder.setTableNameOverride(tableNameOverride);
builder.setTypeConverterFactory(DynamoDBTypeConverterFactory.standard());
return builder.build();
}
In hind sight, I could have setup a DynamoDBTypeConverterFactory bean that returns a standard DynamoDBTypeConverterFactory and inject that into the getDynamoDBMapperConfig() method using the DynamoDBMapperConfig builder. But this will also do the job.

I up voted the other answer but here is an idea:
Create a base class with all your user details:
#MappedSuperclass
public abstract class AbstractUser {
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
private Long id;
private String firstName;
private String lastName;
Create 2 implentations with different table names and spirng profiles:
#Profile(value= {"dev","default"})
#Entity(name = "dev_user")
public class DevUser extends AbstractUser {
}
#Profile(value= {"prod"})
#Entity(name = "prod_user")
public class ProdUser extends AbstractUser {
}
Create a single JPA respository that uses the mapped super classs
public interface UserRepository extends CrudRepository<AbstractUser, Long> {
}
Then switch the implentation with the spring profile
#RunWith(SpringJUnit4ClassRunner.class)
#DataJpaTest
#Transactional
public class UserRepositoryTest {
#Autowired
protected DataSource dataSource;
#BeforeClass
public static void setUp() {
System.setProperty("spring.profiles.active", "prod");
}
#Test
public void test1() throws Exception {
DatabaseMetaData metaData = dataSource.getConnection().getMetaData();
ResultSet tables = metaData.getTables(null, null, "PROD_USER", new String[] { "TABLE" });
tables.next();
assertEquals("PROD_USER", tables.getString("TABLE_NAME"));
}
}

How to run the same SpringBootTests for different applications

I have a SpringBoot multimodule application, something like that:
core
customer1 -> depends on core
customer2 -> depends on core
I want to write integration tests for both, but I don't want to duplicate my core test code. Now I have an abstract class with SpringBootTest(classes = Customer1Application.class) and a lot of test classes, mostly testing the core functionality.
#ContextConfiguration
#SpringBootTest(classes = Customer1Application.class)
#AutoConfigureMockMvc
public abstract class AbstractSpringBootTest
{
#Autowired
protected MockMvc mockMvc;
#Autowired
protected Validator validator;
...
}
I want to check if the changes in Customer2 application break something in core functionality, so I want to run these tests with #SpringBootTest(classes = Customer2Application.class) annotation.
How is it possible to configure the application class in the annotation? Is there a way to run the tests with my other application context without manually changing the annotation or duplicating all the steps?

I don't know if it will work, but I would try removing #SpringBootTest from AbstractSpringBootTest and then defining two test classes as follows:
#SpringBootTest(classes = Customer1Application.class)
class Customer1ApplicationSpringBootTest extends AbstractSpringBootTest {}
#SpringBootTest(classes = Customer2Application.class)
class Customer2ApplicationSpringBootTest extends AbstractSpringBootTest {}
EDIT:
So I dug around Spring Boot sources and came up with this solution.
Essentially to be able to use system property or property file to configure which #SpringBootApplication is supposed to be tested you need to copy the source of class org.springframework.boot.test.context.SpringBootConfigurationFinder to your own test source root and the edit method private Class<?> scanPackage(String source) to look something like this (you do not have to use Lombok of course):
private Class<?> scanPackage(String source) {
while (!source.isEmpty()) {
val components = this.scanner.findCandidateComponents(source);
val testConfig = System.getProperties();
val testConfigFile = "test-config.properties";
val applicationClassConfigKey = "main.application.class";
try {
testConfig.load(this.getClass().getResourceAsStream("/" + testConfigFile));
} catch (IOException e) {
logger.error("Error reading configuration file: {}, using default algorithm", testConfigFile);
}
if (testConfig.containsKey(applicationClassConfigKey)) {
if (!components.isEmpty() && testConfig.containsKey(applicationClassConfigKey) && testConfig.getProperty(applicationClassConfigKey) != null) {
boolean found = false;
val configClassName = testConfig.getProperty(applicationClassConfigKey);
for (BeanDefinition component: components) {
if (configClassName.equals(component.getBeanClassName())) {
found = true;
break;
}
}
Assert.state(found,
() -> "Found multiple #SpringBootConfiguration annotated classes "
+ components + ", none of which are of type " + configClassName);
return ClassUtils.resolveClassName(
configClassName, null);
}
} else {
if (!components.isEmpty()) {
Assert.state(components.size() == 1,
() -> "Found multiple #SpringBootConfiguration annotated classes "
+ components);
return ClassUtils.resolveClassName(
components.iterator().next().getBeanClassName(), null);
}
}
source = getParentPackage(source);
}
return null;
}
Check the link for the entire project.

Did you check?
#SpringBootTest(classes = {Customer1Application.class, Customer2Application.class})

Spring Boot Application Error Page

I'm starting to learn Spring Boot, and I'm following a tutorial on youtube. However, there is an weird thing happening on my project. I just created a Controller called GreetingController. Below is the complete code of the class
#RestController
#EnableAutoConfiguration
public class GreetingController {
private static BigInteger nextId;
private static Map<BigInteger, Greeting> greetingMap;
private static Greeting save(Greeting greeting) {
if (greetingMap == null) {
greetingMap = new HashMap<BigInteger, Greeting>();
nextId = BigInteger.ONE;
}
greeting.setId(nextId);
nextId = nextId.add(BigInteger.ONE);
greetingMap.put(greeting.getId(), greeting);
return greeting;
}
static {
Greeting g1 = new Greeting();
g1.setText("Hello World");
save(g1);
Greeting g2 = new Greeting();
g2.setText("Hola Mundo");
save(g2);
}
#RequestMapping(value = "/api/greetings", method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<Collection<Greeting>> getGreetings() {
Collection<Greeting> greetings = greetingMap.values();
return new ResponseEntity<Collection<Greeting>>(greetings,
HttpStatus.OK);
}
}
The controller is under the following package:
However, when I bootstrap the application with the URL http://localhost:8080/api/greetings the following error appears on my page:
But, when I put the GreetingController in the same package of Application class, as the image below:
And then get the same URL http://localhost:8080/api/greetings, I got the right response:
Can anyone explain me why?

Rename your com.example package to org.example. Spring boot scans for controllers all subpackages of package of class when you place your #SpringBootApplication annotation.
Or put #ComponentScan("org.example") on the same class. This way you tell spring boot where to search your controllers (and other beans).

If you want to support having a controller in another package, you need to include it in the component scan of your Application.
In Application.java, you could add the following:
#ComponentScan({com.example, org.example})
By default, the package that Application is in will be included in the ComponentScan, which is why it was working for you when you had the controller in the same package as Application.
Also, you don't need the #EnableAutoConfiguration annotation on your controller, FYI.