We Keep Coding

How do I set up a #KafkaListener to always reset on startup?

I have a kafka topic which receives messages like
Kafka key: serverId
value: server host name
I just need the latest ones for each serverId so it's a very small segment size and log compaction is enabled. So what I do is the following
#KafkaListener(topics = SERVERS_KAFKA_TOPIC, id = "#{T(java.util.UUID).randomUUID().toString()}",
properties = {
"spring.json.key.default.type=java.lang.String",
"spring.json.value.default.type=java.lang.String"
})
public void registerServer(
#Payload(required = false) String serverHostName
) {
Basically I create a new consumer group per listener because I am going by the assumption that if it is the same consumer group, it will continue where the last one left off and the state across a replica set will not be what I am expecting.
Of course it sort of makes the list of consumer groups very annoying. I was wondering short of creating another shared service like Redis to store the data since I can't really store using Redis as the data that I build from the message is a GRPC client, can I configure the listener to always start from the beginning and ignore the other consumers in the group?

Make your listener bean extend AbstractConsumerSeekAware and in onPartitionsAssigned, call seekToBeginning().
/**
* Seek all assigned partitions to the beginning.
* #since 2.6
*/
public void seekToBeginning() {
https://docs.spring.io/spring-kafka/docs/current/reference/html/
How to implement ConsumerSeekAware in Spring-kafka?
Is there any way to get the eldest available offset for a kafka topic

The seekToBeginning() method that gets provided by AbstractConsumerSeekAware didn't seek from beginning when I tried. However, this code did work
/**
* Reset to the beginning
*
* <p>{#inheritDoc}
*/
#Override
public void onPartitionsAssigned(
#NotNull Map<TopicPartition, Long> assignments,
#NotNull ConsumerSeekCallback callback) {
assignments.keySet().stream()
.filter(topicPartition -> SERVERS_KAFKA_TOPIC.equals(topicPartition.topic()))
.forEach(
topicPartition ->
callback.seekToBeginning(
topicPartition.topic(), topicPartition.partition()));
}
Also I needed it for a specific topic not for ALL topics as such the approach I found also limits it to a specific topic.

How to change message type detection strategy?

I’m using a custom RabbitMQ client in application A and Spring Amqp Library in application B.
I’ve faced with a problem: How to decide witch #RabbitListener to use for different message types?
Problem:
I send custom messages from app A to app B. In a message I set custom header “type” - it’s not a default property and not a default Spring Amqp Header (which is “_ _ TypeId _ _”) - just a new custom header.
In application B (spring amqp) I have to decide which listener to use. As I understood, Spring Amqp uses “_ _ TypeId _ _” as a default mechanism of “message type detection strategy” (I don’t know how to call it properly), but I wanna use my own “strategy”.
I’ve found the next trick, but it seems quite strange and not obvious:
private void determineMessageType(Message message) {
MessageProperties props = message.getMessageProperties();
Map<String, Object> headers = props.getHeaders();
final String type = String.valueOf(headers.get("type"));
if ("popularity-report".equals(type)) {
props.getHeaders().put("__TypeId__",
PopularityReportCommand.class.getName());
}
}
Can I use custom type detection strategy for Spring Amqp application somehow? Or how to solve these the problem properly in Spring Amqp?

The mentioned _ _ TypeId _ _ is used only for JSON message converters. So, if that really a case for you in the part what you really send from that producer, then you can take a look into the AbstractJackson2MessageConverter.setJavaTypeMapper(Jackson2JavaTypeMapper). The default one DefaultJackson2JavaTypeMapper has a property like this:
public String getClassIdFieldName() {
return DEFAULT_CLASSID_FIELD_NAME;
}
Which really is that mentioned above name:
public static final String DEFAULT_CLASSID_FIELD_NAME = "__TypeId__";
so, if you are able to extend this DefaultJackson2JavaTypeMapper and override that getter for your custom header mapper, then that Jackson2JsonMessageConverter can convert properly incoming JSON data into a desired type presented by your custom header. Then #RabbitListener would accept the value.
But you still need to be sure that typePrecedence on that mapper is set to TYPE_ID:
/**
* Set the precedence for evaluating type information in message properties.
* When using {#code #RabbitListener} at the method level, the framework attempts
* to determine the target type for payload conversion from the method signature.
* If so, this type is provided in the
* {#link MessageProperties#getInferredArgumentType() inferredArgumentType}
* message property.
* <p>
* By default, if the type is concrete (not abstract, not an interface), this will
* be used ahead of type information provided in the {#code __TypeId__} and
* associated headers provided by the sender.
* <p>
* If you wish to force the use of the {#code __TypeId__} and associated headers
* (such as when the actual type is a subclass of the method argument type),
* set the precedence to {#link Jackson2JavaTypeMapper.TypePrecedence#TYPE_ID}.
*
* #param typePrecedence the precedence.
* #since 1.6
*/
public void setTypePrecedence(TypePrecedence typePrecedence) {
Otherwise it consults the #RabbitListener method signature.
See more info in docs: https://docs.spring.io/spring-amqp/docs/current/reference/html/#Jackson2JsonMessageConverter-from-message

A RESTful api, multiple types of user access problems?

A RESTful api, multiple types of user access problems? Api as follows:
/ articles / 1 get It's means that getting an article;
/**
* 获得文章
*
* #param authorId
* #param articleId
* #return
*/
#GetMapping("/articles/{articleId}")
public ArticleEntity getArticle(#CurrentAuthor AuthorEntity author,
#PathVariable("articleId") Integer articleId) {
// get one article for author
ArticleEntity article = articleService.get(author.getId(), articleId);
// get one article for visitor
// ArtilceEntity article = articleVisitorService.get(articleId);
return article;
}
For article viewers, you can only get articles that have a status of published.
For the author of the article, you can get a variety of status (draft, published, recyclable) article.
In the program I used the custom annotation #Authorization and the Spring MVC interceptor to specify which controller methods are logged in to access, obviously this method is not suitable for this way.
If you use the above api, how to distinguish between the request which belongs to the user?
For this api, how to use login authentication?
If the design is unreasonable api, then please give me some suggestions?
thank you very much!

Flushing the bean cache

I am working with a system that uses EJB 2. The system consists of two separate applications, one is for user management and the other is the actual application containing business logic.
In the business logic application I have a bean managed entity bean that represents a User.
The application reads information from the user management database, but cannot modify it.
Whenever a user is modified in the user management application, the business logic application is notified that the user has changed. This is implemented as a call to the business application to remove the bean, which causes Weblogic to remove it from the cache and to "delete" it (which does nothing - see code for ejbRemove below). The next time the business application needs the user it will re-load it from the database.
We use the following code to invalidate a single user:
try
{
UserHome home = (UserAuthHome) getHome("User", UserHome.class);
User ua = home.findByPrimaryKey(user);
ua.remove(); // This will remove a single cached User bean in the business logic application
}
catch ...
This works fine, but sometimes (epsecially when doing development) I need to invalidate all cached User beans in the business application. I would like to do this programatically - starting the management console takes too long. There are too many users to do a call for every user.
Possible solutions could include:
--Accessing the bean cache and get a list of the cached User beans.
--Telling WLS to scrap all items in the current User bean cache and re-read them from the database.
Unfortunately I don't know how to do either of these.
I tried to search for a solution, but my internet search karma didn't find anything useful.
Additional information:
Persistance:
<persistence-type>Bean</persistence-type>
<reentrant>false</reentrant>
Caching:
<entity-descriptor>
<entity-cache>
<max-beans-in-cache>500</max-beans-in-cache>
<concurrency-strategy>Exclusive</concurrency-strategy>
<cache-between-transactions>true</cache-between-transactions>
</entity-cache>
<persistence></persistence>
</entity-descriptor>
Bean Code (in the business application):
public void ejbLoad()
{
thisLogger().entering(getUser(m_ctx), "ejbLoad()");
// Here comes some code that connects to the user database and fetches the bean data.
...
}
public void ejbRemove()
{
// This method does nothing
}
public void ejbStore()
{
// This method does nothing
}
public void ejbPostCreate()
{
// This method is empty
}
/**
* Required by EJB spec.
* <p>
* This method always throws CreateException since this entity is read only.
* The remote reference should be obtained by calling ejbFindByPrimaryKey().
*
* #return
* #exception CreateException
* Always thrown
*/
public String ejbCreate()
throws CreateException
{
throw new CreateException("This entity should be called via ejbFindByPrimaryKey()");
}

I did som additional research and was able to find a solution to my problem.
I was able to use weblogic.ejb.CachingHome.invalidateAll(). However, to do so I had to change the concurrency strategy of my bean to ReadOnly. Apparantly, Exclusive concurrency won't make the home interface implement weblogic.ejb.CachingHome:
<entity-descriptor>
<entity-cache>
<max-beans-in-cache>500</max-beans-in-cache>
<read-timeout-seconds>0</read-timeout-seconds>
<concurrency-strategy>ReadOnly</concurrency-strategy>
<cache-between-transactions>true</cache-between-transactions>
</entity-cache>
<persistence></persistence>
</entity-descriptor>
And finally, the code for my new function invalidateAllUsers:
public void invalidateAllUsers() {
logger.entering(getUser(ctx), "invalidateAll()"); // getUser returns a string with the current user ( context.getCallerPrincipal().getName() ).
try {
UserHome home = (UserAuthHome) getHome("User", UserHome.class); // Looks up the home interface
CachingHome cache = (CachingHome)home; // Sweet weblogic magic!
cache.invalidateAll();
} catch (RemoteException e) {
logger.severe(getUser(ctx), "invalidate(date)", "got RemoteException", e);
throw new EJBException(e);
}
}

How to create a custom User-Provider for symfony2?

I'm new to symfony and php5 (I'm Java, Spring, Grails developer).
I'm working on a project which has a java-middleware and a php frontend with symfony 2.
The java middleware stores the users and everything my application needs.
I don't want symfony2 to have it's own database. All of the information symfony2 needs comes from the java-middleware via WSDL and my php-soap-api that I can include into my symfony2 project.
The users need to login in the frontend. So I have to write login and logout functionality.
The java-middleware supplys a login method that I can call in php over the php-soap-api.
How should I implement the login/logout functionality in symfony2? Should I implement a custom User-provider which calls the php-soap-api? If yes, how can I do this? ( http://symfony.com/doc/2.0/cookbook/security/custom_provider.html ) isn't available.

Have a look at the User Provider Interface documentation...I think one way is build your own implementation of the interface, which will act as a wrapper for WSDL calls, and then properly setup your security context (security.yml) to use it.
I've got a similar problem, and I'm trying to build my own User Provider as well.

Yes, you will need create your own user provider and add it as a service your Symfony app. The user provider class you create must implement the UserProviderInterface
To get your class started here is an example of my custom class located in Namespace/Bundle/Security/Provider.php:
namespace CB\WebsiteBundle\Security;
use Symfony\Component\Security\Core\User\UserInterface;
use Symfony\Component\Security\Core\User\UserProviderInterface;
use Symfony\Component\Security\Core\Exception\UsernameNotFoundException;
use CB\WebsiteBundle\Entity\User;
class Provider implements UserProviderInterface {
protected $user;
public function __contsruct (UserInterface $user) {
$this->user = $user;
}
/**
* Loads the user for the given username.
*
* This method must throw UsernameNotFoundException if the user is not
* found.
*
* #throws UsernameNotFoundException if the user is not found
* #param string $username The username
*
* #return UserInterface
*/
function loadUserByUsername($username) {
$user = User::find(array('username'=>$username));
if(empty($user)){
throw new UsernameNotFoundException('Could not find user. Sorry!');
}
$this->user = $user;
return $user;
}
/**
* Refreshes the user for the account interface.
*
* It is up to the implementation if it decides to reload the user data
* from the database, or if it simply merges the passed User into the
* identity map of an entity manager.
*
* #throws UnsupportedUserException if the account is not supported
* #param UserInterface $user
*
* #return UserInterface
*/
function refreshUser(UserInterface $user) {
return $user;
}
/**
* Whether this provider supports the given user class
*
* #param string $class
*
* #return Boolean
*/
function supportsClass($class) {
return $class === 'MC\WebsiteBundle\Entity\User';
}
}
Some key things to note in this class is that it uses a custom User entity that you would define yourself which would facilitate the connection to your Java middleware. In my case, I am connecting to a REST api to get my user information and my user class uses the User::find($criteria) static function to find users by username.
Once you have your own User class that interacts with your middleware, and you have your new provider, you need to add the provider as a service in your bundle configuration: YourBundle/Resources/config.xml:
<parameters>
<parameter key="cb_security_user.class">CB\WebsiteBundle\Entity\User</parameter>
<parameter key="cb_security_provider.class">CB\WebsiteBundle\Security\Provider</parameter>
</parameters>
<services>
<service id="cb_security_user" class="%cb_security_user.class%" />
<service id="cb_security_provider" class="%cb_security_provider.class%">
<argument type="service" id="cb_security_user" />
</service>
</services>
You can find more details on my blog post here: Custom User Providers in Symfony2
Hope this helps!