We Keep Coding

JPA many to many constraint violation exception

I am using JPA to store data and faced two problems during implementation. I have two entities (Station and Commodity) that have many-to-many relationship with intermediate table so that I had to created the third one. When app receives message it converts its data to entites and should save but sometimes app throwing a ConstraintViolationException because there is null value at foreign key field referencing to Commodity entity.
I've tried simple approach: selecting needed commodity from database and saving it if there is no one. Then I started to use bulk searching all commodities of message and then putting it where are needed. None of them did a trick.
In my opinion the problem could be caused by multi-threading read\insert.
The second problem is that service stop running when exception is thrown. App can lost some of transactions that's not a big deal but it simply stops after rollback.
How can I resolve these conflicts?
Here is code of data handling class and diagram of entities :
#Service
#AllArgsConstructor
#Slf4j
public class ZeromqCommoditiesServiceImpl implements ZeromqCommoditesService {
private final CategoryTransactionHandler categoryHandler;
private final CommodityTransactionHandler commodityHandler;
private final EconomyTransactionHandler economyHandler;
private final StationTransactionHandler stationHandler;
private final SystemTransactionHandler systemHandler;
#Override
#Transactional(
isolation = Isolation.READ_COMMITTED,
propagation = Propagation.REQUIRES_NEW,
rollbackFor = Throwable.class)
#Modifying
public void saveData(ZeromqCommodityPayload payload) {
CommodityContent content = payload.getContent();
var station = stationHandler.createOrFindStation(content.getStationName());
var system = systemHandler.createOrFindSystem(content.getSystemName());
var commodityReferences = getMapOfCommodities(content);
station.setSystem(system);
updateEconomies(station, content);
updateProhibited(station, content, commodityReferences);
updateStationCommodities(station, content, commodityReferences);
try {
saveStation(station);
} catch (ConstraintViolationException | PersistentObjectException | DataAccessException e) {
log.error("Error saving commodity info \n" + content, e);
}
}
public void saveStation(StationEntity station) {
stationHandler.saveStation(station);
if (station.getId() != null) {
log.debug(String.format("Updated \"%s\" station info", station.getName()));
} else {
log.debug(String.format("Updated \"%s\" station info", station.getName()));
}
}
private void updateEconomies(StationEntity station, CommodityContent content) {
station.getEconomies().clear();
if (content.getEconomies() != null) {
var economies = content.getEconomies()
.stream()
.map(economy -> {
var stationEconomyEntity = economyHandler.createOrFindEconomy(economy.getName());
Double proportion = economy.getProportion();
stationEconomyEntity.setProportion(proportion != null ? proportion : 1.0);
return stationEconomyEntity;
})
.peek(economy -> economy.setStation(station))
.toList();
station.getEconomies().addAll(economies);
}
}
private void updateProhibited(
StationEntity station,
CommodityContent content,
Map<String, CommodityEntity> commodityEntityMap) {
station.getProhibited().clear();
if (content.getProhibited() != null) {
var prohibitedCommodityEntities = content.getProhibited()
.stream()
.map(prohibited -> {
String eddnName = prohibited.toLowerCase(Locale.ROOT);
CommodityEntity commodityReference = getCommodityEntity(commodityEntityMap, eddnName);
return new ProhibitedCommodityEntity(station, commodityReference);
}
)
.toList();
station.getProhibited().addAll(prohibitedCommodityEntities);
}
}
private void updateStationCommodities(
StationEntity station,
CommodityContent content,
Map<String, CommodityEntity> commodityEntityMap) {
station.getCommodities().clear();
if (content.getCommodities() != null) {
var commodities = content.getCommodities()
.stream()
.map(commodity -> {
CommodityEntity commodityReference = getCommodityEntity(
commodityEntityMap,
commodity.getEddnName());
return StationCommodityEntity.builder()
.commodity(commodityReference)
.buyPrice(commodity.getBuyPrice())
.sellPrice(commodity.getSellPrice())
.demand(commodity.getDemand())
.stock(commodity.getStock())
.station(station)
.build();
})
.toList();
station.getCommodities().addAll(commodities);
}
}
private CommodityEntity getCommodityEntity(Map<String, CommodityEntity> commodityEntityMap, String eddnName) {
return commodityEntityMap.get(eddnName);
}
private Map<String, CommodityEntity> getMapOfCommodities(#NotNull CommodityContent content) {
Set<String> commodities = content.getCommodities()
.stream()
.map(Commodity::getEddnName)
.collect(Collectors.toSet());
if (content.getProhibited() != null && content.getProhibited().size() > 0) {
commodities.addAll(content.getProhibited().
stream()
.map(item -> item.toLowerCase(Locale.ROOT))
.collect(Collectors.toSet()));
}
var commodityReferencesMap = commodityHandler.findAllByEddnName(commodities)
.stream()
.collect(Collectors.toMap(
CommodityEntity::getEddnName,
item -> item
));
commodities.forEach(commodity -> {
if (commodityReferencesMap.get(commodity.toLowerCase()) == null) {
CommodityCategoryEntity category = categoryHandler.createOrFindCategory("Unknown");
CommodityEntity newCommodity = new CommodityEntity(commodity, commodity, category);
CommodityEntity managedCommodity = commodityHandler.saveCommodity(newCommodity);
commodityReferencesMap.put(managedCommodity.getEddnName(), managedCommodity);
}
});
return commodityReferencesMap;
}
}
Thanks in advance

Method keeps running and sending the same information even for different request grpc

I am trying to implement GRPC and when i do so I get the correct response from the server and if I stop the server and run it again and use the other request that I implemented it works however if I try and make a second request straight after making one in from the first request I get the same response. It's like it is looping.
These are the two methods I am using from the client:
public void setSpaces(int id) {
channel =ManagedChannelBuilder.forAddress("localhost", 3000)
// Channels are secure by default (via SSL/TLS). For the example we disable TLS to avoid
// needing certificates.
.usePlaintext()
.build();
blockingStub = carParkServiceGrpc.newBlockingStub(channel);
asyncStub = carParkServiceGrpc.newStub(channel);
logger.info("Will try to get CarPark " + id + " ...");
CarParkToUpdateRequest request = CarParkToUpdateRequest.newBuilder().setDeviceId(id).build();
carParkResponse response;
try {
response = blockingStub.setSpaces(request);
}catch(StatusRuntimeException e) {
logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
return;
}finally {
channel.shutdown();
}
logger.info("Carpark: " + response.getCarPark());
spacesArea.append(response.getCarPark().toString());
}
public void setFull(int id) {
channel =ManagedChannelBuilder.forAddress("localhost", 3000)
// Channels are secure by default (via SSL/TLS). For the example we disable TLS to avoid
// needing certificates.
.usePlaintext()
.build();
blockingStub = carParkServiceGrpc.newBlockingStub(channel);
asyncStub = carParkServiceGrpc.newStub(channel);
logger.info("Will try to get CarPark " + id + " ...");
CarParkToUpdateRequest request = CarParkToUpdateRequest.newBuilder().setDeviceId(id).build();
carParkResponse response;
try {
response = blockingStub.setFull(request);
}catch(StatusRuntimeException e) {
logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
return;
}finally {
channel.shutdown();
}
logger.info("Carpark: " + response.getCarPark());
fullArea.append(response.getCarPark().toString());
}
These two methods are supposed to send a request to the server to change the status of the 'car park' so if I send a request with setFull I get a response saying the carpark is full etc.
These are the methods from the server:
public void setSpaces(CarParkToUpdateRequest request, StreamObserver<carParkResponse> rStreamObserver) {
ArrayList<CarParkOperations.proto.cp.CarPark> carList = Car.getInstance();
for(int i=0; i<carList.size(); i++) {
if(carList.get(i).getCarParkId() == request.getDeviceId()) {
CarParkOperations.proto.cp.CarPark heater_rec = (CarParkOperations.proto.cp.CarPark) carList.get(i);
Car.carparkCar.clear();
Car.carparkCar.add(CarParkOperations.proto.cp.CarPark.newBuilder().setCarParkId(heater_rec.getCarParkId()).setLocation(heater_rec.getLocation()).setStatus("Spaces").build());
}
}
for(CarParkOperations.proto.cp.CarPark heater : Car.carparkCar) {
carParkResponse response = carParkResponse.newBuilder().setCarPark(heater).build();
rStreamObserver.onNext(response);
rStreamObserver.onCompleted();
return;
}
}
public void setFull(CarParkToUpdateRequest request, StreamObserver<carParkResponse> rStreamObserver) {
ArrayList<CarParkOperations.proto.cp.CarPark> carList = Car.getInstance();
for(int i=0; i<carList.size(); i++) {
if(carList.get(i).getCarParkId() == request.getDeviceId()) {
CarParkOperations.proto.cp.CarPark heater_rec = (CarParkOperations.proto.cp.CarPark) carList.get(i);
Car.carparkCar.clear();
Car.carparkCar.add(CarParkOperations.proto.cp.CarPark.newBuilder().setCarParkId(heater_rec.getCarParkId()).setLocation(heater_rec.getLocation()).setStatus("Full").build());
}
}
for(CarParkOperations.proto.cp.CarPark heater : Car.carparkCar) {
carParkResponse response = carParkResponse.newBuilder().setCarPark(heater).build();
rStreamObserver.onNext(response);
rStreamObserver.onCompleted();
return;
}
}
I think it's most likely something to do with the server methods but cant seem to figure it out.
This is where I am storing the data:
package CarParkOperations.proto.cp;
import java.util.ArrayList;
import com.google.rpc.Status;
public class Car extends ArrayList<CarPark>{
public static Car carparkCar;
public static Car getInstance() {
if(carparkCar == null) {
carparkCar = new Car();
}
return carparkCar;
}
public Car() {
this.add(CarParkOperations.proto.cp.CarPark.newBuilder().setCarParkId(1).setStatus("Full").setLocation("Behind Building 1").build());
this.add(CarParkOperations.proto.cp.CarPark.newBuilder().setCarParkId(2).setStatus("Full").setLocation("Behind Building 1").build());
this.add(CarParkOperations.proto.cp.CarPark.newBuilder().setCarParkId(3).setStatus("Full").setLocation("Behind Building 4").build());
this.add(CarParkOperations.proto.cp.CarPark.newBuilder().setCarParkId(4).setStatus("Full").setLocation("Behind Building 3").build());
this.add(CarParkOperations.proto.cp.CarPark.newBuilder().setCarParkId(5).setStatus("Full").setLocation("Behind Building 2").build());
this.add(CarParkOperations.proto.cp.CarPark.newBuilder().setCarParkId(6).setStatus("Full").setLocation("Behind Building 1").build());
this.add(CarParkOperations.proto.cp.CarPark.newBuilder().setCarParkId(7).setStatus("Full").setLocation("Behind Building 1").build());
this.add(CarParkOperations.proto.cp.CarPark.newBuilder().setCarParkId(10).setStatus("Full").setLocation("Behind Building 6").build());
this.add(CarParkOperations.proto.cp.CarPark.newBuilder().setCarParkId(11).setStatus("Full").setLocation("Behind Building 1").build());
this.add(CarParkOperations.proto.cp.CarPark.newBuilder().setCarParkId(12).setStatus("Spaces").setLocation("Behind Building 1").build());
this.add(CarParkOperations.proto.cp.CarPark.newBuilder().setCarParkId(13).setStatus("Spaces").setLocation("Behind Building 1").build());
this.add(CarParkOperations.proto.cp.CarPark.newBuilder().setCarParkId(14).setStatus("Spaces").setLocation("Behind Building 1").build());
}
}
Any suggestions would be much appreciated.

You might need synchronize Car.getInstance() method, because without proper synchronization, if it is called by different threads it may surprisingly return different instances!
public static synchronized Car getInstance() {
if(carparkCar == null) {
carparkCar = new Car();
}
return carparkCar;
}
Also your Car class is not thread-safe because it extends ArrayList which is not thread-safe. You should let your Car class extend something like ConcurrentLinkedQueue instead, or let your Car class compose a field of list = Collections.synchronizedList(new ArrayList()) instead of extending ArrayList.

Handling dead-letter queue message-broker independent way

I have a project that currently uses Spring Cloud Streams and RabbitMQ underneath. I've implemented a logic based on the documentation. See below:
#Component
public class ReRouteDlq {
private static final String ORIGINAL_QUEUE = "so8400in.so8400";
private static final String DLQ = ORIGINAL_QUEUE + ".dlq";
private static final String PARKING_LOT = ORIGINAL_QUEUE + ".parkingLot";
private static final String X_RETRIES_HEADER = "x-retries";
private static final String X_ORIGINAL_EXCHANGE_HEADER = RepublishMessageRecoverer.X_ORIGINAL_EXCHANGE;
private static final String X_ORIGINAL_ROUTING_KEY_HEADER = RepublishMessageRecoverer.X_ORIGINAL_ROUTING_KEY;
#Autowired
private RabbitTemplate rabbitTemplate;
#RabbitListener(queues = DLQ)
public void rePublish(Message failedMessage) {
Map<String, Object> headers = failedMessage.getMessageProperties().getHeaders();
Integer retriesHeader = (Integer) headers.get(X_RETRIES_HEADER);
if (retriesHeader == null) {
retriesHeader = Integer.valueOf(0);
}
if (retriesHeader < 3) {
headers.put(X_RETRIES_HEADER, retriesHeader + 1);
String exchange = (String) headers.get(X_ORIGINAL_EXCHANGE_HEADER);
String originalRoutingKey = (String) headers.get(X_ORIGINAL_ROUTING_KEY_HEADER);
this.rabbitTemplate.send(exchange, originalRoutingKey, failedMessage);
}
else {
this.rabbitTemplate.send(PARKING_LOT, failedMessage);
}
}
#Bean
public Queue parkingLot() {
return new Queue(PARKING_LOT);
}
}
It does what it is expected, however, it is binded to RabbitMQ, and my company is planning to stop using this message broker in one year or two (don't know why, must be some crazy business). So, I want to implement the same thing, but detach it from any message broker.
I tried changing the rePublish method this way, but it does not work:
#StreamListener(Sync.DLQ)
public void rePublish(Message failedMessage) {
Map<String, Object> headers = failedMessage.getHeaders();
Integer retriesHeader = (Integer) headers.get(X_RETRIES_HEADER);
if (retriesHeader == null) {
retriesHeader = Integer.valueOf(0);
}
if (retriesHeader < 3) {
headers.put(X_RETRIES_HEADER, retriesHeader + 1);
String exchange = (String) headers.get(X_ORIGINAL_EXCHANGE_HEADER);
String originalRoutingKey = (String) headers.get(X_ORIGINAL_ROUTING_KEY_HEADER);
this.rabbitTemplate.send(exchange, originalRoutingKey, failedMessage);
}
else {
this.rabbitTemplate.send(PARKING_LOT, failedMessage);
}
}
It fails because the Message class has immutable Headers - throws exception on the put attempt saying you can't change its values (uses org.springframework.messaging.Message class).
Is there a way to implement this dead-letter queue handler in a message broker independent way?

Use
MessageBuilder.fromMessage(message)
.setHeader("foo", "bar")
...
.build();
Note that the message in #StreamListener is a spring-messaging Message<?>, not a spring-amqp Message and can't be sent using the template that way; you need an output binding to send the message to.

How to call a method that returns a list in another method and delete items in this list using iterator

Hey guys am new to this and I would appreciate any help.
I want to call getListTenant() from my save function below and clear the list using iterator before doing my save. Below is the code in my controller:
package controllers;
public class TenantController extends AppController {
Tenant tenant;
FacilityUnit unit;
// list tenants in selected facility
public Result listTenant() {
return ok(Json.toJson(getTenantList()));
}
private List<Tenant> getTenantList() {
List<Tenant> tenants = Tenant.find
.fetch("unit.facility")
.where().eq("unit.facility", currentFacility())
.findList();
return tenants;
}
public Result saveTenant() {
JsonNode submissionNode = request().body().asJson();
JsonNode itemsArray = submissionNode.get("items");
//clear tenant
// create the new tenant
if (itemsArray.isArray()) {
for (JsonNode itemNode : itemsArray) {
JsonNode tenantNode = itemNode.get("tenant");
String tenantId = tenantNode.get("id").asText();
JsonNode unitNode = itemNode.get("unit");
String unitId = unitNode.get("id").asText();
System.out.println("##### Tenant ID IS " + tenantId);
System.out.println("##### unit ID IS " + unitId);
// Tenant.find.where().eq("tenant.id",
// tenant.getTenant().getId()).eq("unit.id", unit.getId()
// ).delete();
// Util.isNotEmpty() &&
if (Util.isNotEmpty(tenantId) && Util.isNotEmpty(unitId)) {
// these two are the minimal criteria for an tenant
Tenant tenant = new Tenant();
tenant.setTenant(Person.find.byId(tenantId));
tenant.setUnit(FacilityUnit.find.byId(unitId));
tenant.save();
System.out.println("##### SAVED A TENANT");
}
}
}
System.out.println("##### DONE");
return ok(infoMessage("Update of " + tenant.getTenant() + "successful"));
}

Iterate over the Tenants from getTenantList() and call delete() on them to clear them.
For example:
private List<Tenant> getTenantList() {
List<Tenant> tenants = Tenant.find
.fetch("unit.facility")
.where().eq("unit.facility", currentFacility())
.findList();
return tenants;
}
public Result saveTenant() {
// Do something...
// ...
// Get the tenants list that we want to clear before saving.
// Best way to do this is loop over it and call delete.
for (Tenant tenant : getTenantList()) {
tenant.delete();
}
// Do some more things...
// ...
}

modifying a map while iterating over its entrySet

In the java docs of the map interface's entrySet() method I found this statement and I really do no understand it.
The set is backed by the map, so changes to the map are reflected in the set, and vice-versa. If the map is modified while an iteration over the set is in progress, the results of the iteration are undefined
what is meant by undefined here?
For more clarification, this is my situation.
I have a web application based on spring & hibernate.
Our team implemented custom caching class called CachedIntegrationClients.
We are using RabbitMQ as a messaging server.
instead of getting our clients each time we want to send a message to the server, we cache the clients using the previous caching class.
The problem is that the messages are sent to the messaging server twice.
Viewing the logs, we found that the method that get the cached clients return the client twice, although this (theoretically) impossible as we store the clients in a map, and the map does not allow duplicate keys.
After some smoke viewing of the code I found that the method that iterates over the cached clients gets a set of the clients from the cached clients map.
So I suspected that while iterating over this set, another request is made by another client and this client may be uncached, so it modifies the map.
Any way this is the CachedIntegrationClients class
public class CachedIntegrationClientServiceImpl {
private IntegrationDao integrationDao;
private IntegrationService integrationService;
Map<String, IntegrationClient> cachedIntegrationClients = null;
#Override
public void setBaseDAO(BaseDao baseDao) {
super.setBaseDAO(integrationDao);
}
#Override
public void refreshCache() {
cachedIntegrationClients = null;
}
synchronized private void putOneIntegrationClientOnCache(IntegrationClient integrationClient){
fillCachedIntegrationClients(); // only fill cache if it is null , it will never refill cache
if (! cachedIntegrationClients.containsValue(integrationClient)) {
cachedIntegrationClients.put(integrationClient.getClientSlug(),integrationClient);
}
}
/**
* only fill cache if it is null , it will never refill cache
*/
private void fillCachedIntegrationClients() {
if (cachedIntegrationClients != null) {
return ;
}
log.debug("filling cache of cachedClients");
cachedIntegrationClients = new HashMap<String, IntegrationClient>(); // initialize cache Map
List<IntegrationClient> allCachedIntegrationClients= integrationDao.getAllIntegrationClients();
if (allCachedIntegrationClients != null) {
for (IntegrationClient integrationClient : allCachedIntegrationClients) {
integrationService
.injectCssFileForIntegrationClient(integrationClient);
fetchClientServiceRelations(integrationClient
.getIntegrationClientServiceList());
}
for (IntegrationClient integrationClient : allCachedIntegrationClients) {
putOneIntegrationClientOnCache(integrationClient);
}
}
}
/**
* fetch all client service
* #param integrationClientServiceList
*/
private void fetchClientServiceRelations(
List<IntegrationClientService> integrationClientServiceList) {
for (IntegrationClientService integrationClientService : integrationClientServiceList) {
fetchClientServiceRelations(integrationClientService);
}
}
private void fetchClientServiceRelations(IntegrationClientService clientService) {
for (Exchange exchange : clientService.getExchangeList()) {
exchange.getId();
}
for (Company company : clientService.getCompanyList()) {
company.getId();
}
}
/**
* Get a client given its slug.
*
* If the client was not found, an exception will be thrown.
*
* #throws ClientNotFoundIntegrationException
* #return IntegrationClient
*/
#Override
public IntegrationClient getIntegrationClient(String clientSlug) throws ClientNotFoundIntegrationException {
if (cachedIntegrationClients == null) {
fillCachedIntegrationClients();
}
if (!cachedIntegrationClients.containsKey(clientSlug)) {
IntegrationClient integrationClient = integrationDao.getIntegrationClient(clientSlug);
if (integrationClient != null) {
this.fetchClientServiceRelations(integrationClient.getIntegrationClientServiceList());
integrationService.injectCssFileForIntegrationClient(integrationClient);
cachedIntegrationClients.put(clientSlug, integrationClient);
}
}
IntegrationClient client = cachedIntegrationClients.get(clientSlug);
if (client == null) {
throw ClientNotFoundIntegrationException.forClientSlug(clientSlug);
}
return client;
}
public void setIntegrationDao(IntegrationDao integrationDao) {
this.integrationDao = integrationDao;
}
public IntegrationDao getIntegrationDao() {
return integrationDao;
}
public Map<String, IntegrationClient> getCachedIntegrationClients() {
if (cachedIntegrationClients == null) {
fillCachedIntegrationClients();
}
return cachedIntegrationClients;
}
public IntegrationService getIntegrationService() {
return integrationService;
}
public void setIntegrationService(IntegrationService integrationService) {
this.integrationService = integrationService;
}
}
and here is the method that iterates over the set
public List<IntegrationClientService> getIntegrationClientServicesForService(IntegrationServiceModel service) {
List<IntegrationClientService> integrationClientServices = new ArrayList<IntegrationClientService>();
for (Entry<String, IntegrationClient> entry : cachedIntegrationClientService.getCachedIntegrationClients().entrySet()) {
IntegrationClientService integrationClientService = getIntegrationClientService(entry.getValue(), service);
if (integrationClientService != null) {
integrationClientServices.add(integrationClientService);
}
}
return integrationClientServices;
}
Also here is the method that calls the previous one
List<IntegrationClientService> clients = integrationService.getIntegrationClientServicesForService(service);
System.out.println(clients.size());
if (clients.size() > 0) {
log.info("Inbound service message [" + messageType.getKey() + "] to be sent to " + clients.size()
+ " registered clients: [" + StringUtils.arrayToDelimitedString(clients.toArray(), ", ") + "]");
for (IntegrationClientService integrationClientService : clients) {
Message<T> message = integrationMessageBuilder.build(messageType, payload, integrationClientService);
try {
channel.send(message);
} catch (RuntimeException e) {
messagingIntegrationService.handleException(e, messageType, integrationClientService, payload);
}
}
} else {
log.info("Inbound service message [" + messageType.getKey() + "] but no registered clients, not taking any further action.");
}
and here is the logs that appears on the server
BaseIntegrationGateway.createAndSendToSubscribers(65) | Inbound service message [news.create] to be sent to 3 registered clients: [Id=126, Service=IntegrationService.MESSAGE_NEWS, Client=MDC, Id=125, Service=IntegrationService.MESSAGE_NEWS, Client=CNBC, Id=125, Service=IntegrationService.MESSAGE_NEWS, Client=CNBC]

Undefined means there is no requirement for any specific behavior. The implementation is free to start WWIII, re-hang all your toilet rolls by the overhand method, sully your grandmother, etc.
The only permitted modification with a specified behaviour is via the iterator.

Have you looked at java.concurrent.ConcurrentHashMap?
EDIT: I looked over your code again this stikes me as odd:
In fillCachedIntegrationClients() you have the following loop:
for (IntegrationClient integrationClient : allCachedIntegrationClients) {
putOneIntegrationClientOnCache(integrationClient);
}
But the putOneIntegrationClientOnCache method itself directly calls fillCachedIntegrationClients();
synchronized private void putOneIntegrationClientOnCache(IntegrationClient integrationClient){
fillCachedIntegrationClients(); // only fill cache if it is null , it will never refill cache
...
}
Something there must go wrong. You are calling fillCachedIntegrationClients() twice. Actually if I am not mistaken this should actually be a never-ending loop since one method calls the other and vice versa. The != null condition is never met during the initialization. Ofcourse you are modifying and iterating in a undefined way so maybe that saves you from an infinite loop.