We Keep Coding

JPA many to many constraint violation exception

I am using JPA to store data and faced two problems during implementation. I have two entities (Station and Commodity) that have many-to-many relationship with intermediate table so that I had to created the third one. When app receives message it converts its data to entites and should save but sometimes app throwing a ConstraintViolationException because there is null value at foreign key field referencing to Commodity entity.
I've tried simple approach: selecting needed commodity from database and saving it if there is no one. Then I started to use bulk searching all commodities of message and then putting it where are needed. None of them did a trick.
In my opinion the problem could be caused by multi-threading read\insert.
The second problem is that service stop running when exception is thrown. App can lost some of transactions that's not a big deal but it simply stops after rollback.
How can I resolve these conflicts?
Here is code of data handling class and diagram of entities :
#Service
#AllArgsConstructor
#Slf4j
public class ZeromqCommoditiesServiceImpl implements ZeromqCommoditesService {
private final CategoryTransactionHandler categoryHandler;
private final CommodityTransactionHandler commodityHandler;
private final EconomyTransactionHandler economyHandler;
private final StationTransactionHandler stationHandler;
private final SystemTransactionHandler systemHandler;
#Override
#Transactional(
isolation = Isolation.READ_COMMITTED,
propagation = Propagation.REQUIRES_NEW,
rollbackFor = Throwable.class)
#Modifying
public void saveData(ZeromqCommodityPayload payload) {
CommodityContent content = payload.getContent();
var station = stationHandler.createOrFindStation(content.getStationName());
var system = systemHandler.createOrFindSystem(content.getSystemName());
var commodityReferences = getMapOfCommodities(content);
station.setSystem(system);
updateEconomies(station, content);
updateProhibited(station, content, commodityReferences);
updateStationCommodities(station, content, commodityReferences);
try {
saveStation(station);
} catch (ConstraintViolationException | PersistentObjectException | DataAccessException e) {
log.error("Error saving commodity info \n" + content, e);
}
}
public void saveStation(StationEntity station) {
stationHandler.saveStation(station);
if (station.getId() != null) {
log.debug(String.format("Updated \"%s\" station info", station.getName()));
} else {
log.debug(String.format("Updated \"%s\" station info", station.getName()));
}
}
private void updateEconomies(StationEntity station, CommodityContent content) {
station.getEconomies().clear();
if (content.getEconomies() != null) {
var economies = content.getEconomies()
.stream()
.map(economy -> {
var stationEconomyEntity = economyHandler.createOrFindEconomy(economy.getName());
Double proportion = economy.getProportion();
stationEconomyEntity.setProportion(proportion != null ? proportion : 1.0);
return stationEconomyEntity;
})
.peek(economy -> economy.setStation(station))
.toList();
station.getEconomies().addAll(economies);
}
}
private void updateProhibited(
StationEntity station,
CommodityContent content,
Map<String, CommodityEntity> commodityEntityMap) {
station.getProhibited().clear();
if (content.getProhibited() != null) {
var prohibitedCommodityEntities = content.getProhibited()
.stream()
.map(prohibited -> {
String eddnName = prohibited.toLowerCase(Locale.ROOT);
CommodityEntity commodityReference = getCommodityEntity(commodityEntityMap, eddnName);
return new ProhibitedCommodityEntity(station, commodityReference);
}
)
.toList();
station.getProhibited().addAll(prohibitedCommodityEntities);
}
}
private void updateStationCommodities(
StationEntity station,
CommodityContent content,
Map<String, CommodityEntity> commodityEntityMap) {
station.getCommodities().clear();
if (content.getCommodities() != null) {
var commodities = content.getCommodities()
.stream()
.map(commodity -> {
CommodityEntity commodityReference = getCommodityEntity(
commodityEntityMap,
commodity.getEddnName());
return StationCommodityEntity.builder()
.commodity(commodityReference)
.buyPrice(commodity.getBuyPrice())
.sellPrice(commodity.getSellPrice())
.demand(commodity.getDemand())
.stock(commodity.getStock())
.station(station)
.build();
})
.toList();
station.getCommodities().addAll(commodities);
}
}
private CommodityEntity getCommodityEntity(Map<String, CommodityEntity> commodityEntityMap, String eddnName) {
return commodityEntityMap.get(eddnName);
}
private Map<String, CommodityEntity> getMapOfCommodities(#NotNull CommodityContent content) {
Set<String> commodities = content.getCommodities()
.stream()
.map(Commodity::getEddnName)
.collect(Collectors.toSet());
if (content.getProhibited() != null && content.getProhibited().size() > 0) {
commodities.addAll(content.getProhibited().
stream()
.map(item -> item.toLowerCase(Locale.ROOT))
.collect(Collectors.toSet()));
}
var commodityReferencesMap = commodityHandler.findAllByEddnName(commodities)
.stream()
.collect(Collectors.toMap(
CommodityEntity::getEddnName,
item -> item
));
commodities.forEach(commodity -> {
if (commodityReferencesMap.get(commodity.toLowerCase()) == null) {
CommodityCategoryEntity category = categoryHandler.createOrFindCategory("Unknown");
CommodityEntity newCommodity = new CommodityEntity(commodity, commodity, category);
CommodityEntity managedCommodity = commodityHandler.saveCommodity(newCommodity);
commodityReferencesMap.put(managedCommodity.getEddnName(), managedCommodity);
}
});
return commodityReferencesMap;
}
}
Thanks in advance

Modify HashMap when CompletableFuture is finished

I have a multi-module system, where one module handles my database storage. This is the method which saves a document:
public CompletableFuture<?> runTransaction() {
return CompletableFuture.runAsync(() -> {
TransactionBody txnBody = (TransactionBody<String>) () -> {
MongoCollection<Document> collection = transaction.getDatabase().getCollection(transaction.getCollection().toString());
collection.insertOne(session, Document.parse(json));
return "Completed";
};
try {
session.withTransaction(txnBody);
} catch (MongoException ex) {
throw new UncheckedMongoException(ex);
}
});
}
the json instance is passed down in the object constructor. However, since this will be used by several modules, with each individual caching system, I'm trying to figure out how the caller can modify data structure, if this method completed without any errors.
For example
public void createClan(Transaction transaction, int id, int maxPlayers) {
MongoTransaction mongoTransaction = (MongoTransaction) transaction;
Clan clan = new Clan(id, maxPlayers);
String json = gson.toJson(clan);
TransactionExecutor executor = new MongoTransactionExecutor(mongoTransaction, json);
executor.runTransaction(); //Returns the completableFuture instance generated by the method. Modify hashmap here.
}
I've tried reading the docs, however it was a bit confusing, any help is appreciated!

As given in the comments, two options can be considered.
First option is to convert the async nature into sync nature using CompletableFuture#get. This way, the code execution is in a blocking context.
public void createClan(Transaction transaction, int id, int maxPlayers) {
MongoTransaction mongoTransaction = (MongoTransaction) transaction;
Clan clan = new Clan(id, maxPlayers);
String json = gson.toJson(clan);
TransactionExecutor executor = new MongoTransactionExecutor(mongoTransaction, json);
try {
Object obj = executor.runTransaction().get();
// HashMap update here
} catch(Exception e) {
//handle exceptions
}
}
Second option is to keep the async nature as is and chain using thenRun (there are many then options available). This way is more a non-blocking context.
public void createClan(Transaction transaction, int id, int maxPlayers) {
MongoTransaction mongoTransaction = (MongoTransaction) transaction;
final Clan clan = new Clan(id, maxPlayers);
String json = gson.toJson(clan);
TransactionExecutor executor = new MongoTransactionExecutor(mongoTransaction, json);
try {
executor.runTransaction().thenRun(() -> updateHashMap(clan));
} catch(Exception e) {
//handle exceptions
}
}

How to do paged request - Spring boot

in my system (Spring boot project) I need to make a request to every 350 people that I search for my data, I need to page and go sending. I looked for a lot of ways to do it and found a lot of it with JPA but I'm using Jooq, so I asked for help with the user's tool and they guided me to use the options of limit and offset.
This is the method where I do the research, I set up my DTO and in the end I return the list of people.
public static ArrayList getAllPeople(Connection connection) {
ArrayList<peopleDto> peopleList = new ArrayList<>();
DSLContext ctx = null;
peopleDto peopleDto;
try {
ctx = DSL.using(connection, SQLDialect.MYSQL);
Result<Record> result = ctx.select()
.from(people)
.orderBy(people.GNUM)
.offset(0)
.limit(350)
.fetch();
for (Record r : result) {
peopleDto = new peopleDto();
peopleDto.setpeopleID(r.getValue(people.GNUM));
peopleDto.setName(r.get(people.SNAME));
peopleDto.setRM(r.get(people.SRM));
peopleDto.setRG(r.get(people.SRG));
peopleDto.setCertidaoLivro(r.get(people.SCERT));
peopleDto.setCertidaoDistrito(r.get(people.SCERTD));
peopleList.add(peopleDto);
}
} catch (Exception e) {
log.error(e.toString());
} finally {
if (ctx != null) {
ctx.close();
}
}
return peopleList;
}
This search without the limitations returns 1,400 people.
The question is how do I send up the limit number then return to this method to continue where I left off last until I reach the total value of records?

Feed your method with a Pageable parameter and return a Page from your method. Something along the lines of ...
public static ArrayList getAllPeople(Connection connection, Pageable pageable) {
ArrayList<peopleDto> peopleList = new ArrayList<>();
DSLContext ctx = null;
peopleDto peopleDto;
try {
ctx = DSL.using(connection, SQLDialect.MYSQL);
Result<Record> result = ctx.select()
.from(people)
.orderBy(people.GNUM)
.offset(pageable.getOffset())
.limit(pageable.getPageSize())
.fetch();
for (Record r : result) {
peopleDto = new peopleDto();
peopleDto.setpeopleID(r.getValue(people.GNUM));
peopleDto.setName(r.get(people.SNAME));
peopleDto.setRM(r.get(people.SRM));
peopleDto.setRG(r.get(people.SRG));
peopleDto.setCertidaoLivro(r.get(people.SCERT));
peopleDto.setCertidaoDistrito(r.get(people.SCERTD));
peopleList.add(peopleDto);
}
} catch (Exception e) {
log.error(e.toString());
} finally {
if (ctx != null) {
ctx.close();
}
}
return new PageImpl(peopleList, pageable, hereyoushouldQueryTheTotalItemCount());
}
Now you can do something with those 350 Users. With the help of the page you can now iterate over the remaining people:
if(page.hasNext())
getAllPeople(connection, page.nextPageable());
Inspired by this article Sorting and Pagination with Spring and Jooq

Trying to insert the Element into ArrayList but, instead of inserting its updating with the current value

I have a below piece of code where I fetch the json data and pass it to the another method. Data will be keep changing on daily basis. Here, I want to retain my old Data, but somehow I am unable to do it.
Code to save the value:
json = getAllHistory(settings.getRapidView(),sprint.getId(),
settings.getCredentials(),settings.getBaseUrl());
List<History> historyList = new ArrayList<>();
Double completedIssues = ClientUtil.getJsonValue(json,sprint.getId(),"completedIssues");
Double allIssues = ClientUtil.getJsonValue(json,sprint.getId(),"allIssues");
Double remainingIssues = completedIssues-allIssues;
if (remainingIssues > 0) {
History history = new History();
history.setMiliseconds(ZonedDateTime.now().toInstant().toEpochMilli());
history.setCompletedIssues(completedIssues);
history.setAllIssues(allIssues);
history.setRemainingIssues(remainingIssues);
historyList.add(history);
sprintdata.gethistory().addAll(historyList);
sprintdata.setHistory(historyList);
}
Code to make the Rest call:
public static String getAllHistory(String rapidView, Long sprintId, String base64Credentials,String baseUrl) {
try
{
String query = String.format(GET_URL_DATA, rapidView, sprintId);
query=baseUrl+query;
HttpEntity<String> entity = new HttpEntity<String>(getHeader(base64Credentials));
RestTemplate restTemplate = new RestTemplate();
ResponseEntity<String> result = restTemplate.exchange(query, HttpMethod.GET, entity, String.class);
String outputJson= result.getBody();
return outputJson;
}
catch (Exception e)
{
// TODO: handle exception
return null;
}
}
Code to get the specific Json value:
public static Double getJsonValue(String json, Long sprintId, String field) {
try{
return new GsonBuilder().
create().
fromJson(json, JsonObject.class).
getAsJsonObject("contents").
getAsJsonObject(field).
get("value").
getAsDouble();
}
catch(Exception ex)
{
return null;
}
}
I can't find the error on my own, so please help me.
I apologize for my mistake.

A list of one element is created.
Then to the old history list of sprintdata: all items of the new list are added (1):
sprintdata.gethistory().addAll(historyList);
Then the old sprintdata history list is replaced with the new one of 1 element:
sprintdata.setHistory(historyList);
So the sole thing to do would be: add one element to the old history list.
sprintdata.gethistory().add(history);

Spring MVC, attempting to edit but creating new objects

I have an MVC app that is creating new offices instead of updating them on when using an edit form. Please help me understand why this is happening.
Search method that populates the search results:
#RequestMapping(value = "/searchResults", method = RequestMethod.POST)
public ModelAndView search(#RequestParam String searchCriteria, HttpServletRequest request) {
List<Office> offices = officeServiceImpl.search(searchCriteria);
return new ModelAndView("searchResults", "offices", offices);
}
Here's what the link to the edit form looks like on the search results page:
Edit Office
Here is the Controller's edit GET method that populates the form with the existing Office:
#RequestMapping(value = "/{officeId}/edit", method = RequestMethod.GET)
#Transactional(noRollbackFor=NoResultException.class)
public ModelAndView initUpdateOfficeForm(
#PathVariable("officeId") Long officeId, Model model) {
Office office = officeServiceImpl.find(officeId);
//prepareEditFormModelAndView(office) just converts some objects to strings for typeahead form population
return prepareEditFormModelAndView(office);
}
Here is the edit POST method:
#RequestMapping(value = "/{officeId}/edit", method = RequestMethod.POST)
public ModelAndView processUpdateOfficeForm(#ModelAttribute("office") #Valid Office office,
BindingResult result, SessionStatus status) {
if (! "united states of america".equals(office.getFolderStrings().toLowerCase())) {
//This portion of code converts the typeahead strings to objects
result = tryCountries(office, result);
result = tryDepartments(office, result);
result = tryEmployees(office, result);
}
if (result.hasErrors()) {
return prepareEditFormModelAndView(office);
} else {
officeServiceImpl.save(office);
status.setComplete();
return new ModelAndView("editResult", "office", office);
}
}
officeServiceImpl calls officeRepositoryImpl method save which looks like:
#Override
public Office save(Office office) {
em.merge(office);
em.flush();
return office;
}
Thanks
Edit: Adding prepareEditFormModelAndView(office), This method attempts to build strings from associated objects:
#Transactional(noRollbackFor={NoResultException.class, IndexOutOfBoundsException.class})
private ModelAndView prepareEditFormModelAndView(Office office) {
String departmentStrings = "";
String employeeStrings = "";
List<OOM> officeOOMs = new ArrayList<OOM>();
StringBuilder sb = new StringBuilder();
try {
officeOOMs = oomServiceImpl.getOOMsForCurrentOffice(office.getId());
} catch (NoResultException e) {
officeOOMs = null;
}
for (OOM o : officeOOMs) {
try {
Employee tempEmployee = employeeServiceImpl.find(o
.getEmployeeId());
sb.append(tempEmployee.getDisplayName() + ", ");
} catch (NoResultException e) {
sb.append("Not found in system");
}
}
employeeStrings = sb.toString();
if ((! "".equals(office.getDepartmentStringsOnForm())) && office.getDepartmentStringsOnForm() != null) {
departmentStrings = office.getDepartmentStringsOnForm();
}
String folderStrings = "";
try {
folderStrings = kmlFolderServiceImpl.getInternationalOfficeString(office.getId());
LOGGER.info("Folder Strings: " + folderStrings);
} catch (NoResultException e) {
folderStrings = "";
LOGGER.info("Folder Strings: " + "no result");
}
boolean isInternational = office.isInternational();
ModelAndView result = new ModelAndView("editOfficeForm", "office", office);
result.addObject("departmentStrings", departmentStrings);
result.addObject("isInternational", isInternational);
result.addObject("folderStrings", folderStrings);
result.addObject("employeeStrings", employeeStrings);
return result;
}

I am adding a previous comment here, for better clarification. According to the OP the following fixes the problem:
When the ID is not in the form then when the model is posted back no ID is set to the entity making the persistence provider believe it is new entity.
Therefor the most obvious solution is to post the ID of the entity as well in the save operation (probably using a hidden field).
Another solution would be to try to load the entity in the database based on some business key
to see if the entity is new or not.