I wrote a Java program that parses JSON to the list of objects, and then I'm simply persisting them into the database using JpaRepository.
The problem with this kind of solution is that I've no Ids from JSON so those are each time unique when I'm parsing data and then I'm saving duplicates.
How can I avoid that?
public void getTestApi() {
ObjectMapper mapper = new ObjectMapper();
mapper.findAndRegisterModules();
try {
URL url = new URL("https://olxdata.azurewebsites.net/olx?city=wszystkie&dataType=mieszkania");
String json = getJsonFromURL(url);
PortalData portalData = mapper.readValue(json, PortalData.class);
List<OlxItem> olxItems = portalData.getOlx();
List<OtodomItem> otodomItems = portalData.getOtodom();
List<OlxDataAzureItem> olxDataAzureItems = new ArrayList<>();
for (var item : olxItems) {
olxDataAzureItems.add(new OlxDataAzureItem("olx", item.getDate(), item.getToRent(), item.getToSell()));
}
for (var item : otodomItems) {
olxDataAzureItems.add(new OlxDataAzureItem("otodom", item.getDate(), item.getToRent(), item.getToSell()));
}
olxDataAzureRepository.saveAll(olxDataAzureItems);
} catch (Exception exception) {
exception.printStackTrace();
}
}
#Entity
#Table(name = "olx_data")
public class OlxDataAzureItem {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private UUID id;
#Column(name = "portal")
private String portal;
#Column(name = "date")
private LocalDateTime date;
#Column(name = "toRent")
private int toRent;
#Column(name = "toSell")
private int toSell;
public OlxDataAzureItem() {
}
public OlxDataAzureItem(String portal, LocalDateTime date, int toRent, int toSell) {
this.portal = portal;
this.date = date;
this.toRent = toRent;
this.toSell = toSell;
}
}
Related
I created a method in my API that receives a list of objects as it's parameter to add and validate multiple objects.
But, I'm thinking about potential validation errors, what are the best approaches to make the response includes errors for any object that errors can occurs in it.
for example:
If 3 of the objects doesn't have phone numbers or one object doesn't have an SSN, how can I trigger these errors and display them in my response?
myDTO:
public class InvoiceDTO {
#NotNull(message = "PersonId can't be null")
#PersonId
private String personId;
#NotNull(message = "invoiceDate can't be null")
// #DateTimeFormat(iso = DateTimeFormatter.ofPattern("yyyy-MM-dd"))
#PastOrPresent
#JsonFormat(pattern = "yyyy-MM-dd")
private LocalDate invoiceDate;
#NotNull(message = "invoiceNumber can't be null")
private String invoiceNumber;
#NotNull(message = "phone number can't be null")
private Long phoneNumber;
#NotNull(message = "invoiceAmount can't be null")
private Double invoiceAmount; //how much does the client ows the org
#NotNull(message = "invoiceNumber can't be null")
private String accountNumber;
private int msdin; //Type of subscription
}
Controller:
#PostMapping("/invoices")
private ResponseEntity<Response> addNewInvoices(#RequestBody #Valid List<InvoiceDTO> invoiceDTO) {
//Todo find a way to through exceptions for every object if validations errors occurs
ResponseEntity<Response> response = null;
for (InvoiceDTO invoices : invoiceDTO) {
Optional<Organization> organization = this.organizationService.getOrganizationFromRequest();
Optional<Madeen> madeen = madeenRepository.findByPersonId(invoices.getPersonId());
if (!madeen.isPresent()) {
Madeen newMadeen = new Madeen();
newMadeen.setCreatedDate(LocalDate.now());
newMadeen.setPersonId(invoices.getPersonId());
newMadeen.setStatus(Status.NOT_MADEEN);
newMadeen.setMadeenTimeTracker(null);
newMadeen.setOrganization(organization.get());
madeenRepository.save(newMadeen);
}
log.info("Add new Invoice : {}", invoiceDTO);
String tcn = UUID.randomUUID().toString();
AuditLog auditLog = new AuditLog();
auditLog.setTcn(tcn);
auditLog.setRequestBody(invoiceDTO.toString());
response = null;
//Add the new amount of the invoice to an existing debts
Optional<Organization> organization1 = this.organizationService.getOrganizationFromRequest();
Optional<Debts> debts = debtsRepository.findDebtsByPersonIdAndOrganization_id(invoices.getPersonId(), organization1.get().getId());
Optional<Madeen> madeenOptional = madeenRepository.findByPersonId(invoices.getPersonId());
if (!debts.isPresent()) {
Debts newDebt = new Debts();
newDebt.setPersonId(invoices.getPersonId());
newDebt.setCreatedDate(LocalDate.now());
newDebt.setUpdatedDate(invoices.getInvoiceDate());
newDebt.setDebtAmount(invoices.getInvoiceAmount());
newDebt.setOrganization(organization1.get());
newDebt.setMadeenId(madeenOptional.get());
debtsRepository.save(newDebt);
}
Optional<Debts> updatedDebts = debtsRepository.findDebtsByPersonIdAndOrganization_id(invoices.getPersonId(), organization1.get().getId());
Madeen madeenCheck = madeenOptional.get();
Debts existingDebts = updatedDebts.get();
if (debts.isPresent()) {
existingDebts.setDebtAmount(existingDebts.getDebtAmount() + invoices.getInvoiceAmount());
existingDebts.setUpdatedDate(invoices.getInvoiceDate());
debtsRepository.save(existingDebts);
}
if (existingDebts.getDebtAmount() < -1) {
madeenCheck.setStatus(Status.STUMBLED);
madeenRepository.save(madeenCheck);
}
//Calculate the amount of debts
List<Debts> findAllDebts = debtsRepository.findDebtsByPersonId(invoices.getPersonId());
double totalDebts = 0; //find how much is it the totalDebts among all organization;
for (Debts item : findAllDebts) {
totalDebts = totalDebts + item.getDebtAmount();
}
if (totalDebts <= -750 && madeenCheck.getMadeenTimeTracker() == null) {
madeenCheck.setMadeenTimeTracker(LocalDate.now());
madeenRepository.save(madeenCheck);
}
if (totalDebts > -750) {
madeenCheck.setMadeenTimeTracker(null);
madeenRepository.save(madeenCheck);
}
if (madeenCheck.getMadeenTimeTracker() != null) {
LocalDate sixtyDaysFromLastInvoice = madeenCheck.getMadeenTimeTracker().plus(Period.ofDays(60));
if (totalDebts <= -750 && LocalDate.now().isAfter(sixtyDaysFromLastInvoice)) {
madeenCheck.setStatus(Status.MADEEN);
madeenRepository.save(madeenCheck);
}
}
if (totalDebts >= 0) {
madeenCheck.setStatus(Status.NOT_MADEEN);
madeenRepository.save(madeenCheck);
}
}
return response;
}
How can I use Spring Specification with Date field? I have no problem with 'normal' fields like Strings. But when I have Date, I have a problem and can't find a solution to solve it.
Here is my TaskEntity.class:
#Entity
#Table(name = "TASKS")
public class TaskEntity {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
private String title;
private String description;
#ManyToOne
private StatusEntity status;
private Date expiryDate;
// ....
}
And here is my TaskSpecification.class:
public class TaskSpecification implements Specification<TaskEntity> {
private static final Logger LOGGER = Logger.getLogger(TaskSpecification.class.getName());
private List<SearchCriteria> searchCriteriaList;
public TaskSpecification() {
this.searchCriteriaList = new ArrayList<>();
}
public void add(SearchCriteria criteria) {
searchCriteriaList.add(criteria);
}
#Override
public Predicate toPredicate(Root<TaskEntity> root, CriteriaQuery<?> query, CriteriaBuilder builder) {
LOGGER.info("toPredicate()");
List<Predicate> predicates = new ArrayList<>();
for (SearchCriteria criteria : searchCriteriaList) {
if (criteria.getValue() instanceof Date) {
// WHAT TO DO HERE?
} else {
predicates.add(
builder.equal(
root.get(criteria.getKey()),
criteria.getValue().toString())
);
}
}
LOGGER.info("toPredicate(...)");
return builder.and(predicates.toArray(new Predicate[0]));
}
}
Ihe same problem I faced a month ago but this solution solved my issue.
public static Date startDate(Date date) {
try {
DateFormat df2 = new SimpleDateFormat("yyyy-MM-dd");
String strDate = df2.format(date) + "T00:00:00";
LocalDateTime localDate = LocalDateTime.parse(strDate);
Instant instant = localDate.atZone(ZoneId.systemDefault()).toInstant();
date = Date.from(instant);
} catch (Exception e) {
e.printStackTrace();
System.out.println(e.getMessage());
}
return date;
}
Create a function and call where you use the date.
like that
values.add(EntitiesSpecification.startDate(fr.getValues().get(0)));
you can get the date value and add the date format in entity like MM-DD-YYYY. Could you please try this way?
I am using mongodb along with a java spring platform as my storage system to store files and documents. As mongo has a limit of 15MB(Bson storage limit) to store files I have used GridFs extension to store my large files. I have implemented this part as follow:
DBObject metaData = new BasicDBObject();
metaData.put("uplad_dateTime", largeDocument.getUploadDateTime());
metaData.put("title", largeDocument.getName());
ObjectId id =gridFsTemplate.store(largeDocument.getData(), largeDocument.getName(), largeDocument.getContentType(), metaData);
largeDocument.setId(id.toString());
The problem is that Gridfs by default uses two fs.chunck and fs.files collections, But I need to store files information with unique file id in a custom document model described here:
#Setter
#Getter
#org.springframework.data.mongodb.core.mapping.Document(collection = "document")
public class Document {
#Id
private String id;
#NotNull
#Column(name = "document_size", nullable = false)
private long size;
#NotNull
#Column(name = "document_name", nullable = false)
private String name;
#NotNull
#Column(name = "content_type", nullable = false)
private String contentType;
#NotNull
#Column(name = "content_data", nullable = false)
private InputStream data;
#NotNull
#Column(name = "upload_date_time", nullable = false)
private LocalDateTime uploadDateTime;
#Column(name = "download_counter", nullable = false)
private long downloadCounter;
public static Builder builder() {
return new Builder();
}
public static final class Builder {
private Document document;
private Builder() {
document = new Document();
document.setUploadDateTime(LocalDateTime.now());
}
public Builder id(String id) {
document.setId(id);
return this;
}
public Builder size(long size) {
document.setSize(size);
return this;
}
public Builder name(String name) {
document.setName(name);
return this;
}
public Builder contentType(String contentType) {
document.setContentType(contentType);
return this;
}
public Builder data(InputStream data) {
document.setData(data);
return this;
}
public Builder uploadDateTime(LocalDateTime uploadDateTime) {
document.setUploadDateTime(uploadDateTime);
return this;
}
public Builder downloadCounter(long downloadCounter) {
document.setDownloadCounter(downloadCounter);
return this;
}
public Document build() {
return document;
}
}
}
How can I change Gridfs to store file information in my model instead of fs.files? I appreciate any help.
First, gridfs provides a facility to associate arbitrary metadata with the uploaded files. This should serve most use cases.
If you require an application object associated with uploaded files, it must be stored in a separate collection.
I'm creating eCommerce for merchants using spring boot with JPA.
I have an issue while creating the order service.
I want to only pass the ID of the nested objects in the request body instead of sending the full nest objects because the size will be extremely big.
Here is my code.
Merchant can do many orders
Order
#Entity
#Table(name = "Orders")
#XmlRootElement
#JsonIgnoreProperties({"hibernateLazyInitializer", "handler"})
public class Order extends BasicModelWithIDInt {
#Basic(optional = false)
#Column(name = "Quantity")
private Integer quantity;
#Basic(optional = false)
#Size(min = 1, max = 150)
#Column(name = "Notes")
private String notes;
#JoinColumn(name = "ProductID", referencedColumnName = "ID")
#ManyToOne(optional = false, fetch = FetchType.LAZY)
#JsonIgnoreProperties
private Product productID;
#JoinColumn(name = "MerchantID", referencedColumnName = "ID")
#ManyToOne(optional = false, fetch = FetchType.LAZY)
private Merchent merchent;
#JoinColumn(name = "OrderSatusID", referencedColumnName = "ID")
#ManyToOne(optional = false, fetch = FetchType.EAGER)
private OrderStatus orderStatus;
// Getters and Setters
}
Order Holder
public class OrderHolder {
#NotNull
private Order order;
public Order getOrder() {
return order;
}
public void setOrder(Order order) {
this.order = order;
}
}
OrderRepo
public interface OrderRepo extends JpaRepository<Order, Integer> {
}
Order Controller
#RestController
#RequestMapping(value = "order", produces = MediaType.APPLICATION_JSON_VALUE)
public class OrderRestController extends BasicController<OrderHolder>{
#Autowired
private OrderRepo orderRepo;
#PostMapping("create")
public ResponseEntity<?> create(#RequestBody #Valid OrderHolder orderHolder, Principal principal) throws GeneralException {
log.debug( "create order {} requested", orderHolder.toString());
Order order = new Order();
order = orderHolder.getOrder();
System.out.println("###############"+order);
try {
order = orderRepo.save(order);
log.info( "Order {} has been created", order );
} catch (Exception e) {
log.error( "Error creating Order: ", e );
e.printStackTrace();
throw new GeneralException( Errors.ORDER_CREATION_FAILURE, e.toString() );
}
return ResponseEntity.ok( order );
}
}
I need request body to look like the below instead of including the full Merchant and Product objects inside the request.
You can make use of JsonView to return only id of product and merchant
public class OrderView {}
...
public class Product{
#Id
#JsonView(OrderView.class)
private Integer id
private String otherFieldWithoutJsonView
...
}
and then in your controller
#PostMapping("create")
#JsonView(OrderView.class) // this will return the product object with one field (id)
public ResponseEntity<?> create(#RequestBody #Valid OrderHolder orderHolder, Principal principal) throws GeneralException {
...
}
hope this can help you
Just have a separate contract class.
public class OrderContract {
private int merchantID;
private String notes;
....
//getter, setters
}
public class OrderHolder {
#NotNull
private OrderContract orderContract;
public OrderContract getOrderContract() {
return orderContract;
}
public void setOrder(OrderContract orderContract) {
this.orderContract = orderContract;
}
}
And before making a call to the Repository , translate from OrderContract to Order.
I would like to share something regarding this.
I have searched a lot on internet and tried lot of things, but the solution given here suited well for this scenario.
https://www.baeldung.com/jackson-deserialization
You need to create a Custom-deserializer for your model by extending StdDeserializer from com.fasterxml.jackson.databind.deser.std.StdDeserializer, where you just want to pass id's and not the whole object in the request.
I have given below example for User Model with Address object.
User(long userId, String name, Address addressId)
Address(long addressId, String wholeAddress)
Writing Deserializer for User class
public class UserDeserializer extends StdDeserializer<User> {
public User() {
this(null);
}
public User Deserializer(Class<?> vc) {
super(vc);
}
#Override
public User deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JacksonException {
JsonNode node = p.getCodec().readTree(p);
long id = 0;
long addressId = (Long) ((IntNode) node.get("addressId")).numberValue().longValue();
return new User(id, name, new Address(addressId, null)
}
Now you have to use
#JsonDeserialize(using = UserDeserializer.class)
public Class User {
...
}
POST request
Before custom deserialization
{
"name" : "Ravi",
"addressId" : { "id" : 1}
}
After custom Deserialization
{
"name" : "Ravi",
"addressId" : 1
}
Also while GET /user/:id call you will get the whole obj like
{
"name" : "Ravi",
"addressId" : { "id" : 1, "wholeAddress" : "Some address"}
}
I am having issues getting Jackson to bind a JSON object's Properties to a POJO. It is returning the JSON's field names instead of the POJOs.
Here is the JSON object I am trying to bind:
"Data": {
"technical_description": "This is a description.",
"technical_version": "1.0",
"technical_last_modified": "2019-03-11T18:23:34",
"is_available_in_source": true,
"availability_description": null,
"is_oii": false,
"is_pii": false,
"grain_description": null,
"grain_attributes": [
"INSTANCE"
],
"source_objects": null
}
Here is the POJO I want it to bind too:
public class Specification {
#JsonProperty(value = "technical_description")
public String description;
#JsonProperty(value = "technical_version")
public String version;
#JsonProperty(value = "technical_last_modified")
#JsonFormat(pattern="yyyy-MM-dd'T'HH:mm:ss")
public Date lastModified;
#JsonProperty(value = "is_available_in_source")
public Boolean availableInSource;
#JsonProperty(value = "availability_description")
public String availabilityDescription;
#JsonProperty(value = "is_oii")
public Boolean oii;
#JsonProperty(value = "is_pii")
public Boolean pii;
#JsonProperty(value = "grain_description")
public String grainDescription;
#JsonProperty(value = "grain_attributes")
public String[] grainAttributes;
#JsonProperty(value = "source_objects")
public String[] sourceObjects;
public Specification(){};
public Specification(String description, String version, Date lastModified, Boolean isAvailableInSource,
String availabilityDescription, Boolean isOii, Boolean isPii, String grainDescription,
String[] grainAttributes, String[] sourceObjects){
this.description=description;
this.version=version;
this.lastModified=lastModified;
this.availableInSource=isAvailableInSource;
this.availabilityDescription=availabilityDescription;
this.oii=isOii;
this.pii=isPii;
this.grainDescription=grainDescription;
this.grainAttributes=grainAttributes;
this.sourceObjects=sourceObjects;
};
}
The specification object is surfaced through a Detail object:
#Entity
public class Detail {
#Id
#Constraints.Required
public String id;
#Column(name = "specifications", updatable = false)
#Convert(converter = HashMapConverter.class)
public Map<String,Specification> technicalSpec;
...
}
Here is the converter I am using:
public class HashMapConverter implements AttributeConverter<Map<String, Specification>, String> {
private ObjectMapper objectMapper = new ObjectMapper().enable(JsonParser.Feature.ALLOW_SINGLE_QUOTES);
#Override
public String convertToDatabaseColumn(Map<String, Specification> techSpecs){
String technicalSpecs = null;
try {
technicalSpecs = objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(techSpecs);
} catch (final JsonProcessingException e) {
System.out.println(e);
}
return technicalSpecs;
}
#Override
public Map<String, Specification> convertToEntityAttribute(String techSpecJSON){
Map<String, Specification> techSpecs = null;
try {
techSpecs = objectMapper.readValue(techSpecJSON, objectMapper.getTypeFactory().constructMapType(HashMap.class, String.class, Specification.class));
} catch (final IOException e) {
System.out.println(e);
}
return techSpecs;
}
}
It will bind the object find through a custom converter, but it will return the same field names from the JSON object, i.e. I would like it to return description instead of technical_description.