I am using mongodb along with a java spring platform as my storage system to store files and documents. As mongo has a limit of 15MB(Bson storage limit) to store files I have used GridFs extension to store my large files. I have implemented this part as follow:
DBObject metaData = new BasicDBObject();
metaData.put("uplad_dateTime", largeDocument.getUploadDateTime());
metaData.put("title", largeDocument.getName());
ObjectId id =gridFsTemplate.store(largeDocument.getData(), largeDocument.getName(), largeDocument.getContentType(), metaData);
largeDocument.setId(id.toString());
The problem is that Gridfs by default uses two fs.chunck and fs.files collections, But I need to store files information with unique file id in a custom document model described here:
#Setter
#Getter
#org.springframework.data.mongodb.core.mapping.Document(collection = "document")
public class Document {
#Id
private String id;
#NotNull
#Column(name = "document_size", nullable = false)
private long size;
#NotNull
#Column(name = "document_name", nullable = false)
private String name;
#NotNull
#Column(name = "content_type", nullable = false)
private String contentType;
#NotNull
#Column(name = "content_data", nullable = false)
private InputStream data;
#NotNull
#Column(name = "upload_date_time", nullable = false)
private LocalDateTime uploadDateTime;
#Column(name = "download_counter", nullable = false)
private long downloadCounter;
public static Builder builder() {
return new Builder();
}
public static final class Builder {
private Document document;
private Builder() {
document = new Document();
document.setUploadDateTime(LocalDateTime.now());
}
public Builder id(String id) {
document.setId(id);
return this;
}
public Builder size(long size) {
document.setSize(size);
return this;
}
public Builder name(String name) {
document.setName(name);
return this;
}
public Builder contentType(String contentType) {
document.setContentType(contentType);
return this;
}
public Builder data(InputStream data) {
document.setData(data);
return this;
}
public Builder uploadDateTime(LocalDateTime uploadDateTime) {
document.setUploadDateTime(uploadDateTime);
return this;
}
public Builder downloadCounter(long downloadCounter) {
document.setDownloadCounter(downloadCounter);
return this;
}
public Document build() {
return document;
}
}
}
How can I change Gridfs to store file information in my model instead of fs.files? I appreciate any help.
First, gridfs provides a facility to associate arbitrary metadata with the uploaded files. This should serve most use cases.
If you require an application object associated with uploaded files, it must be stored in a separate collection.
Related
So I have my LanguageController class in which I have this method:
#GetMapping("/languages")
public ResponseEntity<List<Language>> getAllLanguages(#RequestParam(required = false) String name) {
try {
List<Language> languages = new ArrayList<Language>();
if (name == null) {
languageRepository.findAll().forEach(languages::add);
} else {
languageRepository.findByNameContaining(name).forEach(languages::add);
}
if (languages.isEmpty()) {
return new ResponseEntity<>(HttpStatus.NO_CONTENT);
}
return new ResponseEntity<>(languages, HttpStatus.OK);
} catch (Exception e) {
return new ResponseEntity<>(null, HttpStatus.INTERNAL_SERVER_ERROR);
}
}
And the Language class is (omitted the getters and setters so it's cleaner):
#Entity
#Table(name = "languages")
public class Language {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private long id;
#Column(name = "name")
private String name;
#Column(name = "audio")
#Lob
private Blob audio;
#Column(name = "script")
#Lob
private Blob script;
public Language() {
}
public Language(String name, Blob audio, Blob script) {
this.name = name;
this.audio = audio;
this.script = script;
}
}
And here when I consume the API endpoint, I get this JSON:
[
{
"id": 1,
"name": "Test Language",
"audio": {
"binaryStream": {},
"wrappedBlob": {
"binaryStream": {},
"traceId": 26,
"traceObjectName": "blob26"
}
},
"script": {
"binaryStream": {},
"wrappedBlob": {
"binaryStream": {},
"traceId": 27,
"traceObjectName": "blob27"
}
}
}
]
And here I'm returning the BLOB stream, which isn't super useful.
I would like to return the BLOB encoded in Base64, and I just genuinely don't know where to encode the files.
Thanks for your help!
You might try to use a #JsonSerialize annotation on the Blob fields of your entity class to influence how the blobs are serialized. The SqlBlobSerializer class seems to be doing the conversion to base64 you are looking for.
#Column(name = "audio")
#Lob
#JsonSerialize(using = SqlBlobSerializer.class)
private Blob audio;
https://fasterxml.github.io/jackson-databind/javadoc/2.14/com/fasterxml/jackson/databind/annotation/JsonSerialize.html
https://fasterxml.github.io/jackson-databind/javadoc/2.14/com/fasterxml/jackson/databind/ext/SqlBlobSerializer.html
Instead of returning the #Entity Language you could return a LanguageDTO instance, the DTO being a class you initialize from the Language instance and in which you convert the Blob in Base64.
I wrote a Java program that parses JSON to the list of objects, and then I'm simply persisting them into the database using JpaRepository.
The problem with this kind of solution is that I've no Ids from JSON so those are each time unique when I'm parsing data and then I'm saving duplicates.
How can I avoid that?
public void getTestApi() {
ObjectMapper mapper = new ObjectMapper();
mapper.findAndRegisterModules();
try {
URL url = new URL("https://olxdata.azurewebsites.net/olx?city=wszystkie&dataType=mieszkania");
String json = getJsonFromURL(url);
PortalData portalData = mapper.readValue(json, PortalData.class);
List<OlxItem> olxItems = portalData.getOlx();
List<OtodomItem> otodomItems = portalData.getOtodom();
List<OlxDataAzureItem> olxDataAzureItems = new ArrayList<>();
for (var item : olxItems) {
olxDataAzureItems.add(new OlxDataAzureItem("olx", item.getDate(), item.getToRent(), item.getToSell()));
}
for (var item : otodomItems) {
olxDataAzureItems.add(new OlxDataAzureItem("otodom", item.getDate(), item.getToRent(), item.getToSell()));
}
olxDataAzureRepository.saveAll(olxDataAzureItems);
} catch (Exception exception) {
exception.printStackTrace();
}
}
#Entity
#Table(name = "olx_data")
public class OlxDataAzureItem {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private UUID id;
#Column(name = "portal")
private String portal;
#Column(name = "date")
private LocalDateTime date;
#Column(name = "toRent")
private int toRent;
#Column(name = "toSell")
private int toSell;
public OlxDataAzureItem() {
}
public OlxDataAzureItem(String portal, LocalDateTime date, int toRent, int toSell) {
this.portal = portal;
this.date = date;
this.toRent = toRent;
this.toSell = toSell;
}
}
I want to allow to sort by every field in the class, without having to write switch/ if statements.
My idea was to find the Field that matches given string value by name and then, with Stream API neatly sort. IntelliJ screamed that i need to surround it with try-catch, so it is not so neatly looking, but that's not important, as it does not work.
private List<MyEntity> getSorted(List<MyEntity> list, SearchCriteria criteria) {
Field sortByField = findFieldInListByName(getFieldList(MyEntity.class), criteria.getSortBy());
return list.stream().sorted(Comparator.comparing(entity-> {
try {
return (MyEntity) sortByField.get(entity);
} catch (IllegalAccessException e) {
e.printStackTrace();
}
return entity;
})).collect(Collectors.toList());
}
In the MyEntity class I have added Comparable interface, but I am not sure what should be in the body of Compare(), as I dont want to specify how to compare objects, because it will change based on the selected sorting.
EDIT: Added Entity below:
#Data
#Entity
#Table(name = "role_management", schema = "mdr")
#Builder
#NoArgsConstructor
#AllArgsConstructor
#EqualsAndHashCode
public class MyEntity implements Comparable{
#Id
#Column(name = "uuid", unique = true, insertable = false, updatable = false)
#GeneratedValue(strategy = GenerationType.AUTO)
private UUID uuid;
#ManyToOne
#JoinColumn(name = "user_id", nullable = false)
private UserEntity user;
#Basic
#NonNull
#Column(name = "role")
private String role;
#Basic
#Column(name = "action")
#Enumerated(EnumType.STRING)
private RoleAction action;
#Basic
#Column(name = "goal")
#Enumerated(EnumType.STRING)
private RoleGoal goal;
#Column(name = "date")
private LocalDateTime date;
#OneToOne(cascade = CascadeType.ALL)
#JoinColumn(name = "reporter_id", referencedColumnName = "uuid")
private UserEntity reporter;
#OneToOne(cascade = CascadeType.ALL)
#JoinColumn(name = "authorizer_id", referencedColumnName = "uuid")
private UserEntity authorizer;
#Basic
#Column(name = "ezd")
private String ezd;
#Basic
#Column(name = "is_last")
private boolean isMostRecent;
#Override
public int compareTo(Object o) {
return 0;
}
}
EDIT 2: My code based on the #Sweeper solution:
UserEntity (nullable)
#Override
public int compareTo(UserEntity other) {
if (other == null) {
return 1;
}
return this.getMail().compareTo(other.getMail());
}
Comparator:
public static Comparator getSortComparator(Field sortByField) {
return Comparator.nullsLast(Comparator.comparing(entity -> {
try {
Object fieldValue = sortByField.get(entity);
if (!(fieldValue instanceof Comparable<?>) && fieldValue != null) {
throw new IllegalArgumentException("...");
}
return (Comparable) fieldValue;
} catch (IllegalAccessException e) {
throw new MdrCommonException(e.getLocalizedMessage());
}
}));
}
MyEntity should not implement Comparable. It is the fields, by which you are going to sort the list of MyEntity objects, that needs to be Comparable. For example, if you are sorting by the field user, which is a UserEntity, then UserEntity is the thing that needs to be comparable, not MyEntity.
The lambda's job should just be to check that the fields are indeed Comparable, and throw an exception if they are not.
Since you don't know the types of the fields at compile time, however, you'd have to use a raw type here. The comparing call would look like this:
Comparator.comparing(entity -> {
try {
Object fieldValue = sortByField.get(entity);
// This check still passes if the type of fieldValue implements Comparable<U>,
// where U is an unrelated type from the type of fieldValue, but this is the
// best we can do here, since we don't know the type of field at compile time
if (!(fieldValue instanceof Comparable<?>) && fieldValue != null) {
throw new IllegalArgumentException("Field is not comparable!");
}
return (Comparable)fieldValue;
} catch (IllegalAccessException e) {
throw new RuntimeException(e);
}
})
You can create automatically comparators for any field of any class using reflection but is better create specific comparators (will be typechecked).
Your entity is a normal class with normal fields then, the usual Java sorting machinery should do the job:
Basically, if you define one comparator for every field (even deep fields into your entity):
public final static Comparator<MyEntity> ByField1 = comparing(MyEntity::getField1);
public final static Comparator<MyEntity> ByField2 = comparing(MyEntity::getField2);
public final static Comparator<MyEntity> ByField3 = comparing(MyEntity::getField3);
public final static Comparator<MyEntity> ByDeep1 = comparing(a -> a.getField4().getDeep1());
public final static Comparator<MyEntity> ByDeep2 = comparing(a -> a.getField4().getDeep2());
public final static Comparator<MyEntity> ByDeep3 = comparing(a -> a.getField4().getDeep3());
You can sort using complex sorting expressions:
data.stream()
.sorted(ByField2.reversed().thenComparing(ByDeep2))
.forEach(System.out::println);
a full example could be
public static void main(String[] args) {
List<MyEntity> data =
Stream.of("Row1", "Row2").flatMap(field1 ->
Stream.of(101, 102).flatMap(field2 ->
Stream.of(true, false).flatMap(field3 ->
Stream.of("Row1", "Row2").flatMap(deep1 ->
Stream.of(101, 102).flatMap(deep2 ->
Stream.of(true, false).map(deep3 ->
new MyEntity(field1, field2, field3, new MyDeepField(deep1, deep2, deep3))))))))
.collect(toList());
data.stream()
.sorted(ByField2.reversed().thenComparing(ByDeep2))
.forEach(System.out::println);
}
#Getter
#Setter
#AllArgsConstructor
static class MyDeepField {
private String deep1;
private Integer deep2;
private Boolean deep3;
}
#Getter
#Setter
#AllArgsConstructor
static class MyEntity {
private String field1;
private Integer field2;
private Boolean field3;
private MyDeepField field4;
public final static Comparator<MyEntity> ByField1 = comparing(MyEntity::getField1);
public final static Comparator<MyEntity> ByField2 = comparing(MyEntity::getField2);
public final static Comparator<MyEntity> ByField3 = comparing(MyEntity::getField3);
public final static Comparator<MyEntity> ByDeep1 = comparing(a -> a.getField4().getDeep1());
public final static Comparator<MyEntity> ByDeep2 = comparing(a -> a.getField4().getDeep2());
public final static Comparator<MyEntity> ByDeep3 = comparing(a -> a.getField4().getDeep3());
#Override
public String toString() {
return "MyEntity{" +
"field1='" + field1 + '\'' +
", field2=" + field2 +
", field3=" + field3 +
", deep1=" + field4.getDeep1() +
", deep2=" + field4.getDeep2() +
", deep3=" + field4.getDeep3() +
'}';
}
}
with output
MyEntity{field1='Row1', field2=102, field3=true, deep1=Row1, deep2=101, deep3=true}
MyEntity{field1='Row1', field2=102, field3=true, deep1=Row1, deep2=101, deep3=false}
...
MyEntity{field1='Row2', field2=101, field3=false, deep1=Row2, deep2=102, deep3=true}
MyEntity{field1='Row2', field2=101, field3=false, deep1=Row2, deep2=102, deep3=false}
The criteria field into your SearchCriteria class is some field of type Comparator<MyEntity> or a mapping using an enumeration or parsing string expressions or so...
I cannot resolve a problem. I have a window with TextField and I want to bind a foreign key but have no ideas what am I doing wrong. I've already read the answer here: Binding foreign key in Vaadin (EclipseLink)
I decided to use Converter, but still stumble.
So, there are two entities (simplified): Client and Order. An existent or a new client can have several orders, means that Order.clientID is foreign key
Client entity:
#Entity
#DynamicInsert
#DynamicUpdate
#Table(name = "Clients")
public class Client {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "ClientID", updatable = false, nullable = false)
private Long clientID;
#Column(name = "Name", nullable = false, length = 20)
private String name;
#OneToMany(fetch = FetchType.LAZY, mappedBy = "clientID")
private Set<Order> orders;
public Long getId() { return clientID; }
public void setId(Long clientID) { this.clientID = clientID; }
public String getName() { return firstName; }
public void setName(String name) { this.name = name; }
}
Order entity:
#Entity
#DynamicInsert
#DynamicUpdate
#Table(name = "Orders")
public class Order{
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "OrderID", nullable = false)
private Long orderID;
#ManyToOne(fetch = FetchType.LAZY, cascade = CascadeType.ALL)
#JoinColumn(name = "ClientID",
referencedColumnName = "ClientID",
updatable = false,
nullable = false)
private Client clientID;
#Column(name = "Description", nullable = false, length = 1000)
private String description;
public Long getOrderID() { return orderID; }
//public Long getClientID() { return clientID.getId(); }
public Client getClientID() { return clientID; }
public void setClientID(Client clientID) { this.clientID = clientID; }
public String getDescription() { return description; }
public void setDescription(String description) { this.description = description; }
}
and I want to bind Order.clientID to the TextField. But IDEA highlight the setter setClientID as "Cannot resolve method 'setClientID'"
public class AddOrderModalView extends Window {
private OrderService orderService = new OrderService();
private Order order = new Order();
Binder<Order> binder = new Binder<>(Order.class);
private ChangeHandler changeHandler = new ChangeHandler() {
#Override
public void onChange() {
}
};
private FormLayout formLayout = new FormLayout();
private TextField clientId = new TextField("Client ID");
private TextField description = new TextField("Description");
private Button save = new Button("Save");
private Button cancel = new Button("Cancel");
public AddOrderModalView() {
super("Add a new order");
VerticalLayout subContent = new VerticalLayout();
subContent.setSizeFull();
HorizontalLayout actions = new HorizontalLayout();
actions.addComponents(save, cancel);
formLayout.addComponents(clientId, description);
subContent.addComponent(formLayout);
setContent(subContent);
save.addStyleNames(ValoTheme.BUTTON_SMALL, ValoTheme.BUTTON_PRIMARY);
cancel.addStyleName(ValoTheme.BUTTON_SMALL);
save.addClickListener(e -> save());
cancel.addClickListener(e -> close());
bindingFields();
setModal(true);
}
private void bindingFields() {
binder.forField(clientId)
.withConverter(Long::valueOf, String::valueOf)
.bind(Order::getClientID, Order::setClientID); //the error is here
binder.forField(this.description)
.withValidator(new StringLengthValidator(
"Please add description. The maximum length is 1000 characters", 1, 1000))
.bind(Order::getDescription, Order::setDescription);
binder.bindInstanceFields(this);
binder.setBean(order);
}
public interface ChangeHandler {
void onChange();
}
private void save() {
if (binder.validate().isOk()) {
orderService.persist(order);
close();
changeHandler.onChange();
}
}
}
ClientToClientIdConverter:
public class ClientToClientIdConverter implements Converter<String, Client> {
#Override
public Result<Client> convertToModel(String s, ValueContext valueContext) {
return Result.error("not supported");
}
#Override
public String convertToPresentation(Client client, ValueContext valueContext) {
return Objects.toString(client.getId(), "");
}
}
Could anyone help me with solving the problem?
Answer to the question: How to bind foreign key (or any other nested property) in a textfield (not what you need!)
You can do it by providing lambda expressions to get and set the nested properties.
TextField clientId = new TextField("Client ID");
binder.forField(clientId)
.withConverter(new StringToLongConverter("error message"))
.bind(item -> item.getClient().getId(), (item, value) -> item.getClient().setId(value));
This code can be cause of NullPointerExceptions if the order can have no client at this point. If that is possible, then use this instead (added checking for nullvalues):
TextField clientId = new TextField("Client ID");
binder.forField(clientId)
.withConverter(new StringToLongConverter("error message"))
.bind(
item -> item.getClient() != null ? item.getClient.getId() : null,
(item, value) -> {
if(item.getClient() != null){
item.getClient().setId(value);
}
});
Warning! Please know that manually changing the value in this textfield will change the id of it's already assigned client, and not select/assign a new Client for this Order. If it's the latter that you want, use a ComboBox instead! I'm not sure if it ever makes sense to do the first, but I answered because you asked. I am now sure that you want the latter option, so please follow the next part.
The actual solution to your problem: It seems that you indeed need a ComboBox, because you want to select/assign a client for the order.
So what you need is basically this:
ComboBox<Client> clientSelection = new ComboBox<Client>("client");
clientSelection.setItems(clientService.findAll()); // list/set of possible clients.
// Using clients name for item captions now, but you can also use id or both
clientSelection.setItemCaptionGenerator(Client::getName);
binder.forField(clientSelection)
.bind(Order::getClient, Order::setClient);
This way, you can select a client which then gets set as the bound orders client.
Your binding doesnt work because your setClient method expects an Object of type Client as parameter not an Long. You should change that to:
public void setClientID(Long clientID) { this.clientID = clientID; }
You could also use an ComboBox for the Selection of Customer, there is an example how to bind Objects to a ComboBox on the Vaadin Website.
I am trying to create a springboot usermanagement application.
I have an entity object which contains two blob elements.Here is my entity object.
#Entity
#Table(name="user_meta_profile")
public class UserMetaProfile implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#Column(name = "user_id")
private int user_id;
#Column(name = "resume_file")
#Lob
private Blob resume_file;
#Column(name = "photo")
#Lob
private Blob photo;
#Column(name = "username")
private String username;
public int getUser_id() {
return user_id;
}
public void setUser_id(int user_id) {
this.user_id = user_id;
}
public Blob getResume_file() {
return resume_file;
}
public void setResume_file(Blob resume_file) {
this.resume_file = resume_file;
}
public Blob getPhoto() {
return photo;
}
public void setPhoto(Blob photo) {
this.photo = photo;
}
public void setUsername(String username) {
this.username = username;
}
}
As you can see there are two blob items 'resume_file' and 'photo'.
I want to send back a JSON response to the API call.
My Controller code is as shown below.
#Controller
#RequestMapping("/v1")
public class UsersController {
#Autowired
private IUserMetaProfileService userMetaProfileService;
#GetMapping("MetaProfile/{id}")
public ResponseEntity<UserMetaProfile> getUserMetaProfileById(#PathVariable("id") Integer id) {
UserMetaProfile userMetaProfile = userMetaProfileService.getUsersById(id);
return new ResponseEntity<UserMetaProfile>(userMetaProfile, HttpStatus.OK);
}
}
But when I call the API, I get the exception:
"exception": "org.springframework.http.converter.HttpMessageNotWritableException",
"message": "Could not write JSON document: No serializer found for class java.io.ByteArrayInputStream and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain:
...
...nested exception is com.fasterxml.jackson.databind.JsonMappingException: No serializer found for class java.io.ByteArrayInputStream and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS)
Since JSON cannot contain binary data you need to serialize those fields as something else. You have a couple of options:
If you intend to show the binary as an image (since yours is a photo) you can serialize it as a data uri.
Send links to photos instead and create a controller method that will output the binary data with the appropriate content type (beyond the scope here).
So for Option 1 you can do something like this:
#Entity
#Table(name="user_meta_profile")
public class UserMetaProfile implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#Column(name = "user_id")
private int user_id;
#Column(name = "resume_file")
#Lob
private Blob resume_file;
#Column(name = "photo")
#Lob
private Blob photo;
#Column(name = "username")
private String username;
public int getUser_id() {
return user_id;
}
public void setUser_id(int user_id) {
this.user_id = user_id;
}
#JsonIgnore // disable serializing this field by default
public Blob getResume_file() {
return resume_file;
}
// serialize as data uri insted
#JsonProperty("resumeData")
public String getResume() {
// just assuming it is a word document. you would need to cater for different media types
return "data:application/vnd.openxmlformats-officedocument.wordprocessingml.document;base64," + new String(Base64.getEncoder().encode(resume_file.getBytes()));
}
public void setResume_file(Blob resume_file) {
this.resume_file = resume_file;
}
#JsonIgnore // disable this one too
public Blob getPhoto() {
return photo;
}
// serialize as data uri instead
#JsonProperty("photoData")
public String getPhotoBase64() {
// just assuming it is a jpeg. you would need to cater for different media types
return "data:image/jpeg;base64," + new String(Base64.getEncoder().encode(photo.getBytes()));
}
public void setPhoto(Blob photo) {
this.photo = photo;
}
public void setUsername(String username) {
this.username = username;
}
}
For the photo bit the value of the photoData JSON attribute can be set directly as the src attribute of an img tag and the photo will be rendered in the HTML. With the resume file you can attach it as an href to a <a> tag with a download attribute so it can be downloaded:
<a href={photoData value here} download>Download Resume File</a>
Just as an FYI if the files are large the JSON will be huge and it might also slow down the browser.