Upload multiple blobs to Azure Storage - java

I have the following code to upload single blob to azure storage using azure-storage-blob 12.5.0.
Is there any way to pass a collection of byte arrays and do it in some kind of batch upload?
public void store(final String blobPath, final String originalFileName, final byte[] bytes) {
final BlobClient blobClient = containerClient.getBlobClient(blobPath);
final BlockBlobClient blockBlobClient = blobClient.getBlockBlobClient();
try (ByteArrayInputStream inputStream = new ByteArrayInputStream(bytes)) {
blockBlobClient.upload(inputStream, bytes.length, true);
} catch (BlobStorageException | IOException exc) {
throw new StorageException(exc);
}
}

Is there any way to pass a collection of byte arrays and do it in some
kind of batch upload?
In V8 sdk, i found uploadFromByteArray method supports byte[] parameter.
CloudBlockBlob blob = container.getBlockBlobReference("helloV8.txt");
String str1 = "132";
String str2 = "asd";
ByteArrayOutputStream os = new ByteArrayOutputStream();
os.write(str1.getBytes());
os.write(str2.getBytes());
byte[] byteArray = os.toByteArray();
blob.uploadFromByteArray(byteArray, 0, byteArray.length);
Test:
No such method could be found in V12 sdk,only upload method you used in your question.In fact,in above uploadFromByteArray method inside,it is upload method as well.
If you are referring upload multiple blobs in the batch,i'm afraid it it not supported in the official sdk except using for loop.About bulk writing,you could refer to the Azure CLI and AzCopy scenarios mentioned in this document.

Related

How to create an endpoint which takes a path, load the image and serve it to the client

I want to serve an image to a client by converting it to a byte but for some reason byteArrayOutputStream.toByteArray() is empty. I get a response status of 200 which means it is served. I looked at various documentations on reading an image file from a directory using BufferedImage and then converting BufferedImage to a byteArray from oracle https://docs.oracle.com/javase/tutorial/2d/images/loadimage.html and https://docs.oracle.com/javase/tutorial/2d/images/saveimage.html but for some reason byteArray is still empty
This controller
#GetMapping(path = "/get/image/{name}")
public ResponseEntity<byte[]> displayImage(String name) throws IOException {
String photoPathFromDatabase = productRepository.findPhotoByName(name);
Path path = Paths.get(photoPathFromDatabase);
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
BufferedImage image = ImageIO.read(path.toFile()); // Reading the image from path or file
String fileType = Files.probeContentType(path.toFile().toPath()); // Getting the file type
ImageIO.write(image, fileType, byteArrayOutputStream); // convert from BufferedImage to byte array
byte[] bytes = byteArrayOutputStream.toByteArray();
return ResponseEntity
.ok()
.contentType(MediaType.valueOf(fileType))
.body(bytes);
}
After I debugged the method
You should read the bytes of the file directly, rather than use an excessive amount of methods from different classes. This can be done with the class java.nio.file.Files.
byte[] contentBytes = Files.readAllBytes(path); //Throws IOException
Probably the file extension is not getting set properly.
You can create a new method to get the file extension or you can use FilenameUtils.getExtension from Apache Commons IO.
public static Optional<String> getExtensionByStringHandling(String filename) {
return Optional.ofNullable(filename)
.filter(f -> f.contains("."))
.map(f -> f.substring(filename.lastIndexOf(".") + 1));
}
then change your ImageIO to take this file extention.
String fileExtention= getExtensionByStringHandling(file.getName()).orElseThrow(()->new RuntimeException("File extension not found"));
ImageIO.write(image, fileExtention, byteArrayOutputStream);

How to temporarily create a text file without any file location and send as a response in spring boot at run time?

Need to create a txt file by the available data and then need to send the file as rest response.
the app is deployed in container. i dont want to store it in any location on container or any location in spring boot resources. is there any way where we can create file at runtime buffer without giving any file location and then send it in rest response?
App is production app so i need a solution which is secure
A file is a file. You're using the wrong words - in java, the concept of a stream of data, at least for this kind of job, is called an InputStream or an OutputStream.
Whatever method you have that takes a File? That's the end of the road. A File is a file. You can't fake it. But, talk to the developers, or check for alternate methods, because there is absolutely no reason anything in java that does data processing requires a File. It should be requiring an InputStream or possibly a Reader. Or perhaps even there is a method that gives you an OutputStream or Writer. All of these things are fine - they are abstractions that lets you just send data to it, from a file, a network connection, or made up whole cloth, which is what you want.
Once you have one of those, it's trivial. For example:
String text = "The Text you wanted to store in a fake file";
byte[] data = text.getBytes(StandardCharsets.UTF_8);
ByteArrayInputStream in = new ByteArrayInputStream(data);
whateverSystemYouNeedToSendThisTo.send(in);
Or for example:
String text = "The Text you wanted to store in a fake file";
byte[] data = text.getBytes(StandardCharsets.UTF_8);
try (var out = whateverSystemYouNeedToSendThisTo.getOUtputStream()) {
out.write(data);
}
Take a look at the function below:
Imports
import com.google.common.io.Files;
import org.springframework.http.ContentDisposition;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import java.io.*;
import java.nio.file.Paths;
Function:
#GetMapping(value = "/getFile", produces = MediaType.APPLICATION_OCTET_STREAM_VALUE)
private ResponseEntity<byte[]> getFile() throws IOException {
File tempDir = Files.createTempDir();
File file = Paths.get(tempDir.getAbsolutePath(), "fileName.txt").toFile();
String data = "Some data"; //
try (FileWriter fileWriter = new FileWriter(file)) {
fileWriter.append(data).flush();
} catch (Exception ex) {
ex.printStackTrace();
}
byte[] zippedData = toByteArray(new FileInputStream(file));
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.setContentDisposition(ContentDisposition.builder("attachment").filename("file.txt").build());
httpHeaders.setContentType(MediaType.APPLICATION_OCTET_STREAM);
httpHeaders.setContentLength(zippedData.length);
return ResponseEntity.ok().headers(httpHeaders).body(zippedData);
}
public static byte[] toByteArray(InputStream in) throws IOException {
ByteArrayOutputStream os = new ByteArrayOutputStream();
byte[] buffer = new byte[in.available()];
int len;
// read bytes from the input stream and store them in buffer
while ((len = in.read(buffer)) != -1) {
// write bytes from the buffer into output stream
os.write(buffer, 0, len);
}
return os.toByteArray();
}
In a nutshell, you want to store data in memory. Basic building block for this is array of bytes - byte[].
In JDK there are two classes to connect IO world with byte array - ByteArrayInputStream and ByteArrayOutputStream.
Rest is just same, as when dealing with files.
Example 1
#GetMapping(value = "/image")
public #ResponseBody byte[] getImage() throws IOException {
InputStream in = getClass()
.getResourceAsStream("/com/baeldung/produceimage/image.jpg");
return IOUtils.toByteArray(in);
}
Example 2:
#GetMapping("/get-image-dynamic-type")
#ResponseBody
public ResponseEntity<InputStreamResource> getImageDynamicType(#RequestParam("jpg") boolean jpg) {
MediaType contentType = jpg ? MediaType.IMAGE_JPEG : MediaType.IMAGE_PNG;
InputStream in = jpg ?
getClass().getResourceAsStream("/com/baeldung/produceimage/image.jpg") :
getClass().getResourceAsStream("/com/baeldung/produceimage/image.png");
return ResponseEntity.ok()
.contentType(contentType)
.body(new InputStreamResource(in));
}
Ref: https://www.baeldung.com/spring-controller-return-image-file

Stream content to Google Cloud Storage

I would like to upload a large Set<Integer> to Google Cloud Storage. I can do that with:
Blob result = storage.create(blobInfo, Joiner.on('\n').join(set).getBytes(UTF_8));
But this will create an intermediate String with all the content that might be too large.
I found an example with WriteChannel.write():
Set<Integer> set = ...
String bucketName = "my-unique-bucket";
String blobName = "my-blob-name";
BlobId blobId = BlobId.of(bucketName, blobName);
byte[] content = Joiner.on('\n').join(set).getBytes(UTF_8);
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).setContentType("text/plain").build();
try (WriteChannel writer = storage.writer(blobInfo)) {
writer.write(ByteBuffer.wrap(content, 0, content.length));
} catch (IOException ex) {
// handle exception
}
However, if I do that, the entire set is converted to a String and then to byte[]. The String itself might be too big.
Is there an example how to iterate over the set and transform it to a ByteBuffer? or should I do a loop on chunks of the set?
The most straightforward approach I could think of would be:
try (WriteChannel writer = storage.writer(blobInfo)) {
for(Integer val : set) {
String valLine = val.toString() + '\n';
writer.write(ByteBuffer.wrap(valLine.getBytes(UTF_8));
}
}
Mind you, this isn't very efficient. It creates a lot of small ByteBuffers. You could greatly improve on this by writing into a single larger ByteBuffer and periodically calling writer.write with it.
To avoid creating an intermediate String with all the bytes you can upload from a file. You can find example code to do an upload from a file in various languages here.

Image byte stream is different when file is read in different ways

Under my Spring 4.3/Maven 3.3 project I have an image file, a PNG file, at:
src/main/resources/images/account.png
I have a util java application file that reads in an image, and it writes it to the database field. The code is as follows:
private static String _accountFilename = "src/main/resources/images/account.png";
private byte[] getByteArrayFromFile(String filename)
{
FileInputStream fileInputStream = null;
File file = new File(filename);
byte[] bFile = new byte[(int) file.length()];
try
{
// convert file into array of bytes
fileInputStream = new FileInputStream(file);
fileInputStream.read(bFile);
fileInputStream.close();
for (int i = 0; i < bFile.length; i++)
{
System.out.print((char) bFile[i]);
}
System.out.println("Done");
}
catch (Exception e)
{
e.printStackTrace();
}
return bFile;
}
public String getImageData(byte[] imageByteArray)
{
Base64.Encoder encoder = Base64.getEncoder();
String base64 = encoder.encodeToString(imageByteArray);
base64 = "data:image/png;base64," + base64;
return base64;
}
The String that comes back from "getImageData" works great. I can put that String in the MySQL database, in a table, and the field is defined as TEXT.
I can pull that base64 encoded data back, and display the image.
Now, If I am calling this code from a Spring Service instead of an application, then the image "src/main/resources/images/account.png" is not found.
After researching on the Net for some time, there are many, many examples of getting a file from "resources" and many of these did not work for me. Since I am in Spring, I tried a few things and finally this worked:
#Value(value = "classpath:images/account.png")
private Resource defaultAccountImage;
private byte[] getByteArrayFromFile(Resource image)
{
InputStream inputStream = null;
byte[] bFile = null;
try
{
bFile = new byte[(int) image.contentLength()];
// convert file into array of bytes
inputStream = image.getInputStream();
inputStream.read(bFile);
inputStream.close();
for (int i = 0; i < bFile.length; i++)
{
System.out.print((char) bFile[i]);
}
System.out.println("Done");
}
catch (Exception e)
{
e.printStackTrace();
}
return bFile;
}
private String getImageData(byte[] imageByteArray)
{
Base64.Encoder encoder = Base64.getEncoder();
String base64 = encoder.encodeToString(imageByteArray);
base64 = "data:image/png;base64," + base64;
return base64;
}
public String getDefaultAccountImage()
{
byte[] accountImage = getByteArrayFromFile(defaultAccountImage);
String fileString = getImageData(accountImage);
return fileString;
}
When I look at the String/Image data between the first way with the standalone java app, and the second way with the #Value and inputstream, there is a definite different in the string data.
part of the string data is similar, but then it drastically changes, and they don't match. As a result the text data for the image from the second method doesn't display as an image.
So, I was hoping I could get this image text data, and it would be the same, but it is not. If I can use my web-service, which calls the business service which calls this ImageUtil code where I use the #Value to get the image resource and it saves the text string correctly, that would be great.
If you have any advice, I would very much appreciate it. Thanks!
UPDATE 1:
This is a multi-maven project:
parent-project
entity
dao
service
ws
When I run my test code within the Service layer, the suggested solution works great! The images are found and the byte string gets loaded as it should be. And then I compiled the code into a jar.
The entity.jar gets created first.
The dao.jar gets created and pulls in the entity.jar.
The service.jar gets created and pulls in the dao.jar. This layer also has the /resources/images/account.png file. But this image is now in the jar.
The ws.WAR file pulls in the service.jar file ...
so the code in the answer does not find the image in the resources.
When I run the tests from the ws layer, I get a FileNotFoundException.
So ... now I am researching on how to get an image from jar ...
Does this change how I should be getting my image byte array?
You can get the file from the Resource, and proceed like the first example which works. Seems redundant, but if you can get the file, then you can test a number of things:
Write the file to disk and check the content
Write the file to disk and compare the sizes, etc.
import java.io.File;
import java.io.FileInputStream;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.Resource;
#Value(value = "classpath:images/account.png")
private Resource defaultAccountImage;
private byte[] getByteArrayFromFile(Resource image) {
FileInputStream fileInputStream = null;
byte[] bFile = null;
try {
File file = image.getFile();
bFile = new byte[(int) file.length()];
// convert file into array of bytes
fileInputStream = new FileInputStream(file);
fileInputStream.read(bFile);
fileInputStream.close();
for (int i = 0; i < bFile.length; i++) {
System.out.print((char) bFile[i]);
}
System.out.println("Done");
} catch (Exception e) {
e.printStackTrace();
}
return bFile;
}

Store BLOB in Android's SQLite using SQLOpenHelper

Is there a way to store a BLOB into Android's SQLite using SQLOpenHelper?
My BLOB of type InputStream.
SQLite doesn't support streaming BLOB or CLOB data. You have four options:
Convert the InputStream to a byte[]. This will only work if you have enough memory (see below).
Use FileOutputStream or an Android API that supports streaming
Split the data into small blocks and store then in SQLite
Use a database that works on Android and supports streaming. I only know the H2 database. But be aware the Android support for H2 is very new. In that case, you need to use the JDBC API, and you probably want to enable LOBs in the database, otherwise each large BLOB is stored in a separate file.
To convert an InputStream to a byte array, you could use:
public static byte[] readBytesAndClose(InputStream in) throws IOException {
try {
int block = 4 * 1024;
ByteArrayOutputStream out = new ByteArrayOutputStream(block);
byte[] buff = new byte[block];
while (true) {
int len = in.read(buff, 0, block);
if (len < 0) {
break;
}
out.write(buff, 0, len);
}
return out.toByteArray();
} finally {
in.close();
}
}

Categories