We Keep Coding

Is there a way to put a kind of identificator to asychronous requests?

I have a server and a client that sends requests. I don't need to identify the requests when it works in synch mode. But in asynch mode each request must have an identificator, so I could be sure that the response corresponds to exact request. The server is not to be updated, I have to put the identificator in the client's code. Is there a way do do it? I can't find out any.
Here is my main class. I guess all must be clear, the class is very simple.
public class MainAPITest {
private static int userId = 0;
private final static int NUM_OF_THREADS = 10;
private final static int NUM_OF_USERS = 1000;
public static void main(String[] args) {
Security.addProvider(new GammaTechProvider());
ExecutorService threadsExecutor = Executors.newFixedThreadPool(NUM_OF_THREADS);
for (int i = 0; i < NUM_OF_USERS; i++) {
MyRunnable user = new MyRunnable();
userId++;
user.uId = userId;
threadsExecutor.execute(user);
}
threadsExecutor.shutdown();
}
}
class MyRunnable implements Runnable {
int uId;
#Override
public void run() {
try {
abstrUser user = new abstrUser();
user.setUserId(uId);
user.registerUser();
user.chooseAndImplementTests();
user.revokeUser();
} catch (Exception e) {
e.printStackTrace();
}
}
}
User class describes the user's behaviour. It is long enough, idk if it is needed here. User runs a number of random tests. Each test has its own class that extends abstract test class, where the http connection is established:
import org.json.JSONObject;
import javax.xml.bind.DatatypeConverter;
import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.*;
public abstract class abstractRESTAPITest {
protected String apiUrl;
public abstractRESTAPITest() {
}
protected byte[] sendRequest(JSONObject jsonObject) throws IOException {
return this.sendRequest(jsonObject, (String) null, (String) null);
}
protected byte[] sendRequest(JSONObject jsonObject, String username, String password) throws IOException {
HttpURLConnection httpURLConnection = (HttpURLConnection) (new URL(this.apiUrl)).openConnection();
if (username != null && password != null) {
String userPassword = username + ":" + password;
httpURLConnection.setRequestProperty("Authorization", "Basic " + DatatypeConverter.printBase64Binary(userPassword.getBytes()));
}
httpURLConnection.setRequestProperty("Content-Type", "application/json");
httpURLConnection.setDoOutput(true);
DataOutputStream dataOutputStream = new DataOutputStream(httpURLConnection.getOutputStream());
dataOutputStream.write(jsonObject.toString().getBytes());
dataOutputStream.flush();
System.out.println("REST send: " + jsonObject.toString());
if (httpURLConnection.getResponseCode() != 200) {
System.out.println("REST send error, http code " + httpURLConnection.getResponseCode() + " " + httpURLConnection.getResponseMessage());
throw new IOException();
} else {
byte[] responseBody = null;
StringBuilder data = new StringBuilder();
BufferedReader br = new BufferedReader(new InputStreamReader(httpURLConnection.getInputStream()));
String line = "";
while ((line = br.readLine()) != null) {
data.append(line);
responseBody = data.toString().getBytes();
}
if (br != null) {
br.close();
}
return responseBody;
}
}
public abstract boolean test();
}
Now I am trying to transform the httpUrlConnection part of the code into a kind of this.
Jsonb jsonb = JsonbBuilder.create();
var client = HttpClient.newHttpClient();
var httpRequest = HttpRequest.newBuilder()
.uri(new URI(apiUrl))
.version(HttpClient.Version.HTTP_2)
.header("Content-Type", "application/json")
.POST(HttpRequest.BodyPublishers.ofString(jsonb.toJson(jsonObject)))
.build();
client.sendAsync(httpRequest, HttpResponse.BodyHandlers.ofString());
It must send a JSON request and receive JSON response. HttpClient, introduced in Java11, has sendAsync native method, so I try to use it. But I don't understand fully how it works, so I have no success.

Writing data to an excel(CSV) file

I have created one servlet code. The main goal is to write data from a database to a csv file. In my code, the csv file gets downloaded successfully, but the file remains empty. The project contains javacsv.jar file. I don't have any clue why that is. Please shed some light on how I can achieve this. For a summary please read the points below:
Access data from database table.
Then write that data into the csv file with table format like < td >data here< / td> . [I want this format because I can format that data as per my requirements.]
Please shed some light guys peace :) Refer this Servlet code below:
public class excelServletFile extends HttpServlet {
String name = "";
String email = "";
String eid = "Username";
protected void processRequest(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
response.setContentType("text/html;charset=UTF-8");
try {
String fileName = eid + "Data.csv";
ServletContext context = getServletContext();
String mimeType = context.getMimeType(fileName);
if (mimeType == null) {
mimeType = "application/octet-stream";
}
response.setContentType(mimeType);
String headerKey = "Content-Disposition";
String headerValue = String.format("attachment; filename=\"%s\"", fileName);
response.setHeader(headerKey, headerValue);
OutputStream outStream = response.getOutputStream();
ConnectionClass cn = new ConnectionClass();
Connection con = cn.connectDb();
PreparedStatement ps;
ResultSet rs;
Charset cs = Charset.forName("UTF-8");
Writer writer = new PrintWriter(System.out);
writer.flush();
writer.append("NAME");
writer.append("EMAIL");
ps = con.prepareStatement("select name,email from user");
rs = ps.executeQuery();
while (rs.next()) {
name = rs.getString("name");
email = rs.getString("email");
PrintWriter out = response.getWriter();
out.println(name);
out.println(email);
writer.append(name);
writer.append(email);
}
writer.close();
} catch (Exception e) {
System.out.println(e);
}
}

package servletProject;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.io.OutputStream;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.util.Arrays;
import javax.servlet.ServletContext;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
#WebServlet("/govinds")
public class CSVServlet extends HttpServlet {
/**
*
*/
private static final long serialVersionUID = 1L;
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
processRequest(req, resp);
}
protected void processRequest(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
FileWriter writer = null;
ResultSet rs = null;
Connection con = null;
PreparedStatement ps = null;
try {
String fileName = "D:/Data.csv";
System.out.println(fileName);
ConnectionClass cn = new ConnectionClass();
con = cn.connectDb();
System.out.println("fileName" + fileName);
writer = new FileWriter(fileName);
// Write the CSV file header
CSVUtils.writeLine(writer, Arrays.asList("NAME", "email"));
ps = con.prepareStatement("select firstName,email from employees");
rs = ps.executeQuery();
while (rs.next()) {
System.out.println(rs.getString("firstName"));
CSVUtils.writeLine(writer, Arrays.asList(rs.getString("firstName"), rs.getString("email")));
}
writer.flush();
writer.close();
download(request, response);
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
rs.close();
ps.close();
con.close();
} catch (Exception e) {
}
}
}
public void download(HttpServletRequest request, HttpServletResponse response) throws IOException {
String filePath = "D:/Data.csv";
File downloadFile = new File(filePath);
FileInputStream inStream = new FileInputStream(downloadFile);
// if you want to use a relative path to context root:
String relativePath = getServletContext().getRealPath("");
System.out.println("relativePath = " + relativePath);
// obtains ServletContext
ServletContext context = getServletContext();
// gets MIME type of the file
String mimeType = context.getMimeType(filePath);
if (mimeType == null) {
// set to binary type if MIME mapping not found
mimeType = "application/octet-stream";
}
System.out.println("MIME type: " + mimeType);
// modifies response
response.setContentType(mimeType);
response.setContentLength((int) downloadFile.length());
// forces download
String headerKey = "Content-Disposition";
String headerValue = String.format("attachment; filename=\"%s\"", downloadFile.getName() + ".csv");
response.setHeader(headerKey, headerValue);
// obtains response's output stream
OutputStream outStream = response.getOutputStream();
byte[] buffer = new byte[4096];
int bytesRead = -1;
while ((bytesRead = inStream.read(buffer)) != -1) {
outStream.write(buffer, 0, bytesRead);
}
inStream.close();
outStream.close();
}
}
// DB connection File
public class ConnectionClass {
public Connection connectDb() {
Connection con=null;
try {
Class.forName("com.mysql.jdbc.Driver");
con=DriverManager.getConnection("jdbc:mysql://localhost:3306/classicmodels","root","root");
}catch (Exception e) {
con=null;
}
if(con!=null)
System.out.println("connected");
else
System.out.println("not connected");
return con;
}
}
//CSVUtil Files
package servletProject;
import java.io.IOException;
import java.io.Writer;
import java.util.List;
public class CSVUtils {
private static final char DEFAULT_SEPARATOR = ',';
public static void writeLine(Writer w, List<String> values) throws IOException {
writeLine(w, values, DEFAULT_SEPARATOR, ' ');
}
public static void writeLine(Writer w, List<String> values, char separators) throws IOException {
writeLine(w, values, separators, ' ');
}
// https://tools.ietf.org/html/rfc4180
private static String followCVSformat(String value) {
String result = value;
if (result.contains("\"")) {
result = result.replace("\"", "\"\"");
}
return result;
}
public static void writeLine(Writer w, List<String> values, char separators, char customQuote) throws IOException {
boolean first = true;
// default customQuote is empty
if (separators == ' ') {
separators = DEFAULT_SEPARATOR;
}
StringBuilder sb = new StringBuilder();
for (String value : values) {
if (!first) {
sb.append(separators);
}
if (customQuote == ' ') {
sb.append(followCVSformat(value));
} else {
sb.append(customQuote).append(followCVSformat(value)).append(customQuote);
}
first = false;
}
sb.append("\n");
w.append(sb.toString());
}
}

Google SpeechClient io.grpc.StatusRuntimeException: UNAVAILABLE: Credentials failed to obtain metadata

I'm making an application with Google SpeechClient that has the requirements to set a GOOGLE_APPLICATION_CREDENTIALS environment variable that, once set, you can use the voice to text api.
My application is required to run in linux and windows. In linux it runs perfectly, however, on windows, when running the project, it throws an exception com.google.api.gax.rpc.UnavailableException: "io.grpc.StatusRuntimeException: UNAVAILABLE: Credentials failed to obtain metadata" when trying to run this thread
package Controller.Runnables;
import Controller.GUI.VoxSpeechGUIController;
import Model.SpokenTextHistory;
import com.google.api.gax.rpc.ClientStream;
import com.google.api.gax.rpc.ResponseObserver;
import com.google.api.gax.rpc.StreamController;
import com.google.cloud.speech.v1.*;
import com.google.protobuf.ByteString;
import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.DataLine;
import javax.sound.sampled.TargetDataLine;
import java.io.IOException;
import java.util.ArrayList;
public class SpeechRecognizerRunnable implements Runnable{
private VoxSpeechGUIController controller;
public SpeechRecognizerRunnable(VoxSpeechGUIController voxSpeechGUIController) {
this.controller = voxSpeechGUIController;
}
#Override
public void run() {
MicrofoneRunnable micrunnable = MicrofoneRunnable.getInstance();
Thread micThread = new Thread(micrunnable);
ResponseObserver<StreamingRecognizeResponse> responseObserver = null;
try (SpeechClient client = SpeechClient.create()) {
ClientStream<StreamingRecognizeRequest> clientStream;
responseObserver =
new ResponseObserver<StreamingRecognizeResponse>() {
ArrayList<StreamingRecognizeResponse> responses = new ArrayList<>();
public void onStart(StreamController controller) {}
public void onResponse(StreamingRecognizeResponse response) {
try {
responses.add(response);
StreamingRecognitionResult result = response.getResultsList().get(0);
// There can be several alternative transcripts for a given chunk of speech. Just
// use the first (most likely) one here.
SpeechRecognitionAlternative alternative = result.getAlternativesList().get(0);
String transcript = alternative.getTranscript();
System.out.printf("Transcript : %s\n", transcript);
String newText = SpokenTextHistory.getInstance().getActualSpeechString() + " " + transcript;
SpokenTextHistory.getInstance().setActualSpeechString(newText);
controller.setLabelText(newText);
}
catch (Exception ex){
System.out.println(ex.getMessage());
ex.printStackTrace();
}
}
public void onComplete() {
}
public void onError(Throwable t) {
System.out.println(t);
}
};
clientStream = client.streamingRecognizeCallable().splitCall(responseObserver);
RecognitionConfig recognitionConfig =
RecognitionConfig.newBuilder()
.setEncoding(RecognitionConfig.AudioEncoding.LINEAR16)
.setLanguageCode("pt-BR")
.setSampleRateHertz(16000)
.build();
StreamingRecognitionConfig streamingRecognitionConfig =
StreamingRecognitionConfig.newBuilder().setConfig(recognitionConfig).build();
StreamingRecognizeRequest request =
StreamingRecognizeRequest.newBuilder()
.setStreamingConfig(streamingRecognitionConfig)
.build(); // The first request in a streaming call has to be a config
clientStream.send(request);
try {
// SampleRate:16000Hz, SampleSizeInBits: 16, Number of channels: 1, Signed: true,
// bigEndian: false
AudioFormat audioFormat = new AudioFormat(16000, 16, 1, true, false);
DataLine.Info targetInfo =
new DataLine.Info(
TargetDataLine.class,
audioFormat); // Set the system information to read from the microphone audio
// stream
if (!AudioSystem.isLineSupported(targetInfo)) {
System.out.println("Microphone not supported");
System.exit(0);
}
// Target data line captures the audio stream the microphone produces.
micrunnable.targetDataLine = (TargetDataLine) AudioSystem.getLine(targetInfo);
micrunnable.targetDataLine.open(audioFormat);
micThread.start();
long startTime = System.currentTimeMillis();
while (!micrunnable.stopFlag) {
long estimatedTime = System.currentTimeMillis() - startTime;
if (estimatedTime >= 55000) {
clientStream.closeSend();
clientStream = client.streamingRecognizeCallable().splitCall(responseObserver);
request =
StreamingRecognizeRequest.newBuilder()
.setStreamingConfig(streamingRecognitionConfig)
.build();
startTime = System.currentTimeMillis();
} else {
request =
StreamingRecognizeRequest.newBuilder()
.setAudioContent(ByteString.copyFrom(micrunnable.sharedQueue.take()))
.build();
}
clientStream.send(request);
}
} catch (Exception e) {
System.out.println(e);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
I've been working hard for hours and have not found a solution that solves my problem.
It is worth mentioning that the environment variable is being set correctly.
Has anyone ever had this problem with Google? What should I do to fix this?
This is my envirounment variable creator:
PS: I`ve already tried use all google alternatives to validate credentials, but all return me errors.
package Controller.Autentication;
import java.io.*;
import java.lang.reflect.Field;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
public class GoogleAuthentication {
private static final String GOOGLE_APPLICATION_CREDENTIALS = "GOOGLE_APPLICATION_CREDENTIALS";
private static final String VoxSpeechFolder = ".vox";
private static final String GoogleAuthenticationJsonFile = "VoxAuthentication.json";
public static void setupGoogleCredentials() {
String directory = defaultDirectory();
directory += File.separator+VoxSpeechFolder;
File voxPath = new File(directory);
if (!voxPath.exists()) {
voxPath.mkdirs();
}
ClassLoader classLoader = new GoogleAuthentication().getClass().getClassLoader();
File srcFile = new File(classLoader.getResource(GoogleAuthenticationJsonFile).getFile());
if(srcFile.exists()){
try {
String voxDestPath = defaultDirectory() + File.separator + VoxSpeechFolder +File.separator+ GoogleAuthenticationJsonFile;
File destFile = new File(voxDestPath);
copyFile(srcFile,destFile);
} catch (IOException e) {
e.printStackTrace();
}
}
try {
Map<String,String> googleEnv = new HashMap<>();
String path = defaultDirectory() +File.separator+ VoxSpeechFolder +File.separator+ GoogleAuthenticationJsonFile;
googleEnv.put(GOOGLE_APPLICATION_CREDENTIALS, path);
setGoogleEnv(googleEnv);
} catch (Exception e) {
e.printStackTrace();
}
}
static void copyFile(File sourceFile, File destFile)
throws IOException {
InputStream inStream ;
OutputStream outStream ;
System.out.println(destFile.getPath());
if(destFile.createNewFile()){
inStream = new FileInputStream(sourceFile);
outStream = new FileOutputStream(destFile);
byte[] buffer = new byte[1024];
int length;
while ((length = inStream.read(buffer)) > 0){
outStream.write(buffer, 0, length);
}
inStream.close();
outStream.close();
}
}
static String defaultDirectory()
{
String OS = getOperationSystem();
if (OS.contains("WIN"))
return System.getenv("APPDATA");
else if (OS.contains("MAC"))
return System.getProperty("user.home") + "/Library/Application "
+ "Support";
else if (OS.contains("LINUX")) {
return System.getProperty("user.home");
}
return System.getProperty("user.dir");
}
static String getOperationSystem() {
return System.getProperty("os.name").toUpperCase();
}
protected static void setGoogleEnv(Map<String, String> newenv) throws Exception {
try {
Class<?> processEnvironmentClass = Class.forName("java.lang.ProcessEnvironment");
Field theEnvironmentField = processEnvironmentClass.getDeclaredField("theEnvironment");
theEnvironmentField.setAccessible(true);
Map<String, String> env = (Map<String, String>) theEnvironmentField.get(null);
env.putAll(newenv);
Field theCaseInsensitiveEnvironmentField = processEnvironmentClass.getDeclaredField("theCaseInsensitiveEnvironment");
theCaseInsensitiveEnvironmentField.setAccessible(true);
Map<String, String> cienv = (Map<String, String>) theCaseInsensitiveEnvironmentField.get(null);
cienv.putAll(newenv);
} catch (NoSuchFieldException e) {
Class[] classes = Collections.class.getDeclaredClasses();
Map<String, String> env = System.getenv();
for(Class cl : classes) {
if("java.util.Collections$UnmodifiableMap".equals(cl.getName())) {
Field field = cl.getDeclaredField("m");
field.setAccessible(true);
Object obj = field.get(env);
Map<String, String> map = (Map<String, String>) obj;
map.clear();
map.putAll(newenv);
}
}
}
String genv = System.getenv(GOOGLE_APPLICATION_CREDENTIALS);
System.out.println(genv);
}
}

Spring Boot Unit Testing

I'm currently developing a Rest service in Spring Boot and am attempting to develop unit tests. My Rest controller takes the request and utilizes an Sftp Connector defined within another package to directly interface with our servers. I've defined my test cases to mock the behavior of the Sftp Connector rather than have it execute with the mock data being fed into the Rest endpoints. I've been getting a ton of error logs during testing showing that exceptions are being raised from the Sftp connector. Which makes sense if the mock data sent to the endpoints is still being fed into the connector class. I'm guessing that I'm not mocking the connector properly. I've been unsuccessful so far in locating a method to mock the connector. Is this possible? Or can I only mock the classes defined within the same package as the test class?
Controller:
package RestAPI;
import JSch.SFTP.SftpConnector;
import JSch.SFTP.fileInfo;
import io.swagger.annotations.Api;
import io.swagger.annotations.ApiOperation;
import org.apache.commons.io.FilenameUtils;
import org.apache.commons.io.IOUtils;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;
import java.io.*;
import java.util.Collection;
import java.util.Iterator;
import java.util.Properties;
#RestController
#Api( value = "Test Rest Controller", description = "Basic test for SFTP
connector with very basic operations.")
public class SftpRestController {
private Properties getProperties(){ //used to get connection values from
external file
InputStream input = null;
Properties prop = new Properties(); //new property
try{
if(System.getProperty("os.name").contains("Windows")){
input = new FileInputStream("C:\\tmp\\application.properties"); //get resources stream
}
else{
input = new FileInputStream("application.properties");
}
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
String line;
while((line = reader.readLine()) != null){
String[] split = line.split("=");
prop.put(split[0], split[1]);
}
}catch(IOException IO){
IO.printStackTrace();
}finally{
if(input != null )
try{
input.close(); //close the stream
}catch(IOException IO2){
IO2.printStackTrace();
}
}
System.out.println(prop.getProperty("sftp.port"));
return prop;
}
#ApiOperation(value = "Check that service is up and running.", response = String.class)
#RequestMapping("/")
public #ResponseBody String index(){ return "Rest Test Service is up and running";}
#ApiOperation(value = "Upload a file", response = String.class)
#PostMapping(value="/Upload") //uploads files so long as the files are defined in one structure in the form body
public #ResponseBody String multipartUpload(#RequestBody MultipartFile[] files, #RequestParam("dest") String dest){
String fileMessage = ""; //need something to send back to the caller
SftpConnector connector = new SftpConnector(this.getProperties()); //plugs properties into the constructor and opens a connection
for( int i = 0; i < files.length; i++ ){ //for each file uploaded
connector.uploadFile(files[i], dest); //call the upload method with the file reference and where it's going
fileMessage = fileMessage + files[i].getOriginalFilename() + " has been uploaded to remote directory \n";
}
connector.closeSFTPConnection(); //manually close the connection
return fileMessage;
}
#ApiOperation(value = "Downloads a file over SFTP and writes it to local file system specified", response = String.class)
#GetMapping(value="/Download")
public #ResponseBody String downloadFile(#RequestParam("fileName") String fName, #RequestParam("dest") String dest){
SftpConnector connector = new SftpConnector(this.getProperties()); //new connector
connector.downloadFile(fName, dest); //pull the specified file down
connector.closeSFTPConnection(); //close the connection
return fName + " has been downloaded to the specified local directory";
}
#ApiOperation(value = "Moves all files on a remote server from one directory to another based on file type specified", response = String.class)
#PutMapping(value="/moveFiles")
public #ResponseBody String moveFiles(#RequestParam("dir") String origin, #RequestParam("fileType") String type,#RequestParam("dest") String dest){
SftpConnector connector = new SftpConnector(this.getProperties());
connector.moveFiles(origin, type, dest); //moves a group of files based file type from one directory to another
connector.closeSFTPConnection();
return "All files have been moved.";
}
#ApiOperation(value = "Moves a single specific file", response = String.class)
#GetMapping(value="/moveFile")
public #ResponseBody String moveFile(#RequestParam("dir") String origFile, #RequestParam("dest") String dest){
SftpConnector connector = new SftpConnector(this.getProperties());
connector.moveFile(origFile, dest); //moves a single file. file name must be specified.
connector.closeSFTPConnection();
return FilenameUtils.getName(origFile) + " has been moved to " + dest;
}
#ApiOperation(value = "Gets a specified file stream and returns it as a string", response = String.class)
#GetMapping(value="/readFile")
public #ResponseBody String readFile(#RequestParam("path") String path){
String fileContents;
try{
SftpConnector connector = new SftpConnector(this.getProperties());
InputStream fis = connector.readFile(path); //gets an open file stream
fileContents = IOUtils.toString(fis,"UTF-8"); //takes a file stream, reads into variable in specified encoding
fis.close(); //closes the stream
connector.closeSFTPConnection();
}catch(IOException IO){
fileContents = IO.toString();
}
return fileContents;
}
#ApiOperation(value="Returns a list of file names as a string", response = String.class)
#GetMapping(value="/listFiles")
public #ResponseBody String fileNames(#RequestParam("dir") String origin, #RequestParam("fileType") String type){
String base = "";
try{
SftpConnector connector = new SftpConnector(this.getProperties());
Collection<fileInfo> files = connector.listFiles(origin, type); //gets a list of files with name, type, and an input stream
Iterator<fileInfo> iterator = files.iterator(); //iterator to roll over collection
while(iterator.hasNext()){ //while not empty
fileInfo thisFile = iterator.next(); //get the next fileInfo
base = base + thisFile.getName() + '\n' ; //write the name to the base string
thisFile.getFileStream().close(); //close the file stream
}
connector.closeSFTPConnection();
}catch(Exception ex){
ex.printStackTrace();
}
return base;
}
#ApiOperation(value = "Moves a file by the connectors readFile and writeFile methods rather than SFTP rename", response = String.class)
#GetMapping(value="/test")
public #ResponseBody String StreamTest(#RequestParam("file") String file, #RequestParam("dest") String dest){
SftpConnector connector = new SftpConnector(this.getProperties());
connector.writeFile(dest, FilenameUtils.getName(file), connector.readFile(file));
connector.closeSFTPConnection();
return file + " has been successfully read, and written to destination";
}
}
Sftp Connector:
package JSch.SFTP;
import com.jcraft.jsch.*;
import org.apache.commons.io.FilenameUtils;
import org.springframework.stereotype.Component;
import org.springframework.web.multipart.MultipartFile;
import java.io.*;
import java.util.*;
#Component
public class SftpConnector {
private String sftpHost;
private String sftpUser;
private int sftpPort;
private String sftpPass;
private Session session = null;
private ChannelSftp channelSftp = null;
public String getSftpHost() {
return sftpHost;
}
public void setSftpHost(String sftpHost) {
this.sftpHost = sftpHost;
}
public String getSftpUser() {
return sftpUser;
}
public void setSftpUser(String sftpUser) {
this.sftpUser = sftpUser;
}
public int getSftpPort() {
return sftpPort;
}
public void setSftpPort(int sftpPort) {
this.sftpPort = sftpPort;
}
public String getSftpPass() {
return sftpPass;
}
public void setSftpPass(String sftpPass) {
this.sftpPass = sftpPass;
}
public void setConnectionVars(String host, String user, String password, int port){
this.setSftpHost(host);
this.setSftpPort(port);
this.setSftpUser(user);
this.setSftpPass(password);
}
public SftpConnector(Properties prop){
this.setConnectionVars(prop.getProperty("sftp.host"), prop.getProperty("sftp.user"), prop.getProperty("sftp.pass"), Integer.parseInt(prop.getProperty("sftp.port"))); //set the connection variables
this.openSFTPConnection(); //open the sftp connection
}
public Session getSession() {
return session;
}
public void setSession(Session session) {
this.session = session;
}
public ChannelSftp getChannelSftp() {
return channelSftp;
}
public void setChannelSftp(ChannelSftp channelSftp) {
this.channelSftp = channelSftp;
}
public void openSFTPConnection(){
try{
JSch jsch = new JSch(); //create the new JSch var
Session baseSession = jsch.getSession(this.getSftpUser(), this.getSftpHost(),this.getSftpPort()); //establish the ssh session info
baseSession.setPassword(this.getSftpPass()); //auth over password
baseSession.setConfig("StrictHostKeyChecking","no"); // don't require hosting key check will need to change after testing
baseSession.setConfig("PreferredAuthentications","password"); //used to omit kerberos authentication
this.setSession(baseSession); //set the session to private variable
this.getSession().connect(); //open the session
ChannelSftp tempChannel = (ChannelSftp) this.getSession().openChannel("sftp"); //create an SFTP channel
this.setChannelSftp(tempChannel); //set the channel to the private variable
this.getChannelSftp().connect(); //open the SFTP connection
}catch(JSchException e){
e.printStackTrace();
}
}
public void closeSFTPConnection(){
try{
if(this.getChannelSftp().isConnected()){ //if the channel is open so must be the session close both
this.getChannelSftp().disconnect();
this.getSession().disconnect();
}else{ //if the channel is closed check to see if the session is still open
if(this.getSession().isConnected()){
this.getSession().disconnect();
}
}
}catch(Exception e){
e.printStackTrace();
}
}
public Collection<fileInfo> listFiles(String dir, String filter){
Collection<fileInfo> files = new ArrayList<fileInfo>();
try{
if(this.getChannelSftp() == null || !this.getChannelSftp().isConnected()) //make sure that the JSch sessions been opened.
this.openSFTPConnection();
this.getChannelSftp().cd(dir); //set the working directory
Vector<ChannelSftp.LsEntry> fileList = this.getChannelSftp().ls(filter); //Get a listing of the files in the directory depending on a filter
Iterator<ChannelSftp.LsEntry> iterator = fileList.iterator(); //iterate over the collection
while(iterator.hasNext()) {
ChannelSftp.LsEntry entry = iterator.next();
fileInfo thisFile = new fileInfo(entry.getFilename(), entry.getAttrs().getAtimeString(), FilenameUtils.getExtension(entry.getFilename()), this.getChannelSftp().get(entry.getFilename())); //get a buffered input stream for each file.
files.add(thisFile);
}
}catch(SftpException e){
e.printStackTrace();
}
return files;
}
public InputStream readFile(String filePath){ //built to read get a file stream from a given file path
InputStream inputStream = null;
try{
if(this.getChannelSftp() == null || !this.getChannelSftp().isConnected()) //if the channel isn't open, open it.
this.openSFTPConnection(); //open connection
inputStream = this.getChannelSftp().get(filePath); //get the stream
}catch(SftpException ex){
ex.printStackTrace();
}
return inputStream;
}
public String uploadFile(MultipartFile file, String dest){
try{
System.out.println(this.getSession().getConfig("FileUploadBaseSizeLimit"));
if(this.getChannelSftp() == null || !this.getChannelSftp().isConnected()) //if not connected open the connection
this.openSFTPConnection();
this.getChannelSftp().cd(dest); //set working dir
this.getChannelSftp().put(file.getInputStream(), dest + '/' + file.getOriginalFilename()); //get the input stream from the multipart file and put it in the destination
}catch(IOException IO){
IO.printStackTrace();
}catch(SftpException sftp){
sftp.printStackTrace();
}
return file.getOriginalFilename() + " has been uploaded";
}
public String downloadFile(String orig, String dest){
try{
if(this.getChannelSftp() == null || !this.getChannelSftp().isConnected()) //if not connected
this.openSFTPConnection();
File download = new File(dest + '/' + FilenameUtils.getName(orig)); //create a new local file
OutputStream outputStream = new FileOutputStream(download); //generate an output stream to the file
byte[] buffer = new byte[10000000]; //10MB buffer instance
BufferedInputStream bufferedInputStream = new BufferedInputStream(this.getChannelSftp().get(orig)); //create a buffered input stream off the input stream from the ssh get
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(outputStream); // and a buffered output stream off the output stream we created
int readCount; //integer link to the buffer
while((readCount = bufferedInputStream.read(buffer)) > 0){ //read the buffered input stream into the buffer. while it's not empty...
bufferedOutputStream.write(buffer, 0, readCount); //write the buffer to the output stream
}
bufferedInputStream.close(); //close the streams
bufferedOutputStream.close();
outputStream.close();
}catch(IOException IO){
IO.printStackTrace();
}catch(SftpException sftp){
sftp.printStackTrace();
}
return "File " + FilenameUtils.getName(orig) + " has been downloaded."; //return that we've successfully uploaded the file
}
public void writeFile(String dir, String fileName, InputStream fileIS) { //writes a file given the destination directory, fileName, and an input stream
try{
if(this.getChannelSftp() == null || !this.getChannelSftp().isConnected() ) //if the connection isn't already open, open it
this.openSFTPConnection();
this.getChannelSftp().cd(dir); //set the working dir for better shorthand
this.getChannelSftp().put(fileIS, fileName); //put the file.
fileIS.close(); //close the file stream
}catch(SftpException e){
e.printStackTrace();
}catch(IOException IO){
IO.printStackTrace();
}
}
public void moveFile(String path, String dest){
try{
if(this.getChannelSftp() == null || !this.getChannelSftp().isConnected()){ //open connection if not done already
this.openSFTPConnection();
}
this.getChannelSftp().rename(path, dest + '/' + FilenameUtils.getName(path)); //ssh command to move the file.
}catch(SftpException e){
e.printStackTrace();
}
}
public String moveFiles(String dir, String filter, String dest){
try{
if(this.getChannelSftp() == null || !this.getChannelSftp().isConnected()){ //open if not done already
this.openSFTPConnection();
}
Collection<fileInfo> files = this.listFiles(dir, filter); //invoke the listFiles method and convert to an array
Iterator<fileInfo> iterator = files.iterator(); //iterator
while( iterator.hasNext()){ //while there is another value in the collection
fileInfo thisFile = iterator.next(); // have to store the fileInfo somewhere. can't just keep calling next.
this.getChannelSftp().rename(dir + '/' + thisFile.getName(), dest + '/' + thisFile.getName());
}
this.closeSFTPConnection(); //close the connection
}catch(Exception ex){
ex.printStackTrace();
}
return "all files moved";
}
}
Test Class:
package RestAPI;
import JSch.SFTP.SftpConnector;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mockito;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.boot.test.mock.mockito.MockBean;
import org.springframework.mock.web.MockMultipartFile;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.MvcResult;
import org.springframework.test.web.servlet.RequestBuilder;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;
import java.io.FileInputStream;
import java.io.InputStream;
#RunWith(SpringRunner.class)
#WebMvcTest(value = SftpRestController.class, secure = false)
public class SftpRestUnitTest {
#Autowired
private MockMvc mockMvc;
#MockBean
SftpConnector connector ;
#Test
public void testFileUpload() throws Exception{
MockMultipartFile testFile1 = new MockMultipartFile("data", "WSUID1.csv", "application/csv", "some xml".getBytes());
Mockito.when(
connector.uploadFile(Mockito.any(),Mockito.anyString())
).thenReturn("WSUID1.csv has been uploaded");
MvcResult result = mockMvc.perform(MockMvcRequestBuilders.multipart("/Upload").file(testFile1).param("dest", "/tmp/test")).andReturn();
System.out.println(result.getResponse().getContentAsString());
assert result.getResponse().getContentAsString().equals("WSUID1.csv has been uploaded to remote directory \n");
}
#Test
public void testDownload() throws Exception{
String expected = "TestFile.csv has been downloaded to the specified local directory";
Mockito.when(
connector.downloadFile(Mockito.anyString(), Mockito.anyString())
).thenReturn("File TestFile.csv has been downloaded.");
RequestBuilder request = MockMvcRequestBuilders.get("/Download").param("fileName","/tmp/TestFile.csv").param("dest", "/tmp");
MvcResult result = mockMvc.perform(request).andReturn();
System.out.println(result.getResponse().getContentAsString());
assert result.getResponse().getContentAsString().equals(expected);
}
#Test
public void testMoveFiles() throws Exception{
Mockito.when(
connector.moveFiles(Mockito.anyString(),Mockito.anyString(),Mockito.anyString())
).thenReturn("all files moved");
RequestBuilder request = MockMvcRequestBuilders.put("/moveFiles").param("dir","/IB_Test").param("fileType", "*.csv").param("dest", "/InternatlPgms/INTO/Archive/Receipt");
MvcResult result = mockMvc.perform(request).andReturn();
System.out.println(result.getResponse().getContentAsString());
assert result.getResponse().getStatus() == 200;
}
#Test
public void testReadFile() throws Exception{
String expected = "greetings from java test";
InputStream mockStream = Mockito.mock(FileInputStream.class);
Mockito.when(
connector.readFile(Mockito.anyString())
).thenReturn(mockStream);
RequestBuilder request = MockMvcRequestBuilders.get("/readFile").param("path", "IB_Test");
MvcResult result = mockMvc.perform(request).andReturn();
System.out.println(result.getResponse().getContentAsString());
assert result.equals(expected);
}
}
Error Logs:
The specified file is a directory.
at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2873)
at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1337)
at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1290)
at JSch.SFTP.SftpConnector.readFile(SftpConnector.java:133)
at RestAPI.SftpRestController.readFile(SftpRestController.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:189)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:800)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1038)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:897)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882)
at org.springframework.test.web.servlet.TestDispatcherServlet.service(TestDispatcherServlet.java:71)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.springframework.mock.web.MockFilterChain$ServletFilterProxy.doFilter(MockFilterChain.java:166)
at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:133)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:133)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:92)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:133)
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
Any help is greatly appreciated.

You create a new instance of SftpConnector in every method of your controller. It is not an injected bean so it won't be replaced with your mock.
For an example have a look at this article (section 5):
https://www.baeldung.com/spring-boot-testing

Image compression or conversion

Simple i want to apply image compression using PNG/JPEG/Bitmap file.
Android we have Bitmap.CompressFormat to compressed our bitmap file and use for further operation.
Bitmap.CompressFormat class allow to compress in 3 format as below :
JPEG
PNG
WEBP
My query is i want to compress file in any on of below format :
JBIG2
TIFF G4
TIFF LZW
I have found some image compression library like ImageIo & ImageMagick but didn't get any success. I want to use this file to upload on AmazonServer. Please guide me how to achieve this or is there any other option to upload image on amazon server.
Thanks for your time.

I don't know about those file's compression but i created this class to upload files programatically into an Amazon s3 bucket that uses the Amazon SDK api:
package com.amazon.util;
import com.amazonaws.AmazonClientException;
import com.amazonaws.auth.PropertiesCredentials;
import com.amazonaws.regions.Region;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.model.CannedAccessControlList;
import com.amazonaws.services.s3.model.ObjectMetadata;
import com.amazonaws.services.s3.model.PutObjectRequest;
import com.amazonaws.services.s3.model.PutObjectResult;
import com.amazonaws.services.s3.model.S3ObjectSummary;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URL;
public class AmazonS3Files {
private static final String existingBucketName = "bucketName";
private final String strProperties = "accessKey = MYACESSKEY \n"
+ "secretKey = my+secret+key";
private static final String urlRegion = "https://s3.amazonaws.com/";
public static final String urlPathS3 = urlRegion + existingBucketName;
public String UploadFile(InputStream inFile, String pathDocument, String fileName) {
return UploadFile(inFile, pathDocument, fileName, false);
}
public void deleteObjectsInFolder(String folderPath) {
InputStream inputStreamCredentials = new ByteArrayInputStream(strProperties.getBytes());
folderPath = folderPath.replace('\\', '/');
if (folderPath.charAt(folderPath.length() - 1) == '/') {
folderPath = folderPath.substring(0, folderPath.length() - 1);
}
if (folderPath.charAt(0) == '/') {
folderPath = folderPath.substring(1, folderPath.length());
}
try {
AmazonS3 s3Client = new AmazonS3Client(new PropertiesCredentials(inputStreamCredentials));
for (S3ObjectSummary file : s3Client.listObjects(existingBucketName, folderPath).getObjectSummaries()) {
s3Client.deleteObject(existingBucketName, file.getKey());
}
} catch (IOException | AmazonClientException e) {
System.out.println(e);
}
}
public void deleteFile(String filePath) {
InputStream inputStreamCredentials = new ByteArrayInputStream(strProperties.getBytes());
filePath = filePath.replace('\\', '/');
if (filePath.charAt(0) == '/') {
filePath = filePath.substring(1, filePath.length());
}
try {
AmazonS3 s3Client = new AmazonS3Client(new PropertiesCredentials(inputStreamCredentials));
s3Client.deleteObject(existingBucketName, filePath);
} catch (IOException | AmazonClientException e) {
System.out.println(e);
}
}
public String UploadFile(InputStream inFile, String pathDocument, String fileName, boolean bOverwiteFile) {
InputStream inputStreamCredentials = new ByteArrayInputStream(strProperties.getBytes());
String amazonFileUploadLocationOriginal;
String strFileExtension = fileName.substring(fileName.lastIndexOf("."), fileName.length());
fileName = fileName.substring(0, fileName.lastIndexOf("."));
fileName = fileName.replaceAll("[^A-Za-z0-9]", "");
fileName = fileName + strFileExtension;
pathDocument = pathDocument.replace('\\', '/');
try {
if (pathDocument.charAt(pathDocument.length() - 1) == '/') {
pathDocument = pathDocument.substring(0, pathDocument.length() - 1);
}
if (pathDocument.charAt(0) == '/') {
pathDocument = pathDocument.substring(1, pathDocument.length());
}
amazonFileUploadLocationOriginal = existingBucketName + "/" + pathDocument;
AmazonS3 s3Client = new AmazonS3Client(new PropertiesCredentials(inputStreamCredentials));
s3Client.setRegion(Region.getRegion(Regions.SA_EAST_1));
ObjectMetadata objectMetadata = new ObjectMetadata();
objectMetadata.setContentLength(inFile.available());
if (!bOverwiteFile) {
boolean bFileServerexists = true;
int tmpIntEnum = 0;
while (bFileServerexists) {
String tmpStrFile = fileName;
if (tmpIntEnum > 0) {
tmpStrFile = fileName.substring(0, fileName.lastIndexOf(".")) + "(" + tmpIntEnum + ")" + fileName.substring(fileName.lastIndexOf("."), fileName.length());
}
if (!serverFileExists(urlRegion + amazonFileUploadLocationOriginal + "/" + tmpStrFile)) {
bFileServerexists = false;
fileName = tmpStrFile;
}
tmpIntEnum++;
}
}
String strFileType = fileName.substring(fileName.lastIndexOf("."), fileName.length());
if (strFileType.toUpperCase().equals(".jpg".toUpperCase())) {
objectMetadata.setContentType("image/jpeg");
} else if (strFileType.toUpperCase().equals(".png".toUpperCase())) {
objectMetadata.setContentType("image/png");
} else if (strFileType.toUpperCase().equals(".gif".toUpperCase())) {
objectMetadata.setContentType("image/gif");
} else if (strFileType.toUpperCase().equals(".gmap".toUpperCase())) {
objectMetadata.setContentType("text/plain");
}
PutObjectRequest putObjectRequest = new PutObjectRequest(amazonFileUploadLocationOriginal, fileName, inFile, objectMetadata).withCannedAcl(CannedAccessControlList.PublicRead);
PutObjectResult result = s3Client.putObject(putObjectRequest);
return "/" + pathDocument + "/" + fileName;
} catch (Exception e) {
// TODO: handle exception
return null;
}
}
public boolean serverFileExists(String URLName) {
try {
HttpURLConnection.setFollowRedirects(false);
HttpURLConnection con =
(HttpURLConnection) new URL(URLName).openConnection();
con.setRequestMethod("HEAD");
return (con.getResponseCode() == HttpURLConnection.HTTP_OK);
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
}
And for usage with your file:
BufferedImage img = null;
try {
img = ImageIO.read(new File("file.jpg"));
String strReturn = AmazonS3Files.UploadFile(new ByteArrayInputStream(((DataBufferByte)(img).getRaster().getDataBuffer()).getData()), "path/to/file", "newfilename.jpg"); //Returns null if the upload doesn't work or the s3 file path of the uploaded file
} catch (IOException e) {
//Handle Exception
}