I have two classes DB_Search and Elastic_Search which generate the response from STAGE and PROD, but when I ran these two classes parallel using the testng.xml file, one class got a complete response but the other one got a session-expired-response.
These are the two classes:
public class DB_Search {
#Test
public void FinBlock() throws IOException, ParseException {
SessionFilter session = new SessionFilter();
RestAssured.baseURI = "login_api";
String Response = given().auth().preemptive().basic("******", "****").filter(session).when()
.get(RestAssured.baseURI).then().extract().response().getCookie("RAY_SESSION_ID");
Response resp = given().cookie("RAY_SESSION_ID", Response).filter(session).when().get("API").then().extract()
.response();
FileWriter filewriter = new FileWriter("C:\\Users\\***\\eclipse-workspace\\API_test\\Files\\output.json");
BufferedWriter bufferedWriter = new BufferedWriter(filewriter);
bufferedWriter.write(resp.asString());
bufferedWriter.close();
}
}
public class Elastic_search {
#Test
public void FinBlock() throws IOException, ParseException {
SessionFilter session = new SessionFilter();
RestAssured.baseURI = "Login_API";
String Response = given().auth().preemptive().basic("****", "***").filter(session).when()
.get(RestAssured.baseURI).then().extract().response().getCookie("RAY_SESSION_ID");
Response resp = given().cookie("RAY_SESSION_ID", Response).filter(session).when().get("main_api").then()
.extract().response();
FileWriter filewriter = new FileWriter("C:\\Users\\***\\eclipse-workspace\\API_test\\Files\\output1.json");
BufferedWriter bufferedWriter = new BufferedWriter(filewriter);
bufferedWriter.write(resp.asString());
bufferedWriter.close();
}
}
Both your tests set value for static variable RestAssured.baseURI. Hence your threads interfere.
To fix that you should either to set base path in configuration of your particular request within each your test or use full path to the endpoint in get.
Related
I am very new to Spring Boot project.
I am writing backend code where I have a webmethod url which can download only one file at a time based on fileNo. It is invoked from the front-end when user enters fileNo and submits.
User can enter maximum 5 fileNo(comma-separated) at one time.
In that case I have to take each file no and and set it into my url, invoke it which will download 5 files and put it in one common folder.
Below code is working for one FileNo and downloading the file,
Is there anyway where I can set and invoke all 5 URLs concurrently, download all 5 files and put it in a same folder.
Or If I have to set it one by one in my URL then how to do it. What is the best way to do this. (went through few similar posts but couldn't fine anything for my solution). Looking for a solution here. Thanks
#SneakyThrows
#RequestMapping(value = "/{fileName:.+}", method = RequestMethod.GET)
#ApiResponses(value = {
#ApiResponse(code = 200, message = "Success"),
#ApiResponse(code = 500, message = "Server Error")
})
public ResponseEntity getCollateralDownloadData(#PathVariable("fileName") String fileName) {
String wmURL = "https://www.qa.referencesite.com/invoke/File_appDesigns.process:processTpfQuery?prdType=PTO&tpf_aif=one&SALESNO=&PRODNO=&OASN="+fileName;
try {
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM));
HttpEntity<String> entity = new HttpEntity<>(headers);
ResponseEntity<byte[]> response = restTemplate.build()
.exchange(wmURL, HttpMethod.GET, entity, byte[].class);
return ResponseEntity.ok()
.contentType(MediaType.parseMediaType("application/octet-stream"))
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + fileName + "\"")
.body(response.getBody());
} catch (Exception e) {
e.printStackTrace();
}
return ResponseEntity.status(HttpStatus.OK).body("Collateral Download Created successfully");
}
There are 2 approaches depending on your need :
The client may trigger requests concurrently, so the client has to send a request for each fileNo and you backend deals with it
The client may trigger only one request, so your backend should be modified to trigger subsequent calls, wait for result and build a common response. You may use Spring WebFlux facilities or Java ExecutorService.invokeAll() method to manage subsequent parallels requests.
Regards.
Here is complete Flow of code what I have done.
#RestController
public class DownloadOAFileController {
#Autowired
private RestTemplateBuilder restTemplate;
private static final int MYTHREADS = 30;
#SneakyThrows
#RequestMapping(value = "/downloadzippedFile/{fileName}", method = RequestMethod.GET)
#ApiResponses(value = {
#ApiResponse(code = 200, message = "Success"),
#ApiResponse(code = 500, message = "Server Error")
})
public ResponseEntity<Object> downloadzipFiles(#PathVariable("fileName") String fileName) throws IOException, ExecutionException, InterruptedException {
downloadMultipleFilesToRemoteDiskUsingThread(fileName);
String zipFilename = "/Users/kumars22/Downloads/Files.zip";
File file = new File(zipFilename);
InputStreamResource resource = new InputStreamResource(new FileInputStream(file));
HttpHeaders headers = new HttpHeaders();
headers.add("Content-Disposition",
String.format("attachment; filename=\"%s\"",file.getName()));
headers.add("Cache-Control","no-cache, no-store, no-revalidate");
headers.add("pragma","no-cache");
headers.add("Expires","0");
ResponseEntity<Object> responseEntity = ResponseEntity.ok().headers(headers)
.contentLength(file.length())
.contentType(MediaType.parseMediaType("application/octet-stream"))
.body(resource);
return responseEntity;
}
//Created a Directory where all files will be downloaded (Got some space in AWS for actual implementation.
public static void createDirectory() {
File file = new File("/Users/kumars22/Downloads/Files");
if (!file.exists()) {
file.mkdir();
}
}
public static void downloadMultipleFilesToRemoteDiskUsingThread( String fileName ) throws ExecutionException, InterruptedException, IOException {
createDirectory();
ExecutorService executor = Executors.newFixedThreadPool(MYTHREADS);
String[] serials = fileName.split(",");
String[] wmURLs = new String[serials.length];
for (int i =0;i<serials.length;i++) {
wmURLs[i] = "https://www.qa.referencesite.com/invoke/File_appDesigns.process:processTpfQuery?prdType=PTO&tpf_aif=one&SALESNO=&PRODNO=&OASN="+serials[i];
}
for (int i = 0; i < wmURLs.length; i++) {
String url = wmURLs[i];
Runnable worker = new MultipleCallThreadController(url,fileName,"James");
executor.execute(worker);
}
executor.shutdown();
// Wait until all threads are finish
while (!executor.isTerminated()) {
}
zipFiles("/Users/kumars22/Downloads/Files","/Users/kumars22/Downloads/Files.zip");
System.out.println("\nFinished all threads");
}
//Zip the Folder having all files
public static void zipFiles(String sourceDirPath, String zipFilePath) throws IOException {
Path p = Files.createFile(Paths.get(zipFilePath));
Path pp = Paths.get(sourceDirPath);
try (ZipOutputStream zs = new ZipOutputStream(Files.newOutputStream(p));
Stream<Path> paths = Files.walk(pp)) {
paths
.filter(path -> !Files.isDirectory(path))
.forEach(path -> {
ZipEntry zipEntry = new ZipEntry(pp.relativize(path).toString());
try {
zs.putNextEntry(zipEntry);
Files.copy(path, zs);
zs.closeEntry();
} catch (IOException e) {
System.err.println(e);
}
});
}
}
I have a servlet that offers a CSV file for download:
#RestController
#RequestMapping("/")
public class FileController {
#RequestMapping(value = "/export", method = RequestMethod.GET)
public FileSystemResource getFile() {
return new FileSystemResource("c:\file.csv");
}
}
This works just fine.
Question: how can I offer this file as compressed file? (zip, gzip, tar doesn't matter)?
Based on the solution here (for a plain Servlet), you can also do the same with a Spring MVC based controller.
#RequestMapping(value = "/export", method = RequestMethod.GET)
public void getFile(OutputStream out) {
FileSystemResource resource = new FileSystemResource("c:\file.csv");
try (ZipOutputStream zippedOut = new ZipOutputStream(out)) {
ZipEntry e = new ZipEntry(resource.getName());
// Configure the zip entry, the properties of the file
e.setSize(resource.contentLength());
e.setTime(System.currentTimeMillis());
// etc.
zippedOut.putNextEntry(e);
// And the content of the resource:
StreamUtils.copy(resource.getInputStream(), zippedOut);
zippedOut.closeEntry();
zippedOut.finish();
} catch (Exception e) {
// Do something with Exception
}
}
You created a ZipOutputStream based on the responses OutputStream (which you can simply have injected into the method). Then create an entry for the zipped out stream and write it.
Instead of the OutputStream you could also wire the HttpServletResponse so that you would be able to set the name of the file and the content type.
#RequestMapping(value = "/export", method = RequestMethod.GET)
public void getFile(HttpServletResponse response) {
FileSystemResource resource = new FileSystemResource("c:\file.csv");
response.setContentType("application/zip");
response.setHeader("Content-Disposition", "attachment; filename=file.zip");
try (ZipOutputStream zippedOut = new ZipOutputStream(response.getOutputStream())) {
ZipEntry e = new ZipEntry(resource.getName());
// Configure the zip entry, the properties of the file
e.setSize(resource.contentLength());
e.setTime(System.currentTimeMillis());
// etc.
zippedOut.putNextEntry(e);
// And the content of the resource:
StreamUtils.copy(resource.getInputStream(), zippedOut);
zippedOut.closeEntry();
zippedOut.finish();
} catch (Exception e) {
// Do something with Exception
}
}
Untested but something like that should work:
final Path zipTmpPath = Paths.get("C:/file.csv.zip");
final ZipOutputStream zipOut = new ZipOutputStream(Files.newOutputStream(zipTmpPath, StandardOpenOption.WRITE));
final ZipEntry zipEntry = new ZipEntry("file.csv");
zipOut.putNextEntry(zipEntry);
Path csvPath = Paths.get("C:/file.csv");
List<String> lines = Files.readAllLines(csvPath);
for(String line : lines)
{
for(char c : line.toCharArray())
{
zipOut.write(c);
}
}
zipOut.flush();
zipOut.close();
return new FileSystemResource("C:/file.csv.zip");
use this :
#RequestMapping(value = "/zip", produces="application/zip")
This may solve your issue
I have 3 separate spring web applications
A uses spring 4.x
B uses spring 3.2.0
C uses spring 4.x
B and C exposes REST controllers for uploading files
A reads file and uploads it to B
B sends the request to C without any need to read file content
and then C does whatever it wants with the file.
So the flow would be A->B->C
My question is - is it possible to setup B in such a way so that B wouldn't store whole file in the memory, but would read incoming stream and forward it to C?
What I managed to do is:
A
public void sendFileFromA() throws FileNotFoundException {
final InputStream fis = new FileInputStream(new File("someFile"));
final RequestCallback requestCallback = new RequestCallback() {
#Override
public void doWithRequest(final ClientHttpRequest request) throws IOException {
request.getHeaders().add("Content-type", "application/octet-stream");
IOUtils.copy(fis, request.getBody());
}
};
final RestTemplate restTemplate = new RestTemplate();
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setBufferRequestBody(false);
restTemplate.setRequestFactory(requestFactory);
final HttpMessageConverterExtractor<String> responseExtractor = new HttpMessageConverterExtractor<>(
String.class, restTemplate.getMessageConverters());
restTemplate.execute("http://b_url/upload", HttpMethod.POST, requestCallback, responseExtractor);
}
B
#RequestMapping(value = "/upload", method = RequestMethod.POST)
public #ResponseBody String handleFileUpload(HttpServletRequest request) throws IOException {
final ServletInputStream input = request.getInputStream();
final RequestCallback requestCallback = new RequestCallback() {
#Override
public void doWithRequest(final ClientHttpRequest request) throws IOException {
request.getHeaders().add("Content-type", "application/octet-stream");
try (OutputStream body = request.getBody()) {
IOUtils.copy(input, body);
}
}
};
final RestTemplate restTemplate = new RestTemplate();
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setBufferRequestBody(false);
restTemplate.setRequestFactory(requestFactory);
final HttpMessageConverterExtractor<String> responseExtractor = new HttpMessageConverterExtractor<>(
String.class, restTemplate.getMessageConverters());
restTemplate.execute("http://c_url/upload", HttpMethod.POST, requestCallback, responseExtractor);
return "success";
}
C
#RequestMapping(value = "/upload", method = RequestMethod.POST)
public #ResponseBody String handleFileUpload(HttpServletRequest request) throws IOException {
ServletInputStream input = request.getInputStream();
try (BufferedOutputStream output = new BufferedOutputStream(new FileOutputStream("zibiTest"))) {
IOUtils.copy(input, output);
}
return "success";
}
I can easily copy files over >10GB from A to C using B.
With such a solution we can try to stop A while transferring, B and C should be notified about the error, but sometimes it happens that the error message doesn't reach C - it gets closed with socket timeout exception, any idea why this happens and how to implement it properly?
Is this a valid approach or can it be handled better?
I would try setting smaller socket timeout on C than you have on B. Currently it seems that both have some default value, so if A hangs, both B and C will stop getting data almost the same time. Both start timing out, and maybe it is a race condition, where it depends on the timeout accuracy which one times out first.
I have several servlets, which
take JSON-encoded requests as inputs,
process them and
return responses to the client as JSON-encoded objects.
Up to now I used Android as client (sample Android code see below).
Now I want to write a plain old Java program, which would send requests and receive the responses (do the same as the Android code). For this purpose I wrote a Java test (code see below, section Java code) and ran it.
At the client side I get this error:
21:43:38.930 [main] ERROR r.a.c.t.TestAcceptanceProcedure1 -
java.io.IOException: Server returned HTTP response code: 405 for URL: http://myserver/myapp/rest/GetUserIdService
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.6.0_23]
at testclient.TestAcceptanceProcedure1.test(TestAcceptanceProcedure1.java:47) ~[test-classes/:na]
In the server log, I see this message:
WARNING: No operation matching request path "/myapp/rest/GetUserIdService" is found, Relative Path: /, HTTP Method: GET, ContentType: */*, Accept: text/html,image/gif,image/jpeg,*/*,*/*;q=.2,. Please enable FINE/TRACE log level for more details
Question: How should I change my Java test to fix this error?
Note that the server is up and running (when I execute the Android code, it works).
Android code:
Sending the request and receiving the response:
final GetSimulationStatusRequest request = new GetSimulationStatusRequest();
final String json = Utils.convertToJson(request, getClass());
final String serverUrl = Utils.getServerUrl(this, "GetSimulationStatusService");
final IGetSimulationStatusAsyncTask getSimulationStatusTask =
asyncTaskFactory.createGetSimulationStatusAsyncTask(getWebServiceHelper());
Utils.setRequestAndServerUrl(json, serverUrl, getSimulationStatusTask);
final GetSimulationStatusResponse simulationStatusReponse =
getSimulationStatusTask.get();
Utils.convertToJson:
public static String convertToJson(final Object aRequest, Class<? extends Activity> aClass) {
final ObjectMapper mapper = new ObjectMapper();
String json = null;
try {
json = mapper.writeValueAsString(aRequest);
} catch (final JsonProcessingException exception) {
Log.e(aClass.getSimpleName(), exception.getLocalizedMessage(),
exception);
}
return json;
}
Utils.setRequestAndServerUrl:
public static void setRequestAndServerUrl(final String aJson,
final String aServerUrl, final IAsyncTask aTask) {
aTask.addNameValuePair("request", aJson);
aTask.sendRequest(new String[] { aServerUrl });
}
GetSimulationStatusAsyncTask:
public class GetSimulationStatusAsyncTask
extends AsyncTask<String, String, GetSimulationStatusResponse>
implements IGetSimulationStatusAsyncTask {
private static final String TAG = GetSimulationStatusAsyncTask.class.getSimpleName();
private IWebServiceTaskHelper helper;
private ICcpResponseParser<GetSimulationStatusResponse> responseParser =
new CcpResponseParser<GetSimulationStatusResponse>();
public GetSimulationStatusAsyncTask(final IWebServiceTaskHelper aHelper) {
helper = aHelper;
}
#Override
public void addNameValuePair(final String aName, final String aValue) {
helper.addNameValuePair(aName, aValue);
}
#Override
protected GetSimulationStatusResponse doInBackground(String... aArgs) {
return (GetSimulationStatusResponse)Utils.processResponse(this.helper, TAG, responseParser,
GetSimulationStatusResponse.class, aArgs);
}
#Override
public void sendRequest(final String[] aArgs) {
execute(aArgs);
}
}
Java code:
#Test
public void test() throws JsonProcessingException, MalformedURLException {
final GetUserIdRequest request = new GetUserIdRequest();
request.setDeviceId("PC1");
final String requestAsString = convertToJson(request);
final String serverUrl = getServerUrl("GetUserIdService");
final URL url = new URL(serverUrl);
HttpURLConnection connection = null;
InputStream inputStream = null;
try {
connection = (HttpURLConnection) url.openConnection();
connection.addRequestProperty("request", requestAsString);
connection.connect();
inputStream = connection.getInputStream();
final String responseAsString = IOUtils.toString(inputStream);
LOGGER.debug("responseAsString: " + responseAsString);
} catch (final IOException exception) {
LOGGER.error("", exception);
}
finally
{
IOUtils.closeQuietly(inputStream);
}
}
private String convertToJson(final GetUserIdRequest aRequest) throws JsonProcessingException {
final ObjectMapper mapper = new ObjectMapper();
return mapper.writeValueAsString(aRequest);
}
private String getServerUrl(final String aServiceName)
{
return "http://myserver.com/myapp/rest/" + aServiceName;
}
Update 1 (09.10.2013 12:23 MSK):
#Path("/GetSimulationStatusService")
public class GetSimulationStatusService extends BaseCcpService {
private GetSimulationStatusRequestParser requestParser =
new GetSimulationStatusRequestParser();
#POST
#Produces("text/plain")
public String getSimulationStatus(#FormParam("request") final String aRequestJson)
throws JsonProcessingException
{
final GetSimulationStatusRequest request = requestParser.convert(aRequestJson);
final GetSimulationStatusResponse response = new GetSimulationStatusResponse();
response.setRequestId(request.getId());
response.setCycle(getPersistence().getCurrentCycle(request.getUserId()));
response.setLabourForce(getPersistence().getLabourForceSimulationParameter(
request.getUserId()));
return getObjectMapper().writeValueAsString(response);
}
}
Update 2 (09.10.2013 20:48 MSK): When I change the code like shown below, I get 500 HTTP response. At the server side, the aRequest argument of the method GetUserIdService.getUserId is null.
connection = (HttpURLConnection) url.openConnection();
connection.addRequestProperty("request", requestAsString);
connection.setRequestMethod("POST"); // Added this line
connection.connect();
Update 3 (09.10.2013 23:15): This one works:
#Test
public void test() throws JsonProcessingException, MalformedURLException
{
final GetUserIdRequest request = new GetUserIdRequest();
request.setDeviceId("PC1");
final String requestAsString = convertToJson(request);
final String serverUrl = getServerUrl("GetUserIdService");
final URL url = new URL(serverUrl);
HttpURLConnection connection = null;
InputStream inputStream = null;
OutputStream outputStream = null;
try {
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("POST");
connection.setDoOutput(true);
connection.connect();
outputStream = connection.getOutputStream();
IOUtils.write("request=" + requestAsString, outputStream);
inputStream = connection.getInputStream();
final String responseAsString = IOUtils.toString(inputStream);
LOGGER.debug("responseAsString: " + responseAsString);
} catch (final IOException exception) {
LOGGER.error("", exception);
}
finally
{
IOUtils.closeQuietly(inputStream);
IOUtils.closeQuietly(outputStream);
}
}
The 405 HTTP error code means that given method (GET) is not supported by the endpoint. Probably instead of GET request you want to send POST. I don't know what kind of request is sent by Android client.
Do you have endpoint specification/documentation?
Here you'll find information how to invoke POST using plain Java API. If you can use external libraries in your test then it can be achieved a lot easier using RESTEasy.
I'm pretty new to coding with streams but now I have to do it for more efficient Http coding.
Here is code that I wrote(not working) to get ContentProducer for HttpClient:
public static ContentProducer getContentProducer(final Context context, final UUID gId)
{
return new ContentProducer()
{
public void writeTo(OutputStream outputStream) throws IOException
{
outputStream = new Base64.OutputStream(new FileOutputStream(StorageManager.getFileFromName(context, gId.toString())));
outputStream.flush();
}
};
}
I'm using Base64 streaming encoder from here: http://iharder.sourceforge.net/current/java/base64/
My goal is to use this function to provide data that I read from binary file to HttpClient as base64 encoded stream.
This is how I consume content producers:
private MyHttpResponse processPOST(String url, ContentProducer requestData)
{
MyHttpResponse response = new MyHttpResponse();
try
{
HttpPost request = new HttpPost(serviceURL + url);
HttpEntity entity = new EntityTemplate(requestData);
request.setEntity(entity);
ResponseHandler<String> handler = new BasicResponseHandler();
response.Body = mHttpClient.execute(request, handler);
}
catch (HttpResponseException e)
{
}
catch (Throwable e)
{
}
return response;
}
I have another ContentProducer which works with GSON streamer(and it's working):
public ContentProducer getContentProducer(final Context context)
{
return new ContentProducer()
{
public void writeTo(OutputStream outputStream) throws IOException
{
Gson myGson = MyGsonWrapper.getMyGson();
JsonWriter writer = new JsonWriter(new OutputStreamWriter(outputStream, "UTF-8"));
writer.beginObject();
// stuff
writer.endObject();
writer.flush();
}
};
}
My question is: How to make my first example work. Am I doing it correctly? Right now I get empty post on server side, so it seems like no data coming through.
EDIT:
I believe that the issue is that you are being passed an OutputStream in your ContentProviders writeTo() method, and you are overwriting it with your own OutputStream. The contract of that class/method probably requires you to write your data to the OutputStream passed to you.
Based on looking at the Android Documentation, I do not see a way for you to specify the OutputStream to use, so you will probably need to just write out the data of the file to the OutputStream that is passed in.
Instead, you should do something like this:
public void writeTo(OutputStream outputStream) throws IOException
{
byte[] buf = createByteBufferFromFile(StorageManager.getFileFromName(context, gId.toString()));
outputStream.write(buf);
outputStream.flush();
}
Of course you will need to provide an implementation to the createByteBufferFromFile(...) method that I mention. Again, you should note that it is not likely that you will be using the Base64 OutputStream, so if that is a necessity, then you may have to find a different approach.