I am migrating a webapp from Java 8 to 11 (and Tomcat 8 to 9) and i have a Client that downloads ZIP Archive Files from a Service using the following methods:
public HTTPResponse doGet(String aUrl, HashMap<String,String> aRequestParams, HashMap<String,String> aRequestProperties)
throws Exception
{
try
{
String lUrl = aUrl;
if (aRequestParams != null && aRequestParams.size() > 0)
{
StringBuffer lBodyStringBuffer = new StringBuffer();
for(String lParam : aRequestParams.keySet())
{
String lValue = aRequestParams.get(lParam);
if(lValue != null && !"".equals(lValue.trim()))
{
if(lBodyStringBuffer.length() > 0)
{
lBodyStringBuffer.append("&");
}
lBodyStringBuffer.append(URLEncoder.encode(lParam, sDEFAULTENCODING)).append("=").append(URLEncoder.encode(lValue, sDEFAULTENCODING));
}
}
String lParamString = lBodyStringBuffer.toString();
if (lParamString != null && lParamString.length() > 0)
{
if (!(lUrl.endsWith(sURLPARAMSLEADER) || aUrl.endsWith(sURLPARAMSSEPARATOR)))
{
if (lUrl.indexOf(sURLPARAMSLEADER) > -1)
{
lUrl = lUrl + sURLPARAMSSEPARATOR;
}
else
{
lUrl = lUrl + sURLPARAMSLEADER;
}
}
lUrl = lUrl + lParamString;
}
}
HttpURLConnection lConnection = createConnection(lUrl,sREQUESTETHOD_GET,null, aRequestProperties);
HTTPResponse lReturn = getResponseFromConnection(lConnection);
return lReturn;
}
catch(Exception lException)
{
throw new Exception("Fehler beim Durchführen der Anfrage: " + lException.getMessage(), lException);
}
}
private HTTPResponse getResponseFromConnection(HttpURLConnection aConnection)
throws Exception
{
InputStream lConnectionInputStream = null;
ByteArrayOutputStream lResponseByteArrayOutputStream = null;
try
{
aConnection.setRequestProperty("Accept", "application/zip");
int lStatusCode = aConnection.getResponseCode();
String lResponseCharset = getCharsetFromResponseContentType(aConnection.getContentType());
if (lResponseCharset == null)
{
if (lResponseCharset == null ||lResponseCharset.trim().length() == 0)
{
lResponseCharset = "UTF-8";
}
}
if (HttpURLConnection.HTTP_OK == lStatusCode)
{
lConnectionInputStream = aConnection.getInputStream();
}
else
{
lConnectionInputStream = aConnection.getErrorStream();
}
String lMessage = "";
if (lConnectionInputStream != null)
{
lResponseByteArrayOutputStream = new ByteArrayOutputStream();
int lBufferSize = 4096;
byte[] lBuffer = new byte[lBufferSize];
int lLength = 0;
while ((lLength = lConnectionInputStream.read(lBuffer, 0, lBufferSize)) != -1)
{
lResponseByteArrayOutputStream.write(lBuffer, 0, lLength);
}
byte[] lResponseByte = lResponseByteArrayOutputStream.toByteArray();
lMessage = new String (lResponseByte,lResponseCharset);
}
HTTPResponse lReturn = new HTTPResponse(lStatusCode, lMessage);
return lReturn;
}
catch(Exception lException)
{
throw lException;
}
finally
{
if (lResponseByteArrayOutputStream != null)
{
try{lResponseByteArrayOutputStream.close();}catch(Exception e){}
}
if (lConnectionInputStream != null)
{
try{lConnectionInputStream.close();}catch(Exception e){}
}
}
}
This is how i actually call the service via HTTP Get and save the data:
HTTPResponse lResponse = new HTTPRequest().doGet("http://localService.com/, null, null);
FileOutputStream lFileOutputStream = new FileOutputStream("exampleFile.zip", false);
lFileOutputStream.write(lResponse.getMessage().getBytes());
lFileOutputStream.close();
So this used to Work before and i am getting a headache about what might have changed. When i download the ZIP File with url in my Browser, everything seems fine, so the Service seems to work. But with my client the ZIP Files get corrupted and cannot be opened. The are not empty but they differ in size: surprisingly the corrupted files are about 50% larger than the ones downloaded via browser.
Does anyone know what the problem here is?
Ok,the problem was the return type String of the method getResponseFromConnection. I changed it and now write the file directly with the File.copy() method which VGR suggested.
Related
This question already has answers here:
Find location of a removable SD card
(24 answers)
Closed 4 years ago.
I need SdCard path to save files in it. I have tried some codes but these code didn't work on some devices or Android versions. Now I need a code/path that get SdCard path for all device and all Android versions.
For getting sdcard path,try following code:
public static String getExternalSDCardRootDirectory() {
String cmdMOUNT = "cat /proc/mounts";
Runtime run = Runtime.getRuntime();
List<String> paths = new ArrayList<>();
try {
Process p = run.exec(cmdMOUNT);
BufferedInputStream in = new BufferedInputStream(p.getInputStream());
BufferedReader inBr = new BufferedReader(new InputStreamReader(in));
String lineStr;
while ((lineStr = inBr.readLine()) != null) {
Log.d(TAG, lineStr);
if (lineStr.toLowerCase().contains("sdcard") || lineStr.toLowerCase().contains("ext") ) {
String[] strArray = lineStr.split(" ");
if (strArray.length >= 3 &&
(!strArray[1].contains("/system") &&
!strArray[1].contains("/data") &&
!strArray[1].contains("/cache") &&
!strArray[1].contains("/persist")
)) {
String result = strArray[1].trim();
if((result.contains("ext") || result.contains("1")) && result.contains("storage")) {
paths.add(result);
}
//return result;
}
}
if (p.waitFor() != 0 && p.exitValue() == 1) {
Log.e(TAG, "check mount info failed");
return null;
}
}
inBr.close();
in.close();
} catch (Exception e) {
e.printStackTrace();
return null;
}
if (paths.size() > 0) {
return paths.get(0);
}
else {
return null;
}
}
For getting path you need to call Environment.getExternalStorageState()
I've found an existing post.
Simply change it to...
public static HashSet<String> getExternalMounts() {
final HashSet<String> out = new HashSet<String>();
String reg = "(?i).*vold.*(vfat|ntfs|exfat|fat32|ext3|ext4).*rw.*";
String s = "";
try {
final Process process = new ProcessBuilder().command("mount")
.redirectErrorStream(true).start();
process.waitFor();
final InputStream is = process.getInputStream();
final byte[] buffer = new byte[1024];
while (is.read(buffer) != -1) {
s = s + new String(buffer);
}
is.close();
} catch (final Exception e) {
e.printStackTrace();
}
// parse output
final String[] lines = s.split("\n");
for (String line : lines) {
if (!line.toLowerCase(Locale.US).contains("asec")) {
if (line.matches(reg)) {
String[] parts = line.split(" ");
for (String part : parts) {
if (part.startsWith("/"))
if (!part.toLowerCase(Locale.US).contains("vold"))
out.add(part);
}
}
}
}
return out;
}
The tested solution on different platforms can be found here.
I have a servlet which receives a huge string (apprx 301695 length) as a post parameter.
Every minute, a .net application sends such huge string to the servlet.
Initially I used to get the string as below:
Line 1: String str = request.getParameter("data");
But, after 3-4 hours. I get the following exception:
java.lang.OutOfMemoryError: Java heap space
Then I commented Line: 1. Even though, My servlet code does not receive the string, I get the same exception as mentioned above.
Please guide me. How should I deal with this issue? I have read many blogs and articles related to it, increased the heap size and other things. But, haven't found any solution.
Original code was like below:
private String scanType = "";
private static final String path = "D:\\Mobile_scan_alerts";
private static final String stockFileName = "stock.txt";
private static final String foFileName = "fo.txt";
private static Logger logger = null;
private String currDate = "";
private DateFormat dateFormat;
private StringBuffer stockData;
private StringBuffer foData;
StringBuffer data = new StringBuffer("");
// For average time of received data
private static float sum = 0;
private static float count = 0;
private static float s_sum = 0;
private static float s_count = 0;
private static float fo_sum = 0;
private static float fo_count = 0;
private static final File dir = new File(path);
private static final File stockFile = new File(path + "\\" + stockFileName);
private static final File foFile = new File(path + "\\" + foFileName);
public void init() {
logger = MyLogger.getScanAlertLogger();
if(logger == null) {
MyLogger.createLog();
logger = MyLogger.getScanAlertLogger();
}
}
/**
* Processes requests for both HTTP <code>GET</code> and <code>POST</code>
* methods.
*
* #param request servlet request
* #param response servlet response
* #throws ServletException if a servlet-specific error occurs
* #throws IOException if an I/O error occurs
*/
protected void processRequest(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
PrintWriter out = response.getWriter();
response.setContentType("text/plain");
String strScan = "";
try {
String asof = null;
scanType = request.getParameter("type");
scanType = scanType == null ? "" : scanType;
if(scanType.length() > 0){
if(scanType.equalsIgnoreCase("s")) {
stockData = null;
stockData = new StringBuffer(request.getParameter("scanData"));
stockData = stockData == null ? new StringBuffer("") : stockData;
} else {
foData = null;
foData = new StringBuffer(request.getParameter("scanData"));
foData = foData == null ? new StringBuffer("") : foData;
}
}
asof = request.getParameter("asof");
asof = asof == null ? "" : asof.trim();
// Date format without seconds
DateFormat formatWithoutSec = new SimpleDateFormat("yyyy/MM/dd HH:mm");
dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
Date tmp = new Date();
// format: yyyy/MM/dd HH:mm:ss
currDate = dateFormat.format(tmp);
//format: yyyy/MM/dd HH:mm
Date asofDate = formatWithoutSec.parse(asof);
Date cDate = formatWithoutSec.parse(currDate);
cDate.setSeconds(0);
System.out.println(asofDate.toString()+" || "+cDate.toString());
int isDataExpired = asofDate.toString().compareTo(cDate.toString());
if(isDataExpired > 0 || isDataExpired == 0) {
if(scanType != null && scanType.length() > 0) {
checkAndCreateDir();
strScan = scanType.equalsIgnoreCase("s") ? "Stock Data Recieved at "+currDate
: "FO Data Recieved at "+currDate;
//System.out.println(strScan);
} else {
strScan = "JSON of scan data not received properly at "+currDate;
//System.out.println("GSAS: received null or empty");
}
} else {
strScan = "GSAS: " + scanType + ": Received Expired Data of "+asofDate.toString()+" at "+cDate.toString();
System.out.println(strScan);
}
scanType = null;
} catch (Exception ex) {
strScan = "Mobile server issue for receiving scan data";
System.out.println("GSAS: Exception-1: "+ex);
logger.error("GetScanAlertServlet: processRequest(): Exception: "+ex.toString());
} finally {
logger.info("GetScanAlertServlet: "+strScan);
out.println(strScan);
}
}
private void checkAndCreateDir() {
try {
boolean isStock = false;
Date ddate = new Date();
currDate = dateFormat.format(ddate);
sum += ddate.getSeconds();
count++;
logger.info("Total Average Time: "+(sum/count));
if(scanType.equalsIgnoreCase("s")){ //For Stock
setStockData(stockData);
Date date1 = new Date();
currDate = dateFormat.format(date1);
s_sum += date1.getSeconds();
s_count++;
logger.info("Stock Average Time: "+(s_sum/s_count));
//file = new File(path + "\\" + stockFileName);
isStock = true;
} else if (scanType.equalsIgnoreCase("fo")) { //For FO
setFOData(foData);
Date date2 = new Date();
currDate = dateFormat.format(date2);
fo_sum += date2.getSeconds();
fo_count++;
logger.info("FO Average Time: "+(fo_sum/fo_count));
//file = new File(path + "\\" +foFileName);
isStock = false;
}
if(!dir.exists()) { // Directory not exists
if(dir.mkdir()) {
if(isStock)
checkAndCreateFile(stockFile);
else
checkAndCreateFile(foFile);
}
} else { // Directory already exists
if(isStock)
checkAndCreateFile(stockFile);
else
checkAndCreateFile(foFile);
}
} catch (Exception e) {
System.out.println("GSAS: Exception-2: "+e);
logger.error("GetScanAlertServlet: checkAndCreateDir(): Exception: "+e);
}
}
private void checkAndCreateFile(File file) {
try{
if(!file.exists()){ // File not exists
if(file.createNewFile()){
writeToFile(file);
}
} else { // File already exists
writeToFile(file);
}
} catch (Exception e) {
System.out.println("GSAS: Exception-3: "+e);
logger.error("GetScanAlertServlet: checkAndCreateFile(): Exception: "+e.toString());
}
}
private void writeToFile(File file) {
FileOutputStream fop = null;
try{
if(scanType.equalsIgnoreCase("s")){ //For Stock
data = getStockData();
} else if (scanType.equalsIgnoreCase("fo")) { //For FO
data = getFOData();
}
if(data != null && data.length() > 0 && file != null){
fop = new FileOutputStream(file);
byte[] contentBytes = data.toString().getBytes();
for(byte b : contentBytes){
fop.write(b);
}
//fop.write(contentBytes);
fop.flush();
} else {
System.out.println("GSAS: Data is null/empty string");
logger.info("GSAS: Data is null or empty string");
}
data = null;
} catch (Exception e) {
System.out.println("GSAS: Exception-4: "+e);
logger.info("GetScanAlertServlet: writeToFile(): Exception: "+e.toString());
} finally {
try {
if(fop != null)
fop.close();
} catch (IOException ex) {
java.util.logging.Logger.getLogger(GetScanAlertServlet.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
private String readFromFile(String fileName){
String fileContent = "";
try{
String temp = "";
File file = new File(fileName);
if(file.exists()){
FileReader fr = new FileReader(file);
BufferedReader br = new BufferedReader(fr);
while((temp = br.readLine()) != null)
{
fileContent += temp;
}
br.close();
} else {
System.out.println("GSAS: File not exists to read");
logger.info("GetScanAlertServlet: File not exists to read");
}
temp = null;
file = null;
} catch (Exception e) {
System.out.println("GSAS: Exception-5: "+e);
logger.error("GetScanAlertServlet: readFromFile(): Exception: "+e.toString());
}
return fileContent;
}
public StringBuffer getStockData() {
//String temp="";
//StringBuffer temp = (StringBuffer)scanDataSession.getAttribute("stock");
//if(temp != null && temp.length() > 0) {
// return temp;
//}
if(stockData != null && stockData.length() > 0){
return stockData;
} else {
stockData = null;
stockData = new StringBuffer(readFromFile(path + "\\"+ stockFileName));
return stockData;
}
}
public StringBuffer getFOData(){
//String temp="";
//StringBuffer temp = (StringBuffer)scanDataSession.getAttribute("fo");
//if(temp != null && temp.length() > 0) {
// return temp;
//}
if(foData != null && foData.length() > 0) {
return foData;
} else {
foData = null;
foData = new StringBuffer(readFromFile(path + "\\" + foFileName));
return foData;
}
}
}
Increasing heap size is not a good solution for this problem. Your upstream application should stop sending huge strings to your Servlet.
Your upstream(.net) application should consider writing all the data to a file, just need to send the location of the file as a parameter to your Servlet. Once your servlet receives notification from the upstream, you consider downloading/reading file from the location.
Then I commented Line: 1. Even though, My servlet code does not
receive the string (as commented), I get the same exception as
mentioned above.
The Line: 1 is to read data. If you comment it, you wont receive the String.
You can use apache commons-fileupload library Streaming API, this way, you get your uploaded file as a stream and write it to the file :
ServletFileUpload upload = new ServletFileUpload();
// Parse the request
FileItemIterator iter = upload.getItemIterator(request);
while (iter.hasNext()) {
FileItemStream item = iter.next();
String name = item.getFieldName();
InputStream stream = item.openStream();
if (item.isFormField()) {
System.out.println("Form field " + name + " with value "
+ Streams.asString(stream) + " detected.");
} else {
System.out.println("File field " + name + " with file name "
+ item.getName() + " detected.");
// Process the input stream
...
}
}
Now You have InputStream, so you can write it in the output stream.
But to use this you need your .NET application to upload the bytes to the server instead of sending entire String as request param.
http://commons.apache.org/proper/commons-fileupload/streaming.html
Please check your VM Arguments and modify them approriately if you have no control of the String being passed to the servlet. For examples:
set JAVA_OPTS=-Dfile.encoding=UTF-8 -Xms128m -Xmx1024m -XX:PermSize=64m -XX:MaxPermSize=256m
Check for a complete explanation here.
We used GZip compression/decompression to lower the size of the string. And it worked effectively.
So, the .net service compressed the huge string, sent it to our Servlet. We then decompress it at our server.
I am trying to serve an mp3 using HttpHandler and getting a broken pipe. It works with Google Chrome on my mac and my iPad but Android cause the HttpHander to just hang after getting the IOException and I have to restart. Using very simple code and works fine with images and html.
try {
String requestURI = t.getRequestURI().toString().substring(1);
if(requestURI.equals("") || requestURI.equals("/"))
requestURI = "index.htm";
requestURI = requestURI.replaceAll("%20", " ");
if(requestURI.contains("mp3")) {
urlToResource = new File(System.getProperty("user.home") + "/test/" +
requestURI).toURI().toURL();
}
System.out.println("Modified requestURI:" + requestURI);
if(requestURI.contains("mp3")) {
sContentType = "audio/mpeg";
} else if(requestURI.contains("png")) {
sContentType = "image/png";
} else if(requestURI.contains("jpg")) {
sContentType = "image/jpg";
} else if(requestURI.contains("favicon.ico")) {
sContentType = "content/unknown";
} else if(requestURI.contains("css")) {
sContentType = "text/css";
} else {
sContentType = "text/html";
}
if(!requestURI.contains("mp3")) {
urlToResource = new File("src/com/daford/web/" + requestURI).toURI().toURL();
}
if(urlToResource != null) {
conn = urlToResource.openConnection();
int size = conn.getContentLength();
System.out.println("file " + requestURI + " size is:" + size);
inConnectionReader = conn.getInputStream();
headers = t.getResponseHeaders();
headers.add("Content-Type", sContentType);
t.sendResponseHeaders(200, size);
os = t.getResponseBody();
int iReadByte = inConnectionReader.read();
while (iReadByte != -1) {
os.write(iReadByte);
iReadByte = inConnectionReader.read();
}
} else {
headers = t.getResponseHeaders();
headers.add("Content-Type", "text/html");
String sErrorMessage = "Error getting webpage.";
t.sendResponseHeaders(404, sErrorMessage.length());
os = t.getResponseBody();
os.write(sErrorMessage.getBytes());
}
if(os != null) {
os.close();
}
} catch (Exception e) {
e.printStackTrace();
}
I am trying to save a file by using google drive api.
Code that I am using to save file at the moment looks following:
final java.io.File file = new java.io.File(Environment.getExternalStorageDirectory() + java.io.File.separator + "Json.txt");
try {
file.createNewFile();
if (file.exists()) {
final FileWriter fileWriter = new FileWriter(file);
final String json = gson.toJson(filesEvent);
fileWriter.write(json);
fileWriter.close();
}
} catch (IOException e) {
e.printStackTrace();
}
This is the part that is supposed to get the file contents:
final StringBuilder sb = getStringBuilder(new FileReader(Environment.getExternalStorageDirectory() + java.io.File.separator + "Json.txt"));
File file = gson.fromJson(sb.toString(), File.class);
I assume, you want to save a text file ('json.txt') to Google Drive and your code applies to Android.
First, you do not indicate what API you decided to use, the REST Api or the GDAA.
Starting with java.io.File as an input (your first code block), here are code snippets for both the GDAA and REST Apis.
GDAA: (you may consider turning the 'await' methods into callbacks, or you have to wrap it in non-UI thread)
dependencies {
...
compile 'com.google.android.gms:play-services:7.8.0'
}
com.google.android.gms.common.api.GoogleApiClient mGAC;
...
/**********************************************************************
* create file/folder in GOODrive
* #param prnId parent's ID, (null for root)
* #param titl file name
* #param mime file mime type
* #param file file (with content) to create
* #return file id / null on fail
*/
DriveId createFile(DriveId prnId, String titl, String mime, File file) {
DriveId dId = null;
if (mGAC != null && mGAC.isConnected() && titl != null && mime != null && file != null) {
DriveFolder pFldr = (prnId == null) ?
Drive.DriveApi.getRootFolder(mGAC): Drive.DriveApi.getFolder(mGAC, prnId);
if (pFldr == null) return null; //----------------->>>
MetadataChangeSet meta;
DriveContentsResult r1 = Drive.DriveApi.newDriveContents(mGAC).await();
if (r1 == null || !r1.getStatus().isSuccess()) return null; //-------->>>
meta = new Builder().setTitle(titl).setMimeType(mime).build();
DriveFileResult r2 = pFldr.createFile(mGAC, meta, r1.getDriveContents()).await();
DriveFile dFil = r2 != null && r2.getStatus().isSuccess() ? r2.getDriveFile() : null;
if (dFil == null) return null; //---------->>>
r1 = dFil.open(mGAC, DriveFile.MODE_WRITE_ONLY, null).await();
if ((r1 != null) && (r1.getStatus().isSuccess())) {
Status stts = fileToCont(r1.getDriveContents(), file).commit(mGAC, meta).await();
if ((stts != null) && stts.isSuccess()) {
MetadataResult r3 = dFil.getMetadata(mGAC).await();
if (r3 != null && r3.getStatus().isSuccess()) {
dId = r3.getMetadata().getDriveId();
}
}
}
}
return dId;
}
DriveContents fileToCont(DriveContents driveContents, File file) {
OutputStream oos = driveContents.getOutputStream();
if (oos != null) try {
InputStream is = new FileInputStream(file);
byte[] buf = new byte[4096];
int c;
while ((c = is.read(buf, 0, buf.length)) > 0) {
oos.write(buf, 0, c);
oos.flush();
}
} catch (Exception e) { UT.le(e);}
finally {
try {
oos.close();
} catch (Exception ignore) {
}
}
return driveContents;
}
REST Api: (you have to wrap it in non-UI thread)
dependencies {
...
compile 'com.google.apis:google-api-services-drive:v2-rev105-1.17.0-rc'
compile 'com.google.api-client:google-api-client-android:1.20.0'
compile 'com.google.http-client:google-http-client-gson:1.20.0'
}
com.google.api.services.drive.Drive mGOOSvc;
...
/***************************************************************
* create file/folder in GOODrive
* #param prnId parent's ID, (null or "root") for root
* #param titl file name
* #param mime file mime type
* #param file file (with content) to create
* #return file id / null on fail
*/
static String createFile(String prnId, String titl, String mime, java.io.File file) {
String rsId = null;
if (mGOOSvc != null && titl != null && mime != null && file != null) try {
File meta = new File();
meta.setParents(Arrays.asList(new ParentReference().setId(prnId == null ? "root" : prnId)));
meta.setTitle(titl);
meta.setMimeType(mime);
File gFl = mGOOSvc.files().insert(meta, new FileContent(mime, file)).execute();
if (gFl != null)
rsId = gFl.getId();
}
catch (UserRecoverableAuthIOException uraIOEx) {
// handle uraIOEx;
}
catch (IOException e) {
if (e instanceof GoogleJsonResponseException) {
if (404 == ((GoogleJsonResponseException)e).getStatusCode())
// handle error;
}
} catch (Exception e) {
// handle error;
}
return rsId;
}
Wider context of these methods can be found here and here if you care to dig deeper.
Good Luck
My Code to get file content:
private String readTxt(){
InputStream inputStream = getResources().openRawResource(R.raw.text);
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
int i;
try {
i = inputStream.read();
while (i != -1)
{
byteArrayOutputStream.write(i);
i = inputStream.read();
}
inputStream.close();
}
catch (IOException e)
{
e.printStackTrace();
}
return byteArrayOutputStream.toString();
}
but i want only one specific line on that file to be extracted.
Use BufferedReader instead of ByteArrayOutputStream.
String readLine(int line) throws IOException {
InputStream in = getResources().openRawResource(R.raw.text);
BufferedReader r = new BufferedReader(new InputStreamReader(in));
try {
String lineStr = null;
int currentLine = 0;
while ((lineStr = r.readLine()) != null) {
if (currentLine++ == line) {
return lineStr;
}
}
} finally {
if (r != null) {
r.close();
}
}
throw new IOException("line " + line + " not found");
}