FileUtils.copyURLToFile Getting Stuck - java

I'm using a Jave program to get NSE share price data from NSE's website like this for example:
url = new URL("https://archives.nseindia.com/archives/equities/bhavcopy/pr/PR071122.zip");
f = new File("NSEData.zip");
try {
FileUtils.copyURLToFile(url, f);
} catch (Exception e) {
}
The above code works for dates where market data exists, like 07/11/22 . However, where data does not exist, like on 08/11/22, the url is broken and the copyURLToFile line gets stuck indefinitely during runtime (replacing 071122 with 081122 in the url/code above will cause it to get stuck). Is there an easy way to get the program to recognize that the url for a certain date is broken (eg. https://archives.nseindia.com/archives/equities/bhavcopy/pr/PR081122.zip) and therefore ignore and continue past the try block without getting stuck?
My current workaround is to check whether a certain date is a market holiday using a DayOfWeek check as well as a HashSet containing a list of public holidays, but this is not perfect.

So, basically your URL is returning 500 error upon requesting for invalid date. You can simply use the another method available in FileUtils
https://commons.apache.org/proper/commons-io/javadocs/api-2.5/org/apache/commons/io/FileUtils.html#copyURLToFile(java.net.URL,%20java.io.File,%20int,%20int)
Example code : (Adjust timeouts as per your requirement)
var url = new URL("https://archives.nseindia.com/archives/equities/bhavcopy/pr/PR081122.zip");
var f = new File("NSEData.zip");
try {
FileUtils.copyURLToFile(url, f, 5000, 5000);
} catch (Exception e) {
}

Related

File Fetching from file directory rather than HTTP URL

Previously, my code was developed to fetch the file content from the HTTP URL like "http://storage.googleapis.com/abc". But, requirement just slightly has got changed now and demands that instead of using HTTP URL link, it should need to use file directory location of the server. (i.e., /var/puru/abc).
So, is there any proper or significant way to do and fulfill the requirement without much-doing changes to my existing code drastically?
I have provided the code sample here, which provide the file URL(baseUrl) that is using http URL actually.
private URL getFileUrl(String fileName) throws MalformedURLException {
try {
Date dateObj = dateFormatter.parse(date);
Calendar cal = Calendar.getInstance();
cal.setTime(dateObj);
return new URL(baseUrl, cal.get(Calendar.YEAR) + "/" + fileName);
} catch (ParseException e) {
e.printStackTrace();
}
return null;
}
Please suggest, and guide me.

Sending in-memory generated .docx files from server to client with Spark

I am creating a web application using the Spark Java framework. The front-end is developed using AngularJS.
I want to generate a .docx file on the server (in-memory) and send this to the client for download.
To achieve this I created an angular service with the following function being called after the user clicks on a download button:
functions.generateWord = function () {
$http.post('/api/v1/surveys/genword', data.currentSurvey).success(function (response) {
var element = angular.element('<a/>');
element.attr({
href: 'data:attachment;charset=utf-8;application/vnd.openxmlformats-officedocument.wordprocessingml.document' + response,
target: '_blank',
download: 'test.docx'
})[0].click();
});
};
On the server, this api call gets forwarded to the following method:
public Response exportToWord(Response response) {
try {
File file = new File("src/main/resources/template.docx");
FileInputStream inputStream = new FileInputStream(file);
byte byteStream[] = new byte[(int)file.length()];
inputStream.read(byteStream);
response.raw().setContentType("data:attachment;chatset=utf-8;application/vnd.openxmlformats-officedocument.wordprocessingml.document");
response.raw().setContentLength((int) file.length());
response.raw().getOutputStream().write(byteStream);
response.raw().getOutputStream().flush();
response.raw().getOutputStream().close();
return response;
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
I have tried to solve this in MANY different ways and I always end up with a corrupted 'test.docx' that looks like this:
Solved it by using blobs and specifying the response type as 'arraybuffer' in the $http.post api call. The only bad thing with this solution (as far as I know) is that it doesn't play well with IE, but that's a problem for another day.
functions.generateWord = function () {
$http.post('/api/v1/surveys/genword', data.currentSurvey, {responseType: 'arraybuffer'})
.success(function (response) {
var blob = new Blob([response], {type: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'});
var url = (window.URL || window.webkitURL).createObjectURL(blob);
var element = angular.element('<a/>');
element.attr({
href: url,
target: '_blank',
download: 'survey.docx'
})[0].click();
});
};
I think what went wrong was that the byte stream got encoded as plain text when I tried to create a URL with:
href: 'data:attachment;charset=utf-8;application/vnd.openxmlformats-officedocument.wordprocessingml.document' + response
thus corrupting it.
When using blobs instead, I get a "direct" link to the generated byte stream and no encoding is done on it since the response type is set to 'arraybuffer'.
Note that this is just my own reasoning of why things went wrong with the original code. I might be terribly wrong, so feel free to correct me if that's the case.

Loading XML file into ecore model (Deserialisation)

This is really getting into my nerve...
I already have all the ecore models up and running but I've been unable to load an XML file into those models.This is the code I'm using to do so:
ResultType res = ScheduleTableFactory.eINSTANCE.createResultType();
ByteArrayInputStream is;
try {
/* Read XML file to a string and send it to a buffer */
is = new ByteArrayInputStream((this.xml2String(fileName)).getBytes("UTF-8"));
ResourceSet rs = new ResourceSetImpl();
rs.getResourceFactoryRegistry().getExtensionToFactoryMap().put("xml",
new ScheduleTableResourceFactoryImpl());
Map options = new Properties();
// Just a dummy url to specify the type of the document
URI uri = URI.createURI("http://www.baderous.de/doomz/trankz.xml");
ScheduleTableResourceImpl resource = (ScheduleTableResourceImpl) rs.createResource(uri);
((org.eclipse.emf.ecore.resource.Resource) resource).load(is, options);
}
catch (IOException e) {
e.printStackTrace();
System.exit(0);
}
After a long struggle, now it's reaching the last method inside the try block, but it's giving me this error:
org.eclipse.emf.ecore.resource.Resource$IOWrappedException: Value '2013-04-23.07:55:00' is not legal. (http://www.baderous.de/doomz/trankz.xml, 4, 56)
I wanted to be more precise in this description, but I'm quite new with EMF, so I will just stick to the basics. I would be really grateful if you can help me on this issue.
Thanks in advance!
Try to enclose the value '2013-04-23.07:55:00' in the block CDATA.
The problem was that the date format couldn't be handled, so I had to edit the model in EMf, and now I'm treating dates as Strings.And now everything is working properly!
Hope that in the future someone can benefit from this answer, and save some time.

Printing values of attribute in android

This question may sound similar ,but non of the previous questions have been helpful to me.
In my app there are 2 activities,and 5 .java classes.
I need to know the value of an attribute which is there in the dependent classes (parking.java) (not in Main_Activity).
Till time i have tried putting
System.out.println ,Log.d/e/v/i ,but nothing seems to print my value.
what ever operations I tried,I did only in that particular class.(parking.java)
Also I have set debuggable =true in manifest file.
why the values are not getting printed ,am I putting them in wrong place ? ,or something else needs to be done
Please help
Here is the code
public void setLongitude(String Longitude){
try
{
ExifInterface exif = new ExifInterface(sd.toString());
_longitude=getExifTag(exif,ExifInterface.TAG_GPS_LONGITUDE);
Log.e("Tagname","_longitude");
}
catch(IOException e)
{
e.printStackTrace();
}
}
i have put Tagname as "Nihar"
still no success,
This is the change I have made ,
try
{
ExifInterface exif = new ExifInterface(sd.toString());
_longitude=getExifTag(exif,ExifInterface.TAG_GPS_LONGITUDE);
Log.v("Tagname",_longitude);
}
catch(IOException e)
{
e.printStackTrace();
}
Here is the image

How to use ESAPI to fix Resource Injection (URL) issues

I am new to the Stack Overflow forum. I have a question in remediating the fortify scan issues.
HP Fortify scan reporting the Resource Injection issue for following code.
String testUrl = "http://google.com";
URL url = null;
try {
url = new URL(testUrl);
} catch (MalformedURLException mue) {
log.error("MalformedUrlException URL " + testUrl + " Exception : " + mue);
}
In the above code fortify showing Resource injection in line => url = new URL(testUrl);
I have done following code changes for URL validation using ESAPI to remediate this issue,
String testUrl = "http://google.com";
URL url = null;
try {
String canonURL = ESAPI.encoder().canonicalize(strurl, false, false);
if(ESAPI.validator().isValidInput("URLContext", canonURL, "URL", canonURL.length(), false)) {
url = new URL(canonURL);
} else {
log.error("In Valid script URL passed"+ canonURL);
}
} catch (MalformedURLException mue) {
log.error("MalformedUrlException URL " + canonURL + " Exception : " + mue);
}
However, still Fortify scan reporting as en error. It is not remeditaing this issue. Anything am doing wrong?
Any solution will help lot.
Thanks,
Marimuthu.M
I think that the real issue here is not that the URL may be somehow malformed, but, that the URL may not reference a valid site. More specifically, if I, the bad guy, am able to cause your URL to point to my web site, then you obtain data from my location that is not tested and I can return data that may be used to compromise your system. I might use that to say return a record for "bob the bad guy" that makes bob look like a good guy.
I suspect that in your code you do not set a hard coded value in a string, since this is usually described with words such as
When an application permits a user input to define a resource, like a
file name or port number, this data can be manipulated to execute or
access different resources.
(see https://www.owasp.org/index.php/Resource_Injection)
I think that the proper response will be some combination of:
Do not get the result from the user, but, use the input to choose from your own internal list.
Argue that the value came from a trusted source. For example, read from a strictly controlled database or configuration file.
You do not need to remove the warnings, you need to demonstrate that you understand the risk and indicate why it is OK to use the value in your case.
boolean isValidInput(java.lang.String context,
java.lang.String input,
java.lang.String type,
int maxLength,
boolean allowNull)
throws IntrusionException
type filed in isValidInput function defines a Regular expression or pattern to match with your testUrl.
Like:
try {
ESAPI.validator().getValidInput("URI_VALIDATION", requestUri, "URL", 80, false);
} catch (ValidationException e) {
System.out.println("Validation exception");
e.printStackTrace();
} catch (IntrusionException e) {
System.out.println("Inrusion exception");
e.printStackTrace();
}
It will pass if requestUri matches pattern defined in validation.properties under Validator.URL and its length is less than 80.
Validator.URL=^(ht|f)tp(s?)\:\/\/0-9a-zA-Z(:(0-9))(\/?)([a-zA-Z0-9\-\.\?\,\:\'\/\\\+=&%\$#_])?$
This is piggybacking on Andrew's answer, but the problem Fortify is warning you of is user control of a URL. If your application later decides to make connections to that website, and it is untrusted, this is an issue.
If this is an application where you care more about sharing public URIs, than you'll have to accept the risk, and make sure users are properly trained on the inherent risk, as well as make sure if you redisplay those URLs, that someone doesn't try to embed malicious data.

Categories