I'm looking for a list of common datetime formats used in logs (e.g. webserver, database, etc).
Even better would be a (java) library that can extract date and time from a given string ( < 10KB).
Does anyone know a good one?
this library is likely a good place to start: SimpleDateFormat
The docs contains the an introduction to the standard datetime format strings. But as #Olaf points out, you're going to need to specify what the format is beforehand or there is literally no way differentiate certain dates from one another.
Looks like what you'd want to do is construct a range of date formats that might match, apply all of them to a date string, then see which date is closest to Datetime.now().
Although this doesn't answer your question directly, but Java includes libraries for working with regular expressions. It would be pretty easy to write a library of your own based on that. I've has a lot of success extracting all sorts of data using regular expression. It would certainly be less than 10kb and would require no external dependencies other than the JDK.
Related
I need to store the current time as a String in a database. The time can be in different timezones, so I'm looking at using Java SE 8's new ZonedDateTime class.
I notice that the toString() method automatically outputs:
2016-04-15T17:40:49.305-05:00[America/Chicago]
This also seems to be readable by ZonedDateTime.parse() and convert to the right values.
If all I am doing is storing these values and I don't need to ever convert the value to a user-readable format, is this all I need to do to accurately store data with proper timezones? For example, if I insert two ZonedDateTimes into an SQL database by storing their toString() representations, and I later read in these times by using ZonedDateTime.parse(), can I expect things like isAfter() and isBefore() to work just fine?
Or am I missing a step in between? After trying to figure out timezones in Java 7 this feels almost too easy.
Generally speaking I would avoid relying on parsing the toString() representation of any class in my application.
toString() is meant to provide a human readable version of a class so it's subject to change from relase to release.
I would suggest you to force the format to be the one that you expect, e.g applying:
String toStoreInDb = DateTimeFormatter.ISO_ZONED_DATE_TIME.format(zonedDateTime);
...
ZonedDateTime fromDb =
ZonedDateTime.parse(stringFromDb, DateTimeFormatter.ISO_ZONED_DATE_TIME);
This way your application will resist to any toString() change.
Moreover take a look at this bug:
Bug in ZonedDateTime.parse()
Yes, that will accurately store the date, and using the .parse() method will allow you to use the other methods of ZoneDateTime. Though if you want to be able to use sorting functions with your db then you will need to either manually convert the ZonedDateTime into a timestamp or use your ORM's features to do it for you.
Recently I am being challenged by quite an "easy" problem. Suppose that there is sentences (saved in a String), and I need to find out if there is any date in this String. The challenges is that the date can be in a lot of different formats. Some examples are shown in the list:
June 12, 1956
London, 21st October 2014
13 October 1999
01/11/2003
Worth mentioning that these are contained in one string. So as an example it can be like:
String s = "This event took place on 13 October 1999.";
My question in this case would be how can I detect that there is a date in this string. My first approach was to search for the word "event", and then try to localize the date. But with more and more possible formats of the date this solution is not very beautiful. The second solution that I tried is to create a list for months and search. This had good results but still misses the cases when the date is expressed all in digits.
One solution which I have not tried till now is to design regular expressions and try to find a match in the string. Not sure how much this solution might decrease the performance.
What could be a good solution that I should probably consider? Did anybody face a similar problem before and what solutions did you find?
One thing is for sure that there are no time, so the only interesting part is the date.
Using the natty.joestelmach.com library
Natty is a natural language date parser written in Java. Given a date expression, natty will apply standard language recognition and translation techniques to produce a list of corresponding dates with optional parse and syntax information.
import com.joestelmach.natty.*;
List<Date> dates =new Parser().parse("Start date 11/30/2013 , end date Friday, Sept. 7, 2013").get(0).getDates();
System.out.println(dates.get(0));
System.out.println(dates.get(1));
//output:
//Sat Nov 30 11:14:30 BDT 2013
//Sat Sep 07 11:14:30 BDT 2013
You are after Named Entity Recognition. I'd start with Stanford NLP. The 7 class model includes date, but the online demo struggles and misses the "13". :(
Natty mentioned above gives a better answer.
If it's only one String you could use the Regular Expression as you mentioned. Having to find the different date format expressions. Here are some examples:
Regular Expressions - dates
In case it's a document or a big text, you will need a parser. You could use a Lexical analysis approach.
Depending on the project using an external library as mentioned in some answers might be a good idea. Sometimes it's not an option.
I've done this before with good precision and recall. You'll need GATE and its ANNIE plugin.
Use GATE UI tool to create a .GAPP file that will contain your
processing resources.
Use the .GAPP file to use the extracted Date
annotation set.
Step 2 can be done as follows:
Corpus corpus = Factory.newCorpus("Gate Corpus");
Document gateDoc = Factory.newDocument("This event took place on 13 October 1999.");
corpus.add(gateDoc);
File pluginsHome = Gate.getPluginsHome();
File ANNIEPlugin = new File(pluginsHome, "ANNIE");
File AnnieGapp = new File(ANNIEPlugin, "Test.gapp");
AnnieController =(CorpusController) PersistenceManager.loadObjectFromFile(AnnieGapp);
AnnieController.setCorpus(corpus);
AnnieController.execute();
Later you can see the extracted annotations like this:
AnnotationSetImpl ann = (AnnotationSetImpl) gateDoc.getAnnotations();
System.out.println("Found annotations of the following types: "+ gateDoc.getAnnotations().getAllTypes());
I'm sure you can do it easily with the inbuilt annotation set Date. It is also very enhancable.
To enhance the annotation set Date create a lenient annotation rule in JAPE say 'DateEnhanced' from inbuilt ANNIE annotation Date to include certain kinds of dates like "9/11" and use a Chaining of Java regex on R.H.S. of the 'DateEnhanced' annotations JAPE RULE, to filter some unwanted outputs (if any).
The SolrJ library offers different parsers for Solr's responses.
Namely:
BinaryResponseParser
StreamingBinaryResponseParser
NoOpResponseParser
XMLResponseParser
Sadly the documentation doesn't say much about them, other than:
SolrJ uses a binary format, rather than XML, as its default format.
Users of earlier Solr releases who wish to continue working with XML
must explicitly set the parser to the XMLResponseParser, like so:
server.setParser(new XMLResponseParser());
So it looks like the XMLResponseParser is there mainly for legacy purposes.
What are the differences between the others parsers?
Can I expect performance improvements by using an other parser over the XMLResponseParser?
The Binary Stream Parsers is meant to work directly with the Java Object Format (the binary POJO format) to make the creation of data objects as smooth as possible on the client side.
The XML parser was designed to work with the old response format where there wasn't any real alternatives (as there was no binary response format in Solr). It's a lot more work to consider all the options for an XML format than use the binary format directly.
The StreamingBinaryResponseParser does the same work as the BinaryResponseParser, but has been designed to make streaming documents (i.e. not creating a list of documents and returning that list, but instead return each document by itself without having to hold them all in memory at the same time) possible. See SOLR-2112 for a description of the feature and why it was added.
Lastly, yes, if you're using SolrJ, use the binary response format, unless you have a very good reason for using the XML based one. If you have to ask the question, you're probably better off with the binary format.
I'm trying to get date data types from an excel file, but the output when he's reading is 41306.038888888892.
This value just appear for date, is there any way to get the normal date?
I did not find anything searching in the web.
Hope someone can help
thanks
As with your previous question, I'd strongly suggest you try reading and understanding some of the various examples for this, you'll save yourself a lot of time! The two main ones probably being XSSFEventBasedExcelExtractor in Apache POI and XSSFExcelExtractorDecorator in Apache Tika
If you take the easy route, then you can just use XSSFSheetXMLHandler, which will handle all the pesky formatting stuff for you, and give you nicely formatted strings for your dates
Otherwise, if you want to stay at the low level, then you need to check the formatting rule applied to a cell. If it's a date-based format string, you then need to convert it from a number into a Date object. Handily, there's a POI class DateUtil which can both help you check if a cell is Date formatted, and convert it into a Java Date object for you
I'm trying to find a complete tutorial about formatting strings in java.
I need to create a receipt, like this:
HEADER IN MIDDLE
''''''''''''''''''''''''''''''
Item1 Price
Item2 x 5 Price
Item3 that has a very
long name.... Price
''''''''''''''''''''''''''''''
Netprice: xxx
Grossprice: xxx
VAT: xxx
Shipping cost: xxx
Total: xxx
''''''''''''''''''''''''''''''
FOOTER IN MIDDLE
The format to pass to string.format is documented here:
http://java.sun.com/j2se/1.5.0/docs/api/java/util/Formatter.html#syntax
From the page:
The format specifiers for general,
character, and numeric types have the
following syntax:
%[argument_index$][flags][width][.precision]conversion
The optional argument_index is a
decimal integer indicating the
position of the argument in the
argument list. The first argument is
referenced by "1$", the second by
"2$", etc.
The optional flags is a set of
characters that modify the output
format. The set of valid flags depends
on the conversion.
The optional width is a non-negative
decimal integer indicating the minimum
number of characters to be written to
the output.
The optional precision is a
non-negative decimal integer usually
used to restrict the number of
characters. The specific behavior
depends on the conversion.
The required conversion is a character
indicating how the argument should be
formatted. The set of valid
conversions for a given argument
depends on the argument's data type.
formating string is some what complicated, for this kind of requirement.
so its better to go for some reporting tool using the format you have given.
which would be the better approach.
Either a crystal report or some others which are easy to implement.
Trying to do this with formatting a string will cost you to much time and nerves. I would suggest a templating engine like Stringtemplate or something similar.
with doing these you will separate the presentation from the data and that will be a very good thing in the long run.
See if these classes in java.text package can help..
Format
MessageFormat
Yea as solairaja said if you are planning to create reports or receipts you can go for reporting tools as Crystal reports
Crystal Report Crystal Report Tutorial
Or if you plan to use StringFormatting itself then "StringBuffer" would be the best option coz u can play around with it.
You should probably look at Java templating tools for this sort of multi-line reporting formatting.
Velocity is simple and forgiving of errors. Freemarker is very powerful but more intolerant. I would perhaps look at Velocity initially, and if you have to do more of this sort of work, take a further look at Freemarker.
Looks like the general advice from the community as a better approach to solve your problem is using a reporting tool.
Here you have a detailed list of open source Java charting and reporting tools:
http://java-source.net/open-source/charting-and-reporting
The most well known is, in my opinion, Jasper Reports. A lot of resources about it are available on the web