I'm trying to create something similar to the below HTML form using Play framework with Java:
https://www.w3schools.com/howto/tryit.asp?filename=tryhow_js_filter_list
But instead of hard-coding the selectable values to be searched (like "Adele" in the example I gave above), I plan on querying a large dataset with thousands or 10s of thousands of records and allowing users to search that dataset. Please answer my below 2 questions:
Is this possible to do in the Play Framework
Would doing this be bad practice? Would it be better to have users enter a string and only query when they hit the 'Search' button?
This is 100% possible with Play! The bottleneck will be your dataset. Is it a file, an SQL database, a nosql database, a search engine, a webservice query. This is where the problem usually lies.
Debatable. Implement it any way you like, if you experience bottlenecks tweak it. You could implement a 200ms timeout before firing the ajax request. A very nice tool is select2 which allows you to easily configure this.
General question here. I have a Spring Web MVC Application that allows users to enter data one record at a time. Validation checks are run when adding/editing each individual record (database calls, client side validation, etc)
We want to provide users a way to bulk insert many records on a single load. Right now the obvious choice is importing an excel spreadsheet, however I feel like this will require a ton of redundant work as we will have to provide all the same validation checks, dynamic string building, and preloading drop downs in this excel sheet as we have done in our application. Thus my question is, is there a simple solution of recreating this process via a web interface that would imitate entering data into a spreadsheet (Any tool or framework of sorts)? If this could be done on the front end we would be able to utilize all the functionality we have already implemented
Hope this isn't a poor question, I would just really like to avoid spreadsheets all together
I use http://handsontable.com - it is a javascript component that I use.
You can get quite close to Excel-like behaviour, in a browser. You can also copy / paste to and from Excel with it.
We are using AppEngine and the datastore for our application where we have a moderately large table of information containing a list with entries.
I would like to summarize the list of entries in a report specifying how many times each one appears e.g. normally in SQL I would just use a select distinct for a column, then loop over every entry and just use select count(x) where value = valueOfEntry.
While the count portion is easily done, the distinct problem is "a problem". The only solution I could find remotely close to this is MapReduce and most of the samples are based on Python. There is this blog entry which is very helpful but somewhat outdated since it predated the reduce portion. Then there is the video here and a few more resources I was able to find.
However, its really hard for me to understand how to build he summary table if I can't write to a separate entity and I don't have a reduce stage?
This seems like something trivial and simple to accomplish but requires so many hoops, is there no sample or existing reporting engine I can just plugin to AppEngine without all the friction?
I saw BigQuery, but it seems like a huge hassle to move the data out of app engine and into that store. I tried downloading the data as CSV but ran into many issues with that as well. It doesn't seem like a practical solution in the long run either.
There is a document explaining some of the concepts of the mapreduce for java. Although it is incomplete, it shares most of the architecture with the python version. In that document, there's also a pointer to a complete java sample mapreduce app, that reads from the datastore.
For writing the results, you specify an Output class. To write the results to a new datastore entity you would need to create your own Output Class. But you could also use the blobstore (see BlobFileOutput.java).
Other alternative, is that whenever you write one of your entities, you also write/update another entry to a EntityDistinct data model.
If you plan on performing complex reports and you can anticipate all your needs now, I would suggest you to look again at Big Query. BigQuery is really powerful and works perfectly on very massive datasets. You can inspect http://code.google.com/p/log2bq/ which is a python project that loads the logs into Big Query using mapreduce. Or you could also have a cron job, that every once in a while fetches all new entities and moves them into Big Query.
Related to the friction, remember that this is a no-sql database, and as such has some advantages but some things are inherently different to SQL. Remember you can always use Google Cloud SQL, given that your dataset is of limited size, but you would loose the replication and fault-tolerant capabilities.
I think this could help you: http://jjmpsj.blogspot.ro/2008/05/appengine-output-tricks-reporting.html?m=1
Ok, So I have to generate very complex reports from my DB.
If I am to do it with SQL the query themselves will be complex, and I will have to do some more manipulation on the results later, code level.
How do libraries like Jasper Reports/ Crystal Reports and friends save me time when developing such reports?
What will they give me?
These tools may not necessarily save you that much time writing the SQL, but they will give you a nice clean presentation of the data for your customers.
When it comes to writing the query, they won't give you much. Some of the Business Intelligence tools out there (although none of the open source ones I know of) give you infrastructure for point and click queries. A developer still has to do the hard work of setting them up, especially for complex queries, to provide a view of the data that is simple enough for the user.
What these tools do give you is an engine to take care of a lot of the presentation issues, such as formatting, page breaks, converting to multiple formats (HTML, PDF, etc), running, storing and retrieving the reports and other things in that area.
What ever tool you are using to generate reports, you got to write your complex SQL or define the views out of your data to generate the reports. The benefit of using the tools is
Can generate multiple type of report formats (PDF, EXCEL, TEXT..)
Eliminates a lot of routine chores like setting date/time stamps, header formatting..
Maintains consistency across multiple report pages which is generally a bit hard to acheive
Creates a template layout which can be reused and some fancy functions like sub reports and easy type conversions in some cases..
etc...
If I am to do it with SQL the query themselves will be complex, and I will have to do some more manipulation on the results later, code level.
Jasper could be very effective in manipulating the data before/during generating the report. By design, every variable, parameter or field in JasperReports could be a Java expression, imagine the possibilities.
How do libraries like Jasper Reports/ Crystal Reports and friends save me time when developing such reports?
Both Jasper and Crystal give you the ability to extract the fields from the query, Jasper also gives you abilities like extracting fields from XML or JavaBeans. (i dont know about Crystal)
What will they give me?
Those two are very good tools for making reports, i have made very complicated reports on Jasper, so the tools themselves are very advanced, its how you will use them that counts.
As others have said, what these tools give you is an link between your data and how you want to display it - be that a listing, chart, cross tab (grid of summarized values) etc, with options for your own parameters, calculations and queries.
If you are embedding your reports in your application, I'd say your selection of a reporting platform gets down to what technology are you comfortable with. For Microsoft, look at Crystal. For Java, look at JasperReports and iReport.
JasperReports does have sub reports.
Sherman
Jaspersoft (company behind JasperReports)
If you mean complex as in the data and manipulation, then code-level systems like Jasper and Docmosis will let you manipulate the data in code as well as in the query. You can combine the best of SQL with the best of code algorithms to get your report data ready. This will save you time if maintaining simple SQL + simple code is better in your environment than maintaining complex SQL.
It depends on your company and your skills. If you're in a small environment where you control everything, have access to anything, can download and install whatever you want, and you are a programmer and Reporting is more of an annoyance, Jasper has a lot of options. You need to do the hard work yourself, but you have the capabilities to do it.
But if you are in a company, like many people are, where the DBA controls the database, the SysAdmin controls the servers, you can't download and install whatever you want, things need to be checked in, tested, approved, etc, Reporting is something people need now, not after you figure out a way to code something that Jasper Doesn't offer, and your job is more a part of Reporting and Analysis over programming, then Jasper is a horrible tool.
Plus, like it or not, the other tools have far more support and help. Jasper is a bit useless when it comes to simple help. Search all the Jasper Forums. Look at any other forum about Jasper. Most questions go unanswered. And if you post something, it's very likely your question will never be answered. Or the answer isn't close to what you need.
It's a double edged sword. Jasper gives you a lot of freedom, but a lot of it is a bad tool to use for the real corporate world. Unless you have a job where you don't do much all day, Jasper takes too long for simple things to get done. If it's about getting reports to the right people at the right time, stay away from Jasper. Nobody really has the time or understanding to sit and figure out a work around and write up some code if a report needs to be done yesterday. End users don't care. It's the age old problem. It's built for techies, not for users. And the Reporting world is more about the users than the techies.
It's hard to say with the not so detailed info you provided, but I have 14 years experience with Crystal Reports and can tell you that it's very powerful. You can manipulate the resulting dataset in Crystal Reports in many ways. Crystal Report has a very decent internal program language to do this. And then you'll have subreports, which will save your day if it get hard.
I never found a report builder that can math Crystal Reports, especially looking at it's low price. If money isn't the problem then you can also look at List & Labels, www.combit.net.
One major thing Crystal Reports gives you is the ability to do Subreports--that is, multiple, related queries that return nested result sets. This can really simplify the individual queries your write. Take a look at http://vb.net-informations.com/crystal-report/vb.net_crystal_report_subreport.htm
I'm thinking about making a simple web application to practice custom tags, EL, ...
Now I'm thinking about how to make a simple front page.
I want to have a front page where I'll show a short description of a post and then the user can click it to see the full article.
Further down the line I'd like to attach a poster to it, and even further down the line I'd like to allow people to leave a comment.
Now I see two ways to do this:
a) put it all into a database
b) put the short description and the article into a .tag file and put the comments and users into the database.
Now I'm wondering which way would you go, or would you go for something entirely else?
The first way is probably the easiest but it does require access to the database "often".
The second way is a bit more "sloppy", especially depending on my implementation but it does have the advantage of accessing the database less often.
And any recommendations on keeping the data actual?
I could either load everything each time somebody accesses the news page, or I could put it in the application scope and put the articles into a bean and use a listener.
And do you use hibernate/jdbc/... for a database connection?
I'm getting the feeling that the actual programming will be the easiest part.
Any directions (or book recommendations for that matter) are welcome.
I've read head first servlets & jsp, and while it does a wonderful job of explaining how to develop the application I find it a bit lacking in the when/how to connect with the database and how to optimize it.
Sorry for the long post that possibly doesn't really fall under the scope of this site.
As far as I can see, you are thinking too much about performance. You should not. It is of a little concern in the start. Go what feels right. Tackle performance when its actually lacking.
I would suggest you
You should use some pooling mechanism for database connection. Its very important and make the process very efficient. Take a look at DBCP or C3P0 or something.
to go store your data into the database, even the short description, in some appropriate table.
Moreover, don't load everything when somebody accesses the page, it might go futile and it will take a lot more time and the user gets frustrated.
you can cache data later when you feel its a good idea. Hibernate provide caching real easy, you might try to incorporate Hibernate, as you mentioned it yourself.
you can use AJAX calls wherever appropriate to get rapid request/response.
These are few things I like to mention.