I apologize for asking this question if someone already asked it and for asking a perhaps dumb question but I am new to this and you are experts and I would love to learn from your expert advice and experience.
I want to add paging to an application displaying 200 records per page. The bottom of the page should have numbers. If there are 1050, the first page will display 100 and the bottom of the page will show numbers 1,2,3,4,5 & 6.
What is the general logic to accomplish this?
I know that the database must select 200 every time and I must keep track of the first record.
Is there a way to know how many records will be returned total so that I can know how many numbers to display on the bottom of the page? Does it require selecting a count() statement or something else?
For the 1050 records, The numbers 1,2,3,4,5 & 6 will display and clicking each one requires a call to the server. Is there a way to know how many records will be returned in the next call to the server? Does it require selecting a count() statement or something else?
You can use the JPA Criteria API to accomplish this. Assuming TypedQuery, you would use setFirstResult and setLastResult to limit the records returned from the database. Of course the values for these methods will be dependent on what page was requested, and how many records are getting displayed per page.
first = (page - 1) * page size;
last = (page * size) - 1;
Note: This assumes the first page is 1 (as opposed to zero).
To get a record count you execute a standard Criteria query. As an example:
final CriteriaBuilder builder = entityManager.getCriteriaBuilder();
final CriteriaQuery<Long> countQuery = builder.createQuery(Long.class);
countQuery.select(builder.count(countQuery.from(MyEntity.class)));
final Long count = entityManager.createQuery(countQuery)
.getSingleResult();
You can customize the above code to perform the count relative to another query as well. Finally, you need some way of communicating the total count back to the client. One way of doing this is wrapping the result set of your query in another object that contains an attribute for the count, or returns said object as a result of your DAO call. Or alternatively, you could store the count attribute in an object in your request scope.
public class ResultListWrapper<T> {
private Long count;
private Collection<T> results;
// constructor, getters, setters
}
You will usually perform the same query except using count int the select list instead of the columns prior running the actual query to find how many pages there are. Getting a specific page in hibernate can then be done with something like:
int maxRecords = // page size ...
int startRow = // page number * page size
Query query = session.createQuery('...');
query.setMaxResults(maxRecords);
query.setFirstResult(startRow);
If performing the extra query is too expensive an operation then you could consider just providing next/previous links or alternatively providing extra pages as needed (e.g. As the last of the loaded data comes into view) via ajax.
For displaying paged results including showing page number links I would suggest (assuming you are using jsp pages to display this data) using the display tag JSP tag library. Display tag handles paging and sorting display of tabular data assuming you can get it to the JSP.
Yes, you have to do a select count before getting the rows themselves.
Have a look at Spring Data JPA. It's basically working how Perception suggested earlier, just that you don't have to do it, the framework does that for you. You get paging and sorting by default (you still need to do filtering yourself if you don't want to use Query DSL).
They have a 1h video on the main page that's showing a quick intro, but you'll only need to watch about 15 mins to get an idea of what id does.
Related
I am trying to create a simple pagination in mongodb by below code.
collection.find().skip(n).limit(n);
but doesn't it looks like there will be a performance issue if we see that in java terms first find will return all the records consider i have 2 million records, then it will pass it to skip method then it will be passed to limit method. it means every time this query will be fetching all the db records or mongodb drivers work differently, what i missed?
When talking about pagination in MongoDB, it is easily to write this code:
collection.find().skip(pageSize*(pageNum-1)).limit(pageSize);
Above is the native solution supported by MongoDB, but this is not efficient if there are huge documents in the collection. Suppose you have 100M documents, and you want to get the data from the middle offset(50Mth). MongoDB has to build up the full dataset and walk from the beginning to the specified offset, this will be low performance. As your offset increases, the performance keeps degrade.
The root cause is the skip() command which is not efficient and can not take big benifit from index.
Below is another solution to improve performance on large data pagination:
The typical usage scenario of pagination is that there is a table or list to show data of specified page, and also a 'Previous Page' & 'Next Page' button to load data of previous or next page.
If you got the '_id' of the last document in current page, you can use find() instead of skip(). Use _id > currentPage_LastDocument._id as one of the criteria to find next page data. Here is pseudocode:
//Page 1
collection.find().limit(pageSize);
//Get the _id of the last document in this page
last_id = ...
//Page 2
users = collection.find({'_id': {$gt: last_id}}).limit(pageSize);
//Update the last id with the _id of the last document in this page
last_id = ...
This will avoid MongoDB to walk through large data when using skip().
We are getting 1000 of records from services we need to display all records in jsp page. We have set the data to object and stored in java collections. How can get that collections in java script using Ajax and need to display 10 records in every time based on scroll we can load another 10 records upto completion.
Please suggest the compatible technology.
At this time We are using the struts2 and jquery.
It sounds like you want something along the lines of SlickGrid. It is very fast, and is the data grid that powers SEDE result tables.
Another option, which I have used before with great results, is a YUI DataTable with pagination (server-side or client-side). With client-side pagination — which is typically faster, since all the data is already in the browser — I've created YUI data tables that work with more data than the browser can parse at once, with minimal performance degradation.
You can try implementing a simple pagination technique
int totalRecords;
int maxRecordsPerPage;
int totalPages = ( totalRecords / maxRecordsPerPage );
int displayRecordFrom;
int displayRecordTo;
Total Records : Number of records fetched.
Max Records Per page :
Total Pages : this is optional, either you can display total pages some thing like google or just put next button or link
Display Record From And To : As you are storing records in collection, it can be fetched used get(index)
After fetching results use Jettison or any other Java JSON library to output results into JSON. Instead of working from scratch, Its better to use pre-tested third party Javascript components using JQuery or other library.
We have a simple table with employee data containing a 100k records. We want to display them using pagination on a simple JSP page with helper class/bean.
On one page we'll display 50 records.
Shall I fetch 50 records as per the page number & display them/ Use some other alternative?
What should be the best design approach to this problem?
You might want to do the paging with database & using cache for reduce the data accessing to database.refer: http://onjava.com/pub/a/onjava/excerpt/jebp_3/index2.html "Cache on the server"
There are many paginators for java server page. You can use one of them and display some records on the pages. For paging you can use for example - jsptags paginator
100k records thats mean 50 X 2000 or could i say it need 2000page to see all record. if u use pagination and dispaly pagination completly using first, previous, next, last and number each page it will be harmfull. cause #1st u must count all record, it spend more time in your query to load #2nd u must make a algorith to display it on your view.my advice is using pattern that was facebook and twiit use..on pagination only use next and previous.it will be simple and reduce page load time.
thanks
I am using displaytag 1.0 in my Struts base application. I have 5 million rows to showing reports of all rows data by paging with 50 records. It takes 5 minutes to rendering the data. Can we reduce this one? Please suggest how make it with in one minutes.
It is apprently hauling the complete database into Java's memory instead of only the data of interest (the current page). You'll need to introduce request-based pagination at the DAO level.
To start, in your DAO class do a SELECT stuff FROM data LIMIT firstrow OFFSET rowcount or something like that depending on the DB used. Then in your JSP create a HTML table yourself with help of JSTL c:forEach or Struts' logic:iterate. Keep one or two request parameters in the background in a input type="hidden" element: the first row to be displayed (firstrow) and eventually the amount of rows to be displayed at once (rowcount). Finally provide a bunch of buttons which instructs the server side code to in/decrement the firstrow with rowcount everytime. Just do the math.
You can find a more detailed answer here.
I'm using displaytag to build tables with data from my db. This works well if the requested list isn't that big but if the list size grows over 2500 entries, fetching the result list takes very long (more than 5 min.). I was wondering if this behavior is normal.
How you handle big list / queries which return big results?
This article links to an example app of how to go about solving the problem. Displaytag expects to be passed a full dataset to create paging links and handle sorting. This kind of breaks the idea of paging externally on the data and fetching only those rows that are asked for (as the user pages to them). The project linked in the article describes how to go about setting this type of thing up.
If you're working with a large database, you could also have a problem executing your query. I assume you have ruled this out. If not, you have the SQL as mentioned earlier - I would run it through the DB2 query analyzer to see if there are any DB bottlenecks. The next step up the chain is to run a test of the Hibernate/DAO call in a unit test without displaytag in the mix. Again, from how you've worded things, it sounds like you've already done this.
The Displaytag hauls and stores everything in the memory (the session). Hibernate also does that. You don't want to have the entire DB table contents at once in memory (however, if the slowdown already begins at 2500 rows, it more look like a matter of badly optimized SQL query / DB table; 2500 rows should be peanuts for a decent DB, but OK, that's another story).
Rather create a HTML table yourself with little help of JSTL c:forEach and a shot of EL. Keep one or two request parameters in the background in input type="hidden": the first row to be displayed (firstrow) and eventually the amount of rows to be displayed at once (rowcount).
Then, in your DAO class just do a SELECT stuff FROM data LIMIT firstrow OFFSET rowcount or something like that depending on the DB used. In MySQL and PostgreSQL you can use the LIMIT and/or OFFSET clause like that. In Oracle you'll need to fire a subquery. In MSSQL and DB2 you'll need to create a SP. You can do that with HQL.
Then, to page through the table, just have a bunch buttons which instructs the server side code to in/decrement the firstrow with rowcount everytime. Just do the math.
Edit: you commented that you're using DB2. I've done a bit research and it appears that you can use the UDB OLAP function ROW_NUMBER() for this:
SELECT id, colA, colB, colC
FROM (
SELECT
ROW_NUMBER() OVER (ORDER BY id) AS row, id, colA, colB, colC
FROM
data
) AS temp_data
WHERE
row BETWEEN 1 AND 10;
This example should return the first 10 rows from the data table. You can parameterize this query so that you can reuse it for every page. This is more efficient than querying the entire table in Java's memory. Also ensure that the table is properly indexed.