We Keep Coding

How to batch the fetches when pulling huge number of records from Oracle using JPA?

I have a Java application where we use spring data JPA to query our oracle database. For one use case, I need to fetch all the records present in the table. Now the table has record count of 400,000 thousand and it might grow in the near future. I don't feel comfortable pulling all records into the JVM since we don't know how large they can be. So, I want to configure the code to fetch specific number of records at a time say 50,000 and process before it goes to next 50,000. Is there a way I can achieve this with JPA? I came across this JDBC property that can be used with hibernate hibernate.jdbc.fetch_size. What I am trying to understand is if I use repository.findAll() returning List<Entity>How can a fetch Size work in this case? because List will have all the entities. I was also looking into repository methods returning Stream<>, not sure if I have to use that. Please do suggest. If there can be better solution for this use case?
Thanks

With JPA you can use the Pagination feature, means you tell the Repository how many result should be present at one page. (E.g. 50 000)
For more information follow up here https://www.baeldung.com/jpa-pagination

Create table if they do NOT exist

Currently there are schema actions that let you recreate tables on each startup, but dropping them obviously means you lose all rows of that table.
In CQL you can make a query like
CREATE TABLE IF NOT EXISTS keyspace.tablename(....)
But I can't find any way of achieving a similar result with spring-data-casssandra, one that would let me start my app for the first time and on without changing anything.
Is there any way to create a table defined in a POJO with #Table ONLY if said table does not already exist?

See DATACASS-219.
I just recently added support for CREATE TABLE IF NOT EXISTS keyspace.tablename (..);. This will be available in SD Cassandra 1.5 M1 (Ingals). I'll consider backing porting this to 1.4 for the 1.4.2.RELEASE.
The only other way to accomplish this for the time being (if not using the 1.5.0.BUILD-SNAPSHOT containing the DATACASS-219 fix) is to set your SchemaAction to NONE and provide your own raw CQL, initialization scripts to the CassandraSessionFactoryBean using the setStartupScripts(:List) method.

Hibernate Search query for class

I'm using hibernate search 4.4.0. And I met a problem recently.
E.g, I have 2 classes INDEXING and DATA_PROPERTY. There is no association between 2 of them. And I can't change them or creat a new class to associate 2 of them.
Part of Lucene indexing:
mapping.entity(DatatypeProperty.class).indexed().providedId()
.property("rdfResource",ElementType.FIELD).field().analyze(Analyze.NO).store(Store.YES)
.property("partitionValue", ElementType.FIELD).field().analyze(Analyze.NO)
mapping.entity(Indexing.class).indexed().providedId()
.property("rdfResource",ElementType.FIELD).field().analyze(Analyze.NO).store(Store.YES)
Now in the SQL, I use
SELECT IND.RDF_RESOURCE
FROM INDEXING IND, DATA_PROPERTY DP
WHERE IND.RDF_RESOURCE = DP.RDF_RESOURCE
AND IND.OBJECT_TYPE_ID_INDEXED IN (........)
AND DP.PARTITION_VALUE IN (......)
AND .......
How can I translate IND.RDF_RESOURCE = DP.RDF_RESOURCE in Hibernate Search???
I thought maybe I can use the query to find all the RDF_RESOURCE of class DatatypeProperty and matching all of them in the query for class Indexing. But it seems very inefficiency.
Does anyone has a better way for this?

I have 2 classes INDEXING and DATA_PROPERTY. There is no association
between 2 of them. And I can't change them or create a new class to
associate 2 of them.
In this case you are between a rock and a hard place. You will need to associate the records somehow and the most obvious choice is via an association. Also, you cannot compare a SQL join with a free text based index provided by Lucene.
One potential solution could be to write a custom bridge which at indexing time executes the join and indexes the relevant data, so that you can target it directly via your query. Whether this works for you will depend on your use case. In your example setup, I don't see any field which would benefit from free text search. I can only assume that you are only showing parts of your code. If not, why don't you just stick with SQL?

Get discography of an artist exactly like the overview page on musicbrainz

I am trying to use the musicBrainz api for getting the discography of an artist using the following method: http://www.musicbrainz.org/ws/2/release/?query=artist:eminem but i get a lot of unsorted and repeated data. I know that i can use keywords (AND, OR, etc) but i do not really know how to sort it by date or filter repeated data. Is there a way to do this in the rest call or have i to implement these sort methods in my code?

About repeated data:
It sounds what you want are release groups, not releases. For example a certain album is one release group on MusicBrainz, but usually has multiple different releases in this group due to different editions of an album.
About sorting:
Unfortunately cou can't sort the data while querying the web service yet. There is a ticket for that: MBS-5636.
If you do want to sort the data, you have to fetch all of it and sort it yourself.

Implementing article revision history for Java based web application

Any ideas of how best i can implement article revision history for a Java based web application and save it in AuditLog
StackOverflow already has such a feature allowing one to see the differences from one version to another, almost like SVN clients.
This is more of a design than implementation question.
addition: How would one display these changes on the web page?
addition: Proposed solution
Article
--------------------------------
Integer id
String title
String body
List<Tag> tags
AppUser createdBy
Date createdDate
AuditLog
--------------------------------
Integer id
Integer objectId
Operation operation // enum with UPDATE and DELETE. I won't audit an insert
Date createdDate
AppUser createdBy
String class
String revisionXML
String comment
A Hibernate Interceptor will intercept
the save process and use Castor XML to create an XML string of the old object.
The class and id is used to get the revisions of a particular object.
google-diff-match-patch will be used for creating HTML diff files

The best solution would be to use a database or storage which already supports versions, for example Apache Jackrabbit.
If that's not an option, then you must decide where you want to store the articles. On the file system? Then make each article a directory and save the revisions as numbers (00001, 00002, etc.) and put the number of the last revision in a special file (like current). Then you can quickly find out how many versions there are (just look into current) and go forward and back.
If you use a database, then add a version number field to the article table and add a second table or a flag which says which one the current version is. You could also select with max(version) but those SQL constructs tend to be pretty ugly and confusing. It's much more simple to save this information elsewhere.
[EDIT] To generate diffs, look at this project: google-diff-match-patch

I would use an existing VCS (for instance, SVN) under the hood. There you have the revision history - all that's left to make is an interface from your app to the VCS.