I am confused in how to fetch the paginated data from bigquery in Java,
I followed the docs from bigquery but it's not clear for me
// Identify the table itself
TableId tableId = TableId.of("dataset", "table");
// Page over 100 records. If you don't need pagination, remove the pageSize parameter.
TableResult result = bigQuery.listTableData(tableId, BigQuery.TableDataListOption.pageSize(100));
// Print the records
result
.iterateAll()
.forEach(
row -> {
row.forEach(fieldValue -> System.out.print(fieldValue.toString() + ", "));
System.out.println();
});
I have doubt this query will be giving only first 100 rows, whereas I want to fetch all the data from table, for example in page 1 -> 100 rows, page 2 -> 100 rows and so on untill all data is fetched from table.
Please help me understand where I am wrong.. in the table there is 20 Million records
I have doubts this query will be giving only the first 100 rows.
No, it will give all the rows which are returned by your query.
The function iterateAll() iterates through all the records at once and gives you all the rows from the result. Whereas getValues() function will give only paged records of the current page.
You can try the below code snippet for testing and understanding:
// Page over 100 records. If you don't need pagination, remove the pageSize
// parameter.
TableResult results = bigquery.listTableData(tableId, TableDataListOption.pageSize(100));
// .iterateAll() will iterate through the all the records at once
int rowCount=0;
for (FieldValueList row : results.iterateAll()) {
rowCount++;
}
System.out.println("Total Fetched "+ rowCount +" records");
// .getValues() will only iterate through the all the records of the current page
int pageCount=0;
rowCount=0;
do{
System.out.println("page number: "+(++pageCount));
for (FieldValueList row : results.getValues()) {
rowCount++;
}
System.out.println("Fetched "+rowCount+" records");
results = results.getNextPage();
rowCount=0;
}while(results.hasNextPage());
//for the last page
System.out.println("page number: "+(++pageCount));
for (FieldValueList row : results.getValues()) {
rowCount++;
}
System.out.println("Fetched "+rowCount+" records");
System.out.println("Query ran successfully");
Result:
You can follow this cloud doc for more information.
Related
I am saving some records in my sqlite table and records are being saved absolutely fine. Problem comes when I try to retrieve those records. Let's say I have 3 records in my sqlite so when I retrieve those records, it returns me the first record 3 times. Count is correct but why it returns me only first record 3 times?
This is how I am retrieving the records:
public ArrayList<Cursor> getAllBookings() {
ArrayList<Cursor> bookingsList = new ArrayList<>();
SQLiteDatabase readableDatabase = this.getReadableDatabase();
Cursor cursor = readableDatabase.rawQuery("SELECT * FROM " + BOOKING_TABLE, null);
cursor.moveToFirst();
for (int i = 0; i < cursor.getCount(); i++) {
bookingsList.add(cursor);
cursor.moveToNext();
}
return bookingsList;
}
Please note that I have checked the table in Mozilla SQLiteManager and all records are successfully stored.
The problem is that you are adding the cursor itself to your booking list when you should extract and add the values that each cursor row contains.
So if you have a class Booking the method should be declared as
public List<Booking> getAllBookings() {
Also I think it is better to have a do/while loop here and run it until cursor.moveToNext returns false and inside the loop extract all values using getString or similar methods
List<Booking> bookingsList = new ArrayList<>();
//...
do {
Booking booking = new Booking();
booking.setSomething(cursor.getString("somethingColumn"));
//... other columns
bookingsList.add(booking);
} while (cursor.moveToNext());
If you don't have a custom class then you could use an array or a List for each row.
The Account table contains 2 rows when the query is executed it returns only one row.
when the database is opened in SQLite(Software to view sql database on pc) it shows 2 proper rows.
HashMap<String, Integer> returnAccount(Context context)
{
HashMap<String,Integer> account = new HashMap<String, Integer>();
try
{
// Query in question
String statement = "SELECT * FROM `Account`;";
c = mydatabase.rawQuery(statement,null);
if(c.moveToFirst())
{
do {
Log.e("Count", String.valueOf(c.getCount()));
}while (c.moveToNext());
}
}
I tried deleting the entire table and rechecked the all the queries but nothing helped.
I expected all (2) rows.
2019-07-12 18:06:06.780 5663-5663/com.prasad.budgetmanager E/Count: 1
This are the logged data.
Entire Code is at https://github.com/prasad610/BugetManager/blob/master/app/src/main/java/com/prasad/budgetmanager/ExtraFunctions.java
Actual database content
Remove this line.
//Log.e("Move to next", String.valueOf(c.moveToNext()));
The c.moveToNext() is already moved the cursor to Next value. so the while loop will return false it will not execute the next value.
I have indexed the table "CREATE INDEX Items_idx ON Items(Itemcode,Barcode);"
and inserted more than 400 thousand of lines there . How to increase speed in select and return data faster ?
public static Item instanceFromDBBarcode(SQLiteDatabase db, String itemBarcode) {
String[] args = {itemBarcode};
String sql ="SELECT * FROM Items WHERE "+BARCODE+"=?;
Cursor reader = db.rawQuery(sql, args);
if (reader.moveToNext()) {
Item i = readCurrentItem(reader);
reader.close();
return i;
}
reader.close();
return null;
}
For 400 records your issue will not be whether a column is indexed. A sequential scan of 400 records will be trivial for a modern database.
I believe your index is not doing what you anticipate it to, however. An index on (itemcode, barcode) will not help with queries only querying barcode, you should create an index on just barcode.
I want to create an array for each row.
while (result.next()) {
String[] result1 = {
"Dog ID: " + result.getLong("dogs.id"),
", Dog name: " + result.getString("dogs.first_name"),
", Owner name: "
+ result.getString("owners.first_name"),
", Owner phone: " + result.getString("owners.phone") };
resultList.add(result1);
My code write every one row in one array.
Can i get numbers of columns and put a limit?
while (resultset.next()) {
int i = 1;
while(i <= numberOfColumns) {
It's because i can't send entire table as a result from server to client.
You can query by column number result.getLong(columnIndex) but it doesn't make sense in your case withing a loop because you have columns of different types (unless complicating the code).
If you want to optimize the traffic from server to client the way to go is querying for just the columns you need.
If you want to limit the rows returned, it might be better to put the limiting criteria into the SQL query and only return the rows you want to include.
In order to get number of columns in your ResultSet you can use the following piece of code :
Statement stat = conn.createStatement();
ResultSet rs = stat.executeQuery(myQuery);
ResultSetMetaData metaData = rs.getMetaData();
int numOfColumns = metaData .getColumnCount();
I am new in cassandra. I have to inserted 100 rows. But I need to know the time for inserting those rows. I dont know how to do it.
This is the code for inserting 1 row. I have 100 rows like this which I'll write in a loop.
public void loadData() {
session.execute("INSERT INTO songs (id, title, album, artist, tags) "
+ "VALUES ("
+ "756716f7-2e54-4715-9f00-91dcbea6cf50,"
+ "'La Petite Tonkinoise',"
+ "'Bye Bye Blackbird',"
+ "'Joséphine Baker'," + "{'jazz', '2013'})" + ";");
}
I need to know the time for inserting those rows. Please any one help me....
System.out.println(System.nanoTime()); put this in the start of your Resultset and also end of insertion