We Keep Coding

Reading some of data (not all) from a Firebase database node

I'm working on project in which I need to read the firebase node in this manner:
Suppose I have a node messages in a firebase realtime database. In that node, I have 20, 000 messages.
When a user enter in the message box, I need to show messages. But Its too time consuming to read all 20, 000 messages at a time.
What I want it, to read 30-40 messages at a time and show those to the user.
When user press more message button, I will read another 30-40 messages and show them to users. and so on.... it would faster and convenient.
But I can't find any way to make the things work like this.
I tried with ValueEventListener(), but it reads all the data in that messages node and it's time consuming. I also tried reference.limitToFirst(30) but it read first 30 messages, then what do I do? How can I read next 30 messages?
Could I have present my problem clearly? :(
Thanks in advance.

But it's too time-consuming to read all 20,000 messages at a time.
20,000 messages can be considered a lot of data. So I strongly recommend against loading such an amount of data in one go.
What I want is, to read 30-40 messages at a time and show those to the user. When the user presses more message button, I will read another 30-40 messages and show them to users. and so on.... it would faster and more convenient.
What you're looking for is called pagination. This means that you need to load the data progressively in smaller chunks. You need to load a group of 30-40 messages, one after another. This was already covered a lot of times before, and this question already has a lot of solutions for the Realtime Database:
https://stackoverflow.com/search?q=Firebase+Realtime+Database+Pagination+Android
I also tried reference.limitToFirst(30) but it read the first 30 messages, then what do I do? How can I read the next 30 messages?
I recommend you have a look at some of those questions and give it a try. If you have a hard time implementing it, then show us an MCVE with what you have tried, and this way, I and other Firebase developers can help you.
However, if you consider at some point in time trying using Cloud Firestore, I think that this answer will help:
How to paginate Firestore with Android?
If you want to go further and try using Jetpack Compose, then I think that this resource will help:
How to implement pagination in Firestore using Jetpack Compose?

Would I need to have a database with a java application that frequently consumes a rest api from a site?

I'm creating a simple real-time stock market java desktop application that shows updated information of stocks. This is my first attempt to create an app that takes in real-time data. My idea to tackle this project is to:
Consume JSON rest API from Alpha Vantage.
Parse JSON into objects that are stored into a DB
Read the objects into observable lists and place into a Tableview
on JavaFX gui.
Since the stocks will be updated every few minutes, I feel like there is no need to have a database. Should I just send a GET request to the rest API every time I need to refresh the data or is there a better approach?
Thanks in advance.

I would say it depends on your needs.
Do you just want to view current stock information in a nicely formatted manner? If so, I would say you do not need to do anything with a database.
If you now, or in the future, would like to use past data to do some sort of statistical analysis, you may need to use a database. If you can use the API to find stock price at every hour for the past 24 hours, you may not need a database for some small analysis.

Step 1: Model the data
Most of our service provide data in either XML or JSON format. Unless the responses are really trivial (e.g. lists of strings), you generally don’t save time by accessing them directly.
Step 2: Fetch the data
The actual usage of the JAX-RS client API is dead simple. Still, I wrapped that part into a service class to separate that from the actual UI code. Your UI code doesn’t necessarily need to know whether it is accessing a REST service, a Web Service or a DB. The public API reveals just the reverse engineered WeatherResponse class for the actual UI code.
Step 3: Consume the data
Using POJOs with pretty much any Java technology is so basic stuff that I’ll skip covering that part thoroughly. To complete the example as a runnable application I hooked the service into a Vaadin UI. From VaadinUI you can select three predefined cities and the ForecastDisplay[ForecastDisplay] loops through the daily forecasts and displays the data in a human-readable form.

How can I load more items when scrolling down in listview?

First I want to say: This question is no duplicate!
I already read a lot of questions about loading more items when scrolling down.
This was most helpful for me.
But all questions I read aren't explaining the basic principle.
So, what I mean is:
My app gets data from json and displays it in a ListView, but it's not possible to load all items from database with one request. The app crashes…
The solution is to load only 10 items and on scrolling down 10 items again.
But what is the basic principle to do this?
I thought about these 2 different options:
Set a LIMIT in my PHP file and send for each 10 items a new request from android and set LIMIT +10.
Send one request from android and getting all data from json, but only displaying 10 items.
Code isn't necessary, because I want to know the principle of doing this.

The approach I use in my apps is:
Step # 1: I load first set of data with limit from server and then store the last record data id in a variable.
Step # 2: Implement the on scroll end listener to your listView or RecyclerView.
Step # 3: Start another request inside on scroll end listener and load new records.(Here I set again data id in a variable)
For Example:
Start the request when the activity starts and do what I explained earlier.
GetBusinesses("&isOpen=2&cat="+Prefrences.getSelectedCategoryId());
Then inside your on Scroll End Listener
GetBusinesses("&isOpen=2&cat="+Prefrences.getSelectedCategoryId()+"&limit=10&lastDataId="+BusinessItems[index].mBusinessId)
Edit To avoid duplicate api call inside GetBusinesses() check if the request was started previously, well the idea is create a boolean initially false and then in your GetBusinesses() function make it true before starting the request and once the data is loaded and request is finish make it false again.
I hope this help.

Usually you would load 10 new items from the server each time. Use a page-parameter to identify which 10 items you need and where to place them.
loading all items at once could be way too expensive: The delay could be long and the user's data-plan won't be happy either. Obviously depending on how many items there are and what contents they have.

You will have to find the trade-off. Based on your data size, sometimes it makes sense to parse and save it all in local database.
Just for 200-300 records, you don't want to make another api call after every 50 records in list. Remember with mobile app user scrolls up and down very often. You might be unnecessarily sending multiple requests to your server, which might be an overload(depending on user count).
If you go with option 2, you can make use of something like JobIntentService to silently fetch data and save locally.
This approach will also let your user interact with no internet(offline mode) scenarios.

Saving real time data in Android application to access later

I am currently streaming information in the form of a String from an Arduino to an Android application via Bluetooth. My application currently just displays the data on a series of progress bars as the values come in. I would like to store the data as it comes in to be viewed later but am unsure as to how to achieve this.
I have a BluetoothActionListener which runs when new data is available. This is called up to 20 times per second so I am worried as to whether adding a memory save to a file within this function will have an effect on its performance.
The data arrives as a string via bluetooth and I would like to be able to store the entire session's worth of data (upto 10mins) in the same file on a different line. A simple text file with a timestamp on each line is sufficient.
Would using a internal/external memory write cause my foreground visuals to slow down or even become unresponsive, as each new string arriving causes each of the progress bars to update?

At twenty times a second? Probably not. Just keep the file open rather than closing it, and flush it every few writes.

How to handle activity/fragment lifecycle when dealing with dynamical network loaded data

I'm facing the following problem:
I have a web service of a social network that allows to query a user uploaded videos. The web service signature is:
getUserVideos(String username, int quantity, int offset)
I need to create a gallery with the videos. So basically I request [quantity=10,offset=0] then [quantity=10,offset=10], using a ViewPger for the gallery.
At this point, with the gallery working (and in the position 15 for this example). We need to save the state if the user leaves the app, i.e. pressing the home button. Later, when the user return to the app after some hours, the getUserVideos service may return different data (i.e. if the given username has uploaded new videos in the meanwhile).
I see two different behaviors but find both wrong.
Save the list of videos when leaving the app and restore it when the user comes back, since there is new data in the server and the client data is old, the client offsets and the server offsets will not match, returning repeated videos or never returning others.
Requery to the server keeping the index in the gallery but updating the content (that is return to the video number 15). This will cause that the user returns to see a different video that the one he was seeing before leaving.
What are the common approaches for this kind of problems?

Cause you don't know user have video or not,you must request to server from zero.but when you get data check it with your fetched data and remove duplicate and insert new data to top of your adapter (i don't know if you have adapter or something else).