Firebase not getting new Data, after Android app was idle - java

Info:
I am doing an application where multiple Users need to work on one project. There are a few tasks that need to be completed.
Problem:
When a user turns off his screen, and turns it on around 5 minutes later (Or he went to the Homescreen or any other app for around 5 min ), firebase takes up to a Minute to get the new Data into the App (I have offline storing off btw). It frustrates the Users, because the Data needs to be up to date all the time so they can work with updated Tasks.
EDIT: THIS IS HAPPENING EVERYWHERE IN ALL IMPLEMENTATIONS LIKE THE EXAMPLE BELOW
Question:
Why is it like this?
How can I solve it???
EDIT: Example:
final FirebaseFirestore db = FirebaseFirestore.getInstance();
DocumentReference doc = db.collection("user").document(email).collection("dates").document(date);
doc.get().addOnCompleteListener(new OnCompleteListener<DocumentSnapshot>() {
#Override
public void onComplete(#NonNull Task<DocumentSnapshot> task) {
if (task.isSuccessful()) {
DocumentSnapshot document = task.getResult();
if (document.exists()) {
HashMap CodeMap = (HashMap) document.getData();
Set keySet = CodeMap.keySet();
Object[] key = keySet.toArray();
for (int i = 0; i < keySet.size(); i++) {
if (!key[i].equals("clockIn") && !key[i].equals("clockOut") && !key[i].equals("completeTime") && !key[i].equals("clockPauseFrom") && !key[i].equals("clockPauseTo")) {
userWorkList.add(new User_Work_View_Row(key[i].toString(), email, date, i));
System.out.println(key[i] + " FOUND");
}
}
}else{
Log.d(TAG, "get failed with ", task.getException());
getActivity().finish();
}
}
}
});
SOLUTION:* (Kind of)
I have Implemented the solution of the answer below and did some Testing with the Battery options.
I found out that disabling Battery Optimizations and Background restrictions(if its even there) works! The Data is coming right away. I don't think its a good solution but it works for my App.

What you're observing is likely a result of an exponential backoff in the frequency of connection retries performed by the Firestore SDK. This backoff saves much battery compared to always retrying immediately in a tight loop.
If you want to force a retry, then you could try manually calling disableNetwork() followed by enableNetwork() to get it to retry immediately.

Related

Vert.x performance drop when starting with -cluster option

I'm wondering if any one experienced the same problem.
We have a Vert.x application and in the end it's purpose is to insert 600 million rows into a Cassandra cluster. We are testing the speed of Vert.x in combination with Cassandra by doing tests in smaller amounts.
If we run the fat jar (build with Shade plugin) without the -cluster option, we are able to insert 10 million records in about a minute. When we add the -cluster option (eventually we will run the Vert.x application in cluster) it takes about 5 minutes for 10 million records to insert.
Does anyone know why?
We know that the Hazelcast config will create some overhead, but never thought it would be 5 times slower. This implies we will need 5 EC2 instances in cluster to get the same result when using 1 EC2 without the cluster option.
As mentioned, everything runs on EC2 instances:
2 Cassandra servers on t2.small
1 Vert.x server on t2.2xlarge
You are actually running into corner cases of the Vert.x Hazelcast Cluster manager.
First of all you are using a worker Verticle to send your messages (30000001). Under the hood Hazelcast is blocking and thus when you send a message from a worker the version 3.3.3 does not take that in account. Recently we added this fix https://github.com/vert-x3/issues/issues/75 (not present in 3.4.0.Beta1 but present in 3.4.0-SNAPSHOTS) that will improve this case.
Second when you send all your messages at the same time, it runs into another corner case that prevents the Hazelcast cluster manager to use a cache of the cluster topology. This topology cache is usually updated after the first message has been sent and sending all the messages in one shot prevents the usage of the ache (short explanation HazelcastAsyncMultiMap#getInProgressCount will be > 0 and prevents the cache to be used), hence paying the penalty of an expensive lookup (hence the cache).
If I use Bertjan's reproducer with 3.4.0-SNAPSHOT + Hazelcast and the following change: send message to destination, wait for reply. Upon reply send all messages then I get a lot of improvements.
Without clustering : 5852 ms
With clustering with HZ 3.3.3 :16745 ms
With clustering with HZ 3.4.0-SNAPSHOT + initial message : 8609 ms
I believe also you should not use a worker verticle to send that many messages and instead send them using an event loop verticle via batches. Perhaps you should explain your use case and we can think about the best way to solve it.
When you're you enable clustering (of any kind) to an application you are making your application more resilient to failures but you're also adding a performance penalty.
For example your current flow (without clustering) is something like:
client ->
vert.x app ->
in memory same process eventbus (negletible) ->
handler -> cassandra
<- vert.x app
<- client
Once you enable clustering:
client ->
vert.x app ->
serialize request ->
network request cluster member ->
deserialize request ->
handler -> cassandra
<- serialize response
<- network reply
<- deserialize response
<- vert.x app
<- client
As you can see there are many encode decode operations required plus several network calls and this all gets added to your total request time.
In order to achive best performance you need to take advantage of locality the closer you are of your data store usually the fastest.
Just to add the code of the project. I guess that would help.
Sender verticle:
public class ProviderVerticle extends AbstractVerticle {
#Override
public void start() throws Exception {
IntStream.range(1, 30000001).parallel().forEach(i -> {
vertx.eventBus().send("clustertest1", Json.encode(new TestCluster1(i, "abc", LocalDateTime.now())));
});
}
#Override
public void stop() throws Exception {
super.stop();
}
}
And the inserter verticle
public class ReceiverVerticle extends AbstractVerticle {
private int messagesReceived = 1;
private Session cassandraSession;
#Override
public void start() throws Exception {
PoolingOptions poolingOptions = new PoolingOptions()
.setCoreConnectionsPerHost(HostDistance.LOCAL, 2)
.setMaxConnectionsPerHost(HostDistance.LOCAL, 3)
.setCoreConnectionsPerHost(HostDistance.REMOTE, 1)
.setMaxConnectionsPerHost(HostDistance.REMOTE, 3)
.setMaxRequestsPerConnection(HostDistance.LOCAL, 20)
.setMaxQueueSize(32768)
.setMaxRequestsPerConnection(HostDistance.REMOTE, 20);
Cluster cluster = Cluster.builder()
.withPoolingOptions(poolingOptions)
.addContactPoints(ClusterSetup.SEEDS)
.build();
System.out.println("Connecting session");
cassandraSession = cluster.connect("kiespees");
System.out.println("Session connected:\n\tcluster [" + cassandraSession.getCluster().getClusterName() + "]");
System.out.println("Connected hosts: ");
cassandraSession.getState().getConnectedHosts().forEach(host -> System.out.println(host.getAddress()));
PreparedStatement prepared = cassandraSession.prepare(
"insert into clustertest1 (id, value, created) " +
"values (:id, :value, :created)");
PreparedStatement preparedTimer = cassandraSession.prepare(
"insert into timer (name, created_on, amount) " +
"values (:name, :createdOn, :amount)");
BoundStatement timerStart = preparedTimer.bind()
.setString("name", "clusterteststart")
.setInt("amount", 0)
.setTimestamp("createdOn", new Timestamp(new Date().getTime()));
cassandraSession.executeAsync(timerStart);
EventBus bus = vertx.eventBus();
System.out.println("Bus info: " + bus.toString());
MessageConsumer<String> cons = bus.consumer("clustertest1");
System.out.println("Consumer info: " + cons.address());
System.out.println("Waiting for messages");
cons.handler(message -> {
TestCluster1 tc = Json.decodeValue(message.body(), TestCluster1.class);
if (messagesReceived % 100000 == 0)
System.out.println("Message received: " + messagesReceived);
BoundStatement boundRecord = prepared.bind()
.setInt("id", tc.getId())
.setString("value", tc.getValue())
.setTimestamp("created", new Timestamp(new Date().getTime()));
cassandraSession.executeAsync(boundRecord);
if (messagesReceived % 100000 == 0) {
BoundStatement timerStop = preparedTimer.bind()
.setString("name", "clusterteststop")
.setInt("amount", messagesReceived)
.setTimestamp("createdOn", new Timestamp(new Date().getTime()));
cassandraSession.executeAsync(timerStop);
}
messagesReceived++;
//message.reply("OK");
});
}
#Override
public void stop() throws Exception {
super.stop();
cassandraSession.close();
}
}

Google fit history API won't retrieve dataset?

I am currently trying to display today's current steps through my application. I have the following code below which Authenticates most of the time alright. I have added the SHA1 certificate to the developers console as well.
I keep getting the error "There was a problem inserting the dataset." in my log through with nothing else after that showing up and i'm a bit confused as to why?
Also this seems to be temperamental, sometimes it works and shows 7 data sets with random numbers (not my step count) so my second question is how do I get it to display just today's?
Authentication
private void buildFitnessClient() {
if (mClient == null) {
mClient = new GoogleApiClient.Builder(this)
.addApi(Fitness.HISTORY_API)
.addScope(new Scope(Scopes.FITNESS_ACTIVITY_READ))
.addConnectionCallbacks(
new GoogleApiClient.ConnectionCallbacks() {
#Override
public void onConnected(Bundle bundle) {
Log.i(TAG, "Connected!!!");
// Now you can make calls to the Fitness APIs.
new InsertAndVerifyDataTask().execute();
}
#Override
public void onConnectionSuspended(int i) {
// If your connection to the sensor gets lost at some point,
// you'll be able to determine the reason and react to it here.
if (i == GoogleApiClient.ConnectionCallbacks.CAUSE_NETWORK_LOST) {
Log.i(TAG, "Connection lost. Cause: Network Lost.");
Toast.makeText(getBaseContext(), "Connection lost. Cause: Network Lost.", Toast.LENGTH_LONG).show();
} else if (i
== GoogleApiClient.ConnectionCallbacks.CAUSE_SERVICE_DISCONNECTED) {
Log.i(TAG,
"Connection lost. Reason: Service Disconnected");
}
}
}
)
.enableAutoManage(this, 0, new GoogleApiClient.OnConnectionFailedListener() {
#Override
public void onConnectionFailed(ConnectionResult result) {
Log.i(TAG, "Google Play services connection failed. Cause: " +
result.toString());
Toast.makeText(getBaseContext(), "Google Play services connection failed. Cause: " +
result.toString(), Toast.LENGTH_LONG).show();
}
})
.build();
}
}
Where I think the issue resides.
private class InsertAndVerifyDataTask extends AsyncTask<Void, Void, Void> {
protected Void doInBackground(Void... params) {
//First, create a new dataset and insertion request.
DataSet dataSet = insertFitnessData();
// [START insert_dataset]
// Then, invoke the History API to insert the data and await the result, which is
// possible here because of the {#link AsyncTask}. Always include a timeout when calling
// await() to prevent hanging that can occur from the service being shutdown because
// of low memory or other conditions.
Log.i(TAG, "Inserting the dataset in the History API");
com.google.android.gms.common.api.Status insertStatus =
Fitness.HistoryApi.insertData(mClient, dataSet)
.await(1, TimeUnit.MINUTES);
// Before querying the data, check to see if the insertion succeeded.
if (!insertStatus.isSuccess()) {
Log.i(TAG, "There was a problem inserting the dataset.");
return null;
}
// At this point, the data has been inserted and can be read.
Log.i(TAG, "Data insert was successful!");
// [END insert_dataset]
// Begin by creating the query.
DataReadRequest readRequest = queryFitnessData();
// [START read_dataset]
// Invoke the History API to fetch the data with the query and await the result of
// the read request.
DataReadResult dataReadResult =
Fitness.HistoryApi.readData(mClient, readRequest).await(1, TimeUnit.MINUTES);
// [END read_dataset]
// For the sake of the sample, we'll print the data so we can see what we just added.
// In general, logging fitness information should be avoided for privacy reasons.
printData(dataReadResult);
return null;
}
}
private DataSet insertFitnessData() {
Log.i(TAG, "Creating a new data insert request");
// [START build_insert_data_request]
// Set a start and end time for our data, using a start time of 1 hour before this moment.
Calendar cal = Calendar.getInstance();
Date now = new Date();
cal.setTime(now);
long endTime = cal.getTimeInMillis();
cal.add(Calendar.HOUR_OF_DAY, -1);
long startTime = cal.getTimeInMillis();
// Create a data source
DataSource dataSource = new DataSource.Builder()
.setAppPackageName(this)
.setDataType(DataType.TYPE_STEP_COUNT_DELTA)
.setName(TAG + " - step count")
.setType(DataSource.TYPE_RAW)
.build();
// Create a data set
int stepCountDelta = 1000;
DataSet dataSet = DataSet.create(dataSource);
// For each data point, specify a start time, end time, and the data value -- in this case,
// the number of new steps.
DataPoint dataPoint = dataSet.createDataPoint()
.setTimeInterval(startTime, endTime, TimeUnit.MILLISECONDS);
dataPoint.getValue(Field.FIELD_STEPS).setInt(stepCountDelta);
dataSet.add(dataPoint);
// [END build_insert_data_request]
return dataSet;
}
Got it working with the following code gets todays steps:
Calendar cal = Calendar.getInstance();
Date now = new Date();
cal.setTime(now);
long endTime = cal.getTimeInMillis();
cal.set(Calendar.HOUR_OF_DAY, 0);
cal.set(Calendar.MINUTE, 00);
long startTime = cal.getTimeInMillis();
int steps = 0;
DataSource ESTIMATED_STEP_DELTAS = new DataSource.Builder()
.setDataType(DataType.TYPE_STEP_COUNT_DELTA)
.setType(DataSource.TYPE_DERIVED)
.setStreamName("estimated_steps")
.setAppPackageName("com.google.android.gms").build();
// fill result with just the steps from the start and end time of the present day
PendingResult<DailyTotalResult> result = Fitness.HistoryApi.readDailyTotal(mClient, DataType.AGGREGATE_STEP_COUNT_DELTA);
DailyTotalResult totalResult = result.await(60, TimeUnit.SECONDS);
if (totalResult.getStatus().isSuccess()) {
DataSet totalSet = totalResult.getTotal();
steps = totalSet.isEmpty() ? -1 : totalSet.getDataPoints().get(0).getValue(Field.FIELD_STEPS).asInt();
}
s = String.valueOf(steps);
DataReadRequest readRequest = queryFitnessData();
DataReadResult dataReadResult =
Fitness.HistoryApi.readData(mClient, readRequest).await(1, TimeUnit.MINUTES);
feedItemList.clear();
printData(dataReadResult);
return null;
Sometime, we also want to get data step in a specific time range, and here it's a reason in that case.
There're some probable causes:
1) Have you subscribed this type of data yet?
2) Your app doesn't connect properly with google service. Have you created OAuth Client ID from Google develop console? This's compulsory in instruction of google to connect to its GG Fit service (Please notice that, if you clone another app, on the same computer or not, you need to re-create another OAuth Client ID and one more thing you need 2 separated account, one to login Google develop console to create OAuth Client ID and one to sign in after starting your app, and it will ask you to sign in to accept its permission,... not sure why it's is, but it would work)
Note: Btw you can make a search about Google setting in your device (Setting --> Google), Here you can find which app connects to google service (including GG Fit service). I recommend you disconnect all and delete OAuth Client ID, your app, then re-create all of them!
Mttdat.

ParseQuery fails to get object that exists in backend

reposting this question since I didn't get any responses the first time around. Parse doesn't appear to have any simple means of contacting them about this (which is frustrating), so I really hope someone here can help.
I am currently using Parse to create a messaging app. I have two fundamental ParseObjects in addition to the standard ParseUser, a Chatroom and a Message. A Chatroom contains pointers to the two users in the Chatroom. A Message contains the content of the Message, a pointer to the user who posted it, and a pointer to the Chatroom.
First, I create a list of all the Chatrooms that the current user is in. Then, I'm trying to create a second list of the most recent message in all of these Chatrooms (I have made it impossible to make a Chatroom without sending at least one Message first).
My code looks like this:
TextView mostRecent = (TextView) row.findViewById(R.id.mostRecent);
Date dt1 = null;
ParseQuery lastMessage = ParseQuery.getQuery("Message");
ParseObject chatroom = (ParseObject) roomList.get(position);
Log.w("Chatroom ObjectID...", room.getObjectId());
try {
lastMessage.whereEqualTo("chatroom", chatroom.fetchIfNeeded());
List<ParseObject> allMessages = lastMessage.find();
Log.w("# of Messages...", "Size of the list: " + allMessages.size() + ", Count of query: " + allMessages.count());
if (allMessages.size() > 0) {
mostRecent.setText((String) theList.get(theList.size() - 1).get("content"));
dt1 = allMessages.get(allMessages.size() - 1).getCreatedAt();
}
} catch (ParseException err) {
err.printStackTrace();
Log.w("PARSE ERROR", err.getMessage());
}
This code works for every Chatroom where the current user is the one who created the Chatroom and the Message in that Chatroom. However, whenever I have a different user attempt to start a Chatroom and send a Message to the original user, the code fails.
To be clear, the second user successfully creates both a Chatroom and a Message. I've verified that the Chatroom gets successfully added to the original user's list of Chatrooms, and that the Message contains a pointer to the correct Chatroom. However, for whatever reason, the Logs reveal a list size of 0 and a count of 0. I've even tried querying for the second user's exact message on the original user's account, and it claims the thing doesn't exist.
Any ideas? Could this have something to do with ACL? Thanks in advance!
NOTE: I've confirmed that both users are correctly included in the Message's ACL. What gives? Why isn't this working?
You need to wait for the query to fetch the data.
lastMessage.findInBackground(new FindCallback<ParseObject>() {
public void done(List<ParseObject> allMessages, ParseException e) {
if (e == null) {
Log.w("# of Messages...", "Size of the list: " + allMessages.size() + ", Count of query: " + allMessages.count());
} else {
Log.w("Exception thrown", e.getMessage());
}
}
}

How to find steps with Google Fit API for Android?

I have done a couple of days of research regarding this. All I need is a Simple TextView area in my application to display today's steps.
I have managed to get the Authentication working with the code below. It pops up asks for permission and think I have the right one selected.
But I can not figure out how to simple gain the Step Count information. I hope this is only a couple of lines of code. Any help would be appreciated. Thanks
EDIT 1: I just need to get the Step count number. I can figure out how to display it later. I also have Toasts in just to help me figure out what is going on.
private void buildFitnessClient() {
if (mClient == null) {
mClient = new GoogleApiClient.Builder(this)
.addApi(Fitness.SENSORS_API)
.addScope(new Scope(Scopes.FITNESS_ACTIVITY_READ))
.addConnectionCallbacks(
new GoogleApiClient.ConnectionCallbacks() {
#Override
public void onConnected(Bundle bundle) {
Log.i(TAG, "Connected!!!");
// Now you can make calls to the Fitness APIs.
Toast.makeText(getBaseContext(), "Connected!", Toast.LENGTH_LONG).show();
}
#Override
public void onConnectionSuspended(int i) {
// If your connection to the sensor gets lost at some point,
// you'll be able to determine the reason and react to it here.
if (i == GoogleApiClient.ConnectionCallbacks.CAUSE_NETWORK_LOST) {
Log.i(TAG, "Connection lost. Cause: Network Lost.");
Toast.makeText(getBaseContext(), "Connection lost. Cause: Network Lost.", Toast.LENGTH_LONG).show();
} else if (i
== GoogleApiClient.ConnectionCallbacks.CAUSE_SERVICE_DISCONNECTED) {
Log.i(TAG,
"Connection lost. Reason: Service Disconnected");
}
}
}
)
.enableAutoManage(this, 0, new GoogleApiClient.OnConnectionFailedListener() {
#Override
public void onConnectionFailed(ConnectionResult result) {
Log.i(TAG, "Google Play services connection failed. Cause: " +
result.toString());
Toast.makeText(getBaseContext(), "Google Play services connection failed. Cause: " +
result.toString(), Toast.LENGTH_LONG).show();
}
})
.build();
}
}
GoogleApiClient is deprecated according to the new updates. As well as the HistoryApi is also deprecated. So first have to use the GoogleSignInAccount instead of GoogleApiClient and also use HistoryClient of Google Fit instead of HistoryApi.
Setting Fitness Options
val fitnessOptions: GoogleSignInOptionsExtension = FitnessOptions.builder()
.addDataType(TYPE_STEP_COUNT_DELTA, FitnessOptions.ACCESS_READ).build()
Setting Google SignIn Option
val googleSignInOptions =
GoogleSignInOptions.Builder(GoogleSignInOptions.DEFAULT_SIGN_IN)
.addExtension(fitnessOptions).requestEmail().build()
val googleSignInClient = GoogleSignIn.getClient(requireActivity(), googleSignInOptions)
val signIntent = googleSignInClient.signInIntent
startActivityForResult(signIntent, 0)
Getting Account
override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
super.onActivityResult(requestCode, resultCode, data)
if (requestCode == 0 && resultCode == Activity.RESULT_OK) {
val task = GoogleSignIn.getSignedInAccountFromIntent(data)
val account = task.getResult(ApiException::class.java)
if (account != null) {
getGoogleFitData(account)
}
}
}
Now for getting step data
val response: Task<DataReadResponse> =
Fitness.getHistoryClient(mContext, mSignInAccount)
.readData(
DataReadRequest.Builder()
.read(TYPE_STEP_COUNT_DELTA)
.setTimeRange(
startTime,
endTime,
TimeUnit.MILLISECONDS
)
.build()
)
val readDataResult: DataReadResponse? = Tasks.await(response)
val dataSet: DataSet = readDataResult!!.getDataSet(TYPE_STEP_COUNT_DELTA)
var total = 0
if (!dataSet.isEmpty) {
val dataPoints = dataSet.dataPoints
if (dataPoints.size > 0) {
for (i in 0 until dataPoints.size) {
total += dataSet.dataPoints[i].getValue(Field.FIELD_STEPS).asInt()
}
Log.e(TAG, "Total Steps : $total")
}
}
Here startTime and endTime are timemills of any date and time.
Thank You.
Check out this official documentation from Google on how to read the data from Fit:
// Setting a start and end date using a range of 1 week before this moment.
Calendar cal = Calendar.getInstance();
Date now = new Date();
cal.setTime(now);
long endTime = cal.getTimeInMillis();
cal.add(Calendar.WEEK_OF_YEAR, -1);
long startTime = cal.getTimeInMillis();
java.text.DateFormat dateFormat = getDateInstance();
Log.i(TAG, "Range Start: " + dateFormat.format(startTime));
Log.i(TAG, "Range End: " + dateFormat.format(endTime));
DataReadRequest readRequest = new DataReadRequest.Builder()
// The data request can specify multiple data types to return, effectively
// combining multiple data queries into one call.
// In this example, it's very unlikely that the request is for several hundred
// datapoints each consisting of a few steps and a timestamp. The more likely
// scenario is wanting to see how many steps were walked per day, for 7 days.
.aggregate(DataType.TYPE_STEP_COUNT_DELTA, DataType.AGGREGATE_STEP_COUNT_DELTA)
// Analogous to a "Group By" in SQL, defines how data should be aggregated.
// bucketByTime allows for a time span, whereas bucketBySession would allow
// bucketing by "sessions", which would need to be defined in code.
.bucketByTime(1, TimeUnit.DAYS)
.setTimeRange(startTime, endTime, TimeUnit.MILLISECONDS)
.build();
History API Sample app on GitHub:
Check the sample project on GitHub here.
Direct link to MainActivity.java (in the above sample project) containing the required code: Link

Max 20 Friends Information on Twitter4J

Like you see in this code, I want to get all the information about friends in twitter, people I follow.
But doing this :
PagableResponseList<User> users = twitter.getFriendsList(USER_ID, CURSOR);
... only gives me the first 20 recent friends... What can I do?
Complete code about it :
PagableResponseList<User> users = twitter.getFriendsList(USER_ID, CURSOR);
User user = null;
max = users.size();
System.out.println("Following: "+max);
for (int i = 0 ; i < users.size() ; i++){
user = users.get(i);
System.out.print("\nID: "+user.getId()+" / User: "+user.getName()+" /");
System.out.print("\nFollowers: "+user.getFollowersCount()+"\n");
tid.add(Long.parseLong(String.valueOf(user.getId())));
tusername.add(user.getName());
tfollowers.add(Long.parseLong(String.valueOf(user.getFollowersCount())));
tname.add(user.getScreenName());
}
Thanks..
you can try this code to get the list of people you follow.
long cursor = -1;
PagableResponseList<User> users;
while ((cursor = followers.getNextCursor()) != 0);
{
users = twitter.getFriendsList(userId, cursor);
}
I've taken a peek at the documentation at Twitter4J and Twitter themselves and it's all about that cursor.
To prevent you're getting loaded with a whole bunch of friends at once, Twitter only returns the first 20 results. It doesn't return just the first 20 results, but it also returns a cursor. That cursor is just a random number that's managed by Twitter. When you make a call again and pass this cursor, the next 20 entries (friends) will be returned, again with a cursor that's different now. You can repeat this until the cursor returned is zero. That means there are no more entries available.
In case you want to know more, check these two links: Twitter DEV and Twitter4J documentation.
Concerning your Java, you just need to find a way to get the current cursor, and pass that cursor to your method again, making the app load the next 20 entries. According to this piece of information, that should do the trick.
List<User> allUsers = new ArrayList<User>();
PagableResponseList<User> users;
long cursor = -1;
while (cursor != 0) {
users = twitter.getFriendsList(USER_ID, cursor);
cursor = users.getNextCursor();
allUsers.add(users);
}
You should be able to request up to 200 results at a time:
final PagableResponseList<User> users = twitter.getFriendsList(USER_ID, cursor, 200);
cursor = users.getNextCursor();
If you need to start from where you left off between invocations of your program then you need to store the value of cursor somewhere.
Improvements to Sander's answer!
You can set a count value to the getFriendsList method as in Jonathan's Answer. The maximum value allowed for count is 200. The loop construct will help to collect more than 200 friends now. 200 friends per page or per iteration!
Yet, there are rate limits for any request you make. The getFriendsList method will use this api endpoint: GET friends/list which has a rate limit of 15 hits per 15 minutes. Each hit can fetch a maximum of 200 friends which equates to a total of 3000 friends (15 x 200 = 3000) per 15 minutes. So, there will be no problem if you have only 3000 friends. If you have more than 3000 friends, an exception will be thrown. You can use the RateLimitStatus class to avoid that exception. The following code is an example implementation to achieve this.
Method 1: fetchFriends(long userId)
public List<User> fetchFriends(long userId) {
List<User> friends = new ArrayList<User>();
PagableResponseList<User> page;
long cursor = -1;
try {
while (cursor != 0) {
page = twitter.getFriendsList(userId, cursor, 200);
friends.addAll(page);
System.out.println("Total number of friends fetched so far: " + friends.size());
cursor = page.getNextCursor();
this.handleRateLimit(page.getRateLimitStatus());
}
} catch (TwitterException e) {
e.printStackTrace();
}
return friends;
}
Method 2: handleRateLimit(RateLimitStatus rls)
private void handleRateLimit(RateLimitStatus rls) {
int remaining = rls.getRemaining();
System.out.println("Rate Limit Remaining: " + remaining);
if (remaining == 0) {
int resetTime = rls.getSecondsUntilReset() + 5;
int sleep = (resetTime * 1000);
try {
if(sleep > 0) {
System.out.println("Rate Limit Exceeded. Sleep for " + (sleep / 1000) + " seconds..");
Thread.sleep(sleep);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
By doing so, your program will sleep for some time period based on the rate limiting threshold. It will continue to run from where it left after the sleep. This way we can avoid our program stopping in the midway of collecting friends counting more than 3000.
I have the solution to my post... thanks to Sander, give me some ideas...
The thing was change the for to while ((CURSOR = ids.getNextCursor()) != 0);.
And... user = twitter.showUser(id);
Playing with showUser makes it possible to get, with a slowly time, all the info about all my friends...
That's all. Don't use user.get(i);

Categories