I'm working on writing a simple program to fetch tweets for an information retrieval class (later I will be learning to index and search through them). For now, I just want to gather data. I'm using the twitter4j API and am having trouble understanding the Paging class and how it works. Below is a snippet of code:
TwitterFactory tf = new TwitterFactory(cb.build());
Twitter twitter = tf.getInstance();
Paging paging = new Paging(2,40);
try{
List<Status> statuses = twitter.getUserTimeline("google", paging);
System.out.println(paging);
for(Status status : statuses)
{
System.out.println(status.getText());
}
System.out.println("\n\n\n");
paging.setPage(2);
statuses = twitter.getUserTimeline("google",paging);
for(Status status : statuses)
{
System.out.println(status.getText());
}
}
catch(TwitterException e){
e.printStackTrace();
}
My hope was that the second call to twitter.getUserTimeline would return the next 40 tweets from Google as this example seems to suggest, but upon inspection, both return the same 40 tweets. Can someone then explain what the Paging class actually does?
Your first call should request the first page Paging paging = new Paging(1, 40);, not the second one.
Related
I want to get tweets from certain user timelines using java library twitter4j, currently I have source code which can get ~ 3200 tweets from user time line but I can't get full tweet. I have searched in various sources on the internet but I can't find a solution to my problem. anyone can help me or can anyone provide an alternative to get a full tweet from the user timeline with java programming?
my source code :
public static void main(String[] args) throws SQLException {
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setDebugEnabled(true)
.setOAuthConsumerKey("aaa")
.setOAuthConsumerSecret("aaa")
.setOAuthAccessToken("aaa")
.setOAuthAccessTokenSecret("aaa");
TwitterFactory tf = new TwitterFactory(cb.build());
Twitter twitter = tf.getInstance();
int pageno = 1;
String user = "indtravel";
List statuses = new ArrayList();
while (true) {
try {
int size = statuses.size();
Paging page = new Paging(pageno++, 100);
statuses.addAll(twitter.getUserTimeline(user, page));
System.out.println("***********************************************");
System.out.println("Gathered " + twitter.getUserTimeline(user, page).size() + " tweets");
//get status dan user
for (Status status: twitter.getUserTimeline(user, page)) {
//System.out.println("*********Place Tweets :**************\npalce country :"+status.getPlace().getCountry()+"\nplace full name :"+status.getPlace().getFullName()+"\nplace name :"+status.getPlace().getName()+"\nplace id :"+status.getPlace().getId()+"\nplace tipe :"+status.getPlace().getPlaceType()+"\nplace addres :"+status.getPlace().getStreetAddress());
System.out.println("["+(no++)+".] "+"Status id : "+status.getId());
System.out.println("id user : "+status.getUser().getId());
System.out.println("Length status : "+status.getText().length());
System.out.println("#" + status.getUser().getScreenName() +" . "+status.getCreatedAt()+ " : "+status.getUser().getName()+"--------"+status.getText());
System.out.println("url :"+status.getUser().getURL());
System.out.println("Lang :"+status.getLang());
}
if (statuses.size() == size)
break;
}catch(TwitterException e) {
e.printStackTrace();
}
}
System.out.println("Total: "+statuses.size());
}
Update :
After the answer given by #AndyPiper
the my problem is every tweet that I get will be truncated or not complete. a tweets that I get will be truncated if the length of tweet more than 140 characters. I found the reference tweet_mode=extended, but I do not know how to use it. if you know something please tell me.
Your Configuration should be like this:
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setDebugEnabled(true)
.setOAuthConsumerKey("aaa")
.setOAuthConsumerSecret("aaa")
.setOAuthAccessToken("aaa")
.setOAuthAccessTokenSecret("aaa")
.setTweetModeExtended(true);
It is explained well here
The Twitter API limits the timeline history to 3200 Tweets. To get more than that you would need to use the (commercial) premium or enterprise APIs to search for Tweets by a specific user.
if you are streaming tweets
fist: you have to add .setTweetModeExtended(true); into your configurationbuilder
second(here is the code)
TwitterStream twitterStream = new TwitterStreamFactory(cb.build()).getInstance();
StatusListener listener = new StatusListener(){
public void onStatus(Status status) {
System.out.println("-------------------------------");
if(status.isRetweet()){
System.out.println(status.getRetweetedStatus().getText());
}
else{
System.out.println(status.getText());
}`
its totally works for me.
take care yourself :)
If you are implementing twitter api using twitter4j.properties file and getting truncated timeline text value,simply add the below property in it at the end.
tweetModeExtended=TRUE
I'm trying to retrieve tweets for a given set of search terms using twitter4j.
I can successfully retrieve tweets for a query with only one term, or a query using the OR operator, but when I enter an AND condition in the query I get zero results. I have completed the same query on Twitter's search bar and I get hundreds of results.
It looks like this page references a similar issue, but there is no listed solution.
I would like to get results for the query "#treet java"
Here's my method that completes the search...
// queryString = "#treet" - works
// queryString = "java" - works
// queryString = "#treet OR java" - works
// queryString = "#treet java" - doesn't work
// queryString = "#treet AND java" - doesn't work
public static Treet[] loadAndSaveRemote() {
Twitter twitter = TwitterFactory.getSingleton();
Query query = new Query();
query.setQuery(queryString);
query.setCount(100);
QueryResult result = null;
try {
result = twitter.search(query);
} catch (TwitterException e) {
System.out.println("Getting tweet failed");
e.printStackTrace();
}
List<Status> tweets = result.getTweets();
List<Treet> list = new ArrayList<>();
// With queries using "term1 term2" the output is "0 tweets were retrieved" in all cases
System.out.printf("%d tweets were retrieved %n", tweets.size());
for (Status status : tweets) {
list.add( new Treet(status.getUser().getScreenName(),
status.getText(),
status.getCreatedAt()));
}
Treet[] returning = list.toArray(new Treet[list.size()]);
Treets.save(returning);
return returning;
}
I have looked at the Twitter search api and my query syntax seems correct. I have also seen examples online using a similar format.
Any thoughts would be much appreciated.
I'm using twitter4j version 2.2.5. setPage() doesn't seem to work if I use it with setSince() and setUntil(). See the following code:
Twitter twitter = new TwitterFactory().getInstance();
twitter.setOAuthConsumer(CONSUMER_KEY, CONSUMER_SECRET);
AccessToken accessToken = new AccessToken(ACCESS_TOKEN, ACCESS_TOKEN_SECRET);
twitter.setOAuthAccessToken(accessToken);
int page = 1;
while(true) {
Query query = new Query("from:someUser");
query.setSince("2012-01-01");
query.setUntil("2012-07-05");
query.setRpp(100);
query.setPage(page++);
QueryResult qr = twitter.search(query);
List<twitter4j.Tweet> qrTweets = qr.getTweets();
if(qr.getTweets().size() == 0) break;
for(twitter4j.Tweet t : qrTweets) {
System.out.println(t.getId() + " " + t.getText());
}
}
The code inside the loop is only executed once if I use setSince() and setUntil() methods. But without them, setPage() seems to work and I get more tweet results.
Any ideas why this is happening?
Your code appears to be working for me. It only returns tweets from the past nine days, but that's expected (approximately) according to your comment.
You mentioned in the question that the loop only ran once with rpp=100, and you said in a comment that all 80 of the user's tweets were returned when rpp=40. I think that would indicate that the code is working as expected. If the user only has 80 tweets, the loop only should run once when rpp=100, and it should execute twice and display all their tweets when rpp=40. Try displaying the page number as soon as you enter the while loop so you can see how many times the loop runs.
Hello i want to display a user timeline in a list with twitter4j, because im new to java i just don't know how to do that. I searched the web but couldn't find anything usefull. So my question is: does someone know how to display a user timeline in a list in android?
twitter4j-core-2.2.6
Twitter tw = new TwitterFactory().getInstance();
// some userlist
UserList userList = tw.getUserLists("screen name", -1).get(0);
System.out.println(userList.getName());
int listId = userList.getId();
ResponseList<Status> userListStatuses = tw.getUserListStatuses(listId, new Paging(1));
for (Status status : userListStatuses) {
System.out.println(status.getText());
}
As per the twitter4j Code Examples
Twitter twitter = new TwitterFactory().getInstance();
List<Status> statuses = twitter.getFriendsTimeline();
System.out.println("Showing friends timeline.");
for (Status status : statuses) {
System.out.println(status.getUser().getName() + ":" +
status.getText());
}
Please check official document and code example for twitter4j
I'm trying to search tweets using search method in twitter4j. My code is as
follows,
public List<Tweet> searchTweets(Query searchQuery) {
QueryResult queryResult = twitter.search(searchQuery);
return queryResult != null ? queryResult.getTweets() : new
ArrayList<Tweet>(0);
}
How do I exclude retweets from my search query results
I was looking for how to exclude the replies on the query search, so I found this topic.
To exclude retweets, this worked well for me:
Query query = new Query("from:"+twitterAccount + " +exclude:retweets");
I would rather comment on this but I can't yet so I'll post this as an answer. You need to edit the Query by appending parameters to it. The source code for Query.java can be found in the examples folder in the twitter4j folder.
public Query(String query) {
this.query = query;
}
Check out Twitter search (atom) API - exclude retweets. The search basis is different but the concept is the same, which is appending +exclude:retweets to the end of your string query. Try it out and do let me know if it works!
Query query = new Query(yourQuery + " -filter:retweets");
source
PS. I don't know the difference betweeen +exclude:retweets and -filter:retweets; both seem to do the job similarly.
I used a different approach than above. See my code below:
List<Status> tweets = result.getTweets();
for (Status tweet : tweets )
{
if( !tweet.isRetweet() )
// processing ....
}
It works for me. However, there might be efficiency differences between these approaches.
You can use set in the configuration builder the option setIncludeRTsEnabled to false, so is going to remove the retweets.
ConfigurationBuilder builder = new ConfigurationBuilder();
builder.setIncludeRTsEnabled(false);
Twitter twitter = new TwitterFactory(builder.build()).getInstance();