I'm trying to search tweets using search method in twitter4j. My code is as
follows,
public List<Tweet> searchTweets(Query searchQuery) {
QueryResult queryResult = twitter.search(searchQuery);
return queryResult != null ? queryResult.getTweets() : new
ArrayList<Tweet>(0);
}
How do I exclude retweets from my search query results
I was looking for how to exclude the replies on the query search, so I found this topic.
To exclude retweets, this worked well for me:
Query query = new Query("from:"+twitterAccount + " +exclude:retweets");
I would rather comment on this but I can't yet so I'll post this as an answer. You need to edit the Query by appending parameters to it. The source code for Query.java can be found in the examples folder in the twitter4j folder.
public Query(String query) {
this.query = query;
}
Check out Twitter search (atom) API - exclude retweets. The search basis is different but the concept is the same, which is appending +exclude:retweets to the end of your string query. Try it out and do let me know if it works!
Query query = new Query(yourQuery + " -filter:retweets");
source
PS. I don't know the difference betweeen +exclude:retweets and -filter:retweets; both seem to do the job similarly.
I used a different approach than above. See my code below:
List<Status> tweets = result.getTweets();
for (Status tweet : tweets )
{
if( !tweet.isRetweet() )
// processing ....
}
It works for me. However, there might be efficiency differences between these approaches.
You can use set in the configuration builder the option setIncludeRTsEnabled to false, so is going to remove the retweets.
ConfigurationBuilder builder = new ConfigurationBuilder();
builder.setIncludeRTsEnabled(false);
Twitter twitter = new TwitterFactory(builder.build()).getInstance();
Related
I'm trying to retrieve tweets for a given set of search terms using twitter4j.
I can successfully retrieve tweets for a query with only one term, or a query using the OR operator, but when I enter an AND condition in the query I get zero results. I have completed the same query on Twitter's search bar and I get hundreds of results.
It looks like this page references a similar issue, but there is no listed solution.
I would like to get results for the query "#treet java"
Here's my method that completes the search...
// queryString = "#treet" - works
// queryString = "java" - works
// queryString = "#treet OR java" - works
// queryString = "#treet java" - doesn't work
// queryString = "#treet AND java" - doesn't work
public static Treet[] loadAndSaveRemote() {
Twitter twitter = TwitterFactory.getSingleton();
Query query = new Query();
query.setQuery(queryString);
query.setCount(100);
QueryResult result = null;
try {
result = twitter.search(query);
} catch (TwitterException e) {
System.out.println("Getting tweet failed");
e.printStackTrace();
}
List<Status> tweets = result.getTweets();
List<Treet> list = new ArrayList<>();
// With queries using "term1 term2" the output is "0 tweets were retrieved" in all cases
System.out.printf("%d tweets were retrieved %n", tweets.size());
for (Status status : tweets) {
list.add( new Treet(status.getUser().getScreenName(),
status.getText(),
status.getCreatedAt()));
}
Treet[] returning = list.toArray(new Treet[list.size()]);
Treets.save(returning);
return returning;
}
I have looked at the Twitter search api and my query syntax seems correct. I have also seen examples online using a similar format.
Any thoughts would be much appreciated.
....
StringHandle rawHandle = new StringHandle(jsonString);
RawQueryByExampleDefinition querydef = queryManager.newRawQueryByExampleDefinition(rawHandle);
querydef.setCollections(collection);
StringHandle report = queryManager.validate(querydef, new StringHandle());
LOGGER.info("Query Def valididity: {}",report.toString());
StringHandle resultsHandle = new StringHandle().withFormat(Format.JSON);
queryManager.setPageLength(size);
queryManager.search(querydef, resultsHandle, startIndex);
....
I'm using the code above for MarkLogic search Query By Example, my question is how could you pass in a "sort by" criteria into the RawQueryByExampleDefinition to sort or order the resultset. For example I want the result to be sorted by emailAddress similar to the below query:
{
"$query":
{
"identifier":"user",
"$sort-by":"emailAddress"
}
}
How do I achieve the "sortby" as well as specifying desc or asc?
I think that would be done using a "combined query" from https://docs.marklogic.com/guide/java/searches#id_76144 with the sort-order option from https://docs.marklogic.com/search:search
I tried to implement pagination in google app engine (Java), but I am not able to achieve. It is working only forward pagination and reverse pagination is not able to achieved.
I tried storing the previous cursor value through HTTP request as below:
JSP file:
<a href='/myServlet?previousCursor=${previousCursor}'>Previous page</a>
<a href='/myServlet?nextCursor=${nextCursor}'>Next page</a>
Servlet file:
String previousCursor = req.getParameter("previousCursor");
String nextCursor = req.getParameter("nextCursor");
String startCursor = null;
if(previousCursor != null){
startCursor = previousCursor;
}
if(nextCursor != null){
startCursor = nextCursor;
}
int pageSize = 3;
FetchOptions fetchOptions = FetchOptions.Builder.withLimit(pageSize);
if (startCursor != null) {
fetchOptions.startCursor(Cursor.fromWebSafeString(startCursor));
}
Query q = new Query("MyQuery");
PreparedQuery pq = datastore.prepare(q);
QueryResultList<Entity> results = pq.asQueryResultList(fetchOptions);
for (Entity entity : results) {
//Get the properties from the entity
}
String endCursor = results.getCursor().toWebSafeString();
req.setAttribute("previousCursor", startCursor);
req.setAttribute("nextCursor", endCursor);
With this I am able to retain the previous cursor value, but unfortunately the previous cursor seems to be invalid.
I also tried using reverse() method, but it is of no use. It work same as forward.
So is the any way to implement proper pagination (forward and backword) in google app engine (Java)?
I found similar one that was posted in 2010. Here also the answer was to use Cursor. But as I shown above it is not working.
Pagination in Google App Engine with Java
If you are familiar with JPA you can give it a try.
Have tested it and pagination works in GAE.
I think they support JPA 1.0 as of now.
What I tried was, created an Employee entity.
Created DAO layer and persisted few employee entities.
To have a paginated fetch, did this:
Query query = em.createQuery("select e from Employee e");
query.setFirstResult(0);
query.setMaxResults(2);
List<Employee> resultList = query.getResultList();
(In this example we get first page which has 2 entities. Argument to
setFirstResult would be start index and argument to setMaxResult would be your page size)
You can easily change the arguments to query.setFirstResult and setMaxResults
and have a pagination logic around it.
Hope this helps,
Regards,
As long as I limit my query to:
SolrQuery solrQuery = new SolrQuery();
solrQuery.set("q", query); //where query is solr query string (e.g. *:*)
solrQuery.set("start", 0);
solrQuery.set("rows", 10);
everything works fine - results are returned and so on.
Things are getting worse when I try to group results by my field "Token_group" to avoid duplicates:
SolrQuery solrQuery = new SolrQuery();
solrQuery.set("q", query); //where query is solr query string (e.g. *:*)
solrQuery.set("start", 0);
solrQuery.set("rows", 10);
solrQuery.set("group", true);
solrQuery.set("group.field", "token_group");
solrQuery.set("group.ngroups", true);
solrQuery.set("group.limit", 20);
Using this results in HttpSolrServer no exceptions are being thrown, but trying to access results ends up in NPE.
My querying Solr method:
public SolrDocumentList query(SolrQuery query) throws SolrServerException {
QueryResponse response = this.solr.query(query); //(this.solr is handle to HttpSolrSelver)
SolrDocumentList list = response.getResults();
return list;
}
note that similar grouping (using the very same field) is made in our other apps (PHP) and works fine, so this is not a schema issue.
I solved my issue. In case someone needs this in future:
When you perform a group query, you should use different methods to get and parse results.
While in ungrouped queries
QueryResponse response = this.solr.query(query); //(this.solr is handle to HttpSolrSelver)
SolrDocumentList list = response.getResults();
will work, when you want to query for groups, it won't.
So, how do I make and parse query?
Below code for building query is perfectly fine:
SolrQuery solrQuery = new SolrQuery();
solrQuery.set("q", query); //where query is solr query string (e.g. *:*)
solrQuery.set("start", 0);
solrQuery.set("rows", 10);
solrQuery.set("group", true);
solrQuery.set("group.field", "token_group");
solrQuery.set("group.ngroups", true);
solrQuery.set("group.limit", 20);
where last four lines define that Solr should group results and parameters of grouping. In this case group.limit will define how many maximum results within a group you want, and rows will tell how many max results should be there.
Making grouped query looks like this:
List<GroupCommand> groupCommands = this.solr.query(query).getGroupResponse().getValues();
referring to documentation, GroupCommand contains info about grouping as well as list of results, divided by groups.
Okay, I want to get to the results. How to do it?
Well, in my example there's only one position in List<GroupCommand> groupCommands, so to get list of found groups within it:
GroupCommand groupCommand = groupCommands.get(0);
List<Group> groups = groupCommand.getValues();
This will result in list of groups. Each group contains its own SolrDocumentList. To get it:
for(Group g : groups){
SolrDocumentList groupList = g.getResult();
(...)
}
Having this, well just proceed with SolrDocumentList for each group.
I used grouping query to get list of distinct results. How to do it?
This was exacly my case. It seems easy but there's a tricky part that can catch you if you're refactoring already running code that uses getNumFound() from SolrDocumentList.
Just analyze my code:
/**
* Gets distinct resultlist from grouped query
*
* #param query
* #return results list
* #throws SolrServerException
*/
public SolrDocumentList queryGrouped(SolrQuery query) throws SolrServerException {
List<GroupCommand> groupCommands = this.solr.query(query).getGroupResponse().getValues();
GroupCommand groupCommand = groupCommands.get(0);
List<Group> groups = groupCommand.getValues();
SolrDocumentList list = new SolrDocumentList();
if(groups.size() > 0){
long totalNumFound = groupCommand.getNGroups();
int iteratorLimit = 1;
for(Group g : groups){
SolrDocumentList groupList = g.getResult();
list.add(groupList.get(0));
//I wanted to limit list to 10 records
if(iteratorLimit++ > 10){
break;
}
}
list.setNumFound(totalNumFound);
}
return list;
}
I am trying to get the value of a key from a sub-document and I can't seem to figure out how to use the BasicDBObject.get() function since the key is embedded two levels deep. Here is the structure of the document
File {
name: file_1
report: {
name: report_1,
group: RnD
}
}
Basically a file has multiple reports and I need to retrieve the names of all reports in a given file. I am able to do BasicDBObject.get("name") and I can get the value "file_1", but how do I do something like this BasicDBObject.get("report.name")? I tried that but it did not work.
You should first get the "report" object and then access its contents.You can see the sample code in the below.
DBCursor cur = coll.find();
for (DBObject doc : cur) {
String fileName = (String) doc.get("name");
System.out.println(fileName);
DBObject report = (BasicDBObject) doc.get("report");
String reportName = (String) report.get("name");
System.out.println(reportName);
}
I found a second way of doing it, on another post (didnt save the link otherwise I would have included that).
(BasicDBObject)(query.get("report")).getString("name")
where query = (BasicDBObject) cursor.next()
You can also use queries, as in the case of MongoTemplate and so on...
Query query = new Query(Criteria.where("report.name").is("some value"));
You can try this, this worked for me
BasicDBObject query = new BasicDBObject("report.name", "some value");