How do I parse a Reader into a Calendar? - java

I've tried to parse a reader from an InputStreamReader return into a Calendar with custom events but I got an URISyntaxException due to my icalendar file content. But I can't modify it because my purpose is to get an iCalendar data only (online file.ics) and parse them into a Calendar with iCal4j library: https://github.com/ical4j/ical4j.
So my question is how can I solve or bypass this problem?
My code :
public class AccesHTTP extends AsyncTask<String, String, String> {
private Controle controle;
private String urlServ = null;
public AsyncResponse delegate = null;
public AccesHTTP(String servaddr) {
controle = Controle.getInstance(null);
urlServ = servaddr;
}
#Override
protected String doInBackground(String... strings) {
// Create a TimeZone
System.setProperty("net.fortuna.ical4j.timezone.cache.impl", MapTimeZoneCache.class.getName());
URL url = null;
try {
url = new URL(urlServ);
Reader r = null;
r = new InputStreamReader(url.openStream(), "ISO-8859-15");
CalendarBuilder builder = new CalendarBuilder();
Calendar thisCalendar = builder.build(r);
controle.refreshEdt(thisCalendar);
} catch (IOException | ParserException e) {
e.printStackTrace();
}
return "Completed";
}
#Override
protected void onPostExecute(String s){
Log.d("Return","************ " + s + " ************");
if(s!=null){
delegate.processFinish(s);
};
}
}
My icalendar content format:
BEGIN:VCALENDAR
BEGIN:VEVENT
ATTENDEE;ROLE=CHAIR:CN=LASTNAME Forename :MAILTO:edt-noreply#domain.fr
ATTENDEE;ROLE=REQ-PARTICIPANT:CN=S5A-01_ANG :MAILTO:edt-noreply#domain.fr
SUMMARY:- ang
DESCRIPTION:Professeur:LASTNAME Forename \nGroupe:S5A-01_ANG
DTSTART;TZID=Europe/Paris:20191211T080500
DTEND;TZID=Europe/Paris:20191211T093500
DTSTAMP:20200527T104154
LOCATION:B213
STATUS:CONFIRMED
UID:47468078
END:VEVENT
BEGIN:VEVENT
ATTENDEE;ROLE=CHAIR:CN=LASTNAME Forename :MAILTO:edt-noreply#domain.fr
ATTENDEE;ROLE=REQ-PARTICIPANT:CN=S5A-01_ANG :MAILTO:edt-noreply#domain.fr
...
END:VEVENT
END:VCALENDAR
The error code :
java.net.URISyntaxException: Illegal character in scheme name at index 2: CN=LASTNAME Forename :MAILTO:edt-noreply#domain.fr
at java.net.URI$Parser.fail(URI.java:2875)
at java.net.URI$Parser.checkChars(URI.java:3048)
at java.net.URI$Parser.parse(URI.java:3075)
at java.net.URI.<init>(URI.java:583)
at net.fortuna.ical4j.util.Uris.create(Uris.java:121)
at net.fortuna.ical4j.model.property.Attendee.setValue(Attendee.java:109)
at net.fortuna.ical4j.model.property.Attendee.<init>(Attendee.java:85)
at net.fortuna.ical4j.model.property.Attendee$Factory.createProperty(Attendee.java:175)
at net.fortuna.ical4j.model.PropertyBuilder.build(PropertyBuilder.java:48)
at net.fortuna.ical4j.data.DefaultContentHandler.endProperty(DefaultContentHandler.java:123)
at net.fortuna.ical4j.data.CalendarParserImpl$PropertyParser.parse(CalendarParserImpl.java:292)
at net.fortuna.ical4j.data.CalendarParserImpl$PropertyParser.access$1100(CalendarParserImpl.java:224)
at net.fortuna.ical4j.data.CalendarParserImpl$PropertyListParser.parse(CalendarParserImpl.java:211)
at net.fortuna.ical4j.data.CalendarParserImpl$ComponentParser.parse(CalendarParserImpl.java:427)
at net.fortuna.ical4j.data.CalendarParserImpl$ComponentParser.access$900(CalendarParserImpl.java:404)
at net.fortuna.ical4j.data.CalendarParserImpl$PropertyListParser.parse(CalendarParserImpl.java:209)
at net.fortuna.ical4j.data.CalendarParserImpl.parseCalendar(CalendarParserImpl.java:115)
at net.fortuna.ical4j.data.CalendarParserImpl.parseCalendarList(CalendarParserImpl.java:180)
at net.fortuna.ical4j.data.CalendarParserImpl.parse(CalendarParserImpl.java:149)
... 10 more

There is a syntax error in your iCalendar data.
A semicolon must come after the ROLE parameter, not a colon. For example:
Incorrect: ATTENDEE;ROLE=CHAIR:CN=LASTNAME Forename :MAILTO:edt-noreply#domain.fr
Correct: ATTENDEE;ROLE=CHAIR;CN=LASTNAME Forename :MAILTO:edt-noreply#domain.fr

Thank you #Michael for your help! Here it's my code which works in my situation :
#Override
protected String doInBackground(String... strings) {
// Create a TimeZone
System.setProperty("net.fortuna.ical4j.timezone.cache.impl", MapTimeZoneCache.class.getName());
URL url = null;
try {
url = new URL(urlServ);
Reader r = null;
r = new InputStreamReader(url.openStream(), "ISO-8859-15");
BufferedReader reader = new BufferedReader(r);
String str;
StringBuilder sb = new StringBuilder();
while((str = reader.readLine()) != null){
if(!(str.startsWith("ATTENDEE")))
sb.append(str + "\n");
}
r = new StringReader(sb.toString());
CalendarBuilder builder = new CalendarBuilder();
Calendar thisCalendar = builder.build(r);
controle.refreshEdt(thisCalendar);
} catch (IOException | ParserException e) {
e.printStackTrace();
}
return "Completed";
}

Related

AsyncTask doInBackground to return multiple strings

I'm trying to build a very basic weather app in android studio. I am using AsyncClass to return multiple strings.
As you can see in the code, I used a class named "Wrapper" that is used to store my strings so I can just return a class object and use it in the onPostExecute method of the AsyncTask. The problem I am facing is that when I test the app, all of the returned Strings somehow are undefined (the default for the Wrapper class). This means the strings are not being updated in the doInBackground method and I can't seem to figure out why!
My Activity
#Override
public void onConnectionFailed(ConnectionResult connectionResult) {
Log.i(MainActivity.class.getSimpleName(), "Can't connect to Google Play Services!");
}
private class Wrapper
{
String Temperature = "UNDEFINED";
String city = "UNDEFINED";
String country = "UNDEFINED";
}
private class GetWeatherTask extends AsyncTask<String, Void, Wrapper> {
private TextView textView;
public GetWeatherTask(TextView textView) {
this.textView = textView;
}
#Override
protected Wrapper doInBackground(String... strings) {
Wrapper w = new Wrapper();
String Temperature = "x";
String city = "y";
String country = "z";
try {
URL url = new URL(strings[0]);
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
InputStream stream = new BufferedInputStream(urlConnection.getInputStream());
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(stream));
StringBuilder builder = new StringBuilder();
String inputString;
while ((inputString = bufferedReader.readLine()) != null) {
builder.append(inputString);
}
JSONObject topLevel = new JSONObject(builder.toString());
JSONObject main = topLevel.getJSONObject("main");
JSONObject cityobj = topLevel.getJSONObject("city");
Temperature = String.valueOf(main.getDouble("temp"));
city = cityobj.getString("name");
country = cityobj.getString("country");
w.Temperature= Temperature;
w.city= city;
w.country=country;
urlConnection.disconnect();
} catch (IOException | JSONException e) {
e.printStackTrace();
}
return w;
}
#Override
protected void onPostExecute(Wrapper w) {
textView.setText("Current Temperature: " + w.Temperature + " C" + (char) 0x00B0
+"\n" + "Current Location: "+ w.country +"\n" + "City: "+ w.city );
}
}
}
UPDATE:
turned out that that I was using the wrong url in my code,I was using :
http://api.openweathermap.org/data/2.5/weather?lat=%f&lon=%f&units=%s&appid=%s
Instead I should've been using:
http://api.openweathermap.org/data/2.5/forecast?lat=%f&lon=%f&units=%s&appid=%s
-aka instead of weather I should've been using forcast
Your error starts here
JSONObject main = topLevel.getJSONObject("main");
Probably because the topLevel object has no "main" key.
{
"city":{ },
"cod":"200",
"message":0.1859,
"cnt":40,
"list":[ ]
}
Throw your JSON into here. https://jsonformatter.curiousconcept.com/
You'll notice that there are many, many "main" keys that are within the "list" element, but you have to parse those starting from getJSONArray("list").
Basically, something like this
String city = "undefined";
String country = "undefined";
List<Double> temperatures = new ArrayList<Double>();
try {
JSONObject object = new JSONObject(builder.toString());
JSONObject jCity = object.getJSONObject("city");
city = jCity.getString("name");
country = jCity.getString("country");
JSONArray weatherList = object.getJSONArray("list");
for (int i = 0; i < weatherList.length(); i++) {
JSONObject listObject = weatherList.getJSONObject(i);
double temp = listObject.getJSONObject("main").getDouble("temp");
temperatures.add(temp);
}
} catch (JSONException e) {
e.printStackTrace();
}
return new Wrapper(city, country, temperatures);
After studying your code, either your try block is failing, which is returning your object, but empty, or there is something wrong with your JSON parsing. If you could show us the JSON you are trying to parse that would be a great help.
That being said, the fact that it is still showing as "UNDEFINED" is because that is how you initialised it, and becuase (the JSON parse is likely failing), the object is being returned in an un-edited state.
EDIT:
You are parsing the JSON wrong. You are trying to find an object called "main" in the top directory, however the main object only exists inside of an array called list!
Please look here for a more easy to see and visual representation: http://prntscr.com/dlhlrk
You can use this site to help visualise your JSON and create an appropriate soluton based upon it. https://jsonformatter.curiousconcept.com/
Looking at the API you posted earlier (api.openweathermap.org) you are trying to access variables that don't exist. I suggest you have a look at what the API returns and try getting the variables one by one if you are getting a JSONException
EDIT:
What API you are using? In your initial post you said it was http://api.openweathermap.org/data/2.5/weather but in a comment above you said it was http://api.openweathermap.org/data/2.5/forecast.
If you're using the weather API (as initially stated) you can use the below:
#Override
protected Wrapper doInBackground(String... strings) {
Wrapper w = new Wrapper();
try {
URL url = new URL(strings[0]);
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
InputStream stream = new BufferedInputStream(urlConnection.getInputStream());
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(stream));
StringBuilder builder = new StringBuilder();
String inputString;
while ((inputString = bufferedReader.readLine()) != null) {
builder.append(inputString);
}
Log.d("JSON", builder.toString());
JSONObject topLevel = new JSONObject(builder.toString());
JSONObject main = topLevel.getJSONObject("main");
JSONObject sys = topLevel.getJSONObject("sys");
w.Temperature = String.valueOf(main.getDouble("temp"));
w.city = topLevel.getString("name");
w.country = sys.getString("country");
urlConnection.disconnect();
} catch (IOException | JSONException e) {
e.printStackTrace();
}
return w;
}

How to print json to console in plugin using MessageConsoleStream

I'm creating a plugin for eclipse that takes JSON from the Web of Trust API and prints it into the console. I was able to print the entire JSON but when I try to get specific information, it will not show up in the plugin console.
try {
JSONObject json = readJsonFromUrl("http://api.mywot.com/0.4/public_link_json2?hosts=google.com/&key=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx");
JSONObject msg = (JSONObject) json.getJSONObject("google.com").getJSONObject("categories");
String[] parts = msg.toString().split(",");
String[] parts2 = parts[1].split(":");
// System.out.println("Score:" + parts2[0]);
//System.out.println("Score:" + parts2[0]);
//print result
MessageConsoleStream out = ConsoleManager.getMessageConsoleStream( "Console" );
out.println("Score:" + msg.toString());
}
catch (Exception e) {
}
return null;
}
private static String readAll(Reader rd) throws IOException {
StringBuilder sb = new StringBuilder();
int cp;
while ((cp = rd.read()) != -1) {
sb.append((char) cp);
}
return sb.toString();
}
public static JSONObject readJsonFromUrl(String url) throws IOException, JSONException {
InputStream is = new URL(url).openStream();
try {
BufferedReader rd = new BufferedReader(new InputStreamReader(is, Charset.forName("UTF-8")));
String jsonText = readAll(rd);
JSONObject json = new JSONObject(jsonText);
return json;
} finally {
is.close();
}
}
ConsoleManager Class:
public class ConsoleManager {
public static MessageConsole findConsole( String name ) {
ConsolePlugin plugin = ConsolePlugin.getDefault();
IConsoleManager conMan = plugin.getConsoleManager();
IConsole[] existing = conMan.getConsoles();
for ( int i = 0; i < existing.length; i++ )
if ( name.equals( existing[i].getName() ) ) return ( MessageConsole ) existing[i];
//no console found, so create a new one
MessageConsole myConsole = new MessageConsole( name, null );
conMan.addConsoles( new IConsole[]{myConsole} );
return myConsole;
}
public static MessageConsoleStream getMessageConsoleStream( String name ) {
MessageConsole myConsole = findConsole( name );
MessageConsoleStream out = myConsole.newMessageStream();
return out;
}
}
So I'm trying to print the value from the JSON stored in parts2, however when I try to print it to the plugin console, it doesn't show anything. No errors so I think I'm missing something.
Any help is greatly appreciated.
Is your console the one that is currently being shown?
You probably want to call IConsoleManager.showConsoleView(console) to show your console.

Verify OAuth1a signed request using Twitter joauth with RSA-SHA1?

I have a use case to authenticate OAuth1 request which is signed using RSA Private Key and verified at server end with RSA public key.
I found this library from Twitter which helps us authenticate/verify the Oauth signed requests. https://github.com/twitter/joauth
I want to leverage this library for verifying the request from Jersey or Spring MVC action method. The request from client would have been signed using private key. At my end I would use the public key of the client to verify the request. which means RSA-SHA1 algo.
Twitter joauth seem to be useful but I am missing the code that would transform HttpServletRequest to OAuthRequest
The library read-me file suggests this as facility but I could not find a code that does javax.servlet.http.HttpServletRequest --> com.twitter.joauth.OAuthRequest transformation.
The request verification happens in verify method which has following signature.
public VerifierResult verify(UnpackedRequest.OAuth1Request request, String tokenSecret, String consumerSecret);
Secondly I also want to know which is the most appropriate way to use/read RSA public key with twitter joauth when verify method takes String parameter ?
I have never used any library to authenticate users via Twitter. But I have just looked in the UnpackedRequest.OAuth1Request. You can create an instance of this class by filling all parameters. I have written Twitter OAuth Header creator, so you can just use it to fill those parameters or send POST requests directly without a library.
Here all classes what you need:
Signature - to generate an OAuth Signature.
public class Signature {
private static final String HMAC_SHA1_ALGORITHM = "HmacSHA1";
public static String calculateRFC2104HMAC(String data, String key)
throws java.security.SignatureException
{
String result;
try {
SecretKeySpec signingKey = new SecretKeySpec(key.getBytes(), HMAC_SHA1_ALGORITHM);
Mac mac = Mac.getInstance(HMAC_SHA1_ALGORITHM);
mac.init(signingKey);
byte[] rawHmac = mac.doFinal(data.getBytes());
result = new String(Base64.encodeBase64(rawHmac));
} catch (Exception e) {
throw new SignatureException("Failed to generate HMAC : " + e.getMessage());
}
return result;
}
}
NvpComparator - to sort parameters you need in the header.
public class NvpComparator implements Comparator<NameValuePair> {
#Override
public int compare(NameValuePair arg0, NameValuePair arg1) {
String name0 = arg0.getName();
String name1 = arg1.getName();
return name0.compareTo(name1);
}
}
OAuth - for URL encode.
class OAuth{
...
public static String percentEncode(String s) {
return URLEncoder.encode(s, "UTF-8")
.replace("+", "%20").replace("*", "%2A")
.replace("%7E", "~");
}
...
}
HeaderCreator - to create all needed parameters and generate an OAuth header param.
public class HeaderCreator {
private String authorization = "OAuth ";
private String oAuthSignature;
private String oAuthNonce;
private String oAuthTimestamp;
private String oAuthConsumerSecret;
private String oAuthTokenSecret;
public String getAuthorization() {
return authorization;
}
public String getoAuthSignature() {
return oAuthSignature;
}
public String getoAuthNonce() {
return oAuthNonce;
}
public String getoAuthTimestamp() {
return oAuthTimestamp;
}
public HeaderCreator(){}
public HeaderCreator(String oAuthConsumerSecret){
this.oAuthConsumerSecret = oAuthConsumerSecret;
}
public HeaderCreator(String oAuthConsumerSecret, String oAuthTokenSecret){
this(oAuthConsumerSecret);
this.oAuthTokenSecret = oAuthTokenSecret;
}
public String getTwitterServerTime() throws IOException, ParseException {
HttpsURLConnection con = (HttpsURLConnection)
new URL("https://api.twitter.com/oauth/request_token").openConnection();
con.setRequestMethod("HEAD");
con.getResponseCode();
String twitterDate= con.getHeaderField("Date");
DateFormat formatter = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss Z", Locale.ENGLISH);
Date date = formatter.parse(twitterDate);
return String.valueOf(date.getTime() / 1000L);
}
public String generatedSignature(String url, String method, List<NameValuePair> allParams,
boolean withToken) throws SignatureException {
oAuthNonce = String.valueOf(System.currentTimeMillis());
allParams.add(new BasicNameValuePair("oauth_nonce", oAuthNonce));
try {
oAuthTimestamp = getTwitterServerTime();
allParams.add(new BasicNameValuePair("oauth_timestamp", oAuthTimestamp));
}catch (Exception ex){
//TODO: Log!!
}
Collections.sort(allParams, new NvpComparator());
StringBuffer params = new StringBuffer();
for(int i=0;i<allParams.size();i++)
{
NameValuePair nvp = allParams.get(i);
if (i>0) {
params.append("&");
}
params.append(nvp.getName() + "=" + OAuth.percentEncode(nvp.getValue()));
}
String signatureBaseStringTemplate = "%s&%s&%s";
String signatureBaseString = String.format(signatureBaseStringTemplate,
OAuth.percentEncode(method),
OAuth.percentEncode(url),
OAuth.percentEncode(params.toString()));
String compositeKey = OAuth.percentEncode(oAuthConsumerSecret)+"&";
if(withToken) compositeKey+=OAuth.percentEncode(oAuthTokenSecret);
oAuthSignature = Signature.calculateRFC2104HMAC(signatureBaseString, compositeKey);
return oAuthSignature;
}
public String generatedAuthorization(List<NameValuePair> allParams){
authorization = "OAuth ";
Collections.sort(allParams, new NvpComparator());
for(NameValuePair nvm : allParams){
authorization+=nvm.getName()+"="+OAuth.percentEncode(nvm.getValue())+", ";
}
authorization=authorization.substring(0,authorization.length()-2);
return authorization;
}
}
Explain:
1. getTwitterServerTime
In oAuthTimestamp you need not your time of server but the time of a Twitter server. You can optimize it saving this param if you always send requests in the certain Twitter server.
2. HeaderCreator.generatedSignature(...)
url - logically url to twitter API
method - GET or POST. You must use always "POST"
allParams - Parameters which you know to generate signature ("param_name", "param_value");
withToken - if you know oAuthTokenSecret put true. Otherwise false.
3. HeaderCreator.generatedAuthorization(...)
Use this method after generatedSignature(...) to generate an OAuth header string.
allParams - it is parameters which you have used in generatedSignature(...) plus: nonce, signature, timestamp. Always use:
allParams.add(new BasicNameValuePair("oauth_nonce", headerCreator.getoAuthNonce()));
allParams.add(new BasicNameValuePair("oauth_signature", headerCreator.getoAuthSignature()));
allParams.add(new BasicNameValuePair("oauth_timestamp", headerCreator.getoAuthTimestamp()));
Now you can use it to fill UnpackedRequest.OAuth1Request in your library. Also here an example to authenticate user in SpringMVC without the library:
Requests - to send post requests.
public class Requests {
public static String sendPost(String url, String urlParameters, Map<String, String> prop) throws Exception {
URL obj = new URL(url);
HttpsURLConnection con = (HttpsURLConnection) obj.openConnection();
con.setRequestMethod("POST");
if(prop!=null) {
for (Map.Entry<String, String> entry : prop.entrySet()) {
con.setRequestProperty(entry.getKey(), entry.getValue());
}
}
con.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
wr.writeBytes(urlParameters);
wr.flush();
wr.close();
int responseCode = con.getResponseCode();
BufferedReader in;
if(responseCode==200) {
in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
}else{
in = new BufferedReader(
new InputStreamReader(con.getErrorStream()));
}
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
return response.toString();
}
}
twAuth(...) - put it in your controller. Execute it when an user want to authenticate in your site via Twitter.
#RequestMapping(value = "/twauth", method = RequestMethod.GET)
#ResponseBody
public String twAuth(HttpServletResponse response) throws Exception{
try {
String url = "https://api.twitter.com/oauth/request_token";
List<NameValuePair> allParams = new ArrayList<NameValuePair>();
allParams.add(new BasicNameValuePair("oauth_callback", "http://127.0.0.1:8080/twlogin"));
allParams.add(new BasicNameValuePair("oauth_consumer_key", "2YhNLyum1VY10UrWBMqBnatiT"));
allParams.add(new BasicNameValuePair("oauth_signature_method", "HMAC-SHA1"));
allParams.add(new BasicNameValuePair("oauth_version", "1.0"));
HeaderCreator headerCreator = new HeaderCreator("RUesRE56vVWzN9VFcfA0jCBz9VkvkAmidXj8d1h2tS5EZDipSL");
headerCreator.generatedSignature(url,"POST",allParams,false);
allParams.add(new BasicNameValuePair("oauth_nonce", headerCreator.getoAuthNonce()));
allParams.add(new BasicNameValuePair("oauth_signature", headerCreator.getoAuthSignature()));
allParams.add(new BasicNameValuePair("oauth_timestamp", headerCreator.getoAuthTimestamp()));
Map<String, String> props = new HashMap<String, String>();
props.put("Authorization", headerCreator.generatedAuthorization(allParams));
String twitterResponse = Requests.sendPost(url,"",props);
Integer indOAuthToken = twitterResponse.indexOf("oauth_token");
String oAuthToken = twitterResponse.substring(indOAuthToken, twitterResponse.indexOf("&",indOAuthToken));
response.sendRedirect("https://api.twitter.com/oauth/authenticate?" + oAuthToken);
}catch (Exception ex){
//TODO: Log
throw new Exception();
}
return "main";
}
twLogin(...) - put it in your controller. It is callback from Twitter.
#RequestMapping(value = "/twlogin", method = RequestMethod.GET)
public String twLogin(#RequestParam("oauth_token") String oauthToken,
#RequestParam("oauth_verifier") String oauthVerifier,
Model model, HttpServletRequest request){
try {
if(oauthToken==null || oauthToken.equals("") ||
oauthVerifier==null || oauthVerifier.equals(""))
return "main";
String url = "https://api.twitter.com/oauth/access_token";
List<NameValuePair> allParams = new ArrayList<NameValuePair>();
allParams.add(new BasicNameValuePair("oauth_consumer_key", "2YhNLyum1VY10UrWBMqBnatiT"));
allParams.add(new BasicNameValuePair("oauth_signature_method", "HMAC-SHA1"));
allParams.add(new BasicNameValuePair("oauth_token", oauthToken));
allParams.add(new BasicNameValuePair("oauth_version", "1.0"));
NameValuePair oAuthVerifier = new BasicNameValuePair("oauth_verifier", oauthVerifier);
allParams.add(oAuthVerifier);
HeaderCreator headerCreator = new HeaderCreator("RUesRE56vVWzN9VFcfA0jCBz9VkvkAmidXj8d1h2tS5EZDipSL");
headerCreator.generatedSignature(url,"POST",allParams,false);
allParams.add(new BasicNameValuePair("oauth_nonce", headerCreator.getoAuthNonce()));
allParams.add(new BasicNameValuePair("oauth_signature", headerCreator.getoAuthSignature()));
allParams.add(new BasicNameValuePair("oauth_timestamp", headerCreator.getoAuthTimestamp()));
allParams.remove(oAuthVerifier);
Map<String, String> props = new HashMap<String, String>();
props.put("Authorization", headerCreator.generatedAuthorization(allParams));
String twitterResponse = Requests.sendPost(url,"oauth_verifier="+oauthVerifier,props);
//Get user id
Integer startIndexTmp = twitterResponse.indexOf("user_id")+8;
Integer endIndexTmp = twitterResponse.indexOf("&",startIndexTmp);
if(endIndexTmp<=0) endIndexTmp = twitterResponse.length()-1;
Long userId = Long.parseLong(twitterResponse.substring(startIndexTmp, endIndexTmp));
//Do what do you want...
}catch (Exception ex){
//TODO: Log
throw new Exception();
}
}

FileNotFoundException in collection of tweets from Twitter - Java

I have an algorithm that search old tweets in Twitter between two dates. My objective is to return all tweets. But I am getting an exception.
Code:
public static List<Tweet> getTweets(String username, String since, String until, String querySearch) {
List<Tweet> results = new ArrayList<Tweet>();
try {
String refreshCursor = null;
while (true) {
String response = getURLResponse(username, since, until, querySearch, refreshCursor);
System.out.println(response);
JSONObject json = new JSONObject(response);
refreshCursor = json.getString("min_position");
Document doc = Jsoup.parse((String) json.get("items_html"));
Elements tweets = doc.select("div.js-stream-tweet");
if (tweets.size() == 0) {
break;
}
for (Element tweet : tweets) {
String usernameTweet = tweet.select("span.username.js-action-profile-name b").text();
String txt = tweet.select("p.js-tweet-text").text().replaceAll("[^\\u0000-\\uFFFF]", "");
int retweets = Integer.valueOf(tweet.select("span.ProfileTweet-action--retweet span.ProfileTweet-actionCount").attr("data-tweet-stat-count").replaceAll(",", ""));
int favorites = Integer.valueOf(tweet.select("span.ProfileTweet-action--favorite span.ProfileTweet-actionCount").attr("data-tweet-stat-count").replaceAll(",", ""));
long dateMs = Long.valueOf(tweet.select("small.time span.js-short-timestamp").attr("data-time-ms"));
Date date = new Date(dateMs);
Tweet t = new Tweet(usernameTweet, txt, date, retweets, favorites);
results.add(t);
}
}
} catch (Exception e) {
e.printStackTrace();
}
return results;
}
Method that makes the page request:
private static String getURLResponse(String from, String since, String until, String querySearch, String scrollCursor) throws Exception {
String appendQuery = "";
if (from != null) {
appendQuery += "from:"+from;
}
if (since != null) {
appendQuery += " since:"+since;
}
if (until != null) {
appendQuery += " until:"+until;
}
if (querySearch != null) {
appendQuery += " "+querySearch;
}
String url = String.format("https://twitter.com/i/search/timeline?f=realtime&q=%s&src=typd&max_position=%s", URLEncoder.encode(appendQuery, "UTF-8"), scrollCursor);
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setRequestMethod("GET");
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
return response.toString();
}
Exception:
java.io.FileNotFoundException: https://twitter.com/i/search/timeline?f=realtime&q=+since%3A2014-10-08+until%3A2014-10-10+dilma&src=typd&max_position=TWEET-520341224046469121-520363066366496768-BD1UO2FFu9QAAAAAAAAETAAAAAcAAAASAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
at Manager.TweetManager.getURLResponse(TweetManager.java:58)
at Manager.TweetManager.getTweets(TweetManager.java:121)
at Main.Main.main(Main.java:54)
JSON downloaded: JSON

How to train Name model in OpenNLP?

Iam trying to train the name finder model to detect the Names but it is not giving proper result.
Here is the Code.
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
InputStream is=null;
Resources resources=this.getResources();
assetManager=resources.getAssets();
String trainingDataFile = "en-ner-person.train";
String outputModelFile = "en-ner-person.bin";
String sentence[] = {"Sunil", "61 years old , will join the board as a nonexecutive director Nov. 29" };
train(trainingDataFile, outputModelFile, "person");
try {
predict(sentence, outputModelFile);
}
catch(Exception e)
{
System.out.println("Errror Preditct" + e.getMessage());
}
}
private static void train(String trainingDataFile, String outputModelFile, String tagToFind) {
NameSampleDataStream nss = null;
try {
nss = new NameSampleDataStream(new PlainTextByLineStream(new java.io.FileReader(trainingDataFile)));
} catch (Exception e) {}
TokenNameFinderModel model = null;
try {
model = NameFinderME.train("en", tagToFind, nss, Collections.<String, Object>emptyMap());
} catch(Exception e) {}
try {
File outFile = new File(outputModelFile);
FileOutputStream outFileStream = new FileOutputStream(outFile);
model.serialize(outFileStream);
}
catch (Exception ex) {}
}
private void predict(String sentence[], String modelFile) throws Exception {
InputStream is1 ;
is1 = assetManager.open("en-ner-person.bin",MODE_PRIVATE);
TokenNameFinderModel model1 = new TokenNameFinderModel(is1);
String sd;
NameFinderME nameFinder = new NameFinderME(model1);
Span sp[] = nameFinder.find(sentence);
String a[] = Span.spansToStrings(sp, sentence);
StringBuilder fd = new StringBuilder();
int l = a.length;
for (int j = 0; j < l; j++) {
fd = fd.append(a[j] + "\n");
}
sd = fd.toString();
Log.d("Name Detected:", sd);
}
}
Here is Output iam getting:
D:\Name Detected: [ 07-20 19:35:47.516 8799: 8799 I/Adreno-EGL ]
Content Of en-ner-person.train is:
<START:person> Sunil <END> , 61 years old , will join the board as a nonexecutive director Nov. 29 .
Kindly help.
Try this:
Span[] sp = nameFinder.find(search);
nameFinder.clearAdaptiveData();
for (Span span : sp) {
for (int i=span.getStart(); i<span.getEnd(); i++) {
fd.append(sentence[i] + "\n");
}
}
Also this line seems to be wrong, you don't need to assign the content of append to fd again.
fd = fd.append(a[j] + "\n");
Hope that helps.

Categories