I am running Apache storm topology and I am trying to read the values from properties file. My main function is able to read the values while the bolts are throwing the following error.
Also , I am able to read all the values from a normal class in the same project.
Error
java.lang.NullPointerException at
java.util.Properties$LineReader.readLine(Properties.java:434) at
java.util.Properties.load0(Properties.java:353) at
java.util.Properties.load(Properties.java:341) at
com.StormConsumer.commons.FilePropertyManager.loadPrope
rtyFile(FilePropertyManager.java:54) at
com.StormConsumer.commons.FilePropertyManager.getProper
ty(FilePropertyManager.java:34) at
com.StormConsumer.bolt.SessionIdCounter.prepare(SessionIdCounter.java:42) at
backtype.storm.topology.BasicBoltExecutor.prepare(BasicBoltExecutor.java:43)
at backtype.storm.daemon.executor$fn__4722$fn__4734.invoke(executor.clj:692)
at backtype.storm.util$async_loop$fn__458.invoke(util.clj:461) at
clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:745)
How to solve it ?
This is the class :
public class SessionIdCounter extends BaseBasicBolt {
private static final long serialVersionUID = 1L;
Cluster cluster;
Session session;
/*private static final String KEYSPACE = FilePropertyManager.getProperty(
ApplicationConstants.CASSANDRA_CONSTANTS_FILE,
ApplicationConstants.KEYSPACE);
private static final String CREATE_TABLE = FilePropertyManager.getProperty(
ApplicationConstants.CASSANDRA_QUERY_FILE,
ApplicationConstants.TABLE_SessionID);*/
/**
* creates a table SessionID with fields sessionid, year, month, day, hour,
* dayofyear, weekofyear, count.
*/
public void cleanup() {
}
public void prepare(Map stormConf, TopologyContext context) {
final String KEYSPACE = FilePropertyManager.getProperty(
ApplicationConstants.CASSANDRA_CONSTANTS_FILE,
ApplicationConstants.KEYSPACE);
final String CREATE_TABLE = FilePropertyManager.getProperty(
ApplicationConstants.CASSANDRA_QUERY_FILE,
ApplicationConstants.TABLE_SessionID);
cluster = CassandraConnection.setupCassandraClient();
session = CassandraConnection.getSessionWithRetry(cluster, KEYSPACE);
session.executeAsync(CREATE_TABLE);
}
public void declareOutputFields(OutputFieldsDeclarer declarer) {
}
public void execute(Tuple input, BasicOutputCollector collector) {
String sessionid = input.getString(0);
int year = input.getInteger(1);
int month = input.getInteger(2);
int day = input.getInteger(3);
int hour = input.getInteger(4);
int dayofyear = input.getInteger(5);
int weekofyear = input.getInteger(6);
/*
* Inserting Values In Cassandra
*/
String insertUpdateTable = "UPDATE SessionID SET count = count + 1 "
+ "where sessionid = \'" + sessionid + "\' AND year = " + year
+ " AND month = " + month + " AND day = " + day
+ " AND hour = " + hour + " AND dayofyear = " + dayofyear
+ " AND weekofyear = " + weekofyear + " ;";
session.executeAsync(insertUpdateTable);
}
}
The problem is in your property file. If the ApplicationConstants.CASSANDRA_CONSTANTS_FILE points to a local file not avalaible in all your server, then you've got a problem. Also, check the format of your file.
Related
This the error
I have attached the error above after executing code below mentioned, the code given below is not full code but it targets the main code for which my question is.I am trying to send sms through a variable in which number is stored by fetching from sqlite database. So below code shows that I tried to fetch from dbhelper.java class and store in variable num in sendsms.java class but i think it is not fetching Hence i would request you to see the code and guide where i am wrong for improvement. I hope that now question is clear and sufficient description is given so please help.
sendsms.java
try {
Cursor cursor = databaseHelper.getdata();
while (cursor.moveToNext())
{
String num = cursor.getString(0);
String numm = cursor.getString(1);
String nummm = cursor.getString(2);
}
}
catch(Exception e){
Toast.makeText(getBaseContext(), e.getMessage(),
Toast.LENGTH_SHORT).show();
}
public void run() {
sms.sendTextMessage(num, null, "Help! I've met with an accident at http://maps.google.com/?q=" + String.valueOf(latitude) + "" +
"," + String.valueOf(longitude), null, null);
dbhelper.java
public static final String TABLE_REGISTER = "signin";
public static final String COL_ID = "USER_ID";
public static final String COL_NAME = "NAME";
public static final String COL_PHONE = "PHONE_NUMBER";
public static final String COL_EMAIL = "EMAIL";
public static final String COL_PASSWORD = "PASSWORD";
public static final String COL_CONFIRM_PASSWORD = "CONFIRM_PASSWORD";
public static final String COL_NAMEone_CON = "NAMEONE";
public static final String COL_NUMBERone_CON = "NUMBERONE";
public static final String COL_NAMEtwo_CON = "NAMETWO";
public static final String COL_NUMBERtwo_CON = "NUMBERTWO";
public static final String COL_NAMEthree_CON = "NAMETHREE";
public static final String COL_NUMBERthree_CON = "NUMBERTHREE";
public SQLiteDatabase db;
#Override
public void onCreate(SQLiteDatabase db) {
db.execSQL("CREATE TABLE " + TABLE_REGISTER + "(" + COL_ID + " INTEGER PRIMARY KEY AUTOINCREMENT , "
+ COL_NAME + " TEXT , " + COL_PHONE + " LONG UNIQUE ," + COL_EMAIL + " VARCHAR UNIQUE," + COL_PASSWORD + " VARCHAR , "
+ COL_CONFIRM_PASSWORD + " VARCHAR ," + COL_NAMEone_CON + " TEXT , "
+ COL_NUMBERone_CON + " LONG UNIQUE ," + COL_NAMEtwo_CON + " TEXT ," + COL_NUMBERtwo_CON + " LONG UNIQUE , "
+ COL_NAMEthree_CON + " TEXT ," + COL_NUMBERthree_CON + " LONG UNIQUE " + ")");
}
public Cursor getdata(){
SQLiteDatabase db = this.getWritableDatabase();
Cursor cursor = db.rawQuery("select NUMBERONE, NUMBERTWO,NUMBERTHREE from signin ", null);
return cursor;
}
It seems that you have to call getReadableDatabase or getWritableDatabase before you need the data, this is because it takes time to create the database.So, you have to be aware of that.
Also, I wanted to remark that you must cursor.moveToFirst() before you do cursor.moveToNext() on sendsms.java
Please change your code like this:-
try {
Cursor cursor = databaseHelper.getdata();
if (cursor != null && cursor.getCount() > 0) {
while (cursor.moveToNext()) {
String num = cursor.getString(0);
String numm = cursor.getString(1);
String nummm = cursor.getString(2);
}
}
}
catch(Exception e){
Toast.makeText(getBaseContext(), e.getMessage(),
Toast.LENGTH_SHORT).show();
}
I am trying to store image into SQLite and then retrieve the image back. I have searched other post related to it and got some clues and leads but now i am stuck and getting error which i do not know WHY ?
The image store into the SQLite Database Successfully but when i try to get the image i get the error
Caused by: java.lang.IllegalStateException: Couldn't read row 0, col -1 from CursorWindow. Make sure the Cursor is initialized correctly before accessing data from it.
Below is my code
SQL Helper Class
import android.content.Context;
import android.database.sqlite.SQLiteDatabase;
import android.database.sqlite.SQLiteOpenHelper;
public class SQLHelperClass extends SQLiteOpenHelper {
public static int DATABASE_VERSION = 12;
public static String DATABASE_NAME = "user_image_save.db";
public static String CREATE_TABLE = "CREATE TABLE";
public static String TEXT_TYPE = "TEXT";
public static String BLOB = "BLOB";
public static String INTEGER_TYPE = "INTEGER";
public static String DROP_TABLE = "DROP TABLE IF EXISTS" ;
public static String AUTOINCREMENT = "INTEGER PRIMARY KEY AUTOINCREMENT" ;
public SQLHelperClass(Context context ){
super(context , DATABASE_NAME , null , DATABASE_VERSION);
}
#Override
public void onCreate(SQLiteDatabase sqLiteDatabase) {
sqLiteDatabase.execSQL(CREATE_TABLE + " " + SQL_Constants.SavedIntoSQL.TABLE_SQL_SAVE_IMAGE
+ " ( " + SQL_Constants.SavedIntoSQL._ID + " " + AUTOINCREMENT + " , "
+ SQL_Constants.SavedIntoSQL.COLUMN_OWNER_ID + " " + TEXT_TYPE + " , "
+ SQL_Constants.SavedIntoSQL.COLUMN_NAME + " " + TEXT_TYPE + " , "
+ SQL_Constants.SavedIntoSQL.COLUMN_EMAIL + " " + TEXT_TYPE + " , "
+ SQL_Constants.SavedIntoSQL.COLUMN_IMAGE + " " + BLOB+ ")" );
}
#Override
public void onUpgrade(SQLiteDatabase sqLiteDatabase, int oldVersion, int newVersion) {
sqLiteDatabase.execSQL(DROP_TABLE);
onCreate(sqLiteDatabase);
}
}
Constant Class
import android.provider.BaseColumns;
import android.provider.BaseColumns;
public final class SQL_Constants {
public static abstract class SavedIntoSQL implements BaseColumns {
public static final String TABLE_SQL_SAVE_IMAGE = "imageSave";
public static final String ID_DATA = _ID ;
public static final String COLUMN_OWNER_ID = "owner_id";
public static final String COLUMN_EMAIL = "email";
public static final String COLUMN_NAME = "name";
public static final String COLUMN_IMAGE = "user_image";
}
}
Storing Data into SQLite
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
resource.compress(Bitmap.CompressFormat.PNG, 100, byteArrayOutputStream);
byte[] bytes = byteArrayOutputStream.toByteArray();
Toast.makeText(UpdatedUserProfile.this, "Download Complete", Toast.LENGTH_SHORT).show();
ContentValues contentValues = new ContentValues();
contentValues.put(SQL_Constants.SavedIntoSQL.COLUMN_OWNER_ID, App.getAppInstance().getCurrentUser().getProperty("ownerId").toString());
contentValues.put(SQL_Constants.SavedIntoSQL.COLUMN_NAME, App.getAppInstance().getCurrentUser().getProperty("name").toString());
contentValues.put(SQL_Constants.SavedIntoSQL.COLUMN_EMAIL, App.getAppInstance().getCurrentUser().getEmail());
contentValues.put(SQL_Constants.SavedIntoSQL.COLUMN_IMAGE, bytes);
sqLiteDatabase.insert(SQL_Constants.SavedIntoSQL.TABLE_SQL_SAVE_IMAGE, null, contentValues);
Getting The Data Back
sqlHelperClass = new SQLHelperClass(UpdatedUserProfile.this);
sqLiteDatabase = sqlHelperClass.getWritableDatabase();
sqLiteDatabase = sqlHelperClass.getReadableDatabase();
String projection [] = {
SQL_Constants.SavedIntoSQL.COLUMN_OWNER_ID ,
SQL_Constants.SavedIntoSQL.COLUMN_EMAIL ,
SQL_Constants.SavedIntoSQL.COLUMN_NAME ,
};
cursor = sqLiteDatabase.query(SQL_Constants.SavedIntoSQL.TABLE_SQL_SAVE_IMAGE , projection,
null , null , null , null , null);
cursor.moveToPosition(0);
int statusOwnerId = cursor.getColumnIndex(SQL_Constants.SavedIntoSQL.COLUMN_OWNER_ID);
int statusName = cursor.getColumnIndex(SQL_Constants.SavedIntoSQL.COLUMN_NAME);
int statusEmail = cursor.getColumnIndex(SQL_Constants.SavedIntoSQL.COLUMN_EMAIL);
int statusImage = cursor.getColumnIndex(SQL_Constants.SavedIntoSQL.COLUMN_IMAGE);
String ownerIdAgain = cursor.getString(statusOwnerId);
String name = cursor.getString(statusName);
String email = cursor.getString(statusEmail);
byte [] image = cursor.getBlob(statusImage);
String compareValueOwnerId = sharedPreferencesDatabaseActivity.getString("userId", "");
String compareValueUserName = sharedPreferencesDatabaseActivity.getString("userName", "");
String compareValueEmail = sharedPreferencesDatabaseActivity.getString("userEmail", "");
i am getting all other data i.e name , email , owner_id but when i comes to image i get the error
This is the line that causes error
byte [] image = cursor.getBlob(statusImage);
Any help is appreciated
Your projection does not have COLUMN_IMAGE so cursor.getColumnIndex(SQL_Constants.SavedIntoSQL.COLUMN_IMAGE) returns -1. Attempting to read column -1 causes this exception.
Adding the column to your projection helps if the image blobs are small enough. Android SQLite cursors have a limited-size window and in practice you cannot query rows larger than 2MB. If that's the case for you, consider refactoring your code so that you're not storing the images in the database. Rather store them in the file system and just store their paths in the database.
I’ve a use case where I need to fetch all the records from Cassandra for a given time range and divide it into 30 chunks then further aggregate each chunk, for example let us suppose I’m fetching 60 records for a time range of 30 minutes. Now I need to divide into 30 chunk which will be 2 records per minute. If I’m fetching 600 records for a time range of 1 hour, then 30 chunk will be 20 records per 2 minutes. If I’m fetching 600 records for a time range of 1 week, then 30 chunk will be 20 records per 5.6 hours and so on.
For implementing the same I have written a java code which is giving result in 3 seconds for 100k records. I thought implementing the same in Cassandra UDF will have performance benefit, but the UDF is taking 6-7 seconds(double the time taken by java code) which is shocking for me. Somebody please guide, where I’m off the track, below is my table structure and java as well as UDF code.
Cassandra table schema
CREATE TABLE transactions_data (
app_name text,
api_name text,
app_id text,
start_time timestamp,
duration int,
end_time timestamp,
node_id text,
request_body text,
request_parameter_name1 text,
request_parameter_name2 text,
request_parameter_name3 text,
request_parameter_name4 text,
request_parameter_name5 text,
request_parameter_value1 text,
request_parameter_value2 text,
request_parameter_value3 text,
request_parameter_value4 text,
request_parameter_value5 text,
response_body text,
response_parameter_name1 text,
response_parameter_name2 text,
response_parameter_name3 text,
response_parameter_name4 text,
response_parameter_name5 text,
response_parameter_value1 text,
response_parameter_value2 text,
response_parameter_value3 text,
response_parameter_value4 text,
response_parameter_value5 text,
responsestatus text,
responsestatuscode text,
transaction_id text,
PRIMARY KEY ((app_name, api_name, app_id), start_time)
);
Java code
public class SamplingDataJava {
private static Logger logger = LoggerFactory.getLogger(SamplingDataJava.class);
private static String startTime = "2017-03-21 00:00:00.000";
private static String endTime = "2017-04-25 00:00:00.000";
private final String SELECT_STATEMENT = "select start_time,duration from transactions_data "
+ " where app_name='app_name-abc' and api_name='api_name-1' "
+ " and app_id='app_id-xyz' " + " AND start_time>='"
+ startTime + "' AND start_time<='" + endTime + "' ";
private Cluster cluster;
private Session session;
private String Host = "localhost";
public SamplingDataJava() throws IOException {
// this.query=query;
logger.info("Using CQL3 Writer");
cluster = Cluster.builder().addContactPoints(Host)
.withSocketOptions(new SocketOptions().setConnectTimeoutMillis(2000000)).build();
session = cluster.connect();
}
private class Result {
double duration;
int count;
Result(double duration, int count) {
this.duration = duration;
this.count = count;
}
#Override
public String toString() {
return "Result [duration=" + duration + ", count=" + count + "]";
}
}
public void hashSampling(long interval,long initTime) throws IOException {
HashMap<Long, Result> agg = new HashMap<>();
ResultSet rs = session.execute(SELECT_STATEMENT);
int i = 0;
for (com.datastax.driver.core.Row row : rs) {
i++;
Long hashcode = Math.abs((row.getTimestamp("start_time").getTime() - initTime) / interval);
Result hasResult = agg.get(hashcode);
if (hasResult == null) {
hasResult = new Result(row.getInt("duration"), 1);
} else {
hasResult.duration = (hasResult.duration + row.getInt("duration"));
hasResult.count++;
}
agg.put(hashcode, hasResult);
}
System.out.println("total number of records " + i);
Long code=0L;
while (code<30) {
System.out.println(" code "+agg.get(code));
code++;
}
}
public void close() {
cluster.close();
session.close();
}
public static void main(String[] args) throws IOException {
long beginTime = System.currentTimeMillis();
SamplingDataJava cqp = new SamplingDataJava();
long onlyQueryTime = System.currentTimeMillis();
DateTimeFormatter readPattern = DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss.SSS");
DateTime sTime = readPattern.parseDateTime(startTime);
DateTime eTime = readPattern.parseDateTime(endTime);
long interval = (eTime.getMillis() - sTime.getMillis()) / 30;
System.out.println("start end time :" + eTime.getMillis() + " " + sTime.getMillis());
cqp.hashSampling(interval,sTime.getMillis());
System.out.println("total time without open close " + (System.currentTimeMillis() - onlyQueryTime));
cqp.close();
System.out.println("total time " + (System.currentTimeMillis() - beginTime));
}
}
UDF code
CREATE OR REPLACE FUNCTION txn_group_count_and_sum( txn map<bigint,
frozen<tuple<int,int>>>, start_time bigint, duration int , sample_size
bigint, begin_time bigint )
RETURNS NULL ON NULL INPUT
RETURNS map<bigint, frozen<tuple<int,int>>>
LANGUAGE java AS '
Long hashcode = (start_time - begin_time)/sample_size;
TupleValue tupleValue = txn.get(hashcode);
if (tupleValue == null) {
com.datastax.driver.core.TupleType tupleType =
com.datastax.driver.core.TupleType.of(
com.datastax.driver.core.ProtocolVersion.
NEWEST_SUPPORTED, com.datastax.driver.core.CodecRegistry.DEFAULT_INSTANCE,
com.datastax.driver.core.DataType.cint(),
com.datastax.driver.core.DataType.cint());
tupleValue = tupleType.newValue(1, duration );
}else{
tupleValue.setInt(0, tupleValue.getInt(0) + 1);
tupleValue.setInt(1, tupleValue.getInt(1) + duration);
}
txn.put(hashcode, tupleValue);
return txn; ' ;
CREATE OR REPLACE AGGREGATE group_count_and_sum(bigint, int ,bigint, bigint)
SFUNC txn_group_count_and_sum
STYPE map<bigint, frozen<tuple<int,int>>>
INITCOND {};
query
select group_count_and_sum(toUnixTimestamp(start_time),duration,100800000,1490054400000) from transactions_data
where app_name='app_name-abc' and api_name='api_name-1'
and app_id='app_id-xyz'
AND start_time>='2017-03-21 00:00:00.000' AND start_time<='2017-04-25 00:00:00.000';
Note:-
100800000 = (end_time - start_time)/30
1490054400000 = millisecond of 2017-03-21 00:00:00.000
I have a constructor below. How do I write an array of Event like
Event[] September = new Event[30];
Event[] October = new Event[31];
How do i produce an output such as this
class Event
{
private String TimeofEvent;
private String startingTime;
private int priority;
public Event(String TimeofEvent, String startingTime, int priority)
{
this.TimeofEvent = TimeofEvent;
this.startingTime = startingTime;
this.priority = (int)(Math.random() *3) + 1;
}
public String toString()
{
return "TimeofEvent: " + TimeofEvent + " " + "Starting Time: " + startingTime + " " + "Priority: " + " " +priority;
}
You could attempt to use a HashMap to store your Objects if that is the issue.
To get the output right I would play around with the GridLayout
I have an application that stores information in a database based on the information from 5 text fields. I'm having trouble being able to sort through fields or just even sorting from one field. Also, what method does the ORDER BY clause go in? Snippets of code for the database is below:
public static final String MYDATABASE_NAME = "MY_DATABASE";
public static final String MYDATABASE_TABLE = "MY_TABLE";
public static final int MYDATABASE_VERSION = 1;
public static final String KEY_CONTENT = "Content";
public static final String lastName = "lastName";
public static final String firstName = "firstName";
public static final String school = "school";
public static final String email = "email";
public static final String intrest = "intrest";
Cursor d = sqLiteDatabase.rawQuery("SELECT * from " + MYDATABASE_TABLE + "ORDER BY " + lastName + " ASC" , null);
while (d.moveToNext()) {
int sort = d.getColumnIndex(lastName);
int sort2 = d.getColumnIndex(intrest);
String lastName = d.getString(sort);
String intrest = d.getString(sort2);
System.out.println("GOT STUDENT " + lastName + " LIKES " + intrest);
}
Try to use (cursor.getCount() > 0) method and (cursor != null) condition in your code to check whether you gets data or not. That I have shown below. and try to use Log for your query for the perfection and put that query in your sqlite database to check whether it is working properly or not.
Cursor d = sqLiteDatabase.rawQuery("SELECT * from " + MYDATABASE_TABLE + " ORDER BY " + lastName + " ASC" , null); // here I have edited space before ORDER BY word starts. Please note this
Log.d("query",">>>>"+SELECT * from " + MYDATABASE_TABLE + " ORDER BY " + lastName + " ASC"); // check this query is going to right way or not using local database.
if(d != null){
if(d.getCount() > 0){ // to check you get one or more data
d.moveToFirst();
do{
int sort = d.getColumnIndex(lastName);
int sort2 = d.getColumnIndex(intrest);
String lastName = d.getString(sort);
String intrest = d.getString(sort2);
System.out.println("GOT STUDENT " + lastName + " LIKES " + intrest);
} while (d.moveToNext());
}
}
Try this. Hope it will help you.
I like query method better, because it's more clear:
public static final String MYDATABASE_NAME = "MY_DATABASE";
public static final String MYDATABASE_TABLE = "MY_TABLE";
public static final int MYDATABASE_VERSION = 1;
public static final String KEY_CONTENT = "Content";
public static final String lastName = "lastName";
public static final String firstName = "firstName";
public static final String school = "school";
public static final String email = "email";
public static final String interest = "interest";
public final String[] columns = new String[] {
lastName,
firstName,
school,
email,
interest
};
public void printInOrder(String column) {
Cursor d = sqLiteDatabase.query(MYDATABASE_TABLE, columns, null, null, null, null, column + " ASC");
while (d.moveToNext()) {
int sort = d.getColumnIndex(lastName);
int sort2 = d.getColumnIndex(interest);
String lastName = d.getString(sort);
String interest = d.getString(sort2);
System.out.println("GOT STUDENT " + lastName + " LIKES " + interest);
}
}