I am new to Cassandra and use Cassandra 3.10 and have table like
create table db1.table1 (id text, trip_id text, event_time timestamp, mileage double, primary key(id, event_time));
create table db1.table2 (id text, trip_id text, start_time timestamp, mileage double, primary key(id, start_time));
I need to transfer data from table1 to table2 aggregated by trip_id and sum on mileage and update data in table2
I have written a trigger function to get column name and value
public Collection<Mutation> augment(Partition partition) {
HashMap map = new HashMap();
CFMetaData cfm = partition.metadata();
String tableName = cfm.cfName;
try {
UnfilteredRowIterator it = partition.unfilteredIterator();
while (it.hasNext()) {
Unfiltered un = it.next();
Clustering clt = (Clustering) un.clustering();
Iterator<Cell> cells = partition.getRow(clt).cells().iterator();
while(cells.hasNext()){
Cell cell = cells.next();
map.put(cell.column().name.toString(), cell.value().array());
...
}
}
} catch (Exception e) {
}
...
}
But how can I get Primary key and the value of Primary key? If those are not gettable, how can I use trigger function to do the job?
Yes, It is possible to get primary key and value
To get partition keys column and value use :
List<ColumnDefinition> partitionKeyColumns = cfm.partitionKeyColumns();
ByteBuffer partitionKeyValues = partition.partitionKey().getKey();
To get clustering keys column and value :
List<ColumnDefinition> clusteringKeyColumns = cfm.clusteringColumns();
ByteBuffer[] clusteringKeyValues = clt.getRawValues();
Related
I'm very new to using databases and SQL in general and I'm having some trouble figuring out a function that will allow me to display records from a table in my jdbc database based on data from other tables in the database. I will illustrate below:
Example of "DEMANDS" table (column headers, "ID" is the primary key):
NAME|ADDRESS|DESTINATION|DATE|TIME|ID
Example of "DRIVERS" table ("REGISTRATION" is the primary key):
USERNAME|PASSWORD|REGISTRATION|NAME
Example of "JOURNEY" table ("JID" is the primary key,"REGISTRATION" is a foreign key)
JID|NAME|ADDRESS|DESTINATION|DISTANCE|REGISTRATION|DATE|TIME|STATUS
Below is the code that I have that is used to display tables on a jsp file:
public String retrieve(String query) throws SQLException {
select(query);
return makeTable(rsToList());//results;
}
private void select(String query){
try {
statement = connection.createStatement();
rs = statement.executeQuery(query);
//statement.close();
}
catch(SQLException e) {
System.out.println("way way"+e);
//results = e.toString();
}
}
private String makeTable(ArrayList list) {
StringBuilder b = new StringBuilder();
String[] row;
b.append("<table border=\"3\">");
for (Object s : list) {
b.append("<tr>");
row = (String[]) s;
for (String row1 : row) {
b.append("<td>");
b.append(row1);
b.append("</td>");
}
b.append("</tr>\n");
} // for
b.append("</table>");
return b.toString();
}//makeHtmlTable
private ArrayList rsToList() throws SQLException {
ArrayList aList = new ArrayList();
ResultSetMetaData metaData = rs.getMetaData();
int count = metaData.getColumnCount(); //number of column
String columnName[] = new String[count];
for (int i = 1; i <= count; i++)
{
columnName[i-1] = metaData.getColumnLabel(i);
}
aList.add(columnName);
int cols = rs.getMetaData().getColumnCount();
while (rs.next()) {
String[] s = new String[cols];
for (int i = 1; i <= cols; i++) {
s[i-1] = rs.getString(i);
}
aList.add(s);
} // while
return aList;
} //rsToList
All of this code works fine and if I pass in a query into the "Retrieve" function such as:
String query = "select * from DRIVERS";
It will display all of the records of the "DRIVERS" table.
What I am wanting to do though, is only list drivers from the driver table that are available at the time specified in the demand (meaning their registration is not currently in a record in the journey table at the same time as the demand) If possible, I would also only like to display the "NAME" and "REGISTRATION" columns as oppose to the whole record.
I would really appreciate some help with this as I've searched around for solutions for quite some time and have not been able to work out a function that will achieve the desired outcome.
Cheers,
Creation of tables script:
-- --------------------------------------------------------
--DROP Table Demands;
CREATE TABLE Demands (
Name varchar(20),
Address varchar(60),
Destination varchar(60),
Date date DEFAULT NULL,
Time time DEFAULT NULL,
Status varchar(15) NOT NULL,
id INT primary key
);
-- --------------------------------------------------------
--DROP Table Drivers;
CREATE TABLE Drivers (
username varchar(20),
password varchar(20),
Registration varchar(10),
Name varchar(20),
PRIMARY KEY (Registration)
);
-- --------------------------------------------------------
--DROP Table Journey;
CREATE TABLE Journey (
jid INT primary key
Destination varchar(60),
Distance integer NOT NULL DEFAULT 1,
Registration varchar(10) NOT NULL,
Date date NOT NULL,
Time time DEFAULT NULL
);
The following query may answer your question.
SELECT Drivers.Name, Drivers.Registration
FROM Drivers
LEFT JOIN Journey ON Journey.Registration = Drivers.Registration
LEFT JOIN Demands ON Demands.Date = Journey.Date
WHERE Demands.id IS NULL;
This joins JOURNEY and DRIVER based on the foreign key relation. It then outer-joins DEMANDS and JOURNEY based on an implicit relation that is DATE. Finally we only keep records that fail the outer join condition.
The model has a major flaw though as the relation between DEMANDS and JOURNEY is based on a field of type Date, as far as one can tell by what your provided.
I have a DynamoDB table that contains videos info.
Currently "videoID"is the primary (hash) key and "Category" is the range (sort) key.
I want to get a list of all of the "Categories" (Range keys) so I can allow the user to select from one of the available video categories.
https://www.quora.com/What-are-some-good-ways-to-extract-one-single-column-from-a-DynamoDB-table
I was reading that if you modified change the attribute "Category" to a global secondary index you can return the items for that GSI. But I have not been able to find how to do that.
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GSIJavaDocumentAPI.html
So I guess that gives me three questions:
Is there a way to do to find the items in Category by querying just the range key?
If change Category to a GSI can I fiind the items that way?
or
Is the only way of doing it scanning the whole table?
Thanks in advance for your help
Is the only way of doing it scanning the whole table?
-NO, you can implement GSI to avoid it
Is there a way to do to find the items in Category by querying just the range key?
- Yes, If you don't want to scan entire table then you need to create GSI which will have Category as Hash. This GSI will act as a table in itself and you can query on it by passing category values.
If change Category to a GSI can I find the items that way?
-Yes, you can query on GSI with category values
I was reading that if you modified change the attribute "Category" to a global secondary index you can return the items for that GSI. But I have not been able to find how to do that.
-You need to create GSI when you create table, example is given in the link that you have specified once that is done you can query that GSI
References:http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GSI.html
Here is the sample code to create Videos table with GSI.
Create "Videos" table with GSI:-
#Autowired
private AmazonDynamoDBClient dynamoDBClient;
public Boolean createTableWithGlobalSecondaryIndex(String tableName) {
CreateTableRequest createTableRequest = null;
DynamoDB dynamoDB = new DynamoDB(dynamoDBClient);
try {
ArrayList<AttributeDefinition> attributeDefinitions = new ArrayList<AttributeDefinition>();
attributeDefinitions.add(new AttributeDefinition().withAttributeName("videoid").withAttributeType("S"));
attributeDefinitions.add(new AttributeDefinition().withAttributeName("category").withAttributeType("S"));
ArrayList<KeySchemaElement> keySchema = new ArrayList<KeySchemaElement>();
keySchema.add(new KeySchemaElement().withAttributeName("videoid").withKeyType(KeyType.HASH));
keySchema.add(new KeySchemaElement().withAttributeName("category").withKeyType(KeyType.RANGE));
// Initial provisioned throughput settings for the indexes
ProvisionedThroughput ptIndex = new ProvisionedThroughput().withReadCapacityUnits(150L)
.withWriteCapacityUnits(150L);
GlobalSecondaryIndex videoCategoryGsi = new GlobalSecondaryIndex().withIndexName("VideoCategoryGsi")
.withProvisionedThroughput(ptIndex)
.withKeySchema(new KeySchemaElement().withAttributeName("category").withKeyType(KeyType.HASH),
new KeySchemaElement().withAttributeName("videoid").withKeyType(KeyType.RANGE))
.withProjection(new Projection().withProjectionType(ProjectionType.ALL));
createTableRequest = new CreateTableRequest().withTableName(tableName).withKeySchema(keySchema)
.withAttributeDefinitions(attributeDefinitions)
.withProvisionedThroughput(
new ProvisionedThroughput().withReadCapacityUnits(100L).withWriteCapacityUnits(100L))
.withGlobalSecondaryIndexes(videoCategoryGsi);
Table table = dynamoDB.createTable(createTableRequest);
table.waitForActive();
} catch (ResourceInUseException re) {
if (re.getErrorMessage().equalsIgnoreCase("Cannot create preexisting table")) {
LOGGER.info("Table already exists =============>" + tableName);
} else if (re.getErrorMessage().contains("Table already exists")) {
LOGGER.info("Table already exists =============>" + tableName);
LOGGER.info("Message =============>" + re.getErrorCode() + ";" + re.getErrorMessage());
} else {
throw new RuntimeException("DynamoDB table cannot be created ...", re);
}
} catch (Exception db) {
throw new RuntimeException("DynamoDB table cannot be created ...", db);
}
return true;
}
Query GSI by category:-
Here is the input is just category and it is querying using GSI. In other words, it is not scanning the entire table as well.
public List<String> findVideosByCategoryUsingGlobalSecondaryIndex(String category) {
List<String> videoAsJson = new ArrayList<>();
DynamoDB dynamoDB = new DynamoDB(dynamoDBClient);
Table table = dynamoDB.getTable("Videos");
Index index = table.getIndex("VideoCategoryGsi");
ItemCollection<QueryOutcome> items = null;
QuerySpec querySpec = new QuerySpec();
querySpec.withKeyConditionExpression("category = :val1")
.withValueMap(new ValueMap()
.withString(":val1", category));
items = index.query(querySpec);
Iterator<Item> pageIterator = items.iterator();
while (pageIterator.hasNext()) {
String videoJson = pageIterator.next().toJSON();
System.out.println("Video json ==================>" + videoJson);
videoAsJson.add(videoJson);
}
return videoAsJson;
}
I am working with Cassandra and I am using Hector client to read and upsert the data in Cassandra database. I am trying to retrieve the data from Cassandra database using hector client and I am able to do that if I am trying to retrieve only one column.
Now I am trying to retrieve the data for rowKey as 1011 but with columnNames as collection of string. Below is my API that will retrieve the data from Cassandra database using Hector client-
public Map<String, String> getAttributes(String rowKey, Collection<String> attributeNames, String columnFamily) {
final Cluster cluster = CassandraHectorConnection.getInstance().getCluster();
final Keyspace keyspace = CassandraHectorConnection.getInstance().getKeyspace();
try {
ColumnQuery<String, String, String> columnQuery = HFactory
.createStringColumnQuery(keyspace)
.setColumnFamily(columnFamily).setKey(rowKey)
.setName("c1");
QueryResult<HColumn<String, String>> result = columnQuery.execute();
System.out.println("Column Name from cassandra: " + result.get().getName() + "Column value from cassandra: " + result.get().getValue());
} catch (HectorException e) {
LOG.error("Exception in CassandraHectorClient::getAttributes " +e+ ", RowKey = " +rowKey+ ", Attribute Names = " +attributeNames);
} finally {
cluster.getConnectionManager().shutdown();
}
return null;
}
If you see my above method, I am trying to retrieve the data from Cassandra database for a particular rowKey and for column c1. Now I am trying to retrieve the data from Cassandra database for collection of columns for a particular rowKey.
Meaning something like this-
I want to retrieve the data for multiple columns but for the same rowKey. How can I do this using Hector client? And I don't want to retrieve the data for all the columns and then iterate to find out the individual columns data I am looking for.
Use column name made up with composite key as combination of UTF8Type and TIMEUUID
then after
sliceQuery.setKey("your row key");
Composite startRange = new Composite();
startRange.addComponent(0, "c1",Composite.ComponentEquality.EQUAL);
Composite endRange = new Composite();
endRange.addComponent(0, "c1",Composite.ComponentEquality.GREATER_THAN_EQUAL);
sliceQuery.setRange(startRange,endRange, false, Integer.MAX_VALUE);
QueryResult<ColumnSlice<Composite, String>> result = sliceQuery.execute();
ColumnSlice<Composite, String> cs = result.get();
above code will give you all records for you row key
after that iterate as follows
for (HColumn<Composite, String> col : cs.getColumns()) {
System.out.println("column key's first part : "+col.getName().get(0, HFactoryHelper.stringSerializer).toString());
System.out.println("column key's second part : "+col.getName().get(1, HFactoryHelper.uuidSerializer).toString());
System.out.println("column key's value : "+col.getValue());
}
some where you have to write logic to maintain set of records
Hi I want to insert into this kind of column family row with composite key:
CREATE TABLE my_items (
user_id uuid,
item_id uuid,
description varchar,
PRIMARY KEY (user_id, item_id));
So I try this:
StringSerializer stringSerializer = StringSerializer.get();
UUIDSerializer uuidSerializer = UUIDSerializer.get();
CompositeSerializer compositeSerializer = CompositeSerializer.get();
HColumn<String, UUID> hColumnObj_userID = HFactory.createColumn("user_id", userID, stringSerializer, uuidSerializer);
HColumn<String, UUID> hColumnObj_itemID= HFactory.createColumn("item_id", itemID, stringSerializer, uuidSerializer);
Mutator<Composite> mutator = HFactory.createMutator(
repository.getKeyspace(),
compositeSerializer);
Composite colKey = new Composite();
colKey.addComponent(userID, uuidSerializer);
colKey.addComponent(itemID, uuidSerializer);
mutator.addInsertion(colKey,
"my_items", hColumnObj_userID);
mutator.addInsertion(colKey,
"my_items", hColumnObj_itemID);
mutator.execute();
What's wrong with code above? I keep getting this error: "InvalidRequestException(why:UUIDs must be exactly 16 bytes)". And how can I insert data into column family that I describe above.
Cheers
It looks like Hector was expecting a Composite containing a UUID and a String and found only a string.
Before writing the Hector code you have to translate the create DDL into the actual storage pattern CQL uses. In this case, even though you have two primary keys, only the first, user_id, is used as the row key. That's always the case. Any other primary keys (item_id in this case) are used to form composite column names for every column except the first primary key. That means that when using Hector for your my_items column family you'll have to write two columns, one for item_ID and one for description.
The column name for the item_id value is a composite consisting of the values of primary keys 2...n (item_id in this example) and a constant string name of the value ("item_id").
The column name for the description value is also a composite of the item_id value and the name of the value ("description").
If you wrote 3 CQL table rows, each with the same user_id but having different item_id values then you'd end up with a single column family row whose row key is the common user_id value and which has 6 columns, an item_id column and a description column for each of the 3 CQL table rows.
The code should look like this:
import java.util.UUID;
import me.prettyprint.cassandra.serializers.CompositeSerializer;
import me.prettyprint.cassandra.serializers.IntegerSerializer;
import me.prettyprint.cassandra.serializers.StringSerializer;
import me.prettyprint.cassandra.serializers.UUIDSerializer;
import me.prettyprint.hector.api.Keyspace;
import me.prettyprint.hector.api.beans.Composite;
import me.prettyprint.hector.api.beans.HColumn;
import me.prettyprint.hector.api.beans.AbstractComposite.ComponentEquality;
import me.prettyprint.hector.api.factory.HFactory;
import me.prettyprint.hector.api.mutation.Mutator;
// put this here to make it compile cleanly
Keyspace keyspace = null;
UUID userID = null;
UUID itemID = null;
String description = null;
// Row key is user_id of type UUID
Mutator<UUID> mutator = HFactory.createMutator(
keyspace,
UUIDSerializer.get());
// write column for itemID.
// Column name is composite of itemID value and constant "item_id"
// Row key is value of userID
Composite itemIdColumnName = new Composite();
itemIdColumnName.addComponent(itemID , UUIDSerializer.get());
itemIdColumnName.addComponent("item_id" , StringSerializer.get());
// HFactory.createColumn takes args: column name, column value, serializer for column name, serializer for column value
HColumn<Composite, UUID> hColumnObj_itemID = HFactory.createColumn(itemIdColumnName, userID, new CompositeSerializer(), UUIDSerializer.get());
mutator.addInsertion(userID, "my_items", hColumnObj_itemID);
// write column for description.
// Column name is composite of itemID value and constant "description"
// Row key is value of userID
Composite descriptionColumnName = new Composite();
itemIdColumnName.addComponent(itemID , UUIDSerializer.get());
itemIdColumnName.addComponent("description" , StringSerializer.get());
HColumn<Composite, String> hColumnObj_description = HFactory.createColumn(descriptionColumnName, description , new CompositeSerializer(), StringSerializer.get());
mutator.addInsertion(userID, "my_items", hColumnObj_description);
mutator.execute();
How delete row from table with help jackcess?
I try so, but it's bad:
Table ptabl = db.getTable("person");
int pcount = ptabl.getRowCount();
for (int i = 0; i < pcount; i++) {
Map<String, Object> row2 = ptabl.getNextRow();
if (row2.get("id") == Integer.valueOf(1)) {
ptabl.deleteCurrentRow();
}
}
How set column "id" attribute to autoincrement?
Table newTable = new TableBuilder("diagnosis").
addColumn(new ColumnBuilder("id")
.setSQLType(Types.INTEGER)
.toColumn())
.addColumn(new ColumnBuilder("name")
.setSQLType(Types.VARCHAR)
.toColumn()).toTable(db);
If your id column is indexed, you can use an IndexCursor to quickly find columns:
IndexCursor cursor = new CursorBuilder(ptabl).setIndexByColumnNames("id").toIndexCursor();
if(cursor.findFirstRowByEntry(1)) {
cursor.deleteCurrentRow();
}
If your id column is not indexed, you can use a normal cursor, which is more convenient but effectively no faster than your current code (just does a table scan):
Cursor cursor = new CursorBuilder(ptab1).toCursor();
Column idCol = ptab1.getColumn("id");
if(cursor.findFirstRow(idCol, 1)) {
cursor.deleteCurrentRow();
}
And your own answer indicates you already figured out how to make a column auto increment.
For set autoincrement to column:
Table newTable = new TableBuilder("diagnosis").addColumn(new ColumnBuilder("id").setAutoNumber(true).setSQLType(Types.INTEGER).toColumn()).addColumn(new ColumnBuilder("name").setSQLType(Types.VARCHAR).toColumn()).toTable(db);