Is there any way to list all tables present in a specific Cache and list all caches present on a Apache Ignite Server?
----------------------------------UPDATED--------------------------
Hi,
I am running following code to know Cache name and list all tables present in my cache. This program list outs all cache name present on server. However table listing is printed as blank collection. Meanwhile SQL query present in example is working fine.
public static void main(String[] args) throws Exception {
System.out.println("Run Spring example!!");
Ignition.setClientMode(true);
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setIncludeEventTypes( EVTS_CACHE);
cfg.setPeerClassLoadingEnabled(true);
TcpDiscoveryMulticastIpFinder discoveryMulticastIpFinder = new TcpDiscoveryMulticastIpFinder();
Set<String> set = new HashSet<>();
set.add("hostname:47500..47509");
discoveryMulticastIpFinder.setAddresses(set);
TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi();
discoverySpi.setIpFinder(discoveryMulticastIpFinder);
cfg.setDiscoverySpi(discoverySpi);
cfg.setPeerClassLoadingEnabled(true);
cfg.setIncludeEventTypes(EVTS_CACHE);
Ignite ignite = Ignition.start(cfg);
System.out.println("All Available Cache on server : "+ignite.cacheNames());
CacheConfiguration<String, BinaryObject> cacheConfiguration = new CacheConfiguration<>(CACHE_NAME);
Collection<QueryEntity> entities = cacheConfiguration.getQueryEntities();
System.out.println("All available tables in cache : "+entities);
cacheConfiguration.setIndexedTypes(String.class, BinaryObject.class);
//cacheConfiguration.setCacheMode(CacheMode.PARTITIONED);
IgniteCache<String, BinaryObject> cache = ignite.getOrCreateCache(cacheConfiguration).withKeepBinary();
System.out.println();
QueryCursor<List<?>> query = cache.query(new SqlFieldsQuery("select Field1 from table1 where Field1='TEST'"));
List<List<?>> all = query.getAll();
for (List<?> l : all) {
System.out.println(l);
}
}
Get all cache names: Ignite.cacheNames(). Then use Ignite.cache(String) to get the cache instance.
Get SQL tables:
CacheConfiguration ccfg = cache.getConfiguration(CacheConfiguration.class);
Collection<QueryEntity> entities = ccfg.getQueryEntities();
Each query entity represents a table.
You can read using h2 query.
SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA is the cache name
ClientConfiguration cfg = new ClientConfiguration().setAddresses(host+":"+port).
setUserName(username).setUserPassword(pwd);
private static IgniteClient igniteClient = Ignition.startClient(cfg);
private static ClientCache<Integer, String>
cache=igniteClient.getOrCreateCache(cacheName);
QueryCursor<List<?>> cursor =cache.query(new SqlFieldsQuery("SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA='"+cacheName+"'"));
for (List<?> row : cursor) {
System.out.println(row.get(0));
}
You can get all cache names using Ignite.cacheNames(). In order to get all table names you can use SHOW TABLES command:
QueryCursor<List<?>> cursor = cache.query(new SqlFieldsQuery("SHOW TABLES FROM \""+CACHE_NAME+"\""));
for (List<?> row : cursor) {
System.out.println(row.get(0));
}
More details about the SHOW command you can find here: http://www.h2database.com/html/grammar.html#show
Related
I have the following code:
try (DB db = new DB()) {
db.open(cpds);
// Using the metadata to walk down the tree and identify all
// children nodes to the leaf tables.
// First the metadata for the pk of the target table
List<Integer> recordIdList = new ArrayList<Integer>();
recordIdList.add(recordId);
batchInsertDeletePks(recordIdList, db);
List<Integer> pkValues = new ArrayList<Integer>();
pkValues.add(recordId);
prepareChildDeletes(tablemeta_id, pkValues, recordId, db);
ColumnData.delete("RECORD_ID IN (SELECT ID FROM TEMPDELETE)");
noDeleted = RecordData.delete("ID IN (SELECT ID FROM TEMPDELETE)");
TempDelete.deleteAll();
} catch (Exception ex) {
logger.debug(ex.getMessage());
}
The TEMPDELETE has all of the primary keys I need to delete from COLUMNDATA. The ColumnData.delete is not working, the RecordData.delete is not working and the TempDelete.deleteAll is not working. They give no exceptions. The database is h2-1.4.196
If I debug trace into them and cut and paste the SQL that is created and run that SQL in a sql interpreter all the queries work just fine.
I cannot see what it is about my approach that is different to the examples? Any ideas?
I have a DynamoDB table that contains videos info.
Currently "videoID"is the primary (hash) key and "Category" is the range (sort) key.
I want to get a list of all of the "Categories" (Range keys) so I can allow the user to select from one of the available video categories.
https://www.quora.com/What-are-some-good-ways-to-extract-one-single-column-from-a-DynamoDB-table
I was reading that if you modified change the attribute "Category" to a global secondary index you can return the items for that GSI. But I have not been able to find how to do that.
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GSIJavaDocumentAPI.html
So I guess that gives me three questions:
Is there a way to do to find the items in Category by querying just the range key?
If change Category to a GSI can I fiind the items that way?
or
Is the only way of doing it scanning the whole table?
Thanks in advance for your help
Is the only way of doing it scanning the whole table?
-NO, you can implement GSI to avoid it
Is there a way to do to find the items in Category by querying just the range key?
- Yes, If you don't want to scan entire table then you need to create GSI which will have Category as Hash. This GSI will act as a table in itself and you can query on it by passing category values.
If change Category to a GSI can I find the items that way?
-Yes, you can query on GSI with category values
I was reading that if you modified change the attribute "Category" to a global secondary index you can return the items for that GSI. But I have not been able to find how to do that.
-You need to create GSI when you create table, example is given in the link that you have specified once that is done you can query that GSI
References:http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GSI.html
Here is the sample code to create Videos table with GSI.
Create "Videos" table with GSI:-
#Autowired
private AmazonDynamoDBClient dynamoDBClient;
public Boolean createTableWithGlobalSecondaryIndex(String tableName) {
CreateTableRequest createTableRequest = null;
DynamoDB dynamoDB = new DynamoDB(dynamoDBClient);
try {
ArrayList<AttributeDefinition> attributeDefinitions = new ArrayList<AttributeDefinition>();
attributeDefinitions.add(new AttributeDefinition().withAttributeName("videoid").withAttributeType("S"));
attributeDefinitions.add(new AttributeDefinition().withAttributeName("category").withAttributeType("S"));
ArrayList<KeySchemaElement> keySchema = new ArrayList<KeySchemaElement>();
keySchema.add(new KeySchemaElement().withAttributeName("videoid").withKeyType(KeyType.HASH));
keySchema.add(new KeySchemaElement().withAttributeName("category").withKeyType(KeyType.RANGE));
// Initial provisioned throughput settings for the indexes
ProvisionedThroughput ptIndex = new ProvisionedThroughput().withReadCapacityUnits(150L)
.withWriteCapacityUnits(150L);
GlobalSecondaryIndex videoCategoryGsi = new GlobalSecondaryIndex().withIndexName("VideoCategoryGsi")
.withProvisionedThroughput(ptIndex)
.withKeySchema(new KeySchemaElement().withAttributeName("category").withKeyType(KeyType.HASH),
new KeySchemaElement().withAttributeName("videoid").withKeyType(KeyType.RANGE))
.withProjection(new Projection().withProjectionType(ProjectionType.ALL));
createTableRequest = new CreateTableRequest().withTableName(tableName).withKeySchema(keySchema)
.withAttributeDefinitions(attributeDefinitions)
.withProvisionedThroughput(
new ProvisionedThroughput().withReadCapacityUnits(100L).withWriteCapacityUnits(100L))
.withGlobalSecondaryIndexes(videoCategoryGsi);
Table table = dynamoDB.createTable(createTableRequest);
table.waitForActive();
} catch (ResourceInUseException re) {
if (re.getErrorMessage().equalsIgnoreCase("Cannot create preexisting table")) {
LOGGER.info("Table already exists =============>" + tableName);
} else if (re.getErrorMessage().contains("Table already exists")) {
LOGGER.info("Table already exists =============>" + tableName);
LOGGER.info("Message =============>" + re.getErrorCode() + ";" + re.getErrorMessage());
} else {
throw new RuntimeException("DynamoDB table cannot be created ...", re);
}
} catch (Exception db) {
throw new RuntimeException("DynamoDB table cannot be created ...", db);
}
return true;
}
Query GSI by category:-
Here is the input is just category and it is querying using GSI. In other words, it is not scanning the entire table as well.
public List<String> findVideosByCategoryUsingGlobalSecondaryIndex(String category) {
List<String> videoAsJson = new ArrayList<>();
DynamoDB dynamoDB = new DynamoDB(dynamoDBClient);
Table table = dynamoDB.getTable("Videos");
Index index = table.getIndex("VideoCategoryGsi");
ItemCollection<QueryOutcome> items = null;
QuerySpec querySpec = new QuerySpec();
querySpec.withKeyConditionExpression("category = :val1")
.withValueMap(new ValueMap()
.withString(":val1", category));
items = index.query(querySpec);
Iterator<Item> pageIterator = items.iterator();
while (pageIterator.hasNext()) {
String videoJson = pageIterator.next().toJSON();
System.out.println("Video json ==================>" + videoJson);
videoAsJson.add(videoJson);
}
return videoAsJson;
}
I am using MongoDB with java 3.0 driver. In the manual I only found find() and findOne() which will give me all the document.
I have a scenario like I should get the _id value by querying. for eg in SQL select _id from table name.
{
"_id" : ObjectId("557660c074cd60207e337aed"),
"contactMethodId" : [
{
"contactMethodId" : "contactMethodId",
"contactMethodUsageTypeCode" : null,
"contactMethodTypeCode" : "contactMethodTypeCode",
"contactMethodValue" : "contactMethodValue",
"contactContentTypeCode" : "contactContentTypeCode",
"contactContentMaxSize" : "contactContentMaxSize",
"comment" : "comment",
"preferredInt" : "preferredInd",
"effectiveStartDateOfContact" : "effectiveStartDateOfContact",
"effectiveEndDateOfContact" : "effectiveEndDateOfContact",
"standardizedIndOfContact" : "standardizedIndOfContact",
"lastVerifiedDateOfContact" : "lastVerifiedDateOfContact"
}
]
}
_id is generated by default.
I want something like select _id from table name and I should get ObjectId("557660c074cd60207e337aed") in java. Any suggestions.
MongoIterable<Document> = mongoClient.getDatabase("foo")
.getCollection("bob")
.find()
.projection(new Document("_id", 1))
You can use following :
BasicDBObject projection = new BasicDBObject();
projection.put("_id",1);
DBCursor cursor = collection.find(new BasicDBObject(),projection);
And then read using loop on cursor.
while (cursor.hasNext()) {
System.out.println("Result : -"+cursor.next());
}
Check this code, tested with mongo version 3.0.3 and Mongo Java driver version 3.0.1
import com.mongodb.BasicDBObject;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.DBCursor;
import com.mongodb.Mongo;
public class demo {
public static void main(String[] args) throws Exception {
Mongo mongo = new Mongo("localhost", 27017); // set host and port of mongo
DB db = mongo.getDB("demo"); // set DB name
DBCollection collection = db.getCollection("collectionName"); // set collection Name
BasicDBObject query = new BasicDBObject();
BasicDBObject project = new BasicDBObject();
project.put("_id", 1); // set project to get _id
DBCursor cursorDoc = collection.find(query, project); // find query with projection
while(cursorDoc.hasNext()) {
BasicDBObject object = (BasicDBObject) cursorDoc.next();
String _id = object.get("_id").toString(); // If required convert the _id to String
System.out.println(object); // print _id with object
System.out.println(_id); // print _id as a String
}
}
}
I have 1.2M records at my MongoDB database. And I want to store all of this data at HBase programmatically. Basically I try to put each retrieved record to HBase in a loop. After the operation is finished, I got only 39912 records on HBase.
Here's what I've tried:
Configuration config = HBaseConfiguration.create();
String tableName = "storedtweet";
String familyName = "msg";
String qualifierName = "msg";
HTable table = new HTable(config, tableName);
// using Spring Data MongoDB to interact with MongoDB
List < StoredTweet > storedTweetList = mongoDAO.getMongoTemplate().findAll(StoredTweet.class);
for (StoredTweet storedTweet: storedTweetList) {
Put p = new Put(Bytes.toBytes(storedTweet.getTweetId()));
p.add(Bytes.toBytes(familyName), Bytes.toBytes(qualifierName), Bytes.toBytes(storedTweet.getMsg()));
table.put(p);
table.flushCommits();
}
If some row key exists and you put it again, HBase Put will override the former. I think there are some records having the same tweet id (you set it to the row key) in your data. That's why some records disappear.
Is there a way to retrieve the name of all tables that are managed by the SessionFactory? For instance, all the tables that were added via AnnotationConfiguration.addAnnotatedClass(...))?
Here is howto getting one tableName with getClassMetadata
ClassMetadata cm = sessionFactory.GetClassMetadata(className);
AbstractEntityPersister aep = (AbstractEntityPersister) cm;
String tableName = aep.getTableName();
[EDIT] : you can find all by calling getAllClassMetadata() and find all table names like that
Map m = sessionFactory.GetAllClassMetadata();
/* iterate map*/
AbstractEntityPersister aep = m.get(/*key (className)*/)
String tableName = aep.getTableName();
If you are using JPA instead of direct dependency on hibernate., following code should help in getting all table names
private List<String> getAllTables() {
List<String> tableNames = new ArrayList<>();
Session session = entityManager.unwrap(Session.class);
SessionFactory sessionFactory = session.getSessionFactory();
Map<String, ClassMetadata> map = (Map<String, ClassMetadata>) sessionFactory.getAllClassMetadata();
for(String entityName : map.keySet()){
SessionFactoryImpl sfImpl = (SessionFactoryImpl) sessionFactory;
String tableName = ((AbstractEntityPersister)sfImpl.getEntityPersister(entityName)).getTableName();
tableNames.add(tableName);
}
return tableNames;
}
sessionFactory.GetClassMetadata(className);
is deprecated. Use
Metamodel metamodel = entityManager.getMetamodel();
Set<EntityType<?>> entities = metamodel.getEntities();
entities.forEach(e -> {
System.out.println(e.getName());
});
Your can also get metamodel from SessionFactory
You can try using native sql queries.
session.createSQLQuery("SELECT * FROM user_tables").list();
which gives list of tables owned by loggedin user or else you can use 'all_tables' or 'dba_tables' for all the tables for oracle database.If mysql db is used then replace the query with "SHOW TABLES"