I'm trying to access only partition key of an existing collection in cosmos db
I have gone through official doc and git hub links but no luck. Please help me.
Note: I'm able to successfully read and write data using java by using Document Client(SQL api)
Please try this code:
DocumentClient dClient = new DocumentClient(endPoint,primary_key,null,null);
String collectionLink = String.format("/dbs/%s/colls/%s", databaseId, collectionId);
ResourceResponse<DocumentCollection> response = dClient.readCollection(collectionLink,new RequestOptions());
DocumentCollection documentCollection = response.getResource();
PartitionKeyDefinition partitionKeyDefinition = documentCollection.getPartitionKey();
Collection<String> paths = partitionKeyDefinition.getPaths();
if(paths.iterator().hasNext()){
System.out.println(paths.iterator().next());
}
I found this works:
_container is the db container for cosmos.
var settings = await _container!.ReadContainerAsync(cancellationToken: cancellationToken);
var partitionKeyPath = settings.Resource.PartitionKeyPath[1..];
var partitionProperty = typeof(T).GetProperty(partitionKeyPath);
var partitionKey = partitionProperty!.GetValue(message)!.ToString();
Related
I know that there's probably a better way to do this however I'm completely stumped. I'm writing a Discord bot in which a user is able to add points to other users, however I can't figure out how to replace a user's "points". My code is as follows:
BasicDBObject cursor = new BasicDBObject();
cursor.put(user.getAsMember().getId(), getMongoPoints(user.getAsMember()));
if(cursor.containsKey(user.getAsMember().getId())) {
Document old = new Document(user.getAsMember().getId(), getMongoPoints(user.getAsMember()));
Document doc = new Document(user.getAsMember().getId(), getMongoPoints(user.getAsMember()) + Integer.parseInt(amount.getAsString()));
collection.findOneAndUpdate(old, doc);
}
My getMongoPoints function:
public static int getMongoPoints(Member m) {
ConnectionString connectionString = new ConnectionString("database");
MongoClientSettings settings = MongoClientSettings.builder()
.applyConnectionString(connectionString)
.build();
MongoClient mongoClient = MongoClients.create(settings);
MongoDatabase database = mongoClient.getDatabase("SRU");
MongoCollection<Document> collection = database.getCollection("points");
DistinctIterable<Integer> docs = collection.distinct(m.getId(), Integer.class);
MongoCursor<Integer> result = docs.iterator();
return result.next();
}
I've tried findOneAndReplace, however that simply makes a new entry without deleting the old one. The error I receive is: Invalid BSON field name 262014495440896000
Everything else works, include writing to the database itself which is why I'm stumped. Any help would be greatly appreciated and I apologize if this is written poorly.
BSON field names must be string. From the spec:
Zero or more modified UTF-8 encoded characters followed by '\x00'. The (byte*) MUST NOT contain '\x00', hence it is not full UTF-8.
To use 262014495440896000 as a field name, convert it to string first.
I'm on a heatmap project for my university, we have to get some data (212Go) from a txt file (coordinates, height), then put it in HBase to retrieve it on a web client with Express.
I practiced using a 144Mo file, this is working :
SparkConf conf = new SparkConf().setAppName("PLE");
JavaSparkContext context = new JavaSparkContext(conf);
JavaRDD<String> data = context.textFile(args[0]);
Connection co = ConnectionFactory.createConnection(getConf());
createTable(co);
Table table = co.getTable(TableName.valueOf(TABLE_NAME));
Put put = new Put(Bytes.toBytes("KEY"));
for (String s : data.collect()) {
String[] tmp = s.split(",");
put.addImmutable(FAMILY,
Bytes.toBytes(tmp[2]),
Bytes.toBytes(tmp[0]+","+tmp[1]));
}
table.put(put);
But I now that I use the 212Go file, I got some memory errors, I guess the collect method gather all the data in memory, so 212Go is too much.
So now I'm trying this :
SparkConf conf = new SparkConf().setAppName("PLE");
JavaSparkContext context = new JavaSparkContext(conf);
JavaRDD<String> data = context.textFile(args[0]);
Connection co = ConnectionFactory.createConnection(getConf());
createTable(co);
Table table = co.getTable(TableName.valueOf(TABLE_NAME));
Put put = new Put(Bytes.toBytes("KEY"));
data.foreach(line ->{
String[] tmp = line.split(",");
put.addImmutable(FAMILY,
Bytes.toBytes(tmp[2]),
Bytes.toBytes(tmp[0]+","+tmp[1]));
});
table.put(put);
And I'm getting "org.apache.spark.SparkException: Task not serializable", I searched about it and tried some fixing, without success, upon what I read here : Task not serializable: java.io.NotSerializableException when calling function outside closure only on classes not objects
Actually I don't understand everything in this topic, I'm just a student, maybe the answer to my problem is obvious, maybe not, anyway thanks in advance !
As a rule of thumb - serializing database connections (any type) doesn't make sense. There are not designed to be serialized and deserialized, Spark or not.
Create connection for each partition:
data.foreachPartition(partition -> {
Connection co = ConnectionFactory.createConnection(getConf());
... // All required setup
Table table = co.getTable(TableName.valueOf(TABLE_NAME));
Put put = new Put(Bytes.toBytes("KEY"));
while (partition.hasNext()) {
String line = partition.next();
String[] tmp = line.split(",");
put.addImmutable(FAMILY,
Bytes.toBytes(tmp[2]),
Bytes.toBytes(tmp[0]+","+tmp[1]));
}
... // Clean connections
});
I also recommend reading Design Patterns for using foreachRDD from the official Spark Streaming programming guide.
I need to create BulkConnection for bulk API into salesforce.
We can able to create BulkConnection using ConnectorConfig with basic SOAP authentication.
Code is below,
ConnectorConfig config = new ConnectorConfig();
config.setUsername("USERNAME");
config.setPassword("PASSWORD+TOKEN");
config.setCompression(true);
config.setTraceFile("traceLogs.txt");
config.setTraceMessage(true);
config.setPrettyPrintXml(true);
config.setAuthEndpoint("https://login.salesforce.com/services/Soap/u/39.0");
PartnerConnection connection = new PartnerConnection(config);
String soapEndpoint = config.getServiceEndpoint();
String apiVersion = "39.0";
String restEndpoint = soapEndpoint.substring(0, soapEndpoint.indexOf("Soap/")) + "async/" + apiVersion;
config.setRestEndpoint(restEndpoint);
bulkConnection = new BulkConnection(config);
Above code is working fine.
But in my case, I can't able to collect username & password for each of the customer's salesforce account.
Is this possible to create BulkConnection using accestoken?..
Please give me your suggestion.
Thanks,
Harish
i am doing a task converting VB script written from Powerbuild to java,
i am struggled at converting the DataStore Object into java ,
i have something like this :
lds_appeal_application = Create DataStore
lds_appeal_application.DataObject = "ds_appeal_application_report"
lds_appeal_application.SetTransObject(SQLCA)
ll_row = lds_appeal_application.retrieve(as_ksdyh, adt_start_date, adt_end_date, as_exam_name, as_subject_code)
for ll_rc = 1 to ll_row
ldt_update_date = lds_appeal_application.GetItemDatetime(ll_rc, "sqsj")
ls_caseno = trim(lds_appeal_application.GetItemString(ll_rc, "caseno"))
ls_candidate_no = trim(lds_appeal_application.GetItemString(ll_rc, "zkzh"))
ls_subjectcode = trim(lds_appeal_application.GetItemString(ll_rc, "kmcode"))
ls_papercode = trim(lds_appeal_application.GetItemString(ll_rc, "papercode"))
ls_name = trim(lds_appeal_application.GetItemString(ll_rc, "mc"))
ll_ksh = lds_appeal_application.GetItemDecimal(ll_rc, "ks_h")
ll_kmh = lds_appeal_application.GetItemDecimal(ll_rc, "km_h")
simply speaking, a datasoure is created and a data table is point to it by sql query(ds_appeal_application_report). Finally using a for loop to retrieve information from the table.
in java way of doing, i use an entities manager to createnativequery and the query can result a list of object array. However, i just dont know how to retrieve the information like VB using the DataStore Object.
please give me some advice . Thanks
I am a student and I'm new to Filenet. I am trying to do the test code on file moving.
Document doc = Factory.Document.getInstance(os, ClassNames.DOCUMENT, new Id("{33074B6E-FD19-4C0D-96FC-D809633D35BF}") );
FileStorageArea newDocClassFSA = Factory.FileStorageArea.fetchInstance(os, new Id("{3C6CEE68-D8CC-44A5-AEE7-CADE9752AA77}"), null );
doc.moveContent(dsa);
doc.save(RefreshMode.REFRESH);
The thing is that I can fetch the document by its path like this ,
doc = Factory.Document.fetchInstance(os, "/DEMO/MASTERFILE/ZONE-X/Org.No-XXXXX/XXXX-X-XXXX-X.TIF",null);
but I can't fetch the StorageArea by the path, it only takes ID. Is there a way to move a file easily than this? How can I get ID with path without using queries?
There is no other way to fetch FileStorageArea by its path other than issue a query and filter by RootDirectoryPath property.
You can access the ID of the document using the path:
//Get ID of the Document
StringBuffer propertyNames = new StringBuffer();
propertyNames.append(PropertyNames.ID);
propertyNames.append(" ");
propertyNames.append(PropertyNames.PATH_NAME);
PropertyFilter pf=new PropertyFilter();
FilterElement felement= new FilterElement(Integer.valueOf(0),Long.valueOf(0),Boolean.TRUE,propertyNames.toString(),Integer.valueOf(0));
pf.addIncludeProperty(felement);
Document document = Factory.Document.fetchInstance(os, ruta, pf );
idDocument = document.get_Id().toString();
and in the idDocument string you have it. Hope it helps.