We Keep Coding

Is there any way to write custom or native queries in Java JPA (DocumentDbRepository) while firing a query to azure-cosmosdb?

Connected to azure-cosmosdb and able to fire default queries like findAll() and findById(String Id). But I can't write a native query using #Query annotation as the code is not considering it. Always considering the name of the function in respository class/interface. I need a way to fire a custom or native query to azure-cosmos db. ?!
Tried with #Query annotation. But not working.
List<MonitoringSessions> findBySessionID(#Param("sessionID") String sessionID);
#Query(nativeQuery = true, value = "SELECT * FROM MonitoringSessions M WHERE M.sessionID like :sessionID")
List<MonitoringSessions> findSessions(#Param("sessionID") String sessionID);
findBySessionID() is working as expected. findSessions() is not working. Below root error came while running the code.
Caused by: org.springframework.data.mapping.PropertyReferenceException: No property findSessions found for type MonitoringSessions

Thanks for the response. I got what I exactly wanted from the below link. Credit goes to Author of the link page.
https://cosmosdb.github.io/labs/java/technical_deep_dive/03-querying_the_database_using_sql.html
public class Program {
private final ExecutorService executorService;
private final Scheduler scheduler;
private AsyncDocumentClient client;
private final String databaseName = "UniversityDatabase";
private final String collectionId = "StudentCollection";
private int numberOfDocuments;
public Program() {
// public constructor
executorService = Executors.newFixedThreadPool(100);
scheduler = Schedulers.from(executorService);
client = new AsyncDocumentClient.Builder().withServiceEndpoint("uri")
.withMasterKeyOrResourceToken("key")
.withConnectionPolicy(ConnectionPolicy.GetDefault()).withConsistencyLevel(ConsistencyLevel.Eventual)
.build();
}
public static void main(String[] args) throws InterruptedException, JSONException {
FeedOptions options = new FeedOptions();
// as this is a multi collection enable cross partition query
options.setEnableCrossPartitionQuery(true);
// note that setMaxItemCount sets the number of items to return in a single page
// result
options.setMaxItemCount(5);
String sql = "SELECT TOP 5 s.studentAlias FROM coll s WHERE s.enrollmentYear = 2018 ORDER BY s.studentAlias";
Program p = new Program();
Observable<FeedResponse<Document>> documentQueryObservable = p.client
.queryDocuments("dbs/" + p.databaseName + "/colls/" + p.collectionId, sql, options);
// observable to an iterator
Iterator<FeedResponse<Document>> it = documentQueryObservable.toBlocking().getIterator();
while (it.hasNext()) {
FeedResponse<Document> page = it.next();
List<Document> results = page.getResults();
// here we iterate over all the items in the page result
for (Object doc : results) {
System.out.println(doc);
}
}
}
}

How do you execute a MongoDB query stored as string in Java?

I'm kind of new to the MongoDB Java driver and I was wondering how you could execute a query stored as a string. Is this the best way to execute them, or what would be a better approach?
I've stumbled across the piece of the below on another stackoverflow thread, but haven't been able to get anything useful out of it. The output does not contain the result of the query at all.
The code I'm running right now:
#Test
public void testExecuteStoredQueries() {
String code = "db.getCollection('users').find({})";
final BasicDBObject command = new BasicDBObject();
String formattedCode = String.format("function() { return %s ; }", code);
System.out.println("Formatted code:");
System.out.println(formattedCode);
command.put("eval", formattedCode);
Document result = DbEngine.getInstance().getDatabase().runCommand(command);
System.out.println(result.toJson());
}
Summarized output:
{
"retval": {
"_mongo": "....",
"_db": "...",
"_collection": "...",
"_ns": "cezy.users",
"_query": {},
"_fields": null,
"_limit": 0,
"_skip": 0,
"_batchSize": 0,
"_options": 0,
"_cursor": null,
"_numReturned": 0,
"_special": false
},
"ok": 1
}

I use morphia when i have to deal with objects. As when you retrieve the data from MongoDb, for the long values you get extended Json instead of Json Response. Parsing Extended Json could be a trouble and might break the code. As Gson doesn't support the conversion from Extended Json to Json.
private void createDatastore(boolean createIndexes) {
Morphia morphia = new Morphia();
morphia.map(classname.class);
datastore = morphia.createDatastore(mongoClient, databaseName);
if (createIndexes) {
datastore.ensureIndexes();
}
}
#Override
public Datastore getDatastore() {
return this.datastore;
}
#Test
public void testExecuteStoredQueries() {
String code = "db.getCollection('users').find({})";
String formattedCode = String.format("function() { return %s ; }", code);
final BasicDBObject basicObject = new BasicDBObject(new BasicDBObject("$in", formattedCode));
Query<ClassName> query = getDatastore().createQuery(<Classname>.class).filter("_eval", basicObject);
List<Classname> List = query.asList();
//if you want to access each object and perform some task
List.forEach((cursor) -> {
//perform your task
});
}

Removing the function creation and adding ".toArray()" pretty much solved the problem.
#Test
public void testExecuteStoredQueries() {
String code = "db.users.find({}).toArray();";
final BasicDBObject command = new BasicDBObject();
command.put("eval", code);
Document result = DbEngine.getInstance().getDatabase().runCommand(command);
System.out.println(result.toJson());
assertNotNull(result.get("retval"));
}
The array is in the "retval" field of the response.

ERROR com.basho.riak.client.core.RiakNode - Write failed on RiakNode

I am using Riak KV with Java client and I am unable to write on the RiakNode, although I have created a Bucket with the name of the space I want to create an object on.
I basically have the TasteOfRiak.java class, which has been provided by the basho developer website: https://raw.githubusercontent.com/basho/basho_docs/master/extras/code-examples/TasteOfRiak.java
import com.basho.riak.client.api.RiakClient;
import com.basho.riak.client.api.commands.kv.DeleteValue;
import com.basho.riak.client.api.commands.kv.FetchValue;
import com.basho.riak.client.api.commands.kv.StoreValue;
import com.basho.riak.client.api.commands.kv.UpdateValue;
import com.basho.riak.client.core.RiakCluster;
import com.basho.riak.client.core.RiakNode;
import com.basho.riak.client.core.query.Location;
import com.basho.riak.client.core.query.Namespace;
import com.basho.riak.client.core.query.RiakObject;
import com.basho.riak.client.core.util.BinaryValue;
import java.net.UnknownHostException;
public class TasteOfRiak {
// A basic POJO class to demonstrate typed exchanges with Riak
public static class Book {
public String title;
public String author;
public String body;
public String isbn;
public Integer copiesOwned;
}
// This will allow us to update the book object handling the
// entire fetch/modify/update cycle.
public static class BookUpdate extends UpdateValue.Update<Book> {
private final Book update;
public BookUpdate(Book update){
this.update = update;
}
#Override
public Book apply(Book t) {
if(t == null) {
t = new Book();
}
t.author = update.author;
t.body = update.body;
t.copiesOwned = update.copiesOwned;
t.isbn = update.isbn;
t.title = update.title;
return t;
}
}
// This will create a client object that we can use to interact with Riak
private static RiakCluster setUpCluster() throws UnknownHostException {
// This example will use only one node listening on localhost:10017
RiakNode node = new RiakNode.Builder()
.withRemoteAddress("127.0.0.1")
.withRemotePort(8087)
.build();
// This cluster object takes our one node as an argument
RiakCluster cluster = new RiakCluster.Builder(node)
.build();
// The cluster must be started to work, otherwise you will see errors
cluster.start();
return cluster;
}
public static void main( String[] args ) {
try {
// First, we'll create a basic object storing a movie quote
RiakObject quoteObject = new RiakObject()
// We tell Riak that we're storing plaintext, not JSON, HTML, etc.
.setContentType("text/plain")
// Objects are ultimately stored as binaries
.setValue(BinaryValue.create("You're dangerous, Maverick"));
System.out.println("Basic object created");
// In the new Java client, instead of buckets you interact with Namespace
// objects, which consist of a bucket AND a bucket type; if you don't
// supply a bucket type, "default" is used; the Namespace below will set
// only a bucket, without supplying a bucket type
Namespace quotesBucket = new Namespace("quotes");
// With our Namespace object in hand, we can create a Location object,
// which allows us to pass in a key as well
Location quoteObjectLocation = new Location(quotesBucket, "Iceman");
System.out.println("Location object created for quote object");
// With our RiakObject in hand, we can create a StoreValue operation
StoreValue storeOp = new StoreValue.Builder(quoteObject)
.withLocation(quoteObjectLocation)
.build();
System.out.println("StoreValue operation created");
// And now we can use our setUpCluster() function to create a cluster
// object which we can then use to create a client object and then
// execute our storage operation
RiakCluster cluster = setUpCluster();
RiakClient client = new RiakClient(cluster);
System.out.println("Client object successfully created");
StoreValue.Response storeOpResp = client.execute(storeOp);
System.out.println("Object storage operation successfully completed");
// Now we can verify that the object has been stored properly by
// creating and executing a FetchValue operation
FetchValue fetchOp = new FetchValue.Builder(quoteObjectLocation)
.build();
RiakObject fetchedObject = client.execute(fetchOp).getValue(RiakObject.class);
assert(fetchedObject.getValue().equals(quoteObject.getValue()));
System.out.println("Success! The object we created and the object we fetched have the same value");
// Now update the fetched object
fetchedObject.setValue(BinaryValue.create("You can be my wingman any time."));
StoreValue updateOp = new StoreValue.Builder(fetchedObject)
.withLocation(quoteObjectLocation)
.build();
StoreValue.Response updateOpResp = client.execute(updateOp);
updateOpResp = client.execute(updateOp);
// And we'll delete the object
DeleteValue deleteOp = new DeleteValue.Builder(quoteObjectLocation)
.build();
client.execute(deleteOp);
System.out.println("Quote object successfully deleted");
Book mobyDick = new Book();
mobyDick.title = "Moby Dick";
mobyDick.author = "Herman Melville";
mobyDick.body = "Call me Ishmael. Some years ago...";
mobyDick.isbn = "1111979723";
mobyDick.copiesOwned = 3;
System.out.println("Book object created");
// Now we'll assign a Location for the book, create a StoreValue
// operation, and store the book
Namespace booksBucket = new Namespace("books");
Location mobyDickLocation = new Location(booksBucket, "moby_dick");
StoreValue storeBookOp = new StoreValue.Builder(mobyDick)
.withLocation(mobyDickLocation)
.build();
client.execute(storeBookOp);
System.out.println("Moby Dick information now stored in Riak");
// And we'll verify that we can fetch the info about Moby Dick and
// that that info will match the object we created initially
FetchValue fetchMobyDickOp = new FetchValue.Builder(mobyDickLocation)
.build();
Book fetchedBook = client.execute(fetchMobyDickOp).getValue(Book.class);
System.out.println("Book object successfully fetched");
assert(mobyDick.getClass() == fetchedBook.getClass());
assert(mobyDick.title.equals(fetchedBook.title));
assert(mobyDick.author.equals(fetchedBook.author));
// And so on...
// Now to update the book with additional copies
mobyDick.copiesOwned = 5;
BookUpdate updatedBook = new BookUpdate(mobyDick);
UpdateValue updateValue = new UpdateValue.Builder(mobyDickLocation)
.withUpdate(updatedBook).build();
UpdateValue.Response response = client.execute(updateValue);
System.out.println("Success! All of our tests check out");
// Now that we're all finished, we should shut our cluster object down
cluster.shutdown();
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
}
Whenever Eclipse executes this code: "StoreValue.Response storeOpResp = client.execute(storeOp);
System.out.println("Object storage operation successfully completed");"
I get an error that "ERROR com.basho.riak.client.core.RiakNode - Write failed on RiakNode".
Before running that program I have already created a quotesBucket bucket and have activated it.
Does anyone know where the problem is?

Can you store an object through http? Try this in terminal:
curl -XPUT \
-H "Content-Type: text/plain" \
-d "You're dangerous, Maverick" \
http://localhost:8098/types/default/buckets/quotes/keys/Iceman?returnbody=true

Create an index with MongoDb

I'm beginner with MongoDB and i'm trying some stuff.
I want to store URL and to avoid duplicate URL I create an unique index on the url.
Like that
collection.createIndex(new BasicDBObject("url", type).append("unique", true));
But each time I launch my program the index is create again isn't it ?
Because, now my program is only inserting one url "http://site.com" and if I restart my program this url is insert again like if there isn't index.
Creating the index each time is the wrong way to handle an index ?
Here is an example of my code
mongo.getCollection().ensureIndex(new BasicDBObject("url", 1).append("unique", "true"));
mongo.getCollection().insert(new BasicDBObject("url", "http://site.com").append("crawled", 0));
mongo.getCollection().insert(new BasicDBObject("url", "http://site.com").append("crawled", 0));
And the output:
{ "_id" : { "$oid" : "50d627cf44ae5d6b5e9cf106"} , "url" : "http://site.com" , "crawled" : 0}
{ "_id" : { "$oid" : "50d627cf44ae5d6b5e9cf107"} , "url" : "http://site.com" , "crawled" : 0}
Thanks
EDIT :
Here is my class Mongo which handle MongoDB
import java.net.UnknownHostException;
import java.util.List;
import java.util.Set;
import com.mongodb.BasicDBObject;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.DBObject;
import com.mongodb.MongoClient;
public class Mongo {
private MongoClient mongoClient;
private DB db;
private DBCollection collection;
private String db_name;
public Mongo(String db){
try {
mongoClient = new MongoClient( "localhost" , 27017 );
this.db = mongoClient.getDB(db);
this.db_name = db;
} catch (UnknownHostException e) {
e.printStackTrace();
}
}
public void drop(){
mongoClient.dropDatabase(db_name);
}
public void listCollections(){
Set<String> colls = db.getCollectionNames();
for (String s : colls) {
System.out.println(s);
}
}
public void listIndex(){
List<DBObject> list = collection.getIndexInfo();
for (DBObject o : list) {
System.out.println("\t" + o);
}
}
public void setCollection(String col){
this.collection = db.getCollection(col);
}
public void insert(BasicDBObject doc){
this.collection.insert(doc);
}
public DBCollection getCollection(){
return collection;
}
public void createIndex(String on, int type){
collection.ensureIndex(new BasicDBObject(on, type).append("unique", true));
}
}
And here is my class which handle my program
public class Explorer {
private final static boolean DEBUG = false;
private final static boolean RESET = false;
private Mongo mongo;
private String host;
public Explorer(String url){
mongo = new Mongo("explorer");
mongo.setCollection("page");
if (RESET){
mongo.drop();
System.out.println("Set RESET to FALSE and restart the program.");
System.exit(1);
}
if (DEBUG) {
mongo.listCollections();
}
this.host = url.toLowerCase();
BasicDBObject doc = new BasicDBObject("url", "http://site.com").append("crawled", 0);
mongo.getCollection().ensureIndex(new BasicDBObject("url", 1).append("unique", true));
mongo.getCollection().insert(new BasicDBObject("url", "http://site.com").append("crawled", 0));
mongo.getCollection().insert(new BasicDBObject("url", "http://site.com").append("crawled", 0));
process();
}
private void process(){
BasicDBObject query = new BasicDBObject("crawled", 0);
DBCursor cursor = mongo.getCollection().find(query);
try {
while(cursor.hasNext()) {
System.out.println(cursor.next());
}
} finally {
cursor.close();
}
}
}

You'll need to pass the unique value as the boolean value true, not as a string, and it's the second parameter that are options:
...ensureIndex(new BasicDBObject("url", 1), new BasicDBObject("unique", true));
Also, I tested it manually using the mongo interpreter:
> db.createCollection("sa")
{ "ok" : 1 }
> db.sa.ensureIndex({"url":1},{unique:true})
> db.sa.insert({url:"http://www.example.com", crawled: true})
> db.sa.insert({url:"http://www.example.com", crawled: true})
E11000 duplicate key error index: test.sa.$url_1 dup key: { : "http://www.example.com" }
> db.sa.insert({url:"http://www.example2.com/", crawled: false})
> db.sa.insert({url:"http://www.example.com", crawled: false})
E11000 duplicate key error index: test.sa.$url_1 dup key: { : "http://www.example.com" }
>
There are only the two objects:
> db.sa.find()
{ "_id" : ObjectId("50d636baa050939da1e4c53b"), "url" : "http://www.example.com", "crawled" : true }
{ "_id" : ObjectId("50d636dba050939da1e4c53d"), "url" : "http://www.example2.com/", "crawled" : false }

I don't fully understand your problem but I feel it's very likely that you should use ensureIndex instead of createIndex as the latter always tries to create the index while the former will only ensure that it exists.

Just stumbled over this question and there are some changes since Version 3.0.0
db.collection.ensureIndex(keys, options)
Deprecated since version 3.0.0: db.collection.ensureIndex() is now an
alias for db.collection.createIndex().
Creates an index on the specified field if the index does not already
exist.

To Use the unique index of the mongodb, you should use the method with 2 parameters where 3rd boolean parameter is for the "unique" index.
mongo.getCollection().ensureIndex(new BasicDBObject("url", 1),"unq_url", true));

Also I see that you dont have a collection name specified in getCollection();
What collection would that select? curious

Simple K-Means doesnt handle iris.arff

I have this class below, i build it considering the examples given on the wiki and in a thesis, why can't SympleKMeans handle data? The class can print the Datasource dados, so its nothing wrong with processing file, the error is on the build.
package slcct;
import weka.clusterers.ClusterEvaluation;
import weka.clusterers.SimpleKMeans;
import weka.core.Instance;
import weka.core.Instances;
import weka.core.converters.ConverterUtils.DataSource;
public class Cluster {
public String path;
public Instances dados;
public String[] options = new String[2];
public Cluster(String caminho, int nclusters, int seed ){
this.path = caminho;
this.options[0] = String.valueOf(nclusters);
this.options[1] = String.valueOf(seed);
}
public void ledados() throws Exception{
DataSource source = new DataSource(path);
dados = source.getDataSet();
System.out.println(dados)
if(dados.classIndex()==-1){
dados.setClassIndex(dados.numAttributes()-1);
}
}
public void imprimedados(){
for(int i=0; i<dados.numInstances();i++)
{
Instance actual = dados.instance(i);
System.out.println((i+1) + " : "+ actual);
}
}
public void clustering() throws Exception{
SimpleKMeans cluster = new SimpleKMeans();
cluster.setOptions(options);
cluster.setDisplayStdDevs(true);
cluster.getMaxIterations();
cluster.buildClusterer(dados);
Instances ClusterCenter = cluster.getClusterCentroids();
Instances SDev = cluster.getClusterStandardDevs();
int[] ClusterSize = cluster.getClusterSizes();
ClusterEvaluation eval = new ClusterEvaluation();
eval.setClusterer(cluster);
eval.evaluateClusterer(dados);
for(int i=0;i<ClusterCenter.numInstances();i++){
System.out.println("Cluster#"+( i +1)+ ": "+ClusterSize[i]+" dados .");
System.out.println("Centróide:"+ ClusterCenter.instance(i));
System.out.println("STDDEV:" + SDev.instance(i));
System.out.println("Cluster Evaluation:"+eval.clusterResultsToString());
}
}
}
The error:
weka.core.WekaException: weka.clusterers.SimpleKMeans: Cannot handle any class attribute!
at weka.core.Capabilities.test(Capabilities.java:1097)
at weka.core.Capabilities.test(Capabilities.java:1018)
at weka.core.Capabilities.testWithFail(Capabilities.java:1297)
at weka.clusterers.SimpleKMeans.buildClusterer(SimpleKMeans.java:228)
at slcct.Cluster.clustering(Cluster.java:53)//Here.
at slcct.Clustering.jButton1ActionPerformed(Clustering.java:104)

I believe you need not set the class index, as you are doing clustering and not classification. Try following this guide for programmatic Java clustering.

In your "ledados()" function just remove the code block given below. It will work. Because you have no defined class in your data.
if(dados.classIndex()==-1){
dados.setClassIndex(dados.numAttributes()-1);
}
Your new function:
public void ledados() throws Exception{
DataSource source = new DataSource(path);
dados = source.getDataSet();
System.out.println(dados) }

You would not need a class attribute in the data while doing k clustering