We Keep Coding

How to execute MongoDB shell command in Java application

I installed 4.4 version MongoDB and reproduced replica set, sharding for my Java application.
Now, I am looking for the approach of run MongoDB shell command in my Java application for check and monitoring reproduced MongoDB Cluster's replica set and sharding status. I'm curious how to achieve this.
I tried to find answers. but, I couldn't found clear answer from other stackoverflow question regarding MongoDB. because most questions discuss base on 3.x version.
So, I researched MongoDB document and driver for 4.4 version and Tested that like below.
run MongoDB command
I tried to run "sh.status()" and "rs.status()" command using MongoDB 4.5 driver through MongoClient object and runCommand() method. but, Java return error message.
I realized that runCommand() method only allow "Mongo" commands mentioned on below link and not Mongo Shell Command.
link : https://www.mongodb.com/docs/v4.4/reference/command/
I couldn't found contrasting command with sh.status() from above link.
String uriString = "mongodb://"+userName+":"+passWord+"#"+targetHost+":"+portNo;
uriString = uriString +"/?authSource="+tgDBName+"&authMechanism="+authMechanism;
MongoClient mongoClient = MongoClients.create(uriString);
String commandString = "sh.status()";
Bson command = new BsonDocument(commandString, new BsonInt64(1));
Document commandResult = database.runCommand(command);
com.mongodb.MongoCommandException: Command failed with error 59 (CommandNotFound):
'no such cmd: sh.status()' on server ....
run javascript stored in MongoDB
I found other way to run sh.status() and get metrics.
That is run javascript stored in MongoDB's system.js collection using $eval.
So, I researched above method and found below links.
link 1 : How do I execute a MongoDB js script using the Java MongoDriver
link 2 : https://www.mongodb.com/docs/v4.4/tutorial/store-javascript-function-on-server/
And I reached answer that db.eval function was deprecated from 3.x and removed from 4.2 version.
Finally, I considering access to MongoDB using JSch and run prompt.
But, I think that is not formal approach. so, I want know other way for get metrics
like "run MongoDB Shell Command in java using 4.4 version driver"
Note : when run sh.status() command in mongos, I received metrics successfully like below.
mongos> sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("625513e0838da178377f6900")
}
shards:
{ "_id" : "sh01", "host" : "sh01/WIN-BKEV4AO0KED:27011,WIN-BKEV4AO0KED:27012,WIN-BKEV4AO0KED:27013", "state" : 1 }
{ "_id" : "sh02", "host" : "sh02/WIN-BKEV4AO0KED:27021,WIN-BKEV4AO0KED:27022,WIN-BKEV4AO0KED:27023", "state" : 1 }
active mongoses:
"4.4.13" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:
{ "_id" : "config", "primary" : "config", "partitioned" : true }

rs.status() is a mongo shell helper for db.adminCommand({ replSetGetStatus: 1 })
I don't work with Java, but I guess you must use MongoDatabase.runCommand. Could be similar to this:
MongoDatabase database = mongoClient.getDatabase("admin");
String commandString = "replSetGetStatus";
Bson command = new BsonDocument(commandString, new BsonInt64(1));
Document commandResult = database.runCommand(command);
For sh.status() or db.printShardingStatus() it is a bit more difficult. Documentation says:
The db.printShardingStatus() method run in mongosh does not return JSON. Use db.printShardingStatus() for manual inspection, and Config Database in scripts.
printShardingStatus
function printShardingStatus(configDB, verbose) {
// configDB is a DB object that contains the sharding metadata of interest.
// Defaults to the db named "config" on the current connection.
if (configDB === undefined)
configDB = db.getSiblingDB('config');
var version = configDB.getCollection("version").findOne();
if (version == null) {
print(
"printShardingStatus: this db does not have sharding enabled. be sure you are connecting to a mongos from the shell and not to a mongod.");
return;
}
var raw = "";
var output = function(indent, s) {
raw += sh._shardingStatusStr(indent, s);
};
output(0, "--- Sharding Status --- ");
output(1, "sharding version: " + tojson(configDB.getCollection("version").findOne()));
... many more lines
If you really like to get identical output to db.printShardingStatus(), then you would need to convert the content from that function into Java.

MongoDB: Query has implicit limit(256)?

I've created (in code) a default collection in MongoDB and am querying it, and have discovered that while the code will return all the data when I run it locally, it won't when I query it on a deployment server. It returns a maximum of 256 records.
Notes:
This is not a capped collection.
Locally, I'm running 3.2.5, the remote MongoDB version is 2.4.12
I am not using the limit parameter. When I use it, I can limit both the local and deployment server, but the deployment server will still never return more than 256 records.
The amount of data being fetched from the server is <500K. Nothing huge.
The code is in Clojure, using Monger, which itself just calls the Java com.mongodb stuff.
I can pull in more than 256 records from the remote server using Robomongo though I'm not sure how it does this, as I cannot connect to the remote from the command line (auth failed using the same credentials, so I'm guessing version incompatibility there).
Any help is appreciated.
UPDATE: Found the thing that triggers the problem: When I sort the output, it reduces the output to 256—but only when I pull from Mongo 2.4! I don't know if this is a MongoDB itself, the MongoDB java class, or Monger, but here is the code that illustrates the issue, as simple as I could make it:
(ns mdbtest.core
(:require [monger.core :as mg]
[monger.query :as mq]))
(defn get-list []
(let [coll (mq/with-collection
(mg/get-db
(mg/connect {:host "old-mongo"}) "mydb") "saves"
(mq/sort (array-map :createdDate -1)))] ;;<<==remove sort
coll))

You need to specify a bigger batch-size, the default is 256 records.
Here's an example from my own code:
=> (count (with-db (q/find {:keywords "lisa"})
(q/sort {:datetime 1}) ))
256
=> (count (with-db (q/find {:keywords "lisa"})
(q/sort {:datetime 1})
(q/batch-size 1000) ))
688
See more info here: http://clojuremongodb.info/articles/querying.html#setting_batch_size

How to pass all kind of SQL-Queries via Command Line/Batch Args[] into Java?

I want to write a little .jar which is used as a "translator" for SQL-Queries directed to a z/OS-DB2-Database.
My goal is that the application accepts SQL-Queries as Command Line Arguments manually or via shell script/cron, next to other parameters like IP, Port, User etc.
Is there a way to leave those arguments unaffected while passing them to the jar?
Example:
java -jar db2sql.jar SQL=={SELECT * FROM TABLE1 TAB1, TABLE2 TAB2 WHERE TAB1.XYZ = TAB2.ZYX AND TAB2.ABC LIKE 'blabla' AND TAB1.DATE >= '01.01.2015'} IP=={192.168.0.1} User=={Santa} Password=={CLAUS}
(please ignore that this statement is senseless, but i hope you get the trick)
My Problem is reading out that Command Line parameters, mostly special characters like * , " ' etc.
Questions:
Is there a list of all possible SQL-Parameters which must be escaped?
Is there a special character which can be used as delimiter that will never occur in an SQL-Query?
Is it possible to pass all kind of SQL Statments as ONE argument?
Is it possible to leave special characters unhandled, e.g. Argument "" = String "", and not .classpath etc. ?
Kind Regards

Although I wouldn't recommend what you're trying to do for several reasons, at least in a *NIX environment you could just use the standard way.
java -jar foo.jar -s "SELECT * FROM SOMETHING WHERE foo = 2" -u username -h hostname
You can use additional libraries to parse the parameters, but this way you would use -s to specify the SQL query, and wrap the param value in " to make it a single argument with automatic escape.
In your main method you can then get the full query with (simplified)
if(args[0].equals("-s"))
sqlString = args[1];

BCP exits with code 0

When running BCP from my Java application it exits with status code 0 when 1 is expected.
I run bcp with an invalid combination of data and formatting file and bcp gives the following error:
Starting copy...
SQLState = 22005, NativeError = 0
Error = [Microsoft][SQL Server Native Client 10.0]Invalid character value for cast specification
BCP copy in failed
BCP however exits with exit code 0 and not 1, as i suspect. Now it is extremely difficult to see if something failed while running BCP. Exitting with the right code works once they match to some degree (like same delimiters).
Command
PS C:\Users\feh\Desktop> bcp integrate_test.dbo.AS_LOADER_DELIMITED in .\data.dat -S "10.0.0.161\SQL2K5,1048" -U user -P pass -f .\formatting.ctl -m 1
Starting copy...
SQLState = S1000, NativeError = 0
Error = [Microsoft][SQL Server Native Client 10.0]Unexpected EOF encountered in BCP data-file
0 rows copied.
Network packet size (bytes): 4096
Clock Time (ms.) Total : 1
PS C:\Users\feh\Desktop> $lastexitcode
0
How can i validate a formatting file to the data and get a exit code 1 when they do not match?

If the bcp is "in" the consider using BULK INSERT. This does the same as bcp but can be trapped using SQL Server TRY/CATCH or normal exception handling
Otherwise, are you trapping %ERRORLEVEL% and not some other code?

How can I customize the DDL generated with Hibernate's hbm2ddl (using annotations)?

Here's what I want to do:
I'm using Hibernate (3.3.2) to map my Ingres 10 database. My java entities are generated after a meta model, so we decided to use annotations to ease things.
What we want to do after the code generation is generate the DDL instructions to create the database so we use the hbm2ddl tool and we have something like:
drop table xxx;
create table xxx ...;
What I miss here are extra SQL statement to e.g. add privileges on tables, something like:
drop table xxx;
create table xxx ...;
grant xxx on table xxx;
I know I could use something called database-object to generate such statements but I think it's only available with the XML mapping. Can you confirm this?
If that is confirmed, do you see a better solution for doing such a thing?
Many thanks to all.

A little late, and not the prettiest solution, but here's a quick and dirty bash script to run in conjunction with maven. Hopefully it helps someone else grappling with this issue.
#!/bin/bash
SQL_FILE=$1
GROUP=$2
COMPILED=$1.tmp
SCHEMA_DROP=$3
if [ "$3" == "" ]; then
SCHEMA_DROP="true"
fi
mvn hibernate3:hbm2ddl -Dschema.file=$SQL_FILE -Dschema.drop=$SCHEMA_DROP
if [ "$SQL_FILE" == "" ] || [ "$GROUP" == "" ] ; then
echo "Usage: $0 {SQL_FILE} {GROUP} [DROP_SCHEMA]"
echo "Where: "
echo "SQL_FILE: path to sql file relative to the root of the project"
echo "GROUP: the predefined database group on which to grant access"
echo "DROP_SCHEMA: true|false per the hbm2ddl 'drop' parameter. Defaults to true"
echo "NOTE: In order for this to work properly, the pom configuration of the hibernatetool should be parameterized. For example:"
echo '<hbm2ddl outputfilename="../../../${schema.file}" format="true" drop="${schema.drop}" />'
exit;
fi
echo "" > $COMPILED
GRANT=""
while read line
do
echo $line >> $COMPILED
if [[ $line =~ .*create.table.([A-Za-z_-]*) ]]; then
GRANT="$GRANT\ngrant all on table ${BASH_REMATCH[1]} to group $GROUP;" >> $COMPILED
fi
done < $SQL_FILE
echo -e $GRANT >> $COMPILED
mv $COMPILED $SQL_FILE
I like to put my schema file into a src directory for check-in, such as src/main/db/myschema.sql, hence the dir up values in the outputfilename attribute (see the NOTE in the usage comments). To run this script with file name "genSchema.sh" from the root of the project in cygwin:
./genSchema.sh src/main/db/myschema.sql mygroup
The regexp and generated grant lines are for the postgresql dialect. They'd probably need a slight modification for other dialects.

I don't know Hibernate (only NHibernate) that much and I don't know these annotations stuff very well. But I'm pretty sure that this is only available in XML mapping files.
You may find a way to combine annotations and XML mappings. If this is not possible, consider to switch to XML entirely, it is generally more powerful and you get more control.