We Keep Coding

Read and write to different Mongo collections using Spark with Java

I am a relative newbie to Spark. I need to read from a Mongo collection in Java using Spark, change some field values, let's say I am appending "123" to one field value and write into another collection. Accordingly I had 2 separate Mongo URIs as the input and output URIs configured in Spark. I am then proceeding to read from the input collection. However, what I am not understanding is how would I make the same RDD of documents as output to another collection. This is the input code:
String inputUri = "mongodb://" + kp.getProperty("source.mongo.userid") + ":"
+ Encryptor.decrypt(kp.getProperty("source.mongo.cache")) + "#"
+ kp.getProperty("source.mongo.bootstrap-servers") + "/" + kp.getProperty("source.mongo.database")
+ "." + kp.getProperty("source.mongo.inputCollection") + "?ssl=true&connectTimeoutMS="
+ kp.getProperty("source.mongo.connectTimeoutMS") + "&socketTimeoutMS="
+ kp.getProperty("source.mongo.socketTimeoutMS") + "&maxIdleTimeMS="
+ kp.getProperty("source.mongo.maxIdleTimeMS");
String outputUri = "mongodb://" + kp.getProperty("source.mongo.userid") + ":"
+ Encryptor.decrypt(kp.getProperty("source.mongo.cache")) + "#"
+ kp.getProperty("source.mongo.bootstrap-servers") + "/" + kp.getProperty("source.mongo.database")
+ "." + kp.getProperty("source.mongo.outputCollection") + "?ssl=true&connectTimeoutMS="
+ kp.getProperty("source.mongo.connectTimeoutMS") + "&socketTimeoutMS="
+ kp.getProperty("source.mongo.socketTimeoutMS") + "&maxIdleTimeMS="
+ kp.getProperty("source.mongo.maxIdleTimeMS");
SparkSession spark = SparkSession.builder().master("local[3]").appName(kp.getProperty("spark.app.name"))
.config("spark.mongodb.input.uri", inputUri)
.config("spark.mongodb.output.uri", outputUri)
...;
JavaSparkContext sc = new JavaSparkContext(spark.sparkContext());
JavaMongoRDD<Document> rdd = MongoSpark.load(sc);
System.out.println("Count: " + rdd.count());
System.out.println(rdd.first().toJson());
Please help me in this regard.

I have got the answer myself. I went the Dataset route instead of RDDs which made the modification simpler. So, to load the Mongo colection, I use
Dataset<Row> df = MongoSpark.load(sc).toDF();
Then I create a temporary view upon it in orcder to be able to use Spark SQL:
df.createOrReplaceTempView("Customer");
I register an UDF for operating upon each column value:
spark.udf().register("Test", new TestUDF(), DataTypes.StringType);
the UDF definition is as follows:
public class TestUDF implements UDF1<String, String> {
#Override
public String call(String customer) throws Exception {
return customer + "123";
}
}
Then I call the UDF using the same column name as the original so that the values in the original dataset are replaced:
df = df.withColumn("CustomerName", functions.callUDF("Test", functions.col("CustomerName")));
Then I write it back to Mongo in a separate collection:
MongoSpark.write(df).option("collection", "myCollection").save();

Convert Loadrunner File Parameter to Java string for payload

I have a java virtual user script that is sending a payload request. I am trying to use values from a file to send via a loadrunner file parameter.
here is the payload:
private static final String PAYLOAD =
"<ips_cad_mdt>\n" +
" <SignOnRequest>\n" +
" <DestApplication>hhhh</DestApplication>\n" +
" <OrigApplication>hhh</OrigApplication>\n" +
" <SessionRef>3</SessionRef>\n" +
" <Aliasing>1234</Aliasing>\n" +
" </SignOnRequest>\n" +
"</ips_cad_mdt>";
I would like to use something like the following:
private static final String PAYLOAD =
"<ips_cad_mdt>\n" +
" <SignOnRequest>\n" +
" <DestApplication>hhh</DestApplication>\n" +
" <OrigApplication>hhh</OrigApplication>\n" +
" <SessionRef>3</SessionRef>\n" +
" <Aliasing>”+lr.eval_string(“{AliasId}”)+”</Aliasing>\n" +
" </SignOnRequest>\n" +
"</ips_cad_mdt>";
for some reason i cant see any output for this value. do i need to declare a variable: e.g. lr.save_string("AliasId", "{AliasId}");
an example of this would help loads. Many Thanks

There seems to be an error in the code completion in VuGen. The parameters should be reversed and without the {} in save_string.
lr.save_string("1234","myId");
lr.message(lr.eval_string("{myId}"));
In the documentation it is correct - https://admhelp.microfocus.com/lr/en/12.55/help/function_reference/FuncRef.htm#FuncRef/c_vuser/lrFr_lr_save_string.htm?Highlight=lr_save_string
I asked the responsible team to fix the code completion in VuGen so you will be able to see this change in one of the future releases.

Using JSON data as a String object in Java

I am using HTTP Client to send different kinds of requests (GET, PUT, POST, DELETE) where I am sending the JSON as a data to post with different requests.
I have a JSON data like:
{ "createdByID": "100000", "createdByName": "Admin", "modifiedByID": "100000", "modifiedByName": "Admin" }
Now, to store this JSON into a string, I have to add double quotes wherever necessary so that this can be stored as
String jsonData = "{" + "\"" + "createdByID" + "\"" + ":" + "\"" + "100000" + "\"" + "," + "\"" + "createdByName" + "\"" + ":" + "\"" + "Admin" + "\"" + "," + "\"" + "modifiedByID" + "\"" + ":" + "\"" + "100000" + "\"" + "," + "\"" + "modifiedByName" + "\"" + ":" + "\"" + "Admin" + "\"" + "}"
Does anyone use any tool/utility to convert the JSON data such that it can be stored in a string object?
Please share if anyone has already done this

Hey if you store the json as a sting in your code then only you need to add double codes.
Try to read json from a text file , then you don't need to add double quotes and other stuffs.
More over java does not consider double quotes after compiling your code.

How can I split the data over multiple lines in a JTextArea?

How can I put each line of the host in the message body? I work with a JTextArea.
String host = InetAddress.getLocalHost().getHostName().toString();
texto_recepcion.setText(host + texto_recepcion.getText() + dpRecepcion.getAddress() + " " + mensaje_recibido + "\n");
How it is now:

I resolve my question with append function.
String host = InetAddress.getLocalHost().getHostName().toString();
texto_recepcion.append(host); // ***Use the function append for solve the problem***
texto_recepcion.setText(texto_recepcion.getText() + dpRecepcion.getAddress() + " " + mensaje_recibido + "\n");
Thanks a lot

Why don't you add the newline character "\n" to the beginning of the string?
texto_recepcion.setText("\n" + host + texto_recepcion.getText() + dpRecepcion.getAddress() + " " + mensaje_recibido);

How to pass dynamic value to a JSON String

I have constructed a JSON String this way , but cannot able to pass dynamic values to it
String input = "{\r\n" +
" \"Level\": 0,\r\n" +
" \"Name\": \"String\",\r\n" +
" \"msgName\": \"String\",\r\n" +
" \"ActualMessage\": \"String\",\r\n" +
" \"TimeStamp\": \"/Date(-62135596800000-0000)/\"\r\n" +
"}" ;
String message = "this is value want to pass to the ActualMessage attribute " ;
I need to pass dynamic value to the ActaulMessage atribute
Please tell me how ?
i have tried number of trial and errors but couldn't able to succeed.

Use string concatenation.
String message = "this is value want to pass to the ActualMessage attribute " ;
String input = "{\r\n" +
"\"Level\": 0,\r\n" +
"\"Name\": \"String\",\r\n" +
"\"msgName\": \"String\",\r\n" +
"\"ActualMessage\": \"" + message + "\",\r\n" +
"\"TimeStamp\": \"/Date(-62135596800000-0000)/\"\r\n" +
"}" ;

How about using String.format() for this? for example, to pass a "dynamic value" declare a place holder in the text:
String input = "insert %s in the string"; // here %s is the placeholder
input = String.format(input, "value"); // replace %s with actual value
Now input will contain the string "insert value in the string". In your example, change this line:
" \"msgName\": \"String\",\r\n"
Replace it with this:
" \"msgName\": \"%s\",\r\n"
Now you can perform the substitution:
input = String.format(input, message);
Notice that the first parameter in the format() method has a lot more of options, and that you can pass more than one argument to be replaced. Take a look at the documentation for the Formatter class.

if you want to manipulate Json please consider GSON. your problem can be addressed as follows.
String input = "{\r\n" +
" \"Level\": 0,\r\n" +
" \"Name\": \"String\",\r\n" +
" \"msgName\": \"MessageName\",\r\n" +
" \"ActualMessage\": \"%s\",\r\n" +
" \"TimeStamp\": \"/Date(-62135596800000-0000)/\"\r\n" +
"}" ;
String message = "this is value want to pass to the ActualMessage attribute " ;
String output=String.format(input,message);
//this will replace %s with the content of message variable.