We Keep Coding

How to replace backward slash (\) to forward slash (/) using jmeter BeanShell

I have declared a user defined variable as 'projectHome' using below BeanShell which returns the absolute path of the jmx file.
${__BeanShell(import org.apache.jmeter.services.FileServer; FileServer.getFileServer().getBaseDir();)}
On Windows it returns: projectHome=C:\Users\dd\Desktop\API_Testing
On MacOS it returns:
projectHome=/Users/dd/Desktop/API_Testing
The variable value is working fine in following BeanShell Sampler on MacOS:
import java.io.FileWriter;
import java.util.Arrays;
import java.io.Writer;
import java.util.List;
//Default separator
char SEPARATOR = ',';
//function write line in csv
public void writeLine(FileWriter writer, String[] params, char separator) {
boolean firstParam = true;
StringBuilder stringBuilder = new StringBuilder();
String param = "";
for (int i = 0; i < params.length; i++) {
//get param
param = params[i];
log.info(param);
//if the first param in the line, separator is not needed
if (!firstParam) {
stringBuilder.append(separator);
}
//Add param to line
stringBuilder.append(param);
firstParam = false;
}
//prepare file to next line
stringBuilder.append("\n");
//add to file the line
log.info(stringBuilder.toString());
writer.append(stringBuilder.toString());
}
//get path of csv file (creates new one if its not exists)
String csvFile = "${projectHome}/tenant_details.csv";
String[] params = {"${Email}"};
FileWriter fileWriter = new FileWriter(csvFile, true);
writeLine(fileWriter, params, SEPARATOR);
//proper close to file
fileWriter.flush();
fileWriter.close();
When I am running the script on Windows machine, it is failing because of the path having backslash.
How can I convert the backslash to forward slash within the same BeanShell Sampler so that it can work on Windows as well as MacOS?

You don't need to replace slashes / with backslashes \, slashes will work file for Windows platform as well, you can do something like:
new File("c:/Windows/system32/cmd.exe")
and it will normally resolve.
Don't refer variables like ${projectHome} and/or ${Email} in scripts, use vars shorthand instead. It stands for JMeterVariables class instance and provides read/write access to all JMeter Variables in scope. So you should be using
String csvFile = vars.get("projectHome") + "/tenant_details.csv";
and
String[] params = {vars.get("Email")};
Since JMeter 3.1 it is recommended to use JSR223 Test Elements and Groovy language for scripting so consider migrating to Groovy on next available opportunity as Beanshell (and will) become a performance bottleneck when it comes to high loads. See Apache Groovy - Why and How You Should Use It article for more information.

In beanshell Sampler/Post-PreProcessor, input the path to script folder and script name into the Script File field as:-
${__BeanShell(File.separator,)} will return the / in Unix (Mac, Linux, …) and \ in Windows. So you can consider it as:
– Windows: ${PATH}\Foldername\beanshellfile.bsh
– Unix: ${PATH}/scripts/beanshellfile.bsh

Inside Sampler you can call File.separator which uses relevant OS file separator when you need to add it. In your code change to:
String csvFile = "${projectHome}" + File.separator + "tenant_details.csv";

Here is how I solved this problem using BeanShell:
import org.apache.jmeter.services.FileServer;
String JMXPath = FileServer.getFileServer().getBaseDir();
char[]a = JMXPath.toCharArray();
log.info(JMXPath + " - It is project JMX path");
for (int i = 0; i < a.length; i++)
{
if (a[i] == '\\')
{
a[i] = '/';
}
}
String xyz = String.valueOf(a);
log.info(xyz + " - It is projectHome value");
vars.put("projectHome", xyz);

Execute command from Java code deployed on Apache Tomcat

I was trying to convert a DOCX file to PDF file, found this vb script code which perfectly converts DOCX to PDF file, it uses .bat file for file generation. The code can be executed through java.
I am facing a strange problem, when I execute the code on my local machine, the file is generated, but when I deployed the app on Server , the code executes with no errors but the file is not generated.
Do we need any permission to execute commands through java?
Following is the details:
Server Operating system : Windows Server 2012 R2 Standard
Application server :Apache Tomcat 7.0.75
Code:
1)Java
public static void generatePDF() {
try {
File file = new File("C:\\Docx_To_Pdf_Converter\\errorLog.txt");
PrintStream printStreamToFile = new PrintStream(file);
System.setOut(printStreamToFile);
String docToPdf = "C:\\Docx_To_Pdf_Converter\\doc2pdf.bat";
File docPath = new File("C:\\Docx_To_Pdf_Converter\\Letter1.docx");
File pdfPath = new File("C:\\Docx_To_Pdf_Converter\\LetterPDF.pdf");
String command = String.format("%s %s %s", docToPdf, docPath, pdfPath);
Process process = Runtime.getRuntime().exec(command);
// The next line is optional and will force the current Java
//thread to block until the script has finished its execution.
process.waitFor();
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
and the .bat file code:
#Echo off
pushd %~dp0
cscript C:\Docx_To_Pdf_Converter\doc2pdf.vbs %1 %2
vbscript code which actually converts the docx to pdf
Const wdFormatPDF = 17 ' PDF format.
Const wdFormatXPS = 18 ' XPS format.
Const WdDoNotSaveChanges = 0
Dim arguments
Set arguments = WScript.Arguments
Function CheckUserArguments()
If arguments.Unnamed.Count < 1 Or arguments.Unnamed.Count > 2 Then
WScript.Echo "Use:"
WScript.Echo "<script> input.doc"
WScript.Echo "<script> input.doc output.pdf"
WScript.Quit 1
End If
End Function
// Transforms a doc to a pdf
Function DocToPdf( docInputFile, pdfOutputFile )
Dim fileSystemObject
Dim wordApplication
Dim wordDocument
Dim wordDocuments
Dim baseFolder
Set fileSystemObject = CreateObject("Scripting.FileSystemObject")
Set wordApplication = CreateObject("Word.Application")
Set wordDocuments = wordApplication.Documents
docInputFile = fileSystemObject.GetAbsolutePathName(docInputFile)
baseFolder = fileSystemObject.GetParentFolderName(docInputFile)
If Len(pdfOutputFile) = 0 Then
pdfOutputFile = fileSystemObject.GetBaseName(docInputFile) + ".pdf"
End If
If Len(fileSystemObject.GetParentFolderName(pdfOutputFile)) = 0 Then
pdfOutputFile = baseFolder + "\" + pdfOutputFile
End If
//' Disable any potential macros of the word document.
wordApplication.WordBasic.DisableAutoMacros
// 'from below line the code does not executes
Set wordDocument = wordDocuments.Open(docInputFile)
wordDocument.SaveAs pdfOutputFile, wdFormatPDF
wordDocument.Close WdDoNotSaveChanges
wordApplication.Quit WdDoNotSaveChanges
Set wordApplication = Nothing
Set fileSystemObject = Nothing
End Function
// ' Execute script
Call CheckUserArguments()
If arguments.Unnamed.Count = 2 Then
Call DocToPdf( arguments.Unnamed.Item(0), arguments.Unnamed.Item(1) )
Else
Call DocToPdf( arguments.Unnamed.Item(0), "" )
End If
Set arguments = Nothing

It is not possible to give you a 100% guaranteed answer as we don't have access to your deployment server, but here's what I think is happening there.
If the .bat file were missing or not executable for some reason, then you would get an IOException in your Java code. Since you didn't get an exception, clearly the .bat file was found and executed.
However, whatever is within the .bat file is not being executed as you expect. Either cscript.exe is missing, or the .vbs file is missing. The way your code is written, you would not be aware of this. All you would see is a non-zero status return from the waitfor() method, and you don't bother to check that. Therefore you have no knowledge of what actually happened.
At the very minimum you should change the waitfor() method invocation to:
int rc = process.waitFor();
System.out.printf("Process returned %d\n", rc);
This will tell you the return status from the attempt to execute the .bat file. If it's not zero, then you have a problem, and I'm 99.999% sure you will find this to be non-zero. To troubleshoot this you will need to capture the output from the command. The following is a highly simplified (as in no error handling; that's up to you) example:
Process proc = Runtime.getRuntime().exec(command);
BufferedReader procOutput = new BufferedReader(new InputStreamReader(proc.getInputStream()));
String line;
while((line = procOutput.readLine()) != null)
{
System.out.println(line);
// Or whatever you need to do to in your environment, such
// as log the output or examine it to ensure the script did
// what you want
}
int rc = proc.waitFor();
System.out.printf("Process returned %d\n", rc);
NOTE The above is not production-level code, it is only an example of how to use the API. This works ONLY if the external command requires no redirected input; if it does require input then you need to handle input and output in separate threads to prevent deadlocks.
Also, you should consider using ProcessBuilder as it is much more flexible. For example, it allows you to redirect the process' output to append to a log file, which you should probably implement.

Using displayLogs() in WLSTInterpreter NameError: displayLogs

I am attempting to get a log output from the displayLogs() command and I have trying to do this within the WLST Interpreter. I get the following error which is "NameError: displayLogs" I am able to perform other commands such as domainRuntime() and a number of others, but this one seems to be outside the realm. Do I need to run it with some sort of classes in the classpath when running it? Any help would be appreciated.
The source code that I am using below:
package wlst;
import weblogic.management.scripting.utils.WLSTInterpreter;
import org.python.util.InteractiveInterpreter;
import org.python.core.PyObject;
public class EmbeddedWLST
{
static InteractiveInterpreter interpreter = null;
EmbeddedWLST() {
interpreter = new WLSTInterpreter();
}
private static void connect() {
StringBuffer buffer = new StringBuffer();
buffer.append("connect('USERNAME','PASSWORD','t3://HOSTANAME:PORT')");
interpreter.exec(buffer.toString());
}
public static void main(String[] args) {
new EmbeddedWLST();
connect();
PyObject cmo = interpreter.get("cmo");
String command = getLogs();
System.out.println("Executing Get Logs");
interpreter.exec(command);
System.out.println("Getting Output Object");
PyObject output = interpreter.get("output");
System.out.println(output.getClass());
System.out.println(output);
}
private static String getLogs() {
StringBuffer buf = new StringBuffer();
buf.append( "output = displayLogs(returnData=1)\n" );
return buf.toString();
}
}

UPDATE
Everything you are looking for lives in:
<install dir>/oracle_common/common/wlst
A simple grep -R displayLogs * returned the python module you need:
<install dir>/oracle_common/common/wlst/oracle-logging.py
You will need to include the jars that script needs on your classpath, specifically the logging jar ojdl.jar found under <install dir>/oracle_common/modules/oracle.odl
The above information was found by comparing the scripts below (I am using 10.3.6):
This script <install dir>/wlserver_10.3/common/bin/wlst.sh fails with:
wls:/offline> listLogs()
Traceback (innermost last):
File "<console>", line 1, in ?
NameError: listLogs
This script <install dir>/oracle_common/common/bin/wlst.sh succeeds (and has many more options than the script above) :
wls:/offline> listLogs()
Not connected to a Weblogic server. Connect to a Weblogic server first.
Make sure you have all the same jars and properties set as the second script does.

Using Hadoop to find files that contain a particular string

I have around 1000 files and each file is of the size of 1GB. And I need to find a String in all these 1000 files and also which files contains that particular String. I am working with Hadoop File System and all those 1000 files are in Hadoop File System.
All the 1000 files are under real folder, so If I do like this below, I will be getting all the 1000 files. And I need to find which files contains a particular String hello under real folder.
bash-3.00$ hadoop fs -ls /technology/dps/real
And this is my data structure in hdfs-
row format delimited
fields terminated by '\29'
collection items terminated by ','
map keys terminated by ':'
stored as textfile
How I can write MapReduce jobs to do this particular problem so that I can find which files contains a particular string? Any simple example will be of great help to me.
Update:-
With the use of grep in Unix I can solve the above problem scenario, but it is very very slow and it takes lot of time to get the actual output-
hadoop fs -ls /technology/dps/real | awk '{print $8}' | while read f; do hadoop fs -cat $f | grep cec7051a1380a47a4497a107fecb84c1 >/dev/null && echo $f; done
So that is the reason I was looking for some MapReduce jobs to do this kind of problem...

It sounds like you're looking for a grep-like program, which is easy to implement using Hadoop Streaming (the Hadoop Java API would work too):
First, write a mapper that outputs the name of the file being processed if the line being processed contains your search string. I used Python, but any language would work:
#!/usr/bin/env python
import os
import sys
SEARCH_STRING = os.environ["SEARCH_STRING"]
for line in sys.stdin:
if SEARCH_STRING in line.split():
print os.environ["map_input_file"]
This code reads the search string from the SEARCH_STRING environmental variable. Here, I split the input line and check whether the search string matches any of the splits; you could change this to perform a substring search or use regular expressions to check for matches.
Next, run a Hadoop streaming job using this mapper and no reducers:
$ bin/hadoop jar contrib/streaming/hadoop-streaming-*.jar \
-D mapred.reduce.tasks=0
-input hdfs:///data \
-mapper search.py \
-file search.py \
-output /search_results \
-cmdenv SEARCH_STRING="Apache"
The output will be written in several parts; to obtain a list of matches, you can simply cat the files (provided they aren't too big):
$ bin/hadoop fs -cat /search_results/part-*
hdfs://localhost/data/CHANGES.txt
hdfs://localhost/data/CHANGES.txt
hdfs://localhost/data/ivy.xml
hdfs://localhost/data/README.txt
...

To get the filename you are currently processing, do:
((FileSplit) context.getInputSplit()).getPath().getName()
When you are searching your file record by record, when you see hello, emit the above path (and maybe the line or anything else).
Set the number of reducers to 0, they aren't doing anything here.
Does 'row format delimited' mean that lines are delimited by a newline? in which case TextInputFormat and LineRecordReader work fine here.

You can try something like this, though I'm not sure if it's an efficient way to go about it. Let me know if it works - I haven't tested it or anything.
You can use it like this: java SearchFiles /technology/dps/real hello making sure you run it from the appropriate directory of course.
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Scanner;
public class SearchFiles {
public static void main(String[] args) throws IOException {
if (args.length < 2) {
System.err.println("Usage: [search-dir] [search-string]");
return;
}
File searchDir = new File(args[0]);
String searchString = args[1];
ArrayList<File> matches = checkFiles(searchDir.listFiles(), searchString, new ArrayList<File>());
System.out.println("These files contain '" + searchString + "':");
for (File file : matches) {
System.out.println(file.getPath());
}
}
private static ArrayList<File> checkFiles(File[] files, String search, ArrayList<File> acc) throws IOException {
for (File file : files) {
if (file.isDirectory()) {
checkFiles(file.listFiles(), search, acc);
} else {
if (fileContainsString(file, search)) {
acc.add(file);
}
}
}
return acc;
}
private static boolean fileContainsString(File file, String search) throws IOException {
BufferedReader in = new BufferedReader(new FileReader(file));
String line;
while ((line = in.readLine()) != null) {
if (line.contains(search)) {
in.close();
return true;
}
}
in.close();
return false;
}
}

Reading Java Properties file without escaping values

My application needs to use a .properties file for configuration.
In the properties files, users are allow to specify paths.
Problem
Properties files need values to be escaped, eg
dir = c:\\mydir
Needed
I need some way to accept a properties file where the values are not escaped, so that the users can specify:
dir = c:\mydir

Why not simply extend the properties class to incorporate stripping of double forward slashes. A good feature of this will be that through the rest of your program you can still use the original Properties class.
public class PropertiesEx extends Properties {
public void load(FileInputStream fis) throws IOException {
Scanner in = new Scanner(fis);
ByteArrayOutputStream out = new ByteArrayOutputStream();
while(in.hasNext()) {
out.write(in.nextLine().replace("\\","\\\\").getBytes());
out.write("\n".getBytes());
}
InputStream is = new ByteArrayInputStream(out.toByteArray());
super.load(is);
}
}
Using the new class is a simple as:
PropertiesEx p = new PropertiesEx();
p.load(new FileInputStream("C:\\temp\\demo.properties"));
p.list(System.out);
The stripping code could also be improved upon but the general principle is there.

Two options:
use the XML properties format instead
Writer your own parser for a modified .properties format without escapes

You can "preprocess" the file before loading the properties, for example:
public InputStream preprocessPropertiesFile(String myFile) throws IOException{
Scanner in = new Scanner(new FileReader(myFile));
ByteArrayOutputStream out = new ByteArrayOutputStream();
while(in.hasNext())
out.write(in.nextLine().replace("\\","\\\\").getBytes());
return new ByteArrayInputStream(out.toByteArray());
}
And your code could look this way
Properties properties = new Properties();
properties.load(preprocessPropertiesFile("path/myfile.properties"));
Doing this, your .properties file would look like you need, but you will have the properties values ready to use.
*I know there should be better ways to manipulate files, but I hope this helps.

The right way would be to provide your users with a property file editor (or a plugin for their favorite text editor) which allows them entering the text as pure text, and would save the file in the property file format.
If you don't want this, you are effectively defining a new format for the same (or a subset of the) content model as the property files have.
Go the whole way and actually specify your format, and then think about a way to either
transform the format to the canonical one, and then use this for loading the files, or
parse this format and populate a Properties object from it.
Both of these approaches will only work directly if you actually can control your property object's creation, otherwise you will have to store the transformed format with your application.
So, let's see how we can define this. The content model of normal property files is simple:
A map of string keys to string values, both allowing arbitrary Java strings.
The escaping which you want to avoid serves just to allow arbitrary Java strings, and not just a subset of these.
An often sufficient subset would be:
A map of string keys (not containing any whitespace, : or =) to string values (not containing any leading or trailing white space or line breaks).
In your example dir = c:\mydir, the key would be dir and the value c:\mydir.
If we want our keys and values to contain any Unicode character (other than the forbidden ones mentioned), we should use UTF-8 (or UTF-16) as the storage encoding - since we have no way to escape characters outside of the storage encoding. Otherwise, US-ASCII or ISO-8859-1 (as normal property files) or any other encoding supported by Java would be enough, but make sure to include this in your specification of the content model (and make sure to read it this way).
Since we restricted our content model so that all "dangerous" characters are out of the way, we can now define the file format simply as this:
<simplepropertyfile> ::= (<line> <line break> )*
<line> ::= <comment> | <empty> | <key-value>
<comment> ::= <space>* "#" < any text excluding line breaks >
<key-value> ::= <space>* <key> <space>* "=" <space>* <value> <space>*
<empty> ::= <space>*
<key> ::= < any text excluding ':', '=' and whitespace >
<value> ::= < any text starting and ending not with whitespace,
not including line breaks >
<space> ::= < any whitespace, but not a line break >
<line break> ::= < one of "\n", "\r", and "\r\n" >
Every \ occurring in either key or value now is a real backslash, not anything which escapes something else.
Thus, for transforming it into the original format, we simply need to double it, like Grekz proposed, for example in a filtering reader:
public DoubleBackslashFilter extends FilterReader {
private boolean bufferedBackslash = false;
public DoubleBackslashFilter(Reader org) {
super(org);
}
public int read() {
if(bufferedBackslash) {
bufferedBackslash = false;
return '\\';
}
int c = super.read();
if(c == '\\')
bufferedBackslash = true;
return c;
}
public int read(char[] buf, int off, int len) {
int read = 0;
if(bufferedBackslash) {
buf[off] = '\\';
read++;
off++;
len --;
bufferedBackslash = false;
}
if(len > 1) {
int step = super.read(buf, off, len/2);
for(int i = 0; i < step; i++) {
if(buf[off+i] == '\\') {
// shift everything from here one one char to the right.
System.arraycopy(buf, i, buf, i+1, step - i);
// adjust parameters
step++; i++;
}
}
read += step;
}
return read;
}
}
Then we would pass this Reader to our Properties object (or save the contents to a new file).
Instead, we could simply parse this format ourselves.
public Properties parse(Reader in) {
BufferedReader r = new BufferedReader(in);
Properties prop = new Properties();
Pattern keyValPattern = Pattern.compile("\s*=\s*");
String line;
while((line = r.readLine()) != null) {
line = line.trim(); // remove leading and trailing space
if(line.equals("") || line.startsWith("#")) {
continue; // ignore empty and comment lines
}
String[] kv = line.split(keyValPattern, 2);
// the pattern also grabs space around the separator.
if(kv.length < 2) {
// no key-value separator. TODO: Throw exception or simply ignore this line?
continue;
}
prop.setProperty(kv[0], kv[1]);
}
r.close();
return prop;
}
Again, using Properties.store() after this, we can export it in the original format.

Based on #Ian Harrigan, here is a complete solution to get Netbeans properties file (and other escaping properties file) right from and to ascii text-files :
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.io.Reader;
import java.io.Writer;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Properties;
/**
* This class allows to handle Netbeans properties file.
* It is based on the work of : http://stackoverflow.com/questions/6233532/reading-java-properties-file-without-escaping-values.
* It overrides both load methods in order to load a netbeans property file, taking into account the \ that
* were escaped by java properties original load methods.
* #author stephane
*/
public class NetbeansProperties extends Properties {
#Override
public synchronized void load(Reader reader) throws IOException {
BufferedReader bfr = new BufferedReader( reader );
ByteArrayOutputStream out = new ByteArrayOutputStream();
String readLine = null;
while( (readLine = bfr.readLine()) != null ) {
out.write(readLine.replace("\\","\\\\").getBytes());
out.write("\n".getBytes());
}//while
InputStream is = new ByteArrayInputStream(out.toByteArray());
super.load(is);
}//met
#Override
public void load(InputStream is) throws IOException {
load( new InputStreamReader( is ) );
}//met
#Override
public void store(Writer writer, String comments) throws IOException {
PrintWriter out = new PrintWriter( writer );
if( comments != null ) {
out.print( '#' );
out.println( comments );
}//if
List<String> listOrderedKey = new ArrayList<String>();
listOrderedKey.addAll( this.stringPropertyNames() );
Collections.sort(listOrderedKey );
for( String key : listOrderedKey ) {
String newValue = this.getProperty(key);
out.println( key+"="+newValue );
}//for
}//met
#Override
public void store(OutputStream out, String comments) throws IOException {
store( new OutputStreamWriter(out), comments );
}//met
}//class

You could try using guava's Splitter: split on '=' and build a map from resulting Iterable.
The disadvantage of this solution is that it does not support comments.

#pdeva: one more solution
//Reads entire file in a String
//available in java1.5
Scanner scan = new Scanner(new File("C:/workspace/Test/src/myfile.properties"));
scan.useDelimiter("\\Z");
String content = scan.next();
//Use apache StringEscapeUtils.escapeJava() method to escape java characters
ByteArrayInputStream bi=new ByteArrayInputStream(StringEscapeUtils.escapeJava(content).getBytes());
//load properties file
Properties properties = new Properties();
properties.load(bi);

It's not an exact answer to your question, but a different solution that may be appropriate to your needs. In Java, you can use / as a path separator and it'll work on both Windows, Linux, and OSX. This is specially useful for relative paths.
In your example, you could use:
dir = c:/mydir