I create a process that measure similarity. I use ExampleSet2SimilarityExampleSetoperator and I want to add in input an example set which generated in my code.
The first idea is to write my example set in local repository. Then I will use retrieve operator to read example set and connect the output of operator retrieve to input of operator ExampleSet2SimilarityExampleSet.
So I wonder if I can avoid read/write.
public class Test {
public static void main(String[] args) throws OperatorCreationException, OperatorException{
Test t = new Test();
t.createprocess();
}
public void createprocess() throws OperatorCreationException, OperatorException{
RapidMiner.init();
ExampleSet exampleset = getExampleset();
Operator silimarityOperator = OperatorService.createOperator(ExampleSet2SimilarityExampleSet.class);
// silimarityOperator.setParameter("measure_type", "NumericalMeasures");
// silimarityOperator.setParameter("numerical_measure", "CosineSimilarity");
Process process = new Process();
process.getRootOperator().getSubprocess(0).addOperator(silimarityOperator);
process.getRootOperator().getSubprocess(0).getInnerSources().getPortByIndex(0).connectTo( silimarityOperator.getInputPorts().getPortByName("input"));
// run the process with new IOContainer using the created exampleSet
IOContainer run = process.run(new IOContainer(exampleset));
System.out.println(run.toString());
}
public ExampleSet getExampleset() {
// construct attribute set
Attribute[] attributes = new Attribute[3];
attributes[0] = AttributeFactory.createAttribute("Topic1", Ontology.STRING);
attributes[1] = AttributeFactory.createAttribute("Topic2", Ontology.STRING);
attributes[2] = AttributeFactory.createAttribute("Topic3", Ontology.STRING);
MemoryExampleTable table = new MemoryExampleTable(attributes);
DataRowFactory ROW_FACTORY = new DataRowFactory(0);
Double[] strings = new Double[3];
double a = 0;
for (int i = 0; i < 3; i++) {
a++;
strings[i] = a;
// make and add row
DataRow row = ROW_FACTORY.create(strings, attributes);
table.addDataRow(row);
}
ExampleSet exampleSet = table.createExampleSet();
return exampleSet;
}
}
I found solution. I have error at connection of operator. It is not exist port input, so I get port 0.
process.getRootOperator().getSubprocess(0).getInnerSources().getPortByIndex(0).connectTo(
silimarityOperator.getInputPorts().getPortByIndex(0));
silimarityOperator.getOutputPorts().getPortByIndex(0).connectTo(
process.getRootOperator().getSubprocess(0).getInnerSinks().getPortByIndex(0));
Related
So i've been trying to solve this issue for hours but cant seem to find an answer which would work.
i have an object array which stores flight information and i had to remove flights which had Valstybe: "Maldyvai"
so i made a new object array without them, but when i try to print it i get a memory location.
How do i convert the object array to string array?
even though i have a tostring method in my java class
package com.company;
import java.util.*;
import com.company.Isvestine.OroUostasKeleivis;
public class Main {
public static void main(String[] args) {
// write your code here
OroUostasKeleivis Keleiviai1 = new OroUostasKeleivis("Skrydis","Washington","JAV","Tomas","tomaitis","Washington",5465);
OroUostasKeleivis Keleiviai2 = new OroUostasKeleivis("Skrydis","Washington","Maldyvai","Tomas","tomaitis","Maldyvai",5466);
OroUostasKeleivis Keleiviai3 = new OroUostasKeleivis("Skrydis","Washington","JAV","Tomas","tomaitis","Washington",5467);
OroUostasKeleivis Keleiviai4 = new OroUostasKeleivis("Skrydis","Washington","Maldyvai","Tomas","tomaitis","Maldyvai",5468);
OroUostasKeleivis Keleiviai5 = new OroUostasKeleivis("Skrydis","Washington","JAV","Tomas","tomaitis","Washington",5469);
OroUostasKeleivis Keleiviai6 = new OroUostasKeleivis("Skrydis","Washington","Maldyvai","Tomas","tomaitis","Maldyvai",5470);
OroUostasKeleivis Keleiviai7 = new OroUostasKeleivis("Skrydis","Washington","JAV","Tomas","tomaitis","Washington",5475);
OroUostasKeleivis Keleiviai8 = new OroUostasKeleivis("Skrydis","Washington","Maldyvai","Tomas","tomaitis","Maldyvai",5476);
OroUostasKeleivis Keleiviai9 = new OroUostasKeleivis("Skrydis","Washington","JAV","Tomas","tomaitis","Washington",5477);
OroUostasKeleivis Keleiviai10 = new OroUostasKeleivis("Skrydis","Washington","JAV","Tomas","tomaitis","Washington",5488);
OroUostasKeleivis[] keleiviai = new OroUostasKeleivis[10];
keleiviai[0] = Keleiviai1;
keleiviai[1] = Keleiviai2;
keleiviai[2] = Keleiviai3;
keleiviai[3] = Keleiviai4;
keleiviai[4] = Keleiviai5;
keleiviai[5] = Keleiviai6;
keleiviai[6] = Keleiviai7;
keleiviai[7] = Keleiviai8;
keleiviai[8] = Keleiviai9;
keleiviai[9] = Keleiviai10;
for (OroUostasKeleivis keleiveliai:keleiviai) {
System.out.println(keleiveliai);
}
System.out.println("test debug");
OroUostasKeleivis[] keleiviaibemaldyvu = new OroUostasKeleivis[10];
for (int i = 0; i < 10; i++) {
}
System.out.println(IsstrintiMaldyvus(keleiviai));
String convertedStringObject = IsstrintiMaldyvus(keleiviai) .toString();
System.out.println(convertedStringObject );
}
static Object[] IsstrintiMaldyvus(OroUostasKeleivis[] keleiviai){
OroUostasKeleivis[] keleiviaiBeMaldyvu = new OroUostasKeleivis[10];
int pozicija = 0;
for ( OroUostasKeleivis keleiveliai: keleiviai) {
if (keleiveliai.getValstybe() != "Maldyvai"){
keleiviaiBeMaldyvu[pozicija] = keleiveliai;
pozicija++;
}
}
return keleiviaiBeMaldyvu;
}
}
but when i try to print it i get a memory location
Yes, you will NOT have result as you expected, especially calling toString() with any array. See documentation of java.lang.Object.toString() for more details.
So how can we solve problem?
first, override toString() method in OroUostasKeleivis like this:
class OroUostasKeleivis {
#Override
public String toString() {
// your implementation here
return null; // TODO: change here
}
}
Second, you may do either way:
If you're interested in just print out, you can do that with System.out.println(keleiveliai) in for-each loop like you do.
If you're interested in converting OroUostasKeleivis[] to String[], you can:
// this requires Java 8 or later
String[] converted = Arrays.asList(keleiviai)
.stream()
.map(OroUostasKeleivis::toString)
.toArray(String[]::new);
// then use `converted`
Use System.out.println(Arrays.toString(IsstrintiMaldyvus(keleiviai)))
https://www.geeksforgeeks.org/arrays-tostring-in-java-with-examples/
It will print the array contents similar to how ArrayList would get printed if it had the same content.
Think of it as:
[ obj1.toString(), obj2.toString(), ... ]
Using java.util.Arrays#stream(T[]) filter and convert object array to string array and use java.util.Arrays#toString(java.lang.Object[]) convert array to readable string.
final String[] oroUostasKeleivis = Arrays.stream(keleiviai)
.filter(
k -> k.getValStybe() != "Maldyvai"
)
// or other convert code
.map(OroUostasKeleivis::toString)
.toArray(String[]::new);
System.out.println(Arrays.toString(oroUostasKeleivis));
My problem: I have a model engine that takes a list of parameter configuration, and evaluates a double value that corresponds to the metric associated to that configuration. I have six parameters, and each of them can vary according to a list. I want to find by brute force the best parameter configuration considering the combination that will produce the higher value for the output metric. Since I'm learning Spark, I realized that with the cartesian product operation I can easily generate the combinations, and split the RDD to be processed in parallel. So, I came up with this driver program:
public static void main(String[] args) {
String scriptName = "model.mry";
String scriptStr = null;
try {
scriptStr = new String(Files.readAllBytes(Paths.get(scriptName)));
} catch (IOException ex) {
Logger.getLogger(BruteForceDriver.class.getName()).log(Level.SEVERE, null, ex);
System.exit(1);
}
final String script = scriptStr;
SparkConf conf = new SparkConf()
.setAppName("wordCount")
.setSparkHome("/home/danilo/bin/spark-2.2.0-bin-hadoop2.7")
.setJars(new String[]{"/home/danilo/NetBeansProjects/SparkHello1/target/SparkHello1-1.0.jar",
"/home/danilo/.m2/repository/org/modcs/mercury/4.7/mercury-4.7.jar"})
.setMaster("spark://danilo-desktop:7077");
String baseDir = "/home/danilo/NetBeansProjects/SimulationOptimization/workspace/";
JavaSparkContext sc = new JavaSparkContext(conf);
final int NUM_SERVICES = 6;
final int QTD = 3;
JavaRDD<Service>[] providers = new JavaRDD[NUM_SERVICES];
for (int i = 1; i <= NUM_SERVICES; i++) {
providers[i - 1] = sc.textFile(baseDir + "provider"
+ i
+ ".mat")
.filter((t1) -> !t1.contains("#") && !t1.trim().isEmpty())
.map(Service.createParser("" + i))
.zipWithIndex().filter((t1) -> {
return t1._2 < QTD;
}).keys();
}
JavaPairRDD c = null;
JavaRDD<Service> p = providers[0];
for (int i = 1; i < NUM_SERVICES; i++) {
if (c == null) {
c = p.cartesian(providers[i]);
} else {
c = c.cartesian(providers[i]);
}
}
JavaRDD<List<Service>> cartesian = c.map(new FlattenTuple<>());
final Broadcast<ModelEvaluator> model = sc.broadcast(new ModelEvaluator(script));
JavaPairRDD<Double, List<Service>> results = cartesian.mapToPair(
(t) -> {
try {
double val = model.value().evaluateModel(t);
System.out.println(val);
return new Tuple2<>(val, t);
} catch (Exception ex) {
return null;
}
}
);
results.sortByKey().collect().forEach((t) -> {
System.out.println(t._1 + ", " + t._2);
});
sc.close();
}
The "QTD" variable allows me to control the size the interval which each parameter will vary. For QTD = 3, I'll have 3^6 = 729 combinations. The problem is that it is taking so long to compute all those combinations. I wrote a implementations using only normal Java threads, and the runtime is about 40 seconds. Using my Spark driver program, the runtime more than 6 minutes. Why is my Spark program so slow compared to the plain Java multi-thread program?
Edit:
I put:
results = results.cache();
before sorting the results and now the runtime is 2.5 minutes.
Edit 2:
I created a RDD with the cartesian product of the parameters by hand instead of using the operation provided by the framework. Now my runtime is 1'25''. It does make sense now, since there is some overhead to start the driver and move the jars to the workers.
I want to modify an existing *.rptdesign file and save it under a new name.
The existing file contains a Data Set with a template SQL select statement and several DS parameters.
I'd like to use an actual SQL select statement which uses only part of the DS parameters.
However, the following code results in the exception:
Exception in thread "main" `java.lang.RuntimeException`: *The structure is floating, and its handle is invalid!*
at org.eclipse.birt.report.model.api.StructureHandle.getStringProperty(StructureHandle.java:207)
at org.eclipse.birt.report.model.api.DataSetParameterHandle.getName(DataSetParameterHandle.java:143)
at org.eclipse.birt.report.model.api.DataSetHandle$DataSetParametersPropertyHandle.removeParamBindingsFor(DataSetHandle.java:851)
at org.eclipse.birt.report.model.api.DataSetHandle$DataSetParametersPropertyHandle.removeItems(DataSetHandle.java:694)
--
OdaDataSetHandle dsMaster = (OdaDataSetHandle) report.findDataSet("Master");
HashSet<String> bindVarsUsed = new HashSet<String>();
...
// find out which DS parameters are actually used
HashSet<String> bindVarsUsed = new HashSet<String>();
...
ArrayList<OdaDataSetParameterHandle> toRemove = new ArrayList<OdaDataSetParameterHandle>();
for (Iterator iter = dsMaster.parametersIterator(); iter.hasNext(); ) {
OdaDataSetParameterHandle dsPara = (OdaDataSetParameterHandle)iter.next();
String name = dsPara.getName();
if (name.startsWith("param_")) {
String bindVarName = name.substring(6);
if (!bindVarsUsed.contains(bindVarName)) {
toRemove.add(dsPara);
}
}
}
PropertyHandle paramsHandle = dsMaster.getPropertyHandle( OdaDataSetHandle.PARAMETERS_PROP );
paramsHandle.removeItems(toRemove);
What is wrong here?
Has anyone used the DE API to remove parameters from an existing Data Set?
I had similar issue. Resolved it by calling 'removeItem' multiple times and also had to re-evaluate parametersIterator everytime.
protected void updateDataSetParameters(OdaDataSetHandle dataSetHandle) throws SemanticException {
int countMatches = StringUtils.countMatches(dataSetHandle.getQueryText(), "?");
int paramIndex = 0;
do {
paramIndex = 0;
PropertyHandle odaDataSetParameterProp = dataSetHandle.getPropertyHandle(OdaDataSetHandle.PARAMETERS_PROP);
Iterator parametersIterator = dataSetHandle.parametersIterator();
while(parametersIterator.hasNext()) {
Object next = parametersIterator.next();
paramIndex++;
if(paramIndex > countMatches) {
odaDataSetParameterProp.removeItem(next);
break;
}
}
if(paramIndex < countMatches) {
paramIndex++;
OdaDataSetParameter dataSetParameter = createDataSetParameter(paramIndex);
odaDataSetParameterProp.addItem(dataSetParameter);
}
} while(countMatches != paramIndex);
}
private OdaDataSetParameter createDataSetParameter(int paramIndex) {
OdaDataSetParameter dataSetParameter = StructureFactory.createOdaDataSetParameter();
dataSetParameter.setName("param_" + paramIndex);
dataSetParameter.setDataType(DesignChoiceConstants.PARAM_TYPE_INTEGER);
dataSetParameter.setNativeDataType(1);
dataSetParameter.setPosition(paramIndex);
dataSetParameter.setIsInput(true);
dataSetParameter.setIsOutput(false);
dataSetParameter.setExpressionProperty("defaultValue", new Expression("<evaluation script>", ExpressionType.JAVASCRIPT));
return dataSetParameter;
}
I am using below code which reads a CSV file and passes the object to the method under test called as public void launchWCM1(IBLogonDataCSV data) .
#DataProvider(name = "regCSVData")
public static Object[][] getCSVData() throws IOException {
CSVReader csvReader = new CSVReader(new FileReader(
"C:\\Projects\\Project\\regdata.csv"));
List<String[]> dataList = csvReader.readAll();
String s = "";
Object[][] data = new Object[dataList.size()][1];
List<IBLogonDataCSV> regList = new ArrayList<IBLogonDataCSV>();
for (String[] strArray : dataList) {
IBLogonDataCSV ibLogonData = new IBLogonDataCSV();
ibLogonData.setURL((strArray[0].trim()));
ibLogonData.setApplicationName((strArray[1].trim()));
ibLogonData.setIBLogonURL(strArray[2].trim());
ibLogonData.setWindowName(strArray[3].trim());
ibLogonData.setSnapshotName(strArray[4].trim());
ibLogonData.setRegister(strArray[5].trim());
ibLogonData.setRegisterURL(strArray[6].trim());
ibLogonData.setDemo(strArray[7].trim());
ibLogonData.setDemoURL(strArray[8].trim());
ibLogonData.setSecurity(strArray[9].trim());
ibLogonData.setSecurityURL(strArray[10].trim());
regList.add(ibLogonData);
}
for (int i = 0; i < data.length; i++) {
for (int j = 0; j < data[i].length; j++) {
data[i][j] = regList.get(i);
System.out.println("In Array" + regList.get(i).URL
+ regList.get(i).ApplicationName);
s = s + regList.get(i).URL;
}
}
csvReader.close();
return data;
}
When the method public void launchWCM1(IBLogonDataCSV data) passes or fails the Testng Report displays the object name which is not helpful in knowing what is the String value or the name of the URL failed in order to further debug .
Example:launchWCM1
Test class: com.seleniumtests.test.IBLogonCSV
Parameters: com.seleniumtests.dataobject.IBLogonDataCSV#de1b8a
What should be done to display the parameters in as a String Value ,something which will can be easily read like :
launchWCM1
Test class: com.seleniumtests.test.IBLogonCSV
Parameters: Name of the URL or something like that instead of the object name
You can override toString() and implement it to return readable meaningful vaule of your class.
#Override
public String toString(){
String valueToReturn = "calculate from the properties";
return valueToReturn;
}
Other way is, there are few third party extensions to TestNG are available, You can utilize available data-provider-extension. For example, QAF has in-built data providers for XML/CSV/Excel/Json/DB.
I would like to have Java generate a '0' in case an XPath expression evaluates to 'false'.
I have this Java code:
//Read the input XML document
private SAXBuilder parser = new SAXBuilder();
private Document characters;
private XPath pProbs;
private List<Attribute> probs;
private Double[] dprobs;
private String pathToSourceXml;
private String content1, content2, content3, content4, content5;
characters = parser.build(pathToSourceXml);
pProbs = XPath.newInstance("/n-grams-sorted/n-gram[contains(.,"+content1+") or contains(.,"+content2+") or contains(.,"+content3+") or contains(.,"+content4+") or contains(.,"+content5+")]/#probability");
probs = (List<Attribute>) pProbs.selectNodes(characters);
...
//Return all the values of the #probability attibrutes
public Double[] getProbs(String pathToSourceXml) {
this.pathToSourceXml = pathToSourceXml;
List<Double> theDoubles = new ArrayList();
dprobs = new Double[5];
for (int i=0; i<probs.size(); i++) {
theDoubles.add(Double.parseDouble(probs.get(i).getValue()));
dprobs[i] = theDoubles.get(i);
}
return dprobs;
}
The problem here is that this code:
pProbs = XPath.newInstance("/n-grams-sorted/n-gram[contains(.,"+content1+") or contains(.,"+content2+") or contains(.,"+content3+") or contains(.,"+content4+") or contains(.,"+content5+")]/#probability");
only returns 4 elements because 'content1' returns 'false'. There are no n-gram nodes whose content contains the string '$ $ $'. However, the rest of the content nodes evaluates to 'true'.
If one of these contains() expressions evaluates to 'false', I must draw a '0' in my Xaml code. A '0' must appear on the screen, such as:
'# # #' = 0.0015
'? ? ?' = 0.0047
'k i d' = 0.0012
'$ $ $' = 0
I can't fetch this '0'. I don't know how to say: "return a zero in case that there are no nodes that contain '$ $ $' as content.
Any ideas on how to do this?
Thank you.
Here is the answer:
public class XPathCharacters {
private SAXBuilder parser = new SAXBuilder();
private Document characters;
private XPath pProbs, pContent1, pContent2, pContent3, pContent4, pContent5;
private List<Attribute> probs1, probs2, probs3, probs4, probs5;
private List<List<Attribute>> probs;
private List<String> noNodes;
private Double[] dprobs;
private String pathToSourceXml;
private String content1, content2, content3, content4, content5;
private int noNode;
public XPathCharacters(){
}
public XPathCharacters(String path, String content1,String content2,String content3,String content4,String content5){
setPathToSourceXml(path);
this.content1 = content1;
this.content2 = content2;
this.content3 = content3;
this.content4 = content4;
this.content5 = content5;
noNode = 0;
initiate();
}
public void initiate() {
try {
//Evaluate each XPath expression seperately
characters = parser.build(pathToSourceXml);
pContent1 = XPath.newInstance("/n-grams-sorted/n-gram[contains(.,"+content1+")]/#probability");
pContent2 = XPath.newInstance("/n-grams-sorted/n-gram[contains(.,"+content2+")]/#probability");
pContent3 = XPath.newInstance("/n-grams-sorted/n-gram[contains(.,"+content3+")]/#probability");
pContent4 = XPath.newInstance("/n-grams-sorted/n-gram[contains(.,"+content4+")]/#probability");
pContent5 = XPath.newInstance("/n-grams-sorted/n-gram[contains(.,"+content5+")]/#probability");
//Convert the result of the above XPath expressions to nodes
probs1 = (List<Attribute>) pContent1.selectNodes(characters);
probs2 = (List<Attribute>) pContent2.selectNodes(characters);
probs3 = (List<Attribute>) pContent3.selectNodes(characters);
probs4 = (List<Attribute>) pContent4.selectNodes(characters);
probs5 = (List<Attribute>) pContent5.selectNodes(characters);
} catch (JDOMException jdome) {
System.out.println("Error at XPathInformationgain.initiate(): JDOMException: " + jdome.getMessage());
} catch (IOException ioe) {
System.out.println("Error at XPathInformationgain.initiate(): IOException: " + ioe.getMessage());
}
}
private void setPathToSourceXml(String path) {
this.pathToSourceXml = path;
}
public Double[] getProbs(String pathToSourceXml) {
this.pathToSourceXml = pathToSourceXml;
probs = new ArrayList();
probs.add(probs1);
probs.add(probs2);
probs.add(probs3);
probs.add(probs4);
probs.add(probs5);
List<Double> theDoubles = new ArrayList();
dprobs = new Double[5];
noNodes = new ArrayList();
int j=0;
for (int i=0; i<5; i++) {
if (probs.get(i).size() > 0) {
System.out.println("i: "+i);
System.out.println("j: "+j);
//Add the value of all these nodes to an Array so you can work with them
theDoubles.add(Double.parseDouble(probs.get(i).get(0).getValue()));
dprobs[j] = theDoubles.get(j);
}
else {
//If one of the values happens to be a zero-length array, add 0.0 to the array of values
theDoubles.add(0.0);
}
}
return dprobs;
}
public List<String> getNoNodes() {
return noNodes;
}
}
(This code is not working for what I want to accomplish outside of this question, but it works for this question because I have a way of returning a '0' in java, when the XPath expression yields zero-length).
The solution is this part really:
else {
theDoubles.add(0.0);
}
I would like to have Java generate a '0' in case an XPath expression
evaluates to 'false'.
This XPath expression:
number(boolean(yourExpression))
evaluates to 0 exactly when boolean(yourExpression) is false().
You need simply to substitute yourExpression above with your expression (which isn't at all clear from the question) and to evaluate this XPath expression with the XPath related classes/methods available to you.