Optimize Random Forest parameters in weka? - java

I am trying to optimize random forest parameters using weka, the java class is as the following:
package pkg10foldcrossvalidation;
import weka.core.*;
import weka.classifiers.meta.*;
import weka.classifiers.trees.RandomForest;
import java.io.*;
public class RF_Optimizer {
public static void main(String[] args) throws Exception {
// load data
BufferedReader reader = new BufferedReader(new FileReader("C:\\Prediction Results on the testing set\\Dataset.arff"));
Instances data = new Instances(reader);
reader.close();
data.setClassIndex(data.numAttributes() - 1);
// setup classifier
CVParameterSelection ps = new CVParameterSelection();
ps.setClassifier(new RandomForest());
ps.setNumFolds(10); // using 10-fold CV
ps.addCVParameter("C 0.1 0.5 5");
// build and output best options
ps.buildClassifier(data);
System.out.println(Utils.joinOptions(ps.getBestClassifierOptions()));
}
}
But I am facing difficulty of understanding which parameters should replace the "C" and how the range of each one could be determined? And is it workable to use .addCVParameter several times for several parameters at the same time?
I tried to search for some youtube or website tutorials that explain how to change random forest parameters in java but nothing found.
Thank you

I think what you are describing, -C are the Cross-Validation parameters, not the RandomForest parameters.
Can't you just use the Explorer GUI, open a sample dataset such as glass.arff, and then right-click on the bold RandomForest string at the top of the window, then from the context menu choose "copy configuration to clipboard", and then paste that string into your java code?
After doing this right now, I've copied this string to the clipboard:
weka.classifiers.trees.RandomForest -P 100 -I 100 -num-slots 1 -K 0 -M 1.0 -V 0.001 -S 1
These are the default parameters for Weka's RandomForest learner. What these parameters mean, and which of them is most suitable for optimization, and which range of values to use for optimization I really can't tell. Most likely a very important parameter is numIterations, the -I parameter. Maybe vary it from 100, 200,... to 1000 and plot numIterations vs Accuracy, and check if the curve has smoothed out already.

Related

Program that uses jpbc takes a long time

This is a very simple program I have written that uses jpbc library.
It compiles without any errors, but takes an unusually long time to show the output, or in fact it doesn't show the output at all. (Who in this era will have the patience to wait for nearly half an hour for such a tiny program?) I am using a system with i7 processor but still this is the case.
Could anyone tell what might be wrong with this code?
import it.unisa.dia.gas.jpbc.*;
import it.unisa.dia.gas.plaf.jpbc.pairing.PairingFactory;
import it.unisa.dia.gas.plaf.jpbc.pairing.parameters.*;
import it.unisa.dia.gas.jpbc.PairingParametersGenerator;
import it.unisa.dia.gas.jpbc.PairingParameters;
import it.unisa.dia.gas.plaf.jpbc.pairing.a1.TypeA1CurveGenerator;
public class PairingDemo {
public static void main(String [] args){
try{
int rBits = 160;
int qBits = 512;
PairingParametersGenerator pg = new TypeA1CurveGenerator(rBits, qBits);
PairingParameters params = pg.generate();
Pairing pair = PairingFactory.getPairing("D:\\JPBCLib\\params\\curves\\a1.Properties");
Field Zr = pair.getZr();
int degree = pair.getDegree();
System.out.println("Degree of the pairing : " + degree);
}catch(Exception e){
e.printStackTrace();
}
}
}
There are three issues that you're dealing with here
Generating the pairing parameters takes some time, but this only has to be done once for a system that you're building. You should store the generated pairing parameter for later use.
Since you're not using pg or params, you can remove that code. Instead you're reading precomputed parameters from a file.
jPBC is a complete and pure Java implementation of PBC. It is fully portable and therefore quite slow. jPBC has an option of using a PBCWrapper library which is a wrapper around libpbc which would enable you to get the performance of the native library. I wasn't able to make it work on Windows, but Linux should not be an issue (make sure to check the JNI version or load your own).

How to execute a python script from a Java program using jython? [duplicate]

What's the easiest way to execute a Python script from Java, and receive the output of that script? I've looked for different libraries like Jepp or Jython, but most appear out of date. Another problem with the libraries is that I need to be able to easily include a library with the source code (though I don't need to source for the library itself) if I use a library.
Because of this, would the easiest/most effective way be to simply do something like call the script with runtime.exec, and then somehow capture printed output? Or, even though it would be very painful for me, I could also just have the Python script output to a temporary text file, then read the file in Java.
Note: the actual communication between Java and Python is not a requirement of the problem I am trying to solve. This is, however, the only way I can think of to easily perform what needs to be done.
Not sure if I understand your question correctly, but provided that you can call the Python executable from the console and just want to capture its output in Java, you can use the exec() method in the Java Runtime class.
Process p = Runtime.getRuntime().exec("python yourapp.py");
You can read up on how to actually read the output from this resource:
http://www.devdaily.com/java/edu/pj/pj010016
import java.io.*;
public class JavaRunCommand {
public static void main(String args[]) {
String s = null;
try {
// run the Unix "ps -ef" command
// using the Runtime exec method:
Process p = Runtime.getRuntime().exec("ps -ef");
BufferedReader stdInput = new BufferedReader(new
InputStreamReader(p.getInputStream()));
BufferedReader stdError = new BufferedReader(new
InputStreamReader(p.getErrorStream()));
// read the output from the command
System.out.println("Here is the standard output of the command:\n");
while ((s = stdInput.readLine()) != null) {
System.out.println(s);
}
// read any errors from the attempted command
System.out.println("Here is the standard error of the command (if any):\n");
while ((s = stdError.readLine()) != null) {
System.out.println(s);
}
System.exit(0);
}
catch (IOException e) {
System.out.println("exception happened - here's what I know: ");
e.printStackTrace();
System.exit(-1);
}
}
}
There is also an Apache library (the Apache exec project) that can help you with this. You can read more about it here:
http://www.devdaily.com/java/java-exec-processbuilder-process-1
http://commons.apache.org/exec/
You can include the Jython library in your Java Project. You can download the source code from the Jython project itself.
Jython does offers support for JSR-223 which basically lets you run a Python script from Java.
You can use a ScriptContext to configure where you want to send your output of the execution.
For instance, let's suppose you have the following Python script in a file named numbers.py:
for i in range(1,10):
print(i)
So, you can run it from Java as follows:
public static void main(String[] args) throws ScriptException, IOException {
StringWriter writer = new StringWriter(); //ouput will be stored here
ScriptEngineManager manager = new ScriptEngineManager();
ScriptContext context = new SimpleScriptContext();
context.setWriter(writer); //configures output redirection
ScriptEngine engine = manager.getEngineByName("python");
engine.eval(new FileReader("numbers.py"), context);
System.out.println(writer.toString());
}
And the output will be:
1
2
3
4
5
6
7
8
9
As long as your Python script is compatible with Python 2.5 you will not have any problems running this with Jython.
I met the same problem before, also read the answers here, but doesn't found any satisfy solution can balance the compatibility, performance and well format output, the Jython can't work with extend C packages and slower than CPython. So finally I decided to invent the wheel myself, it took my 5 nights, I hope it can help you too: jpserve(https://github.com/johnhuang-cn/jpserve).
JPserve provides a simple way to call Python and exchange the result by well format JSON, few performance loss. The following is the sample code.
At first, start jpserve on Python side
>>> from jpserve.jpserve import JPServe
>>> serve = JPServe(("localhost", 8888))
>>> serve.start()
INFO:JPServe:JPServe starting...
INFO:JPServe:JPServe listening in localhost 8888
Then call Python from JAVA side:
PyServeContext.init("localhost", 8888);
PyExecutor executor = PyServeContext.getExecutor();
script = "a = 2\n"
+ "b = 3\n"
+ "_result_ = a * b";
PyResult rs = executor.exec(script);
System.out.println("Result: " + rs.getResult());
---
Result: 6
Jep is anther option. It embeds CPython in Java through JNI.
import jep.Jep;
//...
try(Jep jep = new Jep(false)) {
jep.eval("s = 'hello world'");
jep.eval("print(s)");
jep.eval("a = 1 + 2");
Long a = (Long) jep.getValue("a");
}
I've looked for different libraries like Jepp or Jython, but most seem to be very out of date.
Jython is not "a library"; it's an implementation of the Python language on top of the Java Virtual Machine. It is definitely not out of date; the most recent release was Feb. 24 of this year. It implements Python 2.5, which means you will be missing a couple of more recent features, but it is honestly not much different from 2.7.
Note: the actual communication between Java and Python is not a requirement of the aforementioned assignment, so this isn't doing my homework for me. This is, however, the only way I can think of to easily perform what needs to be done.
This seems extremely unlikely for a school assignment. Please tell us more about what you're really trying to do. Usually, school assignments specify exactly what languages you'll be using for what, and I've never heard of one that involved more than one language at all. If it did, they'd tell you if you needed to set up this kind of communication, and how they intended you to do it.
Jython approach
Java is supposed to be platform independent, and to call a native application (like python) isn't very platform independent.
There is a version of Python (Jython) which is written in Java, which allow us to embed Python in our Java programs. As usually, when you are going to use external libraries, one hurdle is to compile and to run it correctly, therefore we go through the process of building and running a simple Java program with Jython.
We start by getting hold of jython jar file:
https://www.jython.org/download.html
I copied jython-2.5.3.jar to the directory where my Java program was going to be. Then I typed in the following program, which do the same as the previous two; take two numbers, sends them to python, which adds them, then python returns it back to our Java program, where the number is outputted to the screen:
import org.python.util.PythonInterpreter;
import org.python.core.*;
class test3{
public static void main(String a[]){
PythonInterpreter python = new PythonInterpreter();
int number1 = 10;
int number2 = 32;
python.set("number1", new PyInteger(number1));
python.set("number2", new PyInteger(number2));
python.exec("number3 = number1+number2");
PyObject number3 = python.get("number3");
System.out.println("val : "+number3.toString());
}
}
I call this file "test3.java", save it, and do the following to compile it:
javac -classpath jython-2.5.3.jar test3.java
The next step is to try to run it, which I do the following way:
java -classpath jython-2.5.3.jar:. test3
Now, this allows us to use Python from Java, in a platform independent manner. It is kind of slow. Still, it's kind of cool, that it is a Python interpreter written in Java.
ProcessBuilder is very easy to use.
ProcessBuilder pb = new ProcessBuilder("python","Your python file",""+Command line arguments if any);
Process p = pb.start();
This should call python. Refer to the process approach here for full example!
https://bytes.com/topic/python/insights/949995-three-ways-run-python-programs-java
You can try using groovy. It runs on the JVM and it comes with great support for running external processes and extracting the output:
http://groovy.codehaus.org/Executing+External+Processes+From+Groovy
You can see in this code taken from the same link how groovy makes it easy to get the status of the process:
println "return code: ${ proc.exitValue()}"
println "stderr: ${proc.err.text}"
println "stdout: ${proc.in.text}" // *out* from the external program is *in* for groovy
First I would recommend to use ProcessBuilder ( since 1.5 )
Simple usage is described here
https://stackoverflow.com/a/14483787
For more complex example refer to
http://www.javaworld.com/article/2071275/core-java/when-runtime-exec---won-t.html
I've encountered problem when launching Python script from Java, script
was producing too much output to standard out and everything went bad.
The best way to achieve would be to use Apache Commons Exec as I use it for production without problems even for Java 8 environment because of the fact that it lets you execute any external process (including python, bash etc) in synchronous and asynchronous way by using watchdogs.
CommandLine cmdLine = new CommandLine("python");
cmdLine.addArgument("/my/python/script/script.py");
DefaultExecuteResultHandler resultHandler = new DefaultExecuteResultHandler();
ExecuteWatchdog watchdog = new ExecuteWatchdog(60*1000);
Executor executor = new DefaultExecutor();
executor.setExitValue(1);
executor.setWatchdog(watchdog);
executor.execute(cmdLine, resultHandler);
// some time later the result handler callback was invoked so we
// can safely request the exit value
resultHandler.waitFor();
Complete source code for a small but complete POC is shared here that addresses another concern in this post;
https://github.com/raohammad/externalprocessfromjava.git

Create a JAR file in eclipse dynamically from the code

I am currently working on a TESTING a game project. Below are the details about my project.
This is my first Maven Project and I really have very less idea on it's working methodology.Please help me!
1] GameProject - Is a Maven Project in Eclipse. The main goals of this project is to create a game player (also called bot) for a First Person Shooter game. When you run this Java Application, it will run the game, with a new player, whose actions will be determined by the code I wrote up in this project.
It has lot of java files, and one main class. There is a class called PlayerInfo.Java, which contains the code to decide, what the player should do while playing the game. It is like the brain of the player. Let's say it has around 40 variables, which decides the behavior of the Player.
Example - One of the variable in the class may be like Player_HEALTH_Level = 50. This means that Player should have atleast 50% of the health to be able to fight with other players. If it's less than this, then He/she should find a health Pack and increase the value. Similarly I have few other 39 parameters.
2] Testing Project - This too is a maven Project. Which is designed to test my Player(bot) performance in the game. Such as, how many times he wins the match, number of weapons he collects, etc.
The way this project is designed that, it requires the path for the JAR file of the GameProject as a input in the Main Method. Then, It runs the bot/player for the given JAR and prints out the results of the game in a separate file.
Example -
In the Main Method I specify something like this
String playerJAR_Path = "C:\Users\Player\Netbeans\GameProject-one-SNAPSHOT-jar.jar"
Note:- The above is the path for GameProject JAR folder. Basically, I just build the GameProject as "Maven Install" and it will create a JAR for me. Then use the path address on the second Project.
Hope this gives you some idea about my project structure.
Issue
The real issue is I have to basically run my GameProject around 500 times and capture the results. I am using Genetic Algorithms to evolve the Player Parameters and improve them over the time.
Currently, I can have a for Loop in my TestingProject, to loop 500 Iterations and it takes cares of executing the GameProject 500 times for me.
In order to apply Genetic Algorithms, After the end of EACH iteration, I have modify the parameters in the Player/bot Behavior (Specifically in file PlayerInfo.Java). Something like, I may change the value for Player_HEALTH_Level=48 for the iteration 2 run, then to 56 on the third run and so on.
In oreder to do this, I need to acheive the follwing in my Testing Project inside my FOR Loop.
Perhaps may be like this
*For int i=1 to 500 do ;
String playerJAR_Path = "C:\\Users\\Player\\Netbeans\\GameProject-one-SNAPSHOT-jar.jar";
Run the Testing();
Modify the parameters in PlayerInfo.Java (Inside GameProject)
Change the values of 40 Variables in that class.
Build the GameProject Project so that, it creates a UPDATED JAR File
End FOR;*
I am really confused, as in how to build the first project automatically, and also how to change the values of the class variables.
Can any one please help me on giving some suggestions on how to achieve the same. I don't mind in what way to do this. Just be able to AUTOMATE This whole testing Process.
Other details:
1. System - Windows 7, Intel Xeon 12 Core Processor, 24GB RAM, 3 TB Flash Drive.
2. For each iteration on the For-Loop, it produces a million record (As in the console window of the Eclipse will contain a Million lines for each iteration in for loop). I also need to know, how to write this to a file, instead of flooding console.
When I tested this initially, Eclipse hang up with Out of Heap Memory Error. Then I have increased all RAM values to 2GB in the eclipse INI file. This somehow managed not to crash my eclipse with memory errors.
EDIT - Adding code to Question.
public class PlayerInfo {
public int minDistanceToFollowEnemy = 200; // This is in UT Units. If Bot sees an enemy over this distance
// He/she tries to follow him and attack.
public int probabilityToFollowEnemy = 50; // Distance between the bot and Enemy will be calculated in real time.
//If the distance is lower, chasing priority is Higher.
public int probabilityOfFlagHolderReturn = 50; // If the Bot holds the Enemy flag, what's the probability that he/she
//Runs to home base directly without being distracted to collect items, fight with enemies?
public int probabilityOfGettingEnemyFlag = 10; // If bot does not carries the enemy flag, it's probability to hunt for enemy flag.
public double maxDistanceToGetHealthPack = 1000 ; // If Bot finds a health pack within this distance && It needs a Health pack
// Then It searches one and grabs it.
public int minHealthLevel = 30; // Min Health level before bot starts looking for Health Pack
public int probabilityToGetHealthPack=80; // Probability that Bot will Pick one health pack if it sees on its path/view.
public int probabilityToSelectItem = 8 ; //Based on the current Bot's view, what's the probability that it will select to choose the item.
}
Main Class PlayerBot.java IN GameProject
public static void main(String args[]) throws PogamutException {
AddNativeBot addbot = new AddNativeBot();
//addbot.addNativeBots(4);
// starts 2 or 4 CTFBots at once
// note that this is the most easy way to get a bunch of bots running at the same time
new UT2004BotRunner<UT2004Bot, UT2004BotParameters>(CTFBot.class, "PlayerInfo").setMain(true)
.startAgents(
new CTFBotParams().setBotSkin("HumanMaleA.MercMaleC").setBotType(0).setSkillLevel(5).setTeam(0).setAgentId(new AgentId("Attacker 1"))
//,new CTFBotParams().setBotSkin("HumanFemaleA.MercFemaleA").setSkillLevel(5).setTeam(0).setAgentId(new AgentId("Attacker 2"))
//,new CTFBotParams().setBotSkin("HumanMaleA.MercMaleA") .setSkillLevel(5).setTeam(0).setAgentId(new AgentId("Attacker 3"))
//,new CTFBotParams().setBotSkin("HumanFemaleA.MercFemaleB").setSkillLevel(5).setTeam(0).setAgentId(new AgentId("Attacker 4"))
//,new CTFBotParams().setBotSkin("HumanFemaleA.MercFemaleB").setSkillLevel(5).setTeam(1).setAgentId(new AgentId("Attacker 5"))
//,new CTFBotParams().setBotSkin("HumanFemaleA.MercFemaleB").setSkillLevel(5).setTeam(1).setAgentId(new AgentId("Attacker 6"))
);
}
//Project 2
public static String[] getArgs_CTF_2v2v1() {
return new String[] {
"-y", // MATCH TYPE
"CTF", // CAPTURE THE FALG
// GENERIC CONFIG
"-u",
"C:\\UT",
"-h", // human-like-log
"-r",
"./results",
"-n",
"Test-CTF-2v2v1", // MATCH NAME
"-s",
"CTFServer",
// CUSTOM BOTS CONFIG
"-a",
"C:\\Users\\Project\\Downloads\\CTFbot\\target\\mavenproject1-1.0-SNAPSHOT.one-jar.jar;C:\\Users\\Project\\Downloads\\CTFbot\\target\\mavenproject1-1.0-SNAPSHOT.one-jar.jar",
"-b",
"CTFBot1;CTFBot2",
"-l",
"1;2",
"-k",
"HumanFemaleA.NightFemaleA;HumanFemaleA.NightFemaleA",
"-i",
"0;1",
// NATIVE BOTS CONFIG
"-c", // NATIVE BOT COUNT
"2",
"-d", // NATIVE BOT NAME
"Native1;Native2",
"-e", // NATIVE BOT SKILL
"5;6",
"-g", // NATIVE BOT TEAMS
"0;1",
// HUMANS CONFIG
"-x",
"1", // HUMAN COUNT
"-z",
"1",
// CAPTURE THE FLAG SPECIFIC CONFIG
"-m",
"CTF-LostFaith",
"-f",
"1", // SCORE LIMIT
"-t",
"5", // TIME LIMIT
};
}
//Main Class
public static void main(String[] args) throws JSAPException {
// -----------
// FOR TESTING
// -----------
//args = getArgs_DM_2v2v1();
//args = getArgs_TDM_2v2v1();
args = getArgs_CTF_2v2v1();
// --------------
// IMPLEMENTATION
// --------------
initJSAP();
header();
readConfig(args);
sanityChecks();
switch (matchType) {
case DM:
executeDeathMatch();
break;
case TDM:
executeTeamDeathMatch();
break;
case CTF:
executeCaptureTheFlag();
break;
case DD:
executeDoubleDomination();
break;
default:
fail("Unsupported match type specified " + matchTypeName + " recognized as " + matchType.shortName + "[" + matchType.name + "].");
}
}
}
Holy wall of text.
Ok, couple of things. Just because Maven created a jar for you, doesn't mean you have to use the jar. You could call the class directly.
So your testing would instantiate the class, instead of trying to run the class from the command line.
Second. You don't have to modify the internal state of the variables directly. My advice is for you to save the changes to a file and have your Player read the file. For the next iteration you modify the file and run the Player again.
Let me know if you have specific questions and I'll improve the answer.

Stanford Topic Modeling Toolbox: Exception

I'm trying to work with the Stanford Topic Modeling Toolbox. I downloaded the "tmt-0.4.0.jar"-File from here: http://nlp.stanford.edu/software/tmt/tmt-0.4/ and I tried to examples.
Example 0 and 1 worked fine, but trying example 2 (no code-changes), I receive the following exception:
[cell] loading pubmed-oa-subset.csv.term-counts.cache.70108071.gz
[Concurrent] 32 permits Exception in thread "Thread-3"
java.lang.ArrayIndexOutOfBoundsException: -1 at
scalanlp.stage.text.TermCounts$class.getDF(TermFilters.scala:64) at
scalanlp.stage.text.TermCounts$$anon$2.getDF(TermFilters.scala:84) at
scalanlp.stage.text.TermMinimumDocumentCountFilter$$anonfun$apply$4$$anonfun$apply$5$$anonfun$apply$6.apply(TermFilters.scala:172)
at
scalanlp.stage.text.TermMinimumDocumentCountFilter$$anonfun$apply$4$$anonfun$apply$5$$anonfun$apply$6.apply(TermFilters.scala:172)
at scala.collection.Iterator$$anon$22.hasNext(Iterator.scala:390) at
scala.collection.Iterator$$anon$22.hasNext(Iterator.scala:388) at
scala.collection.Iterator$class.foreach(Iterator.scala:660) at
scala.collection.Iterator$$anon$22.foreach(Iterator.scala:382) at
scala.collection.IterableViewLike$Transformed$class.foreach(IterableViewLike.scala:41)
at
scala.collection.IterableViewLike$$anon$5.foreach(IterableViewLike.scala:82)
at
scala.collection.TraversableOnce$class.size(TraversableOnce.scala:104)
at
scala.collection.IterableViewLike$$anon$5.size(IterableViewLike.scala:82)
at
scalanlp.stage.text.DocumentMinimumLengthFilter.filter(DocumentFilters.scala:31)
at
scalanlp.stage.text.DocumentMinimumLengthFilter.filter(DocumentFilters.scala:28)
at
scalanlp.stage.generic.Filter$$anonfun$apply$1.apply(Filter.scala:38)
at
scalanlp.stage.generic.Filter$$anonfun$apply$1.apply(Filter.scala:38)
at scala.collection.Iterator$$anon$22.hasNext(Iterator.scala:390) at
edu.stanford.nlp.tmt.data.concurrent.Concurrent$$anonfun$map$2.apply(Concurrent.scala:100)
at
edu.stanford.nlp.tmt.data.concurrent.Concurrent$$anonfun$map$2.apply(Concurrent.scala:88)
at
edu.stanford.nlp.tmt.data.concurrent.Concurrent$$anon$4.run(Concurrent.scala:45)
Why do I receive this exception, and how can this be fixed?
Thanks a lot for your help!
PS: The code is the same as in example 2 of the website:
// Stanford TMT Example 2 - Learning an LDA model
// http://nlp.stanford.edu/software/tmt/0.4/
// tells Scala where to find the TMT classes
import scalanlp.io._;
import scalanlp.stage._;
import scalanlp.stage.text._;
import scalanlp.text.tokenize._;
import scalanlp.pipes.Pipes.global._;
import edu.stanford.nlp.tmt.stage._;
import edu.stanford.nlp.tmt.model.lda._;
import edu.stanford.nlp.tmt.model.llda._;
val source = CSVFile("pubmed-oa-subset.csv") ~> IDColumn(1);
val tokenizer = {
SimpleEnglishTokenizer() ~> // tokenize on space and punctuation
CaseFolder() ~> // lowercase everything
WordsAndNumbersOnlyFilter() ~> // ignore non-words and non-numbers
MinimumLengthFilter(3) // take terms with >=3 characters
}
val text = {
source ~> // read from the source file
Column(4) ~> // select column containing text
TokenizeWith(tokenizer) ~> // tokenize with tokenizer above
TermCounter() ~> // collect counts (needed below)
TermMinimumDocumentCountFilter(4) ~> // filter terms in <4 docs
TermDynamicStopListFilter(30) ~> // filter out 30 most common terms
DocumentMinimumLengthFilter(5) // take only docs with >=5 terms
}
// turn the text into a dataset ready to be used with LDA
val dataset = LDADataset(text);
// define the model parameters
val params = LDAModelParams(numTopics = 30, dataset = dataset,
topicSmoothing = 0.01, termSmoothing = 0.01);
// Name of the output model folder to generate
val modelPath = file("lda-"+dataset.signature+"-"+params.signature);
// Trains the model: the model (and intermediate models) are written to the
// output folder. If a partially trained model with the same dataset and
// parameters exists in that folder, training will be resumed.
TrainCVB0LDA(params, dataset, output=modelPath, maxIterations=1000);
// To use the Gibbs sampler for inference, instead use
// TrainGibbsLDA(params, dataset, output=modelPath, maxIterations=1500);
The answer has been posted by the author of the tool. Please have a look here.
This usually happens when you have a stale .cache file - unfortunately the
error message isn't particularly useful. Try deleting cache in the run
folder and running again.
https://lists.cs.princeton.edu/pipermail/topic-models/2012-July/001979.html

Using/Reading FMOD (.fsb) Files in Java

Heyo Everyone. I am currently working on an Application which should play music files from an FMOD Database. It currently does it by Extracting the files (using a runtime and an external programm called "fsbextract.exe" (link)) as MP3 and then playing them. I am okay with this as it is, but i would now also love to edit/replace the files within the .fsb file. So my question is: Can i somehow directly acces the MP3 files in there without extracting them? I searched the internet for something like this but couldn't find any help.
For reference, here are some informations i can get from within fsbextract:
FileID is |FSB4> | Version is 4.0 | Number of Entries 7412
Global Flags:
0x40 | FMOD_FSB_SOURCE_MPEG_PADDED4 | MPEG frames are aligned to the nearest 4 bytes, 16 bytes for multichannel (Use Frame Verification option)
And from the first (index 0) file within the fsb file:
Format: MP3 (85)
Sample rate: 44100Hz
Channels: Mono
Duration: 01.541
Bitrate: 160,62 kbit/s
Bits/Sample 3.64
Sample Mode Flags [0x10000200]
0x200 | FSOUND_MPEG | Sample is stored in MPEG format
0x10000000 | FSOUND_MPEG_LAYER3 | Samples are stored in MPEG Layer 3 format
I hope someone knows more than me, but thanks in advance
Marenthyu
Jérôme Jouvie wrote a JNI wrapper for FMOD 4+, NativeFmodEx. If your FSB files are not encrypted I think you can use the API to extract streaming data from the FSB banks.
I tried to write my own wrapper as well but I didn't have the time to get past a simple proof of concept.
UPDATE: Sure. So basically what I did was to use JNAerator to wrap the native FMODex libraries and then targeting the resulting glue code to the BridJ runtime environment. That essentially allows you to invoke the native FMOD C library.
I'm not exactly sure if Jerome did something similar or if he created his own JNI stubs to achieve the same result. You'd have to check out and read his project's code.
But essentially, once you're able to make FMOD calls inside your application, you can use the API to do as you please. If you download the latest FMOD studio or FMODex SDK's from the FMOD web site you'll find a help .chm file which contains some documentation for the API.
You should also take a look at the examples in the SDK.
The following code is basically one example translated into Java, using the above mentioned strategy. The code isn't portable, robust or actually useful for an application. It really needs cleanup yet it still illustrates the point.
Like I said, you should be much better off using Jerome's SDK port.
Hope it helps.
package net.unsungstories.fmodex;
import net.unsungstories.fmodex.FmodexLibrary.FMOD_CHANNEL;
import net.unsungstories.fmodex.FmodexLibrary.FMOD_SOUND;
import net.unsungstories.fmodex.FmodexLibrary.FMOD_SYSTEM;
import org.apache.log4j.Logger;
import org.bridj.IntValuedEnum;
import org.bridj.Platform;
import org.bridj.Pointer;
import org.junit.Test;
import static org.bridj.Pointer.*;
import static net.unsungstories.fmodex.FmodexLibrary.*;
import static com.google.common.base.Throwables.*;
import static java.lang.String.format;
public class PlaySound {
private static final Logger log = Logger.getLogger(PlaySound.class);
#Test
public void playSound() {
log.info("Test started...");
String soundPath = "./src/test/resources/picus_get_to_finicular_music_0.fsb";
Platform.addEmbeddedLibraryResourceRoot("net/unsungstories/fmodex/");
IntValuedEnum<FMOD_RESULT> result;
Pointer<Pointer<FMOD_SYSTEM>> ppSystem = allocatePointer(FMOD_SYSTEM.class);
Pointer<Pointer<FMOD_SOUND>> ppSound1 = allocatePointer(FMOD_SOUND.class);
Pointer<Integer> pSubSounds = allocateInt();
Pointer<Pointer<FMOD_CHANNEL>> ppChannel = allocatePointer(FMOD_CHANNEL.class);
#SuppressWarnings("unchecked")
Pointer<FMOD_CREATESOUNDEXINFO> soundExInfo = Pointer.NULL;
Pointer<Byte> targetSoundPath = allocateBytes(soundPath.length() + 1);
targetSoundPath.setCString(soundPath);
result = FMOD_System_Create(ppSystem);
result = FMOD_System_Init(ppSystem.get(), 2, FmodexLibrary.FMOD_INIT_NORMAL, Pointer.NULL);
result = FMOD_System_CreateSound(ppSystem.get(), targetSoundPath, FMOD_HARDWARE, soundExInfo, ppSound1);
result = FMOD_Sound_GetNumSubSounds(ppSound1.get(), pSubSounds);
try {
Pointer<Integer> pChanelPlaying = allocateInt();
for (int k = 0; k < pSubSounds.get(); k++) {
Pointer<Pointer<FMOD_SOUND>> ppSubSound = allocatePointer(FMOD_SOUND.class);
result = FMOD_Sound_GetSubSound(ppSound1.get(), k, ppSubSound);
result = FMOD_System_PlaySound(ppSystem.get(), FMOD_CHANNELINDEX.FMOD_CHANNEL_FREE, ppSubSound.get(), 0, ppChannel);
FMOD_Channel_IsPlaying(ppChannel.get(), pChanelPlaying);
while (ppChannel.getBoolean()) {
log.info("Playing...");
Thread.sleep(1000);
FMOD_Channel_IsPlaying(ppChannel.get(), pChanelPlaying);
}
result = FMOD_Sound_Release(ppSubSound.get());
}
result = FMOD_System_Close(ppSystem.get());
result = FMOD_System_Release(ppSystem.get());
} catch (Exception e) {
log.error(getStackTraceAsString(getRootCause(e)));
}
log.info(format("Finished... %s", result));
}
}
Hope this helps.

Categories