I'm trying to make working example at How to write a custom Protobuf CodeGenerator in Java . When I try to compile a file with
import com.google.protobuf.compiler.PluginProtos;
import java.io.IOException;
public class MyPlugin {
public static void main(String[] args) throws IOException {
CodeGenerator gen = new CodeGenerator();
PluginProtos.CodeGeneratorRequest codeGeneratorRequest = PluginProtos.CodeGeneratorRequest.parseFrom(System.in);
codeGeneratorRequest.getProtoFileList().forEach(gen::handleFile);
// get the response and do something with it
//PluginProtos.CodeGeneratorResponse response = PluginProtos.CodeGeneratorResponse.newBuilder().build();
//response.writeTo(System.out);
}
}
I get compile error because CodeGenerator is unknown. I have in my pom.xml Maven file the following inside "dependencies" tag -
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>3.6.1</version>
</dependency>
What else do I need to add to dependencies make my plugin working? I plan to use Java API from https://developers.google.com/protocol-buffers/docs/reference/java/ .
Yeah, that was a bit stupid - CodeGenerator is a custom class, we need to write it, it's the name which confuses by implying it's from the Google library.
So after writing it could look like this - approximate analogy of Python code at https://www.expobrain.net/2015/09/13/create-a-plugin-for-google-protocol-buffer/ , but without packaging into JSON and cleaning subfields:
import com.google.protobuf.DescriptorProtos;
import com.google.protobuf.compiler.PluginProtos;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
public class ProtocPlugin {
private static List _traverse(String strPackage, List items) {
List<List> res = new ArrayList<>();
for(Object item : items) {
res.add(Arrays.asList(item, strPackage));
if(item instanceof DescriptorProtos.DescriptorProto) {
DescriptorProtos.DescriptorProto dp = (DescriptorProtos.DescriptorProto) item;
for(DescriptorProtos.EnumDescriptorProto e : dp.getEnumTypeList()) {
res.add(Arrays.asList(e, strPackage));
}
for(DescriptorProtos.DescriptorProto nested : dp.getNestedTypeList()) {
String strNestedPackage = strPackage + nested.getName();
for(Object nestedItem : _traverse(strNestedPackage, nested.getNestedTypeList())) {
res.add(Arrays.asList(((List)nestedItem).get(0), strNestedPackage));
}
}
}
}
return res;
}
public static void main(String[] args) throws IOException {
StringBuilder data = new StringBuilder();
PluginProtos.CodeGeneratorRequest codeGeneratorRequest = PluginProtos.CodeGeneratorRequest.parseFrom(System.in);
codeGeneratorRequest.getProtoFileList().forEach((DescriptorProtos.FileDescriptorProto fileDescriptorProto) -> {
String strPackage = fileDescriptorProto.getPackage();
if(strPackage == null || strPackage.isEmpty()) {
strPackage = "<root>";
}
data.append("package: ").append(strPackage).append("\n");
data.append("filename: ").append(fileDescriptorProto.getName()).append("\n");
List<DescriptorProtos.EnumDescriptorProto> enums = fileDescriptorProto.getEnumTypeList();
for(Object pair : _traverse(strPackage, enums)) {
data.append("type: enum").append("\n");
data.append(((List)pair).get(0)).append(((List)pair).get(1)).append(" ");
}
List<DescriptorProtos.DescriptorProto> messageTypes = fileDescriptorProto.getMessageTypeList();
for(Object pair : _traverse(strPackage, messageTypes)) {
data.append("type: message").append("\n");
data.append(((List)pair).get(0)).append(((List)pair).get(1)).append(" ");
}
});
PluginProtos.CodeGeneratorResponse.Builder builder = PluginProtos.CodeGeneratorResponse.newBuilder();
builder.addFileBuilder().setContent(data.toString()).setName("mytest.txt");
PluginProtos.CodeGeneratorResponse response = builder.build();
response.writeTo(System.out);
}
}
The launch of protoc could be with
protoc --plugin=protoc-gen-custom=my-plugin.bat --custom_out=. hello.proto
where my-plugin.bat contains something like
#echo off
java -cp target/classes;c:/users/bover/.m2/repository/com/google/protobuf/protobuf-java/3.6.1/protobuf-java-3.6.1.jar ProtocPlugin
here we assume that our Java plugin compiles ProtocPlugin.class into target/classes directory. The output will be in mytest.txt file. hello.proto is a simple proto file from the Python example above.
Related
Being very new to Java, I'm unable to bring a small concept to a syntactic form. Apologies.
My project structure looks like below & i'm trying to walk thru the sub folders of applications
directory & search for a folder named conduit, if present, create a new folder called base parallel to it.
At best I came up with the below code, post that, kind of struggling.
/home/project_A/applications
|sub_project_A
|target
|conduit
|sub_project_B
|target
|conduit
|sub_project_C
|target
|class
|sub_project_D
|target
|conduit
public class Test
{
public static void main(String[] args)
{
Path currentRelativePath = Paths.get("");
String s = currentRelativePath.toAbsolutePath().toString();
String appDir = s + "/applications";
System.out.println("Directory Exists" + appDir);
}
}
You can use BFS to traverse the sub directories:
import java.io.File;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.LinkedList;
import java.util.Queue;
public class MyClass {
public static void main(String[] args) {
Path currentRelativePath = Paths.get("");
Queue<Path> paths = new LinkedList<>();
paths.add(currentRelativePath);
while(!paths.isEmpty()) {
Path current = paths.poll();
File currentFile = current.toFile();
if(currentFile.isDirectory()) {
if(currentFile.getName().equals("conduit")) {
// Found the directory called conduit, Do what you have to do here
}else {
for(String fileName : currentFile.list()) {
paths.add(Paths.get(currentFile.getAbsolutePath()+"/"+fileName));
}
}
}
}
}
}
Using Files.find is a good way to traverse a series of directories quickly to find files or directories matching any criteria you need. Here is an example which prints off the matches:
BiPredicate<Path, BasicFileAttributes> predicate = (p,a) -> a.isDirectory() && "conduit".equals(p.getFileName().toString());
try(var dirs = Files.find(Path.of("."), Integer.MAX_VALUE, predicate)) {
dirs.forEach(p -> {
System.out.println("Found "+p+", create if not exists: "+p.resolveSibling("base"));
}
);
}
Suppose that I have a maven project located in /home/admin/projects/maven-sample-project, I can package the project to a jar using the following command.
mvn clean package
Now I want to do it with Java and relevant maven-api, is there something like this?
MavenProject mavenProject = new MavenProject("/home/admin/projects/maven-sample-project");
mavenProject.clean();
mavenProject.package(jarLocation = "/home/admin/projects/maven-sample-project/target/maven-sample-project-1.0.0-SNAPSHOT.jar");
As mentioned by khmarbaise, maven-invoker is able to solve this problem.
First add the dependency in your pom.xml file.
<dependency>
<groupId>org.apache.maven.shared</groupId>
<artifactId>maven-invoker</artifactId>
<version>3.0.1</version>
</dependency>
Then create a new class.
import org.apache.maven.shared.invoker.*;
import java.io.File;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
public class MavenRunner {
public static final String MAVEN_COMMAND_CLEAN = "clean";
public static final String MAVEN_COMMAND_PACKAGE = "package";
public static void run(String mavenProjectLocation, List<String> goals) throws MavenInvocationException {
Invoker invoker = getInvoker(mavenProjectLocation);
InvocationRequest request = getInvocationRequest(mavenProjectLocation, goals);
InvocationResult result = invoker.execute(request);
if ( result.getExitCode() != 0 )
{
String exceptionMsgPrefix = String.format("Failed to run maven commands, mavenProjectLocation: %s, goals: %s.", mavenProjectLocation, goals);
if ( result.getExecutionException() != null )
{
throw new MavenInvocationException(exceptionMsgPrefix + "exception: " + result.getExecutionException());
}
else
{
throw new MavenInvocationException(exceptionMsgPrefix + "exit code: " + result.getExitCode());
}
}
}
private static Invoker getInvoker(String mavenProjectLocation) {
Invoker invoker = new DefaultInvoker();
invoker.setLocalRepositoryDirectory(new File(mavenProjectLocation));
// replace it with yours
invoker.setMavenHome(new File("/home/admin/apps/apache-maven-3.5.2"));
return invoker;
}
private static InvocationRequest getInvocationRequest(String mavenProjectLocation, List<String> goals) {
InvocationRequest request = new DefaultInvocationRequest();
request.setBaseDirectory(new File(mavenProjectLocation));
request.setGoals(goals);
return request;
}
}
Usage:
String mavenProjectLocation = "/home/admin/projects/maven-sample-project";
List<String> goals = Arrays.asList(MavenRunner.MAVEN_COMMAND_CLEAN, MavenRunner.MAVEN_COMMAND_PACKAGE);
MavenRunner.run(mavenProjectLocation, goals);
I have gradle script which is creating the version in JIRA using the REST API.
But there is jira-rest-java-client also available. I want to use the java library of jira-rest-java-client and wants to do the same stuff in gradle. Can someone provide an example how could I try this.
How to use the jira-rest-java-client library to make connection with JIRA through example?
In Java I am trying to use this JRCJ Library but getting below error through Intellj
import com.atlassian.jira.rest.client.api.JiraRestClient;
import com.atlassian.jira.rest.client.api.domain.*;
import com.atlassian.jira.rest.client.api.domain.input.ComplexIssueInputFieldValue;
import com.atlassian.jira.rest.client.api.domain.input.FieldInput;
import com.atlassian.jira.rest.client.api.domain.input.TransitionInput;
import com.atlassian.jira.rest.client.internal.ServerVersionConstants;
import com.atlassian.jira.rest.client.internal.async.AsynchronousJiraRestClientFactory;
import com.google.common.collect.Lists;
import org.codehaus.jettison.json.JSONException;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.Arrays;
import java.util.Collection;
import java.util.List;
/**
* A sample code how to use JRJC library
*
* #since v0.1
*/
public class Example1 {
private static URI jiraServerUri = URI.create("http://localhost:2990/jira");
private static boolean quiet = false;
public static void main(String[] args) throws URISyntaxException, JSONException, IOException {
parseArgs(args);
final AsynchronousJiraRestClientFactory factory = new AsynchronousJiraRestClientFactory();
final JiraRestClient restClient = factory.createWithBasicHttpAuthentication(jiraServerUri, "admin", "admin");
try {
final int buildNumber = restClient.getMetadataClient().getServerInfo().claim().getBuildNumber();
// first let's get and print all visible projects (only jira4.3+)
if (buildNumber >= ServerVersionConstants.BN_JIRA_4_3) {
final Iterable<BasicProject> allProjects = restClient.getProjectClient().getAllProjects().claim();
for (BasicProject project : allProjects) {
if (project == TEST){
println(project);}else {
System.out.println("Project" + "Not Found");
}
}
}
// let's now print all issues matching a JQL string (here: all assigned issues)
if (buildNumber >= ServerVersionConstants.BN_JIRA_4_3) {
final SearchResult searchResult = restClient.getSearchClient().searchJql("assignee is not EMPTY").claim();
for (BasicIssue issue : searchResult.getIssues()) {
println(issue.getKey());
}
}
final Issue issue = restClient.getIssueClient().getIssue("TST-7").claim();
println(issue);
// now let's vote for it
restClient.getIssueClient().vote(issue.getVotesUri()).claim();
// now let's watch it
final BasicWatchers watchers = issue.getWatchers();
if (watchers != null) {
restClient.getIssueClient().watch(watchers.getSelf()).claim();
}
// now let's start progress on this issue
final Iterable<Transition> transitions = restClient.getIssueClient().getTransitions(issue.getTransitionsUri()).claim();
final Transition startProgressTransition = getTransitionByName(transitions, "Start Progress");
restClient.getIssueClient().transition(issue.getTransitionsUri(), new TransitionInput(startProgressTransition.getId()))
.claim();
// and now let's resolve it as Incomplete
final Transition resolveIssueTransition = getTransitionByName(transitions, "Resolve Issue");
final Collection<FieldInput> fieldInputs;
// Starting from JIRA 5, fields are handled in different way -
if (buildNumber > ServerVersionConstants.BN_JIRA_5) {
fieldInputs = Arrays.asList(new FieldInput("resolution", ComplexIssueInputFieldValue.with("name", "Incomplete")));
} else {
fieldInputs = Arrays.asList(new FieldInput("resolution", "Incomplete"));
}
final TransitionInput transitionInput = new TransitionInput(resolveIssueTransition.getId(), fieldInputs, Comment
.valueOf("My comment"));
restClient.getIssueClient().transition(issue.getTransitionsUri(), transitionInput).claim();
}
finally {
restClient.close();
}
}
private static void println(Object o) {
if (!quiet) {
System.out.println(o);
}
}
private static void parseArgs(String[] argsArray) throws URISyntaxException {
final List<String> args = Lists.newArrayList(argsArray);
if (args.contains("-q")) {
quiet = true;
args.remove(args.indexOf("-q"));
}
if (!args.isEmpty()) {
jiraServerUri = new URI(args.get(0));
}
}
private static Transition getTransitionByName(Iterable<Transition> transitions, String transitionName) {
for (Transition transition : transitions) {
if (transition.getName().equals(transitionName)) {
return transition;
}
}
return null;
}
}
Error:
xception in thread "main" java.lang.NoClassDefFoundError: com/atlassian/sal/api/executor/ThreadLocalContextManager
at com.atlassian.jira.rest.client.internal.async.AsynchronousJiraRestClientFactory.create(AsynchronousJiraRestClientFactory.java:35)
at com.atlassian.jira.rest.client.internal.async.AsynchronousJiraRestClientFactory.createWithBasicHttpAuthentication(AsynchronousJiraRestClientFactory.java:42)
at Example1.main(Example1.java:34)
Caused by: java.lang.ClassNotFoundException: com.atlassian.sal.api.executor.ThreadLocalContextManager
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 3 more
Moreover I added the JRJC api,core jar in External Libraries but still getting this error?
Could someone tell me what is the issue or where am I missing something?
compile 'com.atlassian.jira:jira-rest-java-client-core:4.0.0'
compile 'com.atlassian.jira:jira-rest-java-client-api:4.0.0'
Simple connection to JIRA:
JiraRestClient restClient = new AsynchronousJiraRestClientFactory().createWithBasicHttpAuthentication(new URI("https://" + jira_domain),
jira_username, jira_password);
I'm trying to create and save a generated model directly from Java. The documentation specifies how to do this in R and Python, but not in Java. A similar question was asked before, but no real answer was provided (beyond linking to H2O doc, which doesn't contain a code example).
It'd be sufficient for my present purpose get some pointers to be able to translate the following reference code to Java. I'm mainly looking for guidance on the relevant JAR(s) to import from the Maven repository.
import h2o
h2o.init()
path = h2o.system_file("prostate.csv")
h2o_df = h2o.import_file(path)
h2o_df['CAPSULE'] = h2o_df['CAPSULE'].asfactor()
model = h2o.glm(y = "CAPSULE",
x = ["AGE", "RACE", "PSA", "GLEASON"],
training_frame = h2o_df,
family = "binomial")
h2o.download_pojo(model)
I think I've figured out an answer to my question. A self-contained sample code follows. However, I'll still appreciate an answer from the community since I don't know if this is the best/idiomatic way to do it.
package org.name.company;
import hex.glm.GLMModel;
import water.H2O;
import water.Key;
import water.api.StreamWriter;
import water.api.StreamingSchema;
import water.fvec.Frame;
import water.fvec.NFSFileVec;
import hex.glm.GLMModel.GLMParameters.Family;
import hex.glm.GLMModel.GLMParameters;
import hex.glm.GLM;
import water.util.JCodeGen;
import java.io.*;
import java.util.Map;
public class Launcher
{
public static void initCloud(){
String[] args = new String [] {"-name", "h2o_test_cloud"};
H2O.main(args);
H2O.waitForCloudSize(1, 10 * 1000);
}
public static void main( String[] args ) throws Exception {
// Initialize the cloud
initCloud();
// Create a Frame object from CSV
File f = new File("/path/to/data.csv");
NFSFileVec nfs = NFSFileVec.make(f);
Key frameKey = Key.make("frameKey");
Frame fr = water.parser.ParseDataset.parse(frameKey, nfs._key);
// Create a GLM and output coefficients
Key modelKey = Key.make("modelKey");
try {
GLMParameters params = new GLMParameters();
params._train = frameKey;
params._response_column = fr.names()[1];
params._intercept = true;
params._lambda = new double[]{0};
params._family = Family.gaussian;
GLMModel model = new GLM(params).trainModel().get();
Map<String, Double> coefs = model.coefficients();
for(Map.Entry<String, Double> entry : coefs.entrySet()) {
System.out.format("%s: %f\n", entry.getKey(), entry.getValue());
}
String filename = JCodeGen.toJavaId(model._key.toString()) + ".java";
StreamingSchema ss = new StreamingSchema(model.new JavaModelStreamWriter(false), filename);
StreamWriter sw = ss.getStreamWriter();
OutputStream os = new FileOutputStream("/base/path/" + filename);
sw.writeTo(os);
} finally {
if (fr != null) {
fr.remove();
}
}
}
}
Would something like this do the trick?
public void saveModel(URI uri, Keyed<Frame> model)
{
Persist p = H2O.getPM().getPersistForURI(uri);
OutputStream os = p.create(uri.toString(), true);
model.writeAll(new AutoBuffer(os, true)).close();
}
Make sure the URI has a proper form otherwise H2O will break on an npe. As for Maven you should be able to get away with the h2o core.
<dependency>
<groupId>ai.h2o</groupId>
<artifactId>h2o-core</artifactId>
<version>3.14.0.2</version>
</dependency>
I'm trying to parse a java file with Java Compiler APIs.
The documents are very poor. After hours of digging I still cannot get the Trees#getElement work for me. Here's my code:
import com.sun.source.tree.*;
import com.sun.source.util.*;
import javax.tools.JavaCompiler;
import javax.tools.JavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;
import java.io.IOException;
import java.nio.CharBuffer;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
class CodeAnalyzerTreeVisitor extends TreePathScanner<Object, Trees> {
#Override
public Object visitClass(ClassTree classTree, Trees trees) {
System.out.println("className " + classTree.getSimpleName());
//prints name of class
TreePath path = getCurrentPath();
printLocationAndSource(trees, path, classTree);
//prints the original source code
while (path != null) {
System.out.println("treepath");
System.out.println(trees.getElement(path));
path = path.getParentPath();
}
//it prints several nulls here
//why?
return super.visitClass(classTree, trees);
}
public static void printLocationAndSource(Trees trees,
TreePath path, Tree tree) {
SourcePositions sourcePosition = trees.getSourcePositions();
long startPosition = sourcePosition.
getStartPosition(path.getCompilationUnit(), tree);
long endPosition = sourcePosition.
getEndPosition(path.getCompilationUnit(), tree);
JavaFileObject file = path.getCompilationUnit().getSourceFile();
CharBuffer sourceContent = null;
try {
sourceContent = CharBuffer.wrap(file.getCharContent(true).toString().toCharArray());
} catch (IOException e) {
e.printStackTrace();
}
CharBuffer relatedSource = null;
if (sourceContent != null) {
relatedSource = sourceContent.subSequence((int) startPosition, (int) endPosition);
}
System.out.println("start: " + startPosition + " end: " + endPosition);
// System.out.println("source: "+relatedSource);
System.out.println();
}
}
public class JavaParser {
private static final JavaCompiler javac
= ToolProvider.getSystemJavaCompiler();
private static final String filePath = "/home/pinyin/Source/hadoop-common/hadoop-yarn-project/hadoop-ya" +
"rn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/ya" +
"rn/server/resourcemanager/ResourceManager.java";
public static void main(String[] args) throws IOException {
StandardJavaFileManager jfm = javac.getStandardFileManager(null, null, null);
Iterable<? extends javax.tools.JavaFileObject> javaFileObjects = jfm.getJavaFileObjects(filePath);
String[] sourcePathParam = {
"-sourcepath",
"/home/pinyin/Source/hadoop-common/hadoop-yarn-project/hadoop-yarn/" +
"hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/"
};
List<String> params = new ArrayList<String>();
params.addAll(Arrays.asList(sourcePathParam));
JavacTask task = (JavacTask) javac.getTask(null, jfm, null, params, null, javaFileObjects);
Iterable<? extends CompilationUnitTree> asts = task.parse();
Trees trees = Trees.instance(task);
for (CompilationUnitTree ast : asts) {
new CodeAnalyzerTreeVisitor().scan(ast, trees);
}
}
}
The lines about params and -sourcepath are added because I thought the compiler is trying to find the source file in the wrong places. They didn't work.
I'm still trying to understand how the Trees, javac and related JSRs work together, are there any recommended documents for beginners?
Thanks for your help.
edit:
The java file I'm trying to analyze is:
https://github.com/apache/hadoop-common/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
The file can be compiled without errors in its maven project, but its dependencies are not passed to javac in my situation. I'm not sure if this is the problem.
The trees.getElement returns null in the middle part of the code above, while the other parts seems to work well.
According to this answer, it seems that the Elements' information is not usable until the compilation is completed.
So calling task.analyze() solved my problem. Although javac is complaining about missing dependencies.
Please correct me if I'm wrong, thanks.