I'm trying to create an IntelliJ plugin that iterates over all files in the project folder and parses all the .java files and then makes some changes in them. The problem is that after reading the documentation I don't have a clear idea how to iterate files over the whole project folder, I think I may use PSI files but I am not sure. Does anyone know or has an idea on how to accomplish this?
To iterate all files in project content, you can use ProjectFileIndex.SERVICE.getInstance(project).iterateContent.
Then you can get PSI files from them (PsiManager#findFile), check if they're Java (instanceof PsiJavaFile) and do whatever you like.
If you don't need PSI, you can just check the file type
(VirtualFile#getFileType == JavaFileType.INSTANCE) and perform the modifications via document (FileDocumentManager#getDocument(file)) or VFS (LoadTextUtil#loadText, VfsUtil#saveText).
A possible way is to use AllClassesGetter, like this:
Processor<PsiClass> processor = new Processor<PsiClass>() {
#Override
public boolean process(PsiClass psiClass) {
// do your actual work here
return true;
}
};
AllClassesGetter.processJavaClasses(
new PlainPrefixMatcher(""),
project,
GlobalSearchScope.projectScope(project),
processor
);
processJavaClasses() will look for classes matching a given prefix in a given scope. By using an empty prefix and GlobalSearchScope.projectScope(), you should be able to iterate all classes declared in your project, and process them in processor. Note that the processor handles instances of PsiClass, which means you won't have to parse files manually. To modify classes, you just have to change the tree represented by these PsiClasses.
Related
I'm looking for a way to access the name of the file being processed during the data transformation within a DoFn.
My pipeline is as shown below:
Pipeline p = Pipeline.create(options);
p.apply(FileIO.match()
.filepattern(options.getInput())
.continuously(Duration.standardSeconds(5),
Watch.Growth.<String>never()))
.apply(FileIO.readMatches()
.withCompression(Compression.GZIP))
.apply(XmlIO.<MyString>readFiles()
.withRootElement("root")
.withRecordElement("record")
.withRecordClass(MyString.class))//<-- This only returns the contents of the file
.apply(ParDo.of(new ProcessRecord()))//<-- I need to access file name here
.apply(ParDo.of(new FormatRecord()))
.apply(Window.<String>into(FixedWindows.of(Duration.standardSeconds(5))))
.apply(new CustomWrite(options));
Each file that is processed is an XML document. While processing the content, I need access to the name of the file being processed too to include in the transformed record.
Is there a way to achieve this?
This post has a similar question, but since i'm trying to use XmlIO I havent found a way to access the file metadata.
Below is the approach I found online, but not sure if there is a way to use it in the pipeline described above.
p.apply(FileIO.match()
.filepattern(options.getInput())
.continuously(Duration.standardSeconds(5),
Watch.Growth.<String>never()))//File Metadata
.apply(FileIO.readMatches()
.withCompression(Compression.GZIP))//Readable Files
.apply(MapElements
.into(TypeDescriptors.kvs(TypeDescriptors.strings(),new TypeDescriptor<ReadableFile>() {} ))
.via((ReadableFile file) -> {
return KV.of(file.getMetadata().resourceId().getFilename(),file);
})
);
Any suggestions are highly appreciated.
Thank you for your time reviewing this.
EDIT:
I took Alexey's advice and implemented a custom XmlIO. It would be nice if we could just extend the class we need and override the appropriate method. However, in this specific case, there was a reference to one method which was protected within the sdk because of which I couldn't easily override what i needed and instead ended up copying a whole bunch of files. While this works for now, I hope in future there is a more straighforward way to access the file metadata in these IO implementations.
I don't think it's possible to do "out-of-box" with a current implementation of of XmlIO since it returns a PCollection<T> where T is a type of your xml record and, if I'm not mistaken, there is no way to add a file name there. Though, you still can try to "reimplement" a ReadFiles and XmlSource in a way that it will return parsed payload and input file metadata.
I am trying to read a file which has name: K2ssal.timestamp.
I want to handle the time stamp part of the file name as wildcard.
How can I achieve this ?
tried * after file name but not working.
var getK2SSal: Iterator[String] = Source.fromFile("C://Users/nrakhad/Desktop/Work/Data stage migration/Input files/K2Ssal.*").getLines()
You can use Files.newDirectoryStream with directory + glob:
import java.nio.file.{Paths, Files}
val yourFile = Files.newDirectoryStream(
Paths.get("/path/to/the/directory"), // where is the file?
"K2Ssal.*" // glob of the file name
).iterator.next // get first match
Misconception on your end: unless the library call is specifically implemented to do so, using a wildcard simply doesn't work like you expect it to.
Meaning: a file system doesn't know about wildcards. It only knows about existing files and folders. The fact that you can put * on certain commands, and that the wildcard is replaced with file names is a property of the tool(s) you are using. And most often, programming APIs that allow you to query the file system do not include that special wild card handling.
In other words: there is no sense in adding that asterisk like that.
You have to step back and write code that actively searches for files itself. Here are some examples for scala.
You can read the directory and filter on files based upon the string.
val l = new File("""C://Users/nrakhad/Desktop/Work/Data stage migration/Input files/""").listFiles
val s = l.filter(_.toString.contains("K2Ssal."))
How do you merge two .odt files? Doing that by hand, opening each file and copying the content would work, but is unfeasable.
I have tried odttoolkit Simple API (simple-odf-0.8.1-incubating) to achieve that task, creating an empty TextDocument and merging everything into it:
private File masterFile = new File(...);
...
TextDocument t = TextDocument.newTextDocument();
t.save(masterFile);
...
for(File f : filesToMerge){
joinOdt(f);
}
...
void joinOdt(File joinee){
TextDocument master = (TextDocument) TextDocument.loadDocument(masterFile);
TextDocument slave = (TextDocument) TextDocument.loadDocument(joinee);
master.insertContentFromDocumentAfter(slave, master.getParagraphByReverseIndex(0, false), true);
master.save(masterFile);
}
And that works reasonably well, however it looses information about fonts - original files are a combination of Arial Narrow and Windings (for check boxes), output masterFile is all in TimesNewRoman. At first I suspected last parameter of insertContentFromDocumentAfter, but changing it to false breaks (almost) all formatting. Am I doing something wrong? Is there any other way?
I think this is "works as designed".
I tried this once with a global document, which imports documents and display them as is... as long as paragraph styles have different names !
Using same named templates are overwritten with the values the "master" document have.
So I ended up cloning standard styles with unique (per document) names.
HTH
Ma case was a rather simple one, files I wanted to merge were generated the same way and used the same basic formatting. Therefore, starting off of one of my files, instead of an empty document fixed my problem.
However this question will remain open until someone comes up with a more general solution to formatting retention (possibly based on ngulams answer and comments?).
I am using the apache commons configuration library to read a configuration xml and it works nicely. However, I am not able to modify the value of the elements or add new ones.
To read the xml I use the following code:
XMLConfiguration config = new XMLConfiguration(dnsXmlPath);
boolean enabled = config.getBoolean("enabled", true));
int size = config.getInt("size");
To write I am trying to use:
config.setProperty("newProperty", "valueNewProperty");
config.save();
If I call config.getString("newProperty"), I obtain "valueNewProperty", but the xml has not been changed.
Obviously it is not the right way or I am missing something, because it does not work.
Could anybody tell me how to do this?
Thanks in advance.
You're modifying xml structure in memory
The parsed document will be stored keeping its structure. The class also tries to preserve as much information from the loaded XML document as possible, including comments and processing instructions. These will be contained in documents created by the save() methods, too.
Like other file based configuration classes this class maintains the name and path to the loaded configuration file. These properties can be altered using several setter methods, but they are not modified by save() and load() methods. If XML documents contain relative paths to other documents (e.g. to a DTD), these references are resolved based on the path set for this configuration.
You need to use XMLConfiguration.html#save(java.io.Writer) method
For example, after you've done all your modifications save it:
config.save(new PrintWriter(new File(dnsXmlPath)));
EDIT
As mentioned in comment, calling config.load() before calling setProperty() method fixes the issue.
I solved it with the following lines. I was missing the config.load().
XMLConfiguration config = new XMLConfiguration(dnsXmlPath);
config.load();
config.setProperty("newProperty", "valueNewProperty");
config.save();
It is true though that you can used the next line instead of config.save() and works the same.
config.save(new PrintWriter(new File(dnsXmlPath)));
In my current project, I am using xText editor for writing my dsl specifications (i.e., voc.mydsl, arch.mydsl, and network.mydsl). I like the xText editor because of its code-completion and other functionalities.
However, I have a separate Java program. This java program takes text files (i.e., voc.txt, arch.txt, network.txt) as inputs, parse these files using ANTLR parser, and generates code using StringTemplate files.
Now, my problem is that currently, I have to follow these steps manually:
(1) I write dsl specifications in XText editor (voc.mydsl, arch.mydsl, and network.mydsl).
(2) I copy-paste these specification into three text files (i.e., voc.txt, arch.txt, network.txt).
(3) Finally, I run the Java program to parse these .txt files and generate code.
Is there any way that I can automize (performed in a single click) all the above three steps? Let me know if you need any detail.
You could write a "special" generator for your DSL. XText will call this generator whenever you edit and save a *.mydsl file. What you actually do in this "Generator" thing is of no interest to Xtext. So your MydslGenerator.xtend generator could look like this:
// whereever Xtext generates your empty version of this file
package mydsl.xtext.generator
// add imports
#Singleton
class MydslGenerator implements IGenerator {
override void doGenerate(Resource resource, IFileSystemAccess fsa) {
// calculate new filename
val newFilename= resource.URI.lastSegment.replaceAll(".mydsl", ".txt")
// get text representation of parsed model
val textContent = resource.contents.map[NodeModelUtils::getNode(it).text].join
// write text content to new file
fsa.generateFile(newFilename, textContent);
// TODO: call ANTLR parser on new file here
}
}
In the last step you can call your "other" program either by calling its main method directly from Eclipse or by invoking a new JVM. The later is only advisable if the other generator works quickly because it is called whenever you save a *.mydsl file. The first method is only advisable when the other program has no memory leaks and has not to many jar dependencies.