Programmatically analyze java heap dump file - java

I want to write a program (preferably in java) that will parse and analyze a java heap dump file (created by jmap). I know there are many great tools that already do so (jhat, eclipse's MAT, and so on), but I want to analyze the heap from a specific perspective to my application.
Where can I read about the structure of the heap dump file, examples how to read it, and so on? Didn't find anything useful searching for it...
Many thanks.

The following was done with Eclipse Version: Luna Service Release 1 (4.4.1) and Eclipse Memory Analyzer Version 1.4.0
Programmatically Interfacing with the Java Heap Dump
Environment Setup
In eclipse, Help -> Install New Software -> install Eclipse Plug-in Development Environment
In eclipse, Window -> Preferences -> Plug-in Development -> Target Platform -> Add
Nothing -> Locations -> Add -> Installation
Name = MAT
Location = /path/to/MAT/installation/mat
Project Setup
File -> new -> Other -> Plug-in Project
Name: MAT Extension
Next
Disable Activator
Disable Contributions to the UI
Disable API analysis
Next
Disable template
Finish
Code Generation
Open plugin.xml
Dependencies -> Add
select org.eclipse.mat.api
Extensions -> Add
select org.eclipse.mat.report.query
right click on report.query -> New
Name: MyQuery
click "impl" to generate the class
Implementing IQuery
#CommandName("MyQuery") //for the command line interface
#Name("My Query") //display name for the GUI
#Category("Custom Queries") //list this Query will be put under in the GUI
#Help("This is my first query.") //help displayed
public class MyQuery implements IQuery
{
public MyQuery{}
#Argument //snapshot will be populated before the call to execute happens
public ISnapshot snapshot;
/*
* execute : only method overridden from IQuery
* Prints out "My first query." to the output file.
*/
#Override
public IResult execute(IProgressListener arg0) throws Exception
{
CharArrayWriter outWriter = new CharArrayWriter(100);
PrintWriter out = new PrintWriter(outWriter);
SnapshotInfo snapshotInfo = snapshot.getSnapshotInfo();
out.println("Used Heap Size: " + snapshotInfo.getUsedHeapSize());
out.println("My first query.")
return new TextResult(outWriter.toString(), false);
}
}
ctrl+shift+o will generate the correct "import" statements.
Custom queries can be accessed within the MAT GUI by accessing the toolbar's "Open Query Browser" at the top of the hprof file you have opened.
The ISnapshot interface
The most important interface one can use to extract data from a heap dump is ISnapshot. ISnapshot represents a heap dump and offers various methods for reading object and classes from it, getting the size of objects, etc…
To obtain an instance of ISnapshot one can use static methods on the SnapshotFactory class. However, this is only needed if the API is used to implement a tool independent of Memory Analyzer. If you are writing extensions to MAT, then your coding will get an instance corresponding to an already opened heap dump either by injection or as a method parameter.
Reference
Built in Command Line Utility
If you're looking to have a program generate the usual reports, there is a command line utility called ParseHeapDump available with any download of Eclipse's MAT tool. You'll be able to get useful html dumps of all the information MAT stores.
> ParseHeapDump <heap dump> org.eclipse.mat.api:suspects org.eclipse.mat.api:top_components org.eclipse.mat.api:overview #will dump out all general reports available through MAT
Hopefully this is enough information to get you started.

I'm not familiar with jhat, but Eclipse's MAT is open source. Their SVN link is available, perhaps you could look through that for their parser, perhaps even use it.

Related

Invoking Remote Systems LPEX Editor using java

We have a set of plugins installed in RDI where they display a list of native RPGLE programs and have got a set of classes that invoke the LPEX editor to allow editing the source file in the traditional Library/SoureFile/Member settings.
I now want to extend this to be able to add/edit RPGLE source in the IFS but cannot find the IBM classes that allow this.
For the normal native source the code is
IQSYSMember memberObj;
IBMiConnection iSeriesConnection = RSEConnectionFactory.getISeriesConnection(connection);
memberObj = iSeriesConnection.getMember(
a.getSourceLibrary(),
a.getSourceFile(),
a.getSourceMember(),
null);
QSYSEditableRemoteSourceFileMember editableMember = new QSYSEditableRemoteSourceFileMember(memberObj);
editableMember.open(Display.getCurrent().getActiveShell(), false);
Obviously for source that are in IFS, all I have is a path (/home/path/QRPGLESRC/nnn.RPGLE).
I know I can go to Remote System Explorer and right hand click and select "Open with Remote Systems LPEX Editor" but need to do this from my plugins.
Anyone know which IBM classes allow this to happen?
Many thanks.
Use the open(boolean,progressmonitor) method on QSYSEditableRemoteSourceFileMember
This has fixed my issue:
FileServiceSubSystem fileService = LpexHelper.getFileServiceSubsystem(iSeriesConnection);
IRemoteFile remoteFile = fileService.getRemoteFileObject(path, new NullProgressMonitor());
SystemEditableRemoteFile file = new SystemEditableRemoteFile(remoteFile);
if (file.exists()) {
file.open(Display.getCurrent().getActiveShell(), true);
}

How to run a GPR file in Java API and run a model GAMS

I have a model with GMS extension. When I run that model with Gams studio, it run perfectly and I obtain the expected results.
I have tried to run the GMS model with Gams IDE but I obtain a lot of errors, so, I have tried something different. I Have opened a file with GPR extension and after of that I have imported the GMS model and everything works perfectly when I run the project.
I think I need to do same thing usinge Gams Java API, but I don't know how to import to my workspace a GPR file.
In this moment I just have the next code:
GAMSWorkspace workspace = new GAMSWorkspace();
workspace.setDebugLevel(DebugLevel.KEEP_FILES);
GAMSJob jobGams = workspace.addJobFromFile("fileModelGms");
jobGams.run();
When I run that code, I obtain an error:
GAMS process returns unsuccessfully with return code : 2 [there was a
compilation error]. Check \_gams_java_gjo1.lst] for more details.
The gpr file has a format that is only understood by the GAMSIDE. You can not pass it to any API. If you get errors calling your model from the API but not from the GAMSIDE, you probably have set certain options using the IDE which you should set now trough the API as well. Though, without seeing the exact error, it is hard to give further hints.
I have solved the problem with Lutz's helper. I needed to Include a dir with input that model uses.
This is my code commented line per line to understand how API Gams works. I used a specific workspace too because API creates a folder in temps file when you run a new Job. I did use for a database GDX too to run my model.
//specific workspace information is created example: C:/Desktop/Workspace
GAMSWorkspaceInfo workspaceInfo = new GAMSWorkspaceInfo();
workspaceInfo.setWorkingDirectory("specificPathWorkspace");
//A new workspace is created with workspaceInfo.
GAMSWorkspace workspace = new GAMSWorkspace(workspaceInfo);
workspace.setDebugLevel(DebugLevel.KEEP_FILES);
//Options where you're going to set input file data.
GAMSOptions options = workspace.addOptions();
//Set path with input Data example: C:/Desktop/InputDate
options.IDir.add("PathWithInputData");
//Using a database where is data to be processed example: db.gdx
GAMSDatabase gdxdb = workspace.addDatabaseFromGDX("db.gdx");
// Creating a JOB to execute the model.
GAMSJob jobGams = workspace.addJobFromFile(entradasModeloGamsDTO.getPathModeloGams());
//Running model
jobGams.run(options,gdxdb);

Programmatically refreshing a file in an eclipse Java application

I have the following program which adds a method to itself when run. But I have to refresh it every time using the F5 button or the refresh option.
Is there a way I could code the refresh in the program itself so that it refreshes itself after the modification? The project I am working on is a Java application and not an eclipse plugin so as far as I know the refreshLocal() method can't be used.
public class Demo {
public static void main(String[] args) throws IOException, CoreException {
File file = new File("/home/kishan/workspace/Roast/src/Demo.java");
if (file.exists()) {
JavaClassSource javaClass = Roaster.parse(JavaClassSource.class,
file);
javaClass.addMethod().setPublic().setStatic(true)
.setName("newMethod").setReturnTypeVoid()
.setBody("System.out.println(\"newMethod created\");")
.addParameter("String[]", "stringArray");
FileWriter writer = new FileWriter(file);
writer.write(javaClass.toString());
writer.flush();
writer.close();
}
}
}
I have tried using the refreshLocal() method defined in the eclipse JDT but since my project is a Java application the ResourcePlugin.getWorkspace() method does not work giving me a "workspace closed" error. Any suggestion is appreciated.
You see, eclipse runs your Java class within its own dedicated JVM. Thus there is no direct programmatic way of enforcing a refresh within eclipse.
You could check this older question; maybe that could lead to a reasonable workarounds.
On the other hand you might step back and ask yourself why exactly you want to achieve that. Your workflow simply doesn't make much sense when looking at it; as in: when generating code that way, shouldn't that generated code better go in its own specific place?
If you intend to "generate" code frequently to then continue to use it in eclipse; well, that somehow smells like a strange idea.
Eclipse has "Refresh using native hooks or polling" which might might help.
You can find it under Window > Prefrences > General > Workspace.
See On Eclipse, what does "Preferences -> General -> Workspace -> Refresh using native hooks or polling" do?

Programmatically list open projects in an eclipse workspace from outside of eclipse

I want to write a Gradle plugin which can inspect an eclipse workspace directory and iterate over the open projects within the workspace and determine the location of each.
Something like
Workspace workspace = EclipseUtils.parseWorkspace("c:/myEclipseWorkspace");
Collection<Project> projects = workspace.getProjects();
for (Project project : projects) {
System.out.println(String.format("name=%s, location=%s, open=%s",
project.getName(), project.getLocation(), project.isOpen()));
}
I've looked at my workspace and can see some .location files under c:\myEclipseWorkspace\.metadata\.plugins\org.eclipse.core.resources\.projects\
But these files are a custom binary format
Is there an eclipse API that I can invoke to parse these? Or some other solution to iterate the open projects in a workspace.
Please note that I want to do this externally to eclipse and NOT within an eclipse plugin.
Reading the Private Description to Obtain Location
Since you are writing in Java, then reuse the Eclipse code from your external location.
i.e. Pull out some of the key code from org.eclipse.core.resources.ResourcesPlugin. Start with the impl of org.eclipse.core.resources.ResourcesPlugin.getWorkspace() and then work your way to org.eclipse.core.resources.IWorkspaceRoot.getProjects()
The above code reads the project description here: org.eclipse.core.internal.resources.LocalMetaArea.readPrivateDescription(IProject, ProjectDescription) and that is called from org.eclipse.core.internal.localstore.FileSystemResourceManager.read(IProject, boolean) which has some logic about default locations.
This is the joy of EPL, as long as your new program/feature is EPL you can reuse Eclipse's core code to do new and wonderful things.
Reading Workspace State to Obtain Open/Close State
When reading workspace state, you are moving into the ElementTree data structures. Reading this without using the ElementTree classes is probably unrealistic. Using the ElementTree classes without full OSGi is probably unrealistic. I provide the following notes to help you on your way.
Working backwards:
ICoreConstants.M_OPEN is the flag value indicating project is open or closed (set for open, clear for closed)
M_OPEN is tested when Project.isOpen() is called
The flags at runtime are in ResourceInfo.flags
The flags are loaded by ResourceInfo.readFrom() called from SaveManager.readElement()
The DataInput input passed to readElement is from the Element Tree stored in the workspace meta directory in .metadata/.plugins/org.eclipse.core.resources/.root/<id>.tree. The specific version (id) of the file to use is recorded in the safe table .metadata/.plugins/org.eclipse.core.resources/.safetable/org.eclipse.core.resources
The safe table is part of the SaveManager's internal state stored in a MasterTable
I've managed to parse the file using this as a reference
protected Location parseLocation(File locationFile) throws IOException {
String name = locationFile.getParentFile().getName();
DataInputStream in = new DataInputStream(new FileInputStream(locationFile));
try {
in.skip(ILocalStoreConstants.BEGIN_CHUNK.length);
String path = in.readUTF();
int numRefs = in.readInt();
String[] refNames = new String[numRefs];
for (int i = 0; i < numRefs; ++ i) {
refNames[i] = in.readUTF();
}
in.skipBytes(ILocalStoreConstants.END_CHUNK.length);
return new Location(name, path, refNames);
} finally {
in.close();
}
}
Unfortunately this doesn't seem to be able to detect if a project is closed or not. Any pointers on getting the closed flag would be much appreciated

Creating a new Java project

I am able to create a default project in PDE just like mentioned here.
However I want to know how to create a Java project. How can I do that?
As a minimum you need to add the Java project nature to the project you create. Use something like:
private void addNatureToProject(IProject proj, String natureId, IProgressMonitor monitor) throws CoreException
{
IProjectDescription description = proj.getDescription();
String[] prevNatures = description.getNatureIds();
String[] newNatures = new String[prevNatures.length + 1];
System.arraycopy(prevNatures, 0, newNatures, 0, prevNatures.length);
newNatures[prevNatures.length] = natureId;
description.setNatureIds(newNatures);
proj.setDescription(description, monitor);
}
The nature id for a Java project is 'org.eclipse.jdt.core.javanature' (also in the JavaCore.NATURE_ID constant).
Note that this does not add the various builders that are normally used in a Java project. The IProjectDescription.setBuildSpec method adds those in a similar way. Create a command for the build spec with:
ICommand command = description.newCommand();
command.setBuilderName(builderID);
where 'builderId' is 'org.eclipse.jdt.core.javabuilder' for the main Java builder (JavaCore.BUILDER_ID constant).
All this information is stored in the '.project' file in the project root folder. This is an XML file which you can look at to see the set up of existing projects.
Window->Open Perspective->Java
then
File->New->Java Project (you may have to go via "Project..." to get to the "Java Project" option)
Please refer to New Project Creation Wizards
On the Plug-in Project page, use com.example.helloworld as the name
for your project and check the box for Create a Java project (this
should be the default). Leave the other settings on the page with
their default settings and then click Next to accept the default
plug-in project structure.

Categories