We Keep Coding

Search entire JVM for value with IntelliJ IDEA debugger at runtime

I have encountered some strange text formatting in our application and I need to find the cause. The problem is I have no idea where to start looking in our Legacy code.Is it possible to make the IntelliJ IDEA debugger search for a value throughout the entire instance of an application in the JVM? As opposed to the usual way of setting a breakpoint with a condition.

Judging from the comment on this question, there is no such feature in IntelliJ.
You could however try performing a heap dump and then searching it using OQL. It can be done in the VisualVM tool which is bundled with the JDK:
Run the jvisualvm command (assuming you have the JDK's bin folder in your PATH).
Start your application, find it in the left column under Local, and select Heap Dump from the right-click menu.
Head to the OQL Console view. In Query Editor, type your query, for example:
select s from java.lang.String s where s.toString().contains("hello")
and execute it.
If your object is on the heap, you should see it. Click it and check the referencing objects in References.
This should get you closer to the classes/objects your object is used by.
There are some tricky parts:
you have to perform the heap dump before the object in question is garbage-collected - that is, as soon as possible after you are sure the object was created (other ways to acquire a heap dump are described here),
the object can be unreachable. In such cases, try to look through the object graph it is part of and find other objects which are still referenced,
the object may not live in the heap at all (escape analysis).
Another tool for heap dump analysis is Eclipse Memory Analyzer (MAT). (For your use case, enable the Keep unreachable objects option before analyzing the heap dump.)
You mentioned that you need to search for the value at runtime. However, if you suspect what you are looking for is a string literal, try just searching the JARs of your application as described in this answer.

Eclipse: Is it Possbile to Set Breakpoint on a Whole Module

I have the following 2 questions regarding Eclipse debugger:
The application that I work on consists of a large number of Maven modules. Is it possible to set a break point on a whole module?
I often have to work on applications that I have no prior knowledge of, and thus don't know where to put a breakpoint. Is there a good place to set a break point in such apps to stop the debugger? Then I can just use the step debugger to trace and see what the code is doing.
Thanks

No, breakpoints are a point, namely a line, in the code where you would like execution to stop. The JVM has no notion of separate Maven modules, so cannot break on that basis.
What information do you start with? Presumably you're not literally looking at a .zip of code you know nothing about. You can place a breakpoint in the main() method to break (almost) as soon as the application starts, or identify other core classes and place breakpoints in them. Other than the actual application entry point (main() and so on), there is no standard "good" place to put a breakpoint in order to see what the code is doing.

Is it possible to set breakpoints at a specific bytecode instruction?

I'm using jdb to remotely debug a Java application of which I don't have the source code. Furthermore, the application jars are obfuscated.
I can set method breakpoints but, is it possible to set breakpoints at a specific bytecode instruction within a method? The idea I have is to use some disassembler like javap to identify the interesting instructions.
Can jdb or other Java debugger do this?

You need line numbers to breakpoint on (something I assume has been removed). You can artificially add line numbers to the file using instrumentation and you will be able to breakpoint at every instruction if you wish.

Real time code compiling?

I was watching the livestream at http://www.humblebundle.com, and I saw them running the application, AND making changes to the code in Eclipse. Is this just changes for the next time they compile and run, or was he actually changing the application as it was running somehow?

Sometimes.
In Debug mode, eclipse can compile and change class files in a running JVM, this is called Hot Code Replace.
The idea is that you can start a debugging session on a given runtime
workbench and change a Java file in your development workbench, and
the debugger will replace the code in the receiving VM while it is
running. No restart is required, hence the reference to "hot".
Source: Eclipse Wiki
There are certain circumstances where this won't work, in which case Eclipse will prompt you to restart.

This is called hot code replace:
The idea is that you can start a debugging session on a given runtime workbench and change a Java file in your development workbench, and the debugger will replace the code in the receiving VM while it is running. No restart is required, hence the reference to "hot".

It's not really possible to tell from the stream. Java's capable of both to some extent - you can, with some restrictions, replace classes loaded by a JVM using the debugger. There's also JRebel, which gets rid of a great deal of those restrictions.

Ability to change application code while running is the feature of debugger. In Eclipse and many other popular IDEs it works "out of the box". Feature has several restrictions: can't change method signatures, add/remove class members, etc.

If you are in debug mode you can make certain changes while the application is running. Whether or not it is valid varies based on if the change is to currently loaded code. For example, you could change a sort function while the program isn't sorting and it will use the new code the next time it sorts.

An alternative of software like VisualVM to programmatically find running java applications' values etc. by searching heap dumps?

I'm not experienced with java applications but I found out that finding static pointers etc. to these applications' memory addresses is often (nearly) impossible, apparently because of the java engine that handles the code (correct me if this way of naming it is wrong please).
Now, I've used VisualVM (https://visualvm.dev.java.net/) and it's great. I can select my java process and create a heap dump. It then shows me all classes and their values.
Can I use this method to continousely poll the heap dump and receive object values, for example the X Y and Z of a game? How would I programmatically interact with such application, and if this should not be done with VisualVM, what would be an alternative?
Edit: this is what I need to do:
I need to be able to find all classes with properties that have a certain value. For example: I'd search for the X coordinate (a float) and it should return the class "PlayerCoordsHandler" (just an example) and the corresponding float with it's value... or alternatively just a way to find this same float again (after restarting for example). This process does not have to be programmatic, aslong as requesting the value of the now known property (x float) can be retrieved programmatically (for example with a command line utility or reading from a file).
Edit2:
The target application is a windows executable (but made with java) and launches it's own java VM. It's not possible to add java parameters for debugging. This does not seem to be required though, as VirtualVM is able to debug the process just fine. Anyone knows how?
Thanks in advance.

It looks like you want to debug running Java applications.
The "official" Java debugger is JDB. I believe it's part of the JDK. It has the ability to set breakpoints, examine heaps, list and display and even change variables, show running threads and so on. The usual debugger stuff. But it's command line, which makes it a pain in the neck to work with.
Instead, it makes a lot of sense to use an IDE with integrated debugger. I use Eclipse. You can do all the usual debuggery things, including displaying windows with variables. You can set conditional breakpoints and there's much more. Specifically in answer to your question, you can set up watch expressions, which will be evaluated during the program's execution and their displays refreshed with new values when they change.
You may not want to run your Java app inside the IDE; or it may be running in a Web application server. That's no problem for JDB or Eclipse (or other IDEs, like NetBeans or IntelliJ Idea): They can connect to a running JVM and debug remotely with the same level of convenience.
A program being debugged like this, remotely or otherwise, run somewhat more slowly than if it were not. Your game, while being debugged, will run at rather bad-looking FPS; but it should still respond more or less normally to gameplay interaction.
Remote debugging:
To be able to attach your EclipseNetBeans debugger to a running Java process you need to start that process with the following Java options…
-Xdebug -Xrunjdwp:transport=dt_socket,address=3704,server=y,suspend=n

Have a look at YourKit. You can monitor CPU, memory and threads live, and generate dumps whenever you want. It can even compare different memory dumps to show you which objects were added/removed.
It's not free though, it has a 15 day (or 30 day?) fully functional eval period. If free is not a real concern it's definitely a great tool.

I good starting point is the jps and jstat tools added in Java 6 (i think). jps gives you the pid and main class for each application. jstat give you more details about process

Triggering a heapdump is usefull for post-mortem analysis of say memory leaks, but as the Java garbage collector moves objects around, you cannot use the memory values of a heapdump to reliably access those objects.
If you need a way to query internal values from outside of the application you could look into setting up an RMI service API via which you can retrieve the values you need.
Another method (if you just need to test something) could be to connect to the process via de Java debugging API.
If you know the JRE location that is used, you could rename java.exe and write a (C/C++) wrapper that adds the debug options listed by Carl and calls the renamed_java.exe in turn.
Another posibility might be to add or update classes in the .jar file of the application. You do not need the source to do this.
Tom, are you trying to reverse engineer an application that specifically tries to obfuscate its working? If so you might get further if you contact the manufacturer and ask them what possibilities they see for what you try to achieve?

You can easily generate a heap dump by creating your own JMX connection to the JVM, just like VisualVM does it. Analyzing the heapdump is very possible (the data is there and totally disconnected from the JVM so there is no interference from the gc).
However, unless it is a very specific scenario you are looking for you are probably much better off giving the heapdump to MAT and find a good workflow in there to use.
Edit: In this particular case it is probably better to create some kind of specific API to access the values from the outside (and maybe publish the values as MBeans using JMX). Taking a heap dump is way to much work if all you want to do is monitoring a few values.
Edit2: Based on your edits, it seems to me like you could really benefit from publishing your own MBean over JMX. I have to run for a meeting but, unless someone else does it while I am away, I will try to remember to give you some pointers later. Either in an edit of this one or in a new post.

If you want to poll the values of specific objects while your Java application is running you would probably find that using JMX is a better and more efficient approach rather than using a heap dump. With JMX you can define what values should be exposed and use tools such as VisualVM or JConsole to view them at runtime.

With VisualVM and heapdump you can find all classes with certain property by OQL:
var out = "";
var cls = filter(heap.classes(), "/java./(it.name)")
while (cls.hasNext()) {
var cl = cls.next();
var fls = cl.fields;
while (fls.hasMoreElements()) {
var fl = fls.nextElement();
if (/size/(fl.name)) {
out = toHtml(cl) + "." + fl.name + "()\n";
}
}
}
out.toString()
and write custom logging for BTrace
It is alternative for debugging.

FusionReactor could be a good alternative. For example;
VisualVM doesn’t give you a lot of insides on application memory
except for the total Heap allocation. Heap is a good metric to start
with, but I feel this is not enough to troubleshoot the actual cause
of a memory-related issue.
FusionReactor will display all of the memory spaces it detects, which
depends on the version of Java you’re running:
Heap allocation Non-Heap allocation CodeHeap (profiled and
non-profiled methods) Compressed Class Space FusionReactor also shows
the amount of memory that each generation takes Eden Space Old Space
Survivor Space
https://www.fusion-reactor.com/blog/java-visualvm-alternatives/