We Keep Coding

Method to iterating through classes in a package using Reflection

I am currently learning about Reflection and I have seen most of the similar posts to my question on stack however, I don't feel they fully answer my question on it.
I want to know if I have a package in Eclipse can I use Reflection to iterate through the classes in the package to find which one implements interface. There is only 1 interface so either they implement it or don't.
Could anyone provide a basic clear example as to how I may go about this? I have been told by some that this is just not possible whilst other people say it is possible using Reflection.
Thank you to any one who could help clear this matter.

Reflection API does not provide directly facility to iterate over classes in specific package. It concentrates on discovery of class once you have it.
To achieve what you want to have to read the class path of your application, iterate over the class path, open jars and go into directories, find files that end with *.class and get them as resource like getResource(full_class_name) or get it directly as class using Class.forName().
This method has a limitation: you will not see classes loaded by custom class loaders.
Unless this is an exercise I'd recommend you to use Reflections library that does everything you need and (probably) even more... :)
Some more details
Java system property "java.class.path" contains class path of your application separated with ; on Windows and : on Unix.
So, this is the way you can get list of class path elements:
System.getProperty("java.class.path").split(File.pathSeparator)
Here is some code sample that can help you to start:
for (String cpElement : System.getProperty("java.class.path").split(File.pathSeparator)) {
File cpFile = new File(cpElement);
if (!cpFile.exists()) {
continue;
}
if (cpFile.isDirectory()) {
findClassesInDirectory(cpFile);
} else {
findClassesInArchive(cpFile);
}
}
Etc, etc. I am leaving implementation of findClassesInDirectory() and findClassesInArchive() for you. Nice exercise. Have fun.

have you heard of guava-libraries for Java.
They provide useful utilities regarding reflection.
For your specific problem, I would have a look a the TypeToken and the method getTypes().
Good luck

yes you can so it. But with eclipse its more of an AST tree translation and not reflections. See if you want to iterate over source code and see which source file implements the interface you probably need AST tree walkers to do that. But on the other hand if you want to introspect at runtime the class files in a given package or a folder which implement the said interface then you need Reflections to do that. Either way its doable. I cant give you the code to try that out as a little time with Google will give you the answers you need though not all at the same place.

Reflection does not provide all operations for a total inspection.
For a known class one can get the physical URL of a class SomeClass using:
CodeSource codeSource = SomeClass.class.getProtectionDomain().getCodeSource();
if (codeSource == null) {
// Run-time class; URL into the rt.jar.
} else {
URL url = codeSource.getLocation();
// "file:/.... /classes/.../SomeClass.class"
// "jar:file:/... /someJar.jar!/.../SomeClass.class"
}
For Java run time classes (rt.jar) codeSource will be null.
On the URL of the jar you can open a (zip) file system, and walk through folders as any real file system.
You'll probably want to inspect classes without $ in the name (embedded, generated anonymous classes): only fileName.matches("[^$]+\\.class").

Programatic code modification (e.g. variable extraction) in Java

I know it's possible to do nice stuff with Reflection, such as invoking methods, or altering the values of fields. Is it possible to do heavier code modification, though, at runtime and programmatically?
For instance, if I have a method:
public void foo(){
this.bar = 100;
}
Can I write a program that modifies the innards of this method, notices that it assigns a constant to a field, and turns it into the following:
public int baz = 100;
public void foo(){
this.bar = baz;
}
Perhaps Java isn't really the language to do this kind of thing in - if not, I'm open to suggestions for languages that would allow me to basically reparse or inspect code in this way, and be able to alter it so precisely. I might be pipe dreaming here though, so please tell me if this is the case also.

Just adding a suggestion from a friend - Apache Commons' BCEL looks excellent:
http://commons.apache.org/bcel/manual.html
The Byte Code Engineering Library (Apache Commons BCEL™) is intended to
give users a convenient way to analyze, create, and manipulate (binary)
Java class files (those ending with .class). Classes are represented by
objects which contain all the symbolic information of the given class:
methods, fields and byte code instructions, in particular.
Such objects can be read from an existing file, be transformed by a
program (e.g. a class loader at run-time) and written to a file again.
An even more interesting application is the creation of classes from
scratch at run-time. The Byte Code Engineering Library (BCEL) may be
also useful if you want to learn about the Java Virtual Machine (JVM)
and the format of Java .class files.

You are looking for software that allows you to do bytecode manipulation, there are several frameworks to achieve this, but the two most known currently are:
ASM
javassist
When performing bytecode modifications at runtime in Java classes keep in mind the following:
If you change a class's bytecode after a class has been loaded by a classloader, you'll have to find a way to reload it's class definition (either through classloading tricks, or using hotswap functionalities)
If you change the classes interface (example add new methods or fields) you will be able only to reach them through reflection.

It's probably fair to say that Java wasn't designed with this purpose in mind, but you can do it potentially. How and when depends a little on the ultimate aim of the exercise. A couple of options:
At the source code level, you can use the Java Compiler API to
compile arbitrary code into a class file (which you can then load).
At the bytecode level, you can write an agent that installs a
ClassFileTransformer to arbitrarily alter a class "on the fly"
as it is loaded. In practice, if you do this, you will also probably
make use of a library such as BCEL (Bytecode Engineering
Library) to make manipulating the class easier.

You want to investigate program transformation systems (PTS), which provide general facilities for parsing and transforming languages at the source level. PTS provide rewrite rules that say in effect, "if you see this pattern, replace it by that pattern" using the surface syntax of the target language. This is done using full parsers so the rewrite rule really operates on language syntax and not text; such rewrite rules obviously won't attempt to modify code-like text in comments, unlike tools based on regexps.
Our DMS Software Reengineering Toolkit is one of these. It provides not only the usual parsing, AST building and prettyprinting (reproducing compilable source code complete with comments), but also supports symbol tables and control and data flow analysis. These are needed for almost any interesting transformations. DMS also has front ends for a variety of dialects of Java as well as many other languages.
Bytecode transformers exist because they are much easier to build; it is pretty easy to "parse" bytecode. Of course, you can't make permanent source changes with a bytecode transformer, so it is lot less useful.

You mean like this?
String script1 = "println(\"OK!\");";
eval( script1 );
script1 += "println(\"... well, maybe NOT OK after all\");";
eval( script2 );
Output:
OK!
OK!
... well, maybe NOT OK after all
... use a scripting extension to Java. Groovy and other things like that would probably allow you to do what you want. I've written a scripting extension which integrates with Java through reflection almost seamlessly myself; contact me if you're interested in the details.

Is Compiling String as code possible?

I have an app that gets the content of an html file.
Lets say the text of the page is:
String[] arr = new String[] {"!","#","#"};
for (String str : arr) {
write(str);
}
Can I somehow compile this text and run the code within my app?
Thanks

Use Janino. It's a java runtime in-memory compiler. Way easier than BCEL and the likes.
From the homepage:
"What is Janino?
Janino is a super-small, super-fast Java™ compiler. Not only can it compile a set of source files to a set of class files like the JAVAC tool, but also can it compile a Java™ expression, block, class body or source file in memory, load the bytecode and execute it directly in the same JVM. Janino is not intended to be a development tool, but an embedded compiler for run-time compilation purposes...

You can use the javac compiler, or the Java Compiler API or the BeanShell library (or similar). You can compile it any number of ways, none terribly simple which often leads to finding another way to solve your problem.
Instead of generating source and compiling its common to generate byte code directly using ASM, Javaassist, BCEL or the like
This appears to be the same as
for(char ch: "!##".toCharArray())
write(ch);
which is likely to be the same as
write("!##");

Since the question is tagged android:
The answers posted so far only apply to the “standard” JVM, not to Android's Dalvik VM. In principle, it is possible on Android too. I don't know if there's an existing Java compiler that you can embed, but you would probably generate the final Dalvik bytecode using dexmaker. It may be possible to combine an existing Java compiler with dexmaker.
But please think twice before attempting anything like this, and be very careful. The last thing you want is a way for an attacker to execute arbitrary code on your user's hardware.

You can try javassist, it's not full Java though.

This is not usually that hard to do, but I have to ask can you give more detail on exactly what it is you are trying to accomplish. I do this type thing all the time. This is just another example of getting information from the user and using it somewhere else in your code. Since your using java maybe look at the string API http://docs.oracle.com/javase/6/docs/api/java/lang/String.html and the string tokenizer http://docs.oracle.com/javase/6/docs/api/index.html?java/lang/package-summary.html
Now you can break the string down into single values one word or other value at a time. From there you can use functions such as isNAN() from the float or double class to determine if it a number or string or whatever it is your testing for. Now you know what you’re dealing with you can reconstructed the data in a usable form.
Note for values if you want to use them as values use Float(string value) constructor. i.e Float x = new Float(myString)

Simple List of All Java Standard Classes and Methods?

I'm building a very simple Java parser, to look for some specific usage models. This is in no way lex/yacc or any other form of interpreter/compiler for puposes of running the code.
When I encounter a word or a set of two words separated by a dot ("word.word"), I would like to know if that's a standard Java class (and method), e.g. "Integer", or some user defined name. I'm not interested in whether the proper classes were included/imported in the code (i.e. if the code compiles well), and the extreme cases of user defined classes that override the names of standard Java classes also does not interest me. In other words: I'm okay with false negative, I'm only interesting in being "mostly" right.
If there a place wher I could find a simple list of all the names of all Java standard classes and methods, in the form easily saved into a text file or database? (J2SE is okay, but J2EE is better). I'm familiar with http://java.sun.com/j2se/ etc, but it seems I need a terrible amount of manual work to extract all the names from there. Also, the most recent JDK is not neccesary, I can live with 1.4 or 1.5.
Clarification: I'm not working in Java but in Python, so I can't use Java-specific commands in my parsing mechanism.
Thanks

What's wrong with the javadoc? The index lists all classes, methods, and static variables. You can probably grep for parenthesis.

To get all classes and methods you can look at the index on
http://java.sun.com/javase/6/docs/api/index-files/index-1.html
This will be 10's of thousands classes and method which can be overwhelming.
I suggest instead you use auto-complete in your IDE. This will show you all the matching classes/methods appropriate based on context.
e.g. say you have a variable
long time = System.
This will show you all the methods in System which return a long value, such as
long time = System.nanoTime();
Even if you know a lot of the method/classes, this can save you a lot of typing.

If you just want to create a list of all classes in Java and their methods (so that you can populate a database or an XML file), you may want to write an Eclipse-plugin that looks at the entire JavaCore model, and scans all of its classes (e.g., by searching all subtypes of Object). Then enumerate all the methods. You can do that technically to any library by including it in your context.

IBM had a tool for creating XML from JavaDocs, if I am not mistaken:
http://www.ibm.com/developerworks/xml/library/x-tipjdoc/index.html

There's also an option to either parse classlist file from jre/lib folder or open the jsse.jar file, list all classes there and make a list of them in dot-separated form by yourself.

When I encounter a word or a set of two words separated by a dot ("word.word"), I would like to know if that's a standard Java class (and method), e.g. "Integer", or some user defined name.
If thats what you're after, you could do without a (limited) list of Java Classes by using some simple reflection:
http://java.sun.com/developer/technicalArticles/ALT/Reflection/
try {
Class.forName("word.word");
System.out.println("This is a valid class!");
} catch (ClassNotFoundException e) {
System.out.println("This is not a valid class.");
}
Something like this should be enough for your purposes, with he added benefit of not being limited to a subset of classes, and extensible by any libraries on the classpath.

Rewriting method calls within compiled Java classes

I want to replace calls to a given class with calls to anther class within a method body whilst parsing compiled class files...
or put another way, is there a method of detecting usages of a given class in a method and replacing just that part of the method using something like javaassist.
for example.. if I had the compiled version of
class A { public int m() { int i = 2; B.multiply(i,i); return i; } }
is there a method of detecting the use of B and then altering the code to perform
class A { public int m() { int i = 2; C.divide(i,i); return i; } }
I know the alternative would be to write a parser to grep the source files for usages but I would prefer a more elegant solution such as using reflection to generate new compiled class files.
Any thoughts ?

As #djna says, it is possible to modify bytecode files before you load them, but you probably do not want to do this:
The code that does the code modification is likely to be complex and hard to maintain.
The code that has been modified is likely to be difficult to debug. For a start, a source level debugger will show you source code that no longer corresponds to the code that you are actually editing.
Bytecode rewriting is useful in certain cases. For example, JDO implementations use bytecode rewriting to replace object member fetches with calls into the persistence libraries. However, if you have access to the source code for these files, you'll get a better (i.e. more maintainable) solution by preprocessing (or generating) the source code.
EDIT: and AOP or Groovy sound like viable alternatives too, depending on the extent of rewriting that you anticipate doing.

BCEL or ASM.
I recently looked at a number of libraries for reading Java class files. BCEL was the fastest, had the least number of dependencies, compiled out of the box, and had a deliciously simple API. I preferred BCEL to ASM because ASM has more dependencies (although the API is reputedly simpler).
AspectJ, as previously mentioned, is another viable option.
BCEL is truly simple. You can get a list of methods in three lines of code:
ClassParser cp = new ClassParser( "A.class" );
JavaClass jc = cp.parse();
Method[] m = jc.getMethods();
There are other API facilities for further introspection, including, I believe, ways to get the instructions in a method. However, this solution will likely be more laborious than AspectJ.
Another possibility is to change the multiply or divide methods themselves, rather than trying to change all instances of the code that calls the operation. That would be an easier road to take with BCEL (or ASM).

The format of byte code for compiled Java is specified and products exist that manipulate it.
This library appears to have the capability you need. I've no idea how easy it is to do these transformations reliably.

If you don't mind using Groovy, you can intercept the call to B.multiply and replace it with C.divide. You can find an example here.

It's much easier to perform these operations ahead-of-time, where the executable on disk is modified before launching the application. Manipulating the code in memory at run time is even more prone to errors than manipulating code in memory in C/C++. Why do you need to do this?