I'm looking for a way to add fields to an Thread on the fly by rewriting the byte code and reloading the class, not sure if it is at all possible. Any pointers welcome. I found some info on modifying and loading a class, and I know JRebel can seamlessly hot swap your code but not sure if the same approach/tools apply here.
The motivation here is exploring a theoretically better alternative to thread local objects. Should the method work I should be able to replace thread local with an annotation and the result should outperform current JDK implementation.
PS: Please save me the "root of all evil speech"
Clarifying use case:
Imagine I have a class with a ThreadLocal:
class A {
ThreadLocal<Counter> counter;
...
counter.get().inc()
}
I'd like to replace that with an annotation:
class A {
#ThreadLocal
Counter counter;
...
counter.inc()
}
But instead of the above code getting generated I'd like to mutate Thread such that Thread will now have an Acounter field and the actual code will be:
class A {
// Nothing here, field is now in Thread
...
Thread.currentThread().Acounter.inc()
}
At present it is impossible to redefine a class at runtime such that the redefinition will result in new methods or fields. This is due to the complexity involved in scanning the heap for all existing instances and transforming them + their references + potential Unsafe field offset base updaters (like AtomicFieldUpdater).
This limitation may be lifted as part of the JEP-159 but as discussed on the concurrency-interest mailing group this is a big impact change so may never happen at all.
Using Javaassist/similar will allow the transformation of a class to a new class with new methods/fields. This class can be loaded by a ClassLoader and used at runtime, but it's definition will not replace existing instances. So it will not be possible to use this method combined with an agent to redefine the class as an instrumentation redefinition is limited such that: "The redefinition may change method bodies, the constant pool and attributes. The redefinition must not add, remove or rename fields ..." see here.
So for now, NO.
If you would like to change the behaviour of "class" at runtime, you could try javassist. API is here
I have seen custom class loading solution that dynamically reloaded JARs - you define one ClassLoader per JAR file and use it to load the classes from that JAR; to reload entire JAR you just "kill" its ClassLoader instance and create another one (after you replace the JAR file).
I don't think it's possible to tweak Java's internal Thread class this way because you don't have control over System ClassLoader. A possible solution is to have a CustomThreadWeaver class that would generate a new class extending Thread with the variables you need and use a custom DynamicWeavedThreadClassLoader to load them.
Good luck and show us your monster when you succeed ;-)
Possible
using instrumentation, and possibly libraries like javassist to modify code on fly. (however, adding and removing fields, methods or constructors are currently not possible)
//Modify code using javassist and call CtClass#toBytecode() or load bytecode from file
byte[] nevcode;
Class<?> clz = Class.forName("any.class.Example");
instrumentationInstace.redefineClasses(new ClassDefinition(clz, nevcode));
Do not forget to add Can-Redefine-Classes: true to your java agent's manifest.
Real example - optimizing java < 9 string.replace(CharSequence, CharSequence) using javassist:
String replace_src =
"{String str_obj = this;\n"
+ "char[] str = this.value;\n"
+ "String find_obj = $1.toString();\n"
+ "char[] find = find_obj.value;\n"
+ "String repl_obj = $2.toString();\n"
+ "char[] repl = repl_obj.value;\n"
+ "\n"
+ "if(str.length == 0 || find.length == 0 || find.length > str.length) {\n"
+ " return str_obj;\n"
+ "}\n"
+ "int start = 0;\n"
+ "int end = str_obj.indexOf(find_obj, start);\n"
+ "if(end == -1) {\n"
+ " return str_obj;\n"
+ "}\n"
+ "int inc = repl.length - find.length;\n"
+ "int inc2 = str.length / find.length / 512;\ninc2 = ((inc2 < 16) ? 16 : inc);\n"
+ "int sb_len = str.length + ((inc < 0) ? 0 : (inc * inc2));\n"
+ "StringBuilder sb = (sb_len < 0) ? new StringBuilder(str.length) : new StringBuilder(sb_len);\n"
+ "while(end != -1) {\n"
+ " sb.append(str, start, end - start);\n"
+ " sb.append(repl);\n"
+ " start = end + find.length;\n"
+ " end = str_obj.indexOf(find_obj, start);\n"
+ "}\n"
+ "if(start != str.length) {\n"
+ " sb.append(str, start, str.length - start);\n"
+ "}\n"
+ "return sb.toString();\n"
+"}";
ClassPool cp = new ClassPool(true);
CtClass clz = cp.get("java.lang.String");
CtClass charseq = cp.get("java.lang.CharSequence");
clz.getDeclaredMethod("replace", new CtClass[] {
charseq, charseq
}).setBody(replace_src);
instrumentationInstance.redefineClasses(new ClassDefinition(Class.forName(clz.getName(), false, null), clz.toBytecode()));
This seems to be a question of using the right tool for the job. A similar question has been asked here: Another Stack Overflow Question and the Javaassist byte code manipulation library was a possible solution.
But without further detail into the reasons why this is being attempted, it seems like the real answer is to use the right tool for the job. For example, with Groovy the ability to dynamically add methods to the language.
You could try creating a JVM Agent that makes use of the java.lang.instrument API and more specifically make use of the retransform method that " facilitates the instrumentation of already loaded classes" and then make use of Javassist (or ASM) as mentioned to deal with the bytecode.
More info on the java.lang.instrument API
To do what you want, the simpler alternative would be to use a subclass of Thread, run it, and then inside that thread execute the code from your example (together with a cast of currentThread() to your subclass).
What you are attempting to do is not possible.
Since you already know about ThreadLocal, you already know what the suggested solution is.
Alternatively, you can sub-class Thread and add your own fields; however, only those threads that you explicitly create of that class will have those fields, so you will still have to be able to "fall back" to using a thread local.
The real question is "why?", as in "why is a thread local insufficient for your requirements?"
Related
In Cucumber 7.4.1+ TypeRegistry is deprecated in favour of annotations.
Indeed, as of today, I have never used anything but #ParameterType to define my ParameterTypes. Searching for alternatives, TypeRegistry is the only one I have found - but if it is "on the way out", of course I'd rather not start using it now.
Given a construct like this I cannot use annotations because those cannot take static parameters:
enum SpecialDate implements Supplier<Date> {
TODAY { #Override public Date get() { return Date(); } },
// YESTERDAY, etc.
;
static String typeSafeRegEx() {
return Arrays.stream(Zeitpunkt.values())
.map(SpecialDate::specName)
.collect(Collectors.joining("|"));
}
static SpecialDate from(final String specName) {
return valueOf(StringUtils.upperCase(specName));
}
String specName() {
return StringUtils.capitalize(StringUtils.lowerCase(this.name()));
}
}
public class ParameterTypes {
// does not compile: "Attribute value must be constant"
#ParameterType("(" + SpecialDate.typeSafeRegEx() + ")")
public Date specialDate(final String specName) {
return SpecialDate.from(specName).get();
}
}
A so-specified regEx is nice, because it will only match values guaranteed to be mappable, so I need no additional error handling code beyond Cucumber's own. The list of allowed values is also maintenance-free (compared to a "classic" switch which would silently grow incorrect when adding new values).
The alternative would be to use an unsafe switch + default: throw, strictly worse because it has to be maintained manually.
(Or, I guess, to just valueOf whatever + wrap into a more specific exception, when it eventually fails.)
To me, Cucumber's native UndefinedStepException appears to be the best outcome on a mismatch, because everyone familiar with Cucumber will immediately recognise it, unlike a project-specific one.
I see that e.g. the ParameterType class is not deprecated but cannot seem to find information how to use it without TypeRegistry.
FWIW:
Updating the libraries or Java would not be an issue. (But downgrading is sadly not viable.)
Business Specialists want to write examples like [Today], [Today + 1], [Yesterday - 3], etc. If this can be realised more elegantly using a different approach, X/Y answers would also be welcome.
An example step looks like this:
And I enter into the field 'Start of Implementation' the <begin>
Examples:
| begin
| [Today]
| [Today + 1]
Context:
I have created a small (java) multithread server for a game. Despite my best efforts of following the best practices it turned out that some methods that were intended to be called from just one thread got called from 2 or more threads. After debugging and analysis I have managed to "fix" my design but I was wondering:
The question:
Is there a tool (or if not - is it possible (and how) to be developed) that allows you to mark some methods with annotations like #SingleThread or #ThreadCount(2) or #ThreadNameLike("my_fancy_thread_group*") which counts/monitors/logs access to these methods like:
#SingleThread - checks if this method is always accessed by only thread
#ThreadCount(2) - is accessed by two threads exactly
#ThreadNameLike - is accessed only by threads with name matching the pattern(s)
The idea is to do a TEST run of the program and get at least log record that annotated condition is violated.
I was thinking that probably AspectJ can do the job to some extend with it's pointcuts but then I realized that some approach similar to Dagger / Dagger2 will be better, i.e. when you want to test your server you will have to turn on an annotation processor (let's called it hypothetically "SafetyFirst") which will generate adapter (wrapper?) classes which contain the monitoring code. Then you will run the server, run some load tests and then check the logs for violations (or in an ideal world - get a report file).
I fully realize that such tool:
will not provide 100% coverage of all potential cases;
will mask/trigger heisenbugs
will slow down the examined program
but at very least it can serve as an early warning system that will clearly advertise violations of the intended design.
I used a similar test with AspectJ load time weaving for intended printing of all function calls within my package.
Best way of load time weaving is you dont dirty your classes like compile time wevaing. When you remove -javaagent:<path to aspectj lib> and your custom astpect lib classpath entry from your run command. Then all all gone, clear.
I made some changes and implemented a test covering #ThreadCount functionality you asked. You need to download and install AspectJ.
Please see code snippets:
aspect Profile {
private static Map<String, AtomicInteger> counterMap = new HashMap<String, AtomicInteger>();
pointcut threadCountPc(test.ThreadCount tc) : execution(* test..*(..)) && #annotation(tc);
Object around(test.ThreadCount tc) : threadCountPc(tc) {
String signature = thisJoinPointStaticPart.getSignature().toString();
AtomicInteger counter = getCounter(signature);
int currentValue = counter.incrementAndGet();
if (currentValue >= tc.value()){
System.out.println("[Thread Name:" + Thread.currentThread().getName() +
"] Method Name:" + signature + ", threadCount:" + currentValue + " exceeds " + tc.value());
}
try{
return proceed(tc);
}finally{
counter.decrementAndGet();
}
}
private static AtomicInteger getCounter(String methodName){
AtomicInteger value = counterMap.get(methodName);
if (value == null){
synchronized (counterMap){
value = counterMap.get(methodName);
if (value == null){
value = new AtomicInteger(0);
counterMap.put(methodName, value);
}
}
}
return value;
}
}
Compiling: "C:/aspectj1.8/bin/ajc.bat" profile_ft.aj -cp C:/aspectj1.8/lib/aspectjrt.jar;../test/. -1.6 -outxml -outjar profile_ft.jar
Running: java -javaagent:C:/aspectj1.8/lib/aspectjweaver.jar -cp aspectj/profile_ft.jar;test test.Test1
I set textures in each individual file, this is the non efficient way to set it
this.setUnlocalizedName("ItemName");
this.setTextureName("MyModName:ItemName");
This way made sense to me, but didn't work:
this.setUnlocalizedName("ItemName");
this.setTextureName(OtherClassName.MODID + ":" + this.getUnlocalizedName());
the 'OtherClassName.MODID' is referring to a variable in another class that contains 'MyModName'
The this.getUnlocalizedName() gets the UnlocalizedName that has been declared, 'ItemName'
Any help? I am not sure why it doesn't work.
getUnlocalizedName is slightly weird - it returns the string you passed into setUnlocalizedName, but with "item." at the start. The joys of working with deobfuscated code...
This would work:
String name = "ItemName";
this.setUnlocalizedName(name);
this.setTextureName(OtherClassName.MODID + ":" + name);
Note that it's not more efficient as in faster to run, but it might be faster to write if you change the item name a lot.
The situation seems to be abnormal, but I was asked to build serializer that will parse an object into string by concatenating results of "get" methods. The values should appear in the same order as their "get" equivalent is declared in source code file.
So, for example, we have
Class testBean1{
public String getValue1(){
return "value1";
}
public String getValue2(){
return "value2";
}
}
The result should be:
"value1 - value2"
An not
"value2 - value1"
It can't be done with Class object according to the documentation. But I wonder if I can find this information in "*.class" file or is it lost? If such data exists, maybe, someone knows a ready to use tool for that purpose? If such information can't be found, please, suggest the most professional way of achieving the goal. I thought about adding some kind of custom annotations to the getters of the class that should be serialized.
If you want that you have to parse the source code, not the byte code.
There are a number of libraries that parse a source file into a node tree, my favorite is the javaparser (hosted at code.google.com), which, in a slightly modified version, is also used by spring roo.
On the usage page you can find some samples. Basically you will want to use a Visitor that listens for MethodDefinitions.
Although reflection does not anymore (as of java 7 I think) give you the methods in the order in which they appear in the source code, the class file appears to still (as of Java 8) contain the methods in the order in which they appear in the source code.
So, you can parse the class file looking for method names and then sort the methods based on the file offset in which each method was found.
If you want to do it in a less hacky way you can use Javassist, which will give you the line number of each declared method, so you can sort methods by line number.
I don't think the information is retained.
JAXB, for example, has #XmlType(propOrder="field1, field2") where you define the order of the fields when they are serialized to xml. You can implemenet something similar
Edit: This works only on concrete classes (the class to inspect has its own .class file). I changed the code below to reflect this. Until diving deeper into the ClassFileAnalyzer library to work with classes directly instead of reading them from a temporary file this limitation exists.
Following approach works for me:
Download and import following libarary ClassFileAnalyzer
Add the following two static methods (Attention! getClussDump() needs a little modification for writing out the class file to a temporary file: I removed my code here because it's very special at this point):
public static String getClassDump(Class<?> c) throws Exception {
String classFileName = c.getSimpleName() + ".class";
URL resource = c.getResource(classFileName);
if (resource == null) {
throw new RuntimeException("Works only for concreate classes!");
}
String absolutePath = ...; // write to temp file and get absolute path
ClassFile classFile = new ClassFile(absolutePath);
classFile.parse();
Info infos = new Info(classFile, absolutePath);
StringBuffer infoBuffer = infos.getInfos();
return infoBuffer.toString();
}
public static <S extends List<Method>> S sortMethodsBySourceOrder(Class<?> c, S methods) throws Exception {
String classDump = getClassDump(c);
int index = classDump.indexOf("constant_pool_count:");
final String dump = classDump.substring(index);
Collections.sort(methods, new Comparator<Method>() {
public int compare(Method o1, Method o2) {
Integer i1 = Integer.valueOf(dump.indexOf(" " + o1.getName() + lineSeparator));
Integer i2 = Integer.valueOf(dump.indexOf(" " + o2.getName() + lineSeparator));
return i1.compareTo(i2);
}});
return methods;
}
Now you can call the sortMethodsBySourceOrder with any List of methods (because sorting arrays is not very comfortable) and you will get the list back sorted.
It works by looking at the class dumps constant pool which in turn can be determined by the library.
Greetz,
GHad
Write your custom annotation to store ordering data, then use Method.getAnnotation(Class annotationClass)
I got a recommendation to use this syntax when logging in java:
if (logger.isLoggable(Log.FINE))
{
logger.fine("bla"+" bla"+" bla");
}
The reason for this is to avoid the redundant construction of the parameter string incase the logging level is lower than "FINE". (in the example above - 5 redundant string object. (" bla"X3, " bla bla" and "bla bla bla").
I'd like to hear what others are doing about this or if you think that this is necessary at all.
Thanks!!
Some newer logging frameworks allow you to specify arguments as parameters, and won't evaluate them if there's no logging.
The example I found is LogBack, the successor to Log4j. Here's the info: http://www.infoq.com/news/2007/08/logback
This gives you the best of both worlds, so to speak. Elegant syntax yet good performance.
Log4j code example:
if( logger.isDebugEnabled() ) {
logger.debug( "User with account " +
user.getAccount() + " failed authentication; " +
"supplied crypted password " + user.crypt(password) +
" does not match." );
}
Equivalent LogBack code:
logger.debug( "User with account {} failed authentication; " +
"supplied crypted password {} does not match.",
user.getAccount(), user.crypt(password) );
This defers the cost of message assembly until LOGBack has ascertained whether or not this message will be viewed. It doesn't defer the cost of retrieving expensive parameters, such as the password crypting in the above example.
String objects are immutable, and repeated concatenation is therefore an expensive operation. It requires repeated memory allocation, object creation and iteration. Considering that some logging calls at the finer log levels can be invoked thousands or millions of times per minute, it might be a considerable performance gain to do as you illustrate. Though, for a smaller application, it might not be worth the extra effort.
As a side note: You can save even more performance, where this is truly critical by using a constant such as this:
public static final boolean DEBUG = false;
If you now wrap the logging code in an if-block such as this, the JVM will be able to completely optimize away the debug calls when running in product mode. This is as close as you get to a C #ifdef.
if (Globals.DEBUG) {
// Logging call
}
Absolutely necessary for debug type logging. It something like 10x quicker to check the log level first than create the string and throw it away.
This is an improvement (good) but it can be improved on a little.
Set up final flags for each logging level (FINE, etc) in a global object used as config, then use a StringBuffer to build up your debugging output -- you can even format numbers into the stream at the same time.
public class MyAppConfig {
public final boolean FINE=true;
// ... other fields
}
public class MyApp {
void someFunction() {
...
int imagesProcessed;
imagesProcessed = processImages();
if (MyAppConfig.FINE) logger.fine(new StringBuffer(35).
append("Count of images processed: ").append(imagesProcessed).toString());
...
}
}
Here the string buffer is set up with an 'initial capacity' of 35 characters. If you know how many characters are going to be generated you can provide hints to StringBuffer.