I have a 32-bit Java service with scalability problems: with high user count we run out of memory because of excessive thread count. In the long term, I plan to switch to 64-bit and to reduce the threads-per-user ratio. In the short term, I'd like to reduce the stack size (-Xss, -XX:ThreadStackSize) to get some more headroom. But this is risky because if I make it too small, I'm going to get StackOverflowErrors.
How can I measure the average and maximum stack size for my application to guide my decision for an optimal -Xss value? I'm interested in two possible approaches:
Measuring a running JVM during integration testing. What profiling tools will report max stack depth?
Static analysis of the application looking for deep call hierarchies. Reflection in dependency injection makes it unlikely that this would work.
Update: I know the long-term right way to fix this problem. Please focus on the question I've asked: how do I measure stack depth?
Update 2: I got a nice answer on a related question specifically about JProfiler: Can JProfiler measure stack depth? (I posted the separate question as per JProfiler's community support recommendations)
You can get an idea of the stack depth with something like an aspect that can be woven to your code (load time weaver to allow advising all loaded code except system class loader). The aspect would work around all executed code and would be able to note when you are calling a method and when you return. You can use this to capture most of your stack usage (you'll miss anything loaded from the system class loader, e.g. java.*). While not perfect, it avoids having to change your code to gather StackTraceElement[] at sample points and also gets you into non-jdk code that you might not have written.
For example (aspectj):
public aspect CallStackAdvice {
pointcut allMethods() : execution(* *(..)) && !within(CallStackLog);
Object around(): allMethods(){
String called = thisJoinPoint.getSignature ().toLongString ();
CallStackLog.calling ( called );
try {
return proceed();
} finally {
CallStackLog.exiting ( called );
}
}
}
public class CallStackLog {
private CallStackLog () {}
private static ThreadLocal<ArrayDeque<String>> curStack =
new ThreadLocal<ArrayDeque<String>> () {
#Override
protected ArrayDeque<String> initialValue () {
return new ArrayDeque<String> ();
}
};
private static ThreadLocal<Boolean> ascending =
new ThreadLocal<Boolean> () {
#Override
protected Boolean initialValue () {
return true;
}
};
private static ConcurrentHashMap<Integer, ArrayDeque<String>> stacks =
new ConcurrentHashMap<Integer, ArrayDeque<String>> ();
public static void calling ( String signature ) {
ascending.set ( true );
curStack.get ().push ( signature.intern () );
}
public static void exiting ( String signature ) {
ArrayDeque<String> cur = curStack.get ();
if ( ascending.get () ) {
ArrayDeque<String> clon = cur.clone ();
stacks.put ( hash ( clon ), clon );
}
cur.pop ();
ascending.set ( false );
}
public static Integer hash ( ArrayDeque<String> a ) {
//simplistic and wrong but ok for example
int h = 0;
for ( String s : a ) {
h += ( 31 * s.hashCode () );
}
return h;
}
public static void dumpStacks(){
//implement something to print or retrieve or use stacks
}
}
And a sample stack might be like:
net.sourceforge.jtds.jdbc.TdsCore net.sourceforge.jtds.jdbc.JtdsStatement.getTds()
public boolean net.sourceforge.jtds.jdbc.JtdsResultSet.next()
public void net.sourceforge.jtds.jdbc.JtdsResultSet.close()
public java.sql.Connection net.sourceforge.jtds.jdbc.Driver.connect(java.lang.String, java.util.Properties)
public void phil.RandomStackGen.MyRunnable.run()
Very slow and has its own memory issues but can be workable to get you the stack information you need.
You can then use the max_stack and max_locals for each method in your stack traces to compute a frame size (see class file format) for the method. Based on the vm spec I believe this should be (max_stack+max_locals)*4bytes for the max frame size for a method (long/double occupy two entries on the operand stack/local vars and is accounted for in max_stack and max_locals).
You can easily javap the classes of interest and see the frame values if you don't have that much in your call stacks. And something like asm provides you with some easy tools to use to do this on a larger scale.
Once you have this computed, you need to estimate additional stack frames for JDK classes that might be called by you at your max stack points and add that to your stack sizes. It wont be perfect but it ought to get you a decent starting point for -Xss tuning without hacking around the JVM/JDK.
One other note: I don't know what JIT/OSR does to frame sizes or stack requirements so do be aware that you may have different impacts from -Xss tuning on a cold vs. warm JVM.
EDIT had a few hours of down time and threw together another approach. This is a java agent that will instrument methods to keep track of a max stack frame size and stack depth. This will be able to instrument most of the jdk classes along with your other code and libraries, giving you better results than the aspect weaver. You need asm v4 for this to work. It was more for the fun of it so file this under plinking java for fun, not profit.
First, make something to track the stack frame size and depth:
package phil.agent;
public class MaxStackLog {
private static ThreadLocal<Integer> curStackSize =
new ThreadLocal<Integer> () {
#Override
protected Integer initialValue () {
return 0;
}
};
private static ThreadLocal<Integer> curStackDepth =
new ThreadLocal<Integer> () {
#Override
protected Integer initialValue () {
return 0;
}
};
private static ThreadLocal<Boolean> ascending =
new ThreadLocal<Boolean> () {
#Override
protected Boolean initialValue () {
return true;
}
};
private static ConcurrentHashMap<Long, Integer> maxSizes =
new ConcurrentHashMap<Long, Integer> ();
private static ConcurrentHashMap<Long, Integer> maxDepth =
new ConcurrentHashMap<Long, Integer> ();
private MaxStackLog () { }
public static void enter ( int frameSize ) {
ascending.set ( true );
curStackSize.set ( curStackSize.get () + frameSize );
curStackDepth.set ( curStackDepth.get () + 1 );
}
public static void exit ( int frameSize ) {
int cur = curStackSize.get ();
int curDepth = curStackDepth.get ();
if ( ascending.get () ) {
long id = Thread.currentThread ().getId ();
Integer max = maxSizes.get ( id );
if ( max == null || cur > max ) {
maxSizes.put ( id, cur );
}
max = maxDepth.get ( id );
if ( max == null || curDepth > max ) {
maxDepth.put ( id, curDepth );
}
}
ascending.set ( false );
curStackSize.set ( cur - frameSize );
curStackDepth.set ( curDepth - 1 );
}
public static void dumpMax () {
int max = 0;
for ( int i : maxSizes.values () ) {
max = Math.max ( i, max );
}
System.out.println ( "Max stack frame size accummulated: " + max );
max = 0;
for ( int i : maxDepth.values () ) {
max = Math.max ( i, max );
}
System.out.println ( "Max stack depth: " + max );
}
}
Next, make the java agent:
package phil.agent;
public class Agent {
public static void premain ( String agentArguments, Instrumentation ins ) {
try {
ins.appendToBootstrapClassLoaderSearch (
new JarFile (
new File ( "path/to/Agent.jar" ) ) );
} catch ( IOException e ) {
e.printStackTrace ();
}
ins.addTransformer ( new Transformer (), true );
Class<?>[] classes = ins.getAllLoadedClasses ();
int len = classes.length;
for ( int i = 0; i < len; i++ ) {
Class<?> clazz = classes[i];
String name = clazz != null ? clazz.getCanonicalName () : null;
try {
if ( name != null && !clazz.isArray () && !clazz.isPrimitive ()
&& !clazz.isInterface ()
&& !name.equals ( "java.lang.Long" )
&& !name.equals ( "java.lang.Boolean" )
&& !name.equals ( "java.lang.Integer" )
&& !name.equals ( "java.lang.Double" )
&& !name.equals ( "java.lang.Float" )
&& !name.equals ( "java.lang.Number" )
&& !name.equals ( "java.lang.Class" )
&& !name.equals ( "java.lang.Byte" )
&& !name.equals ( "java.lang.Void" )
&& !name.equals ( "java.lang.Short" )
&& !name.equals ( "java.lang.System" )
&& !name.equals ( "java.lang.Runtime" )
&& !name.equals ( "java.lang.Compiler" )
&& !name.equals ( "java.lang.StackTraceElement" )
&& !name.startsWith ( "java.lang.ThreadLocal" )
&& !name.startsWith ( "sun." )
&& !name.startsWith ( "java.security." )
&& !name.startsWith ( "java.lang.ref." )
&& !name.startsWith ( "java.lang.ClassLoader" )
&& !name.startsWith ( "java.util.concurrent.atomic" )
&& !name.startsWith ( "java.util.concurrent.ConcurrentHashMap" )
&& !name.startsWith ( "java.util.concurrent.locks." )
&& !name.startsWith ( "phil.agent." ) ) {
ins.retransformClasses ( clazz );
}
} catch ( Throwable e ) {
System.err.println ( "Cant modify: " + name );
}
}
Runtime.getRuntime ().addShutdownHook ( new Thread () {
#Override
public void run () {
MaxStackLog.dumpMax ();
}
} );
}
}
The agent class has the premain hook for instrumentation. In that hook, it adds a class transformer that instruments in the stack frame size tracking. It also adds the agent to the boot class loader so that it can process jdk classes, too. To do that, we need to retransform anything that might be loaded already, like String.class. But, we have to exclude a variety of things that are used by the agent or the stack logging which lead to infinite loops or other problems (some of that was found by trial and error). Finally, the agent adds a shutdown hook to dump the results to stdout.
public class Transformer implements ClassFileTransformer {
#Override
public byte[] transform ( ClassLoader loader,
String className, Class<?> classBeingRedefined,
ProtectionDomain protectionDomain, byte[] classfileBuffer )
throws IllegalClassFormatException {
if ( className.startsWith ( "phil/agent" ) ) {
return classfileBuffer;
}
byte[] result = classfileBuffer;
ClassReader reader = new ClassReader ( classfileBuffer );
MaxStackClassVisitor maxCv = new MaxStackClassVisitor ( null );
reader.accept ( maxCv, ClassReader.SKIP_DEBUG );
ClassWriter writer = new ClassWriter ( ClassWriter.COMPUTE_FRAMES );
ClassVisitor visitor =
new CallStackClassVisitor ( writer, maxCv.frameMap, className );
reader.accept ( visitor, ClassReader.SKIP_DEBUG );
result = writer.toByteArray ();
return result;
}
}
The transformer drives two separate transformations - one to figure out the max stack frame size for each method and one to instrument the method for recording. It might be doable in a single pass but I didn't want to use the ASM tree API or spend more time figuring it out.
public class MaxStackClassVisitor extends ClassVisitor {
Map<String, Integer> frameMap = new HashMap<String, Integer> ();
public MaxStackClassVisitor ( ClassVisitor v ) {
super ( Opcodes.ASM4, v );
}
#Override
public MethodVisitor visitMethod ( int access, String name,
String desc, String signature,
String[] exceptions ) {
return new MaxStackMethodVisitor (
super.visitMethod ( access, name, desc, signature, exceptions ),
this, ( access + name + desc + signature ) );
}
}
public class MaxStackMethodVisitor extends MethodVisitor {
final MaxStackClassVisitor cv;
final String name;
public MaxStackMethodVisitor ( MethodVisitor mv,
MaxStackClassVisitor cv, String name ) {
super ( Opcodes.ASM4, mv );
this.cv = cv;
this.name = name;
}
#Override
public void visitMaxs ( int maxStack, int maxLocals ) {
cv.frameMap.put ( name, ( maxStack + maxLocals ) * 4 );
super.visitMaxs ( maxStack, maxLocals );
}
}
The MaxStack*Visitor classes handle figuring out the max stack frame size.
public class CallStackClassVisitor extends ClassVisitor {
final Map<String, Integer> frameSizes;
final String className;
public CallStackClassVisitor ( ClassVisitor v,
Map<String, Integer> frameSizes, String className ) {
super ( Opcodes.ASM4, v );
this.frameSizes = frameSizes;
this.className = className;
}
#Override
public MethodVisitor visitMethod ( int access, String name,
String desc, String signature, String[] exceptions ) {
MethodVisitor m = super.visitMethod ( access, name, desc,
signature, exceptions );
return new CallStackMethodVisitor ( m,
frameSizes.get ( access + name + desc + signature ) );
}
}
public class CallStackMethodVisitor extends MethodVisitor {
final int size;
public CallStackMethodVisitor ( MethodVisitor mv, int size ) {
super ( Opcodes.ASM4, mv );
this.size = size;
}
#Override
public void visitCode () {
visitIntInsn ( Opcodes.SIPUSH, size );
visitMethodInsn ( Opcodes.INVOKESTATIC, "phil/agent/MaxStackLog",
"enter", "(I)V" );
super.visitCode ();
}
#Override
public void visitInsn ( int inst ) {
switch ( inst ) {
case Opcodes.ARETURN:
case Opcodes.DRETURN:
case Opcodes.FRETURN:
case Opcodes.IRETURN:
case Opcodes.LRETURN:
case Opcodes.RETURN:
case Opcodes.ATHROW:
visitIntInsn ( Opcodes.SIPUSH, size );
visitMethodInsn ( Opcodes.INVOKESTATIC,
"phil/agent/MaxStackLog", "exit", "(I)V" );
break;
default:
break;
}
super.visitInsn ( inst );
}
}
The CallStack*Visitor classes handle instrumenting methods with code to call the stack frame logging.
And then you need a MANIFEST.MF for the Agent.jar:
Manifest-Version: 1.0
Premain-Class: phil.agent.Agent
Boot-Class-Path: asm-all-4.0.jar
Can-Retransform-Classes: true
Finally, add the following to your java command line for the program you want to instrument:
-javaagent:path/to/Agent.jar
You will also need to have the asm-all-4.0.jar in the same directory as the Agent.jar (or change Boot-Class-Path in the manifest to reference the location).
A sample output might be:
Max stack frame size accummulated: 44140
Max stack depth: 1004
This is all a bit crude but works for me to get going.
Note: the stack frame size isn't a total stack size (still don't really know how to get that one). In practice, there are a variety of overheads for the thread stack. I found that I usually needed between 2 and 3 times the reported stack max frame size as a -Xss value. Oh, and be sure to do the -Xss tuning without the agent loaded as it adds to your stack size requirements.
I would reduce the -Xss setting in a test environment until you see a problem. Then add some head room.
Reducing your heap size would give your application more space for thread stacks.
Just switching to a 64-bit OS could give your application more memory as most 32-bit OSes only allow about 1.5 GB for each application, however a 32-bit application on a 64-bit OS can use up to 3-3.5 GB depending on the OS.
There is no readily usable tooling in the Java VM to query the stack depth in bytes. But you can get there. Here are some pointers:
Exceptions contain arrays of stack frames which gives you the methods which were called.
For each method, you can find the Code attribute in the .class file. This attribute contains the frame size per method in the field max_stack.
So what you need is a tool that compiles a HashMap which contains method name + file name + line number as keys and the value max_stack as values. Create an Throwable, fetch the stack frames from it with getStackTrace() and then iterate over the StackTraceElements.
Note:
Each entry on the operand stack can hold a value of any Java virtual machine type, including a value of type long or type double.
So each stack entry is probably 64bits, so you need to multiply max_stack with 8 to get bytes.
Related
I have this code where every person is represented by a special Thread that is instantiated with random ticket data and every 5 seconds a FestivalStatisticsThread wakes up and reads the new data from the gate and generates statistics (how many people have enterd and with what type of ticket).
public class Main {
public static void main(String[] args) throws InterruptedException {
FestivalGate gate = new FestivalGate();
FestivalAttendeeThread festivalAttendee;
FestivalStatisticsThread statsThread = new FestivalStatisticsThread(gate);
TicketType ticketType;
for (int i = 1; i < 85_000; i++) {
ticketType = TicketType.randomTicket();
festivalAttendee = new FestivalAttendeeThread(ticketType, gate);
festivalAttendee.start();
festivalAttendee.join();
}
statsThread.start();
System.out.println(gate.getValidatedTickets().size());
}
}
The problem is that I did not find a way to wake up that thread every 5 second and generate statisitics, the only way I could do it is by generating the statistics only once at the end
The second problem is that I should use join after I start the festivalAttendee thread, otherwise the list where I store the ticket types doesn't store all of them, although is synchronized.
I will also leave here the code that does the inner workings of my code:
public class FestivalGate {
private List<TicketType> validatedTickets = Collections.synchronizedList(new ArrayList<>());
public List<TicketType> getValidatedTickets() {
return validatedTickets;
}
}
public class FestivalAttendeeThread extends Thread {
TicketType ticketType;
FestivalGate festivalGate;
public FestivalAttendeeThread(TicketType ticketType, FestivalGate festivalGate) {
this.ticketType = ticketType;
this.festivalGate = festivalGate;
}
#Override
public synchronized void run() {
this.festivalGate.getValidatedTickets().add(ticketType);
}
}
public class FestivalStatisticsThread extends Thread {
private int NR_OF_FULL;
private int NR_OF_FULL_VIP;
private int NR_OF_FREE_PASS;
private int NR_OF_ONE_DAY;
private int NR_OF_ONE_DAY_VIP;
private int TOTAL_NUMBER_OF_PEOPLE;
private FestivalGate gate;
public FestivalStatisticsThread(FestivalGate gate) {
this.gate = gate;
}
#Override
public void run() {
calculateNumberOfEachTicketType();
calculateNumberOfPeople();
showStats();
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
private void calculateNumberOfEachTicketType() {
synchronized (this.gate.getValidatedTickets()) {
for (TicketType ticketType : this.gate.getValidatedTickets()) {
if (ticketType.equals(TicketType.FULL)) {
NR_OF_FULL += 1;
} else if (ticketType.equals(TicketType.FULL_VIP)) {
NR_OF_FULL_VIP += 1;
} else if (ticketType.equals(TicketType.FREE_PASS)) {
NR_OF_FREE_PASS += 1;
} else if (ticketType.equals(TicketType.ONE_DAY)) {
NR_OF_ONE_DAY += 1;
} else if (ticketType.equals(TicketType.ONE_DAY_VIP)) {
NR_OF_ONE_DAY_VIP += 1;
}
}
}
}
private void calculateNumberOfPeople() {
TOTAL_NUMBER_OF_PEOPLE +=
NR_OF_FULL +
NR_OF_FULL_VIP +
NR_OF_FREE_PASS +
NR_OF_ONE_DAY +
NR_OF_ONE_DAY_VIP
;
}
public void showStats() {
System.out.println(
TOTAL_NUMBER_OF_PEOPLE + " people have entered " + "\n" +
NR_OF_FULL + " have full tickets " + "\n" +
NR_OF_FREE_PASS + " have free passes " + "\n" +
NR_OF_FULL_VIP + " have full vip passes " + "\n" +
NR_OF_ONE_DAY + " have one day passes " + "\n" +
NR_OF_ONE_DAY_VIP + " have one day vip passes"
);
System.out.println("-----------------------------------------------------------");
System.out.println("-----------------------------------------------------------");
}
}
To quote the Oracle tutorial:
t.join() causes the current thread to pause execution until t's thread terminates
So effectively you are not making any real use of threading. On every pass through your for loop, you launch a thread but then wait for it to complete. No point in doing that, you could forgo the threads and just sequentially do your tasks’ work within the current thread.
In modern Java, we rarely need to address the Thread class directly. Instead we submit our Runnable/Callable task to an ExecutorService.
To schedule repeated calculations every 5 seconds, use a ScheduledExecutorService.
ScheduledExecutorService ses = Executors. newSingleThreadScheduledExecutor() ;
…
ses.scheduleAtFixedRate( myStatsCalcRunnable , 1 , 5 , TimeUnit.SECONDS ) ;
Caution: Be sure to eventually shut down your executor services. Otherwise their backing thread pools may continue running indefinitely, like a zombie 🧟♂️.
Change this:
public class FestivalStatisticsThread extends Thread { … }
… with a run method into:
public class FestivalStatistics { … }
… with a recalculate method. Call that recalculate method from the Runnable task submitted to your scheduled executor service.
Or perhaps that functionality should just be a part of the Gate class. (I’m not sure. I’d have to think more on that.)
85,000 simultaneous threads may be too much of a burden on common hardware. Instead, start an executor service with a limited number of threads. Then submit your 85,000 tasks as Runnable/Callable objects. The executor service takes care of feeding those tasks to be executed on those limited number of threads.
In the future, if Project Loom succeeds, you will be able to use 85,000 (or even millions) virtual threads (also known as “fibers”) at a time. To learn more, see the more recent talks by Ron Pressler or other members of the Loom team. Experimental builds based on early-access Java 18 are available now. The Loom team seeks feedback.
All of these topics have been addressed already on Stack Overflow. Search to learn more. I myself have written multiple Answers with code examples, as have others.
By the way, I would make FestivalGate more encapsulated. The other classes should not have access to, or even know about, its internal use of a synchronized List. Add methods to collect and report on tickets.
Here is some quick code I wrote as a demonstration of topics discussed above.
Use at your own risk; I’ve not done any serious testing nor a thoughtful code review.
Perhaps I have oversimplified your domain problem. You could of course expand or rearrange as needed. But hopefully this shows clearly how to use executor service.
We have a basic Ticket class to represent each ticket. We use a record here to define that class. We nest the enum of ticket types.
package work.basil.festival;
import java.util.UUID;
public record Ticket( UUID id , Type type )
{
enum Type
{ FULL_VIP, FREE_PASS, ONE_DAY, ONE_DAY_VIP }
}
We have a Gate class to hold our domain logic. We nest a Statistics class to hold our reported summary of tickets taken so far.
package work.basil.festival;
import java.time.Duration;
import java.time.Instant;
import java.util.*;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.stream.Collectors;
public class Gate
{
final private Set < Ticket > ticketsTaken;
public Gate ( )
{
this.ticketsTaken = new HashSet <>();
}
synchronized public boolean takeTicket ( final Ticket ticket )
{
// Simulate hard work by sleeping some random amount of time.
int millis = ThreadLocalRandom.current().nextInt( 2 );
try { Thread.sleep( millis ); }catch ( InterruptedException e ) { e.printStackTrace(); }
return this.ticketsTaken.add( ticket );
}
synchronized public List < Ticket > ticketsTaken ( )
{
return List.copyOf( this.ticketsTaken ); // Returns unmodifiable list of the `Ticket` objects contained in our private member set.
}
record Statistics( Instant when , int countTicketsTaken , Map < Ticket.Type, Integer > countOfTicketsTakenByType )
{
}
synchronized public Statistics statistics ( )
{
int count = this.countTicketsTaken();
Map < Ticket.Type, Integer > map = this.reportCountOfTicketsTakenByType();
if ( count != map.values().stream().mapToInt( Integer :: intValue ).sum() ) { throw new IllegalStateException( "Counts do not match in Gate.Statistics. Error # 898e905f-9432-4195-a3e0-118bede2872d." ); }
return new Statistics( Instant.now() , count , map );
}
private int countTicketsTaken ( )
{
return this.ticketsTaken.size();
}
private Map < Ticket.Type, Integer > reportCountOfTicketsTakenByType ( )
{
// We use `AtomicInteger` here not for thread-safety, but for the convenience of its `incrementAndGet` method.
Map < Ticket.Type, AtomicInteger > map = new EnumMap < Ticket.Type, AtomicInteger >( Ticket.Type.class );
Arrays.stream( Ticket.Type.values() ).forEach( type -> map.put( type , new AtomicInteger( 0 ) ) ); // Initialize the map, so each ticket-type has an atomic integer set to zero.
this.ticketsTaken.stream().forEach( ticket -> map.get( ticket.type() ).incrementAndGet() );
// Convert the AtomicInteger values to simply `Integer` values.
Map < Ticket.Type, Integer > result = map.entrySet().stream().collect( Collectors.toMap( Map.Entry :: getKey , atomicInteger -> atomicInteger.getValue().intValue() ) );
return Map.copyOf( result ); // Return an unmodifiable map, as a good practice.
}
}
And we have an App class to drive a demonstration.
package work.basil.festival;
import java.util.ArrayList;
import java.util.List;
import java.util.UUID;
import java.util.concurrent.*;
import java.util.stream.IntStream;
/**
* Hello world!
*/
public class App
{
public static void main ( String[] args )
{
System.out.println( "Hello World!" );
App app = new App();
app.demo();
}
private void demo ( )
{
final List < Ticket > ticketsToTake = this.generateTicketsForFestival();
Gate gate = new Gate();
// Report every five seconds on the progress of our gate taking tickets.
ScheduledExecutorService reportingExecutorService = Executors.newSingleThreadScheduledExecutor();
reportingExecutorService.scheduleAtFixedRate( ( ) -> System.out.println( gate.statistics() ) , 0 , 5 , TimeUnit.SECONDS );
ExecutorService ticketTakingExecutorService = Executors.newFixedThreadPool( 7 );
for ( Ticket ticket : ticketsToTake )
{
ticketTakingExecutorService.submit( ( ) -> gate.takeTicket( ticket ) );
}
ticketTakingExecutorService.shutdown();
try { ticketTakingExecutorService.awaitTermination( 10 , TimeUnit.MINUTES ); } catch ( InterruptedException e ) { e.printStackTrace(); }
reportingExecutorService.shutdown();
try { reportingExecutorService.awaitTermination( 10 , TimeUnit.MINUTES ); } catch ( InterruptedException e ) { e.printStackTrace(); }
System.out.println( "« FIN » " + gate.statistics() );
}
private List < Ticket > generateTicketsForFestival ( )
{
List < Ticket > tickets = new ArrayList <>();
for ( int i = 0 ; i < 85_000 ; i++ )
{
tickets.add(
new Ticket(
UUID.randomUUID() , // Randomly generate a UUID, to identify uniquely each ticket.
Ticket.Type.values()[ ThreadLocalRandom.current().nextInt( Ticket.Type.values().length ) ] // Randomly pick one of the ticket types.
)
);
}
return List.copyOf( tickets );
}
}
When run on an 8-core M1 MacBook Pro.
Hello World!
Statistics[when=2021-11-14T02:28:52.746596Z, countTicketsTaken=0, countOfTicketsTakenByType={FREE_PASS=0, ONE_DAY=0, ONE_DAY_VIP=0, FULL_VIP=0}]
Statistics[when=2021-11-14T02:28:57.800514Z, countTicketsTaken=7517, countOfTicketsTakenByType={FREE_PASS=1862, ONE_DAY=1953, ONE_DAY_VIP=1889, FULL_VIP=1813}]
Statistics[when=2021-11-14T02:29:02.804886Z, countTicketsTaken=15128, countOfTicketsTakenByType={FREE_PASS=3791, ONE_DAY=3788, ONE_DAY_VIP=3775, FULL_VIP=3774}]
Statistics[when=2021-11-14T02:29:07.746712Z, countTicketsTaken=22819, countOfTicketsTakenByType={FREE_PASS=5764, ONE_DAY=5653, ONE_DAY_VIP=5703, FULL_VIP=5699}]
Statistics[when=2021-11-14T02:29:12.769943Z, countTicketsTaken=30577, countOfTicketsTakenByType={FREE_PASS=7687, ONE_DAY=7631, ONE_DAY_VIP=7641, FULL_VIP=7618}]
Statistics[when=2021-11-14T02:29:17.803627Z, countTicketsTaken=38146, countOfTicketsTakenByType={FREE_PASS=9553, ONE_DAY=9552, ONE_DAY_VIP=9554, FULL_VIP=9487}]
Statistics[when=2021-11-14T02:29:22.785355Z, countTicketsTaken=45896, countOfTicketsTakenByType={FREE_PASS=11455, ONE_DAY=11497, ONE_DAY_VIP=11499, FULL_VIP=11445}]
Statistics[when=2021-11-14T02:29:27.768809Z, countTicketsTaken=53563, countOfTicketsTakenByType={FREE_PASS=13448, ONE_DAY=13393, ONE_DAY_VIP=13386, FULL_VIP=13336}]
Statistics[when=2021-11-14T02:29:32.739398Z, countTicketsTaken=61189, countOfTicketsTakenByType={FREE_PASS=15358, ONE_DAY=15291, ONE_DAY_VIP=15310, FULL_VIP=15230}]
Statistics[when=2021-11-14T02:29:37.751764Z, countTicketsTaken=68758, countOfTicketsTakenByType={FREE_PASS=17214, ONE_DAY=17136, ONE_DAY_VIP=17226, FULL_VIP=17182}]
Statistics[when=2021-11-14T02:29:42.759303Z, countTicketsTaken=76446, countOfTicketsTakenByType={FREE_PASS=19136, ONE_DAY=19057, ONE_DAY_VIP=19171, FULL_VIP=19082}]
Statistics[when=2021-11-14T02:29:47.768858Z, countTicketsTaken=84030, countOfTicketsTakenByType={FREE_PASS=21086, ONE_DAY=20930, ONE_DAY_VIP=21062, FULL_VIP=20952}]
« FIN » Statistics[when=2021-11-14T02:29:48.406351Z, countTicketsTaken=85000, countOfTicketsTakenByType={FREE_PASS=21321, ONE_DAY=21174, ONE_DAY_VIP=21305, FULL_VIP=21200}]
I have implemented the GetExtendedTcpTable() with JNA, but when I use the function, I receive an error:
java.lang.IndexOutOfBoundsException: Bounds exceeds available space : size=28, offset=52
The definition of my function is as follows:
public interface IPHlpAPIExtended extends IPHlpAPI {
IPHlpAPIExtended INSTANCE = Native.load("IPHlpAPI", IPHlpAPIExtended.class, W32APIOptions.DEFAULT_OPTIONS);
int GetExtendedTcpTable( MIB_TCPTABLE_OWNER_PID pTcpTable, IntByReference pdwSize, boolean bOrder, int ulAf, int TableClass, int Reserved );
#Structure.FieldOrder( { "dwNumEntries", "table" } )
public static class MIB_TCPTABLE_OWNER_PID extends Structure {
public WinDef.DWORD dwNumEntries;
public MIB_TCPROW_OWNER_PID[] table = new MIB_TCPROW_OWNER_PID[]{ new MIB_TCPROW_OWNER_PID() };
public MIB_TCPTABLE_OWNER_PID() {
}
public MIB_TCPTABLE_OWNER_PID( Pointer pointer ) {
super( pointer );
this.read();
}
#Override
public void read() {
super.read();
if ( dwNumEntries.intValue() > 0 ) {
table = ( MIB_TCPROW_OWNER_PID[] ) table[0].toArray( dwNumEntries.intValue() );
} else {
table = new MIB_TCPROW_OWNER_PID[]{ new MIB_TCPROW_OWNER_PID() };
}
}
}
}
And:
IPHlpAPIExtended.MIB_TCPTABLE_OWNER_PID pTcpTable = new IPHlpAPIExtended.MIB_TCPTABLE_OWNER_PID();
final IntByReference pdwSize = new IntByReference( 0 );
try {
if ( IPHlpAPIExtended.INSTANCE.GetExtendedTcpTable( pTcpTable, pdwSize, true, AF_INET,
IPHlpAPIExtended.TCP_TABLE_CLASS.TCP_TABLE_OWNER_PID_ALL.ordinal(), 0 ) == ERROR_INSUFFICIENT_BUFFER ) {
IPHlpAPIExtended.MIB_TCPTABLE_OWNER_PID newPTcpTable = new IPHlpAPIExtended.MIB_TCPTABLE_OWNER_PID( pTcpTable.getPointer() );
assertThat(
IPHlpAPIExtended.INSTANCE.GetExtendedTcpTable( newPTcpTable, pdwSize, true, AF_INET,
IPHlpAPIExtended.TCP_TABLE_CLASS.TCP_TABLE_OWNER_PID_ALL.ordinal(), 0 ) ).isEqualTo( NO_ERROR );
for ( int i = 0; i < newPTcpTable.dwNumEntries.intValue(); i++ ) {
final IPHlpAPIExtended.MIB_TCPROW_OWNER_PID row = newPTcpTable.table[i];
System.out.println( row.dwOwningPid.intValue() );
}
} else {
// TODO getLasError()
Assertions.fail( "GetExtendedTcpTable terminate with errors" );
}
} catch ( Throwable t ) {
t.printStackTrace();
}
The dwNumEntries field is correctly populated. But the row:
table = ( MIB_TCPROW_OWNER_PID[] ) table[0].toArray( dwNumEntries.intValue() );
Raises the exception.
EDIT:
This is the mapping for MIB_TCPROW_OWNER_PID :
#Structure.FieldOrder( { "dwState", "dwLocalAddr", "dwLocalPort", "dwRemoteAddr", "dwRemotePort", "dwOwningPid" } )
public static class MIB_TCPROW_OWNER_PID extends Structure {
public WinDef.DWORD dwState;
public WinDef.DWORD dwLocalAddr;
public WinDef.DWORD dwLocalPort;
public WinDef.DWORD dwRemoteAddr;
public WinDef.DWORD dwRemotePort;
public WinDef.DWORD dwOwningPid;
}
EDIT
My solution has been changed to the signature of the function.
int GetExtendedTcpTable( Memory pTcpTable, IntByReference pdwSize, boolean bOrder, int ulAf, int TableClass, int Reserved );
Now pTcpTable it's a more generic Memory parameter, and the code is now the following:
final IPHlpAPIExtended.MIB_TCPTABLE_OWNER_PID pTcpTable = new IPHlpAPIExtended.MIB_TCPTABLE_OWNER_PID();
final IntByReference pdwSize = new IntByReference( 0 );
final Memory mTcpTable = new Memory( pTcpTable.size() );
if ( IPHlpAPIExtended.INSTANCE.GetExtendedTcpTable( mTcpTable, pdwSize, true, AF_INET,
IPHlpAPIExtended.TCP_TABLE_CLASS.TCP_TABLE_OWNER_PID_ALL.ordinal(), 0 ) == ERROR_INSUFFICIENT_BUFFER ) {
final Memory newMTcpTable = new Memory( pdwSize.getValue() );
assertThat( IPHlpAPIExtended.INSTANCE.GetExtendedTcpTable( newMTcpTable, pdwSize, true, AF_INET,
IPHlpAPIExtended.TCP_TABLE_CLASS.TCP_TABLE_OWNER_PID_ALL.ordinal(), 0 ) ).isEqualTo( NO_ERROR );
final IPHlpAPIExtended.MIB_TCPTABLE_OWNER_PID newPTcpTable = new IPHlpAPIExtended.MIB_TCPTABLE_OWNER_PID( newMTcpTable );
System.out.println( "Table Size: " + newPTcpTable.dwNumEntries.intValue() );
for (int i = 0; i < newPTcpTable.dwNumEntries.intValue(); i++) {
final IPHlpAPIExtended.MIB_TCPROW_OWNER_PID item = newPTcpTable.table[i];
System.out.println( "PID: " + item.dwOwningPid.longValue());
}
}
The problem is that you're using Structure#toArray in a way it was not designed. It's intended to work on memory already alloated for the full size of the array (or not allocated at all, in which case it allocates).
But you are using it in a case where, in your mapping of the MIB_TCPTABLE_OWNER_PID structure, you have defined the second element as an array, with a fixed size of 1, and thus allocated memory for it.
public MIB_TCPROW_OWNER_PID[] table = new MIB_TCPROW_OWNER_PID[]{ new MIB_TCPROW_OWNER_PID() };
This isn't wrong in itself, and you have to declare something there, although there's no need to initialize the element with a Java object. Just assign new MIB_TCPROW_OWNER_PID[1].
JNA allocates memory for the size of a structure immediately on assignment, so the result of the way you assigned it is that the table element already is mapped to native memory with a size of exactly 28 (4 bytes for the DWORD and 4*6 bytes for the single MIB_TCPROW_OWNER_PID element.)
This initial allocation of the single element to table works fine, but when you attempt to reassign that variable using Structure#toArray it fails the memory bounds check because the smaller memory has been allocated, and toArray() checks if memory has been allocated before creating new memory.
It may be possible to reassign the variable defining your own array in plain Java, without using Jna's Structure#toArray() and its bounds checks. This code should work in some circumstances if the native side allocates the memory; however, if you are required to allocate memory yourself, it won't work.
#Override
public void read() {
readField("dwNumEntries");
table = new MIB_TCPROW_OWNER_PID[dwNumEntries.intValue()];
super.read();
}
Another approach is to not bother redefining the array (table) but instead map table as a Pointer or a StructureByReference (a more strongly typed pointer). Then instead of the read() override, you would access the native memory pointed to, and create the array on the fly. You could use a getter function like:
public MIB_TCPROW_OWNER_PID[] getTable() {
Pointer[] array = table.getPointerArray(0, dwNumEntries.intValue());
MIB_TCPROW_OWNER_PID[] rows = new MIB_TCPROW_OWNER_PID[array.length];
for (int i=0; i < rows.length; i++) {
rows[i] = new MIB_TCPROW_OWNER_PID(array[i]);
}
return rows;
}
Finally as you have discovered, if you allocate the memory yourself using a Memory object, and return that, you can then pass the pre-allocated memory to the constructor and your code should work.
I'm working on some homework programs and I'm required to do stuff in Java using functional programming principles as much as possible. These are the important bits of a program that receives a list of numbers and prints the even ones:
public static void main( String args[ ] )
{
ArrayList<Double> listEven = new ArrayList<Double>();
inputRecursion( );
outputRecursion( );
}
public static void inputRecursion( )
{
Scanner in = new Scanner( System.in );
if( in.hasNextDouble() )
{
if( (in.nextDouble() % 2) == 0 )
{
listEven.add( in.nextDouble() );
}
inputRecursion( );
}
}
public static void outputRecursion( )
{
Iterator<Double> it = listEven.iterator();
if( it.hasNext() )
{
System.out.println( it.next() );
outputRecursion( );
}
}
It's a work in progress, but I haven't got to run it and check the logic because I still have two compilation errors in the lines:
listEven.add( in.nextDouble() );
Iterator<Double> it = listEven.iterator();
These two lines throw "cannot find symbol", and I know this is because the ArrayList was declared in a way that makes in inaccessible from methods outside main. I have a rough understanding of the ways of fixing this. I know about setters and getters and they look simple enough from what I researched, but I consider those the last resource since I'm trying to avoid the use of variables (I am aware that my program has other mutable stuff; I'm working on it) in order to meet the restrictions of the exercise as much a possible. I also know that I can make it public static, but then that causes like 15 more errors to appear, and from what I looked up it requires to be "initialized" and it also involves variables.
Are there any other ways to make the ArrayList accessible from those two methods? I'm specially interested in ways that don't involve variables or iteration.
Declare the ArrayList as static but outside of the main function like below:
static ArrayList<Double> listEven = new ArrayList<Double>();
public static void main( String args[ ] )
{
inputRecursion( );
outputRecursion( );
}
public static void inputRecursion( )
{
Scanner in = new Scanner( System.in );
if( in.hasNextDouble() )
{
if( (in.nextDouble() % 2) == 0 )
{
listEven.add( in.nextDouble() );
}
inputRecursion( );
}
}
public static void outputRecursion( )
{
Iterator<Double> it = listEven.iterator();
if( it.hasNext() )
{
System.out.println( it.next() );
outputRecursion( );
}
}
NOTE: I have looked at the other posts, but I'm still quite lost.
This is the code for a private variable that I have in one class:
private int readFile( String fileName)
{
try
{
File f = new File( fileName );
Scanner input = new Scanner( f );
while( input.hasNextLine( ) )
{
String s = input.nextLine( );
String[ ] sArr = s.split( " " );
String animal = sArr[ 0 ];
double cost = Double.parseDouble(sArr [ 1 ] );
boolean penNeeded = Boolean.parseBoolean( sArr[ 2 ] );
boolean available = Boolean.parseBoolean( sArr[ 3 ] );
Pet p = new Pet( animal, cost, penNeeded, available );
if (count < animalList.length )
{
animalList[count] = p;
count++;
}
}
input.close( );
}
catch( Exception e )
{
System.out.println("Error reading the file:");
System.out.println( e );
e.printStackTrace( );
}
return count;
}
I need to access it in this piece of code located in another class:
static public void processTransaction( String fileName, PettingZoo pz )
{
try
{
// variable should be accessed here
}
catch( Exception e )
{
System.out.println("Error reading the file:");
System.out.println( e );
e.printStackTrace( );
}
}
How can I do this? I think that I need to use a modifier of some sort, but I don't know how which one or how to implement it.
You cannot directly access private variables from another class. That's the whole point of declaring it private. What you need to do is use setter and getter methods in class A, then call the get method from class B.
If you want access to a private variable, you can use getter and setter methods.
Example:
private int variable = 5; //<--- your private variable of class A
// a public method (into the same class A)
// that allows the sharing of your private variable
public int getVariable() {
return variable;
}
Now you can call the method getVariable() from an other class (B) and take the value of the private variable (of class A).
As per your comment you can access the private int readFile(String fileName) method by changing the modified of the method. Change the modifier of the method to public or protected. Also since the accessing method is static you need to change the method as static.
So change it as
public static int readFile( String fileName)
{
}
In the processTransaction method invoke it as,
ClassName.readFile("file_name.extn");
What if I'll use switch in getByIntValue()? Is it really neccessary to use a SparseArray?
public enum Gender {
Unknown(0),
Male(1),
Female(2);
private static final SparseArray<Gender> lookupTable = new SparseArray<Gender>();
static {
for (final Gender gender : EnumSet.allOf(Gender.class)) {
lookupTable.put(gender.intValue, gender);
}
}
private final int intValue;
public static Gender getByIntValue(int val) {
return lookupTable.get(val);
}
private Gender(int intValue) {
this.intValue = intValue;
}
public int getIntValue() {
return intValue;
}
}
Since your int values go from 0 to 2, without hole, you could indeed simply use an array. A switch would also be fine, although it would probably be slightly slower than an array lookup. But unless you call the method billions of times, it won't make any noticeable difference. Use what you find the clearest and easiest to understand and maintain.
If you have posted realistic int values, then you don't need to set them explicitly on each enum member, and don't need switch. Just use
Gender.values()[intValue]
List.copyOf( EnumSet.allOf( Gender.class ) )
Caveat: This exercise in optimization seems silly for all but the most extreme scenario, as mentioned by JB Nizet. For real work, I would probably recommend the solution seen in the Answer by Marko Topolnik. But, for fun, I swung a bat at this ball.
Seems the goal is to render a static unmodifiable collection with very fast access by the given numbers 0, 1, 2.
As of Java 10, we have these new implemented (“default”) methods on the List interface: List.of & List.copyOf. These produce an unmodifiable collection. Though the backing implementation is undocumented and subject to change, I will assume it is something akin to an array with similar performance. Performance might even be faster than a conventional array, if the backing implementation detected the presence of an EnumSet and used some kind of bit vector.
I populate the List by passing an EnumSet to List.copyOf( Collection ).
So, this:
private static final SparseArray<Gender> lookupTable = new SparseArray<Gender>();
static {
for (final Gender gender : EnumSet.allOf(Gender.class)) {
lookupTable.put(gender.intValue, gender);
}
}
…becomes this:
private static final List < Gender > lookupTable = List.copyOf( EnumSet.allOf( Gender.class ) );
Entire class, with main for demo.
package com.basilbourque.example;
import java.util.EnumSet;
import java.util.List;
public enum Gender {
UNKNOWN( 0 ),
MALE( 1 ),
FEMALE( 2 );
private static final List < Gender > lookupTable = List.copyOf( EnumSet.allOf( Gender.class ) );
private final int intValue;
public static Gender getByIntValue ( int val ) {
return lookupTable.get( val );
}
public int getIntValue () {
return intValue;
}
// Constructor
private Gender ( int intValue ) {
this.intValue = intValue;
}
public static void main ( String[] args ) {
// Testing.
System.out.println( Gender.UNKNOWN.intValue );
System.out.println( Gender.getByIntValue( 0 ) );
System.out.println( "----" );
System.out.println( Gender.MALE.intValue );
System.out.println( Gender.getByIntValue( 1 ) );
System.out.println( "----" );
System.out.println( Gender.FEMALE.intValue );
System.out.println( Gender.getByIntValue( 2 ) );
}
}
When run.
0
UNKNOWN
1
MALE
2
FEMALE
By the way, as the biological default, FEMALE should come before MALE.