I created a class which is used as a cache provider. It uses a Map, timestamped map entries and it spawns a Thread which performs cleanup every so often. This class is used in a web application. This web application had a problem where POST would take 30 seconds. I traced the problem to this cache class, eliminating it resolves the problem.
I have tried my best to find the error in this class but I can't. Please help me out here.
Assume User class is some kind of POJO describing the user.
public class UserStore implements Thread.UncaughtExceptionHandler {
private static volatile UserStore instance;
private static Thread cleanUpThread;
private static Map<String, TimeStampedToken<User>> tokenMap = new HashMap<String, TimeStampedToken<User>>();
public static UserStore getInstance() {
if (instance == null) {
synchronized(UserStore.class) {
if (instance == null) {
instance = new UserStore();
cleanUpThread = new Thread(new CleanUpWorker());
cleanUpThread.setUncaughtExceptionHandler(instance);
cleanUpThread.start();
}
}
}
return instance;
}
public void uncaughtException(Thread thread, Throwable throwable) {
if (throwable instanceof ThreadDeath) {
cleanUpThread = new Thread(new CleanUpWorker());
cleanUpThread.setUncaughtExceptionHandler(this);
cleanUpThread.start();
throw (ThreadDeath)throwable;
}
}
private static class CleanUpWorker implements Runnable {
private static final long CLEANUP_CYCLE_MS = 300000;
private static final long OBJECT_LIVE_TIME = 299900;
public void run() {
long sleepRemaining;
long sleepStart = System.currentTimeMillis();
sleepRemaining = CLEANUP_CYCLE_MS;
while (true) {
try {
sleepStart = System.currentTimeMillis();
Thread.sleep(sleepRemaining);
cleanUp();
sleepRemaining = CLEANUP_CYCLE_MS;
} catch (InterruptedException e) {
sleepRemaining = System.currentTimeMillis() - sleepStart;
}
}
}
private void cleanUp() {
Long currentTime = System.currentTimeMillis();
synchronized(tokenMap) {
for (String user : tokenMap.keySet()) {
TimeStampedToken<User> tok = tokenMap.get(user);
if (tok.accessed + OBJECT_LIVE_TIME < currentTime) {
tokenMap.remove(user);
}
}
}
}
}
public void addToken(User tok) {
synchronized(tokenMap) {
tokenMap.put(tok.getUserId(), new TimeStampedToken<User>(tok));
}
}
public User getToken(String userId) {
synchronized(tokenMap) {
TimeStampedToken<User> user = tokenMap.get(userId);
if (user != null) {
user.accessed = System.currentTimeMillis();
return user.payload;
} else {
return null;
}
}
}
private static class TimeStampedToken<E> {
public TimeStampedToken(E payload) {
this.payload = payload;
}
public long accessed = System.currentTimeMillis();
public E payload;
}
}
Here is how I would approach it. With multi-threaded code, simplicity is often the best approach as its more likely to work.
(the third parameter trueof the LinkedHashMap means that iterators over this Map follow the order of access rather than order of insertion)
public enum UserStore {
;
interface User {
String getUserId();
}
// a LRU cache with a timestamp.
private static final Map<String, TimeStampedToken<User>> tokenMap = new LinkedHashMap<String, TimeStampedToken<User>>(16, 0.7f, true);
private static final long OBJECT_LIVE_TIME = 299900;
public static synchronized void addToken(User tok) {
final long now = System.currentTimeMillis();
// clean up as we go
for (Iterator<Map.Entry<String, TimeStampedToken<User>>> iter = tokenMap.entrySet().iterator(); iter.hasNext(); ) {
final Map.Entry<String, TimeStampedToken<User>> next = iter.next();
if (next.getValue().accessed + OBJECT_LIVE_TIME >= now)
// the map is ordered by access time so there are no more to clean up.
break;
iter.remove();
}
// add a new entry
tokenMap.put(tok.getUserId(), new TimeStampedToken<User>(tok, now));
}
public static synchronized User getToken(String userId) {
final long now = System.currentTimeMillis();
TimeStampedToken<User> user = tokenMap.get(userId);
if (user == null)
return null;
user.accessed = now;
return user.payload;
}
static class TimeStampedToken<E> {
long accessed;
final E payload;
TimeStampedToken(E payload, long now) {
this.payload = payload;
accessed = now;
}
}
}
This line looks weird to me...
sleepRemaining = System.currentTimeMillis() - sleepStart;
...surely it should be...
sleepRemaining = CLEANUP_CYCLE_MS - (System.currentTimeMillis() - sleepStart);
Related
public class FlinkWindowTest {
public static long timestamp = 1496301598L;
public static void main(String[] args) throws Exception {
// get the execution environment
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
// get input data by connecting to the socket
SourceFunction<String> out = new OutSource();
DataStream<String> text = env.addSource(out);
// parse the data
DataStream<WordWithCount> windowCounts = text
.flatMap(new FlatMapFunction<String, WordWithCount>() {
public void flatMap(String value, Collector<WordWithCount> out) {
for (String word : value.split(" ")) {
out.collect(new WordWithCount(word, 1L));
}
}
});
//assign timestamp
windowCounts = windowCounts.assignTimestampsAndWatermarks(new MyTimestampExtractor(Time.seconds(0)));
windowCounts.keyBy(new MyKeySelector())
.join(windowCounts)
.where(new MyKeySelector()).equalTo(new MyKeySelector())
.window(TumblingEventTimeWindows.of(Time.seconds(10)))
.apply(new JoinFunction<WordWithCount, WordWithCount, Object>() {
public Object join(WordWithCount wordWithCount, WordWithCount wordWithCount2) throws Exception {
System.out.println("start join");
System.out.println(wordWithCount.toString());
System.out.println(wordWithCount2.toString());
WordWithCount wordWithCount3 = new WordWithCount(wordWithCount.word, wordWithCount.count + wordWithCount2.count);
System.out.println(wordWithCount3.toString());
return wordWithCount3;
}
});
env.execute("Window WordCount");
}
public static class MyKeySelector implements KeySelector<WordWithCount, String> {
public String getKey (WordWithCount wordWithCount) throws Exception {
return wordWithCount.word;
}
}
public static class MyTimestampExtractor extends BoundedOutOfOrdernessTimestampExtractor<WordWithCount> {
public MyTimestampExtractor(Time maxOutOfOrderness) {
super(maxOutOfOrderness);
}
public long extractTimestamp(WordWithCount wordWithCount) {
return wordWithCount.getTimeStamp();
}
}
public static class OutSource implements SourceFunction<String> {
private String[] str = {
"aa ff","bb gg","cc hh","dd kk"
};
public void run(SourceContext<String> sourceContext) throws Exception {
int index =0;
while (true) {
if(index == str.length)
index = 0;
sourceContext.collect(str[index]);
index++;
}
}
public void cancel() {
}
}
// Data type for words with count and timestamp
public static class WordWithCount {
public String word;
public long count;
public WordWithCount() {}
public long getTimeStamp() {
return timestamp;
}
public WordWithCount(String word, long count) {
this.word = word;
this.count = count;
++timestamp;
}
#Override
public String toString() {
return word + " : " + count;
}
}
}
This class is a demo. I create a SourceFunction to emit strings, then cut them to words. Finally I use join operation to join the stream itself. I don't care the count result.
The question is that there is no output in my JoinFunction class. I think the output should be
start join
aa : 1
aa : 1
aa : 2
start join
........
but now there is no output, because elements are in the window and not emitted to the join function.
I don't have ideas about this situation. If there is anyone have advice, please tell me here. I expect replies by all.
:)
You forgot to the set the time characteristic to event time:
env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
public class X {
private long timeInactive = 0;
private final long timeStarted = System.currentTimeMillis();
private enum State {
ACTIVE,INACTIVE
};
private State getState() {
if(Math.random <= 0.1) {
return State.ACTIVE;
}
return State.INACTIVE;
}
public static void main(String[] args) {
//code
}
}
Let's pretend I had something like this. How would I input a listener that would time how long the state was inactive before once again becoming active, and do this every single time it went back to being inactive?
Edit: I need to know when getState() has changed value after being changed by random or external factors. For instance, in psvm, I might have something like:
public static void main(String[] args) {
while(true) {
if(value of getState() has changed from inactive to active) {
System.out.println(How long it was inactive prior);
}
}
}
Thanks!
keep the time inside X without Listner
public class X {
private long timeInactive = 0; // total time of being inactive
private long timeLastChanged = System.currentTimeMillis();
private State currentState = State.INACTIVE;
enum State {
ACTIVE, INACTIVE
}
public State getState() {
changeState(); // It shouldn't be called here. I just need to simulate the state changing inside randomly
return currentState;
}
private void changeState() {
State oldState = currentState;
if (Math.random() <= 0.1) {
currentState = State.ACTIVE;
} else {
currentState = State.INACTIVE;
}
long now = System.currentTimeMillis();
if (oldState == State.INACTIVE && currentState == State.ACTIVE) {
long elapse = now - timeLastChanged;
timeInactive += elapse;
}
if (oldState != currentState) {
timeLastChanged = now;
}
}
public long getTotalTimeInActive() {
return timeInactive;
}
public static void main(String[] args) throws InterruptedException {
X x = new X();
while (true) {
x.getState();
System.out.println(x.getTotalTimeInActive());
Thread.sleep(1000);
}
}
}
With a listener:
//Listner.java
interface Listener {
void onStateChange(X2.State currentState);
}
//X2.java
import java.util.ArrayList;
import java.util.List;
public class X2 {
private State currentState = State.INACTIVE;
private List<Listener> listeners = new ArrayList<>();
enum State {
ACTIVE, INACTIVE
}
public State getState() {
changeState(); // It shouldn't be called here. I just need to simulate the state changing inside randomly
return currentState;
}
private void changeState() {
if (Math.random() <= 0.1) {
currentState = State.ACTIVE;
} else {
currentState = State.INACTIVE;
}
for(Listener listener:listeners){
listener.onStateChange(currentState);
}
}
public void registerListener(Listener listener){
listeners.add(listener);
}
public static void main(String[] args) throws InterruptedException {
X2 x = new X2();
TimeInactiveListener timeInactiveListener = new TimeInactiveListener();
x.registerListener(timeInactiveListner);
while (true) {
x.getState();
System.out.println(timeInactiveListner.getTimeInactive());
Thread.sleep(1000);
}
}
}
class TimeInactiveListener implements Listener {
private long timeLastChanged = System.currentTimeMillis();
private X2.State oldState = X2.State.INACTIVE;
private long timeInactive = 0; // total time of being inactive
#Override
public void onStateChange(X2.State currentState) {
long now = System.currentTimeMillis();
if (oldState == X2.State.INACTIVE && currentState == X2.State.ACTIVE) {
long elapse = now - timeLastChanged;
timeInactive += elapse;
}
if (oldState != currentState) {
timeLastChanged = now;
}
oldState = currentState;
}
public long getTimeInactive() {
return timeInactive;
}
}
If you wanted to know when it changed states you would have to know what the last state was so you have something to compare it to. And then just check if the new state is different then the old state, and if so then record the time inactive.
private State state;
public void setState(State newstate){
if (!newstate.equals(state)){
if (state.equals(State.INACTIVE)){
timeInactive = 0;
timeStarted = System.currentTimeMillis();
}
else if (state.equals(State.ACTIVE)){
timeInactive = System.currentTimeMillis()-timeStarted;
}
}
state = newstate;
}
Also to do this you would need to change timeStarted to not be final.
At the moment you have nothing in your code that records the current state. You are defining a private enum but that is nothing more than declaring that a state can have two values: it doesn't actually record the state. To do that you'd need a field with state defined and the last time in changed state. Also note that by making it private it can only be used internally in the class. That might work in this situation (because main is part of the same class) but will generally not be what you want.
Ideally you should encapsulate that logic in a State class rather than have it in the class using the state.
public class State {
public enum Value { ACTIVE, INACTIVE };
private Value value = Value.INACTIVE;
private long inactiveTime = System.currentTimeMillis();
public Value getValue() {
Value newValue = Math.random() < 0.1 ? Value.ACTIVE : Value.INACTIVE;
if (newValue != value) {
if (newValue == Value.ACTIVE)
// notify listener with inactive period current - inactiveTime
else
inactiveTime = System.currentTimeMillis();
value = newValue;
}
return value;
}
}
I am trying to make use of Java 8 and streams and one of the things I am trying to replace is a system we have where we
Use an aspect to measure call latency (per config period of time) to out webservices and then
Feed those results into a Complex Event Processor (esper) so that
We can send out alert notifications
So, one step at a time. For the first step, I need to produce a stream (I think) that allows me to feed those latency numbers into existing listeners. Understanding that, getting the next number in series might have to wait until there is a call.
How can I do that? Here is the latency aspect with comments.
public class ProfilingAspect {
private ProfilingAction action;
public ProfilingAspect(ProfilingAction action) {
this.action = action;
}
public Object doAroundAdvice(ProceedingJoinPoint jp) throws Throwable{
long startTime = System.currentTimeMillis();
Object retVal = null;
Throwable error = null;
try{
retVal = jp.proceed();
}catch (Throwable t){
error = t;
}
Class withinType = jp.getSourceLocation().getWithinType();
String methodName = jp.getSignature().getName();
long endTime = System.currentTimeMillis();
long runningTime = endTime - startTime;
// Let the IntStream know we have a new latency. Or really, we have an object
// stream with all this extra data
action.perform(withinType, methodName, jp.getArgs(), runningTime, error);
if( error != null ){
throw error;
}
return retVal;
}
}
Ok, I have a working example. It doesn't handle the situation where I have to buffer up results though is the stream isn't being read fast enough. I am open to some improvement
public class LatencySupplier implements Supplier<SomeFancyObject> {
private Random r = new Random();
#Override
public SomeFancyObject get() {
try {
Thread.sleep(100 + r.nextInt(1000));
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
return new SomeFancyObject(10 + r.nextInt(1000));
}
}
public class SomeFancyObject {
private static String[] someGroups = {"Group1","Group2","Group3"};
private final String group;
private int value;
public SomeFancyObject(int value) {
this.value = value;
this.group = WSRandom.selectOne(someGroups);
}
public String getGroup() {
return group;
}
public int getValue() {
return value;
}
#Override
public String toString() {
return value + "";
}
}
My next step is to create a stream by time so I can do avg/5 min, etc.
public class Sample {
public static void main(String[] args) throws InterruptedException {
Stream<SomeFancyObject> latencyStream = Stream.generate(new LatencySupplier());
Map<Object,List<SomeFancyObject>> collect = latencyStream.limit(10).collect(Collectors.groupingBy(sfo -> sfo.getGroup()));
System.out.println(collect);
Object o = new Object();
synchronized (o){
o.wait();
}
}
}
I wrote time conversion program(i.e.,seconds to minute and minute seconds etc), but later I found that these classes perform similar operation.Is there any way to relate these classes, if so please give some solution and suggestion. Here is my code....
Second.java
import java.util.concurrent.*;
public class Second {
private long secondsValue;
public Second() {
secondsValue = 0L;
}
public Second(String from, String to, long unitValue) {
unitSelection(from, to, unitValue);
}
private void convertSecondToMinute(long unitValue) {
unitValue = TimeUnit.MINUTES.convert(unitValue, TimeUnit.SECONDS);
secondsValue = unitValue;
}
private void convertSecondToHour(long unitValue) {
unitValue = TimeUnit.HOURS.convert(unitValue, TimeUnit.SECONDS);
secondsValue = unitValue;
}
private void convertSecondToDay(long unitValue) {
unitValue = TimeUnit.DAYS.convert(unitValue, TimeUnit.SECONDS);
secondsValue = unitValue;
}
private void convertSecondToWeek(long unitValue) {
unitValue = unitValue/60/60/24/7;
secondsValue = unitValue;
}
public long getSeconds() {
return secondsValue;
}
private void unitSelection(String from, String to,
long unitValue) {
if( from.equalsIgnoreCase("second")) {
if(to.equalsIgnoreCase("minute")) {
convertSecondToMinute(unitValue);
}
else if(to.equalsIgnoreCase("hour")) {
convertSecondToHour(unitValue);
}
else if(to.equalsIgnoreCase("day")) {
convertSecondToDay(unitValue);
}
else if(to.equalsIgnoreCase("week") ) {
convertSecondToWeek(unitValue);
}
else {
System.out.println("Invalid argument...!");
}
}
}
}
Minute.java
import java.util.concurrent.TimeUnit;
public class Minute {
private long unitMinute;
public Minute() {
unitMinute = 0L;
}
public Minute(String from, String to,
long unitValue) {
unitSelection(from, to, unitValue);
}
private void convertMinuteToSecond(long unitValue) {
unitValue = TimeUnit.SECONDS.convert(unitValue, TimeUnit.MINUTES);
unitMinute = unitValue;
}
private void convertMinuteToHour(long unitValue) {
unitValue = TimeUnit.HOURS.convert(unitValue, TimeUnit.MINUTES);
unitMinute = unitValue;
}
private void convertMinuteToDay(long unitValue) {
unitValue = TimeUnit.DAYS.convert(unitValue, TimeUnit.MINUTES);
unitMinute = unitValue;
}
private void convertMinuteToWeek(long unitValue) {
long value = unitValue /60/24/7;
unitMinute = value;
}
public long getUnitMinute() {
return unitMinute;
}
private void unitSelection(String from, String to,
long unitValue) {
if( from.equalsIgnoreCase("minute")) {
if(to.equalsIgnoreCase("second")) {
convertMinuteToSecond(unitValue);
}
else if(to.equalsIgnoreCase("hour")) {
convertMinuteToHour(unitValue);
}
else if(to.equalsIgnoreCase("day")) {
convertMinuteToDay(unitValue);
}
else if(to.equalsIgnoreCase("week") ) {
convertMinuteToWeek(unitValue);
}
else {
System.out.println("Invalid argument...!");
}
}
}
}
The best way to improve your code is to delete all of your classes and use TimeUnit instead.
TimeUnit has all this functionality (and more) and comes with the JDK.
Don't reinvent the wheel.
When you say "related" is very wide topic. However, the right way to start is using java Packaging mechanism to relate classes at very highest level. You can manage your code (if possible) using Inheritance and Polymorphism . As #Bohemian suggested , use existing java libraries until or unless your are doing sth new and doing it for learning purpose.
I see 2 options:
1. Re-define both as a single class: which suggest the use of a universal value standard (number of milliseconds for example), and to offer what it means in different measures like minutes, hours, ..etc
class Duration
{
private long secondsValue;
public Duration() {
secondsValue = 0L;
}
public Duration(String format, long value) {
if (format.equals("SECONDS")) secondsValue = value;
else if (format.equals("MINUTES")) secondsValue = value*60;
else if (format.equals("HOURS")) secondsValue = value*60*60;
else if (format.equals("DAYS")) secondsValue = value*60*60*24;
else if (format.equals("WEEKS")) secondsValue = value*60*60*24*7;
}
public long asMinutes() {
return value/60;
}
public long asHours() {
return value/(60*60);
}
public long asDays() {
return value/(60*60*24);
}
public long asWeeks() {
return unitValue/(60*60*24*7);
}
public long asSeconds() {
return secondsValue;
}
}
2. use a converter utility class: like how you use TimeUnit.MINUTES.convert(unitValue, TimeUnit.SECONDS); to convert from seconds to minutes
I have been trying to verify if the ThreadLocal members are indeed different in different threads.
This is my TestClass whose object I am sharing among multiple threads.
public class TestClass {
private static Set<Integer> setI;
private static ThreadLocal<Set<String>> setS;
public TestClass() {
Set<String> temp = new HashSet<String>();
for (int i=0; i<=4; i++) {
setI.add(i);
temp.add(Integer.toString(i));
}
setS.set(temp);
}
static {
setI = new HashSet<Integer>();
setS = new ThreadLocal<Set<String>>() {
protected Set<String> initialValue() {
return new HashSet<String>();
}
};
}
public static void addToIntegerSet(int i) {
synchronized(setI) {
setI.add(i);
}
}
public static void addToStringSet(String str) {
Set<String> sets = setS.get();
sets.add(str);
setS.set(sets);
}
}
the following is the class I use to test this out :-
package personal;
import java.util.*;
import personal.TestClass;
import java.lang.reflect.Field;
public class Test2 {
private static TestClass testObj;
private static Set<Set<String>> testStringSet;
private static Set<Set<Integer>> testIntegerSet;
static {
testObj = new TestClass();
testStringSet = new HashSet<Set<String>>();
testIntegerSet = new HashSet<Set<Integer>>();
}
private static void addToStringSet(Set<String> sets) {
synchronized(testStringSet) {
testStringSet.add(sets);
}
}
private static void addToIntegerSet(Set<Integer> sets) {
synchronized(testIntegerSet) {
testIntegerSet.add(sets);
}
}
private static int getTestIntegerSetSize() {
synchronized(testIntegerSet) {
return testIntegerSet.size();
}
}
private static int getTestStringSetSize() {
synchronized(testStringSet) {
return testStringSet.size();
}
}
private static class MyRunnable implements Runnable {
private TestClass tc;
private String name;
public MyRunnable(TestClass tc, int i) {
this.name = "Thread:- " + Integer.toString(i);
this.tc = tc;
}
#Override
public void run() {
try {
Field f1 = tc.getClass().getDeclaredField("setS");
Field f2 = tc.getClass().getDeclaredField("setI");
f1.setAccessible(true);
f2.setAccessible(true);
Set<String> v1 = (Set<String>)(((ThreadLocal<Set<String>>)(f1.get(tc))).get());
Set<Integer> v2 = (Set<Integer>) f2.get(tc);
addToIntegerSet(v2);
addToStringSet(v1);
} catch (Exception exp) {
System.out.println(exp);
}
}
}
public static void main(String[] args) {
for (int i=1; i<=2; i++) {
(new Thread (new MyRunnable(testObj,i))).start();
}
try {
Thread.sleep(5);
} catch (Exception exp) {
System.out.println(exp);
}
System.out.println(getTestStringSetSize());
System.out.println(getTestIntegerSetSize());
}
}
thus the 1st print statement should print out 2 and the second one should print out 1.
how ever the 1st print statement also prints out 1.
what is wrong ?
For a test class, I'd start with something much, much simpler. Just store a String or something in the ThreadLocal to start with, and avoid the reflection calls (setAccessible, etc.). Your issue is most likely in all of this extra code, and nothing due to the ThreadLocal itself.