We Keep Coding

Java netty 4 - how to broadcast message to multiple nodes properly, especially when a node is blocked by network?

I'm using netty 4.
I have multiple nodes in the network, for simplicity, there are 3 nodes called A, B and C.
Every 10 seconds, A will sends a status message to B and C, B and C also do the same thing, B sends to A and C, C sends to A and B.
Because, sometimes, one node can be blocked (not sure about the reason), for example, A cannot send to B in 10 seconds and C will not receive any message from A.
So, I spawn a different thread for different connection to client:
for (final RemoteNode inpeerNode : inpeerNodes) {
log.debug("Total threads: " + java.lang.Thread.activeCount());
Thread thread = new Thread(new Runnable() {
#Override
public void run() {
sendStatusMessage("a big status message");
}
}
thread.start();
}
and in sendStatusMessage method, it creates a new Eventloop which I don't seem it is right as this is expensive:
public void sendMessage(Integer timeoutInMilliseconds) throws InterruptedException {
EventLoopGroup group = new NioEventLoopGroup();
try {
Bootstrap bootstrap = new Bootstrap();
bootstrap.group(group)
.channel(NioSocketChannel.class)
.option(ChannelOption.TCP_NODELAY, true)
.handler(new ChannelInitializer<SocketChannel>() {
#Override
public void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast(new LoggingHandler(LogLevel.TRACE),
new DelimiterBasedFrameDecoder(Integer.MAX_VALUE, Delimiters.lineDelimiter()),
new StringDecoder(),
new StringEncoder(),
new WriteTimeoutHandler(7000),
new ClientCommunicatorHandler(message));
}
});
if (timeoutInMilliseconds != null) {
bootstrap.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, timeoutInMilliseconds);
} else {
bootstrap.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 7000);
}
// Start the connection attempt.
bootstrap.connect(host, port).sync().channel().closeFuture().sync();
} finally {
group.shutdownGracefully();
}
in ClientCommunicatorHandler it has simple code:
class ClientCommunicatorHandler extends SimpleChannelInboundHandler<String> {
private final Logger log = LoggerFactory.getLogger(ClientCommunicatorHandler.class);
private final String messageToSend;
ClientCommunicatorHandler(#NotNull Message message) {
// e.g. send local node's status message to inpeer nodes
this.messageToSend = message.toSerializedString().replace("\n", "").replace("\r", "");
}
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
ServerCommunicatorHandler.clientChannels.add(ctx.channel());
String message = this.messageToSend;
ctx.writeAndFlush(message + System.lineSeparator())
.addListener(ChannelFutureListener.CLOSE);
}
Up to sometimes, the java application on node A cannot send any status messages anymore. I think some channels to send messages to clients (B and C) are blocked.
Any idea how to avoid that problem? For example, node A can still send to node C in 10 seconds, even if node B is blocked.

ScheduledExecutorService task will be executed by multi-thread

This makes me really curious.There is a button which sends a simple post request by ajax on a jsp page,and I use a RESTFUL method to handle this request,but that method will be executed twice or three times.This will only happen on CentOS 7.3,on my laptop I use windows10, multi-thread will not happen.I have searched on Google but nothing helpful
has been found.Here are the codes:
asset.jsp:
<button class="btn btn-default btn-sm" title="allDoMi">
<i class="fa fa-arrow-down">allDoMi</i>
</button>
$("button[title='allDoMi']").click(function () {
var dataparam = "version=1";
if (!confirm('confirm all DoMi？'))
return;
//ajax request
$.ajax({
contentType: "application/json",
url: serviceurl.baseurl + "/asset/doMiAction",
data: dataparam,
beforeSend: function () {
//do nothing
},
error: function () {
//do nothing
},
success: function (info) {
//do nothing
}
});
});
Asset.java
#Service
#Path("/asset")
public class AssetRest {
#Path("/doMiAction")
#POST
#Produces({ MediaType.APPLICATION_JSON,
MediaType.APPLICATION_XML })
public RestfulResult doMiAction(#FormParam("version") String
version) {
logger.info("doMiAction method began....");
//package data for duMiSyncDtos,here only for test
List<DuMiSyncDto> duMiSyncDtos =new List<>();
//this url is for http simulation using HttpURLConnection
final String dumiUrl = "http://someip/someurl";
final Map<String, List<DuMiSyncDto>> map;
//only continue when list is not empty
if (!duMiSyncDtos.isEmpty()) {
//this method used for sync data in a certain order
map = groupList(duMiSyncDtos);
SortedSet<String> ss = new TreeSet<>(map.keySet());
final Iterator<String> iter = ss.iterator();
final ScheduledExecutorService ticker = Executors.newSingleThreadScheduledExecutor();
logger.info("NEW A SINGLETHREADSCHEDULEDEXECUTOR");
//retrieve data from a .property file,I set it 20000,therefore the job will be executed in every 20 seconds
final int DELAY = NumberUtils.toInt(WebUtils.getConfigValue("dumi.delay"));
ticker.scheduleWithFixedDelay(new Runnable() {
private int count;
public void run() {
logger.info("BEGIN RUN METHOD:"+System.identityHashCode(AssetRest.this));
if(iter.hasNext()) {
try {
List<DuMiSyncDto> value = map.get(iter.next());
//this method used for simulating a httprequest using HttpURLConnection to invoke a remote service to get the result info which forms in a JSON string format
String resultmsg = getDuMiReturnMessage(dumiUrl,value);
if(resultmsg!=null && !resultmsg.contains("\"code\":\"0000\"")) {
logger.info("Return code is "+resultmsg+",the sync work will be terminated.");
ticker.shutdown();
return;
}
//this method used for showing some useful infomation on the console using log4j
showSyncInfomation(value);
//this method used for count how many items have been synchronized successfully
int currentcount = getSyncCount(resultmsg);
count += currentcount ;
logger.info("current sync data:"+currentcount+"，summing data"+count+"");
} catch (Exception e) {
logger.error("method[doMiAction]...executing schedule:",e);
}
} else {
ticker.shutdown();
}
}
}, 0, DELAY, TimeUnit.MILLISECONDS);
}
}
After I click the button,all the log info will be shown on Putty console for two or three times,yet I have clicked that button for only ONCE!I have tested for several times,it will happen,but in windows on my laptop,this will not happen at all.Here is a detail might be help:previously the implementation for timed execution is not like this,it has been written like :
for(DuMiSyncDto dto:duMiSyncDtoList){
//do the business code
Thread.sleep(20000);
}
Because there is database synchronization from the remote service,I need to control the interval time not too soon between every two operations:execute in every 20 seconds and 100 data at a time.In this situation,the multi-thread problem occurs,I thought it may be the for loop which aroused so I change the way using a JDK API instead but issues were still there.So WHY all of these?
---------------------------first edit------------------------------------------
private int getSyncCount(String resultmsg) {
int count = 0;
JSONObject obj = JSONObject.fromObject(resultmsg);
String message = obj.getString("message");
if(!WebUtils.isEmpty(message)) {
String[] arr = message.split(" ");
if(arr!=null && arr.length>1) {
count += Integer.parseInt(arr[1].trim());
}
}
logger.info("currentThreadName:"+Thread.currentThread().getName());
return count;
}
Notice in this method,I log the current thread name,and it shows :
...
currentThreadName:pool-1-thread-1
currentThreadName:pool-2-thread-1
currentThreadName:pool-3-thread-1
...
when there are 3 threads.

NullPointerException in YouTube API v3

I have been working with the YouTube API v3 with Java and have encountered a problem.
When trying to set a variable in an array I am getting a NullPointerException. It seems completely random and impossible. I am getting this on this line:
full[0] = snippet.getDisplayMessage().toLowerCase();
in this method:
private static void listChatMessages(
final String liveChatId,
final String nextPageToken,
long delayMs) {
System.out.println(
String.format("Getting chat messages in %1$.3f seconds...", delayMs * 0.001));
Timer pollTimer = new Timer();
pollTimer.schedule(
new TimerTask() {
#Override
public void run() {
try {
// Get chat messages from YouTube
LiveChatMessageListResponse response = youtube
.liveChatMessages()
.list(liveChatId, "snippet, authorDetails")
.setPageToken(nextPageToken)
.setFields(LIVE_CHAT_FIELDS)
.execute();
// Display messages and super chat details
List<LiveChatMessage> messages = response.getItems();
for (int i = 0; i < messages.size(); i++) {
LiveChatMessage message = messages.get(i);
LiveChatMessageSnippet snippet = message.getSnippet();
final String[] full = new String[2];
full[0] = snippet.getDisplayMessage().toLowerCase();
full[1] = message.getId();
ExecutorService es = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
es.submit(new Runnable() {
public void run() {
PressKey.main(full);
}
});
}
// Request the next page of messages
listChatMessages(
liveChatId,
response.getNextPageToken(),
response.getPollingIntervalMillis());
} catch (Throwable t) {
System.err.println("Throwable: " + t.getMessage());
t.printStackTrace();
}
}
}, delayMs);
}
}

I am not entirely sure how the API is implemented, but this NPE happens because the value of
snippet.getDisplayMessage()
Is null, and calling .toLowerCase() on null results in the NPE.
Like I said, I do not know for sure how the API and the container classes are implemented, but it may seem like getDisplayMessage() is not always guaranteed to return any content.
You could fix your NPE by checking for null values and supplying a default value instead:
full[0] = snippet.getDisplayMessage() == null ? "" : snippet.getDisplayMessage().toLowerCase();
Here we assign an empty string to full[0] if getDisplayMessage() is null.

Android video trimming library

I am doing Video Triming and using k4l-video-trimmer library. I am getting an issue. I have downloaded the latest code and integrate it on Android Studio. When i select a video, k4l-video-trimmer successfully prepared the video and correctly shows video info and snapshots. I have set the max duration to 10 sec but when move the progressbar to crop the video at specific duration, the cropping duration which is showing on screen like (01:21 sec - 01:31 sec) for 10 sec will change to (01:21 sec - 01:36 sec) becomes 15 sec duration that is an issue and when I crop the video, it will crop it for 23 sec. I don't know how to resolve this issue. Please help me to resolve this issue

You have to implement MediaRecorder.OnInfoListener to manually stop the recording at 10 seconds. Once its stopped, the MediaRecorder goes back to the initial state and the setup has to be done again to start back recording.
public class VideoCapture extends Activity implements MediaRecorder.OnInfoListener {
public void startVideoRecording() {
// Normal MediaRecorder Setup
recorder.setMaxDuration(10000); // 10 seconds
recorder.setOnInfoListener(this); // very important
}
public void onInfo(MediaRecorder mrc, int mri, int extra) {
if (mri == MediaRecorder.MEDIA_RECORDER_INFO_MAX_DURATION_REACHED) {
Log.v("VIDEOCAPTURE","10 seconds");
mrc.stop();
}
}
}
Now, For the Progress bar you can use a Timer.
//fires once a second, decrease this to fire more frequently
private static final int TIMER_FREQ = 1000;
final ProgressBar progressBar = new ProgressBar(this); //where this is a Context
progressBar.setMax(10000);
Timer progressBarAdvancer = new Timer();
progressBarAdvancer.scheduleAtFixedRate(new TimerTask() {
public void run() {
progressBar.setProgress(progressBar.getProgress() + TIMER_FREQ);
}
},
0, //Delay before first execution
TIMER_FREQ);
By doing this, the progressBar operates on a separate thread from the recording, but will finish within the required 10 seconds. At this pount you can stop the recording and do the rest of the things.
Also, you can use the Video Trimmer based on "k4l-video-trimmer" library which handle various issues on the k4l-video-trimmer.

You can use mobile-ffmpeg Supports API Level 16+
fun scaleVideo(path: String, destinationFilePath: String) {
_loaderVisisble.value = true
viewModelScope.launch {
val cmd = arrayOf(
"-i",
path,
"-vf",
"scale=576:1024:force_original_aspect_ratio=decrease",
destinationFilePath
)
Log.v("str_Cmd", cmd.toString() + "")
val status = executeCommand(cmd)
when (status) {
FFmpeg.RETURN_CODE_SUCCESS -> {
_loaderVisisble.value = false
val mergedFile = File(destinationFilePath)
Log.v(
"target_file_size",
(mergedFile.length() / 1024).toString().toInt().toString() + ""
)
onVideoScaleListener.postValue(destinationFilePath)
}
FFmpeg.RETURN_CODE_CANCEL -> {
_loaderVisisble.value = false
}
else -> {
_loaderVisisble.value = false
}
}
}
}
private suspend fun executeCommand(cmd: Array<String>): Int {
var status = -1
withContext(Dispatchers.Default) {
val rc = FFmpeg.execute(cmd)
when (rc) {
FFmpeg.RETURN_CODE_SUCCESS -> {
Log.i(
Config.TAG,
"Command execution completed successfully."
)
}
FFmpeg.RETURN_CODE_CANCEL -> {
Log.i(
Config.TAG,
"Command execution cancelled by user."
)
}
else -> {
Log.i(
Config.TAG,
String.format(
"Command execution failed with rc=%d and the output below.",
rc
)
)
}
}
status = rc
}
return status
}

How to stop and resume Observable.interval emiting ticks

This will emit a tick every 5 seconds.
Observable.interval(5, TimeUnit.SECONDS, Schedulers.io())
.subscribe(tick -> Log.d(TAG, "tick = "+tick));
To stop it you can use
Schedulers.shutdown();
But then all the Schedulers stops and it is not possible to resume the ticking later. How can I stop and resume the emiting "gracefully"?

Here's one possible solution:
class TickHandler {
private AtomicLong lastTick = new AtomicLong(0L);
private Subscription subscription;
void resume() {
System.out.println("resumed");
subscription = Observable.interval(5, TimeUnit.SECONDS, Schedulers.io())
.map(tick -> lastTick.getAndIncrement())
.subscribe(tick -> System.out.println("tick = " + tick));
}
void stop() {
if (subscription != null && !subscription.isUnsubscribed()) {
System.out.println("stopped");
subscription.unsubscribe();
}
}
}

Some time ago, I was also looking for kind of RX "timer" solutions, but non of them met my expectations. So there you can find my own solution:
AtomicLong elapsedTime = new AtomicLong();
AtomicBoolean resumed = new AtomicBoolean();
AtomicBoolean stopped = new AtomicBoolean();
public Flowable<Long> startTimer() { //Create and starts timper
resumed.set(true);
stopped.set(false);
return Flowable.interval(1, TimeUnit.SECONDS)
.takeWhile(tick -> !stopped.get())
.filter(tick -> resumed.get())
.map(tick -> elapsedTime.addAndGet(1000));
}
public void pauseTimer() {
resumed.set(false);
}
public void resumeTimer() {
resumed.set(true);
}
public void stopTimer() {
stopped.set(true);
}
public void addToTimer(int seconds) {
elapsedTime.addAndGet(seconds * 1000);
}

val switch = new java.util.concurrent.atomic.AtomicBoolean(true)
val tick = new java.util.concurrent.atomic.AtomicLong(0L)
val suspendableObservable =
Observable.
interval(5 seconds).
takeWhile(_ => switch.get()).
repeat.
map(_ => tick.incrementAndGet())
You can set switch to false to suspend the ticking and true to resume it.

Sorry this is in RxJS instead of RxJava, but the concept will be the same. I adapted this from learn-rxjs.io and here it is on codepen.
The idea is that you start out with two streams of click events, startClick$ and stopClick$. Each click occurring on the stopClick$ stream get mapped to an empty observable, and clicks on startClick$ each get mapped to the interval$ stream. The two resulting streams get merge-d together into one observable-of-observables. In other words, a new observable of one of the two types will be emitted from merge each time there's a click. The resulting observable will go through switchMap, which starts listening to this new observable and stops listening to whatever it was listening to before. Switchmap will also start merge the values from this new observable onto its existing stream.
After the switch, scan only ever sees the "increment" value emitted by interval$, and it doesn't see any values when "stop" has been clicked.
And until the first click occurs, startWith will start emitting values from $interval, just to get things going:
const start = 0;
const increment = 1;
const delay = 1000;
const stopButton = document.getElementById('stop');
const startButton = document.getElementById('start');
const startClick$ = Rx.Observable.fromEvent(startButton, 'click');
const stopClick$ = Rx.Observable.fromEvent(stopButton, 'click');
const interval$ = Rx.Observable.interval(delay).mapTo(increment);
const setCounter = newValue => document.getElementById("counter").innerHTML = newValue;
setCounter(start);
const timer$ = Rx.Observable
// a "stop" click will emit an empty observable,
// and a "start" click will emit the interval$ observable.
// These two streams are merged into one observable.
.merge(stopClick$.mapTo(Rx.Observable.empty()),
startClick$.mapTo(interval$))
// until the first click occurs, merge will emit nothing, so
// use the interval$ to start the counter in the meantime
.startWith(interval$)
// whenever a new observable starts, stop listening to the previous
// one and start emitting values from the new one
.switchMap(val => val)
// add the increment emitted by the interval$ stream to the accumulator
.scan((acc, curr) => curr + acc, start)
// start the observable and send results to the DIV
.subscribe((x) => setCounter(x));
And here's the HTML
<html>
<body>
<div id="counter"></div>
<button id="start">
Start
</button>
<button id="stop">
Stop
</button>
</body>
</html>

Here is a another way to do this, I think.
When you check the source code, you will find interval() using class OnSubscribeTimerPeriodically. The key code below.
#Override
public void call(final Subscriber<? super Long> child) {
final Worker worker = scheduler.createWorker();
child.add(worker);
worker.schedulePeriodically(new Action0() {
long counter;
#Override
public void call() {
try {
child.onNext(counter++);
} catch (Throwable e) {
try {
worker.unsubscribe();
} finally {
Exceptions.throwOrReport(e, child);
}
}
}
}, initialDelay, period, unit);
}
So, you will see, if you wanna cannel the loop, what about throwing a new exception in onNext(). Example code below.
Observable.interval(1000, TimeUnit.MILLISECONDS)
.subscribe(new Action1<Long>() {
#Override
public void call(Long aLong) {
Log.i("abc", "onNext");
if (aLong == 5) throw new NullPointerException();
}
}, new Action1<Throwable>() {
#Override
public void call(Throwable throwable) {
Log.i("abc", "onError");
}
}, new Action0() {
#Override
public void call() {
Log.i("abc", "onCompleted");
}
});
Then you will see this:
08-08 11:10:46.008 28146-28181/net.bingyan.test I/abc: onNext
08-08 11:10:47.008 28146-28181/net.bingyan.test I/abc: onNext
08-08 11:10:48.008 28146-28181/net.bingyan.test I/abc: onNext
08-08 11:10:49.008 28146-28181/net.bingyan.test I/abc: onNext
08-08 11:10:50.008 28146-28181/net.bingyan.test I/abc: onNext
08-08 11:10:51.008 28146-28181/net.bingyan.test I/abc: onNext
08-08 11:10:51.018 28146-28181/net.bingyan.test I/abc: onError

You can use takeWhile and loop until conditions is true
Observable.interval(1, TimeUnit.SECONDS)
.takeWhile {
Log.i(TAG, " time " + it)
it != 30L
}
.subscribe(object : Observer<Long> {
override fun onComplete() {
Log.i(TAG, "onComplete " + format.format(System.currentTimeMillis()))
}
override fun onSubscribe(d: Disposable) {
Log.i(TAG, "onSubscribe " + format.format(System.currentTimeMillis()))
}
override fun onNext(t: Long) {
Log.i(TAG, "onNext " + format.format(System.currentTimeMillis()))
}
override fun onError(e: Throwable) {
Log.i(TAG, "onError")
e.printStackTrace()
}
});

#AndroidEx , that's a wonderful answer. I did it a bit differently:
private fun disposeTask() {
if (disposeable != null && !disposeable.isDisposed)
disposeable.dispose()
}
private fun runTask() {
disposeable = Observable.interval(0, 30, TimeUnit.SECONDS)
.flatMap {
apiCall.runTaskFromServer()
.map{
when(it){
is ResponseClass.Success ->{
keepRunningsaidTasks()
}
is ResponseClass.Failure ->{
disposeTask() //this will stop the task in instance of a network failure.
}
}
}