In my code I've a thread waiting for events on a ZeroMQ endpoint, and its main loop looks similar to this:
while (externalCondition) {
byte[] bytes = subscriber.recv(0);
// Do things
}
The problem is that I don't know how to interrupt the call to .recv() method, if I want to exit the thread (for example because I want to shut down the app).
I tried to interrupt the thread but this seems not to work.
Welcome to ZeroMQ and distributed system architectures.
There is no cheap way to interrupt such call ( except a brutal one ), yet
there is an easy way to re-factor the idea, so as it can work in distributed system realms:
/* THIS CODE IS NOT A COPY+PASTE SYNDROME-PROOF, BUT HAS THE DESIGN IDEA CLEAR,
CODE IS NOT A READY TO RUN, AS THE TARGET LANGUAGE
AND THE ACTUAL ZeroMQ BINDING / VERSION
WILL DECIDE ON FURTHER DETAILS TO CODE IN
*/
while ( True ) { // -[NEVER-BLOCK]----------- aSoftRealTIME-control-loop-<body>--
bool ec = externalCondition;
int rc = subscriber.poll( someSoftRealTimeDELAY_in_ms );
if ( ec
& rc == 0
){
//--------------------------------------------------
// DO NOTHING, AS THERE IS NOTHING TO WORK ON
// or sleep, or
// DO ANY SoftRealTime-controlled / yet interruptible
// maintenance workpackage(s)
//-------------------------------------------------------
}
if ( !ec ){
break; // __________________________ J.I.T. / JMP / RET __^
}
if ( rc < 0 ){
// ZeroMQ .poll() Error Analysis & Error Recovery
}
if ( rc > 0 ){
// ZeroMQ has a thing to indeed .recv(), so read it:
byte[] bytes = subscriber.recv( zmq.DONTWAIT );
... // DO things ... on bytes ...
}
} // ---------------[NEVER-BLOCK]----------- aSoftRealTIME-control-loop-<body>--
Nota bene
while this remark might be a fairly well know subject for you, in case you have already got familiar with ZeroMQ internalities and various defaults, yet, let me mention it also here, as not all kind reader may have such deep ZeroMQ practice.
The SUB-side(s) of the Scalable Formal Communication Pattern used above has a dark-side. Unless one explicitly "subscribed" to some newspapers and/or magazine(s), there will nothing arrive into one's SUB-side mailbox, nothing gets delivered at all, as the "delivery service" has not received any subscription from you, telling 'em what you would like to ( receive ) and read
So, always do not forget to submit a call to such a
subscriber.setsockopt( aSubject_What_To_Subscribe_To_Indeed_Receive_STRING );
Not having done so, one will never .recv() a single byte.
Thanks. What would be the "brutal" one? –Alessandro Polverini 23 hours ago
Well, ZeroMQ native API, as of 2018-Q1 permits one, rather brutal way to achieve an enforced interruption of a blocking call to .recv() method - calling a <Context>.term() - so, it works more less like a "Demolition Man" when asking just for a few pieces of paper ... indeed that way, plus there are further pre-requisities not to hang your further code-execution in an uncontrolled zombie-alike state, if not infinitely ( if not having protected all your Socket-instances generated from native-API with an immediately configured .setsockopt( zmq.LINGER, 1 ) ( since native API 2.1+ ~ 4.(?) where a promised change of defaults was published ... respective language binding / wrappers differ in version adoption, so cannot refer in general and best check the jeromq right upon instantiations took place with published .getLinger()-exposed method to see this reactively per-incident :o) ).
Calling a WMD-alike annihilation, using .term()-method, will in those cases, where Socket-instances were not left waiting ( indeterminatly, if not infinitely ) by zmq.LINGER-instance attribute, cancel internal-FSA and exit even the still blocked-{ .recv() | .send() | poll() }-s and return the code-execution control-path to your "Demolition Man"-controls :o)
Doable, but a bit harsh, isn't it?
Related
I am fairly new to Java and extremely new to concurrency. However, I have worked with C# for a while. It doesn't really matter, but for the sake of example, I am trying to pull data off a table on server. I want method to wait until data is completely pulled. In C#, we have async-await pattern which can be used like this:
private async Task<List<ToDoItem>> PullItems ()
{
var newRemoteItems = await (from p in remoteTable select p).ToListAsync();
return newRemoteItems;
}
I am trying to have similar effect in Java. Here is the exact code I'm trying to port (Look inside SynchronizeAsync method.)! However, Java Azure SDK works with callbacks. So, I have a few options:
Use wait and notify pattern. Following code doesn't work since I don't understand what I'm doing.
final List<TEntity> newRemoteItems = new ArrayList<TEntity>();
synchronized( this ) {
remoteTable.where().field("lastSynchronized").gt(currentTimeStamp)
.execute(new TableQueryCallback<TEntity>() {
public void onCompleted(List<TEntity> result,
int count,
Exception exception,
ServiceFilterResponse response) {
if (exception == null) {
newRemoteItems.clear();
for (TEntity item: result) {
newRemoteItems.add(item);
}
}
}
});
}
this.wait();
//DO SOME OTHER STUFF
My other option is to move DO SOME OTHER STUFF right inside the callback's if(exception == null) block. However, this would result in my whole method logic chopped off into the pieces, disturbing the continuous flow. I don't really like this approach.
Now, here are questions:
What is recommended way of doing this? I am completing the tutorial on Java concurrency at Oracle. Still, clueless. Almost everywhere I read, it is recommended to use higher level stuff rather than wait and notify.
What is wrong with my wait and notify?
My implementation blocks the main thread and it's considered a bad practice. But what else can I do? I must wait for the server to respond! Also, doesn't C# await block the main thread? How is that not a bad thing?
Either put DO SOME OTHER STUFF into callback, or declare a semaphore, and call semaphore.release in the callback and call semaphore.aquire where you want to wait. Remove synchronized(this) and this.wait.
I'm using LuaJ to run user-created Lua scripts in Java. However, running a Lua script that never returns causes the Java thread to freeze. This also renders the thread uninterruptible. I run the Lua script with:
JsePlatform.standardGlobals().loadFile("badscript.lua").call();
badscript.lua contains while true do end.
I'd like to be able to automatically terminate scripts which are stuck in unyielding loops and also allow users to manually terminate their Lua scripts while they are running. I've read about debug.sethook and pcall, though I'm not sure how I'd properly use them for my purposes. I've also heard that sandboxing is a better alternative, though that's a bit out of my reach.
This question might also be extended to Java threads alone. I've not found any definitive information on interrupting Java threads stuck in a while (true);.
The online Lua demo was very promising, but it seems the detection and termination of "bad" scripts is done in the CGI script and not Lua. Would I be able to use Java to call a CGI script which in turn calls the Lua script? I'm not sure that would allow users to manually terminate their scripts, though. I lost the link for the Lua demo source code but I have it on hand. This is the magic line:
tee -a $LOG | (ulimit -t 1 ; $LUA demo.lua 2>&1 | head -c 8k)
Can someone point me in the right direction?
Some sources:
Embedded Lua - timing out rogue scripts (e.g. infinite loop) - an example anyone?
Prevent Lua infinite loop
Embedded Lua - timing out rogue scripts (e.g. infinite loop) - an example anyone?
How to interrupt the Thread when it is inside some loop doing long task?
Killing thread after some specified time limit in Java
I struggled with the same issue and after some digging through the debug library's implementation, I created a solution similar to the one proposed by David Lewis, but did so by providing my own DebugLibrary:
package org.luaj.vm2.lib;
import org.luaj.vm2.LuaValue;
import org.luaj.vm2.Varargs;
public class CustomDebugLib extends DebugLib {
public boolean interrupted = false;
#Override
public void onInstruction(int pc, Varargs v, int top) {
if (interrupted) {
throw new ScriptInterruptException();
}
super.onInstruction(pc, v, top);
}
public static class ScriptInterruptException extends RuntimeException {}
}
Just execute your script from inside a new thread and set interrupted to true to stop the execution. The exception will be encapsulated as the cause of a LuaError when thrown.
There are problems, but this goes a long way towards answering your question.
The following proof-of-concept demonstrates a basic level of sandboxing and throttling of arbitrary user code. It runs ~250 instructions of poorly crafted 'user input' and then discards the coroutine. You could use a mechanism like the one in this answer to query Java and conditionally yield inside a hook function, instead of yielding every time.
SandboxTest.java:
public static void main(String[] args) {
Globals globals = JsePlatform.debugGlobals();
LuaValue chunk = globals.loadfile("res/test.lua");
chunk.call();
}
res/test.lua:
function sandbox(fn)
-- read script and set the environment
f = loadfile(fn, "t")
debug.setupvalue(f, 1, {print = print})
-- create a coroutine and have it yield every 50 instructions
local co = coroutine.create(f)
debug.sethook(co, coroutine.yield, "", 50)
-- demonstrate stepped execution, 5 'ticks'
for i = 1, 5 do
print("tick")
coroutine.resume(co)
end
end
sandbox("res/badfile.lua")
res/badfile.lua:
while 1 do
print("", "badfile")
end
Unfortunately, while the control flow works as intended, something in the way the 'abandoned' coroutine should get garbage collected is not working correctly. The corresponding LuaThread in Java hangs around forever in a wait loop, keeping the process alive. Details here:
How can I abandon a LuaJ coroutine LuaThread?
I've never used Luaj before, but could you not put your one line
JsePlatform.standardGlobals().loadFile("badscript.lua").call();
Into a new thread of its own, which you can then terminate from the main thread?
This would require you to make some sort of a supervisor thread (class) and pass any started scripts to it to supervise and eventually terminate if they don't terminate on their own.
EDIT: I've not found any way to safely terminate LuaJ's threads without modifying LuaJ itself. The following was what I came up with, though it doesn't work with LuaJ. However, it can be easily modified to do its job in pure Lua. I may be switching to a Python binding for Java since LuaJ threading is so problematic.
--- I came up with the following, but it doesn't work with LuaJ ---
Here is a possible solution. I register a hook with debug.sethook that gets triggered on "count" events (these events occur even in a while true do end). I also pass a custom "ScriptState" Java object I created which contains a boolean flag indicating whether the script should terminate or not. The Java object is queried in the Lua hook which will throw an error to close the script if the flag is set (edit: throwing an error doesn't actually terminate the script). The terminate flag may also be set from inside the Lua script.
If you wish to automatically terminate unyielding infinite loops, it's straightforward enough to implement a timer system which records the last time a call was made to the ScriptState, then automatically terminate the script if sufficient time passes without an API call (edit: this only works if the thread can be interrupted). If you want to kill infinite loops but not interrupt certain blocking operations, you can adjust the ScriptState object to include other state information that allows you to temporarily pause auto-termination, etc.
Here is my interpreter.lua which can be used to call another script and interrupt it if/when necessary. It makes calls to Java methods so it will not run without LuaJ (or some other Lua-Java library) unless it's modified (edit: again, it can be easily modified to work in pure Lua).
function hook_line(e)
if jthread:getDone() then
-- I saw someone else use error(), but an infinite loop still seems to evade it.
-- os.exit() seems to take care of it well.
os.exit()
end
end
function inithook()
-- the hook will run every 100 million instructions.
-- the time it takes for 100 million instructions to occur
-- is based on computer speed and the calling environment
debug.sethook(hook_line, "", 1e8)
local ret = dofile(jLuaScript)
debug.sethook()
return ret
end
args = { ... }
if jthread == nil then
error("jthread object is nil. Please set it in the Java environment.",2)
elseif jLuaScript == nil then
error("jLuaScript not set. Please set it in the Java environment.",2)
else
local x,y = xpcall(inithook, debug.traceback)
end
Here's the ScriptState class that stores the flag and a main() to demonstrate:
public class ScriptState {
private AtomicBoolean isDone = new AtomicBoolean(true);
public boolean getDone() { return isDone.get(); }
public void setDone(boolean v) { isDone.set(v); }
public static void main(String[] args) {
Thread t = new Thread() {
public void run() {
System.out.println("J: Lua script started.");
ScriptState s = new ScriptState();
Globals g = JsePlatform.debugGlobals();
g.set("jLuaScript", "res/main.lua");
g.set("jthread", CoerceJavaToLua.coerce(s));
try {
g.loadFile("res/_interpreter.lua").call();
} catch (Exception e) {
System.err.println("There was a Lua error!");
e.printStackTrace();
}
}
};
t.start();
try { t.join(); } catch (Exception e) { System.err.println("Error waiting for thread"); }
System.out.println("J: End main");
}
}
res/main.lua contains the target Lua code to be run. Use environment variables or parameters to pass additional information to the script as usual. Remember to use JsePlatform.debugGlobals() instead of JsePlatform.standardGlobals() if you want to use the debug library in Lua.
EDIT: I just noticed that os.exit() not only terminates the Lua script but also the calling process. It seems to be the equivalent of System.exit(). error() will throw an error but will not cause the Lua script to terminate. I'm trying to find a solution for this now.
Thanks to #Seldon for suggesting the use of custom DebugLib. I implemented a simplified version of that by just checking before every instruction if a predefined amount of time is elapsed. This is of course not super accurate because there is some time between class creation and script execution. Requires no separate threads.
class DebugLibWithTimeout(
timeout: Duration,
) : DebugLib() {
private val timeoutOn = Instant.now() + timeout
override fun onInstruction(pc: Int, v: Varargs, top: Int) {
val timeoutElapsed = Instant.now() > timeoutOn
if (timeoutElapsed)
throw Exception("Timeout")
super.onInstruction(pc, v, top)
}
}
Important note: if you sandbox an untrusted script calling load function on Lua-code and passing a separate environment to it, this will not work. onInstruction() seems to be called only if the function environment is a reference to _G. I dealt with that by stripping everything from _G and then adding whitelisted items back.
-- whitelisted items
local sandbox_globals = {
print = print
}
local original_globals = {}
for key, value in pairs(_G) do
original_globals[key] = value
end
local sandbox_env = _G
-- Remove everything from _G
for key, _ in pairs(sandbox_env) do
sandbox_env[key] = nil
end
-- Add whitelisted items back.
-- Global pairs-function cannot be used now.
for key, value in original_globals.pairs(sandbox_globals) do
sandbox_env[key] = value
end
local function run_user_script(script)
local script_function, message = original_globals.load(script, nil, 't', sandbox_env)
if not script_function then
return false, message
end
return pcall(script_function)
end
We know how to force shutdown an computer using Java. For example, the following code works fine for force shutdown:
public static void main(String arg[]) throws IOException{
Runtime runtime = Runtime.getRuntime();
Process proc = runtime.exec("shutdown -s -t 0");
System.exit(0);
}
Now, suppose if I want to force startup a computer (which is in shut down state), at a particular time, is it possible to do in Java or any other language?
You need something to trigger the startup. The best way to trigger this is Wake On Lan.
If you want to do this in Java, this might be a good resource.
In addition to wake on lan, there are IPMI devices that run on some server-grade hardware that is connected to the motherboard and can control power as well as provide serial console output over a network connection. This computer is running all the time, but I'm not familiar with any you can load your own code onto.
You can control this device remotely to power control the server that is off from any language including java.
http://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface
If your BIOS supports Advanced Power Management (APM) version 1.2 or later, it should be possible to wake it from sleep/standy or hibernation based on a timer. On Windows an end user can do this through Task Scheduler, and if you wish to do it programmatically you can use the Task Scheduler interfaces.
I don't know how you would do this through Java, but here is some example C code that will create a task to wake the computer up 2 minutes in the future:
#include <mstask.h>
#include <time.h>
int main() {
HRESULT hr = CoInitialize(NULL);
if (SUCCEEDED(hr)) {
ITaskScheduler *scheduler;
hr = CoCreateInstance(CLSID_CTaskScheduler, NULL, CLSCTX_INPROC_SERVER, IID_ITaskScheduler, (void**)&scheduler);
if (SUCCEEDED(hr)) {
ITask *task;
hr = scheduler->NewWorkItem(L"Wake Timer", CLSID_CTask, IID_ITask, (LPUNKNOWN*)&task);
if (SUCCEEDED(hr)) {
WORD index;
ITaskTrigger *trigger;
hr = task->CreateTrigger(&index, &trigger);
if (SUCCEEDED(hr)) {
time_t t = time(NULL) + 120;
struct tm *ltime = localtime(&t);
TASK_TRIGGER triggertime;
memset(&triggertime, 0, sizeof(triggertime));
triggertime.cbTriggerSize = sizeof(TASK_TRIGGER);
triggertime.wBeginYear = ltime->tm_year+1900;
triggertime.wBeginMonth = ltime->tm_mon+1;
triggertime.wBeginDay = ltime->tm_mday;
triggertime.wStartHour = ltime->tm_hour;
triggertime.wStartMinute = ltime->tm_min;
triggertime.TriggerType = TASK_TIME_TRIGGER_ONCE;
trigger->SetTrigger(&triggertime);
trigger->Release();
}
task->SetFlags(TASK_FLAG_DELETE_WHEN_DONE|TASK_FLAG_SYSTEM_REQUIRED|TASK_FLAG_RUN_ONLY_IF_LOGGED_ON);
task->SetAccountInformation(L"", NULL);
IPersistFile *file;
hr = task->QueryInterface(IID_IPersistFile, (void**)&file);
if (SUCCEEDED(hr)) {
file->Save(NULL, TRUE);
file->Release();
}
task->Release();
}
scheduler->Release();
}
CoUninitialize();
}
return 0;
}
Assumedly if you can do this on Windows, there must be equivalent APIs for other operating systems.
I did manage to find a similar question floating around on the internet, so I'll post the links here to see if you find it helpful. (this was the thread I found: http://www.coderanch.com/t/440680/gc/interact-Windows-Task-Scheduler-Java)
First of all though, I might add that Java is a language that must run in a Virtual Machine - there are no two ways around it. I'm not well versed in 'low-level' programming, such as programming at closer to BIOS level, which is sort of where we are heading with this.
As the question was explicitly about Java, the best I could come up with from research, is (if you're really wanting to use Java for something), using the JAVA-COM (JACOB) http://sourceforge.net/projects/jacob-project/ which allows you to hook into the Windows Task Scheduler http://msdn.microsoft.com/en-us/library/aa383581%28VS.85%29.aspx via the COM language (AF
As far as I am aware, because Java needs to be in a virtual machine to run, there would be no way of getting it to do an action such as turning on a PC - let's not even get into issues of whether such an action would require administrator or above privileges.
Hope that helps.
I work on windows but I am stuck here on Mac. I have the Canon SDK and have built a JNA wrapper over it. It works well on windows and need some help with Mac.
In the sdk, there is a function where one can register a callback function. Basically when an event occurs in camera, it calls the callback function.
On windows, after registering, I need to use User32 to get the event and to dispatch the event by:
private static final User32 lib = User32.INSTANCE;
boolean hasMessage = lib.PeekMessage( msg, null, 0, 0, 1 ); // peek and remove
if( hasMessage ){
lib.TranslateMessage( msg );
lib.DispatchMessage( msg ); //message gets dispatched and hence the callback function is called
}
In the api, I do not find a similar class in Mac. How do I go about this one??
PS: The JNA api for unix is extensive and I could not figure out what to look for. The reference might help
This solution is using the Cocoa framework. Cocoa is deprecated and I am not aware of any other alternative solution. But the below works like charm.
Finally I found the solution using Carbon framework. Here is my MCarbon interface which defines calls I need.
public interface MCarbon extends Library {
MCarbon INSTANCE = (MCarbon) Native.loadLibrary("Carbon", MCarbon.class);
Pointer GetCurrentEventQueue();
int SendEventToEventTarget(Pointer inEvent, Pointer intarget);
int RemoveEventFromQueue(Pointer inQueue, Pointer inEvent);
void ReleaseEvent(Pointer inEvent);
Pointer AcquireFirstMatchingEventInQueue(Pointer inQueue,NativeLong inNumTypes,EventTypeSpec[] inList, NativeLong inOptions);
//... so on
}
The solution to the problem is solved using the below function:
NativeLong ReceiveNextEvent(NativeLong inNumTypes, EventTypeSpec[] inList, double inTimeout, byte inPullEvent, Pointer outEvent);
This does the job. As per documentation -
This routine tries to fetch the next event of a specified type.
If no events in the event queue match, this routine will run the
current event loop until an event that matches arrives, or the
timeout expires. Except for timers firing, your application is
blocked waiting for events to arrive when inside this function.
Also if not ReceiveNextEvent, then other functions as mentioned in MCarbon class above would be useful.
I think Carbon framework documentation would give more insights and flexibilities to solve the problem. Apart from Carbon, in forums people have mentioned about solving using Cocoa, but none I am aware of.
Edit: Thanks to technomarge, more information here
Is there an easy way to read output from subprocess unbuffered? I'm trying to call a C program from Java, but apparently it's stdout block-buffered when connected to pipe and line-buffered only when connected to console. I cannot modify C program.
Maybe there is a way to fool the program into thinking it is connected to console? Bonus points for a solution that works on linux as well.
It's not a great solution, but the runtime library probably doesn't buffer serial ports so if you're desperate enough you could use a null-modem emulator such as com0com or a derivative thereof.
Have you tried using environment variable BUF_1_=0?
OK -
1) "Line buffered input" is a common idiom for console-mode programs, dating back to the original Unix and serial-mode VTxx terminals.
2) You can read "raw, unbuffered" I/O (a keystroke at a time, instead of a line at a time), but the details are OS specific. Whatever you need to do on a specific OS, you almost certainly can do from Java.
3) It sounds like you want to be able to intercept an "arrow up" key or "page down" key, as it happens, on a Windows keyboard. Perhaps for playing a game, or for interacting with a console-mode user interface.
4) There are several options. One you might wish to consider is the "robot" API, used for testing:
http://www.java-tips.org/java-se-tips/java.awt/how-to-use-robot-class-in-java.html
If that's not sufficient, please give more details about exactly how you're trying to get your Java program to interact with the C program (and clarify if the platform is indeed Windows and a DOS prompt).
You could redirect the c output to a file and then tail the file from another thread in java using
org.apache.commons.io.input.Tailer
The best way to do it is using Threads to read the output
What I would do is: using a Callable (a Runnable will also work) where I give the input from the process (process.getInputStream ()) and the output where I want to store the output. The same should be done for the StdErr. The resulting output can be read using whatever you like.
public Object call () throws IOException {
int bytesRead;
byte[] b = new byte [ this.maxBlockSize ];
try {
while ( ( bytesRead = this.input.read ( b ) ) != -1 ) {
this.output.write ( b, 0, bytesRead );
}
} finally {
if ( this.output != null ) {
this.output.close ();
}
}
return null;
}