I am working on homework about thread synchronization. The method reduceLoad() cannot be accessed when I call this method. All print function inside the reduceLoad() method are never called. If I can't run this method, the key will never be released.
It might be caused by Semaphore.accquire() and Semaphore.realse(). So I tried to delete all Semaphore methods that may cause the problem.
Protion of the program result below:
As you can see the program is held after all threads print "disembarks from ferry at port "
..... omit some result here....
Arrive at port 1 with a load of 5 vehicles
Auto 4 arrives at port 0
Ambulance 1 arrives at port 1
Ambulance 1 boards the ferry at port 1
Auto 6 boards on the ferry at port 1
Auto 8 boards on the ferry at port 1
Auto 0 disembarks from ferry at port 1
Auto 2 disembarks from ferry at port 1
Auto 1 disembarks from ferry at port 1
Ambulance 0 disembarks the ferry at port 1
Auto 7 boards on the ferry at port 1
Auto 3 disembarks from ferry at port 1
This is where the function is called
// Arrive at the next port
port = 1 - port;
// wait for ferry arrives
while (fry.getPort() != port) {
try {
sleep(1000);
} catch (Exception e) {
}
}
// disembarking
System.out.println("Auto " + id_auto + " disembarks from ferry at port " + port);
logger.check(fry.getPort() == port, "error unloading at wrong port");
fry.reduceLoad(); // Reduce load
System.out.println("Auto " + id_auto + " successfully disembarks from ferry at port " + port);
This method reduceLoad() should release a key when the ferry is empty, this key is used to notify other threads to continue adding more cars into the ferry
public synchronized void reduceLoad() {
logger.check(load > 0, "error unloading an empty Ferry!");
load = load - 1;
System.out.println("removed load, now " + load);
if (load == 0) {
unloadingDone.release();
}
}
Related
I'm using socket.io-client java library version 1.0.0 (included it via pom.xml)
I want to do a stress/performance-test to a socket.io based server in nodejs (the server seems to work well, not of interest here).
I'm experiencing a very strange behaviour when I establish my connections doing
a Thread.sleep between each new connection.
If establish the connection all at once, there is no problem. The problems occur when I wait (Thead.sleep()) some milliseconds.
Always after the 5th established connection, the connections need unusual long until they are established (connected callback received). The next connection seems to wait until the first one disconnects, but thereafter several connections are established. And a lot of connections aren't established at all, no matter how long I wait. This is really strange. Why does this happen?
It does not matter, if I establish the connections asynychronously (doing each connection in a separate thread starting all threads immediately) or synchronously one after another - it works fine if I don't wait between each new socket/thread creation.
This is the complete code:
import io.socket.client.IO;
import io.socket.client.Socket;
import io.socket.emitter.Emitter;
import java.net.URISyntaxException;
public class SocketStressTest
{
private static final String SOCKET_URI = "http://test.mydomain.com/";
public static int AMOUNT_OF_SOCKETS = 100;
IO.Options opts;
private int connectionCounter = 0;
private int disConnectionCounter = 0;
public static void main(String[] args)
{
new SocketStressTest().start();
}
private void start()
{
opts = new IO.Options();
opts.forceNew = true;
opts.reconnection = false;
Thread thread = new Thread(new Runnable()
{
public void run()
{
initSockets();
}
});
thread.start();
System.out.println("Generated all sockets");
}
private void initSockets()
{
for (int i = 0; i < AMOUNT_OF_SOCKETS; i++)
{
//when I comment out sleeping and try-catch below, everything works fine
try
{
Thread.sleep(400);
}
catch (InterruptedException e)
{
e.printStackTrace();
}
createSocket(i);
}
}
private void createSocket(final int nr)
{
//does not matter if I do this threaded or not:
Thread thread = new Thread(new Runnable()
{
public void run()
{
try
{
System.out.println("nr " + nr + " creating socket");
final Socket socket = IO.socket(SOCKET_URI, opts);
socket
.on(Socket.EVENT_CONNECT, new Emitter.Listener()
{
public void call(Object... args)
{
connectionCounter++;
System.out.println("nr " + nr + " connected id:" + socket.id() + " concounter:" + connectionCounter);
if (connectionCounter == AMOUNT_OF_SOCKETS)
System.out.println("===> ALL connected!");
}
})
.on(Socket.EVENT_DISCONNECT, new Emitter.Listener()
{
public void call(Object... args)
{
disConnectionCounter++;
System.out.println("nr " + nr + " disconnected. discounter=" + disConnectionCounter);
if (disConnectionCounter == AMOUNT_OF_SOCKETS)
System.out.println("<=== ALL DISCONNECTED!");
}
});
socket.connect();
System.out.println("nr " + nr + " connect called");
}
catch (URISyntaxException e)
{
e.printStackTrace();
}
}
});
thread.start();
}
}
This is example output(shortened), waiting 400ms inside the loop:
Generated all sockets
nr 0 creating socket
nr 0 connect called
nr 1 creating socket
nr 1 connect called
nr 0 connected id:7JLvH0hHNF0pg36mAAW3 concounter:1
nr 1 connected id:5fj3I_bFIa1JeUlXAAW4 concounter:2
nr 2 creating socket
nr 2 connect called
nr 2 connected id:RQTLEjftWna2JPuFAAW5 concounter:3
nr 3 creating socket
nr 3 connect called
nr 3 connected id:dg1xL9ddnLqwAlDsAAW6 concounter:4
nr 4 creating socket
nr 4 connect called
nr 4 connected id:y_zIvI4BXdhmEiuwAAW7 concounter:5
nr 5 creating socket
nr 5 connect called
nr 6 creating socket
nr 6 connect called
...
nr 25 creating socket
nr 25 connect called
nr 26 creating socket
nr 26 connect called
nr 0 disconnected. discounter=1
nr 1 disconnected. discounter=2
nr 5 connected id:zCoCg1qG1vJA7pezAAW8 concounter:6
nr 6 connected id:QZJA3yhcXzpRzCwgAAW9 concounter:7
nr 7 connected id:aNZMGdiY8bTeylz3AAW- concounter:8
nr 8 connected id:vitG7xSlEXO5AhnoAAW_ concounter:9
nr 9 connected id:kWirqWwxE5V4ITRiAAXA concounter:10
nr 10 connected id:gCbDdV62pzPRq71qAAXB concounter:11
nr 11 connected id:4ERh1JvC654ky96AAAXC concounter:12
nr 12 connected id:4QMQni7Ohjk0IO7XAAXD concounter:13
nr 27 creating socket
nr 27 connect called
nr 13 connected id:Xb8i-VeDaE_G9N2PAAXE concounter:14
nr 14 connected id:AccfpvKWkWoGY7TEAAXF concounter:15
example output without waiting (commented out the sleep(400) (shortened):
Generated all sockets
nr 0 creating socket
nr 1 creating socket
...
nr 97 creating socket
nr 99 creating socket
nr 24 connect called
nr 21 connect called
nr 52 connect called
...
nr 78 connect called
nr 18 connect called
nr 24 connected id:N4MyGCp4IIWwMtJhAAXV concounter:1
nr 85 connected id:0ahc2QOlGpzPqUwjAAXW concounter:2
nr 68 connected id:W280V6PpH-gUxogOAAXX concounter:3
...
nr 38 connected id:_4c8Ll0cCM_1oQYzAAY2 concounter:98
nr 11 connected id:lIyRiuxK8pmb9voAAAY3 concounter:99
nr 12 connected id:pZkW0Y5DxXgw-Sy6AAY4 concounter:100
===> ALL connected!
I tried it on 2 different machines. Same behaviour.
My goal was to adjust my stresstests with some waiting, but with this behaviour no testing is possible. Anyone any ideas? Am I doing something significantly wrong?
I'm doing the tests on windows 10.
I also created a jar from it and tested it on a debian system. Same behaviour.
I also created a similar javascript test for doing simultanously over 500 socket-connections - works like a charm.
So is there something strange coded in this socket.io java client library that disallows more than 5 connections simultanously?
I am running a local Yarn Cluster with 8 vCores and 8Gb total memory.
The workflow is as such:
YarnClient submits an app request that starts the AppMaster in a container.
AppMaster start, creates amRMClient and nmClient, register itself to the RM and next it creates 4 container requests for worker threads via amRMClient.addContainerRequest
Even though there are enough resources available containers are not allocated (The callback's function onContainersAllocated is never called). I tried inspecting nodemanager's and resourcemanager's logs and I don't see any line related to the container requests. I followed closely apache docs and can't understand what I`m doing wrong.
For reference here is the AppMaster code:
#Override
public void run() {
Map<String, String> envs = System.getenv();
String containerIdString = envs.get(ApplicationConstants.Environment.CONTAINER_ID.toString());
if (containerIdString == null) {
// container id should always be set in the env by the framework
throw new IllegalArgumentException("ContainerId not set in the environment");
}
ContainerId containerId = ConverterUtils.toContainerId(containerIdString);
ApplicationAttemptId appAttemptID = containerId.getApplicationAttemptId();
LOG.info("Starting AppMaster Client...");
YarnAMRMCallbackHandler amHandler = new YarnAMRMCallbackHandler(allocatedYarnContainers);
// TODO: get heart-beet interval from config instead of 100 default value
amClient = AMRMClientAsync.createAMRMClientAsync(1000, this);
amClient.init(config);
amClient.start();
LOG.info("Starting AppMaster Client OK");
//YarnNMCallbackHandler nmHandler = new YarnNMCallbackHandler();
containerManager = NMClient.createNMClient();
containerManager.init(config);
containerManager.start();
// Get port, ulr information. TODO: get tracking url
String appMasterHostname = NetUtils.getHostname();
String appMasterTrackingUrl = "/progress";
// Register self with ResourceManager. This will start heart-beating to the RM
RegisterApplicationMasterResponse response = null;
LOG.info("Register AppMaster on: " + appMasterHostname + "...");
try {
response = amClient.registerApplicationMaster(appMasterHostname, 0, appMasterTrackingUrl);
} catch (YarnException | IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return;
}
LOG.info("Register AppMaster OK");
// Dump out information about cluster capability as seen by the resource manager
int maxMem = response.getMaximumResourceCapability().getMemory();
LOG.info("Max mem capabililty of resources in this cluster " + maxMem);
int maxVCores = response.getMaximumResourceCapability().getVirtualCores();
LOG.info("Max vcores capabililty of resources in this cluster " + maxVCores);
containerMemory = Integer.parseInt(config.get(YarnConfig.YARN_CONTAINER_MEMORY_MB));
containerCores = Integer.parseInt(config.get(YarnConfig.YARN_CONTAINER_CPU_CORES));
// A resource ask cannot exceed the max.
if (containerMemory > maxMem) {
LOG.info("Container memory specified above max threshold of cluster."
+ " Using max value." + ", specified=" + containerMemory + ", max="
+ maxMem);
containerMemory = maxMem;
}
if (containerCores > maxVCores) {
LOG.info("Container virtual cores specified above max threshold of cluster."
+ " Using max value." + ", specified=" + containerCores + ", max=" + maxVCores);
containerCores = maxVCores;
}
List<Container> previousAMRunningContainers = response.getContainersFromPreviousAttempts();
LOG.info("Received " + previousAMRunningContainers.size()
+ " previous AM's running containers on AM registration.");
for (int i = 0; i < 4; ++i) {
ContainerRequest containerAsk = setupContainerAskForRM();
amClient.addContainerRequest(containerAsk); // NOTHING HAPPENS HERE...
LOG.info("Available resources: " + amClient.getAvailableResources().toString());
}
while(completedYarnContainers != 4) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
LOG.info("Done with allocation!");
}
#Override
public void onContainersAllocated(List<Container> containers) {
LOG.info("Got response from RM for container ask, allocatedCnt=" + containers.size());
for (Container container : containers) {
LOG.info("Allocated yarn container with id: {}" + container.getId());
allocatedYarnContainers.push(container);
// TODO: Launch the container in a thread
}
}
#Override
public void onError(Throwable error) {
LOG.error(error.getMessage());
}
#Override
public float getProgress() {
return (float) completedYarnContainers / allocatedYarnContainers.size();
}
Here is output from jps:
14594 NameNode
15269 DataNode
17975 Jps
14666 ResourceManager
14702 NodeManager
And here is AppMaster log for initialization and 4 container requests:
23:47:09 YarnAppMaster - Starting AppMaster Client OK
23:47:09 YarnAppMaster - Register AppMaster on: andrei-mbp.local/192.168.1.4...
23:47:09 YarnAppMaster - Register AppMaster OK
23:47:09 YarnAppMaster - Max mem capabililty of resources in this cluster 2048
23:47:09 YarnAppMaster - Max vcores capabililty of resources in this cluster 2
23:47:09 YarnAppMaster - Received 0 previous AM's running containers on AM registration.
23:47:11 YarnAppMaster - Requested container ask: Capability[<memory:512, vCores:1>]Priority[0]
23:47:11 YarnAppMaster - Available resources: <memory:7680, vCores:0>
23:47:11 YarnAppMaster - Requested container ask: Capability[<memory:512, vCores:1>]Priority[0]
23:47:11 YarnAppMaster - Available resources: <memory:7680, vCores:0>
23:47:11 YarnAppMaster - Requested container ask: Capability[<memory:512, vCores:1>]Priority[0]
23:47:11 YarnAppMaster - Available resources: <memory:7680, vCores:0>
23:47:11 YarnAppMaster - Requested container ask: Capability[<memory:512, vCores:1>]Priority[0]
23:47:11 YarnAppMaster - Available resources: <memory:7680, vCores:0>
23:47:11 YarnAppMaster - Progress indicator should not be negative
Thanks in advance.
I suspect the problem comes exactly from the negative progress:
23:47:11 YarnAppMaster - Progress indicator should not be negative
Note that, since you are using the AMRMAsyncClient, requests are not made immediately when you call addContainerRequest. There is actually an heartbeat function which is run periodically and it is in this function that allocate is called and the pending requests will be made. The progress value used by this function initially starts at 0 but is updated with the value returned by your handler once a response from the acquire is obtained.
The first acquire is supposedly done right after the register so the getProgress function should be called then and update the existing progress. As it is, your progress will be updated to NaN because, at this time, allocatedYarnContainers will be empty and completedYarnContainers will also be 0 and so your returned progress will be the result of 0/0 which is not defined. It just so happens that when the next allocate checks your progress value, it will fail because NaNs return false in all comparisons and so no other allocate function will actually communicate with the ResourceManager because it quits right at that first step with an exception.
Try changing your progress function to the following:
#Override
public float getProgress() {
return (float) allocatedYarnContainers.size() / 4.0f;
}
(note: copied to StackOverflow for posteriority from here)
Thanks to Alexandre Fonseca for pointing out that getProgress() returns a NaN for division by zero when it's called before the first allocation which makes the ResourceManager to quit immediately with an exception.
Read more about it here.
I am developing eclipse plugin for our organization . We are opening multiple servers[minimum 10 servers] on a user machine using this plugin via eclipse . For starting servers we want port numbers which has been not already binded . For that , I am using serversocket to check this . I think it's a costly operation to open a serversocket object . Internally serversocket will check the port is already binded or not It takes minimum 50 milliseconds . Here is my code to return a free port . Is there any way to find already occupied ports without using OS Commands and opening ServerSocket ?
/**
*Tries 100 times
* #param port
* modes
* 1.increment - 1
* This mode increment the port with your start value . But it's costly operation because each time we open a socket and check the port is free .
* 2.decrement - 2
* Invert of increment.
* 3.random - 3
* Randomly choose based on your starting point
* #return
*/
public static String getDefaultPort(int port , int mode){
int retry = 100;
int random = 3;
int increment = 1;
int decrement = 2;
while(true){
//this is for preventing stack overflow error.
if(retry < 1){ //retries 100 times .
break;
}
if(mode==increment){
port++;
}else if(mode == decrement){
port--;
}else if(mode == random){
port = (int) (port+Math.floor((Math.random()*1000)));
}
if(validate(port+"")){
long end = System.currentTimeMillis();
return port+"";
}
}
return "";
}
public boolean validate(String input) {
boolean status = true;
try {
int port = Integer.parseInt(input);
ServerSocket ss = new ServerSocket(port);
ss.close();
}
catch (Exception e) {
status = false;
}
return status;
}
The quickest way would be to run native netstat command and parse the output. It's available on Windows as well and Linux platform. A typical netstat command output is as follows
Proto Local Address Foreign Address State
TCP MYHOST:8080 MYHOST.mydomain.co.in:0 LISTENING
TCP MYHOST:9090 MYHOST.mydomain.co.in:0 LISTENING
TCP MYHOST:3389 MYHOST.mydomain.co.in:0 LISTENING
TCP MYHOST:7717 MYHOST.mydomain.co.in:0 LISTENING
TCP MYHOST:51114 MYHOST.mydomain.co.in:0 LISTENING
TCP MYHOST:netbios-ssn MYHOST.mydomain.co.in:0 LISTENING
TCP MYHOST:netbios-ssn MYHOST.mydomain.co.in:0 LISTENING
TCP MYHOST:2573 hj-lyncfe.mydomain.co.in:5061 ESTABLISHED
TCP MYHOST:2591 mail.mydomain.co.in:8502 ESTABLISHED
TCP MYHOST:2593 mail.mydomain.co.in:8502 ESTABLISHED
The ports of your interest are in the column Local Address with State in LISTENING
This question already has an answer here:
Closed 12 years ago.
Possible Duplicate:
Why do the outputs differ when I run this code using NetBeans 6.8 and Eclipse?
When I am running the following code using Eclipse and NetBeans 6.8. I want to see the available COM ports on my computer. When running in Eclipse it is returning me all available COM ports, but when running it in NetBeans, it does not seem to find any ports ..
public static void test() {
Enumeration lists=CommPortIdentifier.getPortIdentifiers();
System.out.println(lists.hasMoreElements());
while (lists.hasMoreElements()) {
CommPortIdentifier cn =
(CommPortIdentifier)lists.nextElement();
if ((CommPortIdentifier.PORT_SERIAL==cn.getPortType())) {
System.out.println(
"Name is serail portzzzz " +
cn.getName()+
" Owned status " +
cn.isCurrentlyOwned());
try {
SerialPort port1=(SerialPort)cn.open("ComControl",800000);
port1.setSerialPortParams(
9600,
SerialPort.DATABITS_8,
SerialPort.STOPBITS_1,
SerialPort.PARITY_NONE);
System.out.println("Before get stream");
OutputStream out=port1.getOutputStream();
InputStream input=port1.getInputStream();
System.out.println("Before write");
out.write("AT".getBytes());
System.out.println("After write");
int sample=0;
//while((( sample=input.read())!=-1)){
System.out.println("Before read");
//System.out.println(input.read() + "TEsting ");
//}
System.out.println("After read");
System.out.println(
"Receive timeout is " +
port1.getReceiveTimeout());
}
catch(Exception e) {
System.err.println(e.getMessage());
}
}
else {
System.out.println(
"Name is parallel portzzzz " +
cn.getName() +
" Owned status " +
cn.isCurrentlyOwned() +
cn.getPortType() +
" ");
}
}
}
Output with Netbeans,
false
Output using Eclipse,
true
Name is serail portzzzz COM1 Owned status false
Before get stream
Before write
After write
Before read
After read
Receive timeout is -1
Name is serail portzzzz COM2 Owned status false
Before get stream
Before write
After write
Before read
After read
Receive timeout is -1
Name is parallel portzzzz LPT1 Owned status false2
Name is parallel portzzzz LPT2 Owned status false2
An initial guess would be that the library you use use native code enclosed in a DLL and that code cannot be found giving an error earlier you have missed, and the code falls back to a dummy behaviour.
I would have a closer look at the initialization code to see what happens there.
I'm using a Java socket client. In a case where the server is still connected to my client but it does not send a response to my message - I eventually get a read time out exception.
In that case I want to test to see if I should reconnect my socket or just keep it an re-use it.
I use this condition:
if (!socket.isConnected() || socket.isClosed() || !socket.isBound()) {
try {
socket.close();
} catch (IOException e1) {
}
// Wait on a new connection
socket = connectSocket(.....);
}
But I always seem to reconnect. When I log the values of the Boolean properties I see this:
connected: true closed: true bound: true
How can it be connected and closed?
TIA
This thread has some useful discussions on this topic. It turns out that Socket.isConnected returns true if it has (ever) been successfully connected.
From the above thread:
When you use Socket(), which you seem to have overlooked,
Socket.isConnected() tells you whether Socket.connect() has been called
or not. Similarly for isClosed() and close().
Confusion over these methods results from confusing the state of the
socket, which is under the control of the application, with the state
of the overall connection, which is under the control of the protocol.
isConnected() and isClosed() tell what you have done to the socket.
There are no APIs other than read and write for determining the state of
the connection.
The docs says:
Returns true if the socket successfuly connected to a server
and not as one perhaps would expect "returns true if the socket is connected to a server".
The behavior can be confirmed by looking at the source of Socket:
public boolean isConnected() {
// Before 1.3 Sockets were always connected during creation
return connected || oldImpl;
}
You could also run this little test snippet:
Socket s = new Socket();
System.out.println("isConnected: " + s.isConnected() +
" isBound: " + s.isBound() +
" isClosed: " + s.isClosed());
s.connect(new InetSocketAddress("google.com", 80));
System.out.println("isConnected: " + s.isConnected() +
" isBound: " + s.isBound() +
" isClosed: " + s.isClosed());
s.close();
System.out.println("isConnected: " + s.isConnected() +
" isBound: " + s.isBound() +
" isClosed: " + s.isClosed());
Which prints:
isConnected: false isBound: false isClosed: false
isConnected: true isBound: true isClosed: false
isConnected: true isBound: true isClosed: true
I must say that the documentation is quite unclear on this point, and that the method-name is a bit misleading.