I am a newbie to Gridgain and i would like to know how do i add remote nodes to a program. is there some configuration file. i dont see a clear cut example anywhere in the guides. (the worst guide i've ever seen)
By default, GridGain uses multicast discovery, so, if multicast is working in your network, nodes should find each other automatically.
You can configure an alternative multicast group address, if default settings do not work.
You can also configure TCP discovery with a list of IP addresses and ports, where nodes can start. This gives you more control over the discovery process, and is also a good alternative, if multicast discovery does not work.
There are also other means of discovery (including Shared FS and Amazon S3). Check GRIDGAIN_HOME/config/default-spring.xml for examples (search for "discovery"). Also, have a look at GRIDGAIN_HOME/examples/config/spring-cache.xml.
Please ensure you start all the nodes with the same configuration ("localHost" property may differ).
There is work-in-progress book online: http://www.gridgain.com/book/book.html#_taste_of_gridgain
That should answer these basic questions.
Related
I'm trying to implement fully-automatic PCP (according to https://www.rfc-editor.org/rfc/rfc6887) for a decentralized P2P chat system.
That requires PCP configurations to be made, and so far I haven't found anyone having a library or framework for that. Now I'm trying to make my own, and here's the first hurdle:
How do I find the current system's Default Gateway IP on multiple OS types and if possible both IPv4 and IPv6.
I made myself a simple way of receiving and interpreting commands to the CLI of the host OS using
Runtime.getRuntime().exec("[COMMAND LINE]");
as Java itself seems to provide no such information, of if I don't know where.
The current plan would involve getting the ipconfig and ifconfig returns on Windows and Linux (for MAC i'm sure there is a similar command), then search for the keyword of "gateway" and find the next IPv4 and IPv6 addresses, then see if anyone answers there on UDP port 5351 after a few attempts. I'm fairly certain it could work, but before using try-and-error methods I want to look for a more professional solution.
On top of that it would be good having a way to find out if the system is inside a NAT. While it usually isn't a problem with the private A, B and C networks there always is the possibility of custom settings on the router like more configured private networks requiring NAT.
I'm new-ish to networking, and I'm swimming (drowning) in semantics.
I have a VM which runs a Java application. Ideally, it would be fed inputs from the host through a RabbitMQ queue. The Java application would then place the results on another RabbitMQ queue on a different port where it will be used by the host application. After researching it for a bit, it seems like RabbitMQ only exists in the localhost space with listeners on different ports, am I correct in this?
Do I need 2 RabbitMQ servers running in tandem, then, (one on the VM and other on Host) each listening to the same port? Or do I just need one RabbitMQ server running while both applications are pointed to the same IP Address/Port?
Also, I have also read that you cannot connect as 'guest/guest' unless it is on localhost, which I understand, but how is RabbitMQ supposed to be configured/reachable to anything besides localhost?
I've been researching for several hours, but the documentation does not point to a direct answer/how-to guide. Perhaps it is my lack of network experience. If anyone could elaborate on these questions or point me to some articles/helpful guides, I would be much obliged.
P.S. -- I don't even know what code to display to give context. Let me know and I'll edit the code into the post.
RabbitMQ listens to TCP port 5672 on all network interfaces out-of-the-box. This includes the "loopback" interface (to allow fast connections to self) and interfaces visible to other remote hosts (including VMs).
For your use case, you probably need a single RabbitMQ instance for both directions. The application on the host will publish messages to one queue and the Java application in the VM will consume messages from that queue and push the result to a second queue. This second queue can be consumed by the application on the host.
For the user, you need to create a new user with the appropriate rights. This is documented in the access control article. To create the user, you can do it from the management web UI (after you enabled the management plugin) or using the rabbitmqctl command line tool.
The last part is networking between the host and the VM. It really depends on the technology you use. It may work out-of-the-box or you may have to configure how VMs are connected to the network. Refer to the documentation of your hypervisor.
i am working on apache storm which has a topolgy main class. This topology contains the kafkaSpout which listen a kafka topic over a kafka broker. Now before i submit this topology i want to make sure the status of the kafka broker which has the topic. But i didnt found any way to do it? How a kafka brokers status can be known from storm tolopogy class ? Please help...
If you simply want a quick way to know if it is running or not you can just run the start command again with the same config:
bin/kafka-server-start.sh config/server.properties
If it's running then you should get an exception about the port already being in use.
Not foolproof, so better would be to use Zookeeper as mentioned above:
Personally I use intellij which has a Zookeeper plugin which helps you browse the brokers/topics running within it. Probably something similar for Eclipse or other IDEs.
(IntelliJ)
Go to File > Settings > type zookeeper in the search, then install and click ok (may need to restart)
Go to File > Settings > type zookeeper in the search. Click enable then put in the address where your zookeeper server is running and apply changes. (Note you may need to check the port is correct too)
You should now see your zookeeper server as a tab on the left side of the IDE.
This should show you your broker and topics, consumers etc
Hope that helps!
If you have configured storm-ui, then that should give you a brief information about the running cluster, including informations such as currently running topologies, available free slots, supervisor info etc.
Programitically you can write a thrift client to retrieve those information from the storm cluster. You can possibly choose almost any language to develope your own client.
Check out this article for further reference
Depending on what kind of status you want to have, for most cases you would actually retrieve this from Zookeeper. In Zookeeper you can see registered brokers, topics and other useful things which might be what you're looking for.
Another solution would be to deploy a small regular consumer which would be able to perform those checks for you.
I was wondering if someone could give me a clue. The situation is we have one computer that has a server application, the rest have a client application. Now my question is how would i keep track of the server ip address? What I want is have the clients automatically hunt the server application, regardless of the ip address and find it, because using a wireless network computers are using DHCP. So I was wondering is there anything that can be buried in a computer using Java with which other computers can identify the application. Thanks.
The "good" way is to use DNS (imho the ONLY solution in a productive environment), as #Peter Lawrey already mentioned, and i would prefer that solution.
You have two other possibilities:
Let the server broadcast his address on the network.
Let the clients scan the network.
Both this solutions are not ideal, because you create a lot of noise on the network, may be blocked by a firewall, etc. Therefore, i really suggest you to use DNS.
If you can not use the DNS solution i would go with a solution that lets the clients search for the server.
Take a look at this SO question, might get you started.
If they're on the same physical network, your server can broadcast its IP address via UDP to the broadcast address for that sub net.
You could even follow the uPnP standard.
Alternatively, you could host the location of the server from a known other location, like a web server. For example, call an action on the known web server to retrieve the IP address of the server. This allows you to utilize DNS to look up the first host, then go from there.
If you know the DNS-Name of the server you can use that name instead of the IP. Java takes care of the DNS lookup.
Otherwise, I guess the simplest solution is just to scan the IP range of your network and just try to connect to each IP and test if it works.
Other solutions would include broadcasts by the server and would be much more complicated and maybe overkill for a application hosted in a small network.
I have been given a project in which I have to share files with peers without any intermediate server. I want to know how to know how many hosts are online on the network and how to connect to them. I have to roughly make Routing table for the hosts in my computer through Java.
You need not to keep any track of routing. All you need is to know the endpoints addresses.
You may find out if a host is connected to the network by establishing a connection with this host.
You mentioned that there will be no intermediate server, so the NAT hole punching is out of scope of your question.
As the starting point look at java.net.Socket class documentation.
I'm going to assume that this is on a private network. Something that's relatively contained. Letting them find each other over the internet sounds like a nightmare.
So, given this, one model for your peers to find each other might be to select a standardised sequence of ports and an alive signal. Then when a peer searches for others, it simply goes through all the IPs on the network and makes a request on that port number. If it gets an alive signal it adds it to a table. The signalled computer would have to record the new comer as well.
You'd have to select the port sequence such that they're generally unused on the network. (Quite large) I say sequence so that there's a preference in ordering to speed up the search.
To speed up the search even more, when an alive signal is sent, it could also send it's peers table and the IPs it checked (and the ones it received in the same way). Then the new peer would only have to check new IPs on the network since old IPs with a new instance of the program would establish themselves.
Hope that helps
i'd suggest you not to go with mac address,it will be better if you turn your dhcp off of your router (if you can include one of course........but if you only can include an hub you'll be forced i guess to use mac address).and use manual addressing for your ipv4 address,and you will be able to easily connect to the other computers.