I am trying to attach a new volume to my instance using JClouds. But I can't find a way to do it.
final String POLL_PERIOD_TWENTY_SECONDS = String.valueOf(SECONDS.toMillis(20));
Properties overrides = new Properties();
overrides.setProperty(ComputeServiceProperties.POLL_INITIAL_PERIOD, POLL_PERIOD_TWENTY_SECONDS);
overrides.setProperty(ComputeServiceProperties.POLL_MAX_PERIOD, POLL_PERIOD_TWENTY_SECONDS);
Iterable<Module> modules = ImmutableSet.<Module> of(new SshjSshClientModule(), new SLF4JLoggingModule());
//Iterable<Module> modules = ImmutableSet.<Module> of(new SshjSshClientModule());
ComputeServiceContext context = ContextBuilder.newBuilder("aws-ec2")
.credentials("valid user", "valid password")
.modules(modules)
.overrides(overrides)
.buildView(ComputeServiceContext.class);
ComputeService computeService = context.getComputeService();
// Ubuntu AMI
Template template = computeService.templateBuilder()
.locationId("us-east-1")
.imageId("us-east-1/ami-7c807d14")
.hardwareId("t1.micro")
.build();
// This line creates the volume but does not attach it
template.getOptions().as(EC2TemplateOptions.class).mapNewVolumeToDeviceName("/dev/sdm", 100, true);
Set<? extends NodeMetadata> nodes = computeService.createNodesInGroup("m456", 1, template);
for (NodeMetadata nodeMetadata : nodes) {
String publicAddress = nodeMetadata.getPublicAddresses().iterator().next();
//String privateAddress = nodeMetadata.getPrivateAddresses().iterator().next();
String username = nodeMetadata.getCredentials().getUser();
String password = nodeMetadata.getCredentials().getPassword();
// [...]
System.out.println(String.format("ssh %s#%s %s", username, publicAddress, password));
System.out.println(nodeMetadata.getCredentials().getPrivateKey());
}
How can I create an attach a volume to the directory "/var" ?
How could I create the instance with more hard disk space ?
You can use the ElasticBlockStoreApi in jclouds for this. You have created the instance already, so create and attach the volume like this:
// Get node
NodeMetadata node = Iterables.getOnlyElement(nodes);
// Get AWS EC2 API
EC2Api ec2Api = computeServiceContext.unwrapApi(EC2Api.class);
// Create 100 GiB Volume
Volume volume = ec2Api.getElasticBlockStoreApi().get()
.createVolumeInAvailabilityZone(zoneId, 100);
// Attach to instance
Attachment attachment = ec2Api.getElasticBlockStoreApi().get()
.attachVolumeInRegion(region, volume.getId(), node.getId(), "/dev/sdx");
Now, you have an EBS volume attached, and the VM has a block device created for it, you just need to run the right commands to mount it on the /var directory, which will depend on your particular operating system. You can run a script like this:
// Run script on instance
computeService.runScriptOnNode(node.getId(),
Statements.exec("mount /dev/sdx /var"));
Related
My Java program needs to get a list of the drive letters of connected USB devices, but only those that support USB 3.0 (both the device and the USB port it is plugged into, so that it works with high speed).
Currently I try to use WMI through a PowerShell command my Java programm executes.
I already found this: Powershell: Grab USB Drive letter. But it would also list USB 2.0 devices.
Regarding version detection I found this: How to check the version of the available USB ports? - The PowerShell command I tried is Get-WmiObject Win32_USBHub. This brings up several problems. First: It lists far more stuff than only USB drives (I think also all the USB hubs of my PC). Second: Even though there is a field USBVersion for all items in the list it is always empty.
Update
The essence of my research over the last days is, that there are 2 realms of information I need to connect.
Drives / Logical Drives
Drive Letter
BusType (is equal to "USB" for my matter)
USB devices
Vendor ID and Product ID (VID&PID)
bcdUSB (value within the usb device descriptor, indicating USB Version)
For a given drive letter I need to find the bcdUSB value. But I haven't found a way to get the drive corresponding to a USB device.
What I tried so far
WMI over PowerShell
Relevant commands I found are
Get-Disk // Get BusType
gwmi Win32_LogicalDisk // Get drive letter
// Those make the connection between disk and logical disk
gwmi Win32_LogicalDiskToPartition
gwmi Win32_LogicalDiskToPartition
Even though I get the BusType I couldn't make a connection to bcdUSB
usb4java (Link)
I only get information from the USB device realm here. I can load devices and see ther VID&PID and the bcdUSB value, but no way to map this to drives and drive letters.
lsusb via Cygwin
According to this post the linux command is easier to handle than WMI. So I tried to use it under Windows. But I like usb4java I only got VID&PID + bcdUSB, not the mount point (drive letter).
Searching the Windows Registry
I did a few string searchs in the Windows registry. No success.
Reading Windows Event log
I thought about ovserving Windows events to detect what Drive and what USB device connect at the same time. I didn't even find events when plugging in a USB stick.
Maybe this is what you are looking for:
Find Windows Drive Letter of a removable disk from USB VID/PID
At least someone marked the answer as working... :-)
Since the suggested Link solves this problem for C# not Java and leaves out one step, I'll post my final code here.
Summary
In Java
Use USB4Java to find all connected USB devices with bcdUSB=0x0300
Get Vendor ID and Product ID (VID&PID) for that devices
Via Powershell (with jPowerShell)
Get PnPEntity for given VID&PID
Get related USB Controller
Find associator of that USB Controller that is associated with a disk drive
Get that Disk drive
Get related disk partition
Get related logical disk -> LogicalDisk.DeviceID = Drive Letter
Code
Java class:
class UsbDetector {
private PowerShell shell;
#PostConstruct
private void init() {
shell = com.profesorfalken.jpowershell.PowerShell.openSession();
}
#OnDestroy
private void onShutdownHook() {
shell.close();
}
/**
* Get drive letters of USB 3.0 devices.
*/
public List<String> getDriveLettersForUsb3Devices() throws IOException, UsbException {
List<UsbDevice> devicesUSB3 = getAllUsb3Devices();
ImmutableList.Builder<String> driveLetterList = ImmutableList.builder();
for (UsbDevice device : devicesUSB3) {
String vidAndPid = getVidAndPid(device);
String powerShellScript = buildScript(vidAndPid);
String driveLetter = executeOnPowerShell(powerShellScript);
driveLetterList.add(driveLetter);
}
return driveLetterList.build();
}
private String executeOnPowerShell(String powerShellScript) {
InputStream psScriptStream = new ByteArrayInputStream(powerShellScript.getBytes());
BufferedReader psScriptReader = new BufferedReader(new InputStreamReader(psScriptStream));
PowerShellResponse response = shell.executeScript(psScriptReader);
return response.getCommandOutput();
}
private String buildScript(String vidAndPid) throws IOException {
InputStream psScriptStream =
getClass().getClassLoader().getResourceAsStream("GetUsbDrives.ps1");
String psScript = IOUtil.toString(psScriptStream);
psScript = String.format("$input=\"%s\"", vidAndPid) + "\n" + psScript;
return psScript;
}
/**
* The Vendor ID and Product ID are necessary to find the device via WMI.
*/
private String getVidAndPid(UsbDevice device) {
short vendorId = device.getUsbDeviceDescriptor().idVendor();
short productId = device.getUsbDeviceDescriptor().idProduct();
String vendorIdHexString = String.format("%04x", vendorId).toUpperCase();
String productIdHexString = String.format("%04x", productId).toUpperCase();
String vidAndPid = String.format("VID_%s&PID_%s", vendorIdHexString, productIdHexString);
return vidAndPid;
}
/**
* From all Usb devices find those with USB 3.0. The value bcdUsb is a hexadecimal coded number
* telling us the USB version.
*/
private List<UsbDevice> getAllUsb3Devices() throws UsbException {
List<UsbDevice> devicesUSB3 = Lists.newArrayList();
UsbServices services = new org.usb4java.javax.Services();
UsbHub hub = services.getRootUsbHub();
List<UsbDevice> devices = getAllUsbDevices(hub);
for (UsbDevice device : devices) {
UsbDeviceDescriptor descriptor = device.getUsbDeviceDescriptor();
short bcdUsb = descriptor.bcdUSB();
String bcdDecoded = DescriptorUtils.decodeBCD(bcdUsb);
if (Objects.equal(bcdDecoded, "3.00")) {
devicesUSB3.add(device);
}
}
return devicesUSB3;
}
/**
* UsbHubs can either mount UsbDevices or further UsbHubs. This method searches through the tree
* of UsbHubs for UsbDevices and returns them as list.
*/
private List<UsbDevice> getAllUsbDevices(UsbHub hub) {
List<UsbDevice> devices = Lists.newArrayList();
List<UsbDevice> attachedDevices = hub.getAttachedUsbDevices();
for (UsbDevice device : attachedDevices) {
if (device instanceof UsbHub) {
List<UsbDevice> subdevices = getAllUsbDevices((UsbHub) device);
devices.addAll(subdevices);
} else {
devices.add(device);
}
}
return devices;
}
}
PowerShell script:
# $input = "VID_XXXX&PID_XXXX (this line is added in Java Code)
# For given VID and PID of a USB device we search for
# the corresponding logical disk to get the drive letter.
# The chain of objects is:
# PnPEntity (PnP = Plug and Play)
# -> USBController
# -> Some associator of USBController that has a related disk drive
# -> diskDrive
# -> diskPartition
# -> logicalDisk
# Find PnPEntity for given VID and PID
$usbPnPEntity = (gwmi Win32_PnPEntity | where DeviceID -match $input)
# Get USB Controller related to PnP Entity
$usbController = $usbPnPEntity.getRelated("Win32_USBController")
$usbControllerID = $usbController.DeviceID
# Find objects associated with the USB Controller
$query = "ASSOCIATORS OF {Win32_USBController.DeviceID='$usbControllerID'}"
$associators = ([wmisearcher]$query).get()
# Search through associators
foreach ($associator in $associators) {
# Find associator that is related to a disk Drive
$assoDeviceID = $associator.DeviceID
$diskDrive = (gwmi win32_diskdrive | where PNPDeviceID -eq $assoDeviceID)
if($diskDrive){
# Get logical Disk related to the disk drive
$logicalDisk = $diskDrive.getRelated("Win32_DiskPartition").getRelated("Win32_LogicalDisk")
# Print device ID which is the drive letter (e.g. "C:")
$logicalDisk.DeviceID
break
}
}
Maven dependencies:
<dependency>
<groupId>org.usb4java</groupId>
<artifactId>usb4java-javax</artifactId>
<version>1.3.0</version>
</dependency>
<dependency>
<groupId>com.profesorfalken</groupId>
<artifactId>jPowerShell</artifactId>
<version>3.1.1</version>
</dependency>
How I would like to change these parameters from eclipse (not using command line):
jade_core_messaging_MessageManager_poolsize
jade_core_messaging_MessageManager_maxqueuesize
jade_core_messaging_MessageManager_deliverytimethreshold
The first changes the number of threads that handle the queue size, the second changes the max queue size of the received ACL messages, and finally the last changes when to print a warning when the delivery time threshold is exceeded.
bests,
If you start your container and agents programmatically, then something like this
jade.core.Runtime rt = jade.core.Runtime.instance();
Properties properties = new Properties();
properties.put("local-port", "8858");
properties.put("port", "8858");
properties.put("host", "127.0.0.1");
properties.put("local-host", "127.0.0.1");
.... other parameters
properties.put("jade_core_messaging_MessageManager_poolsize", "100");
ProfileImpl p = new ProfileImpl(properties);
rt.setCloseVM(true);
AgentContainer agentContainer = rt.createMainContainer(p);
AgentController ac = agentContainer.createNewAgent("YourAgent", YourAgent.class.getName(), new Object[]{});
ac.start();
I am trying to create an EMR cluster on java, but i can't neither find it on the EMR cluster list, neither can see the instances requested on EC2.
EMR roles do exist:
sqlInjection#VirtualBox:~$ aws iam list-roles | grep EMR
"RoleName": "EMR_DefaultRole",
"Arn": "arn:aws:iam::removed:role/EMR_DefaultRole"
"RoleName": "EMR_EC2_DefaultRole",
"Arn": "arn:aws:iam::removed:role/EMR_EC2_DefaultRole"
and now my java code:
AWSCredentials awsCredentials = new BasicAWSCredentials(awsKey, awsKeySecret);
AmazonElasticMapReduce emr = new AmazonElasticMapReduceClient(awsCredentials);
StepFactory stepFactory = new StepFactory();
StepConfig enabledebugging = new StepConfig()
.withName("Enable debugging")
.withActionOnFailure("TERMINATE_JOB_FLOW")
.withHadoopJarStep(stepFactory.newEnableDebuggingStep());
HadoopJarStepConfig hadoopConfig1 = new HadoopJarStepConfig()
.withJar("s3://foo.bucket/hadoop_jar/2015-01-12/foo.jar")
.withMainClass("com.strackoverflow.DriverFoo") // optional main class, this can be omitted if jar above has a manifest
.withArgs("--input=s3://foo.bucket/logs/,s3://foo.bucket/morelogs/", "--output=s3://foo.bucket/myEMROutput" , "--inputType=text"); // i have custom java code to handle the --input, --output and --inputType parameters
StepConfig customStep = new StepConfig("Step1", hadoopConfig1);
Collection <StepConfig> steps = new ArrayList<StepConfig>();
{
steps.add(enabledebugging);
steps.add(customStep);
}
JobFlowInstancesConfig instancesConfig = new JobFlowInstancesConfig()
.withEc2KeyName("fookey") //not fookey.pem
.withInstanceCount(2)
.withKeepJobFlowAliveWhenNoSteps(false) // on aws example is set to true
.withMasterInstanceType("m1.medium")
.withSlaveInstanceType("m1.medium");
RunJobFlowRequest request = new RunJobFlowRequest()
.withName("java programatic request")
.withAmiVersion("3.3.1")
.withSteps(steps) // on the amazon example is lunched debug and hive, here is debug and a jar
.withLogUri("s3://devel.rui/emr_clusters/pr01/")
.withInstances(instancesConfig)
.withVisibleToAllUsers(true);
RunJobFlowResult result = emr.runJobFlow(request);
System.out.println("toString "+ result.toString());
System.out.println("getJobFlowId "+ result.getJobFlowId());
System.out.println("hashCode "+ result.hashCode());
Where is my cluster? I cannot see it on cluster list, output folder is not created, logs folder stays empty and no instances are visible on EC2.
by the program outputs this
toString {JobFlowId: j-2xxxxxxU}
getJobFlowId j-2xxxxxU
hashCode -1xxxxx4
I had follow the instruction from here to create the cluster
http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/calling-emr-with-java-sdk.html
And this to create the java job
http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-common-programming-sample.html
On the Amazon example, the region is not configured.
After configuring the region the cluster is launched properly.
AmazonElasticMapReduce emr = new AmazonElasticMapReduceClient(awsCredentials);
emr.setRegion(Region.getRegion(Regions.EU_WEST_1));
I am creating JmDNS service from other machine running windows-xp by
JmDNS dns = JmDNS.create("localhost");
dns.regesterService("_sreviceTest._tcp.local.", "Test-Service", 8765, "Description");
If I run other client which will resolve services, Created by JmDNS that finds it, irrespective of machine. But if i try to find same service through avahi-browse. It couldn't find it. and give following output.
avahi-browse -rv _sreviceTest._tcp
Server version: avahi 0.6.30; Host name: ubuntu.local
E Ifce Prot Name Type Domain
+ eth0 IPv4 Test-Service _sreviceTest._tcp local
Failed to resolve service 'Test-Service' of type '_serviceTest._tcp' in domain 'local': Timeout reached
It is a bug in JmDNS version 3.4.1 library. See for detail BUG
this.service_type = "_ros-master._tcp.local.";
this.service_name = "RosMaster";
this.service_port = 8888;
String service_key = "description"; // Max 9 chars
String service_text = "Hypothetical ros master";
HashMap<String, byte[]> properties = new HashMap<String, byte[]>();
properties.put(service_key, text.getBytes());
// service_info = ServiceInfo.create(service_type, service_name, service_port); // case 1, no text
// service_info = ServiceInfo.create(service_type, service_name, service_port, 0, 0, true, service_text); // case 2, textual description
service_info = ServiceInfo.create(service_type, service_name, service_port, 0, 0, true, properties); // case 3, properties assigned textual description
jmdns.registerService(service_info);
Case 1 and case 2 create services detectable by avahi, but they fail to resolve.
Case 3 works fine.
Am trying to convert a VBScript to java using JACOB - Java COM bridge library.
'Create' method in VBScript accepts a [out] param in it's method and it sets it upon method execution and i couldn't figure out how to retrieve it back via JACOB.
VBScript in question:
Function CreateProcess(strComputer, strCommand)
Dim objWMIService, objProcess
Set objWMIService = GetObject("winmgmts:" & "{impersonationLevel=impersonate}!\\" & strComputer & "\root\cimv2")
Set objProcess = objWMIService.Get("Win32_Process")
errReturn = objProcess.Create (strCommand, Null, Null, intProcessID)
Set objWMIService = Nothing
Set objProcess = Nothing
CreateProcess = intProcessID
End Function
intProcessID is [out] param set after method execution. (Create API contract)
Converted java code(incomplete and modified slightly for demonstration):
public static void createProcess() {
String host = "localhost";
String connectStr = String
.format("winmgmts:{impersonationLevel=impersonate}!\\\\%s\\root\\CIMV2",
host);
ActiveXComponent axWMI = new ActiveXComponent(connectStr);
Variant vCollection = axWMI.invoke("get", new Variant("Win32_Process"));
Dispatch d = vCollection.toDispatch();
Integer processId = null;
int result = Dispatch.call(d, "Create", "notepad.exe", null, null, processId)
.toInt();
System.out.println("Result:" + result);
// WORKS FINE until here i.e. notepad launches properly, however processId still seems to be null. Following commented code is wrong - doesn't work
//Variant v = Dispatch.get(d, "processId"); // even ProcessId doesn't work
//int pId = v.getInt();
//System.out.println("process id:"
// + pId);
// what is the right way to get the process ID set by 'Create' method?
}
Would be great if you could provide some pointers or relevant code. Ask me more if needed. Thanks in advance.
Replacing
Integer processId = null;
with
Variant processId = new Variant(0, true);
should solve the problem. You should then have process ID of the notepad.exe process in the processId variant, and it can be fetched by
processId.getIntRef()