In ruby I have:
PTY.spawn("/usr/bin/lxc-monitor -n .+") do |i, o, pid|
# ...
end
How do this in scala/java?
Try JPty or pty4j. These are implementations of pty for Java using JNA.
I don't think that PTY has been ported to java/scala. You can use the built in Runtime from java.
def run() {
val rt = Runtime.getRuntime
val cmds = Array("/usr/bin/lxc-monitor", "-n .+")
val env = Array("TERM=VT100")
val p1 = rt.exec(cmds, env)
}
I used this page as the base for the scala version.
UPDATE:
To get the output you need to get the input stream and read it (I know this sounds backwards but it's input relative to the jvm). The example below uses apache commons to skip some verbose parts of java.
import java.io.StringWriter
import org.apache.commons.io.IOUtils
class runner {
def run() {
val rt = Runtime.getRuntime
val cmds = Array("/usr/bin/lxc-monitor", "-n .+")
val env = Array("TERM=VT100")
val p1 = rt.exec(cmds, env)
val inputStream = p1.getInputStream
val writer = new StringWriter()
IOUtils.copy(inputStream, writer, "UTF-8")
val output = writer.toString()
println(output)
}
}
I got the apache utils idea from here.
Related
I'm having some difficulty in determining the means for loading an "Avro Tools" class and its run method. The issue is somewhere between java and scala interfacing and class loading methods. Due to the fact that avro is used elsewhere in a Spark app with a different version for loading data files, I need to be able to treat this particular method as a siloed call to another version of avro-tools.
The following is my code:
package samples
import java.io.{ByteArrayOutputStream, InputStream}
import org.junit.runner.RunWith
import org.specs2.mutable._
import org.specs2.runner._
import scala.collection.JavaConverters._
#RunWith(classOf[JUnitRunner])
class MySpecTest extends Specification {
"Class Loader" should {
"load an implement a class" in {
var classLoader = new java.net.URLClassLoader(
Array(new java.io.File("./avro-tools-1.9.1.jar").toURI.toURL),
this.getClass.getClassLoader)
var clazzDFRT = classLoader.loadClass("org.apache.avro.tool.DataFileRepairTool")
val objDFRT = clazzDFRT.getConstructor().newInstance()
val toolCmdArgsAsJava = List("-o", "all", "questionable.avro", "fixed.avro").asJava
val stdin : InputStream = null
val out: ByteArrayOutputStream = new ByteArrayOutputStream
val stdout = new PrintStream(out) // added stdout in edit#1
val err = System.err
val toolClassArgsAsJava = List(stdin, stdout, // changed out to stdout in edit#1
err, toolCmdArgsAsJava).asJava
// parameterTypes: Class[_] *
// public int run( InputStream stdin, PrintStream out, PrintStream err, List<String> args)
val paramClasses: Array[Class[_]] = Array(classOf[InputStream], classOf[PrintStream], classOf[PrintStream], classOf[java.util.List[_]])
val method = clazzDFRT.getMethod("run", paramClasses : _*)
// the following produces wrong number of arguments exception
method.invoke(objDFRT.asInstanceOf[Object], toolClassArgsAsJava)
// sidebar: is this the end result for the Unit test - want out str with summary
out.toString("UTF-8").contains("File Summary")
}
}
}
I seem to have some issue in the invoke method part, but maybe the whole solution is a little off - I need to be able to invoke the method as well as load, instantiate or ...
How can I fix this to run the entire code segment (and repair a broken avro)?
It is hard to tell the exact nature of the problem since you didn't include the exception or stack trace. I am not sure why you are loading the avro tools dynamically instead of including the jar statically as part of your build.
// public int run( InputStream stdin, PrintStream out, PrintStream err, List<String> args)
val method = clazzDFRT.getMethod("run", Class[_] : _*)
You are not specifying the parameters correctly.
method.invoke(objDFRT.asInstanceOf[Object], toolClassArgsAsJava)
val params: Array[Class[_]] = Array(classOf[InputStream], classOf[PrintStream], classOf[PrintStream], classOf[java.util.List[_]])
val method = clazzDFRT.getMethod("run", params : _*)
or
val method = clazzDFRT.getMethod("run", classOf[InputStream], classOf[PrintStream], classOf[PrintStream], classOf[java.util.List[_]])
To fix the invoke, you cannot pass the parameters in a list. The invoke method takes variable arguments, you need to pass these in directly.
method.invoke(objDFRT.asInstanceOf[Object], stdin, stdout, stderr, toolCmdArgsAsJava)
or
method.invoke(objDFRT.asInstanceOf[Object], Array(stdin, stdout, stderr, toolCmdArgsAsJava): _*)
Notice the second option uses an Array not a List.
I suggest you read up on the documentation for using var args in Java and Scala
* https://docs.oracle.com/javase/8/docs/technotes/guides/language/varargs.html
* http://daily-scala.blogspot.com/2009/11/varargs.html
I am comparing the memory consumption of two implementations of an API in a big system. In order to minimize impact from other parts of the system, I run the function in a Process started by ProcessBuilder, connect to the new Process using JMX, then use MemoryMXBean to monitor its memory consumption. Here's the code I use:
val pb = new ProcessBuilder("/usr/bin/java",
"-cp", "/home/ubuntu/main.jar",
"-Xmx8G", "dataset.feature.EncMemoryUsageProcess",
param1, param2)
val process = pb.start()
val pidfield = process.getClass.getDeclaredField("pid")
pidfield.setAccessible(true)
val pid = pidfield.get(process).toString
// Attach VM and obtain MemoryMXBean
val vm = VirtualMachine.attach(pid)
var connectorAddr = vm.getAgentProperties.getProperty("com.sun.management.jmxremote.localConnectorAddress")
if (connectorAddr == null) {
val agent = vm.getSystemProperties.getProperty("java.home") + File.separator + "lib" + File.separator + "management-agent.jar"
vm.loadAgent(agent)
connectorAddr = vm.getAgentProperties.getProperty("com.sun.management.jmxremote.localConnectorAddress")
}
val serviceURL = new JMXServiceURL(connectorAddr)
val connector = JMXConnectorFactory.connect(serviceURL)
val mbsc = connector.getMBeanServerConnection
val mbeanName = new ObjectName(ManagementFactory.MEMORY_MXBEAN_NAME)
val memoryMXBean = JMX.newMXBeanProxy(mbsc, mbeanName, classOf[MemoryMXBean])
var maxMemory = 0l
while (process.isAlive) {
Thread.sleep(200l);
val memoryUsage = memoryMXBean.getHeapMemoryUsage.getUsed
maxMemory = Math.max(memoryUsage, maxMemory);
}
return maxMemory
However, the code throws "com.sun.tools.attach.AttachNotSupportedException: Unable to open socket file: target process not responding or HotSpot VM not loaded" when I call VirtualMachine.attach. I googled and noticed this is generally caused by not having a pid file generated, and one common cause is that the new JVM is not created by the same user. However, this seems like not my case as I am creating the new JVM as a subprocess.
Any suggestion is appreciated. Thanks!
Update
I have noticed that this error only occurred when I start multiple Processes like the following(the code listed above is executeAndAttach):
for(int i = 0 ; i < 5;i++) {
executeAndAttach(i, param);
}
The first time I do executeAndAttach, it will succeed, but subsequent calls all throw the exception. More weird, when I debug in the LinuxVirtualMachine.findSocketFile(), where is the place the exception was thrown out, the error disappears.
I'm aware that Scala uses Java sockets, but I don't quite understand the answers from questions that people have had with the same problem but strictly in Java.
Here is my code:
I am trying to send a jar file through a socket, but when I try to open the jar file from the other side, the file seems to be corrupted. How can I fix this?
Server:
object server extends App {
import java.net._
import java.io._
import scala.io._
import scala.io.Source
val server = new ServerSocket(9999)
//Master should ping the slave actor to request for jar file
while (true) {
val s = server.accept()
val in = new BufferedSource(s.getInputStream()).getLines()
val out = new PrintStream(s.getOutputStream())
val filename = "mapReduce.jar"
for (line <- Source.fromFile(filename, "ISO-8859-1").getLines) {
out.println(line)
// println(line)
}
out.flush()
s.close()
}
}
Along with the Client:
object client extends App {
import java.net._
import java.io._
import scala.io._
import java.util.jar._
val s = new Socket(InetAddress.getByName("localhost"), 9999)
lazy val in = new BufferedSource(s.getInputStream()).getLines()
val out = new PrintStream(s.getOutputStream())
out.println("Give me the jar file!")
out.flush()
val file = new File("testmapReduce.jar")
val bw = new BufferedWriter(new FileWriter(file))
while(in.hasNext) {
val buf = in.next()
bw.write(buf)
// println(buf)
}
s.close()
bw.close()
println("Done!")
val jar = new JarFile(file) //this part fails
}
Source, PrintStream etc, are intended to deal with text, not binary data. They convert the data on both read and write in accordance with the character set they are using ("iso-8859-1" in your case).
Do not use them to read/write binary data.
If you just need to send byte, don't bother with interpreting them:
val f = new FileInputStream(filename)
val bos = new BufferedOutputStream(out)
Stream.continually(f.read).takeWhile(_ != -1).foreach(bos.write)
f.close
bos.close
Wrote a function for reading text from a PDF document.
Used scala language, Selenium, PDFBox 2.0.1.
Below is the code:
enter code here
import org.openqa.selenium.firefox.{FirefoxBinary, FirefoxDriver, FirefoxProfile}
import org.apache.pdfbox.pdfparser.PDFParser
import org.apache.pdfbox.text.PDFTextStripper
import java.io.BufferedInputStream
def pdfreaddata {
driver.get("https://www.....pdf")
driver.manage.timeouts.implicitlyWait(50, TimeUnit.SECONDS)
val url: URL = new URL(driver.getCurrentUrl)
println(url)
val fileToParse: BufferedInputStream = new BufferedInputStream(url.openStream())
val parser: PDFParser = new PDFParser(fileToParse)
parser.parse()
val output: String = new PDFTextStripper().getText(parser.getPDDocument)
println("pdf Value" + output)
parser.getPDDocument.close()
driver.manage.timeouts.implicitlyWait(100, TimeUnit.SECONDS)
}
Showing error for PDFParser in val parser: PDFParser = new PDFParser(fileToParse)
Error message:
Cannot resolve constructor
Tried the code in Java too, getting same error.
You are using PDFBox version 2.x, however you are obviously following the docs for version 1.x . In 2.0 there is no such constructor. Some things have changed, including parsing. Follow the migration guide or fall back to 1.8, since it does look much more documented and with more material online.
Using pdfbox 1.8.12 solved the constructor issue. But even the pdf's was not password protected, it was showing as encrypted. Below is the final code using Scala to extract encrypted text from a pdf document. Might be useful for someone in future.
def pdfreaddata {
driver.get("https://www....combo.pdf")
driver.manage.timeouts.implicitlyWait(50, TimeUnit.SECONDS)
val url: URL = new URL(driver.getCurrentUrl)
println(url)
val fileToParse: BufferedInputStream = new BufferedInputStream(url.openStream())
val parser: PDFParser = new PDFParser(fileToParse)
parser.parse()
val cosDocument:COSDocument = parser.getDocument()
val pdDocument:PDDocument = new PDDocument(cosDocument)
if(pdDocument.isEncrypted()) {
val sdm: StandardDecryptionMaterial = new StandardDecryptionMaterial(PDF_OWNER_PASSWORD)//PDF_OWNER_PASSWORD =""
pdDocument.openProtection(sdm)
}
val output: String = new PDFTextStripper().getText(pdDocument)
println("pdf Value" + output)
parser.getPDDocument.close()
driver.manage.timeouts.implicitlyWait(100, TimeUnit.SECONDS)
}
}
I am evaluating Websphere MQ7. I am a traditionally a TibRV guy. One thing I do not like is the fact that the IBM java client libs require C++ libs in order to run. Is there anyway to run the IBM java client libs without requiring the C++ libs? e.g. is there a pure java client library for MQ ?
I have previously written a JMS client to MQSeries v6 (not your version, I know) without needing to install native libs. The only IBM libraries I required were titled:
com.ibm.mq-6.jar
com.ibm.mqbind.jar
com.ibm.mqjms-6.jar
According to this post they come with the client install. I assume you can install it once, then re-use the jars (any licensing issues and expert opinions aside).
EDIT: In response to your comment, here's the client code I hacked up. It is for reading messages from a queue and blatting them to files. It's written in Scala. I hope it helps somewhat.
import com.ibm.mq._
import java.text._
import java.io._
case class QueueDetails(hostname: String, channel: String,
port: Int, queueManager: String, queue: String)
class Reader(details: QueueDetails) {
def read = {
MQEnvironment.hostname = details.hostname
MQEnvironment.channel = details.channel
MQEnvironment.port = details.port
val props = new java.util.Hashtable[String, String]
props.put(MQC.TRANSPORT_PROPERTY, MQC.TRANSPORT_MQSERIES)
MQEnvironment.properties = props
val qm = new MQQueueManager(details.queueManager)
val options = MQC.MQOO_INPUT_AS_Q_DEF | MQC.MQOO_INQUIRE
val q = qm.accessQueue(details.queue, options, null, null, null)
val depth = q.getCurrentDepth
val indexFormat = new DecimalFormat(depth.toString.replaceAll(".", "0"))
def exportMessage(index: Int): Unit = {
if (index < depth) {
val msg = new MQMessage
q.get(msg, new MQGetMessageOptions)
val msgLength = msg.getMessageLength
val text = msg.readStringOfByteLength(msgLength)
val file = new File("message_%s.txt".format(indexFormat.format(index)))
val writer = new BufferedWriter(new FileWriter(file))
writer.write(text)
writer.close
println(file.getAbsolutePath)
exportMessage(index + 1)
}
}
exportMessage(0)
q.close
qm.disconnect
}
}