We Keep Coding

How can I prevent memory leaks in a Scala code?

I have the code below that first read a file and then put these information in a HashMap(indexCategoryVectors). The HashMap contains a String (key) and a Long (value). The code uses the Long value to access a specific position of another file with RandomAccessFile.
By the information read in this last file and some manipulations the code write new information in another file (filename4). The only variable that accumulates information is the buffer (var buffer = new ArrayBuffer[Map[Int, Double]]()) but after each interaction the buffer is cleaned (buffer.clear).
The foreach command should run more than 4 million times, and what I'm realizing there is an accumulation in memory. I tested the code with a million times interaction and the code used more than 32GB of memory. I don't know the reason for that, maybe it's about Garbage Collection or anything else in JVM. Does anybody knows what can I do to prevent this memory leak?
def main(args: Array[String]): Unit = {
val indexCategoryVectors = getIndexCategoryVectors("filename1")
val uriCategories = getMappingURICategories("filename2")
val raf = new RandomAccessFile("filename3", "r")
var buffer = new ArrayBuffer[Map[Int, Double]]()
// Through each hashmap key.
uriCategories.foreach(uri => {
var emptyInterpretation = true
uri._2.foreach(categoria => {
val position = indexCategoryVectors.get(categoria)
// go to position
raf.seek(position.get)
var vectorSpace = parserVector(raf.readLine)
buffer += vectorSpace
//write the information of buffer in file
writeInformation("filename4")
buffer.clear
}
})
})
println("Success!")
}

Play Framework eating up disk space

I am successfully serving videos using the Play framework, but I'm experiencing an issue: each time a file is served, the Play framework creates a copy in C:\Users\user\AppData\Temp. I'm serving large files so this quickly creates a problem with disk space.
Is there any way to serve a file in Play without creating a copy? Or have Play automatically delete the temp file?
Code I'm using to serve is essentially:
public Result video() {
return ok(new File("whatever"));
}

Use Streaming
I use following method for video streaming. This code does not create temp copies of the media file.
Basically this code responds to the RANGE queries sent by the browser. If browser does not support RANGE queries I fallback to the method where I try to send the whole file using Ok.sendFile (internally play also tries to stream the file) (this might create temp files). but this happens very rarely when range queries is not supported by the browser.
GET /media controllers.MediaController.media
Put this code inside a Controller called MediaController
def media = Action { req =>
val file = new File("/Users/something/Downloads/somefile.mp4")
val rangeHeaderOpt = req.headers.get(RANGE)
rangeHeaderOpt.map { range =>
val strs = range.substring("bytes=".length).split("-")
if (strs.length == 1) {
val start = strs.head.toLong
val length = file.length() - 1L
partialContentHelper(file, start, length)
} else {
val start = strs.head.toLong
val length = strs.tail.head.toLong
partialContentHelper(file, start, length)
}
}.getOrElse {
Ok.sendFile(file)
}
}
def partialContentHelper(file: File, start: Long, length: Long) = {
val fis = new FileInputStream(file)
fis.skip(start)
val byteStringEnumerator = Enumerator.fromStream(fis).&>(Enumeratee.map(ByteString.fromArray(_)))
val mediaSource = Source.fromPublisher(Streams.enumeratorToPublisher(byteStringEnumerator))
PartialContent.sendEntity(HttpEntity.Streamed(mediaSource, None, None)).withHeaders(
CONTENT_TYPE -> MimeTypes.forExtension("mp4").get,
CONTENT_LENGTH -> ((length - start) + 1).toString,
CONTENT_RANGE -> s"bytes $start-$length/${file.length()}",
ACCEPT_RANGES -> "bytes",
CONNECTION -> "keep-alive"
)
}

Pinning .jar files to the taskbar in windows 7

My question is an extension on another question already answered, https://superuser.com/questions/257467/windows-7-how-to-pin-a-jar-to-the-taskbar
Is there a way to pin a jar to the taskbar, and have the window generated by the jar register as a different process, thus creating a different icon in the task bar? because as it stands, using any of the methods listed in the answer to the above question, you end up with a shortcut that can be pinned. but it is just a shortcut and only that, not the program itself. Im willing to try anything at this point as it is beginning to be very bothersome. not only does it look unprofessional, it uses up unnecessary screen real estate. As I know someone is going to ask what else I've tried, here's a bit of code i tried to run in c# to launch the jar, but of course, it does the same thing, registering the new process as a new process. (should have thought that one through.)
string strCmdText;
strCmdText = "-jar ImgurDownloader.jar";
Process process = new Process();
process.StartInfo.Arguments = strCmdText;
process.StartInfo.FileName = "javaw";
process.StartInfo.UseShellExecute = false;
process.Start();
so then I tried this:
string strCmdText;
strCmdText = "-jar ImgurDownloader.jar";
Process process = Process.GetCurrentProcess();
process.StartInfo.Arguments = strCmdText;
process.StartInfo.FileName = "javaw";
process.StartInfo.UseShellExecute = false;
process.Start();
and yet still, even if i try and replace the current process, it comes across as a new process, and thus a second icon in the taskbar. Please excuse my possibly short tone, the frustration is starting to kick in after a couple weeks.
Edit: have also tried setting the UAMID (User Application Model ID) using the JNA library to access shel32.dll's functions. The following is the code in the jar
public static void setCurrentProcessExplicitAppUserModelID(final String appID) {
if (SetCurrentProcessExplicitAppUserModelID(new WString(appID)).longValue() != 0)
throw new RuntimeException("unable to set current process explicit AppUserModelID to: " + appID);
}
public static String getCurrentProcessExplicitAppUserModelID() {
final PointerByReference r = new PointerByReference();
if (GetCurrentProcessExplicitAppUserModelID(r).longValue() == 0) {
final Pointer p = r.getValue();
return p.getString(0, true); // here we leak native memory by
// lazyness
}
return "N/A";
}
private static native NativeLong GetCurrentProcessExplicitAppUserModelID(PointerByReference appID);
private static native NativeLong SetCurrentProcessExplicitAppUserModelID(WString appID);
static {
Native.register("shell32");
}
then just call the set method. Tested with the get method, however,
NOTE: getCurrentProcessExplicitAppUserModelID is a lazy method and breaks things later on if used.
then in the C# Wrapper,
[DllImport("shell32.dll")]
public static extern int SetCurrentProcessExplicitAppUserModelID([MarshalAs(UnmanagedType.LPWStr)] string AppID);
static void Main()
{
int der = SetCurrentProcessExplicitAppUserModelID("MAndWorks.ImgurDownloader.ImgurDownloader.2.0.0.0");
string strCmdText;
Console.WriteLine(der);
strCmdText = "-jar ImgurDownloader.jar";
Process process = new Process();
process.StartInfo.Arguments = strCmdText;
process.StartInfo.FileName = "javaw";
process.StartInfo.UseShellExecute = false;
process.Start();
process.WaitForExit();
Console.WriteLine("AWFURA");
}

How to spawn a process in Scala with PTY?

In ruby I have:
PTY.spawn("/usr/bin/lxc-monitor -n .+") do |i, o, pid|
# ...
end
How do this in scala/java?

Try JPty or pty4j. These are implementations of pty for Java using JNA.

I don't think that PTY has been ported to java/scala. You can use the built in Runtime from java.
def run() {
val rt = Runtime.getRuntime
val cmds = Array("/usr/bin/lxc-monitor", "-n .+")
val env = Array("TERM=VT100")
val p1 = rt.exec(cmds, env)
}
I used this page as the base for the scala version.
UPDATE:
To get the output you need to get the input stream and read it (I know this sounds backwards but it's input relative to the jvm). The example below uses apache commons to skip some verbose parts of java.
import java.io.StringWriter
import org.apache.commons.io.IOUtils
class runner {
def run() {
val rt = Runtime.getRuntime
val cmds = Array("/usr/bin/lxc-monitor", "-n .+")
val env = Array("TERM=VT100")
val p1 = rt.exec(cmds, env)
val inputStream = p1.getInputStream
val writer = new StringWriter()
IOUtils.copy(inputStream, writer, "UTF-8")
val output = writer.toString()
println(output)
}
}
I got the apache utils idea from here.

MQ Pure Java Client Lib

I am evaluating Websphere MQ7. I am a traditionally a TibRV guy. One thing I do not like is the fact that the IBM java client libs require C++ libs in order to run. Is there anyway to run the IBM java client libs without requiring the C++ libs? e.g. is there a pure java client library for MQ ?

I have previously written a JMS client to MQSeries v6 (not your version, I know) without needing to install native libs. The only IBM libraries I required were titled:
com.ibm.mq-6.jar
com.ibm.mqbind.jar
com.ibm.mqjms-6.jar
According to this post they come with the client install. I assume you can install it once, then re-use the jars (any licensing issues and expert opinions aside).
EDIT: In response to your comment, here's the client code I hacked up. It is for reading messages from a queue and blatting them to files. It's written in Scala. I hope it helps somewhat.
import com.ibm.mq._
import java.text._
import java.io._
case class QueueDetails(hostname: String, channel: String,
port: Int, queueManager: String, queue: String)
class Reader(details: QueueDetails) {
def read = {
MQEnvironment.hostname = details.hostname
MQEnvironment.channel = details.channel
MQEnvironment.port = details.port
val props = new java.util.Hashtable[String, String]
props.put(MQC.TRANSPORT_PROPERTY, MQC.TRANSPORT_MQSERIES)
MQEnvironment.properties = props
val qm = new MQQueueManager(details.queueManager)
val options = MQC.MQOO_INPUT_AS_Q_DEF | MQC.MQOO_INQUIRE
val q = qm.accessQueue(details.queue, options, null, null, null)
val depth = q.getCurrentDepth
val indexFormat = new DecimalFormat(depth.toString.replaceAll(".", "0"))
def exportMessage(index: Int): Unit = {
if (index < depth) {
val msg = new MQMessage
q.get(msg, new MQGetMessageOptions)
val msgLength = msg.getMessageLength
val text = msg.readStringOfByteLength(msgLength)
val file = new File("message_%s.txt".format(indexFormat.format(index)))
val writer = new BufferedWriter(new FileWriter(file))
writer.write(text)
writer.close
println(file.getAbsolutePath)
exportMessage(index + 1)
}
}
exportMessage(0)
q.close
qm.disconnect
}
}