I downloaded Calabash XML a couple of days back and got it working easily enough from the command prompt. I then tried to run it from Java code I noticed there was no API (e.g. the Calabash main method is massive with code calls to everywhere). To get it working was very messy as I had to copy huge chunks from the main method to a wrapper class, and divert from the System.out to a byte array output stream (and eventually into a String) i.e.
...
ByteArrayOutputStream baos = new ByteArrayOutputStream (); // declare at top
...
WritableDocument wd = null;
if (uri != null) {
URI furi = new URI(uri);
String filename = furi.getPath();
FileOutputStream outfile = new FileOutputStream(filename);
wd = new WritableDocument(runtime,filename,serial,outfile);
} else {
wd = new WritableDocument(runtime,uri,serial, baos); // new "baos" parameter
}
The performance seems really, really slow e.g. i ran a simple filter 1000 times ...
<p:filter>
<p:with-option name="select" select="'/result/meta-data/neighbors/document/title'" />
</p:filter>
On average each time took 17ms which doesn't seem like much but my spring REST controller with calls to Mongo DB and encryption calls etc take on average 3/4 ms.
Has anyone encountered this when running Calabash from code? Is there something I can do to speed things up?
For example, I this is being called each time -
XProcRuntime runtime = new XProcRuntime(config);
Can this be created once and reused? Any help is appreciated as I don't want to have to pay money to use Calamet but really want to get Xproc working from code to an acceptable performance.
For examples on how you could integrate XMLCalabash in a framework, I can mention Servlex by Florent Georges. You'd have to browse the code to find the relevant bit, but last time I looked it shouldn't be too hard to find:
http://servlex.net/
XMLCalabash wasn't build for speed unfortunately. I am sure that if you run profile, and can find some hotspots, Norm Walsh would be interested to hear about it.
Alternative is to look into Quixprox, which is derived from XMLCalabash:
https://code.google.com/p/quixproc/
I am also very sure that if you can send Norm a patch to improve the main class for better integration, he'd be interested to hear about it. In fact, the code should be on github, just fork it, fix it, and do a pull request..
HTH!
Related
I am currentrly working on a Way to visualize Fractals in Java. The mathematics behind it work perfectly fine and I am very happy with how the Pics turn out. Now i want to take these Images and turn them into a Video. I've written a Java Program that produces any number of Pictures and saves them (alphabetically) in a new directory. Now I need a way for Java to convert these Images into a Video.
I know there are solutions such as ffmpeg, however I need this process to be repeatable, so I don't think a Command Line Application would be the best Option.
Is there any Way to implement such a function into Java directly ?
If you're willing to use a third party library and the platform is supported (Windows or Linux, 64bit). You can use Pilecv4j (Full disclosure, I'm the main committer).
There's a test checked in that does exactly this. You can find it here: TestSplitIntoFiles.java
Here is the pertinent function that does this with the minor difference that it's not reading the image file names from the contents on the disk:
private static void encodeFiles(final File imageDir, final long numFrames, final File outputVideo) throws IOException {
try(final CvMat firstFrame = ImageFile.readMatFromFile(new File(imageDir, "image-0.jpg").getAbsolutePath());
final EncodingContext ectx = Ffmpeg2.createEncoder()
.outputStream(outputVideo.getAbsolutePath())
.openVideoEncoder("libx264", "vidEncoder")
.addCodecOptions("preset", "slow")
.addCodecOptions("crf", "40")
.enable(firstFrame, false)
;) {
final VideoEncoder ve = ectx.getVideoEncoder("vidEncoder");
ectx.ready();
LongStream.range(0, numFrames)
.mapToObj(fn -> new File(imageDir, "image-" + fn + ".jpg").getAbsolutePath())
.forEach(frameFile -> {
try(CvMat mat = uncheck(() -> ImageFile.readMatFromFile(frameFile));) {
ve.encode(mat, false);
}
});
}
}
If you decide to give it a try, let me know if you run into any issues. You can use the Issues on GitHub. I've been using it professionally for a while.
Also, see the answer to this question: Read a mp4 and write to anther mp4 creates bigger size
I'm trying to make a program that checks avaliable positions and books the first avaliable one. I started writing it and i ran into a problem pretty early.
The problem is that when I try to connect with the site (which is https) the program doesn't do anything. It doesn't throw an error, it doesn't crash. And the weirdest thing is that it works with some https websites and with some it doesn't.
I've spent countless hours trying to resolve this problem. I tried using htmlunitdriver and it still doesn't work. Please help.
private final WebClient webc = new WebClient(BrowserVersion.CHROME);
webc.getCookieManager().setCookiesEnabled(true);
HtmlPage loginpage = webc.getPage(loginurl);
System.out.println(loginpage.getTitleText());
I'm getting really frustrated with this. Thank you in advance.
As far as i can see this has nothing to do with HttpS. It is a good idea to do some traffic analysis using Charles or Fiddler.
What you can see....
The page returned from the server as response to your first call to https://online.enel.pl/ loads some external javascript. And then the story begins:
This JS looks like
(function() {
var z = "";
var b = "766172205f3078666.....";
eval((function() {
for (var i = 0; i < b.length; i += 2) {
z += String.fromCharCode(parseInt(b.substring(i, i + 2), 16));
}
return z;
})());
})();
As you can see someone likes to hide the real javascript that gets processed.
Next step is to check the javascript after this simple decoding
It is really huge and looks like this
var _0xfbfd = ['\x77\x71\x30\x6b\x77 ....
(function (_0x2ea96d, _0x460da4) {
var _0x1da805 = function (_0x55e996) {
while (--_0x55e996) {
_0x2ea96d['\x70\x75\x73\x68'](_0x2ea96d['\x73\x68\x69\x66\x74']());
}
};
.....
Ok now we have obfuscated javascript. If you like you can start with http://ddecode.com/hexdecoder/ to get some more readable text but this was the step where i have stopped my analysis. Looks like this script does some really bad things or someone still believes in security by obscurity.
If you run this with HtmlUnit, this codes gets interpreted - yes the decoding works and the code runs. Sadly this code runs endless (maybe because of an error or some incompatibility with real browsers).
If you like to get this working, you have to figure out, where the error is and open an bug report for HtmlUnit. For this you can simply start with a small local HtmlFile and include the code from the first external javascript. Then add some log statements to get the decoded version. Then replace this with the decoded version and try to understand what is going on. You can start adding alert statements and check if the code in HtmlUnit follows the same path as browsers do. Sorry but my time is to limited to do all this work but i really like to help/fix if you can point to a specific function in HtmlUnit that works different from real browsers.
Without the URL that you are querying it is dificult to say what could be wrong. However, having worked with HTML unit some time back I found that it was failing with many sites that I needed to get data from. The site owners will do many things to avoid you using programs to access them and you might have to resort to using some lower level library like Apache HTTP components where you have more control over what is going on under the hood.
Also check if the website is constructed using JavaScript which is getting more and more popular but making it increasingly dificult to use programs to interrogate the content.
I'm trying to create an SNMP4j agent and am finding it difficult to understand the process correctly. I have successfully created an agent that can be queried from the command line using snmpwalk. What I am having difficulty with is understanding how I am meant to update the values stored in my implemented MIB.
The following shows the relevant code I use for creating the MIB (I implement Host-Resources-MIB)
agent = new Agent("0.0.0.0/" + port);
agent.start();
agent.unregisterManagedObject(agent.getSnmpv2MIB());
modules = new Modules(DefaultMOFactory.getInstance());
HrSWRunEntryRow thisRow = modules.getHostResourcesMib().getHrSWRunEntry()
.createRow(oidHrSWRunEntry);
final OID ashEnterpriseMIB = new OID(".1.3.6.1.4.1.49266.0");
thisRow.setHrSWRunIndex(new Integer32(1));
thisRow.setHrSWRunName(new OctetString("RunnableAgent"));
thisRow.setHrSWRunID(ashEnterpriseMIB);
thisRow.setHrSWRunPath(new OctetString("All is good in the world")); // Max 128 characters
thisRow.setHrSWRunParameters(new OctetString("Everything is working")); // Max 128 characters
thisRow.setHrSWRunType(new Integer32(HrSWRunTypeEnum.application));
thisRow.setHrSWRunStatus(new Integer32(HrSWRunStatusEnum.running));
modules.getHostResourcesMib().getHrSWRunEntry().addRow(thisRow);
agent.registerManagedObject(modules.getHostResourcesMib());
This appears to be sufficient to create a runnable agent. What I do not understand is how I am meant to change the values stored in the MIB (how do I, for example, change the value of HrSWRunStatus). There seem to be a few kludge ways but they don't seem to fit with the way the library is written.
I have come across numerous references to using/overriding the methods
prepare
commit
undo
cleanup
But cannot find any examples where this is done. Any help would be gratefully received.
In protected void registerManagedObjects(), you need to do something like new MOMutableColumn(columnId, SMIConstants.SYNTAX_INTEGER, MOAccessImpl.ACCESS_READ_WRITE, null); for your HrSWRunStatus. Take a look at the TestAgent.java example of SNMP4J-agent source.
I am trying to be able to pick out frames (video and metadata) from MPEG, MPEG-TS and MPEG-PS files and live streams (network / UDP / RTP streams). I was looking into using JCODEC to do this and I started off by trying to use the FrameGrab / FrameGrab8Bit classes, and ran into an error that those formats are "temporarily unsupported". I looked into going back some commits to see if I could just use older code, but it looks like both of those files have had those formats "temporarily unsupported" since 2013 / 2015, respectively.
I then tried to plug things back into the FrameGrab8Bit class by putting in the below code...
public static FrameGrab8Bit createFrameGrab8Bit(SeekableByteChannel in) throws IOException, JCodecException {
...
SeekableDemuxerTrack videoTrack = null;
...
case MPEG_PS:
MPSDemuxer psd = new MPSDemuxer(in);
List tracks = psd.getVideoTracks();
videoTrack = (SeekableDemuxerTrack)tracks.get(0);
break;
case MPEG_TS:
in.setPosition(0);
MTSDemuxer tsd = new MTSDemuxer(in);
ReadableByteChannel program = tsd.getProgram(481);
MPSDemuxer ptsd = new MPSDemuxer(program);
List<MPEGDemuxerTrack> tstracks = ptsd.getVideoTracks();
MPEGDemuxerTrack muxtrack = tstracks.get(0);
videoTrack = (SeekableDemuxerTrack)tstracks.get(0);
break;
...
but I ran into a packet header assertion failure in the MTSDemuxer.java class in the parsePacket function:
public static MTSPacket parsePacket(ByteBuffer buffer) {
int marker = buffer.get() & 0xff;
int marker = by & 0xff;
Assert.assertEquals(0x47, marker);
...
I found that when I reset the position of the seekable byte channel (i.e.: in.setPosition(0)) the code makes it past the assert, but then fails at videoTrack = (SeekableDemuxerTrack)tstracks.get(0) (tstracks.get(0) cannot be converted to a SeekableDemuxerTrack)
Am I waisting my time? Are these formats supported somewhere in the library and I am just not able to find them?
Also, after going around in the code and making quick test applications, it seems like all you get out of the demuxers are video frames. Is there no way to get the metadata frames associated with the video frames?
For reference, I am using the test files from: http://samples.ffmpeg.org/MPEG2/mpegts-klv/
In case anyone in the future also has this question. I got a response from a developer on the project's GitHub page to this question. Response:
Yeah, MPEG TS is not supported to the extent MP4 is. You can't really seek in TS streams (unless you index the entire stream before hand).
I also asked about how to implement the feature. I thought that it could be done by reworking the MTSDemuxer class to be built off of the SeekableDemuxerTrack so that things would be compatible with the FrameGrab8Bit class, and got the following response:
So it doesn't look like there's much sense to implement TS demuxer on top of SeekableDemuxerTrack. We haven't given much attention to TS demuxer actually, so any input is very welcome.
I think this (building the MTSDemuxer class off of the SeekableDemuxerTrack interface) would work for files (since you have everything already there). But without fully fleshing out that thought, I could not say for sure (it definitely makes sense that this solution would not work for a live MPEG-TS / PS connection).
I am trying to display the result of a Mondrian query using JPivot. Many examples are showing how to use the tag library for JSP but I need to use the Java API, I looked at the documentation but I cannot understand how to use it to display the results in the table. Here is my code
Query query = connection.parseQuery(mdxQuery);
Result result = connection.execute(query);
result.print(new PrintWriter(System.out,true));
I would like to know if I can use the result object to build the jpivot table.
Thanks in advance!
First of all, using JPivot
is a pretty bad idea.
It was discontinued back in 2008.
There is a good project which is intended to replace the JPivot called Pivot4j. Despite it is currently under development (0.8 -> 0.9 version), Pivot4j can actually do the business.
However, if we're talking about your case:
result.print(new PrintWriter(System.out,true));
This string prints the HTML code with OLAP cube into your System.out.
You can write the HTML code in some output stream (like FileOuputStream), and then display it.
OutputStream out = new FileOutputStream("result.html");
result.print(new PrintWriter(out, true));
//then display this file in a browser
However, if you want to have the same interface as in JPivot, I don't think there is an easy way to do it without .jsp. In these case I strongly recommend you to try Pivot4j.
Good luck!