We Keep Coding

How to use an OnlineTSPSource with esig/dss Library?

I'm attempting to use an online timestamp authority (rfc3161) with the Digital Signature Service Java library. However, the following snippet (from their test cases, and similar to the one from their Cookbook):
String tspServer = "http://tsa.belgium.be/connect";
OnlineTSPSource otsp = new OnlineTSPSource(tspServer);
/* tried setting otsp.setDataLoader(new TimestampDataLoader());
too, as it defaults to otsp.setDataLoader(new
NativeHTTPDataLoader()); the exception happens in both cases */
byte[] digest = DSSUtils.digest(DigestAlgorithm.SHA1, "Hello world".getBytes());
TimeStampToken timeStampResponse =
otsp.getTimeStampResponse(DigestAlgorithm.SHA1, digest);
always ends with the following exception:
eu.europa.esig.dss.DSSException:
java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError:
org.apache.commons.io.IOUtils.closeQuietly(Ljava/io/Closeable;)V
Already tried many different public rfc3161 servers (some listed here). Sure there's something wrong going on there, but, as a beginner, I cannot understand what is wrong (what method should be there).
If anyone could put me in the right direction to get the snippet working (or even be kind enough to comment a reliable startup guide on cades/xades/pades with Java's bouncycastle) I would be really grateful.

As stated in the comments by Marteen Bodewes and Mark Rotteveel, there was something wrong with the version of Apache Commons-IO in the classpath. The project is set using Apache Maven and there was an old Commons-IO version declared there as a dependency. In this case, it was enough to remove that declaration, so Maven could download the appropriate version that was declared as an esig/DSS dependency.
esig/DSS version was 5.4 at the time.

Why I got java.lang.nullpointerexception in elasticsearch build node?

I've test my apps using elasticsearch with very simple line of code. Like this :
Node node = nodeBuilder()
.settings(Settings.settingsBuilder().put("cluster.name", "elasticsearch").put("clster.transport.sniff", true).put("path.home", "/home/kenny/Program/Java/elastic"$
.node();
But I got error like this :
Exception in thread "main" java.lang.NullPointerException
at java.io.Reader.<init>(Reader.java:78)
at java.io.InputStreamReader.<init>(InputStreamReader.java:113)
at org.elasticsearch.node.internal.InternalSettingsPreparer.randomNodeName(InternalSettingsPreparer.java:198)
at org.elasticsearch.node.internal.InternalSettingsPreparer.finalizeSettings(InternalSettingsPreparer.java:177)
at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:101)
at org.elasticsearch.node.Node.<init>(Node.java:128)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
at org.elasticsearch.node.NodeBuilder.node(NodeBuilder.java:152)
at TryElastic.main(TryElastic.java:56)
I don't know how to solve this problem, I've try and looking for the solution. Line 56 at error log, refer to ".node()" method above. So, dou you have suggestion or there are something that I've to add in my code
Thanks.....

The only way this can happen is due to a misconfiguration of path.home.
When Elasticsearch tries to generate a random node name for your instance, it looks for a file at {path.home}/config/names.txt
If the file cannot be found, you'll get a (rather unfriendly and unhelpful) NullPointerException.
So the solution is to check that "/home/kenny/Program/Java/elastic" is really the top-level of an ES installation.
See here for docs on the correct directory layout.

ESAPI Symmetric Encryption using JavaEncryptor

I am testing basic stuff in ESAPI, and I ran across this symmetric encryption tutorial and copied and pasted the code, (along with importing the ESAPI 2.1.0 jar file, ESAPI.properties and validation.properties in the 'src' directory in Eclipse)
Modified code from the tutorial:
import org.owasp.esapi.crypto.CipherText;
import org.owasp.esapi.crypto.PlainText;
import org.owasp.esapi.errors.EncryptionException;
import org.owasp.esapi.reference.crypto.JavaEncryptor;
public class ESAPIsymEncTester {
public static void main(String[] args) throws EncryptionException{
String myplaintext = "My plaintext";
CipherText ciphertext =
JavaEncryptor.getInstance().encrypt( new PlainText(myplaintext) );
PlainText recoveredPlaintext = JavaEncryptor.getInstance().decrypt(ciphertext);
assert myplaintext.equals( recoveredPlaintext.toString() );
System.out.println("recovered plaintext: " + recoveredPlaintext.toString());
}
}
However, when I run this in Eclipse Luna using Java 1.8, I get this stack trace:
Exception in thread "main" org.owasp.esapi.errors.EncryptionException: Encryption failure: Invalid key exception.
at org.owasp.esapi.reference.crypto.JavaEncryptor.encrypt(JavaEncryptor.java:526)
at org.owasp.esapi.reference.crypto.JavaEncryptor.encrypt(JavaEncryptor.java:338)
at com.fate.engine.test.ESAPIsymEncTester.main(ESAPIsymEncTester.java:15)
Caused by: java.security.InvalidKeyException: Invalid AES key length: 96 bytes
at com.sun.crypto.provider.AESCipher.engineGetKeySize(AESCipher.java:495)
at javax.crypto.Cipher.passCryptoPermCheck(Cipher.java:1062)
at javax.crypto.Cipher.checkCryptoPerm(Cipher.java:1033)
at javax.crypto.Cipher.init(Cipher.java:1367)
at javax.crypto.Cipher.init(Cipher.java:1301)
at org.owasp.esapi.reference.crypto.JavaEncryptor.encrypt(JavaEncryptor.java:504)
... 2 more
I am not sure if this is a bug in the JavaEncryptor.java code, or if I am pulling something that I misconfigured from the ESAPI.properties file.
I replaced the master key and salt by running the JavaEncryptor and copy/pasting the resultant key/salt.
If it is a bug, I will email the ESAPI guys to get clarification on how I can fix it, since I looked through the JavaEncryptor code and am not entirely clear where all of the pieces are coming from.
Encryptor.MasterKey=WppLubGgsc/p6HhvcPf2LA==
Encryptor.MasterSalt=YokRN9mjMUTZspEbzBY90NA6EC8=
Encryptor.PreferredJCEProvider=
Encryptor.EncryptionAlgorithm=AES
Encryptor.CipherTransformation=AES/CBC/PKCS5Padding
Encryptor.cipher_modes.combined_modes=GCM,CCM,IAPM,EAX,OCB,CWC
Encryptor.cipher_modes.additional_allowed=CBC
Encryptor.EncryptionKeyLength=128
Encryptor.ChooseIVMethod=random
Encryptor.fixedIV=0x000102030405060708090a0b0c0d0e0f
Encryptor.CipherText.useMAC=true
Encryptor.PlainText.overwrite=true
Encryptor.HashAlgorithm=SHA-512 *****
Encryptor.HashIterations=1024
Encryptor.DigitalSignatureAlgorithm=SHA1withDSA
Encryptor.DigitalSignatureKeyLength=1024
Encryptor.RandomAlgorithm=SHA1PRNG
Encryptor.CharacterEncoding=UTF-8
Encryptor.KDF.PRF=HmacSHA1 *****

You forgot to place the most important part of your log into the question:
Dec 11, 2015 8:05:24 AM org.owasp.esapi.reference.JavaLogFactory$JavaLogger log
WARNING: [SECURITY FAILURE Anonymous:null#unknown -> /JavaEncryptor] Encryption key length mismatch. ESAPI.EncryptionKeyLength is 128 bits, but length of actual encryption key is 24 bits. Did you remember to regenerate your master key (if that is what you are using)???
This is a clue that there is something here that the library expects you to do.
It seems to me that you probably have the default encryptor properties set like this in esapi.properties:
Encryptor.MasterKey=owasp1
Encryptor.MasterSalt=testtest
The class JavaEncryptor has a main method that will generate valid properties for you. Run it in eclipse or via the command line. It will give you values to replace in esapi.properties, like this:
Dec 11, 2015 8:10:25 AM org.owasp.esapi.reference.JavaLogFactory$JavaLogger log
OFF: [SECURITY AUDIT Anonymous:null#unknown -> /SecurityProviderLoader] No Encryptor.PreferredJCEProvider specified.
SecurityConfiguration for Encryptor.EncryptionKeyLength not an integer in ESAPI.properties. Using default: 128
Generating a new secret master key
use '-print' to also show available crypto algorithms from all the security providers
SecurityConfiguration for Encryptor.EncryptionKeyLength not an integer in ESAPI.properties. Using default: 128
Copy and paste these lines into your ESAPI.properties
#==============================================================
Encryptor.MasterKey=qW0Qw+8eb1Zu1MBv5djwqA==
Encryptor.MasterSalt=b0VappFU1Hd6LjIt+TGYqQlfrdU=
#==============================================================
Once I did that, your code example runs just fine.

Here's what I'm going to suggest... grab the TEST version of ESAPI.properties from GitHub ("wget https://github.com/ESAPI/esapi-java-legacy/blob/master/src/test/resources/esapi/ESAPI.properties" should work, you use 'git' or save if from your browser), put it in place, and first use it AS-IS. If it fails, then there is a problem in you tweaked code. If it works, there was a problem in your ESAPI.properties file. Many people already suggested what to look for in terms of what might be wrong, but the differences should be minor enough that you should be able to spot them by diff'ing yours versus the TEST version in src/test/resources/esapi/ESAPI.properties. (The production version, incidentally, is under 'configuration/esapi/ESAPI.properties' and is not included with the jar because of some bug in the pom.xml which I don't know how to fix as I am not a Maven guru.)
If you have further questions, contact me at my Gmail account which you should be able to find easily enough via Google with my name and the term "OWASP". Once we figure out an answer that works for you, either you or I can post an answer back to Stack Overflow, but I don't frequent this forum enough to regularly monitor it. (Although, come to think of it, I probably do get notified of replies.)
Hope this helps,
-kevin w. wall / ESAPI crypto developer and co-project lead

DeepLearning4J NoSuchMethodError

I'm new to neural networks and NLP. I've found this library: DeepLearning4J. I'm trying to get it to work but whenever I execute this instruction:
Collection<String> similar = vec.wordsNearest("word_to_search", 10);
If the word I'm searching is mapped into the network I get the following exception:
java.lang.IllegalArgumentException: XERBLA: Error on argument 6 (LDA) in SGEMV
at org.jblas.NativeBlas.sgemv(Native Method)
at org.nd4j.linalg.jblas.blas.JblasLevel2.sgemv(JblasLevel2.java:25)
at org.nd4j.linalg.api.blas.impl.BaseLevel2.gemv(BaseLevel2.java:53)
at org.nd4j.linalg.api.ndarray.BaseNDArray.mmuli(BaseNDArray.java:2569)
at org.nd4j.linalg.api.ndarray.BaseNDArray.mmul(BaseNDArray.java:2377)
at org.deeplearning4j.models.embeddings.wordvectors.WordVectorsImpl.wordsNearest(WordVectorsImpl.java:290)
at org.deeplearning4j.models.embeddings.wordvectors.WordVectorsImpl.wordsNearest(WordVectorsImpl.java:337)
at word2vec.Word2VecTest.main(Word2VecTest.java:74)
Exception in thread "main" java.lang.NoSuchMethodError: org.nd4j.linalg.api.ndarray.INDArray.mean(I)Lorg/nd4j/linalg/api/ndarray/INDArray;
at org.deeplearning4j.models.embeddings.wordvectors.WordVectorsImpl.wordsNearest(WordVectorsImpl.java:283)
at word2vec.Word2VecTest.main(Word2VecTest.java:89)
I know that the NoSuchMethodError may be due to libraries different versions. In this specific case, this is probably caused by nd4j. I've checked the versions lots of time and this is what I'm importing at the moment:
akka-actor_2.11-2.4-M3.jar
akka-cluster_2.11-2.4-M3.jar
akka-remote_2.11-2.4-M3.jar
akka-slf4j_2.11-2.4-M3.jar
byte-buddy-0.6.15.jar
config-1.3.0.jar
deeplearning4j-core-0.0.3.3.4.alpha2.jar
deeplearning4j-nlp-0.0.3.3.4.alpha2.jar
deeplearning4j-scaleout-akka-0.0.3.3.4.alpha2.jar
deeplearning4j-ui-0.0.3.3.4.alpha2.jar
javassist-3.12.1.GA.jar
jblas-1.2.4.jar
jcublas-6.5.jar
lucene-analyzers-common-4.10.3.jar
lucene-core-4.10.3.jar
nd4j-api-0.4-rc3.4.jar
nd4j-bytebuddy-0.4-rc3.4.jar
nd4j-jblas-0.4-rc3.4.jar
nd4j-jcublas-common-0.4-rc3.4.jar
netty-3.10.4.Final.jar
protobuf-java-2.6.1.jar
reflections-0.9.10.jar
scala-library-2.12.0-M2.jar
selenium-server-standalone-2.47.1.jar
Can someone explain to me the problem?

The error is telling you that DeepLearning4J tried to call the method INDArray INDArray.mean(int value) but this method was not found.
Looking at nd4j 0.4-rc3.4 source code, you can see that the mean method actually takes a vararg int... as input. Since this is not int, the error is thrown.
This change was made by this commit when nd4j bumped version from 0.0.3.5.5.5 to 0.4-rc0.
As a result, you need to downgrade nd4j to version 0.0.3.5.5.5. With this downgrade, you will not have any more incompatibility since this is the actual version that DeepLearning4J is depending on. You can see that in the Maven dependencies of deeplearning4j-core-0.0.3.3.4.alpha2.

Java Play Mustache NPE Error

We are getting a mustache play error in production (amazon linux EC2 AMI) but not in development (MACs) and we have tried upgrading the jvm, using the jdk instead, and changing from a tomcat deploy model to match our development environments as much as possible but nothing is working. Please any help would be greatly appreciated. We have lots of shared code in java and javascript using mustache and it would be a big deal to rewrite everything if we had to ditch mustache on the java side.
20:48:52,403 ERROR ~
#6al2dd0po
Internal Server Error (500) for request GET /mystuff/people
Execution exception (In {module:mustache-0.2}/app/play/modules/mustache/MustacheTags.java around line 32)
NullPointerException occured : null
play.exceptions.JavaExecutionException
at play.templates.BaseTemplate.throwException(BaseTemplate.java:90)
at play.templates.GroovyTemplate.internalRender(GroovyTemplate.java:257)
at play.templates.Template.render(Template.java:26)
at play.templates.GroovyTemplate.render(GroovyTemplate.java:187)
at play.mvc.results.RenderTemplate.<init>(RenderTemplate.java:24)
at play.mvc.Controller.renderTemplate(Controller.java:660)
at play.mvc.Controller.renderTemplate(Controller.java:640)
at play.mvc.Controller.render(Controller.java:695)
at controllers.MyStuff.people(MyStuff.java:183)
at play.mvc.ActionInvoker.invokeWithContinuation(ActionInvoker.java:548)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:502)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:478)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:473)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:161)
at Invocation.HTTP Request(Play!)
Caused by: java.lang.NullPointerException
at play.modules.mustache.MustacheTags._template(MustacheTags.java:32)
at play.modules.mustache.MustacheTags$_template.call(Unknown Source)
at /app/views/User/people.html.(line:22)
at play.templates.GroovyTemplate.internalRender(GroovyTemplate.java:232)
... 13 more

Seems the issue is with the threadlocal. In Prod as per my logs, the session gets initialized with the main thread.
[2012-06-30 18:35:38,102] INFO 10097[**main**] - Mustache module initialized
However, MustacheTag tries to access with various thread like this during request.
[2012-06-30 17:48:44,669] INFO 66048[**play-thread-1**] - [{module:mustache-0.2}/app/play/modules/mustache/MustacheTags.java:46] _meta() :: MustachePlugin.session():null
So I changed the implementation of MustachePlugin like this.Changed line commented out:
//private static ThreadLocal<MustacheSession> session_ = new ThreadLocal<MustacheSession>();
private static MustacheSession _session = null;
public static MustacheSession session(){
//return session_.get();
return _session;
}
public void onConfigurationRead(){
// some code
_session = new MustacheSession(compiler, root);
// some code
}
And it is working fine now in prod mode! I see no reason why it should have been in a ThreadLocal in the first place as the session gets initialized at startup!

your issue is difficult to reproduce so i'll give few pointers here. you have tried to eliminate of issue being env issue. so other possible issues could be
data issue: many times reason for production issue is usually difference in actual data and test data. check if its data issue which is casuing NPE.
code issue: Is there something at people.html.(line:22) causing issue. first try removing / altering that to check if thats causing issue. Or can you get source code of mustache (exact version which you are using) and see what object its trying to create and where its failing.
Properties file for different environments: do you have different proprties file for each env? If yes, have u missed on any property for prod env?

You have a NullPointerException on MustacheTags.java at line 32.
This means that you are probably calling a method of the Mustache library and passing a null value.
Try logging all the parameters you transfer to this method (MyStuff.java line 183?).
You can look at the source code of MustacheTags here, it might help you understand what values you are passing and what should be passed.