I have the following method signature
JavaPairRDD<K,Object> countApproxDistinctByKey(double relativeSD)
from class
Class JavaPairRDD<K,V>
Javadoc here
In my code, I do the following
JavaPairRDD<String, String> mapToPair = ... some calculations ...
JavaPairRDD<String, Long> reachedDeviceRDD = mapToPair.countApproxDistinctByKey(0.01);
In Eclipse (Mars) this assignment throws the following error:
Type mismatch: cannot convert from JavaPairRDD<String,Object> to JavaPairRDD<String,Long>
which is correct, given the above signature!
But my problem is that I have been programming for 6+ months with IntellijIdea, and this error has never shown up since now.
It doesn't even come up with NetBeans and it doesn't show up compiling the project in Maven (i.e. javac).
It seems to me that the compiler is "casting automatically" the Object generic parameter to Long, which would be madness.
I don't know what I'm missing.
Additional info:
The classes are from the Apache Spark project, which is written in Scala.
** Edit **
You can find a code example here:
Related
I'm trying to use the EWS Java library (link) with LispWorks 7.1.2 Win32's Java interface. I am somewhat familiar with basic Java concepts but have no experience with the Java language. This is the code I am trying to mimic:
ExchangeService service = new ExchangeService(ExchangeVersion.Exchange2010_SP2);
So I figured I would create an ExchangeVersion object (which is an enum), set it to the value "Exchange2010_SP2", and pass it to another create-java-object expression for the ExchangeService object.
This was my first step:
(create-java-object "microsoft.exchange.webservices.data.core.ExchangeService"
"microsoft.exchange.webservices.data.core.enumeration.misc.ExchangeVersion.Exchange2010_SP2")
However, I get the following error message: constructor of microsoft/exchange/webservices/data/core/ExchangeService first arguments is wrong type, wanted microsoft.exchange.webservices.data.core.enumeration.misc.ExchangeVersion got "microsoft.exchange.webservices.data.core.enumeration.misc.ExchangeVersion.Exchange2010_SP2"
OK, so then I tried this:
(create-java-object "microsoft.exchange.webservices.data.core.ExchangeService"
"microsoft.exchange.webservices.data.core.enumeration.misc.ExchangeVersion")
which gave me the error: constructor of microsoft/exchange/webservices/data/core/ExchangeService first arguments is wrong type, wanted microsoft.exchange.webservices.data.core.enumeration.misc.ExchangeVersion got "microsoft.exchange.webservices.data.core.enumeration.misc.ExchangeVersion". Which doesn't make much sense to me. So finally I tried to create a standalone ExchangeVersion object:
(create-java-object "microsoft.exchange.webservices.data.core.enumeration.misc.ExchangeVersion")
which led to this error message: CREATE-JAVA-OBJECT: : Failed to find constructors for class "microsoft.exchange.webservices.data.core.enumeration.misc.ExchangeVersion".
I wonder if anyone have suggestions on what am I doing wrong?
Many Thanks!
I have some fairly complex code that uses Javaslang. If I compile it into a jar, it runs fine. However, when I try to step into it in Eclipse for debugging, Eclipse flags it as a compilation error and dies when it reaches that line. The particularly weird part is that this worked a week ago, and the code has not changed in the interim.
Things I have tried:
clean project (including unchecking 'build automatically')
delete project from Eclipse, delete .project and .settings, re-import
from scratch
delete project from Eclipse, delete .project, .classpath, .settings, do mvn eclipse:eclipse, reimport
Maven builds this without errors [both within Eclipse and from the command line]. I can run the project this depends on and have it access this code from the JAR, so I know it works. I just cannot have Eclipse access the code from the project, either in 'run' or 'debug' mode.
Seq<Tuple2<StateProbabilityVector, ScenData>> resultStateProbs =
futures.
flatMap(Future::get).
toList();
// Update the target counts.
// THIS ENTIRE STATEMENT IS THE ERROR
Seq<Tuple2<ScenState, Probability>> result =
resultStateProbs.flatMap(tuple -> tuple.apply((spv, baTargetCount) ->
{
return spv.getStateProbList().
peek(sp -> logger.debug("Checking if {} > {}: {}",
sp.getProbability(),
intermediateMinProb,
sp.getProbability().greaterThan(intermediateMinProb))).
filter(sp -> sp.getProbability().greaterThan(intermediateMinProb)).
map(sp -> updateScenarioData(sp, baTargetCount, dupStateInfo));
}));
// signature for updateScenarioData
protected abstract Tuple2<ScenState, Probability> updateScenarioData(StateProbability stateProb,
ScenData scenData,
DSI dupStateInfo);
// truncated def of StateProbabilityVector
#Getter #ToString #Builder
public class StateProbabilityVector {
#NonNull
private final Seq<StateProbability> stateProbList;
}
So the types are all correct, but Eclipse claims:
> Type mismatch: cannot convert from Object to Iterable<? extends
> Object>
> Type mismatch: cannot convert from Seq<Object> to
> Seq<Tuple2<ScenState,Probability>>
As Nándor comments, this is probably down to a difference between the Eclipse compiler and javac, and the problem can probably be solved with a type witness in the right place. To find the right place, I would start by breaking up the functional method chain and extracting some local variables:
Seq<Tuple2<ScenState, Probability>> result =
resultStateProbs.flatMap(tuple -> {
Seq<Tuple2<ScenState, Probability>> filteredAndUpdated =
tuple.apply((spv, baTargetCount) -> {
Seq<StateProbability> stateProbList = spv.getStateProbList();
stateProbList.peek(sp -> {
logger.debug("Checking if {} > {}: {}", sp.getProbability(), intermediateMinProb, sp.getProbability().greaterThan(intermediateMinProb));
});
Seq<StateProbability> filtered = stateProbList.filter(sp ->
sp.getProbability().greaterThan(intermediateMinProb));
Seq<Tuple2<ScenState, Probability>> updated = filtered.map(sp ->
updateScenarioData(sp, baTargetCount, dupStateInfo));
return updated;
});
return filteredAndUpdated;
});
If you use Eclipse's extract variable refactoring, that by itself may tell you where it's inferring the wrong types, and explicitly declaring the correct types of the local variables might be enough to fix the problem all by itself.
If not, it should at least narrow the error down, and show you exactly where in the call chain Eclipse is having trouble. You can then probably fix it with type witnesses or, if all else fails, explicit casts, and then (with that type information added) perhaps inline the variables again, although this code is dense enough that I might leave them in.
Side notes:
peek() will only only debug the first StateProbability -- is that your intent?
consider adding a greaterThan() method to StateProbability so you don't have to repeatedly call getProbability().greaterThan(). (If the answer to #1 is "no", this method would also be a good place to put the debug statement.)
consider adding a method on SceneState that would return a prefiltered list, like Seq<StateProbability> SceneState.allGreaterThan(Probability).
I'm new to neural networks and NLP. I've found this library: DeepLearning4J. I'm trying to get it to work but whenever I execute this instruction:
Collection<String> similar = vec.wordsNearest("word_to_search", 10);
If the word I'm searching is mapped into the network I get the following exception:
java.lang.IllegalArgumentException: XERBLA: Error on argument 6 (LDA) in SGEMV
at org.jblas.NativeBlas.sgemv(Native Method)
at org.nd4j.linalg.jblas.blas.JblasLevel2.sgemv(JblasLevel2.java:25)
at org.nd4j.linalg.api.blas.impl.BaseLevel2.gemv(BaseLevel2.java:53)
at org.nd4j.linalg.api.ndarray.BaseNDArray.mmuli(BaseNDArray.java:2569)
at org.nd4j.linalg.api.ndarray.BaseNDArray.mmul(BaseNDArray.java:2377)
at org.deeplearning4j.models.embeddings.wordvectors.WordVectorsImpl.wordsNearest(WordVectorsImpl.java:290)
at org.deeplearning4j.models.embeddings.wordvectors.WordVectorsImpl.wordsNearest(WordVectorsImpl.java:337)
at word2vec.Word2VecTest.main(Word2VecTest.java:74)
Exception in thread "main" java.lang.NoSuchMethodError: org.nd4j.linalg.api.ndarray.INDArray.mean(I)Lorg/nd4j/linalg/api/ndarray/INDArray;
at org.deeplearning4j.models.embeddings.wordvectors.WordVectorsImpl.wordsNearest(WordVectorsImpl.java:283)
at word2vec.Word2VecTest.main(Word2VecTest.java:89)
I know that the NoSuchMethodError may be due to libraries different versions. In this specific case, this is probably caused by nd4j. I've checked the versions lots of time and this is what I'm importing at the moment:
akka-actor_2.11-2.4-M3.jar
akka-cluster_2.11-2.4-M3.jar
akka-remote_2.11-2.4-M3.jar
akka-slf4j_2.11-2.4-M3.jar
byte-buddy-0.6.15.jar
config-1.3.0.jar
deeplearning4j-core-0.0.3.3.4.alpha2.jar
deeplearning4j-nlp-0.0.3.3.4.alpha2.jar
deeplearning4j-scaleout-akka-0.0.3.3.4.alpha2.jar
deeplearning4j-ui-0.0.3.3.4.alpha2.jar
javassist-3.12.1.GA.jar
jblas-1.2.4.jar
jcublas-6.5.jar
lucene-analyzers-common-4.10.3.jar
lucene-core-4.10.3.jar
nd4j-api-0.4-rc3.4.jar
nd4j-bytebuddy-0.4-rc3.4.jar
nd4j-jblas-0.4-rc3.4.jar
nd4j-jcublas-common-0.4-rc3.4.jar
netty-3.10.4.Final.jar
protobuf-java-2.6.1.jar
reflections-0.9.10.jar
scala-library-2.12.0-M2.jar
selenium-server-standalone-2.47.1.jar
Can someone explain to me the problem?
The error is telling you that DeepLearning4J tried to call the method INDArray INDArray.mean(int value) but this method was not found.
Looking at nd4j 0.4-rc3.4 source code, you can see that the mean method actually takes a vararg int... as input. Since this is not int, the error is thrown.
This change was made by this commit when nd4j bumped version from 0.0.3.5.5.5 to 0.4-rc0.
As a result, you need to downgrade nd4j to version 0.0.3.5.5.5. With this downgrade, you will not have any more incompatibility since this is the actual version that DeepLearning4J is depending on. You can see that in the Maven dependencies of deeplearning4j-core-0.0.3.3.4.alpha2.
I am trying out Spark Programming examples using Java 1.8 in Eclipse Luna and have the following code -
JavaPairRDD<String, Integer> counts = ones
.reduceByKey(new Function2<Integer, Integer, Integer>() {
#Override
public Integer call(Integer i1, Integer i2) {
return i1 + i2;
}
});
List<Tuple2<String, Integer>> output = counts.collect(); //Compilation Error
I am using M2Eclipse to build and create the jar and using spark-submit to execute the jar in my local. The jar is working and printing the correct output but Eclipse always shows the above mentioned line as a compilation error - The type Tuple2 is not generic; it cannot be parameterized with arguments <String, Integer>
Even the programming examples referred in the Spark webpage uses the same notation for Tuple2. https://spark.apache.org/docs/0.9.0/java-programming-guide.html
I am not able to understand why Eclipse is showing it as a compilation error since the return type of the collect call is a List<Tuple2<String,Integer>>
Any help is greatly appreciated.
As mentioned by #Holger in the comments, 2 scala-library jars were added to the build path. Removed the earlier version and compilation errors disappeared.
It helps on my problem on IntelliJ Idea, too. I called a function in which a parameter is a generic type with Tuple2 as a parameter. It always show an error but I can pass the compilation. It confused me for several days. After removing several dependent sharded-jars(in which it contains something related scala-libiary), the error disappeared.
I have a few some strange issues with my class after migrating from JDK5/Tomcat5 to JDK6/Tomcat7 both with MyEclipse 9.
Whenever I try to access 'myclass' via jsp:usebean I got following error
org.apache.jasper.JasperException: javax.servlet.ServletException: java.lang.ClassFormatError: Illegal class modifiers in class myclass: 0x209
The rest of log points nowhere. By trial and error I have trimmed huge class to the following problematic part:
...
rf = store.getDefaultFolder();
f = (IMAPFolder)rf;
final IMAPStore storeNew = store;
Object val = f.doCommand(new IMAPFolder.ProtocolCommand() { <-- problem propably starts here
public Object doCommand(IMAPProtocol p)
throws ProtocolException {
...
The next strange thing is that my colleague is using the same environment with no problems. I have compared our class files and the only difference is in last but one byte. My class ends with '02 09', his '06 09'
After several days I am out of options how to get rid of it.
EDIT:
I have reinstalled/updated to MyEclipse 9.1. No luck so far.
SOLUTION
After another day trying to reproduce problem with a new project with a single file in it, I have realised that only remaining difference is in project/properties/java build path/libraries. There was J2EE 1.4 Libraries in the main project, so I have replaced it. After adding Java EE 6 Libraries voila. I am able to compile and run w/o any problems