I am about to install Apache Spark 2.1.0 on Ubuntu 16.04 LTS. My goal is a standalone cluster, using Hadoop, with Scala and Python (2.7 is active)
Whilst downloading I get the choice: Prebuilt for Hadoop 2.7 and later (File is spark-2.1.0-bin-hadoop2.7.tgz)
Does this package actually include HADOOP 2.7 or does it need to be installed separately (first I assume)?
I have Java JRE 8 installed (Needed for other tasks). As the JDK 8 also seems to be a pre requisite as well, I also did a ' sudo apt install default-jdk', which indeed shows as installed:
default-jdk/xenial,now 2:1.8-56ubuntu2 amd64 [installed]
Checking java -version however doesn't show the JDK:
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
Is this sufficient for the installation? Why doesn't it also show the JDK?
I want to use Scala 2.12.1. Does this version work well with the Spark2.1/Hadoop 2.7 combination or is another version more suitable?
Is the Scala SBT package also needed?
Been going back and forth trying to get everything working, but am stuck at this point.
Hope somebody can shed some light :)
You need to install hadoop-2.7 more to whatever you are installing.
Java version is fine.
The mentioned configuration should work with scala 2.12.1.
Related
Recently, I want to upgrade my java from 8 to 11. Then checking my machine using java -version, it shows as follows:
java version "1.8.0_251"
Java(TM) SE Runtime Environment (build 1.8.0_251-b08)
Java HotSpot(TM) 64-Bit Server VM (build 25.251-b08, mixed mode)
Then, I think there is only java-8 there. But then I use homebrew with brew install java11 it shows this:
==> Formulae
app-engine-java java javacc jslint4java pdftk-java
google-java-format java11 ✔ javarepl libreadline-java
==> Casks
eclipse-java eclipse-javascript oracle-jdk-javadoc homebrew/cask-versions/java-beta
If you meant "java" specifically:
It was migrated from homebrew/cask to homebrew/core.
I am confused now: are these 2 versions installed in my machine, or only one there? If 2 here, how to I specify the path if I want to use java11.
In Homebrew, java11 is an alias for the formula openjdk#11. What you want is a cask, specifically temurin11, which can be installed by running this command:
brew install homebrew/cask-versions/temurin11
(Temurin is the official successor of AdoptOpenJDK.)
I have recently installed java 1.8 on a fresh install of ubuntu 20.04 and seems like I may have a bad java 1.8 setup.
I'm running a docker image from a service called prisma(prisma.io), and it's broken now on this fresh install. I got some java info on the terminal plus the docker java related errors into this pastebin: pastebin.com/y0e7CbpL
I have this same setup running on a separate ubuntu 18.04, also running java 1.8 with no issues. I tried to install it exactly the same way but I guess I missed something...
Is there something obvious that can be spotted there about why java can be badly setup?
Also how likely could it be for the new ubuntu to be causing these problems, if possible at all?
Is there more info that I should provide (and how) to help figure this out?
(side note: I'm also having issues with react-native android project where it cannot build some firebase packages)
Thanks!
IT was indeed a java problem.
I could only solve this problem after copying the entire folder /usr/lib/jvm/ from another computer where this project was running fine and replace it with the one on my machine.
PS: this is the java version:
$java -version
openjdk version "1.8.0_265"
OpenJDK Runtime Environment (build 1.8.0_265-8u265-b01-0ubuntu2~18.04-b01)
OpenJDK 64-Bit Server VM (build 25.265-b01, mixed mode)
I am trying to install eclipse-cpp oxygen on macOS Sierra 10.12. While installing it i get an error that java version is not 1.8, while I had already installed latest JDK package. on terminal it shows java version as below
java version "1.6.0_65"
Java(TM) SE Runtime Environment (build 1.6.0_65-b14-468-11M4833)
Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-468, mixed mode)
Please suggest how can I upgrade my java version.
Apple don't provide a version of Java greater than Java 6. You must install Java 8 from the Oracle site.
In order to run Eclipse you must install the full JDK, the JRE download is not enough as it does not include the java executable.
The Oracle site is here
You can install Java8 from here.
http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
make sure to check the latest binaries of java and javac in PATH. You can override the old java binaries in path.
There is something going wrong with my java configuration and it is really bugging me. I am using IntelliJ IDEA and after downloading and installing java 8 I tried to configure my project to use that SDK but I could only locate version 1.6 under /Library/Java/JavaVirtualMachines/. I am new to OS X and I am really confused with the paths.
Looking on my Java control panel I can see I got installed Java 8 but after running java -version on the terminal I get 1.6.0_65.
And the which java gives back /usr/bin/java.
Please help I am completely lost
Here is an example with several Java versions installed side-by-side ...
ls /Library/Java/JavaVirtualMachines/
jdk1.7.0_25.jdk jdk1.7.0_72.jdk jdk1.8.0_05.jdk jdk1.8.0_25.jdk
User can edit ~/.profile to point explicitly at one
export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_72.jdk/Contents/Home
export PATH=$JAVA_HOME/bin:$PATH
And here is result ...
java -version
java version "1.7.0_72"
Java(TM) SE Runtime Environment (build 1.7.0_72-b14)
Java HotSpot(TM) 64-Bit Server VM (build 24.72-b04, mixed mode)
Well something was wrong with the installation. I completely removed the other versions (not 1.6) and reinstalled 1.8 using the default path (as there is no option to change it during installation. Now IntelliJ can see JDK 1.8
Thanks for the help anyway
I downloaded Eclipse Luna and ran from the terminal in Mac OS X:
> /Applications/Eclipse.app/Contents/MacOS/eclipse
JavaVM: requested Java version (1.8.0) not available. Using Java at "/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home" instead.
However, I have:
> java -version
java version "1.8.0_20"
Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)
What am I doing wrong?
Luna seems currently having a bug: it installs the wrong Java version as its own dependency and then fails claiming that version is wrong.
You need to install Java separately (download official SDK from Oracle website) and make that one default. Be sure you install SDK and not the web plugin. Type java -version to verify the version and then which java to tell the exact path. Edit eclipse.ini as usual and set there the correct path to Java executable.