In the Java API example they create a Datastore by using DatastoreHelper.getOptionsfromEnv
But this creates the warning
WARNING: Not using any credentials
and leads ultimately to:
DatastoreException(null): beginTransaction 401
I set my environment variables to the following:
export DATASTORE_DATASET={Project-ID}
export DATASTORE_HOST="https://www.googleapis.com/datastore/v1/datasets/{Project-ID}"
export DATASTORE_SERVICE_ACCOUNT="{email address}"
export DATASTORE_PRIVATE_KEY_FILE="{path to local p12 keyfile}"
But still when I try to see what the credentials are:
println("Datastore helper: " +DatastoreHelper.getOptionsfromEnv
.dataset(datasetId).build().getCredential)
I get null, what could be missing?
Also is there either a way to set the Credentials inside the project (instead of using the getOptionsfromEnv)?
The problem was that even though I used
source ~/.bash_profile
to refresh my environemnt variables and the echo command showed me that they were indeed updated, apperently I needed to restart my terminal (using Mac OSX) for them to be also updated for sbt and Scala.
I am not sure why this is the case and if this is Scala specific but now I managed to authenticate and communicate with the server.
I managed to figure it out by using the local installation of the Datastore Server and continuing to have the same problems.
Related
When starting for example Elasticsearch 5.5 :
main ERROR Could not register mbeans java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register")
Workaround with Oracle Java 1.8.0_131 is to open file <jre>/lib/security/java.policy and add this line to grant section (i.e. between curly brackets):
permission javax.management.MBeanTrustPermission "register";
Why workaround? The upright solution would be to specify extra grant section which code exactly should get this permission.
got the same error and the answer is here java.security.AccessControlException when using Ant, but runs ok when invoking java from console
Append the grant section in java.policy file with
permission javax.management.MBeanTrustPermission "register";
I had this same issue when moving from a single instance to two instances locally.
I tried what Alice suggested above. Even re-installed Elasticsearch (5.5.0).
I also updated my Java to the latest one for Linux provided by Oracle.
Nothing was working. Then I discovered, that I couldn't just take the elasticsearch-5.5.0/config directory and rename it to elasticsearch-5.5.0/node1.
So... I had to leave that config directory in place and clone it to node1/node2.
EVEN if I configure path.config in the runtime args, ES still needs that base line config directory.
Hope this helps.
I faced same issue on Ubuntu-16.04 system.
Solution:
ElasticSeearch service is not allowed to run for "ROOT" user. That's why change the ownership of elasticsearch folder with below command:
go to Elasticseach installation directory
$ sudo chown -R user_name:user_Group elasticsearch-5.5.0
$/bin/elasticsearch
this will start elasticsearch service. It is working form me perfectly.
I'm trying to connect to my HDFS instance running on Cloudera. My first step was enabling Kerberos and creating Keytabs (as shown here).
In the next step i would like to authenticate with a keytab.
Configuration conf = new Configuration();
conf.set("fs.defaultFS", "hdfs://cloudera:8020");
conf.set("hadoop.security.authentication", "kerberos");
UserGroupInformation.setConfiguration(conf);
UserGroupInformation.loginUserFromKeytab("hdfs#CLOUDERA", "/etc/hadoop/conf/hdfs.keytab");
FileSystem fs = FileSystem.get(conf);
FileStatus[] fsStatus = fs.listStatus(new Path("/"));
for (int i = 0; i < fsStatus.length; i++) {
System.out.println(fsStatus[i].getPath().toString());
}
It fails with the following error
java.io.IOException: Login failure for hdfs#CLOUDERA from keytab
/etc/hadoop/conf/hdfs.keytab:
javax.security.auth.login.LoginException: Unable to obtain password
from user
The question is: how do I correctly handle the keytab? Do i have to copy it to my local machine?
When running a Hadoop client on Windows to reach a kerberized cluster, you need a specific "native library" (i.e. DLL).
As far as I can tell there is no good reason for that, because that lib is not actually used outside of some automated regression tests (!?!) so it's a pain inflicted to Hadoop users by Hadoop committers.
To add extra pain, there is no official build of that DLL (and of the Windows "stub" that enable its use from Java). You must either (a) build it yourself from source code -- good luck -- or (b) search the internet for a downloadable Hadoop-for-Windows runtime, and pray that is does not contain any malware.
The best option (for 64-bit Windows) is here: https://github.com/steveloughran/winutils
...and the ReadMe explains why you can reasonably trust that run-time. But if you are stuck with an older 32-bit Windows, then you are on your own.
Now let's assume you deployed that run-time on your Windows box under
C:\Some Dir\hadoop\bin\(the final bin is required; the embedded space is just extra fun)
You must point the Hadoop client to that run-time with a couple of Java properties:
"-Dhadoop.home.dir=C:/Some Dir/hadoop" "-Djava.library.path=C:/Some Dir/hadoop/bin"
(note the double quotes around Windows args as a whole, to protect embedded spaces in the paths, which have been translated to Java style for extra fun)(in Eclipse, just stuff these props under "VM Arguments", quotes included)
Now, there's the Kerberos config. If your KDC is your corporate Active Directory server, then Java should find the config parameters automatically. But if your KDC is a standalone "MIT Kerberos" install on Linux, then you have to find a valid /etc/krb5.conf file on the cluster, copy it on your Windows box, and have Java use it with an additional property...
"-Djava.security.krb5.conf=C:/Some Other Dir/krb5.conf"
Then let's assume you have created your keytab file on a Linux box, using ktutil (or an Active Directory admin created it for you with some AD command) and you dropped the file under C:\Some Other Dir\foo.keytab
Before anything else, if the keytab is for a real Windows account -- i.e. your own account -- or a Prod service account, then make sure that keytab is secure!! Use the Windows Security dialog box to restrict access to your account only (and maybe System, for backups). Because that file could enable anyone, on any machine, to authenticate on the cluster (and any Kerberos-enabled system, including Windows).
Now you can try to authenticate using
UserGroupInformation.loginUserFromKeytab("foo#BAR.ORG", "C:/Some Other Dir/foo.keytab");
If it does not work, enable the Kerberos debug traces with both an environment variable
set HADOOP_JAAS_DEBUG=true
...and a Java property
-Dsun.security.krb5.debug=true
(in Eclipse, set these in "Environment" and "VM Arguments" respectively)
Do you have set proper permissions?
chown hdfs:hadoop /etc/hadoop/conf/hdfs.keytab
chmod 440 /etc/hadoop/conf/hdfs.keytab
I have a Java program that makes reports from Google AdWords, the authentications are working correctly since I have no issues issuing my reports with the Java API, but now when I run my Java API on the localhost with mvn appengine:devserver, it gives me this exception:
Client ID must be set as api.adwords.clientId in ads.properties.
If you do not have a client ID or secret, please create one in the API console: https://console.developers.google.com/project caused by: [clientId]
Only thing I change from the Java app to the GAE app is the doGet method, I comment it out and change my run() that is called by the doGet to main(String args[]).
Fixed, here's how:
This issue is due to the path of properties file.
First when deploying app, I was using a server so it doesn't read the properties file from the same directory as when it's run (ran?) on eclipse with the play button or with javac your_java_app.java
I moved my ads.properties from resources to src/main/webapp/WEB-INF and changed the OAuth2Credential from
..fromFile().. to
..fromFile("WEB-INF/ads.properties")..
I have downloaded commom-daemon tool and used with a java application. I have created a bat file as shown below
set SERVICE_NAME=sample
set PR_INSTALL=D:\commons-daemon-1.0.15-bin-windows-signed\prunsrv.exe
REM Service log configuration
set PR_LOGPREFIX=%SERVICE_NAME%
set PR_LOGPATH=D:\logs
set PR_STDOUTPUT=D:\logs\stdout.txt
set PR_STDERROR=D:\logs\stderr.txt
set PR_LOGLEVEL=Error
REM Path to java installation
set PR_JVM=C:\Java\jre7\bin\client\jvm.dll
set PR_CLASSPATH=D:\commons-daemon-1.0.15-bin-windows-signed\Daemon.jar
REM Startup configuration
set PR_STARTUP=auto
set PR_STARTMODE=jvm
set PR_STARTCLASS=com.SomeService
set PR_STARTMETHOD=start
REM Shutdown configuration
set PR_STOPMODE=jvm
set PR_STOPCLASS=com.SomeService
set PR_STOPMETHOD=stop
REM JVM configuration
set PR_JVMMS=256
set PR_JVMMX=1024
set PR_JVMSS=4000
set PR_JVMOPTIONS=-Duser.language=DE;-Duser.region=de
In cmd , I install the service using the command
prunsrv.exe //IS//sample
After this, a service named sample become available in the list of services and when I tried to start it it shows:
Windows could not start the sample on Local Computer. For more information review the System event log. If this is a non-Microsoft service, contact the service vendor and refer to the server specific
error code 1
UPDATED
When I run
prunsrv.exe //ES//sample
it shows
The data area passed to a system call is too small.
Failed to start service
Can any one help me in this?
I had the same problem. In my case (not yours exactly), the problem was the jvm.dll path, because the variable %JAVA_HOME% has spaces. So to solve this, I modify the assignment
set CG_PATH_TO_JVM=%JAVA_HOME%\jre\bin\server\jvm.dll
to
set CG_PATH_TO_JVM="%JAVA_HOME%\jre\bin\server\jvm.dll"
and that's all.
Also, you could check the variables assignment with this command:
prunmgr//ES//yourservicename_as_in_windows
To help others troubleshooting.
If you look at:
https://commons.apache.org/proper/commons-daemon/procrun.html
There is a parameter:
--LogPath
which defaults to:
%SystemRoot%\System32\LogFiles\Apache
A log file is generated there which contains some additional error messages and possibly useful information.
The original questioner changed the log path to:
set PR_LOGPATH=D:\logs
So looking there would be the appropriate thing to do in their case.
I also had this cryptic error message 'The data area passed to a system call is too small.' with no further information in either the startup log or the Windows/System32/LogFiles/Apache/ logs on Win 8/Server 2008.
I had renamed my packages and the --StartClass and --StopClass parameters were wrong.
I agreed with OscarSan that a space in %JAVA_HOME% could cause the "error code 1" problem. I solve this problem by re-installing JDK 1.8 to change the installation path from C:\Program Files Java\jdk1.8.0_144 to C:\Java\jdk1.8.0_144. Problem solved.
Well I was trying to follow the first cup tutorial, but I'm stuck with a Glassfish error.
I'm supposed to this:
1.2.4 Getting the Latest Updates to the Tutorial
Check for any updates to this tutorial by using the Update Center
included with the Java EE 7 SDK.
1.2.4.1 Update the Tutorial Through the Update Center Open the Update Center and check for any updates to the tutorial.
In NetBeans IDE, select the Services tab and expand the Servers node.
Right-click the GlassFish Server instance and select View Update
Center to display the Update Tool. In the tree, select Available
Updates to display a list of updated packages. Look for updates to the
First Cup for Java EE 7 (javaee-firstcup-tutorial) package. If there
is an updated version of First Cup, select First Cup 7.0 for Java EE 7
(javaee-firstcup-tutorial) and click Install.
The thing is, when I try to open the Update Center, this is what I get:
The software needed for this command (updatetool) is not installed.
If you choose to install Update Tool, your system will be automatically
configured to periodically check for software updates. If you would like
to configure the tool to not check for updates, you can override the
default behavior via the tool's Preferences facility.
Exception in thread "main" java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:196)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:687)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:658)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
at com.sun.pkg.client.Image.checkRepositoryConnection(Image.java:1225)
at com.sun.pkg.client.Catalog.refresh(Catalog.java:132)
at com.sun.pkg.client.Image.refreshCatalogs(Image.java:1627)
at com.sun.pkg.client.Client.main(Client.java:109)
When this tool interacts with package repositories, some system information
such as your system's IP address and operating system type and version
is sent to the repository server. For more information please see:
http://wikis.oracle.com/display/updatecenter/UsageMetricsUC2
Once installation is complete you may re-run this command.
Would you like to install Update Tool now (y/n):
C:\Program Files\glassfish-4.0>"C:\Program Files\Java\jdk1.7.0_40\bin\java" -Dimage.path="C:\Program Files\glassfish-4.0\bin\\.." -jar "C:\Program Files\glassfish-4.0\bin\\..\pkg/lib/pkg-client.jar" refresh
C:\Program Files\glassfish-4.0>"C:\Program Files\Java\jdk1.7.0_40\bin\java" -Dimage.path="C:\Program Files\glassfish-4.0\bin\\.." -jar "C:\Program Files\glassfish-4.0\bin\\..\pkg/lib/pkg-bootstrap.jar" "C:\Users\Fabio\AppData\Local\Temp\pkg-bootstrap11073.props"
Proxy: Using system proxy settings.
Input/output error: Connection reset
Could not download application packages. This could be because:
- a proxy server is needed to access the internet. Please ensure that
the system proxy server settings in your Internet Options control panel
(under Connections:LAN Settings) are correct, or set the HTTP_PROXY
environment variable to the full URL of the proxy server.
- the package server or network connection is slow.
If you are getting time out errors you can try setting the
PKG_CLIENT_CONNECT_TIMEOUT and PKG_CLIENT_READ_TIMEOUT
environment variables and try again. For example to increase
the timeouts to 300 seconds set them to 300
- the package server is down or otherwise inaccessible or it is
generating invalid data. Please contact the provider of the package
server.
This may be a problem with the slow GlassFish update server. Here are some steps you can try:
You should retry first to see if the error occurs again. You can start a cmd, navigate to the \glassfish\bin directory and run updatetool.bat so you can see the output which is printed after the one you posted.
If you are behind a proxy you should make sure it is setup correctly in the system settings.
You can also try to install the updates through the GlassFish admin console. Start your GlassFish instance and navigate to http:\\localhost:4848. On the bottom of the left menu click on Updatetool. There you can choose and install the desired updates.
Another option is to change the timeouts of the pkg tool. To set the timeout to 300 seconds do the following in a cmd (on Windows):
set PKG_CLIENT_CONNECT_TIMEOUT=300
set PKG_CLIENT_READ_TIMEOUT=300
Your last (ok, there may be other ones) option is to just skip the update process and do the tutorial with the current version. It should work anyway and I can see in my GlassFish update center that there is only an update for the tutorial documentation at the moment.
Just set this line into cmd (like administrator):
set PKG_CLIENT_CONNECT_TIMEOUT=300
set PKG_CLIENT_READ_TIMEOUT=300
Maybe the firstcup example is already existed in the install directory. Try to find it in the directory_install_glassfish/glassfish4/docs. The mine contain even javaee-tutorial and firstcup. For the next part, you maybe continue after guide in firstcup.pdf with the part of : Creating Your First Java by create archetype of java project. Good luck !
For the lazy of you the pkg Unix command is:
export PKG_CLIENT_CONNECT_TIMEOUT=300
export PKG_CLIENT_READ_TIMEOUT=300
After being stuck for a while on the installation of GlassFish update server from Netbeans (in my company network, proxy issues), I got it this way:
- Advanced System Settings > System Properties window > Advanced tab, New System Variable: http_proxy, http://username:password#your_proxy:your_port
- Netbeans (running as admin just in case), Services > Server > Glassfish Server > Right button View Domain Update Center, then install
It worked for me, Hope it helps!