We Keep Coding

Rundeck config file not found when service starts up

I have installed RD 4.8 CE on a Windows 2019 server + MariaDB. The application is up and I can login with the admin account. Service is Windows integrated.
I'm trying now to allow AD users to login and perform activities. I haven't found much documentation on how to configure RD on Windows. So I'm grabbing from here and there. So far I have done the following:
Created a file called jass-multiauth.conf in server/config/ folder as listed below:
multiauth {
com.dtolabs.rundeck.jetty.jaas.JettyCombinedLdapLoginModule sufficient
debug="true"
contextFactory="com.sun.jndi.ldap.LdapCtxFactory"
providerUrl="ldaps://xxxxxxxxxxx:636"
bindDn="CN=xxxxxx,OU=Service Accounts,OU=Admin Users and Groups,OU=xxx,DC=xxx,DC=xxx,DC=xxx"
bindPassword="x$xx<x>xx"
authenticationMethod="simple"
forceBindingLogin="true"
userBaseDn="OU=Admin Users,OU=Admin Users and Groups,OU=xxx,DC=xxx,DC=xxx,DC=xxx"
userRdnAttribute="sAMAccountName"
userIdAttribute="sAMAccountName"
userPasswordAttribute="unicodePwd"
userObjectClass="user"
roleBaseDn="OU=Groups,OU=Admin Users and Groups,OU=xxxx,DC=xxx,DC=xxx,DC=xxx"
roleNameAttribute="cn"
roleMemberAttribute="member"
roleObjectClass="group"
cacheDurationMillis="300000"
timeoutRead="10000"
reportStatistics="true"
supplementalRoles="user";
org.eclipse.jetty.jaas.spi.PropertyFileLoginModule required
debug="true"
file="E:/rundeck/server/config/realm.properties";
};
when I restart rundeck service the AD users are not recognized and no error at all is displayed in the log. I can however login with the admin account. So to troubleshoot this I added:
java %RDECK_CLI_OPTS% %RDECK_SSL_OPTS% -Drundeck.jaaslogin=true -Dloginmodule.conf.name=jaas-multiauth.conf -Djava.security.auth.login.config=E:\rundeck\server\config\jaas-multiauth.conf -Dloginmodule.name=multiauth -jar rundeck.war --skipinstall -d >> %CURDIR%\var\logs\service.log 2>&1
to the start-rundeck.bat file and in fact when I launch it I see an error which is quite weird:
[2022-12-22T18:58:09,409] ERROR config.GrailsApplicationPostProcessor - Error loading spring/resources.groovy file: java.io.IOException: E:\rundeck\server\config\jaas-multiauth.conf (No such file or directory)
java.lang.SecurityException: java.io.IOException: E:\rundeck\server\config\jaas-multiauth.conf (No such file or directory)
the file is there, path is correct and permission is set as on any other file in same directory. I deleted the file and created it again, but same error.
Also, I have a question: I understand that I should not edit the profile file. Hence, when the problem above is fixed, considering that the rundeck service is Windows integrated, where shall I put the line:
java %RDECK_CLI_OPTS% %RDECK_SSL_OPTS% -Drundeck.jaaslogin=true -Dloginmodule.conf.name=jaas-multiauth.conf -Djava.security.auth.login.config=E:\rundeck\server\config\jaas-multiauth.conf -Dloginmodule.name=multiauth -jar rundeck.war --skipinstall -d >> %CURDIR%\var\logs\service.log 2>&1
to refer the jass-multiauth.conf file?

Your multiauth config file is saved as jass-multiauth.conf but the launcher script is looking for jaas-multiauth.conf (jass =! jaas). That is the reason of "No such file or directory" error. Rename the file as jaas-multiauth.conf and relaunch the Rundeck service.

Accessing HDFS on Cloudera with Java and Kerberos Keytab from Windows

I'm trying to connect to my HDFS instance running on Cloudera. My first step was enabling Kerberos and creating Keytabs (as shown here).
In the next step i would like to authenticate with a keytab.
Configuration conf = new Configuration();
conf.set("fs.defaultFS", "hdfs://cloudera:8020");
conf.set("hadoop.security.authentication", "kerberos");
UserGroupInformation.setConfiguration(conf);
UserGroupInformation.loginUserFromKeytab("hdfs#CLOUDERA", "/etc/hadoop/conf/hdfs.keytab");
FileSystem fs = FileSystem.get(conf);
FileStatus[] fsStatus = fs.listStatus(new Path("/"));
for (int i = 0; i < fsStatus.length; i++) {
System.out.println(fsStatus[i].getPath().toString());
}
It fails with the following error
java.io.IOException: Login failure for hdfs#CLOUDERA from keytab
/etc/hadoop/conf/hdfs.keytab:
javax.security.auth.login.LoginException: Unable to obtain password
from user
The question is: how do I correctly handle the keytab? Do i have to copy it to my local machine?

When running a Hadoop client on Windows to reach a kerberized cluster, you need a specific "native library" (i.e. DLL).
As far as I can tell there is no good reason for that, because that lib is not actually used outside of some automated regression tests (!?!) so it's a pain inflicted to Hadoop users by Hadoop committers.
To add extra pain, there is no official build of that DLL (and of the Windows "stub" that enable its use from Java). You must either (a) build it yourself from source code -- good luck -- or (b) search the internet for a downloadable Hadoop-for-Windows runtime, and pray that is does not contain any malware.
The best option (for 64-bit Windows) is here: https://github.com/steveloughran/winutils
...and the ReadMe explains why you can reasonably trust that run-time. But if you are stuck with an older 32-bit Windows, then you are on your own.
Now let's assume you deployed that run-time on your Windows box under
C:\Some Dir\hadoop\bin\(the final bin is required; the embedded space is just extra fun)
You must point the Hadoop client to that run-time with a couple of Java properties:
"-Dhadoop.home.dir=C:/Some Dir/hadoop" "-Djava.library.path=C:/Some Dir/hadoop/bin"
(note the double quotes around Windows args as a whole, to protect embedded spaces in the paths, which have been translated to Java style for extra fun)(in Eclipse, just stuff these props under "VM Arguments", quotes included)
Now, there's the Kerberos config. If your KDC is your corporate Active Directory server, then Java should find the config parameters automatically. But if your KDC is a standalone "MIT Kerberos" install on Linux, then you have to find a valid /etc/krb5.conf file on the cluster, copy it on your Windows box, and have Java use it with an additional property...
"-Djava.security.krb5.conf=C:/Some Other Dir/krb5.conf"
Then let's assume you have created your keytab file on a Linux box, using ktutil (or an Active Directory admin created it for you with some AD command) and you dropped the file under C:\Some Other Dir\foo.keytab
Before anything else, if the keytab is for a real Windows account -- i.e. your own account -- or a Prod service account, then make sure that keytab is secure!! Use the Windows Security dialog box to restrict access to your account only (and maybe System, for backups). Because that file could enable anyone, on any machine, to authenticate on the cluster (and any Kerberos-enabled system, including Windows).
Now you can try to authenticate using
UserGroupInformation.loginUserFromKeytab("foo#BAR.ORG", "C:/Some Other Dir/foo.keytab");
If it does not work, enable the Kerberos debug traces with both an environment variable
set HADOOP_JAAS_DEBUG=true
...and a Java property
-Dsun.security.krb5.debug=true
(in Eclipse, set these in "Environment" and "VM Arguments" respectively)

Do you have set proper permissions?
chown hdfs:hadoop /etc/hadoop/conf/hdfs.keytab
chmod 440 /etc/hadoop/conf/hdfs.keytab

ORDS and Tomcat - images prefix path might not be correct

I have installed JDK, Tomcat 8 and ORDS on the server. I installed ORDS as standalone and then copied the images folder to j folder undet Tomcat dir/webapps
I am getting the following error when trying to access `http:\localhost:8090\ords
There is a problem with your environment because the Application Express files have not been loaded. Please verify that you have copied the images directory to your application server as instructed in the Installation Guide. In addition, please verify that your image prefix path is correct. Your current path is /j/ (it should contain both starting and ending forward slashes, such as the default /i/). Use the SQL script reset_image_prefix.sql if you need to change it
First I have noticed that my ORDS standalone.properties file did not have an ending \ at the end of the path so I added it but still getting the same error.
Also another developer used /j/ instead of /i/ and I am thinking it might be a problem but then I created /i/ directory and changed standalone.properties but nothing has changed - still getting an error.
My database is on another server and I assume so is the script reset_image_prefix.sql. I might not be able to run it myself, will have to ask a DBA but maybe that is not the problem and I can make a change on my own on the current server?
Can anyone help?

#ElenaDBA, could you please answer me the following questions?
Version of ords?
Are you migrating the Apex or is a fresh installation?
Is it a prod or the dev installation?
Is your database PDB or not?
I will provide the solution according to your answer?
Here are the steps without using SQL Developer?
Install java 7+ and TOMCAT
Setup JAVA_HOME
Make sure tomcat is configured correctly by running the tomcat server without ords.war.
Create user for ORDS users
ALTER USER APEX_PUBLIC_USER identified by password123 ACCOUNT UNLOCK;
create USER ORDS_PUBLIC_USER identified by password123;
alter USER ORDS_PUBLIC_USER identified by password123 ACCOUNT UNLOCK;
grant connect to ORDS_PUBLIC_USER;
create folders /opt/ords/war_file /opt/ords/config
download ords.war into /opt/ords/war_file folder
create directory /opt/ords/config
execute the following:
cd /opt/ords/war_file
java -jar ords.war
You will be prompted for the oracle database server name, SID, sysdba user password.
Say NO or SKIP to apex installation.
copy /opt/ords/war_file/ords.war to /opt/tomcat/apache-tomcat/webapps/ folder
run tomcat and browse to
http://server-ip:8080/ords
you should get the ords page with 404 Not Found
ORDS is installed and up and running. Now its time to enable some schema and its object.
Enable Schema:
BEGIN
ORDS.ENABLE_SCHEMA(p_enabled => TRUE,
p_schema => '<schema-name>',
p_url_mapping_type => 'BASE_PATH',
p_url_mapping_pattern => '<schema-alias>',
p_auto_rest_auth => FALSE);
commit;
END;
/
Enable Object/Table:
BEGIN
ORDS.ENABLE_OBJECT(p_enabled => TRUE,
p_schema => '<schema-name>',
p_object => '<table-name>',
p_object_type => 'TABLE',
p_object_alias => '<table-alias-name>',
p_auto_rest_auth => FALSE);
commit;
END;
/
Now browse to
http://server-ip:8080/ords/schema-alias/table-alias
To Disable Schema
BEGIN
ORDS.ENABLE_SCHEMA(
p_enabled => FALSE,
p_schema => '<schema-name>');
end;
/
Uninstall ORDS
cd /opt/ords/war_file
java -jar ords.war uninstall

Problem solved! The issue was that images folder copied was not complete, so once we copied images folder that had all the files

Play Framework deploy failed: "the application secret has not been set, and we are in prod mode. Your application is not secure"

I'm deploying on Heroku a web application developed with Play Framework.
The deploy fails returning this error:
[error] p.a.l.c.CryptoConfigParser - The application secret has not been set, and we are in prod mode. Your application is not secure.
What does it means that "we are in prod mode"? Where do I set the mode? I have to change this because it's not a production environment.
Thanks

Put this into your conf/application.conf
Play 2.5
play.crypto.secret="changethissosomethingsecret"
Play 2.6+
play.http.secret.key="changethissosomethingsecret"
You can also start your application with a source with a secret key as a parameter.

you need to add an application secret in your conf file.
play.http.secret.key="changethissosomethingsecret"

You need to set up different keys based on the Play version that your application is using.
What version of play are you using?
Either look at com.typesafe.play:sbt-plugin version inside your project's plugins.sbt or if you have a GNU compatible shell, execute the following command:
find . -name "plugins.sbt" -exec grep -PHin --color=always 'com.typesafe.play.*sbt-plugin.*%\s*"\K.*?(?=")' {} \;
Proceed to the next step, once you know the version number of play used by your app.
What is the key to use in application.conf based on play version?
Play 2.3.x: application.secret (defaults to changeme)
Play 2.4.x: play.crypto.secret (defaults to changeme)
Play 2.5.x: play.crypto.secret (defaults to changeme)
Play 2.6.x: play.http.secret.key (defaults to changeme)
As you can see it is not consistent, but what is consistent is:
In the above versions you can override the secret using APPLICATION_SECRET environment variable/property (so, something like APPLICAITION_SECRET="changed" sbt "runProd 9001")
The default value in each case is changeme
The next two options were tested on Play v2.6.x, but according to the documentation, should work with the same commands for v2.4.x and v2.5.x.
The only exception is v2.3.x which uses play-generate-secret and play-update-secret respectively
How do I get sbt to generate a key for me?
Run the following command from within the directory of your play application (tested with sbt v1.1.6 and play v2.6.x):
$ sbt playGenerateSecret | grep -i 'generated'
[info] Generated new secret: ^7UubY[rFXzkN:v6TB9WL/lfGsP61/vzAHA9tdZNZ#nALH=TEztKAlTC>xz;VUXw
You can manually copy the above as the value for the key (based on play framework version as noted before) OR, you can use the suggestion below.
How do I get sbt to update existing key in application.conf automatically?
Same as above, run the following command from the root directory of your play framework app:
$ sbt playUpdateSecret | grep -i "secret"
[info] Generated new secret: Kz?fHm_I[wt^Onp[#cr<:`ttrQi]KMsdDs>22hEF?RhkoanQ7gA6NAjL33EV2^Xt
[info] Updating application secret in /Users/blah/someplayapp/conf/application.conf
[warn] Did not find application secret in /Users/blah/someplayapp/conf/application.conf
[warn] Adding application secret to start of file
Couple of things to note about this command:
playUpdateSecret will update the existing secret or add a secret to application.conf if it is not present.
This commands looks for the key and/or adds/updates it in application.conf only. This is important if you are following the Best Practice.
This best practice refers to creating a separate production.conf that overrides this key from application.conf:
include "application"
play.http.secret.key="somesecretkey"
and then run your app like so:
/path/to/yourapp/bin/yourapp -Dconfig.file=/path/to/production.conf
As you can see, if you are using playUpdateSecret to generate a new secret then it is going to have no effect since it is being overridden by the secret specified in production.conf.

Before you run your application in production mode, you need to generate an application secret
You need generate token for application in production mode, using activator playGenerateSecret
Configuration Application Secret
Link documentation

[GAE Java]: WARNING: Not using any credentials

In the Java API example they create a Datastore by using DatastoreHelper.getOptionsfromEnv
But this creates the warning
WARNING: Not using any credentials
and leads ultimately to:
DatastoreException(null): beginTransaction 401
I set my environment variables to the following:
export DATASTORE_DATASET={Project-ID}
export DATASTORE_HOST="https://www.googleapis.com/datastore/v1/datasets/{Project-ID}"
export DATASTORE_SERVICE_ACCOUNT="{email address}"
export DATASTORE_PRIVATE_KEY_FILE="{path to local p12 keyfile}"
But still when I try to see what the credentials are:
println("Datastore helper: " +DatastoreHelper.getOptionsfromEnv
.dataset(datasetId).build().getCredential)
I get null, what could be missing?
Also is there either a way to set the Credentials inside the project (instead of using the getOptionsfromEnv)?

The problem was that even though I used
source ~/.bash_profile
to refresh my environemnt variables and the echo command showed me that they were indeed updated, apperently I needed to restart my terminal (using Mac OSX) for them to be also updated for sbt and Scala.
I am not sure why this is the case and if this is Scala specific but now I managed to authenticate and communicate with the server.
I managed to figure it out by using the local installation of the Datastore Server and continuing to have the same problems.