I'm trying to produce the same hashes found in the linux shadow file using the MessageDigest, given the password, salt value and hashing algorithm, although the results do not match with what I get from the function below.
Hash Algorithm = 6
Password = mandar
Salt Value = 5H0QpwprRiJQR19Y
Expected Output = $6$5H0QpwprRiJQR19Y$bXGOh7dIfOWpUb/Tuqr7yQVCqL3UkrJns9.7msfvMg4ZOPsFC5Tbt32PXAw9qRFEBs1254aLimFeNM8YsYOv.
Actual Output = ca0d04319f273d36f246975a4f9c71d0184c4ca7f3ba54bc0b3e0b4106f0eefca1e9a122a536fb17273b1077367bf68365c10fa8a2b18285a6825628f3614194
I have this function for generating the hash value
public String getSha512Hash(String password, String saltValue) throws NoSuchAlgorithmException{
String text = saltValue + password ;
MessageDigest messageDigest = MessageDigest.getInstance("SHA-512");
byte[] bytes = messageDigest.digest( text.getBytes() );
StringBuilder sb = new StringBuilder();
for (int i = 0; i < bytes.length; ++i) {
sb.append(Integer.toHexString((bytes[i] & 0xFF) | 0x100).substring(1,3));
}
return sb.toString();
}
I'm referring to this website.
The passwords in /etc/shadow are hashed using the crypt(3) system call (man crypt).
You can use the Apache Commons implementation which should mimic the same behavior.
The fundamental problem is that the site you are referring to uses Perl's crypt() which seems a direct call to libc crypt(). In the manual of crypt is not specified how the SHA-512 hash is actually computed, but I searched GitHub and found this ~400 LOC source file sha512-crypt.c.
I read throught it and can't tell if it refers to some standard or if it's the only program using that algorithm. Since the SHA-512 thing also seems a proprietary extension to the POSIX standard, it's absolutely not unlikely.
You could ask the maintainer or the mailing list and report your findings back, otherwise if you absolutely need that functionality, you could write a native extension (don't know if there are Java libraries already available).
I have a Dovecot server with MySQL database for storing usernames and passwords. The passwords in the database are in SHA512-CRYPT scheme.
I am inserting the hashed passwords in the database using a script.
doveadm pw -s SHA512-CRYPT -p password -r 500000
I want to hash the passwords using a JAVA application. I found this questions and I tried to create the same resulting hash using same password firstpassword and salt FooBarBaz. For some reason the resulting hash I get is different, although I am using the same hashing algorithm, salt and password.
Here is my Java code:
byte[] password = "firstpassword".getBytes();
byte[] salt = "FooBarBaz".getBytes();
MessageDigest digest = MessageDigest.getInstance("SHA-512");
digest.reset();
digest.update(salt);
byte[] hashed = digest.digest(password);
String encodedHash = Base64.getEncoder().encodeToString(hashed);
System.out.printf("{SHA512-CRYPT}$6$%s$%s", "FooBarBaz",encodedHash);
This outputs the hash:
{SHA512-CRYPT}$6$FooBarBaz$5WPtOnXVI/a6f003WByGKIcsfa6x0ansxiyE8uEfJ0TE5pI+Rv9kcMLgdZboKg7ZSWQgWFg+pIqruvdg6aiP/g==
I also tried swapping the order of salt + password to make it:
digest.update(password);
byte[] hashed = digest.digest(salt);
this gives me:
{SHA512-CRYPT}$6$FooBarBaz$QWS8+W5EWhModF+uO2tcsd55tDxzdzGJ5FurIbEgwVCwKfT5UqwIvBNG1Oyws8bZEFdeGgyD0u6zS1KArvGf9Q==
Does anyone have any idea how can I accomplish the same hash results in Java if I use the same password and salt?
The hash I am looking for is:
{SHA512-CRYPT}$6$FooBarBaz$.T.G.7FRJqZ6N2FF7b3BEkr5j37CWhwgvPOOoccrr0bvkBbNMmLCxzqQqKJbNhnhC.583dTBLEuZcDuQe7NEe.
doveadm uses the Unix crypt family of functions to generate the hash and outputs the hash as a Base64 encoded string. The alphabet used for the encoding (by crypt) is [a-zA-Z0-9./] (as mentioned on the man page for the functions). However, the alphabet used by the java.util.Base64 class is [A-Za-z0-9+/] (compliant with RFC 4648, as mentioned on the JavaDoc page for the Base64 class). Therefore, even if the hashed values are the same, they will get encoded differently.
A reliable option is to use the Crypt class from Apache Commons Codec as Crypt.crypt("firstpassword", "$6$FooBarBaz") (The prefix $6$ is mandatory to instruct Crypt that the SHA512-CRYPT algorithm needs to be used). This will generate your expected hash value.
i try to find the good way for the best technology/method for hidden password in a file, but without use external jar or library.
Actually i use one object that represent a list of user name and password. Convert my list in a xml (only in memory) and after that, i store in a file with AES.
Use only java 7, no external library.
Is a good/secure method?
If this operation is no good, is possible to create dynamically xml encrypted?
thanks
You can use a FileOutputStream wrapped in a CipherOutputStream.
It's not really secure to save passwords encrypted with AES because:
1) Where do you store the key? If you store it in the server, if an attacker violates the server and finds the key, he will have complete acces to the users information.
2) Do you really need to know the users' passwords? In many application, for security reasons, it's better to keep only the hash of the password. The username can be stored in plaintext and you can also add a salt to the password to enforce it. You can do that with some algorithms offered by Java7 platform. In this way, even if someone enters your server, he can't use users login informations without breaking the hash function.
Here's an example that worked for me:
public byte[] getHash(String password, byte[] salt, String algorithm) throws NoSuchAlgorithmException, UnsupportedEncodingException {
MessageDigest digest = MessageDigest.getInstance(algorithm);//The String rapresents the alg you want to use: for example "SHA-1" or "SHA-256"
digest.reset();
digest.update(salt);
return digest.digest(password.getBytes("UTF-8"));
}
You can also look at this link for a more complete example: https://www.owasp.org/index.php/Hashing_Java
Assuming that I'm trying to pull from a RESTful api that uses basic authentication / basic certificates, what would be the best way to store that user name and password in my program? Right now it's just sitting there in plaintext.
UsernamePasswordCredentials creds = new UsernamePasswordCredentials("myName#myserver","myPassword1234");
Is there some way of doing this that is more security minded?
Thanks
Important note:
If you're designing the authentication system as a whole, you shouldn't store passwords, even if they're encrypted. You store a hash, and check if passwords provided during login match the same hash. That way, a security breach on your database avoids getting your users' passwords exposed.
With that said, for situations where you are going to store data as-is (in this case passwords), then with an inner-to-outer mindset, here are some steps to protect your process:
First step, you should change your password-handling from String to character array.
The reason for this is that a String is an immutable object, and so it's data will not be cleansed immediately even if the object is set to null; The data is set for garbage-collection instead, and this poses security problems because malicious programs might gain access to that String (password) data before it is cleaned.
This is the main reason why Swing's JPasswordField's getText() method is deprecated, and why getPassword() uses character arrays.
The second step is to encrypt your credentials, only decrypting them temporarily during the authentication process. Or to hash them server-side, store that hash, and "forget" the original password.
This, similarly to the first step, makes sure your vulnerability-time is as small as possible.
It is recommended that your credentials are not hard-coded, and that instead, you store them in a centralized, configurable and easily-maintainable manner, such as a configuration or properties file, or a database.
You should encrypt your credentials before saving the file, and additionally, you can apply a second encryption to the file itself (2-layer encryption to the credentials, and 1-layer to other file contents).
Note that each of the two encryption processes mentioned above can be multiple-layered themselves. Each encryption can be an individual application of Triple Data Encryption Standard (AKA TDES and 3DES), as a conceptual example.
After your local environment is properly protected (but remember, it's never ever "safe"!), the third step is apply basic protection to your transmission process, by using TLS (Transport Layer Security) or SSL (Secure Sockets Layer).
The forth step is to apply other protection methods.
For example, applying obfuscation techniques to your "to-use" compile, to avoid (even if shortly) the exposure of your security measures in case your program is obtained by Ms. Eve, Mr. Mallory, or someone else (the bad-guys) and decompiled.
UPDATE 1:
By #Damien.Bell 's request, here is an example that covers the first and second steps:
//These will be used as the source of the configuration file's stored attributes.
private static final Map<String, String> COMMON_ATTRIBUTES = new HashMap<String, String>();
private static final Map<String, char[]> SECURE_ATTRIBUTES = new HashMap<String, char[]>();
//Ciphering (encryption and decryption) password/key.
private static final char[] PASSWORD = "Unauthorized_Personel_Is_Unauthorized".toCharArray();
//Cipher salt.
private static final byte[] SALT = {
(byte) 0xde, (byte) 0x33, (byte) 0x10, (byte) 0x12,
(byte) 0xde, (byte) 0x33, (byte) 0x10, (byte) 0x12,};
//Desktop dir:
private static final File DESKTOP = new File(System.getProperty("user.home") + "/Desktop");
//File names:
private static final String NO_ENCRYPTION = "no_layers.txt";
private static final String SINGLE_LAYER = "single_layer.txt";
private static final String DOUBLE_LAYER = "double_layer.txt";
/**
* #param args the command line arguments
*/
public static void main(String[] args) throws GeneralSecurityException, FileNotFoundException, IOException {
//Set common attributes.
COMMON_ATTRIBUTES.put("Gender", "Male");
COMMON_ATTRIBUTES.put("Age", "21");
COMMON_ATTRIBUTES.put("Name", "Hypot Hetical");
COMMON_ATTRIBUTES.put("Nickname", "HH");
/*
* Set secure attributes.
* NOTE: Ignore the use of Strings here, it's being used for convenience only.
* In real implementations, JPasswordField.getPassword() would send the arrays directly.
*/
SECURE_ATTRIBUTES.put("Username", "Hypothetical".toCharArray());
SECURE_ATTRIBUTES.put("Password", "LetMePass_Word".toCharArray());
/*
* For demosntration purposes, I make the three encryption layer-levels I mention.
* To leave no doubt the code works, I use real file IO.
*/
//File without encryption.
create_EncryptedFile(NO_ENCRYPTION, COMMON_ATTRIBUTES, SECURE_ATTRIBUTES, 0);
//File with encryption to secure attributes only.
create_EncryptedFile(SINGLE_LAYER, COMMON_ATTRIBUTES, SECURE_ATTRIBUTES, 1);
//File completely encrypted, including re-encryption of secure attributes.
create_EncryptedFile(DOUBLE_LAYER, COMMON_ATTRIBUTES, SECURE_ATTRIBUTES, 2);
/*
* Show contents of all three encryption levels, from file.
*/
System.out.println("NO ENCRYPTION: \n" + readFile_NoDecryption(NO_ENCRYPTION) + "\n\n\n");
System.out.println("SINGLE LAYER ENCRYPTION: \n" + readFile_NoDecryption(SINGLE_LAYER) + "\n\n\n");
System.out.println("DOUBLE LAYER ENCRYPTION: \n" + readFile_NoDecryption(DOUBLE_LAYER) + "\n\n\n");
/*
* Decryption is demonstrated with the Double-Layer encryption file.
*/
//Descrypt first layer. (file content) (REMEMBER: Layers are in reverse order from writing).
String decryptedContent = readFile_ApplyDecryption(DOUBLE_LAYER);
System.out.println("READ: [first layer decrypted]\n" + decryptedContent + "\n\n\n");
//Decrypt second layer (secure data).
for (String line : decryptedContent.split("\n")) {
String[] pair = line.split(": ", 2);
if (pair[0].equalsIgnoreCase("Username") || pair[0].equalsIgnoreCase("Password")) {
System.out.println("Decrypted: " + pair[0] + ": " + decrypt(pair[1]));
}
}
}
private static String encrypt(byte[] property) throws GeneralSecurityException {
SecretKeyFactory keyFactory = SecretKeyFactory.getInstance("PBEWithMD5AndDES");
SecretKey key = keyFactory.generateSecret(new PBEKeySpec(PASSWORD));
Cipher pbeCipher = Cipher.getInstance("PBEWithMD5AndDES");
pbeCipher.init(Cipher.ENCRYPT_MODE, key, new PBEParameterSpec(SALT, 20));
//Encrypt and save to temporary storage.
String encrypted = Base64.encodeBytes(pbeCipher.doFinal(property));
//Cleanup data-sources - Leave no traces behind.
for (int i = 0; i < property.length; i++) {
property[i] = 0;
}
property = null;
System.gc();
//Return encryption result.
return encrypted;
}
private static String encrypt(char[] property) throws GeneralSecurityException {
//Prepare and encrypt.
byte[] bytes = new byte[property.length];
for (int i = 0; i < property.length; i++) {
bytes[i] = (byte) property[i];
}
String encrypted = encrypt(bytes);
/*
* Cleanup property here. (child data-source 'bytes' is cleaned inside 'encrypt(byte[])').
* It's not being done because the sources are being used multiple times for the different layer samples.
*/
// for (int i = 0; i < property.length; i++) { //cleanup allocated data.
// property[i] = 0;
// }
// property = null; //de-allocate data (set for GC).
// System.gc(); //Attempt triggering garbage-collection.
return encrypted;
}
private static String encrypt(String property) throws GeneralSecurityException {
String encrypted = encrypt(property.getBytes());
/*
* Strings can't really have their allocated data cleaned before CG,
* that's why secure data should be handled with char[] or byte[].
* Still, don't forget to set for GC, even for data of sesser importancy;
* You are making everything safer still, and freeing up memory as bonus.
*/
property = null;
return encrypted;
}
private static String decrypt(String property) throws GeneralSecurityException, IOException {
SecretKeyFactory keyFactory = SecretKeyFactory.getInstance("PBEWithMD5AndDES");
SecretKey key = keyFactory.generateSecret(new PBEKeySpec(PASSWORD));
Cipher pbeCipher = Cipher.getInstance("PBEWithMD5AndDES");
pbeCipher.init(Cipher.DECRYPT_MODE, key, new PBEParameterSpec(SALT, 20));
return new String(pbeCipher.doFinal(Base64.decode(property)));
}
private static void create_EncryptedFile(
String fileName,
Map<String, String> commonAttributes,
Map<String, char[]> secureAttributes,
int layers)
throws GeneralSecurityException, FileNotFoundException, IOException {
StringBuilder sb = new StringBuilder();
for (String k : commonAttributes.keySet()) {
sb.append(k).append(": ").append(commonAttributes.get(k)).append(System.lineSeparator());
}
//First encryption layer. Encrypts secure attribute values only.
for (String k : secureAttributes.keySet()) {
String encryptedValue;
if (layers >= 1) {
encryptedValue = encrypt(secureAttributes.get(k));
} else {
encryptedValue = new String(secureAttributes.get(k));
}
sb.append(k).append(": ").append(encryptedValue).append(System.lineSeparator());
}
//Prepare file and file-writing process.
File f = new File(DESKTOP, fileName);
if (!f.getParentFile().exists()) {
f.getParentFile().mkdirs();
} else if (f.exists()) {
f.delete();
}
BufferedWriter bw = new BufferedWriter(new FileWriter(f));
//Second encryption layer. Encrypts whole file content including previously encrypted stuff.
if (layers >= 2) {
bw.append(encrypt(sb.toString().trim()));
} else {
bw.append(sb.toString().trim());
}
bw.flush();
bw.close();
}
private static String readFile_NoDecryption(String fileName) throws FileNotFoundException, IOException, GeneralSecurityException {
File f = new File(DESKTOP, fileName);
BufferedReader br = new BufferedReader(new FileReader(f));
StringBuilder sb = new StringBuilder();
while (br.ready()) {
sb.append(br.readLine()).append(System.lineSeparator());
}
return sb.toString();
}
private static String readFile_ApplyDecryption(String fileName) throws FileNotFoundException, IOException, GeneralSecurityException {
File f = new File(DESKTOP, fileName);
BufferedReader br = new BufferedReader(new FileReader(f));
StringBuilder sb = new StringBuilder();
while (br.ready()) {
sb.append(br.readLine()).append(System.lineSeparator());
}
return decrypt(sb.toString());
}
A full example, addressing every protection step, would far exceed what I think is reasonable for this question, since it's about "what are the steps", not "how to apply them".
It would far over-size my answer (at last the sampling), while other questions here on S.O. are already directed on the "How to" of those steps, being far more appropriate, and offering far better explanation and sampling on the implementation of each individual step.
If you are using basic auth, you should couple that with SSL to avoid passing your credentials in base64 encoded plain text. You don't want to make it easy for someone sniffing your packets to get your credentials. Also, don't hard code your credentials in your source code. Make them configurable. read them from a config file. You should encrypt the credentials before storing them in a config file and your app should decrypt the credentials once it reads them from the config file.
Why not store credentials in source code
It is generally a good idea to avoid to store credentials in source code.
The issue is, that access to code and who should have access to the credentials often changes over time. Once a project becomes more mature there are usually some developers who do not need to know and hence should not know certain credentials. In addition code may be reused for slightly different purposes or even become open source. Also, as the code base grows more complex, it becomes very tedious to identify credentials being buried somewhere in the middle of the code.
It seems safe to say that hundreds of millions of users have already been affected by issues caused by hard-coded credentials. Here is an article with some examples.
How to provide credentials to your app
If credentials are not part of the code, this raises the question how you can provide credentials to your application. This depends on the platform on which your application runs. For instance if you host your application on some cloud service, this service will have a mechanism to store credentials in a save way and inject them into the operation system environment of your application. To provide a specific example, here is the documentation how to provide credentials for an app hosted on Heroku.
In you application code you can then access them from the environment. E.g. for Java you could use getenv
String apiPassword = getenv("API_PASSWORD");
Here API_PASSWORD needs to be provided in the environment by the hosting mechanism of your app.
Further Reading
I have written a blog article about the topic that covers this topic in more detail: Keep passwords out of source code - why and how.
It's generally not good advice to encrypt credentials. Something that is encrypted can be decrypted. Common best practice is to store passwords as a salted hash.A hash cannot be decrypted. The salt is added to defeat brute force guessing with Rainbow Tables. As long as every userId has its own random salt, an attacker would have to generate a set of tables for every possible value of the salt, quickly making this attack impossible within the lifespan of the universe. This is the reason why websites generally can't send you your password if you have forgotten it, but they can only 'reset' it. They don't have your password stored, only a hash of it.
Password hashing is not very difficult to implement yourself, but it's such a common problem to solve that countless others have done it for you. I've found jBcrypt easy to use.
As an extra protection against brute force guessing of passwords, it is common best practice to force a userId or remote IP to wait a few seconds after a certain number of login attempts with the wrong password. Without this, a brute force attacker can guess as many passwords per second as your server can handle. There is a huge difference between being able to guess 100 passwords per 10 second period or a million.
I get the impression that you have included the username/password combination in your source code. This means that if you ever want to change the password, you'll have to recompile, stop and restart your service, and it also means that anyone who gets a hold of your source code, also has your passwords. Common best practice is never to do this, but to store the credentials (username, password hash, password salt) in your datastore
secure computer that initializes the request (your computer). if that machine is insecure, nothing will protect you. that's completely separate topic (up-to-date software, properly configured, strong passwords, encrypted swap, hardware sniffers, physical security etc)
secure your storage
the medium you use for storing your credentials should be encrypted. decrypted credentials should be stored only in ram of your secured machine
people that maintain that hardware must be trusted (probably the weakest link)
they also should know as few as possible. that's a protection from rubber-hose cryptanalysis
your credentials should fulfil all the security recommendation (proper length, randomness, single purpose etc)
your connection to remote service must be secured (SSL etc)
your remote service must be trusted (see points 1-4). plus it should be hacking prone (if your data/service is insecure then securing your credentials is pointless). plus it should not store your credentials
plus probably thousand things i forgot about :)
If you cannot trust the environment your program is running in, but need to authenticate via plain passwords or certificates, there is nothing you can do to secure your credentials. The most you can do is obfuscate them with the methods described in the other answers.
As a workaround, I'd run all requests to the RESTful api through a proxy that you can trust and do the cleartext password authentication from there.
why are people talking about hashing. OP wants to store his users credentials to access external resource. Hashing his password will not help.
Now that's out of the way. I would just simple best practices for every layer.
1 . Storing your password in java app. : Store it as Char Array. Create a password store class and store the password as hashmap with key as the resource that you want to access and value as some object that contains of username and password. Restrict the entry point to this api with some authentication Ex: accept the logged in users' credential to validate access level of that user for that resource (simply map user to list of passwords they can access. If you've a lot create a group and map passwordmap key to that group) Anything beyond this to store password depends on how paranoid you're about jvm itself to leak it.
to transmit the password ensure you're sending it on secured portocols (Ex: Https is good, http is bad). If you really must transmit over insecure protocol encrypt it and encode it to say base64. Ensure the recipient decodes and can decrypt your password.