Related
I am implementing ECDH and wanted to ask how the shared secret that is derived from one's private key and another's public key should be best used.
I got the fact that both parties using this end up with the same secret, which is fine. The secret gets compromised once an attacker gets hold of a private key of any of the two parties involved in a communication (Alice or Bob).
It appears that the (shared) secret is always the same when calculating the key agreement using a certain algorithm (in my case AES).
Then you use the secret and use AES/GCM to encrypt whatever one wants.
I think that this is flawed as you always use the same key for communication between the same partner making it possible to record the secret communication and there might be a chance to derive the secret by just cracking AES or whatever is used.
So what I want to do is deriving keys from it instead without ever using the entire secure key as is. So instead of using the secret key both are able to generate independently, I would like to use the secret key to derive a number of keys passing along kind of an offset both sides use to create unlimited one time keys.
My question is what Key Derivation function is usually used (as it is key based)?
I currently implemented a SipHash128 as it promises what I need - keeping the actual secret secret but allow for infinite symmetric secrets by exchanging random numbers as messages (one part from the device and one part from the server) to derive the sip hash from by using the shared secret key avoiding the device being able to always send the same 'random' messages.
PS: I want a Java Backend talking to Flutter and JavaScript apps.
PSS: Yes I have a Master (a colleague challenged me on this already) in Computer Science but feel kind of stupid doing it this way, but it was/is the easiest solution I could come up with.
I am writing a secure file sharing application in Java. The general architecture looks like this:
User wishes to encrypt a file for secure sharing between multiple users.
The application generates a random UUID on the client and uses this as the AES 256 password, and encrypts the data with the UUID.
The UUID is then RSA encrypted with each person's public key. Once per shared user.
Each encrypted UUID packet is stored as part of the file in a custom file header.
The file is then uploaded to a server where others can access it.
The user's can each use their private key to read the AES encryption key and decrypt the file.
Here is the catch. The user's private key must be encrypted and stored on our servers in our database so that the files can be accessed from multiple locations. The private key will be encrypted with a user selected password on the client prior to being uploaded to the server.
I would like to do this using AES 256 bit encryption. And I would like to do the entire thing without relying on BouncyCastle libraries or any 3rd party libraries. It needs to use the standard Java 5 libraries, which is why I have chosen to use AES 256 encryption and RSA rather than something like PGP.
Can anyone find anything inherently insecure with this approach, or think of a more efficient way to do this?
Edit:
OK, I'm updating the question because all of the answers I am getting are suggesting that I not transmit the private key to the server. The reason I need the private key on the server is because the user's need to be able to access their data from multiple clients and multiple locations (ie: their iphone, their ipad, their work laptop, their home pc). They do not want to have to manage and copy their keys from device to device, which is even more insecure than storing their keys on our server because they would just end up emailing them to themselves at that point.
The big problem with this is using UUIDs. Although UUIDs are (sort of) guaranteed to be unique, quite a bit of what they contain is quite predictable; substantial amounts remain constant across all the UUIDs generated on a single machine. As such, if a person gets access to (for example) their own key, they can probably guess many other people's keys fairly easily.
The other part that's problematic is storing user's private keys on the server. This makes the whole rest of the scheme relatively fragile, since access to those keys obviously gives access to all the rest of the data. It also (apparently) means you'll normally be decrypting the data on the server, so when the user accesses that data across the network, it'll either need to be re-encrypted for transmission, and decrypted on the users's machine, or else you'll be transmitting the data in the clear (thus rendering most of the encryption useless).
Edit: As to how I think I'd do this:
I'd have a list of public keys on the server. When a client wants to share a file with some other clients, it obtains the public keys for those clients from the server. It then generates a secure random key, and encrypts the data with that key. It then encrypts the random key with the public keys of all the other clients that are supposed to be able to access the data. Put those together into a stream, and transmit them to the server. The other clients can then download the stream, decrypt the key with their private key, and use that to decrypt the data itself.
This means each client's private key remains truly private -- it never has to leave their machine in any form. All they ever have to share with the rest of the world is their public key (which, by definition, shouldn't cause a security problem).
With that, the two obvious lines of attack are against the random number generator, and against RSA itself. For the random number generator, I'd use Java's SecureRandom -- this is exactly the sort of purpose for which it's intended, and if memory serves it's been pretty carefully examined and significant breaks against it seem fairly unlikely.
I won't try to comment on the security of RSA itself. For now, I think your primary concern is with the protocol, not the encryption algorithm proper. Suffice to say that if RSA were significantly broken, you'd obviously need to change your code, but you'd have a lot of company.
With this, it's pretty much up to the client to store their private keys securely. I like smart cards for that job, but there are quite a few alternatives. From the viewpoint of the server and protocol, it's no longer really a factor at all though.
Edit 2: As for dealing with multiple devices, I think I'd simply treat each device as a separate user, with its own public/private key pair. I'd then (probably) group those together by the actual users, so I can easily choose "Joe Blow" to give him access on all his devices -- but with a hierarchical display, I could also pretty easily restrict access to a subset of those, so if I want to share it with Joe on his office machine, but it's sensitive enough that I don't want it going where somebody might look over his shoulder while he looks at it, I can pretty easily do that too.
This keeps life simple for the users, but retains the same basic security model (i.e., private keys remain private).
The scheme you outline is equivalent to CMS (the standard underlying S/MIME) and PGP; fundamentally, it is secure. In CMS, this mode is called "key transport". You could also use multi-party "key agreement," with an algorithm like DH or ECDH.
The only problem is that you are using poorly chosen keys for AES.
I can't think of any reason to use a random UUID, which contains non-random bits. Just use the normal key generation mechanism of the Java Cryptography Architecture. Keys, plaintext, and ciphertext should all be represented as byte sequences, unless you need to accommodate some external storage or transport that only accommodates text.
Iterable<Certificate> recipients = null;
KeyGenerator gen = KeyGenerator.getInstance("AES");
gen.init(256);
SecretKey contentEncryptionKey = gen.generateKey();
Initialize the AES cipher and let the provider choose an IV.
Cipher contentCipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
contentCipher.init(Cipher.ENCRYPT_MODE, contentEncryptionKey);
AlgorithmParameters params = contentCipher.getParameters();
byte[] iv = params.getParameterSpec(IvParameterSpec.class).getIV();
For each recipient, initialize the RSA cipher and encrypt the AES key.
Cipher keyEncryptionCipher = Cipher.getInstance("RSA/ECB/PKCS1Padding");
for (Certificate recipient : recipients) {
keyEncryptionCipher.init(Cipher.WRAP_MODE, recipient);
byte[] encryptedKey = keyEncryptionCipher.wrap(contentEncryptionKey);
/* Store the encryptedKey with an identifier for the recipient... */
}
/* Store the IV... */
/* Encrypt the file... */
Having users select and remember passwords that give 256 bits of effective strength is unreasonable. To get that strength, you'd have to randomly choose passwords, encode them as text, and have users write them down on a card. If you really need that much strength, you could check out a smart-card–based solution for storing the users' RSA keys.
I'd highly recommend using a CMS library to store your files. It will increase your chances that the protocol you're using is safe, the code you are using has had more review, and that other tools, libraries, and systems can inter-operate with the encrypted messages. BouncyCastle's API is a little obscure, but it might be worth learning it.
(I can't remember if Java 5 supports "RSA/ECB/OAEPWithSHA-512AndMGF1Padding"; if it does, you should use that instead of PKCS1Padding.)
OK, this question is asking for a protocol discussion, so it is not completely according to stackoverflow's standards. That said, let's see if we can make some remarks anyway :) :
The Bouncy Castle PGP libraries have a very permissive license, so you can even copy them into a sub-package within your code;
Besides PGP there are also other standard container formats such as CMS or XML encryption, although the latter might not be such a good general purpose format;
Instead of UUID's, I would strongly suggest to use a well seeded PRNG such as the Java JCE "SHA1PRNG" to create the AES keys - I don't see any strong reason why you should rely on something like an UUID in your scheme;
AES keys are supposed to consist of random bits to have enough entropy, thinking of them as "passwords" is leading into a trap: you cannot use a String as a secure AES key;
The user will have to trust your application and server, you are acting as a trusted third party: you can send user's passwords to your server, you can send incorrect public keys to the users etc. etc. etc.
Your scheme is not protected against any man in the middle attacks (and many argue this cannot be done without using SSL)
Instead of directly encrypting with a password, you should look into something like Password Based Encryption PBKDF2 to encrypt the RSA private key;
Try and add integrity protection when encrypting, from Java 7 onwards you may use AES in GCM mode, it's well worth it.
It all depends on how "secure" you want the encryption to be. Obviously RSA is a well document/accepted standard for PKI. That being said, any time you provide the plaintext as well as the encrypted text, it makes it significantly easier for a hacker to decrypt the ciphertext knowing part of the plaintext. Here, you are doing precisely that. Although you are only transmitting the encrypted UUID, by having the same plaintext encrypted with multiple keys gives an attacker significant insight into the payload. Furthermore, if the hacked is actually one of the recipients, he is able to decode the UUID, and thereby automatically knows the plaintext that is being encrypted by the other users' public keys.
This is not likely a critical issue for you, but just thought I would point out a security risk.
I am not entirely sure why you need to store the user's private key, however. Furthermore, by using a simple password to encrypt the private key, you have basically reduced the overall security of the entire system to the strength of the user's password. Finally, if the user loses his password, he is toast; no way to recover any data.
I did something similar in the past but stored the results in a DB. However, I used the BouncyCastle libraries at the time, so am not sure how to accomplish this without them.
I have a question relating to the use of an Initialization Vector in AES encryption. I am referencing the following articles / posts to build encryption into my program:
[1] Java 256-bit AES Password-Based Encryption
[2] http://gmailassistant.sourceforge.net/src/org/freeshell/zs/common/Encryptor.java.html
I was originally following erickson's solution from the first link but, from what I can tell, PBKDF2WithHmacSHA1 is not supported on my implementation. So, I turned to the second link to get an idea for my own iterative SHA-256 hash creation.
My question comes in how the IV is created. One implementation ([1]) uses methods from the Cypher class to derive the IV where are the other ([2]) uses the second 16 bytes of the hash as the IV. Quite simply, why the difference and which is better from a security standpoint? I am kinda confused to the derivation and use of IVs as well (I understand what they are used for, just not the subtler differences), so any clarification is also very welcome.
I noticed that the second link uses AES-128 rather than AES-256 which would suggest to me that I would have to go up to SHA-512 is I wanted to use this method. This seems like it would be an unfortunate requirement as the user's password would have to be 16 characters longer to ensure a remotely secure hash and this app is destined for a cell phone.
Source is available on request, though it is still incomplete.
Thank you in advance.
The IV should not be generated from the password alone.
The point of the IV that even with the same key and plaintext is re-used, a different ciphertext will be produced. If the IV is deterministically produced from the password only, you'd get the same ciphertext every time. In the cited example, a salt is randomly chosen, so a new key is generated even with the same password.
Just use a random number generator to choose an IV. That's what the cipher is doing internally.
I want to stress that you have to store either the IV (if you use the first method) or a salt (if you use the second method) together with the ciphertext. You won't have good security if everything is derived from the password; you need some randomness in every message.
Cryptographers should generate IVs using a secure pseudo-random random number generator.
Application developers should use existing, off the shelf cryptography. I suggest that you use SSL with certificates to secure your network traffic and GPG to secure file data.
There are so many details that can make an implementation insecure, such as timing attacks. When an application developer is making decisions between AES 128 and AES 256 it is nearly always pointless since you've likely left a timing attack that renders the extra key bits useless.
My understanding is that Initialization Vector is just random input to encryption algorithm, otherwise you would always get same result for same input. Initialization Vector is stored together with cipher text, it's not secret in any way. Just use secure random function to generate initialization vector. PBKDF* algorithms are used to derive secret keys of desired length for encryption algorithms from user-entered passwords.
First implementation that you link to simply lets Cipher object to generate Initialization Vector. Then it fetches this generated IV to store it together with cipher text.
Second one uses part of hash bytes. Any approach that generates non-repeating IVs is good enough.
Most important property of IV is that it doesn't repeat (very often).
The IV is just a consequence of the use of block chaining. I presume that this is more than a simple API design question. I assume that you know that the reasoning for using it is so that the same plaintext will not show up as the same ciphertext in multiple blocks.
Think about recursion from the last block where the Nth ciphertext block depends in some way on the (N-1)th block, etc. When you get to the first block, 0th block, you need some data to get started. It doesn't matter what that data is as long as you know it before you attempt to decrypt. Using non-secret random data as an initialization vector will cause identical messages encrypted under the same key to come out as completely different ciphertext.
It's similar in concept to salting a hash. And that source code looks a little fishy to me. An IV should simply be fresh-at-encryption-time random bits dependent upon nothing, like a nonce. The IV is basically part of the encrypted message. If you re-encrypt identical data with identical key, you should not be able to correlate the messages. (Hey, think about the consequences of correlating by ciphertext length as well.)
As with everyone else here, I've always known IVs to be just chosen randomly using the standard algorithms for doing so.
The second reference you provided, though, doesn't seem to be doing that. Looks like he salts a password and hashes it. Then takes that hash and splits it up into halves. One half is the encryption key, one is the IV. So the IV is derived from the password.
I don't have any strong breaks for such a method, but it's bad design. The IV should be independent and random all on its own. Maybe if there's a weakness in the hashing algorithm or if you choose a weak password. You don't want to be able to derive the IV from anything else or it's conceivable to launch pre-computation attacks.
I need to develop a feature in the system which allows unregistered users to get one-off system access via URL token that is generated/sent by an authenticated user.
For example, a user logs in and wants to share a piece of information so the system generates a URL like http://host/page?token=jkb345k4b5234k54kh5345kb34kb34. Then this URL is sent to an unregistered user who would follow the URL to get some limited access to normally protected data.
First question - are there any standards (RFC? IETF? others?) that would be defining URL generation? The only ones I was able to find are RFC2289 and OpenToken, but none of these are directly related to what I need to do and the latter is only in a second draft state.
There is another design consideration: whether to use one way crypto hash functions and store the payload in a local data store VS using private-public key pairs and encode all necessary payload in the unique string itself.
At the moment I am heavily leaning towards one way hash as it would give me much more freedom (no dependency between payload size and generated string) and less potential problems in the future (e.g. what if I decide to add more payload - how to ensure backwards compatibility). Last but not least, accidental exposure of server-side private key would require massive efforts in key regeneration, update of all live instances, etc. None of these problems are relevant if choosing one-way hash option, but maybe there's something I overlook? RFC2289 prefers one way crypto function whereas OpenToken chooses the key pair option.
And finally, is anybody aware of any Java library for generating these?
Thanks in advance.
Also have a look at http://en.wikipedia.org/wiki/Universally_unique_identifier and RFC4122. Inside the backend you would need to attach the generated uuid to your entity so verification based on the UUID can be done later.
Apart from that most often the token could include some data (e.g. versioning+userdata) and then a secure MD5-hash is used to 'obfuscate/anonymize' it. Later then the data is concatenated by server and the hash-values are compared again.
Regarding java-lib and uuid have a look at UUID-javadoc.
Generate random strings and store them in a database with credentials.
The codes generated need to have two properties: complexity and uniqueness. Complexity ensures that they cannot be guessed and uniqueness ensures that the same code can never be generated twice. Beyond this, the specific method doesn't matter.
Generate token strings with two parts to them. The first part is time-dependent, where the key will increment and change in a predictable way with each millisecond. The second part is completely random. Combined, this will give you a long string that is unique and complex.
When you generate the token, store it in the database with the credentials that are granted when this token is used. It's important that these credentials are not encoded into the string, since this ensures that the strings cannot be hacked.
When the user click on the link with the token, mark that token as used in the database. Even better is to set a timestamp for the use, so that it can be expired, perhaps, 24 hours after the first click. This approach gives you the flexibility to implement this specific part of the requirement as necessary for your project.
I've used this solution before in many different cases for not only one-off system access, but also for ticket admission codes, gift certificate codes, and anything that's one-time use. It doesn't matter so much what you use to generate the token, so much as you can guarantee its complexity and uniqueness.
Here's how I would have done it:
Create a token (you could use a UUID for this) and add it to your database along with creation time and what resource the token should grant access to
Send an email to the user with the url http://www.myserver.com/page?token=
When the user navigates to the url, create a new session with the desired timeout and mark that session as authorized to view whatever the database says the user should be able to see (If the token isn't too old. Check the creation time against current time)
Either delete the token from the database, or mark it as expired
You only need a token when a user shares one piece of information. So, can't you just generate a random token, and associate this with the piece of information (e.g. a database field)? It's a lot simpler than doing any crypto stuff...
I am making a licensing system when clients ask my server for a license and I send them a license if they are permitted to have one.
On my current system I encrypt the license using a single private key and have the public key embedded into the client application that they use to decrypt the license. It works!
Others have told me that I should be encrypting with the public key on the server and distributing the private key to clients. I have searched the web and can see that sometimes they use the private key to encrypt and other times they use the public key to encrypt.
In this case what am I supposed to do?
Others have told me that I should be
encrypting with the public key on the
server and distributing the private
key to clients.
Those people are wrong. The name private key implies that it is private meaning that only you should have access to it.
In this case what am I supposed to do?
Use digital signatures. Sign the license file with your private key and use your public key in your application to verify that the signature on the license came from you.
Congratulations, you just invented the RSA signature. (Which is what you should be using, anyway.) To communicate with a public key system, you need to use the private key once and the public key once, but RSA supports two different orders:
1) Encrypt with the public key, decrypt with the private: The recipient doesn't know anything about the source of the message, but the sender knows that only the recipient (the holder of the private key) can read it. This is classical "encryption".
2) "Encrypt" with the private key, then "decrypt" with the public. This is a digital signature, and provides authentication. Anyone can read the message, but only the private key holder could have sent it.
Assuming your license is customized to the client (which could be as simple as including a copy of a client-generated random number), then it's useless to anyone else, but the client can be sure that the server sent it.
The symmetry isn't quite that neat in practice; the different modes of operation have different weaknesses and gotchas, so the implementation is typically significantly different, but that's the general idea.
One of the first and most important lessons in cryptology is understanding authentication and when to use it. It's needed at least as often as encryption, and not knowing when to use it leaves you in a Midvale School for the Gifted situation.
If you are encrypting something that is only to be read by a single recipient, then you encrypt with that recipients public key and they use their private key to read it.
If you are encrypting for multiple recipients, then you can encrypt with your private key and distribute your public key to those which you want to be able to read it. This is usually called "signing" as anyone who has access to your public key can read it, so it's not really a form of private communication.
An overall more robust solution for you would be for your app to generate a key pair per installation, send the public key that it generated back to the server, which you would then use to encrypt so that only that single install could use the license that you created (by decrypting it with its private key).
At least in a typical public key encryption algorithm (e.g., RSA) there's not really a major difference between the public and the private key. When you generate keys, you get two keys. You keep one private and publish the other -- but it doesn't matter much which one you publish and which one you keep private.
Anything you encrypt with one key can be decrypted with the other key. For normal purposes, you publish one key, which lets anybody encrypt something that only you can decrypt. From a technical viewpoint, the reverse works fine though: if you encrypt something with your private key, anybody with the public key can decrypt it. This is typically used for things like signature verification (i.e., anybody with the public key can verify that the signature had to have been created with the private key). You usually want to use separate key pairs for encryption and signing though.
For your case, it's open to some question what you're really going to accomplish. You can certainly encrypt some data necessary to use the program, so the user needs the key to decrypt it and use the program -- but if the user is willing to give a copy of the code to an unauthorized person, they probably won't hesitate at giving a copy of the key to them as well. As such, even though the encryption/decryption will do it's job, it's unlikely to provide any real protection.
A more typical licensing scheme is tied to something like a specific IP address, so you do something like encrypting the IP address, then use the result as a key to decrypt data necessary to use the program. If the IP address is wrong, the data isn't decrypted correctly, and the program doesn't work. As long as the user has a static IP address this can work well -- but will cause problems in conjunction with DHCP.
My immediate advice would to just not do this at all. If you insist on doing it anyway, don't do it yourself -- get something like FlexNet to handle it for you. You're better off without it, but at least this way you'll get something that sort of works, and you won't waste time and effort on it that could be put to better purposes like improving your software.
If you are using public-private (asymmetric) encryption, you always encrypt with the recipient's public key, who decrypts with their private key. For digital signatures, you sign with your private key and the recipient verifies the signature with their public key.
The question then arises, how do you make a secure DRM system? If you use encryption, and give the recipients a private key, they can distribute either the key or the decrypted content. If you use signatures, they can simply strip out the signature verification part of your program.
The answer is that it's impossible. The concept of DRM is fundamentally flawed.
Hope this link from wikipedia helps. PKI is based on mutual trust. However the private key has to be protected by the owner. Public as the name implies is open to all. The entire architecture is made inorder to help the scenario as defined above in your question.