My application uses Google protocol buffers to send sensitive data between client and server instances. The network link is encrypted with SSL, so I'm not worried about eavesdroppers on the network. I am worried about the actual loading of sensitive data into the protobuf because of memory concerns explained in this SO question.
For example:
Login login = Login.newBuilder().setPassword(password)// problem
.build();
Is there no way to do this securely since protocol buffers are immutable?
Protobuf does not provide any option to use char[] instead of String. On the contrary, Protobuf messages are intentionally designed to be fully immutable, which provides a different kind of security: you can share a single message instance between multiple sandboxed components of a program without worrying that one may modify the data in order to interfere with another.
In my personal opinion as a security engineer -- though others will disagree -- the "security" described in the SO question to which you link is security theater, not actually worth pursuing, for a number of reasons:
If an attacker can read your process's memory, you've already lost. Even if you overwrite the secret's memory before discarding it, if the attacker reads your memory at the right time, they'll find the password. But, worse, if an attacker is in a position to read your process's memory, they're probably in a position to do much worse things than extract temporary passwords: they can probably extract long-lived secrets (e.g. your server's TLS private key), overwrite parts of memory to change your app's behavior, access any and all resources to which your app has access, etc. This simply isn't a problem that can be meaningfully addressed by zeroing certain fields after use.
Realistically, there are too many ways that your secrets may be copied anyway, over which you have no control, making the whole exercise moot:
Even if you are careful, the garbage collector could have made copies of the secret while moving memory around, defeating the purpose. To avoid this you probably need to use a ByteBuffer backed by non-managed memory.
When reading the data into your process, it almost certainly passes through library code that doesn't overwrite its data in this way. For example, an InputStream may do internal buffering, and probably doesn't zero out its buffer afterwards.
The operating system may page your data out to swap space on disk at any time, and is not obliged to zero that data afterwards. So even if you zero out the memory, it may persist in swap. (Encrypting swap ensures that these secrets are effectively gone when the system shuts down, but doesn't necessarily protect against an attacker present on the local machine who is able to extract the swap encryption key out of the kernel.)
Etc.
So, in my opinion, using mutable objects in Java specifically to be able to overwrite secrets in this way is not a useful strategy. These threats need to be addressed elsewhere.
Related
Recently we have added a tool to find security holes in our organization. One of the issues that was found is that when connecting to a database (ex. using Hikari), we have to provide a String containing the password (encrypted, of course, which will be decrypted when used).
Now, keeping passwords in Strings is not safe, because it can be extracted, until garbage collector comes and clears it.
So we started changing our code to use char[] and byte[] (not sure it's the best, but the idea is that we can clear the array after usage, and not wait for garbage collector to clear it for us) to set our passwords on Hikari, but the last part of the flow is setting an unencrypted password String to Hikari. So all this fuss to find out that Hikari is keeping the password inside a String..
So am I supposed to change Hikari code and recompile it as our own organization implementation of Hikari which use passwords from a char[]? or what?
How can we avoid this?
Now, keeping passwords in Strings is not safe, because it can be extracted, until garbage collector comes and clears it.
Only if someone has sufficient access to capture what is in memory (or swap space on disk). If someone can do that, they can probably also do one or more of the following:
modify your application at the bytecode level to inject code to capture the secret
attach a debugger and use it to set a breakpoint at the point where the secret is used
read the secret from the file system, database, whatever
find the private key for your server's SSL certs and use it to snoop on the HTTPS traffic to your server,
walk out of your machine room with your hard drives, etc and then attack them at their leisure
and so on.
Spending a lot of effort to use char[] for handling passwords won't address any of those other ways of stealing the secrets.
And it won't address various other security blunders ... like porous firewalls, unencrypted backups saved to the cloud, keys on a stolen devops laptop, successful spear phishing, etc.
So am I supposed to change Hikari code and recompile it as our own organization implementation of Hikari which use passwords from a char[]? or what?
That is what you would need to do if you wanted to address this attack vector. Don't ever hold passwords in String objects, and make sure that you clear the char[] or byte[] or whatever that you use to hold them as soon as possible.
Are you "supposed" to do that? Shrug.
My advice would be to do a full security risk assessment, look at all of the issues and decide whether or not addressing this one will make a significant difference to overall security. Balance that against the costs of creating and maintaining the Hikari patches. On the flip-side, quantify the costs to your organization if (these) passwords were stolen.
But it is not up to us to decide what you should. And it is not even possible to give you an objective recommendation, because we don't understand the full context.
I learned about using char[] to store passwords back in the Usenet days in comp.lang.java.*.
Searching Stack Overflow, you can also easily find highly upvoted questions like this:
Why is char[] preferred over String for passwords? which agrees with what I learned a long, long time ago.
I still write my APIs to use char[] for password. But are that just hollow ideals now?
For example, look at Atlassian Jira's Java API:
LoginManager.authenticate which takes your password as a String.
Or Thales' Luna Java API:
login() method in LunaSlotManager. Of all people, an HSM vendor using String for the HSM slot password.
I think I've also read somewhere that the internals of URLConnection (and many other classes) uses String internally to handle the data. So if you ever send a password (although the password is encrypted by TLS over the wire), it will be in a String in your server's memory.
Is accessing server memory an attack factor so difficult to achieve that it is okay to store passwords as String now? Or is Thales' doing it because your password will end up in a String anyway due to classes written by others?
First, let’s recall the reason for the recommendation to use char[] instead of String: Strings are immutable, so once the string is created, there is limited control over the contents of the string until (potentially well after) the memory is garbage collected. An attacker that can dump the process memory can thus potentially read the password data. Meanwhile the contents of the char[] object can be overwritten after it has been created. Assuming that this is done, and that the GC hasn’t moved the object to another physical memory location in the interim, this means that the password contents can be destroyed (somewhat) deterministically after it has been used. An attacker reading the process memory after that point won’t be able to get the password.
So using char[] instead of String lets you prevent a very specific attack scenario, where the attacker has full access to the process memory,1 but only at specific points in time rather than continuously. And even under this scenario, using char[] and overwriting its contents does not prevent the attack, it just reduces its chance of success (if the attacker happens to read the process memory between the creation and the erasure of the password, they can read it).
I am not aware of any evidence that shows (a) how frequent this scenario is, nor (b) how much this mitigation reduces the probability of success under that scenario. As far as I know, this is pure guesswork.
In fact, on most systems, this scenario likely does not exist at all: an attacker who can get access to another process’ memory can also gain full tracing access. For instance, on both Linux and Windows any process that can read another process’ memory can also inject arbitrary logic into that process (e.g. via LD_PRELOAD and similar mechanisms2). So I would say that this mitigation at best has a limited benefit, and potentially none at all.
… Actually I can think of one specific counter-example: an application that loads an untrusted plugin library. As soon as that library is loaded via conventional means (i.e. in the same memory space), it has access to the parent application. In this scenario, it might make sense to use char[] instead of String, and overwrite its contents when done with it, if the password is handled before the plugin is loaded. But a better solution would be not to load untrusted plugins into the same memory space. A common alternative is to launch it in a separate process and communicate via IPC.
(See the answer by Gilles for more vulnerable scenarios. I still think that the benefit is relatively limited, but it’s clearly not nil.)
1 As shown in Gilles’ answer, this is not correct: no full memory access is required to mount a successful attack.
2 Although LD_PRELOAD specifically requires the attacker to not only have access to another process but either to launch that process, or to have access to its parent process.
(Note: I am a security expert but not a Java expert.)
Yes, there is a significant security advantage in using char[] rather than strings for passwords. This also applies to some extent to other highly confidential data, although most highly confidential data (e.g. cryptographic keys) tends to be bytes and not characters.
The old, and still valid, reason to use char[] is to clean up memory as soon as it is used, which is not possible with String. This is a very firmly established security practice. For example, in the (in)famous FIPS 140 requirements for cryptographic processing, which are generally considered to be security requirements, there are in fact extremely few security requirements at level 1 (the easiest level). Just two, in fact: one is that you may only used approved cryptographic algorithms, and the other one is that keys, passwords and other sensitive data must be wiped after use.
This practice is one of the reason why production implementations of cryptographic primitives are usually implemented in languages with manual memory management such as C, C++ or Rust: cryptography implementers want to retain control of where sensitive data goes, and to be sure to wipe all copies of sensitive material.
As an example of what can go wrong, consider the (in)famous Heartbleed bug. It allowed anyone on the Internet connecting to a vulnerable server to dump some of the memory of the server, without being detected. The attacker didn't get much control over which part of the memory, but could try again and again. An attacker could make requests that would cause the dumpable part to move around the heap, and thus could potentially dump the whole memory.
Are such bug common? No. This one got a lot of buzz because it was in a very popular software and the consequences were bad. But such bugs do exist and it's good to protect against them.
In addition, since Java 8, there is another reason, which is to avoid string deduplication. String deduplication means that if two String objects have the same content, they may be merged. String deduplication is problematic if an attacker can mount a side channel attack when the deduplication is attempted. The attack does not require the password to be deduplicated (although it is easier in this case): there's a problem as soon as some code compares the password against another string.
The usual way to compare strings for equality is:
If the lengths are different, return false.
Otherwise compare the characters one by one. As soon as there are different characters at one position, return false.
If the end of the strings is reached without encountering a difference, return true.
This has a timing side channel: the time of the middle step depends on the number of identical characters at the beginning of the string. Suppose that an attacker can measure this time, and can upload some strings for comparison (e.g. by making legitimate requests to a server). The attacker notices that comparing with sssssssss takes slightly longer than comparing with aaaaaaaaa, so the password must begin with s. Then the attacker tries to vary the second character, and finds that comparing with swwwwwwww takes again slightly longer. And thus, in relatively short time, the attacker can reconstruct the password character by character.
In the context of string deduplication, the attack is harder, because (as far as I know) the deduplication code first hashes the strings to compare. This may mean that the attacker has to first guess the hash value. But the total number of hash values in a given hash table (that's the number of hash buckets, not the full range of the hash method) is small enough that it's practical to enumerate.
This is not an easy attack, to be sure. But I would absolutely not rule it out, especially with a local attacker, but even with a remote attacker. Remote timing attacks are practical (still).
In conclusion, yes, you should not use String for passwords. Read them as char[], keep careful track of any copies, hash them as soon as possible if you're verifying them, and wipe all copies.
If you need to store a password for a third-party service, it's a good idea to store it in encrypted form even if there is no separate access control for the encryption key. Copies of an encrypted password are less prone to leaking through side channels than copies of the password itself, which is a printable string with low entropy.
I think I've also read somewhere that the internals of URLConnection (and many other classes) uses String internally to handle the data. So if you ever send a password (although the password is encrypted by TLS over the wire), it will be in a String in your server's memory.
I'm not a Java expert, but this doesn't sound right: the plaintext of a connection (TLS or otherwise) is a byte stream, not a character stream. It should be arrays of 8-bit bytes, not arrays of Unicode code points.
Or that your password will end up in a String anyway due to classes written by others, is that why Thales' doing it.
Possibly. Or possibly because they aren't Java experts, or because the people who write the high-level layers are often not the foremost security experts.
Almost everyone else's answer plus one additional point:
Swap space on a storage media.
If the JVM heap is ever paged out to disk and the password is still in memory as a string (immutable and not GC'd), it will be written to the swap file. This swap file can then be scanned for password values, so, essentially another attack vector that's time based and still rather difficult to utilize but obviously not that difficult because we're here :D.
Wiping the mutable array at least reduces the time where the password is in memory.
The story I heard was that if an attacker can attack your process (like a DDOS) and trigger the process to swap out, it's somewhat easier to attack the swap space than the memory, AND swap space is preserved across boots/crashes/etc. This allows for yet another attack vector where the attacker pulls the swap drive out to scan the swap space.
Lots of detail in the answers but here's the short of it: yes, in theory, putting the password in an array and wiping it provides security benefits. In practice, that only helps if you can avoid the password ever being stored in a String. That is, if you take a password stored in a String and put the contents of the String into a char[], it doesn't magically make the String disappear from the heap. The necessary requirement is that the password never is placed in a String at all. I'd be interested to see that successfully implemented in a real Java application.
It's not an idea of the moment of transfer over the network. There indeed you're indeed better off using a String as it's just more convient to use to send over the network, of course making sure it's properly encrypted.
For using passwords in applications it's different due to stack-dumps and reverse engineering, and the problem of the String being immutable:
In case the password has been entered, even if the reference to the variable is changed to another string, there is no certainty about when the garbage collector will actually remove the String from the heap. So a hacker being able to see the dump will also be able to see the password. Using an array of char prevents this as you can change the data in the array directly without relying on the garbage collector.
Now you might say: well then when sending it over the network as a String it'll still be visible no? Well yes, but that's why encrypting it before sending it is important. Never send plain text passwords over the network when possible.
Strings in Java are kept around for a while, potentially a long while. This is a good thing, unless that String contains a user's actual password. Character arrays are suggested because they aren't immutable and can be cleared faster. (Let's hope there's never a "Bitter Coffee" attack that works like Heartbleed but against the JVM (remote heap dump)).
I notice that Spring Security PasswordEncoder takes a CharSequence not a String, possibly for this reason. However I'm not sure what object I should use to keep the password in memory pre-hashing. Would StringBuilder be appropriate? what would that look like? I'm even less sure if I'm creating a REST API (with Jackson under the hood, via either Spring Data or Spring MVC) how I can keep that from ever being a String.
How can I code a JSON REST API for creating/updating passwords whilst being as secure as possible and avoiding the various problems with using Strings?
How can I code a JSON REST API for creating/updating passwords whilst
being as secure as possible and avoiding the various problems with
using Strings?
Well API keys are not true passwords. You have control on how API keys are created so you can create some random string which will have a very low collision (ie double UUID) and very low common substring (in the case of dedup). After the client logs in through the REST API using the key you could use temporary tokens thus improving the likelihood of the API key getting garbage collected.
As for dealing with real passwords which is the case for a human logging in (perhaps to reset the API key) you don't really have much options given that almost every servlet container will turn request parameters into strings. One cheesy option is to have the client through Javascript (or whatever your clients are) Base64 encode the password and then add a separator and then add a randomly generated number or string to the password. This is not really for obfuscation but again to lower the probability of keeping the same string around. You'll have to be careful of course to decode into char or byte array and then remove the random suffix by manipulating the char or byte array (see CharBuffer).
Another complicated option is the microservice cloud approach. Just make an authentication service composed of a couple of tiny round robin instances that only do authentication. Have those JVM instances get restarted frequently (to flush memory). Or if they are small enough they will hopefully garbage collect more frequently.
I'll assume of course that your data repository has salted passwords (otherwise this safety precaution is pretty moot).
To be honest though there are so many other threats that I don't really think its worth the effort for most use cases in a HTTP server environment.
The reason why Java Swing uses char[] for password because Swing is used for a desktop environment. Desktop environments are far more likely to have malicious programs such as virus/spyware that could do some memory probing for passwords.
With that in mind its really the clients you should worry about and not the server.
Memory mapped files are (according to the spec) largely dependent on the actual implementation of the OS and a number of these unknown aspects are already explained in the javadoc. However I have some additional questions and not sure where to turn to for answers.
Suppose application A maps a file to memory from position=0 to size=10.
I would assume the OS needs a continuous piece of memory to map it? Or is this dependent on implementation?
Now suppose we have an application B that maps from position=0 to size=11.
Are the first 10 bytes shared or is it an entirely different mapping? This relates back to the continuous memory question.
If we want to use mapped files for IPC, we need to know how the data is reflected in other applications, so if B writes to memory, does A see this?
However as I read the spec, this depends on the OS. This makes it dangerous to use for general purpose IPC as it destroys portability right?
Additionally suppose the OS does support it, so B writes to memory, A sees the change, what happens if we do this:
B.write("something");
A.write("stuff");
A.read();
What exactly will A read?
Or put otherwise:
how are the file pointers managed?
How does it work with concurrency, is there cross application locking?
You can assume that every operating system will perform the memory mapping in terms of blocks which usually have a size which is either a power of two of a multiple of a power of two and significantly larger than 11 bytes.
So regardless of whether you map from 0 to 10 or from 1 to 11, the underlying system will likely establish a mapping from 0 to blocksize to a logical address X which will be perfectly hidden to the Java programmer as the returned ByteBuffer has its own address pointer and capacity so it can always be adjusted so that, e.g. position 0 yield to address X + 1. But whether the underlying system or Java’s MappedByteBuffer performs the necessary translation is not important.
Usually, operating systems will end up using the same physical memory block for a mapping to the same region of the same file so it is a reasonable way of establishing IPC, but as you have already guessed, that is indeed OS dependent and not portable. Still, it might be useful if you make it optional and let the user who knows that his system supports it, can enable it.
Regarding your question about the two writes, of course, if two applications write to the same location concurrently, the result is entirely unpredictable.
Mapping a file region is independent from locking but you may use the file channel API to lock the region you are mapping to gain exclusive access.
Is there anyway in Java to delete data (e.g., a variable value, object) and be sure it can't be recovered from memory? Does assigning null to a variable in Java delete the value from memory? Any ideas? Answers applicable to other languages are also acceptable.
Due to the wonders virtual memory, it is nearly impossible to delete something from memory in a completely irretrievable manner. Your best bet is to zero out the value fields; however:
This does not mean that an old (unzeroed) copy of the object won't be left on an unused swap page, which could persist across reboots.
Neither does it stop someone from attaching a debugger to your application and poking around before the object gets zeroed or crashing the VM and poking around in the heap dump.
Store sensitive data in an array, then "zero" it out as soon as possible.
Any data in RAM can be copied to the disk by a virtual memory system. Data in RAM (or a core dump) can also be inspected by debugging tools. To minimize the chance of this happening, you should strive for the following
keep the time window a secret is
present in memory as short as
possible
be careful about IO pipelines (e.g.,
BufferedInputStream) that internally
buffer data
keep the references to the secret on the stack and out of the heap
don't use immutable types, like
String, to hold secrets
The cryptographic APIs in Java use this approach, and any APIs you create should support it too. For example, KeyStore.load allows a caller to clear a password char[], and when the call completes, as does the KeySpec for password-based encryption.
Ideally, you would use a finally block to zero the array, like this:
KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType());
InputStream is = …
char[] pw = System.console().readPassword();
try {
ks.load(is, pw);
}
finally {
Arrays.fill(pw, '\0');
}
Nothing gets deleted, its just about being accessible or not to the application.
Once inaccessible, the space becomes a candidate for subsequent usage when need arises and the space will be overwritten.
In case of direct memory access, something is always there to read but it might be junk and wont make sense.
By setting your Object to null doesn't mean that your object is removed from memory. The Virtual Machine will flag that Object as ready for Garbage Collection if there are no more references to that Object. Depending on your code it might still be referenced even though you have set it to null in which case it will not be removed. (Essentially if you expect it to be garbage collected and it is not you have a memory leak!)
Once it is flagged as ready for collection you have no control over when the Garbage Collector will remove it. You can mess around with Garbage Collection strategies but I wouldn't advise it.
Profile your application and look at the object and it's id and you can see what is referencing it. Java provide VisualVM with 1.6.0_07 and above or you can use NetBeans
As zacherates said, zero out the sensitive fields of your Object before removing references to it. Note that you can't zero out the contents of a String, so use char arrays and zero each element.
Nope, unless you have direct answer to hardware. There is a chance that variable will be cached somewhere. Sensitive data can even be stored in swap :) If you're concerning only about RAM, you can play with garbage collector. In high level langs usually you don't have a direct access to memory, so it's not possible to control this aspect. For example in .NET there is a class SecureString which uses interop and direct memory access.
I would think that your best bet (that isn't complex) is to use a char[] and then change each position in the array. The other comments about it being possible for it to be copied in memory still apply.
Primitive data (byte, char, int, double) and arrays of them (byte[], ...) are erasable by writing new random content into them.
Object data have to be sanitized by overwriting their primitive properties; setting a variable to null just makes the object available for GC, but not immediately dead. A dump of VM will contain them for anyone to see.
Immutable data such as String cannot be overwritten in any way. Any modification just makes a copy. You shall avoid keeping sensitive data in such objects.
P.S. If we talk about passwords, it's better to use crypto-strong hash functions (MD5, SHA1, ...), and never ever work with passwords in clear text.
If you're thinking about securing password/key management, you could write some JNI code that uses platform-specific API to store the keys in a secure way and not leak the data into the memory managed by the JVM. For example, you could store the keys in a page locked in physical memory and could prevent the IO bus from accessing the memory.
EDIT: To comment on some of the previous answers, the JVM could relocate your objects in memory without erasing their previous locations, so, even char[], bytes, ints and other "erasable" data types aren't an answer if you really want to make sure that no sensitive information is stored in the memory managed by the JVM or swapped on to the hard drive.
Totally and completely irretrievable is something almost impossible in this day and age.
When you normally delete something, the onlything that happens is that the first spot in your memory is emptied. This first spot used to contain the information as to howfar the memory had to be reserved for that program or something else.
But all the other info is still there untill it's overwritten by someone else.
i sudgest either TinyShredder, or using CCleaner set to the Gutmann-pass