I think i'm looking for some sort of basic file encryption but don't know where to start.
I'm looking for someone to tell me where to start looking or, even better, offer some code.
I've written a game that currently saves data to a general text file. This of course could be changed by anyone who wished to do so.
What i need is to create a file that can store integers and strings that is difficult if not impossible to be edited outside of the game.
In my searching i came across .dat files but they seemed more complicated that what i'm looking for.
All help is appreciated, Alex.
You can write your data to a ByteBuffer and then you can distort your data by a simple algorithm. For example, assume that the data you want to save is a String array, you can do this:
String[] data; // the data you want to save
int byteLength = 0;
byte[][] bytes = new byte[data.length][];
// Calculate the length of the content.
for(int i=0; i<data.length; i++) {
bytes[i] = data[i].getBytes();
byteLength += bytes[i].length;
byteLength += 4; // this is for an integer, which is for the length of the String
}
// Transfer the content to a ByteBuffer object
ByteBuffer buffer = ByteBuffer.allocate(byteLength);
for(int i=0; i<bytes.length; i++) {
// Put the length of the current byte array
buffer.putInt(bytes[i].length);
for(int j=0; j<bytes[i].length; j++) {
// Reverse the byte so that it can't be understood
buffer.put((byte)(~bytes[i][j]));
}
}
After writing all of your content to the ByteBuffer object, you can take the resulting byte array and write it down to a file.
FileOutputStream fos = new FileOutputStream("YourFileName.anyExtension");
fos.write(buffer.array());
fos.close();
While reading the file back, you should first read an integer, which is the length of the data you should read as byte array, then you should read this byte array.
FileInputStream fis = new FileInputStream("YourFileName.anyExtension");
DataInputStream dis = new DataInputStream(fis);
ArrayList<String> list = new ArrayList<String>();
byte[] bytes;
while(dis.available()) {
int length = dis.readInt();
bytes = new byte[length];
for(int i=0; i<length; i++) {
// Those bytes were reversed, right?
bytes[i] = (byte)(~dis.readByte());
}
// Convert byte array to String
String str = new String(bytes);
list.add(str);
}
Now you have an ArrayList of your String data.
Of course this is not the best, the safest, and the fastest algorithm. You can always find or create faster. But I think this is a good example of doing those kind of things.
If you are using Java you can just try and create a class that implements Serializable This way you can just create an object with all your meta info stored inside, serialize it, and when you wanna load it just deserialize it again.
Its not very safe though since you only need to know have the class it was made with, to deserialize it. But it is something to begin with.
Look into digital signatures, specifically HMACs. Those are pretty much exactly what you need, and the Java Crypto framework should make things fairly straightforward. Here's a potentially relevant SO entry: How to generate an HMAC in Java equivalent to a Python example?
You could pass your file writing stream thru a CipherOutputStream
Generate a random string, or number or anything. get its byte array, produce a key, and use it to encrypt your file.
byte password[] = (WHAT YOUR WANT. STRING, NUMBER, etc.).getBytes();
DESKeySpec desKeySpec;
try {
desKeySpec = new DESKeySpec(password);
SecretKeyFactory keyFactory = SecretKeyFactory.getInstance("DES");
SecretKey key = keyFactory.generateSecret(desKeySpec);
Cipher desCipher = Cipher.getInstance("DES/ECB/PKCS5Padding");
desCipher.init(Cipher.ENCRYPT_MODE, key);
// Create stream
FileOutputStream fos = new FileOutputStream("Your file here");
BufferedOutputStream bos = new BufferedOutputStream(fos);
CipherOutputStream cos = new CipherOutputStream(bos, desCipher);
}
Now you can write to the file using cos
Reading the file is done the same way using the SecretKey object
SecretKey key = loadKey(); // Deserialize your SecretKey object
try {
Cipher cipher = Cipher.getInstance("DES/ECB/PKCS5Padding");
cipher.init(Cipher.DECRYPT_MODE, key);
FileInputStream fis = new FileInputStream("Your file here");
BufferedInputStream bis = new BufferedInputStream(fis);
CipherInputStream cis = new CipherInputStream(bis, cipher);
now you can read using cis
The downside is you need to keep the SecretKey object (Serialize it or something) it wouldn't be a problem for any low level hacker to get the data (since the key is stored on the device) but it wouldn't allow just changing your data using a text editor.
Related
I'm trying to decrypt an encrypted data which is stored in a text file. I use the same key and IV to encrypt and decrypt, and transfer it via configuration file.
When I print CipherInputStream to the console I do get some content, but when I try to write it to a text file I don't get any content in it.
This is the piece of code refferes to my problem:
File encryptedData = new File("C:\\Users\\Victoria\\Desktop\\encryptedData.txt");
File decryptedData = new File("C:\\Users\\Victoria\\Desktop\\decryptedData.txt");
FileInputStream inputStream = new FileInputStream(encryptedData);
byte[] inputBytes = new byte[(int) decryptedData.length()];
inputStream.read(inputBytes);
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
cipher.init(Cipher.DECRYPT_MODE, newkey, newiv, SecureRandom.getInstance("SHA1PRNG"));
CipherInputStream cipherInputStream = new CipherInputStream(inputStream, cipher);
FileOutputStream outputStream = new FileOutputStream(decryptedData);
System.out.println("cipherInputStream: " + cipherInputStream);
// Writing the decrypted content to an output file
byte[] buff = new byte[1024 * 10];
int length;
while ((length = cipherInputStream.read(buff)) > 0) {
outputStream.write(buff, 0, length);
}
bufin.close();
outputStream.close();
cipherInputStream.close();
Any solutions?
Thank you!
Encrypted data which is stored in a text file
This is already a contradiction in terms. Encrypted data is binary, not text, and should not be stored in files with the .txt extension.
byte[] inputBytes = new byte[(int) decryptedData.length()];
This line of code is meaningless. You don't yet know how long the decrypted data will be. The decrypted file may not even exist, in which case this will produce a zero length array; or it may be different from what is about to be produced, in which case it is the wrong length.
inputStream.read(inputBytes);
Remove this line and the one before it.
It reads into an array which is at best sized to the size of the decrypted data, which is the wrong size for encrypted data, and at worst is just the wrong size, or even zero length, as shown above.
It reads the input probably until that wrongly sized buffer is full, and you then (a) completely ignore the data read and (b) attempt to read the same stream further, which will fail in the decryption loop, or at best produce incorrect output, as you may not be decrypting all the data.
When I print CipherInputStream to the console I do get some content
No you don't. You get a piece of data of the general form CipherInputStream#0011223344, which is just the result of calling CipherInputStream.toString(), which does not contain any 'content'.
I am using AES to encrypt files. The problem first came when i tried to encrypt a large file. So i did some reading online and figured that i need to use a buffer and only encrypt bytes of data at a time.
I divided my plaintext into chunks of 8192 bytes of data and then applied the encryption operation on each of these chunks but I am still getting the out of memory error.
public static File encrypt(File f, byte[] key) throws Exception
{
System.out.println("Starting Encryption");
byte[] plainText = fileToByte(f);
SecretKeySpec secretKey = new SecretKeySpec(key, ALGORITHM);
Cipher cipher = Cipher.getInstance(ALGORITHM);
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
System.out.println(plainText.length);
List<byte[]> bufferedFile = divideArray(plainText, 8192);
System.out.println(bufferedFile.size());
List<byte[]> resultByteList = new ArrayList<>();
for(int i = 0; i < bufferedFile.size(); i++)
{
resultByteList.add(cipher.doFinal(bufferedFile.get(i)));
}
ByteArrayOutputStream baos = new ByteArrayOutputStream();
for(byte[] b : resultByteList)
baos.write(b);
byte[] cipherText = baos.toByteArray();
File temp = byteToFile(cipherText, "D:\\temp");
return temp;
}
The fileToByte() takes a file as input and returns a byte array
the divideArray() takes a byte array as input and divides it into an arraylist consisting of smaller byte arrays.
public static List<byte[]> divideArray(byte[] source, int chunkSize) {
List<byte[]> result = new ArrayList<byte[]>();
int start = 0;
while (start < source.length) {
int end = Math.min(source.length, start + chunkSize);
result.add(Arrays.copyOfRange(source, start, end));
start += chunkSize;
}
return result;
}
Here is the error I get
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3236)
at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
at java.io.OutputStream.write(OutputStream.java:75)
at MajorProjectTest.encrypt(MajorProjectTest.java:61)
at MajorProjectTest.main(MajorProjectTest.java:30)
I am not getting this error if I use a file of a smaller size, but then again, the sole purpose of using buffers was to eliminate the out of memory problem.
Thanks in advance. Any help is appreciated.
One problem is holding arrays and copies of arrays in memory.
Read and write in blocks.
Then doFinal should not be repeated. Use update instead. Many examples just use a single doFinal which is misleading.
So:
public static File encrypt(File f, byte[] key) throws Exception
{
System.out.println("Starting Encryption");
SecretKeySpec secretKey = new SecretKeySpec(key, ALGORITHM);
Cipher cipher = Cipher.getInstance(ALGORITHM);
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
System.out.println(plainText.length);
Path outPath = Paths.get("D:/Temp");
byte[] plainBuf = new byte[8192];
try (InputStream in = Files.newInputStream(f.toPath());
OutputStream out = Files.newOutputStream(outPath)) {
int nread;
while ((nread = in.read(plainBuf)) > 0) {
byte[] enc = cipher.update(plainBuf, 0, nread);
out.write(enc);
}
byte[] enc = cipher.doFinal();
out.write(enc);
}
return outPath.toFile();
}
Explanation
Encryption of some byte blocks goes as:
Cipher.init
Cipher.update block[0]
Cipher.update block[1]
Cipher.update block[2]
...
Cipher.doFinal(block[n-1])
Or instead of the last doFinal:
Cipher.update(block[n-1])
Cipher.doFinal()
Every update or doFinal yielding a portion of the encrypted data.
doFinal also "flushes" final encryption data.
If one has only a single block of bytes, it suffices to call
byte[] encryptedBlock = cipher.doFinal(plainBlock);
Then no calls to cipher.update are needed.
For the rest I used the try-with-resources syntax which automatically closes input and output streams, even should a return happen, or an exception have been thrown.
Instead of File the newer Path is a bit more versatile, and in combination with Paths.get("...") and the very nice utility class Files can provide powerful code: like Files.readAllBytes(path) and much more.
Look at these four variables: plainText, bufferedFile, resultByteList, cipherText. All of them contain your entire file in a slightly different format, which means that each of them is 1.2GB big. Two of them are Lists which means they are likely to be even bigger, because you didn't set the initial size of ArrayLists and they resize automatically when needed. So we are talking about over 5GB of memory needed.
Actually, you add chunks into ByteArrayOutputStream baos, which means that it must store it internally, before you call toByteArray() on it. So it's 5 copies of your data, meaning 6GB+. The ByteArrayOutputStream is internally using an array so it grows similarly to ArrayLists so it will use more memory than needed (see the stacktrace - it tried to resize).
All these variables are in the same scope, never are assigned null which means that they cannot be garbage collected.
You can increase the maximum heap limit (see Increase heap size in Java), but this will be a serious limitation on your program.
Your program throws out of memory error when writing to ByteArrayOutputStream. This is the 4th time you copy all your data, which means that 3.6GB is already allocated. From this I deduce that your heap is set to 4GB (which is a maximum you can set on 32 bit operating system).
What you should do is have a loop, read part of the file, encrypt it and write to another file. This will avoid loading entire file into memory. Lines like List<byte[]> bufferedFile = divideArray(plainText, 8192); or resultByteList.add(...) is something that you shouldn't have in your code - you end up storing entire file in memory. The only thing that you need to keep track of is a cursor (i.e. position which says what bytes you already processed), which is O(1) memory complexity. Then you only need as much memory as the chunk your are encoding - which is far smaller than entire file.
As you're iterating over the file, keep a counter to track the number of bytes:
int encryptedBytesSize = 0;
for(int i = 0; i < bufferedFile.size(); i++) {
resultByteList.add(cipher.doFinal(bufferedFile.get(i)));
encryptedBytesSize += resultByteList.get(resultByteList.size() - 1).length;
}
Then use the constructor which takes a size parameter to create the output buffer:
ByteArrayOutputStream baos = new ByteArrayOutputStream(encryptedBytesSize);
This will avoid the internal buffer from having to grow. Growth could be non-linear so as more bytes are added each iteration even more space is allocated the next time it grows.
But this still might not work, depending on the file size. Another approach would be to:
Read a little chunk of the unencrypted file
Encrypt the chunk
Write to the encrypte file
This avoids having all of the regular and encrypted files in memory at the same time.
So I have a server side public key and private key, my aim is to send the client the public key, the client will encrypt a string with the key, then send the bytes through a stream, and the server will decrypt the byte array.
Exception:
javax.crypto.BadPaddingException: Decryption error
Code:
Sending the encoded key.
handler.getOos().writeObject(publicKey.getEncoded());
handler.getOos().flush();
Receiving the byte array (of the encoded key):
Object o = ois.readObject();
if (o instanceof byte[]) {
JChat.get().setServerPublicKey(KeyFactory.getInstance("RSA").generatePublic(new X509EncodedKeySpec((byte[]) o)));
JChat.get().go();
}
The go() method (here I use a DataOutputStream to send the byte array):
public void go() {
String text = "hello darkness my old friend";
byte[] encrypted = encrypt(text, serverPublicKey);
try {
handler.getDos().write(encrypted);
handler.getDos().flush();
} catch (IOException e) {
e.printStackTrace();
}
}
Reading the byte array, on the server side:
int count = dis.available();
byte[] in = new byte[count];
dis.readFully(in);
System.out.println(Server.decrypt(in, Server.get().getPrivateKey()));
The decryption method throws this exception:
javax.crypto.BadPaddingException: Decryption error
at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:380)
at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:291)
at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:363)
at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:389)
at javax.crypto.Cipher.doFinal(Cipher.java:2165)
at com.archiepking.Server.decrypt(Server.java:97)
at com.archiepking.net.ClientHandler$1.run(ClientHandler.java:44)
at java.lang.Thread.run(Thread.java:745)
Any suggestions as to what I am doing wrong? Please note:
Dos = DataOutputStream Dis = DataInputStream Oos = ObjectOutputStream
Ois = ObjectInputStream
I am using two different sockets, one for sending objects and one for datatypes (as my chat application will need both).
What can I do to fix this error?
FURTHER INFORMATION:
Generation of keys:
KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance("RSA");
keyPairGenerator.initialize(1024);
KeyPair keyPair = keyPairGenerator.genKeyPair();
byte[] publicKeyBytes = keyPair.getPublic().getEncoded();
FileOutputStream fosPublic = new FileOutputStream("public");
fosPublic.write(publicKeyBytes);
fosPublic.close();
byte[] privateKeyBytes = keyPair.getPrivate().getEncoded();
FileOutputStream fosPrivate = new FileOutputStream("private");
fosPrivate.write(privateKeyBytes);
fosPrivate.close();
publicKey = keyPair.getPublic();
privateKey = keyPair.getPrivate();
The problem is that you are using DataInputStream.available() to determine how many bytes to read. That method does not do what you apparently think that it does.
From the Javadoc of this method:
Returns an estimate of the number of bytes that can be read (or
skipped over) from this input stream without blocking by the next
caller of a method for this input stream. The next caller might be the
same thread or another thread. A single read or skip of this many
bytes will not block, but may read or skip fewer bytes.
It just returns the number of bytes that can be read without blocking, which can be far less than the actual number of bytes that you sent, especially if you are using network Sockets to send/receive that data.
The solution:
before writing the bytes, write an int with the writeInt method that contains the number of bytes that you're writing
before reading the bytes, call readInt to read the number of bytes that will follow, and construct a byte array of the right length from that number.
If you are using an ObjectOutputStream why bother converting the public key to a byte array using getEncoded? You can serialize the object directly. e.g.
handler.getOos().writeObject(publicKey);
Or if you have to use the encoded version, then remove the ObjectOutputStream and use ByteArrayOutputStream instead.
I am trying to figure out how to read multiple digests (md5, sha1, gpg) based on the same InputStream using DigestInputStream. From what I've checked in the documentation, it seems to be possible by cloning the digest. Could somebody please illustrate this?
I don't want to be re-reading the stream in order to calculate the checksums.
You could wrap a DigestInputStream around a DigestInputStream and so on recursively:
DigestInputStream shaStream = new DigestInputStream(
inStream, MessageDigest.getInstance("SHA-1"));
DigestInputStream md5Stream = new DigestInputStream(
shaStream, MessageDigest.getInstance("MD5"));
// VERY IMPORTANT: read from final stream since it's FilterInputStream
byte[] shaDigest = shaStream.getMessageDigest().digest();
byte[] md5Digest = md5Stream.getMessageDigest().digest();
The Javadoc is pretty clear. You can use clone only to calculate different intermediate digests using the same algorithm. You cannot use DigestInputStream to calculate different digest algorithms without reading the stream multiple times. You must use a regular InputStream and multiple MessageDigest objects; read the data once, passing each buffer to all MessageDigest objects to get multiple digests with different algorithms.
You could easily encapsulate this in your own variant of DigestInputStream, say MultipleDigestInputStream that follows the same general approach but accepts a collection of MessageDigest objects or algorithm names.
Pseudojava (error handling omitted)
MessageDigest sha = MessageDigest.getInstance("SHA-1");
MessageDigest md5 = MessageDigest.getInstance("MD5");
InputStream input = ...;
byte[] buffer = new byte[BUFFER_SIZE];
int len;
while((len = input.read(buffer)) >= 0)
{
sha.update(buffer,0,len);
md5.update(buffer,0,len);
...
}
byte[] shaDigest = sha.digest();
byte[] md5Digest = md5.digest();
I have written code in vb.net to encrypt a file from a memory stream. I also decrypt the file as well as copy the memory stream to a file to assure encryption/ decryption works. My vb solution works.
However my need is to decrypt using Java. When I decrypt my file, I always get an extra "?" character at the very beginning of the file, but other than that the resullts are perfect. Has anyone seen anything like this before? I must admit, my results are from using only one set of data, but I've encrypted it twice using new keys and vectors both times.
A few details. I'm using AES, PKCS7 padding in vb, and PKCS5 padding in Java. The file can be of arbitrary length. Any help is appreciated.
I am posting this from my phone, and don't have the code handy. I can add it tomorrow. I'm just hoping that this description rings a bell with someone.
Thanks,
SH
When I wrote to the MemoryStream in VB, I declared a StreamWriter like so:
Writer = New IO.StreamWriter(MS, System.Text.Encoding.UTF8)
Here's my VB.NET encryption function.
Public Shared Function WriteEncryptedFile(ms As MemoryStream, FileName As String) As List(Of Byte())
Try
Dim original() As Byte
Dim myAes As System.Security.Cryptography.Aes = Aes.Create()
myAes.KeySize = 128
myAes.Padding = PadMode
Dim keys As New List(Of Byte())
keys.Add(myAes.Key)
keys.Add(myAes.IV)
original = ms.ToArray
Dim encryptor As ICryptoTransform = myAes.CreateEncryptor(myAes.Key, myAes.IV)
Using FileEncrypt As New FileStream(FileName, FileMode.Create, FileAccess.Write)
Using csEncrypt As New CryptoStream(FileEncrypt, encryptor, CryptoStreamMode.Write)
csEncrypt.Write(original, 0, original.Length)
csEncrypt.FlushFinalBlock()
FileEncrypt.Flush()
FileEncrypt.Close()
csEncrypt.Close()
End Using
End Using
Return keys
Catch e As Exception
MsgBox("Error during encryption." & vbCrLf & e.Message)
End Try
Return Nothing
End Function
And here's the Java decryption:
public static void DecryptLIGGGHTSInputFile(String fileIn, String fileOut, String base64Key, String base64IV) throws Exception
{
// Get the keys from base64 text
byte[] key = Base64.decodeBase64(base64Key);
byte[] iv= Base64.decodeBase64(base64IV);
// Read fileIn into a byte[]
int len = (int)(new File(fileIn).length());
byte[] cipherText = new byte[len];
FileInputStream bs = new FileInputStream(fileIn);
bs.read(cipherText, 1, len-1);
System.out.println(cipherText.length);
System.out.println((double)cipherText.length/128);
bs.close();
// Create an Aes object
// with the specified key and IV.
Cipher cipher = null;
cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
// Encrypt the message.
SecretKey secret = new SecretKeySpec(key, "AES");
/*
cipher.init(Cipher.ENCRYPT_MODE, secret, ivspec);
cipherText = cipher.doFinal("Hello, World!".getBytes("UTF-8"));
System.out.println(cipherText);
*/
cipher.init(Cipher.DECRYPT_MODE, secret , new IvParameterSpec(iv));
String plaintext = new String(cipher.doFinal(cipherText), "UTF-8");
System.out.println(plaintext.length());
FileWriter fw = new FileWriter(fileOut);
fw.write(plaintext);
fw.close();
}
It was a BOM problem. When I created the MemoryStream with VB, I initialized it in UTF-8 encoding. The very first character in my file boosted the size and position of the stream from 0 bytes to 4 bytes, when it should have only been one. The solution was to create an encoding based on UTF-8 without Byte Order Marks, like so:
Dim UTF8EncodingWOBOM As New System.Text.UTF8Encoding(False) 'indicates to omit BOM
Writer = New IO.StreamWriter(MS, UTF8EncodingWOBOM)
I read here that there are frequently issues with encoding incompatibilities between platforms due to the presence or lack of byte order mark, as it is neither recommended or required. It's not right to use one, it's not wrong to use one. You basically have to find a way to deal with them. A plethora of other articles and postings suggested different ways to do it. The gist was, either identify them and deal with them if they exist. Since I have control of both the writing and the reading, it makes about as much sense to do away with them entirely.
SH