I'm looking for a low-level encryption to write questions/answers for a test/exam application in Java. Both the questions and exam are objects. Basically, I need a way to serialize a object, write it to a file, whilst encrypting everything so no one can read the question/answers without my program.
I've heard of Protocol Buffers (http://code.google.com/apis/protocolbuffers/docs/javatutorial.html), but not sure if there is something newer/better out there or if it is the next best thing.
Cheers
-Tanner
You need two steps - serialization/deserialization, which converts an object to a representation which can be stored on disk; and encryption/decryption, which enciphers the on-disk representation.
In Java you can use an ObjectOutputStream to perform the serialization, and a CipherOutputStream to perform the encryption. First obtain a FileOutputStream, then pass that to the constructor of a CipherOutputStream, then pass that to the constructor of an ObjectOutputStream. Then you can just hand your Serializable object(s) to the ObjectOutputStream, and they'll end up serialized, encrypted and written to a file. (You will of course need to perform additional setup on at least the CipherOutputStream object, but that's the basic idea).
However, there is a rather large caveat to all of this. The encryption you're doing is no more than obfuscation - if you give someone the encrypted data and a program that can decrypt it, that person has all the information they need to decrypt the data themselves. There's no way to get around this - if your program can decrypt it, then your program can be pulled apart and its secrets found.
Judging from your description of the application, performance is not a concern. So a solution that serializes to XML (e.g. using XStream) and then encrypts the XML would be satisfy your requirements.
However, I should warn you that there is a significant risk that student with sufficient incentive could hack your encryption. I can think of only ways you can guard against this:
don't store the encrypted data on the user's machine at all, or
use an asymmetric cipher and don't ever do any decryption on the user's machine.
If you cannot do one or the other, your application is vulnerable. Hacking it is not quite as simple as reading the files with a text editor, but it is not all that hard either ... even if you feed the application through an JAR file obfuscator.
Related
I'm working on a proprietary TCP protocol. This protocol sends and receive messages with a specific sequence of bytes.
I should be complaiant to this protocol, and i cant change it.
So my input / output results are something like that :
\x01\x08\x00\x01\x00\x00\x01\xFF
\x01 - Message type
\x01 - Message type
\x00\x01 - Length
\x00\x00\x01 - Transaction
\xFF - Body
The sequence of field is important. And i want only the values of the fields in my serialization, and nothing about the structure of the class.
I'm working on a Java controller that use this protocol and I've thought to define the message structures in specific classes and serialize/deserialize them, but I was naive.
First of all I tried ObjectOutputStream, but it output the entire structure of the object, when I need only the values in a specific order.
Someone already faced this problem:
Java - Object to Fixed Byte Array
and solved it with a dedicated Marshaller.
But I was searching for a more flexible solution.
For text serialization and deserialization I've found:
http://jeyben.github.io/fixedformat4j/
that with annotation defines the schema of the line. But it outputs a String, not a byte[]. So 1 is output like "1" that is represented differently based on encoding, and often with more bytes.
What I was searching for is something that given the order of my class properties will convert each property in a bunch of bytes (based on the internal representation) and append them to a byte[].
Do you know some library used for that purpose?
Or a simple way to do that, without coding a serialization algorithm for each of my entities?
Serialization just isn't easy; it sounds from your question like you feel you can just invoke something and out rolls compact, simple, versionable, universal data you can then put on the wire. What you need to fix is to scratch the word 'just' from that sentence. You're going to have to invest some time and care.
As you figured out already, java's baked in serialization has a ton of downsides. Don't use that.
There are various serializers. The popular ones are things like GSON or Jackson, which lets you serialize java objects into JSON. This isn't particularly efficient, and is string based. This sounds like crucial downsides but they really aren't, see below.
You can also spend a little more time specifying the exact format and use protobuf which lets you write a quite lean and simple data protocol (and protobuf is available for many languages, if eventually you want to write an participant in this protocol in non-java later).
So, those are the good options: Go to JSON via Jackson or GSON, or, use protobuf.
But JSON is a string.
You can turn a string to bytes trivially using str.getBytes(StandardCharsets.UTF_8). This cannot fail due to charset encoding differences (as long as you also 'decode' in the same fashion: Turn the bytes into a string with new String(theBytes, StandardCharsets.UTF_8). UTF-8 is guaranteed to be available on all JVMs; if it is not there, your JVM is as broken as a JVM that is missing the String class - not something to worry about.
But JSON is inefficient.
Zip it up, of course. You can trivially wrap an InputStream and an OutputStream so that gzip compression is applied which is simple, available on just about every platform, and fast (it's not the most efficient cutting edge compression algorithm, but usually squeezing the last few bytes out is not worth it) - and zipped-up JSON can often be more efficient that carefully handrolled protobuf, even.
The one downside is that it's 'slow', but on modern hardware, note that the overhead of encrypting and decrypting this data (which you should obviously be doing!!) is usually multiple orders of magnitude more involved. A modern CPU is simply very, very fast - creating JSON and zipping it up is going to take 1% of CPU or less even if you are shipping the collected works of shakespeare every second.
If an arduino running on batteries needs to process this data, go with uncompressed, unencrypted protobuf-based data. If you are facebook and writing the whatsapp protocol, the IAAS creds saved by not having to unzip and decode JSON is tiny and pales in comparison to the creds you spend just running the servers, but at that scale its worth the development effort.
In just about every other case, just toss gzipped JSON on the line.
I have searched around for the best methods for encryption in terms of what ciphers to use, methods/etc. I'v decided on using 128Bit AES for the time being. My question lies more in what method is best for encrypting various types of data. In my example, this is for a small game I am making that has map data and associated image data.
I can save the data in any format, but would prefer something simple to read (when un-encrypted). Should I actually save this to the file itself, or should I change the file itself in some way?
Likewise in terms of the image files associated to the game that are saved as PNG or BMP files, any recommendations on how to encrypt those on top of the rest of the files?
My current method for the game data is to just encrypt it in singular lines - and have it load from a file line-by-line. Each line formatting to a different data value to be put into the system (e.g. load part of a map, maybe some item data, etc). Is there a better/faster method of accomplishing this that I may not have found yet?
I was also wondering about actually taking the entire class with the data saved and serializing it - then encrypting it. That way I could load the entire thing in one go straight into a class. Would this be a reasonable idea?
For the images: you can read them using a CipherInputStream / CipherOuputStream. Note that at least older Java versions ignored padding errors for CipherInputStream.
You cannot directly encrypt to lines as the output of a cipher is binary. So you need to encode (using e.g. Base64) if you want to store the result as lines. You may ask yourself if you want things like game data to be in text. I would recommend CBC mode with random IV prefixed to the ciphertext.
If the data is serializable then serializing data could be an option. Personally I'm not a huge fan of serialization in Java due to the many pitfalls. But yes, you can certainly encrypt serialized data easily, it's binary after all.
I am just getting in to writing networked code using Sockets in Java. I'm just making some test programs. Originally I was going to send data as comma separated values, but I recently discovered ObjectOutputStream. Which method would be faster or more bandwidth efficient? For example, if I'm making a game where I have to send x and y coordinates very often, should I send it through PrintWriter separated by a comma, or make a Position class and send an instance over ObjectOutputStream. What if I change my code and need to send a lot more data?
What are the pros and cons of sending data as CSV over PrintWriter vs as fields in an object over ObjectOutputStream?
An ad-hoc binary format has a good chance of being more bandwidth-efficient than the default serialization format, which should be (but it's a wild guess, and it depends on the nature and amount of data: you should measure it if it matters) more or less as bandwidth efficient than a text-based format.
But bandwidth efficiency is not the only thing that matters.
Using serialization, the client and the server must be written in Java, and have the classes of the serialized objects in their classpath. If you intend to have clients written in any language, you shouldn't consider it.
If serialization is OK, it's of course a really easy way to transform almost any Java object into bytes, which allows you to avoid defining a format.
Note that there are alternatives that provide almost the same flexibility, but don't have the Java-only disadvantage of serialization. For example, JSON, XML, or protobuf.
I think CSV is smaller.
If you want to check data size,please try to output to a File.
and I don't recommend ObjectOutputStream to you by other reason.
Because you have to keep Objects compatibility.
Did you research about serialize and serialVersionUID?
Please check java.io.Serializable
I am encrypting some data from codeigniter using Encrypt class and this will be send to a java program and need to be decoded there.
php code:
$this->load->library('encrypt');
$this->encrypt->set_key(SERVER_ENCRYPT_KEY);
$this->encrypt->set_mode(MCRYPT_MODE_CFB);
$this->data = $this->encrypt->encode($this->input->post('data'));
where SERVER_ENCRYPT_KEY is the key.
I found this: MCrypt for Java but I could not make it work.
There are other libs that can do this? or maybe an example how to do this using the java version of MCrypt.
Take a look in the CI_Encrypt class. In addition to calling the mcrypt library, it does various non standard things. As an example it runs encrypted data through an own invented _add_cipher_noise method. Also, it seems that the encrypted format have changed with different versions - indicating that this could happen again.
If you are going to use the encrypted data outside codeignite you should not use what looks like codeignites own packaging of mcrypt encrypted data.
I decided to avoid CI_Encrypt and use this PHP-Java-AES-Encrypt with small changes: add noise and use 2 types of keys. Also I build a tool to convert the old encrypted data to the new format.
How to exchange the public key to the client place. i have encrypted a document(text file) using RSA algorithm by using private key and then stored the public key as an java.security.Key object in a file using serialization.I want to know about the integrity of the serialized public key object whether it is safe option to do or any other option available.
A public key is usually just exchanged as a piece of text. It is then imported into a keystore. The exact method of doing is depends on the implementation (I've always used PGP).
I wouldn't expose the key as a serialized form of java.security.Key because it's not really standard. The key in it's simple form is the standard form of interchange.
On exposing the key as a download: it's public, so there's nothing an intruder could do by downloading your key. The only thing that could go wrong is that someone could fake your server and host a different key. Then sign with that key's private key and claim to be you. Of course you could have the same issue if you mailed it to somebody. But then at least you would know who you mailed it to.
The safest approach is to spread the key out-of-bound. Like on a usb stick.
Depending on your cause, I think you can live with the risk.
Joeri has answered on the security aspects - I have nothing to add about that.
The main problem you might have is the serialized representation changing between library versions, which is a potential issue with any use of Java serialization.
Of course there are other ways to do it. The RSA public key should have an industry-standard encoding which you can access using the getEncoded() method. This gives you an array of bytes which you can write to a FileOutputStream. So it's really easy.