We Keep Coding

Signing JSON objects

I have to exchange JSON objects between different platforms and implementations of a service and make its integrity verifiable via digital signatures. So a platform A would create such an object and create a digital signature. Said signature is then included into the object and sent to platform B. The JSON objects can contain arbitrary attributes and data.
E.g. in PHP:
function signObject($jsonObjectToSign, $privateKey) {
$jsonObjectToSign->signature = "";
$msgToSign = json_encode($jsonObjectToSign);
openssl_sign($msgToSign, $jsonObjectToSign->signature, $privateKey, OPENSSL_SLGO_SHA1);
return $jsonObjectToSign;
}
Problem is, that e.g. in Java, there is no way to tell whether the attributes of a JSON object will be in the same order you added them (via JSONObject.put()). So, if I do a
$json = json_encode('{"a":1, "b":2}');
in PHP, sign this object as stated above, transfer it to a java based server, decode the json object and then try to verify the signature, I'd probably get a different order of the object's attributes.
So what I need, is a reliable way to create a String from a JSONObject, independent of the language or platform used.
The example object above needs always to output {"a":1, "b":2} and NEVER {"b":2, "a":1}. Unfortunately, this is the usual case e.g. in Java.
Is there any "best practice" to sign JSON Objects in a secure way?
But let me describe the problem in another way:
Let's say I want to do this in Java (or any other language):
JSONObject j = new JSONObject();
j.put("a", 1);
j.put("b", 2);
Now, I need a serialization function, that outputs always the same string representation for this object, no matter how and with what language this object is created.

Signing and encryption of JSON objects is specified in the JOSE suite of specifications where JOSE stands for Javascript Object Signing and Encryption, see http://jose.readthedocs.org/en/latest/ JOSE uses a detached signature calculated over a base64url encode representation of the JSON object. The signature is not part of the JSON object itself so no re-ordering is required to validate it.

As there is AFAIK no official (neither unofficial) standard on JSON Signing yet, I'd probably do a custom implementation. I'd define a new JSON object, e.g.
{
"original": "..." // original JSON as a Base64 encoded string
"signature": "..." // the signature
}
and implement a signing / signature verification layer on both/all sides of my system.

This is how I solved it now. It is somehow similar to what JOSE does, except for the header. But JOSE seems to bring a lot of overhead (and features) I dont need. So I decided to go with the following:
class Signature
{
private static $algorithm = OPENSSL_ALGO_SHA512;
private static $signaturePrefix = '-----BEGIN SIGNATURE-----';
private static $signaturePostfix = '-----END SIGNATURE-----';
public static function createSignature($message, $privateKey)
{
$signature = null;
openssl_sign($message, $signature, $privateKey, self::$algorithm);
return self::$signaturePrefix . base64_encode($signature) . self::$signaturePostfix;
}
public static function verifySignature($message, $publicKey, $signature)
{
$signature = str_replace(self::$signaturePrefix, '', $signature);
$signature = str_replace(self::$signaturePostfix, '', $signature);
return openssl_verify($message, base64_decode($signature), $publicKey, self::$algorithm);
}
public static function signJSON($jsonToSign, $privateKey)
{
if(gettype($jsonToSign) != 'string')
$jsonToSign = json_encode($jsonToSign);
$signedJSON = json_decode('{}');
$sigedJSON->signature = self::createSignature($message, $privateKey);
$signedJSON->object = $jsonToSign;
return $signedJSON;
}
public static function verifyJSONSignature($jsonObject, $publicKey)
{
if(gettype($jsonObject->object) == 'string')
throw new Exception('Value $jsonObject->object must be a String, is a ' . gettype($jsonObject->object));
return self::verifySignature($jsonObject->object, $publicKey, $jsonObject->signature);
}
}

We encountered a similar issue with hashing JSON-encoded payloads. In our case we use the following methodology:
Convert data into JSON object
Encode JSON payload in base64
Message digest (HMAC) the generated base64 payload
transmit base64 payload (with the generated signature) .
See the details in the link below
linked answer here :
How to cryptographically hash a JSON object?

JsonObject js = new JsonObject();
js.addProperty("c", "123");
js.addProperty("t", "yyyy-MM-dd'T'HH:mm:ss.SSSZ");
System.out.println("Json object == " + js);
GenerateSignature util = new GenerateSignature();
String ecodedToken = util.signJSONObject(js);

Passing java object to python

I am prototyping an interface to our application to allow other people to use python, our application is written in java. I would like to pass some of our data from the java app to the python code but I am unsure how to pass an object to python. I have done a simple java->python function call using simple parameters using Jython and found it very useful for what I am trying to do. Given the class below, how can I then use it in Python/Jython as an input to a function/class:
public class TestObject
{
private double[] values;
private int length;
private int anotherVariable;
//getters, setters
}

One solution. You could use some sort of message system, queue, or broker of some sort to serialize/deserialize, or pass messages between python and java. Then create some sort workers/producer/consumers to put work on the queues to be processed in python, or java.
Also consider checking out for inspiration: https://www.py4j.org/
py4j is used heavily by/for pyspark and hadoop type stuff.
To answer your question more immediately.
Example using json-simple.:
import org.apache.commons.io.FileUtils;
import org.json.simple.JSONObject;
//import org.json.simple.JSONObject;
public class TestObject
{
private double[] values;
private int length;
private int anotherVariable;
private boolean someBool;
private String someString;
//getters, setters
public String toJSON() {
JSONObject obj=new JSONObject();
obj.put("values",new Double(this.values));
obj.put("length",new Integer(this.length));
obj.put("bool_val",new Boolean(this.SomeBool));
obj.put("string_key",this.someString);
StringWriter out = new StringWriter();
obj.writeJSONString(out);
return out.toString();
}
public void writeObject(){
Writer writer = new BufferedWriter(
new OutputStreamWriter(
new FileOutputStream("anObject.json"), "utf-8")
)
)
writer.write(this.toJSON());
}
public static void setObject(){
values = 100.134;
length = 12;
anotherVariable = 15;
someString = "spam";
}
}
And in python:
class DoStuffWithObject(object):
def __init__(self,obj):
self.obj = obj
self.changeObj()
self.writeObj()
def changeObj(self):
self.obj['values'] = 100.134;
self.obj['length'] = 12;
self.obj['anotherVariable'] = 15;
self.obj['someString'] = "spam";
def writeObj(self):
''' write back to file '''
with open('anObject.json', 'w') as f:
json.dump(self.obj, f)
def someOtherMethod(self, s):
''' do something else '''
print('hello {}'.format(s))
import json
with open('anObject.json','r') as f:
obj = json.loads(f.read())
# print out obj['values'] obj['someBool'] ...
for key in obj:
print(key, obj[key])
aThing = DoStuffWithObject(obj)
aThing.someOtherMethod('there')
And then in java read back the object. There are solutions that exist implementing this idea (JSON-RPC, XML-RPC, and variants). Depending, you may may also want to consider using something like ( http://docs.mongodb.org/ecosystem/drivers/java/ ) the benefit being that mongo does json.
See:
https://spring.io/guides/gs/messaging-reactor/
http://spring.io/guides/gs/messaging-rabbitmq/
http://spring.io/guides/gs/scheduling-tasks/
Celery like Java projects
Jedis
RabbitMQ
ZeroMQ
A more comprehensive list of queues:
http://queues.io/
Resources referenced:
http://www.oracle.com/technetwork/articles/java/json-1973242.html
How do I create a file and write to it in Java?
https://code.google.com/p/json-simple/wiki/EncodingExamples

Agree with the answer below. I think that the bottom line is that "Python and Java are separate interpreter-environments." You therefore shouldn't expect to transfer "an object" from one to the other. You shouldn't expect to "call methods." But it is reasonable to pass data from one to another, by serializing and de-serializing it through some intermediate data format (e.g. JSON) as you would do with any other program.
In some environments, such as Microsoft Windows, it's possible that a technology like OLE (dot-Net) might be usable to allow environments to be linked-together "actively," where the various systems implement and provide OLE-objects. But I don't have any personal experience with whether, nor how, this might be done.
Therefore, the safest thing to do is to treat them as "records," and to use serialization techniques on both sides. (Or, if you got very adventurous, run (say) Java in a child-thread.) An "adventurous" design could get out-of-hand very quickly, with little return on investment.

You need to make the python file to exe using py2exe , Refer the link : https://www.youtube.com/watch?v=kyoGfnLm4LA. Then use the program in java and pass arguements:
Please refer this link it will be having the details:
Calling fortran90 exe program from java is not executing

Java Jersey REST Request Parameter Sanitation

I'm trying to make sure my Jersey request parameters are sanitized.
When processing a Jersey GET request, do I need to filter non String types?
For example, if the parameter submitted is an integer are both option 1 (getIntData) and option 2 (getStringData) hacker safe? What about a JSON PUT request, is my ESAPI implementation enough, or do I need to validate each data parameter after it is mapped? Could it be validated before it is mapped?
Jersey Rest Example Class:
public class RestExample {
//Option 1 Submit data as an Integer
//Jersey throws an internal server error if the type is not Integer
//Is that a valid way to validate the data?
//Integer Data, not filtered
#Path("/data/int/{data}/")
#GET
#Produces(MediaType.TEXT_HTML)
public Response getIntData(#PathParam("data") Integer data){
return Response.ok("You entered:" + data).build();
}
//Option 2 Submit data as a String, then validate it and cast it to an Integer
//String Data, filtered
#Path("/data/string/{data}/")
#GET
#Produces(MediaType.TEXT_HTML)
public Response getStringData(#PathParam("data") String data) {
data = ESAPI.encoder().canonicalize(data);
if (ESAPI.validator().isValidInteger("data", data, 0, 999999, false))
{
int intData = Integer.parseInt(data);
return Response.ok("You entered:" + intData).build();
}
return Response.status(404).entity("404 Not Found").build();
}
//JSON data, HTML encoded
#Path("/post/{requestid}")
#POST
#Consumes({MediaType.APPLICATION_FORM_URLENCODED, MediaType.APPLICATION_JSON})
#Produces(MediaType.TEXT_HTML)
public Response postData(String json) {
json = ESAPI.encoder().canonicalize(json);
json = ESAPI.encoder().encodeForHTML(json);
//Is there a way to iterate through each JSON KeyValue and filter here?
ObjectMapper mapper = new ObjectMapper();
DataMap dm = new DataMap();
try {
dm = mapper.readValue(json, DataMap.class);
} catch (Exception e) {
e.printStackTrace();
}
//Do we need to validate each DataMap object value and is there a dynamic way to do it?
if (ESAPI.validator().isValidInput("strData", dm.strData, "HTTPParameterValue", 25, false, true))
{
//Is Integer validation needed or will the thrown exception be good enough?
return Response.ok("You entered:" + dm.strData + " and " + dm.intData).build();
}
return Response.status(404).entity("404 Not Found").build();
}
}
Data Map Class:
public class DataMap {
public DataMap(){}
String strData;
Integer intData;
}

The short answer is yes, though by "filter" I interpret it as "validate," because no amount of "filtering" will EVER provide you with SAFE data. You can still run into integer overflows in Java, and while those may not have immediate security concerns, they could still put parts of your application in an unplanned for state, and hacking is all about perturbing the system in ways you can control.
You packed waaaaay too many questions into one "question," but here we go:
First off, the lines
json = ESAPI.encoder().canonicalize(json);
json = ESAPI.encoder().encodeForHTML(json);
Aren't doing what you think they're doing. If your JSON is coming in as a raw String right here, these two calls are going to be applying mass rules across the entire string, when you really need to handle these with more surgical precision, which you seem to at least be subconsciously aware of in the next question.
//Is there a way to iterate through each JSON KeyValue and filter
here?
Partial duplicate of this question.
While you're in the loop discussed here, you can perform any data transformations you want, but what you should really be considering is using the JSONObject class referenced in that first link. Then you'll have JSON parsed into an object where you'll have better access to JSON key/value pairs.
//Do we need to validate each DataMap object value and is there a
dynamic way to do it?
Yes, we validate everything that comes from a user. All users are assumed to be trained hackers, and smarter than you. However if you handled filtering before you do your data mapping transformation, you don't need to do it a second time. Doing it dynamically?
Something like:
JSONObject json = new JSONObject(s);
Iterator iterator = json.keys();
while( iterator.hasNext() ){
String data = iterator.next();
//filter and or business logic
}
^^That syntax is skipping typechecks but it should get you where you need to go.
/Is Integer validation needed or will the thrown exception be good
enough?
I don't see where you're throwing an exception with these lines of code:
if (ESAPI.validator().isValidInput("strData", dm.strData, "HTTPParameterValue", 25, false, true))
{
//Is Integer validation needed or will the thrown exception be good enough?
return Response.ok("You entered:" + dm.strData + " and " + dm.intData).build();
}
Firstly, in java we have autoboxing which means this:
int foo = 555555;
String bar = "";
//the code
foo + bar;
Will be cast to a string in any instance. The compiler will promote the int to an Integer and then silently call the Integer.toString() method. Also, in your Response.ok( String ); call, THIS is where you're going to want to encodeForHTML or whatever the output context may be. Encoding methods are ALWAYS For outputting data to user, whereas canonicalize you want to call when receiving data. Finally, in this segment of code we also have an error where you're assuming that you're dealing with an HTTPParameter. NOT at this point in the code. You'll validate http Parameters in instances where you're calling request.getParameter("id"): where id isn't a large blob of data like an entire JSON response or an entire XML response. At this point you should be validating for things like "SafeString"
Usually there are parsing libraries in Java that can at least get you to the level of Java objects, but on the validation side you're always going to be running through every item and punting whatever might be malicious.
As a final note, while coding, keep these principles in mind your code will be cleaner and your thought process much more focused:
user input is NEVER safe. (Yes, even if you've run it through an XSS filter.)
Use validate and canonicalize methods whenever RECEIVING data, and encode methods whenever transferring data to a different context, where context is defined as "Html field. Http attribute. Javascript input, etc...)
Instead of using the method isValidInput() I'd suggest using getValidInput() because it will call canonicalize for you, making you have to provide one less call.
Encode ANY time your data is going to be passed to another dynamic language, like SQL, groovy, Perl, or javascript.

Java Object Serialization issue

I have some input that I add to a serialized object.
Now when I read the serialized object, I want to check if it exists... If not loop till it has a value in it.
How do i modify the deserialization function to handle that.
There is basically a delay in populating my serializable object. So in the meantime if i were to read that object, it is going to be empty. I want to put a check to read only when it has data in it. if not it should wait till it has some data
public String _displayResults(){
String SomeData = "";
try {
FileInputStream fis = new FileInputStream("SomeDataobj");
ObjectInputStream ois = new ObjectInputStream(fis);
SomeData = (String)ois.readObject();
ois.close();
}
catch(Exception e) {
System.out.println("Exception during deserialization: ");
}
return SomeData;
}
What I tried:
added a wait condition for 2 secs for 10 times... Is there a cleaner way.
while ( ois.readObject().toString().equalsIgnoreCase("") && i <10){
Thread.sleep(2000);
i++;
}

Java provides an API called Externalizable, which allows you to customize the (de) serialization. Serialiazable is marker interface and that indicates the object can be wrote to output stream. Externalizable provides two methods readExternal() and writeExternal() where you can override the behavior.
Your question is not so clear about what you want to achieve, so I am not sure if the above information is helpful for you

Select object dynamically

Here's the situation :
I have 3 objects all named **List and I have a method with a String parameter;
gameList = new StringBuffer();
appsList = new StringBuffer();
movieList = new StringBuffer();
public void fetchData(String category) {
URL url = null;
BufferedReader input;
gameList.delete(0, gameList.length());
Is there a way to do something like the following :
public void fetchData(String category) {
URL url = null;
BufferedReader input;
"category"List.delete(0, gameList.length());
, so I can choose which of the lists to be used based on the String parameter?

I suggest you create a HashMap<String, StringBuffer> and use that:
map = new HashMap<String, StringBuffer>();
map.put("game", new StringBuffer());
map.put("apps", new StringBuffer());
map.put("movie", new StringBuffer());
...
public void fetchData(String category) {
StringBuffer buffer = map.get(category);
if (buffer == null) {
// No such category. Throw an exception?
} else {
// Do whatever you need to
}
}

If the lists are fields of your object - yes, using reflection:
Field field = getClass().getDeclaredField(category + "List");
List result = field.get();
But generally you should avoid reflection. And if your objects are fixed - i.e. they don't change, simply use an if-clause.

The logically simplest way taking your question as given would just be:
StringBuffer which;
if (category.equals("game"))
which=gameList;
else if (category.equals("apps"))
which=appList;
else if (category.equals("movie"))
which=movieList;
else
... some kind of error handling ...
which.delete();
As Jon Skeet noted, if the list is big or dynamic you probably want to use a map rather than an if/else/if.
That said, I'd encourage you to use integer constant or an enum rather than a String. Like:
enum ListType {GAME, APP, MOVIE};
void deleteList(ListType category)
{
if (category==GAME)
... etc ...
In this simple example, if this is all you'd ever do with it, it wouldn't matter much. But I'm working on a system now that uses String tokens for this sort of thing all over the place, and it creates a lot of problems.
Suppose you call the function and by mistake you pass in "app" instead of "apps", or "Game" instead of "game". Or maybe you're thinking you added handling for "song" yesterday but in fact you went to lunch instead. This will successfully compile, and you won't have any clue that there's a problem until run-time. If the program does not throw an error on an invalid value but instead takes some default action, you could have a bug that's difficult to track down. But with an enum, if you mis-spell the name or try to use one that isn't defined, the compiler will immediately alert you to the error.
Suppose that some functions take special action for some of these options but not others. Like you find yourself writing
if (category.equals("app"))
getSpaceRequirements();
and that sort of thing. Then someone reading the program sees a reference to "app" here, a reference to "game" 20 lines later, etc. It could be difficult to determine what all the possible values are. Any given function might not explicitly reference them all. But with an enum, they're all neatly in one place.

You could use a switch statement
StringBuffer buffer = null;
switch (category) {
case "game": buffer = gameList;
case "apps": buffer = appsList;
case "movie": buffer = movieList;
default: return;
}