We Keep Coding

How to mask a specific value without knowing exact key, within a JSON string, in Java

I am receiving a JSON in the form of a string, need to mask a piece of information, however the JSON strucutre and key-names are always different but value's pattern is recognizable. Question being, what is an efficient way to traverse through String/JSONObject to mask that piece of data.
I've tried turning the String into a JSONObject and traverse through every embedded JSONObject/Array, detect the pattern, and replace that original value with its masked version. But this seems very time consuming when Logging this information out to console.
Value's pattern for reference is a 9-digit (Long) number.
Structure always varies from "{"key1":[{"innerKey1":123456789}]}" to "{"key1":"value1", "key2":{"innerKey1":123456789}"
Sample result : "{"key1":[{"innerKey1":"XXXXXX789"}]}"

If the JSON structure is always provided as optimized single line string you could just find the value in the string and replace it or get even more elaborate and use a regular expression to find the innerKey1:12345 match and replace that.
If this is just for logging purposes you can even implement this as a filter, depending on your logging framework it might be even configurable instead of having to code it.

Does bson require more space then json when sending decimal data? [duplicate]

I am comparing JSON and BSON for serializing objects. These objects contain several arrays of a large number of integers. In my test the object I am serializing contains a total number of about 12,000 integers. I am only interested in how the sizes compare of the serialized results. I am using JSON.NET as the library which does the serialization. I am using JSON because I also want to be able to work with it in Javascript.
The size of the JSON string is about 43kb and the size of the BSON result is 161kb. So a difference factor of about 4. This is not what I expected because I looked at BSON because I thought BSON is more efficient in storing data.
So my question is why is BSON not efficient, can it be made more efficient? Or is there another way of serializing data with arrays containing large number of integers, which can be easily handled in Javascript?
Below you find the code to test the JSON/BSON serialization.
// Read file which contain json string
string _jsonString = ReadFile();
object _object = Newtonsoft.Json.JsonConvert.DeserializeObject(_jsonString);
FileStream _fs = File.OpenWrite("BsonFileName");
using (Newtonsoft.Json.Bson.BsonWriter _bsonWriter = new BsonWriter(_fs)
{ CloseOutput = false })
{
Newtonsoft.Json.JsonSerializer _jsonSerializer = new JsonSerializer();
_jsonSerializer.Serialize(_bsonWriter, _object);
_bsonWriter.Flush();
}
Edit:
Here are the resulting files
https://skydrive.live.com/redir?resid=9A6F31F60861DD2C!362&authkey=!AKU-ZZp8C_0gcR0

The efficiency of JSON vs BSON depends on the size of the integers you're storing. There's an interesting point where ASCII takes fewer bytes than actually storing integer types. 64-bit integers, which is how it appears your BSON document, take up 8 bytes. Your numbers are all less than 10,000, which means you could store each one in ASCII in 4 bytes (one byte for each character up through 9999). In fact, most of your data look like it's less than 1000, meaning it can be stored in 3 or fewer bytes. Of course, that deserialization takes time and isn't cheap, but it saves space. Furthermore, Javascript uses 64-bit values to represent all numbers, so if you wrote it to BSON after converting each integer to a more appropriate dataformat, your BSON file could be much larger.
According to the spec, BSON contains a lot of metadata that JSON doesn't. This metadata is mostly length prefixes so that you can skip through data you aren't interested in. For example, take the following data:
["hello there, this is an necessarily long string. It's especially long, but you don't care about it. You're just trying to get to the next element. But I keep going on and on.",
"oh man. here's another string you still don't care about. You really just want the third element in the array. How long are the first two elements? JSON won't tell you",
"data_you_care_about"]
Now, if you're using JSON, you have to parse the entirety of the first two strings to find out where the third one is. If you use BSON, you'll get markup more like (but not actually, because I'm making this markup up for the sake of example):
[175 "hello there, this is an necessarily long string. It's especially long, but you don't care about it. You're just trying to get to the next element. But I keep going on and on.",
169 "oh man. here's another string you still don't care about. You really just want the third element in the array. How long are the first two elements? JSON won't tell you",
19 "data_you_care_about"]
So now, you can read '175', know to skip forward 175 bytes, then read '169', skip forward 169 bytes, and then read '19' and copy the next 19 bytes to your string. That way you don't even have to parse the strings for delimiters.
Using one versus the other is very dependent on what your needs are. If you're going to be storing enormous documents that you've got all the time in the world to parse, but your disk space is limited, use JSON because it's more compact and space efficient.
If you're going to be storing documents, but reducing wait time (perhaps in a server context) is more important to you than saving some disk space, use BSON.
Another thing to consider in your choice is human readability. If you need to debug a crash report that contains BSON, you'll probably need a utility to decipher it. You probably don't just know BSON, but you can just read JSON.
FAQ

Java - Mapping a description to a word with Map/HashMap

I'm working on a small project in which I want the following to happen: I want to be able to give descriptions to words in such a way that I am able to get both pieces of information later on. Using a String array seems like it would be time consuming, and, by the way I'm interpreting what I'm reading, using the following piece of code only gives me the second written information, which is "Example Description.", because it maps the second value to the first "value" (which is a "key").
Map<String, String> exampleDictionaryThing = new HashMap<String, String>() {{
put("Example Word", "Example Description.");
}};
Just to make sure I'm understood, as English isn't really my best language, I'm trying to make a bunch of dictionary-like things where I'm able to grab both pieces of information, the two pieces of info. being a word and a description for it, but doing a bunch of String array's would A) take a lot of time and B) would be, what seems to me, an incredibly inefficient way of going about this issue.
Anybody have any ideas?
Update #1
I knew my wording would cause confusion; sorry xD. From what I've read on how I'm going about this, the line of code I've given above only allows me access to the "Example Description." line if I call the get() function. What I need is something to store two Strings and give me access to both (hopefully) without using a String array. My question is: How would I go about storing two Strings in that fashion?

If you have
HashMap<String, String> mp = new HashMap<String, String>();
which is properly populated (basically in the way you described), you can do the following in order to iterate through it (this way you have both the key and the value in your loop):
for (String word : mp.keySet()){
System.out.println("Word: [" + word + "]; Word Description: [" + mp.get(word) + "].");
}

If you want to "get both pieces of information later on" as you say, maybe you should take a look at Guava's BiMap. This data structure would allow you to recover the description given the word, or the opposite.
http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/BiMap.html

Optimized way of doing string.endsWith() work.

I need to look for all web requests received by Application Server to check if the URL has extensions like .css, .gif, etc
Referred how tomcat is listening for every request and they pick the right configured Servlet to serve.
CharChunk , MessageBytes , Mapper
Here is my idea to implement:
Load all the extensions we like to compare and get the byte
representation of them.
get a unique value for this xtension by summing up the bytes in the byte Array // eg: "css".getBytes()
Add the result value to Sorted List
Whenever we receive the request, get the byte representation of the URL // eg: "flipkart.com/eshopping/images/theme.css".getBytes()
Start summing the bytes from the byte array's last index and break when we encounter "." dot byte value
Search for existence of the value thus summed with the Sorted List // Use binary Search here
Kindly give your feed backs about the implementation and issues if any.
-With thanks, Krishna

This sounds way more complicated than it needs to be.
Use String.lastIndeXOf to find the last dot in the URL
Use String.substring to get the extension based on that
Have a HashSet<String> for a set of supported extensions, or a HashMap<String, Whatever> if you want to map the extension to something else
I would be absolutely shocked to discover that this simple approach turned out to be a performance bottleneck - and indeed I suspect it would be more efficient than the approach you suggested, given that it doesn't require the entire URL to be converted into a byte array... (It's not clear why your approach uses byte arrays anyway instead of forming the hash from char values.)
Fundamentally, my preferred approach to performance is:
Do up-front design and testing around things which are hard to change later, architecturally
For everything else:
Determine the performance criteria first so you know when you can stop
Write the simplest code that works
Test it with realistic data
If it doesn't perform well enough, use profilers (etc) to work out where the bottleneck is, and optimize that making sure that you can prove the benefits using your existing tests

Reflection, hashtable or methods for performance?

I'm trying to write a Java program to decode and encode Ogg streams. I've got a decoder working but I didn't like the fact that I had duplicate code so I started writing something like that:
Decoder oggDecoder = new Decoder(
new StringDecoder( "Ogg" ),
new IntDecoder( "something" )//, ...
);
I wrote encoders and decoders for some "types" and then use them to build the whole thing.
But then I don't know how to store the result. I have 3 options I know:
- keep the data in an array of bytes and provide a get( String name ) and set( String name, Object value ) methods that will work directly on the bytes.
- use a dictionary.
- use a class and use reflection to set the properties.
I'm not that much into performance and if it's slow I don't really care as long as it's fast enough to read music. Meaning that I know writing the functions myself would make it faster but I want to write just one function working for all the properties.
So what do you think would be the fastest way of doing this?
Another way to ask this question would be:
Given a set of field names as an array of String, what is the most appropriate data structure to store the corresponding values that got decoded from a byte stream:
- keep them as byte
- store them in a dictionary
- store them in a class using reflexion
Thanks in advance for your answer.

KISS - just use a HashMap<String, byte[]>. No reflection needed.
Update
I don't think I understood at first what you want, but now I think what you are looking for is a hetergeneous map structure.
Here's a question that might be of more use to you.