We Keep Coding

Understanding rainbow table Java

I'm taking a cryptography course but some of the things my teacher describes are really not clear and badly explained.
He asked me to create an algorithm in Java to generate a RT table (hash/text) and to have a file (test.txt) that has 100 hashes to 'crack'. So I am at the stage where I have to compare the two files. But it seems to me too 'simple' I could look in my course and we talk about function reduction but I don't see how (when) to implement it.
I could already do the file reading and I could read line by line my big file and compare each hash with my small one. I don't know where and especially how to implement the function reduction in my algorithm and what it consists in.
Thank you very much for your help, if needed I put my code.
private static void bufferedReaderFilePasswordFirst() throws IOException {
Path path = Paths.get("C:\\Users\\basil\\OneDrive - Haute Ecole Bruxelles Brabant (HE2B)\\Documents\\NetBeansProjects\\sha256\\passwords.txt");
int nbOfLine = 0;
StringBuffer oui = new StringBuffer();
List<String> test = hashMap();
final DecimalFormat df = new DecimalFormat();
final DecimalFormatSymbols ds = df.getDecimalFormatSymbols();
ds.setGroupingSeparator('_');
df.setDecimalFormatSymbols(ds);
try (BufferedReader readerPasswordGenerate = Files.newBufferedReader(path, Charset.forName("UTF-8"));) {
String currentLinePassword = null;
long start = System.nanoTime();
while(((currentLinePassword = readerPasswordGenerate.readLine()) != null)){
String firstWord = currentLinePassword.substring(0, currentLinePassword.indexOf(":"));
int indexList = test.indexOf(firstWord);
if(indexList!=-1){
System.out.println(indexList);
String secondWord = currentLinePassword.substring(currentLinePassword.lastIndexOf(":") + 1);
oui.append(secondWord).append(System.lineSeparator());
}
nbOfLine++;
if(nbOfLine%1_000_000==0){
System.out.printf(
"%s / %s%n",
df.format(nbOfLine),
df.format(10000000));
}
}
System.out.println(oui);
final long consumed = System.nanoTime() - start;
final long totConsumed = TimeUnit.NANOSECONDS.toMillis(consumed);
final double tot = (double) totConsumed;
System.out.printf("Done. Took %s seconds", (tot / 1000));
} catch (IOException ex) {
ex.printStackTrace(); //handle an exception here
}
}
The test list is only a list of the 100 hash to crack

in my course and we talk about function reduction but I don't see how (when) to implement it.
I think you're confused about what rainbow tables even are, and how they differ from a simple table of passwords and the hashes of those passwords.
Rainbow tables are a way to save storage space in exchange for increasing the time needed to check each password hash against a candidate password compared to a full precomputed table of passwords and hashes.
Rainbow tables use a "reduction function". The reduction function takes in a hash and the column number (see below) and uses it to generate a possible password. Essentially it's a random password generator which uses a hash and column as its input seed.
Rainbow tables take a plaintext password candidate input, then repeatedly
hash the current plaintext
reduce the hash to a new plaintext
until some pre-chosen number of repetitions has finished, and hashes the final plaintext. Then you store only the starting password candidate and the final hash. Each iteration is a "column". Most aren't stored, but their numbers are used to vary the reduction function's outupt and prevent some problems.
To look up a password from its hash:
Look for the hash in the table. If it's there, break out of the loop.
If it's not there, reduce the hash to another plaintext, hash that, and go back to the start using that hash.
You now know which chain (if any) in the table this particular hash belongs to.
Take that chain's starting plaintext, and start hashing & reducing it until you come to the known hash. The previous plaintext is the password!
Rainbow tables don't work against any password hashing system that uses "salt" values. The first such system was Unix's crypt in 1976, so this tends to be a useless attack. It's still taught because the name is cool, and there are still people who make systems which don't salt their password hashes.
This article is a decent explanation. Wikipedia has more detail.

How can I maintain probability across multiple executions in Java

Firstly I am not the greatest with Math, so please excuse any ignorance relating to that. I am trying to maintain probability based randomness across multiple executions but I am failing. I have this input in a JSONObject
{
"option1": 25,
"option2":25,
"option3" :10,
"option4" :40
}
This is my function that selects a value from the above JSONObject based on the probability assigned:
public static String selectRandomoptions(JSONObject options) {
String selectedOption = null;
if (options != null) {
int maxChance = 0;
for (String option : options.keySet()) {
maxChance += options.getInt(option);
}
if (maxChance < 100) {
maxChance = 100;
}
Random r = new Random();
Integer randomValue = r.nextInt(maxChance);
int chance = 0;
for (String option : options.keySet()) {
chance += options.getInt(option);
if (chance >= randomValue) {
selectedOption = options.toLowerCase();
break;
}
}
}
}
the function behaves within a reasonable error margin if I call it x amount of times in a single execution ( tested 100+ calls), the problem is that I am running this every hour to generates some sample data in an event-driven app to verify our analytics process/data but we need it to be somewhat predictable, at least within a reasonable margin?
Has anyone any idea how I might approach this? I would rather not have to persist anything but I am not opposed to it if it makes sense or reduces complexity/time.

The values returned by Random.nextInt() are uniformly distributed, so that shouldn't be a problem.
I you would like to make random results repeatable, then you may want to use Random with seed.

Rather than create a new Random() object each time you want a new random number, just create the Random object once per run, and use the Random.nextInt() object once per run.
Looking at the documentation of Random() constructor,
This constructor sets the seed of the random number generator to a
value very likely to be distinct from any other invocation of this
constructor.it only guarantees it to be different
that's a bit of a weaker contract than the number you get from nextInt().
If you want to get the same sequence of numbers on each run, use the Random(long seed) or the setSeed(long seed) method of the random object. Both these methods set the seed of the generator. If you used the same seed for each invocation it's guaranteed that you will get the same sequence of numbers from the generator.
Random.setSeed(long).

Java: Is This Code Safe?

I'm making an Android App and have currently created some code to be used to generate a code. The generated code will be checked agaisnt the database to see if the code is in use. If it is it will re-generate another code until it finds a code which is not in use. This is done using a do-while loop, whilst there are no codes there will be no noticeable delay to the user. However if there are loads of codes there will be a noticeable delay won't there? The code is below:
public static String generateCode(DBAdapter myDB, String mgrName){
String[] name = mgrName.split(" +");
String fName = name[0];
String lName = name[1];
String fLetter = fName.substring(0, 1).toUpperCase();
String lLetter = lName.substring(0, 3).toUpperCase();
int randomPIN = (int) (Math.random() * 9000) + 1000;
String pin = String.valueOf(randomPIN);
String letters = new StringBuilder().append(fLetter).append(lLetter).append(pin).toString();
Boolean result = checkCode(myDB, letters);
if(result == true){
return(letters);
}
else{
String code = "";
Boolean resultfail = false;
do{
int randomPINFail = (int) (Math.random() * 9000) + 1000;
String generatedCode = new StringBuilder().append(fLetter).append(lLetter).append(randomPINFail).toString();
Boolean check = checkCode(myDB, generatedCode);
if(check){
resultfail = true;
code = generatedCode;
}
}while(!resultfail);
return code;
}
}
public static Boolean checkCode(DBAdapter myDB, String code){
Cursor cursor = myDB.getRowComplaint(code);
if(cursor.getCount() == 0){
return true;
}
else{
return false;
}
}
My Question is what is the chance that the generator will choose a code that is already in use so many times that the user will notice a delay? Bearing in mind the generator will use different manager names as well as different numbers. And is this code safe to use? If not what can be done to make it safe?
EDIT: I can't use UUID as the user has requested the code to contain four letters and four digits. The code is used to retrieve data from the db and that's why it needs to be unique.

As with any performance related question, there's no way for us to answer - you should profile it yourself by creating a large number of existing rows and then seeing how slow it is.
In terms of safety, that's a very broad term and I don't know anything about what you're using these codes for, so I couldn't comfortably tell you that the code is safe. But there don't seem to be any horrible problems with the way you're accessing the database.

Just use UUID class or any other built-in pseudo-random number generator - don't reinvent the wheel. In theory, they will provide such a small collision rate that in absolute most cases you'll generate unique id on a first try. But once again, it depends on your use case. I assume that you're doing something sane and not generating&storing millions of millions of those codes on mobile device.
Be sure to not invoke this routine from the main thread - in this case use might notice the delay even if your DB is empty.

Deterministic shuffle in Objective C

This code in Java is the implementation of Knuth's shuffle, but a deterministic one, controllable by the seed to the random number generator.
public String shuffleString(String data, long shuffleSeed) {
if(shuffleSeed!=0) {
Random rnd = new Random(shuffleSeed);
StringBuilder sb = new StringBuilder(data);
int n = data.length();
while(n>1) {
int k = rnd.nextInt(n--);
char t = sb.charAt(n);
sb.setCharAt(n, sb.charAt(k));
sb.setCharAt(k, t);
}
return sb.toString();
}
else {
return data;
}
}
How can I implement a deterministic shuffle in Objective C that outputs the same shuffle order given the same seed? I am using srandom(_shuffleSeed); and random()%(n--) knowing that arc4_random is better but that cannot be seeded.
- (NSString*) shuffleString:(NSString*) data withShuffleSeed:(int) shuffleSeed {
if(shuffleSeed!=0) {
srandom(_shuffleSeed);
NSMutableString *result = [[NSMutableString alloc] initWithString:data];
unsigned long n = data.length;
while(n>1) {
unsigned long k = random()%(n--);
unichar t = [result characterAtIndex:n];
NSRange r1 = {n,1};
[result replaceCharactersInRange:r1 withString:[NSString stringWithFormat:#"%c", [result characterAtIndex:k]]];
NSRange r2 = {k,1};
[result replaceCharactersInRange:r2 withString:[NSString stringWithFormat:#"%c", t]];
}
return result;
}
else {
return data;
}
}
Currently, the two shuffle methods do not generate the same result for the same input parameters. I am sure I am missing something!

There are many pseudo-random number generating algorithms that use a seed. You can't assume that the one in the Java standard library uses exactly the same algorithm as srandom/random in Objective C.
The Java random generator uses:
The class uses a 48-bit seed, which is modified using a linear
congruential formula. (See Donald Knuth, The Art of Computer
Programming, Volume 3, Section 3.2.1.)
It doesn't make any more guarantees, although it is never changed for backward compatibility reasons.
Your options are:
Take the Java source and convert it to Objective-C (or hope that somebody else has done this before). Note that the Java source is licensed under the GPL, or a restrictive Oracle license. If you take the version under the GPL license, that has an impact on the license that you can use for your own code.
Search for the source of the random generator in Objective-C and convert that to Java. (Which may also have license restrictions, and the source may not be available). Or maybe the algorithm is more properly specified so you can implement it in Java solely from the documentation.
Find another random generator that has a Java and an Object-C implementation that give identical results (or write one)

How to create user friendly unique IDs, UUIDs or other unique identifiers in Java

I usually use the UUID class to generate unique IDs. This works fine if these IDs are used by technical systems only, they don't care how long they are:
System.out.println(UUID.randomUUID().toString());
> 67849f28-c0af-46c7-8421-94f0642e5d4d
Is there a nice way to create user friendly unique IDs (like those from tinyurl) which are a bit shorter than the UUIDs? Usecase: you want to send out IDs via Mail to your customers which in turn visit your site and enter that number into a form, like a voucher ID.
I assume that UUIDs get generated equally through the whole range of the 128 Bit range of the UUID. So would it be sage to use just the lower 64 Bits for instance?
System.out.println(UUID.randomUUID().getLeastSignificantBits());
Any feedback is welcome.

I assume that UUIDs get generated
equally through the whole range of the
128 Bit range of the UUID.
First off, your assumption may be incorrect, depending on the UUID type (1, 2, 3, or 4). From the Java UUID docs:
There exist different variants of
these global identifiers. The methods
of this class are for manipulating the
Leach-Salz variant, although the
constructors allow the creation of any
variant of UUID (described below).
The layout of a variant 2 (Leach-Salz)
UUID is as follows: The most
significant long consists of the
following unsigned fields:
0xFFFFFFFF00000000 time_low
0x00000000FFFF0000 time_mid
0x000000000000F000 version
0x0000000000000FFF time_hi
The least significant long consists of
the following unsigned fields:
0xC000000000000000 variant
0x3FFF000000000000 clock_seq
0x0000FFFFFFFFFFFF node
The variant field contains a value
which identifies the layout of the
UUID. The bit layout described above
is valid only for a UUID with a
variant value of 2, which indicates
the Leach-Salz variant.
The version field holds a value that
describes the type of this UUID. There
are four different basic types of
UUIDs: time-based, DCE security,
name-based, and randomly generated
UUIDs. These types have a version
value of 1, 2, 3 and 4, respectively.
The best way to do what you're doing is to generate a random string with code that looks something like this (source):
public class RandomString {
public static String randomstring(int lo, int hi){
int n = rand(lo, hi);
byte b[] = new byte[n];
for (int i = 0; i < n; i++)
b[i] = (byte)rand('a', 'z');
return new String(b, 0);
}
private static int rand(int lo, int hi){
java.util.Random rn = new java.util.Random();
int n = hi - lo + 1;
int i = rn.nextInt(n);
if (i < 0)
i = -i;
return lo + i;
}
public static String randomstring(){
return randomstring(5, 25);
}
/**
* #param args
*/
public static void main(String[] args) {
System.out.println(randomstring());
}
}
If you're incredibly worried about collisions or something, I suggest you base64 encode your UUID which should cut down on its size.
Moral of the story: don't rely on individual parts of UUIDs as they are holistically designed. If you do need to rely on individual parts of a UUID, make sure you familiarize yourself with the particular UUID type and implementation.

Here is another approach for generating user friendly IDs:
http://thedailywtf.com/Articles/The-Automated-Curse-Generator.aspx
(But you should go for the bad-word-filter)

Any UUID/Guid is just 16 Bytes of data. These 16 bytes can be easily encoded using BASE64 (or BASE64url), then stripped off all of the "=" characters at the end of the string.
This gives a nice, short string which still holds the same data as the UUID/Guid. In other words, it is possible to recreate the UUID/Guid from that data if such becomes necessary.

Here's a way to generate a URL-friendly 22-character UUID
public static String generateShortUuid() {
UUID uuid = UUID.randomUUID();
long lsb = uuid.getLeastSignificantBits();
long msb = uuid.getMostSignificantBits();
byte[] uuidBytes = ByteBuffer.allocate(16).putLong(msb).putLong(lsb).array();
// Strip down the '==' at the end and make it url friendly
return Base64.encode(uuidBytes)
.substring(0, 22)
.replace("/", "_")
.replace("+", "-");
}
For your use-case, it would be better to track a running count of registered user, and for each value, generate a string-token like this:
public static String longToReverseBase62(long value /* must be positive! */) {
final char[] LETTERS = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ".toCharArray();
StringBuilder result = new StringBuilder(9);
do {
result.append(LETTERS[(int)(value % 62)]);
value /= 62l;
}
while (value != 0);
return result.toString();
}
For security reasons, it would be better if you make the values non-sequential, so each time a user registers, you can increment the value let's say by 1024 (This would be good to generate uuids for 2^64 / 2^10 = 2^54 users which is quite certainly more than you'd ever need :)

At the time of this writing, this question's title is:
How to create user friendly unique IDs, UUIDs or other unique identifiers in Java
The question of generating a user-friendly ID is a subjective one. If you have a unique value, there are many ways to format it into a "user-friendly" one, and they all come down to mapping unique values one-to-one with "user-friendly" IDs — if the input value was unique, the "user-friendly" ID will likewise be unique.
In addition, it's not possible in general to create a random value that's also unique, at least if each random value is generated independently of any other. In addition, there are many things you should ask yourself if you want to generate unique identifiers (which come from my section on unique random identifiers):
Can the application easily check identifiers for uniqueness within the desired scope and range (e.g., check whether a file or database record with that identifier already exists)?
Can the application tolerate the risk of generating the same identifier for different resources?
Do identifiers have to be hard to guess, be simply "random-looking", or be neither?
Do identifiers have to be typed in or otherwise relayed by end users?
Is the resource an identifier identifies available to anyone who knows that identifier (even without being logged in or authorized in some way)?
Do identifiers have to be memorable?
In your case, you have several conflicting goals: You want identifiers that are unique, random, and easy to type by end users. But other things you should think about are:
Are other users allowed to access the resource identified by the ID, whenever they know the ID? If not, then additional access control or a longer key length will be necessary.
Can your application tolerate the risk of duplicate keys? If so, then the keys can be completely randomly generated (such as by a cryptographic RNG such as java.security.SecureRandom in Java). If not, then your goal will be harder to achieve, especially for keys intended for security purposes.
Also, if you want IDs that have to be typed in by end users, you should consider choosing a character set carefully or allowing typing mistakes to be detected.

Only for you :) :
private final static char[] idchars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789".toCharArray();
private static String createId(int len) {
char[] id = new char[len];
Random r = new Random(System.currentTimeMillis());
for (int i = 0; i < len; i++) {
id[i] = idchars[r.nextInt(idchars.length)];
}
return new String(id);
}

How about this one? Actually, this code returns 13 characters(numbers and lowercase alphabets) max.
import java.nio.ByteBuffer;
import java.util.UUID;
/**
* Generate short UUID (13 characters)
*
* #return short UUID
*/
public static String shortUUID() {
UUID uuid = UUID.randomUUID();
long l = ByteBuffer.wrap(uuid.toString().getBytes()).getLong();
return Long.toString(l, Character.MAX_RADIX);
}