We Keep Coding

How to get the optimal cluster number using the elbow method for java?

I use haifengl/smile and I need to get the optimal cluster number.
I am using CLARANS where I need to specify the number of clusters to create. I think maybe there is some solution to sort out for example from 2 to 10 clusters, see the best result and choose the number of clusters with the best result. How can this be done with the Elbow method?

To determine the appropriate number of clusters such that elements within the cluster are similar to each other and dissimilar to elements in other groups, can be found by applying a variety of techniques like;
Gap Statistic- compares the total within intra-cluster variation for different values of k with their expected values under null reference distribution of the data.
Silhouette Method The optimal number of clusters k is the one that maximizes the average silhouette over a range of possible values for k.
Sum of Square method
For more details, read the sklearn documentation on this subject.

The Elbow method is not automatic.
You compute the scores for the desired range of k, plot this, and then visually try to find an "elbow" - which may or may not work.
Because x and y have no "correct" relation to each other, beware that the interpretation of the plot (and any geometric attempt to automate this) depend on the scaling of the plot and are inherently subjective. In the end, the entire concept of an "elbow" likely is flawed and not sound in this form. I'd rather look for more advanced measures where you can argue for the maximum or minimum, although some notion of "significantly better k" would be desirable.

Ways to find clusters:
1- Silhouette method:
Using separation and cohesion or just using an implemented method the optimal number of clusters is the one with the maximum silhouette coefficient. * silhouette coefficient range from [-1,1] and 1 is the best value.
Example of the silhouette method with scikit-learn.
2- Elbow method (You can use the elbow method automatically)
The elbow method is a graph between the number of clusters and the average square sum of the distances.
To apply it automatically in python there is a library Kneed in python to detect the knee in a graph.Kneed Repository

Genetic Algorithm - Grouping people: Only find solutions containing criteria X,Y and Z

I am trying to solve the following problem:
I have a list of 30 people.
These people need to be divided into groups of 6.
Each person has given the names of 3 other people who they would like to be in a group with.
I thought of solving this problem using a genetic algorithm.
The fitness function could evaluate all the groups, and assign a fitness score based on how many people per room have all their preferences met. (or a scoring system similar to that)
Example:
One of the generated solutions is: 1,3,19,5,22,2,7,8,11,12,13,14,15,13,17....etc
I would assume the first 5 people are in the first group, and the next 5 in the the next group and calculate a fitness value from that.
I think that this solution would work - does anyone see a better way of doing this?
My main question is this:
If I want to make sure person A and B are definitely in the same group, I could implement the fitness function to check for this and assign a terrible fitness if this condition isn't met. Is this the best way to do it? It seems quite inefficient.
Is there a way to 'lock' certain parts of the solution ("certain genes") and just solve or the remainder?
Any help or insights will be appreciated.
Thanks in advance.
AK

Just to clarify a bit, your problem isn't about genetic programming but genetic algorithms, which are two different things. Genetic programming is about generating (using evolutionary algorithms) executable individuals that will generate your solutions while genetic algorithms individuals represent directly your solutions.
That being said, your two assumptions are corrects. Data representation is a key element of evolutionary algorithms in general and a bad representation may hinder efficient solution space exploration. Your current data representation seems correct to me, given groups are only allowed to have exactly 5 individuals. Your second thought about the way to enforce some criteria is also right. Putting a large fitness value (preferably one that can't represent a potentially valid even if bad solution) such as infinity (if your library / language allows it easily) is the preferred way to express invalid solutions in literature. This has multiple advantages over simply deleting invalid individuals: During the selection stage, bad individuals won't be selected and thus the solution space they represent won't be explored as much as interesting ones, which is computationally good because it surely won't contain optimal solutions. Knowing a solution is bad is good knowledge, after all. At the same time, genetic diversity is really important in evolutionary algorithms in order to avoid stagnation. At least some bad individual should be kept for the sake of genetic diversity in order to explore solution spaces between currently represented zones.
The goal of genetic algorithms is to compute solutions that are either impossible or too hard to compute analytically or by brute-force. Trying to dynamically lock down some genes with heuristics would require much knowledge about the inner working of your problems as well as the underlying evolution mechanisms and would be defeating the purpose of using evolutionary algorithms. The effective goal of evolutionary algorithms is to lock down genes that seems correct.
In fact, if you are a priori absolutely positively certain that some given genes must have a given value, don't represent them in your individuals. For instance, make your first group 3 individuals long if you are sure that the 2 others must be of some given value. You can then code your evaluation function as if there was 5 individuals in the first group but won't be evolving / searching to replace the 2 fixed ones.

What does your crossover operation look like? The way you have it laid out in your description, I'm not sure how you implement it cleanly. For instance if you have two solutions:
1, 2, 3, 4, 5, ....., 30
and
1, 2, 30, 29,......,10
Assuming you're using single point crossover function, you would have the potential to get multiple assignments for the same people and other people not being assigned at all using the genomes above.
I would have a genome with 30 values, where each value defines a person's group assignment (1-6). It would look like 656324113255632....etc. So person 1 is assigned group 6, person 2 group 5, etc. This would make the crossover operation easier to implement, because you don't have to ensure that after crossover each new solution is a valid assignment regardless of whether it's optimal.
The fitness function would assign a penalty for each group not having the proper number of members (5), and additional penalties for group member assignments that are suboptimal. I would make the first penalty significantly larger than the second, and then adjust these to get the results you're looking for.

This can be modeled as a generalized quadratic assignment problem (GQAP). This problem allows to specify a number of equipment (people) that demand a certain capacity, a number of locations (groups) that offer a capacity and the weights matrix that specifies the closeness between equipment and the distance matrix specifying the distance between locations. Additionally, there are install costs, but these are not required for your problem. I have implemented this problem in HeuristicLab. It's not part of the trunk, but I can send you the plugin if you're interested (or you compile it yourself).

It seems that the most challenging part of using a genetic algorithm for this problem is implementing the crossover. Here's how I would do it:
First choose a constant, C. C will stay constant throughout all generations, and I will explain it's purpose in a moment.
I will use a smaller example than 5 groups of 6 to demonstrate this crossover, but the premise is the same. Say we have 2 parents, each consisting of 3 groups of 3. Let's make one [[1,2,3],[4,5,6],[7,8,9]], and the other [[9,4,3],[5,7,8],[6,1,2]].
Make a list of possible numbers (1 through total number of people), in this case it is simply [1,2,3,4,5,6,7,8,9]. Remove 1 random number from the list. Let's say we remove 2. The list becomes [1,3,4,5,6,7,8,9]
We assign each remaining number a probability. The probability starts at 1, and goes up by C for any matches with the parents. For example, in parent 1, 3 and 2 are in the same group so 3 would have a probability of 1+C. Same thing with 6 because it forms a match in parent 2. 1 would have a probability of 1+2C, because it is in the same group as 2 in both parents. Based on these probabilities, use a roulette wheel type selection. Let's say we pick 6.
Now, we have 2 and 6 in the same group. We similarly look for matches with these numbers and make probabilities. For each parent, we add C if it matches with only 2 or only 6, and 2C if it matches with both. Continue this until the group is done (for 3x3 this is the last selection, but for 5x6 there would be a few more)
4.Choose a new random number that has not been picked and continue for other groups
One of the good things about this crossover, is that it basically includes mutations already. There are chances built in to group people that were not grouped in their parents
Credit: I adapted the idea from the Omicron Genetic Algorithm

How to implement equivalence class in Java?

What would be the simple way to implement equivalence class in Java? Is there any library for that purpose?
The bothering part is how to write an efficient and non-naive "equal" operator.
Let S = {x,y,z,w,h}. If we use a mapping x->1, y->1, z->1, w->2, h->2 for the equivalence class of S, one has to consider the mapping x->10, y->10, z->10, w->20, h->20 as the same equivalence class.
Naive "equal" operator can quickly become time-consuming when the cardinal of the set S becomes large.
What would be the simple way? Any idea?
[EDITED] To clarify, the specific problem can be formalized as follows:
Let S be a non-empty set. We denote by M a set of partial mappings from V to integers. It is relatively easy to show that the binary relation \sim defined below derives an equivalence relation on M.
For m1 and m2 two partial mappings of M, m1 \sim m2 if and only if,
for any a of V, m1(a) is defined if and only if m2(a) is defined
for any a,b of V, m1(a) and m1(b) are both defined to be the same
integer value 'z1' if and only if m2(a) and m2(b) are both defined
to the same integer value 'z2' (which may or may not differ from
'z1')
Example.
a->9,b->9,w->1 \sim a->10,b->10,w->0
But it is not correct to say
a->5 \sim b->9
Thanks.

From what I understand from your question you can find the greatest common divisor (Euclid's algorithm recursively) for a set once and map the quotients with that instead - if they're exactly equal with another set it's equal, else not. This will only work if the sets are equal in size and mappings.

If I understand you correct, you could apply vector normalization. A 3d vector for example is normalized to a length of 1 by dividing all of its components separately with the vectors length. If two normalized vector's components are equal, their original (non-normalized) vectors point in the same direction (which is what I think you define as 'equal')
x,y,z,w,h would in your case be a 5-dimensional vector. They belong to the same class when the show into the same direction, but may have an arbitrary length.

Aside: I assume that the set S is actually the set V in your definition.
I think Uli is on the right track, although I would not assume that Set(Set(E)).equals() is efficient for your purposes. (Sorry, I couldn't get the lt or gt symbols to come through)
The default implementation of Set(E).equals() is likely O(nlog n) or O(n^2). Set(E).equals() almost certainly involves sorting; O(nlog n) is as good as it gets. I suggest you look at radix sort. It's O(n*log n), but grows very slowly.

Addition Chains [duplicate]

How can you compute a shortest addition chain (sac) for an arbitrary n <= 600 within one second?
Notes
This is the programming competition on codility for this month.
Addition chains are numerically very important, since they are the most economical way to compute x^n (by consecutive multiplications).
Knuth's Art of Computer Programming, Volume 2, Seminumerical Algorithms has a nice introduction to addition chains and some interesting properties, but I didn't find anything that enabled me to fulfill the strict performance requirements.
What I've tried (spoiler alert)
Firstly, I constructed a (highly branching) tree (with the start 1-> 2 -> ( 3 -> ..., 4 -> ...)) such that for each node n, the path from the root to n is a sac for n. But for values >400, the runtime is about the same as for making a coffee.
Then I used that program to find some useful properties for reducing the search space. With that, I'm able to build all solutions up to 600 while making a coffee. But for n, I need to compute all solutions up to n. Unfortunately, codility measures the class initialization's runtime, too...
Since the problem is probably NP-hard, I ended up hard-coding a lookup table. But since codility asked to construct the sac, I don't know if they had a lookup table in mind, so I feel dirty and like a cheater. Hence this question.
Update
If you think a hard-coded, full lookup table is the way to go, can you give an argument why you think a full computation/partly computed solutions/heuristics won't work?

I have just got my Golden Certificate for this problem. I will not provide a full solution because the problem is still available on the site.I will instead give you some hints:
You might consider doing a deep-first search.
There exists a minimal star-chain for each n < 12509
You need to know how prune your search space.
You need a good lower bound for the length of the chain you are looking for.
Remember that you need just one solution, not all.
Good luck.

Addition chains are numerically very important, since they are the
most economical way to compute x^n (by consecutive multiplications).
This is not true. They are not always the most economical way to compute x^n. Graham et. all proved that:
If each step in addition chain is assigned a cost equal to the product
of the numbers at that step, "binary" addition chains are shown to
minimize the cost.
Situation changes dramatically when we compute x^n (mod m), which is a common case, for example in cryptography.
Now, to answer your question. Apart from hard-coding a table with answers, you could try a Brauer chain.
A Brauer chain (aka star-chain) is an addition chain where each new element is formed as the sum of the previous element and some element (possibly the same). Brauer chain is a sac for n < 12509. Quoting Daniel. J. Bernstein:
Brauer's algorithm is often called "the left-to-right 2^k-ary method",
or simply "2^k-ary method". It is extremely popular. It is easy to
implement; constructing the chain for n is a simple matter of
inspecting the bits of n. It does not require much storage.
BTW. Does anybody know a decent C/C++ implementation of Brauer's chain computation? I'm working partially on a comparison of exponentiation times using binary and Brauer's chains for both cases: x^n and x^n (mod m).

Markov Chain Text Generation

We were just assigned a new project in my data structures class -- Generating text with markov chains.
Overview
Given an input text file, we create an initial seed of length n characters. We add that to our output string and choose our next character based on frequency analysis..
This is the cat and there are two dogs.
Initial seed: "Th"
Possible next letters -- i, e, e
Therefore, probability of choosing i is 1/3, e is 2/3.
Now, say we choose i. We add "i" to the output string. Then our seed becomes
hi and the process continues.
My solution
I have 3 classes, Node, ConcreteTrie, and Driver
Of course, the ConcreteTrie class isn't a Trie of the traditional sense. Here is how it works:
Given the sentence with k=2:
This is the cat and there are two dogs.
I generate Nodes Th, hi, is, ... + ... , gs, s.
Each of these nodes have children that are the letter that follows them. For example, Node Th would have children i and e. I maintain counts in each of those nodes so that I can later generate the probabilities for choosing the next letter.
My question:
First of all, what is the most efficient way to complete this project? My solution seems to be very fast, but I really want to knock my professor's socks off. (On my last project A variation of the Edit distance problem, I did an A*, a genetic algorithm, a BFS, and Simulated Annealing -- and I know that the problem is NP-Hard)
Second, what's the point of this assignment? It doesn't really seem to relate to much of what we've covered in class. What are we supposed to learn?

On the relevance of this assignment with what you covered in class (Your second question). The idea of a 'data structures' class is to expose students to the very many structures frequently encountered in CS: lists, stacks, queues, hashes, trees of various types, graphs at large, matrices of various creed and greed, etc. and to provide some insight into their common implementations, their strengths and weaknesses and generally their various fields of application.
Since most any game / puzzle / problem can be mapped to some set of these structures, there is no lack of subjects upon which to base lectures and assignments. Your class seems interesting because while keeping some focus on these structures, you are also given a chance to discover real applications.
For example in a thinly disguised fashion the "cat and two dogs" thing is an introduction to statistical models applied to linguistics. Your curiosity and motivation prompted you to make the relation with markov models and it's a good thing, because chances are you'll meet "Markov" a few more times before graduation ;-) and certainly in a professional life in CS or related domain. So, yes! it may seem that you're butterflying around many applications etc. but so long as you get a feel for what structures and algorithms to select in particular situations, you're not wasting your time!
Now, a few hints on possible approaches to the assignment
The trie seems like a natural support for this type of problem. Maybe you can ask yourself however how this approach would scale, if you had to index say a whole book rather than this short sentence. It seems mostly linearly, although this depends on how each choice on the three hops in the trie (for this 2nd order Markov chain) : as the number of choices increase, picking a path may become less efficient.
A possible alternative storage for the building of the index is a stochatisc matrix (actually a 'plain' if only sparse matrix, during the statistics gathering process, turned stochastic at the end when you normalize each row -or column- depending on you set it up) to sum-up to one (100%). Such a matrix would be roughly 729 x 28, and would allow the indexing, in one single operation, of a two-letter tuple and its associated following letter. (I got 28 for including the "start" and "stop" signals, details...)
The cost of this more efficient indexing is the use of extra space. Space-wise the trie is very efficient, only storing the combinations of letter triplets effectively in existence, the matrix however wastes some space (you bet in the end it will be very sparsely populated, even after indexing much more text that the "dog/cat" sentence.)
This size vs. CPU compromise is very common, although some algorithms/structures are somtimes better than others on both counts... Furthermore the matrix approach wouldn't scale nicely, size-wize, if the problem was changed to base the choice of letters from the preceding say, three characters.
None the less, maybe look into the matrix as an alternate implementation. It is very much in spirit of this class to try various structures and see why/where they are better than others (in the context of a specific task).
A small side trip you can take is to create a tag cloud based on the probabilities of the letters pairs (or triplets): both the trie and the matrix contain all the data necessary for that; the matrix with all its interesting properties, may be more suited for this.
Have fun!

You using bigram approach with characters, but usually it applied to words, because the output will be more meaningful if we use just simple generator as in your case).
1) From my point of view you doing all right. But may be you should try slightly randomize selection of the next node? E.g. select random node from 5 highest. I mean if you always select node with highest probability your output string will be too uniform.
2) I've done exactly the same homework at my university. I think the point is to show to the students that Markov chains are powerful but without extensive study of application domain output of generator will be ridiculous