We Keep Coding

KTable data - Bytes printing

I have a KTable in Kafka that when I debug I get the data in bytes.
If I want it in String, what should I do? I have attached an snippet as well.

The key of the LinkedHashMap is a string. If you expand any String in the debugger, you'll see its byte representation.
That highlighted "value" is not the value of the map's keys, that would be the value field in the bottom of the screenshot.
Plus, you're expanding the topology builder, which is not the actual values in the Table. If you want to see the data in the table, table.toStream().print()

Something like:
public static String byteArrayToHexString(byte[] rawBytes) {
final String HEXCHARS = "0123456789abcdef";
StringBuilder sb = new StringBuilder(rawBytes.length * 2);
for (int i = 0; i < rawBytes.length; i++) {
int ix = (rawBytes[i] >> 4) & 0xF;
sb.append(HEXCHARS.charAt(ix));
ix = rawBytes[i] & 0xF;
sb.append(HEXCHARS.charAt(ix));
}
return sb.toString();
}

Java: How do I loop through characters in a string that have a surrogate pair and print them?

I tried this to loop through the characters in my string and print them. All of them are printing fine except the Deseret Long I (𐐀). I have no idea if there are other ways to do this so that the 𐐀 is printed correctly. Here is my code:
package javaapplication13;
public class JavaApplication13 {
public static void main(String[] args) {
String s = "h𤍡y𐐀\u0500";
System.out.println(s);
final int length = s.length();
for (int offset = 0; offset < length;) {
final int codepoint = s.codePointAt(offset);
System.out.println((char) (codepoint));
offset += Character.charCount(codepoint);
}
}
}
The output looks like this (Netbeans):
run:
h𤍡y𐐀Ԁ
h
䍡
y
Ѐ
Ԁ
BUILD SUCCESSFUL (total time: 0 seconds)

Your problem is caused by the fact that you try to convert int to char (4 bytes to 2 bytes). The value in the codepoint variable cannot fit in one char in case of surrogate pair. Look, it is called pair, because it is a pair of chars. I think the simplest way how you can print it is by using String.Substring() method. Or you can convert it to array of char's this way: char[] ch = Character.toChars(codepoint); and you can convert this array back to string by simple new String(ch).

java: perform set, setbit, get and getbit in hashmap

Edited
I am trying to implement the following functions in my program using hashmap.
get
getBit
set
setBit
Here is the code.
static HashMap<String, Object> hashmap = new HashMap<String, Object>();
public static String dictGet(String key){
String val = (String) hashmap.get(key);
return val;
}
public static int dictGet(String key, int pos){
String val = (String) hashmap.get(key);
byte[] bytes = val.getBytes();
byte byt = bytes[pos /8];
System.out.println("Byte : "+byt);
return (byt >> (pos % 8) ) & 1;
}
public static void dictSet(String key, String val){
hashmap.put(key, val);
}
public static void dictSet(String key, int pos, byte bitval){
String val = (String) hashmap.get(key);
byte[] bytes = val.getBytes();
byte byt = bytes[pos /8];
if(bitval == 1){
bytes[pos /8] = (byte) (bytes[pos /8]| (1 << (pos % 8)));
}
else if(bitval == 0){
bytes[pos /8] = (byte) (bytes[pos /8] & ~(1 << (pos % 8)));
}
hashmap.put(key, new String(bytes));
}
I want to achieve O(1) complexity for each of these functions. But currently getBit() and setBit() functions are taking O(n) time complexity.
Any help is appreciated.

This is very inefficient. val.getBytes() will allocate new array (actually more than one) and encode whole string into it every time you call it. This is likely going to be the most expensive operation in this snippet.
Moreover, this code mixes two completely different representations. To make your intentions clear and avoid mistakes, you should stick to one. If you want to operate on bytes, store byte[] in your hash map. If you want to operate on characters or code points use charAt or codePointAt. If you need to convert between those representation, do it once during initialization.
Notice also, that getBytes encodes the string using platform's default charset. This, most likely, is not what you want and your code will break, if UTF-16 is used.
Here is an example using StringBuilder, Note that in Java char has 16 bits and this code uses that full range:
package test;
import java.util.HashMap;
import java.util.Map;
public class Test {
Map<String, StringBuilder> map = new HashMap<>();
void setBit(final String key, final int n) {
final StringBuilder s = this.map.get(key);
final char c = s.charAt(n / 16);
s.setCharAt(n / 16, (char) (c | 1 << n % 16));
}
int getBit(final String key, final int n) {
final StringBuilder s = this.map.get(key);
final int c = s.charAt(n / 16);
return (c << n % 16 & 0x8000) != 0 ? 1 : 0;
}
CharSequence getValue(final String key) {
return this.map.get(key);
}
public static void main(final String[] args) {
final Test t = new Test();
t.map.put("x", new StringBuilder("abc"));
t.map.put("y", new StringBuilder("abc"));
t.setBit("x", 3);
t.setBit("y", 4);
// many methods accept any CharSequence not just String
final CharSequence xValue = t.getValue("x");
final CharSequence yValue = t.getValue("y");
System.out.println(String.join(",", xValue, yValue));
}
}

No, the complexity of this is not O(1), it is O(n), where n is the length of the string. The reason for this is that bytes that you receive from getBytes() must always be a copy; otherwise, String's immutability would be compromised.
This can be confirmed by looking at the source code: getBytes calls encode, which calls Arrays.copyOf:
static byte[] encode(Charset cs, char[] ca, int off, int len) {
StringEncoder se = new StringEncoder(cs, cs.name());
char[] c = Arrays.copyOf(ca, ca.length); // <<=== Copying an array is O(n)
return se.encode(c, off, len);
}
To get O(1) amortized complexity (assuming that you are going to access most bits of a string) you would need additional storage - a HashMap<String,byte[]> which would be a "parallel map" to the original hash map, storing "cached" values of getBytes() calls. Start by looking for a cached representation; if it is not there, go for the actual value, call getBytes(), and cache the result. The rest of the code would remain the same.
byte[] bytes = cache.get(key);
if (bytes == null) {
String val = (String) hashmap.get(key);
bytes = val.getBytes();
cache.put(key, bytes);
}
byte byt = bytes[pos /8];
System.out.println("Byte : "+byt);
return (byt >> (pos % 8) ) & 1;
Note that the maintenance of cache is on you: each time that you modify hashmap you need to remove the corresponding key from the cache.

I know this not related to your answer, but if you are using Java, you should be using generic types so that the cast is not required.
Coming to your answer, the complexity of String.getBytes() depends on the length of the String. Think in this way, the getBytes() method convert every character in the String to a byte. Hence, the complexity is O(L), where L is the length of the String.

How to treat a string as a variable in java?

Remember playing the game when you indexed each of the words of your name and added it all together to form a secret number? Like aay would be 1+1+25=27
I tried to do the same thing by various methods in java but I failed.Let me share my script first and then tell what all i tried.
class test{
public static void main(String args[]){
int c = 3;
String s = "c";
///now this is what all i tried:
int value1 = (int)(s);
///i tried a cast, but that failed.
String d = "4";
int value2 = Integer.parseInt(d);
///when i try this, it correctly converts the string to an integer.
Integer.parseInt(s);
///but when i try the above, it fails to do the same.
}
}
In this code, I try 2 different methods both of which are working in a similar way that I want to but not exact.
The problem is that it couldn't reorganize that c is a variable that has an integer value.
So, is there any shortcut to do it?
Also, right now the string is only 1 digit long, once the user inputs his or her name, I shall use a for loop to loop all of the letters completely.
If there isn't any shortcut, is the only option left with me is to make an if statement like:
if(letter=a;){
value=1;
}
Or something like that?
thanks for helping!

You can't convert a String directly into an integer, you need to take one character at a time and subtract the value of the char 'a' then add 1 :
public static void main(String[] a) {
String s = "test";
for (char c : s.toLowerCase().toCharArray()){
System.out.println(charToInt(c));
}
}
private static int charToInt(char c){
return c - 'a' + 1;
}

In Java like most other C-eqsue languages, char is a Number type.
char: The char data type is a single 16-bit Unicode character. It has a minimum value of '\u0000' (or 0) and a maximum value of '\uffff'
(or 65,535 inclusive).
Just Alphabetic Characters
So all you need to do is convert each character to its equivilent char and you have its ASCII/Unicode value; I won't go into Unicode here, because ASCII maps over Unicode into the correct place.
// this will give you a Zero based int of the
// UPPPERCASE alphabet which starts at `65` and
// ends at `90`.
// See where the lowercases starts and ends?
char c = "My Name".charAt(0) - 65;
// c = 'M' in this case which is 77 - 65 = 12
The ASCII codes are easy to translate.
(source: asciitable.com)
For UPPERCASE letters it is just an exercise of looping through all the chars of a String and getting their code and subtracting 65 to get a 0 based value. It gets a little more complicated to exclude other characters and process lowercase as well, but you get the idea from the table.
Printable Characters
Notice "printable" characters start at 32 and run through 126 which is usually what you just do, is subtract 32 instead of 65 and just convert all "printable" characters to a Zero base.
Example
The following will print out all the "printable" characters.
public class Main
{
public static void main(final String[] args)
{
for (char i = 32; i <= 126; i++)
{
System.out.println(new Character(i));
}
}
}
Solution
Here is a complete solution that will create a secret code, of all the *"printable" characters possible.
public class Main
{
public static void main(final String[] args)
{
int hash = 0;
final String s = args[0];
for (int c = 0; c < s.length(); c++)
{
hash = hash + s.charAt(c) - 32;
}
System.out.printf("Secret code for %s is %d", s, hash);
}
}
the result for my name is
Secret code for Jarrod is 418

A particular type of hash on a String concatenation

I need a specialised hash function h(X,Y) in Java with the following properties.
X and Y are strings.
h(X,Y) = h(Y,X).
X and Y are arbitrary length strings and there is no length limit on the result of h(X,Y) either.
h(X,Y) and h(Y,X) should not collide with h(A,B) = h(B,A) if X is not equal to A and Y is not equal to B.
h() does not need to be a secure hash function unless it is necessary to meet the aforementioned requirements.
Fairly high-performant but this is an open-ended criterion.
In my mind, I see requirements 2 and 4 slightly contradictory but perhaps I am worrying too much.
At the moment, what I am doing in Java is the following:
public static BigInteger hashStringConcatenation(String str1, String str2) {
BigInteger bA = BigInteger.ZERO;
BigInteger bB = BigInteger.ZERO;
for(int i=0; i<str1.length(); i++) {
bA = bA.add(BigInteger.valueOf(127L).pow(i+1).multiply(BigInteger.valueOf(str1.codePointAt(i))));
}
for(int i=0; i<str2.length(); i++) {
bB = bB.add(BigInteger.valueOf(127L).pow(i+1).multiply(BigInteger.valueOf(str2.codePointAt(i))));
}
return bA.multiply(bB);
}
I think this is hideous but that's why I am looking for nicer solutions. Thanks.
Forgot to mention that on a 2.53GHz dual core Macbook Pro with 8GB RAM and Java 1.6 on OS X 10.7, the hash function takes about 270 micro-seconds for two 8 (ASCII) character Strings. I suspect this would be higher with the increase in the String size, or if Unicode characters are used.

why not just add their hashCode's together?

Today I've decided to add my solution for this hash function problem. It was not tested very good and I did not measure its performance, so you can feed me back with your comments. My solution is situated below:
public abstract class HashUtil {
//determines that we want hash, that has size of 32 integers ( or 32*32 bits )
private static final int hash_size = 32;
//some constants that can be changed in sake of avoiding collisions
private static final BigInteger INITIAL_HASH = BigInteger.valueOf(7);
private static final BigInteger HASH_MULTIPLIER = BigInteger.valueOf(31);
private static final BigInteger HASH_DIVIDER = BigInteger.valueOf(2).pow(32*hash_size);
public static BigInteger computeHash(String arg){
BigInteger hash = new BigInteger(INITIAL_HASH.toByteArray());
for (int i=0;i<arg.length()/hash_size+1;i++){
int[] tmp = new int[hash_size];
for(int j=0;j<Math.min(arg.length()-32*i,32);j++){
tmp[i]=arg.codePointAt(i*hash_size+j);
}
hash = hash.multiply(HASH_MULTIPLIER).add(new BigInteger(convert(tmp)).abs()).mod(HASH_DIVIDER);
}
//to reduce result space to something meaningful
return hash;
}
public static BigInteger computeHash(String arg1,String arg2){
//here I don't forgot about reducing of result space
return computeHash(arg1).add(computeHash(arg2)).mod(HASH_DIVIDER);
}
private static byte[] convert(int[] arg){
ByteBuffer byteBuffer = ByteBuffer.allocate(arg.length*4);
IntBuffer intBuffer = byteBuffer.asIntBuffer();
intBuffer.put(arg);
return byteBuffer.array();
}
public static void main(String[] args){
String firstString="dslkjfaklsjdkfajsldfjaldsjflaksjdfklajsdlfjaslfj",secondString="unejrng43hti9uhg9rhe3gh9rugh3u94htfeiuwho894rhgfu";
System.out.println(computeHash(firstString,secondString).equals(computeHash(secondString,firstString)));
}
}
I suppose that my solution should not produce any collision for single string with length less then 32 (to be more precise, for single string with length less then hash_size variable value). Also it is not very easy to find collisions (as I think). To regulate hash conflicts probability for your particular task you can try another prime numbers instead of 7 and 31 in INITIAL_HASH and HASH_MULTIPLIER variables. What do you think about it? Is it good enought for you?
P.S. I think that it will be much better if you'll try much bigger prime numbers.

3) h(X,Y) and h(Y,X) should not collide with h(A,B) = h(B,A) if X is not equal to A and Y is not equal to B.
I think that this requirement rules any hash function that produces numbers that are smaller (on average) than the original Strings.
Any requirement of no collisions runs into the roadblock of the Pigeonhole Principle.

How strict are you being with requirement 4? If the answer is 'not completely strict' then you could just concatenate the two strings putting the smaller one first (this would result in a collision for h('A', 'B') and h('AB', ''))
If there are any characters which you are sure would never appear in the string values then you could use a single instance as a separator, which would fix the collision above.

From 4-th point we can get that h(x,"") should never collide with h(y,"") until x.equals(y) is true. So, you have no size limits on what produce h(x,y), cause it shoud produce unique result for each unique x. But there are infinite number of unique strings. This is not a correct hash function, I think.

Building on String#hashCode, this is not a perfect hash function, so it does not fulfill condition 4.
public static long hashStringConcatenation(String str1, String str2) {
int h1 = str1.hashCode();
int h2 = str2.hashCode();
if ( h1 < h2 )
{
return ((long)h1)<<32 & h2;
}
else
{
return ((long)h2)<<32 & h1;
}
}

Okay, #gkuzmin's comment made me think why I am doing the powers of 127. So, here's a slightly simpler version of the code. The changes are as follows:
I am no longer doing the powers of 127 but actually concatenating the codePointAt numbers as strings, converting the result into BigInteger for each input string and then adding the two BigIntegers.
To compact the answer, I am doing a mod 2^1024 on the final answer.
Speed is not any better (perhaps a little worse!) but then I think the way I am measuring the speed is not right because it probably also measures the time taken for the function call.
Here's the modified code. Does this fulfill all conditions, albeit 4 for such unfortunate cases where repetitions may occur over the 2^1024 result space?
public static BigInteger hashStringConcatenation(String str1, String str2) {
if(str1==null || str1.isEmpty() || str2 == null || str2.isEmpty()) {
return null;
}
BigInteger bA, bB;
String codeA = "", codeB = "";
for(int i=0; i<str1.length(); i++) {
codeA += str1.codePointAt(i);
}
for(int i=0; i<str2.length(); i++) {
codeB += str2.codePointAt(i);
}
bA = new BigInteger(codeA);
bB = new BigInteger(codeB);
return bA.add(bB).mod(BigInteger.valueOf(2).pow(1024));
}

I've decided to add another answer because #Anirban Basu have proposed another solution. So, I do not know how to provide link to his post and if somebody know how to do it - correct me.
Anirban's solution looks like this:
public static BigInteger hashStringConcatenation(String str1, String str2) {
if(str1==null || str1.isEmpty() || str2 == null || str2.isEmpty()) {
return null;
}
BigInteger bA, bB;
String codeA = "", codeB = "";
for(int i=0; i<str1.length(); i++) {
codeA += str1.codePointAt(i);
}
for(int i=0; i<str2.length(); i++) {
codeB += str2.codePointAt(i);
}
bA = new BigInteger(codeA);
bB = new BigInteger(codeB);
return bA.add(bB).mod(BigInteger.valueOf(2).pow(1024));
}
Your new solution now looks like a hash function, but it still has some problems. I suggest that you should think about this:
Maybe it will be better to throw NullPointerException or IllegalArgumentException when null was used as function argument? Are you sure, that you do not want to compute hash for empty strings?
To concatenate large amount of strings it is better to use StringBuffer instead of + operator. Use of this class will produce huge positive impact on your code performance.
Your hash function is not very secure - it is realy easy to compute strings, which will produce conflict.
You can try this code to check algorithm that can can demonstrate your hash function collision.
public static void main(String[] args){
String firstString=new StringBuffer().append((char)11).append((char)111).toString();
String secondString=new StringBuffer().append((char)111).append((char)11).toString();
BigInteger hash1 = hashStringConcatenation(firstString,"arbitrary_string");
BigInteger hash2 = hashStringConcatenation(secondString,"arbitrary_string");
System.out.println("Is hash equal: "+hash1.equals(hash2));
System.out.println("Conflicted values: {"+firstString+"},{"+secondString+"}");
}
So, It is realy easy to break your hash function. Moreover, it is good that it has 2^1024 result space, but a lot of real life conflicts for your implementation lies in very close and simple strings.
P.S. I think that you should read something about already developed hash algorithms, hash function that failed in a real life (like java String class hash function which computed hash using only 16 first characters in the past) and try to examine your solutions according to your requirements and real life. At least you can try to find hash conflict manually and if you succeed then your solution most likely already has some problems.

Here's my changed code according to #gkuzmin's suggestion:
public static BigInteger hashStringConcatenation(String str1, String str2) {
BigInteger bA = BigInteger.ZERO, bB = BigInteger.ZERO;
StringBuffer codeA = new StringBuffer(), codeB = new StringBuffer();
for(int i=0; i<str1.length(); i++) {
codeA.append(str1.codePointAt(i));
}
for(int i=0; i<str2.length(); i++) {
codeB.append(str2.codePointAt(i));
}
bA = new BigInteger(codeA.toString());
bB = new BigInteger(codeB.toString());
return bA.multiply(bB).mod(BigInteger.valueOf(2).pow(1024));
}
Note that in the result, I now multiply bA with bB instead of adding.
Also, added #gkuzmin's suggested test function:
public static void breakTest2() {
String firstString=new StringBuffer().append((char)11).append((char)111).toString();
String secondString=new StringBuffer().append((char)111).append((char)11).toString();
BigInteger hash1 = hashStringConcatenation(firstString,"arbitrary_string");
BigInteger hash2 = hashStringConcatenation(secondString,"arbitrary_string");
System.out.println("Is hash equal: "+hash1.equals(hash2));
System.out.println("Conflicted values: {"+firstString+"},{"+secondString+"}");
}
and another test with strings having only numeric values:
public static void breakTest1() {
Hashtable<String,String> seenTable = new Hashtable<String,String>();
for (int i=0; i<100; i++) {
for(int j=i+1; j<100; j++) {
String hash = hashStringConcatenation(""+i, ""+j).toString();
if(seenTable.contains(hash)) {
System.out.println("Duplication for " + seenTable.get(hash) + " with " + i + "-" + j);
}
else {
seenTable.put(hash, i+"-"+j);
}
}
}
}
The code runs. Of course, it is not an exhaustive check, but the breakTest1() function does not have any issues. #gkuzmin's function displays the following:
Is hash equal: true
Conflicted values: { o},{o }
Why do the two strings produce the same hash? Because they are effectively working with strings '11111arbitrary_string' in both cases. This is a problem.

How about the slightly modified function now?
public static BigInteger hashStringConcatenation(String str1, String str2) {
BigInteger bA = BigInteger.ZERO, bB = BigInteger.ZERO;
StringBuffer codeA = new StringBuffer(), codeB = new StringBuffer();
for(int i=0; i<str1.length(); i++) {
codeA.append(str1.codePointAt(i)).append("0");
}
for(int i=0; i<str2.length(); i++) {
codeB.append(str2.codePointAt(i)).append("0");
}
bA = new BigInteger(codeA.toString());
bB = new BigInteger(codeB.toString());
return bA.multiply(bB).mod(BigInteger.valueOf(2).pow(1024));
}
Here, we add a separator character "0" between each character codes, so the combination for characters 11 111 and 111 11 will no longer confuse the function because the concatenation will produce 110111 and 111011. However, it still will not break requirement 2 of the original question.
So does this now solve the problem albeit within the limits of the 2^1024 range?