We Keep Coding

Why ssh-keygen does not output 32 bytes but instead 1612 chars?

My goal is to create my own TLS handshake server, so I can understand how encryption between server-client works. I am following this reference. In the Server Key Exchange Generation section from the reference, the private key length are said to be 32 bytes. But the example length is 64 bytes. Also, ssh-keygen output 1612 chars for private.key and 567 chars for private.pub.
My expected result: The private.pub should be used as is in Server Hello section of the reference which sends the public key with length 32 bytes to the client.
My actual result: The ssh-keygen or even keytool (I can't read the file with IntelliJ IDEA, so right now, I am avoiding it) output more than 32 bytes.
The cmd used
ssh-keygen
keytool -genkeypair -keyalg RSA -validity 7 -keystore keystore
test\experiment\tls\tls\TLSHandshake.java
class TLSHandshakeTest {
#Test
void doHandshake() {
String key = "MC4CAQAwBQYDK2VuBCIEIJCRkpOUlZaXmJmam5ydnp+goaKjpKWmp6ipqqusra6v";
byte[] bytes = key.getBytes();
System.out.println(Arrays.toString(bytes));
// output: 64 instead of 32, does this mean that the `private.pub` file must be processed with an encoder?
System.out.println(bytes.length);
}
}

The 32 byte (128 bit) secret key produced in the "server key exchange generation" step is not the RSA private key. It is actually the key for the symmetric encryption algorithm that will be used to protect application data.
What happens is that the RSA asymmetric algorithm is used to establish identities, and to protect the negotiation. During this process the symmetric algorithm is agreed and 128 bit secret keys are generated and exchanged (securely). Once the session is established, the 128 bit keys are use to encrypt and decrypt the data sent over the connection.
Why don't they use RSA for everything?
Basically, it is too computationally expensive to use RSA (or any other asymmetric algorithm) for bulk data encryption / decryption. The symmetric (block cypher) algorithms are much more efficient. (So you could view the initial negotiation as a "key distribution mechanism" for the session private keys.)
As for the size of the private and public key files generated by ssh-keygen:
The files contain more than just one key:
The public key file contains the value of the modulus and the public exponent e
The private key file usually contains the values of the modulus and both e and the private exponent d and several more values (p, q, dp, dq, qinvp). This is part of why the private key file is bigger than the public key file!
The file contents are encoded in ASN.1 which adds some extra type headers.
The (binary) ASN.1 encoded data is then base64 encoded which uses roughly one ASCII character for each 6 bits of data (plus some padding).
Some additional stuff is added to that:
The private key has line breaks added, along with "PEM" header and trailer lines.
The public key has the name of the algorithm (e.g. "ssh-rsa") and a "comment" added which serves to identify the person the key belongs to. (The comment is largely ignored by SSH ... but it is useful when you are manually adding or removing keys from an "authorized keys" file.)
For later versions of OpenSSH, OpenSSH key formats may be used instead of OpenSSL key formats. These use XDR encoding in place of ASN.1. This may be relevant to you if you are trying to write your own SSH implementation1.
As we can see, the "private key" file generated by ssh-keygen actually contains both the public and private keys for the keypair. Indeed it is possible to recover the keypair's public key from a private key file; e.g.
ssh-keygen -f ~/.ssh/id_rsa -y > ~/.ssh/id_rsa.pub
Refer to Create a public ssh key from the private key for more details.
1 - To be honest, I can't see the point of doing that. It is a lot of effort ... just to learn a bunch of details that you most likely don't need to know.

how to SHA512-CRYPT for Dovecot in JAVA?

I have a Dovecot server with MySQL database for storing usernames and passwords. The passwords in the database are in SHA512-CRYPT scheme.
I am inserting the hashed passwords in the database using a script.
doveadm pw -s SHA512-CRYPT -p password -r 500000
I want to hash the passwords using a JAVA application. I found this questions and I tried to create the same resulting hash using same password firstpassword and salt FooBarBaz. For some reason the resulting hash I get is different, although I am using the same hashing algorithm, salt and password.
Here is my Java code:
byte[] password = "firstpassword".getBytes();
byte[] salt = "FooBarBaz".getBytes();
MessageDigest digest = MessageDigest.getInstance("SHA-512");
digest.reset();
digest.update(salt);
byte[] hashed = digest.digest(password);
String encodedHash = Base64.getEncoder().encodeToString(hashed);
System.out.printf("{SHA512-CRYPT}$6$%s$%s", "FooBarBaz",encodedHash);
This outputs the hash:
{SHA512-CRYPT}$6$FooBarBaz$5WPtOnXVI/a6f003WByGKIcsfa6x0ansxiyE8uEfJ0TE5pI+Rv9kcMLgdZboKg7ZSWQgWFg+pIqruvdg6aiP/g==
I also tried swapping the order of salt + password to make it:
digest.update(password);
byte[] hashed = digest.digest(salt);
this gives me:
{SHA512-CRYPT}$6$FooBarBaz$QWS8+W5EWhModF+uO2tcsd55tDxzdzGJ5FurIbEgwVCwKfT5UqwIvBNG1Oyws8bZEFdeGgyD0u6zS1KArvGf9Q==
Does anyone have any idea how can I accomplish the same hash results in Java if I use the same password and salt?
The hash I am looking for is:
{SHA512-CRYPT}$6$FooBarBaz$.T.G.7FRJqZ6N2FF7b3BEkr5j37CWhwgvPOOoccrr0bvkBbNMmLCxzqQqKJbNhnhC.583dTBLEuZcDuQe7NEe.

doveadm uses the Unix crypt family of functions to generate the hash and outputs the hash as a Base64 encoded string. The alphabet used for the encoding (by crypt) is [a-zA-Z0-9./] (as mentioned on the man page for the functions). However, the alphabet used by the java.util.Base64 class is [A-Za-z0-9+/] (compliant with RFC 4648, as mentioned on the JavaDoc page for the Base64 class). Therefore, even if the hashed values are the same, they will get encoded differently.
A reliable option is to use the Crypt class from Apache Commons Codec as Crypt.crypt("firstpassword", "$6$FooBarBaz") (The prefix $6$ is mandatory to instruct Crypt that the SHA512-CRYPT algorithm needs to be used). This will generate your expected hash value.

Translating Java RSA encryption routine to Objective-C

I am trying to duplicate an encryption process that is working in Java over to iOS/OSX.
My Java code is as follows:
PublicKey publicKey = KeyFactory.getInstance("RSA").
generatePublic(new RSAPublicKeySpec(firstKeyInteger, secondKeyInteger));
// This always results in the public key OpenSSLRSAPublicKey{modulus=2b3b11f044.....58df890,publicExponent=10001}
Cipher cipher = Cipher.getInstance("RSA/ECB/OAEPWITHSHA1ANDMGF1PADDING");
String stringToEncode = "EncodeThisString";
byte[] bytesToEncode = stringToEncode.getBytes("UTF-8");
cipher.init(cipher.PUBLIC_KEY, publicKey);
byte[] encrypted = cipher.doFinal(plain);
The first challenge i'm struggling with is how to use the public key in iOS. Can I just dump the modulus into NSData and use it? Or must I store it in the keychain first? (I don't really need to use the keychain unless I must). Or is there a method similar to generatePublic() were I can recreate the public key using the 2 integers?
Then would I use SecKeyEncrypt to encrypt? Whenever I add this to my project I get Implicit declaration warnings even though I import the Security framework.
Thanks
EDIT -----
I think I have managed to get a Base64 encoded public key, which I believe is what is in a PEM certificate. Now, how to use it.

Java Python Message Signature and Verification

My server is coded in Python and I am making a java client for this server.
I am Signing a message(data) using:
public static byte[] Sign(PrivateKey privateKey, byte[] data) throws Exception{
System.out.println("Signing the key inside RSACrypto#Sign");
Signature dsa = Signature.getInstance("SHA1withRSA");
SecureRandom secureRandom =null;
dsa.initSign(privateKey,secureRandom);
dsa.update(data);
return dsa.sign();
}
This returns a byteArray(named signed_data) , now I am encoding this signature using Base64.encodeBase64 and converting the byteArray(named my_byte_array) so formed into string using :
String str = new String(my_byte_array)
and sending this string to Server.
On server side, I receive this string , then Server verifies the signature using:
publicKey.verify(str(data), (long(base64.b64decode(my_byte_array)),))
using library http://gdata-python-client.googlecode.com/hg/pydocs/gdata.Crypto.PublicKey.RSA.html
when I try to print my_byte_array on both side they are same, and so is signed_data and base64.b64decode(my_byte_array)
but I am getting this error:
ValueError: invalid literal for long() with base 10: '\x8b\xa1\xbb\x19fO\xea\xe7\xa4B\xd4\xd2\xa1\xe3\xb9\xd0\x89n\xa2\xfe\xb5\xedsL\x02\xba\xad\x12!qjp\x0c%+Z\t\xa7\x12\x08\x90\xfaTk\xca\xd0\xae\xd8\xa9\xfa\xbb]>9\x1c\x80\xd0
As far as I can think, this error is arising because Java signs the message into byte and python expects it to be in Long.
Is there a way to solve this problem?

You have actually 2 problems.
The first one is that - according to the Java Cryptograpy Architecture API - the SHA1withRSA algorithm involves PKCS#1 v1.5 padding. At the Python side, you must use the same padding scheme to verify the signature; that can be achieved with PyCrypto's PKCS#1 v1.5 signature module (Crypto.Signature.PKCS1_v1_5).
The second problem is the one you point out: the verify method of an RSA PyCrypto object
oddly requires the signature to be encoded as an integer. However, by using the module I mentioned above, the problem will go away, since it accepts byte strings.

Veryfing signature generated by BouncyCastle on PC using native Java JCE

I am using BouncyCastle to generate a DSA signature but using the native JCE to verify the it.
NOTE: I am working with a j2me client that does not natively support signing hence the need for BouncyCastle)
So, on the client the signature is generated as follows:
DSASigner sig = new DSASigner();
sig.init(true, privateKey);
String plaintext = "This is the message being signed";
BigInteger[] sigArray = sig.generateSignature(plaintext.getBytes());
...
sigArray contains 2 BigIntegers r and s.
This signature then has to be transmitted to a server which uses native JCE to verify the sig. On the server side, using the native Java JCE, it should be possible to verify a signature as follows:
...
Signature sig = Signature.getInstance("SHA1withDSA");
byte[] sigbytes = Base64.decode(signature);
sig.initVerify(publicKey);
sig.update(plaintext.getBytes());
sig.verify(sigbytes)
The problem am having is: how do i encode sigArray into a format that can be sent to the pc/server as a single Base64 string (instead of separately as r and s) that can then be verified on the server using the native JCE method show in the second snippet of code?
So far i have tried to create DERObjects from the r,s arrays (separately, together as one array, encoded) but still no luck. Anybody faced this before? How did you tackle it?

According to Cryptographic Message Syntax Algorithms (RFC 3370) the DSA signature encoding is an ASN.1 sequence containing both integers r and s:
Dss-Sig-Value ::= SEQUENCE {
r INTEGER,
s INTEGER }