get 15 bits CRC from any size bit array - java

I transfer data like this: [DATA - any size bit array][CRC 15 bits].
How i can get 15 bits CRC from any size bit array to detect accidental changes to raw data?
Here start of code:
byte crc[15];
int data_length = *any size*;
byte data[data_length]; // for example data = {1,1,1,0,1,0,1,0,1,1,1,1,0,1,0,1,1,1,0,1,0,1,1,1,1,0,1,0,1,1}
crc = get_crc(crc);
get_crc - ?

As a commenter said, your code example has no bits but bytes. I assume you´ve really want the last 15 bit out of some byte array named data and that you checked that data has at least 2 bytes.
It´s as simple as
short crc = (data[data_length - 2] & 0x7f) | data[data_length - 1];
Side note: Maybe you are aware of it already, but 15bit CRCs are not exactly the most reliable thing (but fast). If you data not completely unimportant, some better algorithm could make sense (SHA´s, or depending on the use case something else like ReedSolomon-encoding etc.etc.)

Related

Implementing a writeBit method in Java

So I know that in Java you cannot write out individual bits to a file and that you have to use writeByte. I have some understanding that there is a way to implement a writeBit method that makes use of writeByte by calling writeByte once 8 'bits' are concatenated together. I was hoping to implement this like:
public void writeBit(char bit) {
try {
//functionality here
} catch (IOException e) {
System.out.println(e);
}
}
But I just cannot seem to get started. I understand that I should probably have some attribute that keeps track of how many bits I have concatenated, but other than that I'm lost as to how to implement this.
I guess my big question here is how can I continuously call writeBit without losing my concatenated String of bits, and what would an implementation of writeBit look like, if it were to make use of writeByte?
As a side note, I am using a DataOutputStream here if this was not clear.
I've noticed a couple people have talked about using two instance variables, one to store the byte as you add bits to it and another to keep track of how many bits have been added so far. While this is a perfectly good way to do it, I'd like to show why you don't need a second instance variable.
Theory
There's no need to keep track of how many bits have been added so far. The only piece of information we need is "has the byte been filled yet?". Instead of initializing your byte to 0, try initializing it to a value of 1. Then each time you add a bit, shift the bits of the byte to the left one place (using the bitshift operator <<), and then add the new bit in the rightmost place.
In practice, it would look something like this, where X is the newly added bit:
Initialize the byte to a value of 1: 00000001
To insert a new bit, shift the bits to the left: 00000010
And add the new bit X in the rightmost place: 0000001X
Shift left: 000001X0
Add the new bit: 000001XX
Eventually, you'd have 7 bits written from your method and the leftmost bit would be 1: 1XXXXXXX
So in your method, you can check to see if the leftmost bit is set every time it's called. If it is, then you know you're ready to write the byte to the file on this iteration. You would start by doing the same thing, shifting left and then adding the new bit, so now you have XXXXXXXX. Then you would write the now-full byte to the file, and then reset the byte to a value of 1 so the cycle can start over again.
Writing the code
First you'll need an instance variable to keep track of these bits. It will need to be type byte, and I'll just call it buffer.
To shift the bits to the left one place, we can use the bitshift operator, <<. And, to make our lives even easier, there's even a bitshift assignment operator, <<=, so we can perform the bitshift and assign the new value back to the variable all in one operation. This leaves us with:
buffer <<= 1;
The next thing we'll need to do is add the new bit. If you OR a value with 1, the rightmost bit will be set, and the rest of the bits will be unaffected. If you OR a value with 0, none of the bits are affected. We can use this trick to only set the rightmost bit if the new bit is a 1 (The |= is the OR assignment operator):
buffer |= bit ? 1 : 0;
Then, the last piece of this code is writing the if statement to check if the leftmost bit is set. If it is, then when we AND it with 10000000, we will get 10000000. If not, we will get 00000000. 10000000 is 128 in decimal (or -128, or 256, all will work), so our expression is:
(buffer & 128) == 128
Result
Putting all these pieces together, we get:
// Notice bit is type boolean
public void writeBit(boolean bit) {
// If the leftmost bit in buffer is set:
if ((buffer & 128) == 128) {
// Shift all the bits in buffer to the left 1 place
buffer <<= 1;
// Add the new bit in the rightmost place
buffer |= bit ? 1 : 0;
// Write the now-full byte to the file
// I'm just calling your DataOutputStream "dos" here
try {
dos.writeByte(buffer);
} catch (IOException e) {
throw new RuntimeException();
}
// Reset buffer to a value of 1
buffer = 1;
} else {
// Shift all the bits in buffer to the left 1 place
buffer <<= 1;
// Add the new bit in the rightmost place
buffer |= bit ? 1 : 0;
}
}
Make a class with two instance variables, the first with the bits that you have accumulated so far, and the second with how many bits you have accumulated. Use the shift and or operations to insert a bit into the buffer (initialized to zero), and increment the number of bits. Once you have eight bits, write the buffer, then zero out the buffer and the count.
At the end you will need to flush any remaining bits if the count is not zero by writing the buffer to a byte, even though it contains less than eight bits. The format of the sequence of bits needs to be able to deal with this eventuality, unless it assures that a multiple of eight bits is always written.

Varying value bit length array

I have recently come across the problem of creating arrays with values that have a specified bit length. Say an array with 13bits instead of 8,16,32 etc. I tried to look for a good tutorial/article about it as I am new to bit operations. Though I am not really sure of what to search for. I presume the array would work with a backing array of bytes or longs...
My ultimate question is if you can show me if there is a duplicate question or tutorial out there.
If not perhaps show me an example. AND if you got the time write a short explanation.
Thank you.
EDIT: The purpose is not to make an array of say longs but only use 40% of it. I want it to be packed together to save space to be compatible with the thing im making.
It's not possible to "create your own primitive types" in java. Also I don't think there is any library around here to do what you want. I think most people would go with the overhead of losing some memory, especially at bit level. Maybe C or Cpp would have been a wiser choice (and I'm not even sure).
You'll have to create your own bit manipulation library. There are many ways to do it, I'll give you one. I began using a byte[] but it's more complex. As a rule, use the biggest normal type (ex: for a 48bit elements, use 32 bit types as storage). so let's go with an int array (16 bits) for 100 of your 13bits types. I'll use big-endian-style storage.
int intArraySize = 100 * 16 / 13 + 1; // + 1 is just to be sure...
int[] intArray = new int[byteArraySize];
Now, how do you access the sixth value for example. You'll always need at least and at most two int of your array and an integer to store it.
int pos = 6;
int buffer = 0;
int firstPart = int Array[ (pos * 13) /16]; // 1010 0110 1100 0011
int secondPart = int Array[ (pos * 13) /16 + 1]; // 1001 1110 0101 1111
int begin = pos * 13 % 16;
The variable begin = 14 is the bit at which your number begins. So that means on your 13bits elements there are (16-14) 3 bits in the first (left) int and the rest (13-3 = 10) in the second (right).
The number you want is 1010 0110 1100 0{011 and 1001 1110} 0101 1111.
You're gonna put these two ints into one now. Right shift the secondPart 3 times (so it's the right part of your final number), and left shift the firstPart 10 times, add them in the buffer. Because it's a 13bits elements, you'll need to clean ( with a bitmask ) the 3 first elements of your 16 bit in the buffer, and voila !
I'll let you guess how to insert a value in the array (try doing the same step, but in reverse) and be carefull not to erase other values. And if you haven't looked yet: https://docs.oracle.com/javase/tutorial/java/nutsandbolts/op3.html
Disclaimer: I didn't try the code, but you get the general idea. There might be some errors, maybe you'll have to add or remove 1 to begin. But you get the general idea. The first thing you should do is make a function that prints/log any integer (or byte, or whatever) into it's binary representation. Multiple possibilities here: Print an integer in binary format in Java because you're gonne need them to test every step of your code.
I still think it's a bad idea to store your special number this way, (seriously memory is rarely gonna be an issue), but I found the exercise interesting, and maybe you really need taht kind of storage. If your curious, take a look at the ByteArrayOutputStream, I'm not sure you'll ever need this for what you're doing but who knows.

How do I combine two AudioInputStream?

The file format is "PCM_SIGNED 44100.0 Hz, 16 bit, stereo, 4 bytes/frame, little-endian", and I want to add them together while amplifying one of the two files. I plan to read the two wav get put them into two audioinputstream instances, then store the instances into two byte[] array, manipulate in the arrays, and get return as another audioinputstream instance.
I have done a lot of research but I have got no good results.
I know that is a class from www.jsresources.org mixing two audioinputstream, but it doesn't allow me to modify either of the two streams before mixing while I want to decrease one of the streams before mixing them. What do you think I should do?
To do this, you can convert the streams to PCM data, multiply the channel whose volume you wish to change by the desired factor, add the PCM data from the results together, then convert back to bytes.
To access the AudioStreams on a per-byte basis, check out the first extended code fragment at the Java Tutorials section on Using Files and Format Converters. This shows how to get an array of sound byte data. There is a comment that reads:
// Here, do something useful with the audio data that's
// now in the audioBytes array...
At this point, iterate through the bytes, converting to PCM. A set of commands based on the following should work:
for (int i = 0; i < numBytes; i += 2)
{
pcmA[i/2] = audioBytesA[i] & 0xff ) | ( audioBytesA[i + 1] << 8 );
pcmB[i/2] = audioBytesB[i] & 0xff ) | ( audioBytesB[i + 1] << 8 );
}
In the above, audioBytesA and audioBytesB are two input streams (names based on the code from the example), and pcmA and pcmB could be either int arrays or short arrays, holding values that fit within the range of a short. It might be best to make pcm arrays floats since you will be doing some math that will result in fractions. Using floats as in the example below only adds one place worth of accuracy (better rounding than when using int), and int would perform faster. I think using floats is more often done if the audio data gets normalized for use with additional processing.
From there, the best way to change volume is to multiply every PCM value by the same amount. For example, to increase volume by 25%,
pcmA[i] = pcmA[i] * 1.25f;
Then, add pcmA and pcmB, and convert back to bytes. You might also want to put in min or max functions to ensure that the volume & merging do not exceed values that can fit in the format's 16 bits.
I use the following to convert back to bytes:
for (int i = 0; i < numBytes; i++)
{
outBuffer[i*2] = (byte) pcmCombined[i];
outBuffer[(i*2) + 1] = (byte)((int)pcmCombined[i] >> 8 );
}
Above assumes pcmCombined[] is a float array. The conversion code can be a bit simpler if it is a short[] or int[] array.
I cut and pasted the above from dev work I did for programs posted at my website, and edited it for your scenario, so if there is a typo or bug crept in, please let me know in the comments and I will fix it.

Difference between storing images in byte array and binary (BLOB) and which one is faster

I want to insert and select images from sql server in jdbc. I am confused whether BLOB and byte are the same thing or different. I have used Blob in my code and the application loads slow as it has to select the images stored in Blob and convert it pixel by pixel. I want to use byte array but I don't know whether they are same or different. My main aim is to load the image faster.
Thank you
Before going further, we may need to remember about basic concepts about bit, byte and binary, BLOB.
Bit: Abbreviation of binary digit. It is the smallest storage unit. Bits can take values of 0 or 1.
Byte: Second smallest storage which is commonly (nibble is not mentioned since it is not very common term) used. It includes eight bits.
Binary: Actually, it is a numbering scheme that each digit of a number can take a value of 0 or 1.
BLOB: Set of binary data stored in a database. Also, type of a column which stores binary data inside.
To sum up definitions: Binary format is a scheme that which include bits.
To make it more concrete, we can observe results with the code below.
import java.nio.ByteBuffer;
public class TestByteAndBinary{
public static void main(String []args){
String s = "test"; //a string, series of chars
System.out.println(s);
System.out.println();
byte[] bytes = s.getBytes(); //since each char has a size of 1 byte, we will have an array which has 4 elements
for(byte b : bytes){
System.out.println(b);
}
System.out.println();
for(byte b : bytes){
String c = String.format("%8s", Integer.toBinaryString(b)).replace(' ', '0'); //each element is printed in its binary format
System.out.println(c);
}
}
}
Output:
$javac TestByteAndBinary.java
$java -Xmx128M -Xms16M TestByteAndBinary
test
116
101
115
116
01110100
01100101
01110011
01110100
Let's go back to the question:
If you really want to store an image inside a database, you have to use the BLOB type.
BUT! It is not the best practice.
Because databases are designed to store data and filesystems are
designed to store the files.
Reading image from disk is a simple thing. But reading an image from
the database need more time to accomplished (querying data,
transforming to an array and vice versa).
While an image is being read, it will cause the database to suffer
from lower performance since it is not simple textual or numerical read.
An image file doesn't benefit from characteristical features of a database (like indexing)
At this point, it is best practice to store that image on a server and store its path on the database.
As far as I can see on enterprise level projects, images are very very rarely stored inside the database. And it is the situation that those images were needed to store encrypted since they were including very sensual data. According to my humble opinion, even in that situation, those data had not to be stored in a database.
Blob simply means (Binary Large Object) and its the way database stores byte array.
hope this is simple and it answers your question.

Java multi-bit / compact small integer array

I am working on implementing some bloom filter variants, and a very useful data structure for this would be a compact multi-bit array; that is, an array where each element is a compact integer of around 4 bits.
Space efficiency is of the utmost importance here, so while a plain integer array would give me the functionality I want, it would be bulkier than necessary.
Before I try to implement this functionality myself with bit arithmetic, I was wondering if anyone knows of a library out there that already provides such a data structure.
Edit: Static size is fine.
The ideal case would be an implementation that is flexible with regard to the number of bits per cell. That might be a bit much to hope for though (no pun intended?).
If you aren't modifying the array after creation, java.util.BitSet does all the bit masking for you but is slow to access since you have to fetch each bit individually and do the masking yourself to re-create the int from 4 bits.
Having said that writing it yourself might be the best way to go. Doing the bit arithmetic yourself isn't that difficult since it's only 2 values per byte so decoding the high bits are (array[i] & 0xF0) >> 4 and the low bits are array[i] & 0x0F
Take a look at the compressed BitSet provided by http://code.google.com/p/javaewah/, it allows to set bits freely and will ensure that it uses memory efficiently via compression algorithms being used.
I.e. something like
EWAHCompressedBitmap32 set = new EWAHCompressedBitmap32();
set.set(0);
set.set(1000000);
will still only occupy a few bytes, not one MB as with the Java BitSet...
You should be able to map the 4-bit integer to the BitSet by multiplying the index into the BitSet accordingly

Categories