We Keep Coding

In netty, how we can send string messages with different arbitrary length?

I want to send string message by netty. To do that I need to use StringDecoder and encoder as follows:
ch.pipeline().addLast("frameDecoder", new LineBasedFrameDecoder(**maxLength**));
ch.pipeline().addLast("stringDecoder", new StringDecoder(CharsetUtil.UTF_8));
// Encoder
ch.pipeline().addLast("stringEncoder", new StringEncoder(CharsetUtil.UTF_8));
It is nice, but the only part that make me disappointed in the maxLengh?! I don't know the maximum size of my string messages? Why it needs that, it can find string by a delimiter or \r\n why it need the maxLength?
Is there any way to send string messages without specifying a length for them? And by the way, if I set the limit to a large number, and only use the small portion, do I loose anything? Am I wasting space?
I am writing a distributed key-value store, servers in my system replicate new write to each other, I don't know how large is a the value of a key.
Thanks

Everything has a beginning must have an end. What if no LF is sent to the server? That's why there is a max length. I believe you should set a limit of line length when designing this kind of system.
If you set a very large number, and only short lines are received, no space is wasted.

I found another post that can be used for sending string:
Netty channel.write not writing message
So using the method suggested there, I came up with this solution, but I don't if it is efficient.
Send:
String str = "Hello";
ctx.writeAndFlush(Unpooled.copiedBuffer(str, CharsetUtil.UTF_8)); // (3)
Receive:
I first deifne an StringBuffer in the class as a member:
StringBuffer sBuf = new StringBuffer("");
Then in the channelRead I will have:
public void channelRead(ChannelHandlerContext ctx, Object msg) {
System.out.println("Inside channelRead");
ByteBuf in = (ByteBuf) msg;
while (in.isReadable()) {
byte b = in.readByte();
char c = (char)b;
System.out.print(c);
if (c == '\n') {
System.out.println("Client received: " + sBuf.toString());
sBuf = new StringBuffer("");
} else {
sBuf.append(c);
}
}
ctx.close();
in.release();
}
I know, it does not looks efficient, but I check the java code for readLine() method of the BufferReader that I used to use in in the traditional way here:
http://developer.classpath.org/doc/java/io/BufferedReader-source.html
I think it is doing the same thing (which I don't like) it uses the StringBuffer and append method, it looks complex though, maybe that makes it more efficient than my simple idea :p
So what do you thing guys? I tested it, it works fine, the only concern is the efficiency.

Why does creating a string from multiple objects fail after joining the first two objects

Today, I encountered a problem when I tried to combine multiple "variables" or objects, like the following example code:
String stuff = "blah blah" + amazingness.getName() + " fantabulous code: " + address.
The above example works fine normally, but it does not work in my case.
So basically, I have a server which receives data via UDP means. Now, in order to use UDP, I have to create many threads, so the application does not hang. I don't know if the problem has anything to do with multiple threading, since the processor randomly runs the threads in a random order.
I am using java version 1.8 update 05.
Here is the code in my method.
String message = "Client" + c.name + "(" + c.getID() + ") #" + c.address.toString() + ":" + c.port + "disconnected";
Where:c.name is "Classic", c.getID() returns a random number, c.address.toString() returns the client's ip address, and c.port is the client's port.
When I run the code, I get a message of "Client Classic" and nothing else.
The expected message is: "Client Classic(1932) # 70.40.423.110:8112 disconnected"
Here is a screen shot:
Where has the other values gone?
Now I start to think it may be because referencing other classes messes up the process of adding more values to the string.
So I try to get the values before setting the message string
String message = "";
String name = c.name;
int cID = c.getID();
String address = c.address.toString();
int port = c.port;
message = "Client " + name + "(" + cID + ") # " + address + ":" + port + "disconnected";
System.out.println(message);
Where:String name is Classic, cIDis a random number, address is an ip address, and port is a port number. All of the variables are not static. Still does not output the desired message:
NOW, I test it again without the threads, just a standalone class:
IT WORKS PERFECTLY! Now, my question is why does the above result occur and what did I do wrong?
I appreciate the time you have spent reading my question and I hope you can understand it.
Edit 1: I noticed, in the value of message, the end quotations were never added, will look into it.
Edit 2: looks like the fault is in the name variable, although I have no idea why.
Edit 3: Looks like the name variable contains multiple "null" characters, I will look into it.
Edit: I believe I have fixed the problem, I'll contain a rundown:
Since I created a byte array of length 1024 to send through the network, I never trimmed the array before sending.
Calling "string.trim()" deletes the excess whitespace characters created when I set the variable "name".
Even though I had found this out without seeing the answer from David, I'll credit him with the correct answer.

Do you only see this behavior in Eclipse? I suspect that what is happening is that your name variable contains a null (0) character, which is not a problem for Java per se, but may be a problem for SWT (the widget toolkit used by Eclipse).
For example, the following code creates an array filled with zeros, overwrites the first few 0's with some letters, and then constructs a string similarly to your example.
public class TestMain {
public static void main(String args[]) {
byte[] badChars = new byte[10]; // all bytes are 0
String test = "Middle";
for (int i = 0;i < test.length() && i < badChars.length; ++i) {
badChars[i] = (byte) test.charAt(i);
}
String a = new String(badChars);
System.out.println(a);
String p = "Before " + new String(badChars) + " After";
System.out.println(p);
}
}
The output of this in eclipse is
Middle
Before Middle
But when I run this within Intellij IDEA, I get
Middle
Before Middle After
Just as you would expect.
When I add the following code just after the last println above:
StringBuffer b = new StringBuffer();
for (int i = 0; i < p.length(); ++i) {
char ch = p.charAt(i);
if (ch > 0) {
b.append(ch);
}
}
System.out.println(b.toString());
I get the following in Eclipse
Middle
Before Middle
Before Middle After
So I think it is possible that your 'name' contains a null char, which is causing SWT problems when displayed (such as the debugger, or even the console). SWT uses native widgets (and probably native C style strings, meaning null terminated).
Hope this makes sense - try stripping nulls from your string before you print them, even though they shouldn't be significant in Java.

Can you try:
String message = String.format("Client %s (%d) # %s:%d disconnected", name, cID, address, port);

Dart How to parse bytes from a String

My server side Dart application receives a JSON String from a socket. The string is generated by Java code. When the string is sent to the socket in the Java code it is encoded to UTF8 and two bytes, a short int, are prepended to the string. The value of this short is the number of bytes in the string + 2.
I need to extract that value as an int in order to handle the string but nothing I've tried has worked. It dies at JSON.decode (below) because it encounters the start of another JSON string. The first byte of the second string is the start short with the length of the second JSON string. The two strings are each less then 40 characters long.
(I will need to append the length to strings sent from Dart to Java as well.)
Java line
out.writeUTF(json); // converts and writes to the socket stream.
Dart server side code method
handleJavaSocket(Socket javasocket){
javasocket.transform(UTF8.decoder).listen((String socketString){
var truncated = socketString.substring(2);
String message = JSON.decode(truncated); // dies here
// more code
}, onError: (error) {
print('Bad JavaSocket request');
});
}
One of the JSON strings before encoding
{"target":"DOOR","command":"OPEN"}

So you're always sending as [sizeuper8, sizelower8, ..utf8-string...] as your message boundaries? UTF8 decode doesn't expect length as a parameter and sees the two bytes as unicode (probably null followed by a character).
I'm currently working on a StreamBuffer for Quiver (https://pub.dartlang.org/packages/quiver) that will let you pipe the socket to a buffer that gives you:
read(2).then((bytes) => read(bytes[1]<<8|bytes[2]).then((string) => UTF8.decode(string)));
You can can post the decoded string for whatever you like after that, but it should demux your data.
Current pull request (work in progress): https://github.com/google/quiver-dart/pull/117

I couldn't find a way to convert the two byte header to an integer especially since one byte could be 0 and some string methods look at that as a string terminator.
Since a message from the Java application will always consist of the header followed by the message I ignore the header and parse the message(s) from the Stream adding them to a list (in order to echo them back as verification),
I then traverse the list and simulate the header by writing a 0 byte followed the message size to the stream stream followed by the message. This won't work as is with message lengths greater than 255 but I don't expect any even close to that size.
it should be noted that the Dart application and the Java application are on the same machine.
handleJavaSocket(Socket javasocket){
// echo socket
javasocket.transform(UTF8.decoder).forEach((item) {
var start = 2;
var end = -1;
var messages = new List<String>();
while((++end < item.length) && (end = item.indexOf('}',end)) != -1) {
messages.add(item.substring(start, ++end));
start = end + 2;
}
for(var message in messages) {
// header message length as two bytes
javasocket.writeCharCode(0); // max length 254
javasocket.writeCharCode(message.length);
javasocket.write(message); // <== send
}
});
}

java - how to pass multiple parameters over serial port to Arduino mega

Through my java program. i want to pass a byte value to the Arduino mega to blink an Led
and also at the same time i want to pass a string value to the Arduino to be displayed in the lcd.
How can I separately get above 2 inputs from the java program to Arduino and use them in different processes inside Arduino..
Below is the arduino code
LiquidCrystal lcd (12, 11, 10, 9, 8, 7);
int operation;
void setup() {
lcd.begin(16, 2);
Serial.begin(9600);
Serial1.begin(9600);
Serial2.begin(9600);
pinMode(3, OUTPUT);
pinMode(2, OUTPUT);
}
int count = 0;
void loop() {
//LCD start
if (Serial.available()) {
// wait a bit for the entire message to arrive
delay(50);
// clear the screen
lcd.clear();
delay(10);
// read all the available characters
while (Serial.available() > 0) {
// display each character to the LCD
lcd.write(Serial.read());
}
}
//LCD end
//LED Blink start
if (Serial.available() > 0)
delay(10);
{
operation = Serial.read();
}
if(operation == '2')
{
digitalWrite(2, LOW);
digitalWrite(3, HIGH);
delay(50);
digitalWrite(3, LOW);
delay(50);
}
if(operation == '1')
{
digitalWrite(3, LOW);
digitalWrite(2, HIGH);
delay(50);
digitalWrite(2, LOW);
delay(50);
}
//LED Blink end
// Recieve rfid tag numbers
if(Serial1.available()) {
int x = Serial1.read();
Serial1.print(x);
}
if(Serial2.available()) {
int x = Serial2.read();
Serial1.print(x);
}
}
Below is the Java code to send data
Code to send number 1 to Arduino
String buf = "1";
char buf2[] = buf.toCharArray();
output.write((byte)buf2[0]);
Code to send string to display in lcd
output.write("Hellow world. this is a String from java".getBytes());
When I run these codes separately it works well without any interference.. but when I do them both together... sometimes value 1 or 2 is displayed in the lcd.. and led doesnot blink properly . how to get two inputs from java to arduino and process them separately inside the Arduino?

I think there might be several issues working in concert.
First of all, Strings are represented internally in Java as UTF-16 encoded characters. I don't remember what Arduino "C" operates on per default, but I am pretty sure it is not UTF-16.
From the JavaDoc
"A String represents a string in the UTF-16 format in which
supplementary characters are represented by surrogate pairs (see the
section Unicode Character Representations in the Character class for
more information). Index values refer to char code units, so a
supplementary character uses two positions in a String."
http://docs.oracle.com/javase/6/docs/api/java/lang/String.html
Second,
the method getBytes() return a platform specific encoded array, so depending on what platform you run the program on, the returned bytes can vary.
Try looking into using public byte[] getBytes(String charsetName), which will give you predictable values back.
Try something like bytes[] asciiBytes = new String("Hello World").getBytes("US-ASCII");.
See http://docs.oracle.com/javase/6/docs/api/java/nio/charset/Charset.html for more info on Character sets.

You could concatenate the data.
So, if you are passing "Hello" and "200" as 2 items then combine them before the send to Arduino and send "Hello%200" and split on the % inside the Arduino.

I am having trouble using FileReader to write a txt file to an array (Java), what am I doing wrong?

Scanner s = null;
try {
s = new Scanner(new BufferedReader(new FileReader("rates.txt")));
for (int i=0; i<9; i++){
while(s.hasNext()){rates[i] = s.next();}
System.out.println(rates[i]);
}
}catch (IOException e){
System.out.println(e);
}
finally {
if (s != null) {
s.close();
}
}
When I run this code, it reads the last chunk of characters in my txt file, places them in rates[0], sticks null in 1-9. I'm not sure why it's reading the end of my file first. The contents of the txt are below..
USD 1.34
EUR 1.00
JPY 126.28
GBP 0.88
INR 60.20
It reads the 60.20, which is all it is recording in the array. Any help would be appreciated. I guess I could give you the results of running this code:
run:
60.20
null
null
null
null
null
null
null
null
BUILD SUCCESSFUL (total time: 0 seconds)

while(s.hasNext()){rates[i] = s.next();}
In plain english, this says: While there are tokens left, put the next token into rates[i].
So it will put the first token into rates[i], then the next token into rates[i], then the next token into rates[i], ..., and finally the last token into rates[i]. Since i is not modified, all the values are written into the same element of the array, overwriting the previously read values.

I recommend:
Using List instead of array
More flexible, much easier to work with, takes advantage of Java Collections Framework, etc
Not storing the currency symbol and the numeric exchange rate all in one mixed bag
...but using a class to encapsulate the pair
Using Scanner.nextDouble() to read the numeric exchange rate (which presumably you'll want to convert to double anyway)
So, something like this:
List<ExchangeRate> allRates = new ArrayList<ExchangeRate>();
while (sc.hasNext()) {
String symbol = sc.next();
double rate = sc.nextDouble();
allRates.add(new ExchangeRate(symbol, rate));
}
Note how:
You no longer need to know how many elements to allocate in an array
The symbol and the rate aren't all thrown into one mixed bag
List.add means no counter that you need to keep track of and manage
i.e. the bug in your original question!

I think the problem is that line 5, which contains your while loop, reads the entire file input. So you read your entire file on the first for loop iteration where i = 0; The next time your for loop there is nothing left to read.
You probably want something like this instead:
List rates = new ArrayList();
while (s.hasNext()) {
rates.add(s.next());
}

One other potential problem: FileReader uses the platform default encoding. This can be appropriate to process user-supplied files, but if the files are part of the application, they can get corrupted when the application is run on a system with an incompatible default encoding (and no, using only ASCII characters does not protect you completely against this).
To avoid the problem, use new InputStreamReader(new FileInputStream(filename), encoding) instead - and realize that you actually have to pick an encoding for your file.