We Keep Coding

Reading System.in character by character without buffering

I know that similar questions have been asked before, but not exactly what I'm asking. To begin with, let me explain my purpose. I'm trying to write a kind of "remote shell" that will take in characters from the console (System.in) on character at a time and then send those to a remote session on another machine, write them to that machine and gather any characters it might output to return to my shell to display back to the user.
So, the issue is that System.in, no matter what I do, doesn't really support a "raw" mode where any type of reader is able to read just one character at a time UNTIL a terminator character is entered, typically new line.
Things I have tried, Using Scanner, using a buffered reader, creating a FileDescriptor.in and creating a fileInputStream from that, using a FileChannel and reading into a ByteBuffer that is one character long, etc. In all cases, it seems, System.in only makes characters available to the java application after a terminator character has been entered by the user. I'm convinced there is not a "java" way to do this, so the question is, does anyone have some native code, wrapped in a java library to do this? Its hard to find such a thing just searching GitHub.
BTW, for the remote console, I'm using the pty4J package. I've seen sample projects that connect to that code using other langauages, for example javaScript running in a browser to create a web based shell. Other languages all you to do a simple "get_char" on standard in.
Some examples of the code I've tried:
Scanner scanner = new Scanner(System.in);
FileDescriptor fd = FileDescriptor.in;
FileInputStream fis = new FileInputStream(fd);
FileChannel fc = fis.getChannel();
while(process.isAlive()) {
System.out.println(scanner.next());
// ByteBuffer bb = ByteBuffer.allocate(1);
// int c = fc.read(bb);
// int c = fis.read();
// System.err.println("Read " + c);
// if (c == 1) {
// os.write(bb.get());
// }
}
You can see that I've tried various methods to read the input: scanner.next(), fc.read(byteBuffer), fileInputStream.read(), etc. All attempts "wait" till a terminator character is entered.
Additionally, I have tried using the "useDelimiter" and "next(pattern)" methods on the scanner too. That's still not working.
Any pointer or help is much appreciated.

Below is an example of reading one character at a time until end of stream is reached. On linux, you type control-d to signal the end input. I think on Windows, you type control-c to end of input.
import java.io.*;
class Test {
public static void main(String[] args) throws IOException {
int c = 0;
while( (c=System.in.read()) != -1){
System.out.println((char) c);
}
}
}

java android application cannot make http get

Can someone helps me to get json from the web.In the end of function jsonResponse is empty. I use this method to do it:
private String getJson() {
jsonResponsce = "";
AsyncTask.execute(new Runnable() {
#Override
public void run() {
try{
URL httpbinEndpoint = new URL(webPage);
HttpsURLConnection myConnection = (HttpsURLConnection) httpbinEndpoint.openConnection();
myConnection.setRequestMethod("GET");
// Enable writing
myConnection.setDoOutput(true);
String internetData = "";
// Write the data
myConnection.getOutputStream().write(internetData.getBytes());
jsonResponsce = internetData;
} catch (IOException e) {
e.printStackTrace();
}
}
});
return jsonResponsce;
}
I set an Internet permission to the manifest. I try go get Json from the next address: https://shori-dodjo-mobile-app.firebaseio.com/.json. Full code is placed here: https://github.com/GenkoKaradimov/Shori-Dodjo-Android-App/

You are executing the request asynchronously so the method starts the execution and then completes and therefore there is no result. The result will be there in a second but by that time the method getJson has already completed. You most probably need to put the code that uses the json at the end of the run method.
In addition your code for reading from the stream seems wrong. It should probably be something like
BufferedReader br = new BufferedReader(new InputStreamReader(myConnection.getInputStream()));
jsonResponsce = br.lines().collect(Collectors.joining("\n"));
(I haven't tested this)

There are multiple issues in your code:
First, AsyncTask means it's async(hronous), so you can't return the result right away. Instead, override AsyncTask's onPostExecute and do what you need to do with the data there. Here is the sample implementation.
Second, you're using getOutputStream, which is intended for writing to the connection, i.e. sending data to the server. In your case you need to getInputStream and read from it. Easiest way is to wrap it in a BufferedReader and read until it returns -1 (marking end of stream), and then convert to string.
There are a few quirks: You should handle, or at least recognize errors by checking HTTP status code, handle encodings (the convert-bytes-to-string part), and handle cases when response is compressed, e.g. using DEFLATE or gzip. I've implemented that in a pure Java way (reference code, warning: outdated docs), but I'd seriously recommend using one of the established libraries such as Retrofit or Volley.

Json objects usually get return as HashMaps.
So you might need something like, HashMap yourMap = new HashMap<~>();
then
yourMap.get("the objects name on the other side", the var its getting saved too.);
right now it looks like all you are trying to do is save the byte data, but this byte data needs to have a type. Hope this helps.

XStream not sent via sockets

I used already working code for save/load game for sending a player state via sockets. And I encountered a problem that game save is correct, but server is not receiving client's player state.
Here is the base code that is tested and working:
int retval = fc.showSaveDialog(givenComponent);
if (retval == JFileChooser.APPROVE_OPTION) {
File file = fc.getSelectedFile();
try {
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(file), "UTF-8"));
XStream xs = new XStream();
GameSave gs = new GameSave();
ArrayList<PlayerSerialize> listps = new ArrayList<PlayerSerialize>();
for (Player tempplayer : Players.players) {
PlayerSerialize ps = new PlayerSerialize();
ps.getPlayerData(tempplayer);
listps.add(ps);
}
gs.playersSerialize = listps;
gs.gamedate = Dateutils.gamedate;
String s = xs.toXML(gs);
bw.write(s);
bw.close();
} catch (IOException ex) {
Logger.getLogger(DialogMainField.class.getName()).log(Level.SEVERE, null, ex);
}
}
Here is the client side code that is not sending anything to server:
XStream xs = new XStream();
GameSave gs = new GameSave();
ArrayList<PlayerSerialize> listps = new ArrayList<PlayerSerialize>();
PlayerSerialize ps = new PlayerSerialize();
ps.getPlayerData(Players.players.get(1));
listps.add(ps);
gs.playersSerialize = listps;
gs.gamedate = Dateutils.gamedate;
String s = xs.toXML(gs);
out.println("clientplayertoserver");
out.println(s);
Here is the server side just in case:
if (strIn.contains("clientplayertoserver")) {
strIn = in.readLine();
XStream xs = new XStream();
GameSave gs = (GameSave) xs.fromXML(strIn);
Players.players.get(1).getPlayerSerializeData(gs.playersSerialize.get(0));
}
I need some kind of clue because I'm stuck investigating the problem. Are there any XStream limitations? Or the error is in the working with sockets? The same code is working in one place and is not working in another - I greatly thank in advance for any help with this weird situation.

Well, you are doing two different things here:
1) Saving the data to a file, which is ok.
2) Sending data via a socket. You seem to assume that all your data (the XStream serialized object) is actually in one line. This will usually not be the case. Even if you configure XStream to serialize all data without identation, you still cannot be sure you won't have linebreaks in the serialized data (your variables).
So solve your issue, you should separate your concerns here.
1st serialize / deserialize your objects to String and back (that seems to be working for you.
2nd send this data to a medium, like a file (which you already have) or to a server.
For sending string data to a server, you'll need some kind of protocol. Either you can reuse an existing protocol, like HTTP (POST request to a server), Web Service, Rest Call or whatever else your server is running.
If you want to implement your own protocol (as you have tried above), you must ensure that the server knows what to expect and how to treat it properly. Usually you should split your request in a header and a payload section or something like that.
Include in your header what you want to do (e.g save player state) and the meta information of that (e.g how many bytes payload you are sending).
After the header, send the payload.
The server must now read the header 1st (like everything until the first newline), parse the header to understand what is going on (e.g save player state, 543 bytes data) and act on it (read the data, transform it to a string, deserialize the XStream object and store it in a local database or whatever the server should do with that).
So and after all this information, please adapt your question. As you have seen you do not really have a question about XStream, but about how to send some data from client to a custom server.

How receive JSON object using serial event listener?

I'm sending a JSON object from an Arduino board to a java application on my PC via the USB serial connection. The java application receives "chunks" from the serial port in a serial event listener.
This is the body of the listener:
public synchronized void serialEvent(SerialPortEvent oEvent) {
if (oEvent.getEventType() == SerialPortEvent.DATA_AVAILABLE) {
try {
int available = input.available();
byte chunk[] = new byte[available];
input.read(chunk, 0, available);
// buffering of bytes must happen here to
// ultimately de-serialize into JSON object
} catch (Exception e) {
System.err.println(e.toString());
}
}
}
The problem I have is that it's not clear when the end of the JSON object has been received. From what I've researched, there needs to be some sort of event that causes the attempt to read from serial port to return a -1. The problem is that with this model, I only ever enter the listener when there's data available to be read.
I added some code to check the return value of the call to input.read and it seems to return the same value as the number of bytes available and I've also tested for other events and none other appear to happen.
Unless there's an easy way to modify this to detect the end of a serial transmission, I'm thinking this model of serial input isn't going to work for receiving something like a JSON object.
Should I move to a design that creates a separate thread with a blocking "readline" call to the serial port?
I'm stumped.

Since whitespace is allowed in JSON (see json.org), you can put a newline character after the very last character of the JSON string. Make sure you don't use a newline character anywhere else in the JSON string. Once you encounter the newline character, that's your signal that you received the entire string.
I've done this before with serial streaming JSON. In my situation, there's a device continuously streaming a JSON string. Data values vary within the JSON string (it's environmental data like temperature and humidity). So I find the start of the string by looking for an identifier (in my case, it's {"environment data": and I find the end by looking for a newline character (which is used nowhere else in the string).

Client-side string encoding java

My team and I have this nasty problem with parsing a string received from our server. The server is pretty simple socket stuff done in qt here is the sendData function:
void sendData(QTcpSocket *client,QString response){
QString text = response.toUtf8();
QByteArray block;
QDataStream out(&block, QIODevice::WriteOnly);
out << (quint32)0;
out << text;
out.device()->seek(0);
out << (quint32)(block.size() - sizeof(quint32));
try{
client->write(block);
}
catch(...){...
The client is in Java and is also pretty standard socket stuff, here is where we are at now after trying many many different ways of decoding the response from the server:
Socket s;
try {
s = new Socket(URL, 1987);
PrintWriter output = new PrintWriter(s.getOutputStream(), true);
InputStreamReader inp = new InputStreamReader(s.getInputStream(), Charset.forName("UTF-8"));
BufferedReader rd = new BufferedReader( inp );
String st;
while ((st = rd.readLine()) != null){
System.out.println(st);
}...
If a connection is made with the server it sends a string "Send Handshake" with the size of the string in bytes sent before it as seen in the first block of code. This notifies the client that it should send authentication to the server. As of now the string we get from the server looks like this:
������ ��������S��e��n��d�� ��H��a��n��d��s��h��a��k��e
We have used tools such as string encode/decode tool to try and assess how the string is encoded but it fails on every configuration.
We are out of ideas as to what encoding this is, if any, or how to fix it.
Any help would be much appreciated.

At a glance, the line where you convert the QString parameter to a Utf8 QByteArray and then back to a QString seems odd:
QString text = response.toUtf8();
When the QByteArray returned by toUtf8() is assigned to text, I think it is assumed that the QByteArray contains an Ascii (char*) buffer.

I'm pretty sure that QDataStream is intended to be used only within Qt. It provides a platform-independent way of serializing data that is then intended to be deserialized with another QDataStream somewhere else. As you noticed, it's including a lot of extra stuff besides your raw data, and that extra stuff is subject to change at the next Qt version. (This is why the documentation suggests including in your stream the version of QDataStream being used ... so it can use the correct deserialization logic.)
In other words, the extra stuff you are seeing is probably meta-data included by Qt and it is not guaranteed to be the same with the next Qt version. From the docs:
QDataStream's binary format has evolved since Qt 1.0, and is likely to
continue evolving to reflect changes done in Qt. When inputting or
outputting complex types, it's very important to make sure that the
same version of the stream (version()) is used for reading and
writing.
If you are going to another language, this isn't practical to use. If it is just text you are passing, use a well-known transport mechanism (JSON, XML, ASCII text, UTF-8, etc.) and bypass the QDataStream altogether.