I have a client-server topology in which the client asks for a listing of the directories or files in the current working directory on the server, and the server replies with the appropriate information.
See client code
controlSocket.writeByte(LSDIR);
int dirCount = controlSocket.readInt();
Map<String, Long> dirMap = new HashMap<>();
for (int i = 0; i < dirCount; i++) {
dirMap.put(controlSocket.readString(), controlSocket.readLong());
}
and server code
dir = new File(cwd);
output.writeInt(dir.listFiles(File::isDirectory).length);
for (File file : dir.listFiles(File::isDirectory)){
output.writeUTF(file.getName());
output.writeLong(file.lastModified());
}
Now when I don't change the directory on the server the directory listing works just fine, no matter how many times I call it. However, if I cd using this client code
controlSocket.writeByte(CD);
controlSocket.writeString(path);
and this server code
String inputDir = input.readUTF();
if (inputDir.equals("..")) {
cwd = cwd.substring(0, cwd.lastIndexOf("/"));
} else {
cwd += "/" + inputDir;
}
the directory listing runs but the integer that is read from the socket is not what the server sends (ex. on the server I see 1 is sent, but on the client something like 16777216 is read). The server reads the directory content with no problem, so there is no issue on this side.
It seems like the Data I/O Stream is not consistent, or else I'm missing out on something. Note that both the client and the server run on the same machine.
The problem was that the server wrote an additional confirmation boolean after changing the directory, which was read by the client as the integer.
So the code up here is working without any problem.
Related
I'm trying to read data from the server with SSH protocol. For this, I'm using the j2ssh library. My server connects with the other server in ssh without any problem. The problem is when I try to read any data from the shell command line. Whatever "command" I send to program "read = in.read(buffer)" never get any data, I tried with "ls" with "cat filename.txt" and other commands.
Only one command works fine and is "tail -f filename.txt". With this command, I can see the buffer is not empty, this contain the text of file, but the tail command does not close and while listening, sends the program in loop.
Can Anyone help me to know why I can't get any data from othere command?
This is my code:
private String exec(String cmd) throws SSHHandlerException {
String result = null;
session = ssh.openSessionChannel();
if(session.startShell())
{
session.getOutputStream().write((cmd+"\n").getBytes());
session.getOutputStream().close();
result = read(session,log);
}
session.close();
ssh.disconnect();
return result;
}
private static String read(SessionChannelClient session, ProcessLogger log) throws Exception{
byte buffer[] = new byte[255];
int read;
StringBuffer out=new StringBuffer();
InputStream in = session.getInputStream();
while((read = in.read(buffer)) > 0) {
out.append(new String(buffer, 0, read));
}
return out.toString();
}
If your goal is to transfer files, you should be using an SFTP client instead. SFTP is exactly what you're looking for: a file transfer protocol on top of SSH. It's much, much more efficient than using some command on the host and redirecting the stream.
J2SSH has an SftpClient implementation that can be constructed with an SshClient. Just use one of the get methods. Javadocs are here.
Edit after learning that you're not trying to transfer files:
You need to request a pseudo-terminal before you start the shell. From the docs:
The remote process may require a pseudo terminal. Call this method before executing a command or starting a shell.
Also, because it appears that you're using a Linux environment, I would recommend using terminal type "xterm" rather than their example of "vt100".
The reason that tail was working and not the other commands was because you were calling tail interactively. The interactive command creates its own pseudo-terminal of sorts. If, instead, you call tail -n 16 filename.txt then you will get the same results as with the other commands because it won't be interactive.
I'm using apache commons-net FTPClient (ver 3.3) and below error was produced on one of my clients machine (I've tried reproducing the error on my dev machine without luck using testing folder with the same requests on the same server with the same login)
I have a process that check's the remote FTP server for new requests in form of an XML-files. After listing all of those files i proceed in loop to check if they're in XML format. If the file is in this format I do first change their name by changing format from *.xml to *.xmlProcessing, retrive them to a input stream, parse them to my object and create a request in my queue and finally change the name and move them to subfolder working as an archive.
After downloading random amount of files I get stuck while calling retrieveFileStream on the next file, without a timeout or IO Exception.
I've managed to get logs from FTP server and it just says it can't open a data connection
05.06.2019 12:45:45 - > RNFR /folder/file.xml
05.06.2019 12:45:45 - > 350 File exists, ready for destination name.
05.06.2019 12:45:45 - > RNTO /folder/file.xmlProcessing
05.06.2019 12:45:45 - > 250 file renamed successfully
05.06.2019 12:45:45 - > PORT *ports*
05.06.2019 12:45:45 - > 200 Port command successful
05.06.2019 12:45:45 - > RETR /folder/file.xmlProcessing
05.06.2019 12:45:45 - > 150 Opening data channel for file download from server of "/folder/file.xmlProcessing"
05.06.2019 12:45:45 - > 425 Can't open data connection for transfer of "/folder/file.xmlProcessing"
I've already tried diffrent FTP modes. Active local, remote, passive etc. (currently stuck with passive local mode).
I've tried the data timeout but it looks like while i finally got stuck on one of the files the method took more than 1 minute on that file despite me setting the timeout on 30s.
ftp = new FTPClient();
ftp.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out)));
ftp.connect(server, port);
int reply = ftp.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply)) {
ftp.disconnect();
throw new IOException("Exception in connecting to FTP Server");
}
ftp.login(user, password);
ftp.enterLocalPassiveMode();
ftp.setKeepAlive(true);
ftp.setDataTimeout(30000);
Collection<String> listOfFiles = listFiles(FOLDER_PATH);
for(String filePath : listOfFiles){
if (filePath != null && filePath.endsWith(".xml")) {
ftp.rnfr(folder + filePath);
ftp.rnto(folder + filePath + "Processing");
InputStream fileInputStream = ftp.retrieveFileStream(folder + filePath + "Processing");
ftp.completePendingCommand();
//Parsing file to an instance of my object and creating request
ftp.rnfr(folder + filePath + "Processing");
ftp.rnto(archiveFolder + filePath);
if(fileInputStream != null){
fileInputStream.close();
}
}
}
Is there's a bigger likehood that this is fault of the FTP Server, Firewall issues or something else ?
I've runed the same code from my dev machine and it processed all files from test folder (there were around 400 of them) i don't know if im being unlucky for error not occuring on my local dev machine or is it actually something wrong with communication of my contractor with the remote server ?
Everytime you do a file transfer or a directory listing with FTP, the server (or client if using an active mode) assigns a random port number out of a configured range to that transfer. The port number is not released immediately, when the transfer completes. There's some cooldown interval. If you do too many file transfers in a short time interval, it can happen that the server runs out of the available ports – Because all ports end up in the cooldown state.
If you can, check the server configuration and configure a larger range of ports.
Or as a workaround, you can try to slow down the transfer rate.
For some background, see:
How many data channel ports do I need for an FTP?
Why does FTP passive mode require a port range as opposed to only one port?
Though this is just a guess, you should check the server's log, as it can show more details.
Another possibility is, that there's simply a limited number of transfers the server allows for a specific user or source address in some time interval.
i need to send some messages from my java web application to some servers using Diameter protocol, in particular CCR-CCA scenario. I had a look at jdiameter opensource project, but my usecase does not require such complexity, since that i just need to send a single request and log the response (actually i don't even need the CER-CEA part).
So i thought i could just have used Seagull running under my webapp. I downloaded Seagull (for Windows), and what i'm trying to do is basically to run the .bat file coming from Seagull for the diameter environment from my java environment.
That's what i've done till now..
1) A simple test to invoke the client.. Here wrapper simply sets working dir and starts the process
public static void main(String[] args) {
List<String> cmd=new ArrayList<>();
cmd.add("cmd.exe");
cmd.add("/c");
cmd.add("my_start_client.bat");
JavaProcessBuilderWrapper wrapper = new JavaProcessBuilderWrapper();
Process p = wrapper.createProcess(RedirectErrorsTo.STDERR,
new HashMap<>(), new File("my_working_dir"), cmd);
BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
StringBuilder output = new StringBuilder();
String line;
try {
while ((line = reader.readLine()) != null) {
output.append(line);
}
System.out.println(line);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
2) I modified the client's and server's .bat files coming from Seagull to use CCR-CCA protocol.
Running Java main with this configuration caused a
Fatal: Keyboard saved configuration failure error
on my logs.
3) So, as mentioned here i further modified my client's .bat file to run in background mode, adding -bg at the end. Now my client's bat look like this
#ECHO OFF
rem
"Diameter Start Script Sample"
"Local env"
SET RUN_DIR=C:\Program Files\Seagull
set PATH=%PATH%;%RUN_DIR%
set LD_LIBRARY_PATH=%RUN_DIR%
set RUN_DIR=%RUN_DIR%\diameter-env\run
cd %RUN_DIR%
cls
mode 81,25
echo "Seagull Diameter Client Sample Start"
seagull -conf ..\config\conf.client.xml -dico ..\config\base_ro_3gpp.xml -scen ..\scenario\ccr-cca.ro.client.xml -log ..\logs\ccr-cca.client.log -llevel ETM -bg
pause
Since i was facing some troubles, to keep things simple, i just tried to make it work at least via cmd (not using my java method), but i think background mode is messing around, because now when i start my server and then my client in bg mode, sometimes i get a
Fatal: Forking error
but the most of the times, the client send a single message and then on my console i see that my software is causing connection abort (error code -1), and from the log i see that the channel just get closed, and my client does not even receive an answer. (NB for now i left the configuration files untouched)
Has any of you faced this behaviour? Is something else closing the connection (firewall perhaps)? Do i have to provide other configurations to make this work?
Once i can get this working, can i use my java web app (with a method similar to the one i already mentioned) to make diameter calls?
Thanks in advance, any help is really welcomed.
I am hoping someone can help me (once again).
I have a very large number of smmll files (over 4000) each only a few K.
I have writen an FTP program in java which will transfer each file individually but it is taking a very long time. Also the handshaking overhead seems to make the problem worse.
What I would like to be able to do is open the FTP connection send all the files then close it again.
I know that this is possible in FTP but quite how to acheive this in java is beyond me.
I currently have the filenames in an array so parsing through them is no problem. I have tried calling the following class and passing it the filename but after several hours it was still moving about 1 file per second.
package website;
import java.io.BufferedOutputStream;
import java.io.FileInputStream;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
public class ftpUpload {
public ftpUpload(String target, String savename, String localFilePath) {
URL url;
try {
url = new URL(target + savename + ";type=i");
URLConnection con = url.openConnection();
BufferedOutputStream out =
new BufferedOutputStream(con.getOutputStream());
FileInputStream in =
new FileInputStream(localFilePath + savename);
int i = 0;
byte[] bytesIn = new byte[1024];
while ((i = in.read(bytesIn)) >= 0) {
out.write(bytesIn, 0, i);
}
out.close();
in.close();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Is there a way I can open the connection with the ftp site username and password,
then send it the files
and finally close the connection?
This would seem to me easier than creating multiple threads to send files concurrently.
Any advice greatfully received.
Paul
I don't think it's possible to send multiple files in one session using URLConnection, this means you get the overhead of opening and closing the session for every file.
FTPClient from commons net does support multiple operations in one session. For example (exception handling ommitted):
FTPClient ftp = new FTPClient();
ftp.connect("ftp.example.com");
ftp.login("admin", "secret");
ftp.setFileType(FTPClient.BINARY_FILE_TYPE);
for(File file : files) {
InputStream in = new FileInputStream(file);
ftp.storeFile(file.getName(), in);
in.close();
}
ftp.disconnect();
This should help.
If you still need better performance, I don't see any other alternative than using multiple threads.
After plenty of testing I have found reliability issues with multiple ftp threads to public servers which is what I need in this case. Most (if not all) ftp servers limit the maximum number of connections and also limit the maximum number of concurrent connections from the same IP address. Two concurrent connections from the same IP seems to be the only guaranteed maximum you are allowed. The realistic option as suggested above is to zip the files and ftp a single file. You can unzip the file when it gets there using an php script (as long as the server supports unzip you will need to check if this included in the php build). Finally if like me you need to upload in excess of 10,000 files many ftp servers will not show more than 9998 files (10,000 inlcuding . and ..)
If anyone knows of an ftp server free or cheap that supports ZipArchive in the php build and will list more than 9998 files when requesting a file listing in ftp can you please let me know.....
I want to connect to a number of different sockets/webservices on the command line and send data back and forth in the standard output/input.
I have been doing this using a variety of different languages and approaches so far: System.Net.Sockets in C#, flash.net.sockets in Flash and java.net.sockets in Java, depending on the protocol being used by the socket and the language being used in the client example given by the companies who've written the sockets. I've had enough of moving from language to language to do this (using the provided client socket example in each case), and will probably all the clients to java.
In the meantime, I want a way to connect to a socket on the command line in windows, see what's return in the standard output, send text to the socket on the command line (or a very, very simple GUI) and see what's returned back. I don't need any extra functionality like a periodic ping to keep the socket alive or anything.
What tools can I do this on Windows with? I've tried opening a telnet session to the socket i.e. push.domain.com 1234, and also tried using Putty to connect also, to no avail.
I'm trying to emulate the way a flash client connects to this socket and sends and receives data:
theSocket.addEventListener(Event.CONNECT, connectHandler);
theSocket.connect(theHost, thePort);
* * *
private function connectHandler(event:Event) : void
{
if (theSocket.connected)
{
bytes = new ByteArray();
bytes.writeByte(35);
bytes.writeByte(1);
bytes.writeByte(23);
bytes.writeByte(7);
bytes.writeUTFBytes(theTopic);
bytes.writeByte(0);
theSocket.writeBytes(bytes);
theSocket.flush();
theSocket.addEventListener(ProgressEvent.SOCKET_DATA, handshakeHandler);
* * *
private function handshakeHandler(event:ProgressEvent) : void
{
var zero:int = 0;
theSocket.removeEventListener(ProgressEvent.SOCKET_DATA, handshakeHandler);
theConnectionTimer.stop();
var bytes:* = new ByteArray();
var counter:int = 0;
theSocket.readUTFBytes(theSocket.bytesAvailable));
var a:* = theSocket.readByte();
var b:* = theSocket.readByte(); // the second byte should be 1????
var response:* = theSocket.readByte(); // this is the reponse identifier. . . ???
theMessageSize = theSocket.readByte(); // is this byte the size??????
switch(response)
{
case 100:
{
while ((zero = theSocket.readByte()) != 0)
{
var temp = counter++;
bytes[temp] = _loc_5;
};
theClientID = bytes.toString();
trace("The client ID is: " + theClientID);
How can I send the bytes values of 35, 1, 23, 7 and 0, as well as the value of the variable, Topic, to the socket using Hercules (or any other tool). Ideally, I'd like to connect with Hercules, send those bytes and the topic, and get something back containing the clientID like in the code. Although, I don't know if hercules will render the bytes in the response into text for me.
I'd appreciate any pointers at all on this.
Thanks.
I was thinking in Hercules and searching for the website I found out that there's already an answer here in stackoverflow.
I think it does what you need and more.
Uhm, I'm nost sure I understood completely what you are asking, but I don't see why telnet could not help you in this case.