We Keep Coding

FTPClient throws Connection reset by peer during storeFile

I am trying to upload file to ftp server using Java.
What i got so far is
fun uploadData(): Boolean {
val ftpClient = FTPSClient()
ftpClient.addProtocolCommandListener(
PrintCommandListener(PrintWriter(OutputStreamWriter(System.out, "UTF-8")), true)
)
ftpClient.connect(ftpProperties.server)
try {
ftpClient.enterLocalPassiveMode()
ftpClient.login(ftpProperties.username, ftpProperties.password).takeIf { !it }?.let {
log.error("cannot login to ftp")
throw Exception("cannot login to ftp")
}
ftpClient.enterLocalPassiveMode()
ftpClient.soTimeout = 10000
ftpClient.dataTimeout = Duration.ofSeconds(10)
ftpClient.execPBSZ(0)
ftpClient.execPROT("P")
ftpClient.changeWorkingDirectory("/")
ftpClient.setFileType(FTP.BINARY_FILE_TYPE)
ftpClient.enterLocalPassiveMode()
} catch (ie: IOException) {
log.error("ftp initialization error", ie)
throw Exception("ftp initialization error")
}
val remoteFile = "top.txt.gz"
val data = FileInputStream(File("top.txt.gz"))
var done = false
try {
done = ftpClient.storeFile(remoteFile, data)
} catch (e: Exception) {
log.error(e) { "error" }
}
data.close()
if (done) {
return true
}
log.error { "${ftpClient.replyCode} ${ftpClient.replyString}" }
throw RuntimeException("File not stored $remoteFile")
}
Code does not seem to be working properly. After starting it i get error code on line with code:
done = ftpClient.storeFile(remoteFile, data)
org.apache.commons.net.io.CopyStreamException: IOException caught while copying.
Caused by: java.net.SocketException: Connection reset by peer
I could not find anything wrong in ftp log
220 Private FTP server
AUTH TLS
234 Proceed with negotiation.
USER *******
331 Please specify the password.
PASS *******
230 Login successful.
PBSZ 0
200 PBSZ set to 0.
PROT P
200 PROT now Private.
CWD /
250 Directory successfully changed.
TYPE I
200 Switching to Binary mode.
PASV
227 Entering Passive Mode (x,x,x,x,x,x).
STOR top.txt.gz
150 Ok to send data.
I am able to connect and upload files to that ftp server from the same machine that i start Java/Kotlin code, using filezilla.
Filezilla log looks like that:
Status: Connection established, waiting for welcome message...
Response: 220 Private FTP server
Command: AUTH TLS
Response: 234 Proceed with negotiation.
Status: Initializing TLS...
Status: Verifying certificate...
Status: TLS connection established.
Command: USER k8s_search
Response: 331 Please specify the password.
Command: PASS ******************
Response: 230 Login successful.
Status: Server does not support non-ASCII characters.
Command: PBSZ 0
Response: 200 PBSZ set to 0.
Command: PROT P
Response: 200 PROT now Private.
Status: Logged in
Status: Starting upload of /home/x/x/top.txt.gz
Command: CWD /
Response: 250 Directory successfully changed.
Command: TYPE I
Response: 200 Switching to Binary mode.
Command: PASV
Response: 227 Entering Passive Mode (x,x,x,x,x,x).
Command: STOR top.txt.gz
Response: 150 Ok to send data.
Response: 226 Transfer complete.
Status: File transfer successful, transferred 2.6 MB in 1 second
I connot figure out if problem is in my code or is it anything else. Filezilla working correctly indicates that problem lies in code.

Sending OID in SNMP Trap header

I have a java application which sends SNMP traps using SNMP4J. The problem is that OID is sent in trap body. All data I'm setting is successfully sent, but in trap body. I want Oid to be sent in trap header.
How can I send Oid in Trap header?
UdpAddress managerUdpAddress = new UdpAddress("address");
CommunityTarget ctarget = new CommunityTarget();
ctarget.setAddress(managerUdpAddress);
ctarget.setRetries(retryCount);
ctarget.setCommunity(new OctetString(community));
ctarget.setTimeout(timeout);
ctarget.setVersion(SnmpConstants.version2c);
PDU trap = new PDU();
OID oid = new OID(myOid);
trap.add(new VariableBinding(SnmpConstants.snmpTrapOID, oid));
trap.add(new VariableBinding(SnmpConstants.sysUpTime, new TimeTicks(5000)));
trap.add(new VariableBinding(SnmpConstants.sysDescr, new OctetString(
"System Description")));
trap.add(new VariableBinding(oid, new OctetString(message)));
DefaultUdpTransportMapping transport = new DefaultUdpTransportMapping();
Snmp snmp = new Snmp(transport);
snmp.notify(trap, ctarget);
When UPS is sending SNMP trap, OID is presented in SNMP trap header. Here are examples:
Trap from UPS:
Mon Mar 18 04:13:18 2019 .1.3.6.1.4.1.935.0.49 Normal "SNMP EVENT"
x.x.x.x - UPS_212_bypass_ac_normal SNMP TRAP: Bypass AC Normal
Trap from JAVA:
Mon Mar 18 05:25:36 2019 .0.00 Critical "SNMP EVENT" x.x.x.x - my application snmp errors: System Description General error. Size=2"

The SNMP TRAP format has fixed structure defined in RFC 1157 or RFC 3412 (in case of SNMPv3). This structure consists of header and PDU (Packet Data Unit). The PDU is basically a set of so called variable bindings. Each binding has OID, Syntax and value. So you can only change the PDU part. The header structure cannot be changed.

I did it by adding this code:
trap.setType(PDU.TRAP);
trap.add(new VariableBinding(oid));
Now SNMP trap sent from Java looks like this:
Thu Mar 21 15:16:51 2019 .1.3.6.1.6.3.1.1.7.1.6 Critical "SNMP EVENT"
x.x.x.x - my application snmp errors: System Description General
error. Size=2"

Signature length not correct: got 127 but was expecting 128

I am facing a strange problem after java 1.8 upgrade. I am using jsch-0.1.54.jar in one of our utility programs to download files from various places. This particular utility was being used for almost 4-5 years without any problem(back then it jsch-0.1.48). At that time environment was java 1.6. Recently we upgraded to java 1.8 and as a result we upgraded this particular utility. Now we are encountering a strange problem and it occurs occasionally, and most of the time the download of files are prefect.
Error log
INFO: SSH_MSG_KEXDH_INIT sent
INFO: expecting SSH_MSG_KEXDH_REPLY
INFO: Disconnecting from SRV2000 port 22
2016-10-28 08:42:18:0576 ERROR [main] net.AerisAbstractMethod - Failed to open connection
com.jcraft.jsch.JSchException: Session.connect: java.security.SignatureException: Signature length not correct: got 127 but was expecting 128
at com.jcraft.jsch.Session.connect(Session.java:565)
at com.jcraft.jsch.Session.connect(Session.java:183)
at com.aeris.net.AerisSFTPMethod.connectToServer(AerisSFTPMethod.java:65)
at com.aeris.net.AerisAbstractMethod.getListOfFiles(AerisAbstractMethod.java:143)
at com.aeris.worker.AerisUploaderDownloader.performUploadDownloadListing(AerisUploaderDownloader.java:112)
at com.aeris.main.AerisCommonSftpUtility.main(AerisCommonSftpUtility.java:102)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.simontuffs.onejar.Boot.run(Boot.java:340)
at com.simontuffs.onejar.Boot.main(Boot.java:166)
Success Log: (in most of the cases it is success)
INFO: SSH_MSG_KEXDH_INIT sent
INFO: expecting SSH_MSG_KEXDH_REPLY
INFO: ssh_rsa_verify: signature true
WARN: Permanently added 'SRV2000' (RSA) to the list of known hosts.
INFO: SSH_MSG_NEWKEYS sent
INFO: SSH_MSG_NEWKEYS received
INFO: SSH_MSG_SERVICE_REQUEST sent
INFO: SSH_MSG_SERVICE_ACCEPT received
INFO: Authentications that can continue: publickey,password,keyboard-interactive
INFO: Next authentication method: publickey
INFO: Authentication succeeded (publickey).
2016-10-28 08:36:48:0794 INFO [main] net.AerisAbstractMethod - Session connected to server
2016-10-28 08:36:48:0794 INFO [main] net.AerisAbstractMethod - Opening SFTP channel..
2016-10-28 08:36:48:0810 INFO [main] net.AerisAbstractMethod - Connecting to server through channel.
2016-10-28 08:36:48:0857 INFO [main] net.AerisAbstractMethod - Connection successful.
2016-10-28 08:36:48:0857 INFO [main] net.AerisAbstractMethod - Changing to directory:C:/interfaces/ib/wf/work
2016-10-28 08:36:48:0888 INFO [main] net.AerisAbstractMethod - Start file Listing of the remote directory:C:/interfaces/ib/wf/work
0 Oct 28, 2016 04:15 ./
0 Oct 28, 2016 04:15 ../
I did a complete analysis with Vandyke (sftp software provider) but did not find any error at that end. I also tried to sftp using different tools but I am not getting any error. Following is the code snippet to connect of SFTP server. Can any one help in this matter?
protected void connectToServer() throws AerisConnectionException {
JSch jSch =(JSch)this.client;
try {
session = jSch.getSession(config.getUsername(), config.getRemoteserver(), config.getPort());
LOGGER.info("Creating SSH Session using Username:"+config.getUsername()+ " Server :" +config.getRemoteserver()+ " at PORT:"+config.getPort());
if(config.getAuth().equalsIgnoreCase("PASSWD")||config.getAuth().equalsIgnoreCase("KEYPASS")){
LOGGER.info("Setting password ...");
session.setPassword(config.getPassword());
}
Properties jShconfig = new Properties();
jShconfig.put("StrictHostKeyChecking", "no");
jShconfig.put("PreferredAuthentications",
"publickey,password,keyboard-interactive");
jShconfig.put("LogLevel", "VERBOSE");
LOGGER.info("Setting timeout to "+config.getTimeOut());
session.setTimeout(config.getTimeOut()*1000);
session.setConfig(jShconfig);
session.connect();
LOGGER.info("Session connected to server");
this.connected=true;
} catch (JSchException e) {
LOGGER.error("Failed to open connection ",e);
throw new AerisConnectionException("Failed to open connection.");
}
}

Although it would be nice to have the stacktrace to confirm, I'll bet the server is using an RSA 'host' key to authenticate and is wrongly 'trimming' leading zero in rare cases.
RSA signature values (and encrypted values also) defined by PKCS#1, which SSH uses (as do many other things including SSL), are required to be encoded as an octet string of fixed length 'k' equal to the length required to encode the modulus, or informally 'the same size as the modulus'. However, since the underlying mathematical value is a large nonnegative (aka unsigned) integer, specifically modexp(s,d,n), historically some implementations have omitted leading zero octets -- an omission which is valid when treating the value as an integer -- resulting in an encoded value that is sometimes shorter than it should be.
The RSA signature (or encrypted) value is effectively a uniform random number in (1,n). Thus when the RSA key used by the server has a 'round binary' size like 1024 as here, this trimming will happen approximately 1 in 200 times at random, or 400 if trimmed as a signed number.
I did not know, but on testing I confirm that (Oracle) Java 6 does accept such a 'short' value for Signature type RSA or as actually used here SHA1withRSA, both of which imply the PKCS1-v1_5 scheme, but Java 7 and 8 throw the exception you saw.
OTOH both OpenSSH and PuTTY (also used by WinSCP and FileZilla) do accept 'short' values, while always sending correct length values; this Postelian behavior may make it more difficult to detect when a peer is misbehaving this way. (Note: I checked OpenSSH 5.5 and 7.3, the oldest and newest I have conveniently at hand, but only the current PuTTY 0.67 because that's all I keep online.)
You can try pointing the server software implementor to the published standards, but it may not do any good. You could ask jcraft to special-case this; they already have logic in the DSA and ECDSA cases for mpint/ASN.1 that I might argue is equally ugly.
Or if the server has another (usable) key, request that by configuring "server_host_key" to NOT include ssh-rsa -- easiest just get the existing/default list, split, remove "ssh-rsa" (and check not empty) and rejoin, instead of possibly confusing your users and/or (co)maintainers by listing today's specific algorithms.

buffering issue with java communicating vlc

I'm a newbie to Socket communication, so I may be wrong, but please advice or at least give the direction!
I'm implementing an RTSP server according to http://www.csee.umbc.edu/~pmundur/courses/CMSC691C/lab5-kurose-ross.html#appendix taking a look to the similar code from http://www.java2s.com/Open-Source/Android/UnTagged/mynpr/com/webeclubbin/mynpr/RTSPserver.java.htm
At the moment I'm implementing responce to the OPTIONS request. To make it easy in the first approach, I decided to hardcode the answer according to the sample RTSP request/response log done for some real communication between vlc and gstreamer rtsp.
So, the log recorded with vlc URL -vvv says:
Sending request: OPTIONS rtsp://localhost:8554/test RTSP/1.0
CSeq: 2
User-Agent: LibVLC/2.0.8 (LIVE555 Streaming Media v2013.04.30)
Received 183 new bytes of response data.
Received a complete OPTIONS response:
RTSP/1.0 200 OK
CSeq: 2
Public: OPTIONS, DESCRIBE, GET_PARAMETER, PAUSE, PLAY, SETUP, SET_PARAMETER, TEARDOWN
Server: GStreamer RTSP server
Date: Tue, 10 Sep 2013 19:56:53 GMT
Sending request: DESCRIBE rtsp://localhost:8554/test RTSP/1.0
CSeq: 3
User-Agent: LibVLC/2.0.8 (LIVE555 Streaming Media v2013.04.30)
Accept: application/sdp
i.e.
RTSP/1.0 200 OK
CSeq: 2
Public: OPTIONS, DESCRIBE, GET_PARAMETER, PAUSE, PLAY, SETUP, SET_PARAMETER, TEARDOWN
Server: GStreamer RTSP server
Date: Tue, 10 Sep 2013 19:56:53 GMT
part is 183 bytes length
I'm writing to the buffer right according to the example:
try{
System.out.println("S -> C");
System.out.println("RTSP/1.0 200 OK");
System.out.println("CSeq: "+RTSPSeqNb);
//System.out.println("Session: "+RTSP_ID);
if (responceType==OPTIONS) {System.out.println("Public: OPTIONS, DESCRIBE, GET_PARAMETER, PAUSE, PLAY, SETUP, SET_PARAMETER, TEARDOWN");};
if (responceType==OPTIONS) {System.out.println("Server: GStreamer RTSP server"); };
if (responceType==OPTIONS) {System.out.println("Date: Tue, 10 Sep 2013 19:56:53 GMT");};
RTSPBufferedWriter.write("RTSP/1.0 200 OK"+CRLF);
RTSPBufferedWriter.write("CSeq: "+RTSPSeqNb+CRLF);
//RTSPBufferedWriter.write("Session: "+RTSP_ID+CRLF);
if (responceType==OPTIONS) {RTSPBufferedWriter.write("Public: OPTIONS, DESCRIBE, GET_PARAMETER, PAUSE, PLAY, SETUP, SET_PARAMETER, TEARDOWN"+CRLF);};
if (responceType==OPTIONS) {RTSPBufferedWriter.write("Server: GStreamer RTSP server"+CRLF); };
if (responceType==OPTIONS) {RTSPBufferedWriter.write("Date: Tue, 10 Sep 2013 19:56:53 GMT"+CRLF); };
RTSPBufferedWriter.write("Session: "+RTSP_ID+"\r"+CRLF);
RTSPBufferedWriter.flush();
//RTSPBufferedWriter.newLine();
System.out.println("RTSP Server - Sent response to Client.");
}
catch(IOException ex)
{
System.out.println("Exception caught: "+ex.getStackTrace());
// System.exit(0);
}
and the vlc log says
Opening connection to 127.0.0.1, port 6666...
...remote connection opened
Sending request: OPTIONS rtsp://127.0.0.1:6666/autostream.mjpg RTSP/1.0
CSeq: 2
User-Agent: LibVLC/2.0.8 (LIVE555 Streaming Media v2013.04.30)
Received 193 new bytes of response data.
[0x7fd01c001178] live555 demux debug: connection timeout
[0x7fd01c001178] live555 demux error: Failed to connect with rtsp://127.0.0.1:6666/autostream.mjpg
Where CRLF is '\n'. Before I tried CRLF="\r\n" (and no +"\r"+ in the last line) with
Received 198 new bytes of response data.
So, what is wrong there? What vlc is waiting for? Why default delimeters from the example doesn't work for it?

I looks like I always find the answer to my questions after I publish them to stackoverflow...
DOUBLE CRLF should stand after the last header for the usual RTSP protocol 9not the customized one used in the example).

Dealing with "[HOST_KEY_NOT_VERIFIABLE] Could not verify `ssh-rsa` host key with fingerprint" in sshj

I having a strange issue with sshj (am using sshj v0.6.0) for which I would need some help from someone.
Authentication with public key works fine on some machines but doesnt work fine on other machines and I see the below error.
The only difference that I could make out was that the UNIX ID in question viz coonradt seems to have the below listed configuration setup under ~/.ssh/config only on the box on which the below errors are being triggered
Host *
Protocol 1,2
FallBackToRsh no
ForwardAgent yes
ForwardX11 yes
PasswordAuthentication yes
RhostsAuthentication no
RhostsRSAAuthentication no
RSAAuthentication yes
NoHostAuthenticationForLocalhost yes
StrictHostKeyChecking no
KeepAlive yes
From the above config file I learnt that the ID in question is supposed to make use of Protocol 1,2 and I suspect that this might have something to do with my failures (I am not very sure about it, but this is just a hunch)
For all other UNIX IDs for which this works fine, I dont have any such config file.
PS : I cannot alter the config of the UNIX ID "coonradt" since this ID is being used by the central hudson servers.
Would appreciate if someone could please help me suggest as to what might be wrong here
Following is the error that I am seeing :
Oct 24, 2011 2:30:37 AM net.schmizz.sshj.DefaultConfig initCipherFactories
WARNING: Disabling high-strength ciphers: cipher strengths apparently limited by JCE policy
Oct 24, 2011 2:30:38 AM net.schmizz.sshj.transport.TransportImpl init
INFO: Client identity string: SSH-2.0-SSHJ_0_6_0
Oct 24, 2011 2:30:38 AM net.schmizz.sshj.transport.TransportImpl init
INFO: Server identity string: SSH-1.99-OpenSSH_4.3
Oct 24, 2011 2:30:38 AM net.schmizz.sshj.transport.KeyExchanger sendKexInit
INFO: Sending SSH_MSG_KEXINIT
Oct 24, 2011 2:30:38 AM net.schmizz.sshj.transport.KeyExchanger handle
INFO: Received SSH_MSG_KEXINIT
Oct 24, 2011 2:30:38 AM net.schmizz.sshj.transport.kex.AbstractDHG init
INFO: Sending SSH_MSG_KEXDH_INIT
Oct 24, 2011 2:30:38 AM net.schmizz.sshj.transport.KeyExchanger handle
INFO: Received kex followup data
Oct 24, 2011 2:30:38 AM net.schmizz.sshj.transport.kex.AbstractDHG next
INFO: Received SSH_MSG_KEXDH_REPLY
Oct 24, 2011 2:30:38 AM net.schmizz.sshj.transport.TransportImpl die
SEVERE: Dying because - net.schmizz.sshj.transport.TransportException: [HOST_KEY_NOT_VERIFIABLE] Could not verify `ssh-rsa` host key with fingerprint `ca:0b:b3:7f:53:5a:e3:bc:bf:44:63:d8:2d:26:c0:41` for `mymachine.domain.com` on port 22
Oct 24, 2011 2:30:38 AM net.schmizz.concurrent.Promise tryRetrieve
SEVERE: <<kex done>> woke to: net.schmizz.sshj.transport.TransportException: [HOST_KEY_NOT_VERIFIABLE] Could not verify `ssh-rsa` host key with fingerprint `ca:0b:b3:7f:53:5a:e3:bc:bf:44:63:d8:2d:26:c0:41` for `mymachine.domain.com` on port 22
Oct 24, 2011 2:30:38 AM net.schmizz.sshj.transport.TransportImpl setService
INFO: Setting active service to null-service
Oct 24, 2011 2:30:38 AM com.test.jaws.execution.ssh.impl.SSHJClientImpl$ExceptionHandler handleSevereCondition
SEVERE: mymachine.domain.com is not added to your /x/home/coonradt/.ssh/known_hosts file.
Throwable occurred: net.schmizz.sshj.transport.TransportException: [HOST_KEY_NOT_VERIFIABLE] Could not verify `ssh-rsa` host key with fingerprint `ca:0b:b3:7f:53:5a:e3:bc:bf:44:63:d8:2d:26:c0:41` for `mymachine.domain.com` on port 22
at net.schmizz.sshj.transport.KeyExchanger.verifyHost(KeyExchanger.java:222)
at net.schmizz.sshj.transport.KeyExchanger.handle(KeyExchanger.java:373)
at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:477)
at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
at net.schmizz.sshj.transport.Reader.run(Reader.java:72)

You may set the SSH client to accept all keys without any verification (ignores host key verification)
SSHClient sshClient = new SSHClient();
sshClient.addHostKeyVerifier(new PromiscuousVerifier());
...

How about adding a HostKeyVerifier for this machine?
sshClient.addHostKeyVerifier("ca:0b:b3:7f:53:5a:e3:bc:bf:44:63:d8:2d:26:c0:41");
The reason it doesn't happen automatically is probably because the known_hosts file isn't at $(user.home)/.ssh/known_hosts. You can also explicitly load known hosts from a specific location.
sshClient.loadKnownHosts(new File("path_to_known_hosts"));

try {
ssh.connect(envConf.getIp(), port);
} catch (TransportException e) {
if (e.getDisconnectReason() == DisconnectReason.HOST_KEY_NOT_VERIFIABLE) {
String msg = e.getMessage();
String[] split = msg.split("`");
String vc = split[3];
ssh = new SSHClient();
ssh.addHostKeyVerifier(vc);
ssh.connect(envConf.getIp(), port);
} else {
throw e;
}
}
ssh.authPassword(envConf.getName(), envConf.getPw());
ssh.newSCPFileTransfer().download(envConf.getHomePath() + FilePath, toPath);

For an alternative answer ensure that the hostname you are trying to connect to is exactly a match in your known_hosts file. An example mistake that I was making was trying to connect to the full URL bob.insidenetwork.pvt but my known_hosts file had only bob as an entry because when I ssh manually I'm far too lazy to type the entire URL...

If the server accepts keyboard-interactive verification, you can also do this, at least with recent versions of SSHJ:
client.addHostKeyVerifier(new ConsoleKnownHostsVerifier(new File(userKnownHostsFile), System.console()) {
#Override
protected boolean hostKeyUnverifiableAction(String hostname, PublicKey key) {
try {
entries().add(new HostEntry(null, hostname, KeyType.fromKey(key), key));
write();
} catch (IOException e) {
throw new RuntimeException(e);
}
return true;
}
});
This overrides the "Are you sure you want to continue connecting (yes/no)? " prompt that the user is supposed to reply to, so that you don't need to provide any input into a console to continue, while satisfying the keyboard-interactive authentication when a host is unknown and avoiding using a PromiscuousVerifier.
You can, of course, tweak the overriding method to your heart's content to add logging, and if you want to be prompted then just use a plain ConsoleKnownHostsVerifier.
If the server doesn't accept keyboard-interactive authentication and you still don't want to use the promiscuous verifier, then you could do something similar to the above for the OpenSSHKnownHosts class; e.g.:
HostKeyVerifier hkv = new OpenSSHKnownHosts(new File("~/.ssh/known_hosts") {
#Override
protected boolean hostKeyUnverifiableAction(String hostname, PublicKey key) {
return true;
}
};
client.addHostKeyVerifier(hkv);
Inside OpenSSHKnownHosts, all the hostKeyUnverifiableAction(...) method does is return false, so change that and you're good.