i need to connect to ssh server and pull out data of a huge file.
iam able to do it with cat command , and it takes nearly 10 -15 mins time.
The challange here is, i cant user sftp , exe
i.e Channel channel = session.openChannel("sftp"); & Channel channel = session.openChannel("exec"); not allowed in my prd environment.
The only option left behind is "shell".
Can anyone tell me how to do pg , until a specific pattern is found in the file.
Please help me with a code example !!
Hope you got my query . Thanks in advance.
How about following sample program?
https://gist.github.com/4482587
It will remotely exec "gzip -c", and gunzip the given data stream.
Related
I am trying to run a JMX file from non GUI mode on linux server. Using SFTP protocol trying to upload file to object store. Below is the script and configuration used to run the script.
Test.jmx
import com.jcraft.jsch.*;
import java.io.*;
def jsch = new JSch()
def session = jsch.getSession("user1", "xyz", 4000)
session.setConfig("StrictHostKeyChecking", "no")
session.setPassword("password")
def sftpSession = session.connect()
def channel = session.openChannel("sftp")
channel.connect();
def channelSftp = (ChannelSftp)channel;
log.info("SFTP Connection with host is acquired" + channelSftp)
channelSftp.cd("/0002/test/upload/r9a1");
def f1 = new File("/home/dc-user/Files/test_exact5mb.txt");
channelSftp.put(new java.io.FileInputStream(f1), f1.getName()+ Math.random());
session.disconnect()
Configuration used is
No of Threads - 150
Ramp up period - 1
Loop Count -1
After running the script, out of 150 files sometimes 126 or 129 or 141 files gets uploaded but not 150.
Below is the error i see on terminal
The JVM should have exited but did not.
The following non-daemon threads are still running (DestroyJavaVM is OK):
Thread[Connect thread 10.157.147.242 session,5,main], stackTrace:java.net.SocketInputStream#socketRead0
java.net.SocketInputStream#socketRead at line:116
java.net.SocketInputStream#read at line:171
java.net.SocketInputStream#read at line:141
com.jcraft.jsch.IO#getByte at line:82
com.jcraft.jsch.Session#read at line:908
com.jcraft.jsch.Session#run at line:1378
java.lang.Thread#run at line:74
Please let me know if i am missing any configuration
I fail to see how this code can work as your i is not defined anywhere so it will not even compile
It seems that some thread(s) cannot be finished because they are still reading something, I believe you have some form of SLAs or NFRs which define the maximum acceptable time for the file upload operation and your test seems to be exceeding this time.
So before calling def sftpSession = session.connect() I would recommend placing the Session.setTimeout() function call like:
session.setTimeout(5000)
this way if the operation is not finished in 5 seconds your sampler will fail
If you're not too comfortable with Groovy scripting using SSH SFTP sampler might be much easier, check out Load Testing FTP and SFTP Servers Using JMeter article for more details.
I am working on Network Monitoring. I wish to monitor the data and check the network usage details of the client PC from the admin PC.
I have figured that I can fulfill this with the help of captured the packets of the browser as shown below. (This is the packet of FB request)
New packet1459690210:290428 /192.168.0.138->/31.13.79.220 protocol(6) priority(0) hop(128) offset(0) ident(2687) TCP 49281 > 443 seq(1708554008) win(258) ack 938747547 P
But, how can I extract the ipaddress of destination from the captured packets.
I wish to store the data in the database and display as below in the image.
I also wish to have data usage in MB.
Many thanks for help.
(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\d*
This is what I got. This regex retrieves the ip addresses.
I still have to figure out how to get the used MBs. If any one can help, I shall be greatful
I have a user account, but on trying using this am getting exception for rssh. Is there any possibility to give permission of directory using other way other than ChannelExec. Based on the exception I come to know this account can't use ChannelExec to give permission of a directory or file. Thus is there any other way this account could give file permission without getting access to this user account for rssh. Please give you thoughts.
Code :
channelSftp = (ChannelSftp) channel;
ChannelExec channelexe = (ChannelExec) session.openChannel("exec");
channelexe.setCommand("chmod 777 -R " + depDir);
channelexe.connect();
System.out.println("channelexe.getExitStatus:"+channelexe.getExitStatus());
Output :
channelexe.getExitStatus:-1:
This account is restricted by rssh.
Allowed commands: scp sftp
If you believe this is in error, please contact your system administrator.
There's no need to use the "exec" channel for this.
Use the ChannelSftp.chmod:
public void chmod(int permissions, String path)
Note that the method takes the permissions as an integer. So you cannot use 777, as that's an octal representation of the permissions.
An equivalent decimal representation is 511 (= 7*8^2 + 7*8^1 + 7*8^0).
See also Decimal to Octal Conversion.
Though the ChannelSftp.chmod cannot set permissions recursively. The SFTP protocol does not support recursive operations of any kind.
The code below works to two other sites I've tried, but will not work with my domain hosted by 1and1. The return code is always 500 - Permanent Negative Completion reply.
I know I'm connecting because FTPReply.isPositiveCompletion(reply) returns true. I tried it with both reading file off phone storage and sending it from a byte array populated early in the code. This is the byte array. Both return 500. Both work on the other sites.
If I don't use enterLocalPassiveMode() the code stops executing on the storeFile call. No exception, no socket time-out. It just ends there and the async task will not call again in that session. The code does not do that on the other sites.
I've tried both ASCII and BINARY file types. Both return 500. 1and1 site says to use my domain and port 21. I can connect with CoreFTP and read and write using both of the accounts I've set up.
I also tired ftp4j and had the same response with all scenarios so went back to Apache because the code was already written with robust error trapping.
I've tried both mydomain.com and ftp.mydomian.com. 500 on both. I also tried the dot quad I can see in the CoreFTP window, but i get "cannot resolve host name" with the Apache Java code. Maybe not a static IP?
This is what CoreFTP does. It connects on port 21 and then goes in to passive mode and an ASCII data connection.
It's a long shot, but has anyone else ever FTPed to their 1and1 domain using Java in Android Studio?
Greg
Resolving mydomain.com...
Connect socket #5684 to xx.xx.xx.xxx, port 21...
220 Microsoft FTP Service
USER ftp79815757-0
331 Password required for ftp79815757-0.
PASS **********
230 User logged in.
SYST
215 Windows_NT
Keep alive off...
PWD
257 "/ftp79815757-0" is current directory.
PASV
227 Entering Passive Mode (xx,xxx,xx,xxx,xxx,xxx).
LIST
Connect socket #5700 to xx.xx.xx.xx, port 62894...
150 Opening ASCII mode data connection.
226 Transfer complete.
Transferred 51 bytes in 0.094 seconds
FTPClient mFtpClient = new FTPClient();
String ip = "my domain dot com";
String userName = "ftp79815757-0";
String pass = "password";
mFtpClient.connect(InetAddress.getByName(ip));
mFtpClient.login(userName, pass);
int reply = mFtpClient.getReplyCode();
if (FTPReply.isPositiveCompletion(reply)) {
mFtpClient.setFileType(FTP.BINARY_FILE_TYPE);
//one thread said this would do the trick
mFtpClient.enterLocalPassiveMode();
mFtpClient.enterRemotePassiveMode();
InputStream stream = new ByteArrayInputStream(imageData);
//I have two accounts. One points to images_in
/*if (!mFtpClient.changeWorkingDirectory("images_in")) {
Log.e("ChangeDir", String.valueOf(mFtpClient.getReplyCode()));
}*/
if (!mFtpClient.storeFile("remoteName.jpg", stream)) {
Log.e("FTPUpload", String.valueOf(mFtpClient.getReplyCode()));
}
stream.close();
mFtpClient.disconnect();
}
Finally got it. The main problem was that I was using an old version of the Apache library. The Jar I was using was commons-net-1.4.jar. Someone in another thread pointed me to commons-net-3.3.jar.
I commented out both mFtpClient.enterLocalPassiveMode() and mFtpClient.enterRemotePassiveMode(), and with some trail and error it worked with FTP.BINARY_FILE_TYPE and not ASCII_FILE_TYPE. ASCII got the file there, but it was garbage.
I am attempting to send multiple commands to a device using the SSHJ library. There is the option of sending multiple commands in this format:
Command command = sshSession.exec("show line; show ip interface brief;");
This works, but it is not always usable in my case. I have found other suggestions such as the second answer here.
When I attempt this suggestion the first command works fine and then it cycles between this error:
net.schmizz.sshj.connection.ConnectionException: Broken transport; encountered EOF
...
Caused by: net.schmizz.sshj.transport.TransportException: Broken transport; encountered EOF
or
Exception in thread "main" java.lang.IllegalStateException: Not connected
at net.schmizz.sshj.SSHClient.checkConnected(SSHClient.java:713)
at net.schmizz.sshj.SSHClient.startSession(SSHClient.java:651)
The code used is:
sshSession = sshClient.startSession();
Command command = sshSession.exec("sho ip int brie");
System.out.println(IOUtils.readFully(command.getInputStream()));//Just to see the reply while testing
command.join(5, TimeUnit.SECONDS);
sshSession = sshClient.startSession();
Command command2 = sshSession.exec("sho line");
System.out.println(IOUtils.readFully(command2.getInputStream()));//Just to see the reply while testing
A note if needed, the device I am connecting to, and majority of devices that it will connect to are Cisco networking equipment.
Thank you for any assistance.
-Jarrod
Never found a resolution for the exact problem. But I worked around the issue by using the DefaultPTY and providing my own streams with all the data I wanted to send. Playing around with this example.