Please, help.
I have a java program that currently reads a list of files from physical directory (shared) and while reading, creating output files - also on the shared location.
So, for several files read, there can be a a single output file.
New requirements came to take the files from that output location and ftp them to a given ftp server
The caveat is: The ftp procedure is done by microservice that most likely runs on the machines which do not have access to the output folder. So, I am making a call to a service, not using FTP API (Appache, or etc..) internally.
Is it possible to send a OutputStream (byteOutputStream, or ObjectOutputStream) over an http call? If so, how?
I know how to ftp the files from within my application, but the goal is .. to make a microservice call (webservice call, for what it matters, since my application is a plain old core java app).
I had a (rather different) project recently, however it also had the requirement to write to a http response.
I used this approach which works in my case:
try (OutputStream os = response.getOutputStream()) {
final PrintWriter w = new PrintWriter(new OutputStreamWriter(os, "UTF-8"));
Then after that, I write to the PrintWriter w.
Related
I have created Spring project which writes into a .csv file and then sends the file back to be downloaded from frontend. this works fine while testing at localhost. But when compiled to JAR the file paths change and result is IOException file not found
At the moment i have calculation.csv in my resources folder.
I also understand that i could write csv file outside of the JAR, but since this application needs to be portable and i have no guarantee of writing permissions, it makes this quite hard to accomplish.
is there a way, using maybe something like just input/output streams to make this work? I have no need to save the file for longer times, i just need temp file to hold CSV information while it's being written and then be able to return this same file to user (from endpoint).
current implimation is just a basic File which lives in resources/csv/calculation.csv
File file = new File("csv/calculation.csv");
Response.ResponseBuilder response = Response.ok(file);
transactionManagement.writeCSV(detailedList);
return response.build();
transactionManagement.writeCSV(detailedList);
is just a very simple service that writes csv line by line from data
I am trying to find a way to read just end of huge and dynamic log file (like 20-30 lines from end) via SFTP from server and to save the point until where I read, and if I need more lines, to read more from this point upper.
Everything I've tried takes too long time, I've tried to copy this file on machine and after this to read from end using ReversedLinesFileReader because this method need the File object, when via SFTP you will get only InputStream, takes a lot to download file.
Also tried to count lines and to read from n line but also takes too long and throws exception because sometime in this time file is modified. Another way I tried to connect via SSH and used tail -100 and get the desired result, but just for one time, because next time I will get also new logs, but I need to go upper. Is there a fast way to get the end of file and to save the point and to read more upper of this point later? Any idea?
You don't say what SFTP library you're using, but the most widely used Java SSH/SFTP library is JSch, so I'll assume you're using that.
The SFTP protocol has operations to perform random-access I/O on remote files. Unfortunately, the JSch SFTP client doesn't expose the full range of operations. However, it does have versions of the get operation (for getting a file from the remote server) which permit skipping over the first part of the remote file. You can use one of these operations to read for example the last 10 KB of a file.
Several of the JSch get operations return an InputStream. You can read the contents of the remote file from the input stream. If you want to access the remote file line-by-line, you can convert it to Reader using InputStreamReader.
So, a process might do the following:
Call stat() on the remote file to get its size.
Figure out where in the file you want to start reading from. You could keep track of where you stopped reading last time, or you could guess based on the amount of data you're willing to download and the expected size in bytes of these last 20-30 lines.
Call get() to start reading it.
Process data read from the InputStream returned by the get() call.
Best would be to have a kind of rotating log files, possibly with compression.
Hower rsync is a unidirectional synchronisation, that can transmit only the changed parts of a file: for a log the new end.
I am not sure whether it works sufficiently performant in your case, and ssh is a prerequisite.
I'm using JSP and I have a complex serializable object that I want to be downloaded into a flat file format to the client computer. How can this be done?
I can get the object to save locally to the JSP server using ObjectOutputStream but I need the object to be stored remotely to the JSP server and never locally (e.g. to the clients storage)
Once the object is stored on the clients computer locally, I then want to be able to upload it back to the JSP server and opened/read into an object, how can this be done?
Kind Regards,
Tim
I think, it is not possible for a JSP server to write automatically to a network location. This is a security restriction and may not be possible without a manual request to application. You may want to use below strategies:
Create a shared location at client machine (assuming, it is on network) and mount the location on your server, then code the JSP to save the file at that location.
Save the file on JSP server location, then an FTP transfer script can be written to transfer file to client machine; using a job scheduler (needs FTP communication to be established).
Code the application in such a way that it could be downloaded using the application website (like a music file is downloaded). Then at client side, you may run a program, which will connect to the app server and programatically save the file at the client machine.
Your problem is sending a binary object over http.
You can use the following code snippet in the JSP code.
response.setContentType("binary/octet-stream");
response.setHeader("Content-Transfer-Encoding", "binary");
response.setHeader("Content-Disposition","attachment; filename=myjavaobject.obj");
ObjectOutputStream out = new ObjectOutputStream(response.getOutputStream());
oos.writeObject(yourJavaObject); //Assuming yourJavaObject is the java object you want to send.
out.flush();
out.close();
Do not forget to make sure that yourJavaObject is serializable.
i am making a client server program in java, a portion of which requires to send
a complete file (.txt or .docx) file from client to server. I am not sure how to do
it, i have tried using this webpage
http://www.rgagnon.com/javadetails/java-0542.html
, but it seems that it don't work correctly
in form + multi-threaded application, is there any other way to send a complete
file from client to server?
Any help/suggestions are appreciated...
regards
usama
In principle you have to read the file on your client application, open an OutputStream to the server, and write the contents of the file to this stream.
On the server you read from the other end of the stream.
Depending on your architecture, you either will use a new socket-pair for the file transfer, or somehow embed it into your existing communication protocol.
It would be better to use a Messaging Application for your purpose. I would recommend using JMS. I personally use ActiveMQ . If this does not suit your need then try out Apache Mina. It would abstract you from the Network Programming.
I need a simple application, preferably a cross-platform one, that enables sending of files between two computers.
It just need to accept and send the files, and show a progress bar. What applications could I use or how could I write one?
Sending and Receiving Files
The sending and receiving of a file basically breaks down to two simple pieces of code.
Recieving code:
ServerSocket serverSoc = new ServerSocket(LISTENING_PORT);
Socket connection = serverSoc.accept();
// code to read from connection.getInputStream();
Sending code:
File fileToSend;
InputStream fileStream = new BufferedInputStream(fileToSend);
Socket connection = new Socket(CONNECTION_ADDRESS, LISTENING_PORT);
OutputStream out = connection.getOutputStream();
// my method to move data from the file inputstream to the output stream of the socket
copyStream(fileStream, out);
The sending piece of code will be ran on the computer that is sending the code when they want to send a file.
The receiving code needs to be put inside a loop, so that everytime someone wants to connect to the server, the server can handle the request and then go back to waiting on serverSoc.accept().
To allow sending files between both computers, each computer will need to run the server (receiving code) to listen for incoming files, and they will both need to run the sending code when they want to send a file.
Progress Bar
The JProgressBar in Swing is easy enough to use. However, getting it to work properly and show current progress of the file transfer is slightly more difficult.
To get a progress bar to show up on a form only involves dropping it onto a JFrame and perhaps setting setIndeterminate(false) so hat it shows that your program is working.
To implement a progress bar correctly you will need to create your own implementation of a SwingWorker. The Java tutorials have a good example of this in theirlesson in concurrency.
This is a fairly difficult issue on its's own though. I would recommend asking this in it's own question if you need more help with it.
Woof is a cool Python script that might work for you:
http://www.home.unix-ag.org/simon/woof.html
I would strongly consider using FTP. Apache has a FTP client and a server
Edit: spdenne's suggestion of HTTP is also good, especially if everyone has Java 6. If not, you can use something like Tiny Java Web Server.
You can write one by using Socket programming in Java. You would need to write a Server and a Client program. The server would use a ServerSocket to listen for connections, and the Client would use a Socket to connect to that server on the specified port.
Here's a tutorial: http://www.javaworld.com/jw-12-1996/jw-12-sockets.html
Sun's Java 6 includes a light-weight HTTP server API and implementation. You could fairly easily use this to serve your file, using URLConnection to obtain it.
Check out this tutorial, it's a really basic example. You would probably also want to send control headers prior to the actual file being sent, containing the size of the file, filename, etc.
Alternatively, base it on an existing protocol, like this project.
Can you install FTP servers on (one of) your machines ?
If you can, you will just have to use a FTP client (FileZilla for example, which have a progress bar).
Two popular apps are "scp" and "rsync". These are standard on Linux, are generally available on Unix and can be run on Windows under cygwin, although you may be able to find windows-native apps that can do it as well. (PuTTY can serve as an SCP client).
For any sort of pc-to-pc file transfer, you need to have a listener on the destination PC. This can be a daemon app (or Windows system process), or it can be a Unix-style "superserver" that's configured to load and run the actual file-copy app when someone contacts the listening port.
SCP and one of the rsync modes do require that there be some sort of remote login capability. Rsync can also publish resources that it will handle directory. Since the concept of a Windows "remote login" isn't as well-established as it is under Linux, this may be preferable. Plus it limits remote access to defined sources/targets on the destination machine instead of allowing access to any (authorized) part of the filesystem.
To transfer over a network more efficiently. Take a look at this article that explains efficient data transfer through zero copy