We Keep Coding

Why does when sending a file in multipart/form-data in java we have to use both a Writer and a OutputStream?

Hi I have seen many sample codes working for sending a file in multipart/form-data in java.
But they have used both Writer and an OutputStream.
Why can't they use just use one of them?
Here is the sample code they have sent
import java.io.*;
import java.net.HttpURLConnection;
import java.net.URL;
public class MainClass_External2 {
public static void main(String[] args){
try{
// Connect to the web server endpoint
URL serverUrl =
new URL("http://posttestserver.com/post.php?dir=example");
HttpURLConnection urlConnection = (HttpURLConnection)serverUrl.openConnection();
String boundaryString = "----SomeRandomText";
String fileUrl = "abc.txt";
File logFileToUpload = new File(fileUrl);
// Indicate that we want to write to the HTTP request body
urlConnection.setDoOutput(true);
urlConnection.setRequestMethod("POST");
urlConnection.addRequestProperty("Content-Type","multipart/form-data; boundary=" + boundaryString);
// Indicate that we want to write some data as the HTTP request body
urlConnection.setDoOutput(true);
OutputStream outputStreamToRequestBody = urlConnection.getOutputStream();
BufferedWriter httpRequestBodyWriter =
new BufferedWriter(new OutputStreamWriter(outputStreamToRequestBody));
// Include value from the myFileDescription text area in the post data
httpRequestBodyWriter.write("\n\n--" + boundaryString + "\n");
httpRequestBodyWriter.write("Content-Disposition: form-data; name=\"myFileDescription\"");
httpRequestBodyWriter.write("\n\n");
httpRequestBodyWriter.write("Log file for 20150208");
// Include the section to describe the file
httpRequestBodyWriter.write("\n--" + boundaryString + "\n");
httpRequestBodyWriter.write("Content-Disposition: form-data;"
+ "name=\"myFile\";"
+ "filename=\""+ logFileToUpload.getName() +"\""
+ "\nContent-Type: text/plain\n\n");
httpRequestBodyWriter.flush();
// Write the actual file contents
FileInputStream inputStreamToLogFile = new FileInputStream(logFileToUpload);
int bytesRead;
byte[] dataBuffer = new byte[1024];
while((bytesRead = inputStreamToLogFile.read(dataBuffer)) != -1){
outputStreamToRequestBody.write(dataBuffer, 0, bytesRead);
}
// Mark the end of the multipart http request
httpRequestBodyWriter.write("\n--" + boundaryString + "--\n");
httpRequestBodyWriter.flush();
// Close the streams
outputStreamToRequestBody.close();
httpRequestBodyWriter.close();
// Read response from web server, which will trigger the multipart HTTP request to be sent.
BufferedReader httpResponseReader =
new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
String lineRead;
while((lineRead = httpResponseReader.readLine()) != null) {
System.out.println(lineRead);
}
}catch (Exception e){
e.printStackTrace();
}
}
}

Basically, the response contains both text and binary data, so using both a Writer and an OutputStream makes perfect sense.
The writer just wraps the output stream, and is used to write text. The output stream itself is used to write the binary data.
Why can't they use just use one of them?
Using just an OutputStream would make it more painful to write the text. Using just a Writer would be inappropriate when binary data needs to be written.

Opening file saved on FTP server

I have a functionality for add attachment. When we attach any file, it is uploaded to FTP server. For viewing purpose, there is a link that shows the file content from FTP server within the browser itself. I can open the file after downloading it from FTP server but i don't want to download or save the file on local machine. is there any way to open the file from FTP server without downloading or saving it on local machine?
The code that i wrote.
FTPClient ftpClient = new FTPClient();
ftpClient.connect(ftpServerIp);
boolean connectionStatus = ftpClient.login(ftpServerUserName, ftpServerPassword);
if(connectionStatus){
log.info("Connected successfully.");
}
ftpClient.changeWorkingDirectory(ftpServerUploadPath);
fos = new FileOutputStream(fileName);
boolean result = ftpClient.retrieveFile(fileName, fos);
log.info("result==>"+result);
Here the result is coming as true while retrieving the file. but i am not able to display the file contents. Is there any way to achieve it?
Any help is appreciated. Thanks in advance.

I have solved the problem by using below code.
FTPClient ftpClient = new FTPClient();
ftpClient.connect(ftpServerIp);
boolean connectionStatus = ftpClient.login(ftpServerUserName, ftpServerPassword);
if(connectionStatus){
log.info("Connected successfully.");
}
ftpClient.changeWorkingDirectory(ftpServerUploadPath);
InputStream stream = ftpClient.retrieveFileStream(fileName);
int length = 0;
ServletContext context = getServletConfig().getServletContext();
String mimetype = context.getMimeType( fileName );
response.setContentType( (mimetype != null) ? mimetype : "application/octet-stream" );
response.setHeader( "Content-Disposition", "attachment; filename=\"" + fileName + "\"" );
byte[] bbuf = new byte[fileName.length()];
DataInputStream in = new DataInputStream(stream);
while ((in != null) && ((length = in.read(bbuf)) != -1))
{
out.write(bbuf,0,length);
}
in.close();
out.flush();
out.close();

Sending HTTP request's

I am writing a web client. I have the following code.
public class Connection extends Thread{
public final static int PORT = 1337;
private ServerSocket svrSocket = null;
private Socket con = null;
public Connection(){
try{
svrSocket = new ServerSocket(PORT);
System.out.println("Conected to: " + PORT);
}catch(IOException ex)
{
System.err.println(ex);
System.out.println("Unable to attach to port");
}
}
public void run(){
while(true)
{
try{
con = svrSocket.accept();//on this part the program stops
System.out.println("Client request accepted");
PrintWriter out = new PrintWriter(con.getOutputStream());
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
out.println("GET /<index.html> HTTP/1.1");
out.println("***CLOSE***");
System.out.println(in.readLine());
/*
String s;
while((s = in.readLine()) != null)
{
System.out.println(s);
}*/
out.flush();
out.close();
in.close();
con.close();
System.out.println("all closed");
}
catch(IOException ex)
{
ex.printStackTrace();
}
}
}
}
The run method will be used latter on. That I have is a file called index.html. This file is in the same file as the java code. What I am trying to do with the request is send the HTML file. But if I run this program on a web browser localhost:1337 the following gets displayed.
GET /<index.html> HTTP/1.1
***CLOSE***
This should not get displayed. The page that results of the HTML code in the index.html should get displayed.
Index.html code:
<html>
<head>
<title> </title>
</head>
<body bgcolor = "#ffffcc" text = "#000000">
<h1>Hello</h1>
<p>This is a simple web page</p>
</body>
</html>
How do I get this html page to display in the browser?
Thank you

t seems that all is good on your code, it seems you need to read the HTTP header from the input stream so you can get the requested file name and then use the Socket output stream to write the response from the file.
OutputStream output = con.getOutputStream();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String fileName = readHeader(in);
String baseDir = System.getProperty("my.base.dir", "/home/myname/httpserver");
boolean exist = true;
InputStream fileIn = null;
try {
File requestedFile = new File(baseDir, fileName);
fileIn = new FileInputStream(requestedFile);
} catch(Exception e){
exist = false;
}
String server = "Java Http Server";
String statusLine = null;
String typeLine = null;
String body = null;
String lengthLine = "error";
if (exist) {
statusLine = "HTTP/1.0 200 OK" + "\r\n";
//get content type by extension
typeLine = "Content-type: html/text \r\n";
lengthLine = "Content-Length: " + (new Integer(fileIn.available())).toString() + "\r\n";
} else {
statusLine = "HTTP/1.0 404 Not Found" + CRLF;
typeLine = "text/html";
body = "<HTML>" + "<HEAD><TITLE>404</TITLE></HEAD>" + "<BODY>404 Not Found"+"</BODY></HTML>";
}
output.write(statusLine.getBytes());
output.write(server.getBytes());
output.write(typeLine.getBytes());
output.write(lengthLine.getBytes());
output.write("\r\n".getBytes());
if (exist) {
byte[] buffer = new byte[1024];
int bytes = 0;
while ((bytes = fileIn.read(buffer)) != -1) {
output.write(buffer, 0, bytes);
}
} else {
output.write(body.getBytes());
}
//close sreams

You are confusing a couple of things. First of all: what you are writing is a server, not a client.
Second: You are not following the HTT Protocol.
The line GET /<index.html> HTTP/1.1 (which is wrong, it should be GET /index.html HTTP/1.1) is a request that is sent by the client (like a web browser). Instead, it is your server sending this.
A quick solution:
Instead of sending this static text (the line with the GET and the one with the ***CLOSE***), read the content of your index.html file and print it to your out stream.
EDIT: Here's a quick overview of the http data flow:
The client (e.g. a browser) connects to the server
The client sends it's request, something like
GET /theFileIWant.html HTTP/1.1\r\n
Host: localhost\r\n
\r\n
at this point, the client usually stops sending anything and waits for the server to respond. That is called the "request/response" model.
The server reads the request data and finds out what it has to do.
The output (in this case: a file's content) is sent to the client, preceded by HTTP response headers.
The connection can be kept open or closed, depending on the HTTP headers of both client's request and server's response.

Resumable upload from Java client to Grails web application?

After almost 2 workdays of Googling and trying several different possibilities I found throughout the web, I'm asking this question here, hoping that I might finally get an answer.
First of all, here's what I want to do:
I'm developing a client and a server application with the purpose of exchanging a lot of large files between multiple clients on a single server. The client is developed in pure Java (JDK 1.6), while the web application is done in Grails (2.0.0).
As the purpose of the client is to allow users to exchange a lot of large files (usually about 2GB each), I have to implement it in a way, so that the uploads are resumable, i.e. the users are able to stop and resume uploads at any time.
Here's what I did so far:
I actually managed to do what I wanted to do and stream large files to the server while still being able to pause and resume uploads using raw sockets. I would send a regular request to the server (using Apache's HttpClient library) to get the server to send me a port that was free for me to use, then open a ServerSocket on the server and connect to that particular socket from the client.
Here's the problem with that:
Actually, there are at least two problems with that:
I open those ports myself, so I have to manage open and used ports myself. This is quite error-prone.
I actually circumvent Grails' ability to manage a huge amount of (concurrent) connections.
Finally, here's what I'm supposed to do now and the problem:
As the problems I mentioned above are unacceptable, I am now supposed to use Java's URLConnection/HttpURLConnection classes, while still sticking to Grails.
Connecting to the server and sending simple requests is no problem at all, everything worked fine. The problems started when I tried to use the streams (the connection's OutputStream in the client and the request's InputStream in the server). Opening the client's OutputStream and writing data to it is as easy as it gets. But reading from the request's InputStream seems impossible to me, as that stream is always empty, as it seems.
Example Code
Here's an example of the server side (Groovy controller):
def test() {
InputStream inStream = request.inputStream
if(inStream != null) {
int read = 0;
byte[] buffer = new byte[4096];
long total = 0;
println "Start reading"
while((read = inStream.read(buffer)) != -1) {
println "Read " + read + " bytes from input stream buffer" //<-- this is NEVER called
}
println "Reading finished"
println "Read a total of " + total + " bytes" // <-- 'total' will always be 0 (zero)
} else {
println "Input Stream is null" // <-- This is NEVER called
}
}
This is what I did on the client side (Java class):
public void connect() {
final URL url = new URL("myserveraddress");
final byte[] message = "someMessage".getBytes(); // Any byte[] - will be a file one day
HttpURLConnection connection = url.openConnection();
connection.setRequestMethod("GET"); // other methods - same result
// Write message
DataOutputStream out = new DataOutputStream(connection.getOutputStream());
out.writeBytes(message);
out.flush();
out.close();
// Actually connect
connection.connect(); // is this placed correctly?
// Get response
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String line = null;
while((line = in.readLine()) != null) {
System.out.println(line); // Prints the whole server response as expected
}
in.close();
}
As I mentioned, the problem is that request.inputStream always yields an empty InputStream, so I am never able to read anything from it (of course). But as that is exactly what I'm trying to do (so I can stream the file to be uploaded to the server, read from the InputStream and save it to a file), this is rather disappointing.
I tried different HTTP methods, different data payloads, and also rearranged the code over and over again, but did not seem to be able to solve the problem.
What I hope to find
I hope to find a solution to my problem, of course. Anything is highly appreciated: hints, code snippets, library suggestions and so on. Maybe I'm even having it all wrong and need to go in a totally different direction.
So, how can I implement resumable file uploads for rather large (binary) files from a Java client to a Grails web application without manually opening ports on the server side?

HTTP GET method have special headers for range retrieval: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35 It's used by most downloaders to do resumable download from server.
As I understand, there are no standard practice for using this headers for POST/PUT request, but it's up to you, right? You can make pretty standard Grails controller, that will accept standard http upload, with header like Range: bytes=500-999. And controller should put this 500 uploaded bytes from client into file, starting at position 500
At this case you don't need to open any socket, and make own protocols, etc.
P.S. 500 bytes is just a example, probably you're using much bigger parts.

Client Side Java Programming:
public class NonFormFileUploader {
static final String UPLOAD_URL= "http://localhost:8080/v2/mobileApp/fileUploadForEOL";
static final int BUFFER_SIZE = 4096;
public static void main(String[] args) throws IOException {
// takes file path from first program's argument
String filePath = "G:/study/GettingStartedwithGrailsFinalInfoQ.pdf";
File uploadFile = new File(filePath);
System.out.println("File to upload: " + filePath);
// creates a HTTP connection
URL url = new URL(UPLOAD_URL);
HttpURLConnection httpConn = (HttpURLConnection) url.openConnection();
httpConn.setDoOutput(true);
httpConn.setRequestMethod("POST");
// sets file name as a HTTP header
httpConn.setRequestProperty("fileName", uploadFile.getName());
// opens output stream of the HTTP connection for writing data
OutputStream outputStream = httpConn.getOutputStream();
// Opens input stream of the file for reading data
FileInputStream inputStream = new FileInputStream(uploadFile);
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead = -1;
while ((bytesRead = inputStream.read(buffer)) != -1) {
System.out.println("bytesRead:"+bytesRead);
outputStream.write(buffer, 0, bytesRead);
outputStream.flush();
}
System.out.println("Data was written.");
outputStream.flush();
outputStream.close();
inputStream.close();
int responseCode = httpConn.getResponseCode();
if (responseCode == HttpURLConnection.HTTP_OK) {
// reads server's response
BufferedReader reader = new BufferedReader(new InputStreamReader(
httpConn.getInputStream()));
String response = reader.readLine();
System.out.println("Server's response: " + response);
} else {
System.out.println("Server returned non-OK code: " + responseCode);
}
}
}
Server Side Grails Programme:
Inside the controller:
def fileUploadForEOL(){
def result
try{
result = mobileAppService.fileUploadForEOL(request);
}catch(Exception e){
log.error "Exception in fileUploadForEOL service",e
}
render result as JSON
}
Inside the Service Class:
def fileUploadForEOL(request){
def status = false;
int code = 500
def map = [:]
try{
String fileName = request.getHeader("fileName");
File saveFile = new File(SAVE_DIR + fileName);
System.out.println("===== Begin headers =====");
Enumeration<String> names = request.getHeaderNames();
while (names.hasMoreElements()) {
String headerName = names.nextElement();
System.out.println(headerName + " = " + request.getHeader(headerName));
}
System.out.println("===== End headers =====\n");
// opens input stream of the request for reading data
InputStream inputStream = request.getInputStream();
// opens an output stream for writing file
FileOutputStream outputStream = new FileOutputStream(saveFile);
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead = inputStream.read(buffer);
long count = bytesRead
while(bytesRead != -1) {
outputStream.write(buffer, 0, bytesRead);
bytesRead = inputStream.read(buffer);
count += bytesRead
}
println "count:"+count
System.out.println("Data received.");
outputStream.close();
inputStream.close();
System.out.println("File written to: " + saveFile.getAbsolutePath());
code = 200
}catch(Exception e){
mLogger.log(java.util.logging.Level.SEVERE,"Exception in fileUploadForEOL",e);
}finally{
map <<["code":code]
}
return map
}
I have tried with above code it is worked for me(only for file size 3 to 4MB, but for small size files some bytes of code missing or not even coming but in request header content-length is coming, not sure why it is happening.)

How-to Test/"Drive" HttpRequest.java (Web Server)?

I am writing simple, unsophisticated web-server code in java. It seems to be finished, but I'm not quite sure how to test it. Could someone point me in the right direction? All the coding is finished, I just need to test the code. I tried running it from the terminal, and then connecting to localhost with a specified port, but I only get 404 NOT FOUNDs. I reiterate, I don't think this is a problem with the code, but with my guessing at methods by which to test drive said code. Ideas?
import java.io.*;
import java.net.*;
import java.util.*;
final class HttpRequest implements Runnable {
final static String CRLF = "\r\n";
Socket socket;
// Constructor
public HttpRequest(Socket socket) throws Exception {
this.socket = socket;
}
// Implement the run() method of the Runnable interface.
public void run() {
try {
processRequest();
} catch (Exception e) {
System.out.println(e);
}
}
private static void sendBytes(FileInputStream fis, OutputStream os)
throws Exception {
// Construct a 1K buffer to hold bytes on their way to the socket.
byte[] buffer = new byte[1024];
int bytes = 0;
// Copy requested file into the socket's output stream.
while((bytes = fis.read(buffer)) != -1 ) {
os.write(buffer, 0, bytes);
}
}
private static String contentType(String fileName) {
if(fileName.endsWith(".htm") || fileName.endsWith(".html")) {
return "text/html";
}
if(fileName.endsWith(".jpeg") || fileName.endsWith(".jpg")) {
return "image/jpeg";
}
if(fileName.endsWith(".gif")) {
return "image/gif";
}
return "application/octet-stream";
}
private void processRequest() throws Exception {
// Get a reference to the socket's input and output streams.
InputStream is = socket.getInputStream();
DataOutputStream os = new DataOutputStream(socket.getOutputStream());
// Set up input stream filters.
BufferedReader br = new BufferedReader(new InputStreamReader(is));
// Get the request line of the HTTP request message.
String requestLine = new String(br.readLine());
// Display the request line.
System.out.println();
System.out.println(requestLine);
// Get and display the header lines.
String headerLine = null;
while ((headerLine = br.readLine()).length() != 0) {
System.out.println(headerLine);
}
// Extract the filename from the request line.
StringTokenizer tokens = new StringTokenizer(requestLine);
tokens.nextToken(); // skip over the method, which should be "GET"
String fileName = tokens.nextToken();
// Prepend a "." so that file request is within the current directory.
fileName = "." + fileName;
// Open the requested file.
FileInputStream fis = null;
boolean fileExists = true;
try {
fis = new FileInputStream(fileName);
} catch (FileNotFoundException e) {
fileExists = false;
}
// Construct the response message.
String statusLine = null;
String contentTypeLine = null;
String entityBody = null;
if (fileExists) {
statusLine = "200 OK" + CRLF;
contentTypeLine = "Content-type: " +
contentType( fileName ) + CRLF;
} else {
statusLine = "404 NOT FOUND" + CRLF;
contentTypeLine = "Content Not Found!" + CRLF;
entityBody = "<HTML>" +
"<HEAD><TITLE>Not Found</TITLE></HEAD>" +
"<BODY>Not Found</BODY></HTML>";
}
// Send the status line.
os.writeBytes(statusLine);
// Send the content type line.
os.writeBytes(contentTypeLine);
// Send a blank line to indicate the end of the header lines.
os.writeBytes(CRLF);
// Send the entity body.
if (fileExists) {
sendBytes(fis, os);
fis.close();
} else {
os.writeBytes("File DNE: Content Not Found!");
}
// Close streams and socket.
os.close();
br.close();
socket.close();
}
public static void main(String[] args) throws Exception {
final ServerSocket ss = new ServerSocket(8080);
while (true)
new HttpRequest(ss.accept()).run();
}
}
Solution:
SOLVED. It turns out, to test it, all you need to do is:
1) run the program from the terminal as per usual,
2) place the file you want to try to retrieve (lets say "example.html") into the same folder as your .java file(s),
3) in a separate terminal, run the command $ wget localhost:PORT/FILE.EXTENSION
(I used port 8080 here, so $ wget localhost:8080/example.html)
You should now see, in the folder you are currently sending the wget command from, an html response file "200 OK" or "404 File Not Server", along with the contents of the file if the former is true.
I was over-complicating this, as were the comments/replies... But it's done.
Guessing and checking ftw.

SOLVED. It turns out, to test it, all you need to do is:
1) run the program from the terminal as per usual,
2) place the file you want to try to retrieve (lets say "example.html") into the same folder as your .java file(s),
3) in a separate terminal, run the command $ wget localhost:PORT/FILE.EXTENSION
(I used port 8080 here, so $ wget localhost:8080/example.html)
You should now see, in the folder you are currently sending the wget command from, a response file "200 OK" or "404 File Not Server", along with the contents of the file if the former is true.
I was over-complicating this, as were the comments/replies... But it's done.
Guessing and checking ftw.