I am working on an assignment for my Networks class and we have to use sockets to download an image from the web. The idea is to open 5 separate sockets, run them concurrently in separate threads and have them each download a part of the file then stitch them back together after getting the different parts. What I am confused about is how would the second thread know to start the stream at the point the first one ended? Like I have:
InputStream is = new BufferedInputStream(socket.getInputStream());
DataOutputStream dout = new DataOutputStream(socket.getOutputStream());
String request = "GET " + path + " HTTP/1.0\n\r\n\r";
dout.write(request.getBytes());
Then the loop that reads the stream:
while ((bytes = is.read(buffer)) != -1)
But if this is happening every time a run a new thread and create a new socket/TCP connection, how can set the stream to start at the point the last stream stopped? To clarify a bit more, the first thread should create a file called image.jpg.part1 then the second image.jpg.part2 and so on. Then we combine the parts into the original image.
Thanks!
Related
Hello stack overflow world, I've been struggling with the most straight forward and common problem within Java IO, for some time, and now need your help to tackle it.
Check out this piece of code I have in a try block, within a thread.run():
// connect to client socket, and setup own server socket
clientSocket = new Socket(serverHostname, CLIENT_PORT);
//send a test command to download a file
String downloadFileName = "sample.txt";
DataOutputStream dataOutputStream = new DataOutputStream(clientSocket.getOutputStream());
System.out.println("Sending a request to download file : " + downloadFileName + " from user: Arsa node"); //todo: replace with node user later
dataOutputStream.writeUTF("D/sample.txt");
//close socket if host isn't detected anymore, and if socket doesn't become null suddenly
dataOutputStream.flush();
dataOutputStream.close();
System.out.println("****File has been sent****");
in = new DataInputStream(clientSocket.getInputStream());
byte[] retrievedFileData = new byte[8036];
if (in.readInt() > 0) {
System.out.println("Starting file download!");
in.read(retrievedFileData);
System.out.println("File data has been read, converting to file now");
//closing input stream will close socket also
in.close();
}
clientSocket.close();
2 Main questions that have been confusing me to death:
Why does dataOutputStream.close() need to be run for writeUTF to actually send my string to the server socket, I find that when I don't have dos.close(), data isn't retrieved on the other side, further because I close it, I no longer can read from the socket - as it seems the socket connection becomes closed when the Output Stream is previously closed...
What's a better way, following some sort of pattern to do this? For context, all I'm trying to do is write the filename I'm looking to download to my client, then read the response right away, which I expect to be bytes with the file, any error handling I will consider as a part of my development.
Overall, it shouldn't be complicated to write something to a socket, then read and ingest it's response...which doesn't seem to be the case here,
any help would be greatly appreciated! If the ServerSocket code snippet is needed I'm happy to share.
The observed behavior is just a side-effect of close(), as it calls flush() before closing to make sure any buffered data is sent. To solve your problem, you need to call the flush() method instead of closing.
This behavior is not unique to DataOutputStream: a lot of other OutputStream (or Writer) implementations apply buffering, and you will need to flush when you want to ensure the data is sent to the client, written to disk or otherwise processed.
BTW: The DataOutputStream and DataInputStream is for a very specific type of data serialization protocol that is particular to Java. You may want to consider carefully if this is the right protocol to use.
I want to write a "simple" Java Server-type application to stream videos to different clients. My first step would be an again "simple" Android App containing a VideoView and a MediaPlayer set to Video Streaming (More Information on Android SDK - MediaPlayer) though later i might add desktop java application too.
What i'm not sure is how i would actually do the streaming on the server. I already wrote a little Http Server processing HTTP GET requests from a client over TCP.
There i write/"stream" the files back using this coding:
FileInputStream fs = new FileInputStream(f);
final byte[] buffer = new byte[1024];
int count = 0;
//add the header information to the respone
while ((count = fs.read(buffer)) >= 0) {
os.write(buffer, 0, count);
}
os.flush();
fs.close();
os.close();
os being the OutputStream of the Response i get through the tcp socket and f being the requested file.
This seems to send the file almost completly at once though and not like i want to, stream it "in chunks".
So my questions are:
What do i, or do i have to change my coding to actually stream the video, or is it already correct this way?
When i want to make it using UDP instead of TCP would i then only put the buffer byte arrays read from the fileinputstream directly into the DatagramPacket and the MediaPlayer would know what to do with it?
PS: I know there are several questions on here about streaming in Java but none of them actually cover the server side but mainly the (in this case Android-) client side.
I'm trying to write a server in Java. I know very little Java. I've found an example using Selector.
It looks good, but it behaves strangely. When I do my_socket_output_stream.writeBytes("hello world") in client code, the server reads this message one byte at a time. Shouldn't I be notified only when the complete message is sent? Now I'd have to check my buffer after getting every byte to know if I can already work with it. Seems terribly inefficient.
I wonder if that's due to Selector or is that just how sockets work (it's been a long time since I used them). Could I make them wait for the full message somehow? Also, can I associate some objects with a channel? Right now all sockets use the same buffer. I'm sure you see how that is a problem..
The reason I want to use a Selector is that my server is only going to do io with a HashTable. Multiple threads would just be constantly waiting. And I only have one core anyway. Though maybe a combination of ThreadPoolExecutor and ConcurrentHashMap would be a good choice? It would surely enable me to have a buffer per socket..
I'd appreciate suggestions.
I faced the same problem a long time ago. I solved by first sending the number of bytes of the message, then sending the message itself byte by byte. Then I expanded it to line by line.
At the sender's side:
// code at sender side
StreamConnectionNotifier service = (StreamConnectionNotifier) Connector.open( url );
//System.out.println("opened");
StreamConnection con = (StreamConnection) service.acceptAndOpen();
OutputStream outputStream = con.openOutputStream();
// file to send
Scanner in = new Scanner(inFile);
//just count lines
String s=null;
int countLines=0;
while(in.hasNext()) {
s=in.nextLine();
countLines++;
}
//send num of lines
outputStream.write(Integer.toHexString(countLines).getBytes());
try{Thread.sleep(100);} catch(InterruptedException e){}
//send lines
in = new Scanner(inFile);
for(int i=0; i<countLines; i++) {
s=in.nextLine()+"\n";
outputStream.write(s.getBytes());
Thread.sleep(100);
}
At the receiver's side:
// code at receiver side
byte buffer[] = new byte[80];
int bytes_read = inputStream.read( buffer );
String received = new String(buffer, 0, bytes_read);
try{Thread.sleep(100);} catch(InterruptedException e){}
int receiveLines = Integer.parseInt(received);
PrintWriter out = new PrintWriter(new FileOutputStream("received.txt"));
for(int i=0; i<receiveLines; i++) {
bytes_read = inputStream.read( buffer );
received = new String(buffer, 0, bytes_read);
out.println(received);
Thread.sleep(100);
}
I hope this helps :)
Unless you have gained some skill in understanding of the issues of multi-threading and synchronization, avoid NIO. It is good stuff, but you are (currently) not properly equipped to debug it, much less fully appreciate and understand its synchronization needs.
Write a Runnable class that wraps a ServerSocket in a while loop, allowing the loop to block on the accept method. Then grab that return socket and construct a "client handler" thread which will handle whatever data came in the NIC.
This resource will give you some pointers on writing this older, slightly slower, and much more understandable server listening loop. I linked to a "middle" page in the article as that's the code listing, but you might want to read the entire article.
This uses the older "one Thread to handle the request" model of network processing. It's not terribly bad, but it can encounter scalability issues.
The alternative is to take the deep dive and do it with non-blocking NIO. It's not terribly hard, but it does require you to completely structure your server code in a manner that's not straightforward. Effectively you get "pools" of worker Threads than can perform various tasks, and then you synchronize on passing the data from worker to worker.
Shouldn't I be notified only when the complete message is sent?
No. Without specifying how messages should be separated from each other, the API can only give you one byte at a time (or all available bytes). The easiest way to separate strings would be to use a java.io.PrintStream on the side that is sending the message and a java.io.BufferedReader on the side that is receiving, like so:
// code that sends strings
OutputStream out = ...; // get the output stream from the socket
PrintStream sender = new PrintStream(out);
sender.println("Hello, world.");
// code that receives strings
InputStream in = ...; // get the input stream from the socket
BufferedReader receiver = new BufferedReader(new InputStreamReader(in));
String message = receiver.readLine(); // reads "Hello, world."
I am trying to implement a multisocket program (both client and server). After a little googling, I found that a good idea to do it is to implement Runnable. Now I suppose that each thread I create and use .start() is a different client (correct me if I'm wrong).
What I find difficult is to understand 2 things:
-Which is the exact line that accepts data? I guess it's not the Server.accept() method since this method is used to initiate a connection with the specific client (by making a new thread as mentioned before).
-How can I accept more than 1 packet (let's say it's a string)?
A little correction, every new thread you create and start with start() will be a new server thread handling a new client.
Which is the exact line that accepts data?
To accept data from client, you'll have to wrap the client's input stream into some input stream and then call the input stream's respective read function.
void readx(Socket con)
{
String line=new String();
BufferedReader bin= new BufferedReader(new InputStreamReader(con.getInputStream());
while((line = bin.readLine()) != null) //Read new lines coming from the server
System.out.println(line);
}
This is just an example, you can have other InputStream wrappers like DataInputStream and their respective read functions.
How can I accept more than 1 packet (let's say it's a string)?
The above snippet will continuously accept data from client(can be any number of lines) till the client sends an End of Stream character.
I am attempting to transfer files (MP3s about six megabytes in size) between two PCs using SPP over Bluetooth (in Java, with the BlueCove API). I can get the file transfer working fine in one direction (for instance, one file from the client to the server), but when I attempt to send any data in the opposite direction during the same session (i.e., send a file from the server to the client), the program freezes and will not advance.
For example, if I simply:
StreamConnection conn;
OutputStream outputStream;
outputStream = conn.openOutputStream();
....
outputStream.write(data); //Data here is an MP3 file converted to byte array
outputStream.flush();
The transfer works fine. But if I try:
StreamConnection conn;
OutputStream outputStream;
InputStream inputStream;
ByteArrayOutputStream out = new ByteArrayOutputStream();
outputStream = conn.openOutputStream();
inputStream = conn.openInputStream();
....
outputStream.write(data);
outputStream.flush();
int receiveData;
while ((receiveData = inputStream.read()) != -1) {
out.write(receiveData);
}
Both the client and the server freeze, and will not advance. I can see that the file transfer is actually happening at some point, because if I kill the client, the server will still write the file to the hard drive, with no issues. I can try to respond with another file, or with just an integer, and it still will not work.
Anyone have any ideas what the problem is? I know OBEX is commonly used for file transfers over Bluetooth, but it seemed overkill for what I needed to do. Am I going to have to use OBEX for this functionality?
It could be as simple as both programs stuck in blocking receive calls, waiting for the other end to say something... try adding a ton of log statements so you can see what "state" each program is in (ie, so it gives you a running commentary such as "trying to recieve", "got xxx data", "trying to reply", etc), or set up debugging, wait until it gets stuck and then stop one of them and single step it.
you can certainly use SPP to transfer file between your applications (assuming you are sending and receiving at both ends using your application). From the code snippet it is difficult to tell what is wrong with your program.
I am guessing that you will have to close the stream as an indication to the other side that you are done with sending the data .. Note even though you write the whole file in one chunk, SPP / Bluetooth protocol layers might fragment it and the other end could receive in fragments, so you need to have some protocol to indicate transfer completion.
It is hard to say without looking at the client side code, but my guess, if the two are running the same code (i.e. both writing first, and then reading), is that the outputStream needs to be closed before the reading occurs (otherwise, both will be waiting for the other to close their side in order to get out of the read loop, since read() only returns -1 when the other side closes).
If the stream should not be closed, then the condition to stop reading cannot be to wait for -1. (so, either change it to transmit the file size first, or some other mechanism).
Why did you decide to use ByteArrayOutputStream? Try following code:
try {
try {
byte[] buf = new byte[1024];
outputstream = conn.openOutputStream();
inputStream = conn.openInputStream();
while ((n = inputstream.read(buf, 0, 1024)) > -1)
outputstream.write(buf, 0, n);
} finally {
outputstream.close();
inputstream.close();
log.debug("Closed input streams!");
}
} catch (Exception e) {
log.error(e);
e.printStackTrace();
}
And to convert the outputStream you could do something like this:
byte currentMP3Bytes[] = outputStream.toString().getBytes();
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream(currentMP3Bytes);