We Keep Coding

reading data from website into variable in Java using java.util.Scanner

I'm attempting to fetch data from an url and dump it into "content" instance variable. Both the url and the content should be initialized in the contructor. The hasNextLine() and nextLine() are also involved but being completely new to Java I can't make sense of that. Here's the code:
public class NewsFinder {
// Instance variables
private String url;
private String content;
private Scanner s;
// Getter methods
public String getUrl() {
return url;
}
public String getContent() {
return content;
}
// Constructor
public NewsFinder(String url) {
this.url = url;
try {
Scanner s = new Scanner(new URL(url).openStream());
if (s.hasNextLine()) {
this.s = s.nextLine();
}
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public boolean isInNews(Object o) {
if (((String) o).contains(content)) {
return true;
} else {
return false;
}
}
Any suggestions?

Scanner s = new Scanner(new URL(url).openStream());
if (s.hasNextLine()) {
this.s = s.nextLine();
}
Logically, this part of code should be replaced by :
Scanner s = new Scanner(new URL(url).openStream());
while (s.hasNextLine()) {
this.s += s.nextLine();
}
Personnaly, I would use an InputStream and a BufferedReader to achieve this.
Example
URL url; InputStream is; BufferedReader br; String line; StringBuilder sb;
try{
url = new URL("http://stackoverflow.com");
is = url.openStream();
br = new BufferedReader(new InputStreamReader(is));
sb = new StringBuilder();
while ((line = br.readLine()) != null){
sb.append(line);
}
}catch(Exception e){
e.printStackTrace();
}finally{
try{
if(is != null){
is.close();
}catch(Exception e){
e.printStackTrace();
}
}

Why does multithreading my single-threaded simple echo server cause it to stop working?

This single threaded echo server works fine.
public class Server {
public static void main(String[] args) throws IOException {
try (ServerSocket sc = new ServerSocket(1111)) {
while (true) {
try (Socket dataSocket = sc.accept();
BufferedReader is = new BufferedReader(
new InputStreamReader(
dataSocket.getInputStream()));
PrintWriter out = new PrintWriter(
dataSocket.getOutputStream());) {
String line;
while ((line = is.readLine()) != null) {
System.out.println(line);
out.println(line);
out.flush();
if (line.equals("Bye."))
break;
}
}
}
}
}
}
But why does this multi-threaded version not work? It simply passes input and output streams to construct a TestServer1 thread and start it. Nothing special. But somehow when a client connects to this server, A "Stream close" exception is thrown in run() and "error in run" is printed.
public class TestServer1 extends Thread{
BufferedReader in;
PrintWriter out;
public TestServer1(BufferedReader in, PrintWriter out){
this.in=in;
this.out=out;
}
#Override
public void run(){
String line;
try{
while ((line = in.readLine()) != null) {
System.out.println(line);
out.println(line);
out.flush();
if (line.equals("Bye."))
break;
}
} catch (IOException e){
System.out.println("error in run");
e.printStackTrace();
}
}
public static void main(String[] args) throws IOException {
try (ServerSocket sc = new ServerSocket(1111)) {
while (true) {
try (Socket dataSocket = sc.accept();
BufferedReader in = new BufferedReader(
new InputStreamReader(
dataSocket.getInputStream()));
PrintWriter out = new PrintWriter(
dataSocket.getOutputStream());) {
TestServer1 ts1=new TestServer1(in, out);
ts1.start();
}
}
}
}
}
Here is the stacktrace
error in run
java.io.IOException: Stream closed
at java.io.BufferedReader.ensureOpen(BufferedReader.java:115)
at java.io.BufferedReader.readLine(BufferedReader.java:310)
at java.io.BufferedReader.readLine(BufferedReader.java:382)
at server.TestServer1.run(TestServer1.java:22)

Java 7: Try with resources
With Java 7, you can create one or more “resources” in the try
statement. A “resources” is something that implements the
java.lang.AutoCloseable interface. This resource would be
automatically closed and the end of the try block.
See more at: http://www.vineetmanohar.com/2011/03/java-7-try-with-auto-closable-resources/#sthash.cnvRzGIZ.dpuf
From javadocs:
static String readFirstLineFromFile(String path) throws IOException {
try (BufferedReader br =
new BufferedReader(new FileReader(path))) {
return br.readLine();
}
}
In this example, the resource declared in the try-with-resources statement is a BufferedReader. The declaration statement appears within parentheses immediately after the try keyword. The class BufferedReader, in Java SE 7 and later, implements the interface java.lang.AutoCloseable. Because the BufferedReader instance is declared in a try-with-resource statement, it will be closed regardless of whether the try statement completes normally or abruptly (as a result of the method BufferedReader).
Link : http://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html
So probably this can be reason , your streams are closed automatically when it goes out of scope.

the try () is the reason. the stream will be closed after the try (.....) block.
Please test this code:
while (true) {
try {
Socket dataSocket = sc.accept();
BufferedReader in = new BufferedReader(
new InputStreamReader(
dataSocket.getInputStream()));
PrintWriter out = new PrintWriter(
dataSocket.getOutputStream());
TestServer1 ts1 = new TestServer1 (in, out);
ts1.start();
} catch (Exception e) {
e.printStackTrace();
}
}
P.S: You need to close the stream in the thread run() method.

Try this version out below - its untested. But as I said in my comment, I believe you are closing the output stream prematurely in your TryWithResources
public class TestServer1 extends Thread{
Socket connection;
public TestServer1(Socket connection){
this.connection = connection;
}
#Override
public void run(){
try (BufferedReader in = new BufferedReader(
new InputStreamReader(connection.getInputStream()));
PrintWriter out = new PrintWriter(connection.getOutputStream());) {
String line;
try {
while ((line = in.readLine()) != null) {
System.out.println(line);
out.println(line);
out.flush();
if (line.equals("Bye."))
break;
} catch (IOException e){
System.out.print("error in run");
}
}
}
public static void main(String[] args) throws IOException {
try (ServerSocket sc = new ServerSocket(1111)) {
while (true) {
try (Socket dataSocket = sc.accept()) {
TestServer1 ts1=new TestServer1(dataSocket);
ts1.start();
}
}
}
}
}

Based on the answers from #Amir, #LFF and #Abhijeet, I put together the following version, and it works. The main take aways is: "Socket dataSocket = sc.accept()" should not be put into "try()" as a resource; otherwise, the main thread will close it. It should be closed by the child thread in "run()".
Thank you all for your help.
public class ThreadedServer extends Thread {
private Socket dataSocket;
public ThreadedServer(Socket dataSocket) throws IOException {
this.dataSocket = dataSocket;
}
#Override
public void run() {
String line;
try (BufferedReader in = new BufferedReader(new InputStreamReader(
dataSocket.getInputStream()));
PrintWriter out = new PrintWriter(dataSocket.getOutputStream());) {
while ((line = in.readLine()) != null) {
System.out.println(line);
out.println(line);
out.flush();
if (line.equals("Bye."))
break;
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
dataSocket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
public static void main(String[] args) throws IOException {
try (ServerSocket sc = new ServerSocket(1111)) {
while (true) {
try {
Socket dataSocket = sc.accept();
ThreadedServer ts = new ThreadedServer(dataSocket);
ts.start();
} finally {
}
}
}
}

Making Java I / O and change the file to split in java

I'm making a project where using java I / O
I have a file with the following data:
170631|0645| |002014 | 0713056699|000000278500
155414|0606| |002014 | 0913042385|000001220000
000002|0000|0000|00000000000|0000000000000000|000000299512
and the output I want is as follows:
170631
0645
002014
file so that the data will be decreased down
and this is my source code:
public class Tes {
public static void main(String[] args) throws IOException{
File file;
BufferedReader br =null;
FileOutputStream fop = null;
try {
String content = "";
String s;
file = new File("E:/split/OUT/Berhasil.RPT");
fop = new FileOutputStream(file);
br = new BufferedReader(new FileReader("E:/split/11072014/01434.RPT"));
if (!file.exists()) {
file.createNewFile();
}
while ((s = br.readLine()) != null ) {
for (String retVal : s.split("\\|")) {
String data = content.concat(retVal);
System.out.println(data.trim());
byte[] buffer = data.getBytes();
fop.write(buffer);
fop.flush();
fop.close();
}
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
I want is to generate output as above from the data that has been entered
File Input -> Split -> File Output
thanks :)

I think you forgot to mention what problem are you facing. Just by looking at the code it seems like you are closing the fop(FileOutputStream) every time you are looping while writing the split line. The outputStream should be closed once you have written everything, outside the while loop.
import java.io.*;
public class FileReadWrite {
public static void main(String[] args) {
try {
FileReader inputFileReader = new FileReader(new File("E:/split/11072014/01434.RPT"));
FileWriter outputFileWriter = new FileWriter(new File("E:/split/11072014/Berhasil.RPT"));
BufferedReader bufferedReader = new BufferedReader(inputFileReader);
BufferedWriter bufferedWriter = new BufferedWriter(outputFileWriter);
String line;
while ((line = bufferedReader.readLine()) != null) {
for (String splitItem : line.split("|")) {
bufferedWriter.write(splitItem + "\n");
}
}
bufferedWriter.flush();
bufferedWriter.close();
bufferedReader.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}

Read a file from a FTP server but InputStream is always null

I want read files in a directory. I want add in:
List<String> nomi = new ArrayList<String>();
the linestring of file Nomi.txt.
With debug i view correctly the files in links(001.jpg 002.jpg 003.jpg) and ft(Nomi.txt), but in stream i have always null;
InputStream stream = f.retrieveFileStream(/*url_ftp + "/photo/"+ft*/ft);
my complete code is this:
private static abstract class GetLinksTask extends AsyncTask<String, Void, List<String>> {
protected List<String> doInBackground(String... urls) {
List<String> links = new ArrayList<String>();
String ft=null;
BufferedReader reader = null;
List<String> nomi = new ArrayList<String>();
FTPClient f = new FTPClient();
try {
int reply;
f.connect(url_ftp);
f.login(username,password );
reply = f.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply)) {
f.disconnect();
System.err.println("FTP server refused connection.");
}
FTPListParseEngine engine = f.initiateListParsing("photo");
while (engine.hasNext()) {
FTPFile[] files = engine.getNext(25); // "page size" you want
//FTPFile[] files = engine.getFiles(filter);
for (FTPFile file : files) {
if(file.getName().substring(file.getName().length()-3,file.getName().length()).equals("jpg")){
System.out.println(file.getName());
links.add(file.getName());
}else{
ft=file.getName();
InputStream stream = f.retrieveFileStream(/*url_ftp + "/photo/"+ft*/ft);
reader = new BufferedReader(new InputStreamReader(stream, "UTF-8"));
//nomi.add(reader.readLine());
}
System.out.println(file.getName());
}
//names=nomi;
}
}catch (IOException e) {
e.printStackTrace();
}finally {
if (reader != null)
try {
reader.close();
}catch (IOException logOrIgnore) {
}
}
return links;
}
protected abstract void postExecute(List<String> links);
protected void onPostExecute(List<String> lists) {
postExecute(lists);
}
}
Some tips?
thanks

It is not enough to create a Reader
reader = new BufferedReader(new InputStreamReader(stream, "UTF-8"));
and to close it:
reader.close();
Somewhere, in between, you'll actually have to read the data:
String line;
while( (line = reader.readLine()) != null ){
nomi.add( line );
}

I am explaining the full code of getting inputStream from FTP server and then how to read data from that inputstream. I am assuming, you are using TLS/SSL security layer.
public FTPSClient makeFTPConnection(String vserver, int vport, String vuser, String vpassword) {
LOGGER.debug("ENTRY");
try {
ftpClient = new FTPSClient();
} catch (NoSuchAlgorithmException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
try {
ftpClient.connect(vserver, vport);
ftpClient.login(vuser, vpassword);
ftpClient.execPBSZ(0);
ftpClient.execPROT("P");
ftpClient.changeWorkingDirectory("/");
ftpClient.changeWorkingDirectory("/feeds");
ftpClient.enterLocalPassiveMode();
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
} catch (Exception e) {
LOGGER.error(e.getMessage());
}
/* // int reply=ftpClient.getReply();
String replyStr=ftpClient.getReplyString();
ftpClient.getAuthValue();*/
LOGGER.debug("EXIT");
return ftpClient;
}
Then after making the connection, we will check weather file exist at ftp or not.
public InputStream checkWOFileExistAtFTP(FTPSClient ftpClient, String host,String user, String filePath) throws IOException {
int returnCode;
// filePath="ftp://"+user+"#"+host+"/"+filePath;
InputStream inputStream = ftpClient.retrieveFileStream(filePath);
String dd=ftpClient.getReplyString();
returnCode = ftpClient.getReplyCode();
if (inputStream == null || returnCode == 550) {
return null;
}
return inputStream;
}
Now we already got the inputStream in above method now its time to read data from it.
BufferedReader br = new BufferedReader(new InputStreamReader(inputStream));
System.out.println("Reading file start.");
char[] charBuffer = new char[8 * 1024];
StringBuilder builder = new StringBuilder();
int numCharsRead;
while ((numCharsRead = br.read(charBuffer, 0, charBuffer.length)) != -1) {
builder.append(charBuffer, 0, numCharsRead);
}
//will print all data
system.out.println(builder.toString());

br.readline() gets stuck while br.read() works

I am making a simple ftp client/server program which on command from the clients lists files, tells the current directory, downloads files
My client code works fine since i have already tested it with a working server. However the server that i have designed gets stuck in the run() function on the line String message = br.readline(); If instead i use the br.read(), then it works but i need command in form of a string to know which file i have to download whereas br.read() returns int. Here's my code, i have used threading.
public class Myserver {
static final int PortNumber = 108;
static ServerSocket MyService;
static Socket clientSocket = null;
/**
* #param args
* #throws IOException
*/
public static void main(String[] args) throws IOException {
File directory;
directory = new File(System.getProperty("user.home"));
try {
MyService = new ServerSocket(PortNumber);
String cd = directory.toString();
System.out.println(cd);
System.out.println("Listening on " + PortNumber);
while(true) {
clientSocket = MyService.accept();
Connecthandle a = new Connecthandle(clientSocket, directory);
a.run();
}
}
catch (IOException e) {
System.out.println(e);
}
}
static class Connecthandle extends Thread {
File Directory;
Socket clientsocket;
// Constructor for class
Connecthandle(Socket clients, File dir) {
clientsocket = clients;
Directory = dir;
}
// Works Fine
void listfiles() throws IOException {
String []Listfile = Directory.list();
String send = "";
for (int j = 0; j < Listfile.length; j++) {
send = send + Listfile[j] + ",";
}
DataOutputStream GoingOut = new DataOutputStream(clientsocket.getOutputStream());
GoingOut.writeBytes(send);
GoingOut.flush();
GoingOut.close();
}
// Works Fine
void currentdirectory() throws IOException {
String cd = Directory.toString();
String cdd = "resp," + cd;
System.out.println(cdd);
DataOutputStream GoingOut = new DataOutputStream(clientsocket.getOutputStream());
GoingOut.writeBytes(cdd);
GoingOut.flush();
GoingOut.close();
System.exit(0);
}
void sendfiles(String fileName) {
try {
File nfile = new File(fileName);
DataOutputStream GoingOut = new DataOutputStream(clientsocket.getOutputStream());
if ( (! nfile.exists()) || nfile.isDirectory() ) {
GoingOut.writeBytes("file not present");
} else {
BufferedReader br = new BufferedReader(new FileReader(nfile));
String line;
while ((line = br.readLine()) != null) {
line = br.readLine();
GoingOut.writeBytes(line+"\n");
}
GoingOut.flush();
GoingOut.close();
br.close();
}
} catch (IOException e) {
System.out.println("Unable to send!");
}
}
#SuppressWarnings("deprecation")
public void run() {
try {
DataInputStream comingin = new DataInputStream(clientsocket.getInputStream());
InputStreamReader isr = new InputStreamReader(comingin, "UTF-8");
BufferedReader br = new BufferedReader(isr);
System.out.println("here");
// if (br.ready())
String message = br.readLine(); // Code gets stuck here, if i use br.read() it works, but i need string output.
if (message.equals("listfiles\n")) {
listfiles();
} else if (message.equals("pwd")) {
currentdirectory();
} else if (message.contains("getfile,")) {
String fileName = new String(message.substring(8, message.length()));
sendfiles(fileName);
}
} catch (Exception e) {
e.printStackTrace();
}
finally {
try {
clientsocket.close();
} catch (IOException e) {}
}
}
}
}

If readLine() is blocking and you are sending data, you aren't sending a newline.