I am trying to create a simple command line program that will determine if a playlist is a media playlist or master based on the tag returned. Unfortunately both type of playlist first line tags are the same so I was wondering is their a way I could adjust my code to read the text starting at the second line?
private static String getPlaylistUrl(String theUrl) throws
FileNotFoundException, MalformedURLException, IOException{
String content = "";
//Creates a url variable
URL url = new URL(theUrl);
//Cretes a urlConnection variable
URLConnection urlConnection = (HttpURLConnection) url.openConnection();
//Wraps the urlConnection in a BufferedReader
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
String line;
while ((line = bufferedReader.readLine()) != null) {
content += line + "\n";
}
bufferedReader.close();
return content;
}
Just read the first line before the loop starts.
private static String getPlaylistUrl(String theUrl) throws IOException {
try (InputStream is = new URL(theUrl).openConnection().getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
Stream<String> stream = reader.lines()) {
return stream
// skip the first line
.skip(1)
// join all other lines using a new line delimiter
.collect(Collectors.joining("\n"));
}
}
Skip the header like this
String line;
bool IsHeader=true;
while ((line = bufferedReader.readLine()) != null) {
if (IsHeader){
IsHeader=false; //skip header..
}else{
content += line + "\n";
}
}
bufferedReader.close();
Related
I've modified the following code from an example on the internet. Currently it POSTs and returns the response line by line. How can I modify the code so it returns the entire response in one line, so I can parse it more easily.
static void updateIp() throws MalformedURLException, IOException {
String urlParameters = "name=sub&a=rec_edit&id=9001";
URL url = new URL("http://httpbin.org/post");
URLConnection con = url.openConnection();
con.setDoOutput(true);
BufferedReader reader;
try (OutputStreamWriter writer = new OutputStreamWriter(con.getOutputStream())) {
writer.write(urlParameters);
writer.flush();
String line;
reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
}
reader.close();
}
Any help would be greatly appreciated!
You can't determine how many lines the URL response will be over, so you need to join them all together yourself in one line using StringBuilder:
static void updateIp() throws MalformedURLException, IOException {
String urlParameters = "name=sub&a=rec_edit&id=9001";
URL url = new URL("http://httpbin.org/post");
URLConnection con = url.openConnection();
con.setDoOutput(true);
BufferedReader reader;
try (OutputStreamWriter writer = new OutputStreamWriter(con.getOutputStream())) {
writer.write(urlParameters);
writer.flush();
String line;
StringBuilder urlResponse = new StringBuilder();
reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
while ((line = reader.readLine()) != null) {
urlResponse.append(line);
}
String response = urlResponse.toString();
System.out.println(response);
}
reader.close();
}
The response string variable will now contain all the output in a single line.
I am having trouble getting the html text from this html file via ftp. I use beautiful soup to read an html file via http/https but for some reason I cannot download/read from an ftp. Please help!
Here is the url.
a link
Here is my code so far.
BufferedReader reader = null;
String total = "";
String line;
ur = "ftp://ftp.legis.state.tx.us/bills/832/billtext/html/house_resolutions/HR00001_HR00099/HR00014I.htm"
try {
URL url = new URL(ur);
URLConnection urlc = url.openConnection();
InputStream is = urlc.getInputStream(); // To download
reader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
while ((line = reader.readLine()) != null)
total += reader.readLine();
} finally {
if (reader != null)
try { reader.close();
} catch (IOException logOrIgnore) {}
}
This code working for me, Java 1.7.0_25. Notice that you were storing one of every two lines, calling reader.readLine() both in the condition and in the body of the while loop.
public static void main(String[] args) throws MalformedURLException, IOException {
BufferedReader reader = null;
String total = "";
String line;
String ur = "ftp://ftp.legis.state.tx.us/bills/832/billtext/html/house_resolutions/HR00001_HR00099/HR00014I.htm";
try {
URL url = new URL(ur);
URLConnection urlc = url.openConnection();
InputStream is = urlc.getInputStream(); // To download
reader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
while ((line = reader.readLine()) != null) {
total += line;
}
} finally {
if (reader != null) {
try {
reader.close();
} catch (IOException logOrIgnore) {
}
}
}
}
First thought this is related to a wrong path resolution as discussed here but this does not help.
I don't know what is exactly going wrong here but I can only reproduce this error on this ftp-server and with the MacOS Java 1.6.0_33-b03-424. I can't reproduce it with Java 1.7.0_25. So perhaps you check for a Java update.
Or you could use commons FTPClient to retrieve the file:
FTPClient client = new FTPClient();
client.connect("ftp.legis.state.tx.us");
client.enterLocalPassiveMode();
client.login("anonymous", "");
client.changeWorkingDirectory("bills/832/billtext/html/house_resolutions/HR00001_HR00099");
InputStream is = client.retrieveFileStream("HR00014I.htm");
I have strange problem with BufferedReader reading from web.
This URL content is different in browsers than in pasted Java code.
In content fetched using Java first elements result is empty in browser it is not.
My code:
public static void main(String[] args) {
try {
String url = "https://api.freebase.com/api/service/mqlread?queries={\"q1\":{\"query\":[{\"name\":\"Pulp Fiction\",\"*\":null,\"type\":\"/film/film\"}]},\"q3\":{\"query\":[{\"name\":\"Portal\",\"*\":null,\"type\":\"/cvg/computer_videogame\"}]}}";
URL u = new URL(url);
System.out.println(u.toString());
URLConnection urlConn = u.openConnection();
InputStreamReader is = new InputStreamReader(urlConn.getInputStream());
BufferedReader br = new BufferedReader(is);
String line = null;
String data = "";
while ((line = br.readLine()) != null) {
data += line + "\n";
}
br.close();
System.out.println(data);
} catch (Exception ex) {
System.err.println(ex);
}
}
EDIT: Ahh. Figured it out. No space characters in URLs. Just replace them with %20.
I'm trying to read from a text/plain file over the internet, line-by-line. The code I have right now is:
URL url = new URL("http://kuehldesign.net/test.txt");
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
LinkedList<String> lines = new LinkedList();
String readLine;
while ((readLine = in.readLine()) != null) {
lines.add(readLine);
}
for (String line : lines) {
out.println("> " + line);
}
The file, test.txt, contains ¡Hélló!, which I am using in order to test the encoding.
When I review the OutputStream (out), I see it as > ¬°H√©ll√≥!. I don't believe this is a problem with the OutputStream since I can do out.println("é"); without problems.
Any ideas for reading form the InputStream as UTF-8? Thanks!
Solved my own problem. This line:
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
needs to be:
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream(), "UTF-8"));
or since Java 7:
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream(), StandardCharsets.UTF_8));
String file = "";
try {
InputStream is = new FileInputStream(filename);
String UTF8 = "utf8";
int BUFFER_SIZE = 8192;
BufferedReader br = new BufferedReader(new InputStreamReader(is,
UTF8), BUFFER_SIZE);
String str;
while ((str = br.readLine()) != null) {
file += str;
}
} catch (Exception e) {
}
Try this,.. :-)
I ran into the same problem every time it finds a special character marks it as ��. to solve this, I tried using the encoding: ISO-8859-1
BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream("txtPath"),"ISO-8859-1"));
while ((line = br.readLine()) != null) {
}
I hope this can help anyone who sees this post.
If you use the constructor InputStreamReader(InputStream in, Charset cs), bad characters are silently replaced. To change this behaviour, use a CharsetDecoder :
public static Reader newReader(Inputstream is) {
new InputStreamReader(is,
StandardCharsets.UTF_8.newDecoder()
.onMalformedInput(CodingErrorAction.REPORT)
.onUnmappableCharacter(CodingErrorAction.REPORT)
);
}
Then catch java.nio.charset.CharacterCodingException.
I want to ask a question about Java. I have use the URLConnection in Java to retrieve the DataInputStream. and I want to convert the DataInputStream into a String variable in Java. What should I do? Can anyone help me. thank you.
The following is my code:
URL data = new URL("http://google.com");
URLConnection dataConnection = data.openConnection();
DataInputStream dis = new DataInputStream(dataConnection.getInputStream());
String data_string;
// convent the DataInputStream to the String
import java.net.*;
import java.io.*;
class ConnectionTest {
public static void main(String[] args) {
try {
URL google = new URL("http://www.google.com/");
URLConnection googleConnection = google.openConnection();
DataInputStream dis = new DataInputStream(googleConnection.getInputStream());
StringBuffer inputLine = new StringBuffer();
String tmp;
while ((tmp = dis.readLine()) != null) {
inputLine.append(tmp);
System.out.println(tmp);
}
//use inputLine.toString(); here it would have whole source
dis.close();
} catch (MalformedURLException me) {
System.out.println("MalformedURLException: " + me);
} catch (IOException ioe) {
System.out.println("IOException: " + ioe);
}
}
}
This is what you want.
You can use commons-io IOUtils.toString(dataConnection.getInputStream(), encoding) in order to achieve your goal.
DataInputStream is not used for what you want - i.e. you want to read the content of a website as String.
If you want to read data from a generic URL (such as www.google.com), you probably don't want to use a DataInputStream at all. Instead, create a BufferedReader and read line by line with the readLine() method. Use the URLConnection.getContentType() field to find out the content's charset (you will need this in order to create your reader properly).
Example:
URL data = new URL("http://google.com");
URLConnection dataConnection = data.openConnection();
// Find out charset, default to ISO-8859-1 if unknown
String charset = "ISO-8859-1";
String contentType = dataConnection.getContentType();
if (contentType != null) {
int pos = contentType.indexOf("charset=");
if (pos != -1) {
charset = contentType.substring(pos + "charset=".length());
}
}
// Create reader and read string data
BufferedReader r = new BufferedReader(
new InputStreamReader(dataConnection.getInputStream(), charset));
String content = "";
String line;
while ((line = r.readLine()) != null) {
content += line + "\n";
}