When I type this following URL into my browser, Bugzilla answers with XML:
http://bugzilla.mycompany.local/buglist.cgi?ctype=rdf&bug_status=CONFIRMED&product=MyProduct
I want to process this XML in a Java program. But when I use the exact same URL in my Java program, Bugzilla answers with HTML instead of XML.
This is my program:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
public class Test {
public static void main(String[] args)
throws IOException {
URL url = new URL("http://bugzilla.mycompany.local/buglist.cgi?ctype=rdf&bug_status=CONFIRMED&product=MyProduct");
URLConnection connection = url.openConnection();
final StringBuilder response = new StringBuilder(1024);
try(InputStreamReader isr = new InputStreamReader(connection.getInputStream())) {
try(BufferedReader reader = new BufferedReader(isr)) {
String inputLine = null;
while((inputLine = reader.readLine()) != null) {
response.append(inputLine);
response.append('\n');
}
}
}
System.out.println(response);
}
}
What am I doing wrong?
The resulting HTML is not the result of the query. It's Bugzillas log-in form. Duh!
Related
below code is for getting html web page
import java.net.*;
import java.io.*;
import java.io.File; // Import the File class
import java.io.IOException; // Import the IOException class to handle errors
public class TestClass2 {
public static void main(String[] args) throws Exception {
try{
URL url = new URL("https://stackoverflow.com/");
HttpURLConnection urlConnection=(HttpURLConnection)url.openConnection();
BufferedReader reader = new BufferedReader( new InputStreamReader(url.openStream()));
String line;
while ((line = reader.readLine()) != null)
{
System.out.println(line+"\n");
}
reader.close();
}catch(Exception ex){
System.out.println(ex);
}
}
}
but when compile and run that below error occur:
javax.net.ssl.SSLException: Received fatal alert: protocol_version.
how can fix it?
thanks.
This could be that the SSL Certificate is out of date? Have you tried using HttpsURLConnection? Try this first
Revised Code
import java.net.*;
import java.io.*;
import java.io.File; // Import the File class
import java.io.IOException; // Import the IOException class to handle errors
public class TestClass2 {
public static void main(String[] args) throws Exception {
try{
URL url = new URL("https://stackoverflow.com/");
HttpsURLConnection urlConnection=(HttpsURLConnection)url.openConnection();
BufferedReader reader = new BufferedReader( new InputStreamReader(url.openStream()));
String line;
while ((line = reader.readLine()) != null)
{
System.out.println(line+"\n");
}
reader.close();
}catch(Exception ex){
System.out.println(ex);
}
}
}
I am practising JUnit test cases and currently working on a problem which is as follows:
To read HTML from any website say "http://www.google.com" ( Candidate can use any API of inbuilt APIs in Java like URLConnection ).
Print on console the HTML from the URL above and save it to a file ( web-content.txt) in local machine.
Write JUnit test cases for the above program.
I've successfully achieved first steps but when I am running JUnit Test Case its showing Failure.
ReadFile.java
package com.test;
import java.io.BufferedInputStream;
import java.io.BufferedReader;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.Reader;
import java.net.HttpURLConnection;
import java.net.URL;
public class ReadFile
{
static void display(String input,OutputStream fos)
{
try
{
URL url = new URL(input);
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
InputStream stream = new BufferedInputStream(urlConnection.getInputStream());
Reader reader = new InputStreamReader(stream);
int data=0;
while((data=reader.read())!=-1)
{
System.out.print((char)data);
fos.write((char)data);
}
}
catch(Exception e)
{
System.out.println(e);
}
}
public static void main(String[] args)
{
BufferedReader reader = new BufferedReader(new InputStreamReader(System.in));
String input =null;
FileOutputStream fos =null;
System.out.println("Please enter any url");
try
{
input = reader.readLine();
fos = new FileOutputStream("src/web-context.txt");
display(input,fos);
}
catch(Exception e)
{
System.out.println(e);
}
}
}
ReadFileTest.java
package com.test;
import static org.junit.Assert.*;
import java.io.ByteArrayOutputStream;
import org.junit.Test;
public class ReadFileTest {
#Test
public void test() {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ReadFile.display("http://google.co.in", baos);
assertTrue(baos.toString().contains("http://google.co.in"));
}
}
I am getting following error while running JUnit Test in Eclipse:
java.lang.AssertionError
java.lang.AssertionError at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertTrue(Assert.java:52) at com.test.ReadFileTest.test(ReadFileTest.java:15) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown
I want that the JUnit Test Case will return true.
What's not working here is :
assertTrue(baos.toString().contains("http://google.co.in"));
and what would work is
assertTrue(baos.toString().contains("google.co.in")); // note the difference
Make something like that:
static String display(String input) {
try {
URL url = new URL(input);
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
InputStream stream = new BufferedInputStream(urlConnection.getInputStream());
Reader reader = new InputStreamReader(stream);
int data = 0;
StringBuilder builder = new StringBuilder();
while ((data = reader.read()) != -1) {
builder.append((char) data);
}
return builder.toString();
} catch(Exception e) {
e.printStackTrace();
return null;
}
}
I don't know why you use ByteArrayOutputStream
And now for your test case:
#Test
public void test() {
String data = ReadFile.display("http://google.co.in");
assertTrue(data != null);
assertTrue(data.contains("http://google.co.in"));
}
In the following code, the content of HTML is displayed in the console. What I want to do is how can I just show the content of some part of the HTML, for example the HTML content of stock prices?
import java.io.InputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLConnection;
import java.util.Scanner;
public class ShowStock {
public static void main(String[] args) throws IOException {
String urlString;
if(args.length == 1)
urlString = args[0];
else
{
urlString = "https://www.google.com/finance/historical?cid=22144&startdate=Jan+1%2C+2014&enddate=Dec+31%2C+2015&num=30&ei=m-JzVqm2L9fJUaOphsAF";
System.out.println("Reading data from " + urlString );
}
// Open connection
URL u = new URL(urlString);
URLConnection connection = u.openConnection();
// check to make sure the page exists
HttpURLConnection httpConnection = (HttpURLConnection) connection;
int code = httpConnection.getResponseCode();
String message = httpConnection.getResponseMessage();
System.out.println(code + " " + message);
if (code != HttpURLConnection.HTTP_OK)
return;
// Read server response
InputStream instream = connection.getInputStream();
Scanner in = new Scanner(instream);
// display server response to console
while (in.hasNextLine())
{
String input = in.nextLine();
System.out.println(input);
}
}
}
If it is XHTML (html like xml), you can use many xml libraries
If not, use an html parser jsoup, htmlcleaner, ...
see this:
Which HTML Parser is the best?
I am currently trying to build a java webapp where I would have to retrieve SOME information (a few fields in a HTML table) from a page, but after login. I do have a basic java code to return the html code of a webpage, but what I am looking for is to be able to Log in, and then parse the text from a URL (certain HTML fields), which otherwise would not be accessible. I have looked into tools like Selenium WebDriver and jTidy, but could not find what I was looking for.
Example code I use to get HTML code:
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
public class WebCrawler{
public static void main(String[] args) {
try {
URL google = new URL("http://stackoverflow.com");
URLConnection yc = google.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(yc.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null) {
System.out.println(inputLine);
}
in.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
I have been working with java to make it where i check if a certain user if live and it will say true or false if the user is streaming...im working with minimal json.
here is my code
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
import com.eclipsesource.json.JsonObject;
public class hostbot {
public static void main(String[] args) throws Exception {
Twitchbot bot = new Twitchbot();
bot.setVerbose(true);
bot.connect("irc.twitch.tv", 6667, "something");
}
public boolean isStreamLive()
{
try
{
URL url = new URL("https://api.twitch.tv/kraken/streams/rexephon");
URLConnection conn = url.openConnection();
BufferedReader br = new BufferedReader( new InputStreamReader( conn.getInputStream() ));
String inputLine = br.readLine();
br.close();
JsonObject jsonObj = JsonObject.readFrom(inputLine);
return ( jsonObj.get("stream").isNull() )?false:true;
}
catch (IOException e)
{
e.printStackTrace();
}
return false;
}
}
when i return false is that suppose to print in the log the word false? or something else?