I've studied up on the Oracle documentation and examples and still can't get this to work.
I have a Java Applet that is simply trying to send a text field to a PHP script via POST, using URLConnection and OutputStreamWriter. The Java side seems to work fine, no exceptions are thrown, but PHP is not showing any output on my page. I am a PHP noob so please bear with me on that part.
Here is the relevant Java portion:
try {
URL url = new URL("myphpfile.php");
URLConnection con = url.openConnection();
con.setDoOutput(true);
out = new OutputStreamWriter(con.getOutputStream());
String outstring = "field1=" + field1 + "&field2=" + field2;
out.write(outstring);
out.close();
}
catch (Exception e) {
System.out.println("HTTPConnection error: " + e);
return;
}
and here is the relevant PHP code:
<?php
$field1= $_POST['field1'];
$field2= $_POST['field2'];
print "<table><tr><th>Column1</th><th>Column2</th></tr><tr><td>" .
$field1 . "</td><td>" . $field2 . "</td></tr></table>";
?>
All I see are the table headers Column1 and Column2 (let's just keep these names generic for testing purposes). What am I doing wrong? Do I need to tell my PHP script to check when my Java code does the write?
Not USE $_POST ,USE $_REQUEST OR $_GET
WHERE TO SET $field1 and $field2 in your php script?
Try URL url = new URL("myphpfile.php?field1=" + field1 + "&field2=" + field2);
Well, I feel like I've tried every possible thing that can be tried with PHP, so I eventually went with JSObject. Now THAT was easy.
Working Java code:
JSObject window = JSObject.getWindow(this);
// invoke JavaScript function
String result = "<table><tr><th>Column1</th><th>Column2</th></tr><tr><td>"
+ field1 + "</td><td>" + field2 + "</td></tr></table>";
window.call("writeResult", new Object[] {result});
Relevant working Javascript:
function writeResult(result) {
var resultElem =
document.getElementById("anHTMLtagID");
resultElem.innerHTML = result;
}
From here I can even send the results from Javascript to PHP via Ajax to do database-related actions. Yay!
Related
First of all i am sorry if i am wrong that the response is fake JSON ...
the api i am using is ticker api of unocoin
https://www.unocoin.com/trade?all
I have been working on a website which takes the rate from various indian bitcoin exchanges and plot the graphs for easy visualization.So far i have added 3 exchanges and got their rate from their TICKER API,the response i got is just plane text and no other surprises..
all these exchanges like
ZEBPAY: https://www.zebapi.com/api/v1/market/ticker/btc/inr
Koinex: https://koinex.in/api/ticker
made my life easier but
making a get request to unocoin api gives me a html page with only an iframe in body tag and i am not able to directly(or indirectly) use data in my code.
there is an alternate method to get access to many features but it requires me to register and feed my ACCESS TOKEN in every request which i don't prefer right now.
to make api calls i am using java and code is given belowe:
private static String sendGet(String host,String apiEndpoint) throws Exception {
URL obj = new URL(host+apiEndpoint);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
// optional default is GET
con.setRequestMethod("GET");
//add request header
con.setRequestProperty("User-Agent", USER_AGENT);
int responseCode = con.getResponseCode();
System.out.println(responseCode);
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
return(response.toString());
}
just a note: i got google recaptcha if i make a lot of request in small time frame
the result from above code is
<html><head><META NAME="robots" CONTENT="noindex,nofollow"><script src="/_Incapsula_Resource?SWJIYLWA=2977d8d74f63d7f8fedbea018b7a1d05"></script><script>(function() { var z="";var b="7472797B766172207868723B76617220743D6E6577204461746528292E67657454696D6528293B766172207374617475733D227374617274223B7661722074696D696E673D6E65772041727261792833293B77696E646F772E6F6E756E6C6F61643D66756E6374696F6E28297B74696D696E675B325D3D22723A222B286E6577204461746528292E67657454696D6528292D74293B646F63756D656E742E637265617465456C656D656E742822696D6722292E7372633D222F5F496E63617073756C615F5265736F757263653F4553324C555243543D363726743D373826643D222B656E636F6465555249436F6D706F6E656E74287374617475732B222028222B74696D696E672E6A6F696E28292B222922297D3B69662877696E646F772E584D4C4874747052657175657374297B7868723D6E657720584D4C48747470526571756573747D656C73657B7868723D6E657720416374697665584F626A65637428224D6963726F736F66742E584D4C4854545022297D7868722E6F6E726561647973746174656368616E67653D66756E6374696F6E28297B737769746368287868722E72656164795374617465297B6361736520303A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2072657175657374206E6F7420696E697469616C697A656420223B627265616B3B6361736520313A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2073657276657220636F6E6E656374696F6E2065737461626C6973686564223B627265616B3B6361736520323A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2072657175657374207265636569766564223B627265616B3B6361736520333A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2070726F63657373696E672072657175657374223B627265616B3B6361736520343A7374617475733D22636F6D706C657465223B74696D696E675B315D3D22633A222B286E6577204461746528292E67657454696D6528292D74293B6966287868722E7374617475733D3D323030297B706172656E742E6C6F636174696F6E2E72656C6F616428297D627265616B7D7D3B74696D696E675B305D3D22733A222B286E6577204461746528292E67657454696D6528292D74293B7868722E6F70656E2822474554222C222F5F496E63617073756C615F5265736F757263653F535748414E45444C3D313539373232303738303038363836383835372C31313637303136303238393537363439373530392C373430383533373634363033313237303235322C353332303936222C66616C7365293B7868722E73656E64286E756C6C297D63617463682863297B7374617475732B3D6E6577204461746528292E67657454696D6528292D742B2220696E6361705F6578633A20222B633B646F63756D656E742E637265617465456C656D656E742822696D6722292E7372633D222F5F496E63617073756C615F5265736F757263653F4553324C555243543D363726743D373826643D222B656E636F6465555249436F6D706F6E656E74287374617475732B222028222B74696D696E672E6A6F696E28292B222922297D3B";for (var i=0;i<b.length;i+=2){z=z+parseInt(b.substring(i, i+2), 16)+",";}z = z.substring(0,z.length-1); eval(eval('String.fromCharCode('+z+')'));})();</script></head><body><iframe style="display:none;visibility:hidden;" src="//content.incapsula.com/jsTest.html" id="gaIframe"></iframe></body></html>
i just want the response just like i get in my browser after visiting
https://www.unocoin.com/trade?all
The website is protected by an anti-scraping script called Incapsula that tries to run a small Javascript bit, but since you are using Java it won't be able to run it, unless you are using Selenium or like the V8 engine, but this is a bit not recommended because you are somehow breaking the rules of what they considered to be intrusive for them, but my recommendation:
Talk with the guys from unocoin.com and ask them to whitelist your IP if they are okay with you scraping their site.
Instead of using the API, you can do it by scraping the Unocoin Ticker API All Rates webpage. This would break if there is some change in the website, but till then it works.
It can be implemented via WebKit using WKWebView, WKNavigationDelegate protocol and then injecting some JavaScript.
import UIKit
import WebKit
class ViewController: UIViewController, WKNavigationDelegate {
#IBOutlet weak var webView: WKWebView!
override func viewDidLoad() {
super.viewDidLoad()
webView.isHidden = true
webView.navigationDelegate = self
let myURL = URL(string: "https://www.unocoin.com/trade?all")
let myRequest = URLRequest(url: myURL!)
webView.load(myRequest)
}
// For checking if website has loaded
func webView(_ webView: WKWebView, didFinish navigation: WKNavigation!) {
// Injecting JS to fetch HTML inside <body>
webView.evaluateJavaScript("document.body.innerHTML", completionHandler: {
(html: Any?, error: Error?) in
if error == nil && html != nil {
// Perform string manipulation and parse JSON to get data
} else {
// Error while fetching data
}
})
}
}
I am creating a web application using the Spark Java framework. The front-end is developed using AngularJS.
I want to generate a .docx file on the server (in-memory) and send this to the client for download.
To achieve this I created an angular service with the following function being called after the user clicks on a download button:
functions.generateWord = function () {
$http.post('/api/v1/surveys/genword', data.currentSurvey).success(function (response) {
var element = angular.element('<a/>');
element.attr({
href: 'data:attachment;charset=utf-8;application/vnd.openxmlformats-officedocument.wordprocessingml.document' + response,
target: '_blank',
download: 'test.docx'
})[0].click();
});
};
On the server, this api call gets forwarded to the following method:
public Response exportToWord(Response response) {
try {
File file = new File("src/main/resources/template.docx");
FileInputStream inputStream = new FileInputStream(file);
byte byteStream[] = new byte[(int)file.length()];
inputStream.read(byteStream);
response.raw().setContentType("data:attachment;chatset=utf-8;application/vnd.openxmlformats-officedocument.wordprocessingml.document");
response.raw().setContentLength((int) file.length());
response.raw().getOutputStream().write(byteStream);
response.raw().getOutputStream().flush();
response.raw().getOutputStream().close();
return response;
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
I have tried to solve this in MANY different ways and I always end up with a corrupted 'test.docx' that looks like this:
Solved it by using blobs and specifying the response type as 'arraybuffer' in the $http.post api call. The only bad thing with this solution (as far as I know) is that it doesn't play well with IE, but that's a problem for another day.
functions.generateWord = function () {
$http.post('/api/v1/surveys/genword', data.currentSurvey, {responseType: 'arraybuffer'})
.success(function (response) {
var blob = new Blob([response], {type: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'});
var url = (window.URL || window.webkitURL).createObjectURL(blob);
var element = angular.element('<a/>');
element.attr({
href: url,
target: '_blank',
download: 'survey.docx'
})[0].click();
});
};
I think what went wrong was that the byte stream got encoded as plain text when I tried to create a URL with:
href: 'data:attachment;charset=utf-8;application/vnd.openxmlformats-officedocument.wordprocessingml.document' + response
thus corrupting it.
When using blobs instead, I get a "direct" link to the generated byte stream and no encoding is done on it since the response type is set to 'arraybuffer'.
Note that this is just my own reasoning of why things went wrong with the original code. I might be terribly wrong, so feel free to correct me if that's the case.
I'm building a simple news readers app and I am using HTMLCleaner to retrieve and parse the data. I've sucessfully gotten the data I need using the commandline version of HTMLCleaner and using xmllint for example:
java -jar htmlcleaner-2.6.jar src=http://www.reuters.com/home nodebyxpath=//div[#id=\"topStory\"]
and
curl www.reuters.com | xmllint --html --xpath //div[#id='"topStory"'] -
both return the data I want. Then when I try to make this request using HTMLCleaner in my code I get no results. Even more troubling is that even basic queries like //div only return 8 nodes in my app while command line reports 70+ which is correct.
Here is the code I have now. It is in an Android class extending AsyncTask so its performed in the background. The final code will actually get the text data I need but I'm having trouble just getting it to return a result. When I Log Title Node the node count is zero.
I've tried every manner of escaping the xpath query strings but it makes no difference.
The HTMLCleaner code is in a separate source folder in my project and is (at least I think) compiled to dalvik with the rest of my app so an incompatible jar file shouldn't be the problem.
I've tried to dump the HTMLCleaner file but it doesn't work well with LogCat and alot of the page markup is missing when I dump it which made me think that HTMLCleaner was parsing incorrectly and discarding most of the page but how can that be the case when the commandline version works fine?
Also the app does not crash and I'm not logging any exceptions.
protected Void doInBackground(URL... argv) {
final HtmlCleaner cleaner = new HtmlCleaner();
TagNode lNode = null;
try {
lNode = cleaner.clean( argv[0].openConnection().getInputStream() );
Log.d("LoadMain", argv[0].toString());
} catch (IOException e) {
Log.d("LoadMain", e.getMessage());
}
final String lTitle = "//div[#id=\"topStory\"]";
// final String lBlurp = "//div[#id=\"topStory\"]//p";
try {
Object[] x = lNode.evaluateXPath(lTitle);
// Object[] y = lNode.evaluateXPath(lBlurp);
Log.d("LoadMain", "Title Nodes: " + x.length );
// Log.d("LoadMain", "Title Nodes: " + y.length);
// this.mBlurbs.add(new BlurbView (this.mContext, x.getText().toString(), y.getText().toString() ));
} catch (XPatherException e) {
Log.d("LoadMain", e.getMessage());
}
return null;
}
Any help is greatly appreciated. Thank you.
UPDATE:
I've narrowed down the problem to being something to do with the http request. If I load the html source as an asset I get what I want so clearly the problem is in receiving the http request. In other words using lNode = cleaner.clean( getAssets().open("reuters.html") ); works fine.
Problem was that the http request was being redirected to the mobile website. This was solved by changing the User-Agent property like so.
private static final String USER_AGENT = "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0";
HttpURLConnection lConn = (HttpURLConnection) argv[0].openConnection();
lConn.setRequestProperty("User-Agent", USER_AGENT);
lConn.connect();
lNode = cleaner.clean( lConn.getInputStream() );
Here is my sample url:
url.com/data?format=json&pro={%22merchanturl%22:%22http://url.com/logo.png%22,%22price%22:599,%22productDesc%22:%22Apple%2032GBBlack%22,%22prodID%22:%22913393%22,%22merchant%22:%224536%22,%22prourl%22:%22http://url.com/data%22,%22name%22:%22Apple%2032GB%20%2D%20Black%22,%22productUrl%22:%22http://www.url.com/image.jpg%22,%22myprice%22:550,%22mercname%22:%22hello%22,%22mybool%22:false}
I have an android app. I need to post this url to server. So that server responds back with a token. I am doing the httppost through app. But I am not getting any response/exception. If I copy the same url and paste it in browser, that works very well. I hope I am doing mistake with the encoding part. Can anyone point out my issue?
Here is my encoding method:
private String encodeString(String input) {
String output = new String(input.trim().replace(" ", "%20")
.replace("&", "%26").replace(",", "%2c").replace("(", "%28")
.replace(")", "%29").replace("!", "%21").replace("=", "%3D")
.replace("<", "%3C").replace(">", "%3E").replace("#", "%23")
.replace("$", "%24").replace("'", "%27").replace("*", "%2A")
.replace("-", "%2D").replace(".", "%2E").replace("/", "%2F")
.replace(":", "%3A").replace(";", "%3B").replace("?", "%3F")
.replace("#", "%40").replace("[", "%5B").replace("\\", "%5C")
.replace("]", "%5D").replace("_", "%5F").replace("`", "%60")
.replace("{", "%7B").replace("|", "%7C").replace("}", "%7D")
.replace("\"", "%22"));
return output;
}
Update:
The reason why I am doing like this is, I need to send the data as in this format. The parameters part of the url is a json data. If I encode the complete url, that is not working.
Try using URLEncoder, encode only the part after ?
String query = URLEncoder.encode(queryPart, "utf-8");
String url = "http://server.com/search?q=" + query;
Although a self-written encoding isn't bad, I recommend using built-in Java methods that have been proven to be working.
TextUtils contains a method htmlEncode(String s) just for this.
http://developer.android.com/reference/android/text/TextUtils.html#htmlEncode%28java.lang.String%29
I'm not very well-versed in how the internets work, so I'm not really sure what this java method is doing or how to best replicate it in python. I have tried several different methods including urllibs and sockets, but nothing seems to work. The only time I get a response I end up getting a huge HTML document when the response should only be about 5 lines of xml.
Any help would be greatly appreciate, thanks guys :).
try {
URL url = new URL( sPROTOCOL, sHOSTNAME, sPAGENAME );
URLConnection url_con = url.openConnection();
url_con.setDoInput(true);
url_con.setDoOutput(true);
url_con.setUseCaches (false);
url_con.setRequestProperty ("content-type", "application/x-www-form-urlencoded");
String input_xml = make_XML( sAppID, sAppPassword, sUserID, sPassword );
if (bDEBUG) {
System.out.println( "\nINPUT XML------------------\n" + input_xml );
System.out.println( "\nEND INPUT XML--------------\n" );
}
BufferedWriter writebuf = new BufferedWriter(new OutputStreamWriter(url_con.getOutputStream()));
writebuf.write("XMLData=");
writebuf.write( URLEncoder.encode( input_xml, "UTF-8" ) ); //Java 1.4.x and later
//writebuf.write( URLEncoder.encode( input_xml ) ); //Java 1.3.1 and earlier
writebuf.flush();
writebuf.close();
writebuf = null;
HashMap hm = parseResp(url_con);
it looks like it's opening a connection to sHOSTNAME, sending the XML data generated by make_XML (apparently as a single POST parameter called XMLData, so sPROTOCOL must be HTTP), and then processing the response in parseResp.
in python you would use httplib. the final example at http://docs.python.org/library/httplib.html is doing something similar (but is sending three parameters). note that the code you posted is kind of ugly, in that it explicitly writes the POST contents - in python you would just give the XML as a parameter.