I am trying to update a Confluence page using this code:
https://bitbucket.org/jaysee00/confluence-rest-api-example/src/master/src/main/java/com/atlassian/api/examples/Main.java
Code is:
public class Confluence {
/**
* Demonstrates how to update a page using the Conflunence 5.5 REST API.
*/
private static final Logger LOGGER = Logger.getLogger(Confluence.class);;
private static final String BASE_URL = "http://confluence:8080";
private static final String USERNAME = "admin";
private static final String PASSWORD = "admin";
private static final String ENCODING = "utf-8";
private String getContentRestUrl(Long contentId, String[] expansions)
throws UnsupportedEncodingException {
String expand = URLEncoder.encode(StringUtils.join(expansions, ","),
ENCODING);
return String
.format("%s/rest/api/content/%s?expand=%s&os_authType=basic&os_username=%s&os_password=%s",
BASE_URL, contentId, expand,
URLEncoder.encode(USERNAME, ENCODING),
URLEncoder.encode(PASSWORD, ENCODING));
}
public void publish() throws ClientProtocolException, IOException, Exception {
final long pageId = 36307446;
HttpClient client = new DefaultHttpClient();
// Get current page version
String pageObj = null;
HttpEntity pageEntity = null;
try {
String restUrl = getContentRestUrl(pageId,
new String[] { "body.storage", "version", "ancestors" });
HttpGet getPageRequest = new HttpGet(restUrl);
HttpResponse getPageResponse = client.execute(getPageRequest);
pageEntity = getPageResponse.getEntity();
pageObj = IOUtils.toString(pageEntity.getContent());
LOGGER.info("Get Page Request returned "
+ getPageResponse.getStatusLine().toString());
LOGGER.info(pageObj);
LOGGER.info((int)pageObj.trim().charAt(0));
} finally {
if (pageEntity != null) {
EntityUtils.consume(pageEntity);
}
}
// Parse response into JSON
JSONObject page = new JSONObject(pageObj.trim());
// Update page
// The updated value must be Confluence Storage Format
// NOT HTML.
page.getJSONObject("body").getJSONObject("storage")
.put("value", "hello, world");
int currentVersion = page.getJSONObject("version").getInt("number");
page.getJSONObject("version").put("number", currentVersion + 1);
// Send update request
HttpEntity putPageEntity = null;
try {
HttpPut putPageRequest = new HttpPut(getContentRestUrl(pageId,
new String[] {}));
StringEntity entity = new StringEntity(page.toString());
entity.setContentType("application/json");
putPageRequest.setEntity(entity);
HttpResponse putPageResponse = client.execute(putPageRequest);
putPageEntity = putPageResponse.getEntity();
System.out.println("Put Page Request returned "
+ putPageResponse.getStatusLine().toString());
System.out.println("");
System.out.println(IOUtils.toString(putPageEntity.getContent()));
} finally {
EntityUtils.consume(putPageEntity);
}
}
}
The response is alway 'HTTP 404 - Page not found'. I have changed the page id to one I know exists in Confluence.
An exception follows when it tries to parse the response into a JSON object:
avvvaorg.json.JSONException: A JSONObject text must begin with '{' at character 1
at org.json.JSONTokener.syntaxError(JSONTokener.java:496)
at org.json.JSONObject.<init>(JSONObject.java:180)
at org.json.JSONObject.<init>(JSONObject.java:403)
at com.openet.report.publish.Confluence.publish(Confluence.java:74)
at com.openet.report.miner.ReportMiner.generateSummary(ReportMiner.java:268)
at com.openet.report.miner.ReportMiner.runReport(ReportMiner.java:251)
at com.openet.report.miner.ReportMiner.main(ReportMiner.java:138)
Updating confluence pages using REST is not supported by Confluence 4.3.1. The API is much more limited:
https://docs.atlassian.com/atlassian-confluence/REST/4.3.1/
You can however update confluence using XML RPC:
public void publish() throws IOException {
DateFormat df = new SimpleDateFormat("MM/dd/yyyy HH:mm:ss");
Date today = Calendar.getInstance().getTime();
XWikiXmlRpcClient rpc = new XWikiXmlRpcClient(CONFLUENCE_URI);
try {
rpc.login(USER_NAME, PASSWORD);
//The info macro would get rendered an info box in the Page
Page page = new Page();
page.setSpace("Some space");
page.setTitle("Testing XML RPC calls in confluence_" + df.format(today));
//page.setContent(
String s = String.format("||Heading 1||Heading 2||Heading 3||%s|col A1|col A2|col A3|", "\r\n");
page.setContent(s);
page.setParentId(PAGEID);
rpc.storePage(page);
} catch (XmlRpcException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// TODO Auto-generated catch block
}
This requires the following libraries:
import org.apache.xmlrpc.XmlRpcException;
import org.codehaus.swizzle.confluence.Page;
import org.w3c.dom.Document;
import org.xwiki.xmlrpc.XWikiXmlRpcClient;
Note that these libraries are not in the standard maven repository. You will have to update your repository manager (artifactory in my case) to sync with the XWiki maven repo. You will also need the service rocket plugin (https://community.servicerocket.com/servicerocket/topics/the-license-could-not-be-verified-there-is-no-license-certificate-installed-for-customware-scaffolding-plugin-for-confluence) configured correctly on Confluence.
Related
I am looking for a way how to get app versionCode and VersionName from google play with package name via java app in PC.
I have seen: https://androidquery.appspot.com/ but it not working anymore and also https://code.google.com/archive/p/android-market-api/ started to making problems and also stopped working, and it requer device ID.
Can you help me with some simple solution or API for this?
Very important, i need versionCode and VersionName and VersionName is relatively easy to get by parsing html google play app site. The versionCode is very important.
There is no official Google Play API, Playstore uses an internal protobuf API which is not documented and not open. IMHO, you could :
use an open source library that reverse engineer the API
scrap apk download sites that have already extracted this information (most likely via the same protobuf Google Play API)
Note that there is a Google Play developer API but you can't list your apks, versions, apps. It's essentially used to manage the app distribution, reviews, edits etc..
Google play internal API
play-store-api Java library
This library uses Google Play Store protobuf API (undocumented and closed API) and requires an email/password to generate a token that can be reused to play with the API :
GplaySearch googlePlayInstance = new GplaySearch();
DetailsResponse response = googlePlayInstance.getDetailResponse("user#gmail.com",
"password", "com.facebook.katana");
AppDetails appDetails = response.getDocV2().getDetails().getAppDetails();
System.out.println("version name : " + appDetails.getVersionString());
System.out.println("version code : " + appDetails.getVersionCode());
with this method :
public DetailsResponse getDetailResponse(String email,
String password,
String packageName) throws IOException, ApiBuilderException {
// A device definition is required to log in
// See resources for a list of available devices
Properties properties = new Properties();
try {
properties.load(getClass().getClassLoader().getSystemResourceAsStream("device-honami" +
".properties"));
} catch (IOException e) {
System.out.println("device-honami.properties not found");
return null;
}
PropertiesDeviceInfoProvider deviceInfoProvider = new PropertiesDeviceInfoProvider();
deviceInfoProvider.setProperties(properties);
deviceInfoProvider.setLocaleString(Locale.ENGLISH.toString());
// Provide valid google account info
PlayStoreApiBuilder builder = new PlayStoreApiBuilder()
.setDeviceInfoProvider(deviceInfoProvider)
.setHttpClient(new OkHttpClientAdapter())
.setEmail(email)
.setPassword(password);
GooglePlayAPI api = builder.build();
// We are logged in now
// Save and reuse the generated auth token and gsf id,
// unless you want to get banned for frequent relogins
api.getToken();
api.getGsfId();
// API wrapper instance is ready
return api.details(packageName);
}
device-honami.properties is device property file that is required to identify device characteristics. You have some device.properties file sample here
The OkHttpClientAdapter can be found here
Dependencies used to run this example :
allprojects {
repositories {
...
maven { url 'https://jitpack.io' }
}
}
dependencies {
compile 'com.github.yeriomin:play-store-api:0.19'
compile 'com.squareup.okhttp3:okhttp:3.8.1'
}
Scrap third part apk download sites
http://apk-dl.com
You could get the version name & version code from http://apk-dl.com (of course unofficial) by scraping the page with jsoup for the required package name :
String packageName = "com.facebook.katana";
Document doc = Jsoup.connect("http://apk-dl.com/" + packageName).get();
Elements data = doc.select(".file-list .mdl-menu__item");
if (data.size() > 0) {
System.out.println("full text : " + data.get(0).text());
Pattern pattern = Pattern.compile("(.*)\\s+\\((\\d+)\\)");
Matcher matcher = pattern.matcher(data.get(0).text());
if (matcher.find()) {
System.out.println("version name : " + matcher.group(1));
System.out.println("version code : " + matcher.group(2));
}
}
https://apkpure.com
Another possibility is scrapping https://apkpure.com :
String packageName = "com.facebook.katana";
Elements data = Jsoup.connect("https://apkpure.com/search?q=" + packageName)
.userAgent("Mozilla")
.get().select(".search-dl .search-title a");
if (data.size() > 0) {
Elements data2 = Jsoup.connect("https://apkpure.com" + data.attr("href"))
.userAgent("Mozilla")
.get().select(".faq_cat dd p");
if (data2.size() > 0) {
System.out.println(data2.get(0).text());
Pattern pattern = Pattern.compile("Version:\\s+(.*)\\s+\\((\\d+)\\)");
Matcher matcher = pattern.matcher(data2.get(0).text());
if (matcher.find()) {
System.out.println("version name : " + matcher.group(1));
System.out.println("version code : " + matcher.group(2));
}
}
}
https://api-apk.evozi.com
Also, https://api-apk.evozi.com has an internal JSON api but :
sometimes it doesn't work (return Ops, APK Downloader got access denied when trying to download) mostly for non popular app
it has mechanism in place against scraping bot (random token generated in JS with a random variable name)
The following is returning the version name and code with https://api-apk.evozi.com FWIW :
String packageName = "com.facebook.katana";
String data = Jsoup.connect("https://apps.evozi.com/apk-downloader")
.userAgent("Mozilla")
.execute().body();
String token = "";
String time = "";
Pattern varPattern = Pattern.compile("dedbadfbadc:\\s+(\\w+),");
Pattern timePattern = Pattern.compile("t:\\s+(\\w+),");
Matcher varMatch = varPattern.matcher(data);
Matcher timeMatch = timePattern.matcher(data);
if (varMatch.find()) {
Pattern tokenPattern = Pattern.compile("\\s*var\\s*" + varMatch.group(1) + "\\s*=\\s*'(.*)'.*");
Matcher tokenMatch = tokenPattern.matcher(data);
if (tokenMatch.find()) {
token = tokenMatch.group(1);
}
}
if (timeMatch.find()) {
time = timeMatch.group(1);
}
HttpClient httpclient = HttpClients.createDefault();
HttpPost httppost = new HttpPost("https://api-apk.evozi.com/download");
List<NameValuePair> params = new ArrayList<NameValuePair>();
params.add(new BasicNameValuePair("t", time));
params.add(new BasicNameValuePair("afedcfdcbdedcafe", packageName));
params.add(new BasicNameValuePair("dedbadfbadc", token));
params.add(new BasicNameValuePair("fetch", "false"));
httppost.setEntity(new UrlEncodedFormEntity(params, "UTF-8"));
HttpResponse response = httpclient.execute(httppost);
JsonElement element = new JsonParser().parse(EntityUtils.toString(response.getEntity()));
JsonObject result = element.getAsJsonObject();
if (result.has("version") && result.has("version_code")) {
System.out.println("version name : " + result.get("version").getAsString());
System.out.println("version code : " + result.get("version_code").getAsInt());
} else {
System.out.println(result);
}
Implementation
You could implement it on a backend of yours that communicates directly with your Java application, this way you could maintain the process of retrieving version code/name if one of the above method fails.
If you are only interested in your own apps, a cleaner solution would be :
to set up a backend which will store all your current app version name / version code
all developer/publisher in your company could share a publish task (gradle task) which will use the Google Play developer API to publish apk and that gradle task would include a call to your backend to store the version code / version name entry when the app is published. The main goal would be to automate the whole publication with storage of the app metadata on your side.
Apart from using JSoup, we can alternatively do pattern matching for getting the app version from playStore.
To match the latest pattern from google playstore ie
<div class="BgcNfc">Current Version</div><span class="htlgb"><div><span class="htlgb">X.X.X</span></div>
we first have to match the above node sequence and then from above sequence get the version value. Below is the code snippet for same:
private String getAppVersion(String patternString, String inputString) {
try{
//Create a pattern
Pattern pattern = Pattern.compile(patternString);
if (null == pattern) {
return null;
}
//Match the pattern string in provided string
Matcher matcher = pattern.matcher(inputString);
if (null != matcher && matcher.find()) {
return matcher.group(1);
}
}catch (PatternSyntaxException ex) {
ex.printStackTrace();
}
return null;
}
private String getPlayStoreAppVersion(String appUrlString) {
final String currentVersion_PatternSeq = "<div[^>]*?>Current\\sVersion</div><span[^>]*?>(.*?)><div[^>]*?>(.*?)><span[^>]*?>(.*?)</span>";
final String appVersion_PatternSeq = "htlgb\">([^<]*)</s";
String playStoreAppVersion = null;
BufferedReader inReader = null;
URLConnection uc = null;
StringBuilder urlData = new StringBuilder();
final URL url = new URL(appUrlString);
uc = url.openConnection();
if(uc == null) {
return null;
}
uc.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; WindowsNT 5.1; en-US; rv1.8.1.6) Gecko/20070725 Firefox/2.0.0.6");
inReader = new BufferedReader(new InputStreamReader(uc.getInputStream()));
if (null != inReader) {
String str = "";
while ((str = inReader.readLine()) != null) {
urlData.append(str);
}
}
// Get the current version pattern sequence
String versionString = getAppVersion (currentVersion_PatternSeq, urlData.toString());
if(null == versionString){
return null;
}else{
// get version from "htlgb">X.X.X</span>
playStoreAppVersion = getAppVersion (appVersion_PatternSeq, versionString);
}
return playStoreAppVersion;
}
I got this solved through this. Hope that helps.
Jsoup takes too long, its inefficient, for short easy way with pattermatching:
public class PlayStoreVersionChecker {
public String playStoreVersion = "0.0.0";
OkHttpClient client = new OkHttpClient();
private String execute(String url) throws IOException {
okhttp3.Request request = new Request.Builder()
.url(url)
.build();
Response response = client.newCall(request).execute();
return response.body().string();
}
public String getPlayStoreVersion() {
try {
String html = execute("https://play.google.com/store/apps/details?id=" + APPIDHERE!!! + "&hl=en");
Pattern blockPattern = Pattern.compile("Current Version.*([0-9]+\\.[0-9]+\\.[0-9]+)</span>");
Matcher blockMatch = blockPattern.matcher(html);
if(blockMatch.find()) {
Pattern versionPattern = Pattern.compile("[0-9]+\\.[0-9]+\\.[0-9]+");
Matcher versionMatch = versionPattern.matcher(blockMatch.group(0));
if(versionMatch.find()) {
playStoreVersion = versionMatch.group(0);
}
}
} catch (IOException e) {
e.printStackTrace();
}
return playStoreVersion;
}
}
public class Store {
private Document document;
private final static String baseURL = "https://play.google.com/store/apps/details?id=";
public static void main(String[] args) {
}
public Store(String packageName) {
try {
document = Jsoup.connect(baseURL + packageName).userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:64.0) Gecko/20100101 Firefox/64.0").get();
} catch (IOException ex) {
ex.printStackTrace();
}
}
public String getTitle() {
return document.select("h1.AHFaub > span").text();
}
public String getDeveloper() {
return document.selectFirst("span.UAO9ie > a").text();
}
public String getCategory() {
Elements elements = document.select("span.UAO9ie > a");
for (Element element : elements) {
if (element.hasAttr("itemprop")) {
return element.text();
}
}
return null;
}
public String getIcon() {
return document.select("div.xSyT2c > img").attr("src");
}
public String getBigIcon() {
return document.select("div.xSyT2c > img").attr("srcset").replace(" 2x", "");
}
public List<String> getScreenshots() {
List<String> screenshots = new ArrayList<>();
Elements img = document.select("div.u3EI9e").select("button.Q4vdJd").select("img");
for (Element src : img) {
if (src.hasAttr("data-src")) {
screenshots.add(src.attr("data-src"));
} else {
screenshots.add(src.attr("src"));
}
}
return screenshots;
}
public List<String> getBigScreenshots() {
List<String> screenshots = new ArrayList<>();
Elements img = document.select("div.u3EI9e").select("button.Q4vdJd").select("img");
for (Element src : img) {
if (src.hasAttr("data-src")) {
screenshots.add(src.attr("data-srcset").replace(" 2x", ""));
} else {
screenshots.add(src.attr("srcset").replace(" 2x", ""));
}
}
return screenshots;
}
public String getDescription() {
return document.select("div.DWPxHb > span").text();
}
public String getRatings() {
return document.select("div.BHMmbe").text();
}
}
Imports
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
This script will return
Category (Personalization for example)
Developer Name
App Icon
App Name
Screenshots (Thumbnail and Full preview)
Description
You can also check the full source code here
First, I want to say thanks to everyone that took their time to help me figure this out because I was searching for more than a week for a solution to my problem. Here it is:
My goal is to start a custom workflow in Alfresco Community 5.2 and to set some custom properties in the first task trough a web script using only the Public Java API. My class is extending AbstractWebScript. Currently I have success with starting the workflow and setting properties like bpm:workflowDescription, but I'm not able to set my custom properties in the tasks.
Here is the code:
public class StartWorkflow extends AbstractWebScript {
/**
* The Alfresco Service Registry that gives access to all public content services in Alfresco.
*/
private ServiceRegistry serviceRegistry;
public void setServiceRegistry(ServiceRegistry serviceRegistry) {
this.serviceRegistry = serviceRegistry;
}
#Override
public void execute(WebScriptRequest req, WebScriptResponse res) throws IOException {
// Create JSON object for the response
JSONObject obj = new JSONObject();
try {
// Check if parameter defName is present in the request
String wfDefFromReq = req.getParameter("defName");
if (wfDefFromReq == null) {
obj.put("resultCode", "1 (Error)");
obj.put("errorMessage", "Parameter defName not found.");
return;
}
// Get the WFL Service
WorkflowService workflowService = serviceRegistry.getWorkflowService();
// Build WFL Definition name
String wfDefName = "activiti$" + wfDefFromReq;
// Get WorkflowDefinition object
WorkflowDefinition wfDef = workflowService.getDefinitionByName(wfDefName);
// Check if such WorkflowDefinition exists
if (wfDef == null) {
obj.put("resultCode", "1 (Error)");
obj.put("errorMessage", "No workflow definition found for defName = " + wfDefName);
return;
}
// Get parameters from the request
Content reqContent = req.getContent();
if (reqContent == null) {
throw new WebScriptException(Status.STATUS_BAD_REQUEST, "Missing request body.");
}
String content;
content = reqContent.getContent();
if (content.isEmpty()) {
throw new WebScriptException(Status.STATUS_BAD_REQUEST, "Content is empty");
}
JSONTokener jsonTokener = new JSONTokener(content);
JSONObject json = new JSONObject(jsonTokener);
// Set the workflow description
Map<QName, Serializable> params = new HashMap();
params.put(WorkflowModel.PROP_WORKFLOW_DESCRIPTION, "Workflow started from JAVA API");
// Start the workflow
WorkflowPath wfPath = workflowService.startWorkflow(wfDef.getId(), params);
// Get params from the POST request
Map<QName, Serializable> reqParams = new HashMap();
Iterator<String> i = json.keys();
while (i.hasNext()) {
String paramName = i.next();
QName qName = QName.createQName(paramName);
String value = json.getString(qName.getLocalName());
reqParams.put(qName, value);
}
// Try to update the task properties
// Get the next active task which contains the properties to update
WorkflowTask wfTask = workflowService.getTasksForWorkflowPath(wfPath.getId()).get(0);
// Update properties
WorkflowTask updatedTask = workflowService.updateTask(wfTask.getId(), reqParams, null, null);
obj.put("resultCode", "0 (Success)");
obj.put("workflowId", wfPath.getId());
} catch (JSONException e) {
throw new WebScriptException(Status.STATUS_BAD_REQUEST,
e.getLocalizedMessage());
} catch (IOException ioe) {
throw new WebScriptException(Status.STATUS_BAD_REQUEST,
"Error when parsing the request.",
ioe);
} finally {
// build a JSON string and send it back
String jsonString = obj.toString();
res.getWriter().write(jsonString);
}
}
}
Here is how I call the webscript:
curl -v -uadmin:admin -X POST -d #postParams.json localhost:8080/alfresco/s/workflow/startJava?defName=nameOfTheWFLDefinition -H "Content-Type:application/json"
In postParams.json file I have the required pairs for property/value which I need to update:
{
"cmprop:propOne" : "Value 1",
"cmprop:propTwo" : "Value 2",
"cmprop:propThree" : "Value 3"
}
The workflow is started, bpm:workflowDescription is set correctly, but the properties in the task are not visible to be set.
I made a JS script which I call when the workflow is started:
execution.setVariable('bpm_workflowDescription', 'Some String ' + execution.getVariable('cmprop:propOne'));
And actually the value for cmprop:propOne is used and the description is properly updated - which means that those properties are updated somewhere (on execution level maybe?) but I cannot figure out why they are not visible when I open the task.
I had success with starting the workflow and updating the properties using the JavaScript API with:
if (wfdef) {
// Get the params
wfparams = {};
if (jsonRequest) {
for ( var prop in jsonRequest) {
wfparams[prop] = jsonRequest[prop];
}
}
wfpackage = workflow.createPackage();
wfpath = wfdef.startWorkflow(wfpackage, wfparams);
The problem is that I only want to use the public Java API, please help.
Thanks!
Do you set your variables locally in your tasks? From what I see, it seems that you define your variables at the execution level, but not at the state level. If you take a look at the ootb adhoc.bpmn20.xml file (https://github.com/Activiti/Activiti-Designer/blob/master/org.activiti.designer.eclipse/src/main/resources/templates/adhoc.bpmn20.xml), you can notice an event listener that sets the variable locally:
<extensionElements>
<activiti:taskListener event="create" class="org.alfresco.repo.workflow.activiti.tasklistener.ScriptTaskListener">
<activiti:field name="script">
<activiti:string>
if (typeof bpm_workflowDueDate != 'undefined') task.setVariableLocal('bpm_dueDate', bpm_workflowDueDate);
if (typeof bpm_workflowPriority != 'undefined') task.priority = bpm_workflowPriority;
</activiti:string>
</activiti:field>
</activiti:taskListener>
</extensionElements>
Usually, I just try to import all tasks for my custom model prefix. So for you, it should look like that:
import java.util.Set;
import org.activiti.engine.delegate.DelegateExecution;
import org.activiti.engine.delegate.DelegateTask;
import org.apache.log4j.Logger;
public class ImportVariables extends AbstractTaskListener {
private Logger logger = Logger.getLogger(ImportVariables.class);
#Override
public void notify(DelegateTask task) {
logger.debug("Inside ImportVariables.notify()");
logger.debug("Task ID:" + task.getId());
logger.debug("Task name:" + task.getName());
logger.debug("Task proc ID:" + task.getProcessInstanceId());
logger.debug("Task def key:" + task.getTaskDefinitionKey());
DelegateExecution execution = task.getExecution();
Set<String> executionVariables = execution.getVariableNamesLocal();
for (String variableName : executionVariables) {
// If the variable starts by "cmprop_"
if (variableName.startsWith("cmprop_")) {
// Publish it at the task level
task.setVariableLocal(variableName, execution.getVariableLocal(variableName));
}
}
}
}
I am trying to programmatically search for a word meaning in google & save its meaning in a file in my computer. I have successfully called the page & get the response in Document (org.jsoup.nodes.Document). Now I do not know how to get only the word meaning from this Document. Please find the screenshot where I have indicated the part of data that I need.
The response html is so big that I can't understand from which element I will get my desired data. Please help. Here is what I have done so far:
public class Search {
private static Pattern patternDomainName;
private Matcher matcher;
private static final String DOMAIN_NAME_PATTERN
= "([a-zA-Z0-9]([a-zA-Z0-9\\-]{0,61}[a-zA-Z0-9])?\\.)+[a-zA-Z]{2,6}";
static {
patternDomainName = Pattern.compile(DOMAIN_NAME_PATTERN);
}
public static void main(String[] args) {
Search obj = new Search();
Set<String> result = obj.getDataFromGoogle("debug%20meaning");
for(String temp : result){
System.out.println(temp);
}
System.out.println(result.size());
}
public String getDomainName(String url){
String domainName = "";
matcher = patternDomainName.matcher(url);
if (matcher.find()) {
domainName = matcher.group(0).toLowerCase().trim();
}
return domainName;
}
private Set<String> getDataFromGoogle(String query) {
Set<String> result = new HashSet<String>();
String request = "https://www.google.com/search?q=" + query + "&num=20";
System.out.println("Sending request..." + request);
try {
// need http protocol, set this as a Google bot agent :)
Document doc = Jsoup
.connect(request)
.userAgent(
"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)")
.timeout(5000).get();
/**********Here comes my data fetching logic*****************
* Dont know where to fing my desired data in such a big html
*/
/*
String sc = doc.html().replaceAll("\\n", "");
System.out.println(doc.html());
*/
} catch (IOException e) {
e.printStackTrace();
}
return result;
}
}
Google Dictionary API is deprecated!
But instead scraping through google search URI,which is what you are doing currently, you can do the same thing using this http://google-dictionary.so8848.com/ service which preferably more easy to scrape data from, with what you are doing currently.
In GWT how do I display an image from the appengine server side blobstore given the string version of the key?
I think I have stored an image as a blob on the appengine. can someone tell me if it's correct?
try
{
FileService fileService = FileServiceFactory.getFileService();
AppEngineFile file = fileService.createNewBlobFile(content_type, fileName);
boolean lock = true;
FileWriteChannel writeChannel = fileService.openWriteChannel(file, lock);
byte[] b1 = new byte[BUFFER_SIZE];
int readBytes1;
while ((readBytes1 = is.read(b1)) != -1)
{
writeChannel.write(ByteBuffer.wrap(b1, 0, readBytes1));
}
writeChannel.closeFinally();
item_image_blob_key = fileService.getBlobKey(file).getKeyString();
}
catch (Exception e)
{
System.out.println(e.getLocalizedMessage());
e.printStackTrace(response.getWriter());
}
I sent the key back to the client and I am trying to present the image. I tried using :
ImagesService imagesService = ImagesServiceFactory
.getImagesService();
// Get the image serving URL
String imageUrl = imagesService.getServingUrl(blobKey);
but it is deprecated so I tried:
ImagesService imagesService = ImagesServiceFactory.getImagesService();
ServingUrlOptions suo = ServingUrlOptions.Builder.withBlobKey(blobKey);
String image_url = imagesService.getServingUrl(suo);
item.setProperty("image_url", image_url);
Now I get a URL which looks like this:
http://0.0.0.0:8888/_ah/img/5nXYgHwfiMmblDFldDXSew
and get create an image on the client thus:
String image_url = result.get_image_url();
System.out.println("image url is: " + image_url);
Image image = new Image();
image.setUrl(image_url);
RootPanel.get("dynamicDate").add(image);
Only a default image icon appears on the UI
so I created a servlet which accesses the blobstore thus:
import java.io.IOException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import com.google.appengine.api.blobstore.BlobKey;
import com.google.appengine.api.blobstore.BlobstoreService;
import com.google.appengine.api.blobstore.BlobstoreServiceFactory;
public class ImageServlet extends HttpServlet
{
private static final long serialVersionUID = 1L;
#Override
public void doGet(HttpServletRequest req, HttpServletResponse res) throws IOException
{
String blob_key = req.getParameter("blob-key");
if (blob_key != null)
{
BlobKey blobKey = new BlobKey(blob_key);
BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();
blobstoreService.serve(blobKey, res);
}
else
{
res.sendError(400, "One or more parameters are not set");
}
}
}
and a client http request:
RequestBuilder requestBuilder = new RequestBuilder(RequestBuilder.GET, "/itemmanager/image");
try
{
Request request = requestBuilder.sendRequest(null, new RequestCallback() {
public void onError(Request request, Throwable exception)
{
System.out.println(exception.getMessage());
}
public void onResponseReceived(Request request, Response response)
{
System.out.println("so far so good");
System.out.println(response.getHeadersAsString());
if (200 == response.getStatusCode())
{
}
else
{
// Handle the error. Can get the status text from response.getStatusText()
}
}
});
} catch (RequestException e) {
// Couldn't connect to server
}
I seem to be getting text how do i get an input stream or something i can get an image with?
Finally after two days of scouring Google and stack-overflow and trying I got it!
On the server I got the upload url thus:
ImagesService imagesService = ImagesServiceFactory.getImagesService();
ServingUrlOptions suo = ServingUrlOptions.Builder.withBlobKey(blobKey);
String image_url = imagesService.getServingUrl(suo);
item.setProperty("image_url", image_url);
The appengine API produced a url which didn't work on the local mode [I think it has to do with sop cross platform issues]
http://0.0.0.0:8888/_ah/img/mR9SOTSEizec4gZYsRnuEw
but it provided a clue: namely the /_ah/img/ part of the String
So I decided to try it and gave this URL to the image "/_ah/img/mR9SOTSEizec4gZYsRnuEw"
Here is the client side code.
String imageKey = result.get_image_key();
System.out.println("category is: " + result.get_category() + " image blob key is: " + imageKey);
String image_url = result.get_image_url();
System.out.println("image url is: " + image_url);
//This doesn't work at least locally
/*Image image = new Image();
image.setUrl(image_url);
RootPanel.get("dynamicDate").add(image);*/
Image image2 = new Image();
image2.setUrl("/_ah/img/" + imageKey);
RootPanel.get("dynamicDate").add(image2);
I have a project where you can ask for resources that are served by jax-rs in the json format. Everything works properly in the browser when I query the rest URL the json appears.
Now I want my GWT project to request those resources and process them and show them in my interface. The simplest way I found to do so is using a request builder and an overlay. Code is lower. The problem is, it seems when the code is running it never goes into the actual RequestCallback(). The status string is never changed. I thought it could be a SOP so I added the <add-linker name="xs"/> but still doesn't work. Any ideal?
package com.workoutcell.client;
//import com.google.gwt.core.client.JavaScriptObject;
import com.google.gwt.core.client.JsArray;
import com.google.gwt.http.client.*;
import com.google.gwt.http.client.Request;
import com.google.gwt.http.client.RequestBuilder;
import com.google.gwt.http.client.RequestCallback;
import com.google.gwt.http.client.RequestException;
import com.google.gwt.http.client.Response;
/**
*
* #author
*/
public class RestToInfoSession{
String queryReturn = null;
JsArray<InfoJSO> arrayOfInfo = null;
String host = "http://localhost:8080/mysite";
String restModule = "/calendar/getinfo";
String id = null;
String year = null;
String month = null;
String status = "Not Initialized";
public RestToInfoSession(String id, String year, String month){
this.id =id;
this.year = year;
this.month = month;
String url = host + restModule + "/"+this.id + "/"+this.year + "/"+this.month;
RequestBuilder builder = new RequestBuilder(RequestBuilder.GET, url);
try {
status = "Initialized at Url " + builder.getUrl();
Request request = builder.sendRequest(null, new RequestCallback() {
public void onError(Request request, Throwable exception) {
// Couldn't connect to server (could be timeout, SOP violation, etc.)
status = "Error on connecting to Server";
}
public void onResponseReceived(Request request, Response response) {
if (200 == response.getStatusCode()) {
// arrayOfInfo = jsonToJsArray(response.getText());
status = "JSON has been Fetched. Result is:" + response.getText();
} else if(0 == response.getStatusCode()) {
status = "Error is 0";
} else {
status = "Error in JSON Request:" + response.getStatusCode();
//response.getStatusText();
}
}
});
} catch (RequestException ex) {
status = "Error in Request Builder Startup";
}
}
//get an jso object in array
private final native JsArray<InfoJSO> jsonToJsArray(String json) /*-{
return eval(json);
}-*/;
public JsArray<InfoJSO> getInfoArray (){
return arrayOfInfo;
}
}
UPDATE: My problem is the same as Referring to a non-final variable data inside an inner class . I wasn't aware of asynchronous calls working mechanism. I still don't know how to pass my response.getText() to update a label that isn't part of my RestToInfoSession class any ideas?
Consider using the RestyGWT project. It will make calling JAXRS JSON resources as easy as using GWT-RPC. Plus you can typically reuse the same request response DTOs from the server side on the client side.
I have put a timer that checks every 1000ms if my json string has updated from null to the xhttp requested data. This works, but I got a feeling there is a more elegant way of resolving this problem.