We Keep Coding

Deserialising string list in GSON?

I am trying to save a string list in a jsonobject and deserialise it on load. When I am saving the list, I am creating a JsonObject and adding it as a property by using the GSON.toJson. I have created a String arraylist serialiser to serialise the item and when I check my mongo database it appears to be saving as a jsonArray but when I try to deserialise it, it tells me that it is not a JsonArray.
Here is my code:
Order object:
import lombok.Getter;
import lombok.RequiredArgsConstructor;
import lombok.Setter;
import java.time.LocalDate;
import java.util.ArrayList;
import java.util.List;
import java.util.UUID;
#Getter
#Setter
#RequiredArgsConstructor
public class Order {
private final UUID id;
private final String table;
private short numberOfPeople;
private LocalDate timeOfOrder = LocalDate.now();
private String staffInCharge = "";
private List<String> order = new ArrayList<>();
private String note = "";
}
The #RequiredArgumentsConstructor just creates a constructor for me so I dont have to do public Order() etc
Imports of the Serialise and Deserialise class:
import com.google.gson.*;
import lombok.Getter;
import lombok.RequiredArgsConstructor;
import me.oscar.orderbird.OrderBird;
import me.oscar.orderbird.gson.GsonUtil;
import me.oscar.orderbird.mongo.MongoUtils;
import me.oscar.orderbird.profile.food.Food;
import me.oscar.orderbird.profile.order.Order;
import me.oscar.orderbird.profile.staff.Staff;
import me.oscar.orderbird.profile.table.Table;
import org.bson.Document;
import java.util.*;
Serialise:
if (!this.orders.isEmpty()){
JsonArray orderArray = new JsonArray();
this.orders.values().forEach(order -> {
JsonObject ordersObect = new JsonObject();
ordersObect.addProperty("id", order.getId().toString());
ordersObect.addProperty("name", order.getTable());
ordersObect.addProperty("note", order.getNote());
ordersObect.addProperty("numberOfPeople", order.getNumberOfPeople());
ordersObect.addProperty("staffInCharge", order.getStaffInCharge());
ordersObect.addProperty("orderItems", GsonUtil.GSON.toJson(order.getOrder()));
orderArray.add(ordersObect);
});
document.put("orders", orderArray.toString());
}
Deserialise:
if (document.containsKey("orders")) {
if (document.get("orders") instanceof String) {
JsonArray ordersArray = PARSER.parse(document.getString("orders")).getAsJsonArray();
for (JsonElement jsonElement : ordersArray) {
JsonObject jsonObject = jsonElement.getAsJsonObject();
try {
Order order = new Order(UUID.fromString(jsonObject.get("id").getAsString()), jsonObject.get("name").getAsString());
order.setNote(jsonObject.get("note").getAsString());
order.setNumberOfPeople(jsonObject.get("numberOfPeople").getAsShort());
order.setStaffInCharge(jsonObject.get("staffInCharge").getAsString());
order.setOrder(GsonUtil.GSON.fromJson(jsonObject.get("orderItems").getAsJsonObject(), ArrayList.class));
this.orders.put(jsonObject.get("name").getAsString(), order);
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
}
ArrayListAdapter:
package me.oscar.orderbird.gson.impl;
import com.google.gson.*;
import java.lang.reflect.Type;
import java.util.ArrayList;
import java.util.List;
public class StringArrayAdapter implements JsonDeserializer<List<String>>, JsonSerializer<List<String>> {
#Override
public List<String> deserialize(JsonElement jsonElement, Type type, JsonDeserializationContext jsonDeserializationContext) throws JsonParseException {
JsonArray jsonArray = jsonElement.getAsJsonArray();
List<String> values = new ArrayList<>();
for (int i = 0; i < jsonArray.size(); i++){
values.set(i, jsonArray.get(i).getAsJsonPrimitive().getAsString());
}
return values;
}
#Override
public JsonElement serialize(List<String> stringList, Type type, JsonSerializationContext jsonSerializationContext) {
JsonArray jsonArray = new JsonArray();
for (String string : stringList) {
jsonArray.add(new JsonPrimitive(string));
}
return jsonArray;
}
}
When I attempt to set the order I am getting the error:
java.lang.IllegalStateException: Not a JSON Object: "[\n \"Steak\"\n]"

There are two issues with your implementation
While changing from Object to Document, you need to preserve property class type for orderItems
change
ordersObect.addProperty("orderItems", GsonUtil.GSON.toJson(order.getOrder()));
TO
ordersObect.addProperty("orderItems", GSON.toJson(order.getOrder(), ArrayList.class));
While reading from Document, need to read as the same class type
change
GsonUtil.GSON.fromJson(jsonObject.get("orderItems").getAsJsonObject(), ArrayList.class)
TO
GSON.fromJson(jsonObject.get("orderItems").getAsString(),ArrayList.class)
So working code is as below:
import com.Order;
import com.StringArrayAdapter;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import org.bson.Document;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;
JsonParser PARSER = new JsonParser();
Gson GSON = new GsonBuilder()
.registerTypeHierarchyAdapter(ArrayList.class, new StringArrayAdapter())
.setPrettyPrinting()
.serializeNulls()
.create();
Document document = new Document();
Map<String, Order> orders = new HashMap<>();
for (int i = 0; i < 2; i++) {
Order order = new Order(UUID.randomUUID(), "" + i);
orders.put(order.getTable(), order);
}
// SO Post Code
if (!orders.isEmpty()) {
JsonArray orderArray = new JsonArray();
orders.values().forEach(order -> {
JsonObject ordersObect = new JsonObject();
ordersObect.addProperty("id", order.getId().toString());
ordersObect.addProperty("name", order.getTable());
ordersObect.addProperty("note", order.getNote());
ordersObect.addProperty("numberOfPeople", order.getNumberOfPeople());
ordersObect.addProperty("staffInCharge", order.getStaffInCharge());
ordersObect.addProperty("orderItems", GSON.toJson(order.getOrder(), ArrayList.class));
orderArray.add(ordersObect);
});
document.put("orders", orderArray.toString());
}
System.out.println("document: " + document);
Map<String, Order> readOrders = new HashMap<>();
if (document.containsKey("orders")) {
if (document.get("orders") instanceof String) {
JsonArray ordersArray = PARSER.parse(document.getString("orders")).getAsJsonArray();
for (JsonElement jsonElement : ordersArray) {
JsonObject jsonObject = jsonElement.getAsJsonObject();
try {
Order order = new Order(UUID.fromString(jsonObject.get("id").getAsString()), jsonObject.get("name").getAsString());
order.setNote(jsonObject.get("note").getAsString());
order.setNumberOfPeople(jsonObject.get("numberOfPeople").getAsShort());
order.setStaffInCharge(jsonObject.get("staffInCharge").getAsString());
order.setOrder((ArrayList<String>)GSON.fromJson(jsonObject.get("orderItems").getAsString(), ArrayList.class));
readOrders.put(jsonObject.get("name").getAsString(), order);
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
}
System.out.println("Orders read : " + readOrders);
OUTPUT:
document: Document{{orders=[{"id":"e853617a-b516-4daf-ba6c-e5d731c556de","name":"0","note":"","numberOfPeople":0,"staffInCharge":"","orderItems":"[]"},{"id":"31004bbd-164d-4e64-b468-7021e1c21c39","name":"1","note":"","numberOfPeople":0,"staffInCharge":"","orderItems":"[]"}]}}
Orders read : {0=Order(id=e853617a-b516-4daf-ba6c-e5d731c556de, table=0, numberOfPeople=0, timeOfOrder=2021-06-16, staffInCharge=, order=[], note=), 1=Order(id=31004bbd-164d-4e64-b468-7021e1c21c39, table=1, numberOfPeople=0, timeOfOrder=2021-06-16, staffInCharge=, order=[], note=)}

Yandex Api not translating properly in eclipse

Here is my code in TranslateAPI.java:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLConnection;
import java.util.HashMap;
import java.util.Map;
public class TranslateAPI {
public static final String API_KEY = "pdct.1.1.20180924T090857Z.3e14b8b207704aef.9bdc409229b123003526815bb7062ed42616f26a";
private static String request(String URL) throws IOException {
URL url = new URL(URL);
HttpURLConnection urlConn = (HttpURLConnection) url.openConnection();
InputStream inStream = urlConn.getInputStream();
String recieved = new BufferedReader(new InputStreamReader(inStream)).readLine();
System.setProperty("http.agent", "Chrome");
String agent = java.security.AccessController.doPrivileged(new sun.security.action.GetPropertyAction("http.agent"));
inStream.close();
return recieved;
}
public static Map<String, String> getLangs() throws IOException {
String langs = request("https://translate.yandex.net/api/v1.5/tr.json/getLangs?key=" + API_KEY + "&ui=en");
langs = langs.substring(langs.indexOf("langs")+7);
langs = langs.substring(0, langs.length()-1);
String[] splitLangs = langs.split(",");
Map<String, String> languages = new HashMap<String, String>();
for (String s : splitLangs) {
String[] s2 = s.split(":");
String key = s2[0].substring(1, s2[0].length()-1);
String value = s2[1].substring(1, s2[1].length()-1);
languages.put(key, value);
}
return languages;
}
public static String translate(String text, String sourceLang, String targetLang) throws IOException {
String response = request("https://translate.yandex.net/api/v1.5/tr.json/translate?key=" + API_KEY + "&text=" + text + "&lang=" + sourceLang + "-" + targetLang);
return response.substring(response.indexOf("text")+8, response.length()-3);
}
AND in workerthread.java:
String s=TranslateAPI.detectLanguage(abc);
System.out.println(s);
However,I am getting the follwing errror:
Server returned HTTP response code: 403 for URL: https://translate.yandex.net/api/v1.5/tr.json/detect?key=pdct.1.1.20180924T090857Z.3e14b8b207704aef.9bdc409229b123003526815bb7062ed42616f26a&text=cat
Can you please help? Thanks in advance

You are getting a 401 Error thus Your API Key is Invalid,
You can always go to Yandex's Developers page to get a new one.
Its always great to publish your private API here :D

Deploy a .bpmn file via REST

i want to deploy a BPMN-file with the REST-API of Activiti. But i always get a Bad Request (400) error...
Has anybody an idea what i'm doing wrong??? I use restlet to upload my code. My code is below.
Thank your very much :)
import java.io.File;
import java.io.IOException;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
import org.restlet.data.ChallengeScheme;
import org.restlet.data.Disposition;
import org.restlet.data.Form;
import org.restlet.data.MediaType;
import org.restlet.representation.FileRepresentation;
import org.restlet.representation.Representation;
import org.restlet.resource.ClientResource;
public class REST {
private static String REST_URI = "http://localhost:8080/activiti-rest/service";
private static ClientResource getClientResource(String uri) {
ClientResource resource = new ClientResource(uri);
resource.setChallengeResponse(ChallengeScheme.HTTP_BASIC, "kermit", "kermit");
return resource;
}
public static JSONArray postDeployments() throws JSONException, IOException {
String deploymentURI = REST_URI + "/repository/deployments";
File file = new File("C:\\Users\\Test\\Desktop\\process.bpmn");
FileRepresentation fr = new FileRepresentation(file, MediaType.TEXT_PLAIN);
System.out.println("Size sent: " + fr.getSize());
try{
Representation response = getClientResource(deploymentURI).post(file, MediaType.MULTIPART_FORM_DATA);
JSONObject object = new JSONObject(response.getText());
if (object != null) {
JSONArray dataArray = (JSONArray) object.get("data");
return dataArray;
}
}
catch(Exception e){
System.out.println("Error:");
e.printStackTrace();
}
return null;
}
}

You are not generating the multipart-form-data request properly with Restlet.
You could have done the following to deploy a business process to activiti REST API:
String deploymentURI = REST_URI + "/repository/deployments";
File file = new File("/home/toto/fake-process.bpmn20.xml");
FileRepresentation entity = new FileRepresentation(file, MediaType.MULTIPART_FORM_DATA);
FormDataSet fds = new FormDataSet();
FormData fd = new FormData("upload_file", entity);
fds.getEntries().add(fd);
fds.setMultipart(true);
try {
Representation response = getClientResource(deploymentURI).post(fds);
} catch (Exception e) {
System.out.println("Error:");
e.printStackTrace();
}
See also this post

Different Result on DBPedia Spotlight by using the code and DBPedia Spotlight endpoint

This is the main class in which query is being fired
package extractKeyword;
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import org.apache.commons.httpclient.Header;
import org.apache.commons.httpclient.methods.GetMethod;
import org.dbpedia.spotlight.exceptions.AnnotationException;
import org.dbpedia.spotlight.model.DBpediaResource;
import org.dbpedia.spotlight.model.Text;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
import java.io.File;
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.util.LinkedList;
import java.util.List;
public class db extends AnnotationClient {
//private final static String API_URL = "http://jodaiber.dyndns.org:2222/";
private static String API_URL = "http://spotlight.dbpedia.org/";
private static double CONFIDENCE = 0.0;
private static int SUPPORT = 0;
// private static String powered_by ="non";
// private static String spotter ="CoOccurrenceBasedSelector";//"LingPipeSpotter"=Annotate all spots
//AtLeastOneNounSelector"=No verbs and adjs.
//"CoOccurrenceBasedSelector" =No 'common words'
//"NESpotter"=Only Per.,Org.,Loc.
//private static String disambiguator ="Default";//Default ;Occurrences=Occurrence-centric;Document=Document-centric
//private static String showScores ="yes";
#SuppressWarnings("static-access")
public void configiration(double CONFIDENCE,int SUPPORT)
//, String powered_by,String spotter,String disambiguator,String showScores)
{
this.CONFIDENCE=CONFIDENCE;
this.SUPPORT=SUPPORT;
// this.powered_by=powered_by;
//this.spotter=spotter;
//this.disambiguator=disambiguator;
//showScores=showScores;
}
public List<DBpediaResource> extract(Text text) throws AnnotationException {
// LOG.info("Querying API.");
String spotlightResponse;
try {
String Query=API_URL + "rest/annotate/?" +
"confidence=" + CONFIDENCE
+ "&support=" + SUPPORT
// + "&spotter=" + spotter
// + "&disambiguator=" + disambiguator
// + "&showScores=" + showScores
// + "&powered_by=" + powered_by
+ "&text=" + URLEncoder.encode(text.text(), "utf-8");
//LOG.info(Query);
GetMethod getMethod = new GetMethod(Query);
getMethod.addRequestHeader(new Header("Accept", "application/json"));
spotlightResponse = request(getMethod);
} catch (UnsupportedEncodingException e) {
throw new AnnotationException("Could not encode text.", e);
}
assert spotlightResponse != null;
JSONObject resultJSON = null;
JSONArray entities = null;
try {
resultJSON = new JSONObject(spotlightResponse);
entities = resultJSON.getJSONArray("Resources");
} catch (JSONException e) {
//throw new AnnotationException("Received invalid response from DBpedia Spotlight API.");
}
LinkedList<DBpediaResource> resources = new LinkedList<DBpediaResource>();
if(entities!=null)
for(int i = 0; i < entities.length(); i++) {
try {
JSONObject entity = entities.getJSONObject(i);
resources.add(
new DBpediaResource(entity.getString("#URI"),
Integer.parseInt(entity.getString("#support"))));
} catch (JSONException e) {
//((Object) LOG).error("JSON exception "+e);
}
}
return resources;
}
}
The extended class
package extractKeyword;
import org.apache.commons.httpclient.DefaultHttpMethodRetryHandler;
import org.apache.commons.httpclient.Header;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpException;
import org.apache.commons.httpclient.HttpMethodBase;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.params.HttpMethodParams;
import org.dbpedia.spotlight.exceptions.AnnotationException;
import org.dbpedia.spotlight.model.DBpediaResource;
import org.dbpedia.spotlight.model.Text;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.text.ParseException;
import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
import java.util.logging.Logger;
import javax.ws.rs.HttpMethod;
/**
* #author pablomendes
*/
public abstract class AnnotationClient {
//public Logger LOG = Logger.getLogger(this.getClass());
private List<String> RES = new ArrayList<String>();
// Create an instance of HttpClient.
private static HttpClient client = new HttpClient();
public List<String> getResu(){
return RES;
}
public String request(GetMethod getMethod) throws AnnotationException {
String response = null;
// Provide custom retry handler is necessary
( getMethod).getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
new DefaultHttpMethodRetryHandler(3, false));
try {
// Execute the method.
int statusCode = client.executeMethod((org.apache.commons.httpclient.HttpMethod) getMethod);
if (statusCode != HttpStatus.SC_OK) {
// LOG.error("Method failed: " + ((HttpMethodBase) method).getStatusLine());
}
// Read the response body.
byte[] responseBody = ((HttpMethodBase) getMethod).getResponseBody(); //TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
// Deal with the response.
// Use caution: ensure correct character encoding and is not binary data
response = new String(responseBody);
} catch (HttpException e) {
// LOG.error("Fatal protocol violation: " + e.getMessage());
throw new AnnotationException("Protocol error executing HTTP request.",e);
} catch (IOException e) {
//((Object) LOG).error("Fatal transport error: " + e.getMessage());
//((Object) LOG).error(((HttpMethodBase) method).getQueryString());
throw new AnnotationException("Transport error executing HTTP request.",e);
} finally {
// Release the connection.
((HttpMethodBase) getMethod).releaseConnection();
}
return response;
}
protected static String readFileAsString(String filePath) throws java.io.IOException{
return readFileAsString(new File(filePath));
}
protected static String readFileAsString(File file) throws IOException {
byte[] buffer = new byte[(int) file.length()];
#SuppressWarnings("resource")
BufferedInputStream f = new BufferedInputStream(new FileInputStream(file));
f.read(buffer);
return new String(buffer);
}
static abstract class LineParser {
public abstract String parse(String s) throws ParseException;
static class ManualDatasetLineParser extends LineParser {
public String parse(String s) throws ParseException {
return s.trim();
}
}
static class OccTSVLineParser extends LineParser {
public String parse(String s) throws ParseException {
String result = s;
try {
result = s.trim().split("\t")[3];
} catch (ArrayIndexOutOfBoundsException e) {
throw new ParseException(e.getMessage(), 3);
}
return result;
}
}
}
public void saveExtractedEntitiesSet(String Question, LineParser parser, int restartFrom) throws Exception {
String text = Question;
int i=0;
//int correct =0 ; int error = 0;int sum = 0;
for (String snippet: text.split("\n")) {
String s = parser.parse(snippet);
if (s!= null && !s.equals("")) {
i++;
if (i<restartFrom) continue;
List<DBpediaResource> entities = new ArrayList<DBpediaResource>();
try {
entities = extract(new Text(snippet.replaceAll("\\s+"," ")));
System.out.println(entities.get(0).getFullUri());
} catch (AnnotationException e) {
// error++;
//LOG.error(e);
e.printStackTrace();
}
for (DBpediaResource e: entities) {
RES.add(e.uri());
}
}
}
}
public abstract List<DBpediaResource> extract(Text text) throws AnnotationException;
public void evaluate(String Question) throws Exception {
evaluateManual(Question,0);
}
public void evaluateManual(String Question, int restartFrom) throws Exception {
saveExtractedEntitiesSet(Question,new LineParser.ManualDatasetLineParser(), restartFrom);
}
}
The Main Class
package extractKeyword;
public class startAnnonation {
public static void main(String[] args) throws Exception {
String question = "What is the winning chances of BJP in New Delhi elections?";
db c = new db ();
c.configiration(0.25,0);
//, 0, "non", "AtLeastOneNounSelector", "Default", "yes");
c.evaluate(question);
System.out.println("resource : "+c.getResu());
}
}
The main problem is here when I am using DBPedia spotlight using spotlight jar (above code)then i am getting different result as compared to the dbpedia spotlight endpoint(dbpedia-spotlight.github.io/demo/)
Result using the above code:-
Text :-What is the winning chances of BJP in New Delhi elections?
Confidence level:-0.35
resource : [Election]
Result on DBPedia Spotlight endpoint(//dbpedia-spotlight.github.io/demo/)
Text:-What is the winning chances of BJP in New Delhi elections?
Confidence level:-0.35
resource : [Bharatiya_Janata_Party, New_Delhi, Election]
Why also the spotlight now don't have support as a parameter?

Not able to take screenShot using HtmlUnitDriver [Selenium WebDriver java]

I am wanting to take a screenshot of a page using HtmlUnitDriver I came across this Link where this guy has made a custom HTML unit driver to take the screenshot.
But unfortunately, while implementing that I am getting an exception.
"Exception in thread "main" java.lang.ClassCastException: [B cannot be cast to java.io.File
at Test.main(Test.java:39)"
My code is as follows-
import java.io.File;
import java.io.IOException;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.WebDriver;
import com.gargoylesoftware.htmlunit.BrowserVersion;
public class Test extends ScreenCaptureHtmlUnitDriver {
public static void main(String[] args) throws InterruptedException, IOException {
WebDriver driver = new ScreenCaptureHtmlUnitDriver(BrowserVersion.FIREFOX_38);
driver.get("https://www.google.com/?gws_rd=ssl");
try{
File scrFile = ((ScreenCaptureHtmlUnitDriver) driver).getScreenshotAs(OutputType.FILE);
FileUtils.copyFile(scrFile, new File("D:\\TEMP.PNG"));
}catch (Exception e) {
e.printStackTrace();
}
}
}
HtmlUnit driver which I am using(the one which is in the link) is this-
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.net.URL;
import java.util.Collections;
import java.util.HashMap;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
import org.apache.commons.io.FilenameUtils;
import org.apache.commons.io.IOUtils;
import org.openqa.selenium.Capabilities;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.htmlunit.HtmlUnitDriver;
import org.openqa.selenium.internal.Base64Encoder;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
import com.gargoylesoftware.htmlunit.BrowserVersion;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.WebRequest;
import com.gargoylesoftware.htmlunit.WebWindow;
import com.gargoylesoftware.htmlunit.html.HtmlElement;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
public class ScreenCaptureHtmlUnitDriver extends HtmlUnitDriver implements TakesScreenshot {
private static Map<String, byte[]> imagesCache = Collections.synchronizedMap(new HashMap<String, byte[]>());
private static Map<String, String> cssjsCache = Collections.synchronizedMap(new HashMap<String, String>());
// http://stackoverflow.com/questions/4652777/java-regex-to-get-the-urls-from-css
private final static Pattern cssUrlPattern = Pattern.compile("background(-image)?[\\s]*:[^url]*url[\\s]*\\([\\s]*([^\\)]*)[\\s]*\\)[\\s]*");// ?<url>
public ScreenCaptureHtmlUnitDriver() {
super();
}
public ScreenCaptureHtmlUnitDriver(boolean enableJavascript) {
super(enableJavascript);
}
public ScreenCaptureHtmlUnitDriver(Capabilities capabilities) {
super(capabilities);
}
public ScreenCaptureHtmlUnitDriver(BrowserVersion version) {
super(version);
DesiredCapabilities var = ((DesiredCapabilities) getCapabilities());
var.setCapability(CapabilityType.TAKES_SCREENSHOT, true);
}
//#Override
#SuppressWarnings("unchecked")
public <X> X getScreenshotAs(OutputType<X> target) throws WebDriverException {
byte[] archive = new byte[0];
try {
archive = downloadCssAndImages(getWebClient(), (HtmlPage) getCurrentWindow().getEnclosedPage());
} catch (Exception e) {
}
if(target.equals(OutputType.BASE64)){
return target.convertFromBase64Png(new Base64Encoder().encode(archive));
}
if(target.equals(OutputType.BYTES)){
return (X) archive;
}
return (X) archive;
}
// http://stackoverflow.com/questions/2244272/how-can-i-tell-htmlunits-webclient-to-download-images-and-css
protected byte[] downloadCssAndImages(WebClient webClient, HtmlPage page) throws Exception {
WebWindow currentWindow = webClient.getCurrentWindow();
Map<String, String> urlMapping = new HashMap<String, String>();
Map<String, byte[]> files = new HashMap<String, byte[]>();
WebWindow window = null;
try {
window = webClient.getWebWindowByName(page.getUrl().toString()+"_screenshot");
webClient.getPage(window, new WebRequest(page.getUrl()));
} catch (Exception e) {
window = webClient.openWindow(page.getUrl(), page.getUrl().toString()+"_screenshot");
}
String xPathExpression = "//*[name() = 'img' or name() = 'link' and (#type = 'text/css' or #type = 'image/x-icon') or #type = 'text/javascript']";
List<?> resultList = page.getByXPath(xPathExpression);
Iterator<?> i = resultList.iterator();
while (i.hasNext()) {
try {
HtmlElement el = (HtmlElement) i.next();
String resourceSourcePath = el.getAttribute("src").equals("") ? el.getAttribute("href") : el
.getAttribute("src");
if (resourceSourcePath == null || resourceSourcePath.equals(""))
continue;
URL resourceRemoteLink = page.getFullyQualifiedUrl(resourceSourcePath);
String resourceLocalPath = mapLocalUrl(page, resourceRemoteLink, resourceSourcePath, urlMapping);
urlMapping.put(resourceSourcePath, resourceLocalPath);
if (!resourceRemoteLink.toString().endsWith(".css")) {
byte[] image = downloadImage(webClient, window, resourceRemoteLink);
files.put(resourceLocalPath, image);
} else {
String css = downloadCss(webClient, window, resourceRemoteLink);
for (String cssImagePath : getLinksFromCss(css)) {
URL cssImagelink = page.getFullyQualifiedUrl(cssImagePath.replace("\"", "").replace("\'", "")
.replace(" ", ""));
String cssImageLocalPath = mapLocalUrl(page, cssImagelink, cssImagePath, urlMapping);
files.put(cssImageLocalPath, downloadImage(webClient, window, cssImagelink));
}
files.put(resourceLocalPath, replaceRemoteUrlsWithLocal(css, urlMapping)
.replace("resources/", "./").getBytes());
}
} catch (Exception e) {
}
}
String pagesrc = replaceRemoteUrlsWithLocal(page.getWebResponse().getContentAsString(), urlMapping);
files.put("page.html", pagesrc.getBytes());
webClient.setCurrentWindow(currentWindow);
return createZip(files);
}
String downloadCss(WebClient webClient, WebWindow window, URL resourceUrl) throws Exception {
if (cssjsCache.get(resourceUrl.toString()) == null) {
cssjsCache.put(resourceUrl.toString(), webClient.getPage(window, new WebRequest(resourceUrl))
.getWebResponse().getContentAsString());
}
return cssjsCache.get(resourceUrl.toString());
}
byte[] downloadImage(WebClient webClient, WebWindow window, URL resourceUrl) throws Exception {
if (imagesCache.get(resourceUrl.toString()) == null) {
imagesCache.put(
resourceUrl.toString(),
IOUtils.toByteArray(webClient.getPage(window, new WebRequest(resourceUrl)).getWebResponse()
.getContentAsStream()));
}
return imagesCache.get(resourceUrl.toString());
}
public static byte[] createZip(Map<String, byte[]> files) throws IOException {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ZipOutputStream zipfile = new ZipOutputStream(bos);
Iterator<String> i = files.keySet().iterator();
String fileName = null;
ZipEntry zipentry = null;
while (i.hasNext()) {
fileName = i.next();
zipentry = new ZipEntry(fileName);
zipfile.putNextEntry(zipentry);
zipfile.write(files.get(fileName));
}
zipfile.close();
return bos.toByteArray();
}
List<String> getLinksFromCss(String css) {
List<String> result = new LinkedList<String>();
Matcher m = cssUrlPattern.matcher(css);
while (m.find()) { // find next match
result.add( m.group(2));
}
return result;
}
String replaceRemoteUrlsWithLocal(String source, Map<String, String> replacement) {
for (String object : replacement.keySet()) {
// background:url(http://org.com/images/image.gif)
source = source.replace(object, replacement.get(object));
}
return source;
}
String mapLocalUrl(HtmlPage page, URL link, String path, Map<String, String> replacementToAdd) throws Exception {
String resultingFileName = "resources/" + FilenameUtils.getName(link.getFile());
replacementToAdd.put(path, resultingFileName);
return resultingFileName;
}
}
UPDATE
Code provided by Andrew works- but I wanted to know if there is a way by which we can download only selected resources. For eg this website I would like to download only captcha image those id is "//*[#id='cimage']" because downloading all the resources will take a long time. Is there a way by which we can download only the specific resource. Because with the existing code provided
below all the resources get downloaded.
byte[] zipFileBytes = ((ScreenCaptureHtmlUnitDriver) driver).getScreenshotAs(OutputType.BYTES);
FileUtils.writeByteArrayToFile(new File("D:\\TEMP.PNG"), zipFileBytes);

The error says that the code is trying to convert a byte[] to a File. It's easy to see why if you just strip out the unused paths from getScreenshotAs:
public <X> X getScreenshotAs(OutputType<X> target) throws WebDriverException {
byte[] archive = new byte[0];
try {
archive = downloadCssAndImages(getWebClient(), (HtmlPage) getCurrentWindow().getEnclosedPage());
} catch (Exception e) {
}
return (X) archive;
}
There's no way you can get a File out of that. OutputType.FILE is not supported, so you have to handle file output yourself. Luckily, that's easy. You can change your code to:
byte[] zipFileBytes = ((ScreenCaptureHtmlUnitDriver) driver).getScreenshotAs(OutputType.BYTES);
FileUtils.writeByteArrayToFile(new File("D:\\TEMP.PNG"), zipFileBytes);
See FileUtils.writeByteArrayToFile() for more.

Check this out this may helpful for you
File scrFile = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
FileUtils.copyFile(scrFile, new File("C:/Users/home/Desktop/screenshot.png"));// copy it somewhere