We Keep Coding

INVALID_ARGUMENT response when signingBlob using IamCredentialsClient

I'm following this google guide on how to sign urls manually since I need to create a signed URL in other to let other user upload files without having to authenticate manually and while I manage to follow all steps required and showed in the python example, my java code always receives an io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Request contains an invalid argument.
I'm getting crazy trying to find what is the issue and I'm not sure if I'm doing it correctly.
The dependency I'm using to signBlob is the google-cloud-iamcredentials. Is there something I'm missing out?
Here you can check the code
try (IamCredentialsClient iamCredentialsClient = IamCredentialsClient.create()){
final LocalDateTime currentTime = LocalDateTime.now();
final String completeTimeStamp = currentTime.format(DateTimeFormatter.ofPattern("yyyyMMdd HHmmss"))
.replace(" ", "T") + 'Z';
final String pathToResource = "/" + filename;
final String credentialScope = currentTime.format(DateTimeFormatter.BASIC_ISO_DATE) + "/eu/storage/goog4_request";
String credential = clientEmail + '/' + credentialScope;
credential = credential.replace("#", "%40");
credential = credential.replace("/", "%2F");
final String canonicalHeader = "host:" + bucketName + ".storage.googleapis.com\n";
final String signedHeader = "host";
final String canonicalQueryString =
"X-Goog-Algorithm=GOOG4-RSA-SHA256&" +
"X-Goog-Credential=" + credential + '&' +
"X-Goog-Date=" + completeTimeStamp + '&' +
"X-Goog-Expires=" + expiration + '&' +
"X-Goog-SignedHeaders=" + signedHeader;
final String canonicalRequest =
"PUT" + '\n' +
pathToResource + '\n' +
canonicalQueryString + '\n' +
canonicalHeader + '\n' +
signedHeader + '\n' +
"UNSIGNED-PAYLOAD";
final String hashedCanonicalRequest =
DigestUtils.sha256Hex(canonicalRequest.getBytes(StandardCharsets.UTF_8));
final String stringToSign = "GOOG4-RSA-SHA256" + '\n' +
completeTimeStamp + '\n' +
credentialScope + '\n' +
hashedCanonicalRequest;
final String signature = iamCredentialsClient.signBlobCallable()
.call(
SignBlobRequest.newBuilder()
.setName(ServiceAccountName.of(projectId, clientEmail).toString())
.addAllDelegates(new ArrayList<>())
.setPayload(ByteString.copyFrom(stringToSign.getBytes(StandardCharsets.UTF_8)))
.build()
).toString();
return String.format(CLOUD_URL, pathToResource,
canonicalQueryString, "&X-Goog-Signature=".concat(signature));

I just found my issue after a long time debugging and an even longer time having a break from this problem.
Turns out I was creating the ServiceAccountName.of wrong, according to this resource https://cloud.google.com/iam/docs/reference/credentials/rest/v1/projects.serviceAccounts/signBlob.
You can see under the parameter section this comment
Required. The resource name of the service account for which the credentials are requested, in the following format: projects/-/serviceAccounts/{ACCOUNT_EMAIL_OR_UNIQUEID}. The - wildcard character is required; replacing it with a project ID is invalid.
Meaning my code was not constructing this field in the correct format, I changed ServiceAccountName.of(projectId, clientEmail).toString() to ServiceAccountName.of("-", clientEmail).toString() and it worked!
I leave this as the answer for my question in case anyone in the future encounters with this exact issue.

Why -from Java- JSON returns this error? Clearly is in the JSON deserialization, but I cannot find exactly where it occurs

Situation:
[![from Postman, JSON returns the data correctly][1]][1]
[1]: https://i.stack.imgur.com/4Kk0J.png
Server side:
Enables the reading of the request body for all routes under /api/articulos.
Router router = Router.router(vertx);
router.route("/api/articulos*").handler(BodyHandler.create());
router.get("/api/articulos/:cantcomp1/:cantcomp2/:tipoprod/:prodpadre").handler(bizArticulo::getOneReadingBarcode);
Business
private static final String SELECT_CBA = "select art.leyenda, $1 :: numeric as cantidad, uni.abreviatura, "
+ "round(((art.precio_costo * (art.utilidad_fraccionado/100)) + art.precio_costo) * ($2),2) as totpagar "
+ "FROM public.articulos art join public.unidades uni on uni.idunidad = art.idunidad "
+ "WHERE (substring(art.codigobarra,1,2) = ($3) and substring(art.codigobarra,3,6) = ($4))";
public void getOneReadingBarcode(RoutingContext routingContext) {
Double cantComprada1 = Double.parseDouble(routingContext.request().getParam("cantcomp1"));
Double cantComprada2 = Double.parseDouble(routingContext.request().getParam("cantcomp2"));
String tipoProducto = routingContext.request().getParam("tipoprod");
String productoPadre = routingContext.request().getParam("prodpadre");
HttpServerResponse response = routingContext.response();
pgClient
.preparedQuery(SELECT_CBA)
.execute(Tuple.of(cantComprada1, cantComprada2, tipoProducto, productoPadre), ar -> {
if (ar.succeeded()) {
RowSet<Row> rows = ar.result(); // return always ONE ARTICLE
List<Articulo> articulos = new ArrayList<>();
rows.forEach(row -> {
articulos.add(fromBarCode(row));
});
response.putHeader("content-type", "application/json; charset=utf-8")
.setStatusCode(200)
.end(Json.encodePrettily(articulos));
} else {
System.out.println("Failure: " + ar.cause().getMessage());
response.putHeader("content-type", "application/json; charset=utf-8")
.end(Json.encodePrettily(ar.cause().getMessage()));
}
});
}
private static Articulo fromBarCode(Row row) {
String leyenda = row.getString("leyenda");
BigDecimal cantComprada = row.getBigDecimal("cantidad");
String abreviatura = row.getString("abreviatura");
BigDecimal total_a_pagar = row.getBigDecimal("totpagar");
Articulo articulo = new Articulo();
articulo.setLeyenda(leyenda);
articulo.setCant_comprada(cantComprada);
articulo.setAbreviatura(abreviatura);
articulo.setTot_a_pagar(total_a_pagar);
return articulo;
}
Client-side
private void validarArticulo() {
switch (txtCodigoBarra.getText().substring(0,2)) {
case "20":
Integer d = Integer.parseInt(txtCodigoBarra.getText().substring(8, 12));
Double decimal = d * 0.001; // convert to kilos
System.out.println(
String.format("%.3f", decimal) + " | " +
String.format("%.3f", decimal) + " | " +
txtCodigoBarra.getText().substring(0, 2) + " | " +
txtCodigoBarra.getText().substring(2, 8)
);
PosAccess
.getCodigoBarra(
decimal,
decimal,
txtCodigoBarra.getText().substring(0, 2),
txtCodigoBarra.getText().substring(2, 8));
case "77":
}
public static ObservableList<Articulo> getCodigoBarra(Double cantComp1, Double cantComp2, String tipoProducto, String productoPadre) {
ObservableList<Articulo> itemsArticulo = FXCollections.observableArrayList();
WebClient client = WebClient.create(Vertx.vertx());
client
.get(PORT, HOST, "/api/conceptos/" + cantComp1 + "/" + cantComp2 + "/" + tipoProducto + "/" + productoPadre)
// .get(PORT, HOST, "/api/conceptos/")
// .setQueryParam("cantcomp1", cantComp1)
// .addQueryParam("cantcomp2", cantComp2)
// .addQueryParam("tipoproducto", tipoProducto)
// .addQueryParam("productopadre", productoPadre)
.send(ar -> {
if (ar.succeeded()) {
HttpResponse<Buffer> response = ar.result();
response.bodyAsJsonArray().forEach(articulo -> {
JsonObject jo = (JsonObject) articulo;
itemsArticulo.add(new Articulo(jo.getString("leyenda"), jo.getDouble("cant_comprada"), jo.getString("abreviatura"), jo.getDouble("tot_a_pagar")));
});
System.out.println("Received response with status code " + response.statusCode());
System.out.println(response.bodyAsJsonArray());
} else {
System.out.println("Something went wrong " + ar.cause().getMessage());
}
});
return itemsArticulo;
}
Ok, that's the code. When scan the barcode I get this error
0,750 | 0,750 | 20 | 021162
oct. 11, 2020 6:33:18 A. M. io.vertx.core.impl.ContextImpl
SEVERE: Unhandled exception
io.vertx.core.json.DecodeException: Failed to decode:Unexpected character ('<' (code 60)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
at [Source: (io.netty.buffer.ByteBufInputStream); line: 1, column: 2]
at io.vertx.core.json.jackson.JacksonCodec.fromParser(JacksonCodec.java:100)
at io.vertx.core.json.jackson.JacksonCodec.fromBuffer(JacksonCodec.java:67)
at io.vertx.ext.web.codec.impl.BodyCodecImpl.lambda$static$2(BodyCodecImpl.java:51)
at io.vertx.ext.web.client.impl.HttpResponseImpl.bodyAsJsonArray(HttpResponseImpl.java:116)
at consumer.PosAccess.lambda$0(PosAccess.java:45)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code 60)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
at [Source: (io.netty.buffer.ByteBufInputStream); line: 1, column: 2]
... 22 more
Anyboy can help me?
TIA
Ernesto

You are calling the wrong endpoint (i.e. /api/conceptos/ instead of /api/articulos).
In the screenshot provided in the post, which illustrates a correct response, you are showing a response for url
192.168.0.15:8092/api/articulos/0.75/0.75/20/021162
but in the code you have
client.get(PORT, HOST, "/api/conceptos/" + cantComp1 + "/" + cantComp2 + "/" + tipoProducto + "/" + productoPadre)

According to my experiences, this error message:
io.vertx.core.json.DecodeException: Failed to decode:Unexpected character ('<'
usualy happens when you try to deserialize an api response into a json, but indeed the response is something like:
<html><body><h1>Resource not found</h1></body></html>
or
<html><body><h1>Internal server error</h1></body></html>
and the parsing fails at the first char <
For your case, this happens at response.bodyAsJsonArray() probably.
please make a debug output for the url what you are calling:
String url = "/api/conceptos/" + cantComp1 + "/" + cantComp2 + "/" + tipoProducto + "/" + productoPadre;

Directing the search depths in Crawler4j Solr

I am trying to make the crawler "abort" searching a certain subdomain every time it doesn't find a relevant page after 3 consecutive tries. After extracting the title and the text of the page I start looking for the correct pages to submit to my solr collection. (I do not want to add pages that don't match this query)
public void visit(Page page)
{
int docid = page.getWebURL().getDocid();
String url = page.getWebURL().getURL();
String domain = page.getWebURL().getDomain();
String path = page.getWebURL().getPath();
String subDomain = page.getWebURL().getSubDomain();
String parentUrl = page.getWebURL().getParentUrl();
String anchor = page.getWebURL().getAnchor();
System.out.println("Docid: " + docid);
System.out.println("URL: " + url);
System.out.println("Domain: '" + domain + "'");
System.out.println("Sub-domain: '" + subDomain + "'");
System.out.println("Path: '" + path + "'");
System.out.println("Parent page: " + parentUrl);
System.out.println("Anchor text: " + anchor);
System.out.println("ContentType: " + page.getContentType());
if(page.getParseData() instanceof HtmlParseData) {
String title, text;
HtmlParseData theHtmlParseData = (HtmlParseData) page.getParseData();
title = theHtmlParseData.getTitle();
text = theHtmlParseData.getText();
if ( (title.toLowerCase().contains(" word1 ") && title.toLowerCase().contains(" word2 ")) || (text.toLowerCase().contains(" word1 ") && text.toLowerCase().contains(" word2 ")) ) {
//
// submit to SOLR server
//
submit(page);
Header[] responseHeaders = page.getFetchResponseHeaders();
if (responseHeaders != null) {
System.out.println("Response headers:");
for (Header header : responseHeaders) {
System.out.println("\t" + header.getName() + ": " + header.getValue());
}
}
failedcounter = 0;// we start counting for 3 consecutive pages
} else {
failedcounter++;
}
if (failedcounter == 3) {
failedcounter = 0; // we start counting for 3 consecutive pages
int parent = page.getWebURL().getParentDocid();
parent....HtmlParseData.setOutgoingUrls(null);
}
my question is, how do I edit the last line of this code so that i can retrieve the parent "page object" and delete its outgoing urls, so that the crawl moves on to the rest of the subdomains.
Currently i cannot find a function that can get me from the parent id to the page data, for deleting the urls.

The visit(...) method is called as one of the last statements of processPage(...) (line 523 in WebCrawler).
The outgoing links are already added to the crawler's frontier (and might be processed by other crawler processes as soon as they are added).
You could define the behaviour described by adjusting the shouldVisit(...) or (depending on the exact use-case) in shouldFollowLinksIn(...) of the crawler

java result set only writing one line instead of all selected into .csv

I have a java function that is meant to take strings from jlist called "readyList" and pulling data from mysql workbench tables with the intent to write a line for each string in a .csv file. With the current code it sucessfully pulls the data one at a time like i intended but it only writes the last line instead of all the lines. I want to have all the lines written in the .csv file. Please help!
int[] selectedIx = readyList.getSelectedIndices();
for (int i = 0; i < selectedIx.length; i++) {
// while (i < selectedIx.length) {
Object sel = readyList.getModel().getElementAt(selectedIx[i]);
Statement s1 = DBConnect.connection.createStatement();
String selTable01 = "SELECT Sku As s, Qty As q, Orig_Retail As prce, Orig_Sku As orgsk, Store As strcd "
+ "FROM completed_lines WHERE Form_Name = '" + sel + "' AND Warranty = 'true'";
s1.execute(selTable01);
try (ResultSet rs01 = s1.getResultSet()) {
fWriter = new FileWriter("Exports/Transfers/" + /* frmNm.replace(":", "_") */"EBW_" + GtDates.fdate + ".csv", false);
writer = new BufferedWriter(fWriter);
String header = "slip_number,slip_type,out_store,in_store,item_number,quantity,price,comment,to_num";
writer.write(header);
writer.newLine();
while (rs01.next()) {
String strcode = rs01.getString("strcd");
String sku = rs01.getString("s");
String qty = rs01.getString("q");
String price = rs01.getString("prce");
String orgsku = rs01.getString("orgsk");
//System.out.println(frmNm.split("_")[1] + qty + "," + sku + "," + vendor + "," + desc1 + "," + reas + "," + descdmg + "," + orgR + "," + nwsku + "," + desc2 + "," + qtyI);
String line = ""+","+"out"+","+strcode+","+"RTV"+","+sku+","+qty+","+price+","+"EBW"+","+orgsku;
writer.write(line);
writer.newLine();
}
}
// JOptionPane.showMessageDialog(frame, "All Data from Selected Forms has been Exported");
}
// FormCompelted();
writer.close();
}
}

A few issues with this code. The reason you're only getting the last result is because of this line:
fWriter = new FileWriter("Exports/Transfers/" + /* frmNm.replace(":", "_") */"EBW_" + GtDates.fdate + ".csv", false);
This line is inside your loop. The false as the last parameter tells FileWriter not to append. In other words, a false means overwrite the file if it exists. Since this is in your loop, each result overwrites the file that the last result created. You should create the FileWriter outside of your loop, probably in a try with resources. That will allow you to remove your writer.close() call, which should have been in a finally block anyway.
Not related to your original question but something you should be aware of: You're creating a new Statement with each loop iteration. This can be an expensive operation. You should use a PreparedStatement instead. Create it outside your loop and then just set the parameter and execute it inside the loop. It also implements AutoCloseable, so you can create it in a try with resources too, probably the same one you create your FileWriter in.

Using ACE with WT

UPDATE 3
Final working code below. YOU NEED THE ace.js FROM THE src FOLDER! It will not work from the libs, you need the pre-packaged version from their site.
WText *editor = new WText(root());
editor->setText("function(){\n hello.abc();\n}\n");
editor->setInline(false);
The above code can set the contents of the ACE window.
MyClass::MyClass(const WEnvironment& env)
: WApplication(env)
{
wApp->require("ace-builds/src/ace.js");
// A WContainerWidget is rendered as a div
WContainerWidget *editor = new WContainerWidget(root());
editor->resize(500, 500);
std::string editor_ref = editor->jsRef(); // is a text string that will be the element when executed in JS
std::string command =
editor_ref + ".editor = ace.edit(" + editor_ref + ");" +
editor_ref + ".editor.setTheme(\"ace/theme/monokai\");" +
editor_ref + ".editor.getSession().setMode(\"ace/mode/javascript\");";
editor->doJavaScript(command);
JSignal <std::string> *jsignal = new JSignal<std::string>(editor, "textChanged");
jsignal->connect(this, &MyClass::textChanged);
WPushButton *b = new WPushButton("Save", root());
command = "function(object, event) {" +
jsignal->createCall(editor_ref + ".editor.getValue()") +
";}";
b->clicked().connect(command);
}
void MyClass::textChanged(std::string incoming)
{
}
UPDATE 2
Here is what my project looks like atm, still getting a white screen with a red "Loading..." message from WT in the top right hand corner. More notes below.
MyClass::MyClass(const WEnvironment& env)
: WApplication(env)
{
wApp->require("lib/ace/ace.js");
// A WContainerWidget is rendered as a div
WContainerWidget *editor = new WContainerWidget(root());
editor->resize(500, 500);
std::string editor_ref = editor->jsRef(); // is a text string that will be the element when executed in JS
std::string command =
editor_ref + ".editor = ace.edit(" + editor_ref + ");" +
editor_ref + ".editor.setTheme(\"ace/theme/monokai\");" +
editor_ref + ".editor.getSession().setMode(\"ace/mode/javascript\");";
editor->doJavaScript(command);
JSignal <std::string> *jsignal = new JSignal<std::string>(editor, "textChanged");
jsignal->connect(this, &MyClass::textChanged);
WPushButton *b = new WPushButton("Save", root());
command = "function(object, event) {" +
jsignal->createCall(editor_ref + ".editor.getValue()") +
";}";
b->clicked().connect(command);
}
void MyClass::textChanged(std::string incoming)
{
}
"command" variable is equal to the following when it is used for editor->doJavaScript(command)
"Wt3_3_0.$('oy4ycjy').editor = ace.edit(Wt3_3_0.$('oy4ycjy'));Wt3_3_0.$('oy4ycjy').editor.setTheme('ace/theme/monokai');Wt3_3_0.$('oy4ycjy').editor.getSession().setMode('ace/mode/javascript');"
"command" variable is equal to the following when it is used for b->clicked().connect(command);
"function(object, event) {Wt.emit('oy4ycjy','textChanged',Wt3_3_0.$('oy4ycjy').editor.getValue());;}"
UPDATE 1
Added the suggested code to my constructor, however the page does not change from a solid white screen. I am doing nothing else in this WT project, only this code is running.
wApp->require("lib/ace/ace.js");
// A WContainerWidget is rendered as a div
WContainerWidget *editor = new WContainerWidget(root());
std::string editor_ref = editor->jsRef(); // is a text string that will be the element when executed in JS
editor->doJavaScript(
editor_ref + ".editor = ace.edit('" + editor_ref + "');" +
editor_ref + ".editor.setTheme('ace/theme/monokai');" +
editor_ref + ".editor.getSession().setMode('ace/mode/javascript');"
);
The value of editor_ref is "Wt3_3_0.$('oumvrgm')" minus the quotes.
Also tried adding the code below, and the page is still blanked out.
JSignal <std::string> *jsignal = new JSignal<std::string>(editor, "textChanged");
jsignal->connect(this, &MyClass::textChanged);
WPushButton *b = new WPushButton("Save", root());
b->clicked().connect("function(object, event) {" +
jsignal->createCall(editor->jsRef() + ".editor.getValue()") +
";}");
I have also found that commenting out
editor_ref + ".editor = ace.edit('" + editor_ref + "');" +
makes the button show up, but there is a red "Loading..." note at the top right of the screen so WT is waiting on something.
I have textChanged as a do nothing function at the moment.
ORIGINAL POST
So, my problem is this. How can I get ACE http://ace.ajax.org/#nav=about in WT http://www.webtoolkit.eu/wt. More specifically, ACE in a WT Wt::WTextArea or Wt::WTabWidget, the text area would be preferred. I have been trying to do this for a few days now and have not had much success.
I've been able to embed ACE in an HTML page no problem, as their site says "just copy and paste it into your page" and it really is that simple. However, I need to load it locally through WT and into a container. I downloaded their repos from GIT to my machine and have tried using
require("lib/ace/ace.js");
and
doJavaScript(...);
with various commands to no success... I am not nearly as strong in Java and HTML as C++ so I will ask for as much detail as possible if this involves a lot of Java/HTML. Thanks in advance mates!

Maybe this puts you in the right direction:
wApp->require("lib/ace/ace.js")
// A WContainerWidget is rendered as a div
WContainerWidget *editor = new WContainerWidget(parent);
// editor->jsRef() is a text string that will be the element when executed in JS
editor->doJavaScript(
editor->jsRef() + ".editor = ace.edit(" + editor->jsRef() + ");" +
editor->jsRef() + ".editor.setTheme('ace/theme/monokai');" +
editor->jsRef() + ".editor.getSession().setMode('ace/mode/javascript');"
);
That should decorate the editor. Wt does not automatically send the modifications to a div to the server, so you do this manually through a JSignal (emits a signal from JS to C++):
JSignal <std::string> *jsignal = new JSignal<std::string>(editor, "textChanged");
jsignal->connect(this, MyClass::textChanged);
WPushButton *b = new WPushButton("Save", parent);
b->clicked().connect("function(object, event) {" +
jsignal->createCall(editor->jsRef() + ".editor.getValue()") +
";}");
(code above is not tested so you may need to adjust a bit)
I have integrated CodeMirror in an earlier JWt (java) project like this:
import eu.webtoolkit.jwt.WApplication;
import eu.webtoolkit.jwt.WContainerWidget;
import eu.webtoolkit.jwt.WTextArea;
public class CodeMirrorTextArea extends WContainerWidget {
private WTextArea textArea;
public CodeMirrorTextArea(WContainerWidget parent) {
super(parent);
textArea = new WTextArea(this);
WApplication app = WApplication.getInstance();
app.require(app.resolveRelativeUrl("codemirror-2.32/lib/codemirror.js"));
app.require(app.resolveRelativeUrl("codemirror-2.32/mode/groovy/groovy.js"));
//TODO:
//We save the editor state to the text area on each key stroke,
//it appears to be not a performance issue,
//however it might very well become one when editing larger fragments of code.
//A better solution would be to save this state to the text area only when
//the form is submitted, currently this is not yet possible in Wt???.
String js =
"var e = " + textArea.getJsRef() + ";" +
"var cm = CodeMirror.fromTextArea(e, {" +
" onKeyEvent : function (editor, event) {" +
" editor.save();" +
" }," +
" lineNumbers: true" +
" });" +
"var self = " + getJsRef() + ";" +
"self.cm = cm;";
this.doJavaScript(js);
}
public CodeMirrorTextArea() {
this(null);
}
public void setText(String text) {
textArea.setText(text);
}
public String getText() {
return textArea.getText();
}
public void setMarker(int line, String htmlMarker) {
String js =
"var self = " + getJsRef() + ";" +
"self.cm.setMarker(" + line + ", " + jsStringLiteral(htmlMarker +
"%N%") + ");";
this.doJavaScript(js);
}
public void clearMarker(int line) {
String js =
"var self = " + getJsRef() + ";" +
"self.cm.clearMarker(" + line + ");";
this.doJavaScript(js);
}
}