I have a Java application running on Spring and defining multiple gRPC endpoints. These endpoints are meant to be queried from multiple clients, one of which being in PHP, so I used the PHP lib for gRPC. Now I wonder how to properly get the metadata from the server in case of an invalid request, this metadata containing mostly constraint violations built by the Java validator and transformed into a collection of gRPC FieldViolation objects. In this example, the server is supposed to return one single field violation as metadata, with the key "violationKey" and the description "violationDescription":
try {
// doStuff
} catch (ConstraintViolationException e) {
Metadata trailers = new Metadata();
trailers.put(ProtoUtils.keyForProto(BadRequest.getDefaultInstance()), BadRequest
.newBuilder()
.addFieldViolations(FieldViolation
.newBuilder()
.setField("violationKey")
.setDescription("violationDescription")
.build()
)
.build()
);
responseObserver.onError(Status.INVALID_ARGUMENT.asRuntimeException(trailers));
}
On the PHP side, this is the implementation to retrieve the metadata:
class Client extends \Grpc\BaseStub
{
public function callService()
{
$call = $this->_simpleRequest(
'MyService/MyAction',
$argument,
['MyActionResponse', 'decode'],
$metadata, $options
);
list($response, $status) = $call->wait();
var_dump($status->metadata); // A
var_dump($call->getMetadata()); // B
}
}
Result: "A" outputs an empty array, "B" outputs the proper metadata, formatted as follows:
array(1) {
["google.rpc.badrequest-bin"]=>
array(1) {
[0]=>
string(75) "
I
testALicense plate number is not in a valid format for country code FR"
}
}
Why is the metadata in the status empty, and why is the metadata retrieved by $call->getMetadata() is formatted that way ("I" followed by the violation key, then "A" and finally the violation description) ? How can I avoid to make potentially tedious transformation of this metadata client-side?
Can you please log an issue on our grpc/grpc Github repo so that we can better follow up there? Thanks.
Related
Is there a way to read only specific fields of a Kafka topic?
I have a topic, say person with a schema personSchema. The schema contains many fields such as id, name, address, contact, dateOfBirth.
I want to get only id, name and address. How can I do that?
Currently I´m reading streams using Apache Beam and intend to write data to BigQuery afterwards. I am trying to use Filter but cannot get it to work because of Boolean return type
Here´s my code:
Pipeline pipeline = Pipeline.create();
PCollection<KV<String, Person>> kafkaStreams =
pipeline
.apply("read streams", dataIO.readStreams(topic))
.apply(Filter.by(new SerializableFunction<KV<String, Person>, Boolean>() {
#Override
public Boolean apply(KV<String, Order> input) {
return input.getValue().get("address").equals(true);
}
}));
where dataIO.readStreams is returning this:
return KafkaIO.<String, Person>read()
.withTopic(topic)
.withKeyDeserializer(StringDeserializer.class)
.withValueDeserializer(PersonAvroDeserializer.class)
.withConsumerConfigUpdates(consumer)
.withoutMetadata();
I would appreciate suggestions for a possible solution.
You can do this with ksqlDB, which also work directly with Kafka Connect for which there is a sink connector for BigQuery
CREATE STREAM MY_SOURCE WITH (KAFKA_TOPIC='person', VALUE_FORMAT=AVRO');
CREATE STREAM FILTERED_STREAM AS SELECT id, name, address FROM MY_SOURCE;
CREATE SINK CONNECTOR SINK_BQ_01 WITH (
'connector.class' = 'com.wepay.kafka.connect.bigquery.BigQuerySinkConnector',
'topics' = 'FILTERED_STREAM',
…
);
You can also do this by creating a new TableSchema by yourself with only the required fields. Later when you write to BigQuery, you can pass the newly created schema as an argument instead of the old one.
TableSchema schema = new TableSchema();
List<TableFieldSchema> tableFields = new ArrayList<TableFieldSchema>();
TableFieldSchema id =
new TableFieldSchema()
.setName("id")
.setType("STRING")
.setMode("NULLABLE");
tableFields.add(id);
schema.setFields(tableFields);
return schema;
I should also mention that if you are converting an AVRO record to BigQuery´s TableRow at some point, you may need to implement some checks there too.
I'm using ElasticSearch as a search service in Spring Web project which using Transport Client to communicate with ES.
I'm wondering if there exists a method which can construct a QueryBuilder from a JSON DSL. for example, convert this bool query DSL JSON to a QueryBuilder.
{
"query" : {
"bool" : {
"must" : { "match" : {"content" : "quick"},
"should": { "match": {"content" : "lazy"}
}
}
}
I need this method because I have to receive user's bool string input from web front-side, and parse this bool string to a QueryBuilder. However it not suit to use QueryBuilders.boolQuery().must(matchQB).should(shouldQB).must_not(mustNotQB). Because we may need several must or non must query.
If there exist a method can construct a QueryBuilder from JSON DSL or there exists alternative solutions, it will much easier.
PS: I have found two method which can wrap a DSL String to a QueryBuilder for ES search.
One is WrapperQueryBuilder, see details here. http://javadoc.kyubu.de/elasticsearch/HEAD/org/elasticsearch/index/query/WrapperQueryBuilder.html
Another is QueryBuilders.wrapperQuery(String DSL).
You can use QueryBuilders.wrapperQuery(jsonQueryString);
You can use setQuery, which can receive a json format string.
/**
* Constructs a new search source builder with a raw search query.
*/
public SearchRequestBuilder setQuery(String query) {
sourceBuilder().query(query);
return this;
}
Note this: only part of the DSL is needed, the {"query": } part is omitted, like this:
SearchResponse searchResponse = client.prepareSearch(indices).setQuery("{\"term\": {\"id\": 1}}").execute().actionGet();
It might be worth investigating low level rest client. With this you can do:
RestClient esClient = RestClient.builder(new HttpHost("localhost", 9200, "http")).build();
Request request = new Request("POST", "/INDEX_NAME/_doc/_search");
request.setJsonEntity(yourJsonQueryString);
Response response = esClient.performRequest(request);
String jsonResponse = EntityUtils.toString(response.getEntity());
I'm doing a Dynamics CRM integration from a Java application and I've followed the example from the CRM training kit and managed successfully to connect and create accounts and contacts.
Now I'm having some problems with adding some more fields in the account creation and when connecting a contact with an account.
For instance I cannot create accounts with "address1_freighttermscode" that is a picklist.
My code is the following:
private static OrganizationServiceStub.Guid createAccount(OrganizationServiceStub serviceStub, String[] args) {
try {
OrganizationServiceStub.Create entry = new OrganizationServiceStub.Create();
OrganizationServiceStub.Entity newEntryInfo = new OrganizationServiceStub.Entity();
OrganizationServiceStub.AttributeCollection collection = new OrganizationServiceStub.AttributeCollection();
if (! (args[0].equals("null") )) {
OrganizationServiceStub.KeyValuePairOfstringanyType values = new OrganizationServiceStub.KeyValuePairOfstringanyType();
values.setKey("name");
values.setValue(args[0]);
collection.addKeyValuePairOfstringanyType(values);
}
if (! (args[13].equals("null"))){
OrganizationServiceStub.KeyValuePairOfstringanyType incoterm = new OrganizationServiceStub.KeyValuePairOfstringanyType();
incoterm.setKey("address1_freighttermscode");
incoterm.setValue(args[13]);
collection.addKeyValuePairOfstringanyType(incoterm);
}
newEntryInfo.setAttributes(collection);
newEntryInfo.setLogicalName("account");
entry.setEntity(newEntryInfo);
OrganizationServiceStub.CreateResponse createResponse = serviceStub.create(entry);
OrganizationServiceStub.Guid createResultGuid = createResponse.getCreateResult();
System.out.println("New Account GUID: " + createResultGuid.getGuid());
return createResultGuid;
} catch (IOrganizationService_Create_OrganizationServiceFaultFault_FaultMessage e) {
logger.error(e.getMessage());
} catch (RemoteException e) {
logger.error(e.getMessage());
}
return null;
}
When it executes, I get this error
[ERROR] Incorrect attribute value type System.String
Does anyone have examples on how to handle picklists or lookups?
To connect the contact with the account I'm filling the fields parentcustomerid and parentcustomeridtype with the GUID from the account and with "account", but the contact does not get associated with the account.
To set a picklist value you must use an OptionSet and for a lookup you must use an EntityReference. See the SDK's C# documentation, should work the same way using the Axis generated Java code.
incoterm.setKey("address1_freighttermscode")
//assuming the arg is an integer value that matches a picklist value for the attribute
OptionSetValue freight = new OptionSetValue();
freight.Value = args[13];
incoterm.setValue(freight);
collection.addKeyValuePairOfstringanyType(incoterm);
I haven't worked with Java for over a decade (and never towards an MS creation like Dynamics) so it might be way off from what you like. :)
You could use the REST web service and call directly to CRM creating your instances. As far I know, that's platform independent and should work as long as you can connect to the exposed service OrganizationData.
I want to use a custom authentication module conforming to JSR 196 in GlassFish 3. The interface javax.security.auth.message.ServerAuth has the method:
AuthStatus validateRequest(
MessageInfo messageInfo,
javax.security.auth.Subject clientSubject,
javax.security.auth.Subject serviceSubject
)
AuthStatus can be one of several constants like FAILURE or SUCCESS.
The question is: How can I get the roles from a "role datebase" with JSR 196?
Example: The server receives a request with a SSO token (CAS token for example), checks whether the token is valid, populates the remote user object with roles fetches from a database via JDBC or from REST service via http.
Is the role fetching in the scope of JSR 196? How could that be implemented?
Do I have to use JSR 196 together with JSR 115 to use custom authentication and a custom role source?
This is a code example from my JSR-196OpenID Implementation.
The method set the roles stored in a String Array for the current CallerPrincipal:
private boolean setCallerPrincipal(String caller, Subject clientSubject) {
boolean rvalue = true;
boolean assignGroups = true;
// create CallerPrincipalCallback
CallerPrincipalCallback cPCB = new CallerPrincipalCallback(
clientSubject, caller);
if (cPCB.getName() == null && cPCB.getPrincipal() == null) {
assignGroups = false;
}
try {
handler.handle((assignGroups ? new Callback[] {
cPCB,
new GroupPrincipalCallback(cPCB.getSubject(),
assignedGroups) } : new Callback[] { cPCB }));
logInfo(DEBUG_JMAC, "jmac.caller_principal:" + cPCB.getName() + " "
+ cPCB.getPrincipal());
} catch (Exception e) {
// should not happen
logger.log(Level.WARNING, "jmac.failed_to_set_caller", e);
rvalue = false;
}
return rvalue;
}
I call this method during the validateRequest() method.
You can see the complete code here:
http://code.google.com/p/openid4java-jsr196/source/browse/trunk/src/main/java/org/imixs/openid/openid4java/OpenID4JavaAuthModule.java
Also this page will be helpfull :
http://code.google.com/p/openid4java-jsr196/
Here's how I map users to roles:
I have 3 roles in my web.xml and also I have 3 role-to-group mappings in my sun-web.xml which map those roles several groups. Then I have a database with table Users that has a column called "group". That group corresponds to the group that is mapped to a role. I also use JSR 196-based custom auth module with OpenID. So basically whenever a user is logged in their group is read from the db and then my app assigns them the corresponding role. This is all done using the standard declarative security model of J2EE.
For my custom auth module I use a library called AuthenticRoast which makes things quite a bit simpler.
Here's also a related post...
Hope this helps.
I am using BlazeDS java client to get info from this page.
This page has a form in the middle that when you select a type, the location combo on the button gets updated.
I am trying to use BlazeDS to get those values in java.
I have been using Charles web proxy to debug, and this are the screenshots from the request and the response:
My code so far is the following:
// Create the AMF connection.
AMFConnection amfConnection = new AMFConnection();
// Connect to the remote url.
String url = "http://orlandoinfo.com/flex2gateway/";
try
{
amfConnection.connect(url);
}
catch (ClientStatusException cse)
{
System.out.println(cse);
return;
}
// Make a remoting call and retrieve the result.
try
{
// amfConnection.registerAlias("flex.messaging.io.ArrayCollection", "flex.messaging.io.ArrayCollection");
amfConnection.call("ColdFusion.getLocations", new Object[] {"consumer", "attractions", "ATTR"});
}
catch (ClientStatusException cse)
{
System.out.println(cse);
}
catch (ServerStatusException sse)
{
System.out.println(sse);
}
// Close the connection.
amfConnection.close();
When I run it I get a:
ServerStatusException
data: ASObject(15401342){message=Unable to find source to invoke, rootCause=null, details=null, code=Server.Processing}
HttpResponseInfo: HttpResponseInfo
code: 200
message: OK
Can anyone spot what's wrong?
Thanks for reading!
I ended up using Charles Web Proxy. Sniffing AMF parameters and running my code with -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=8888
I compare both calls and modify to look alike.
The working code looks like this:
String url = "http://www.theGateWayurl.com";
// Generates the connection to the amf gateway.
AMFConnection amfConnection = new AMFConnection();
// Must register the class that this library will use to load the
// AMF object information.
// The library will read AMF object variables and use setters from
// the java bean stated in this line.
AMFConnection.registerAlias("", new LabelData().getClass().getName());
try {
// Do the connection.
amfConnection.connect(url);
// This page requires a certain headers to function.
// The Content-type is used to sniff with Charles Web Proxy.
amfConnection.addHttpRequestHeader("Content-type", "application/x-amf");
// The Referer is used by the webpage to allow gathering information.
amfConnection.addHttpRequestHeader("Referer", "http://orlandoinfo.com/ws/b2c/sitesearch/customtags/comSearch.swf");
// The rest of the HTTP POST sent by this library is wrapped
// inside a RemotingMessage.
// Prepare the msg to send.
RemotingMessage msg = new RemotingMessage();
// The method called in the server.
msg.setOperation("getLocations");
// Where the request came from. Similar to referer.
msg.setSource("ws.b2c.sitesearch.components.myService");
// The destination is a needed parameter.
msg.setDestination("ColdFusion");
// Create the body with the parameters needed to call the
// operation set with setOperation()
msg.setBody(new Object[] {"consumer", "attractions"});
// This is needed but not used.
msg.setMessageId("xxxxxxxxxx");
// Send the msg.
AcknowledgeMessage reply = (AcknowledgeMessage) amfConnection.call("null", msg);
// Parse the reply from the server.
ArrayCollection body = (ArrayCollection) reply.getBody();
for (Object obj : body) {
LabelData location = (LabelData) obj;
// Do something with the info.
}
} catch (ClientStatusException cse) {
// Do something with the exception.
} catch (ServerStatusException sse) {
// Do something with the exception.
} finally {
amfConnection.close();
}
The LabelData is just a java bean with with two vars: Data and Label.
I tried to comment every line for a better understanding.
Take into account what Stu mention in previous comments about crossdomain.xml to see if you have the rights to do this kind of things.