We Keep Coding

How to replace one document with WebFlux and MongoDB Reactive frameworks?

I have an old project that I would like to modernize using WebFlux and MongoDB Reactive frameworks but I'm absolutely stuck at the update method. I have no clue what to pass in the subscribe method. The replaceOne call returns a Publisher but how I should handle it? One of the tips I read was about using the SubscriberHelpers class from mongo-java-driver-reactivestreams but it's missing from the recent versions (4.4). Any help would be appreciated.
mongoDatabase.getCollection("players", Player.class).replaceOne(
Filters.eq("email", player.getEmail()),
player,
new ReplaceOptions().upsert(true)
).subscribe( ??? );
Update: I created a DummySubscriber class to manage the subscription which seems working properly.
mongoDatabase.getCollection("players", Player.class).replaceOne(
Filters.eq("email", player.getEmail()),
player,
new ReplaceOptions().upsert(true)
).subscribe(new DummySubscriber<>());
private static class DummySubscriber<T> implements Subscriber<T> {
#Override
public void onSubscribe(Subscription subscription) {
subscription.request(Integer.MAX_VALUE);
}
#Override
public void onNext(T t) {
}
#Override
public void onError(Throwable throwable) {
System.out.println("Error while updating data");
}
#Override
public void onComplete() {
System.out.println("Data has been updated");
}
}

After looking at your example i could see that the problem is that you are not using the ReactiveMongoTemplate when doing your calls against the database.
You decided to use the raw MongoDatabase object, which is more low level which does not provide you with the webflux api.
MongoDatabase only implement the standard reactivestreams specification, which does only support something implementing Subscriber<T> as subscribers.
I recommend not using the low level MongoDatabase object and instead use the higher level abstractions.
If you are going to use spring webflux you should be using either the ReactiveRepositories that spring provides, or if you want to be low level use the ReactiveMongoTemplate
private final ReactiveMongoTemplate reactiveMongoTemplate;
#Autowired
public PlayerRepository(ReactiveMongoTemplate reactiveMongoTemplate) {
this.reactiveMongoTemplate = reactiveMongoTemplate;
}
public void savePlayers(Player player) {
final Query query = new Query();
query.addCriteria(Criteria.where("email").is(player.getEmail()));
final Update update = new Update();
update.set("name", player.getName());
update.set("email", player.getEmail());
reactiveMongoTemplate.findAndModify(query, update, Player.class)
.subscribe(
updatedPlayer -> System.out.println("updated player"),
error -> System.out.println("Something went wrong: " + error.getCause()),
() -> System.out.println("update is finished"));
}
I strongly suggest you learn how to use regular MongoTemplate without webflux before you go on to use webflux and the ReactiveMongoTemplate
There are several tutorials, about using MongoTemplate and the ReactiveMongoTemplate
https://www.baeldung.com/spring-data-mongodb-reactive
https://hantsy.github.io/spring-reactive-sample/data/data-mongo.html

Using protobuf with flink

I'm using flink to read data from kafka and convert it to protobuf. The problem I'm facing is when I run the java application I get the below error. If I modify the unknownFields variable name to something else, it works but it's hard to make this change on all protobuf classes.
I also tried to deserialize directly when reading from kafka but I'm not sure what should be the TypeInformation to be returned for getProducedType() method.
public static class ProtoDeserializer implements DeserializationSchema{
#Override
public TypeInformation getProducedType() {
// TODO Auto-generated method stub
return PrimitiveArrayTypeInfo.BYTE_PRIMITIVE_ARRAY_TYPE_INFO;
}
Appreciate all the help. Thanks.
java.lang.RuntimeException: The field protected com.google.protobuf.UnknownFieldSet com.google.protobuf.GeneratedMessage.unknownFields is already contained in the hierarchy of the class com.google.protobuf.GeneratedMessage.Please use unique field names through your classes hierarchy
at org.apache.flink.api.java.typeutils.TypeExtractor.getAllDeclaredFields(TypeExtractor.java:1594)
at org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1515)
at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1412)
at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
at org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
at org.apache.flink.api.java.typeutils.TypeExtractor.privateCreateTypeInfo(TypeExtractor.java:437)
at org.apache.flink.api.java.typeutils.TypeExtractor.getUnaryOperatorReturnType(TypeExtractor.java:306)
at org.apache.flink.api.java.typeutils.TypeExtractor.getFlatMapReturnTypes(TypeExtractor.java:133)
at org.apache.flink.streaming.api.datastream.DataStream.flatMap(DataStream.java:529)
Code:
FlinkKafkaConsumer09<byte[]> kafkaConsumer = new FlinkKafkaConsumer09<>("testArr",new ByteDes(),p);
DataStream<byte[]> input = env.addSource(kafkaConsumer);
DataStream<PBAddress> protoData = input.map(new RichMapFunction<byte[], PBAddress>() {
#Override
public PBAddress map(byte[] value) throws Exception {
PBAddress addr = PBAddress.parseFrom(value);
return addr;
}
});

Maybe you should try this follow:
env.getConfig().registerTypeWithKryoSerializer(PBAddress. class,ProtobufSerializer.class);
or
env.getConfig().registerTypeWithKryoSerializer(PBAddress. class,PBAddressSerializer.class);
public class PBAddressSerializer extends Serializer<Message> {
final private Map<Class,Method> hashMap = new HashMap<Class, Method>();
protected Method getParse(Class cls) throws NoSuchMethodException {
Method method = hashMap.get(cls);
if (method == null) {
method = cls.getMethod("parseFrom",new Class[]{byte[].class});
hashMap.put(cls,method);
}
return method;
}
#Override
public void write(Kryo kryo, Output output, Message message) {
byte[] ser = message.toByteArray();
output.writeInt(ser.length,true);
output.writeBytes(ser);
}
#Override
public Message read(Kryo kryo, Input input, Class<Message> pbClass) {
try {
int size = input.readInt(true);
byte[] barr = new byte[size];
input.read(barr);
return (Message) getParse(pbClass).invoke(null,barr);
} catch (Exception e) {
throw new RuntimeException("Could not create " + pbClass, e);
}
}
}

try this:
public class ProtoDeserializer implements DeserializationSchema<PBAddress> {
#Override
public TypeInformation<PBAddress> getProducedType() {
return TypeInformation.of(PBAddress.class);
}

https://issues.apache.org/jira/browse/FLINK-11333 is the JIRA ticket tracking the issue of implementing first-class support for Protobuf types with evolvable schema. You'll see that there was a pull request quite some time ago, which hasn't been merged. I believe the problem was that there is no support there for handling state migration in cases where Protobuf was previously being used by registering it with Kryo.
Meanwhile, the Stateful Functions project (statefun is a new API that runs on top of Flink) is based entirely on Protobuf, and it includes support for using Protobuf with Flink: https://github.com/apache/flink-statefun/tree/master/statefun-flink/statefun-flink-common/src/main/java/org/apache/flink/statefun/flink/common/protobuf. (The entry point to that package is ProtobufTypeInformation.java.) I suggest exploring this package (which includes nothing statefun specific); however, it doesn't concern itself with migrations from Kryo either.

Use Retrofit to consume Server Sent Events

I am trying to consume a rest api [1] that sends server sent events to the client.
I am currently using retrofit from square to consume this but I am not sure how to do it.
Can some one who has prior experience working with retrofit help ?
If not retrofit please suggest other Java libraries that can do that.
[1] https://mesosphere.github.io/marathon/docs/rest-api.html#get-v2-events

Try this library : oksee.
OkSse is an extension library for OkHttp to create a Server-Sent Event (SSE) client
As I experienced the same problem,from my research it is the best option for now, as Retrofit doesn't support it.
https://github.com/heremaps/oksse

I know that it is old question. But I didn't find a whole example and now try to provide it by my code. We use only retrofit and coroutines
1.In retrofit API interface need to add code. Pay attention what we use #Streaming and return type Call<ResponseBody>
#POST("/v1/calc/group-prices")
#Streaming
fun calculateGroupPrices(#Body listOptions: List<GroupCalculatorOptions>): Call<ResponseBody>
2.In your repository class need to add this code. Pay attention what we use flow and read a stream. To understand that a message with a payload has arrived, it must begin with "data:"
fun loadGroupDeliveryRateInfos(listOptions: List<GroupCalculatorOptions>) = flow {
coroutineScope {
val response = restApi.calculateGroupPrices(listOptions).execute()
if (response.isSuccessful) {
val input = response.body()?.byteStream()?.bufferedReader() ?: throw Exception()
try {
while (isActive) {
val line = input.readLine() ?: continue
if (line.startsWith("data:")) {
try {
val groupDeliveryRateInfo = gson.fromJson(
line.substring(5).trim(),
GroupDeliveryRateInfo::class.java
)
emit(groupDeliveryRateInfo)
} catch (e: Exception) {
e.printStackTrace()
}
}
}
} catch (e: IOException) {
throw Exception(e)
} finally {
input.close()
}
} else {
throw HttpException(response)
}
}
}
3.Final step we need to collect our data in ViewModel. We just need to call the method from repository
repository.loadGroupDeliveryRateInfos(it.values.toList())
.collect { info ->
handleGroupDeliveryRateInfo(info)
}
And that's all, no additional libraries are needed.

There's no real need to conflate Retrofit and SSE. Use retrofit to get an inputstream, then find (or write) an inputstream parser that chunks up the SSE events.
In retrofit I have this:
public interface NotificationAPI {
#GET("notifications/sse")
Call<InputStream> getNotificationsStream(#retrofit2.http.Query("Last-Event-ID") String lastEventId);
}
I wrote a quick converter factory for InputStream:
public class InputStreamConverterFactory extends Converter.Factory {
private static class InputStreamConverter implements Converter<ResponseBody, InputStream> {
#Nullable
#Override
public InputStream convert(ResponseBody value) throws IOException {
return value.byteStream();
}
}
#Override
public #Nullable
Converter<ResponseBody, ?> responseBodyConverter(Type type, Annotation[] annotations, Retrofit retrofit) {
if (type.equals(InputStream.class)) {
return new InputStreamConverter();
}
return null;
}
}
My client code looks like this:
var cinputStream = api.getNotificationsStream(null);
var inputStream = cinputStream.execute().body();
try(var sseStream = new MySSEStreamParser(inputStream)) {
//handle the stream here...
}
There is an OkHttp SSE parser, which you probably could use. However:
The OkHttp SSE code comes with threads. Chances are you'd want to bring your own threading model.
The actual OkHttp SSE parser is an internal package. Which doesn't make it a great candidate for lifting.

Play Framework 2.x: How to update a displayed view from server side?

I have some problems with the play! framework.
The framework is new for me and my first and only practise with it, is this tutorial here.
I have a view which lists up some data. A Controller which have one method to show the view and another method which can receive some data with JSON. Now i would like to update the view when new data(JSON) is received.
I found out, that i can use either a comet socket, a websocket or server-sent events to implement this.
(A permanent AJAX-polling ist not a option for me to do it.)
What is the best way to do this?
I tried to solve it with websockets, but it does not work as i want...
Here is my code:
public class MyController extends Controller{
public static Result showMyView() {
return ok(views.html.showMyView.render(Data.all()));
}
#BodyParser.Of(play.mvc.BodyParser.Json.class)
public static Result jsonReceiver() {
JsonNode json = request().body().asJson();
String rcvData = json.findPath("someData").textValue();
if(rcvData != null) {
/*FLAG*/
//SEND SOME DATA OVER THE WEBSOCKET FROM HERE?! ..To update the view
//something like: out.write("Now from here!");
return ok();
} else {
return badRequest();
}
}
public static WebSocket<String> websocket() {
return new WebSocket<String>() {
public void onReady(WebSocket.In<String> in, WebSocket.Out<String> out) {
out.write("Hello!");
}
};
}
}
Is there any way to send messages or other data over the socket from the position /*FLAG*/ in the code above?
My view looks like this:
#(dataContent: List[Data])
#main(title = "MyViews Observation Page") {
#*
SOME HTML CODE ...
*#
<script type="text/javascript">
var socket = new WebSocket("ws://" + location.host + "/path/to/my/websocket")
socket.onmessage = function(event) {
alert("Received: " + event.data); // only for testing... will be replaced later
}
</script>
}
Can anybody help me here? I don't know how to update the view, when new data is received.
I would be very grateful if you could show it on the basis of my example code above.
if it is easier to make it in a different way(comet or server-sent events), I am also very grateful for examples.
In my previous investigation i have found a few examples for comet and server-sent events but they were only in scala and i haven't understood them.
I'm using play framework 2.2.2 with java.

Each client will generate a WebSocket.Out on your server.
Store them in a List, and then call WebSocket.Out.write(objectNode)
public class MyController extends Controller{
//1. you store your client channels here
private static ArrayList<WebSocket.Out> channels=new ArrayList<>();
public static Result showMyView() {
return ok(views.html.showMyView.render(Data.all()));
}
#BodyParser.Of(play.mvc.BodyParser.Json.class)
public static Result jsonReceiver() {
JsonNode json = request().body().asJson();
String rcvData = json.findPath("someData").textValue();
if(rcvData != null) {
//3. you can write in your channels :)
for(WebSocket.Out channel : channels){
channel.write("blah");
}
return ok();
} else {
return badRequest();
}
}
public static WebSocket<String> websocket() {
return new WebSocket<String>() {
public void onReady(WebSocket.In<String> in, WebSocket.Out<String> out) {
//2. each time a client connects, you save a reference to its channel
channels.add(out);
}
};
}
}

Networking in Android/Java - how do I callback to a API class after a call back from the server?

So I'm writing an Android application in Java based on an iOS application that I am also working on, but this question is more asking about how to communicate callback mechanism (like blocks in Objective-C 2.0) in Java.
This application involves networking, authenticating and communicating with a server via an API.
I am using this framework: https://github.com/loopj/android-async-http
I am trying to encapsulate all of the networking model into classes to make everything clean and easy (it seems so easy in iOS with delegates and blocks, but java doesn't appear to have ANY of these conveniences). So, I am using this as a guide for callbacks: http://www.gdgankara.org/2013/03/25/android-asynchronous-http-client-a-callback-based-http-client-library-for-android-and-android-smart-image-view/
Now lets say I don't want to make a call from an Activity class, but an API class, which can be called from an Activity class, how can I do this? I know easily how to do this with blocks and delegates in iOS, but how can I do this with interfaces?
For Example:
In iOS (using a common networking framework called AFNetworking), I have 4 classes:
HTTPClient.h/m
+(id)sharedHTTPClient
{
static dispatch_once_t pred = 0;
__strong static id __httpClient = nil;
dispatch_once(&pred, ^{
NSString *baseURL = http://baseurl.com;
__httpClient = [[self alloc] initWithBaseURL:[NSURL URLWithString:baseURL]];
[__httpClient setParameterEncoding:AFJSONParameterEncoding];
});
return __httpClient;
}
APILogin.h/m
-(void)loginWithSuccessBlock:(void (^)(NSArray *responseArray))loginSuccess {
HTTPClient *httpClient = [HTTPClient sharedHTTPClient];
NSURLRequest *request = [httpClient requestWithMethod:#"GET" path:#"/api/login" parameters:nil];
AFJSONRequestOperation *operation = [AFJSONRequestOperation JSONRequestOperationWithRequest:request success:^(NSURLRequest *request, NSHTTPURLResponse *response, id JSON) {
NSArray *response = [self.jsonParser parseResponseFromJSON:JSON];
if (loginSuccess) {
loginSuccess(response);
}
} failure:^(NSURLRequest *request, NSHTTPURLResponse *response, NSError *error, id JSON) {
[APIErrorHandler handleError:error withHTTPResponse:response];
}];
[operation start];
}
LoginObject.h/m
-(id)init
{
self = [super init];
if(self) {
[self.apiLogin loginWithSuccessBlock:^ void (NSArray *loginArray) {
//process the array
}];
}
}
LoginVC.h/m
...
LoginObject *loginObj = [[LoginObject alloc] init];
...
So, now what I have so far, using the Android-Async-Http library is:
HTTPClient.java
public class HTTPClient extends AsyncHttpClient {
public static HTTPClient sharedHTTPClient;
public static String baseUrl = "http://baseurl.com";
public HTTPClient {
super();
}
...
}
APILogin.java
public class APILogin {
public void loginWithSuccessBlock() {
HTTPClient httpClient = QHTTPClient.sharedHTTPClient;
httpClient.get("/api/login", new JsonHttpResponseHandler() {
#Override
public void onSuccess(JSONArray response) {
// Successfully got a response
ArrayList<LoginObject> loginInfo = this.jsonParser.parseLoginFromJSON(response);
**NEED TO DO A CALL BACK!! Like with the blocks in iOS**
}
#Override
public void onSuccess(JSONObject response) {
// Successfully got a response
// shouldn't be an object
throw new IllegalArgumentException();
}
#Override
public void onFailure(Throwable e, String response) {
// Response failed :(
}
});
}
LoginObject.java
public class LoginObject {
public LoginObject {
this.apiLogin.loginWithSuccessBlock(**something something for a callback**);
}
}
Hopefully, I've made it a little clearer what I am trying to achieve. I want to be able to execute some kind of callback block on the object that called the api call, on success. However, it will not always be the same object. LoginObject may have an instance of APILogin.java and so might a different object, so I can't use the second link above in which you can specific a specific class and pass it in and call a method on it, because the classes will be of different types, and Java doesn't have a generic pointer (id or void*) object.

So I've discovered my own answer after trying many things and scouring the web for a possible solution. What I've come up with is to basically chain the response handlers.
So for:
public class APILogin {
public void loginWithSuccessBlock(**final JsonHttpResponseHandler handler**) {
HTTPClient httpClient = QHTTPClient.sharedHTTPClient;
httpClient.get("/api/login", handler);
}
}
public class LoginObject {
public LoginObject {
this.apiLogin.loginWithSuccessBlock(new JsonHttpResponseHandler(){
...
);
}
}
which isn't very robust because it doesn't let me do much customization, but it'll do.