Java - Flink sending empty object on kafka sink - java

On my flink script I have a stream that I'm getting from one kafka topic, manipulate it and sending it back to kafka using the sink.
public static void main(String[] args) throws Exception {
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setParallelism(1);
env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
Properties p = new Properties();
p.setProperty("bootstrap.servers", servers_ip_list);
p.setProperty("gropu.id", "Flink");
FlinkKafkaConsumer<Event_N> kafkaData_N =
new FlinkKafkaConsumer("CorID_0", new Ev_Des_Sch_N(), p);
WatermarkStrategy<Event_N> wmStrategy =
WatermarkStrategy
.<Event_N>forMonotonousTimestamps()
.withIdleness(Duration.ofMinutes(1))
.withTimestampAssigner((Event, timestamp) -> {
return Event.get_Time();
});
DataStream<Event_N> stream_N = env.addSource(
kafkaData_N.assignTimestampsAndWatermarks(wmStrategy));
The part above is working fine no problems at all, the part below instead is where I'm getting the issue.
String ProducerTopic = "CorID_0_f1";
DataStream<Stream_Blocker_Pojo.block> box_stream_p= stream_N
.keyBy((Event_N CorrID) -> CorrID.get_CorrID())
.map(new Stream_Blocker_Pojo());
FlinkKafkaProducer<Stream_Blocker_Pojo.block> myProducer = new FlinkKafkaProducer<>(
ProducerTopic,
new ObjSerializationSchema(ProducerTopic),
p,
FlinkKafkaProducer.Semantic.EXACTLY_ONCE); // fault-tolerance
box_stream_p.addSink(myProducer);
No errors everything works fine, this is the Stream_Blocker_Pojo where I'm mapping a stream manipulating it and sending out a new one.(I have simplify my code, just keeping 4 variables and removing all the math and data processing).
public class Stream_Blocker_Pojo extends RichMapFunction<Event_N, Stream_Blocker_Pojo.block>
{
public class block {
public Double block_id;
public Double block_var2 ;
public Double block_var3;
public Double block_var4;}
private transient ValueState<block> state_a;
#Override
public void open(Configuration parameters) throws Exception {
state_a = getRuntimeContext().getState(new ValueStateDescriptor<>("BoxState_a", block.class));
}
public block map(Event_N input) throws Exception {
p1.Stream_Blocker_Pojo.block current_a = state_a.value();
if (current_a == null) {
current_a = new p1.Stream_Blocker_Pojo.block();
current_a.block_id = 0.0;
current_a.block_var2 = 0.0;
current_a.block_var3 = 0.0;
current_a.block_var4 = 0.0;}
current_a.block_id = input.f_num_id;
current_a.block_var2 = input.f_num_2;
current_a.block_var3 = input.f_num_3;
current_a.tblock_var4 = input.f_num_4;
state_a.update(current_a);
return new block();
};
}
This is the implementation of the Kafka Serialization schema.
public class ObjSerializationSchema implements KafkaSerializationSchema<Stream_Blocker_Pojo.block>{
private String topic;
private ObjectMapper mapper;
public ObjSerializationSchema(String topic) {
super();
this.topic = topic;
}
#Override
public ProducerRecord<byte[], byte[]> serialize(Stream_Blocker_Pojo.block obj, Long timestamp) {
byte[] b = null;
if (mapper == null) {
mapper = new ObjectMapper();
}
try {
b= mapper.writeValueAsBytes(obj);
} catch (JsonProcessingException e) {
}
return new ProducerRecord<byte[], byte[]>(topic, b);
}
}
When I open the messages that i sent from my Flink script using kafka, I find that all the variables are "null"
CorrID b'{"block_id":null,"block_var1":null,"block_var2":null,"block_var3":null,"block_var4":null}
It looks like I'm sending out an empty obj with no values. But I'm struggling to understand what I'm doing wrong. I think that the problem could be into my implementation of the Stream_Blocker_Pojo or maybe into the ObjSerializationSchema, Any help would be really appreciated. Thanks

There are two probable issues here:
Are You sure the variable You are passing of type block doesn't have null fields? You may want to debug that part to be sure.
The reason may also be in ObjectMapper, You should have getters and setters available for Your block otherwise Jackson may not be able to access them.

Related

How to convert an input into a method that the program will understand

I'm currently working on my homework about client and server topic:
I created a class called VehileRequest.java which will take three variables (year, make, model) from Client.java and pass them to Server.java then the Server will get information from VehicleRespone.java and display the information about price, miles vv...
As I understand, the program didn't recognize the request that pass to the respond file.
I got stuck at passing the Request to Response so that the Response will understand. Any help please. thank you
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.databind.ObjectMapper;
public class VehicleResponse {
private VehicleRequest request;
private int milesOnVehicle;
private int price;
private int numberOfSeats;
private int numberOfDoors;
private String[] options;
#JsonIgnore
private static final ObjectMapper objectMapper = new ObjectMapper();
public static String toJSON(VehicleResponse vehicle) throws Exception {
return objectMapper.writeValueAsString(vehicle);
}
public static VehicleResponse fromJSON(String input) throws Exception{
return objectMapper.readValue(input, VehicleResponse.class);
}
protected VehicleResponse() {}
public VehicleResponse(VehicleRequest request, int milesOnVehicle,int price, int numberOfSeats, int numberOfDoors,String[] options) {
this.request=request;
this.milesOnVehicle=milesOnVehicle;
this.price=price;
this.numberOfSeats=numberOfSeats;
this.numberOfDoors=numberOfDoors;
this.options=options;
}
#Override
public String toString() {
return String.format(
"Vehicle request:[miles=%d, price=%d, number of seats=%d,number of doors=%d, option='%s']",
milesOnVehicle,price,numberOfSeats,numberOfDoors,options);
}
public VehicleRequest getRequest() {return request;}
public int getMilesOnVehicle(){return milesOnVehicle;};
public int getPrice(){return price;}
public int getNumberOfDoors() {return numberOfDoors;}
public int getNumberOfSeats() {return numberOfSeats;}
public String[] getOptions() {return options;}
public void setRequest(VehicleRequest request) {
this.request = request;
}
public void setMilesOnVehicle(int milesOnVehicle) {
this.milesOnVehicle = milesOnVehicle;
}
public void setPrice(int price) {
this.price = price;
}
public void setNumberOfSeats(int numberOfSeats) {
this.numberOfSeats = numberOfSeats;
}
public void setNumberOfDoors(int numberOfDoors) {
this.numberOfDoors = numberOfDoors;
}
public void setOptions(String[] options) {
this.options = options;
}
}
Here is the VehicleRequest file
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.databind.ObjectMapper;
public class VehicleRequest {
private int year;
private String make;
private String model;
#JsonIgnore
private static final ObjectMapper objectMapper = new ObjectMapper();
public static String toJSON(VehicleRequest vehicle) throws Exception {
return objectMapper.writeValueAsString(vehicle);
}
public static VehicleRequest fromJSON(String input) throws Exception{
return objectMapper.readValue(input, VehicleRequest.class);
}
protected VehicleRequest() {}
public VehicleRequest(int year, String make, String model) {
this.year = year;
this.make =make;
this.model=model;
}
#Override
public String toString() {
return String.format(
"Vehicle: [year=%d, make='%s', model='%s']",
year,make,model);
}
public int getYear() {
return year;
}
public String getMake(){return make;}
public String getModel(){return model;}
public void setYear(int year) {
this.year = year;
}
public void setMake(String make){
this.make=make;
}
public void setModel(String model){
this.model=model;
}
}
Here is the Server
public class Server {
private ServerSocket serverSocket;
private Socket clientSocket;
private PrintWriter out;
private BufferedReader in;
public void start(int port) throws Exception {
serverSocket = new ServerSocket(port);
clientSocket = serverSocket.accept();
out = new PrintWriter(clientSocket.getOutputStream(), true);
in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null) {
VehicleRequest request = VehicleRequest.fromJSON(inputLine);
VehicleResponse response = new VehicleResponse(request,10000,12000,5,4, null);
out.println(VehicleResponse.toJSON(response));
}
}
public void stop() throws IOException {
in.close();
out.close();
clientSocket.close();
serverSocket.close();
}
public static void main(String[] args) {
Server server = new Server();
try {
server.start(4444);
server.stop();
} catch(Exception e) {
e.printStackTrace();
}
}
}
Here is the client
public class Client {
private Socket clientSocket;
private PrintWriter out;
private BufferedReader in;
public void startConnection(String ip, int port) throws IOException {
clientSocket = new Socket(ip, port);
out = new PrintWriter(clientSocket.getOutputStream(), true);
in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
}
public VehicleRequest sendRequest() throws Exception {
out.println(VehicleRequest.toJSON(new VehicleRequest(2008,"Honda","Civic")));
return VehicleRequest.fromJSON(in.readLine());
}
public void stopConnection() throws IOException {
in.close();
out.close();
clientSocket.close();
}
public static void main(String[] args) {
Client client = new Client();
try {
client.startConnection("127.0.0.1", 4444);
System.out.println(client.sendRequest().toString());
client.stopConnection();
} catch(Exception e) {
e.printStackTrace();
}
}
}
I got the result was
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "request" (class edu.sdccd.cisc191.template.VehicleRequest), not marked as ignorable (3 known properties: "model", "year", "make"])
at [Source: (String)"{"request":{"year":2008,"make":"Honda","model":"Civic"},"milesOnVehicle":10000,"price":12000,"numberOfSeats":5,"numberOfDoors":4,"options":null}"; line: 1, column: 13] (through reference chain: edu.sdccd.cisc191.template.VehicleRequest["request"])
at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61)
at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:855)
at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1212)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1604)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1582)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:299)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:156)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4482)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3434)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3402)
at edu.sdccd.cisc191.template.VehicleRequest.fromJSON(VehicleRequest.java:17)
at edu.sdccd.cisc191.template.Client.sendRequest(Client.java:32)
at edu.sdccd.cisc191.template.Client.main(Client.java:44)
}
In the line VehicleRequest.fromJSON(in.readLine()); you are trying to parse whatever it is the input, which seems to be:
{
"request":{
"year":2008,
"make":"Honda",
"model":"Civic"
},
"milesOnVehicle":10000,
"price":12000,
"numberOfSeats":5,
"numberOfDoors":4,
"options":null
}
However, you are expecting this to be parseable to VehicleRequest which is not possible because it contains only 3 parameters and not all that. Either you parse this as VehicleResponse as follows:
VehicleResponse.fromJSON(in.readLine());
Or you change the input to be something that can be parsed to VehicleRequest:
{
"year":2008,
"make":"Honda",
"model":"Civic"
}
If I am understanding your code correctly you are trying to make communication between Client and Server. In that case, you will need to change your Client code:
public class Client {
private Socket clientSocket;
private PrintWriter out;
private BufferedReader in;
public void startConnection(String ip, int port) throws IOException {
clientSocket = new Socket(ip, port);
out = new PrintWriter(clientSocket.getOutputStream(), true);
in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
}
public VehicleResponse sendRequest() throws Exception {
out.println(VehicleRequest.toJSON(new VehicleRequest(2008,"Honda","Civic")));
return VehicleResponse.fromJSON(in.readLine());
}
public void stopConnection() throws IOException {
in.close();
out.close();
clientSocket.close();
}
public static void main(String[] args) {
Client client = new Client();
try {
client.startConnection("127.0.0.1", 4444);
System.out.println(client.sendRequest().toString());
client.stopConnection();
} catch(Exception e) {
e.printStackTrace();
}
}
}
Although João Dias already solved your lapse by fixing the response-type, I would like to share some practical advice on JSON-conversion especially when implementing web-API facades like your Client / Server classes:
pure-functions with parameters make code more testable: send(request)
content-conversion (from/into formats, like JSON) is a (cross-cutting) separate concern: inject a reusable component like ObjectMapper into client and server (representation layer) to make your request/response classes format-independent
The resulting code might appeal to you as
more readable
easier to comprehend and debug
less coupled
easier to maintain and extend
Request to response: send as pure function
Usually the responsibility of a client is to send multiple different requests. So parameterize your send method.
Design issues
Before: No parameter but static hard-coded request. Thus not testable.
public VehicleRequest sendRequest() throws Exception {
out.println(VehicleRequest.toJSON(new VehicleRequest(2008,"Honda","Civic")));
return VehicleRequest.fromJSON(in.readLine());
}
Note: the method signature ...Request sendRequest() can be read like "output = produceInput()". This style of designing methods or functions is called pure function: it usually converts an input to a new output - it produces something, like a response.
Design issues:
Here no input can be given. The returned product or answer could be expected to be always the same. Wouldn't you like to ask different questions or send different requests. So that the expected answer depends on the given request.
Improve
After: Added parameter to make testable with different requests. Method name simplified.
public VehicleResponse send(VehicleRequest request) throws Exception {
out.println(VehicleRequest.toJSON(request));
return VehicleResponse.fromJSON(in.readLine());
}
This can be called in your main or in a test like response = client.send(new VehicleRequest(2008,"Honda","Civic")).
Object representation: JSON-conversion as separate-concern
There are some basic representation formats like String that are tightly coupled to each class. In Java this concern is inherited from Object class's methods toString() and for some wrapper-classes like Integer.valueOf(String s).
However more complex representations, especially REST- or HTTP-related like JSON, XML, HTML, but also binary formats are usually treated by a design principle called Separation of Concerns. Especially in web-applications (like your simplified client-server architecture) this leads to separate layers (best-practice):
Representation layer (concern of UI, Web, Communication) .. e.g. your Client / Server classes and Jackson's ObjectMapper handling JSON-conversion
Business layer (concern of Logic) .. e.g. your Vehicle.. classes
Persistence layer (concern of Storage, Database, etc.)
See these separate concerns illustrated as layers, e.g. in Wikipedia Multi-tier Architecture or in Robert C. Martin's Clean Architecture.
Design issues
The external dependency Jackson is duplicated part of your DTO-classes like ..Request and ..Response. So both depend on Jackson and JSON-format.
Before: Concern of representation layer is mixed into DTO-classes, which might change along with Business layer. Now they need to change with each format, the client can support, too (strong-coupling and 2 reasons to change).
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.databind.ObjectMapper;
// ..
#JsonIgnore
private static final ObjectMapper objectMapper = new ObjectMapper();
// duplicated code, implemented similarly for both classes
public static String toJSON(VehicleRe.. vehicle) throws Exception {
return objectMapper.writeValueAsString(vehicle);
}
public static VehicleRe.. fromJSON(String input) throws Exception{
return objectMapper.readValue(input, VehicleRe.. .class);
}
Note: You could move this cross-cutting (used by many classes) concern into the client. This way, your (entity) classes stay independent from the format/mapping-framework and the client can be easily extended to support other formats (like XML) in the future.
Improve
After: In REST formats are concern of the representation layer. The client is responsible of content-negotiation and conversion.
class Client {
private final ObjectMapper mapper;
// Mapper can be injected as dependency, configurable from ouside
public Client(ObjectMapper mapper) {
this.mapper = mapper;
// optional remaining initialization, could add ip, port, etc.
}
public VehicleResponse send(VehicleRequest vehicle) throws Exception {
String request = objectMapper.writeValueAsString(vehicle); // to <textual-format>
out.println(request); // request as String (now JSON, future XML)
String response = in.readLine(); // request as String (now JSON, future XML)
return objectMapper.readValue(response); // from <textual-format>
}
}
This way you can in your main-method simply create a special XML-capable client by configuring like
Client xmlClient = new Client(new XmlMapper()) or use as-is (with JSON as content-exchange format): Client client = new Client(new ObjectMapper()).
For your CS-course and homework this might be too much and over-engineering.
However in "real code", as seen in professional web-frameworks like Spring, this is good or best-practice and contributes to extensible and maintainable software, see SOLID .

Modify HashMap when CompletableFuture is finished

I have a multi-module system, where one module handles my database storage. This is the method which saves a document:
public CompletableFuture<?> runTransaction() {
return CompletableFuture.runAsync(() -> {
TransactionBody txnBody = (TransactionBody<String>) () -> {
MongoCollection<Document> collection = transaction.getDatabase().getCollection(transaction.getCollection().toString());
collection.insertOne(session, Document.parse(json));
return "Completed";
};
try {
session.withTransaction(txnBody);
} catch (MongoException ex) {
throw new UncheckedMongoException(ex);
}
});
}
the json instance is passed down in the object constructor. However, since this will be used by several modules, with each individual caching system, I'm trying to figure out how the caller can modify data structure, if this method completed without any errors.
For example
public void createClan(Transaction transaction, int id, int maxPlayers) {
MongoTransaction mongoTransaction = (MongoTransaction) transaction;
Clan clan = new Clan(id, maxPlayers);
String json = gson.toJson(clan);
TransactionExecutor executor = new MongoTransactionExecutor(mongoTransaction, json);
executor.runTransaction(); //Returns the completableFuture instance generated by the method. Modify hashmap here.
}
I've tried reading the docs, however it was a bit confusing, any help is appreciated!
As given in the comments, two options can be considered.
First option is to convert the async nature into sync nature using CompletableFuture#get. This way, the code execution is in a blocking context.
public void createClan(Transaction transaction, int id, int maxPlayers) {
MongoTransaction mongoTransaction = (MongoTransaction) transaction;
Clan clan = new Clan(id, maxPlayers);
String json = gson.toJson(clan);
TransactionExecutor executor = new MongoTransactionExecutor(mongoTransaction, json);
try {
Object obj = executor.runTransaction().get();
// HashMap update here
} catch(Exception e) {
//handle exceptions
}
}
Second option is to keep the async nature as is and chain using thenRun (there are many then options available). This way is more a non-blocking context.
public void createClan(Transaction transaction, int id, int maxPlayers) {
MongoTransaction mongoTransaction = (MongoTransaction) transaction;
final Clan clan = new Clan(id, maxPlayers);
String json = gson.toJson(clan);
TransactionExecutor executor = new MongoTransactionExecutor(mongoTransaction, json);
try {
executor.runTransaction().thenRun(() -> updateHashMap(clan));
} catch(Exception e) {
//handle exceptions
}
}

How to Validate ui data with rest api responses

I need to validate my ui data and api responses are same,
here is my code I tried,
private ValidateContentPage cp = new ValidateContentPage();
public void getTitle() {
String UITitle = driver.findElement(titlepage).getText();
System.out.println(UITitle);
Assert.assertEquals(UITitle, cp.getAPICall(),"Passed");
}
here im getting my api responses,
public class ValidateContentPage {
public common cm = new common();
public Properties prop;
public void baseURI() {
prop = cm.getProperties("./src/test/API/IndiaOne/propertyfile/EndpointURL.properties");
RestAssured.baseURI = prop.getProperty("baseURI");
}
public String getAPICall() {
objectpojo ps = given().expect().defaultParser(Parser.JSON).when().get(prop.getProperty("resources")).as(objectpojo.class, cm.getMapper());
int number = ps.getPosts().size();
System.out.println(number);
System.out.println(ps.getPosts().get(0).getTitle());
return ps.getPosts().get(0).getTitle();
}
If i validate both using testng assertion it throwing null pointer exception, anyone help me on how to validate my ui data and api responses.
You need to call your ValidateContentPage from #Test itself or from #BeforeTest
#Test
public void getTitle() {
String UITitle = driver.findElement(titlepage).getText();
System.out.println(UITitle);
ValidateContentPage cp = new ValidateContentPage();
Assert.assertEquals(UITitle, cp.getAPICall(),"Passed");
}

Handling dead-letter queue message-broker independent way

I have a project that currently uses Spring Cloud Streams and RabbitMQ underneath. I've implemented a logic based on the documentation. See below:
#Component
public class ReRouteDlq {
private static final String ORIGINAL_QUEUE = "so8400in.so8400";
private static final String DLQ = ORIGINAL_QUEUE + ".dlq";
private static final String PARKING_LOT = ORIGINAL_QUEUE + ".parkingLot";
private static final String X_RETRIES_HEADER = "x-retries";
private static final String X_ORIGINAL_EXCHANGE_HEADER = RepublishMessageRecoverer.X_ORIGINAL_EXCHANGE;
private static final String X_ORIGINAL_ROUTING_KEY_HEADER = RepublishMessageRecoverer.X_ORIGINAL_ROUTING_KEY;
#Autowired
private RabbitTemplate rabbitTemplate;
#RabbitListener(queues = DLQ)
public void rePublish(Message failedMessage) {
Map<String, Object> headers = failedMessage.getMessageProperties().getHeaders();
Integer retriesHeader = (Integer) headers.get(X_RETRIES_HEADER);
if (retriesHeader == null) {
retriesHeader = Integer.valueOf(0);
}
if (retriesHeader < 3) {
headers.put(X_RETRIES_HEADER, retriesHeader + 1);
String exchange = (String) headers.get(X_ORIGINAL_EXCHANGE_HEADER);
String originalRoutingKey = (String) headers.get(X_ORIGINAL_ROUTING_KEY_HEADER);
this.rabbitTemplate.send(exchange, originalRoutingKey, failedMessage);
}
else {
this.rabbitTemplate.send(PARKING_LOT, failedMessage);
}
}
#Bean
public Queue parkingLot() {
return new Queue(PARKING_LOT);
}
}
It does what it is expected, however, it is binded to RabbitMQ, and my company is planning to stop using this message broker in one year or two (don't know why, must be some crazy business). So, I want to implement the same thing, but detach it from any message broker.
I tried changing the rePublish method this way, but it does not work:
#StreamListener(Sync.DLQ)
public void rePublish(Message failedMessage) {
Map<String, Object> headers = failedMessage.getHeaders();
Integer retriesHeader = (Integer) headers.get(X_RETRIES_HEADER);
if (retriesHeader == null) {
retriesHeader = Integer.valueOf(0);
}
if (retriesHeader < 3) {
headers.put(X_RETRIES_HEADER, retriesHeader + 1);
String exchange = (String) headers.get(X_ORIGINAL_EXCHANGE_HEADER);
String originalRoutingKey = (String) headers.get(X_ORIGINAL_ROUTING_KEY_HEADER);
this.rabbitTemplate.send(exchange, originalRoutingKey, failedMessage);
}
else {
this.rabbitTemplate.send(PARKING_LOT, failedMessage);
}
}
It fails because the Message class has immutable Headers - throws exception on the put attempt saying you can't change its values (uses org.springframework.messaging.Message class).
Is there a way to implement this dead-letter queue handler in a message broker independent way?
Use
MessageBuilder.fromMessage(message)
.setHeader("foo", "bar")
...
.build();
Note that the message in #StreamListener is a spring-messaging Message<?>, not a spring-amqp Message and can't be sent using the template that way; you need an output binding to send the message to.

Java netty rxtx pipeline blocks after writeAndFlush()

I'm currently trying to achieve a somewhat stable connection between a micro-controller and a Java-application using netty 4.0.44.Final and rxtx. From time to time the controller asks for a time-stamp, otherwise it is just forwarding sensor data to my application. The application is able to receive as many packages as I want to until i call writeAndFlush() somewhere in the pipeline (i.e. answering a time-request). The pipeline correctly writes data on the outputstream (when writeAndFlush() is called) and from that point onwards my application is never receiving data again and I have no idea why.
public class WsnViaRxtxConnector extends AbstractWsnConnector{
private static final Logger LOG = LoggerFactory.getLogger(WsnViaRxtxConnector.class);
private String port;
private Provider<MessageDeserializer> deserializerProvider;
private ChannelFuture channelFuture;
public ChannelKeeper keeper;
#Inject
public WsnViaRxtxConnector(Provider<MessageDeserializer> deserializerProvider, ChannelKeeper keeper) {
this.deserializerProvider = deserializerProvider;
this.port = Configuration.getConfig().getString("rest.wsn.port");
this.keeper = keeper;
System.setProperty("gnu.io.rxtx.SerialPorts", this.port);
}
#Override
protected void run() throws Exception
{
EventLoopGroup group = new OioEventLoopGroup();
//final EventExecutorGroup group2 = new DefaultEventExecutorGroup(1500);
try {
Bootstrap b = new Bootstrap();
b.group(group)
.channel(RxtxChannel.class)
.handler(new ChannelInitializer<RxtxChannel>() {
#Override
public void initChannel(RxtxChannel ch) throws Exception {
ch.pipeline().addLast(new DleStxEtxFrameDecoder(), new DleStxEtxFrameEncoder());
ch.pipeline().addLast(new IntegrityCheck(),new IntegrityCalculation());
ch.pipeline().addLast(new AesCcmDecrypter(),new AesCcmEncrypter());
ch.pipeline().addLast(deserializerProvider.get(),new MessageSerializer());
ch.pipeline().addLast(new TimeStampJockel());
}
})
.option(RxtxChannelOption.BAUD_RATE, 19200);
ChannelFuture f = b.connect(new RxtxDeviceAddress(this.port)).sync();
f.channel().closeFuture().sync();
} finally {
group.shutdownGracefully();
}
}
The handlers are all pretty much standard implementations and seem to work when receiving packages only. The pipeline should first generate an object from the raw data, checkCRC, decrypt, deserialize and then compute some logic (aka generate a time-response).
public class TimeStampJockel extends ChannelInboundHandlerAdapter{
private static final Logger LOG = LoggerFactory.getLogger(TimeStampJockel.class);
private EventBus bus;
private ChannelKeeper keeper;
#Inject
public TimeStampJockel(){
this.bus = GlobalEventBus.getInstance();
this.keeper = keeper;
}
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg){
LOG.debug("Creating packet from received data");
RawPacket raw = (RawPacket)msg;
//EventExecutor ex = ctx.executor();
//LOG.debug("inexecutor.EventLoop(1):" + ex.inEventLoop());
//keeper.addChannelHandlerContext(raw.getSource(),ctx);
ByteBuf buf = raw.getContent();
LOG.debug("\tBuffer: {}", HelperFunctions.getBufferAsHexString(buf));
UnsignedLong mac = UnsignedLong.fromLongBits(21);
while(buf.readerIndex()<buf.writerIndex())
{
int type = buf.readShort();
int length = buf.readShort();
ByteBuf value = buf.readBytes(length);
if(PacketType.getEnum(type).equals(PacketType.MAC))
{
mac = UnsignedLong.valueOf(value.readLong());
}
else
{
AbstractPacket packet = PacketFactory.createPacket(PacketType.getEnum(type), raw.getVersion(), raw.getPacketType(), raw.getSource(), raw.getSource(), raw.getDestination(), mac, value);
if(packet instanceof TimeReqPacket) {
TimeReqPacket timeReqPacket = (TimeReqPacket) packet;
Duration d = HelperFunctions.timeSinceYear2000();
TimeRespPacket newPacket = new TimeRespPacket(Constants.PROTOCOL_VERSION, PacketType.TIME_RESP.getValue(), packet.getGatewayAdr(),UnsignedLong.valueOf(Configuration.getConfig().getLong("rest.wsn.mac", Long.MAX_VALUE)),timeReqPacket.getMac(),timeReqPacket.getMac(),d.getStandardSeconds(),HelperFunctions.getMillisOfDuration(d));
ctx.write(newPacket);
} else {
bus.post(packet);
}
}
}
}
The received sensor data is pushed to a Guava-bus (unless its a time-request) and processed by other components. If the incoming package is a time-request-packet, the previously displayed component should generate a time-stamp-packet and writeAndFlush() is down the pipeline. Any ideas what may cause that issue? I'm pretty much out of ideas - I have been googling the last 10 hours without meaningful results and I have no unchecked resources left. I'm using ubuntu 16.04, thanks in advance.
[EDIT] I tried checking the ChannelFuture, by adding the following code-snippet to the last pipeline handler
ChannelFuture f = ctx.writeAndFlush(newPacket);
f.addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future) throws Exception {
if (!future.isSuccess()) {
LOG.error("Server failed to send message", future.cause());
future.channel().close();
}
}
[EDIT2] Found my error. It was a netty version conflict. I am working with multiple versions of netty in different projects and was using an older netty version (4.0.13) instead of netty 4.044.final. I have no idea what changed between those versions but I am glad that everything is working properly now.

Categories