Is it possible to use MailDev with Java? - java

I have a MailDev instance installed on a remote Server.
I'm trying to understand if is it possibile to send email with Java (using the standard JavaMailSender) using this fake SMTP server.
The config needs only the URL and the port but, in my case, it doesn't work.
It returns always:
Mail server connection failed; nested exception is com.sun.mail.util.MailConnectException: Couldn't connect to host ...
The WebUI is running correctly and I can see the empty inbox on server.
Thanks.

It seems like you are experience connections issues. Although I cannot tell you what the cause there is, perhaps I can offer an alternative solution to test your emails?
Using Wiser, you can run an embedded SMTP server and query that inside your junit test. I've used this a lot in my open source project Simple Java Mail and created a Rule for this:
package testutil.testrules;
import org.jetbrains.annotations.NotNull;
import org.junit.rules.ExternalResource;
import org.subethamail.smtp.server.SMTPServer;
import org.subethamail.wiser.Wiser;
import org.subethamail.wiser.WiserMessage;
import javax.mail.MessagingException;
import javax.mail.internet.MimeMessage;
import java.io.IOException;
import java.io.InputStream;
import java.io.PrintStream;
import java.util.Iterator;
import java.util.List;
/**
* SmtpServerRule - a TestRule wrapping a Wiser instance (a SMTP server in Java), started and stopped right before and after each test.
* <br>
* SmtpServerRule exposes the same methods as the {#link Wiser} instance by delegating the implementation to the instance. These methods, however, can not be
* used outside a JUnit statement (otherwise an {#link IllegalStateException} is raised).
* <br>
* The {#link Wiser} instance can be directly retrieved but also only from inside a JUnit statement.
*/
public class SmtpServerRule extends ExternalResource {
private final Wiser wiser = new Wiser();
private final int port;
public SmtpServerRule(#NotNull Integer port) {
this.port = port;
}
#Override
protected void before() {
this.wiser.setPort(port);
this.wiser.start();
}
#Override
protected void after() {
this.wiser.stop();
}
#NotNull
public Wiser getWiser() {
checkState("getWiser()");
return this.wiser;
}
#NotNull
public List<WiserMessage> getMessages() {
checkState("getMessages()");
return wiser.getMessages();
}
#NotNull
public MimeMessage getOnlyMessage(String envelopeReceiver)
throws MessagingException {
checkState("getMessages()");
List<WiserMessage> messages = getMessages();
assertThat(messages).hasSize(1);
Iterator<WiserMessage> iterator = messages.iterator();
WiserMessage wiserMessage = iterator.next();
assertThat(wiserMessage.getEnvelopeReceiver()).isEqualTo(envelopeReceiver);
MimeMessage mimeMessage = wiserMessage.getMimeMessage();
iterator.remove();
return mimeMessage;
}
#NotNull
public MimeMessageAndEnvelope getOnlyMessage()
throws MessagingException {
checkState("getMessages()");
List<WiserMessage> messages = getMessages();
assertThat(messages).hasSize(1);
Iterator<WiserMessage> iterator = messages.iterator();
WiserMessage wiserMessage = iterator.next();
iterator.remove();
return new MimeMessageAndEnvelope(wiserMessage.getMimeMessage(), wiserMessage.getEnvelopeSender());
}
#NotNull
public MimeMessage getMessage(String envelopeReceiver)
throws MessagingException {
checkState("getMessages()");
List<WiserMessage> messages = getMessages();
Iterator<WiserMessage> iterator = messages.iterator();
while (iterator.hasNext()) {
WiserMessage wiserMessage = iterator.next();
if (wiserMessage.getEnvelopeReceiver().equals(envelopeReceiver)) {
MimeMessage mimeMessage = wiserMessage.getMimeMessage();
iterator.remove();
return mimeMessage;
}
}
throw new AssertionError("message not found for recipient " + envelopeReceiver);
}
#NotNull
public SMTPServer getServer() {
checkState("getServer()");
return wiser.getServer();
}
public boolean accept(String from, String recipient) {
checkState("accept(String, String)");
return wiser.accept(from, recipient);
}
public void deliver(String from, String recipient, InputStream data)
throws IOException {
checkState("deliver(String, String, InputStream)");
wiser.deliver(from, recipient, data);
}
public void dumpMessages(PrintStream out)
throws MessagingException {
checkState("dumpMessages(PrintStream)");
wiser.dumpMessages(out);
}
private void checkState(String method) {
if (this.wiser == null) {
throw new IllegalStateException(format("%s must not be called outside of a JUnit statement", method));
}
}
}
Then I use it like this:
public class MailerLiveTest {
private static final Integer SERVER_PORT = 251;
#Rule
public final SmtpServerRule smtpServerRule = new SmtpServerRule(SERVER_PORT);
#Before
public void setup() {
mailer = MailerBuilder.withSMTPServer("localhost", SERVER_PORT).buildMailer();
}
#Test
public void createMailSession_EmptySubjectAndBody() {
// send mail using mailer, which goes to localhost:251
MimeMessageAndEnvelope receivedMimeMessage = smtpServerRule.getOnlyMessage();
Email receivedEmail = EmailConverter.mimeMessageToEmail(receivedMimeMessage.getMimeMessage());
// perform assertions on Email object...
}
}

Check if mail.smtp.auth and mail.smtp.starttls.enable are set to false. In my case it worked (I also set localhost as host and 1025 as port).

Related

kafka streams exception Could not find a public no-argument constructor for org.apache.kafka.common.serialization.Serdes$WrapperSerde

getting the below error stack trace while working with kafka streams
UPDATE: as per #matthias-j-sax, have implemented my own Serdes with default constructor for WrapperSerde but still getting the following exceptions
org.apache.kafka.streams.errors.StreamsException: stream-thread [streams-request-count-4c239508-6abe-4901-bd56-d53987494770-StreamThread-1] Failed to rebalance.
at org.apache.kafka.streams.processor.internals.StreamThread.pollRequests (StreamThread.java:836)
at org.apache.kafka.streams.processor.internals.StreamThread.runOnce (StreamThread.java:784)
at org.apache.kafka.streams.processor.internals.StreamThread.runLoop (StreamThread.java:750)
at org.apache.kafka.streams.processor.internals.StreamThread.run (StreamThread.java:720)
Caused by: org.apache.kafka.streams.errors.StreamsException: Failed to configure value serde class myapps.serializer.Serdes$WrapperSerde
at org.apache.kafka.streams.StreamsConfig.defaultValueSerde (StreamsConfig.java:972)
at org.apache.kafka.streams.processor.internals.AbstractProcessorContext.<init> (AbstractProcessorContext.java:59)
at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.<init> (ProcessorContextImpl.java:42)
at org.apache.kafka.streams.processor.internals.StreamTask.<init> (StreamTask.java:136)
at org.apache.kafka.streams.processor.internals.StreamThread$TaskCreator.createTask (StreamThread.java:405)
at org.apache.kafka.streams.processor.internals.StreamThread$TaskCreator.createTask (StreamThread.java:369)
at org.apache.kafka.streams.processor.internals.StreamThread$AbstractTaskCreator.createTasks (StreamThread.java:354)
at org.apache.kafka.streams.processor.internals.TaskManager.addStreamTasks (TaskManager.java:148)
at org.apache.kafka.streams.processor.internals.TaskManager.createTasks (TaskManager.java:107)
at org.apache.kafka.streams.processor.internals.StreamThread$RebalanceListener.onPartitionsAssigned (StreamThread.java:260)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete (ConsumerCoordinator.java:259)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded (AbstractCoordinator.java:367)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup (AbstractCoordinator.java:316)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll (ConsumerCoordinator.java:290)
at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce (KafkaConsumer.java:1149)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll (KafkaConsumer.java:1115)
at org.apache.kafka.streams.processor.internals.StreamThread.pollRequests (StreamThread.java:827)
at org.apache.kafka.streams.processor.internals.StreamThread.runOnce (StreamThread.java:784)
at org.apache.kafka.streams.processor.internals.StreamThread.runLoop (StreamThread.java:750)
at org.apache.kafka.streams.processor.internals.StreamThread.run (StreamThread.java:720)
Caused by: java.lang.NullPointerException
at myapps.serializer.Serdes$WrapperSerde.configure (Serdes.java:30)
at org.apache.kafka.streams.StreamsConfig.defaultValueSerde (StreamsConfig.java:968)
at org.apache.kafka.streams.processor.internals.AbstractProcessorContext.<init> (AbstractProcessorContext.java:59)
at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.<init> (ProcessorContextImpl.java:42)
at org.apache.kafka.streams.processor.internals.StreamTask.<init> (StreamTask.java:136)
at org.apache.kafka.streams.processor.internals.StreamThread$TaskCreator.createTask (StreamThread.java:405)
at org.apache.kafka.streams.processor.internals.StreamThread$TaskCreator.createTask (StreamThread.java:369)
at org.apache.kafka.streams.processor.internals.StreamThread$AbstractTaskCreator.createTasks (StreamThread.java:354)
at org.apache.kafka.streams.processor.internals.TaskManager.addStreamTasks (TaskManager.java:148)
at org.apache.kafka.streams.processor.internals.TaskManager.createTasks (TaskManager.java:107)
at org.apache.kafka.streams.processor.internals.StreamThread$RebalanceListener.onPartitionsAssigned (StreamThread.java:260)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete (ConsumerCoordinator.java:259)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded (AbstractCoordinator.java:367)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup (AbstractCoordinator.java:316)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll (ConsumerCoordinator.java:290)
at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce (KafkaConsumer.java:1149)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll (KafkaConsumer.java:1115)
at org.apache.kafka.streams.processor.internals.StreamThread.pollRequests (StreamThread.java:827)
at org.apache.kafka.streams.processor.internals.StreamThread.runOnce (StreamThread.java:784)
at org.apache.kafka.streams.processor.internals.StreamThread.runLoop (StreamThread.java:750)
at org.apache.kafka.streams.processor.internals.StreamThread.run (StreamThread.java:720)
Here's my usecase:
I will be getting json responses as input to the stream, I want to count requests whose status codes are not 200. Initially, I went through the documentation of kafka streams in official documentation as well as confluent, then implemented WordCountDemo which is working very fine, then I tried to wrote this code, but getting this exception, I am very new to kafka streams, I went through the stack trace, but couldn't understood the context, hence came here for help!!!
Here's my code
LogCount.java
package myapps;
import java.util.Properties;
import java.util.concurrent.CountDownLatch;
import org.apache.kafka.common.serialization.Serde;
import org.apache.kafka.common.serialization.Serdes;
import org.apache.kafka.streams.KafkaStreams;
import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.streams.Topology;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.kstream.Produced;
import myapps.serializer.JsonDeserializer;
import myapps.serializer.JsonSerializer;
import myapps.Request;
public class LogCount {
public static void main(String[] args) {
Properties props = new Properties();
props.put(StreamsConfig.APPLICATION_ID_CONFIG, "streams-request-count");
props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
JsonSerializer<Request> requestJsonSerializer = new JsonSerializer<>();
JsonDeserializer<Request> requestJsonDeserializer = new JsonDeserializer<>(Request.class);
Serde<Request> requestSerde = Serdes.serdeFrom(requestJsonSerializer, requestJsonDeserializer);
props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());
props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, requestSerde.getClass().getName());
final StreamsBuilder builder = new StreamsBuilder();
KStream<String, Request> source = builder.stream("streams-requests-input");
source.filter((k, v) -> v.getHttpStatusCode() != 200)
.groupByKey()
.count()
.toStream()
.to("streams-requests-output", Produced.with(Serdes.String(), Serdes.Long()));
final Topology topology = builder.build();
final KafkaStreams streams = new KafkaStreams(topology, props);
final CountDownLatch latch = new CountDownLatch(1);
System.out.println(topology.describe());
// attach shutdown handler to catch control-c
Runtime.getRuntime().addShutdownHook(new Thread("streams-shutdown-hook") {
#Override
public void run() {
streams.close();
latch.countDown();
}
});
try {
streams.cleanUp();
streams.start();
latch.await();
} catch (Throwable e) {
System.exit(1);
}
System.exit(0);
}
}
JsonDeserializer.java
package myapps.serializer;
import com.google.gson.Gson;
import org.apache.kafka.common.serialization.Deserializer;
import java.util.Map;
public class JsonDeserializer<T> implements Deserializer<T> {
private Gson gson = new Gson();
private Class<T> deserializedClass;
public JsonDeserializer(Class<T> deserializedClass) {
this.deserializedClass = deserializedClass;
}
public JsonDeserializer() {
}
#Override
#SuppressWarnings("unchecked")
public void configure(Map<String, ?> map, boolean b) {
if(deserializedClass == null) {
deserializedClass = (Class<T>) map.get("serializedClass");
}
}
#Override
public T deserialize(String s, byte[] bytes) {
if(bytes == null){
return null;
}
return gson.fromJson(new String(bytes),deserializedClass);
}
#Override
public void close() {
}
}
JsonSerializer.java
package myapps.serializer;
import com.google.gson.Gson;
import org.apache.kafka.common.serialization.Serializer;
import java.nio.charset.Charset;
import java.util.Map;
public class JsonSerializer<T> implements Serializer<T> {
private Gson gson = new Gson();
#Override
public void configure(Map<String, ?> map, boolean b) {
}
#Override
public byte[] serialize(String topic, T t) {
return gson.toJson(t).getBytes(Charset.forName("UTF-8"));
}
#Override
public void close() {
}
}
As I mentioned, I will be getting JSON as input, the structure is like this,
{
"RequestID":"1f6b2409",
"Protocol":"http",
"Host":"abc.com",
"Method":"GET",
"HTTPStatusCode":"200",
"User-Agent":"curl%2f7.54.0",
}
The corresponding Request.java file looks like this
package myapps;
public final class Request {
private String requestID;
private String protocol;
private String host;
private String method;
private int httpStatusCode;
private String userAgent;
public String getRequestID() {
return requestID;
}
public void setRequestID(String requestID) {
this.requestID = requestID;
}
public String getProtocol() {
return protocol;
}
public void setProtocol(String protocol) {
this.protocol = protocol;
}
public String getHost() {
return host;
}
public void setHost(String host) {
this.host = host;
}
public String getMethod() {
return method;
}
public void setMethod(String method) {
this.method = method;
}
public int getHttpStatusCode() {
return httpStatusCode;
}
public void setHttpStatusCode(int httpStatusCode) {
this.httpStatusCode = httpStatusCode;
}
public String getUserAgent() {
return userAgent;
}
public void setUserAgent(String userAgent) {
this.userAgent = userAgent;
}
}
EDIT: when I exit from kafka-console-consumer.sh, it's saying Processed a total of 0 messages.
As the error indicate, a class is missing a non-argument default constructor for Serdes$WrapperSerde:
Could not find a public no-argument constructor
The issue is this construct:
Serde<Request> requestSerde = Serdes.serdeFrom(requestJsonSerializer, requestJsonDeserializer);
props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, requestSerde.getClass().getName());
Serdes.serdeFrom return WrapperSerde that does not have an empty default constructor. Thus, you cannot pass it into the StreamsConfig. You can use Serdes generate like this only if you pass objects into the corresponding API calls (ie, overwrite default Serde for certain operators).
To make it work (ie, to be able to set the Serde in the config), you would need to implement a proper class that implement Serde interface.
The requestSerde.getClass().getName() did not work for me. I needed to provide my own WrapperSerde implementation in an inner class. You probably need to do the same with something like:
public class MySerde extends WrapperSerde<Request> {
public MySerde () {
super(requestJsonSerializer, requestJsonDeserializer);
}
}
Instead of specifying in properties, add the custom serde in streams creation
KStream<String, Request> source = builder.stream("streams-requests-input",Consumed.with(Serdes.String(), requestSerde));

Google Pub/Sub Java examples

I'm not able to find a way to read messages from pub/sub using java.
I'm using this maven dependency in my pom
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-pubsub</artifactId>
<version>0.17.2-alpha</version>
</dependency>
I implemented this main method to create a new topic:
public static void main(String... args) throws Exception {
// Your Google Cloud Platform project ID
String projectId = ServiceOptions.getDefaultProjectId();
// Your topic ID
String topicId = "my-new-topic-1";
// Create a new topic
TopicName topic = TopicName.create(projectId, topicId);
try (TopicAdminClient topicAdminClient = TopicAdminClient.create()) {
topicAdminClient.createTopic(topic);
}
}
The above code works well and, indeed, I can see the new topic I created using the google cloud console.
I implemented the following main method to write a message to my topic:
public static void main(String a[]) throws InterruptedException, ExecutionException{
String projectId = ServiceOptions.getDefaultProjectId();
String topicId = "my-new-topic-1";
String payload = "Hellooooo!!!";
PubsubMessage pubsubMessage =
PubsubMessage.newBuilder().setData(ByteString.copyFromUtf8(payload)).build();
TopicName topic = TopicName.create(projectId, topicId);
Publisher publisher;
try {
publisher = Publisher.defaultBuilder(
topic)
.build();
publisher.publish(pubsubMessage);
System.out.println("Sent!");
} catch (IOException e) {
System.out.println("Not Sended!");
e.printStackTrace();
}
}
Now I'm not able to verify if this message was really sent.
I would like to implement a message reader using a subscription to my topic.
Could someone show me a correct and working java example about reading messages from a topic?
Anyone can help me?
Thanks in advance!
Here is the version using the google cloud client libraries.
package com.techm.data.client;
import com.google.cloud.pubsub.v1.AckReplyConsumer;
import com.google.cloud.pubsub.v1.MessageReceiver;
import com.google.cloud.pubsub.v1.Subscriber;
import com.google.cloud.pubsub.v1.SubscriptionAdminClient;
import com.google.common.util.concurrent.MoreExecutors;
import com.google.pubsub.v1.ProjectSubscriptionName;
import com.google.pubsub.v1.ProjectTopicName;
import com.google.pubsub.v1.PubsubMessage;
import com.google.pubsub.v1.PushConfig;
/**
* A snippet for Google Cloud Pub/Sub showing how to create a Pub/Sub pull
* subscription and asynchronously pull messages from it.
*/
public class CreateSubscriptionAndConsumeMessages {
private static String projectId = "projectId";
private static String topicId = "topicName";
private static String subscriptionId = "subscriptionName";
public static void createSubscription() throws Exception {
ProjectTopicName topic = ProjectTopicName.of(projectId, topicId);
ProjectSubscriptionName subscription = ProjectSubscriptionName.of(projectId, subscriptionId);
try (SubscriptionAdminClient subscriptionAdminClient = SubscriptionAdminClient.create()) {
subscriptionAdminClient.createSubscription(subscription, topic, PushConfig.getDefaultInstance(), 0);
}
}
public static void main(String... args) throws Exception {
ProjectSubscriptionName subscription = ProjectSubscriptionName.of(projectId, subscriptionId);
createSubscription();
MessageReceiver receiver = new MessageReceiver() {
#Override
public void receiveMessage(PubsubMessage message, AckReplyConsumer consumer) {
System.out.println("Received message: " + message.getData().toStringUtf8());
consumer.ack();
}
};
Subscriber subscriber = null;
try {
subscriber = Subscriber.newBuilder(subscription, receiver).build();
subscriber.addListener(new Subscriber.Listener() {
#Override
public void failed(Subscriber.State from, Throwable failure) {
// Handle failure. This is called when the Subscriber encountered a fatal error
// and is
// shutting down.
System.err.println(failure);
}
}, MoreExecutors.directExecutor());
subscriber.startAsync().awaitRunning();
// In this example, we will pull messages for one minute (60,000ms) then stop.
// In a real application, this sleep-then-stop is not necessary.
// Simply call stopAsync().awaitTerminated() when the server is shutting down,
// etc.
Thread.sleep(60000);
} finally {
if (subscriber != null) {
subscriber.stopAsync().awaitTerminated();
}
}
}
}
This is working fine for me.
The Cloud Pub/Sub Pull Subscriber Guide has sample code for reading messages from a topic.
I haven't used google cloud client libraries but used the api client libraries. Here is how I created a subscription.
package com.techm.datapipeline.client;
import java.io.IOException;
import java.security.GeneralSecurityException;
import com.google.api.client.googleapis.json.GoogleJsonResponseException;
import com.google.api.client.http.HttpStatusCodes;
import com.google.api.services.pubsub.Pubsub;
import com.google.api.services.pubsub.Pubsub.Projects.Subscriptions.Create;
import com.google.api.services.pubsub.Pubsub.Projects.Subscriptions.Get;
import com.google.api.services.pubsub.Pubsub.Projects.Topics;
import com.google.api.services.pubsub.model.ExpirationPolicy;
import com.google.api.services.pubsub.model.Subscription;
import com.google.api.services.pubsub.model.Topic;
import com.techm.datapipeline.factory.PubsubFactory;
public class CreatePullSubscriberClient {
private final static String PROJECT_NAME = "yourProjectId";
private final static String TOPIC_NAME = "yourTopicName";
private final static String SUBSCRIPTION_NAME = "yourSubscriptionName";
public static void main(String[] args) throws IOException, GeneralSecurityException {
Pubsub pubSub = PubsubFactory.getService();
String topicName = String.format("projects/%s/topics/%s", PROJECT_NAME, TOPIC_NAME);
String subscriptionName = String.format("projects/%s/subscriptions/%s", PROJECT_NAME, SUBSCRIPTION_NAME);
Topics.Get listReq = pubSub.projects().topics().get(topicName);
Topic topic = listReq.execute();
if (topic == null) {
System.err.println("Topic doesn't exist...run CreateTopicClient...to create the topic");
System.exit(0);
}
Subscription subscription = null;
try {
Get getReq = pubSub.projects().subscriptions().get(subscriptionName);
subscription = getReq.execute();
} catch (GoogleJsonResponseException e) {
if (e.getStatusCode() == HttpStatusCodes.STATUS_CODE_NOT_FOUND) {
System.out.println("Subscription " + subscriptionName + " does not exist...will create it");
}
}
if (subscription != null) {
System.out.println("Subscription already exists ==> " + subscription.toPrettyString());
System.exit(0);
}
subscription = new Subscription();
subscription.setTopic(topicName);
subscription.setPushConfig(null); // indicating a pull
ExpirationPolicy expirationPolicy = new ExpirationPolicy();
expirationPolicy.setTtl(null); // never expires;
subscription.setExpirationPolicy(expirationPolicy);
subscription.setAckDeadlineSeconds(null); // so defaults to 10 sec
subscription.setRetainAckedMessages(true);
Long _week = 7L * 24 * 60 * 60;
subscription.setMessageRetentionDuration(String.valueOf(_week)+"s");
subscription.setName(subscriptionName);
Create createReq = pubSub.projects().subscriptions().create(subscriptionName, subscription);
Subscription createdSubscription = createReq.execute();
System.out.println("Subscription created ==> " + createdSubscription.toPrettyString());
}
}
And once you create the subscription (pull type)...this is how you pull the messages from the topic.
package com.techm.datapipeline.client;
import java.io.IOException;
import java.security.GeneralSecurityException;
import java.util.ArrayList;
import java.util.List;
import com.google.api.client.googleapis.json.GoogleJsonResponseException;
import com.google.api.client.http.HttpStatusCodes;
import com.google.api.client.util.Base64;
import com.google.api.services.pubsub.Pubsub;
import com.google.api.services.pubsub.Pubsub.Projects.Subscriptions.Acknowledge;
import com.google.api.services.pubsub.Pubsub.Projects.Subscriptions.Get;
import com.google.api.services.pubsub.Pubsub.Projects.Subscriptions.Pull;
import com.google.api.services.pubsub.model.AcknowledgeRequest;
import com.google.api.services.pubsub.model.Empty;
import com.google.api.services.pubsub.model.PullRequest;
import com.google.api.services.pubsub.model.PullResponse;
import com.google.api.services.pubsub.model.ReceivedMessage;
import com.techm.datapipeline.factory.PubsubFactory;
public class PullSubscriptionsClient {
private final static String PROJECT_NAME = "yourProjectId";
private final static String SUBSCRIPTION_NAME = "yourSubscriptionName";
private final static String SUBSCRIPTION_NYC_NAME = "test";
public static void main(String[] args) throws IOException, GeneralSecurityException {
Pubsub pubSub = PubsubFactory.getService();
String subscriptionName = String.format("projects/%s/subscriptions/%s", PROJECT_NAME, SUBSCRIPTION_NAME);
//String subscriptionName = String.format("projects/%s/subscriptions/%s", PROJECT_NAME, SUBSCRIPTION_NYC_NAME);
try {
Get getReq = pubSub.projects().subscriptions().get(subscriptionName);
getReq.execute();
} catch (GoogleJsonResponseException e) {
if (e.getStatusCode() == HttpStatusCodes.STATUS_CODE_NOT_FOUND) {
System.out.println("Subscription " + subscriptionName
+ " does not exist...run CreatePullSubscriberClient to create");
}
}
PullRequest pullRequest = new PullRequest();
pullRequest.setReturnImmediately(false); // wait until you get a message
pullRequest.setMaxMessages(1000);
Pull pullReq = pubSub.projects().subscriptions().pull(subscriptionName, pullRequest);
PullResponse pullResponse = pullReq.execute();
List<ReceivedMessage> msgs = pullResponse.getReceivedMessages();
List<String> ackIds = new ArrayList<String>();
int i = 0;
if (msgs != null) {
for (ReceivedMessage msg : msgs) {
ackIds.add(msg.getAckId());
//System.out.println(i++ + ":===:" + msg.getAckId());
String object = new String(Base64.decodeBase64(msg.getMessage().getData()));
System.out.println("Decoded object String ==> " + object );
}
//acknowledge all the received messages
AcknowledgeRequest content = new AcknowledgeRequest();
content.setAckIds(ackIds);
Acknowledge ackReq = pubSub.projects().subscriptions().acknowledge(subscriptionName, content);
Empty empty = ackReq.execute();
}
}
}
Note: This client only waits until it receives at least one message and terminates if it's receives one (up to a max of value - set in MaxMessages) at once.
Let me know if this helps. I'm going to try the cloud client libraries soon and will post an update once I get my hands on them.
And here's the missing factory class ...if you plan to run it...
package com.techm.datapipeline.factory;
import java.io.IOException;
import java.security.GeneralSecurityException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.logging.Level;
import java.util.logging.Logger;
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.pubsub.Pubsub;
import com.google.api.services.pubsub.PubsubScopes;
public class PubsubFactory {
private static Pubsub instance = null;
private static final Logger logger = Logger.getLogger(PubsubFactory.class.getName());
public static synchronized Pubsub getService() throws IOException, GeneralSecurityException {
if (instance == null) {
instance = buildService();
}
return instance;
}
private static Pubsub buildService() throws IOException, GeneralSecurityException {
logger.log(Level.FINER, "Start of buildService");
HttpTransport transport = GoogleNetHttpTransport.newTrustedTransport();
JsonFactory jsonFactory = new JacksonFactory();
GoogleCredential credential = GoogleCredential.getApplicationDefault(transport, jsonFactory);
// Depending on the environment that provides the default credentials (for
// example: Compute Engine, App Engine), the credentials may require us to
// specify the scopes we need explicitly.
if (credential.createScopedRequired()) {
Collection<String> scopes = new ArrayList<>();
scopes.add(PubsubScopes.PUBSUB);
credential = credential.createScoped(scopes);
}
logger.log(Level.FINER, "End of buildService");
// TODO - Get the application name from outside.
return new Pubsub.Builder(transport, jsonFactory, credential).setApplicationName("Your Application Name/Version")
.build();
}
}
The message reader is injected on the subscriber. This part of the code will handle the messages:
MessageReceiver receiver =
new MessageReceiver() {
#Override
public void receiveMessage(PubsubMessage message, AckReplyConsumer consumer) {
// handle incoming message, then ack/nack the received message
System.out.println("Id : " + message.getMessageId());
System.out.println("Data : " + message.getData().toStringUtf8());
consumer.ack();
}
};

Websockets Tomcat 7 does not work in existing project

I try to run a websocket server in a Java project that was running on Tomcat6. I have set up a Tomcat 7 server where the project now is running on.
First I tried to run the socket example of Tomcat7. This run perfectly. I copied this class to my old project. When I run the old project again all the functionalities are working like before but only the websocket server doe not work.
This is the ChatAnnotation class that I have copied from the examples from Tomcat to my old project.
import java.io.IOException;
import java.util.Set;
import java.util.concurrent.CopyOnWriteArraySet;
import java.util.concurrent.atomic.AtomicInteger;
import javax.websocket.OnClose;
import javax.websocket.OnError;
import javax.websocket.OnMessage;
import javax.websocket.OnOpen;
import javax.websocket.Session;
import javax.websocket.server.ServerEndpoint;
import org.apache.log4j.Logger;
#ServerEndpoint(value = "/websocket/chat")
public class ChatAnnotation {
private static Logger logger = Logger.getLogger(ChatAnnotation.class);
private static final String GUEST_PREFIX = "Guest";
private static final AtomicInteger connectionIds = new AtomicInteger(0);
private static final Set<ChatAnnotation> connections = new CopyOnWriteArraySet<ChatAnnotation>();
private final String nickname;
private Session session;
public ChatAnnotation() {
nickname = GUEST_PREFIX + connectionIds.getAndIncrement();
logger.info("ws instance");
}
#OnOpen
public void start(Session session) {
this.session = session;
connections.add(this);
String message = String.format("* %s %s", nickname, "has joined.");
broadcast(message);
}
#OnClose
public void end() {
connections.remove(this);
String message = String.format("* %s %s", nickname, "has disconnected.");
broadcast(message);
}
#OnMessage
public void incoming(String message) {
// Never trust the client
String filteredMessage = String.format("%s: %s", nickname, message.toString());
broadcast(filteredMessage);
}
#OnError
public void onError(Throwable t) throws Throwable {
logger.error("Chat Error: " + t.toString(), t);
}
private static void broadcast(String msg) {
for (ChatAnnotation client : connections) {
try {
synchronized (client) {
client.session.getBasicRemote().sendText(msg);
}
} catch (IOException e) {
logger.debug("Chat Error: Failed to send message to client", e);
connections.remove(client);
try {
client.session.close();
} catch (IOException e1) {
// Ignore
}
String message = String.format("* %s %s", client.nickname, "has been disconnected.");
broadcast(message);
}
}
}
}
I have noting added in my web.xml. In my old project are also tcpsockets used can this be the problem?
Can anyone help me with this problem?
EDIT
Class added:
import java.util.HashSet;
import java.util.Set;
import javax.websocket.Endpoint;
import javax.websocket.server.ServerApplicationConfig;
import javax.websocket.server.ServerEndpointConfig;
import org.apache.log4j.Logger;
public class ExamplesConfig implements ServerApplicationConfig {
private static Logger log = Logger.getLogger(ChatAnnotation.class);
public Set<ServerEndpointConfig> getEndpointConfigs(Set<Class<? extends Endpoint>> endpointClasses) {
Set<ServerEndpointConfig> result = new HashSet<ServerEndpointConfig>();
log.info("getEndpointConfigs");
return result;
}
public Set<Class<?>> getAnnotatedEndpointClasses(Set<Class<?>> scanned) {
log.info("getAnnotatedEndpointClasses");
return scanned;
}
}
Java websocket server use return value of ServerApplicationConfig interface to deploy programmatic endpoints and for annotated endpoints.
For Tomcat example, if you change the package name of ChatAnnotation. You have to modify websocket.ExamplesConfig too.
public Set<Class<?>> getAnnotatedEndpointClasses(Set<Class<?>> scanned) {
// Deploy all WebSocket endpoints defined by annotations in the examples
// web application. Filter out all others to avoid issues when running
// tests on Gump
Set<Class<?>> results = new HashSet<>();
for (Class<?> clazz : scanned) {
String name = clazz.getPackage().getName();
boolean ok = name.startsWith("websocket.");
if (ok) {
results.add(clazz);
}
}
return scanned;
}
The getAnnotatedEndpointClasses(scanned) only return classes which package name start with websocket. Unmatched classes will not deployed even they have #ServerEndpoint declarations.

Grizzly Http Server - accepting only one connection at a time

I have a Grizzly Http Server with Async processing added. It is queuing my requests and processing only one request at a time, despite adding async support to it.
Path HttpHandler was bound to is: "/"
Port number: 7777
Behavior observed when I hit http://localhost:7777 from two browsers simultaneously is:
Second call waits till first one is completed. I want my second http call also to work simultaneously in tandom with first http call.
EDIT Github link of my project
Here are the classes
GrizzlyMain.java
package com.grizzly;
import java.io.IOException;
import java.net.URI;
import javax.ws.rs.core.UriBuilder;
import org.glassfish.grizzly.http.server.HttpServer;
import org.glassfish.grizzly.http.server.NetworkListener;
import org.glassfish.grizzly.nio.transport.TCPNIOTransport;
import org.glassfish.grizzly.strategies.WorkerThreadIOStrategy;
import org.glassfish.grizzly.threadpool.ThreadPoolConfig;
import com.grizzly.http.IHttpHandler;
import com.grizzly.http.IHttpServerFactory;
public class GrizzlyMain {
private static HttpServer httpServer;
private static void startHttpServer(int port) throws IOException {
URI uri = getBaseURI(port);
httpServer = IHttpServerFactory.createHttpServer(uri,
new IHttpHandler(null));
TCPNIOTransport transport = getListener(httpServer).getTransport();
ThreadPoolConfig config = ThreadPoolConfig.defaultConfig()
.setPoolName("worker-thread-").setCorePoolSize(6).setMaxPoolSize(6)
.setQueueLimit(-1)/* same as default */;
transport.configureBlocking(false);
transport.setSelectorRunnersCount(3);
transport.setWorkerThreadPoolConfig(config);
transport.setIOStrategy(WorkerThreadIOStrategy.getInstance());
transport.setTcpNoDelay(true);
System.out.println("Blocking Transport(T/F): " + transport.isBlocking());
System.out.println("Num SelectorRunners: "
+ transport.getSelectorRunnersCount());
System.out.println("Num WorkerThreads: "
+ transport.getWorkerThreadPoolConfig().getCorePoolSize());
httpServer.start();
System.out.println("Server Started #" + uri.toString());
}
public static void main(String[] args) throws InterruptedException,
IOException, InstantiationException, IllegalAccessException,
ClassNotFoundException {
startHttpServer(7777);
System.out.println("Press any key to stop the server...");
System.in.read();
}
private static NetworkListener getListener(HttpServer httpServer) {
return httpServer.getListeners().iterator().next();
}
private static URI getBaseURI(int port) {
return UriBuilder.fromUri("https://0.0.0.0/").port(port).build();
}
}
HttpHandler (with async support built in)
package com.grizzly.http;
import java.io.IOException;
import java.util.Date;
import java.util.concurrent.ExecutorService;
import javax.ws.rs.core.Application;
import org.glassfish.grizzly.http.server.HttpHandler;
import org.glassfish.grizzly.http.server.Request;
import org.glassfish.grizzly.http.server.Response;
import org.glassfish.grizzly.http.util.HttpStatus;
import org.glassfish.grizzly.threadpool.GrizzlyExecutorService;
import org.glassfish.grizzly.threadpool.ThreadPoolConfig;
import org.glassfish.jersey.server.ApplicationHandler;
import org.glassfish.jersey.server.ResourceConfig;
import org.glassfish.jersey.server.spi.Container;
import com.grizzly.Utils;
/**
* Jersey {#code Container} implementation based on Grizzly
* {#link org.glassfish.grizzly.http.server.HttpHandler}.
*
* #author Jakub Podlesak (jakub.podlesak at oracle.com)
* #author Libor Kramolis (libor.kramolis at oracle.com)
* #author Marek Potociar (marek.potociar at oracle.com)
*/
public final class IHttpHandler extends HttpHandler implements Container {
private static int reqNum = 0;
final ExecutorService executorService = GrizzlyExecutorService
.createInstance(ThreadPoolConfig.defaultConfig().copy()
.setCorePoolSize(4).setMaxPoolSize(4));
private volatile ApplicationHandler appHandler;
/**
* Create a new Grizzly HTTP container.
*
* #param application
* JAX-RS / Jersey application to be deployed on Grizzly HTTP
* container.
*/
public IHttpHandler(final Application application) {
}
#Override
public void start() {
super.start();
}
#Override
public void service(final Request request, final Response response) {
System.out.println("\nREQ_ID: " + reqNum++);
System.out.println("THREAD_ID: " + Utils.getThreadName());
response.suspend();
// Instruct Grizzly to not flush response, once we exit service(...) method
executorService.execute(new Runnable() {
#Override
public void run() {
try {
System.out.println("Executor Service Current THREAD_ID: "
+ Utils.getThreadName());
Thread.sleep(25 * 1000);
} catch (Exception e) {
response.setStatus(HttpStatus.INTERNAL_SERVER_ERROR_500);
} finally {
String content = updateResponse(response);
System.out.println("Response resumed > " + content);
response.resume();
}
}
});
}
#Override
public ApplicationHandler getApplicationHandler() {
return appHandler;
}
#Override
public void destroy() {
super.destroy();
appHandler = null;
}
// Auto-generated stuff
#Override
public ResourceConfig getConfiguration() {
return null;
}
#Override
public void reload() {
}
#Override
public void reload(ResourceConfig configuration) {
}
private String updateResponse(final Response response) {
String data = null;
try {
data = new Date().toLocaleString();
response.getWriter().write(data);
} catch (IOException e) {
data = "Unknown error from our server";
response.setStatus(500, data);
}
return data;
}
}
IHttpServerFactory.java
package com.grizzly.http;
import java.net.URI;
import org.glassfish.grizzly.http.server.HttpServer;
import org.glassfish.grizzly.http.server.NetworkListener;
import org.glassfish.grizzly.http.server.ServerConfiguration;
/**
* #author smc
*/
public class IHttpServerFactory {
private static final int DEFAULT_HTTP_PORT = 80;
public static HttpServer createHttpServer(URI uri, IHttpHandler handler) {
final String host = uri.getHost() == null ? NetworkListener.DEFAULT_NETWORK_HOST
: uri.getHost();
final int port = uri.getPort() == -1 ? DEFAULT_HTTP_PORT : uri.getPort();
final NetworkListener listener = new NetworkListener("IGrizzly", host, port);
listener.setSecure(false);
final HttpServer server = new HttpServer();
server.addListener(listener);
final ServerConfiguration config = server.getServerConfiguration();
if (handler != null) {
config.addHttpHandler(handler, uri.getPath());
}
config.setPassTraceRequest(true);
return server;
}
}
It seems the problem is the browser waiting for the first request to complete, and thus more a client-side than a server-side issue. It disappears if you test with two different browser processes, or even if you open two distinct paths (let's say localhost:7777/foo and localhost:7777/bar) in the same browser process (note: the query string partecipates in making up the path in the HTTP request line).
How I understood it
Connections in HTTP/1.1 are persistent by default, ie browsers recycle the same TCP connection over and over again to speed things up. However, this doesn't mean that all requests to the same domain will be serialized: in fact, a connection pool is allocated on a per-hostname basis (source). Unfortunately, requests with the same path are effectively enqueued (at least on Firefox and Chrome) - I guess it's a device that browsers employ to protect server resources (and thus user experience)
Real-word applications don't suffer from this because different resources are deployed to different URLs.
DISCLAIMER: I wrote this answer based on my observations and some educated guess. I think things may actually be like this, however a tool like Wireshark should be used to follow the TCP stream and definitely assert this is what happens.

Set MessageId in header before sending mail

I am sending mails from my application using JAVA mail on smtp server, port 465. My need is that, I have to set Message-ID before sending mail. I did some R&D and found the code below. I had override the method updateMessageID() of MimeMessage
import javax.mail.MessagingException;
import javax.mail.Session;
import javax.mail.internet.MimeMessage;
public class CustomMimeMessage extends MimeMessage {
public CustomMimeMessage(Session session) {
super(session);
}
#Override
protected void updateMessageID() throws MessagingException {
setHeader("Message-ID", "message id");
}
}
And then I had made an instance of CustomMimeMessage in my service and then invoke updateMessageID() method using that instance, but I still get the Message-ID generated by gmail.
In your code
setHeader("Message-ID", "message id");
you are trying to set "message id" as Message-ID which is quite wrong you have to set a unique id that qualify all the rules of the message id (Read This).
Try this..,.
import javax.mail.MessagingException;
import javax.mail.Session;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
public class CustomMimeMessage extends MimeMessage {
Session session;
private static int id = 0;
public CustomMimeMessage(Session session) {
super(session);
this.session=session;
}
#Override
protected void updateMessageID() throws MessagingException {
setHeader("Message-ID", "<" + getUniqueMessageIDValue(session) + ">");
}
public static String getUniqueMessageIDValue(Session ssn) {
String suffix = null;
InternetAddress addr = InternetAddress.getLocalAddress(ssn);
if (addr != null)
suffix = addr.getAddress();
else {
suffix = "javamailuser#localhost"; // worst-case default
}
StringBuffer s = new StringBuffer();
// Unique string is <hashcode>.<id>.<currentTime>.JavaMail.<suffix>
s.append(s.hashCode()).append('.').append(getUniqueId()).append('.').
append(System.currentTimeMillis()).append('.').
append("JavaMail.").
append(suffix);
return s.toString();
}
private static synchronized int getUniqueId() {
return id++;
}
}
I am doing something similar but sending from the local host instead. This Might help.
import java.util.Properties;
import javax.mail.Message;
import javax.mail.Session;
import javax.mail.internet.MimeMessage;
import javax.mail.Transport;
import javax.mail.MessagingException;
import javax.mail.internet.AddressException;
import javax.mail.internet.InternetAddress;
public class SendEmail {
/**
* Sends an email based on paramaters passed to it.
*
* #param toWho - the recipiants email address
* #param fromWho - the senders email address
* #param subject - the subject line of the email
* #param body - the email message body
* #return void
* #throws AddressException
* #throws MessageingException
*/
public void sendMail(String toWho, String subject, String body, String fromWho) throws AddressException, MessagingException {
// Setting Properties
Properties props = System.getProperties();
props.put("mail.imaps.ssl.trust", "*"); // trusting all server certificates
props.setProperty("mail.store.protocol", "imaps");
// Get the default Session object.
Session session = Session.getDefaultInstance(props, null);
// Create a default MimeMessage object.
MimeMessage message = new MimeMessage(session);
// Set From header
message.setFrom(new InternetAddress(fromWho));
// Set to header
message.addRecipient(Message.RecipientType.TO, new InternetAddress(toWho));
// Header set subject
message.setSubject(subject);
// Message Body
message.setContent(body, "text/html; charset=utf-8");
// Send message
Transport.send(message);
}
}
You can set message-id to MimeMessage before call Transport.send() by extending MimeMessage like this.
import javax.mail.MessagingException;
import javax.mail.Session;
import javax.mail.internet.MimeMessage;
public class MyMimeMessage extends MimeMessage {
public MailorMimeMessage(Session session) {
super(session);
}
#Override
protected void updateMessageID() throws MessagingException {
if (getHeader("Message-Id") == null) {
super.updateMessageID();
}
}
}
And set custom message-id.
message.setHeader("Message-Id","<MY-MESSAGE-ID>");
You can try custom id with .(dot) before left side of '#'. It worked for me.
Check the following code snippet:
public class CustomMimeMessage extends MimeMessage {
Session session;
private static int id = 0;
public CustomMimeMessage(Session session) {
super(session);
this.session = session;
}
protected void updateMessageID() throws MessagingException {
debugLog("Calling updateMessageID()");
setHeader("Message-ID", "<" + getUniqueMessageIDValue(session) + ">");
}
public static String getUniqueMessageIDValue(Session ssn) {
String suffix = null;
InternetAddress addr = InternetAddress.getLocalAddress(ssn);
if (addr != null) {
testLog("InternetAddress = " + addr.toString());
String address = addr.getAddress();
if (address.contains("#")) {
address = address.substring(address.lastIndexOf("#"), address.length());
suffix = address;
}
}
if (suffix == null) {
suffix = "#mail";// worst-case default
}
testLog("suffix Address = " + suffix);
StringBuffer s = new StringBuffer();
s.append(System.currentTimeMillis()).append("")
.append(getUniqueId()).append(suffix).append(".sm");
testLog("NEW MESSAGE-ID: " + s.toString());
return s.toString();
}
private static synchronized int getUniqueId() {
return id++;
}
For more details, Please refer the following RFC's. These are really helpful for creating syntax for Custom message-id.
RFC 2822
RFC 5322
Hope this would also work for you.

Categories