Qpid receiver on Azure EventHub - java

I have already working application based on Azure EventHub. Now I need write java receiver that connects to the existing infrastructure.
Existing configuration:
Event Hub > SomeName > Consumer Group > SomeGroupName
In the administrative console I cannot see any QUEUE or TOPIC definitions. Analyzing working c# code I can see that hub-name + group-name is enough to connect.
I have reconstructed url that allows me to connect over java (and connection works so far).
amqps://SomeName.servicebus.windows.net
So my questions:
1) When instead of queue /topic I specify group-name then I get exception The messaging entity 'sb://SomeName.servicebus.windows.net/SomeGroupName' could not be found. What is the model used there instead of queue/topic?
2) How to work with such infrastructure from Apache-qpid?

Are you using the Event Hub created in the old portal or one created using the new portal?
EventHub is not a Message Bus, so there are no Queues or Topics, that is correct.
The consumer group is not a part of the address. The address is build using the namespace and the name of the eventhub in that namespace.
So the address becomes:
sb://SomeNameSpaceName.servicebus.windows.net/SomeEventHubName
Can you post the c# code you've analyzed? Since you have an already working application maybe we can workout the differences that prevents it from working now.

The greatest hint for resolve the question gave me following link: http://theitjourney.blogspot.com/2015/12/sendreceive-messages-using-amqp-in-java.html
So No queue neither topic in this model. You need to connect to specific provider and specify correct EventHub as following:
application.properties:
connectionfactory.SBCF=amqps://<PolicyName>:<PolicyKey>#<DomainName>.servicebus.windows.net
queue.EventHub=<EventHubName>/ConsumerGroups/$Default/Partitions/0
Where:
After that following code allowed me to create MessageConsumer:
Hashtable<String, String> env = new Hashtable<>();
env.put(Context.INITIAL_CONTEXT_FACTORY,
"org.apache.qpid.amqp_1_0.jms.jndi.PropertiesFileInitialContextFactory");
env.put(Context.PROVIDER_URL,
getClass().getResource("/application.properties").toString());
Context context = null;
context = new InitialContext(env);
// Look up ConnectionFactory
ConnectionFactory cf = (ConnectionFactory) context.lookup("SBCF");
Destination queue = (Destination) context.lookup("EventHub");
// Create Connection
Connection connection = cf.createConnection();
// Create receiver-side Session, MessageConsumer
Session receiveSession = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
MessageConsumer receiver = receiveSession.createConsumer(queue);

Related

Setting Trusted Packages (ActiveMQ)

I am trying to send receive ActiveMQ messages. But I see the messages come back with this message in them.
JMSException in onMessage(): javax.jms.JMSException: Failed to build body from content. Serializable class not available to broker. Reason: java.lang.ClassNotFoundException: Forbidden class com.logicalprovisioning.common.gtc.shared.GTCMessage! This class is not trusted to be serialized as ObjectMessage payload. Please take a look at http://activemq.apache.org/objectmessage.html for more information on how to configure trusted classes.
So I read the link in the message. And I tried to follow the instructions. Although I must say its not very well written on where to put the configuration.
So what I did was:
1. I edited the ACTIVEMQ_OPTS line in activemq.bat file in the bin folder to
if "%ACTIVEMQ_OPTS%" == "" set ACTIVEMQ_OPTS=-Xms1G -Xmx1G -Djava.util.logging.config.file=logging.properties -Djava.security.auth.login.config="%ACTIVEMQ_CONF%\login.config" -Dorg.apache.activemq.SERIALIZABLE_PACKAGES=com.logicalprovisioning.common.gtc.shared.GTCMessage
It didn't work.
I added the above line in the activemq.bat of the win64 folder as well. It didn't work.
I modified my subscriber object creation to add the Trusted Packages. Like:
String providerEndpoints = "tcp://" + host + ":" + port + "?wireFormat.maxInactivityDuration=7200000";
// Set the trusted packages/classes to move back and forth on the ActiveMQ JMS service.
ArrayList<String> trustedClasses = new ArrayList<String>();
trustedClasses.add("com.logicalprovisioning.common.gtc.shared.GTCMessage");
// Obtain the factory
ActiveMQConnectionFactory activeMQConnectionFactory = new ActiveMQConnectionFactory();
activeMQConnectionFactory.setBrokerURL(providerEndpoints);
// Add the trusted packages/classes to the ActiveMQ consumer.
activeMQConnectionFactory.setTrustedPackages(trustedClasses);
//Create the connection
setConnection(activeMQConnectionFactory.createQueueConnection());
getConnection().setClientID(this.getName());
// Make a session
setSession(getConnection().createQueueSession(false, Session.AUTO_ACKNOWLEDGE));
getSession().createQueue(jmsDestination);
// Create the destination
Destination destination = getSession().createQueue(jmsDestination);
String selector = "JMSCorrelationID = '" + getActionRequest().getOriginId() + "_" + getActionRequest().getRequestId() + "'" ;
setConsumer(getSession().createConsumer(destination, selector));
getConsumer().setMessageListener(new DefaultMessageListener(this));
// Start ...
gtcMessages = new GTCMessageQueue<GTCMessage>(); // We'll need a message store now
getConnection().start();
And I added something similar in my producer as well for good measure:
Context initialContext = new InitialContext();
Context environmentContext = (Context) initialContext.lookup("java:comp/env");
String queueConnectionFactoryNameLookup = PalInit.getProperty("jms.queue.connection.factory");
// Set the trusted packages/classes to move back and forth on the ActiveMQ JMS service.
ArrayList<String> trustedClasses = new ArrayList<String>();
trustedClasses.add("com.logicalprovisioning.common.gtc.shared.GTCMessage");
ActiveMQConnectionFactory activeMQConnectionFactory = (ActiveMQConnectionFactory) environmentContext.lookup(queueConnectionFactoryNameLookup);
activeMQConnectionFactory.setTrustedPackages(trustedClasses);
// Create connection
QueueConnection queueConnection = activeMQConnectionFactory.createQueueConnection();
queueConnection.start();
// Create session and producer
setSession(queueConnection.createSession(false, Session.AUTO_ACKNOWLEDGE));
String queueName = PalInit.getProperty("jms.destination");
Queue jmsQueue = getSession().createQueue(queueName);
setProducer(getSession().createProducer(jmsQueue));
setQueueConnection(queueConnection);
// Set Message "Time to Live" to the request timeout plus 10 minutes
getProducer().setTimeToLive(getTimeout() + (10 * 60 * 1000L));
But nothing seems to work. I have the ActiveMQ-All jar in the Tomcat's lib folder and so too the jar where the GTCMessage class resides. Can anyone please tell me what am I doing wrong? Is it a problem of a missing class or my bad configuration? Any help would be appreciated. Thanks!
The application is running on Tomcat 9, JAVA 1.8 and Active MQ 5.15.11.
I think your problem is that you're setting the name of the specific class rather than the package of the class. The code looks at the package name, not the class name. Try this:
// Set the trusted packages to move back and forth on the ActiveMQ JMS service.
ArrayList<String> trustedPackages = new ArrayList<String>();
trustedPackages.add("com.logicalprovisioning.common.gtc.shared");
ActiveMQConnectionFactory activeMQConnectionFactory = (ActiveMQConnectionFactory) environmentContext.lookup(queueConnectionFactoryNameLookup);
activeMQConnectionFactory.setTrustedPackages(trustedPackages);
I don't think you'll need to set this on the broker itself.
Aside from that I would strongly discourage you from using JMS ObjectMessage. They depend on Java serialization to marshal and unmarshal their object payload. This process is generally considered unsafe, because a malicious payload can exploit the host system. Lots of CVEs have been created for this which is why most JMS providers force users to explicitly whitelist packages that can be exchanged using ObjectMessage messages.
There are also a number of other issues with using JMS ObjectMessage not related to security that you should read about. This article has a good suggestion for how to replace ObjectMessage - define a data representation for the payload (JSON, protobuf, XML) and use TextMessage or BytesMessage to carry it.

How to use WAS JMS resourses for IIB MQ Queue from a Java Application on WAS (using JNDI)

I am new to MQ and JNDI and I looking for some simple sample Java code that resolves my WAS JMS configuration and can write to and read from two Message Queues.
specifically I would like JAVA code to:
run code on IBM WebSphere Application Server Network Deployment (WAS ND 8.5.5)
write to, and read from, 2 IBM Integration Bus (IIB) Message Queues on an external system
In WAS I configured JMS resources as follows:
for the connection factory - gave it a JNDI name of "jms/MQCONN.FACTORY"
for the queue #1 - gave it a JNDI name of "jms/MQUEUE1.DEST"
for the queue #2 - gave is a JNDI name of "jms/MQUEUE2.DEST"
I set up JAAS - J2C authentication data credentials.
Note: I was unable to test the connection to MQ for connection factory, because the security settings are added to after the wizard completes and the you can only test from the wizard. I believe the WAS configuration is correct including the credentials.
I especially do not understand how to code the JNDI part (i.e. How to store the environment variable that tells JNDI which initial context to use, and where to find the provider.)
Grateful for any assistance!
Sibyl , Once you have configured these Managed Objects (QueueConnectionFactory . Queue) , you should be able to lookup these from code that you can deploy on the application server.
You will have to get
a) InitialContext (when you deploy a ear on the server , you can use the default constructor)
b) Lookup queue connection factory (context.lookup(xxx))
c) Lookup queue (context.lookup(yyyy))
d) Create a message producer
e) Create a Queue Session , text message and send the message directly
You can get some more idea here (http://www.webspheretools.com/sites/webspheretools.nsf/docs/Creating%20a%20Queue%20Connection%20Factory%20and%20Queue%20for%20connectivity%20to%20MQ)
Basically post configuration it is a lot of Boilerplate JMS coding
Here is little help for you.
You don't need to provide extra configuration after creating resources in WAS.
Queue myQueue;
QueueConnectionFactory myQueueFactory;
QueueConnection connection = null;
QueueSession session = null;
try{
InitialContext jndi = new InitialContext();
myQueueFactory = (QueueConnectionFactory) jndi.lookup("jms/MQCONN.FACTORY");
myQueue = (Queue) jndi.lookup("jms/MQUEUE1.DEST");
connection=myQueueFactory.createQueueConnection();
session = connection.createQueueSession(true, Session.AUTO_ACKNOWLEDGE);
QueueSender sender = session.createSender(myQueue);
connection.start();
TextMessage textMessage = session.createTextMessage(event);
textMessage.setStringProperty("messageType", "file");
sender.send(textMessage);
sender.close();
if (session != null) {
session.close();
}
if (connection != null) {
connection.close();
}
} catch (JMSException e) {
e.printStackTrace();
}

Get a Topic in JMS using Java SE

I am using Java SE. I create a topic when the app first starts like so:
connectionFactory = new ActiveMQConnectionFactory("vm://localhost");
conn = connectionFactory.createTopicConnection();
session = conn.createTopicSession(false,
TopicSession.AUTO_ACKNOWLEDGE);
conn.start();
session.createTopic(name);
I am confused on how to retrieve this Topic in my classes.
Say for example I have a class, and it connects to the JMS Service just like above using:
connectionFactory = new ActiveMQConnectionFactory("vm://localhost");
conn = connectionFactory.createTopicConnection();
session = conn.createTopicSession(false,
TopicSession.AUTO_ACKNOWLEDGE);
conn.start();
How can I then get a reference to the Topic I created a app startup to send messages?
I would imagine something along the lines of:
session.getTopic(name);
Would exist, but I cant find anything on it.
I have read how to do this using Java EE using JNDI lookup, but this service is not available to me since I am not running in a container.
You don't 'retrieve' a topic. A Topic instance is just a piece of information. You construct an instance of it in your client if you want to subscribe to a topic (or queue), like is demonstrated in the ActiveMQ hello world example:
http://activemq.apache.org/hello-world.html
ex:
// the name should of course be the same as it exists on the producer side
Destination destination = session.createTopic("TEST.FOO");
// Create a MessageConsumer from the Session to the Topic or Queue
MessageConsumer consumer = session.createConsumer(destination);
This is all governed by the standardized and very mature JMS API, so you should refer to the JEE documentation. Any book on JMS will work for you as well.
Further reading: https://docs.oracle.com/javaee/6/tutorial/doc/bncdr.html
API docs: http://docs.oracle.com/javaee/6/api/javax/jms/package-summary.html

How do I verify that a JMS queue exists using Java?

How can I check whether a queue exists on a JMS server using the Java API? I don't want to send or receive any data to the queue for now, just verify that the queue exists. Also, the queue may be empty.
Here is my code sample. I have removed the error handling for simplicity.
Connection connection = null;
Session session = null;
connection = factory.createConnection();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
//I was hoping this next line would throw an exception if the queue does not exist
Queue queue = session.createQueue(queueName);
My JMS server is TIBCO EMS. I'm hoping for a solution that works on versions 5-7.
Solution
I followed the recommendation in the accepted answer but created a browser instead. The following line threw an exception as desired:
QueueBrowser browser = session.createBrowser(queue);
This is dependent on the provider, but you wont know in most cases until you create the session type, such as session.createConsumer. Simply creating a consumer this way will not consume any messages until you do a receive. And it is here the behavior may change from provider to provider and configuration of the server.
For example with ActiveMQ, assuming there are no permissions blocking the user you are connecting with, the queue is created automatically when you create the session type.
With WebSphere MQ, the queue has to be defined by an admin. If it does not exist, the queue manager will return an exception with a reason code of 2085 (UNKNOWN_OBJECT_NAME).
Outside of this, you'd need to see if the particular provider had a way to access a list of queues. Using the above examples, ActiveMQ you can get the list of queues using JMX, with WebSphere MQ, you can do this if you have permissions to send PCF commands to the queue manager.
Try creating a consumer or producer off the Session passing in the queue object you just created:
session.createConsumer(queue);
This should throw an InvalidDestinationException if the queue (or topic) does not exist.

Apache ActiveMQ Connect to a topic non-dynamically

I've set up ActiveMQ and I can create/connect to a topic using "dynamicTopics/MyTopic". This works great:
Properties env = new Properties( );
env.setProperty(Context.INITIAL_CONTEXT_FACTORY,"org.apache.activemq.jndi.ActiveMQInitialContextFactory");
env.setProperty(Context.PROVIDER_URL,"tcp://myhostname:xxxxx");
javax.naming.Context ctx = new InitialContext(env);
InitialContext jndi = new InitialContext(env);
// Look up a JMS connection factory
TopicConnectionFactory conFactory = (TopicConnectionFactory)jndi.lookup("TopicConnectionFactory");
// Create a JMS connection
TopicConnection connection = conFactory.createTopicConnection(username,password);
Topic chatTopic = (Topic)jndi.lookup("dynamicTopics/MyTopic");
I would like to connect to a topic already exists without dynamically creating the topic if it doesn't exist, how can I do so?
Cheers,
Pete
While you don't need to create destinations manually with AMQ you always have that option. Basically you can alter the security settings to allow only an admin to create destinations. See the AMQ FAQ for this:
http://activemq.apache.org/how-do-i-create-new-destinations.html
Regards
Tim
www.fusesource.com
What you are actually doing is just connecting to topic. ActiveMQ is so nice that it creates topic for you if it did not exist before.
So, to connect to already existing topic just do the same that you are already doing.

Categories