We Keep Coding

How to send events to all instances of the application in PCF

I am not able to find a way to send/broadcast a message to all application instances in Pivotal Cloud Foundry. How can we notify to all app instances of some events? If we use the HTTP request, PCF router will dispatch it to a single instance of the app. How can we solve this problem?

What #Florian said is probably the safer option, but if you want something quick and easy, you can send HTTP requests directly to an app instance by using the X-CF-APP-INSTANCE header. The format for the header is YOUR-APP-GUID:YOUR-INSTANCE-INDEX.
https://docs.cloudfoundry.org/concepts/http-routing.html#app-instance-routing
So given an app guid, you could iterate over the number of instances, say 0 to 5, and send an HTTP request to each one. Make sure to check the response to confirm that each one succeeded.
This also requires that you know the app guid for your app (i.e. cf app <name> --guid) and the number of instances of your app.

CF, out of the box, does not provide any event queue mechanism where apps can subscribe to.
What I would do (assuming you've two app instances A and B):
Provide an event endpoint in your application code, e.g. POST /api/event (alternatively, if the event should arise from another app (e.g. another microservice), this one could directly send messages onto the queue)
All app instances are listening on an internal event queue for new events
instance A receives the call from the CF router and processes it by issuing an event on an internal event queue, the instance will not react to the event, yet
When A publishes the event, A and B receives the event and processes it accordingly
Now, the internal event queue you can use highly depends on your deployment. On AWS you probably can use SQS or SNS or something similar. PCF, as I know, may also provide a messaging system which would suit here as well, rabbitmq. You could also use features of other services that would allow you to subscribe to events, such as redis (pub/sub commands) or similar.
If you provide more information about what you want to achieve more concretely, more detailed answer would be possible, though.

Be notified when a RDS instance was created

I'm using AWS SDK for Java.
Imagine I create a RDS instance as described in the AWS documentation.
AmazonRDS client = AmazonRDSClientBuilder.standard().build();
CreateDBInstanceRequest request = new CreateDBInstanceRequest().withDBInstanceIdentifier("mymysqlinstance").withAllocatedStorage(5)
.withDBInstanceClass("db.t2.micro").withEngine("MySQL").withMasterUsername("MyUser").withMasterUserPassword("MyPassword");
DBInstance response = client.createDBInstance(request);
If I call instance.getEndpoint() right after making the request it will return null to me, because AWS is still creating the database. I need to know this endpoint when it becomes available, but I'm not figuring out how to do it.
Is there a way, using the AWS SDK, to be notified when the instance was finally created?

You can use the RDS SNS notifications:
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_Events.html#USER_Events.Messages
Subscribing to Amazon RDS Event Notification
You can create an Amazon
RDS event notification subscription so you can be notified when an
event occurs for a given DB instance, DB snapshot, DB security group,
or DB parameter group. The simplest way to create a subscription is
with the RDS console. If you choose to create event notification
subscriptions using the CLI or API, you must create an Amazon Simple
Notification Service topic and subscribe to that topic with the Amazon
SNS console or Amazon SNS API. You will also need to retain the Amazon
Resource Name (ARN) of the topic because it is used when submitting
CLI commands or API actions. For information on creating an SNS topic
and subscribing to it, see Getting Started with Amazon SNS.

Disclaimer: Opinionated Answer
IMO creating infrastructure at runtime in code like this is devil's work. Stacks are the way to go here, much more modular and you will get some of the following benefits:
If you start creating more than one table per customer you will be able to logically group them into a stack and clean then up easier as needed
If for some reason the creation of a resource fails you can see this very easily in the stack console
Management is much easier to search through stacks as you have a console already built for you
Updating a stack in AWS is much easier as well than updating tables individually
MOST IMPORTANT: If an error occurs the stack functionality already has rollback and redundancy functionality built in, which you control the behaviour of. If something happens in your code during your on boarding process it will be a mess to clean up, what if one table succeeded and the other not? You will have to troll through logs (if they exist) to find out what happened.
You can also combine this approach with using something like AWS Pipelines or even AWS Simple Workflow Service to add custom steps in your custom on-boarding process, eg run a lambda function, send a notification when completed, wait for some payment. This builds on my last point that if this pipeline does fail, you will be able to see which step failed, and why it failed. You will also be able to see if things timeout.
Lastly I want to advise caution in creating infrastructure per customer. It's much more work and adds allot more ways in which things can break. Make sure you put limits in AWS as well that you don't have a situation in which your bill sky-rockets because of some bug creating infrastructure.

azure service bus & dynamics crm integration

I am currently working on a POC where I am draining messages from an azure queue - that is populated through a dynamics CRM plugin during DB CRUD operations.The queue drain operation using a java client fails with the message :
<Error>
<Code>500</Code>
<Detail>
The service was unable to process the request; please retry the operation. For more information on exception types and proper exception handling, please refer to http://go.microsoft.com/fwlink/?LinkId=761101. TrackingId:add85f1a-a249-4e69-b284-ad879cd29968_G27, SystemTracker:scsqueue1-ns:Queue:scsqueue1, Timestamp:7/6/2016 7:21:49 PM
</Detail>
</Error>
The java client can be found here :
https://github.com/sharpcodes/scs-bus-demo
However using a C# client works

I can't see the obvious issue in your code on GitHub. However, per my experience, I suggest that you can try to use the tool Fiddler for debugging the rest request from your code, because the Azure Service Bus SDK for Java is wrapped the REST APIs, such as Receive and Delete Message (Destructive Read) for draining messages.
Meanwhile, there are three ways below in Java for draining messages via the operation Receive and Delete Message on Azure.
Using Azure Service Bus SDK for Java, please refer to the section Receive messages from a queue of the tutorial "How to use Service Bus queues" to do with default mode ReceiveAndDelete.
Using Azure Service Bus REST API for Java with http client, please refer to the section Receive and delete a message from the queue of the tutorial "Service Bus brokered messaging REST tutorial".
Using JMS API with AMQP 1.0 in Java, please refer to the section Coding Java applications of the tutorial "How to use the Java Message Service (JMS) API with Service Bus and AMQP 1.0" to see the function SimpleSenderReceiver().

Thanks to peter for guiding on this..
Looks like there is an issue with the dynamics crm-azure bus plugin that prevents java/node-js clients from draining the queue using SDKs or RestFul calls.
Using JMS client seems to do the trick. Another way would be to have a C# middleware that could be wrapped by Java/Node.
Here is the reference to the issue : https://github.com/Azure/azure-sdk-for-php/issues/823

The issue is closed now as a fix had to be made by Azure Java SDK team to escape characters correctly..

i want to send notification to my all authenticated app user

I have an application that contains order data . I want to send this to all users of this app as notification.My target is to send Push Notifications to all of the users of my applications. How would I accomplish that?

To begin with your app mush already be using GCM and listening to the correct topic. Otherwise you have to roll out an update with the new GCM feature and hope that all users update it.
If you want to see some code for this take a look at this sample i crated some time ago. What you need to implement from the sample is the "SubscribeTopic" part. And to test if its working you can use this java program.
The good thing about topics is that you don’t need to save the users registration tokens and the message is sent to everyone listening for that exact topic.

Use Apache kafka
The original use case for Kafka was to be able to rebuild a user
activity tracking pipeline as a set of real-time publish-subscribe
feeds. This means site activity (page views, searches, or other
actions users may take) is published to central topics with one topic
per activity type. These feeds are available for subscription for a
range of use cases including real-time processing, real-time
monitoring, and loading into Hadoop or offline data warehousing
systems for offline processing and reporting.

To start with, note that a full GCM implementation requires both a client implementation and a server implementation. Before you can write client apps that use GCM, you must have an application server that meets the following criteria:
Able to communicate with your client.
Able to send properly formatted requests to the GCM connection
server.
Able to handle requests and resend them using exponential back-off.
Able to securely store the API key and client registration tokens.
Note: never include the API key in any client code.
For XMPP, the server must be able to generate message IDs to uniquely
identify each message it sends (GCM HTTP connection server generates
message IDs and returns them in the response). XMPP message IDs
should be unique per sender ID.
Complete documentation, how-to-guides for sending messages and links to examples can be found from Cloud Messaging - Messaging Concepts and Options.
You may also check ANDROID AND GCM – BROADCAST YOURSELF for the tutorial and demonstration on the use of GCM for the broadcast of messages to an Android client from a Tomcat server and use of sending broadcastintents from a service and receiving those broadcastintents from an app's activity.

How can we save Java Message Queues for reference?

How can we keep track of every message that gets into our Java Message Queue? We need to save the message for later reference. We already log it into an application log (log4j) but we need to query them later.

You can store them
in memory - in a collection or in an in-memory database
in a standalone database

You could create a database logging table for the messages, storing the message as is in a BLOB column, the timestamp that it was created / posted to the MQ and a simple counter as primary key. You can also add fields like message type etc if you want to create statistical reports on messages sent.
Cleanup of the tabe can be done simply by deleting all message older than the retention period by using the timestamp column.

I implemented such a solution in the past, we chose to store messages with all their characteristics in a database and developed a search, replay and cancel application on top of it. This is the Message Store pattern:
(source: eaipatterns.com)
We also used this application for the Dead Letter Channel.
(source: eaipatterns.com)
If you don't want to build a custom solution, have a look at the ReplayService for JMS from CodeStreet.

The best way to do this is to use whatever tracing facility your middleware provider offers. Or possibly, you could set up an intermediate listener whose only job was to log messages and forward on to your existing application.
In most cases, you will find that the middleware provider already has the ability to do this for you with no changes or awareness by your application.

I would change the queue to a topic, and then keep the original consumer that processes the messages, and add another consumer for auditing the messages to a database.
Some JMS providers cater for topic-to-queue-bridge definitions, the consumers then receive from their own dedicated queues, and don't have to read past messages that are left on the queue due to other consumers being inactive.
Alternatively, you could write a log4j appender, which writes your logged messages to a database.