So I'm trying to finally grasp how cloud-based, enterprise applications work, and what their architectures typically look like. Say I use a cloud provider like Amazon. I assume (please correct me if I'm wrong) that I would be paying for 1+ virtual machines that would house a stack of software per my application's needs.
I'm confused with how frameworks like jclouds or Terracotta fit into the picture. jclouds advertises itself as "an open source library that helps you get started in the cloud", and lists off a number of huge features that don't mean much to me without meaningful examples. Terracotta boasts itself as a high-scaling clustering framework. Why would I need to use something like jclouds? What specific, concrete scenarios would I use it for?
Again, if I'm using Amazon as my cloud provider, wouldn't they already be highly-scaled? Why would I need Terracotta in the cloud?
Taking an app "into the cloud" has at least two aspects.
Firstly you have to manage the nodes: deploy your app on all nodes, monitor them, start new nodes to actually scale, detect and replace failed nodes, realize some update scenario for new app versions, and so on. Usually this can't be done reasonably without tools. JClouds fits in here, since it covers some of these points.
Secondly your app itself must be "cloud ready". You can't take an arbitrary app, put it on multiple nodes and expect it to scale well. The main point here is to define how to scale the access to the shared data between all nodes (SQL database, NoSQL datastore, potentially session replication, ...). Usually you use some existent framework/appserver/datastore to manage your shared state. Terracotta is one of them, basically it provides an efficient way to share memory between JVM instances on multiple nodes.
So you have your Linux machine (virtual instance) and it is working OK. But suddenly you need to scale - that is you need to fire up more instances as demand go high and shut them as it goes down. So what you can do is basically use Amazon's API to start EC2 instances - provision them with everything you can do from the administrative console (and even more). But using amazon's API's basically ties your hands to amazon. With frameworks such as JCloud what you do is something like (this is pseudo code):
CloudProvider provider = new CloudProvider.getProvider("Amazon");
provider.authenticate("username", "password");
provider.startInstance("some option", numOfInstances);
So say you have to scale and you are deployed on Amazon using JClouds - you are going to use something like the above BUT suddenly you decide to move from amazon to Rackspace so instead of re-engineering all the logic of your app which has to do with provisioning instances and working with them you can just change the
CloudProvider provider = new CloudProvider.getProvider("Amazon");
to something like
CloudProvider provider = new CloudProvider.getProvider("RackSpace");
and continue using the authenticate method and startInstance but then the library would take of how to actually "translate" this library method to the specific method which the given cloud provider supports. Basically it is a way of abstracting the code which has to deal with the underlying cloud provider - you shouldn't care who it is as long as it is providing the service, right?
Related
I am trying to replace a requirement our dev teams have where they manually have to fill out a form that includes a list of their app's external connections (for example any database connections, calls to other services/applications, backing services, etc...). This is required in order to get approval to deploy to production. Mgmt/Security and our last mile folks use this information to determine risk level and to make sure that any scheduled dependencies are looked at (e.g., make sure the deployment is not scheduled for a time when one of the backing services is down so all the integration tests don't fail). Any suggestions to capture this automatically by scanning the code in Git? Or can Dynatrace provide this information if we have it monitoring in the lower environments pre-prod? Some other tool?
A little background in case you need it - we are using Jenkins with OpenShift to deploy docker containers to AWS PaaS. Code is stored in Git, we use Bitbucket. In the pipeline we have SonarQube scanning and a tool that scans third party libraries the app is using (e.g., struts, cucumber, etc..). We have dynatrace to monitor the app in production (but we can also use it in dev if we want). Mostly Java apps but we also have Node and Python and .NET.
I can't suggest a way to automate this. I suspect there isn't one.
I would have thought it was advisable that the dev teams did this by hand anyway. Surely they should have a handle on what external connections the apps ought to be making. Expecting the production / security team to take care of it all means that they need to develop a deeper understanding of the app's functionality and architecture so that they can make a reasoned decision on whether particular access is necessary,
I do have one suggestion though. You could conceivably do your testing on machines with firewalls that block out-going connections for all but a set of white-listed hosts and ports. That white-list could be the starting point for the forms you need to fill in.
Have you looked into tagging? Manual or environment based variable set up looks painful (which is why I have avoided), but might be worthwhile? https://www.dynatrace.com/support/help/how-to-use-dynatrace/tags-and-metadata/
I'm trying to make simple application and deploy it on Google Cloud Platform Flexible App Engine, which will contain two main parts:
Front end application (simple Web UI based on Java 8 (Spring + Thymeleaf) with OAuth authorization from different external sites)
Back end application (monitoring several resources in separate threads, based on logged in users and reacting to their input in a certain way (behavioral changes))
Initially I was planning to make them as one app, but I think that potentially heavy background processing may cause failures in my front end application part + App Engine docs says that deployed services behave similar to microservice architecture.
My questions are:
Do I really need to separate front end from back end, if I need to react to user input as fast as possible? (but delays up to 2 seconds aren't that critical)
If I do need to separate them (and I strongly believe that I do) - how to I set up interaction between applications?
Each resource must be tracked exactly by one thread on back end - what are the best practices about this? I thought about having a SQL table with a list of acquired resources, but the flaw I see there is if an instance will fail I will need to make some kind of clean up on that table and redetermine which resources are actually acquired.
Your proposed architecture sounds like the right approach in separating the two into different services for the following reasons:
Can deploy code for each separately, rollback versions separately, and split traffic separately for experiments or phased rollouts.
Can adjust machine types and memory allocations for each service to better suit its needs. If you're doing work that is memory intensive on the backend, you can adjust that service's settings to allocate more memory per instance.
Allow each type of service to scale independently based on demands, which should result in better utilization of the services and less waste. This should also lower your overall spending than if you tried to go for a one-sized fits all approach in a single monolithic service.
You can mix different runtime environments across services. For example, you can mix language runtimes within a project OR you could even mix between standard and flexible environments. Say your front-end code is more cost efficient in standard, designate that service as a standard environment service and your backend as a flexible environment service. Or say you need a customer docker file with Perl in it, you could do that as a flexible environment custom runtime and have your front-end in Java 8.
You can still share common services like Cloud SQL, PubSub, Cloud Tasks (currently in alpha) or Redis for in memory caching. Your works don't need t reside in App Engine, they could reside in a different product if that better suits your needs.
Overall, you get much better control over your application to split it apart. The biggest benefit likely comes down to optimizing your application for spending only on what you need.
I think that you are likely to be able to deploy everything as an appengine app except if you use some exotic Java libraries that are not whitelisted. It could still be good to deploy it with compute engine for increased configurability and versatility.
You can create one front-end instance and one back-end instance in compute engine and divide the resources between them like that. Google's documentation has an example where you can do that.
I made a web based application by using the java language, and I would like to monitor its performance periodically (e.g. response time). Also I want to display this information on the homepage of my application. Is that possible? Can I have any idea about how this can be made.
Thanks.
You can take a look at stagemonitor. It is a open source java web application performance monitor. It captures response time metrics, JVM metrics, request details (including a call stack captured by the request profiler) and more. The overhead is very low.
Optionally, you can use the great timeseries database graphite with it to store a long history of datapoints that you can look at with fancy dashboards.
Example:
Take a look at the github page to see screenshots, feature descriptions and documentation.
Note: I am the developer of stagemonitor
Depending on your environment, I would use a cron job or task that measures the response time to request your app using something like HttpClient. Then drop that information into a database table accessible by your app.
The answer here is the simplest way you can measure the time: How do I time a method's execution in Java?
Why not checkout Munin monitoring? The website says
Munin the monitoring tool surveys all your computers and remembers
what it saw. It presents all the information in graphs through a web
interface. Its emphasis is on plug and play capabilities. After
completing a installation a high number of monitoring plugins will be
playing with no more effort.
SLAC at the Stanford university also keeps a large, quite well sorted list with various solutions for network monitoring among other things. SLACs list of Network Monitoring Tools, check for instance "Public domain or free network monitoring tools".
You can also consider to create your own custom web application monitor. Therfore, use the ProxyPattern and and create a concreate monitor. By using Spring framework you can easily swich on and off the monitor during runtime without re- deployment or restart of the web application. Furthermore you can create a lot of different specific monitors by yourself and are able to control what is beeing monitored. This gives you a maximum of flexibility, but requires a bit of work.
It is possible.
The clearest way to go about it, providing true numbers is to simulate a client that performs some sort of activity that mimics the real usage. Then have that client periodically use the website.
This presupposes that your website has a means to accept inputs that do not impact the real back end business. Crafting such interfaces requires some thought, but is not beyond the ability of a person who could put together the web site in the first place. The key points are to attempt to emulate as much using the real website as possible, but guard against real business impact. Basically it is designing for a special user (the tester).
So you might have a special user that when logged in, all purchases are bound to a special account that actually is filtered out to appropriately not demand payment and not ship goods. Provided the systems you integrate with all share an understanding of this live testing account, you can simultaneously test alongside of real production post-deployment.
Such a structure provides a huge benefit. You get performance of the real, live running system. Performance tends to change over time, and is subject to the environment. By fetching your performance numbers on the live system, in the same environment, you get a much better view of what real users might be encountering. Also, you can differentiate and track performance for different activities.
Yes, it is a lot more to design and set up; however, if you are in it for the long run, the benefits are huge.
I guess JavaMelody is the most appropriate solution for you. It can be built into a Java application and due to this feature, it monitors the functionality inside the app. Using this platform, it’s possible to get much more specific parameters for your Java app, than via external monitoring. In addition, it allows you displaying some statistics on your app homepage. Moreover, you can build in the app the graphs from JavaMelody, that essentially facilitates the app monitoring.
Take a look at the detailed overview of JavaMelody: http://cases.azoft.com/enterprise-system-monitoring-solutions-business-apps/
I have been comparing Java PaaSes carefully and am really starting to like CloudBees. I only have one big concern with them, and that is their SLA/uptime.
After scouring through all of their documentation, I can only find one paper they offer on SLAs which states:
If you are using the CloudBees PaaS without taking advantage of high availability options, then CloudBees can only offer uptime that approaches the base uptime SLA of
the infrastructure cloud provider.
As the same paper also mentions, Amazon seems to offer a 99.95% uptime, and I know that CloudBees runs - largely - on AWS/EC2 instances itself.
So this spawns a number of closely-related SLA questions:
If I don't take advantage of "high availability" options, then can I assume that CloudBees doesn't even guarantee 99.95%? Or is there documentation elsewhere that does state what their uptime is, and remedies for failing to meet that uptime?
What High Availability options are they talking about here? I just read their entire developer docs and never saw anything about HA.
What are my remedies if a partner service (like SendGrid for mail, or MemCachier for caching) goes down? One thing I do like about GAE is its CapabilitiesService where, before you go to use their Email API, or Caching API, you first check with the master CapabilitiesService to make sure those services are operating. I'd like to do the same with CloudBees, but seems like I'd need to build it myself. That's fine, but not sure if CloudBees even offers a mechanism (API call, etc.) to determine if a particular service partner is on or offline.
Thanks in advance!
CloudBees does not offer an SLA on availability nor remedies in the form of credits if a particular level of uptime is not met in a month. This is AFAIK common for other offerings on AWS (e.g., Heroku). CloudBees does offer standard response-time based SLAs via a support agreement. As discussed in the white paper you reference, we also employ practices for our own usage of AWS and external providers that has helped to isolate our users from some specific Amazon issues.
The availability features you can make use of include:
Using multiple instances (and potentially auto-scale). App instances are spread by CloudBees across different EC2 instances, so you can avoid downtime in the event of an EC2 instance failure.
Using the session store. You can share session state in a separate tier from your app instance using our offering or a partner offering like Memcachier.
Using dedicated servers that CloudBees sets up in multiple AWS availability zones.
Ensuring the database used with your app is set up in a highly available configuration. For example, RDS is simple to use with CloudBees and supports standbys and read replicas in multiple AZs.
Using app monitoring solutions from partners like New Relic and AppDynamics to alert you of any issues.
The main point of the comment about using "high availability options" was to warn people that simply deploying an app on CloudBees does not make it highly available. If an EC2 instance fails underneath your single-instance deployment, your users will experience downtime while our internal machinery redeploys to a working instance, whereas a multi-instance deployment will likely only experience slower responses until a new instance is deployed. Similarly with single-instance databases without standbys or replicas across AZs. While this is just stating the blindingly obvious for a lot of people, you might be surprised how many people just assume some magic is happening.
Good point on the CapabilitiesService! We have some ideas kicking around in this area, but you would have to do something like this on your own for now.
I have a task to build a prototype for a massively scalable distributed shared memory (DSM) app. The prototype would only serve as a proof-of-concept, but I want to spend my time most effectively by picking the components which would be used in the real solution later on.
The aim of this solution is to take data input from an external source, churn it and make the result available for a number of frontends. Those "frontends" would just take the data from the cache and serve it without extra processing. The amount of frontend hits on this data can literally be millions per second.
The data itself is very volatile; it can (and does) change quite rapidly. However the frontends should see "old" data until the newest has been processed and cached. The processing and writing is done by a single (redundant) node while other nodes only read the data. In other words: no read-through behaviour.
I was looking into solutions like memcached however this particular one doesn't fulfil all our requirements which are listed below:
The solution must at least have Java client API which is reasonably well maintained as the rest of app is written in Java and we are seasoned Java developers;
The solution must be totally elastic: it should be possible to add new nodes without restarting other nodes in the cluster;
The solution must be able to handle failover. Yes, I realize this means some overhead, but the overall served data size isn't big (1G max) so this shouldn't be a problem. By "failover" I mean seamless execution without hardcoding/changing server IP address(es) like in memcached clients when a node goes down;
Ideally it should be possible to specify the degree of data overlapping (e.g. how many copies of the same data should be stored in the DSM cluster);
There is no need to permanently store all the data but there might be a need of post-processing of some of the data (e.g. serialization to the DB).
Price. Obviously we prefer free/open source but we're happy to pay a reasonable amount if a solution is worth it. In any way, paid 24hr/day support contract is a must.
The whole thing has to be hosted in our data centers so SaaS offerings like Amazon SimpleDB are out of scope. We would only consider this if no other options would be available.
Ideally the solution would be strictly consistent (as in CAP); however, eventual consistence can be considered as an option.
Thanks in advance for any ideas.
Have a look at Hazelcast. It is pure Java, open source (Apache license) highly scalable in-memory data grid product. It does offer 7X24 support. And it does solve all of your problems I tried to explain each of them below:
It has a native Java Client.
It is 100% dynamic. Add and remove nodes dynamically. No need to change anything.
Again everything is dynamic.
You can configure number of backup nodes.
Hazelcast support persistency.
Everything that Hazelcast offers is free(open source) and it does offer enterprise level support.
Hazelcast is single jar file. super easy to use. Just add jar to your classpath. Have a look at screen cast in main page.
Hazelcast is strictly consistent. You can never read stale data.
I suggest you to use Redisson - Redis based In-memory Data Grid for Java. Implements (BitSet, BloomFilter, Set, SortedSet, Map, ConcurrentMap, List, Queue, Deque, BlockingQueue, BlockingDeque, ReadWriteLock, Semaphore, Lock, AtomicLong, CountDownLatch, Publish / Subscribe, RemoteService, ExecutorService, LiveObjectService, SchedulerService) on top of Redis server! It supports master/slave, sentinel and cluster server modes. Automatic cluster/sentinel servers topology discovery supported also. This lib is free and open-source.
Perfectly works in cloud thanks to AWS Elasticache support
Depending of what you prefer, i would surely follow the others by suggesting Hazelcast if you're towards AP from the CAP Theorem but if you need CP, i would choose Redis
Have a look at Terracotta's JVM clustering, it's OpenSource ;)
It has no API while it works efficent at JVM level, when you store the value in a replicated object it is sent to all other nodes.
Even locking and all those things work transparent and without adding any new code.
You may want to checkout Java-specific solutions like Coherence: http://www.oracle.com/global/ru/products/middleware/coherence/index.html
However, I consider such solutions to be too complex and prefer to use solutions like memcached. Big disadvantage of memcached for your purpose is lack of record lock it seems and there is no built in way to replicate data for failover. That is why I would look into the key-value data stores. Many of them would satisfy your need completely.
Here is a list of key-value data stores that may help you with your task:
http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores
Just pick one that you fill comfortable with.
I am doing a similar project, but instead targeting the .NET platform. Apart from the already mentioned solutions, I think you should take a look at ScaleOut StateServer and Alachisoft NCache. I am afraid neither of these alternatives are cheap, but they are a safer bet than open source for commercial solutions according to my judgement.
Both provide Java client APIs, even though I have only played around with the .NET APIs.
StateServer features self-discovery of new cache nodes, and NCache has a management console where new cache nodes can be added.
Both should be able to handle failovers seamlessly.
StateServer can have 1 or 2 passive copies of the data. NCache features more caching topologies to choose between.
If you mean write-through/write-behind to a database that is available in both.
I have no idea how many cache servers you plan to use, but here are the full price specs:
ScaleOut StateServer
Alachisoft NCache
Both are installed and configured locally on your server and they both have GUI Management.
I am not sure exactly what strictly consistent involves, so I'll leave that for you to investigate..
Overall, StateServer is the best option if you want to skip configuring every little detail in the cache cluster, while NCache features very many features and caching topologies to choose from.
Depending on the behaviour of data towards the clients (if the data is read many times from the same client) it might be a good idea to mix local caching on the clients with the distributed caching in the cluster (available for both NCache and StateServer), just a thought.
The specified use case seems to fit into Netflix's Hollow. This is a read-only replicated cache with a single producer and multiple consumers.
Have you tought about using a standard messaging solution like rabbitmq ?
RabbitMQ is an open source implementation of the AMQP protocol.
Your application seems more or less like a Publish/subscribe system.
The Publisher node is the one that does the processing and puts messages (processed data) in a queue in the servers.
Subscribers can get messages from the server in various ways. AMQP decouples the producer and the consumer of messages and is very flexible in how you can combine the two sides.