I have database driven web site that needs more than one MySQL Sever to handle the expected demand
I also need to implement back up system (of some type) to keep data safe.
I'm using java but that that’s not critical
What options are available to me from projects out their
I'm thinking of daisy chaining project with the MYSQL server's somehow and then when one is busy go to the next and they all be written data to. I know they can measure time used they must be able to measure when they are in use.
You might want to look into clustering.
http://www.mysql.com/products/cluster/
How about deploying a Cluster in the cloud?
http://www.mysqlconf.com/mysql2009/public/schedule/detail/6912
Related
What are recommended strategies for building Java application that will be run on "desktop", not in browser. Characteristics of the application would be:
1. Multiple application instances would be running on different machines
2. Applications must communicate in real-time (if one user make changes,
in another application data must be refreshed)
Do you want to create a networking application maybe? based on sockets and so on? Regarding your 2 questions, I have implemented that scenario some time ago and I am working in something similar for my job, it is not complex at all, but I will answer to you according the two issues that concern to you.
Multiple application instances would be run on different machines.
If you are going to install an instance of the application in the people's desktop, I'd suggest to be very careful with "paths", do not hard code any path, since the resources loading will be dynamic.
Check carefully what is the network architecture in which your application will be installed. Maybe it is just a LAN, or maybe it will work in a big network and access through VPN, etc. Check what is the scenario.
Once you make sure your application works fine in different machines without any path conflict or resource loading conflict, you can export your jar, generate it using maven, ant, etc.
Also, if you want to move forward, you can create an installer using any Install wizard creation and create a batch file (.exe) for Windows or (.sh) for Linux distr. But these are only suggestions for the installation stage.
On the other hand, if you wanna execute the application as a Java desktop but using an URL to launch it, you can take a look to JNLP.
Applications must communicate in real-time (if one user make changes, then other will be able to see that)
If you want to do that, you will need, for sure, a server to provide and store information. The server can be a physical machine set up in the office or a remote one.
You have two options here:
Use Java Networking: Create an application that works as a server that provides and saves the information (it should be a concurrent environment since many people will perform transactions or queries over it). Check how can you create a basic server - client application using Sockets to understand better how it works and then you will not have problems to add the complexity of the requirements your environment demands.
You can simply, develop a Java REST Based application and make your Client application connect to the machine (or machines if you plan to implement load balancing) and consume those REST. You can take a look to Jersey libraries in order to implement your scenario. Make sure to add security to these Web Services and make the server private access for the network in which your application instances will work.
Well, that's what I can tell you regarding the scenario you try to implement, based on what I've done and what I'm doing now so far.
Maybe if you need additional or further information, you can reply in the comments, and it will be great to help you.
Regards and happy coding :)
you want to look into using sockets, TCP or UDP, and also figure out if you want a central authoritative server ( what if two users change the same thing in different ways, whose data is saved?)
read this article from Oracle/Java hereJava Custom Networking
Here is my situation:
I wrote two REST clients in Java which are running on my server. And these clients are packaged into Runnable Jar files. I set a schedule for running it. Every time data has been synced around 3MB.
Recently, I needed to write two more clients to sync from other resources. Before that, I didn't have software architecture experience to build an efficient client System.
My problem is my server is not good for running Microsoft windows server 2003 R2. The hardware info is following:
CPU: Intel Xeon E5649 2.53GHZ
RAM: 2GB
I use MySQL database, basically the sql writes per second around 20. It is very slow. From now, doing one synchronization takes 2 hours. I could not imagine 4 REST clients running on Monday.
I need help with how to deal with four clients running on a low capacity server. However, please don't convince me to change a new powerful server :)
I have been thinking for a long time, could I only build a client which sync data from different resource? Or build four clients which running on a different schedule? The other problem is in the future more resources will be added. I don't know how to build a system with strong scalability because of my lack of knowledge.
If you could give me some advice to push my learning a little bit, it will be very appreciated. Thank you.
Further information:
The goal is to grasp data from different RESTful servers with different API and GET query rules and insert these data into Database. Thus, basically the application's job is to put data into MySQL via RESTful call.
In addition, I only focus on it. I do not need to consider about how to deal with inserted data. The structure is simply:
Get Restful call and get JSON format result
parsing JSON
insert into DB
I used (JAX-RS) Jersey api to implement RESTful call, and use JDBC to manage the database, but I am working on the next version which will use Hibernate to implement insert, delete, update, and search functions.
This application implementation does not use any application server. I package the program to runnable jar file, and set schedule to run it on windows server 2003.
Being a recently graduated student, that is my first REST client, but with my deep research and studying, I know more about it. However, I don't have experience on it. I just want to make it work better. Any suggestion, I appreciate it.
Use a profiler to gain insight into the actual memory and CPU usage of your application. You can then decide how to improve it using the appropriate means. (F.e. multithreading, caching, compression, ...)
I made a web based application by using the java language, and I would like to monitor its performance periodically (e.g. response time). Also I want to display this information on the homepage of my application. Is that possible? Can I have any idea about how this can be made.
Thanks.
You can take a look at stagemonitor. It is a open source java web application performance monitor. It captures response time metrics, JVM metrics, request details (including a call stack captured by the request profiler) and more. The overhead is very low.
Optionally, you can use the great timeseries database graphite with it to store a long history of datapoints that you can look at with fancy dashboards.
Example:
Take a look at the github page to see screenshots, feature descriptions and documentation.
Note: I am the developer of stagemonitor
Depending on your environment, I would use a cron job or task that measures the response time to request your app using something like HttpClient. Then drop that information into a database table accessible by your app.
The answer here is the simplest way you can measure the time: How do I time a method's execution in Java?
Why not checkout Munin monitoring? The website says
Munin the monitoring tool surveys all your computers and remembers
what it saw. It presents all the information in graphs through a web
interface. Its emphasis is on plug and play capabilities. After
completing a installation a high number of monitoring plugins will be
playing with no more effort.
SLAC at the Stanford university also keeps a large, quite well sorted list with various solutions for network monitoring among other things. SLACs list of Network Monitoring Tools, check for instance "Public domain or free network monitoring tools".
You can also consider to create your own custom web application monitor. Therfore, use the ProxyPattern and and create a concreate monitor. By using Spring framework you can easily swich on and off the monitor during runtime without re- deployment or restart of the web application. Furthermore you can create a lot of different specific monitors by yourself and are able to control what is beeing monitored. This gives you a maximum of flexibility, but requires a bit of work.
It is possible.
The clearest way to go about it, providing true numbers is to simulate a client that performs some sort of activity that mimics the real usage. Then have that client periodically use the website.
This presupposes that your website has a means to accept inputs that do not impact the real back end business. Crafting such interfaces requires some thought, but is not beyond the ability of a person who could put together the web site in the first place. The key points are to attempt to emulate as much using the real website as possible, but guard against real business impact. Basically it is designing for a special user (the tester).
So you might have a special user that when logged in, all purchases are bound to a special account that actually is filtered out to appropriately not demand payment and not ship goods. Provided the systems you integrate with all share an understanding of this live testing account, you can simultaneously test alongside of real production post-deployment.
Such a structure provides a huge benefit. You get performance of the real, live running system. Performance tends to change over time, and is subject to the environment. By fetching your performance numbers on the live system, in the same environment, you get a much better view of what real users might be encountering. Also, you can differentiate and track performance for different activities.
Yes, it is a lot more to design and set up; however, if you are in it for the long run, the benefits are huge.
I guess JavaMelody is the most appropriate solution for you. It can be built into a Java application and due to this feature, it monitors the functionality inside the app. Using this platform, it’s possible to get much more specific parameters for your Java app, than via external monitoring. In addition, it allows you displaying some statistics on your app homepage. Moreover, you can build in the app the graphs from JavaMelody, that essentially facilitates the app monitoring.
Take a look at the detailed overview of JavaMelody: http://cases.azoft.com/enterprise-system-monitoring-solutions-business-apps/
First of all, I have a conceptual question, Does the word "distributed" only mean that the application is run on multiple machines? or there are other ways where an application can be considered distributed (for example if there are many independent modules interacting togehter but on the same machine, is this distributed?).
Second, I want to build a system which executes four types of tasks, there will be multiple customers and each one will have many tasks of each type to be run periodically. For example: customer1 will have task_type1 today , task_type2 after two days and so on, there might be customer2 who has task_type1 to be executed at the same time like customer1's task_type1. i.e. there is a need for concurrency. Configuration for executing the tasks will be stored in DB and the outcomes of these tasks are going to be stored in DB as well. the customers will use the system from a web browser (html pages) to interact with system (basically, configure tasks and see the outcomes).
I thought about using a rest webservice (using JAX-RS) where the html pages would communicate with and on the backend use threads for concurrent execution.
Questions:
This sounds simple, But am I going in the right direction? or i should be using other technologies or concepts like Java Beans for example?
2.If my approach is fine, do i need to use a scripting language like JSP or i can submit html forms directly to the rest urls and get the result (using JSON for example)?
If I want to make the application distributed, is it possible with my idea? If not what would i need to use?
Sorry for having many questions , but I am really confused about this.
I just want to add one point to the already posted answers. Please take my remarks with a grain of salt, since all the web applications I have ever built have run on one server only (aside from applications deployed to Heroku, which may "distribute" your application for you).
If you feel that you may need to distribute your application for scalability, the first thing you should think about is not web services and multithreading and message queues and Enterprise JavaBeans and...
The first thing to think about is your application domain itself and what the application will be doing. Where will the CPU-intensive parts be? What dependencies are there between those parts? Do the parts of the system naturally break down into parallel processes? If not, can you redesign the system to make it so? IMPORTANT: what data needs to be shared between threads/processes (whether they are running on the same or different machines)?
The ideal situation is where each parallel thread/process/server can get its own chunk of data and work on it without any need for sharing. Even better is if certain parts of the system can be made stateless -- stateless code is infinitely parallelizable (easily and naturally). The more frequent and fine-grained data sharing between parallel processes is, the less scalable the application will be. In extreme cases, you may not even get any performance increase from distributing the application. (You can see this with multithreaded code -- if your threads constantly contend for the same lock(s), your program may even be slower with multiple threads+CPUs than with one thread+CPU.)
The conceptual breakdown of the work to be done is more important than what tools or techniques you actually use to distribute the application. If your conceptual breakdown is good, it will be much easier to distribute the application later if you start with just one server.
The term "distributed application" means that parts of the application system will execute on different computational nodes (which may be different CPU/cores on different machines or among multiple CPU/cores on the same machine).
There are many different technological solutions to the question of how the system could be constructed. Since you were asking about Java technologies, you could, for example, build the web application using Google's Web Toolkit, which will give you a rich browser based client user experience. For the server deployed parts of your system, you could start out using simple servlets running in a servlet container such as Tomcat. Your servlets will be called from the browser using HTTP based remote procedure calls.
Later if you run into scalability problems you can start to migrate parts of the business logic to EJB3 components that themselves can ultimately deployed on many computational nodes within the context of an application server, like Glassfish, for example. I don think you don't need to tackle this problem until you run it to it. It is hard to say whether you will without know more about the nature of the tasks the customer will be performing.
To answer your first question - you could get the form to submit directly to the rest urls. Obviously it depends exactly on your requirements.
As #AlexD mentioned in the comments above, you don't always need to distribute an application, however if you wish to do so, you should probably consider looking at JMS, which is a messaging API, which can allow you to run almost any number of worker application machines, readying messages from the message queue and processing them.
If you wanted to produce a dynamically distributed application, to run on say, multiple low-resourced VMs (such as Amazon EC2 Micro instances) or physical hardware, that can be added and removed at will to cope with demand, then you might wish to consider integrating it with Project Shoal, which is a Java framework that allows for clustering of application nodes, and having them appear/disappear at any time. Project Shoal uses JXTA and JGroups as the underlying communication protocol.
Another route could be to distribute your application using EJBs running on an application server.
I am developing a web application with php that needs to synchronize a local mysql database that a java desktop version of the web application is interacting with. At the same time i also need the local db to synchronize with the remote db. How do i do this without using other software like MySQL Compare. I will really appreciate the help. Thanx guys.
You clearly have a significant architecture issue. It needs to be planned very well. Two-way replication clearly isn't going to work unless you have thought it out very carefully and understand how to do conflict resolution and what impact that will have on your application. In particular, you can forget using AUTO_INCREMENT.
For one-way replication, you can use mk-table-sync, or use mysql replication in some way (there are a variety of possiblities).
You can also run another mysql instance on the server, use mk-table-sync to periodically synchronise it locally, and use mysql replication on that. This has some benefits, particularly if there are some tables you don't want to replicate.
You really need to think about how it's going to work, if you plan to do two-way synchronisation. It is possible that you may end up writing custom code to do it, as the conflict resolution mechanism may mandate it.