I am thinking of working on an online homework submission system where students can upload their text documents that can viewed by somebody(Teacher/Professor) with viewing rights. This person can annotate/mark up/highlight this document and then can pass this document on for further review. After the review is done, this document is sent back to the student with proper feedback.
There are a lot of moving pieces here and I am having a tough time trying to figure out where to start. I am using this project as an exercise to design a scalable/secure/fast web-application using entirely open source tools that supports desktop like user experience but besides the UI-framework(Vaadin) and language(Java), I am having a little bit of trouble figuring out the roadmap needed to make some progress with this.
It will be great if the SO gurus can mentor me through this or provide me with a nudge in the right direction.
Edit: Thanks for the response. This is the standard three-tiered architecture that was described in the response. I need massive scalability and since the application will be primarily document centric and I may have to retrofit search at some later date, I would like to steer away from RDBMS. Since we have a massive amount of users posting their documents at any given time(lets say *.txt) files for the time being, I would need some kind of message queue to process this massive information influx. There has to be some kind of a fast transformation layer that takes the documents in all its formats and displays it in a format suited for annotations and markup....and the list goes on. Starting with the Domain model and moving down would be ideal but I am a little bit of a skeptic.
I'm a Java guy. How I would approach this problem will be,
Figure out the data model. What are the Objects that the system will be using.
Based on that design your database. You can use ORM frameworks like Ibatis or Hibernate ( Ibatis has a code generator which generates pretty much all the DAO's to access the tables. It gives you the CRUD methods and you can add on top of it).
Once you are done with that, then you can start designing your service layer. Its not a good practice to expose the DAO's directly to the controllers( MVC pattern). This is where your business logic should go in.
The choose one of the web frameworks available. The most popular java web frameworks are Spring MVC. You can also try Google Guice.
Final step will be to design your front end. I think your project will include a lot of javascript. So look into JQuery or EXT JS
I hope this helps you to get started with.
Not really a "full answer" but hopefully some food for thought.
I am using this project as an exercise to design a scalable/secure/fast web-application using entirely open source tools...
That's a great thing to be doing - so my question would be why worry about the architecture? I admit this is a bit strange coming from someone who see's themselves as an advocate for architecture, but "learning architecture" and learning new technologies can both be demanding. Maybe you just want to get up-to speed with one first, then tackle the other?
Architecting, designing and building a system that is "scalable/secure/fast" isn't trivial. In a "real-world" business driven case you'd have a system context to fit in with, and non-functional requirements to hit: these might suggest a different technology stack. Different key drivers will have a huge affect on how you approach things and what decisions get made - and of course the architecure would be built around that.
Edit:
I'd start with what was more important to you; then as I start on the "other" topic I'd be constantly checking what I'd learnt / my assumptions.
Depending on the two subjects there might be some "synergies" which suggest a different approach. Part of me wants to recommend doing enough on one to get a good basic grasp and then bring the other one up to a similar level - then move on. That way you're kind of more consistent.
Another part of me says - just do what's most interesting!
Should I plumb the depths of a given technology stack before moving forward or get the blueprint right from a tool agnostic standpoint?
As a start, you'd definately want to get a broad but shallow view; maybe go into some specific areas in a little more detail if they were of particular importance to you.
Or is my blueprint going to change because of the technology constraints?
By this I'm assuming you mean that: if you discover a "given technology works in such a way" and how it works it different form what you were expecting - then will this affect the blueprint?
It could do - that's why a broad but shallow view of the relevant technology should help you avoid the worst / most obviously bad mistakes. I'm also assuming that you have enough overal experience to know what you're looking at when you start doing this - that you can recognise an area that needs more detailed study before committing to a design.
At a high level, all you're looking to do is ensure you're not restricting options further down the path.
Related
I'm involved in a project at the institute where I study that consists in build an thematic online dictionary. I'm thinking in make it from zero, constructing an MySQL database and using it with servlets and jsps. I'm not professional and that's my first experience with a real project, so I'm afraid that this way may, in the future, cause problems such as difficulties in maintenance, for example. I know that there are many CMSs, but I would to use this experience to put all my theorical abilities in practical. Yes, I've time to waste. What I need to know is: the fact that I don't know things like Struts or Spring precludes that I make a functional system?
Thank you.
If you are willing to learn new technologies and break you head for a few hours in the beginning learning and applying it, then no, not knowing spring or struts does not preclude you from creating a functional system. Maintenance is a whole new ballgame/discussion. Even the best programmers in the world cannot create a zero maintenance system. The only thing you can do is learn the system well enough (and be prepared to make the learning on going) so that you can put in the best guard rails around it and also be able to debug issues. Now, if this system that you are planning on building needs to be a professional grade system, then I would recommend hiring/getting someone who has built web applications in the past and can maintain it. That person can help carry you until you can take over. If this is a smallish project which can live with a few initial issues/downtimes (which you can work on to improve), then I say go for it.
The best way (IMO) to learn is to take a sample application, deploy it and start reading code. See if you can trace the request from the browser (I am assuming your application is a web applications) into the framework, to your controller/models and back to the view. Grab a sample spring mvc application (for e.g. http://static.springsource.org/docs/Spring-MVC-step-by-step/) and see if you can make sense of it.
Is it wise to develop a prototype GUI before designing other part of the system?
I am using Java for this small project. It will be a program with GUI and database connection. Say the database has table A and B, the user can choose which table to interact with. The program then display the contents of, say, table A in the GUI, and allows the user to change the content and submit the changes, or delete, or insert.
I think GUI should be developed first before any back-end development starts. There are couple of reason to do this:
You gain clarity on how model objects should interact.
Usability poses lots of restrictions on the way you want to pull data. You will probably want to develop and architect after you're 100% sure what constraints are there.
On business point, managers like to have a dumb function UI before any development start. Many times, the feedback leads in major changes in back-end assumptions. Which is a lot less pain than the case when you get a change request after the back-end development is over.
My personal experience goes that simultaneous development of GUI and back-end is a bit messy. Plus GUI provides solid expectation of behavior from back-end. Moreover, this approach makes sure all the developers, your client and your manager on the same page.
I agree with Joel Spolsky that it is a great idea to write a functional spec before writing code. Part of that spec should include a collection of screen mockups. #O.D. is right, Balsamiq is a great tool. It has saved me a lot of time in the past.
Once you have a functional spec in place that the business users are happy with, you will then have a better idea of how to design your system to meet the requirements. e.g. is high performance a requirement, domain model vs simple crud etc.
Then you should start by taking a single use case and building a vertical slice of your application. Build a GUI, service layer, persistence layer, database schema in one iteration. This will hopefully point out any problems with your design and give you the chance to modify it before you start building out the horizontal functionality.
I'd say yes and no.
No because you should design you application to be modularized enough so that your logic and data do not depend on UI design.
Yes because it is always smart to design everything before you actually start implementing it.
So what I mean is that you should make a concept, but not let your UI concept 'tie your hands' when you implement your logic. So if your managers clients don't like your conceptual UI, you can always change it without actually changing your application logic.
Well showing you GUI brfore starting to program is a very a good Idea, specially that you enable the enduser (Customer) to check if the UI is up to his expectations, which can save you lots of time.
In order to do that you dont necessarily need to develope a "real" prototype, you can use programms which enable you to fast design the UI of your App, including a minimal workflow simulation instead of full funcionality.
i had a very good experience with: Balsamiq can really recommend it
Writing spec before your code is always a good idea, because it makes you think. But most specs I have seen are not that good. And if the spec is too technical, users will at the end sign-off your spec without really understanding what are they going to get.
I have seen best results when either presenting the User Manual to the client, or by discussing mockups of the system one scenario at a time.
Note that half-baked mockups won't do the trick. You need your mockups to be fully populated with relevant data (Ever tried to discuss some screens with accounting while the numbers on the screen don't match? There's no way at all you could explain to them these are only dummy numbers...)
And the caveat of using mockups is that users will more often than not believe the app is "almost finished", whatever you do or say. It must be some subconscious thing, I'm not sure. But to avoid that, most of specialized tools have either only "black&white" look and feel or multiple skins you can switch to and from.
There is a pretty complete list of mockup tools here. Many of them are free:
http://c2.com/cgi/wiki?GuiPrototypingTools
My own tool is pretty popular: http://MockupScreens.com, I created it a long time ago exactly because of my own frustration with above mentioned problems.
I am looking to start work on a brand-new project, something I've been thinking about for a while as my first independent sellable project.
It's broadly speaking a web-based service application, and my first choice, server-language is quite easy... I know Java pretty well from working on Java web-apps in the past.
However my experience doing web-apps involved JSP, Servlets and JSTL... I know the ideas behind newer technologies like Hibernate/Spring but have never used them. So we wrote our own DAOs, handled AJAX by writing special mini-JSP pages that generated XML/JSON pages, etc.
I'm not hugely into the idea that Spring/Hibernate are the 'only' or 'right' way to do any Java web-project, but they are widely used. On the other hand, not only would trying to learn these increase initial development time, but I'd be using my learning attempts to build a production system.
I remember one of Joel's early articles said (I'll paraphrase since I can't find it)
"regardless what's cool, always use
the technologies that the lead
developer (or dev team?) knows best"
I wondered what people thought about that?
ps: should this be CW?
I work as a consultant, and I've seen a lot of projects where the devs started out with servlets+JSP because that's what they knew, and it's pretty simple to get started with. However, it gives the team an opportunity/excuse to write a platform of their own, which is more fun than using someone else's and just writing an application.
As the project grows, the team reinvents more and more wheels, quite a few of which end up square. That's where I enter the picture - adding new stuff to this semi-flexible platform has become so complicated that the devs can't keep up adding features and fixing bugs without calling in reinforcements. Just to add insult to injury, the internal devs are usually the ones who get assigned to do the boring bug fixes because bug fixes require more knowledge of the gory entrails of what has become the team's proprietary persistence-and-web framework, and so those gosh-danged consultants get to do the new, fun stuff.
Now, you shouldn't use a framework just because people have been regurgitating each other's blog posts about the awesomeness of it, but you should also realize that there are very good reasons why those frameworks exist (and why they're used). If you haven't used any web frameworks at all, I'd recommend you to take Spring MVC, Wicket or whatever for a test drive. They don't solve all problems, and they do cause some of their own, but the grand total is usually a productivity increase, especially if you're making advanced user interfaces.
I have been on projects where plain JDBC has been quite sufficient for persistence, and where no more advanced web frameworks than servlets+JSP have been needed, but those projects are a minority. Without having used a framework or two, you'll never whether your project is part of that minority that doesn't need one, or if it is part of the grand majority that does.
Don't try everything all at once - take on one new technology at a time.
Beware the lure of cool new frameworks! I'm currently hacking on a tiny little web app that just has a login, a few mostly static pages, and a few forms to request some information by email. It would have taken me maybe two days to do as traditional Servlet/JSP in MVC style. Instead, since there was slack in the schedule, I decided to use this project to get up to speed in Spring, Spring MVC, and Spring WebFlow. While it's quite possible that I'm just dense, it took me several weeks to get my head around the right way of doing things, I'm still not totally confident that I'm doing everything correctly, and the application is still not done. Fortunately, due to slack, I'm not in danger of the overall project schedule slipping, but I'm always asking myself if I'm going to have to scrap it and start over.
I have learned my lesson, though: next time, I won't be the one pushing a new framework unless its one I've used for production projects before. That said, I'm glad I now understand Spring (or at least I think I do) and will not hesitate to use it again next time.
So how would I learn a new framework next time? If there's a project lead (in this case I'm a project lead of a team of one, no help there) I'd use the framework that they put in place. If there isn't, or if I want to learn a framework that the project lead isn't using, I'd use it for a side project on my own time. Learning is good. Putting company work at risk by throwing untested technology at it is not so good.
I can say for sure that Spring is worth considering. It gives you as much as you can take, but it doesn't bother you with things you don't need.
For example, at the very beginning you probably need dependency injection only. Then you'll need help with database interactions and transaction management. Then you'll decide to apply MVC patter to your web-application. After that may be you'll realize that components of your system are to send JMS to each other. And so on and so forth.
For all this cases Spring has it's own simple, intuitive, light-weight solution.
When starting a new project limit the number of unfamiliar technologies / frameworks to use. Every framework takes time to learn and every framework has issues especially if not implemented correctly.
If you can I would recommend you look into the Play framework. It is a web framework for Java that focuses on developer productivity. You can choose to use Spring / Hibernate if you want but you are not bound by that. It has a very easy to learn implementation and you should be able to get a good idea within a day of playing around with it if it is what you are looking for.
It depends what the customer wants (in the world of consultancy).
You have to learn new technologies. Does the customer wants to pay for that?
Not all the caveats of the new ones are known, whereas the older ones are proven a lot more.
Of course, if everybody thought like this we would all be stuck with VB these days. You have to look for the right balance, and learn a lot yourself too so you can get an objective view on the technologies available, their up and downsides.
i personally would definitely recommend looking into spring, i've found it's saved me countless hours. hibernate is also useful if you need an ORM layer (and spring has nice integrations with hibernate too). they're certainly not the 'only' or even the 'right' way to do things (that's quite subjective) but they have both saved me time and effort, especially spring.
There is one major trap with any unknown technology. You do not know where the dragons are, and you do not know how to rub the new technology "with the hairs".
Learning that will take time, and you need to have that in your estimates. Also your estimates will most likely be too low...
The project which I am going to start is my first project and it is very big. Though it's a great opportunity for me, I don't want me to trapped myself in a messed-code in the end so I have made a whole design of the software (software architecture) divided it into three tiers:
Presentation Tier ---- (will be implemented through Java Swing)
BusinessLogic Tier -- (will be implemented through EJB technology)
DatabaseLayer ------- (will be implemented with the help of Hibernate)
Q1. Which Tier should I select to start with?
I don't have any experience of Standard Product Development environment but I am sure that there is some specific order which is better than other.
Q2. I think that these things come under Good Design Principles and Best Practices. I have searched the internet for these kinds of resources and have found some good resources too but I would be grateful if you recommend me some resources that you know have short, to-the-point and quality content?
"my first project and it is very big"
Please don't do that.
Please do a small project first.
Write the "model" (business logic) first. This will be very difficult, since it's your first project. Keep it small and focused on just business logic that you can test and prove that it works.
Throw that away.
Now do another project.
Write the "model" (business logic) first. Based on lessons learned during the first project this will be much better. This will be hard, since it's your second project. Keep it small and focused on just business logic that you can test and prove that it works.
Write the persistence and the object-relational mapping second. Adding database persistence will be very hard, since it's only your first database project, and only your second project.
Throw that away.
Now, you have some idea what you're doing. Start a third project.
Write the "model" (business logic) first. Based on lessons learned during the first two projects this will be much better. By now, this will still be a lot of work because you finally understand what you're doing. The work, however, will no longer involve technical issues, it will now involve real issues around use cases and what the application actually does.
Write the persistence and the object-relational mapping second. Based on lessons learned during the first projects this will be much better. This will still be hard. Nothing makes this easy. It's only your second time, so you'll still make mistakes; there will be fewer mistakes.
Write the presentation last. Always.
This is actually the fastest way to do a large project when you don't already know the technology.
Do the implementation in vertical stripes: implement one capability of the project from the three layer so that the design could be validated from end-to-end.
The database layer could be skipped first, data could either be hard coded, not persisted, or simply store in test files first.
There are two approaches to this and both have advantages and disadvantages:
Top-down
First develop the UI and implement the data access and buisness logic where required for the UI.
Bottum-up
Develop the Database Layer first and then add the UI functionality later.
The first has the advantage that you have a (at least partially) working UI fast which is something the customers like, so it's the de-facto industry standard. The latter one provides a stable and complete Database and business logic layer but they usually have some overhead because it is not clear how the data is later required/accessed.
I think you should start with the business logic layer as this layer should be the one that dictates which functionality the other layers should have. If you start with the data access layer there's a risk that you will end up with a lot of YAGNI code, and furthermore any application should be centered around business logic and not persistence or presentation logic.
You should look into such best practices as DDD and TDD, and you could probably benefit from knowledge about MVVM or similar pattern too.
I would add to the existing answers:
Do not use lots of technologies you're unfamiliar with or only have a basic undersyanding; EJB, Hibernate, etc. They give you no appreciation of what's going on under the covers and add to the learning curve, pushing out your deadline. Instead go for something a lot simpler; e.g. RMI client-server application with JDBC or Spring/JDBC for persistence. You can always rework elements later but better to deliver incrementally than not at all.
Q1. Take a pen and a lot of paper and start designing. That way you make "plans" before writing code.
Q2. Using paper first help designing for quality; regular refinement, refactoring and review help maintaining quality.
Design Principles:
Start here
http://www.butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod
I've done lots of java web development using jsps and servlets, and I have found this approach to be straightforward and flexible. Some of the groundwork involved though - such as managing database connections - is rather tedious, and it takes a fair amount of work just to get a new web app off the ground.
I'm therefore considering using a framework for the first time, but my impression of frameworks is that they're mostly used for large J2EE applications and involve a lot of complex configuration. What I'm looking for is something simple that (after the initial learning curve) will enable me to get up and running with a new web app as quickly as possible.
So my question is - does it make sense to use a framework for a simple java web app?
Note that I'm not asking which framework to use (if indeed a framework is recommended), as it has already been asked here.
If you don't use a web framework you'll usually end up writing one - poorly.
It makes a lot of sense. My team has spent the better part of five years with our open source stack and no matter and we have a "seed" project (works like appfuse) that we use to create all new web apps. Even the simple two pagers from the pov of maintaining the app, it looks like every other app, just smaller.
The short is you won't get any return on the investment right now, but you will as the project evolves and you maintain it.
Yes, I would use a web framework for the following reasons:
Increased navigation capabilities and controls. Even though you may not need them, they are there for you to use should you require them at any time
As others have indicated, apps grow over time and you will feel the need for the framework at some point in time in the future. When you need to add additional pages and navigation
Features that allow you to plug into other frameworks such as security and DB access frameworks. Spring is a prime example in the Java world. You never have to use Spring but it plugs in so well with Struts, Spring MVC, Hibernate, Acegi etc. It ends up saving you the hassle of doing all of the plumbing on your own.
Support!!! Good frameworks almost always have a vibrant community to support them and ask and answer questions.
It may seem like too much hassle initially but definitely saves you much time in the future
I'd say it actually matters quite a lot which framework you go for. Something like Spring MVC is fairly unobtrusive in your code and allows you existing stuff to run quite a lot as-is. Other frameworks have much more specific ideas about how you should do things.
Yes it does make sense. Apps can grow and change and might require something a framework can provide easily in the future.
For example at my workplace we have simple jsp / servlet app. It needs to be rewritten just because of what I explained above. If someone would have taken the time to just get the framework setup we would be in better shape today.
Yes it makes sense. However implicit in your question is that the wrong (for you) framework can be more pain than it's worth - and that's true. There is a world of difference between some heavy J2EE framework and something light and cheerful like grails.
What's the alternative? Rolling your own? Embedding all the navigation and logic in the JSPs?
I agree with those who say that a web framework is worth it. There are literally hundreds now (e.g., Struts, JSF, Spring, Wicket, etc.). Pick one that suits you.
There are two kinds of applications:
1) the kind that you throw away and never use again, and therefore should not worry about modularity, maintainability and clarity.
2) the real kind.
It may sometimes seem that your app may never have to grow, scale, or service a larger feature set / user base than you currently are planning for....I assure you, that perception is always wrong.
Frameworks, specifically things like Struts for MVC in Java, Spring for MVC and Dependency Injection, Hibernate for Object-Relational Modeling are all extremely valuable tools that lead to modularity, maintainability and clarity in your code. So, to answer your original question.... Yes, emphatically.
No it doesn't unless:
you don't have any java experience
in your team
you are doing just a prototype that
must be ready in hours
you don't trust your developers
enough to let them write your
application
you don't plan to let your
developers learn and improve
Danger in using frameworks include but not limited to:
you'll get all framework defects on
top on your own
you'll be unable to do realistic estimates until you
know perfectly the framework
your team won't learn how to program a
web server
you'll find your team passing more
and more time on google instead that
solving problems by writing code.
The only thing that is worst that choosing a open source framework is having a separate R&D team in the company who should create the "big complete ultimate company framework".
but my impression of frameworks is that they're mostly used for large J2EE applications and involve a lot of complex configuration
Not necessarily true. The good frameworks are built to scale well so that they will take you from small apps to very large apps. Many of the frameworks today are moving towards zero configuration so you will find them easier and easier to use.
You are right that it does take an initial effort to learn the framework itself but that investment pays for itself in the very first application you build. And meta frameworks like AppFuse makes it even easier to get started since it pre-configures the frameworks for you.
Frameworks make sense for most applications. It may be necessary for you to build your own or adopt some other framework. The most important questions is the granularity of your data structures. What I mean by this is do you need to just enter data or do you need to parse, compile and execute dynamic code?
If you think of a frame works as a gauge and to the left is an all ready existing frame work open source or closed and to the right is all custom framework then super impose your code on top of that frame work where the level of complexity increases to the right. The farther right you go the more of a stuggle you will have with the frame work (IMO).
However, you can also slowly migrate from existing to custom framework.
There is also the question about your business. If you work for a business that does not look at software as it's core compentency such as a bank or hospital then you need to way that into how much of a frame work you want to build.
Bottom line a framework of some type will always be useful.
Using a framework may add some overhead to your web-app development due to the learning curve. However, depending on the framework you choose, you may be able to realize the following benefits:
Scalability
Maintainability
Clear Division of your model (e.g. MVC)
Out-of-the-box components(SessionManagement,authentication,etc )
Third party plugins
Modularity inherited from the framework
Many others.
Besides, the learning curve is a variable factor. Some frameworks may be easier to learn than others depending on your skills.
It definitely makes sense to look into using a framework even if for no other reason than it'll give you a new area of knowledge. If you've got the time to invest in learning a framework then you may well find in time that it's easier and faster to use that for new projects no matter how simple they are.
Also, if I could vote for SamBeran's post then I would. Getting started with a framework will involve several moments of "oh great! I don't need to write all that boilerplate again!".
I'd say use one - like you said, it's rather tedious to get stuff up and running for even a simple java web app. If the framework helps you get the job done faster and provides the services you need, I'd say it makes sense.