At every moment in the world things change, said some guy.
We're sitting on a big fat REST API server which is becoming more and more of a burden to maintain because good Java developers are hard to find, expensive, development takes much more time, so we're slowly switching various application parts to Rails, which is... well, nevermind.
There are way too many APIs (and some are poorly documented) to just clone functionality by looking at the docs, so my idea is to find a way to log everything that's passing through this service for a while, in a machine-readable format we can later index, process and report, so we can gradually rewrite the code.
Is there a "clean" not so "hacky" way of doing this with Tomcat? Or should I go down the wireshark/reverse proxy way?
TIA!
You can create a payload logging filter and configure it in your web.xml. You can also use an output filter for the response. This and this should be helpful working examples.
Related
I am used to adding logging to standalone Java applications and writing the logs to the files using log4j and sl4j. I am moving some applications to a Java web start format and I am not clear on what is the best way to perform the logging to monitor the behaviour of the application. I have thought of two options
Write the log to the local machine and provide an option to send the information to the central server under some condition (time, error etc..)
Send the output of the log to the server directly
What is best practice?
I've seen 1. implemented by many programs.
But 2. seems bandwidth intensive, intrusive, and overkill.
Agreed, 2 seems like it's not such a good option. An error with webservices wouldn't be logged in that case. I was wondering if there was any other option but I can't think of any.
I was thinking of entirely local sources of problems connecting to the server, but good point.
What is best practice?
Stick with the majority and use method 1. Unless you have a marvelous inspiration about how the entire logging/reporting system can be improved, I'd go with "tried and tested". It is likely to be easiest, best supported by existing frameworks, and should your code falter, has the greatest number of people who have 'been there, done that' to potentially help.
Our development team hosts many different applications both .Net and Java based. Currently, we handle our error logging with Log4J and use emails to alert the development team when problems arise. Currently, we get thousands of alerts a day and it's becoming a little tedious to maintain.
We've been discussing creating a central dashboard for all our apps. The ideal tool would track errors, warnings, info etc. over the life of an application (it doesn't necessarily need to be db driven). The idea is that the data can be viewed on a dashboard, drillable to specific errors with the capability of alerting via emal when triggers and or thresholds are met.
Elmah is good for .Net but we need a tool that could also work for Java EE? What is the best way to go about this? Should we:
Just use Elmah for the .Net apps and find something similar for Java and build our own dashboard to create a united look & feel?
OR
Is there a tool that already exists that we can leverage to do this cross platform?
I've tried looking in Sourceforge but it's difficult to describe what I'm looking for.
I don't think you have a logging problem, I think that you have an integration problem, no matter if it is logging, or any other area your root issue is the same... How do I make my completely different components talk to each other?
There is a lot of approaches, but probably the easiest to implement for different technologies is Web services or REST... You will probably need to have a central logger that you need to implement independently, and then build a Web service/REST interface to which you are going to have to connect to...
Maybe a different line of investigation for you is to see if there is a logging product out in the market that takes web service calls... If that's the case, you only need to change your components to make a service call every time.
Something else that you need to consider is that your remote logging should never superseed your local logging, that's it do both, the reason is very simple, remote calls can fail, so code as if they will fail.
We have been using http://www.exceptional.io/ for error tracking for some time now: it's cheap and extremely simple.
To report errors you just post a json document to its endpoint.
I have a project whose core domain is dramatically changing. It's possible to use 50% of the core functionality from this site and just add the 50% new functionality, but I am starting to consider that maybe it might be faster to simply redo the product in Rails. Development speed is very important.
There are some things I really like about java - the performance and scalability are very good. I am not a crappy Java developer, so my apps tend to run very well - better than the Rails sites I've seen. I've always accepted the idea that people probably just throw a little more money at the problem when it comes to using Rails, which probably works itself out in the end because of the insane productivity benefits.
I am actually quite agile with Java. I know it will still take me longer to add a basic entity to the system, but I am quick at it and I don't mind it that much. At least it's easy and straight-forward to do.
What I do mind is:
having to start/stop the server just to fix a route, lazy load exception, controller is going to wrong view, etc.
putting up with the fact that unit/integration tests sometimes have different results than the production environment (because annotations on controllers can't be tested, or lazy-loading exceptions occur during asynchronous service calls, or things like that). Knowing if your Jackson is marshaling your data properly is another Tomcat-only thing because it's handled by Spring. There are lots of things that go wrong after you have tested all that you can, and this frankly annoys the crap out of me.
putting up with the occasional maven/classloader problem that doesn't rear its ugly head until you deploy into tomcat. It gives the false impression that everything is "a-okay" when you are in your IDE.
having to put more effort to do database migrations than the ruby people ever have to.
putting up with framework bugs in Spring that block (it's happened about 5 times on this project since 2009) or Hibernate. I also don't like upgrading Spring Security and having them constantly change the configuration, apis and tag libraries over and over again. This is annoying.
wasting so much time uploading 58 MB war files to the server! These take me 12 minutes to upload whenever I need to deploy changes. If I forgot to do 'mvn clean' before I upload, Spring might complain that 2 beans exist with the same name because I moved one to a new package... and then I have to re-upload the whole stupid war file again. Why isn't "clean" run by default whenever you do 'mvn package' for?!?! Sometimes these frameworks and tools use the stupidest default settings. This is just so common in the Java world.
Having to spend hour(s) to figure out where a framework wants to plug-in your own custom implementation for something. This is very annoying. You can spend 2 hours sifting through Google and crappy documentation trying to figure out how to override Spring Security's authentication mechanism for example... and then spend only 5 minutes writing the actual implementation. Of course, they wrote paragraphs upon paragraphs explaining the architecture and how awesome it is, but nobody cares. For something so common, why not just give example source code and be done with it?
Waiting 10-15 seconds for Spring to start up whenever you want to run your integration tests. This is a drag.
There are a few things I like about Java though. Role-based access is very easy to do with Spring Security. Authentication is never that big of a gain, but I like the implementation inside of Spring.
I also like Spring's form-backing objects and #ModelAttribute. These are huge wins when it comes to controllers, and I don't know if Rails can do these things. I honestly never liked passing request parameters around in every action - Spring MVC is actually a lot easier to use when it comes to this common bloat.
Being able to cache really massive structures in memory and have them stay in memory when you start the application is also highly desirable, especially for this application actually. I have an in-memory thesaurus and grammar checker that needs to get called hundreds of times per request, so in memory is pretty much the fastest option for me.
Even still, I think I could rebuild what I have in 2-3 weeks, and then add all of the new features in a few weeks using rails.
On the bright side, all of the really well-designed css, html and javascript could be ported over with very little problems.
I'd appreciate some advice on the subject before I continue.
PS: I could also go to Spring-ROO... but that would also be a considerable rework. I was never using JPA - I was using Hibernate directly. I am also not using JSP's - I am using Freemarker.
It takes more time to get good at Ruby, and Rails. I worked as an independent contractor as Spring and Hibernate expert myself, but I felt strangled by java and it's web frameworks so I decided to learn Ruby on Rails.
I would advice you to learn Ruby, from what I read you would probably master it, although get pretty frustrated with the very different way the use the ORM. I had issues with it, used to working on aggregate roots in Hibernate to the ActiveRecord one class one table kind of pattern. But hey, you could easily try out MongoDB to have some real fun.
Ruby is
less code
it's fast and scalable (slower than java on the specific tasks, but you get rid of stacks of layers.)
the problems are more often; which gem should I use. Luxorious!
a unique, big, sharing and caring open source community
nice frameworks, as Rails and Sinatra
powerful.
fun!
Would I advice you to do the project you describe in Ruby.
NO.
Not if speed of development matters. You will be slower, trust me. There's a lot to learn, it's conventions are not familiar to a java programmer and when you get stuck, lots of hours fly by.
The best option would be to hire a senior ruby developer to pair up with you and teach you. Be a good apprentice and you'll learn fast. Faster than me, I had to learn most by myself, which is really inefficient.
Good luck!
Check out Playframework. Its fun to develop, and you can use your Java experience to develop features way quicker (given than you have 2 weeks) than any other Java-based frameworks out there.
You do not have to start/stop a server. You fix the code in Eclipse and hit refresh on the browser. No dealing with WAR files till you have to actually deploy in production. Do everything from within Eclipse. Easily perform TDD process if thats what you want as you develop code. From an architecture standpoint, it is a fully stateless, RESTful framework from the get-go. Fully JPA compliant (even for NoSQL like Mongo), so you will not have to write complex JDBC code. On the front-end, it has a full featured templating engine, using Groovy as a templating language.
I can go on and on, but I'd recommend going through the site and take a look.
You should take a look at Grails.
You can continue to leverage a lot of your Java code but use a scripting language (Groovy) and many of the paradigms of Rails. E.g. lots of time saved by using convention rather than configuration.
Grails is used by some pretty big web sites E.g. BSkyB the UK satellite broadcaster.
It doesn't really help with some of the startup speed aspects. If you really prize development speed that much - get a faster machine or buy an SSD and fit in your machine. If you work for a big company - sell it to your manager as the cheaper option (E.g. buy a $2000 machine rather than spend 3 weeks rewriting something to save 10 minutes a day).
Java will scale better in the long run than Rails. The Hotspot technology in the JVM is one of the wonders of modern technology.
Also worth checking out is Tapestry5. It allows you to make code changes on the fly (no server restart required) and is easily the fastest & leanest framework to develop with in Java I've used.
I would still give Spring Roo a shot, it will the same rework as with Ruby on Rails or Grails or even less, but you will still stay with something that you are familiar with, which is often the biggest consideration
It has the scaffolding concepts of Ruby on Rails and Grails, but it gives you zero lock in code, just simple, well written (massive use of AOP is matter of taste though) of Spring + Hibernate / JPA (I think you can use Freemarker for the views, Roo has a miriad of plugins, but I'm not 100% sure)
I'm trying to figure out wether I should use SOAP or REST for a specific project. I will explain what the project is or needs to do, and I hope some of you will give me useful information about why I should use one over the other.
The project I need to work on is totally web service based. Every frond end implementation needs to access webservices to get all the information it needs. But Beyond that, we also have an Indesign plugin that talks to these webservices to upload documents and check validations on the document etc.
So to make it short, I need to access the web services via front end web gui's and via an indesign plugin. The latter makes me wonder wether I should use REST or SOAP.
Thanks alot for reading this and posting you feedback.
I found this really great article on the advantages and disadvantages of SOAP vs REST: http://geeknizer.com/rest-vs-soap-using-http-choosing-the-right-webservice-protocol/
And my own humble opionion:
I don't like SOAP, I never did. It's been great at the time, but it just never felt "right". If you've ever tried to read one of these huge overloaded XML-documents and compared it to the REST output (which is JSON most of the time) you know what I mean. IMHO just feels "better".
One problem is that you have to document your REST API better, but that shouldn't be that big problem as long as it is not open to everyone.
Especially if you handle all your data with Webservices, I think you should go with the lighter and less bandwith consuming option: REST.
Also REST is the new and hip technology that everyone uses at the moment... ;)
About your indesign plugin: You should be able to include any library you want, so that shouldn't be a problem.
I'm trying to write a program in Scala, that will accept SOAP-requests, get the response from the real server (or read it from the local disc) and return the data to the original client.
I'm new to the java/scala ecosystem, so I have no clue, what libraries to choose.
I heard Scala's XML-handling is quite nice, so I don't know, whether I should use some enterprisey soap-library/framework like jax-ws, jboss-ws, axis, cxf, xmlbeans, etc.
Basically, I just need
a library, that accepts the requests (currently, I'm looking at jetty, but I'd prefer something that natively supports actors. scala-http seems to cover that, but isn't production-ready or maintained, for that matter)
some library to request the data from the other server (something like curl, libwww-perl for java/scala)
a build system (ant? sbt?)
an IDE (I'm used to eclipse, but IntelliJ's scala support is supposed to be better)
a tool to test it (currently, I'm using SoapUI)
SOAP is truly a hideous specification, with lots of potential for unusual fringe behaviour. While it's true that XML support in Scala would help you write such a library from scratch, it would still be a major effort (depending on how much of the spec you need).
Likewise, Jetty has years of development behind it; dealing with performance demands and other unexpected behaviour that you probably haven't considered... Even Scala's best-known web framework, Lift, runs atop a Java web server for these very reasons. It still works very happily with actors.
So, at this moment in time, you're almost certainly better off using a tried and tested solution with a Java web server and an off-the-shelf Java SOAP Library. The effort to add a thin Scala wrapper around these will be far less than the effort to build these things from scratch.
For the build system, sbt is the most powerful tool presently available for Scala, but you may need to fall back to Maven if this is needed for code generation by your chosen SOAP library.
Finally, for the choice of editor. If you're happy using Emacs, then the Ensime plugin is just amazing. If a more conventional Java IDE is to your liking, then IntelliJ currently seems to be the most stable option, though be aware that this could change very quickly.
Just a partial answer.
Look at:
HttpClient for making HTTP requests
Build system, if you have no prior experience with ant I would recommend sbt
For the IDE, I had good success with IntelliJ a few months ago. I believe Eclipse has improved but I don't know how much.
SoapUI would still work perfectly
Proxy servers are a classic use case for asynchronous/non-blocking IO. If I were embarking on this project, I'd start by taking a long look at Netty's HTTP support and build a simple reverse proxy (i.e. forwards front-end requests to backend servers, and backend responses to frontend clients) first before moving on to protocol translation.
When it comes time to work on protocol translation, you'll incur the wrath of XML parsers. Unfortunately, to my knowledge there isn't a good, high-performance, low-footprint parser that natively handles asynchronous IO; quite a few of them probably exist but are embedded in commercial products. See this thread for more information.
However, you can "cheat", at the cost of additional thread use, by using a SAX parser, which typically relies on blocking IO, to consume the output of a "push-me-pull-you" pipeline. Since the HTTP parts of your server are non-blocking, you can probably afford to use a few dozen threads just for shoveling bytes around.
As it happens, I've been there, and done that. :)