Google App Engine uses java.util.logging.Logger (JUL) for all logging. Thus to log anything (and then subsequently retrieve it via the LogService), you just log like you normally do with JUL:
private Logger logger = Logger.getLogger(this.class.getName());
// ...
public void doWhatever() {
logger.info("This will be logged.");
}
But if you read over the GAE tutorials/guides for their various service APIs (Memcache, Mail, Datastore, etc.), they all reiterate that you should always code for the possibility that one of their services are down. GAE even provides a CapabilitiesService that you can check before calling any service method to see if that service is currently enabled or not.
So I ask: is there ever a chance that JUL logging operation will ever fail:
logger.info("Can I ever fail and not get logged?");
If not, why? And if so, what can I do to "failover" in the case that JUL has choked? Thanks in advance.
logger.info("Can I ever fail and not get logged?");
Of course it can fail.
Configuration behind this innocently looking line may:
Write a message to console ( console writes can be not initialized yet, or already shutdown )
Append a message to file ( can fail for many file I/O related reasons )
Send an email ( can fail for many socket I/O related reasons )
Write to DB ( can fail for many DB related reasons )
I've ran into this same problem, and yes the logging service can fail without errors. The best you're going to get (until GAE improves the logging service API), is to cron a job to wake up, say, every minute, and perform a logger.info(...).
Then run a LoggingService#fetchLogs(...), filtered to only retrieve the AppLogLine containing the most recent logger call, and check to make sure you can retrieve it. If you can't, then the logger.info(...) failed, and you can have your app react however you like.
I always expose a secure servlet on my GAE apps that pings the Capabilities Service and asks for a status check on each service. If the service is disabled or down for maintenance, I have an external monitor (that checks this URL every 5 mins) send me a text message. You can tie this "log checking" cron job into that kind of a service check.
One option, if you're consistently crashing and trying to figure something out, is to send async HTTP calls (as a log mechanism) to another server.
Also, because this gave me a smile during several weeks of crashing hell: https://groups.google.com/d/msg/google-appengine/js5CeRWLQZ0/KrW2CpJ4JuYJ
In most systems the Uptime is 100% minus the summation of the downtime of
all other systems. The exception to this rule is logging. When Logging
fails to record the downtime, Uptime goes up. As a result Google has been
working hard to build a logging system that goes down just ahead of all
other systems, and comes up shortly after.
Related
I have a Spring Boot app (jar file) that is running on Windows server and is used to sync data between some tables in a database and other parts of infrastructure (consumer apps via ActiveMQ).
It is crucial to have it running 24/7 without any downtime (or with very little).
I am currently trying to find the best way to do this as our current solution is to run multiple instances of the same app and define one to be active and ping it continuously (via an entry in database where it writes every 15 seconds), while other instances are just running and do nothing (inactive state, cause lock is taken). If an active instance has stopped to update lock entry (freeze or crashed) in database one of the available instances will take its place and start to handle data.
I have a feeling, that it is not so flexible solution, especially when I need to prepare different part of my code to check lock entry and sync all those instances. It adds complexity to the code and I want to avoid it.
Is there any better solution? Plugins, implementation pattern or tools?
PS:
I read about health endpoints that are available in a SpringBootApplication and think that it can help me somehow (ping\check them from some other Watchdog software\tool, maybe?), but don't know how.
In case of a crash you still have a delay of 15 seconds while a request can fail
I would go with a zuul router from netflix (open source)
It will balance the load between instances and will retry your request on another instance if the first call has failed
I'm pretty sure it's already done but use windows services to restart instance in case of hard crash
Maybe I'm overthinking this but I'd like some advice. Customers can place an order inside my GWT application and on a secondary computer I want to monitor those submittals inside th eGWT application and flash an alarm every time an order is submitted, provided the user has OK'd this. I cant figure out the best way to do this. Orders are submitted to a mysql database if that makes any difference. Does anyone have a suggestion on what to do or try?
There are two options: 1) polling or 2) pushing which would allow your server (in the servlet handling the GWT request) to notify you (after the order is successfully placed).
In 1) polling, the client (meaning the browser you are using to monitor the app) will periodically call the server to see if there is data waiting. It may be more resource intensive as many calls are made for infrequent data. It may also be slower due to the delay between calls. If only your monitoring client is calling though it wouldn't be so resource intensive.
In 2) pushing, the client will make a request and the request will be held open until there is data. It is less resource intensive and can be faster. Once data is returned, the client sends another request (this is long polling). Alternatively, streaming is an option where the server doesn't sent a complete request and just keeps sending data. This streaming option requires a specific client-/browser-specific implementation though. If it's just you monitoring though, you should know the client and could set it up specifically for that.
See the demo project in GWT Event Service
Here is the documentation (user manual) for it.
Also see GWT Server Push FAQ
There are other ways of doing it other than GWT Event Service of course. Just google "GWT server push" and you'll find comet, DWR, etc., and if you are using Google's App Engine the Channel API
Application - Struts 1.2, Tomcat 6
Action class calls a service which through DAO executes a query and returns results.
The query is not long running and gives results in seconds when run directly over the database (through SQL client say SQL Developer), but, when the user browses through application front end and same query is run in background through the application, the system hangs and the response either times out or takes a lot of time.
Issue is specific to one particular screen implying that app server to db server connectivity is ok.
Is there a way to enable debug logging of Tomcat/ Struts without any code change, to identify one out of the two scenarios below or any other scenarios possible?
The query is taking the time.
The response is not being sent back to the browser.
P.S. - Debugging or code change to add logging is not an immediate option.
Something to look at is a "Java Profiler". The one that I've used and have liked is YourKit.
In a recent job interview I was asked a J2EE debugging question. The question was As Follows:
"You are not getting same data as expected from your server how do you debug it?"
What or how should I answered this question that would make the interviewer happy??
Please suggest....
On the top of my head, usually you would
check the request and compare it with API - is the request being done correctly
check the logs for any problems on the server
confirm that the version of server application matches the one expected
check the database data status
if else fails, try to reproduce the problem locally or in a lower environment or step through the server app execution path with a debugger. Increasing the log level or hooking up to debug interface might be relevant as well.
When you have to debug code to server there is a common way on how to debug the code. You see server logs. Now, if you cannot find any errors, you have to see what the API returns at every single step. If you have not logs for every step, put log.debug("Some text that means something"); and rerun. If something is unusual, then you have to check the specific step.
The question is too general and opens a way for you to literally bombard the interviewer with questions and that's probably what he expects you to do.
Usually when sh. hits the fan I want to know: does the user receive any data at all? If not, app log, server log and db log is where I look. Some apps we use got app log located on the disk where the app is running, some are using db based logs and some are using the default log which can be accessed through admin console on the server (Glassfish for example).
On the other hand if the user received incorrect data I start tracking through the app how the data is "made" which usually means going through several db queries and such where I'm trying to determine what's going on. After that I compare the result I expect with the result user received and according to the difference I decide what went wrong.
But hey this question is too general in this environment so you either let the interviewer specify the problem or create your own scenario for him.
We are in the process of creating a training mode for our ColdFusion (9) sites.
The system will allow our users, after logging in, to switch from production mode to training mode by clicking on a link.
When they switch, the data-sources will be switched allowing the data to be safely modified.
We are also going to implement a test SMTP server, using the SubEthaSMTP Java project, in order to capture the emails that are sent from the training mode and display them to the user in a web page.
We can launch the SMTP server as a stand alone process or service without much trouble.
The nicer solution would be to launch server as part of the ColdFuson runtime at the point that the user switches to training mode.
We would create a true Java thread that would persist on a Server level scope for the length of any training sessions and then some arbitrary time out period. If the server times out and a new training session is initiated we would initiate a new SMTP server.
My essential question is, therefore, is it a bad idea to run an ongoing thread in the ColdFusion runtime this way?
I can't see a problem with doing this, although you ought to test to see what resources SubEthaSMTP uses and make sure it's not going to cause you issues. It looks to have minimal dependencies (essentially just SLF4J, which ColdFusion 9 & 10 already provide)
From the example page it looks to be pretty easy to set up and drop into a long-running scope. I think you're right to pick the server scope, as you may have problems using application or anything more volatile, as there'll be a situation where application scope would timeout and be reset, but you'd loose all references to the Mail Server instance.
Please update the post with your findings, as I'd be interested in seeing what you find.