I'm working with core java and IBM Websphere MQ 6.0. We have a standalone module say DBcomponent that hits the database and fetches a resultset based on the runtime query. The query is passed to the application via MQ messaging medium. We have a trigger configured for the queue which invokes the DBComponent whenever a message is available in the queue. The DBComponent consumes the message, constructs the query and returns the resultset to another queue. In this overall process we use log4j to log statements on a log file for auditing.
The connection is pooled to the database using Apache pool. I am trying to check whether the log messages are logged correctly using a sample program. The program places the input message to the queue and checks for the logs in the log file. Its expected for the trigger method invocation to complete before i try to check for the message in log file, but every time my program to check for log message gets executed first leading my check to failure.
Even if i introduce a Thread.sleep(time) doesn't solves the case. How can i make it to keep my method execution waiting until the trigger operation completes?
Any suggestion will be helpful.
I suggest you go and read up about the concurrency primitives that Java offers you. http://tutorials.jenkov.com/java-concurrency/index.html seems to cover the bases, the Thread Signalling chapter in particular.
I would recommend against relying on log4j (or any logging functionality) even in a simple test program.
Have your test run as you would expect it to, putting debugging/tracing statements in the log as you see fit (be liberal about it, log4j is very fast!) Then, when it's done, check the log yourself.
Writing log parsing will only complicate your goals.
Write your test, view the result, view the logs. If you want automated testing, consider setting up a functional test. You can set up tests free using Selenium. (http://seleniumhq.org/) There's no need to write your own functional testing/parsing stuff when there's easy to configure, easy to use, easy to customize frameworks out there! :-)
Related
I am trying to implement row level security so our application can enforce more stringent access control.
One of the technologies we are looking into is Oracle's Virtual Private Database, which allows row level security by basically augmenting all queries against specific tables with a where clause predicate. Since we are in a web environment, we need to set up a special context within Oracle, inside a single request's thread. We use connection pooling with a service account.
I started to look into Eclipse Link and Hibernate. Eclipse Link seems to have events that fit perfectly into this model.
This would involve us migrating from hibernate, which is not a problem, but we would then be bound to EL for these events.
Oracle seems to imply that they implement at the data source level in Web Logic product.
The context is set and cleared by the WebLogic data source code.
Question: Is it more appropriate to do this at the DataSource level with some series of events. What are the events or methods that I should pay the most attention too?
Added Question: How would I extend a connection pool to safely initialize an oracle context with some custom data? I am digging around in Apache, and it seems like extending BasicDataSource doesn't give me access to anything that would allow me to clean up the connection when Spring is done with it.
I need to set up a connection, and clean up a connection as the exit / enter the connection pool. I am hoping for an implementation that is so simple, no one can mess it up by breaking some delicate balance of products.
- Specifically we are currently using Apache Commons DBCP Basic Data Source
This would allow us to use various ways to connect to the database and still have our security enforced. But I don't see a great example or set of events to work with, and rolling my own security life cycle is never a good idea.
I eventually solved my problem by extending some of the Apache components.
First I extended org.apache.commons.pool.impl.GenericObjectPool and overrode both borrowObject() and returnObject(). I knew the type of the objects in the pool (java.sql.Connection) so I could safely cast and work with them.
Since for my case I was using Oracle VPD, I was able to set information in the Application context. I recommend you read about that in more depth. It is a little complicated and there are a lot of different options to hide or share data at various contexts level, and across RAC nodes. Start
In essence what I did was generate a nonce and use it to instantiate a session within oracle, and then set the access level of the user to a variable in that session, that the Oracle VPD policy would then read and use to do the row level filtering.
I instantiated and destroyed that information in my overridden borrowObject() and returnObject() The SQL I ran was something like this:
CallableStatement callStat =
conn.prepareCall("{call namespace.cust_ctx_pkg.set_session_id(" + Math.random() + ")}");
callStat.execute();
Note math.random() isn't a good nonce.
Next was to simply extend org.apache.commons.dbcp.BasicDataSource and set my object pool by overriding createConnectionPool(). Note that the way I did this disabled some functionality I did not need, so you may need to rewrite more or less than I did.
You can try any object level security mechanism for simplicity, like Spring Security ACL.
You will want to do this at the application layer. You will want a pre-commit hook and a post read hook.
The pre-commit hook is used to ensure that data from the client is being presented by a user authorized to modify that data. This prevents an unauthorized user from overwriting data that they shouldn't be able to access.
It's not intuitive, but the post read hook is used to keep the client from accessing data the user shouldn't be allowed to view. This happens post-view because this is being enforced at the application layer, not at the data layer. The application has no way to know if the caller is allowed to access the data until it's been retrieved from the data layer. In the post read hook you evaluate the credential on each row returned against the credential of the logged in user in order to determine whether or not access is allowed. If access is denied on any row then an exception would be raised and the data would not be returned to the client.
Application level security done in this way requires that you have a way to connect each row in a table to a permission/role required to access it and a way to evaluate a user's permissions on the server at runtime.
Hope that helps.
You will get better control by using one of the other Commons DBCP Datasources.
The Basic one is just that: basic :)
The ones in org.apache.commons.dbcp.datasources package gives you more fine-grained control.
In our project we use Activemq (jms template) - to publish many events from one webapp to another.
we use logging aspect (spring aop) as well - mainly we log errors and entering\quitting methods.
Now, sometimes we face racing conditions on the flow of the systems. i.e. an entity is being created on one web app, an event is fired to notify another webapp, but the handling of the other webapp requires a different event to be handled first, so if such scenario happens, the handling fails (for example, an id is missing ) and immediately retried (jms re-delivery), on the 2nd time of the retry its usually works (never more then 3 retries are required).
So basically, we have exceptions as part of our day to day flow, but:
our log files are huge and messy because of the exceptions thrown by such scenarios, any idea how can we not log the first few retries exceptions and only on a later exception we will log? Maybe another approach you can recommend?
Thanks.
You can use JMSXDeliveryCount property of Message to get the redelivery count. See http://activemq.apache.org/activemq-message-properties.html
I have a log4j with several loggers, appenders used in a multi-threaded application. In one scenario, I will try to connect to a remote service. If the connection fails, I will try again repeatedly.
I would like that only the first time log4j uses its original configuration. But for every other subsequent attempts, I want to use a less verbose configuration. This should not change the logging configuration of the other threads that might operate on the same objects. Note that I cannot know in advance which loggers are used inside the call to connect to the remote service.
So, is there a way to alter the logging globally for the duration of one call without changing the behavior of other concurrent threads?
Look at this part of the API:
http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/Logger.html
You can call the setLevel(Level lev) method and do some changes. However, I'm not 100% sure that you won't affect other threads as normally the Logger is based on a class.
I would think that you would have to get a Logger object for each user session, but I'm not 100% sure if it could cause heap issues.
Maybe controlling the output from your code (only send some messages to the log if it's the first attempt for instance).
Regards
I have used Spring Integration in my current successfully for some of the needs. Awesome..
There is some weird behavior observed on a heavy load where-in the same message seems to be processed more than once. I can confirm that because there are multiple rows in the database which is typically the last command on the chain that is configured over the channel.
Digging into the manual further, it looks seems like load-balancing is done automatically by spring. The manual says that the message is balanced between multiple message handlers.
Question is:
How many handlers are present on a channel by default? The spring XML that gets loaded does not seem to have that configuration. All i do is this (per the recommendation in the manual):
<int:channel id="SwPath.Channel"/>
<int:chain id="SwPath.chain" input-channel="SwPath.Channel">
</int:chain>
I can disable the fail-over but I am curious to know how many are present by default.
It's been a while since I worked on those load balancers, but I remember that the default number of threads in the thread pool was somewhere between 2 and 10.
It is possible that you have found a concurrency bug.
If you turn on TRACE logging the load balancer will give you a lot of information, but that could easily hide the problem.
If you would create a JIRA issue with a JUnit test case, I'm sure it would be much easier to figure out what happens exactly.
I am using Log4j logging framework to insert the log into oracle database.But the insert query in the log4j properties file is taking a lot of time to execute and making the application very slow.When I removed the logging statements from the java code, the application worked fine.At first, I thought that the insertion into DataBase is taking time , but writing the log on an external file also takes a lot of time.
Can anyone please suggest a solution?
Thank You,
Dhaval Maheshwari.
If you application is under development then log level should be debug and before logging you should check for isDebugEnabled() and then log your string.
but If your application is in production then log level should be info and you must log minimal information in log file.
Always use atleast two log level in your application one for debuggnig
mode(for development environment) and another for production mode and
production log should be minimal.
This is the way you can speed up you applicaiton.
and second thing if you want to persist your logs into database then
create a scheduler task whose responsibility would be reading logs
from flat file and persisting them into database and schedule this to
run only once in a day.
I suggest not to follow the technique u r following now.
First of all I am not sure why u r trying to log the output of log4j in DB.
Anyways if it is that necessary try something like this. Let the logfile write into a file as it is and later run a thread to dump this file from the disk when file is closed to the database as a batch process.
In this case your application will be separated from the latency of DB.
There are other solutions also using a JMS.
Where you can write it to a JMS queue and the consumer on the other hand can read the queue and write it a DB.
It depends on the kind of problem you are trying to solve though.
See of it helps
In Logging there are levels included in. For example in production only log application level exceptions and errors[ERROR level].
If it's tracking logs(Such as user actions) don't write them to files, directly add them to database. Hope this helps.