I would like to develop a Java EE web application that requires Prolog, via JPL, for certain search related tasks.
The web application will be deployed in the JBoss application server.
The Prolog engine can be either YAP or SWI (afaik the only Prolog engines compatible with JPL at the moment).
The Prolog queries depend on information stored in a (potentially large) database.
If someone has tried this or something similar, could you please give me feedback about the following questions?:
What is the best way to manage concurrent http sessions that need access to the Prolog engine?. Is it possible -desirable?- to assign to each separate session its own Prolog engine ?. If this solution works, is it possible to implement something similar to a 'Prolog engine pooling' to quickly assign prolog engines to new sessions ? . Or the best solution is to have a single Prolog engine that will manage all the query requests synchronously ? (and slowly).
How could be managed the interaction of Prolog with the database ?. If the data is changing often in the database and Prolog needs this data to solve its queries, what is the best strategy to keep the facts in the Prolog engine synchronized with the data in the database ?. The navy option of starting from scratch at each new session (e.g., reloading all the data from the database as Prolog facts) does not seem to be a good idea if the database grows large.
Any other expected issues/difficulties related to the java-prolog-database interaction during the implementation ?
Thanks in advance!
What is the best way to manage concurrent http sessions that need access to the Prolog engine?.
If I look at the source of JPL, it looks like it uses an engine pool. The query data type implements the enumerator pattern plus a close() operation. I guess an engine is automatically assigned to a query as long as it is active.
So each http request can independently access the Prolog system via new query objects. If you don't want to close your query object during a http request, I guess you can also attach it to a http session. And reuse it an another request.
How could be managed the interaction of Prolog with the database ?
This depends on the usage pattern of the data in the database and the available access paths. It could be that you can quickly access very large databases during a request and refetch the data during each request. For example if the needed matching data set is small and if the database has good indexes, so that the matching data can be quickly accessed.
Otherwise you would need to implement some intelligent caching. I am currently working at a solution where I use a kind of a check-in/check-out pattern. But this is not suitable for a web server, where you have multiple users. I am using this pattern for a standalone solution where there is one user and one checked out data junk in memory. For a web server with varying multiple users the junks could overflow the webserver memory.
So caching only works if you can limit and throttle the junks or if you have a very large webserver memory. Maybe you can find such an invariant for your application. Otherwise the conclusion could be that you cannot go Java EE independent of whether you use Prolog or not.
Related
I have 3 different applications
ASP.NET web application
Java Desktop application
Android Studio mobile application
These 3 applications have the same database and and they need to connect from any part of the world with an internet connection. They share almost all the information, so, if you move something in one application it has to update the information in the other 2 applications.
I have the database on a physical server and I want to know how best to make this connection.
I have searched but I couldn't find if I have to connect directly to the server with some SQL Server, using Web Service, or something like that.
I hope someone could help.
Thank you.
I believe the best way is to first create a Web API layer (REST/SOAP) that will be used to perform all the relative operations in the centralized DB. Once that is setup, any of your applications written in any language can use the exposed web API methods to manipulate the data of the same DB.
If you are looking at a global solution - will you have multiple copies of the applications in different parts of the world as well?
In this scenario you should be looking at a cloud-hosted database with some form of geo-replication so that you can keep latency to a minimum.
There are no restrictions on the number of applications that can connect to a specific database - you do not have to create a different database for each and you may be able to reuse Stored Procedures between applications if they perform the same task.
I would however look at the concept of schemas - any database objects that are specific to one app should be separated from other - so put them in a schema for "App1". Shared objects can be in a shared schema.
I read somewhere use of webservcies in apps. After a lot of research I am able to create one Webservice which will accept Json and JsonP both format as request and response accordingly. I developed the webservcies using Java, Apache Axis2, Hibernate and MySQL as database. there are few problems and I dont know how to solve ?
Insert or delete option, sometimes if at a time more than two users call that service that is insert or delete any row the queries go in sleep mode and next time someone tries to fetch that service he couldnt. Accroding to server log it says error SQL Lockout State. If I checks Processlist in MYSQL it is showing that query in Sleep, I have to kill to resume.
The performance of webservice doesnt seems to be upto mark, it takes time some more time as what i experienced it shouldn't. In simple words how to obtain better performance by the services
How to implement security feature such that if a user logins he/she can be provided an id and validation of that id so that unauthorized access can be prevented
Or just guide me what should be the most appropriate and optmized Webservice methodology that can be used using Java
Answer to this question is not specific to Android. Below are my investigations which might be useful for you.
For the point about MySQL connections going to sleep mode, you can do the following.
Debug the datasource used by Hibernate, try to increase the pool size & check for any issues in it.
Define a timeout period for connections. JBoss has several configurations related to this like blocking-timeout-millis, idle-timeout-minutes etc.
Declare a mechanism to validate periodically the connection resources in the pool for activeness. You can explore OracleStaleConnectionChecker for options.
Configure miniumn connections in the pool. This is important because when all the stale connections are discarded, empty pool needs to be pre-filled & ready with active connections.
Coming to performance of Insert/Delete operations & SQL Lockout State, please try to re-order the sequence of the queries which you are firing to DB at every request. This may not be a deadlock situation but sequencing DB queries correctly will definitely lead to less lockout time and better performance.
This answer may be of use for you. Hibernate: Deadlock found when trying to obtain lock
Web-services which you have developed may require some performance optimization to make them upto the mark. Below are first few steps you can take to bring the performance up.
Avoid nested loops. Every extra parameter in the iterated lust increase the order of the lopp exponentially.
Remove early initialization of objects. This may lead to long unwanted GC cycles.
Apart from above optimizations, there are several frameworks & tools at your service to evaluate the code quality & its performance. PMD, FindBugs, JMeter, Java profiler are few of them to name.
Shishir
You are going to have to profile your server and see where the time is spent. I really like YourKit for doing thread profile. visualvm which comes with the JDK can help also.
There are all sorts of reasons your web service can be slow:
Latency from client to server
Handling the HTTP request on the server
Handling the HTTP response on the client
Making the database call (sounds like you already have some kind of locking / blocking going on there)
You are going to have to get markers to tell you how long it took to go from A to B to C to D back to C back to B back to A kind of thing. We would be speculating heavily from here on what is exactly going on in your program, but we can give you the ideas / tools to figure it out.
If you use YourKit, connect it to your server process. Have nothing else connecting to your server (for instance your client is not sending requests). Try it with your client requesting, you should see your accepting threads receive the HTTP request and then delegate to either your processing thread or do the processing itself. You can use YourKit to see how much time is spent in different functions during that call time.
Try it with your client making the call.
Try it using a simple HTTP request tool like wget or maybe your IDE has a webservice test tool (for instance intellij does), or you can download a simple HTTP test tool.
By testing it in a simple tool that just outputs the response, you can eliminate any client processing issues. You can also achieve a similar test in Chrome or Firefox and use the developer tools to see time to fulfill request.
In my experience, the framework for handling the requests and delegating can introduce some performance issues. I ripped Grails out of a production environment because of its performance issues (before any Grails / Groovy flames come my way, we were operating at a much higher rate than typical web applications, and I am sure Grails has made some headway in the last couple years... alas, it was not for my need at that time)
BTW, I doubt you are operating a load where you will be critiquing the web service framework you chose to use. I have been happy with Spring MVC and DropWizard (Jersey JAX-RS), and Grails is easy to use too.
You should make a simple static content response in your webservice and see how quickly that returns vs a request that makes a database call.
Also, what kind of table are you using in MySQL? InnoDB? MyISAM? They have different locking schemes. That could be causing your MySQL issue.
The key to all of it, break the problem up into parts, and measure each and eliminate parts one by one till you go, everytime I do X it is slower (like everytime I make a database call its slower)
In Java the the way you will be able to find more support online via documentation/forums is to develop the web service as a REST web service using Spring MVC.
You can base yourself on this resource and take it from there:
Spring MVC REST Hello World Web Service
Using Spring you can create a RestFul webservice easily and spring does all the ground work you needed. As others had mentioned you can consume the webservice in any type of client - including Android.
A detailed guide available here:
https://spring.io/guides/gs/rest-service/
Here are my suggestions:
Make APIs only read or write database. If an API combines reading and writing, it is possible to cause deadlock;
Use a light-weight HTTP server. Powerful HTTP server is possibly consuming more.
Make use of thread. Have more threads could be helpful when you are facing a ton of users.
Make more things static. You could avoid unnecessary queries.
I think mhoglan's answer is detailed enough.
we have to develop a local server which will load itself with the real-time data of a industry (particularly time stamped data points like the temperature of a boiler,pressure values etc) which are stored in industrial server and we want to fetch them and populate our server with it, the data is not streamed at server end so how to fetch it continuously and populate the server...
we would like to store only past 2-3 days of history data as time advances, any recommendations about the server and the back end process to be used to fetch data are welcome, we don't have any idea were to start..
please help...
As others have stated,
You need to provide more information on how do you intend to populate your server.
What API do you have for the "real time server"?
I worked on a management system for solar engery devices
(i.e - devices that produce electricity from solar energy - they are called photo-volatic cells if I remember correctly).
In my case these devices had an FTP access , which provided me files with time-based information.
I constructed a java server that used the following technologies:
A. Apache tomcat web container - This web container allowed me on one hand to hold java logic, and on the other hand to expose HTTP-based interface to the customer.
The Java logic was located in a Servlet- which exposes methods to handle HTTP requests (and allows writing returned data using response objects).
B. The servlet has an init method, I used it to perform some initialization, such as starting a quartz periodic task to probe the ftp servers of the devices.
C. I used a database (postgresql database, which is an open source database) to store configuration for the application, and also to store results.
D. I used another periodic task to archive old data in an archiving table, so the main data table will hold relatively new data.
I ran the archiving task once in a few days, and it simple checked for record that were "too" old, inserted them to the archiving table, and deleted them from the main data table. In order to peform this efficiently I have have decided to use a function that I coded on the database.
E. In order to access the database from the application, I used the Hibernate object relational mapping technology.
This technology allowed me to define mappings between tables and their relations to java objects, and gave me generated create,read (by-id), delete and updated SQL statements.
Using the HQL query language, I wrote some more complex queries.
F. For presentation/client side - I used plain JSP.
You may choose other alternatives such as :
GWT, Apache Wicket, JSF
You may consider using some MVC framework to have some seperation between the logic and the presentation. Such frameworks can be:
Spring-MVC , Struts, and many others.
To conclude, you must understand that Java offers you a variety of technologies, you must define requirements well, and then start investigating which technology can meet your needs.
In a web application I am developing, I am using a third party Java library (JPL) that uses JNI to connect to an external application: a Prolog engine.
For the nature of my problem, I need to have one Prolog engine per http session. But as far as I know the library I am using only let me work with one Prolog engine per java VM.
In order to solve this issue I came up with the idea of trying to configure JBoss to launch a new process (instead of just a new thread) per each http session, a bit like CGI where normally one process is started per http request.
In this way, certain servlets could use the required JNI based library without having to worry about synchronization issues in its side, since as I expect (and hope not be wrong about that), each of them will have an independent Prolog engine with different state (e.g., different asserted Prolog facts).
Is possible to configure JBoss (or other servlet container) in this way? Any feedback or pointer will be highly appreciated!.
To my knowledge this is not possible. However looking at the documentation http://www.swi-prolog.org/packages/jpl/java_api/high-level_interface.html#Multi-Threaded%20Queries the only problem seems to be that you can have only one open query per VM.
I am designing an enterprise security server for our company - we own many different applications, most written in java and a few written in PHP. I could provide a remote API that would give each application access to the server. I could also create 'agents' that each application could include that would do all the work for them, but allow my server control over their sessions and thus their authentications/authorizations. Issue is I would probably be better to write the agent in java because 80% or more of our apps are in java.
If I wrote the agent in java does anyone know if there was a way this program could access the php session? If not does anyone have a suggestion regarding a better way to go about doing this?
The session data is stored as a (php) serialized array in a temporary folder. The locations for these are set in the php.ini file.
But you can change both the format of the data and the place it is stored (e.g. to a database or shared memory or somewhere else) by writing your own handler.
A quick google suggests that several people have written [de]serializers in Java for PHP data. e.g. http://hurring.com/scott/code/java/serialize/
If you have problems with the built-in PHP serialize function - have a google for WDDX (which IIRC comes as standard) and serializes data into XML.
You might want to think about how you keep the session data appearing to be active to PHP if you want the agent to continue independently of the web session.
C.
You can hook into PHP's session handling using session_set_save_handler() (an example for a simple but complete custom handler is included in the manual). You should be able to synchronize PHP's session management with a central Java server that way.
Your PHP application would receive a session ID through a cookie ($_COOKIE["SESSION_ID"] or whatever).
Your custom session_save_handler would, instead of maintaining a session store of its own, pass that session ID to your central Java-based security server, and get all the session data in return. Writing into a session from PHP would be routed the same way.
You could of course also go the other way, and poll PHP's internal session data from the outside, but wouldn't quite understand what exactly for. If that is the case, can you go into more detail there?