I am a student building a http proxy server. I want to cache those requests that are frequently accessed. May I get any idea about this? Especially in java.
To figure out what you need to implement, read and understand the HTTP specification. Focus particularly on the sections on how a proxy is supposed to behave.
You could possibly base part of the implementation on the Apache HttpClient library, but I have a feeling that the APIs will prove to be unsuitable for the proxy server use-case.
I'd also like to point out that a more practical way to implement an HTTP proxy server would be to simply deploy an existing server like Squid.
Related
I've been asked to look into adding an LDAP interface to an existing Java web application - that is, to make it possible for LDAP clients to connect to the application as if it's an LDAP server. I could write code to listen on a dedicated port and implement the LDAP protocol, and hook that into the existing database... but I'd have to understand the protocol first and then there are potential security issues if I write that from the ground up (not to mention the time it could take).
What I'm looking for is a library of existing code - something that handles the ports and protocols, and lets me focus on writing just the back end. There are plenty of client-side libraries out there, as you'd expect, but I've had no luck in finding something to help with server-side development. So the question is, does anyone here know of such a library that would help with this?
Yes you will most probably find many more client implementations than server, however LDAP is a request response protocol, so with a bit of playing around you should be able to use the same classes and their serialization capabilities. Instead of sending the request you would be receiving it, and responding with the response you would otherwise expect from the client.
You could look at the Apache Directory. https://directory.apache.org/api/
It has an embedded directory server project as part of it, which claims to be extensible and embeddable in your application. https://directory.apache.org/apacheds/
So maybe that could be the answer to your needs.
I'm working for a company and they require me to code an Android application that will handle sensitive data stored on a centralized database (MySQL), so I need to code a web service to communicate the Android app with the DB, and since in that connection will flow sensitive data I need to use a secure connection for that.
So, I decided to use json-rpc for the commucation, that's why I'm looking for two json-rpc implementations: the server-side in PHP (actually the hosting that they have is running PHP 4.3), and client side in Java (Compatible with Android) compatible between them and with SSL support.
I'm sure someone might make some suggestions, but this probably isn't the best venue to get help with library selection.
If, on the other hand, you ask concrete questions about certain libraries/implementations you might get intelligent answers. A common problem with that approch is that the asker doesn't know what questions to ask! (And that leads to re-inventions of wheels and commonly, poor decisions around security).
The JSON-RPC (1.0/1.1/2.0) group of protocols are extremely simple. There are hundreds of PHP implementations. I don't know much about evaluating Android libraries but this prior SO question gives a suggested client-side implementation for Android.
I'll primarily deal with the server-side part of your question by giving you concrete answers you can ask about a particular implementation you might be evaluating.
Considerations for the server-side PHP JSON-RPC implementation:
How will you handle transport authentication?
How will you handle RPC-method authorization?
How will you handle transport security?
Will you need to handle positional/named/both style method arguments
Will you need to handle/send notification messages?
Will you need to handle batch requests? Will you need to send batch responses?
How does the PHP implementation behave under a variety of exception/warning conditions?
How does the PHP implementation advise against or mitigate the PHP array set/list duality?
Will you need to implement SMD?
Will your web-service also need to perform other HTTP-level routing? (e.g: to static resources)
How will you handle transport authentication?
Assuming you use HTTP as the transport-layer protocol, how will you authenticate HTTP requests? Will you use password/HTTP-auth/certificate/SRP/other authentication? Will you continue to trust an open socket? Will you use cookies to extend trust after authentication?
These aren't really JSON-RPC related questions, but many server-side implementations make all sorts of assumptions about how you'll do authentication. Some just leave that entirely up to you. Your choice of library is likely be greatly informed by your choices here.
How will you handle RPC-method authorization?
You might decide that once a user is authenticated that they're allowed to call all exposed methods. If not, then you need a mechanism to allow individual users (or groups of them) to execute some methods but not others.
While these concerns live inside the JSON-RPC envelope, they're not directly addressed by the JSON-RPC protocol specifications.
If you do need to implement a method-"authorization" scheme, you might find that some server implementations make this difficult or impossible. Look for a library that exposes hooks or callbacks prior to routing JSON-RPC requests to individual PHP functions so that you can inspect the requests and combine them with authentication-related information to make decisions about how/whether to handle those requests. If the library leaves you to your own devices here, you'll likely have to duplicate lots of user/method checking code across your JSON-RPC exposed PHP functions.
How will you handle transport security?
Assuming you use HTTP as your transport-level protocol, the obvious answer to this is TLS. All the usual SSL/TLS security concerns apply here and many SE/SO questions already deal with these.
Since you control the client-side implementation, you should at least consider:
Choosing a strong cypher & disallowing poor cyphers
Dealing (or not dealing!) with re-negotiation in a sane way
Avoiding known-problematic implementations/features (e.g: OpenSSL 1.0.1-1.0.1f, heartbeat[CVE-2014-160]).
Will you need to handle positional/named/both style method arguments
Some JSON-RPC server-side implementations prefer that the params argument of the JSON-RPC envelope looks like params: [412, "something"] (positional), while others prefer that the params argument of the JSON-RPC envelope looks like params: { "type": "something", "num_events": 412 } (named). Still others can handle either style without issue. Your choice of client & server libraries should be informed by this issue of 'compatibility'.
Will you need to handle/send notification messages?
Notification messages are sent by the client or the server to the opposite party in order to communicate some updated/completed state over time. The sending party (ostensibly) shouldn't expect a response to a "notification" message.
Most JSON-RPC implementations use HTTP(S) for their transport protocol; as HTTP doesn't implement bi-directional asynchronous communications, you can't prooperly implement "notification" messages.
As your client side will be Android, you could use plain JSON text over TCP sockets as the transport protocol (rather than HTTP); as TCP does allow bi-directional asynchronous communications (once the socket is established), then you can properly implement "notification" messages.
Some libaries implement client-to-server notifications over HTTP transport by enqueing notification messages and then piggy-backing the notifications on request-style messages (possibily using "batch" requests -- see next question). Similarly some libraries implement server-to-client notifications over HTTP transport by piggy-backing the notifications on response-style messages (possibly using "batch" responses).
Still other libaries make use of HTTP by using WebSockets as their transport protocol. As this does allow bi-directional (essentially) asynchronous communications, then these libaries can properly implement "notification" messages.
If you don't need this you'll have significantly more choice in your selection of transport protocols (and therefore implementations).
Will you need to handle batch requests? Will you need to send batch
responses?
Sometimes it is desirable to send/handle groups of requests/responses/notifications in a single request/response (so called "batch" messages). The id argument of the JSON-RPC envelope is used to distinguish requests/responses in a batch.
If you don't need this you'll have significantly more choice in your selection of implementations ("batch messages" is the most commonly ommitted feature of JSON-RPC implementations).
How does the PHP implementation behave under a variety of exception / warning conditions?
PHP has many ways of propagating error conditions to the programmer and also has different types of error condition. Does the server-side implementation handle 'thrown' exceptions and PHP-level errors in a consistent manner? Is the error_reporting configuration appropriate (and does the library change it locally) ? Will the library interact poorly with debugging libraries (e.g: xdebug)?
Does the server-side implementation properly isolate JSON-RPC level errors from HTTP level errors? (i.e: does it prevent errors/exceptions inside a request lifecycle from becoming an HTTP-level error like 500: Internal Server Error).
Does the server-side implementation properly interact with your authentication & authorization mesures? (i.e: it might be desirable to promote related errors to 401: Unauthorized and 403: Forbidden HTTP statuses respectively).
Does the server-side implementation 'leak' sensitive information about your implementation or data-sources when delivering errors, either via HTTP or JSON-RPC?
These are pretty important questions to ask of a JSON-RPC webservice that will be used in a security minded setting. It's pretty hard to evaluate this for a given implementation by reading its documentation. You'll likely need to:
delve into its source code to look at the error handling strategies employed, and
perform extensive testing to avoid leaks and undesired propagation of error conditions.
How does the PHP implementation advise against or mitigate the PHP array set/list duality?
PHP is a crappy language. There. I said it :-)
One of the common issues JSON-RPC implementors deal with is how to properly map JSON arrays and JSON objects to language-native datastructures. While PHP uses the same data structure for both indexed and associative arrays, most languages do not. That includes JavaScript, whose language features/limitations informed the JSON (and therefore JSON-RPC) specifications.
As in PHP there is no way to distingish an empty indexed array from an empty associative array, and as various naughty things can be done to muddy existing arrays (e.g: set associative key on existing indexed array) various solutions have been proposed to deal with this issue.
One common mitigation is that the JSON-RPC implementation might force the author to cast all intended-associative arrays to (object) before returning them, and then reject at run-time any (implicity or intended) indexed arrays that have non-confirming keys or non-sequential indexes. Some other server-side libraries force authors to annotate their methods such that the intended array semantics are known at 'compile' time; some try to determine typing through automatic introspection(bad bad bad). Still other sever-side libraries leave this to chance.
The mitigations present/missing in the server-side PHP JSON-RPC implementation will probably be quite indicative as to its quality :-)
Will you need to implement SMD?
SMD is a not-very-standardized extension to JSON-RPC "-like" protocols to allow publishing of WSDL-like information about the webservice endpoint and the functions (no classes, although conventions for namespacing can be found out there) it exposes.
Sometimes 'typing' and 'authorization' concerns are layered with the class/function "annotations" used to implement SMD.
This 'standard' is not well supported in either client or server implementations :-)
Will your web-service also need to perform other HTTP-level routing?
(e.g: to static resources)
Some server-side implementations make all sorts of assumptions about how soon during the HTTP request cycle they'll be asked to get involved. This can sometimes complicate matters if the JSON-RPC implementation either implements the HTTP server itself, or constrains the content of all received HTTP messages.
You can often work around these issues by proxying known-JSON-RPC requests to a separate web server, or by routing non-JSON-RPC requests to a separate web-server.
If you need to serve JSON-RPC and non-JSON-RPC resources from the same webserver, ask yourself if the server-side JSON-RPC implemenation makes this impossible, or whether it makes you jump through hoops to work around it.
I have a web application in which two of the major components are the website (implemented in Groovy and Grails) and a backend RESTful web service (implemented using JAX-RS (Jersey) and Spring). Both of these will be running in Glassfish. The website will make calls to the RESTful web service. In many cases, these components will reside on separate servers, so the website will make calls over the network to the RESTful web service. If, however, I run both applications in the same Glassfish server, are there any optimizations that can be made to avoid the network call? In other words, I'm looking for some equivalent of EJB's remote/local interfaces for REST. Thanks!
Don't sweat the network call. Your traffic will generally never leave the local interface so you won't be consuming any bandwidth. You lose a bit of performance from serialization/deserialization, but you'll need to ask yourself if reducing the impact of this is worth developing a complicated proxy architecture. I think it most cases you'll find the answer to be no.
Not sure you will find any trivial solutions: you could of course add your own additional proxy layer, but I really wouldn't worry about it. Local network I/O (localhost or 127.0.0.1) is so heavily optimized anyway that you really won't notice.
Depending on your implementation Spring does support a number of remoting technologies (an old list is at http://static.springsource.org/spring/docs/2.0.x/reference/remoting.html), but you will find that key to all of these is the network transfer: they wrap it up in a variety of different ways but ultimately almost all turnkey remoting technologies drop into the network at some point in time. You may gain SOME efficiency by not having to use HTTP, but you will probably lose some of the loose coupling you gained by using Jersey.
If you are not too afraid to tightly couple maybe you can put the actual objects you are exposing via Jersey into a Glassfish-wide Spring context and invoke the methods directly: much tighter coupling though, so I'd say stick with the HTTP calls.
Yes, you can avoid a network call if your server and client both reside in the same JVM. You should be able to use Jersey Client API to create your own implementation of Connector to override default HTTP calls and handle request/response. Here is the blog that can get you started - http://www.theotherian.com/2013/08/jersey-2.0-server-side-client-in-memory-connector.html
IMHO, an unnecessary network overhead should be avoided at all cost. Even though this overhead would be only a few milliseconds, but while building features for your web application, you would increase such services call and all these milliseconds will add up to a good amount of latency on your application.
I am working on an application that needs to be able to post to HTTPS and keep track of the session that is created by authenticating to the https server. Is there anything in java and android that handles this better than just using the http methods offered by java? Like the HttpsURLConnection.
Thanks!
This depends on what you mean by "better". HTTPURLConnection works well for many cases but if this is not enough, you may look into HTTP Core from Apache. I understand that HTTP Core can work on Android.
I'm in the process of writing a client/server application which should work message based. I would like re-use as much as possible instead of writing another implementation and curious what others are using.
Features the library should offer:
client and server side functionality
should work message based
support multi-threading
should work behind load balancer / firewalls
I did several tests with HTTPCore, but the bottom line is that one has to implement both client and server, only the transport layer would be covered. RMI is not an option either due to the network related requirements.
Any ideas are highly appreciated.
Details
My idea is to implement a client/server wrapper which handles the client communication (including user/password validation) and writes incoming requests to a JMS queue:
#1 User --> Wrapper (Check for user/password) --> JMS --> "Server"
#2 User polls Wrapper which polls JMS
Separate processes will handle the requests and can reply via wrapper to the clients. I'd like to use JMS because:
it handles persistence quite well
load balancing - it's easy to handle peaks by adding additional servers as consumer
JMSTimeToLive comes in handy too
Unfortunately I don't see a way to use JMS on it's own, because clients should only have access to their messages and the setup of different users on JMS side doesn't sound feasible either.
Well, HTTP is probably the best supported in terms of client and server code implementing it - but it may well be completely inappropriate based on your requirements. We'll need to actually see some requirements (or at least a vague idea of what the application is like) before we can really advise you properly.
RMI works nicely for us. There are limitations, such as not being able to call back to the client unless you can connect directly to that computer (does not work if client is behind a firewall). You can also easily wrap your communication in SSL or tunnel it over HTTP which can be wrapped in SSL.
If you do end up using this remember to always set the serial version of a class that is distributed to the client. You can set it to 1L when you create it, or if the client already has the class use serialver.exe to discover the existing class's serial. Otherwise as soon as you change or add a public method or variable compatibility with existing clients will break.
static final long serialVersionUID = 1L
EDIT: Each RMI request that comes into the server gets its own thread. You don't have to handle this yourself.
EDIT: I think some details were added later in the question. You can tunnel RMI over HTTP, then you could use a load balancer with it.
I've recently started playing with Hessian and it shows a lot of promise. It natively uses HTTP which makes it simpler than RMI over HTTP and it's a binary protocol which means it's faster than all the XML-based protocols. It's very easy to get Hessian going. I recently did this by embedding Jetty in our app, configuring the Hessian Servlet and making it implement our API interface. The great thing about Hessian is it's simplicity... nothing like JMS or RMI over HTTP. There are also libraries for Hessian in other languages.
I'd say the best-supported, if not best-implemented, client/server communications package for Java is Sun's RMI (Remote Method Invocation). It's included with the standard Java class library, and gets the job done, even if it's not the fastest option out there. And, of course, it's supported by Sun. I implemented a turn-based gaming framework with it several years ago, and it was quite stable.
It is difficult to make a suggestion based on the information given but possibly the use of TemporaryQueues e.g. dynamically created PTP destinations on a per client basis might fit the problem?
Here is a reasonable overview.
Did you tried RMI or CORBA? With both of them you can distribute your logic and create Sessions
Use Spring....Then pick and choose the protocol.
We're standardizing on Adobe's AMF as we're using Adobe Flex/AIR in the client-tier and Java6/Tomcat6/BlazeDS/Spring-Framework2.5/iBATIS2.3.4/ActiveMQ-JMS5.2 in our middle-tier stack (Oracle 10g back-end).
Because we're standardizing on Flex client-side development, AMF and BlazeDS (now better coupled to Spring thanks to Adobe and SpringSource cooperating on the integration), are the most efficient and convenient means we can employ to interact with the server-side.
We also heavily build on JMS messaging in the data center - BlazeDS enables us to bridge our Flex clients as JMS topic subscribers. That is extremely powerful and effective.
Our Flex .swf and Java .class code is bundled into the same .jar file for deployment. That way the correct version of the client code will be deployed to interact with the corresponding middle-tier java code that will process client service calls (or messaging operations). That has always been a bane of client-server computing - making sure the correct versions of the respective tiers are hooked up to each other. We've effectively solved that age-old problem with our particular approach to packaging and deployment.
All of our client-server interactions work over HTTP/HTTPS ports 80 and 443. Even the server-side messaging push we do with BlazeDS bridged to our ActiveMQ JMS message broker.