I have a query which looks something like this:
(|(mail=andrew*)(cn=andrew*)(sn=andrew*)(telephoneNumber=andrew*))
i.e. it takes a search term from a UI and looks for a match against term* across a bunch of attributes.
The user enters andrew in this case and the app adds the wildcard. If the user enters andrew` (trailing back tick) the app looks for andrew`*.
I've noticed that if telephoneNumber is included in the searched attributes the query fails with a javax.naming.InvalidAttributeValueException, if it's excluded then the query works without error.
I'm not particularly interested in the backtick alone, but as it's not a special character in LDAP searches I'm wondering why I get this behaviour and if other characters will produce similar results. Is there going to be something in the schema that explains this if I can figure out how to query it, or will it be something else?
If it matters, accessing via a Spring library in a Java app.
The query probably fails because of attribute value constraints on the telephoneNumber attribute. The syntax for telephoneNumber attributes is described in this RFC. Back tick does indeed seem to be an invalid character in telephoneNumber values.
Now, I might be wrong, but reading your question it appears you are trying to construct filters using string concatenation. Please note that you should never, ever build queries of any sort using string concatenation, especially when parts of the queries come from user input. I'm sure you know this is the case for SQL queries, and it's equally true when you're using LDAP.
Spring LDAP provides two ways to help you build LDAP queries. The preferred approach is the use the LDAP query API documented here and (advanced usage) here. The old deprecated - but still functioning - way to do it is using the filter classes, documented in the old reference documentation here.
Using these utilities you won't need to keep track of which characters need to be encoded and when. You also eliminate any risk for query injection attacks.
Related
In Spring-Data-Rest an object's ID is the URI returned with the object. For example, if I have a User, it might have a URI like:
http://example.com/users/1
In an authentic REST api, this URI is the id of the object, you are not supposed to use just '1' as the id.
Give that, how do I search for all Orders that belong to that User?
http://example.com/orders/search/findByUser?user={{XXX}}
Specifically, what do I use for {{XXX}}?
I know I could do the opposite search:
http://example.com/users/1/orders
But in my case I need to search for matching jobs so I can add additional parameters which are also keys.
I can export /orders/search/findByUser by creating this function definition on OrderRepository:
List findByUser(User user);
and findByUser will be exported by Spring-Data-REST, but how do I specify the User as a GET parameter?
Again, I'm specifically looking for the pure REST solution, since the Spring Data Rest project is trying to encourage purity.
You might take a look at the Query annotation of Spring Data. It enables you to execute custom queries without the need of a custom controller.
Edit:
Query parameters are a good way to filter a resource by simple properties. As SDR serializes all complex types as relations, it is even clearer that filtering only applies to the remaining (simple) properties.
If you have only one relation, you correctly mentioned the way of doing the 'inverse' search as you called it by using /users/1/orders.
If you want to search by multiple relations I suggest you define a seperate search (sub-)resource and perform the search by issuing a POST-request to this resource.
For example:
POST /orders/search
{
"user": "http://example.org/users/1",
...
}
This way, SDR will correctly translate the URIs into entities. However, I think you will need to use a custom controller here but it should be possible to still use the Spring Data repository and provide the user and additional entities as parameter.
For further information, see the following SO-questions:
How to design RESTful search/filtering?
RESTful URL design for search
Edit2:
Addressing the point that using a POST for searching violates the REST spec:
REST was designed to be simple. One of the key advantages of REST is that you are not forced to do anything. You can adapt the spec until it fits your needs. Of course, this could mean that your API is less RESTful but you should consider if it is worth it to strictly stick to the spec if it introduces an unnecessary overhead for the consumers of your API.
Of course you can design the above the idea to fully meet the REST spec. This would involve creating a seperate search entity, persisting it to the database and later retrieve the result of the search with a call to a subresource like /result or something like that. However, the question is, if it is worth it.
In your concrete example, I would just require the client to parse the ID out of the link and provide it as query parameter. If you are scaling up your application later, you could introduce a feature like named searches and apply the above mentioned solution.
If you use a controller, like it seems to be your case, you can pass it any parameter(s) you consider necessary. Take a look at this question:
Spring Data REST custom query integration
See https://jira.spring.io/browse/DATAREST-502
Depending of your version of Spring Data, it would work as you want or not.
By the way, I still think the POST should be an option too, it would be much cleaner.
The project I'm working on has a REST API written in JRuby/Java, with an endpoint that hits a MySQL database to retrieve a number of records.
We need to allow the client to filter those records using one or more columns, including boolean checks and range values.
The easiest way we can do this is to add a string parameter to the API, then add it into the SQL statement.
Collectively, the development team agree that this is a bad idea but the alternative is to provide an almost identical syntax for filtering, which is translated into SQL. The allure of the SQL injection parameter is strong.
So my question is, are there any circumstances under which this is a safe thing to do?
In particular, might we consider using the WHERE clause safely if it's been fully parsed beforehand, and identified as such. Or at the very least, checking for certain trigger words such as DROP, SELECT etc.
Also if anyone knows of a good library that could act as a go-between (translating or parsing an external expression into a WHERE clause) that would be great.
The OData and GData protocols already implement this functionality in a safe and standard way. You can find server and client implementations for both, for Ruby, PHP, MySQL etc. Check here for the OData libraries
Leaving aside the SQL injection issue, you'll expose your inner implementation (both the database chosen - MySQL and your table structure) directly in the form of your API.
e.g. if you change to some NoSQL-type implementation at the backend, your public-facing API will break immediately. Similarly if you restructure your database. I wouldn't do this even in an environment in which I wasn't worried about the probability/severity of injection attacks.
Besides the security implications, allowing an arbitrary WHERE clause is a bad idea because it takes the "I" out of "API" -- it's not an interface. The API is supposed free the user of the need to know details of the implementation. Like table and column names.
If clients are interacting with your data by constructing their own WHERE clauses, then you can never change the database. There might be code out there with those statements programmed in. If a bug or new feature required you to alter the DB in a way that would break existing client interactions you'd be stuck. The API should provide the filtering capability and translate requests into calls to backend in a way that lets you change the backend without breaking the API.
There are numerous ORM's for this purpose, especially in ruby (activerecord, sequel)
The most basic thing you need to do is escape the string input, which will pretty much prevent sequel injection if you are doing it properly.
It helps to not directly insert parameters directly into the sequel statement if you dont have to either, instead, check their validity and then map them to logical ones (this isn't always possible). For example, if there is an html dropdown list, and when you submit the form it passes some parameter 'firstitem', map 'firstitem' to an id or otherwise that you will then use to search on, versus using the user supplied version (assuming this mapping doesn't involve the db).
We have a Java web app with a hibernate backend that provides REST resources. Now we're facing the task to implement a generic search that is controlled by the query parameters in our get request:
some/rest/resource?name_like=foo&created_on>=2012-09-12&sort_by_asc=something
or similar.
We don't want to predefine all possible parameters(name, created_on,
something)
We don't want to have to analyze the request String to pick up control characters (like >=)
nor do we don't want to implement our own grammar to reflect things like _eq _like _goe and so on (as an alternative or addition to control characters)
Is there some kind of framework that provides help with this mapping from GET request parameters to database query?
Since we know which REST resource we're GETing we have the entity / table (select). It probably will also be necessary to predefine the JOINs that will be executed in order to limit the depths of a search.
But other than that we want the REST consuming client to be able to execute any search without us having to predefine how a certain parameter and a certain control sequence will get translated into a search.
Right now I'm trying some semi automatic solution building on Mysemas QueryDSL. It allows me to predefine the where columns and sort columns and I'm working on a simple string comparison to detect things like '_like', '_loe', ... in a parameter and then activate the corresponding predefined part of the search. Not much different from an SQL String except that it's SQL injection proof an type save.
However I still have to tell my search object that it should be able to potentially handle a query "look for a person with name like '???'". Right now this is okay as we only consume the REST resource internally and isolate the actual search creation quite well. If we need to make a search do more we can just add more predefinitions for now. But should we make our REST resources public at some time in the future that won't be so great.
So we're wondering, there has to be some framework or best practice or recommended solution to approaching this. We're not the first who want this. Redmine for example offers all of its resource via a REST interface and I can query at will. Or facebook with its Graph API. I'm sure those guys didn't just predefine all possibilities but rather created some generic grammar. We'd like to save as much as possible on that effort and use available solutions instead.
Like I said, we're using Hibernate so an SQL or HQL solution would be fine or anything that builds on entities like QueryDsl. (Also there's the security issue concerning SQL injection)
Any suggestions? Ideas? Will we just have to do it all ourselves?
From a .NET perspective the closest thing I can think of would be a WCF data service.
Take a look at the uri-conventions specified on the OData website. There is some good information on the section on 4.5 Filter System Query Option. You'll notice that a lot of the examples on this site are .NET related, but there are other suggestions for getting this to work with Java.
I have a (REST) application that handles CRUD operations initiated by a 3rd party, and basically my problem is in returning an easily understood error message in the case of a constraint violation. I'm using Hibernate (though JPA solutions would be better) and the ConstraintViolationException it throws is not terribly useful, it doesn't even have the name of the constraint that has been violated. I've so far been returning constraintViolation.getSQLException().getMessage() which returns a half decent message along the lines of Duplicate entry '<value>' for key 2.
The problem is a new requirement to improve the error returned to include which entity failed to commit as well as the value (and what kind of failure it was as a custom error code; DUPLICATE_VALUE, MISSING_FOREIGN_KEY, or something like that). Since the above string comes from the SQLException I assume it depends on the DB/driver, so while I could pick out '' from that string and search through every field in every object I just committed to try to find the right one it doesn't exactly seem the robust option (especially as we need to support multiple databases).
So is there any way I can find this information based on the information at hand? (Which is basically just the ConstraintViolationException w/SQLException and the object I tried to persist).
This is why you should validate provided data in business logic rather than depending on Hibernate to do it. Web frameworks provide validation mechanisms for just this reason.
It should be possible to capture that exception specifically and return a meaningful response, instead of generic exceptions and parsing it to understand , it's worth the effort to re-factor this. just my 2c's
I have a number of domain/business objects, which when used in a hibernate criteria are referenced by the field name as a string, for example :
Criteria crit = session.createCriteria(User.class);
Order myOrdering = Order.desc("firstname");
crit.addOrder(myOrdering);
Where firstname is a field/property of User.class.
I could manually create an Enum and store all the strings in there; is there any other way that I am missing and requires less work(I'll probably forget to maintain the Enum).
I'm afraid there is no a good way to do that.
Even if you decide to use reflections, you'll discover the problem only when the query will run.
But there is a little bit better solution how to discover the problem early: if you use the Named Queries (javax.persistence.NamedQueries) you'll get all your queries compiled as soon as your entities are processed by Hibernate, so basically it will happen during the server's start-up. So if some object was changed breaking the query, you'll know about it the next time you start the server and not when the query is actually run.
Hope it helps.
This is one of the things that irritates me about Hibernate.
In any case, I've solved this in the past using one of two mechanisms, either customizing the templates used to generate base classes from Hibernate config files, or interrogating my Hibernate classes for annotations/properties and generating appropriate enums, classes, constants, etc. from that. It's pretty straight-forward.
It adds a step to the build process, but IMO it was exactly what I needed when I did it. (The last few projects I haven't done it, but for large, multi-dev things I really like it.)