There can be potentially up to 1000 strings in total. Should these be hardcoded or stored in database? These are frequently accessed because everytime user wants to register or checkout an item, they are going to need to see list of area/suburb/province/countries.
If i have bunch of Enums, i think the performance should be fast because there is a max number of strings ~1-2k max.
On the other hand, if i store them in database, there's going to be latency accessing the database as well as cpu/memory consumption.
Which option do you choose?
1000 isn't a huge amount, and I would put this information into a text file and read them into the program on start-up.
Regardless, this is data, not code, and so should not be an enum (code). Why not enum? It's a lot easier and more flexible to update/change data than it is to change code, should this need to be changed in the future.
If you will definitely be updating and changing this information with time, especially if through multiple sources, then a database is surely the way to go.
It all depends on you. There is no proper convention. Below are 3 ways along with their pros and cons.
Create a class with static final string variables.
Pros:
a. Very easy to use.
b. Developers can do look ups from within IDEs.
Cons:
a. Every time you need to add/delete something, code will have to be recompiled. However, this will not be much problem if you have ci-cd in place.
Add everything in properties file and load at runtime.
Pros:
a. Modifying things will be a breeze. No code recompilation required.
Cons:
a. This would still need re-deployment and server restart.
b. Developers will be unhappy as they will have to refer the txt file every now and then. Also this could lead to mistake if developers use wrong codes which are not present in properties file.
Use database
Pros:
a. Highly configurable.
b. No need of re-deployment.
Cons:
a. Service restart will be required.
As you can see, service restart will be required for all of them as you will definitely going to use caching in case 2 and 3. My suggestion would be to use first option if they are literally never going to change as it is quite developer friendly.
Related
I try to figure out the best solution for a use case I'm working on. However, I'd appreciate getting some architectural advice from you guys.
I have a use case where the frontend should display a list of users assigned to a task and a list of users who are not assigned but able to be assigned to the same task.
I don't know what the better solution is:
have one backend call which collects both lists of users and sends them
back to the frontend within a new data class containing both lists.
have two backend calls which collect one of the two lists and send them
back separately.
The first solution's pro is the single backend call whereas the second solution's pro is the reusability of the separate methods in the backend.
Any advice on which solution to prefer and why?
Is there any pattern or standard I should get familiar with?
When I stumble across the requirement to get data from a server I start with doing just a single call for, more or less (depends on the problem domain), a single feature (which I would call your task-user-list).
This approach saves implementation complexity on the client's side and saves protocol overhead for transactions (TCP header, etc.).
If performance analysis shows that the call is too slow because it requests too much data (user experience suffers) then I would go with your 2nd solution.
Summed up I would start with 1st approach. Optimize (go with more complex solution) when it's necessary.
I'd prefer the two calls because of the reusability. Maybe one day you need add a third list of users for one case and then you'd need to change the method if you would only use one method. But then there may be other use cases which only required the two lists but not the three, so you would need to change code there as well. Also you would need to change all your testing methods. If your project gets bigger this makes your project hard to update or fix. Also all the modifications increase the chances of introducing new bugs as well.
Seeing the methods callable by the frontend of the backend like an interface helps.
In general an interface should be open for extension but closed on what the methods return and require. As otherwise a slight modification leads to various more modifications.
ok so this is a general question ... in my app on GAE I need to keep a 2 types of static data, parameters that could change like user names and passwords (for external services not my user database) so the best way is not to hard code them and keep them in the datastore so I can change them from the admin system settings panel. However, how about stuff that will NEVER change, like a list of countries for selectors in forms. Here are my options:
Option 1:
Keep them in the datastore under a dataUitls entity. However, this will have a toll on my datastore quotations.
Option 2:
Hard coded in a class. This will not affect my datastore quotations, and will serve much faster in a JSP loop as I dont have to wait for datastore transactions, however, this will substantially affect my memory and instances. Example is as follows
package system.Tools;
import java.util.ArrayList;
import java.util.List;
public class SystemConstants
{
public static String[] tmp = {"String1","String2"};
public static List<String>Countries = get(tmp);
private static List<String> get(String[]countries)
{
List<String>result = new ArrayList<String>();
for(String tmp:countries)
{
result.add(tmp);
}
return result;
}
// getters and setters for whatever other parameters I have here
}
Both will work no doubt, but I need a professional opinion (preferably someone who has tried both) telling me which is a better practice for this particular situation.
Thanks alot guys !! Keep em coming !!
Hard coded constants have far less processor overhead than Datastore queries and also use less memory. The memory overhead of Datastore access classes and objects will exceed that of a fairly large number of hard coded constants.
The only possible advantage that Datastore may have, in an extreme case, is slightly less memory usage when you need only a small subset of lots of data. A list of countries is not that extreme.
The best choice will depend on specific details of your situation. Rather than speculate, measure and compare.
It depends on your requirement. If you believe your data will NEVER change, then no barrier to hardcode. Whenever you want to add multi language support, you may end up with lots of changes due to hardcoding.
Don't be too concerned about memeory usage for just a static variable. its negligible compared to other memory usages within you program.
Also there is a third way that you can use.
write these into a .property file and bundle it with your program. at the startup u read the file and load values.
In Summary, Don't think alot about memory. If your list is in the DB, you have to load it into the memory before you use. Database solution is good if those are changeable via a GUI. property file based approach is efficient and easy to program.
You have some options, depending of the case and the data:
Harcode: if the data is not changing at all and the structure is easy and it is not growing (more and more and...), it could be ok.
Properties file: if your data change from one environment to another, it could give you advantage to get the data in the a file. Change the file is easy from one environment to another, more than recompile.
Json or similar: if the data structure could change (new properties) or it is "complex", to have all in a json is more easy to manage. Parse using Jackson or similar it is one or to lines of code if you have the DTOs/classes. And the maintenance is better than hardcode data.
Database: if the data is not changing is "static" to put in the database doesn't give you any advantage, IMO.
Of course, whatever you decide, you can parse/read in the start up of the server and write the data in the memory or memcached.
I am looking at changing the way some large objects which maintain the data for a large website are reloaded, they contain data relating to catalogue structure, products etc and get reloaded daily.
After changing how they are reloaded I need to be able to see whether there is any difference in the resulting data so the intention is to reload both and compare the content.
There may be some issues(ie. lists used when ordering is not imporatant) that make the comparison harder so I would need to be able to alter the structure before comparison. I have tried to serialise to json using gson but I run out of memory. I'm thinking of trying other serialisation methods or writing my own simple one.
I imagine this is something that other people will have wanted to do when changing critical things like this but I haven't managed to find anythign about it.
In this special case (separate VMs) I suggest adding something like a dump method to each class which writes the relevant content into a file (human readable text). This method calls dump on each aggregated object as well.
In the end you have to files from each VM, and then you can compare them using an MD5 checksum for example.
This is probably a lot of work, but if you encounter any differences, you can use diff on both files, and this will be a great help.
You can start with a simple version, and refine it step-by-step by adding more output.
Adding (complete) serialization later to a class is cumbersome. There might be tools which simplify this (using reflection etc.), but in my experience you have to tweak your classes: Exclude fields which are not relevant, define a sort order for lists, cyclic relations etc.
Actually I use a similar approach for the same reasons (to check whether a new version still returns the same result): The application contains multiple services (for each version), the results are always data transfer objects, serialization is added immediately to the DTOs, and DTOs must provide a comparison method dedicated for this purpose.
Looking at the complications and memory issues, also as you have mentioned you dont want to maintain versions, i would look to use database for comparison.
It will need some effort in terms of mapping your data in jvm to db table but once you have done that, it will be staright forward. You can dump data from one large object in db tables and then you can simply run a check from 2nd object in db.
Creating a stored proc can simplify things. This solution can support data check from any number of jvms.
I have a lot of existing data in my database already, and want to develop a points mechanism that computes a score for each user based on what actions they do.
I am implementing this functionality in a pluggable way, so that it is independent of the main logic, and relies on Spring events being sent around, once an entity gets modified.
The problem is what to do with the existing data. I do not want to start collecting points from now, but rather include all the data until now.
What is the most practical way to do this? Should I design my plugins in such a way as to provide for an index() method, which will force my system to fetch every single entity from the database, send an EntityDirtyEvent, to fire the points plugins, for each one, and then update it, to let points get saved next to each entity. That could result in a lot of overhead, right?
The simplest thing would be to create a complex stored procedure, and then make the index() call that stored procedure. That however, seems to me like a bad thing either. Since I will have to write the logic for computing the points in java anyway, why have it once again in SQL? Also, in general I am not a fan of splitting business logic into the different layers.
Has anyone done this before? Please help.
First let's distinguish between the implementation strategy and business rules.
Since you already have the data, consider obtaining results directly from the data. This forms the data domain model. Design the data model to store all your data. Then, create a set of queries, views and stored procedures to access and update the data.
Once you have those views, use a data access library such as Spring JDBC Template to fetch this data and represent them into java objects (lists, maps, persons, point-tables etc).
What you have completed thus far does not change much, irrespective of what happens in the upper layers of the system. This is called Model.
Then, develop a rule base or logic implementation which determines, under what inputs, user actions, data conditions or for all other conditions, what data is needed. In mathetical sense, this is like a matrix. In programming sense, this would be a set of logic statements. If this and this and this is true, then get this data, else get that data, etc. This encompasses the logic in your system. Hence it is called "Controller".
Do not move this logic into the queries/stored procedure/views.
Then finally develop a front-end or "console" for this. In the simplest case, develop a console input system, which takes a .. and displays a set of results. This is your "view" of the system.
You can eventually develop the view into a web application. The above command-line view can still be viable in the form of a Restful API server.
I think there is one problem here to be considered: as I understand there's huge data in the Database so the idea to create only one mechanism to calculate the point system could not be the best approach.
In fact if you don't want to start collecting points but include all the data, you must process and calculate the information you have now. Yes, the first time you will run this can result an overhead, but as you said, you need this data calculated.
By other hand you may include another mechanism that attends changes in an entity and launches a different process capable of calculate the new pointing diffence that applies to this particular modification.
So, you can use one Service responsible of calculate the pointing system, one for a single entity and another, may be longer to finish, capable of calculate the global points. Even, if you don't need to be calculated in real-time you can create a scheduled job responsible of launch it.
Finally, I know it's not a good approach to split the business logic in two layers (Db + Java) but sometimes is a requirement do it, for example, if you need to reply quickly to a request that finally works with a lot of registries. I've found some cases that there's no other option than add business logic to the database (as a stored procedures, etc) to manage a lot of data and return the final result to the browser client (ex: calculation process in one specific time).
You seem to be heading in the right direction. You know you want your "points" thing decoupled from the main application. Since it is implied you are already using hibernate (by the tag!), you can tap into the hibernate event system (see here section 14.2). Depending upon the size/complexity of your system, you can plugin your points calculations here (if it is not a large/complex system), or you can publish your own event to be picked up by whatever software is listening.
The point in either design approach is that neither knows or cares about your point calculations. If you are, as I am guessing, trying to create a fairly general purpose plugin mechanism, then you publish your own events to that system from this tie-in point. Then if you have no plug-ins on a given install/setup, then no one gets/processes the events. If you have multiple plug-ins on another install/setup, then they each can decide what processing they need to do based upon the event received. In the case of the "points plugin" it would calculate it's point value and store it. No stored proc required....
You're trying to accomplish "bootstrapping." The approach you choose should depend on how complicated the point calculations are. If stored procedures or plain update statements are the simplest solution, do that.
If the calculations are complicated, write a batch job that loads your existing data, probably orders it oldest first, and fires the events corresponding to that data as if they've just happened. The code which deals with an event should be exactly the same code that will deal with a future event, so you won't have to write any additional code other than the batch jobs themselves.
Since you're only going to run this thing once, go with the simplest solution, even if it is quick and dirty.
There are two different ways.
One is you already know that - poll the database for for changed data. In that case you are hitting the database when there may not be change and it may slow down your process.
Second approach - Whenever change happens in database, the database will fire the event. That you can to using CDC (Change Data Capture). It will minimize the overhead.
You can look for more options in Spring Integration
The description may sound like just a bunch of words so here is a more detailed explanation. I have a User object which is mapped to database table.
I want users to be in different roles. There will be a bunch of those - and they technically will be the same users in same table but to them will apply different roles. Say user in role A will have to have two fields as required, and will have to have certain restrictions to the length and contents on his password, as well as the time expiration of his password, etc.
While I can hardcore those rules I am very interested to find out of there is an other way to define the rules and may be store in database so it's easier to load/apply and the main idea - to change and update them -- without redeploying the codebase.
Technically the stupidest and straightforward solution is to implement class, serialized, store in db, then load, deserialze, call methods on it which will execute rules. The problem is in changes to the ruleset ( read "interface" of the rule class ) and that generally solution sounds like a hack.
Anytihing else? Any frameworks? Other approaches?
UPDATE: probably was not clear. say, I have class User.java
I need to define different rules say:
1. do we need to verify length of password, and what should it be?
2. do we need to require some properties to be required?
3. do we need to track login attempts for this user?
4. if we do track, how many login attempts allowed?
5. do we expire password?
6. if we do, then in how many days? or months? or weeks?
7. ...
and so on and so on.
so questions ARE.
- how do I define those rules and operate on User object WITHOUT modifying and redeploying code base?
- how do I store those set of rules?
Drools, jBPM, etc. do not seem like a fit for that task. But any advice would help!
JRuleengine is good I heard, sometime back I planned to use it for similar application.
There are many other Rule Engines though.
Well there are some good rules engines out there include jrules, drools I think is popular too. One thing to keep in mind is the relationship between a rule and the data it examines. After all you can have the rules in a word document, but when they execute they need examine data, and that is also a factor in choosing a rule engine or architecture. generally its if (a > b) then do y. Means you need to examine a and b in the rule execution. That is the real issue is how to get the parameters into the rule and engine.