I have to prepare a logic where in a table there are two types of columns for a record.
One that identifies a unique record means key fields
that are non key fields
If key fields are changing then I have to update a record with Delete/Add operator. Whereas if non key fields are updating I have to use Change operator.
I get the records in List of object which using comparator I am able to detect change in Key Fields and use Add/Delete operator.
Comparator logic is compare old Key fields in && with new key fields if there is a change Delete Operator for Old record and Add for new record added. This is straight.
But for Change operator what will be the best way to know which field is updated? Accordingly 'Change' will be added. Using comparator I can validate if something is changed or not, but what changed how to know using comparator or other thing?
As a example in below record ID, Open Date and Status are Key fields where as rest are non key fields
ID, Open Date, Closed Date, Status, Est Closed Date
I can compare complete record for Non key comparision to know if something changed but what is changed I can't know because I have to update the same existing record with Update operator instead of Delete/Add.
I will run Add/Delete comparator check first and then will validate for non key fields changes. Please guide
Update: I think I answered my own question, first I will do Key Field comparision for a record to generate Add/Delete records on basis of updates in key fields combination and then for same record i will do complete key comparision if change is detected update operator will be added and no Insert/Delete will be generated. I don't need to detect which exact non key field changed.
Thanks.
I think I answered my own question, first I will do Key Field comparison for a record and for same record i will do complete key comparison if change is detected update operator will be added and no Insert/Delete will be generated. I don't need to detect which exact non key field changed.
Thanks for looking my question and your time.
Related
I am unable to grasp the concept of the lookup table.
I am currently working on a project wherein I am using two tables.
The first table consists of two columns- name(varchar) and value(varchar).
The second table also has two rows- Result(varchar) and value(varchar).
Result is used to store the values which are obtained from a Java code. Whenever the Result of the Java code matches the name in the first table, I need to update the second table with the corresponding value in the first table.
Does using lookup table help in any way? If it does, can it be explained with an example?If not, is there any other way?
Just imagine a table person with a column GenderIsMale BIT. You can set this value to 1 (yes, it is a boy) or to 0 (no, a girl). This was easy in earlier days.
Now we have more categories. According to this link facebook offers more than 50 differing categories...
There the lookup-table comes into play: You create a table which has - as minium - a unique key and a value. In most cases this is an ID INT IDENTITY and a Content VARCHAR(100) NOT NULL. You can add more columns like Abbreviation or any other additional content (e.g. other languages or codes of external code systems read about mapping tables also) directly bound to this value.
The next step is, to take the GenderIsMale-column away and replace it with a
GenderID INT NOT NULL
CONSTRAINT FK_Person_GenderID FOREIGN KEY REFERENCES GenderLookUpTable(GenderID)
The person table will store the GenderID only, the related values are stored in the side table and can be looked up.
The simple lookup table is the basic construct of how to create a relational database model in min. 3.NF or BCNF (which should be a minium reuqirement for professional database design).
Whenever the Result of the Java code matches the name in the first
table, I need to update the second table with the corresponding value
in the first table.
That's a perfect use case for database trigger, which can be used to perform various things when a change (insert, update, delete) happens in a table.
Assuming you're inserting the value of your Java calculations to your (result, value) table (let's call it foo, and the other table is bar), you can write a trigger that replaces the value being written with the value from the other table. Example given for Postgres, if using another db refer to your particular RDBMS manual to see the syntax.
CREATE FUNCTION get_value_from_lookup_table() RETURNS trigger AS $$
BEGIN
IF EXISTS (SELECT 1 FROM bar WHERE name = NEW.result) THEN
RETURN SELECT name, value FROM bar WHERE name = NEW.result;
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER lookup_value
INSTEAD OF INSERT ON foo
FOR EACH ROW
EXECUTE PROCEDURE get_value_from_lookup_table();
Every time an INSERT is done on foo, a check is done to see if a row exists in bar where name=result. If so, that row is inserted, otherwise the insert goes on normally. That's the basic gist of it. The actual solution depends on table constraints, whether you need to handle inserts and updates, etc.
I have mentioned a sequence generation strategy as IDENTITY on my entity class for the primary key of a table in AS400 system.
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
#Column(name = "SEQNO")
private Integer seqNo;
The table's primary key column is defined as GENERATED ALWAYS AS IDENTITY in database.
SEQNO BIGINT NOT NULL GENERATED ALWAYS AS IDENTITY(START WITH 1, INCREMENT BY 1)
My understanding of IDENTITY strategy is that it will leave the primary key generation responsibility to the table itself.
The problem that I am facing is that somehow in one environment, while inserting record in the table it gives me [SQL0803] Duplicate Key value specified.
Now there are couple of questions in my mind:
Is my understanding correct for #GeneratedValue(strategy=GenerationType.IDENTITY)?
In which scenario table will generate Duplicate key?
I figured out there are sequence values missing in the table, i.e. after 4, the sequence till 20 is missing and I do not know if someone manually deleted it or not, but could this be related to duplicate key generation?
YES. IDENTITY means use in-datastore features like "AUTO_INCREMENT", "SERIAL", "IDENTITY". So any INSERT should omit the IDENTITY column, and will pull the value back (into memory, for that object) after the INSERT is executed.
Should never get a duplicate key. Check the INSERT statement being used.
Some external process using the same table? Use the logs to see SQL and work it out.
I don't use JPA, but what you have seems reasonable to me.
As far as the DB2 for i side...
Are you sure you're getting the duplicate key error on the identity column? Are there no other columns defined as unique?
It is possible to have a duplicate key error on an identity column.
What you need to realize is that the next identity value is stored in the table object; not calculated on the fly. When I started using Identities, I got bit by a CMS package that routinely used CPYF to move data between newly created versions of a table. The new version of the table would have a next identity value of 1, even though there might be 100K records in it. (the package has since gotten smarter :) But the point remains that CPYF for instance, doesn't play nice with identity columns.
Additionally, it is possible to override the GENERATED ALWAYS via the OVERRIDING SYSTEM VALUE or OVERRIDING USER VALUE clauses of the INSERT statement. But inserting with an override has no effect on the stored next identity value. I suppose one could consider CPYF as using OVERRIDING SYSTEM VALUE
Now, as far as your missing identities...
Data was deleted
Data was copied in with overridden identities
Somebody ALTER TABLE <...> ALTER COLUMN <...> RESTART WITH
You lost the use of some values
Let me explain #4. For performance reasons, DB2 for i by default will cache 20 identity values for a process to use. So if you have two processes adding records, one will get values 1-20 the other 20-40. This allows both process to insert concurrently. However, if process 1 only inserts 10 records, then identity values 11-20 will be lost. If you absolutely must have continuous identity values, then specify NO CACHE during the creation of the identity.
create table test
myid int generated always
as identity
(start with 1, increment by 1, no cache)
Finally, with respect to the caching of identity values. While confirming a few things for this answer, I noticed that the use of ALTER TABLE to add a new column seemed to cause a loss of the cached values. I inserted 3 rows, did the alter table and the next row got an identity value of 21.
I have an HashMap(String,Object). The key is combination of more than 1 unique ID. I have an input, a string which is part of the key(1 unique ID). I need to take the value in HashMap using that part of the key i have without iterating thousands of values in HashMap.
Can we achieve it using any Regex statement in HashMap.get()?
My Key is xxx.yyy.zzz where combination of xxx.zzz is unique throughout the Map. I have xxx and zzz as input. Also i have set of possible values of yyy(5-6 possibilities which may increase as well)for a given zzz.
I have two options to solve this now.
Map.Entry to check whether key starts and ends with xxx and zzz respectively
Trial and Error Method
i. Form key xxx.yyy.zzz with all possible yyys and check for whether the key is present or not using .contains()
ii. But this way, if i do .contains() 5-6 times for each call, won't it loop through 5-6 times at the worst case?
iii. Also i am creating more strings in stringpool.
Which one should i prefer?
The only way to retrieve a value from a HashMap without iterating over the entries/keys (which you don't want) is by searching for the full key.
If you require efficient search via a partial key, you should consider having a HashMap whose key is that partial key.
No, it's not possible to use partial keys with a HashMap.
With TreeMap this can be achieved with a partial prefix of the wanted key, as it allows you to use tailMap(String key) to return a part of the map that would follow a specific key (i.e. your keypart). You'd still need to process the entries to see which ones would match the partial key.
If your keys are like xxx.yyy.zzz and you want to use xxx.* type access then you could consider my MapFilter class.
It allows you to take a Map and filter it on a certain key prefix. This will do the searching for specific prefixes and retain the results of that search for later.
Can we achieve it using any Regex statement in HashMap.get()?
No.You can't. You need to pass the exact key to get the associated value.
Alternatively, you should itertate ober keys and get the values matched to it. They you can have regex to match your input string against key.
You cannot do this using a HashMap. However, you can use a TreeMap which will internally store the keys according to their natural order. You can write a custom search method which will find the matching key, if it exists, in the set using the regex. If written correctly, this will take O(lgN) time, which is substantially better than linear. The problem reduces to searching for a String in an ordered list of Strings.
As #Thilo pointed out, this solution assumes that you are trying to match a fragment of a key which starts at the beginning, and not anywhere else.
HashMap works on hashing algorithm that maintains hash buckets of hash code of keys and based on that hash code hash map retrieves corresponding value. For the you need to override equals() and hashcode() method for custom objects.
So
If you will try to get the value of a key, then key's hash code value get generated and further fetch operation happen based on that hash code.
If you would not give a exact match of key how HashMap will find out that bucket with a wrong hashcode ?
I'm trying to insert a new record using UpdatableRecords in jOOQ 3.4.2. The pattern is extremely concise and pleasant to use, except that the INSERT reads null values as no value and ignores default values or a generated index. How can I use the UpdatableRecord to do an insert that respects default values and generated indexes?
Here's my table:
CREATE TABLE aragorn_sys.org_person (
org_person_id SERIAL NOT NULL,
first_name CHARACTER VARYING(128) NOT NULL,
last_name CHARACTER VARYING(128) NOT NULL,
created_time TIMESTAMP WITH TIME ZONE DEFAULT current_timestamp NOT NULL,
created_by_user_id INTEGER,
last_modified_time TIMESTAMP WITH TIME ZONE,
last_modified_by_user_id INTEGER,
org_id INTEGER NOT NULL,
CONSTRAINT PK_org_person PRIMARY KEY (org_person_id)
);
Note my primary key and default values. Now here's my jOOQ code:
// orgPerson represents a POJO filled with my values to be inserted and null for everything else
// Note that orgPerson.orgPersonId is null
OrgPersonRecord orgPersonRecord = create.newRecord( ORG_PERSON, orgPerson );
Integer orgPersonId = create.executeInsert( orgPersonRecord );
But when I run this, I get the error null value in column "org_person_id" violates not-null constraint.
I noticed the jOOQ docs say that calling newRecord automatically sets all the internal "changed" flags to true on the UpdatableRecord. So then I tried this:
// orgPerson represents a POJO filled with my values to be inserted and null for everything else
// Note that orgPerson.orgPersonId is null
OrgPersonRecord orgPersonRecord = create.newRecord( ORG_PERSON, orgPerson );
orgPersonRecord.changed( ORG_PERSON.ORG_PERSON_ID, false );
orgPersonRecord.changed( ORG_PERSON.CREATED_TIME, false );
orgPersonRecord.insert()
Integer orgPersonId = orgPersonRecord.getOrgPersonId();
But that gives me the error ERROR: duplicate key value violates unique constraint "pk_org_person". And when I do this repeatedly, the values seem to keep increasing by 1. This doesn't really make sense to me, but my greater question is: Is there a good way I can do an INSERT based on my object values, or better yet, simply include only the non-null columns?
I saw JOOQ ignoring database columns with default values, but that doesn't seem to resolve this. Any recommendations on the most concise way to handle this?
By the way, jOOQ has been fantastic to work with so far. Lukas, thank you for this awesome tool!
UPDATE #1:
The "not null issue" is addressed by Lukas's answer below, and that's an easy fix.
For the duplicate primary keys, I am definitely not confusing INSERT with UPDATE. When I run the above code (slight update since original post), jOOQ seems to arbitrarily pick a "starting" primary key value for OrgPersonId. For example, when I first load up my environment, jOOQ might start with "11" for OrgPersonId.
Then, when I do an INSERT, jOOQ will attempt to supply a value of "11" for OrgPersonId, I'll get the ERROR: duplicate key value and the INSERT will fail. If I then repeat the INSERT, jOOQ uses "12", then "13". It succeeds or fails based on whether that ID is available, but it's not "starting" with the right ID.
The manual (http://www.jooq.org/doc/3.4/manual/sql-execution/crud-with-updatablerecords/identity-values/) says that If you're using jOOQ's code generator, the above table will generate a org.jooq.UpdatableRecord with an IDENTITY column. This information is used by jOOQ internally, to update IDs after calling store().
UPDATE #2:
Ok, I just tried the generated query directly in Postgres and it fails there, too, with the same issue. So, clearly this is a Postgres issue and not a jOOQ issue. I'll post the final resolution on that when I find it in case anyone else runs into this.
UPDATE #3:
Issue has been resolved. We use FlywayDB (another awesome tool) to automate our database schema migration, and we had a bunch of INSERT statements in our Flyway scripts that manually INSERTED the id number. This was convenient because we wanted to create a bunch of dummy data and wanted to guarantee the right foreign key relationships.
But manually specifying the primary key increment does not advance the Postgres sequence! Hence, we had to cycle through the Postgres sequence before (correctly operating) jOOQ would get the right sequence value.
Solution is to remove all our manual inserts of the primary keys in our demo data migration scripts.
violates not-null constraint
The first part that you're describing is a flaw (#3582), which is related to a previous issue (#2700), which enforced storing null values loaded from POJOs into jOOQ Records for database columns that are NOT NULL. The fix will be in jOOQ 3.5.0, 3.4.3, 3.3.4, and 3.2.7
duplicate key value violates unique constraint "pk_org_person"
The second part probably is caused by the fact that you are really loading an existing record and then calling executeInsert() on it (observe the INSERT, which will always execute an INSERT statement). You might want to call executeUpdate(), instead
I've got a DynamoDB table with a an alpha-numeric string as a hash key (e.g. "d4ed6962-3ec2-4312-a480-96ecbb48c9da"). I need to query the table based on another field in the table, hence I need my query to select all the keys such as my field x is between dat x and date y.
I know I need a condition on the hash key and another on a range key, however I struggle to compose a hash key condition that does not bind my query to specific IDs.
I thought I could get away with a redundant condition based on the ID being NOT_NULL, but when I use it I get the error:
Query key condition not supported
Below is the conditions I am using, any idea how to achieve this goal?
Condition hashKeyCondition = new Condition()
.withComparisonOperator(ComparisonOperator.NOT_NULL.toString());
Condition rangeCondition = new Condition()
.withComparisonOperator(ComparisonOperator.BETWEEN.toString())
.withAttributeValueList(new AttributeValue().withS(dateFormatter.print(lastScanTime())),
new AttributeValue().withS(dateFormatter.print(currentScanTime)));
Map<String, Condition> keyConditions = new HashMap<String, Condition>();
keyConditions.put("userId", hashKeyCondition);
keyConditions.put("lastAccesTime", rangeCondition);
Thanks in advance to everyone helping.
In DynamoDB you can get items with 3 api:
. Scan (flexible but expensive),
. Query (less flexible: you have to specify an hash, but less expensive)
. GetItem (by Hash and, if your table has one, by range)
The only way to achieve what you want is by either:
Use Scan, and be slow or expensive.
Use another table (B) as an index to the previous one (A) like:
B.HASH = 'VALUES'
B.RANGE = userid
B.lastAccesTime = lastAccesTime (with a secondary index)
Now you have to maintain that index on writes, but you can use it with the Query operation,
to get your userIds. Query B: hash='VALUES', lastaccessTime between x and y, select userid.
Hope this helps.
The NOT_NULL comparison operator is not valid for the hash key condition. The only valid operator for the Hash key condition on a query is EQ. More information can be found here:
http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html
And what this means is that a query will not work, at least as your table is currently constructed. You can either use a Scan operation or you can create a separate table that stores the data by Date (hash) and User ID (range).
Good luck!
I ended up scanning the table and enforcing a filter.
Thanks to everyone taking time for helping out!
You could add Global Secondary Index with, for example, year and month of your date and make it your hash key, range key for that index would be your date then you could query any data range in a certain month. It will help you avoid expensive full scan.
E.g.
Global Secondary Index:
Hash key: month_and_year for example '2014 March'
Range key: full_date
Hope it helps!
You need to create GSI if you want to query other than Partition Key. Scan is very expensive in terms of cost and performance.