Im trying to store a list,collection of data objects in Hbase. For example ,a User table where a the userId is the Rowkey and column family Contacts with column Contacts:EmailIds where EmailIds is a list of emails as
{abcd#example.com,bpqrs#gmail.com....etc}
How do we model this in Hbase ? How do we do this in Java?/Python?Ive tried pickling and unpickling data in Python but this is one solution which I do not want to use due to performance issues.
You can use it in the following manner:
| userid | contacts |
| test | c:email1=test#example.com; c:email2=te.st#example.com |
or
| userid | contacts |
| test | c:test#example.com=1; c:te.st#example.com=2 |
This way you can use versioning, add/remove as much email addresses as you want, use filters, and it is really easy to iterate over these KV pairs in the client code
Related
I am using Microsoft SQL Server with already stored data.
In one of my tables I can find data like:
+--------+------------+
| id | value |
+--------+------------+
| 1 | 12-34 |
| 2 | 5678 |
| 3 | 1-23-4 |
+--------+------------+
I realized that the VALUE column was not properly formatted when inserted.
What I am trying to achieve is to get id by given value:
SELECT d.id FROM data d WHERE d.value = '1234';
Is there any way to format data in column just before SELECT clause?
Should I create new view and modify column in that view or maybe use complicated REGEX to get only digits (with LIKE comparator)?
P.S. I manage database in Jakarta EE project using Hibernate.
P.S.2. I am not able to modify stored data.
One method is to use replace() before the comparison:
WHERE REPLACE(d.value, '-', '') = '1234'
I need to design a Table in Oracle/SQL & data will be upload via Java/C# application via CSV with 50 fields (mapped to columns of Table). How to design Table/DB with below constraints during data importing from CSV
CSV may have new fields being added to existing 50 fields.
In that case instead of adding column to table manually & load data. How can we design table for smooth/automatic file handling with dynamic fields
EX:
CSV has S_ID, S_NAME, SUBJECT, MARK_VALUE fields in it
+------+---------+-------------+------------+
| S_ID | S_NAME | SUBJECT | MARK_VALUE |
+------+---------+-------------+------------+
| 1 | Stud | SUB_1 | 50 |
| 2 | Stud | SUB_2 | 60 |
| 3 | Stud | SUB_3 | 70 |
+------+---------+-------------+------------+
What if CSV has new field "RANK" (similar more fields) added to it & i need to store all new fields in Table.
Please suggest DB design for this consideration
So there are few approaches come to my mind, one of the way would be having metadata(Record) information in one table (column name, data type, any constraint) and have another free form table with large enough no. of columns which will hold the data. Use the metadata table while inserting data into this table to maintain data integrity and other stuff.
Suppose I have the following tables, in an Oracle DB
Foo:
+--------+---------+---------+
| id_foo | string1 | string2 |
+--------+---------+---------+
| 1 | foo | bar |
| 2 | baz | bat |
+--------+---------+---------+
Bar:
+--------+-----------+--------+
| id_bar | id_foo_fk | string |
+--------+-----------+--------+
| 1 | 1 | boo |
| 2 | 1 | bum |
+--------+-----------+--------+
When I insert into Foo, by using a Dataset and JDBC, such as
Dataset<Row> fooDataset = //Dataset is initialized
fooDataset.write().mode(SaveMode.Append).jdbc(url, table, properties)
an ID is auto-generated by the database. Now when I need to save Bar, using the same strategy, I want to be able to link it to Foo, via id_foo_fk.
I looked into some possibilities, such as using monotonically_increasing_id() as suggested in this question, but it won't solve the issue, as I need the ID generated by the database. I tried what was suggested in this question, but it leads to the same issue, of unique non-database IDs
It's also not possible to select from the JDBC again, as string1 and string2 may not be unique. Nor is it possible to change the database. For instance, I can't change it to be UUID, and I can't add a trigger for it. It's a legacy database that we can only use
How can I achieve this? Is this possible with Apache Spark?
I'm not a Java specialist so you will have to look into the database layer on how to proceed exactly but there are 3 ways you can do this:
You can create a store procedure if the database server you are using is capable of (most do) and call it from your code.
Create a trigger that returns the id number on the first insertion and use it in your next DB insertion.
Use UUID and use this as the key instead of the database auto generated key.
Is it possible index a complete database without mentioning the table names explicitly in the data-config.xml as new tables are added everyday and I cannot change the data-config.xml everyday to add new tables.
Haven table names based on the date smells like there is something wrong in your Design. But given this requirement in your question you can add Data to your solr server without telling you have a DB. You just have to make sure you hav a unique ID for the data record in you solr Server with whom you can identify the corresponding record in your DB, something like abcd_2011_03_19.uniqueid. You can post the data to solr in Java in solrj or just plain xml or json.
Example:
--------------
| User Input |
--------------
|post
V
-----------------------------------
| My Backend (generate unique id) |
-----------------------------------
|post(sql) |post (e.g. solrj)
V V
------ --------
| DB | | solr |
------ --------
My ascii skillz are mad :D
I have a project with a jTable called AchTable, that is like this:
+-------+------+
| File | Type |
+-------+------+
| | |
| | |
| | |
+--------------+
And I have a mySQL table that is like the same, then I want to know how could I populate the jTable.
So what is the problem, creating a table or creating an SQL query?
Read the section from the Swing tutorial on How to Use Tables.
Read the tutorial on JDBC Database Access.
Put the two together and you've got your problem solved. That is first create your query and create a ResultSet. Then you use the meta data to get the column names. Then you loop through the ResultSet and add rows of data to your table. You can use a DefaultTableModel for this.