I'm using the Object API with OrientDB, and I need to change one of my fields. I currently have something like:
class Book { String author }
But I want to change it to:
class Book { List<String> authors }
My question is: how do I persist this list of Strings in OrientDB? Do I have to declare the list as #Embedded? Do I have to define the Schema as LINKLIST?
I tried the latter, which resulted in:
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to com.orientechnologies.orient.core.db.record.OIdentifiable
And, if I make the type in the database an Embbed, then it results in the error:
Caused by: java.lang.IllegalArgumentException: Type EMBEDDED must be a multi value type (collection or map)
Which doesn't provide to much information unfortunately.
So, how can I best fix this?
Start point:
I created a new db from the Java API, and I saved a list of authors.
CLASS BOOK
public class Book {
private List<String> authors;
public void setAuthors (List<String> pAuthors){
this.authors = pAuthors;
}
public List<String> getAuthors(){
return this.authors;
}
}
CLASS MAIN
public class DatabaseTipoObject {
private static String remote="remote:localhost/";
public static void main(String[] args) {
String nomeDb="Object";
String path=remote+nomeDb;
try {
OServerAdmin serverAdmin = new OServerAdmin(path).connect("root", "root");
if(serverAdmin.existsDatabase()){
System.out.println("Database '"+nomeDb +"' exist..");
}
else{
serverAdmin.createDatabase(nomeDb, "object", "plocal");
System.out.println(" Database '"+nomeDb +"' created!..");
}
OObjectDatabaseTx db = new OObjectDatabaseTx (path);
db.open("root","root");
db.getEntityManager().registerEntityClass(Book.class);
Book book = db.newInstance(Book.class);
List<String> myAuthors = new ArrayList();
myAuthors.add("Archimede");
myAuthors.add("Pitagora");
book.setAuthors(myAuthors);
db.save(book);
System.out.println("Data inserted!" );
//get info by query
for (Book book_retrive : db.browseClass(Book.class)) {
System.out.println("#: " +book_retrive.getAuthors().get(0) );
System.out.println("#: " +book_retrive.getAuthors().get(1) );
}
db.close();
serverAdmin.close();
} catch (IOException e) {
e.printStackTrace();
}
}
From Studio:
the object 'Book' was created, and it has the field 'authors' as embeddedlist. (Created automatically)
Related
I have written a code which fetched the S3 objects from AWS s3 using S3 sdk and stores the same in our DB, the only problem is the task is repeated for three different services, the only thing is changed is the instance of service class.
I have copy and pasted code in each service layer just to changes the instance for an instance.
The task is repeated for service classes VehicleImageService, MsilLayoutService and NonMsilLayoutService, every layer is having its own repository.
I am trying to identify a way to accomplish the same by placing that snippet in one place and on an runtime using Reflection API I wish to pass the correct instance and invoke the method, but I want to achieve the same using best industry practices and pattern. I.e. I want to refactor into generic methods for other services, so instance can be passed at runtime.
So kindly assist me on the same.
public void persistImageDetails() {
log.info("MsilVehicleLayoutServiceImpl::persistImageDetails::START");
String bucketKey = null; //common param
String modelCode = null;//common param
List<S3Object> objList = new ArrayList<>(); //common param
String bucketName = s3BucketDetails.getBucketName();//common param
String bucketPath = s3BucketDetails.getBucketPrefix();//common param
try {
//the layoutRepository object can be MSILRepository,NonMSILRepository and VehilceImageRepository
List<ModelCode> modelCodes = layoutRepository.findDistinctAllBy(); // this line need to take care of
List<String> modelCodePresent = modelCodes.stream().map(ModelCode::getModelCode)
.collect(Collectors.toList());
List<CommonPrefix> allKeysInDesiredBucket = listAllKeysInsideBucket(bucketName, bucketPath);//common param
synchDB(modelCodePresent, allKeysInDesiredBucket);
if (null != allKeysInDesiredBucket && !allKeysInDesiredBucket.isEmpty()) {
for (CommonPrefix commonPrefix : allKeysInDesiredBucket) {
bucketKey = commonPrefix.prefix();
modelCode = new File(bucketKey).getName();
if (modelCodePresent.contains(modelCode)) {
log.info("skipping iteration for {} model code", modelCode);
continue;
}
objList = s3Service.getBucketObjects(bucketName, bucketKey);
if (null != objList && !objList.isEmpty()) {
for (S3Object object : AppUtil.skipFirst(objList)) {
saveLayout(bucketName, modelCode, object);
}
}
}
}
log.info("MSIL Vehicle Layout entries has been successfully saved");
} catch (Exception e) {
log.error("Error occured", e);
e.printStackTrace();
}
log.info("MsilVehicleLayoutServiceImpl::persistImageDetails::END");
}
private void saveLayout(String bucketName, String modelCode, S3Object object) {
log.info("Inside saveLayout::Start preparing entity to persist");
String resourceUri = null;
MsilVehicleLayout vehicleLayout = new MsilVehicleLayout();// this can be MsilVehicleLayout. NonMsilVehicleLayout, VehicleImage
vehicleLayout.setFileName(FilenameUtils.removeExtension(FilenameUtils.getName(object.key())));
vehicleLayout.setModelCode(modelCode);
vehicleLayout.setS3BucketKey(object.key());
resourceUri = getS3ObjectURI(bucketName, object.key());
vehicleLayout.setS3ObjectUri(resourceUri);
vehicleLayout.setS3PresignedUri(null);
vehicleLayout.setS3PresignedExpDate(null);
layoutRepository.save(vehicleLayout); //the layoutRepository object can be MSILRepository,NonMSILRepository and VehilceImageRepository
log.info("Exiting saveLayout::End entity saved");
}
I am struggling with importing data into Mongodb from a Json file.
I can do the same in command line by using mongoimport command.
I explored and tried lot but not able to import from Json file using java.
sample.json
{ "test_id" : 1245362, "name" : "ganesh", "age" : "28", "Job" :
{"company name" : "company1", "designation" : "SSE" }
}
{ "test_id" : 254152, "name" : "Alex", "age" : "26", "Job" :
{"company name" : "company2", "designation" : "ML" }
}
Thank for your time.
~Ganesh~
Suppose you can read the JSON string respectively. For example, you read the first JSON text
{ "test_id" : 1245362, "name" : "ganesh", "age" : "28", "Job" :
{"company name" : "company1", "designation" : "SSE" }
}
and assign it to a variable (String json1), the next step is to parse it,
DBObject dbo = (DBObject) com.mongodb.util.JSON.parse(json1);
put all dbo into a list,
List<DBObject> list = new ArrayList<>();
list.add(dbo);
then save them into database:
new MongoClient().getDB("test").getCollection("collection").insert(list);
EDIT:
In the newest MongoDB Version you have to use Documents instead of DBObject, and the methods for adding the object look different now. Here's an updated example:
Imports are:
import com.mongodb.MongoClient;
import com.mongodb.client.MongoDatabase;
import org.bson.Document;
The code would like this (refering to the text above the EDIT):
Document doc = Document.parse(json1);
new MongoClient().getDataBase("db").getCollection("collection").insertOne(doc);
you can also do it the way with the list. but then you need
new MongoClient().getDataBase("db").getCollection("collection").insertMany(list);
But I think there is a problem with this solution. When you type:
db.collection.find()
in the mongo shell to get all objects in the collection, the result looks like the following:
{ "_id" : ObjectId("56a0d2ddbc7c512984be5d97"),
"test_id" : 1245362, "name" : "ganesh", "age" : "28", "Job" :
{ "company name" : "company1", "designation" : "SSE"
}
}
which is not exactly the same as before.
Had a similar "problem" myself and ended up using Jackson with POJO databinding, and Morphia.
While this sound a bit like cracking a nut with a sledgehammer, it is actually very easy to use, robust and quite performant and easy to maintain code wise.
Small caveat: You need to map your test_id field to MongoDB's _id if you want to reuse it.
Step 1: Create an annotated bean
You need to hint Jackson how to map the data from a JSON file to a POJO. I shortened the class a bit for the sake of readability:
#JsonRootName(value="person")
#Entity
public class Person {
#JsonProperty(value="test_id")
#Id
Integer id;
String name;
public Integer getId() {
return id;
}
public void setId(Integer id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
As for the embedded document Job, please have a look at the POJO data binding examples linked.
Step 2: Map the POJO and create a datastore
Somewhere during your application initialization, you need to map the annotated POJO. Since you already should have a MongoClient, I am going to reuse that ;)
Morphia morphia = new Morphia();
morphia.map(Person.class);
/* You can reuse this datastore */
Datastore datastore = morphia.createDatastore(mongoClient, "myDatabase");
/*
* Jackson's ObjectMapper, which is reusable, too,
* does all the magic.
*/
ObjectMapper mapper = new ObjectMapper();
Do the actual importing
Now importing a given JSON file becomes as easy as
public Boolean importJson(Datastore ds, ObjectMapper mapper, String filename) {
try {
JsonParser parser = new JsonFactory().createParser(new FileReader(filename));
Iterator<Person> it = mapper.readValues(parser, Person.class);
while(it.hasNext()) {
ds.save(it.next());
}
return Boolean.TRUE;
} catch (JsonParseException e) {
/* Json was invalid, deal with it here */
} catch (JsonMappingException e) {
/* Jackson was not able to map
* the JSON values to the bean properties,
* possibly because of
* insufficient mapping information.
*/
} catch (IOException e) {
/* Most likely, the file was not readable
* Should be rather thrown, but was
* cought for the sake of showing what can happen
*/
}
return Boolean.FALSE;
}
With a bit of refatcoring, this can be converted in a generic importer for Jackson annotated beans.
Obviously, I left out some special cases, but this would out of the scope of this answer.
With 3.2 driver, if you have a mongo collection and a collection of json documents e.g:
MongoCollection<Document> collection = ...
List<String> jsons = ...
You can insert individually:
jsons.stream().map(Document::parse).forEach(collection::insertOne);
or bulk:
collection.insertMany(
jsons.stream().map(Document::parse).collect(Collectors.toList())
);
I just faced this issue today and solved it in another different way while none here satisfied me, so enjoy my extra contribution. Performances are sufficient to export 30k documents and import them in my Springboot app for integration test cases (takes a few seconds).
First, the way your export your data in the first place matters.
I wanted a file where each line contains 1 document that I can parse in my java app.
mongo db --eval 'db.data.find({}).limit(30000).forEach(function(f){print(tojson(f, "", true))})' --quiet > dataset.json
Then I get the file from my resources folder, parse it, extract lines, and process them with mongoTemplate. Could use a buffer.
#Autowired
private MongoTemplate mongoTemplate;
public void createDataSet(){
mongoTemplate.dropCollection("data");
try {
InputStream inputStream = Thread.currentThread().getContextClassLoader().getResourceAsStream(DATASET_JSON);
List<Document> documents = new ArrayList<>();
String line;
InputStreamReader isr = new InputStreamReader(inputStream, Charset.forName("UTF-8"));
BufferedReader br = new BufferedReader(isr);
while ((line = br.readLine()) != null) {
documents.add(Document.parse(line));
}
mongoTemplate.insert(documents,"data");
} catch (Exception e) {
throw new RuntimeException(e);
}
}
List<Document> jsonList = new ArrayList<Document>();
net.sf.json.JSONArray array = net.sf.json.JSONArray.fromObject(json);
for (Object object : array) {
net.sf.json.JSONObject jsonStr = (net.sf.json.JSONObject)JSONSerializer.toJSON(object);
Document jsnObject = Document.parse(jsonStr.toString());
jsonList.add(jsnObject);
}
collection.insertMany(jsonList);
Runtime r = Runtime.getRuntime();
Process p = null;
//dir is the path to where your mongoimport is.
File dir=new File("C:/Program Files/MongoDB/Server/3.2/bin");
//this line will open your shell in giving dir, the command for import is exactly same as you use mongoimport in command promote
p = r.exec("c:/windows/system32/cmd.exe /c mongoimport --db mydb --collection student --type csv --file student.csv --headerline" ,null,dir);
public static void importCSV(String path) {
try {
List<Document> list = new ArrayList<>();
MongoDatabase db = DbConnection.getDbConnection();
db.createCollection("newCollection");
MongoCollection<Document> collection = db.getCollection("newCollection");
BufferedReader reader = new BufferedReader(new FileReader(path));
String line;
while ((line = reader.readLine()) != null) {
String[] item = line.split(","); // csv file is "" separated
String id = item[0]; // get the value in the csv assign keywords
String first_name = item[1];
String last_name = item[2];
String address = item[3];
String gender = item[4];
String dob = item[5];
Document document = new Document(); // create a document
document.put("id", id); // data into the database
document.put("first_name", first_name);
document.put("last_name", last_name);
document.put("address", address);
document.put("gender", gender);
document.put("dob", dob);
list.add(document);
}
collection.insertMany(list);
}catch (Exception e){
System.out.println(e);
}
}
I am having trouble getting data from a database I know exists and I know the format of.
In the code snippet below the "if conn != null" is just a test to verify the database name, table name, etc are all correct, and they DO verify.
The last line below is what generates the exception
public static HashMap<Integer, String> getNetworkMapFromRemote(DSLContext dslRemote, Connection conn, Logger logger) {
HashMap<Integer,String> remoteMap = new HashMap<Integer, String>();
// conn is only used for test purposes
if (conn != null) {
// test to be sure database is ok
try
{
ResultSet rs = conn.createStatement().executeQuery("SELECT networkid, name FROM network");
while (rs.next()) {
System.out.println("TEST: nwid " + rs.getString(1) + " name " + rs.getString(2));
}
rs.close();
}
catch ( SQLException se )
{
logger.trace("getNetworksForDevices SqlException: " + se.toString());
}
}
// ----------- JOOQ problem section ------------------------
Network nR = Network.NETWORK.as("network");
// THE FOLLOWING LINE GENERATES THE UNKNOWN TABLE
Result<Record2<Integer, String>> result = dslRemote.select( nR.NETWORKID, nR.NAME ).fetch();
This is the output
TEST: nwid 1 name Network 1
org.jooq.exception.DataAccessException: SQL [select `network`.`NetworkId`, `network`.`Name` from dual]; Unknown table 'network' in field list
at org.jooq.impl.Utils.translate(Utils.java:1288)
at org.jooq.impl.DefaultExecuteContext.sqlException(DefaultExecuteContext.java:495)
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:327)
at org.jooq.impl.AbstractResultQuery.fetch(AbstractResultQuery.java:330)
at org.jooq.impl.SelectImpl.fetch(SelectImpl.java:2256)
at com.nvi.kpiserver.remote.KpiCollectorUtil.getNetworkMapFromRemote(KpiCollectorUtil.java:328)
at com.nvi.kpiserver.remote.KpiCollectorUtilTest.testUpdateKpiNetworksForRemoteIntravue(KpiCollectorUtilTest.java:61)
.................
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown table 'network' in field list
.................
For the sake of completness here is part of the JOOQ generated class file for Network
package com.wbcnvi.intravue.generated.tables;
#javax.annotation.Generated(value = { "http://www.jooq.org", "3.3.1" },
comments = "This class is generated by jOOQ")
#java.lang.SuppressWarnings({ "all", "unchecked", "rawtypes" })
public class Network extends org.jooq.impl.TableImpl<com.wbcnvi.intravue.generated.tables.records.NetworkRecord> {
private static final long serialVersionUID = 1729023198;
public static final com.wbcnvi.intravue.generated.tables.Network NETWORK = new com.wbcnvi.intravue.generated.tables.Network();
#Override
public java.lang.Class<com.wbcnvi.intravue.generated.tables.records.NetworkRecord> getRecordType() {
return com.wbcnvi.intravue.generated.tables.records.NetworkRecord.class;
}
public final org.jooq.TableField<com.wbcnvi.intravue.generated.tables.records.NetworkRecord, java.lang.Integer> NWID = createField("NwId", org.jooq.impl.SQLDataType.INTEGER.nullable(false), this, "");
public final org.jooq.TableField<com.wbcnvi.intravue.generated.tables.records.NetworkRecord, java.lang.Integer> NETWORKID = createField("NetworkId", org.jooq.impl.SQLDataType.INTEGER.nullable(false).defaulted(true), this, "");
public final org.jooq.TableField<com.wbcnvi.intravue.generated.tables.records.NetworkRecord, java.lang.String> NAME = createField("Name", org.jooq.impl.SQLDataType.CHAR.length(40).nullable(false).defaulted(true), this, "");
public final org.jooq.TableField<com.wbcnvi.intravue.generated.tables.records.NetworkRecord, java.lang.Integer> USECOUNT = createField("UseCount", org.jooq.impl.SQLDataType.INTEGER.nullable(false).defaulted(true), this, "");
public final org.jooq.TableField<com.wbcnvi.intravue.generated.tables.records.NetworkRecord, java.lang.Integer> NETGROUP = createField("NetGroup", org.jooq.impl.SQLDataType.INTEGER.nullable(false).defaulted(true), this, "");
public final org.jooq.TableField<com.wbcnvi.intravue.generated.tables.records.NetworkRecord, java.lang.String> AGENT = createField("Agent", org.jooq.impl.SQLDataType.CHAR.length(16), this, "");
public Network() {
this("network", null);
}
public Network(java.lang.String alias) {
this(alias, com.wbcnvi.intravue.generated.tables.Network.NETWORK);
}
..........
Based on the "unknown table" exception I thought there was a problem connected to the wrong database or wrong server, but the console output is correct for a JDBC query.
Any thoughts are appreciated, perhaps something else can be the root cause or the DSLContext is not valid (but I would think that would generate a different exception).
The answer ends up being simple, I did not include the .from() method
Result<Record2<Integer, String>> result = dslRemote.select( nR.NETWORKID, nR.NAME )
.from(nR)
.fetch();
That is why the table was unknown, I never put the from method in.
I have a simple entity class and it is supposed to include unique names on it.
#Entity
class Package {
#PrimaryKey(sequence = "ID")
public Long id;
#SecondaryKey(relate = Relationship.ONE_TO_ONE)
public String name;
private Package() {}
public Package(String name) { this.name = name; }
#Override
public String toString() { return id + " : " + name; }
}
I want to use deferred writing option because of extensive modification. Here is the test i tried and its output.
final String dbfilename = "test01";
new File(dbfilename).mkdirs();
EnvironmentConfig config = new EnvironmentConfig().setAllowCreate(true);
Environment environment = new Environment(new File(dbfilename), config);
StoreConfig storeConfig = new StoreConfig().setAllowCreate(true).setDeferredWrite(true);
EntityStore store = new EntityStore(environment, "", storeConfig);
PrimaryIndex<Long, Package> primaryIndex = store.getPrimaryIndex(Long.class, Package.class);
try {
primaryIndex.put(new Package("package01")); // will be put.
primaryIndex.put(new Package("package01")); // throws exception.
} catch (UniqueConstraintException ex) {
System.out.println(ex.getMessage());
}
store.sync(); // flush them all
// expecting to find one element
SortedMap<Long,Package> sortedMap = primaryIndex.sortedMap();
for (Package entity : sortedMap.values()) {
System.out.println(entity);
}
Output
(JE 5.0.73) Unique secondary key is already present
1 : package01
2 : package01
So my question is that even if it throws exception while putting second package, why does it lists two packages. Any way to avoid this without using transactions?
Thanks.
had several apps with jdbc and Oracle 10g. Now I´m changing the apps for use c3p0. But I have some problems working with Oracle types.
I Have this Oracle type:
CREATE OR REPLACE
TYPE DATAOBJ AS OBJECT
(
ID NUMBER,
NAME VARCHAR2(50)
)
And this Oracle function:
CREATE OR REPLACE FUNCTION F_IS_DATA_OBJECT (datar in DATAOBJ) RETURN varchar2 IS
tmpVar varchar2(150);
BEGIN
tmpVar := 'Data object:';
if datar.id is not null then
tmpVar := tmpVar || 'id=' || datar.ID;
end if;
if datar.name is not null then
tmpVar := tmpVar || 'name=' || datar.name;
end if;
return tmpVar;
EXCEPTION
WHEN NO_DATA_FOUND THEN
NULL;
WHEN OTHERS THEN
RAISE;
END F_IS_DATA_OBJECT;
then I have a app in Java with c3p0 with next classes:
Dataobj.class to represent the object type:
package c3p0pruebas.modelo;
import java.io.Serializable;
import java.sql.SQLData;
import java.sql.SQLException;
import java.sql.SQLInput;
import java.sql.SQLOutput;
public class Dataobj implements SQLData, Serializable {
private String name;
private Integer id;
public Dataobj() {
}
public String getSQLTypeName() {
return "DATAOBJ";
}
public void writeSQL(SQLOutput stream) throws SQLException {
stream.writeInt(id.intValue());
stream.writeString(name);
}
public void readSQL(SQLInput stream, String typeName) throws SQLException {
id = new Integer(stream.readInt());
name = stream.readString();
}
public void setName(String name) {
this.name = name;
}
public String getName() {
return name;
}
... and its gets and sets ....
And the main class and main method:
Connection connection = DBConnectionManager.getInstance().getConnection("Mypool"); //I use a class to get connection
CallableStatement cs = null;
String error = "";
try {
/*
//First I made a NativeExtractor of the connection, but the result is the same, I got it from Spring framework.
//C3P0NativeJdbcExtractor extractor = new C3P0NativeJdbcExtractor();
//OracleConnection newConnection = (OracleConnection) extractor.getNativeConnection(connection);
//cs = (OracleCallableStatement) newConnection.prepareCall("{? = call F_IS_DATA_OBJECT(?)}");
*/
//Creates the object
Dataobj obj = new Dataobj();
obj.setId(new Integer(33));
obj.setName("myName");
cs = connection.prepareCall("{? = call F_IS_DATA_OBJECT(?)}");
cs.registerOutParameter(1, OracleTypes.VARCHAR);
cs.setObject(2, obj);
cs.execute();
error = cs.getString(1);
System.out.println("Result: " + error);
} catch (SQLException e) {
e.printStackTrace();
} catch (Exception ex) {
ex.printStackTrace();
} finally {
closeDBObjects(null,cs,null);
}
closeDBObjects(null, null, connection); //Close connection
The execution gets:
Data object: id=33.
I cant get the String (Varchar2) value, the name string.
With oracle arrays of object type, I have the same problem, It worked nice with JDBC. When I worked with Arrays, also, it hasn´t the string values:
//Here I use a NativeConnection ...
Dataobj arrayOfData[] = new Dataobj[myDataObj.size()];
... //Makes the array of DataObj.
ArrayDescriptor descriptor = ArrayDescriptor.createDescriptor("OBJ_ARRAY", newConnection);
ARRAY arrayDatas = new ARRAY(descriptor, newConnection, arrayOfData);
//In this step, objects of arrayDatas haven´t the name string...
Thanks!!!
OK, It finally works.
Searching, We found out the answer:
We change data definition in the database and now it works:
CREATE OR REPLACE
TYPE "DATAOBJ" AS OBJECT
(
vid NUMBER,
vname NCHAR(50)
)
Thanks!
I had the same problem and i solved without change VARCHAR2 to NCHAR, because for me, the NCHAR doesn't appear the String in the Oracle, stay "?" in all the positions.
I changed the oracle driver of the WAR to the version of my database, in my case was 11.2.0.1.0:
http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-112010-090769.html
And i put another driver, that is the NLS for Oracle Objects and Collections:
http://download.oracle.com/otn/utilities_drivers/jdbc/112/orai18n.jar
With this, i solved the problem and the VARCHAR2 worked fine.
Good luck.