I am trying to send a transaction through the Java driver in my spring application.
The following is the simplified code.
#Test
public void rawTransactionTest(){
var appContext = new AnnotationConfigApplicationContext(DataLoaderApplication.class);
var arangoOperations = appContext.getBean(ArangoOperations.class);
String action = "function(){\n" +
" db = require(\"#arangodb\").db; \n" +
"db._query(\"LET doc = {title: \\\"Hello\\\"} "+
"UPSERT { _key: doc._key } INSERT doc._key == null ? UNSET(doc, \\\"_key\\\") : doc " +
"REPLACE doc IN Books OPTIONS { ignoreRevs: false } RETURN NEW\");\n" +
"return \"Success\"; \n" +
"}";
System.out.println(action);
var tOpts = new TransactionOptions();
tOpts.writeCollections("Books");
tOpts.waitForSync(true);
var result = arangoOperations.driver().db().transaction(action, String.class, tOpts);
System.out.println("Commit");
}
This returns the return value "Success" in the variable result. But the database remains unchanged. Doing the same thing in ArangoShell works perfectly fine. The ArangoShell code is as follows -
db._executeTransaction({
collections: {
write: ["Books"]
},
action: function(){
db = require("#arangodb").db;
db._query("LET doc = {title: \"Hello\"} UPSERT { _key: doc._key } "+
"INSERT doc._key == null ? UNSET(doc, \"_key\") : doc REPLACE doc"+
" IN Books OPTIONS { ignoreRevs: false } RETURN NEW");
return "Success";
}
});
This code works fine from the shell. Other non-transaction queries work fine from he same Spring-container.
What might be the problem?
The .db() only points to the _system database. Had to pass the database name to fix it.
Related
1.Development environment:
IDEA,ORACLE 12g,Mybatis
2.The code where has this question
#Transactional(rollbackFor = Exception.class,propagation = Propagation.REQUIRED )
#Override public void syncSmartGateOrgList() {
log.debug("syncSmartGateOrgList() called");
//remove all of the current data if it's debug
if (GlobalProperties.isDebug()) {
int deleteLocalOrgResults = mSmartGateOrgServiceI.getDao().deleteByEntity(SmartGateOrg.builder().build());
log.info("syncSmartGateOrgList deleteLocalOrgResults = " + deleteLocalOrgResults);
}
//validate params
ValidateParams validateParams = SmartGateSignTool.getValidateParams(SmartGateConstants.getPlatformToken());
log.info("syncSmartGateOrgList validateParams = " + validateParams);
//get org list
OrganizationResult organizationList = mSmartGateManager.getOrganizationList(SmartGateBody.builder().unitid(SmartGateConstants.getPlatformUnitId()).build(), validateParams.getSignature(), validateParams.getTimestamp(), validateParams.getNonce());
//then import the data to the db
if (Objects.equals(organizationList.getErrmsg(), OK)) {
List<OrganizationResult.UnitsBean> units = organizationList.getUnits();
log.info("syncSmartGateOrgList origin units = " + units.size());
List<SmartGateOrg> collect = units.parallelStream().map(unitsBean -> {
SmartGateOrg smartGateOrg = new SmartGateOrg();
smartGateOrg.setUnitName(unitsBean.getUnitname());
smartGateOrg.setUnitId(unitsBean.getUnitid());
String unitPath = Joiner.on(",").join(unitsBean.getUnitpath());
smartGateOrg.setUnitPath(unitPath);
List<OrganizationResult.UnitsBean.ParentunitsBean> parentunits = unitsBean.getParentunits();
smartGateOrg.setParentId(!Emptys.isEmpty(parentunits) ? parentunits.get(0).getUnitid() : "");
smartGateOrg.setParentOrder(!Emptys.isEmpty(parentunits) ? String.valueOf(parentunits.get(0).getOrder()) : "");
smartGateOrg.setParentPriority(!Emptys.isEmpty(parentunits) ? String.valueOf(parentunits.get(0).getPriority()) : "");
smartGateOrg.setGovWxId(!Emptys.isEmpty(Optional.ofNullable(unitsBean.getExtend()).map(OrganizationResult.UnitsBean.ExtendBean::getWeworkpartyid)) ? String.valueOf(unitsBean.getExtend().getWeworkpartyid().get(0)) : "");
smartGateOrg.setDeptType(!Emptys.isEmpty(unitsBean.getSystemtype()) ? unitsBean.getSystemtype().get(0) : "");
smartGateOrg.setCreateBy("timer");
return smartGateOrg;
})
//check the data if it's existed before insert
.filter(smartGateOrg ->{
//fixme the data still could be found whatever the all of the data have been delete on the above
SmartGateOrg localEntity = mSmartGateOrgServiceI.getByEntity(SmartGateOrg.builder().unitId(smartGateOrg.getUnitId()).build());
// SmartGateOrg localEntity = mSmartGateOrgServiceI.getDao().selectOne(SmartGateOrg.builder().unitId(smartGateOrg.getUnitId()).build());
boolean aNull = Objects.isNull(localEntity);
if (!aNull) {
log.info("syncSmartGateOrgList localEntity = " + localEntity);
}
log.info("syncSmartGateOrgList aNull = " + aNull);
return aNull;
})
.collect(Collectors.toList());
log.info("syncSmartGateOrgList after filter units = " + collect.size());
if (Emptys.isNotEmpty(collect)) {
int result = mSmartGateOrgServiceI.addBatch(collect);
log.info("syncSmartGateOrgList() returned: " + result);
}
} else {
log.warn("syncSmartGateOrgList organizationList = " + organizationList);
}
}
3.The sql which be used here:
<!--delete data by condition-->
<delete id="deleteByEntity">
delete from T_SMART_GATE_ORG
<where>
<include refid="whereConditionFull"/>
</where>
</delete>
<!--get entity by condition-->
<select id="getByEntity" resultMap="SmartGateOrgResultMap"
parameterType="com.zx.lodging.modules.pojo.entity.smartgate.SmartGateOrg" >
select
<include refid="table_field"/>
from T_SMART_GATE_ORG
<where>
<include refid="whereConditionFull"/>
</where>
</select>
4.The issues:
a.first, the all data actually be deleted in this transaction becasue of the result returned was expected
b. not all of the data could still be found,most of the data was not found,and some of the data was still could be found.that's the problem is.
c. the question sql was actually executed and return nothing.The problem is that the 'selected' on the code still returns the data.I don't know if it's the cache problem
After I have debug for a while,I found this question was cause by the parallelStream where from the java1.8.I'm not sure what the root cause for this question.
But when I replace the parallelStream with "stream" or "parallelStream.sequential",this problem is solved.
My XPage gathers information which I use to populate a document in a different Domino database. I use a link button (so I can open another XPage after submission). The onClick code is as follows:
var rtn = true
var util = new utilities()
var hostURL = configBean.getValue("HostURL");
var userAttachment;
//set up info needed for checking duplicates
var attachName=getComponent("attachmentIdentifier").getValue();
var serialNbr = getComponent("serialNumber").getValue();
userAttachment = user+"~"+attachName;
var userSerial = user+"~"+serialNbr;
//Done setting info needed
//check for duplicates
rtn = utilBean.checkAttachmentName(userAttachment, userSerial)
//done
if(rtn==true){
var doc:Document = document1;
dBar.info("ALL IS GOOD");
var noteID:String=document1.getNoteID();
dBar.info("Calling saveNewAttachment using NoteID " + noteID )
rtn=utilBean.saveNewAttachment(session,noteID ); //<<< I get error here
dBar.info("rtn = " + rtn)
return "xsp-success";
view.postScript("window.open('"+sessionScope.nextURL+"')")
}else if (rtn==false){
errMsgArray = utilBean.getErrorMessages();
for(err in errMsgArray){
//for (i=0; i < errMsgArray.size(); i++){
dBar.info("err: "+ err.toString());
if (err== "nameUsed"){
//send message to XPXage
facesContext.addMessage(attachmentIdentifier.getClientId(facesContext) , msg(langBean.getValue("duplicateName")));
}
if(err=="serialUsed"){
//send message to XPXage
facesContext.addMessage(serialNumber.getClientId(facesContext) , msg(langBean.getValue("duplicateSerial")));
}
}
return "xsp-failure";
}
And the java code that delivers the error is this
public boolean saveNewAttachment(Session ses, String noteID)
throws NotesException {
debugMsg("Entering saveNewAttachment and NOTEID = "+noteID);
// this is used when the user saves an attachment to to the
// user profiles db
boolean rtn = false;
Document doc;
ConfigBean configBean = (ConfigBean)
ExtLibUtil.resolveVariable(FacesContext.getCurrentInstance(),
"configBean");
String dbName = (String) configBean.getValue("WebsiteDbPath");
debugMsg("A");
Database thisDB = ses.getDatabase(ses.getServerName(), dbName, false);
String value;
try {
debugMsg("noteID: "+noteID);
The next line throws the NotesException error
doc = thisDB.getDocumentByID("noteID");
debugMsg("C");
} catch (Exception e) {
debugMsg("utilitiesBean.saveAttachment: " + e.toString());
e.printStackTrace();
System.out.println("utilitiesBean.saveAttachment: " + e.toString());
throw new RuntimeException("utilitiesBean.saveAttachment: "
+ e.toString());
}
return rtn;
}
I might be going about this wrong. I want to save the document which the data is bound to the User Profile database but if I submit it I need to redirect it to a different page. That is why I am using a link, however, I am having a hard time trying to get the document saved.
Has document1 been saved before this code is called? If not, it's not in the backend database to retrieve via getDocumentByID().
I'm assuming this line has been copied into here incorrectly, because "noteID" is not a NoteID or a variable holding a NoteID, it's a string.
doc = thisDB.getDocumentByID("noteID");
I'm using Java to download HTML contents of websites whose URLs are stored in a database. I'd like to put their HTML into database, too.
I'm using Jsoup for this purpose:
public String downloadHTML(String byLink) {
String htmlInPage = "";
try {
Document doc = Jsoup.connect(byLink).get();
htmlInPage = doc.html();
} catch (org.jsoup.UnsupportedMimeTypeException e) {
// process this and some other exceptions
}
return htmlInPage;
}
I'd like to download websites concurrently and use this function:
public void downloadURL(int websiteId, String url,
String categoryName, ExecutorService executorService) {
executorService.submit((Runnable) () -> {
String htmlInPage = downloadHTML(url);
System.out.println("Category: " + categoryName + " " + websiteId + " " + url);
String insertQuery =
"INSERT INTO html_data (website_id, html_contents) VALUES (?,?)";
dbUtils.query(insertQuery, websiteId, htmlInPage);
});
}
dbUtils is my class based on Apache Commons DbUtils. Details are here: http://pastebin.com/iAKXchbQ
And I'm using everything mentioned above in a such way: (List<Object[]> details are explained on pastebin, too)
public static void main(String[] args) {
DbUtils dbUtils = new DbUtils("host", "db", "driver", "user", "pass");
List<String> categoriesList =
Arrays.asList("weapons", "planes", "cooking", "manga");
String sql = "SELECT lw.id, lw.website_url, category_name " +
"FROM list_of_websites AS lw JOIN list_of_categories AS lc " +
"ON lw.category_id = lc.id " +
"where category_name = ? ";
ExecutorService executorService = Executors.newFixedThreadPool(10);
for (String category : categoriesList) {
List<Object[]> sitesInCategory = dbUtils.select(sql, category );
for (Object[] entry : sitesInCategory) {
int websiteId = (int) entry[0];
String url = (String) entry[1];
String categoryName = (String) entry[2];
downloadURL(websiteId, url, categoryName, executorService);
}
}
executorService.shutdown();
}
I'm not sure if this solution is correct but it works. Now I want to modify code to save HTML not from all websites in my database, but only their fixed ammount in each category.
For example, download and save HTML of 50 websites from the "weapons" category, 50 from "planes", etc. I don't think it's necessary to use sql for this purpose: if we select 50 sites per category, it doesn't mean we save them all, because of possibly incorrect syntax and connection problems.
I've tryed to create separate class implementing Runnable with fields: counter and maxWebsitesPerCategory, but these variables aren't updated. Another idea was to create field Map<String,Integer> sitesInCategory instead of counter, put each category as a key there and increment its value until it reaches maxWebsitesPerCategory, but it didn't work, too. Please, help me!
P.S: I'll also be grateful for any recommendations connected with my realization of concurrent downloading (I haven't worked with concurrency in Java before and this is my first attempt)
How about this?
for (String category : categoriesList) {
dbUtils.select(sql, category).stream()
.limit(50)
.forEach(entry -> {
int websiteId = (int) entry[0];
String url = (String) entry[1];
String categoryName = (String) entry[2];
downloadURL(websiteId, url, categoryName, executorService);
});
}
sitesInCategory has been replaced with a stream of at most 50 elements, then your code is run on each entry.
EDIT
In regard to comments. I've gone ahead and restructured a bit, you can modify/implement the content of the methods I've suggested.
public void werk(Queue<Object[]> q, ExecutorService executorService) {
executorService.submit(() -> {
try {
Object[] o = q.remove();
try {
String html = downloadHTML(o); // this takes one of your object arrays and returns the text of an html page
insertIntoDB(html); // this is the code in the latter half of your downloadURL method
}catch (/*narrow exception type indicating download failure*/Exception e) {
werk(q, executorService);
}
}catch (NoSuchElementException e) {}
});
}
^^^ This method does most of the work.
for (String category : categoriesList) {
Queue<Object[]> q = new ConcurrentLinkedQueue<>(dbUtils.select(sql, category));
IntStream.range(0, 50).forEach(i -> werk(q, executorService));
}
^^^ this is the for loop in your main
Now each category tries to download 50 pages, upon failure of downloading a page it moves on and tries to download another page. In this way, you will either download 50 pages or have attempted to download all pages in the category.
I am using the FilterTable addon of vaadin.
I am getting a NullPointerException on the following code, but unable to find the reason of this.
cont = new IndexedContainer()
cont.addContainerProperty("Patient", String.class, null);
cont.addContainerProperty("Date Of Service", String.class,null);
cont.addContainerProperty("Provider", String.class, null);
Session session = HibernateUtil.getSessionFactory().openSession();
Iterator<?> iterator = session.createQuery("FROM ConvertVisitToBillingV WHERE ready_for_billing = '0'").list().iterator();
while(iterator.hasNext()){
ConvertVisitToBillingV var = (ConvertVisitToBillingV) iterator.next();
Visits v = (Visits) session.load(Visits.class, var.getVisitId());
Appointments app = (Appointments)session.load(Appointments.class, v.getAppointmentId());
t_id= var.getVisitId();
cont.addItem(t_id);
Resources res = (Resources)session.load(Resources.class, v.getReferredBy());
cont.getContainerProperty(t_id, "Patient").setValue(var.getFirstName() + " " + var.getLastName());
cont.getContainerProperty(t_id, "Date Of Service").setValue(new SimpleDateFormat("MM/dd/yyyy").format(v.getVisitDt()));
cont.getContainerProperty(t_id, "Provider").setValue(res.getResourceFirstName()+" "+res.getResourceLastName());
}
When it executes the line "cont.getContainerProperty(t_id,property).setValue()
It occasionally throws NullPointerException. Not getting the reason behind it.
What can be the reason behind this , any help!
Thanks!
Without more details I would say either :
v.getVisitDt() is null for some `v``
session.load(Resources.class, v.getReferredBy()); returns null for some v and so res is null.
This will probably fix the problem:
cont.getContainerProperty(t_id, "Patient").setValue(var.getFirstName() + " " + var.getLastName());
if(v.getVisitDt()!=null){
cont.getContainerProperty(t_id, "Date Of Service").setValue(new SimpleDateFormat("MM/dd/yyyy").format(v.getVisitDt()));
} else {
cont.getContainerProperty(t_id, "Date Of Service").setValue("?");
}
if(res!=null){
cont.getContainerProperty(t_id, "Provider").setValue(res.getResourceFirstName()+" "+res.getResourceLastName());
else{
cont.getContainerProperty(t_id, "Provider").setValue("?");
}
I am trying to fetch page content with phantomjs. In many examples on the official site (eg.: https://github.com/ariya/phantomjs/blob/master/examples/imagebin.js) the function page.open() is used.
In my script though it does not seem to work. I used reflection to look at all defined methods of the page object:
for ( var prop in page) {
if (typeof page[prop] == 'function') {
log("method in page: " + prop);
}
}
and the open() method did not show up. (close(), render(), etc... did show up)
also when I am trying to execute a script:
// include plugins
var system = require('system');
var fileSystem = require('fs');
var page = require('webpage').create();
// global errorhandler
phantom.onError = function(msg, trace) {
console.log("ERROR!!!!! \n" + msg);
phantom.exit(1);
};
// read json input and remove single outer quotes if set
var jsonin = system.args[1];
if (jsonin.charAt(0) == "'") {
jsonin = jsonin.substr(1, jsonin.length - 2);
}
// make object of json
var data = eval('(' + jsonin + ')');
// optional url
var url = system.args[2];
// transfer file
var dest = system.args[3];
console.log("systemargs[1]: data -> " + data);
console.log("systemargs[2]: url -> " + url);
console.log("systemargs[3]: dest -> " + dest);
openRoot();
/*
* open site
*/
function openRoot() {
page.onConsoleMessage = function(msg) {
console.log('INNER ' + msg);
};
page.open(url, function(status) {
if (status === "success") {
if (loadCount == 0) { // only initial open
console.log("opened successfully.");
page.injectJs("./jquery-1.8.3.min.js");
} else {
console.log("page open error.");
console.log('skip refresh ' + loadCount);
}
} else {
console.log("error opening: " + status);
}
});
}
phantom.exit(0);
it does not execute the open function. The log does not show any messages inside the open() method.
Any advice on what I might do wrong would be greatly appreciated. If there is additional information required, please let me know.
Regards,
Alex
Edit:
The line
console.log(typeof (page.open));
outputs: function which is not what I expected, given the previous list of methods I wrote to the log, where open does not exist. Hmm.
After hours of senseless searching I found the mistake. Stupid me. At the end of the script I call phantom.exit() where I should not.
The working code includes an Interval which checks on an object, in my case content and a member of that content.isFinished. If I set this to true, then phantom.exit() gets called.
My bad, absolutely my fault.
Working code:
var url = system.args[2];
// transfer file
var dest = system.args[3];
content = new Object();
content.isFinished = false;
console.log("systemargs[1]: data -> " + data);
console.log("systemargs[2]: url -> " + url);
console.log("systemargs[3]: dest -> " + dest);
openRoot();
/*
* open site
*/
function openRoot() {
page.onConsoleMessage = function(msg) {
console.log('INNER ' + msg);
};
page.open(url, function(status) {
if (status === "success") {
if (loadCount == 0) { // only initial open
console.log("opened successfully.");
page.injectJs("./jquery-1.8.3.min.js");
// do stuff
content.isFinished = true;
} else {
console.log("page open error.");
console.log('skip refresh ' + loadCount);
content.isFinished = true
}
} else {
console.log("error opening: " + status);
}
});
}
/*
* wait for completion
*/
var interval = setInterval(function() {
if (content.isFinished) {
page.close();
f = fileSystem.open(dest, "w");
f.writeLine(out);
f.close();
// exit phantom
phantom.exit();
} else {
console.log('not finished - wait.');
}
}, 5000);
Regards,
Alex