We Keep Coding

Solr export query into CSV File

Hie, I m using java and solr as search engine to export the data into csv format.The sequence of step which i m following
--Based on the input parameter i m making a query Let say Q.
--After that i m creating a java Process p to execute that query.
--The Response which i got from solr server i need to push it into a csv file at any location.
enter code here
String Q = "http://pcam-stg-app-03:9999/solr/brm-royalty/select?
q=VENDOR_NAME:\"SOME_NAME\"&f1=vendor_name,vendor_id&wt=csv";
Process p = Runtime.getRuntime().exec(Q);
p.getInputStream();
This stream i need to write in csv file any idea ?

Beware of just paging through the query to get all results...that is an antipattern. If the query returned many docs, and you want all of them, you need to use cursorMark.

A combination of the CSV response writer and SOLRJ to page through all of the results sending it to something like apache commons fileutils:
FileUtils.writeStringToFile(new File(output.csv), outputLine ("line.separator"), true);
Would be quiet quick to knock up in Java.

Capture generated output file path and name using CSSDK

We are in the process of converting over to using the XSLT compiler for page generation. I have a Xalan Java extention to exploit the CSSDK and capture some meta data we have stored in the Extended Attributes for output to the page. No problems in getting the EA's rendered to the output file.
The problem is that I don't know how to dynamically capture the file path and name of the output file.
So just as POC, I have the CSVPath hard coded to the output file in my Java extension. Here's a code sample:
CSSimpleFile sourceFile = (CSSimpleFile)client.getFile(new CSVPath("/some-path-to-the-output.jsp"));
Can someone point me in the CSSDK to where I could capture the output file?

I found the answer.
First, get or create your CSClient. You can use the examples provided in the cssdk/samples. I tweaked one so that I captured the CSClient in the method getClientForCurrentUser(). Watch out for SOAP vs Java connections. In development, I was using a SOAP connection and for the make_toolkit build, the Java connection was required for our purposes.
Check the following snippet. The request CSClient is captured in the static variable client.
CSSimpleFile sourceFile = (CSSimpleFile)client.getFile(new CSVPath(XSLTExtensionContext.getContext().getOutputDirectory().toString() + "/" + XSLTExtensionContext.getContext().getOutputFileName()));

Using PHP to supply an in memory pdf from a Java generator

I am creating a web front end for clients to download their reports. The program that the company uses is written in Java and consists of 31 mysql tables with the 4 reports needed having over 140 points of data each. I am making the website in PHP and report generator in Java but I am having an issue finding the information on how to get the in memory PDF to load directly in the clients browser. I figured on using a TCP client/server going from Java to PHP but how do I code it so that it doesn't have to be written to the server drive and be supplied as a link. I also have no wish to rewrite 17,000 lines of Java to PHP as I am new to PHP. Is there anyway to get this done?
Thank you in advance.

Depending on how long the java app would take to run, you could consider using proc_open to create a pipe to the java programme:
<?php
$desc = array(array('pipe','r'),array('pipe','w'));
$resource= proc_open('bash',$desc,$pipes);
if (!is_resource($resource))
{
throw new Exception('PDF stream creation failed');
}
usleep(5);//for safety
fwrite($pipes[0],'java mkPDF paras'."\n");//double quoted \n is crucial
fclose($pipes[0]);
usleep(100);//while java app is running;
$pdf = stream_get_contents($pipes[1]);
fclose($pipes[1]);
proc_close($resource);
?>
This is just a basic example, that gets the streamed pdf in one big lump, so it's far from perfect. What you -IMO- should look into is getting the stream in chunks using a while loop. The php man page contains a couple of examples of this, basically, repleace
usleep(100);//while java app is running;
$pdf = stream_get_contents($pipes[1]);
with:
usleep(10);//while java app is running;
$pdf = '';
while (($pdfBuff = fgets($pipes[1],1024)) !== null)
{
$pdf.=$pdfBuff;//or echo, or whatever
}
fclose($pipes[1]);
The latter is untested, so I'm not sure if that will work in your case... but you get the idea

As said by #Elias directly send web request to java application and stream the pdf to php.
Using web services,
I. Develop some web services on java side that will send only data in any of the format like XML, jSon etc.
II. Write a code to consume these web services and develop your code to generate the pdf and stream these pdf's.
There is one pdf generating lib. Please find the link here

PHP Problem : filesize() return 0 with file containing few data?

I use PHP to call a Java command then forward its result into a file called result.txt. For ex, the file contains this:
"Result is : 5.0"
but the function filesize() returns 0 and when I check by 'ls -l' command it's also 0. Because I decide to print the result to the screen when file size != 0 so nothing is printed. How can I get the size in bit ? or another solution available?

From the docs, when you call filesize, PHP caches this result in the stat cache.
Have you tried clearing the stat cache?
clearstatcache();
If it does not work, possible workaround is to open the file, seek to its end, and then use ftell.
$fp = fopen($filename, "rb");
fseek($fp, 0, SEEK_END);
$size = ftell($fp);
fclose($fp);
If you are actually planning to display the output to the user, you can instead read the entire file and then strlen.
$data = file_get_contents($filename);
$size = strlen($data);

Which function do you use ?
Because exec() can directly assign result to a variable, so maybe there's no need to save output to a file, if you just want to load it in PHP.

You say:
I use PHP to call a Java command then
forward its result into a file called
result.txt.
Who does the result writing?
1.The JAVA programm?
2.Do you catch the output in PHP and write it to the file.
3.Do you just redirect the output from command line?
If 1 and 3 you might have a delay between when the result is written in the file so, practically, when you read the file in PHP, if you don't wait for the execution to finish, you read it befor it even was written with the result.

Import CSV file to oracle DB

Is there a simple Java library or approach that will take a SQL query and load data in a CSV file to oracle database. Pls help

You don't have to use Java to load a data file into a table unless it is absolutely necessary. Instead, I'd recommend Oracle's command-line SQL*Loader utility which was designed specially for this purpose.

For similar tasks I usually use Groovy scripts as it's really easy and quick to write and runs on the JVM off course.
...an example:
import groovy.sql.Sql
def file1 = new File(/C:\Documents and Settings\USER\Desktop\Book1.csv/)
def reader = new FileReader(file1)
def sql = Sql.newInstance("jdbc:oracle:thin:#XXXXXX:XXXX:XXX", "SCHEMA",
"USER", "oracle.jdbc.driver.OracleDriver")
reader.each { line ->
fields = line.split(';')
sql.executeInsert("insert into YOUR_TABLE values(${fields[0]},${fields[1]},${fields[2]})")
}
It's a basic example, if you have double quotes and semi columns in your csv you will probably want to use something like OpenCSV to handle that.

You could transform each line in the CSV with regular expressions, to make an insert query, and then send to Oracle (with JDBC).

I think this tool will help you for any type of database import-export problem.
http://www.dmc-fr.com/home_en.php

Do you have that CSV in a file on the database server or can you store it there? Then you may try to have Oracle open it by declaring a DIRECTORY object for the path the file is in and then create an EXTERNAL TABLE which you can query in SQL afterwards. Oracle does the parsing of the file for you.

If you are open to Python you can do bulk load using SQL*Loader
loadConf=('sqlldr userid=%s DATA=%s control=%s LOG=%s.log BAD=%s.bad DISCARD=%s.dsc' % (userid,datafile, ctlfile,fn,fn,fn)).split(' ')
p = Popen(loadConf, stdin=PIPE, stdout=PIPE, stderr=PIPE, shell=False, env=os.environ)
output, err = p.communicate()
It's will be much faster that row insert.
I uploaded basic working example here.