I have a sql script file, i need to execute commands present in that through java. I searched in internet regarding the same, i got some code defining parsers to split the SQL statements and executing that. But none of them worked for my sql script file.Because my script file contains both create statements and alter statements without semicolon at the end[Instead it has GO]Can anybody suggest a solution to execute the script file?
Thanks,
Mahesh
For simple scripts I generally use this class from ibatis - ScriptRunner. Alternative you can spawn a new db client process from Java and feed in the script you wan't execute. This will work for all scripts, as simple solutions like ScriptRunner don't work well when the delimiters in the sql files get changed for instance.
Here's an example how to feed the sql as a string to a spawed db client process:
private void runSql(String pSql) {
String tCommand = "mysql -u " + username + (password != null ? " -p" + password : "") + " " + dbName;
System.out.println(tCommand);
try {
Process tProcess = Runtime.getRuntime().exec(tCommand);
OutputStream tOutputStream = tProcess.getOutputStream();
Writer w = new OutputStreamWriter(tOutputStream);
System.out.println(pSql);
w.write(pSql);
w.flush();
Scanner in = new Scanner(tProcess.getErrorStream());
String errorMessage = "";
while (in.hasNext()) {
errorMessage += in.next() + " ";
}
if (errorMessage.length() > 0) {
System.out.println(errorMessage);
throw new ClientSqlExecutionException(errorMessage);
}
} catch (IOException e) {
e.printStackTrace();
}
}
Have a look at Mybatis Migrations code, it does something like the one you need:
http://code.google.com/p/mybatis/wiki/Migration
You need to change the parser so it produces executable statements. But I'm not sure I understand what you mean when you say "execute through Java".
Java won't execute those SQL statements - the database you connect to will. Java can connect to a database using JDBC and send the SQL statements from the file.
I don't see why you have to parse the SQL, unless you want Java to validate them before sending them on to the database server. The server will parse and validate them again, so it feels like you're doing extra work for nothing.
The simplest solution I can present to you is this, presuming I understand your question.
1) Read text file into a string or array via Java IO.
2) Pass string or array to MySQL via JDBC.
Read from file example,
import java.io.*;
class FileRead
{
public static void main(String args[])
{
try{
// Open the file that is the first
// command line parameter
FileInputStream fstream = new FileInputStream("textfile.txt");
// Get the object of DataInputStream
DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String strLine;
//Read File Line By Line
while ((strLine = br.readLine()) != null) {
// Print the content on the console
System.out.println (strLine);
}
//Close the input stream
in.close();
}catch (Exception e){//Catch exception if any
System.err.println("Error: " + e.getMessage());
}
}
}
acquired from, http://www.roseindia.net/java/beginners/java-read-file-line-by-line.shtml
The simplest way is to simply get the statements and check if they need they semi-column at the end: (this is an example and only works if it is a statement by line:
public void executeScript(String script) {
BufferedReader in = new BufferedReader(new FileReader(script));
while (in.read() > 0) {
String statement = in.readLine();
statement = statement.trim().toLowerCase();
String command = statement.split("[ ]+")[0]; // split the statement.
if (command.equals("insert") || command.equals("update") /* || any other */) {
statement = statement + ";";
}
// execute statement using jdbc
}
}
If you do not know how to use jdbc, just ask :-)
Use this slightly modified version of the com.ibatis.common.jdbc.ScriptRunner class which is fully self contained, i.e. you do not need to have any third party jar dependencies.
It is possible to change the delimiter from ; to GO. I think that should do the trick.
Here is an example:
Reader reader = new BufferedReader(*** Your FileReader instance ***);
try
{
ScriptRunner runner = new ScriptRunner(connection, false, true);
runner.setDelimiter("GO", true);
runner.runScript(reader);
}
finally
{
reader.close();
}
Apache Ant SQL Task worked for me.
https://ant.apache.org/manual/Tasks/sql.html
Related
I am trying to create an authentication system of sorts that uses a file called Users.dat to store user data. Currently, I am developing a method to remove users by rewriting the Users.dat file, omitting the user specified. The code below works in a basic environment with an all-encompassing directory containing the .java files and the Users.dat file in the same spot. The old Users.dat file is deleted and Users.dat.tmp is renamed to User.dat. (No problems here, everything works as intended).
public static boolean RemoveUser(String userName) {
// TODO remove username from Users.dat
try {
File originalFile = new File("Users.dat");
System.out.println(originalFile.getAbsolutePath());
BufferedReader read = new BufferedReader(new FileReader("Users.dat"));
String line = null;
while ((line = read.readLine()) != null) {
if (line.indexOf(userName) != -1) {
break;
}
}
String[] userInfo = line.split(", ");
if (!userName.equals(userInfo[2])) {
System.out.println("Username not found. No users removed.");
read.close();
return false;
}
File tempFile = new File(originalFile.getAbsolutePath() + ".tmp");
PrintWriter print = new PrintWriter(new FileWriter(tempFile));
String lineToRemove = line;
BufferedReader read2 = new BufferedReader(new FileReader("Users.dat"));
while ((line = read2.readLine()) != null) {
if (!line.trim().equals(lineToRemove)) {
print.println(line);
print.flush();
}
}
print.close();
read.close();
read2.close();
System.out.println(originalFile.getAbsolutePath());
originalFile.delete(); //This line is not executing correctly
tempFile.renameTo(originalFile); //Nor is this line
} catch (FileNotFoundException ex) {
ex.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}
return true;
}
Users.dat file format:
Joe, Last, jlast, 58c536ed8facc2c2a293a18a48e3e120, true
Sam, sone, samsone, 2c2a293a18a48e3e12058c536ed8facc, false
Jane, Best, jbest, 293a18a48e3e12052058c536ed8facc2c, false
Andrew, Estes, Aestes, 63a490d69aa544fd1272a976014ad570, true
Test, User, tuser, 63a490d69aa544fd1272a976014ad570, true
I have two System.out.println(originalFile.getAbsolutePath()) statements, one at the beginning, one at the end to make sure the path isn't getting screwed up in the process of everything somehow.
Like I said, the code works, however, when I try to implement it in my project, it creates the Users.dat.tmp and it writes the correct data to it, but it does not delete the old Users.dat file, nor does it rename the Users.dat.tmp file to replace Users.dat. I'm certain the directory is correct, as I am literally displaying it as the code executes. I can't figure out any other reason why originalFile.delete() and tempFile.renameTo(originalFile) aren't functioning properly.
EDIT:
Using java.nio.file, I was able to produce an error message. it reads:
java.nio.file.FileSystemException: C:\Path\Users.dat: The process cannot access the file because it is being used by another process.
I don't have the file open when this error message is shown, and I don't get this error using java.nio in my testing environment mentioned at the beginning. I'm not sure what other process the message is referring to.
EDIT 2:
I tried running the code on other machines, one a Mac, the other a Windows laptop, and the code functioned on the Mac just fine, but I was still seeing the same issue on the Windows laptop.
I had the similar issue. My problem was not closing all the streams I read and written to the file. Thanks for your Edit #1, that was helpful
When you wrap
BufferedReader read = new BufferedReader(new FileReader("Users.dat"));
don't you need to close the inner readers too?
If not for the author, but for those who stambled upon this question (like me), hope this suggestion will be useful
I had an earlier function that I was calling in main that was accessing Users.dat, but I never closed the BufferredReader in that function.
I want to use an external tool while extracting some data (loop through lines).
For that I first used Runtime.getRuntime().exec() to execute it.
But then my extraction got really slow. So I am searching for a possibility to exec the external tool in each instance of the loop, using the same instance of shell.
I found out, that I should use ProcessBuilder. But it's not working yet.
Here is my code to test the execution (with input from the answers here in the forum already):
public class ExecuteShell {
ProcessBuilder builder;
Process process = null;
BufferedWriter process_stdin;
BufferedReader reader, errReader;
public ExecuteShell() {
String command;
command = getShellCommandForOperatingSystem();
if(command.equals("")) {
return; //Fehler! No error handling yet
}
//init shell
builder = new ProcessBuilder( command);
builder.redirectErrorStream(true);
try {
process = builder.start();
} catch (IOException e) {
System.out.println(e);
}
//get stdout of shell
reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
errReader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
//get stdin of shell
process_stdin = new BufferedWriter(new OutputStreamWriter(process.getOutputStream()));
System.out.println("ExecuteShell: Constructor successfully finished");
}
public String executeCommand(String commands) {
StringBuffer output;
String line;
try {
//single execution
process_stdin.write(commands);
process_stdin.newLine();
process_stdin.flush();
} catch (IOException e) {
System.out.println(e);
}
output = new StringBuffer();
line = "";
try {
if (!reader.ready()) {
output.append("Reader empty \n");
return output.toString();
}
while ((line = reader.readLine())!= null) {
output.append(line + "\n");
return output.toString();
}
if (!reader.ready()) {
output.append("errReader empty \n");
return output.toString();
}
while ((line = errReader.readLine())!= null) {
output.append(line + "\n");
}
} catch (Exception e) {
System.out.println("ExecuteShell: error in executeShell2File");
e.printStackTrace();
return "";
}
return output.toString();
}
public int close() {
// finally close the shell by execution exit command
try {
process_stdin.write("exit");
process_stdin.newLine();
process_stdin.flush();
}
catch (IOException e) {
System.out.println(e);
return 1;
}
return 0;
}
private static String getShellCommandForOperatingSystem() {
Properties prop = System.getProperties( );
String os = prop.getProperty( "os.name" );
if ( os.startsWith("Windows") ) {
//System.out.println("WINDOWS!");
return "C:/cygwin64/bin/bash";
} else if (os.startsWith("Linux") ) {
//System.out.println("Linux!");
return"/bin/sh";
}
return "";
}
}
I want to call it in another Class like this Testclass:
public class TestExec{
public static void main(String[] args) {
String result = "";
ExecuteShell es = new ExecuteShell();
for (int i=0; i<5; i++) {
// do something
result = es.executeCommand("date"); //execute some command
System.out.println("result:\n" + result); //do something with result
// do something
}
es.close();
}
}
My Problem is, that the output stream is always empty:
ExecuteShell: Constructor successfully finished
result:
Reader empty
result:
Reader empty
result:
Reader empty
result:
Reader empty
result:
Reader empty
I read the thread here: Java Process with Input/Output Stream
But the code snippets were not enough to get me going, I am missing something. I have not really worked with different threads much. And I am not sure if/how a Scanner is of any help to me. I would really appreciate some help.
Ultimatively, my goal is to call an external command repeatetly and make it fast.
EDIT:
I changed the loop, so that the es.close() is outside. And I wanted to add, that I do not want only this inside the loop.
EDIT:
The problem with the time was, that the command I called caused an error. When the command does not cause an error, the time is acceptable.
Thank you for your answers
You are probably experiencing a race condition: after writing the command to the shell, your Java program continues to run, and almost immediately calls reader.ready(). The command you wanted to execute has probably not yet output anything, so the reader has no data available. An alternative explanation would be that the command does not write anything to stdout, but only to stderr (or the shell, maybe it has failed to start the command?). You are however not reading from stderr in practice.
To properly handle output and error streams, you cannot check reader.ready() but need to call readLine() (which waits until data is available) in a loop. With your code, even if the program would come to that point, you would read only exactly one line from the output. If the program would output more than one line, this data would get interpreted as the output of the next command. The typical solution is to read in a loop until readLine() returns null, but this does not work here because this would mean your program would wait in this loop until the shell terminates (which would never happen, so it would just hang infinitely).
Fixing this would be pretty much impossible, if you do not know exactly how many lines each command will write to stdout and stderr.
However, your complicated approach of using a shell and sending commands to it is probably completely unnecessary. Starting a command from within your Java program and from within the shell is equally fast, and much easier to write. Similarly, there is no performance difference between Runtime.exec() and ProcessBuilder (the former just calls the latter), you only need ProcessBuilder if you need its advanced features.
If you are experiencing performance problems when calling external programs, you should find out where they are exactly and try to solve them, but not with this approach. For example, normally one starts a thread for reading from both the output and the error stream (if you do not start separate threads and the command produces large output, everything might hang). This could be slow, so you could use a thread pool to avoid repeated spawning of processes.
I am creating a wrapper for a executable that runs on the windows command line. The executable takes a few commands then attempts to connect to another device. then it outputs and ERROR! or Ready For "Device Name" i do not get this message until the app exits. The problem is this app is a tunnel allowing me to run telnet on the external box but i need to make sure the Device is ready this is my code.
public void startUDPTunnel() {
//TODO Pull Amino serial number from webportal
Properties prop = new Properties();
InputStream inConfig = getClass().getClassLoader().getResourceAsStream("config.properties");
try {
prop.load(inConfig);
} catch (IOException e) {
System.out.println(e.getMessage());
}
String server = prop.getProperty("server");//config.GetProp("server");
System.out.println(server);
String port = prop.getProperty("port");//config.GetProp("port");
System.out.println(port);
String location = prop.getProperty("location");//config.GetProp("location");
System.out.println(location);
String url = prop.getProperty("URL");
System.out.println(url);
String input = "";
try {
input = getSerial(url);
System.out.println(input);
Process p = Runtime.getRuntime().exec(location+"udptunnel.exe -c 127.0.0.1 23 "+input+" "+server+" "+port+" 127.0.0.1 23");
threadSleep();
BufferedReader in = new BufferedReader(
new InputStreamReader(p.getInputStream()));
String line = null;
while ((line = in.readLine()) != null) {
if(line.equals("ERROR!")){
System.out.println("There was an ERROR");
}
if(line.equals("Ready for \""+input+"\"")){
System.out.println("Load Telnet");
}
}
p.destroy();
} catch (IOException e) {
e.printStackTrace();
}
}
Sorry there is a lot of debug code left in this function.
EDIT
OK I am pretty sure know what the issue is bufferReader.readLine() requires a \n or \r or just hangs is there anyway to watch the stream with out the buffer?
You should use a ProcessBuilder, and then use redirectErrorStream(). I think this will cause stdout of the process to be unbuffered. And even if it doesn't, you'll only have to read from one InputStream to get both stdout and stderr.
I have figured out my problem the applications that i am executing with java do not have a EOL at the end of the line in fact they just hang on the line For example telnet waits for the username then the password. i am not sure this is proper but it works and is what i am going to use for now
while((i=br.read())!=-1){
ch += (char)i;
}
This outputs every char as they come in when then i just make sure the string contains what i am looking for!
First, this is my code :
import java.io.*;
import java.util.Date;
import com.banctecmtl.ca.vlp.shared.exceptions.*;
public class PowershellTest implements Runnable {
public static final String PATH_TO_SCRIPT = "C:\\Scripts\\ScriptTest.ps1";
public static final String SERVER_IP = "XX.XX.XX.XXX";
public static final String MACHINE_TO_MOD = "MachineTest";
/**
* #param args
* #throws OperationException
*/
public static void main(String[] args) throws OperationException {
new PowershellTest().run();
}
public PowershellTest(){}
#Override
public synchronized void run() {
String input = "";
String error = "";
boolean isHanging = false;
try {
Runtime runtime = Runtime.getRuntime();
Process proc = runtime.exec("powershell -file " + PATH_TO_SCRIPT +" "+ SERVER_IP +" "+ MACHINE_TO_MOD);
proc.getOutputStream().close();
InputStream inputstream = proc.getInputStream();
InputStreamReader inputstreamreader = new InputStreamReader(inputstream);
BufferedReader bufferedreader = new BufferedReader(inputstreamreader);
proc.waitFor();
String line;
while (!isHanging && (line = bufferedreader.readLine()) != null) {
input += (line + "\n");
Date date = new Date();
while(!bufferedreader.ready()){
this.wait(1000);
//if its been more then 1 minute since a line has been read, its hanging.
if(new Date().getTime() - date.getTime() >= 60000){
isHanging = true;
break;
}
}
}
inputstream.close();
inputstream = proc.getErrorStream();
inputstreamreader = new InputStreamReader(inputstream);
bufferedreader = new BufferedReader(inputstreamreader);
isHanging = false;
while (!isHanging && (line = bufferedreader.readLine()) != null) {
error += (line + "\n");
Date date = new Date();
while(!bufferedreader.ready()){
this.wait(1000);
//if its been more then 1 minute since a line has been read, its hanging.
if(new Date().getTime() - date.getTime() >= 60000){
isHanging = true;
break;
}
}
}
inputstream.close();
proc.destroy();
} catch (IOException e) {
//throw new OperationException("File IO problem.", e);
} catch (InterruptedException e) {
//throw new OperationException("Script thread problem.",e);
}
System.out.println("Error : " + error + "\nInput : " + input);
}
}
I'm currently trying to run a powershell script that will start/stop a vm (VMWARE) on a remote server. The script work from command line and so does this code. The thing is, I hate how I have to use a thread (and make it wait for the script to respond, as explained further) for such a job. I had to do it because both BufferedReader.readline() and proc.waitFor() hang forever.
The script, when ran from cmd, is long to execute. it stall for 30 sec to 1 min from validating authentification with the server and executing the actual script. From what I saw from debugging, the readline hang when it start receiving those delays from the script.
I'm also pretty sure it's not a memory problem since I never had any OOM error in any debugging session.
Now I understand that Process.waitFor() requires me to flush the buffer from both the error stream and the regular stream to work and so that's mainly why I don't use it (I need the output to manage VM specific errors, certificates issues, etc.).
I would like to know if someone could explain to me why it hangs and if there is a way to just use the typical readline() without having it to hang so hard. Even if the script should have ended since a while, it still hang (I tried to run both the java application and a cmd command using the exact same thing I use in the java application at the same time, left it runingfor 1h, nothing worked). It is not just stuck in the while loop, the readline() is where the hanging is.
Also this is a test version, nowhere close to the final code, so please spare me the : this should be a constant, this is useless, etc. I will clean the code later. Also the IP is not XX.XX.XX.XXX in my code, obviously.
Either explanation or suggestion on how to fix would be greatly appreciated.
Ho btw here is the script I currently use :
Add-PSSnapin vmware.vimautomation.core
Connect-VIServer -server $args[0]
Start-VM -VM "MachineTest"
If you need more details I will try to give as much as I can.
Thanks in advance for your help!
EDIT : I also previously tested the code with a less demanding script, which job was to get the content of a file and print it. Since no waiting was needed to get the information, the readline() worked well. I'm thus fairly certain that the problem reside on the wait time coming from the script execution.
Also, forgive my errors, English is not my main language.
Thanks in advance for your help!
EDIT2 : Since I cannot answer to my own Question :
Here is my "final" code, after using threads :
import java.io.*;
public class PowershellTest implements Runnable {
public InputStream is;
public PowershellTest(InputStream newIs){
this.is = newIs;
}
#Override
public synchronized void run() {
String input = "";
String error = "";
try {
InputStreamReader inputstreamreader = new InputStreamReader(is);
BufferedReader bufferedreader = new BufferedReader(inputstreamreader);
String line;
while ((line = bufferedreader.readLine()) != null) {
input += (line + "\n");
}
is.close();
} catch (IOException e) {
//throw new OperationException("File IO problem.", e);
}
System.out.println("Error : " + error + "\nInput : " + input);
}
}
And the main simply create and start 2 thread (PowerShellTest instances), 1 with the errorStream and 1 with the inputStream.
I believe I made a dumb error when I first coded the app and fixed it somehow as I reworked the code over and over. It still take a good 5-6 mins to run, which is somehow similar if not longer than my previous code (which is logical since the errorStream and inputStream get their information sequentially in my case).
Anyway, thanks to all your answer and especially Miserable Variable for his hint on threading.
First, don't call waitFor() until after you've finished reading the streams. I would highly recommend you look at ProcessBuilder instead of simply using Runtime.exec, and split the command up yourself rather than relying on Java to do it for you:
ProcessBuilder pb = new ProcessBuilder("powershell", "-file", PATH_TO_SCRIPT,
SERVER_IP, MACHINE_TO_MOD);
pb.redirectErrorStream(true); // merge stdout and stderr
Process proc = pb.start();
redirectErrorStream merges the error output into the normal output, so you only have to read proc.getInputStream(). You should then be able to just read that stream until EOF, then call proc.waitFor().
You are currently waiting to complete reading from inputStream before starting to read from errorStream. If the process writes to its stderr before stdout maybe you are getting into a deadlock situation.
Try reading from both streams from concurrently running threads. While you are at it, also remove proc.getOutputStream().close();. It shouldn't affect the behavior, but it is not required either.
i'm not a java coder but need a commnad that will do
cut -d "/" -f1,2,3 MyFile
Any ideas?
Read the file. Split each line on / and then print out the first three parts.
BufferedReader in = null;
try{
in = new BufferedReader(new FileReader("MyFile"));
String line = "" ;
while((line = in.readLine()) != null){
String[] fields = line.split("/");
System.out.printf("%s/%s/%s\n", fields[0], fields[1], fields[2]) ;
}
}
catch(Exception e){
e.printStackTrace();
}
finally{
if (in != null) {
try {
in.close();
} catch (IOException e) {
}
}
}
The main difference between Java coding and that script example is that they use different programming paradigm lest say that Java use Object Programing and that example can be assigned to declarative programing (You only express the logic, not the way to get the result) so no commands in Java.
So there is no command that You can use for this type of functionality.
There might be some package with this type of static procedure that works like that cut program. But still You will have to write a program to use it.
So if You would describe us the purpose of usage of this program (cut) )we could be able to provide You a good answer.