I write a database mid-ware for mysql. I need to determine a sql statement isRead statement or isWrite statement and extract the schema and tables information. Some parser I found through google only parse the DML statement. But I need a parser can parse all statement what mysql server can do.
Do you think that is a good idea I call the mysql parser directly in java or is that possible?
Do you think that is a good idea I call the mysql parser directly in java or is that possible?
No I don't think it is a good idea. Calling C / C++ code from Java is rarely a good idea. And in this case, the parser is embedded in a whole bunch of other code, and it is likely to be able separate it out. On top of that, it will emit the parse tree as complicated native data structures that will be difficult to use from Java.
A better approach would be to take an existing open source Java parser for SQL, and (over time) add support for parsing MySQL-specific statements.
You may want to consider using code from Alibaba's Druid project. Although designed as a sophisticated connection pooling library, this project supports a very advanced parser and AST for ANSI SQL and non-ANSI dialects such as MySQL, Oracle, SQL Server, etc. The project is open source and bears the very liberal Apache License Version 2.0.
The main entry points into this part of the library is SQLUtils.java.
Related
I am completely new to Programming language translation and don't know how to start. I want to convert stored procedures written in PL/SQL to Java, to make our application database independent.
There are about 50 stored procedures, each having about 10000-15000 lines of code. I am looking for a way to automate this.
I have got a reference of ANTLR parser, and have found some parsers on GitHub, but have no clue, how to go about it.
First of all you must know some theory, and also you have to learn something about ANTLR. The ANTLR parser will give you "only" AST (Abstract syntax tree), which is just a small 1st step. The rest is up to you. There is an article called "Life after parsing" describing the problems with automatic translation. If you do not have an experience with this, you probably should not start this way.
You can use ANTRL for:
generation of PL/SQL code call-tree. To get some overview of code's business logic.
extract SQL statements from PL/SQL code
extract table names from SQLs
This can give you some overview which "module" accesses particular tables.
You can also serialize the AST tree into some "fake" java code, where original PL/SQL will be stored as comments. But the rest it up to you. PL/SQL programming style is different from Java one.
PS: You will also find that some of Oracle's proprietary SQLs can not be easily translated into ISO SQL92, and some databases do not even support ISO SQL92 (or higher). So as I wrote before most of the work must be done manually. So maybe rewriting it from scratch is not so bad idea.
I am looking for SQL query parser for MySQL queries. Using which I can parse the query, modify the query object and print back modified query
JSQL Parser was exactly what I needed but It has 2 main issues while escaping single quote inside column values
https://github.com/JSQLParser/JSqlParser/issues/167
https://github.com/JSQLParser/JSqlParser/issues/166
So I am looking for open source alternative which can help me with the task
Presto-parser I tried was not able to parse Update queries
If anyone else is aware of any other reliable SQL Parsing library please let me know
Regarding JSqlParser:
Issue 166 already fixed.
The escaping of single quotes is not supported, but using double single quotes is. Therefore someone could replace all \' using '' before parsing.
EDIT: Issue 167 fixed within actual Snapshot 0.9.5 of JSqlParser.
One year ago i search for the same thing but finally i use another approach. Those were the projects that i found to parse sql.
ANTLR. Is commonly used generic parser. It's a kind of parser generator. From a set of rules it writes the java code. There are rules for a lot of thins like spanish, java or sql.
JParsec. Seems similar to the previous one. Seems that already exists SQL rules, but does not seem easy to use.
ZQL. Uses JavaCC to auto-generate Java code that parses the SQL. I test this one but i can not get it working as i want. And i think that the project is discontinued.
sql-parser. It looks nice, but when i tested it i can not get it working. The link seems to not work, but i think that this is a fork of the original project.
jOOQ is a great library for those that don't need or like an ORM and also has a parser.
I am working with Jsbs and want to select a DBMS for my application that require a native XML database. Can you people guide me?
1) how many DBMS are supported by Java (is it true almost all DBMS are supported by java?)
2) Which one will be the best selection for XML storage and retrieval?
Thanks in advance.
Here is a list of JDBC Drivers and the DBs they work for. Probably every database out there has a JDBC driver.
As far as supporting XML it depends if you want to be able to do queries against the XML or not. Most modern DBMSs support XML to some degree. Do you have one you are already using, or that you are recommended to use ?
1) Yes, there are JDBC drivers for all the major DBMS (Oracle, MySQL, Postgres, DB2...) and also some interesting java DBMS like HSQL
2) As far as I know Oracle, DB2, PostgreSQL (and probably MySQL) all have XML column types
It is not obvious to me that you need anything more than support for Blobs or Clobs to implement simple XML storage and retrieval. You'd only need special XML support if you needed to perform queries against the data contained in the XML.
What you are talking about is an "XML enabled" RDBMS. Depending on your actual requirements, you may also want to look into native XML databases (NXDs). There is even a standard Java API (XQJ) for querying NXDs, though not all vendors support it.
Most modern databases have JDBC-drivers, which is what is needed for Java programs to connect to the database. You generally want type 4 drivers which do not depend on native code.
For starting I would recommend Apache Derby, which is written in Java and can be part of your program, which keeps it simple. http://db.apache.org/derby/. If you later find you need another database you replace the JDBC-driver, and double-check your SQL-statements.
Hi is there any tool available in Java world that will parse/read a source file and pull SQL statements out in to a text file. This is a complex task given that you can write SQL statements in different fashion within the source (ex: using + sign or using .append()) or even conditional building of SQL.
We've been considering doing this. This requires:
1) A full Java parser
2) Full control and data flow analysis, so that you can
track how values propagate through the code
3) Recognition of SQL (JDBC) calls, tracking data flows for the
SQL query back to their origins, and symbolically intrepreting the
set of operations along the data flow paths to determine
the apparant SQL operation.
The DMS Software Reengineering Toolkit is a generic engine for parsing langauges. DMS has a full Java front end (meets condition 1) that computes local control and global data data flow analysis (condition 2). DMS is configurable to extract code properties, so it could be configured we think to do 3. Still not a trivial task.
But alas, the requested tool is not available yet.
Not that I've ever come across. The few times that people have had to do something like this, they have ended up writing a program to do it, because it's not really something you can generalise.
Have you considered having the SQL for the prepared statements in a property file (or language resource bundle) and read them from there?
Check www.antlr.org, they have grammars for PL/SQL, MySQL and Java 1.6. If you only interested in SQL, probably SQL grammar could be something you can build upon.
There could be easier ways about it thought, like reading pseudo SQL from JDBC driver log/wire/from database server log will produce more consistent syntax.
I've found numerous posts about reading CSV with Java and the APIs they were pointing at all had a line-oriented approach when it came to reading a CSV file. Something like "while you get a line, get the values of every column".
I'd appreciate a higher-level API, like in Perl where DBI allows you to use SQL on CSV like if it where a DB table. Otherwise I'll have to implement lots of access logic by myself.
Is there such an API? Am I missing something? There are some references about JDBC drivers but most are projects that haven't been updated the last 5 years.
You can use HSQL in order to do it, see the following links from the docs and a blog post describing exactly that.
You could give H2Database a go - it is rather heavy weight, but at least it is maintained.
Are you trying to avoid using a regular database for accessing CSV? MySQL supports CSV as one of it's table types if you are open to using a database system.
yes, JDBC with a CSV driver. You can try implement yourself or just try something like HXTT