This is actually a re-do of an older question of mine that I have completely redone because my old question seemed to confuse people.
I have written a Java program that Queries a database and is intended to retrieve several rows of data. I have previously written the program in Informix-4GL and I am using a sql cursor to loop through the database and store each row into a "dynamic row of record". I understand there are no row of records in Java so I have ended up with the following code.
public class Main {
// DB CONNECT VARIABLE ===========================
static Connection gv_conn = null;
// PREPARED STATEMENT VARIABLES ==================
static PreparedStatement users_sel = null;
static ResultSet users_curs = null;
static PreparedStatement uinfo_sel = null;
static ResultSet uinfo_curs = null;
// MAIN PROGRAM START ============================
public static void main(String[] args) {
try {
// CONNECT TO DATABASE CODE
} catch(Exception log) {
// YOU FAILED CODE
}
f_prepare(); // PREPARE THE STATEMENTS
ArrayList<Integer> list_id = new ArrayList<Integer>();
ArrayList<String> list_name = new ArrayList<String>();
ArrayList<Integer> list_info = new ArrayList<String>();
ArrayList<String> list_extra = new ArrayList<String>();
try {
users_sel.setInt(1, 1);
users_curs = users_sel.executeQuery();
// RETRIEVE ROWS FROM USERS
while (users_curs.next()) {
int lv_u_id = users_curs.getInt("u_id");
String lv_u_name = users_curs.getString("u_name");
uinfo_sel.setInt(1, lv_u_id);
uinfo_curs = uinfo_sel.executeQuery();
// RETRIEVE DATA FROM UINFO RELATIVE TO USER
String lv_ui_info = uinfo_curs.getString("ui_info");
String lv_ui_extra = uinfo_curs.getString("ui_extra");
// STORE DATA I WANT IN THESE ARRAYS
list_id.add(lv_u_id);
list_name.add(lv_u_name);
list_info.add(lv_ui_info);
list_extra.add(lv_ui_extra);
}
} catch(SQLException log) {
// EVERYTHING BROKE
}
// MAKING SURE IT WORKED
System.out.println(
list_id.get(0) +
list_name.get(0) +
list_info.get(0) +
list_extra.get(0)
);
// TESTING WITH ARBITRARY ROWS
System.out.println(
list_id.get(2) +
list_name.get(5) +
list_info.get(9) +
list_extra.get(14)
);
}
// PREPARE STATEMENTS SEPARATELY =================
public static void f_prepare() {
String lv_sql = null;
try {
lv_sql = "select * from users where u_id >= ?"
users_sel = gv_conn.prepareStatement(lv_sql);
lv_sql = "select * from uinfo where ui_u_id = ?"
uinfo_sel = gv_conn.prepareStatement(lv_sql)
} catch(SQLException log) {
// IT WON'T FAIL COZ I BELIEEEVE
}
}
}
class DBConn {
// connect to SQLite3 code
}
All in all this code works, I can hit the database once, get all the data I need, store it in variables and work with them as I please however this does not feel right and I think it's far from the most suited way to do this in Java considering I can do it with only 15 lines of code in Informix-4GL.
Can anyone give me advice on a better way to achieve a similar result?
In order to use Java effectively you need to use custom objects. What you have here is a lot of static methods inside a class. It seems that you are coming from a procedural background and if you try to use Java as a procedural language, you will not much value from using it. So first off create a type, you can plop it right inside your class or create it as a separate file:
class User
{
final int id;
final String name;
final String info;
final String extra;
User(int id, String name, String info, String extra)
{
this.id = id;
this.name = name;
this.info = info;
this.name = name;
}
void print()
{
System.out.println(id + name + info + extra);
}
}
Then the loop becomes:
List<User> list = new ArrayList<User>();
try {
users_sel.setInt(1, 1);
users_curs = users_sel.executeQuery();
// RETRIEVE ROWS FROM USERS
while (users_curs.next()) {
int lv_u_id = users_curs.getInt("u_id");
String lv_u_name = users_curs.getString("u_name");
uinfo_sel.setInt(1, lv_u_id);
uinfo_curs = uinfo_sel.executeQuery();
// RETRIEVE DATA FROM UINFO RELATIVE TO USER
String lv_ui_info = uinfo_curs.getString("ui_info");
String lv_ui_extra = uinfo_curs.getString("ui_extra");
User user = new User(lv_u_id, lv_u_name, lv_ui_info, lv_ui_extra);
// STORE DATA
list.add(user);
}
} catch(SQLException log) {
// EVERYTHING BROKE
}
// MAKING SURE IT WORKED
list.get(0).print();
This doesn't necessarily address the number of lines. Most people who use Java don't interact with databases with this low-level API but in general, if you are looking to get down to the fewest number of lines (a questionable goal) Java isn't going to be your best choice.
Your code is actually quite close to box stock JDBC.
The distinction is that in Java, rather than having a discrete collection of arrays per field, we'd have a simple Java Bean, and a collection of that.
Some examples:
public class ListItem {
Integer id;
String name;
Integer info;
String extra;
… constructors and setters/getters ellided …
}
List<ListItems> items = new ArrayList<>();
…
while(curs.next()) {
ListItem item = new ListItem();
item.setId(curs.getInt(1));
item.setName(curs.getString(2));
item.setInfo(curs.getInfo(3));
item.setExtra(curs.getString(4));
items.add(item);
}
This is more idiomatic, and of course does not touch on the several frameworks and libraries available to make DB access a bit easier.
Related
I'm trying to get data from mySQL to List in java using sql2o lib.
But for some reason I just fail to understand how to use it properly (it looks like).
Here is the faulty code:
List<String> returning = new ArrayList<String>();
String date = "";
String playerList = "";
String playerCount = "";
String playerMax = "";
con.createQuery(sql)
.throwOnMappingFailure(true).addColumnMapping("date", date)
.addColumnMapping("playerList", playerList)
.addColumnMapping("playerCount", playerCount)
.addColumnMapping("playerMax", playerMax).executeAndFetch(String.class);
returning.add(date);
returning.add(playerList);
returning.add(playerCount);
returning.add(playerMax);
And here is error I get:
org.sql2o.Sql2oException: Could not map date to any property.
at org.sql2o.DefaultResultSetHandlerFactory.newResultSetHandler0(DefaultResultSetHandlerFactory.java:199)
at org.sql2o.DefaultResultSetHandlerFactory.access$200(DefaultResultSetHandlerFactory.java:17)
at org.sql2o.DefaultResultSetHandlerFactory$5.evaluate(DefaultResultSetHandlerFactory.java:160)
at org.sql2o.DefaultResultSetHandlerFactory$5.evaluate(DefaultResultSetHandlerFactory.java:156)
at org.sql2o.tools.AbstractCache.get(AbstractCache.java:49)
at org.sql2o.DefaultResultSetHandlerFactory.newResultSetHandler(DefaultResultSetHandlerFactory.java:173)
at org.sql2o.PojoResultSetIterator.<init>(PojoResultSetIterator.java:20)
at org.sql2o.Query$14.iterator(Query.java:547)
at org.sql2o.Query.executeAndFetch(Query.java:588)
at org.sql2o.Query.executeAndFetch(Query.java:574)
at lol.discordbot.database.QueryServerInfo.getCurrent(QueryServerInfo.java:31)
at lol.discordbot.command.Query.execute(Query.java:20)
at lol.discordbot.command.CommandsListener.onMessageReceived(CommandsListener.java:39)
I think you misunderstand what column mappings are. Column mappings are used to map column names to object-field names.
You should first create a data class to hold the result of your query. From your code above, I assume that you are trying to fetch players.
public class Player {
public String date;
public String playerList;
public String playerCount;
public String playerMax
}
(Consider to use better data types. Date for dates, int for counts, etc)
Then you can use sql2o to fetch data
List<Player> players = con.createQuery(sql).executeAndFetch(Player.class);
There is a much better way now.
.setAutoDeriveColumnNames(true)
Example
try (Connection con = sql2o.open()) {
List<Player> l = con.createQuery(sql)
.setAutoDeriveColumnNames(true)
.executeAndFetch(Player.class);
}
https://groups.google.com/g/sql2o/c/3H4XJIv-i04
I am trying to retrieve and process code from JIRA, unfortunately the pieces of information (which are in the Metadata-Plugin) are saved in a column, not a row.
Picture of JIRA-MySQL-Database
The goal is to save this in an object with following attributes:
public class DesiredObject {
private String Object_Key;
private String Aze.kunde.name;
private Long Aze.kunde.schluessel;
private String Aze.projekt.name;
private Long Aze.projekt.schluessel
//getters and setters here
}
My workbench is STS and it's a Spring-Boot-Application.
I can fetch a List of Object-Keys with the JRJC using:
JiraController jiraconnect = new JiraController();
List<JiraProject> jiraprojects = new ArrayList<JiraProject>();
jiraprojects = jiraconnect.findJiraProjects();
This is perfectly working, also the USER_KEY and USER_VALUE are easily retrievable, but I hope there is a better way than to perform
three SQL-Searches for each project and then somehow build an object from all those lists.
I was starting with
for (JiraProject jp : jiraprojects) {
String SQL = "select * from jira_metadata where ENRICHED_OBJECT_KEY = ?";
List<DesiredObject> do = jdbcTemplateObject.query(SQL, new Object[] { "com.atlassian.jira.project.Project:" + jp.getProjectkey() }, XXX);
}
to get a list with every object, but I'm stuck as i can't figure out a ObjectMapper (XXX) who is able to write this into an object.
Usually I go with
object.setter(rs.getString("SQL-Column"));
But that isn't working, as all my columns are called the same. (USER_KEY & USER_VALUE)
The Database is automatically created by JIRA, so I can't "fix" it.
The Object_Keys are unique which is why I tried to use those to collect all the data from my SQL-Table.
I hope all you need to enlighten me is in this post, if not feel free to ask for more!
Edit: Don't worry if there are some 'project' and 'projekt', that's because I gave most of my classes german names and descriptions..
I created a Hashmap with the Objectkey and an unique token in brackets, e.g.: "(1)JIRA".
String SQL = "select * from ao_cc6aeb_jira_metadata";
List<JiraImportObjekt> jioList = jdbcTemplateObject.query(SQL, new JiraImportObjektMapper());
HashMap<String, String> hmap = new HashMap<String, String>();
Integer unique = 1;
for (JiraImportObjekt jio : jioList) {
hmap.put("(" + unique.toString() + ")" + jio.getEnriched_Object_Key(),
jio.getUser_Key() + "(" + jio.getUser_Value() + ")");
unique++;
}
I changed this into a TreeMap
Map<String, String> tmap = new TreeMap<String, String>(hmap);
And then i iterated through that treemap via
String aktuProj = new String();
for (String s : tmap.keySet()) {
if (aktuProj.equals(s.replaceAll("\\([^\\(]*\\)", ""))) {
} else { //Add Element to list and start new Element }
//a lot of other stuff
}
What I did was to put all the data in the right order, iterate through and process everything like I wanted it.
Object hinfo = hmap.get(s);
if (hinfo.toString().replaceAll("\\([^\\(]*\\)", "").equals("aze.kunde.schluessel")) {
Matcher m = Pattern.compile("\\(([^)]+)\\)").matcher(hinfo.toString());
while (m.find()) {
jmo[obj].setAzeKundeSchluessel(Long.parseLong(m.group(1), 10));
// logger.info("AzeKundeSchluessel: " +
// jmo[obj].getAzeKundeSchluessel());
}
} else ...
After the loop I needed to add the last Element.
Now I have a List with the Elements which is easy to use and ready for further steps.
I cut out a lot of code because most of it is customized for my problem.. the roadmap should be enough to solve it though.
Good luck!
I'm using Java to download HTML contents of websites whose URLs are stored in a database. I'd like to put their HTML into database, too.
I'm using Jsoup for this purpose:
public String downloadHTML(String byLink) {
String htmlInPage = "";
try {
Document doc = Jsoup.connect(byLink).get();
htmlInPage = doc.html();
} catch (org.jsoup.UnsupportedMimeTypeException e) {
// process this and some other exceptions
}
return htmlInPage;
}
I'd like to download websites concurrently and use this function:
public void downloadURL(int websiteId, String url,
String categoryName, ExecutorService executorService) {
executorService.submit((Runnable) () -> {
String htmlInPage = downloadHTML(url);
System.out.println("Category: " + categoryName + " " + websiteId + " " + url);
String insertQuery =
"INSERT INTO html_data (website_id, html_contents) VALUES (?,?)";
dbUtils.query(insertQuery, websiteId, htmlInPage);
});
}
dbUtils is my class based on Apache Commons DbUtils. Details are here: http://pastebin.com/iAKXchbQ
And I'm using everything mentioned above in a such way: (List<Object[]> details are explained on pastebin, too)
public static void main(String[] args) {
DbUtils dbUtils = new DbUtils("host", "db", "driver", "user", "pass");
List<String> categoriesList =
Arrays.asList("weapons", "planes", "cooking", "manga");
String sql = "SELECT lw.id, lw.website_url, category_name " +
"FROM list_of_websites AS lw JOIN list_of_categories AS lc " +
"ON lw.category_id = lc.id " +
"where category_name = ? ";
ExecutorService executorService = Executors.newFixedThreadPool(10);
for (String category : categoriesList) {
List<Object[]> sitesInCategory = dbUtils.select(sql, category );
for (Object[] entry : sitesInCategory) {
int websiteId = (int) entry[0];
String url = (String) entry[1];
String categoryName = (String) entry[2];
downloadURL(websiteId, url, categoryName, executorService);
}
}
executorService.shutdown();
}
I'm not sure if this solution is correct but it works. Now I want to modify code to save HTML not from all websites in my database, but only their fixed ammount in each category.
For example, download and save HTML of 50 websites from the "weapons" category, 50 from "planes", etc. I don't think it's necessary to use sql for this purpose: if we select 50 sites per category, it doesn't mean we save them all, because of possibly incorrect syntax and connection problems.
I've tryed to create separate class implementing Runnable with fields: counter and maxWebsitesPerCategory, but these variables aren't updated. Another idea was to create field Map<String,Integer> sitesInCategory instead of counter, put each category as a key there and increment its value until it reaches maxWebsitesPerCategory, but it didn't work, too. Please, help me!
P.S: I'll also be grateful for any recommendations connected with my realization of concurrent downloading (I haven't worked with concurrency in Java before and this is my first attempt)
How about this?
for (String category : categoriesList) {
dbUtils.select(sql, category).stream()
.limit(50)
.forEach(entry -> {
int websiteId = (int) entry[0];
String url = (String) entry[1];
String categoryName = (String) entry[2];
downloadURL(websiteId, url, categoryName, executorService);
});
}
sitesInCategory has been replaced with a stream of at most 50 elements, then your code is run on each entry.
EDIT
In regard to comments. I've gone ahead and restructured a bit, you can modify/implement the content of the methods I've suggested.
public void werk(Queue<Object[]> q, ExecutorService executorService) {
executorService.submit(() -> {
try {
Object[] o = q.remove();
try {
String html = downloadHTML(o); // this takes one of your object arrays and returns the text of an html page
insertIntoDB(html); // this is the code in the latter half of your downloadURL method
}catch (/*narrow exception type indicating download failure*/Exception e) {
werk(q, executorService);
}
}catch (NoSuchElementException e) {}
});
}
^^^ This method does most of the work.
for (String category : categoriesList) {
Queue<Object[]> q = new ConcurrentLinkedQueue<>(dbUtils.select(sql, category));
IntStream.range(0, 50).forEach(i -> werk(q, executorService));
}
^^^ this is the for loop in your main
Now each category tries to download 50 pages, upon failure of downloading a page it moves on and tries to download another page. In this way, you will either download 50 pages or have attempted to download all pages in the category.
I improved the structure after a good observation from [http://stackoverflow.com/users/1690199/v-k] I am still getting a token error even though the syntax looks correct to me. More comments and critiques will be useful and acknowledged here.
import de.bezier.data.sql.*;
PostgreSQL pgsql;
Float val;
void setup()
{
size( 100, 100 );
println(val);
}
Token error identified in Processing 2 at Class Database.
Class Database
{
String user = "user";
String pass = "pass";
String database = "db";
Float val;
Database (Float col) {
val = col;
}
void database_connection( col )
{
//sets up database
pgsql = new PostgreSQL( this, "127.0.0.1", database, user, pass );
if ( pgsql.connect() )
{
pgsql.query( "SELECT col FROM table ORDER BY col DESC LIMIT 1; " );
return( pgsql.getFloat("col") );
}
else
{
println ("failed to connect to the database");
}
}
}
OLD ISSUE: Class structure addressed after a great observation from [http://stackoverflow.com/users/1690199/v-k]
import de.bezier.data.sql.*;
.....
.....
Old code removed for clarity of this issue.
Classes don't take arguments. Also it is class not Class... Am i missing something? Look, a general sample:
class Database {
String user = "user";
String pass = "pass";
String database = "db";
float val; //by convention no Caps for vars...
// a constructor, which get partameters
Database (float v) {
val = v;
}
// a method
void database_setup() {
//whateverq
}
}//end of Database class
First, you should create a new question if you have a second question. Second, you never create the variable pgsql, you just start using it immediately. Move this line:
PostgreSQL pgsql;
to this group of lines:
String user = "user";
String pass = "pass";
String database = "db";
float val;
It's a variable that gets used in that class, so put it in that class. Also, use "float" with a lowercase "f" :)
In my Java code I have embedded a SQL query which fetches data from a database and stores it in a result-set. I want to add a function or a piece of code which will take only non-negative data from the result-set for further processing.
Assumption: The result set can contain positive/negative/zero data values as well as characters. Also i cannot change the SQL query as its out of my scope.
try something like this, i think it will do the job
private ArrayList getNegativeNumbers(ResultSet rs, String coulumnName ) throws SQLException
{
ArrayList ret = new ArrayList();
while(rs.next()){
try {
int x = rs.getInt(coulumnName);
if(x>=0){
ret.add(new Integer(x));
}
} catch (Exception ex) {
String x = rs.getString(coulumnName);
ret.add(x);
}
}
return ret;
}
UPDATED 2. Sorry for my edits, i missread the question
while (resultSet.next()) {
if(resultSet.getInt("Column name") > 0);
Processmethod(resultSet.getInt("Column name") );
}