DBLinks ORA-24778: cannot open connections - java

I have a stored procedure which opens the cursor to fetch the data from remote databse
Cursor crs1 is select distinct ID FROM xxxx#link where xxxx and ID = v_id;
Open crs1;
loop fetch crs1 into v_N_ID;
exit when crs1%notfound;
end loop;
This week we started getting this ORA-24778: cannot open connections error. After restarting the websphere server and call the stored procedure it works 2 or 3 times and then again it fails.
We are using XA and the OPEN_LINKS_PER_INSTANCE is set to default 4.
I dont find any answers based on the discussion here
ORA-24778: cannot open connections
How to fix this issue?

Related

SQL server hanged suddenly - all DB connections are active but no response - SQL server 2016

I am a Java developer. Since I am new to SQL server, I have limited knowledge on it. I am trying to find out the root cause why our SQL server suddenly hanged and It became normal after restart
Symptoms:
~ Java threads started getting stuck, figured out that the Java JDBC connections started hanging without any response from DB which caused threads to stuck
~ All connections (around 100 ) were active until SQL server was restarted. Finally, DB connections were closed by DB after restart. Java JDBC connection received 'Connection reset by peer' AFTER DB was restarted
Impact duration : 5 hours (until restart)
Tech stack:
Java spring boot running on weblogic. ORM: hibernate
SQL server 2016
Limitation:
Team had restarted the SQL server before we can export any statistics from DB and even doubtful whether we were able to run statistic SQL queries before SQL server restarted as the DB was hanged already
Findings/actions:
After DB was restarted, I tried to extract statistics from dm_exec_query_stats, however, it tracks queries based on last run time only. There was no result for affected period. Same scenario for dm_os_waiting_tasks as well.
Server team say that the CPU and Memory usage were normal (I still to receive complete report)
Could see no error/problem from Windows event log and cluster logs. They look normal
Some Google sources say that some queries may consume complete CPU which may make SQL server to hang, some others say that some queries might have made blocking.
It may look simple or common for SQL server experts/DBA, however, I have googled for finding out relevant issue and resolution, however they doesn't seem to help
Just guiding me to refer any document or expert advise will be great. Let me know if additional info needed. Thanks in advance !
Tried these queries but no joy
SELECT deqs.last_execution_time AS [Time], dest.TEXT AS [Query], dbid, deqs.*
FROM sys.dm_exec_query_stats AS deqs
CROSS APPLY sys.dm_exec_sql_text(deqs.sql_handle) AS dest
where deqs.last_execution_time between '2020-09-29 13:16:52.710' and '2020-09-29 23:16:52.710'
ORDER BY deqs.last_execution_time DESC ;
SELECT
qs.sql_handle,
qs.execution_count,
qs.total_worker_time AS Total_CPU,
total_CPU_inSeconds = --Converted from microseconds
qs.total_worker_time/1000000,
average_CPU_inSeconds = --Converted from microseconds
(qs.total_worker_time/1000000) / qs.execution_count,
qs.total_elapsed_time,
total_elapsed_time_inSeconds = --Converted from microseconds
qs.total_elapsed_time/1000000,
st.text,qs.query_hash,
qp.query_plan
FROM
sys.dm_exec_query_stats AS qs
CROSS APPLY
sys.dm_exec_sql_text(qs.sql_handle) AS st
CROSS APPLY
sys.dm_exec_query_plan (qs.plan_handle) AS qp
where qs.last_execution_time between '2020-09-29 13:16:52.710' and '2020-09-29 23:16:52.710'
ORDER BY
qs.total_worker_time DESC;
--View waiting tasks per connection
SELECT st.text AS [SQL Text], c.connection_id, w.session_id,
w.wait_duration_ms, w.wait_type, w.resource_address,
w.blocking_session_id, w.resource_description, c.client_net_address, c.connect_time
FROM sys.dm_os_waiting_tasks AS w
INNER JOIN sys.dm_exec_connections AS c ON w.session_id = c.session_id
CROSS APPLY (SELECT * FROM sys.dm_exec_sql_text(c.most_recent_sql_handle)) AS st
WHERE w.session_id > 50 AND w.wait_duration_ms > 0
ORDER BY c.connection_id, w.session_id
GO
-- View waiting tasks for all user processes with additional information
SELECT 'Waiting_tasks' AS [Information], owt.session_id,
owt.wait_duration_ms, owt.wait_type, owt.blocking_session_id,
owt.resource_description, es.program_name, est.text,
est.dbid, eqp.query_plan, er.database_id, es.cpu_time,
es.memory_usage*8 AS memory_usage_KB
FROM sys.dm_os_waiting_tasks owt
INNER JOIN sys.dm_exec_sessions es ON owt.session_id = es.session_id
INNER JOIN sys.dm_exec_requests er ON es.session_id = er.session_id
OUTER APPLY sys.dm_exec_sql_text (er.sql_handle) est
OUTER APPLY sys.dm_exec_query_plan (er.plan_handle) eqp
WHERE es.is_user_process = 1
ORDER BY owt.session_id;
GO;

Stop, restore and restart MS SQL server

I have a build deployed on my local and I am able to access the application using https://localhost:8443/dashboard. Now, as a part of my junits, I am supposed to take the backup of the database used by this application and restore it to its earlier state as the application inserts some data across multiple tables as a part of the tests. So, I need to restore the DB before the tests run. Now, I am able to backup and restore it using java program (junit) but the problem is coming when the server is running. It says DB is already in use and can't restore. Is there a way to disconnect the db connection (user) used in the application, do the restoration and then connect back again so that I don't have to shutdown the server and start it manually again.
Using commands similar to following -
BACKUP DATABASE [Store] TO DISK = N'"+backuppath+"' WITH NOFORMAT, NOINIT,
NAME = 'demodb-full', SKIP, NOREWIND, NOUNLOAD, STATS = 10";
And
RESTORE DATABASE ["+dbName+"] FROM DISK='"+database_backup_location+"' WITH
REPLACE, MOVE '"+mdfLogicalName+"' TO
'"+getPropertyByKey("databse.dbfiles.location")+mdfLogicalName+".mdf', MOVE
'"+ldfLogicalName+"' TO
'"+getPropertyByKey("databse.dbfiles.location")+ldfLogicalName+".ldf';
Error that I am getting -
org.springframework.jdbc.UncategorizedSQLException: StatementCallback;
uncategorized SQLException for SQL [RESTORE DATABASE ....error code [3101];
Exclusive access could not be obtained because the database is in use.; nested
exception is com.microsoft.sqlserver.jdbc.SQLServerException: Exclusive access
could not be obtained because the database is in use.
I am already using the BACKUP and RESTORE commands. RESTORE command
gives error because the database is in use.
I've already answered, you just need to set your database offline this way:
alter database myDB set offline with rollback immediate
Doing this you'll disconnect all the users from your db and now you can make a RESTORE
You need to do below steps :
Backup the database
BACKUP DATABASE [Store] TO DISK = N'"+backuppath+"' WITH NOFORMAT, NOINIT,
NAME = 'demodb-full', SKIP, NOREWIND, NOUNLOAD, STATS = 10";
Kill existing connections
USE master;
DECLARE #kill varchar(8000); SET #kill = '';
SELECT #kill = #kill + 'kill ' + CONVERT(varchar(5), spid) + ';'
FROM master..sysprocesses
WHERE dbid = db_id('Store')
EXEC(#kill);
Then run the restore script.

processBuilder failing to load data on different host?

On host 'A' I start a transaction by inserting values to a table in DB. As soon as I insert, I call processBuilder to refresh host 'B' which in-turn should load the updated values from same DB table to cache. But the values are not getting loaded.
Is there any relation between processBuilder and transaction? Because the transaction is yet to complete on the host from where I am calling processBuilder.
I tried fetching values from DB before calling processBuilder on host 'A' and it is returning the values which was recently inserted (result set returns 10 rows) where as on host 'B' which is calling same select statement return 9 rows.
"as soon as I insert" red flag.
Yes, there is a relationship between the transaction and processBuilder. If the transaction is not committed, then all other sessions will not be able to see the changes. If you're used to programming in a DB app environment where autocommit is enabled and switch to a DB app environment where autocommit is disabled, then you are likely to run across this kind of problem.

How to check number of active connections in mysql database using java and close it

I want to check the number of active open connections in mysql database using java and close it.
I tried using query only
SHOW STATUS LIKE 'threads_connected';
With this can anyone tell me how to find and close the connection. I know when a database connection is established we need to close it immediately but for some other requirement i want to know active connections list and delete that list.
Any idea please suggest
I pulled aout the strings used bei HeidiSQL
First for getting a list with active connections(Processes):
SELECT ID, USER, HOST, DB, COMMAND, TIME, STATE, LEFT(INFO, 51200) AS Info FROM information_schema.PROCESSLIST
The for deleting a processes:
KILL {The ID of the Process};
It would probably be enough to SELECT only the ID of the Process but with that query you get all Information needed.

How to count open db connections?

I'm developing a web app using Java servlet to access Mysql db, how can I get the number of connections to my DB that is currently open ?
Edit :
I tried "show processlist", it showed me : 2695159, but that's not right, I'm just developing this new project, I'm the only user, couldn't have that many processes running, what I want is the number of users accessing my project's DB, not the number of all db users, but just the ones logged in to my database which has only one table.
Depending on your MySQL version, you can perform a select on
SELECT COUNT(*) FROM information_schema.PROCESSLIST;
and you can do a where between the user, database, and host IP.
For example:
USE information_schema;
SELECT COUNT(*) FROM PROCESSLIST WHERE db ="mycase" AND HOST LIKE "192.168.11.174%"
You could use the MySQL command show processlist to get the number of connections.
However that'll also show you any connections made with the same userID to the database which may not be coming from your servlet.
In general I would suggest that you're probably better off using a Connection Pool object (see http://java-source.net/open-source/connection-pools) to manage your connections to the MySQL server. This can increase performance by making DB connections persistent, so you don't always have the overhead of a new DB connection for each page load.
If your servlet needs to know the number of connections then your Connection Pool should come with a method that tells you how many connections are currently active.
show status like 'Threads_connected'
or
show global status like 'Threads_connected'
Not sure about the difference between those two in a user-context, and you might still suffer from the problem that you would see all connections, not only those from your app.
you can even check Threads_running to only see running threads (e.g not sleeping).
Run the following query, it lists out host name and no. of connections from each host:
SELECT host,count(host) FROM information_schema.processlist GROUP BY host;
show processlist
You can only select from Information_Schema.Processlist the data that belongs to you. It means you can use it for monitoring ONLY if you're logged in as root, otherwise you will be seeing the connections coming from your user you got logged in with.
If you want proper monitoring SQL, it will be:
SELECT variable_value
FROM INFORMATION_SCHEMA.GLOBAL_STATUS
WHERE variable_name='threads_connected'
You also can count open connection by show the status from Threads_connected variable name like this:
SHOW STATUS WHERE variable_name = 'Threads_connected';
Or you can also count the process list directly from information_schema.PROCESSLIST like below:
SELECT COUNT(*) FROM information_schema.PROCESSLIST;
You may use this
SHOW GLOBAL STATUS;
or
show global status like "Threads_connected";
from Connections status you can findout total number of connections.

Categories