Am getting error :
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: org.bouncycastle.jce.exception.ExtCertPathValidatorException: Additional certificat
e path checker failed.
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:362)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:259)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1496)
... 73 more
Caused by: org.bouncycastle.jce.exception.ExtCertPathValidatorException: Additional certificate path checker failed.
at org.bouncycastle.jce.provider.RFC3280CertPathUtilities.wrapupCertF(Unknown Source)
at org.bouncycastle.jce.provider.PKIXCertPathValidatorSpi.engineValidate(Unknown Source)
at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292)
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:357)
... 79 more
Caused by: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on signature algorithm: SHA1WithRSAEncryption
at sun.security.provider.certpath.AlgorithmChecker.check(AlgorithmChecker.java:278)
... 83 more
during a handshake
I have located my java.security file in the path jdk/jre/lib/security
and changed properties
jdk.certpath.disabledAlgorithms=
jdk.jar.disabledAlgorithms=
jdk.tls.disabledAlgorithms=
So now no algorithmus are on the list, but I still getting same error !?
My certificate is signed with SHA1WITHRSA / RSA 1024 Bits , and I need to use that one.
Anyone have any Idea what could I try also , java version 1.8.0_141, running on JBoss 6.2
Related
I am loading one of the ssl certificate from a .crt file to access one resource. But when I make the call to that intranet resource. I get following exception. Can someone suggest how to overcome this problem? Also can I disable a DNS check
...
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Subtree check for certificate subject alternative name failed.
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:362)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:270)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1496)
... 81 more
Caused by: java.security.cert.CertPathValidatorException: Subtree check for certificate subject alternative name failed.
at org.bouncycastle.jce.provider.RFC3280CertPathUtilities.processCertBC(Unknown Source)
at org.bouncycastle.jce.provider.PKIXCertPathValidatorSpi.engineValidate(Unknown Source)
at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292)
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:357)
... 87 more
Caused by: org.bouncycastle.jce.provider.PKIXNameConstraintValidatorException: DNS is not from a permitted subtree.
at org.bouncycastle.jce.provider.PKIXNameConstraintValidator.checkPermittedDNS(Unknown Source)
at org.bouncycastle.jce.provider.PKIXNameConstraintValidator.checkPermitted(Unknown Source)
I found answer to this problem, sharing if it saves time for anybody else.
What is been observed is that in some cases only CA root certificate is not sufficient. But when other intermediate certs are also loaded then this issue didnt occur. So basically the whole chain certificates were needed to overcome this problem. As a combination of bouncy castle library upgrade and including whole chain certs worked for me.
For sometime I am noticing following error consistently, when servers are trying to connect other services using https. What action is to be taken to resolve this. Interestingly, it is happening only on few servers.
{http://xml.apache.org/axis/}stackTrace:javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on signature algorithm: SHA256WithRSAEncryption
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1959)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
....
....
at com.xyz.rest.XyzService.call(XyzService.java:1239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on signature algorithm: SHA256WithRSAEncryption
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1959)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1514)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:961)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1072)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
at org.apache.axis.components.net.JSSESocketFactory.create(JSSESocketFactory.java:186)
at org.apache.axis.transport.http.HTTPSender.getSocket(HTTPSender.java:191)
at org.apache.axis.transport.http.HTTPSender.writeToSocket(HTTPSender.java:404)
at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:138)
... 22 more
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on signature algorithm: SHA256WithRSAEncryption
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:362)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:270)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1496)
... 33 more
Caused by: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on signature algorithm: SHA256WithRSAEncryption
at org.bouncycastle.jce.provider.RFC3280CertPathUtilities.prepareNextCertO(Unknown Source)
at org.bouncycastle.jce.provider.PKIXCertPathValidatorSpi.engineValidate(Unknown Source)
at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292)
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:357)
... 39 more
Thanks all, but I found the answer to this problem. Had an older version of bouncy castle jar, after upgrading the the jar bcprov-jdk15on-1.58.jar, the "Algorithm constraints check failed on signature algorithm: SHA256WithRSAEncryption" error is no longer occurring.
Please, review the java configuration related to TLS in the affected servers, the SDK uses the security properties jdk.certpath.disabledAlgorithms and jdk.tls.disabledAlgorithm to disable algorithms during TLS protocol negotiation, and maybe SHA256WithRSAEncryption has been disabled.
You can find the value of these properties in the <install_dir>/jre/lib/security/java.security file of your Java installation.
Please, consider read this related article, it is for IBM SDK 8, but the behavior described should be the same or very similar for other SDK versions and vendors.
Also, consider to enable the java.security.debug environment property with a value of certpath or all when running your program in the affected servers:
-Djava.security.debug=certpath
It will provide you guidance and valuable information about what is going on and why Java is complaining about the algorithm.
I have two different key pair values which generated using Java keytool and stored in two different files called keystore1.jks and keystore2.jks.
What I did is I have imported the key pair from keystore2.jks to keystore1.jks by the below command
keytool -importkeystore -srcstoretype JKS -srckeystore <source_keystorfile> -deststoretype JKS -destkeystore <keystorfile_to_import_keypair>
I have added the keystore1.jks to server to listen in ssl using this keystore.
Now I have import the public key from the file keystore2.jks to truststore file named truststore.jks
From this truststore.jks file when I try to connect to the server which is listen in ssl using keystore1.jks, server unfortunately not accepting the connection and throwing an exception as below
javax.jms.JMSException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: signature check failed
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1298)
at org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1382)
at org.apache.activemq.ActiveMQConnection.createSession(ActiveMQConnection.java:309)
at com.sample.ssl.job.handler.MessageQueueLocator.getJmsSession(Unknown Source)
at com.sample.ssl.job.handler.MessageQueueLocator.sendMessageToGeneralQueue(Unknown Source)
at com.sample.ssl.communication.JobResposeDispatcherInvoker.dispatchStartupMessage(Unknown Source)
at com.sample.ssl.job.MessageDispatchJob.dispatchStartupMessage(Unknown Source)
at com.sample.ssl.job.MessageDispatchJob.execute(Unknown Source)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:529)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: signature check failed
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1731)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:241)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:235)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1206)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:136)
at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:593)
at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:529)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:925)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1170)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:637)
at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:89)
at org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:115)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:181)
at org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:255)
at org.apache.activemq.transport.WireFormatNegotiator.sendWireFormat(WireFormatNegotiator.java:168)
at org.apache.activemq.transport.WireFormatNegotiator.sendWireFormat(WireFormatNegotiator.java:84)
at org.apache.activemq.transport.WireFormatNegotiator.start(WireFormatNegotiator.java:74)
at org.apache.activemq.transport.failover.FailoverTransport.doReconnect(FailoverTransport.java:844)
at org.apache.activemq.transport.failover.FailoverTransport$2.iterate(FailoverTransport.java:135)
at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: signature check failed
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:289)
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:263)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:184)
at sun.security.validator.Validator.validate(Validator.java:218)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:126)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:209)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:249)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1185)
... 21 more
Caused by: java.security.cert.CertPathValidatorException: signature check failed
at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:139)
at sun.security.provider.certpath.PKIXCertPathValidator.doValidate(PKIXCertPathValidator.java:330)
at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:178)
at java.security.cert.CertPathValidator.validate(CertPathValidator.java:250)
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:275)
... 28 more
Caused by: java.security.SignatureException: Signature does not match.
at sun.security.x509.X509CertImpl.verify(X509CertImpl.java:421)
at sun.security.provider.certpath.BasicChecker.verifySignature(BasicChecker.java:133)
at sun.security.provider.certpath.BasicChecker.check(BasicChecker.java:112)
at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:117)
... 32 more
I don't know in which I have made a mistake. In my case half portion of clients use trust store of the key pair from keystore1.jks and remaining use keystore2.jks.
The clients those use trust store of keystore1.jks connecting to the server fine. But those clients use trust store of keystore2.jks is unable to connect.
Please help me in this context to resolve. If I have made anything wrong in my way please give a correct path. Thanks in advance.
I did not understand some things from you..
You have two files that have passwords.. You import the second file to the first.
So. why do you need to use a second file?
I think the command should be: (backup first)
keytool -importkeystore -srckeystore keystore2.jks -deskeystore keystore1.jks
I'm new one for web site crawling process in java, currently I'm facing the issue SSLHandshakeException, while crawling the site's like http:// based no issues on moving to https:// having the issues, most wide I've searching the result always being certificate issue, still i didn't get specific solution for it!, please some one give the solution for it. Here my output:
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1520)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:182)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:176)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:975)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:123)
at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:511)
at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:449)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:817)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1029)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1056)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1040)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:405)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:170)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:133)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:512)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:534)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:493)
at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:205)
at org.jsoup.helper.HttpConnection.get(HttpConnection.java:194)
at jobcrawl.jobcrawl.main(jobcrawl.java:112)
Caused by:
sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:251)
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:234)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:148)
at sun.security.validator.Validator.validate(Validator.java:218)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:126)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:209)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:249)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:954)
... 16 more
Caused by:
java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors
at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:195)
at java.security.cert.CertPathValidator.validate(CertPathValidator.java:250)
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:246)
I tried to install Oracle Entitlements Server Client.
When I call
config.cmd -smConfigId Sample-SM -prpFileName C:\oracle\product\11.1.2\as_1\oessm\SMConfigTool\smconfig.java.controlled.prp
I got this Exception:
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Algorithm constraints check failed: MD5withRSA
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1884)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:804)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1016)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:702)
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:122)
at java.io.OutputStream.write(OutputStream.java:75)
at oracle.security.oes.enroll.EnrollmentClient.writeToSocket(EnrollmentClient.java:330)
at oracle.security.oes.enroll.EnrollmentClient.enroll(EnrollmentClient.java:161)
at oracle.security.oes.enroll.EnrollmentClient.main(EnrollmentClient.java:478)
at oracle.security.oes.tools.EnrollmentTool.doEnroll(EnrollmentTool.java:103)
at oracle.security.oes.tools.SMConfigTool.doEnrollment(SMConfigTool.java:1192)
at oracle.security.oes.tools.SMConfigTool.run(SMConfigTool.java:617)
at oracle.security.oes.tools.SMConfigTool.main(SMConfigTool.java:546)
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Algorithm constraints check failed: MD5withRSA
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:350)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:260)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:231)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:126)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1323)
... 15 more
Caused by: java.security.cert.CertPathValidatorException: Algorithm constraints check failed: MD5withRSA
at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:159)
at sun.security.provider.certpath.PKIXCertPathValidator.doValidate(PKIXCertPathValidator.java:351)
at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:191)
at java.security.cert.CertPathValidator.validate(CertPathValidator.java:279)
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:345)
... 21 more
sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Algorithm constraints check failed: MD5withRSA
Can you help me to find a reason?
The problem is caused by Oracle disabling hash algorithms which are no longer considered to be secure. Take a look at
JRE_HOME/lib/security/java.security
It contains the following properties:
jdk.certpath.disabledAlgorithms
jdk.tls.disabledAlgorithms
You can adjust them appropriately. For example, remove MD5 from the former and MD5withRSA from the latter.
Hint for docker images:
there is additional config file /etc/crypto-policies/back-ends/java.config in some docker images like keycloak in my case which overrides values in java.security
keyser gave direction for answer in comment.
Problem was in key's length. In short: "Starting from 7u40, the use of x.509 certificates with RSA keys less than 1024 bits in length is restricted."
So the right way to solve this problem it is using certificates with key's length at least 2048 bits.