I have problem with https services in my Gateway application ( which uses zuul )
it works great when proxying http services, but i have problems with proxying https services
I have exception
java.lang.IllegalStateException: Could not create URI object: Expected scheme-specific part at index 6: https:
at org.springframework.web.util.HierarchicalUriComponents.toUri(HierarchicalUriComponents.java:430) ~[spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.cloud.netflix.ribbon.RibbonClientConfiguration$OverrideRestClient.reconstructURIWithServer(RibbonClientConfiguration.java:184) ~[spring-cloud-netflix-core-1
at com.netflix.client.AbstractLoadBalancerAwareClient$1.call(AbstractLoadBalancerAwareClient.java:106) ~[ribbon-loadbalancer-2.1.5.jar:2.1.5]
at com.netflix.loadbalancer.reactive.LoadBalancerCommand$3$1.call(LoadBalancerCommand.java:303) ~[ribbon-loadbalancer-2.1.5.jar:2.1.5]
at com.netflix.loadbalancer.reactive.LoadBalancerCommand$3$1.call(LoadBalancerCommand.java:287) ~[ribbon-loadbalancer-2.1.5.jar:2.1.5]
at rx.internal.util.ScalarSynchronousObservable$4.call(ScalarSynchronousObservable.java:223) ~[rxjava-1.1.5.jar:1.1.5]
at rx.internal.util.ScalarSynchronousObservable$4.call(ScalarSynchronousObservable.java:220) ~[rxjava-1.1.5.jar:1.1.5]
at rx.Observable.unsafeSubscribe(Observable.java:8460) ~[rxjava-1.1.5.jar:1.1.5]
... 150 common frames omitted
aused by: java.net.URISyntaxException: Expected scheme-specific part at index 6: https:
at java.net.URI$Parser.fail(URI.java:2848) ~[na:1.8.0_92]
at java.net.URI$Parser.failExpecting(URI.java:2854) ~[na:1.8.0_92]
at java.net.URI$Parser.parse(URI.java:3057) ~[na:1.8.0_92]
at java.net.URI.<init>(URI.java:673) ~[na:1.8.0_92]
My Gateway config
server:
ssl:
key-store: classpath:my.jks
key-store-password: secret
key-password: secret
spring:
application:
name: mille-gateway
cloud:
config:
discovery:
enabled: true
serviceId: mille-config-server
eureka:
client:
healthcheck:
enabled: true
ribbon:
IsSecure: true
zuul:
ignoredServices: '*'
routes:
test:
path: /test/**
serviceId: mille-test2
test:
ribbon:
ReadTimeout: 5000
MaxAutoRetries: 2
IsSecure: true
My Registry ( Eureka ) server
server:
port: 8761
eureka:
instance:
hostname: localhost
client:
registerWithEureka: false
fetchRegistry: false
serviceUrl:
defaultZone: http://${eureka.instance.hostname}:${server.port}/eureka/
server:
enableSelfPreservation: false
My client configuration
spring:
application:
name: mille-test2
cloud:
config:
discovery:
enabled: true
serviceId: mille-config-server
eureka:
client:
healthcheck:
enabled: true
server:
port: 50000
ssl:
key-store: classpath:my.jks
key-store-password: secret
key-password: secret
eureka:
client:
enabled: true
serviceUrl:
defaultZone: http://localhost:8761/eureka/
instance:
nonSecurePortEnabled: false
securePortEnabled: true
securePort: ${server.port}
instanceId: ${spring.application.name}:${spring.application.instance_id:${random.value}}
statusPageUrl: https://${eureka.hostname}:${server.port}/info
healthCheckUrl: https://${eureka.hostname}:${server.port}/health
homePageUrl: https://${eureka.instance.hostname}:${server.port}/
secureVirtualHostName: ${spring.application.name}
metadataMap:
hostname: ${eureka.instance.hostname}
securePort: ${server.port}
What could be a problem ?
I have traced this problem in the debugger in Brixton.SR6.
The FeignLoadBalancer.reconstructURIWithServer(...) calls RibbonUtils.updateToHttpsIfNeeded(...) before calling the superclass method that actually adds the server to the uri.
The uri passed in is "", which is the normal case when the incoming url matches the zuul mapping exactly and using service discovery to get the server/port.
updateToHttpsIfNeeded() adds "https" and calls UriComponentsBuilder via build(true) to create an instance of HierarchicalUriComponents.
It then calls toUri() on that instance, which, because of the true passed in to build(), follows the encode == true path and calls new URI(toString()) (line 415). That toString() returns "https:" since we have set the scheme, but not the server or port.
The URI does not like that at all and throws java.net.URISyntaxException: Expected scheme-specific part at index 6: https:.
As to solving the problem, perhaps FeignLoadBalancer should not ensure https until after the server and port are set in the URI? Or perhaps RibbonUtils.updateToHttpsIfNeeded should set the server and port in the UriComponentsBuilder before calling toUri() - it has the Server instance.
This accounts for why it only happens for secure connections. I have not yet found any workaround when using exact url matches in the mapping, other than reverting to Brixton.SR5, which works fine, because the call to build() does not pass the true flag, so new URI() is not called. HTH
Edit: see also spring-cloud-netflix issue 1099.
spring-cloud spring-cloud-netflix netflix-zuul netflix-ribbon
Not sure if you were able to resolve this but I just ran into the same thing. It appears to only occur though when Zuul attempts to proxy a request via https (as you've indicated) to a service endpoint "root".
For instance, taking your Zuul config above - hitting the endpoint /test will likely cause that exception, however hitting any other endpoint proxied to the mille-test2 service (that isn't mapped to the root) will probably work test/something. At least it has for me. So as a temporary work around I've simply created different endpoints that extend the root such as /test/new. Kludgy, yes - but workable.
I'm curently using spring cloud Brixton.SR4.
Related
I've been trying to make two ssl to secure the communication between microservices. It is actually working when service A want to register into EurekaServer. But when I try to register eureka into the same eureka server or other instance of eureka I always got a bad certificate exception.
This is my application.yml from eureka server
server:
port: ${SERVER_SSL_PORT:${PORT:10100}}
ssl:
enabled: true
client-auth: need
key-alias: ${JKS_ALIAS}
key-password: ${JKS_KEY_PASSWORD}
key-store-type: ${JKS_TYPE}
key-store-provider: ${JKS_PROVIDER:SUN}
key-store: ${JKS_PATH}
key-store-password: ${JKS_KEY_PASSWORD}
trust-store: ${JKS_PATH}
trust-store-password: ${JKS_KEY_PASSWORD}
trust-store-provider: ${JKS_PROVIDER:SUN}
trust-store-type: ${JKS_TYPE}
## EUREKA CONFIGURATION
eureka:
client:
enabled: true
register-with-eureka: true
fetch-registry: false
service-url:
defaultZone: ${SVC_REGISTRY:https://localhost:14102/eureka}
prefer-same-zone-eureka: true
healthcheck:
enabled: true
tls:
enabled: true
key-store: ${JKS_PATH}
key-password: ${JKS_KEY_PASSWORD}
key-store-type: ${JKS_TYPE}
key-store-password: ${JKS_KEYSTORE_PASSWORD}
trust-store: ${JKS_PATH}
trust-store-password: ${JKS_KEY_PASSWORD}
trust-store-type: ${JKS_TYPE}
instance:
instance-id: ${SERVER_NAME:${spring.application.name}:${server.port}}#${eureka.instance.hostname}
hostname: ${SERVER_HOST:localhost}
home-page-url: https://${eureka.instance.hostname}:${server.port}${server.contextPath:}
status-page-url: https://${eureka.instance.hostname}:${server.port}${server.contextPath:}${management.endpoints.web.base-path:}/info
health-check-url: https://${eureka.instance.hostname}:${server.port}${server.contextPath:}${management.endpoints.web.base-path:}/health
secure-health-check-url: https://${eureka.instance.hostname}:${server.port}${server.contextPath:}${management.endpoints.web.base-path:}/health
lease-renewal-interval-in-seconds: 10
prefer-ip-address: true
metadata-map:
instanceId: ${eureka.instance.instance-id}
zone: ${SERVER_ZONE:local}
non-secure-port-enabled: false
secure-port-enabled: true
secure-port: ${server.port}
I also try to add this configuration to modify the JerseyClient
#Bean
#SneakyThrows
public DiscoveryClient.DiscoveryClientOptionalArgs discoveryClientOptionalArgs2() {
DiscoveryClient.DiscoveryClientOptionalArgs args = new DiscoveryClient.DiscoveryClientOptionalArgs();
EurekaJerseyClientImpl.EurekaJerseyClientBuilder clientBuilder = new EurekaJerseyClientImpl.EurekaJerseyClientBuilder()
.withClientName("DiscoveryClient-HTTPClient-Custom")
.withUserAgent("Java-EurekaClient")
.withConnectionTimeout(config.getEurekaServerConnectTimeoutSeconds() * 1000)
.withReadTimeout(config.getEurekaServerReadTimeoutSeconds() * 1000)
.withMaxConnectionsPerHost(config.getEurekaServerTotalConnectionsPerHost())
.withMaxTotalConnections(config.getEurekaServerTotalConnections())
.withConnectionIdleTimeout(config.getEurekaConnectionIdleTimeoutSeconds() * 1000)
.withEncoderWrapper(CodecWrappers.getEncoder(config.getEncoderName()))
.withDecoderWrapper(CodecWrappers.resolveDecoder(config.getDecoderName(), config.getClientDataAccept()))
.withCustomSSL(sslContext());
EurekaJerseyClient jerseyClient = clientBuilder.build();
args.setEurekaJerseyClient(jerseyClient);//Provide custom EurekaJerseyClient to override default one
return args;
}
#Bean
public SSLContext sslContext() throws Exception {
log.info("initialize ssl context bean with keystore {} ", env.getProperty("JKS_PATH"));
char[] password = env.getProperty("JKS_KEY_PASSWORD").toCharArray();
return new SSLContextBuilder()
.loadTrustMaterial(ResourceUtils.getFile(env.getProperty("JKS_PATH")), password).build();
}
But I keep getting this error
2022-09-01 18:44:01.522 INFO 26829 --- [tbeatExecutor-0]
c.n.d.s.t.d.RedirectingEurekaHttpClient : Request execution error.
endpoint=DefaultEndpoint{
serviceUrl='https://XXXXXXXXXXXXXXXXXXXX:DDDD/eureka/},
exception=javax.net.ssl.SSLHandshakeException: Received fatal alert:
bad_certificate
stacktrace=com.sun.jersey.api.client.ClientHandlerException:
javax.net.ssl.SSLHandshakeException: Received fatal alert:
bad_certificate
at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:187)
at com.sun.jersey.api.client.Client.handle(Client.java:652)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.put(WebResource.java:529)
at
Certificate should be fine since it is the same cert I used to make 2way SSL between service A and registry. I am also testing this on a real server with valid certificate.
Any clue on this?
Thanks in advance! Appreciate it
I found the root cause of my problem. I had this property as true
prefer-ip-address: true
but it should be false since my certificates have hostnames register, not IPs. So when it try to validate the certs it was comparing hostnames vs ip which are different and it was throwing bad_certificate exception.
I have a custom gateway filter that use the route written in application.yml,
When I want to change the route predicate from /test/myTest to /public the request return 404 it seemes like the "/" is not reconized because I tried to change /test-myTest to /public and it's work.
Any idea please how can I use a predicate with this format /name/name/**
this is my application.yml :
myTest-gateway:
default-uri: https://app
apps:
/test/myTest: mymicroservice-name
spring:
cloud:
gateway:
enabled: false
x-forwarded:
proto-enabled: false
routes:
- id: appsPrefixPathPublic
uri: ${myTest-gateway.default-uri}
predicates:
- AppPath=/test/myTest
filters:
- StripPrefix=1
- PrefixPath=/public
this is the error that I got it :
No content
< 404 NOT_FOUND Not Found < Content-Type: [application/json] <
Content-Length: [147]
{"timestamp":"2020-08-26T15:44:12.023+0000","path":"/test/myTest/whatever","status":404,"error":"Not
Found","message":null,"requestId":"e7baeeeb-6"}
java.lang.AssertionError: Status expected:<200 OK> but was:<404
NOT_FOUND>
From the documentation :
spring:
cloud:
gateway:
routes:
- id: nameRoot
uri: https://nameservice
predicates:
- Path=/name/**
filters:
- StripPrefix=2
When a request is made through the gateway to /name/blue/red, the request made to nameservice looks like nameservice/red.
You should change 1 to 2.
I have uploaded a saved configuration file in a BeanStalk application in a region to another BeanStalk application in another region.
While loading that config I got an error
Stack named 'awseb-e-sme7w3eym3-stack' aborted operation. Current
state: 'CREATE_FAILED' Reason: The following resource(s) failed to
create: [AWSEBLoadBalancer]
Creating load balancer failed Reason: Property Listeners cannot be
empty Any idea about this issue ?
See the config file
AWSConfigurationTemplateVersion: 1.1.0.0
EnvironmentConfigurationMetadata:
DateCreated: '1580272974000'
DateModified: '1580273310143'
Description: xxxxxxxxxxxxxxxxxxxxx
EnvironmentTier:
Name: WebServer
Type: Standard
OptionSettings:
AWSEBAutoScalingGroup.aws:autoscaling:updatepolicy:rollingupdate:
MaxBatchSize: '1'
MinInstancesInService: '1'
RollingUpdateEnabled: true
RollingUpdateType: Health
AWSEBAutoScalingLaunchConfiguration.aws:autoscaling:launchconfiguration:
EC2KeyName: xxxxxxxxxxxxxxxxxxx
AWSEBCloudwatchAlarmHigh.aws:autoscaling:trigger:
UpperThreshold: '60'
AWSEBCloudwatchAlarmLow.aws:autoscaling:trigger:
BreachDuration: '2'
LowerThreshold: '25'
MeasureName: CPUUtilization
Period: '1'
Statistic: Maximum
Unit: Percent
AWSEBLoadBalancerSecurityGroup.aws:ec2:vpc:
VPCId: vpc-xxxxxxxxxxxxxxxx
AWSEBV2LoadBalancerListener.aws:elbv2:listener:default:
ListenerEnabled: false
AWSEBV2LoadBalancerListener443.aws:elbv2:listener:443:
SSLCertificateArns: arn:aws:acm:us-east-2:xxxxxxxxxxx:certificate/xxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxxx
AWSEBV2LoadBalancerTargetGroup.aws:elasticbeanstalk:environment:process:default:
HealthCheckPath: /rest/account/ping
MatcherHTTPCode: '200'
Port: '80'
Protocol: HTTP
aws:autoscaling:launchconfiguration:
IamInstanceProfile: aws-elasticbeanstalk-ec2-role
SecurityGroups:
- sg-xxxxxxxxxxxxx
aws:ec2:instances:
InstanceTypes: t2.small
aws:ec2:vpc:
ELBSubnets: subnet-xxxxxxxxxxxxxxxxxx,subnet-xxxxxxxxxxxxxxxxxx,subnet-xxxxxxxxxxxxxxx
Subnets: subnet-xxxxxxxxxxxxxxxxx,subnet-xxxxxxxxxxxxxxxxx,subnet-xxxxxxxxxxxxxxxxx
aws:elasticbeanstalk:application:environment:
JDBC_CONNECTION_STRING: jdbc:mysql://xxxxxxxxxxxxxxxxxxxxxxxxxxxx?user=xxxxxxxx&password=xxxxxxxxxxx&rewriteBatchedStatements=true&characterEncoding=UTF-8
aws.accessKeyId: xxxxxxxxxxxxxxxxxx
aws.secretKey: xxxxxxxxxxxxxxxxxxxx
com.aws.secretManger.secret.name: xxxxxxxxxxxxxxx
com.aws.secretManger.secret.region: us-east-2
com.decsond.loggly.token: xxxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx#xxxxx
com.decsond.metakey: xxxxxxxxxxxxxxxxx/XXX==
com.decsond.mode: debug
com.decsond.server.db.environment: aws
com.decsond.server.dpBinaryColumn: xxxxxxxxxxxx
com.decsond.server.environment: xxxxxxxxxx
com.decsond.server.type: pms
aws:elasticbeanstalk:container:tomcat:jvmoptions:
JVM Options: -XX:+CMSClassUnloadingEnabled -Dmvel.disable.jit=true -Ddrools.permgenThreshold=0
Xms: 512m
Xmx: 1024m
aws:elasticbeanstalk:environment:
LoadBalancerType: application
ServiceRole: arn:aws:iam::xxxxxxxxxxxxxx:role/aws-elasticbeanstalk-service-role
aws:elasticbeanstalk:healthreporting:system:
SystemType: enhanced
aws:elasticbeanstalk:managedactions:
ManagedActionsEnabled: true
PreferredStartTime: SAT:03:01
aws:elasticbeanstalk:managedactions:platformupdate:
InstanceRefreshEnabled: true
UpdateLevel: minor
aws:elasticbeanstalk:xray:
XRayEnabled: true
aws:elbv2:listener:443:
DefaultProcess: default
ListenerEnabled: true
Protocol: HTTPS
Rules: ''
SSLPolicy: ELBSecurityPolicy-2016-08
Platform:
PlatformArn: arn:aws:elasticbeanstalk:us-east-2::platform/Tomcat 8.5 with Java 8
running on 64bit Amazon Linux/3.3.1
Any idea about the issue ?
The most likely reason is that you are referencing objects in the region from where the config was saved from.
Is this the first EB application / environment in the new region?
If it is, it's worth first creating a test application and environment, using the features you want ... that will give EB a chance to create all the region specific behind-the-scenes magic it relies on.
I am using Spring cloud stream with RabbitMQ.
I want to be able to configure message and query properties from source code and not from a property file (as they mention in their docs).
For example with the classic Java Client for RabbitMq i can do something like that to create a queue with the properties i want:
//qName, passive, durable, exclusive auto-delete
channel.queueDeclare("myQueue", true, false, false, , false , null);
Any ideas on how can i achieve the same thing using Spring cloud stream?
Inside of "application.yml" you can add all this values , following is example
spring:
cloud:
stream:
instance-count: 1
bindings:
input:
consumer:
concurrency: 2
maxAttempts: 1
group: geode-sink
destination: jdbc-event-result
binder: rabbit
rabbit:
bindings:
input:
consumer:
autoBindDlq: true
republishToDlq: true
requeueRejected: false
rabbitmq:
username: ur-user-name
password: ur-password
host: rabbitmq-url-replace-here
port: 5672
datasource:
platform: mysql
url: jdbc:mysql-url-replace-here
username: ur-user-name
password: ur-password
driverClassName: com.mysql.jdbc.Driver
datasource:
tomcat:
max-wait: 300
min-idle: 10
max-idle: 100
aggregator:
groupCount: 2
batchSize: 1000
batchTimeout: 1000
Updated :
https://cloud.spring.io/spring-cloud-static/spring-cloud-stream-binder-rabbit/2.2.0.M1/spring-cloud-stream-binder-rabbit.html
https://github.com/spring-projects/spring-xd/blob/master/spring-xd-dirt/src/main/resources/application.yml
After digging in their documentation and with the help of #vaquar khan I found out that the only way to do it is from your property file.
application.yml
spring:
cloud:
stream:
bindings:
queue_name :
destination: queue_name
group: your_group_name
durable-subscription : true
This will declare a durable, non-deleting and non-exclusive queue.
I'm trying to set up RabbitMQ with Spring Cloud Stream Support
I have a couple consumers and producers. One of the producers should produce messages to a separate virtual host on a same RabbitMQ instance (later it might be different physical instances).
application.yaml
spring:
cloud:
stream:
binders:
binder1:
type: rabbit
defaultCandidate: false
inheritEnvironment: false
environment:
spring:
rabbitmq:
host: localhost
port: 5672
virtual-host: virtual-host-1
username: guest
password: guest
binder2:
type: rabbit
defaultCandidate: false
inheritEnvironment: false
environment:
spring:
rabbitmq:
host: localhost
port: 5672
virtual-host: virtual-host-2
username: guest
password: guest
bindings:
test:
binder: binder1
coordinates:
destination: coordinates
binder: binder1
events:
destination: events
binder: binder1
events_output:
destination: events
binder: binder1
tasks:
destination: tasks
binder: binder2
The goal is that binding tasks should use vhost virtual-host-2. Other bindings should use vhost virtual-host-1.
However binder value seems to be ignored and default rabbit binder is taken into account with default settings on application startup.
I noticed it while debugging the runtime:
The binder value on each binding is NULL. Although the value is explicitly provided in properties.
If I set defaultCandidate of any of the binders to true then that binder settings will be used as a replacement for default one.
Is something misconfigured?
This is one of the reasons why I don't like yaml. It's hard to track what may be misconfigured. In any event, here is the working example I just tried.
spring:
cloud:
stream:
bindings:
input:
binder: rabbit1
group: vhost1-group
destination: vhost1-queue
output:
binder: rabbit2
destination: vhost2-queue
binders:
rabbit1:
type: rabbit
environment:
spring:
rabbitmq:
host: localhost
virtual-host: vhost1
rabbit2:
type: rabbit
environment:
spring:
rabbitmq:
host: localhost
virtual-host: vhost2
I just copied/pasted your config; fixed some indentation and it worked fine for me...
spring:
cloud:
stream:
binders:
binder1:
type: rabbit
defaultCandidate: false
inheritEnvironment: false
environment:
spring:
rabbitmq:
host: localhost
port: 5672
virtual-host: virtual-host-1
username: guest
password: guest
binder2:
type: rabbit
defaultCandidate: false
inheritEnvironment: false
environment:
spring:
rabbitmq:
host: localhost
port: 5672
virtual-host: virtual-host-2
username: guest
password: guest
bindings:
test:
binder: binder1
tasks:
destination: tasks
binder: binder2
#SpringBootApplication
#EnableBinding(Foo.class)
public class So56462671Application {
public static void main(String[] args) {
SpringApplication.run(So56462671Application.class, args);
}
}
interface Foo {
#Input
MessageChannel test();
#Input
MessageChannel tasks();
}
and
defaultCandidate: false
inheritEnvironment: false
were incorrectly indented, but I got a YAML parse error.