We Keep Coding

Httpclient and Zipkin Java

My and project team are looking to add Zipkin logging and tracing to our current project. We are working in an microservice environment using Spring Boot (Java 17) and cloud foundery. For the communication between Microservices we are using HttpClient. From what I've gathered from the documentation Zipkin requires an RestTemplate to function. However we don't have time to change this.
We were able to implement Zipkin in every individual project. However, every call generates their own Trace ID. I think we need to configure the HttpClient to work in tandem with Zipkin, however the documentation is not very clear and I have been unable to find anything that explains how to do this.
What can I try on this? I've included the config and dependencies below.
spring:
application:
name: Application_1
zipkin:
baseUrl: http://localhost:9411
sleuth:
sampler:
probability: 1.0
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-sleuth-zipkin</artifactId>
<version>3.1.3</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-sleuth</artifactId>
<version>3.1.3</version>
</dependency>

Spring Dependency Mess - conflict with Spring Boot 2.5.4 and Spring Cloud 3.0.3

I am trying to update an application which already pulls in the kitchen sink (or perhaps a few, they're joined at the hip) and I am sorting through version conflicts.
I want to update to Spring Boot 2.5+ and also use Spring Cloud Consul - I am attempting to pull in:
spring-cloud-starter-consul-discovery:3.0.3
spring-boot:2.5.4
For bonus points, within spring-cloud-starter-consul-discovery, I am seeing that it pulls in reactor-core:3.4.6 and at the same time reactor-extra:3.4.3 (which pulls in reactor-core:3.4.5). The list goes on and on ...
https://search.maven.org/artifact/org.springframework.cloud/spring-cloud-starter-consul-discovery/3.0.3/jar - original point of contention is that it pulls in spring boot 2.4.6 ... it was advertised as supporting 2.5+, then shouldn't the version reference 2.5+?
https://search.maven.org/artifact/org.springframework.cloud/spring-cloud-loadbalancer/3.0.3/jar - this to me is just plain laziness, right below reactor-core is reactor-extra, why wouldn't the Spring developers make extra pull in the same version of core? See: https://search.maven.org/artifact/io.projectreactor.addons/reactor-extra/3.4.3/jar
While this is a trivial problem to solve, it shouldn't be my problem. Am I missing something, or is this just the way it is and I shouldn't expect more?

First of all, you need to look at this compatibility matrix between cloud and boot dependencies. Then, you need (for example) to generate your bom, where you import
the correct cloud dependencies bom
spring boot dependencies bom
These boms, internally, either import other boms, like for example consul, the one you are interested in, which is at version 2.2.8.RELEASE. Look in the properties tag in that file and see this:
<spring-cloud-consul.version>2.2.8.RELEASE</spring-cloud-consul.version>
specifically:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-consul-dependencies</artifactId>
<version>${spring-cloud-consul.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
You can then look at the specific consul bom and see that the version consul-discovery is:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-consul-discovery</artifactId>
<version>${project.version}</version>
</dependency>
Same pattern to find out what version is where can be done for reactor dependencies.
From my 10 minutes investing into this, I don't see a version of spring-cloud-starter-consul-discovery:3.0.3 that would be included in a cloud-dependecies.
You could still try to force a certain version of a dependency. We just recently had such a problem in spring-cloud-kubernetes, internally.
This may or may not work, though.

Classpath issue preventing pipeline to be run on Google Dataflow

I'm new to a project where we have a spring-boot application running on GKE receiving (via Kafka) and publishing events via Pub/Sub. Consumers of these events might want to have these events replayed and we want them to request this via the REST API of our application. Since the application stores the events in GCS before publishing, we thought Apache Beam pipelines run with DataFlow should do the trick.
One "replay request" might result in multiple pipelines, since the events in GCS are stored in folder structures containing the date (e.g. gs://<entity>/2020/12/13/event.json) and depending on how much history the consumer needs, we create a pipeline per day of events.
I'm fairly confident that the logic of defining and submitting pipelines is correct, since the application is able to perform this on a local Kubernetes cluster with the DirectRunner.
On DataFlow I run into the issue summarized here. Spawning a worker (org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness) fails due to a classpath issue:
Caused by: java.lang.NoClassDefFoundError: org/apache/beam/sdk/options/PipelineOptions
I can see that my jar that should have the correct dependencies on the classpath when DataFlow spawns the worker (Omitted most parameters):
java
-cp
/opt/google/dataflow/batch/libshuffle_v1.jar:/opt/google/dataflow/batch/dataflow-worker.jar:/opt/google/dataflow/slf4j/jcl_over_slf4j.jar:/opt/google/dataflow/slf4j/log4j_over_slf4j.jar:/opt/google/dataflow/slf4j/log4j_to_slf4j.jar:/var/opt/google/dataflow/app-6BkavP-0nx4wHMC__85sdbCjJQa7QcQcOxGSQL5huMU.jar
...
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness
After playing around with different scopes of the beam dependencies, because I suspected a clash with the google-dataflow.jar, I haven't seen any change. I'm a bit clueless on where to continue looking. I'm using beam library version 2.27.0 and these are the ones referred to in my pom.xml:
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-direct-java</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-extensions-google-cloud-platform-core</artifactId>
<version>${beam.version}</version>
</dependency>
Any advice is much appreciated.

The class org/apache/beam/sdk/options/PipelineOptions is found in the core Java SDK. The artifact is beam-sdks-java-core. This is not baked in to the Dataflow worker, but is part of the expected staged files.
The DataflowRunner by default will attempt to stage every file that it finds on the classpath. If there is anything about your environment or application that affects its ability to do this, you will need to add the SDK dependency yourself.

Camel AMQP - AMQConnectionFactory ClassNotFound

I'm using Camel 2.13.3 and trying to establish a connection via AMQP to a remote ActiveMQ instance.
According to the Camel AMQP docs is should be sufficient to add the following dependency
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-amqp</artifactId>
<version>2.13.1</version>
</dependency>
It then indicates that you should configure the jms component to use a connection factory supplied by the QPID project. The docs page uses org.apache.qpid.amqp_1_0.jms.impl.ConnectionFactoryImpl, and the results of other google searches indicate that org.apache.qpid.client.AMQConnectionFactory could be used.
However, the org.apache.qpid dependencies do not appear to have been added to the project and, unsurprisingly, I get a ClassNotFoundException when I run it.
I considered downloading the qpid dependency separately, but their web site seems to indicate that the qpid client project has been deprecated and replaced by something else ( QPID Messaging API if I remember correctly )
Can anyone point me in the right direction?

should be sufficient
The Camel docs you linked to does not state that. It just says this dependency is needed, doesn't say anything about additional dependencies. Just looked inside the jar you're using, and it does not contain qpid-client classes. You should add that dependency to your pom as well. For AMQP 0.x, there is a good chance you'll need JMS spec dependency as well:
<dependency>
<groupId>org.apache.qpid</groupId>
<artifactId>qpid-client</artifactId>
<version>0.32</version> <!-- replace with appropriate version -->
</dependency>
<dependency>
<groupId>org.apache.geronimo.specs</groupId>
<artifactId>geronimo-jms_1.1_spec</artifactId>
<version>1.0</version>
</dependency>
If you're using AMQP 1.0,
<dependency>
<groupId>org.apache.qpid</groupId>
<artifactId>qpid-jms-client</artifactId>
<version>0.3.0</version>
</dependency>

Support for X509PKIPathv1 in xws-security for Spring-WS

I'm trying to send a request to an existing webservice. This webservice is not governed by me. The security policy of this webservice requires me to send my complete certificate chain in my SOAP request. My certificate chain contains 3 certificates. There are no issues with the setup of the certificate chain, as I'm able to test it's validity (and have done so).
The security configuration for this setup (= sending the complete certificate chain in the request), is:
<xwss:Sign id="signature">
<xwss:X509Token
certificateAlias="alias"
keyReferenceType="Direct"
valueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509PKIPathv1" />
</xwss:Sign>
I'm trying to achieve this through Spring-WS. Spring-WS uses spring-ws-security for security. Spring-ws-security delegates to xws-security.
<dependency>
<groupId>org.springframework.ws</groupId>
<artifactId>spring-ws-security</artifactId>
<version>2.1.0.RELEASE</version>
<exclusions>
<exclusion>
<groupId>org.apache.ws.security</groupId>
<artifactId>wss4j</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.xml.wsit</groupId>
<artifactId>xws-security</artifactId>
</exclusion>
</exclusions>
</dependency>
Xws-security comes in 2 flavors:
<dependency>
<groupId>com.sun.xml.wsit</groupId>
<artifactId>xws-security</artifactId>
<version>1.3.1</version>
</dependency>
and
<dependency>
<groupId>com.sun.xml.wss</groupId>
<artifactId>xws-security</artifactId>
<version>3.0</version>
</dependency>
The first one is used by Spring WS Security. The second is legacy.
Applying my XWSS configuration in xws-security is done in a class called BinarySecurityToken. BinarySecurityToken has a field called
valueType
The JavaDoc of valueType says it has support for X509PKIPathv1 (among others). However, it does not, as stated by this setter:
protected void setValueType(String valueType) {
if (!(MessageConstants.X509v3_NS.equals(valueType)||MessageConstants.X509v1_NS.equals(valueType))) {
log.log(Level.SEVERE,"WSS0342.valtype.invalid");
throw new RuntimeException("Unsupported value type: " + valueType);
}
this.valueType = valueType;
}
The class MessageConstants does not (even) have a static for X509PKIPathv1. When I run my code, I get the expected result:
Unsupported value type: http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509PKIPathv1
I was able to look at the source code of the legacy com.sun.xml.wss.xws-security:3.0. Despite my efforts, I have not found the source code of com.sun.xml.wsit.xws-security-1.3.1. However I believe the code is the same. I tried both libraries and both give me the same exception. I tried it, using the default spring-ws-security and using explicit dependency declarations to both libraries (one at a time).
My questions:
Has anyone been able to use xws-security for generating an X509 signature with a valueType of X509PKIPathv1 and a keyReferenceType that is Direct?
Do other xws-security implementations exist that offer this? Or should I look at a completely different approach like Wss4j?
I have considered re-writing BinarySecurityToken, but that would probably also imply rewriting the X509 signing of SignatureProcessor in DSIG.

Interesting problem you got there.
As far as I could tell with my Google-fu, there exists support for #X509PKIPathv1 in some projects (e.g., Oracle's XMLSec or Open SAML), however it is not widespread and even application like Soap UI don't support it for SOAP-WS.
Not only that, but other languages/frameworks have the same lack of support, like Delphi and .NET, IBM JRE.
What you could do, based on this SO and especially this SO is implementing your own WebServiceTemplate / WebServiceMessageSender.

The valueType can be #X509v3, #X509PKIPathv1
That is found here
XWS-SecurityIntro4
Have you tried those values specifically instead of a URL?

This pull request will enable the ability to use X509PKIPathv1.