We Keep Coding

Buildx failed with error: "No such file or directory" when trying to push docker on Github

I'm very new to Docker and I'm trying to get my Github repository setup with a docker when I push an update. However, when the github workfile script goes to push my code to Docker I get the following error.
Error: buildx failed with: error: failed to solve: failed to compute cache key: failed to walk /var/lib/docker/tmp/buildkit-mount1164878607/target: lstat /var/lib/docker/tmp/buildkit-mount1164878607/target: no such file or directory
It appears Docker can't access the file I set for it in my Dockerfile yet no amount of tweaking with the code appears to fix it, aside for me completely removing the COPY tag which renders my Docker unrunnable, though it does send. I'm using maven compiler plugin along with maven shade plugin and have setup my pom.xml file accordingly.
Here is my Github WorkFile:
name: Docker
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Set up JDK
uses: actions/setup-java#v3.3.0
with:
java-version: 18
distribution: adopt
- name: Build with Maven
run: mvn clean package
- name: Login to DockerHub
uses: docker/login-action#v2.0.0
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_TOKEN }}
- name: Build and push Docker images
uses: docker/build-push-action#v3.0.0
with:
push: true
tags: sudden/discordbot:latest
Here is my DockerFile:
FROM openjdk:11-jdk-slim
WORKDIR /tmp
COPY target/discordbot-1.0-SNAPSHOT-shaded.jar app.jar
EXPOSE 8080
ENTRYPOINT ["java","-jar","/app.jar"]
What I've tried:
Copying another random file in my "src" directory.
Changing the name of my jar file and changing the path.
Changing my WORKDIR.
Changed the Docker's "context" and other tags in my Workfile.
Any help is appreciated. This might be a simple fix, but I've been stuck on it for a day despite my best efforts so asking here is a last resort. Thanks.

Make sure that you haven't excluded your target folders in your .dockerignore file.
That's what fixed it for me after several days of despairing search...

Configure CircleCI for Maven project using Nexus repo

I'm using external Nexus repository for java maven project and I trying to config circleCI config. At first I need to somehow say CircleCI to look into env variables for credentials during build.
In grade driven java projects all going fine without any extra configs, but in maven it says that can't have access to nexus.
Have anybody experience with it?
It's my first time configuring CircleCI
UPD1:
For authorisation I added env variables for username, password, server url and attributes to context on CircleCI Organization Settings page
I using this ORB and there I have found the names of env variables I have to use as default names.
Created circleci config file:
version: 2.1
orbs:
nexus-platform-orb: sonatype/nexus-platform-orb#1.0.28
workflows:
main:
jobs:
- nexus-platform-orb/nexusjob:
context: MyContext
And have this error:
Caught: java.io.FileNotFoundException: /tmp/workspace (Is a directory)
java.io.FileNotFoundException: /tmp/workspace (Is a directory)
at NexusPublisher.run(NexusPublisher.groovy:59)
Exited with code exit status 1
But when I added command to create file inside and used it, I steel have the same problem.
For a which file or directory circleci is looking for?
UPD1:
I a bit updated my config file:
version: 2.1
orbs:
maven: circleci/maven#1.3.0
nexus-platform-orb: sonatype/nexus-platform-orb#1.0.28
jobs:
install-nexus:
executor: nexus-platform-orb/nexus-platform-cli
steps:
- checkout
- nexus-platform-orb/install
workflows:
main:
jobs:
- install-nexus
- nexus-platform-orb/nexusjob:
context: MyContext
workspace: tmp/workspace/
requires:
- install-nexus
As a result I have an error running this job:
/bin/sh: curl: not found
Because command nexus-platform-orb/install contains this line:
curl -L
https://groovy.jfrog.io/artifactory/libs-release-local/org/codehaus/groovy/groovy-binary/<<
parameters.groovy-version >>/groovy-binary-<< parameters.groovy-version
>>.zip -o apache-groovy-binary.zip
But adding command installing curl gives me one another error:
steps:
- checkout
- run: apk update && apk add curl curl-dev bash
- nexus-platform-orb/install
#!/bin/sh -eo pipefail
apk update && apk add curl curl-dev bash
ERROR: Unable to lock database: Permission denied
ERROR: Failed to open apk database: Permission denied
Exited with code exit status 99
And I have no idea how to fix it, because only way I found is to add RUN root to the Dockerfile. But it not works.
UPD2:
I spent more than week to understand how to fix it and posted my resolution as an answer.
Hope it will save time for someone.

The resolution of this problem is to manually say what exactly file to use for getting credentials.
This is how my config.yml file looks after all:
version: 2.1
orbs:
maven: circleci/maven#1.3.0
jobs:
build:
executor: maven/default
working_directory: ~/my-project
steps:
- checkout
- restore_cache:
keys:
- v1-dependencies-{{ checksum "pom.xml" }}
- v1-dependencies-
- run:
name: Test
command: |
mvn -s ./settings.xml clean test
- save_cache:
paths:
- ~/.m2
key: v1-dependencies-{{ checksum "pom.xml" }}
workflows:
main:
jobs:
- build
I do not need nexus orb anymore. I used maven orb for an executor.
Added command mvn -s ./settings.xml clean test to run tests and says in which file nexus credentials are.
settings.xml file:
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
https://maven.apache.org/xsd/settings-1.0.0.xsd">
<servers>
<server>
<id>MYPROJECT</id>
<username>*****</username>
<password>*****</password>
</server>
</servers>
</settings>
And link to Nexus repository I added in pom.xml file:
<repositories>
<repository>
<id> MYPROJECT </id>
<name>*****</name>
<url>https://nexus.*****/</url>
</repository>
</repositories>
Hope this case will help someone.

DB connection during integration tests works on local, but not working on Google Cloud Build

I'm deploying Java 11 REST API to GKE using GitHub, Gradle, and Docker.
The following errors are only happened on Google Cloud Build, not on the local environment. According to the error, it seems the app can't find the DB server(Google Cloud SQL) from Google Cloud Build. I tried both public and private IP, but the results were the same:
...
Step #0 - "Build": 2021-03-11 04:12:04.644 INFO 115 --- [ Test worker] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
Step #0 - "Build": 2021-03-11 04:12:35.855 ERROR 115 --- [ Test worker] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Exception during pool initialization.
Step #0 - "Build":
Step #0 - "Build": com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
Step #0 - "Build":
Step #0 - "Build": The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
...
Step #0 - "Build": Caused by: java.net.SocketTimeoutException: connect timed out
...
This happened after I added integration tests. The app deployed successfully after I removed the tests. So, I can remove the integration tests to avoid this issue. The thing is, I want to keep the tests if possible because there are things that we can't test with unit tests.
This is the Dockerfile I'm using for deployments to GKE. RUN gradle build --no-daemon -i --stacktrace is where the error occurs during the test task:
ARG APP_NAME=test-api
ARG GRADLE_USER_HOME_PATH=/home/gradle/cache_home/
#cache dependencies to reduce downloads
FROM gradle:6.8-jdk11 AS cache
ARG APP_NAME
ARG GRADLE_USER_HOME_PATH
WORKDIR /${APP_NAME}/
RUN mkdir -p ${GRADLE_USER_HOME_PATH}
ENV GRADLE_USER_HOME ${GRADLE_USER_HOME_PATH}
COPY --chown=gradle:gradle build.gradle /${APP_NAME}/
RUN gradle clean build --no-daemon -i --stacktrace -x bootJar
#build
FROM gradle:6.8-jdk11 AS build
ARG APP_NAME
ARG GRADLE_USER_HOME_PATH
WORKDIR /${APP_NAME}/
#Copies cached dependencies
COPY --from=cache ${GRADLE_USER_HOME_PATH} /home/gradle/.gradle/
#Copies the Java source code inside the container
COPY --chown=gradle:gradle . /${APP_NAME}/
#Compiles the code and runs unit tests (with Gradle build)
RUN gradle build --no-daemon -i --stacktrace
#Discards the Gradle image with all the compiled classes/unit test results etc.
#Starts again from the JRE image and copies only the JAR file created before
FROM openjdk:11-jre-slim
ARG APP_NAME
COPY --from=build /${APP_NAME}/build/libs/${APP_NAME}.jar /${APP_NAME}/${APP_NAME}.jar
ENTRYPOINT ["java","-jar","/test-api/test-api.jar"]
How to implement integration tests that using DB to GKE? Or maybe I need to change my approach?

I managed to solve the problem referencing this Q&A: Run node.js database migrations on Google Cloud SQL during Google Cloud Build
I had to add 2 steps(Cloud SQL Proxy and Test) on cloudbuild.yaml to use Cloud SQL Proxy. The other steps were auto-generated by GKE:
steps:
- name: gradle:6.8.3-jdk11
entrypoint: sh
args:
- '-c'
- |-
apt-get update && apt-get install -y wget \
&& wget "https://storage.googleapis.com/cloudsql-proxy/v1.21.0/cloud_sql_proxy.linux.amd64" -O cloud_sql_proxy \
&& chmod +x cloud_sql_proxy \
|| exit 1
id: Cloud SQL Proxy
- name: gradle:6.8.3-jdk11
entrypoint: sh
args:
- '-c'
- |-
(./cloud_sql_proxy -instances=<CONNECTION_NAME>=tcp:<PORT> & sleep 2) \
&& gradle test --no-daemon -i --stacktrace \
|| exit 1
id: Test
- name: gcr.io/cloud-builders/docker
args:
- build
- '-t'
- '$_IMAGE_NAME:$COMMIT_SHA'
- .
- '-f'
- $_DOCKERFILE_NAME
dir: $_DOCKERFILE_DIR
id: Build
- name: gcr.io/cloud-builders/docker
args:
- push
- '$_IMAGE_NAME:$COMMIT_SHA'
id: Push
- name: gcr.io/cloud-builders/gke-deploy
args:
- prepare
- '--filename=$_K8S_YAML_PATH'
- '--image=$_IMAGE_NAME:$COMMIT_SHA'
- '--app=$_K8S_APP_NAME'
- '--version=$COMMIT_SHA'
- '--namespace=$_K8S_NAMESPACE'
- '--label=$_K8S_LABELS'
- '--annotation=$_K8S_ANNOTATIONS,gcb-build-id=$BUILD_ID'
- '--create-application-cr'
- >-
--links="Build
details=https://console.cloud.google.com/cloud-build/builds/$BUILD_ID?project=$PROJECT_ID"
- '--output=output'
id: Prepare deploy
- name: gcr.io/cloud-builders/gsutil
args:
- '-c'
- |-
if [ "$_OUTPUT_BUCKET_PATH" != "" ]
then
gsutil cp -r output/suggested gs://$_OUTPUT_BUCKET_PATH/config/$_K8S_APP_NAME/$BUILD_ID/suggested
gsutil cp -r output/expanded gs://$_OUTPUT_BUCKET_PATH/config/$_K8S_APP_NAME/$BUILD_ID/expanded
fi
id: Save configs
entrypoint: sh
- name: gcr.io/cloud-builders/gke-deploy
args:
- apply
- '--filename=output/expanded'
- '--cluster=$_GKE_CLUSTER'
- '--location=$_GKE_LOCATION'
- '--namespace=$_K8S_NAMESPACE'
id: Apply deploy
...
And Dockerfile:
ARG APP_NAME=test-api
ARG APP_HOME=/test-api
FROM openjdk:11-jdk-slim AS build
USER root
ARG APP_HOME
WORKDIR ${APP_HOME}/
COPY . .
# test is performed from Test step from cloudbuild.yaml
RUN ./gradlew build --no-daemon -i --stacktrace -x test
FROM openjdk:11-jdk-slim
ARG APP_NAME
ARG APP_HOME
WORKDIR ${APP_HOME}/
COPY --from=build ${APP_HOME}/build/libs/${APP_NAME}.jar ./${APP_NAME}.jar
EXPOSE 8080
ENTRYPOINT ["java","-jar","/test-api/test-api.jar"]
While I solved the questioned problem, this script has a little problem: there will be 2 separate Gradle dependencies downloads(Test and Build). I couldn't manage to use Cloud SQL Proxy on gcr.io/cloud-builders/docker, so I workaround by using the Test step instead of the Build step. Maybe this can be solved using either docker run --network="host" or host.docker.internal, but I didn't try.

Google Cloud Debugger can't find Spring Boot web app deployed as Cloud Run service

I followed the guide (https://cloud.google.com/debugger/docs/setup/java#cloud-run) to setup the cloud debugger for my cloud run services. Everything looks like it should work. However the debugger UI tells me it can not find any application.
However I did everything that was suugested in the documentation
Source code is in Source Repositories
Cloud Build triggers on master push
I download the cdbg-java-agent.so
I run the java application with the -agentpath option
Cloud Run service is being deployed via cloudbuild.yaml
Cloud Run service starts healthy and works
However as you can see in the screenshot my application can not be found after successful deployment.
Here are some details of my configuration:
Spring Boot v2.4.2
Kotlin 1.4.21
This is my Dockerfile
FROM gradle:6.7.0-jdk14 as build
USER root
WORKDIR /dist
ADD . /dist
RUN gradle assemble
FROM alpine as agent
USER root
WORKDIR /agent
RUN wget -qO- https://storage.googleapis.com/cloud-debugger/compute-java/debian-wheezy/cdbg_java_agent_gce.tar.gz | tar xvz
FROM openjdk:14-slim
WORKDIR /app
COPY --from=agent /agent/cdbg_java_agent.so agent.so
COPY --from=build /dist/build/libs/user-service.jar app.jar
EXPOSE 8080
CMD ["java", "-agentpath:/app/agent.so", "-Dcom.google.cdbg.breakpoints.enable_canary=false", "-jar", "app.jar"]
Here is the docker build output
Deploying '<unknown> Dockerfile: Dockerfile'...
Building image...
Preparing build context archive...
[==================================================>]1855/1855 files
Done
Sending build context to Docker daemon...
[==================================================>] 76.19MB
Done
Step 1/15 : FROM gradle:6.7.0-jdk14 as build
---> 5af4d25725b2
Step 2/15 : USER root
---> Using cache
---> c97e9145e3e1
Step 3/15 : WORKDIR /dist
---> Using cache
---> 468fff36a2e7
Step 4/15 : ADD . /dist
---> c435bd2ffde9
Step 5/15 : RUN gradle assemble
---> Running in 70f5c69b7dde
Welcome to Gradle 6.7!
Here are the highlights of this release:
- File system watching is ready for production use
- Declare the version of Java your build requires
- Java 15 support
For more details see https://docs.gradle.org/6.7/release-notes.html
Starting a Gradle Daemon (subsequent builds will be faster)
> Task :compileKotlin
> Task :compileJava NO-SOURCE
> Task :processResources
> Task :classes
> Task :bootJarMainClassName
> Task :bootJar
> Task :inspectClassesForKotlinIC
> Task :jar SKIPPED
> Task :assemble
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.7/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 1m 48s
5 actionable tasks: 5 executed
Removing intermediate container 70f5c69b7dde
---> 5bbd9485f873
Step 6/15 : FROM alpine as agent
---> 28f6e2705743
Step 7/15 : USER root
---> Using cache
---> 4a0572e8ff4a
Step 8/15 : WORKDIR /agent
---> Using cache
---> 0374735af05d
Step 9/15 : RUN wget -qO- https://storage.googleapis.com/cloud-debugger/compute-java/debian-wheezy/cdbg_java_agent_gce.tar.gz | tar xvz
---> Using cache
---> ef141a7d5acc
Step 10/15 : FROM openjdk:14-slim
---> 38fca30874ab
Step 11/15 : WORKDIR /app
---> Using cache
---> 4dd576d38d65
Step 12/15 : COPY --from=agent /agent/cdbg_java_agent.so agent.so
---> Using cache
---> ff5fc93d2d5e
Step 13/15 : COPY --from=build /dist/build/libs/user-service.jar app.jar
---> 4b009124345f
Step 14/15 : EXPOSE 8080
---> Running in 99f71fb86534
Removing intermediate container 99f71fb86534
---> 38fcd9d00999
Step 15/15 : CMD ["java", "-agentpath:/app/agent.so", "-Dcom.google.cdbg.breakpoints.enable_canary=false", "-jar", "app.jar"]
---> Running in 309f35aed48f
Removing intermediate container 309f35aed48f
---> 43757d33b2ee
Successfully built 43757d33b2ee
Existing container found: 94d3fab4505461e8e694b61909ba1f7d827d8f7078b4dd8d1d36d4c4e0a0ac08, removing...
Creating container...
Container Id: b99675b5f6c00a0e5bec76999db04abf92eab29d257ee887bcea123a02789850
Container name: '/quirky_stonebraker'
Attaching to container '/quirky_stonebraker'...
Starting container '/quirky_stonebraker'
'<unknown> Dockerfile: Dockerfile' has been deployed successfully.
The container runs fine as already stated with the mentioned deployed cloud run service.
So I am out of ideas on what to do to make this work. I hope anybody can help me out here
I also add my cloudbuild.yaml in case that is relevant
steps:
- name: 'gradle:6.8.3-jdk11'
entrypoint: 'gradle'
args: [ 'check', '-x', 'test']
- name: 'gradle:6.8.3-jdk11'
entrypoint: 'gradle'
args: [ 'test']
- name: 'gradle:6.8.3-jdk11'
entrypoint: 'gradle'
args: [ 'integration-test']
- name: 'gcr.io/cloud-builders/docker'
args: [ 'build', '-t', 'eu.gcr.io/$PROJECT_ID/user-service:$BUILD_ID', '.' ]
- name: 'gcr.io/cloud-builders/docker'
args: [ 'push', 'eu.gcr.io/$PROJECT_ID/user-service:$BUILD_ID' ]
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
entrypoint: gcloud
args:
- 'alpha'
- 'run'
- 'deploy'
- 'user-service'
- '--image=eu.gcr.io/$PROJECT_ID/user-service:$BUILD_ID'
- '--concurrency=10'
- '--cpu=1'
- '--memory=512Mi'
- '--region=europe-west3'
- '--max-instances=2'
- '--platform=managed'
- '--port=8080'
- '--timeout=3000'
- '--set-env-vars=SQL_CONNECTION=10.28.96.3, SQL_USER=test, SQL_PASSWORD=test'
- '--set-env-vars=AUTH0_DOMAIN=prototype.eu.auth0.com, AUTH0_CLIENT_ID=123, AUTH0_CLIENT_SECRET=123'
- '--set-env-vars=^#^SPRING_PROFILES_ACTIVE=prod'
- '--allow-unauthenticated'
- '--ingress=internal'
- '--vpc-connector=cloud-run'
- '--vpc-egress=private-ranges-only'
- '--set-cloudsql-instances=$PROJECT_ID:europe-west1:prototype'
images:
- 'eu.gcr.io/$PROJECT_ID/user-service:$BUILD_ID'
timeout: 3000s

If you look what's inside the zipped agent file, you'll notice that there's a few files in it called:
cdbg_java_agent.so
cdbg_java_agent_internals.jar
version.txt
You're getting this problem because you missed to copy the internals.jar file to /app dir. The shared object file has a dependency and that can explain why users are instructed to create a separate directory for the Debugger.
To fix it on this case, add the missing file on to your Dockerfile like this:
COPY --from=agent /agent/cdbg_java_agent.so agent.so
COPY --from=agent /agent/cdbg_java_agent_internals.jar cdbg_java_agent_internals.jar
COPY --from=build /dist/build/libs/user-service.jar app.jar
Then wait for approx. ~5 minutes until your service appears on the Debugger. Note that the source code will not appear automatically unless it's located on a Git repo. For that you have to select the source code manually.

Azure Linux Webapp cannot seem to find deployed Spring .jar from DevOps repo

I have a Spring Web Application in a DevOps repository, with a .yml that looks like this (as generated by the tool in the DevOps web client):
# Build your Java project and deploy it to Azure as a Linux web app
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/java
trigger:
- master
variables:
Version: '0.0.1'
# Azure Resource Manager connection created during pipeline creation
azureSubscription: '-snipped-'
# Web app name
webAppName: 'SlackCentralTestApp'
# Environment name
environmentName: 'SlackCentralTestApp'
# Agent VM image name
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: MavenPackageAndPublishArtifacts
displayName: Maven Package and Publish Artifacts
pool:
vmImage: $(vmImageName)
steps:
- task: Maven#3
displayName: 'Maven Package'
inputs:
mavenPomFile: 'pom.xml'
publishJUnitResults: true
testResultsFiles: '**/surefire-reports/TEST-*.xml'
javaHomeOption: 'JDKVersion'
jdkVersionOption: '1.11'
mavenVersionOption: 'Default'
mavenAuthenticateFeed: false
effectivePomSkip: false
sonarQubeRunAnalysis: false
- task: CopyFiles#2
displayName: 'Copy Files to artifact staging directory'
inputs:
SourceFolder: '$(System.DefaultWorkingDirectory)'
Contents: '**/target/SlackbotTest-$(Version)-SNAPSHOT.?(war|jar)'
TargetFolder: $(Build.ArtifactStagingDirectory)
- upload: $(Build.ArtifactStagingDirectory)
artifact: drop
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: DeployLinuxWebApp
displayName: Deploy Linux Web App
environment: $(environmentName)
pool:
vmImage: $(vmImageName)
strategy:
runOnce:
deploy:
steps:
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy: SlackCentralTestApp'
inputs:
azureSubscription: 'Azure for Students (4998490e-1bc4-43fc-a370-80744706d1f5)'
appType: 'webAppLinux'
appName: 'SlackCentralTestApp'
package: '$(Pipeline.Workspace)/drop/target/SlackbotTest-$(Version)-SNAPSHOT.jar'
runtimeStack: 'JAVA|11-java11'
startUpCommand: 'java -jar $(Pipeline.Workspace)/drop/target/SlackbotTest-$(Version)-SNAPSHOT.jar'
The deployment process seems to be successful as seen in this screenshot, yet when I take a look at the server log, I sadly get greeted with the following error:
2020-04-14T09:03:55.658599508Z Initializing App Insights applicationinsights-agent-codeless-2.5.0.jar....
2020-04-14T09:03:55.669449167Z STARTUP_FILE=
2020-04-14T09:03:55.676322304Z STARTUP_COMMAND=java -jar /home/vsts/work/1/drop/target/SlackbotTest-0.0.1-SNAPSHOT.jar
2020-04-14T09:03:55.676350305Z No STARTUP_FILE available.
2020-04-14T09:03:55.676428905Z Running STARTUP_COMMAND: java -jar /home/vsts/work/1/drop/target/SlackbotTest-0.0.1-SNAPSHOT.jar
2020-04-14T09:03:55.681352232Z Error: Unable to access jarfile /home/vsts/work/1/drop/target/SlackbotTest-0.0.1-SNAPSHOT.jar
2020-04-14T09:03:55.694784805Z Finished running startup command 'java -jar /home/vsts/work/1/drop/target/SlackbotTest-0.0.1-SNAPSHOT.jar'. Exiting with exit code 1.
This would lead me to conclude that the path as given in the 'startUpCommand' property from the .yml file is incorrect, yet I have not been able to find what the correct path should be.
I have attempted the following:
Specify no directories, only the filename. Leads to the same result, sadly.
Use the 'find' command in Bash to find any .jars on the Web App, which tells me that there are none.
Manually building the application from the Web App does not seem to be an option either, as it's a Java 11 application and the java SE that seems to be included is version 1.8, regardless of the version I specify during the creation of the Web App resource in Azure.