We Keep Coding

ExifTool: Removing all metadata of an image except one in java application

I want to remove all the metadata of an image except "Copyright".I am using exiftool of this .The command for this is "exiftool -all= -tagsFromFile # -copyright Tunis_Bab_Souika_1899.jpg".But when I do this in java application .It somehow deletes all the tags.
here is the code snippet -
val outputConsumer = ArrayListOutputConsumer()
val exiftoolCmd = ExiftoolCmd()
exiftoolCmd.setOutputConsumer(outputConsumer)
val operation = ETOperation()
println("File name" + sourceImage.toFile().absolutePath)
operation.addImage(sourceImage.toFile().absolutePath)
// exiftool -all= -tagsFromFile # -copyright Tunis_Bab_Souika_1899.jpg
operation.addRawArgs("-all=")
operation.addRawArgs("-tagsFromFile #")
operation.addRawArgs("-copyright")
println("About to execute")
try {exiftoolCmd.run(operation)
println("Inside try")
} catch (e: java.lang.Exception) {
throw RuntimeException(e)
}
val output = outputConsumer.output
.stream()
.map { obj: String -> obj.trim { it <= ' ' } }
.collect(toList())
println("ooutput$output")

Answering my question - This worked for me-
operation.delTags("all")
operation.tagsFromFile("#")
operation.getTags("copyright")
operation.addImage(sourceImage.toFile().absolutePath)

Dynamic gradle command for Exec task fails

I am trying to publish Lambda zip file to s3- s3://my-aws-lambda/<projectName>/[release|SNAPSHOT]/. The task defined below, publishToS3 fails with the message
Caused by: java.io.FileNotFoundException: /Users/me/my-lambda/build/distributions
when I run
./gradlew clean build -x test -x release
Appreciate any help. Thanks.
task buildZip(type: Zip) {
from compileJava
from processResources
into('lib') {
from configurations.runtimeClasspath
}
}
task publishToS3(type: Exec, dependsOn: buildZip) {
onlyIf { file("${project.buildDir}/distributions").exist() }
def artifacts = new FileNameByRegexFinder().getFileNames("${project.buildDir}/distributions", /.*\.zip/)
assert artifacts.size() == 1
def isSnapShot = artifacts[0].endsWith('-SNAPSHOT.zip')
def releaseCmd = ("aws s3 cp " +
"${artifacts[0]} " +
"s3://my-aws-lambdas/${project.name}/${isSnapShot ? 'SNAPSHOT' : 'release'}/ ").trim().tokenize(' ') as List
workingDir "${project.buildDir}/distributions"
commandLine releaseCmd
}
build.dependsOn buildZip

If you are on UNIX environment, you can use the find command to search the files and use it's output.
task scanFiles() {
def a= "find ${project.buildDir} -name *.zip".execute().text
String[] files=a.split('\n')
if(files.length == 1){
println("We'v got a file :"+a)
}
else if(files.length ==0){
println("We've not no files ")
}
else{
println("We've got multiple files\n"+a)
}
}

How to read a properties files and use the values in project Gradle script?

I am working on a Gradle script where I need to read the local.properties file and use the values in the properties file in build.gradle. I am doing it in the below manner. I ran the below script and it is now throwing an error, but it is also not doing anything like creating, deleting, and copying the file. I tried to print the value of the variable and it is showing the correct value.
Can someone let me know if this is the correct way to do this? I think the other way is to define everything in the gradle.properties and use it in the build.gradle. Can someone let me know how could I access the properties in build.gradle from build.properties?
build.gradle file:
apply plugin: 'java'
// Set the group for publishing
group = 'com.true.test'
/**
* Initializing GAVC settings
*/
def buildProperties = new Properties()
file("version.properties").withInputStream {
stream -> buildProperties.load(stream)
}
// If jenkins build, add the jenkins build version to the version. Else add snapshot version to the version.
def env = System.getenv()
if (env["BUILD_NUMBER"]) buildProperties.test+= ".${env["BUILD_NUMBER"]}"
version = buildProperties.test
println "${version}"
// Name is set in the settings.gradle file
group = "com.true.test"
version = buildProperties.test
println "Building ${project.group}:${project.name}:${project.version}"
Properties properties = new Properties()
properties.load(project.file('build.properties').newDataInputStream())
def folderDir = properties.getProperty('build.dir')
def configDir = properties.getProperty('config.dir')
def baseDir = properties.getProperty('base.dir')
def logDir = properties.getProperty('log.dir')
def deployDir = properties.getProperty('deploy.dir')
def testsDir = properties.getProperty('tests.dir')
def packageDir = properties.getProperty('package.dir')
def wrapperDir = properties.getProperty('wrapper.dir')
sourceCompatibility = 1.7
compileJava.options.encoding = 'UTF-8'
repositories {
maven { url "http://arti.oven.c:9000/release" }
}
task swipe(type: Delete) {
println "Delete $projectDir/${folderDir}"
delete "$projectDir/$folderDir"
delete "$projectDir/$logDir"
delete "$projectDir/$deployDir"
delete "$projectDir/$packageDir"
delete "$projectDir/$testsDir"
mkdir("$projectDir/${folderDir}")
mkdir("projectDir/${logDir}")
mkdir("projectDir/${deployDir}")
mkdir("projectDir/${packageDir}")
mkdir("projectDir/${testsDir}")
}
task prepConfigs(type: Copy, overwrite:true, dependsOn: swipe) {
println "The name of ${projectDir}/${folderDir} and ${projectDir}/${configDir}"
from('${projectDir}/${folderDir}')
into('${projectDir}/$configDir}')
include('*.xml')
}
build.properties file:
# -----------------------------------------------------------------
# General Settings
# -----------------------------------------------------------------
application.name = Admin
project.name = Hello Cool
# -----------------------------------------------------------------
# ant build directories
# -----------------------------------------------------------------
sandbox.dir = ${projectDir}/../..
reno.root.dir=${sandbox.dir}/Reno
ant.dir = ${projectDir}/ant
build.dir = ${ant.dir}/build
log.dir = ${ant.dir}/logs
config.dir = ${ant.dir}/configs
deploy.dir = ${ant.dir}/deploy
static.dir = ${ant.dir}/static
package.dir = ${ant.dir}/package
tests.dir = ${ant.dir}/tests
tests.logs.dir = ${tests.dir}/logs
external.dir = ${sandbox.dir}/FlexCommon/External
external.lib.dir = ${external.dir}/libs

If using the default gradle.properties file, you can access the properties directly from within your build.gradle file:
gradle.properties:
applicationName=Admin
projectName=Hello Cool
build.gradle:
task printProps {
doFirst {
println applicationName
println projectName
}
}
If you need to access a custom file, or access properties which include . in them (as it appears you need to do), you can do the following in your build.gradle file:
def props = new Properties()
file("build.properties").withInputStream { props.load(it) }
task printProps {
doFirst {
println props.getProperty("application.name")
println props.getProperty("project.name")
}
}
Take a look at this section of the Gradle documentation for more information.
Edit
If you'd like to dynamically set up some of these properties (as mentioned in a comment below), you can create a properties.gradle file (the name isn't important) and require it in your build.gradle script.
properties.gradle:
ext {
subPath = "some/sub/directory"
fullPath = "$projectDir/$subPath"
}
build.gradle
apply from: 'properties.gradle'
// prints the full expanded path
println fullPath

We can use a separate file (config.groovy in my case) to abstract out all the configuration.
In this example, we're using three environments viz.,
dev
test
prod
which has properties serverName, serverPort and resources. Here we're expecting that the third property resources may be same in multiple environments and so we've abstracted out that logic and overridden in the specific environment wherever necessary:
config.groovy
resources {
serverName = 'localhost'
serverPort = '8090'
}
environments {
dev {
serverName = 'http://localhost'
serverPort = '8080'
}
test {
serverName = 'http://www.testserver.com'
serverPort = '5211'
resources {
serverName = 'resources.testserver.com'
}
}
prod {
serverName = 'http://www.productionserver.com'
serverPort = '80'
resources {
serverName = 'resources.productionserver.com'
serverPort = '80'
}
}
}
Once the properties file is ready, we can use the following in build.gradle to load these settings:
build.gradle
loadProperties()
def loadProperties() {
def environment = hasProperty('env') ? env : 'dev'
println "Current Environment: " + environment
def configFile = file('config.groovy')
def config = new ConfigSlurper(environment).parse(configFile.toURL())
project.ext.config = config
}
task printProperties {
println "serverName: $config.serverName"
println "serverPort: $config.serverPort"
println "resources.serverName: $config.resources.serverName"
println "resources.serverPort: $config.resources.serverPort"
}
Let's run these with different set of inputs:
gradle -q printProperties
Current Environment: dev
serverName: http://localhost
serverPort: 8080
resources.serverName: localhost
resources.serverPort: 8090
gradle -q -Penv=dev printProperties
Current Environment: dev
serverName: http://localhost
serverPort: 8080
resources.serverName: localhost
resources.serverPort: 8090
gradle -q -Penv=test printProperties
Current Environment: test
serverName: http://www.testserver.com
serverPort: 5211
resources.serverName: resources.testserver.com
resources.serverPort: 8090
gradle -q -Penv=prod printProperties
Current Environment: prod
serverName: http://www.productionserver.com
serverPort: 80
resources.serverName: resources.productionserver.com
resources.serverPort: 80

Another way... in build.gradle:
Add :
classpath 'org.flywaydb:flyway-gradle-plugin:3.1'
And this :
def props = new Properties()
file("src/main/resources/application.properties").withInputStream { props.load(it) }
apply plugin: 'flyway'
flyway {
url = props.getProperty("spring.datasource.url")
user = props.getProperty("spring.datasource.username")
password = props.getProperty("spring.datasource.password")
schemas = ['db_example']
}

This is for Kotlin DSL (build.gradle.kts):
import java.util.*
// ...
val properties = Properties().apply {
load(rootProject.file("my-local.properties").reader())
}
val prop = properties["myPropName"]
In Android projects (when applying the android plugin) you can also do this:
import com.android.build.gradle.internal.cxx.configure.gradleLocalProperties
// ...
val properties = gradleLocalProperties(rootDir)
val prop = properties["propName"]

Just had this issue come up today. We found the following worked both locally and in our pipeline:
In build.gradle:
try {
apply from: 'path/name_of_external_props_file.properties'
} catch (Exception e) {}
This way when an external props file which shouldn't get committed to Git or whatever (as in our case) you are using is not found in the pipeline, this 'apply from:' won't throw an error in it. In our use case we have a file with a userid and password that should not get committed to Git. Aside from the problem of file-reading: we found that the variables we had declared in the external file, maven_user and maven_pass, had in fact to be declared in gradle.properties. That is they simply needed to be mentioned as in:
projectName=Some_project_name
version=1.x.y
maven_user=
maven_pass=
We also found that in the external file we had to put single-quotes around these values too or Gradle got confused. So the external file looked like this:
maven_user='abc123'
maven_pass='fghifh7435bvibry9y99ghhrhg9539y5398'
instead of this:
maven_user=abc123
maven_pass=fghifh7435bvibry9y99ghhrhg9539y5398
That's all we had to do and we were fine. I hope this may help others.

Gradle creating duplicate start scripts into bin directory

I am trying to create multiple start script files through gradle. But somehow one particular start script file is getting duplicated.
startScripts.enabled = false
run.enabled = false
def createScript(project, mainClass, name) {
project.tasks.create(name: name, type: CreateStartScripts) {
outputDir = new File(project.buildDir, 'scripts')
mainClassName = mainClass
applicationName = name
classpath = jar.outputs.files + project.configurations.runtime
doLast {
def windowsScriptFile = file getWindowsScript()
def unixScriptFile = file getUnixScript()
windowsScriptFile.text = windowsScriptFile.text.replace('%APP_HOME%\\lib\\conf', '%APP_HOME%\\conf')
unixScriptFile.text = unixScriptFile.text.replace('$APP_HOME/lib/conf', '$APP_HOME/conf')
}
}
project.tasks[name].dependsOn(project.jar)
project.applicationDistribution.with {
into("bin") {
from(project.tasks[name])
fileMode = 0755
}
}
}
// Call this for each Main class you want to expose with an app script
createScript(project, 'com.main.A', 'A')
createScript(project, 'com.main.B', 'B')
in bin directory I can see,
A.sh
A.sh
A.bat
A.bat
B.sh
B.bat
What am I missing here? How to fix this?
Thank you for help.

I solved this problem. Actually it was a mistake from my side and thanks to #Opal. I somehow forgot to delete 'mainClassName="com.main.A"' line from the header.
Also I have to add
distZip {
duplicatesStrategy = 'exclude'
}

Delete directory recursively in Scala

I am writing the following (with Scala 2.10 and Java 6):
import java.io._
def delete(file: File) {
if (file.isDirectory)
Option(file.listFiles).map(_.toList).getOrElse(Nil).foreach(delete(_))
file.delete
}
How would you improve it ? The code seems working but it ignores the return value of java.io.File.delete. Can it be done easier with scala.io instead of java.io ?

With pure scala + java way
import scala.reflect.io.Directory
import java.io.File
val directory = new Directory(new File("/sampleDirectory"))
directory.deleteRecursively()
deleteRecursively() Returns false on failure

Try this code that throws an exception if it fails:
def deleteRecursively(file: File): Unit = {
if (file.isDirectory) {
file.listFiles.foreach(deleteRecursively)
}
if (file.exists && !file.delete) {
throw new Exception(s"Unable to delete ${file.getAbsolutePath}")
}
}
You could also fold or map over the delete if you want to return a value for all the deletes.

Using scala IO
import scalax.file.Path
val path = Path.fromString("/tmp/testfile")
try {
path.deleteRecursively(continueOnFailure = false)
} catch {
case e: IOException => // some file could not be deleted
}
or better, you could use a Try
val path: Path = Path ("/tmp/file")
Try(path.deleteRecursively(continueOnFailure = false))
which will either result in a Success[Int] containing the number of files deleted, or a Failure[IOException].

From
http://alvinalexander.com/blog/post/java/java-io-faq-how-delete-directory-tree
Using Apache Common IO
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.filefilter.WildcardFileFilter;
public void deleteDirectory(String directoryName)
throws IOException
{
try
{
FileUtils.deleteDirectory(new File(directoryName));
}
catch (IOException ioe)
{
// log the exception here
ioe.printStackTrace();
throw ioe;
}
}
The Scala one can just do this...
import org.apache.commons.io.FileUtils
import org.apache.commons.io.filefilter.WildcardFileFilter
FileUtils.deleteDirectory(new File(outputFile))
Maven Repo Imports

Using the Java NIO.2 API:
import java.nio.file.{Files, Paths, Path, SimpleFileVisitor, FileVisitResult}
import java.nio.file.attribute.BasicFileAttributes
def remove(root: Path): Unit = {
Files.walkFileTree(root, new SimpleFileVisitor[Path] {
override def visitFile(file: Path, attrs: BasicFileAttributes): FileVisitResult = {
Files.delete(file)
FileVisitResult.CONTINUE
}
override def postVisitDirectory(dir: Path, exc: IOException): FileVisitResult = {
Files.delete(dir)
FileVisitResult.CONTINUE
}
})
}
remove(Paths.get("/tmp/testdir"))
Really, it's a pity that the NIO.2 API is with us for so many years and yet few people are using it, even though it is really superior to the old File API.

Expanding on Vladimir Matveev's NIO2 solution:
object Util {
import java.io.IOException
import java.nio.file.{Files, Paths, Path, SimpleFileVisitor, FileVisitResult}
import java.nio.file.attribute.BasicFileAttributes
def remove(root: Path, deleteRoot: Boolean = true): Unit =
Files.walkFileTree(root, new SimpleFileVisitor[Path] {
override def visitFile(file: Path, attributes: BasicFileAttributes): FileVisitResult = {
Files.delete(file)
FileVisitResult.CONTINUE
}
override def postVisitDirectory(dir: Path, exception: IOException): FileVisitResult = {
if (deleteRoot) Files.delete(dir)
FileVisitResult.CONTINUE
}
})
def removeUnder(string: String): Unit = remove(Paths.get(string), deleteRoot=false)
def removeAll(string: String): Unit = remove(Paths.get(string))
def removeUnder(file: java.io.File): Unit = remove(file.toPath, deleteRoot=false)
def removeAll(file: java.io.File): Unit = remove(file.toPath)
}

Using java 6 without using dependencies this is pretty much the only way to do so.
The problem with your function is that it return Unit (which I btw would explicit note it using def delete(file: File): Unit = {
I took your code and modify it to return map from file name to the deleting status.
def delete(file: File): Array[(String, Boolean)] = {
Option(file.listFiles).map(_.flatMap(f => delete(f))).getOrElse(Array()) :+ (file.getPath -> file.delete)
}

To add to Slavik Muz's answer:
def deleteFile(file: File): Boolean = {
def childrenOf(file: File): List[File] = Option(file.listFiles()).getOrElse(Array.empty).toList
#annotation.tailrec
def loop(files: List[File]): Boolean = files match {
case Nil ⇒ true
case child :: parents if child.isDirectory && child.listFiles().nonEmpty ⇒
loop((childrenOf(child) :+ child) ++ parents)
case fileOrEmptyDir :: rest ⇒
println(s"deleting $fileOrEmptyDir")
fileOrEmptyDir.delete()
loop(rest)
}
if (!file.exists()) false
else loop(childrenOf(file) :+ file)
}

This one uses java.io but one can delete directories matching it with wildcard string which may or may not contain any content within it.
for (file <- new File("<path as String>").listFiles;
if( file.getName() matches("[1-9]*"))) FileUtils.deleteDirectory(file)
Directory structure e.g.
* A/1/, A/2/, A/300/ ... thats why the regex String: [1-9]*, couldn't find a File API in scala which supports regex(may be i missed something).

Getting little lengthy, but here's one that combines the recursivity of Garrette's solution with the npe-safety of the original question.
def deleteFile(path: String) = {
val penultimateFile = new File(path.split('/').take(2).mkString("/"))
def getFiles(f: File): Set[File] = {
Option(f.listFiles)
.map(a => a.toSet)
.getOrElse(Set.empty)
}
def getRecursively(f: File): Set[File] = {
val files = getFiles(f)
val subDirectories = files.filter(path => path.isDirectory)
subDirectories.flatMap(getRecursively) ++ files + penultimateFile
}
getRecursively(penultimateFile).foreach(file => {
if (getFiles(file).isEmpty && file.getAbsoluteFile().exists) file.delete
})
}

This is recursive method that clean all in directory, and return count of deleted files
def cleanDir(dir: File): Int = {
#tailrec
def loop(list: Array[File], deletedFiles: Int): Int = {
if (list.isEmpty) deletedFiles
else {
if (list.head.isDirectory && !list.head.listFiles().isEmpty) {
loop(list.head.listFiles() ++ list.tail ++ Array(list.head), deletedFiles)
} else {
val isDeleted = list.head.delete()
if (isDeleted) loop(list.tail, deletedFiles + 1)
else loop(list.tail, deletedFiles)
}
}
}
loop(dir.listFiles(), 0)
}

What I ended up with
def deleteRecursively(f: File): Boolean = {
if (f.isDirectory) f.listFiles match {
case files: Array[File] => files.foreach(deleteRecursively)
case null =>
}
f.delete()
}

os-lib makes it easy to delete recursively with a one-liner:
os.remove.all(os.pwd/"dogs")
os-lib uses java.nio under the hood, just doesn't expose all the Java ugliness. See here for more info on how to use the library.

You can do this by excute external system commands.
import sys.process._
def delete(path: String) = {
s"""rm -rf ${path}""".!!
}