How to find unused properties in a pom - java

After inheriting a maven project, I would like to check for unused properties and remove them.
One way I dont want to take is to remove them one by one and see the build fail. Another way would be to count the occurences in the whole codebase (to make sure properties for filters and resources are not wrongly seen as unused) with a custom script. Before I do that, I would like to make sure I'm not reinventing the wheel.
There is a way to do that for the dependencies explained here.
Is there something similar for properties that I have missed? Or a better way?
Thank you
I am using maven 3.0.3

It's a bit complicated but here's a bash script that will parse the properties element from a pom.xml, check if each property is used in the pom, and optionally checks all project files (a deep search).
#!/bin/bash
cmd=$(basename $0)
read_dom () {
local IFS=\>
read -d \< entity content
local retval=$?
tag=${entity%% *}
attr=${entity#* }
return $retval
}
parse_dom () {
# uncomment this line to access element attributes as variables
#eval local $attr
if [[ $tag = "!--" ]]; then # !-- is a comment
return
elif [[ $tag = "properties" ]]; then
in=true
elif [[ $tag = "/properties" ]]; then
in=
elif [[ "$in" && $tag != /* ]]; then #does not start with slash */
echo $tag
fi
}
unused_terms () {
file=$1
while read p; do
grep -m 1 -qe "\${$p}" $file
if [[ $? == 1 ]]; then echo $p; fi
done
}
unused_terms_dir () {
dir=$1
while read p; do
unused_term_find $dir $p
done
}
unused_term_find () {
dir=$1
p=$2
echo -n "$p..."
find $dir -type f | xargs grep -m 1 -qe "\${$p}" 2> /dev/null
if [[ $? == 0 ]]; then
echo -ne "\r$(tput el)"
else
echo -e "\b\b\b "
fi
}
if [[ -z $1 ]]; then
echo "Usage: $cmd [-d] <pom-file>"
exit
fi
if [[ $1 == "-d" ]]; then
deep=true
shift
fi
file=$1
dir=$(dirname $1)
if [ $deep ]; then
while read_dom; do
parse_dom
done < $file | unused_terms $file | unused_terms_dir $dir
else
while read_dom; do
parse_dom
done < $file | unused_terms $file
fi
The script uses XML parsing code from this thread.
This script won't find any properties that are used by maven or maven plugins directly. It simply looks for the pattern ${property} within project files.
It works on my mac; your milage may vary.

Unfortunately there is no better way instead of doing it manually ...

If you are using IntelliJ, you could just search for usages of the property. Just put your cursor in the tag, and right click -> Find Usages (keyboard shortcut in my case: Alt+F7).
Haven't found a better way yet.

dependency_cleaner https://github.com/junaidbs/dependency_cleaner
This jar will help to identify and then remove the unwanted dependency from pom .
It will automate the process of Removing a dependency and run then check whether dependency needful
It's a better way than doing manualy

Related

Will this NGS data annotation code pipeline work?

threads=36
task() {
NEWNAME="${1/%.norm.vcf.gz/.snv.indel.vcf.gz}"
java -Xmx8g -jar /home/ubuntu/snpEff/SnpSift.jar annotate /home/ubuntu/data/spliceai_scores.raw.snv.hg19.vcf.gz $1 | java -Xmx8g -jar /home/ubuntu/snpEff/SnpSift.jar annotate /home/ubuntu/data/spliceai_scores.raw.indel.hg19.vcf.gz > $NEWNAME1
}
for file in $(ls *norm.vcf.gz)
do
if [ $(jobs -r | wc -l) -ge $threads ]; then
wait $(jobs -r -p | head -1)
fi
task "$file" &
done
wait
I am now learning NGS pipeline and I would like to annotate two DBs(snv,indel) using SnpSift. Currently I am using 36 threads CPU. Since there are two commands in task() function, annotate snv and indel, will my code wait $(jobs -r -p | head -1) work fine?
Are there any other recommendations to use CPU efficiently?

How to return a value printed by Java to the bash script it is called from? [duplicate]

I have a pretty simple script that is something like the following:
#!/bin/bash
VAR1="$1"
MOREF='sudo run command against $VAR1 | grep name | cut -c7-'
echo $MOREF
When I run this script from the command line and pass it the arguments, I am not getting any output. However, when I run the commands contained within the $MOREF variable, I am able to get output.
How can one take the results of a command that needs to be run within a script, save it to a variable, and then output that variable on the screen?
In addition to backticks `command`, command substitution can be done with $(command) or "$(command)", which I find easier to read, and allows for nesting.
OUTPUT=$(ls -1)
echo "${OUTPUT}"
MULTILINE=$(ls \
-1)
echo "${MULTILINE}"
Quoting (") does matter to preserve multi-line variable values; it is optional on the right-hand side of an assignment, as word splitting is not performed, so OUTPUT=$(ls -1) would work fine.
$(sudo run command)
If you're going to use an apostrophe, you need `, not '. This character is called "backticks" (or "grave accent"):
#!/bin/bash
VAR1="$1"
VAR2="$2"
MOREF=`sudo run command against "$VAR1" | grep name | cut -c7-`
echo "$MOREF"
Some Bash tricks I use to set variables from commands
Sorry, there is a loong answer, but as bash is a shell, where the main goal is to run other unix commands and react on result code and/or output, ( commands are often piped filter, etc... ).
Storing command output in variables is something basic and fundamental.
Therefore, depending on
compatibility (posix)
kind of output (filter(s))
number of variable to set (split or interpret)
execution time (monitoring)
error trapping
repeatability of request (see long running background process, further)
interactivity (considering user input while reading from another input file descriptor)
do I miss something?
First simple, old (obsolete), and compatible way
myPi=`echo '4*a(1)' | bc -l`
echo $myPi
3.14159265358979323844
Compatible, second way
As nesting could become heavy, parenthesis was implemented for this
myPi=$(bc -l <<<'4*a(1)')
Using backticks in script is to be avoided today.
Nested sample:
SysStarted=$(date -d "$(ps ho lstart 1)" +%s)
echo $SysStarted
1480656334
bash features
Reading more than one variable (with Bashisms)
df -k /
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/dm-0 999320 529020 401488 57% /
If I just want a used value:
array=($(df -k /))
you could see an array variable:
declare -p array
declare -a array='([0]="Filesystem" [1]="1K-blocks" [2]="Used" [3]="Available" [
4]="Use%" [5]="Mounted" [6]="on" [7]="/dev/dm-0" [8]="999320" [9]="529020" [10]=
"401488" [11]="57%" [12]="/")'
Then:
echo ${array[9]}
529020
But I often use this:
{ read -r _;read -r filesystem size using avail prct mountpoint ; } < <(df -k /)
echo $using
529020
( The first read _ will just drop header line. ) Here, in only one command, you will populate 6 different variables (shown by alphabetical order):
declare -p avail filesystem mountpoint prct size using
declare -- avail="401488"
declare -- filesystem="/dev/dm-0"
declare -- mountpoint="/"
declare -- prct="57%"
declare -- size="999320"
declare -- using="529020"
Or
{ read -a head;varnames=(${head[#]//[K1% -]});varnames=(${head[#]//[K1% -]});
read ${varnames[#],,} ; } < <(LANG=C df -k /)
Then:
declare -p varnames ${varnames[#],,}
declare -a varnames=([0]="Filesystem" [1]="blocks" [2]="Used" [3]="Available" [4]="Use" [5]="Mounted" [6]="on")
declare -- filesystem="/dev/dm-0"
declare -- blocks="999320"
declare -- used="529020"
declare -- available="401488"
declare -- use="57%"
declare -- mounted="/"
declare -- on=""
Or even:
{ read _ ; read filesystem dsk[{6,2,9}] prct mountpoint ; } < <(df -k /)
declare -p mountpoint dsk
declare -- mountpoint="/"
declare -a dsk=([2]="529020" [6]="999320" [9]="401488")
(Note Used and Blocks is switched there: read ... dsk[6] dsk[2] dsk[9] ...)
... will work with associative arrays too: read _ disk[total] disk[used] ...
Other related sample: Parsing xrandr output: and end of Firefox tab by bash in a size of x% of display size? or at AskUbuntu.com Parsing xrandr output
Dedicated fd using unnamed fifo:
There is an elegent way! In this sample, I will read /etc/passwd file:
users=()
while IFS=: read -u $list user pass uid gid name home bin ;do
((uid>=500)) &&
printf -v users[uid] "%11d %7d %-20s %s\n" $uid $gid $user $home
done {list}</etc/passwd
Using this way (... read -u $list; ... {list}<inputfile) leave STDIN free for other purposes, like user interaction.
Then
echo -n "${users[#]}"
1000 1000 user /home/user
...
65534 65534 nobody /nonexistent
and
echo ${!users[#]}
1000 ... 65534
echo -n "${users[1000]}"
1000 1000 user /home/user
This could be used with static files or even /dev/tcp/xx.xx.xx.xx/yyy with x for ip address or hostname and y for port number or with the output of a command:
{
read -u $list -a head # read header in array `head`
varnames=(${head[#]//[K1% -]}) # drop illegal chars for variable names
while read -u $list ${varnames[#],,} ;do
((pct=available*100/(available+used),pct<10)) &&
printf "WARN: FS: %-20s on %-14s %3d <10 (Total: %11u, Use: %7s)\n" \
"${filesystem#*/mapper/}" "$mounted" $pct $blocks "$use"
done
} {list}< <(LANG=C df -k)
And of course with inline documents:
while IFS=\; read -u $list -a myvar ;do
echo ${myvar[2]}
done {list}<<"eof"
foo;bar;baz
alice;bob;charlie
$cherry;$strawberry;$memberberries
eof
Practical sample parsing CSV files:
As this answer is loong enough, for this paragraph,
I just will let you refer to
this answer to How to parse a CSV file in Bash?, I read a file by using an unnamed fifo, using syntax like:
exec {FD}<"$file" # open unnamed fifo for read
IFS=';' read -ru $FD -a headline
while IFS=';' read -ru $FD -a row ;do ...
... But using bash loadable CSV module.
On my website, you may find the same script, reading CSV as inline document.
Sample function for populating some variables:
#!/bin/bash
declare free=0 total=0 used=0 mpnt='??'
getDiskStat() {
{
read _
read _ total used free _ mpnt
} < <(
df -k ${1:-/}
)
}
getDiskStat $1
echo "$mpnt: Tot:$total, used: $used, free: $free."
Nota: declare line is not required, just for readability.
About sudo cmd | grep ... | cut ...
shell=$(cat /etc/passwd | grep $USER | cut -d : -f 7)
echo $shell
/bin/bash
(Please avoid useless cat! So this is just one fork less:
shell=$(grep $USER </etc/passwd | cut -d : -f 7)
All pipes (|) implies forks. Where another process have to be run, accessing disk, libraries calls and so on.
So using sed for sample, will limit subprocess to only one fork:
shell=$(sed </etc/passwd "s/^$USER:.*://p;d")
echo $shell
And with Bashisms:
But for many actions, mostly on small files, Bash could do the job itself:
while IFS=: read -a line ; do
[ "$line" = "$USER" ] && shell=${line[6]}
done </etc/passwd
echo $shell
/bin/bash
or
while IFS=: read loginname encpass uid gid fullname home shell;do
[ "$loginname" = "$USER" ] && break
done </etc/passwd
echo $shell $loginname ...
Going further about variable splitting...
Have a look at my answer to How do I split a string on a delimiter in Bash?
Alternative: reducing forks by using backgrounded long-running tasks
In order to prevent multiple forks like
myPi=$(bc -l <<<'4*a(1)'
myRay=12
myCirc=$(bc -l <<<" 2 * $myPi * $myRay ")
or
myStarted=$(date -d "$(ps ho lstart 1)" +%s)
mySessStart=$(date -d "$(ps ho lstart $$)" +%s)
This work fine, but running many forks is heavy and slow.
And commands like date and bc could make many operations, line by line!!
See:
bc -l <<<$'3*4\n5*6'
12
30
date -f - +%s < <(ps ho lstart 1 $$)
1516030449
1517853288
So we could use a long running background process to make many jobs, without having to initiate a new fork for each request.
You could have a look how reducing forks make Mandelbrot bash, improve from more than eight hours to less than 5 seconds.
Under bash, there is a built-in function: coproc:
coproc bc -l
echo 4*3 >&${COPROC[1]}
read -u $COPROC answer
echo $answer
12
echo >&${COPROC[1]} 'pi=4*a(1)'
ray=42.0
printf >&${COPROC[1]} '2*pi*%s\n' $ray
read -u $COPROC answer
echo $answer
263.89378290154263202896
printf >&${COPROC[1]} 'pi*%s^2\n' $ray
read -u $COPROC answer
echo $answer
5541.76944093239527260816
As bc is ready, running in background and I/O are ready too, there is no delay, nothing to load, open, close, before or after operation. Only the operation himself! This become a lot quicker than having to fork to bc for each operation!
Border effect: While bc stay running, they will hold all registers, so some variables or functions could be defined at initialisation step, as first write to ${COPROC[1]}, just after starting the task (via coproc).
Into a function newConnector
You may found my newConnector function on GitHub.Com or on my own site (Note on GitHub: there are two files on my site. Function and demo are bundled into one unique file which could be sourced for use or just run for demo.)
Sample:
source shell_connector.sh
tty
/dev/pts/20
ps --tty pts/20 fw
PID TTY STAT TIME COMMAND
29019 pts/20 Ss 0:00 bash
30745 pts/20 R+ 0:00 \_ ps --tty pts/20 fw
newConnector /usr/bin/bc "-l" '3*4' 12
ps --tty pts/20 fw
PID TTY STAT TIME COMMAND
29019 pts/20 Ss 0:00 bash
30944 pts/20 S 0:00 \_ /usr/bin/bc -l
30952 pts/20 R+ 0:00 \_ ps --tty pts/20 fw
declare -p PI
bash: declare: PI: not found
myBc '4*a(1)' PI
declare -p PI
declare -- PI="3.14159265358979323844"
The function myBc lets you use the background task with simple syntax.
Then for date:
newConnector /bin/date '-f - +%s' #0 0
myDate '2000-01-01'
946681200
myDate "$(ps ho lstart 1)" boottime
myDate now now
read utm idl </proc/uptime
myBc "$now-$boottime" uptime
printf "%s\n" ${utm%%.*} $uptime
42134906
42134906
ps --tty pts/20 fw
PID TTY STAT TIME COMMAND
29019 pts/20 Ss 0:00 bash
30944 pts/20 S 0:00 \_ /usr/bin/bc -l
32615 pts/20 S 0:00 \_ /bin/date -f - +%s
3162 pts/20 R+ 0:00 \_ ps --tty pts/20 fw
From there, if you want to end one of background processes, you just have to close its fd:
eval "exec $DATEOUT>&-"
eval "exec $DATEIN>&-"
ps --tty pts/20 fw
PID TTY STAT TIME COMMAND
4936 pts/20 Ss 0:00 bash
5256 pts/20 S 0:00 \_ /usr/bin/bc -l
6358 pts/20 R+ 0:00 \_ ps --tty pts/20 fw
which is not needed, because all fd close when the main process finishes.
As they have already indicated to you, you should use `backticks`.
The alternative proposed $(command) works as well, and it also easier to read, but note that it is valid only with Bash or KornShell (and shells derived from those),
so if your scripts have to be really portable on various Unix systems, you should prefer the old backticks notation.
I know three ways to do it:
Functions are suitable for such tasks:**
func (){
ls -l
}
Invoke it by saying func.
Also another suitable solution could be eval:
var="ls -l"
eval $var
The third one is using variables directly:
var=$(ls -l)
OR
var=`ls -l`
You can get the output of the third solution in a good way:
echo "$var"
And also in a nasty way:
echo $var
Just to be different:
MOREF=$(sudo run command against $VAR1 | grep name | cut -c7-)
When setting a variable make sure you have no spaces before and/or after the = sign. I literally spent an hour trying to figure this out, trying all kinds of solutions! This is not cool.
Correct:
WTFF=`echo "stuff"`
echo "Example: $WTFF"
Will Fail with error "stuff: not found" or similar
WTFF= `echo "stuff"`
echo "Example: $WTFF"
If you want to do it with multiline/multiple command/s then you can do this:
output=$( bash <<EOF
# Multiline/multiple command/s
EOF
)
Or:
output=$(
# Multiline/multiple command/s
)
Example:
#!/bin/bash
output="$( bash <<EOF
echo first
echo second
echo third
EOF
)"
echo "$output"
Output:
first
second
third
Using heredoc, you can simplify things pretty easily by breaking down your long single line code into a multiline one. Another example:
output="$( ssh -p $port $user#$domain <<EOF
# Breakdown your long ssh command into multiline here.
EOF
)"
You need to use either
$(command-here)
or
`command-here`
Example
#!/bin/bash
VAR1="$1"
VAR2="$2"
MOREF="$(sudo run command against "$VAR1" | grep name | cut -c7-)"
echo "$MOREF"
If the command that you are trying to execute fails, it would write the output onto the error stream and would then be printed out to the console.
To avoid it, you must redirect the error stream:
result=$(ls -l something_that_does_not_exist 2>&1)
This is another way and is good to use with some text editors that are unable to correctly highlight every intricate code you create:
read -r -d '' str < <(cat somefile.txt)
echo "${#str}"
echo "$str"
You can use backticks (also known as accent graves) or $().
Like:
OUTPUT=$(x+2);
OUTPUT=`x+2`;
Both have the same effect. But OUTPUT=$(x+2) is more readable and the latest one.
Here are two more ways:
Please keep in mind that space is very important in Bash. So, if you want your command to run, use as is without introducing any more spaces.
The following assigns harshil to L and then prints it
L=$"harshil"
echo "$L"
The following assigns the output of the command tr to L2. tr is being operated on another variable, L1.
L2=$(echo "$L1" | tr [:upper:] [:lower:])
Mac/OSX nowadays come with old Bash versions, ie GNU bash, version 3.2.57(1)-release (arm64-apple-darwin21). In this case, one can use:
new_variable="$(some_command)"
A concrete example:
newvar="$(echo $var | tr -d '123')"
Note the (), instead of the usual {} in Bash 4.
Some may find this useful.
Integer values in variable substitution, where the trick is using $(()) double brackets:
N=3
M=3
COUNT=$N-1
ARR[0]=3
ARR[1]=2
ARR[2]=4
ARR[3]=1
while (( COUNT < ${#ARR[#]} ))
do
ARR[$COUNT]=$((ARR[COUNT]*M))
(( COUNT=$COUNT+$N ))
done

"Too late for "-C" option" error With Perl and Shell scripts

I have a jar application that has several functions, one of which is to convert from HTML to XML. When I try to run a simple command such as:
java -jar lt4el-cmd.jar send -l en "l2:https://en.wikipedia.org/wiki/Personal_computer"
I get the following errors:
ERROR [Thread-1]: html2base/html2base-wrapper.sh: Too late for "-C" option at html2base/html2xml.pl line 1.
/tmp/lpc.30872.html: failed
cat: /tmp/lpc.30872.xml: No such file or directory
(LpcControl.java:229)
ERROR [Thread-1]: ana2ont/ana2ont.sh ${lang}: -:1: parser error : Document is empty
-:1: parser error : Start tag expected, '<' not found
Tokenization/tagging failed
^
-:1: parser error : Document is empty
unable to parse -
-:1: parser error : Document is empty
unable to parse -
(LpcControl.java:229)
ERROR [Thread-1]: Error in conversion: Error running conversion script (ana2ont/ana2ont.sh ${lang}): 6 (AppInterface.java:159)
This is the html2base-wrapper.sh script which seems to be where the first error occurs.
#!/bin/bash
if [ "$1" == "check" ]; then
. common.sh
check_binary perl || exit 1
check_perl_module HTML::TreeBuilder || exit 1
check_perl_module XML::LibXML || exit 1
check_binary tidy || exit 1
check_binary xmllint || exit 1
check_binary xsltproc || exit 1
exit
fi
cat >"$TMPDIR/lpc.$$.html"
html2base/html2base.sh -d html2base/LT4ELBase.dtd -x html2base/LT4ELBase.xslt -t "$TMPDIR/lpc.$$.html" >&2
cat "$TMPDIR/lpc.$$.xml";
rm -f "$TMPDIR"/lpc.$$.{ht,x}ml
And the html2base.sh script:
#!/bin/bash
#
# Sample script for automated HTML -> XML conversion
#
# Miroslav Spousta <spousta#ufal.mff.cuni.cz>
# $Id: html2base.sh 462 2008-03-17 08:37:14Z qiq $
basedir=`dirname $0`;
# constants
HTML2XML_BIN=${basedir}/html2xml.pl
ICONV_BIN=iconv
TIDY_BIN=tidy
XMLLINT_BIN=xmllint
XSLTPROC_BIN=xsltproc
DTDPARSE_BIN=dtdparse
TMPDIR=/tmp
# default values
VERBOSE=0
ENCODING=
TIDY=0
VALIDATE=0
DTD=${basedir}/LT4ELBase.dtd
XSLT=${basedir}/LT4ELBase.xslt
usage()
{
echo "usage: html2base.sh [options] file(s)"
echo "XML -> HTML conversion script."
echo
echo " -e, --encoding=charset Convert input files from encoding to UTF-8 (none)"
echo " -d, --dtd=file DTD to be used for conversion and validation ($DTD)"
echo " -x, --xslt=file XSLT to be applied after conversion ($XSLT)"
echo " -t, --tidy Run HTMLTidy on input HTML files"
echo " -a, --validate Validate output XML files"
echo " -v, --verbose Be verbose"
echo " -h, --help Print this usage"
exit 1;
}
OPTIONS=`getopt -o e:d:x:tahv -l encoding:,dtd:,xlst,tidy,validate,verbose,help -n 'convert.sh' -- "$#"`
if [ $? != 0 ]; then
usage;
fi
eval set -- "$OPTIONS"
while true ; do
case "$1" in
-e | --encoding) ENCODING=$2; shift 2 ;;
-d | --dtd) DTD=$2; shift 2 ;;
-x | --xslt) XSLT=$2; shift 2 ;;
-t | --tidy) TIDY=1; shift 1;;
-a | --validate) VALIDATE=1; shift 1;;
-v | --verbose) VERBOSE=1; shift 1 ;;
-h | --help) usage; shift 1 ;;
--) shift ; break ;;
*) echo "Internal error!" ; echo $1; exit 1 ;;
esac
done
if [ $# -eq 0 ]; then
usage;
fi
DTD_XML=`echo "$DTD"|sed -e 's/\.dtd/.xml/'`
if [ "$VERBOSE" -eq 1 ]; then
VERBOSE=--verbose
else
VERBOSE=
fi
# create $DTD_XML if necessary
if [ ! -f "$DTD_XML" ]; then
if ! $DTDPARSE_BIN $DTD -o $DTD_XML 2>/dev/null; then
echo "cannot run dtdparse, cannot create $DTD_XML";
exit 1;
fi;
fi
# process file by file
total=0
nok=0
while [ -n "$1" ]; do
file=$1;
if [ -n "$VERBOSE" ]; then
echo "Processing $file..."
fi
f="$file";
result=0;
if [ -n "$ENCODING" ]; then
$ICONV_BIN -f "$ENCODING" -t utf-8 "$f" -o "$file.xtmp"
result=$?
error="encoding error"
f=$file.xtmp
fi
if [ "$result" -eq 0 ]; then
if [ "$TIDY" = '1' ]; then
$TIDY_BIN --force-output 1 -q -utf8 >"$file.xtmp2" "$f" 2>/dev/null
f=$file.xtmp2
fi
out=`echo $file|sed -e 's/\.x\?html\?$/.xml/'`
if [ "$out" = "$file" ]; then
out="$out.xml"
fi
$HTML2XML_BIN --simplify-ws $VERBOSE $DTD_XML -o "$out" "$f"
result=$?
error="failed"
fi
if [ "$result" -eq 0 ]; then
$XSLTPROC_BIN --path `dirname $DTD` $XSLT "$out" |$XMLLINT_BIN --noblanks --format -o "$out.tmp1" -
result=$?
error="failed"
mv "$out.tmp1" "$out"
if [ "$result" -eq 0 -a "$VALIDATE" = '1' ]; then
tmp=`dirname $file`/$DTD
delete=0
if [ ! -f $tmp ]; then
cp $DTD $tmp
delete=1
fi
$XMLLINT_BIN --path `dirname $DTD` --valid --noout "$out"
result=$?
error="validation error"
if [ "$delete" -eq 1 ]; then
rm -f $tmp
fi
fi
fi
if [ "$result" -eq 0 ]; then
if [ -n "$VERBOSE" ]; then
echo "OK"
fi
else
echo "$file: $error "
nok=`expr $nok + 1`
fi
total=`expr $total + 1`
rm -f $file.xtmp $file.xtmp2
shift;
done
if [ -n "$VERBOSE" ]; then
echo
echo "Total: $total, failed: $nok"
fi
And the beginning part of the html2xml.pl file:
#!/usr/bin/perl -W -C
# Simple HTML to XML (subset of XHTML) conversion tool. Should always produce a
# valid XML file according to the output DTD file specified.
#
# Miroslav Spousta <spousta#ufal.mff.cuni.cz>
# $Id: html2xml.pl 461 2008-03-09 09:49:42Z qiq $
use HTML::TreeBuilder;
use HTML::Element;
use HTML::Entities;
use XML::LibXML;
use Getopt::Long;
use Data::Dumper;
use strict;
I can't seem to figure where the problem is. And what exactly does ERROR [Thread-1] mean?
Thanks
The error comes from having -C on the shebang (#!) line of a Perl script, but not passing the -C to perl. This type of error happens when someone does
perl html2base/html2xml.pl ...
instead of
html2base/html2xml.pl ...
The error was from the the html2xml.pl script as other users rightly mentioned. I'm running ubuntu 16.04.2 system which comes with a default perl 5.22 version. And as this post mentions, using the -C option (as from perl 5.10.1) on the #! line requires you to also specify it on the command line at execution time, which I wasn't sure how to do because I was running a jar file. I installed perlbrew, instead, which I used to get an earlier version of perl and modified my perl script to:
#!/usr/bin/path/to/perlbrew/perl -W -C
# Simple HTML to XML (subset of XHTML) conversion tool. Should always produce a
# valid XML file according to the output DTD file specified.
#
# Miroslav Spousta <spousta#ufal.mff.cuni.cz>
# $Id: html2xml.pl 461 2008-03-09 09:49:42Z qiq $
This might also come in handy in setting up shell scripts when using perlbrew.
Thanks for the efforts in contribution.

Search and replace with sed

Last week I accidently externalized all my strings of my eclipse project. I need to revert this and my only hope is sed. I tried to create scripts but failed pathetically because I'm new with sed and this would be a very complicated operation. What I need to do is this:
Strings in class.java file is currently in the following format(method) Messages.getString(<key>). Example :
if (new File(DataSource.DEFAULT_VS_PATH).exists()) {
for (int i = 1; i <= c; i++) {
if (!new File(DataSource.DEFAULT_VS_PATH
+ Messages.getString("VSDataSource.89") + i).exists()) { //$NON-NLS-1$
getnewvfspath = DataSource.DEFAULT_VS_PATH
+ Messages.getString("VSDataSource.90") + i; //$NON-NLS-1$
break;
}
}
}
The key and matching Strings are in messages.properties file in the following format.
VSDataSource.92=No of rows in db =
VSDataSource.93=Verifying db entry :
VSDataSource.94=DB is open
VSDataSource.95=DB is closed
VSDataSource.96=Invalid db entry for
VSDataSource.97=\ removed.
key=string
So I need the java file back in this format:
if (new File(DataSource.DEFAULT_VS_PATH).exists()) {
for (int i = 1; i <= c; i++) {
if (!new File(DataSource.DEFAULT_VS_PATH
+ "String 2" + i).exists()) { //$NON-NLS-1$
getnewvfspath = DataSource.DEFAULT_VS_PATH
+ "String 1" + i; //$NON-NLS-1$
break;
}
}
}
How can I accomplish this with sed? Or is there an easier way?
This might work for you (GNU sed?):
sed 's|^\([^=]*\)=\(.*\)|s#Messages.getString("\1")#"\2"#g|;s/\\/\\\\/g' messages.properties |
sed -i -f - *.java
To repeat my comment on the question - I think that Java problems are best solved in Java :) Though this arguably is an Eclipse-helped problem caused by you :)
Make a Java program in which you can:
Read the properties,
Traverse all your project .java files,
For each file:
Read each file line by line,
Replace all the strings by using regexps, keying from the loaded properties,
Save when done reading all lines.
Not a 2-minute job, but easy enough.
But if you really want to use sed ;)
mkt.sh
$ cat mkt.sh
# Test structure
rm -rf a b
mkdir a
mkdir b
cat > a/A.java <<EOF
my plans for replace
this will be left alone
EOF
cat > b/B.java <<EOF
propery ginger
broccoli tomato potato
EOF
display() {
for i in a/A.java b/B.java; do
echo --- $i
cat $i
done
}
display
# Prop change
echo 'echo --- Replacing in: $1' > replace.sh
sed -r 's/([^=]+)=(.+)/sed -i '\''s#\1#\2#'\'' $1/' sample.properties >> replace.sh
chmod u+x replace.sh
# Replace
find -type f -name "*.java"|xargs -n1 ./replace.sh
# Test
display
Run:
$ ./mkt.sh
--- a/A.java
my plans for replace
this will be left alone
--- b/B.java
propery ginger
broccoli tomato potato
--- Replacing in: ./a/A.java
--- Replacing in: ./b/B.java
--- a/A.java
my plans for world domination
this will be left alone
--- b/B.java
propery ginger
steak tomato potato
This should work properly on your .java files, but do make a copy before ;) You will have some issues if # is in the strings, but you can solve this by removing these from properties file, doing a replace, bringing them back and changing this line:
sed -r 's/([^=]+)=(.+)/sed -i '\''s#\1#\2#'\'' $1/' sample.properties >> replace.sh
to e.g.:
sed -r 's/([^=]+)=(.+)/sed -i '\''s+\1+\2+'\'' $1/' sample.properties >> replace.sh
where + is not a remaining character. A bit of a hassle, but...
Hope this helps.
makesed.awk:
BEGIN {
FS="=";
print "for i in *.java"
print "do"
print "sed \\"
}
{
msg = "Messages.getString(\"" $1 "\")";
gsub("/","\\/",$2);
print "-e 's/" msg "/\"" $2 "\"/g' \\"
}
END {
print "$i > $$"
print "mv $$ $i"
print "done"
}
Run:
awk -f makesed.awk yourpropertiesfile.dat > process.sh
This gives you a shell script:
for i in *.java
do
sed \
-e 's/Messages.getString("VSDataSource.92")/"No of rows in db "/g' \
-e 's/Messages.getString("VSDataSource.93")/"Verifying db entry : "/g' \
-e 's/Messages.getString("VSDataSource.94")/"DB is open"/g' \
-e 's/Messages.getString("VSDataSource.95")/"DB is closed"/g' \
-e 's/Messages.getString("VSDataSource.96")/"Invalid db entry for "/g' \
-e 's/Messages.getString("VSDataSource.97")/"\ removed."/g' \
$i > $$
mv $$ $i
done
Then go in to your respective Java directories and run:
sh process.sh
That will "fix" all of the java files in that directory.
If your properties file is long, you may very well run in to a command line limit with sed. Simply split the file up in to chunks until the script is happy.
Obviously this doesn't work with any escape character, if you have "=" in your messages you'll suffer some pain as well. If you're fool enough to run this on code that isn't backed up, then you certainly deserve whatever happens to you.
But it should be a good first start.
You don't need to program anything. Right-click on one of your modified files and select "Replace With >", "Previous from Local History". Repeat as necessary.

How to use Flash content in a Java Micro Edition application?

Despite my luddite tendencies, I now have a phone with Java support - but no Flash support. I also have a copy of Macromedia Flash MX2004, though I'm unlikely to upgrade any time soon.
What I'd like to be able to do is develop some content (including vector animations) in Flash, then use those resources in a Java Micro Edition application. I don't need all features of Flash - in particular, I don't care about ActionScript support. But I do want to be able to load a SWF file (or, perhaps better, an alternative file format that can be generated using a converter tool), and to be able to display animations and use other resources (particularly play sounds) from in that file.
Is there a good library and toolkit to support this kind of thing? Obviously (from the MX2004) it doesn't need to be completely up to date.
On knowledge level - I've been a programmer for decades, and my everyday language these days is C++. However, I have a very limited knowledge of Java, and virtually no knowledge (yet) of Micro Edition and its libraries.
I've already heard of Flash to J2ME converters, but so far as I can see they generate complete applications in one step, rather than treating the SWF file as a source of resources to be controlled from separately written Java code.
EDIT
I get the feeling that this is (with slight modifications) probably quite easy. Java Mobile Edition supports SVG vector graphics. SVG supports animations. There are (I'm pretty certain) ways to convert flash animations to SVG - probably a simple export-to-SVG in the application, though I've not checked.
This in itself doesn't give me a convenient bundle-of-media resources file format, but that's a relatively simple problem to solve, so long as there's a way to "load" SVG and other media files from some kind of non-file stream class that gets its data in turn from the bundle-of-media file.
I've never used Java ME before, so I won't be able to help on that side, but I use actionscript/flash on a daily basis.
The 'easiest' thing I can think of is a 2 step process:
Export your animation as vector sequence via File > Export > Export Movie and choosing the right format (e.g. .ai/.eps/.dxf).
Convert the vector sequence to svg. Inkscape has a few handy SVG conversion tools.
A long winded way would be to write a JSFL script in Flash MX 2004.
You would traverse the shapes for each frame, then write the path data to SVG.
Another slightly different way would be to export the vector sequence as explained above (unfortunately there is no JSFL functionality to automate that), then from JSFL read and loop through each file, parse it and write an SVG.
The only advantage this would give you though is not having to install Inkscape and you wouldn't need to switch to another application.
I wouldn't recommend this though because:
You would need to write a parser (dxf/eps might be the simplest)
You will need to make an SVG and you only have Strings at your disposal (E4X XML support was added in Flash CS3)
I'm not saying it's impossible, it just seems impractical.
Found this thread on the Inkscape forum sharing a bash script that
extracts SWF objects to an SVG file
using SWFTools, but haven't tried that yet. For reference, here is hadi's script:
#!/bin/bash
#USAGE ./swf2svg.sh /path/to/file.swf > output.svg
FILE=$1;
DUMP="dump.txt"
echo '<?xml version="1.0" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg width="100%" height="100%" version="1.1"
xmlns="http://www.w3.org/2000/svg">
';
swfdump -s $FILE > $DUMP
fillCols=();
lineCols=();
lineWidth=();
FILLREGEX="[0-9]+(\s*)SOLID [0-f]{8}";
LINEREGEX="[0-9]+(\s*)[0-9]\.[0-9]{0,2} [0-f]{8}";
lastStartPoint="";
pathClosedTag="";
firstGroup="TRUE";
firstPath="TRUE";
cat $DUMP | while read line
do
#Remove ( and )
line=`echo $line | sed "s/[()]//g"`
#tmp=`echo $line | egrep -o "DEFINE(SHAPE|SPRITE)"`;
tmp=`echo $line | egrep -o "DEFINE(SHAPE|SPRITE)[0-9]? defines id [0-9]+"`;
if [ "$tmp" != "" ]
then
if [ "$firstGroup" == "TRUE" ]
then
firstGroup="FALSE";
else
if [ "$firstPath" == "FALSE" ]
then
if [ "$lastStartPoint" != "" ]
then
if [ "$lastStartPoint" == "$curPoint" ]
then
pathClosedTag="Z";
fi
fi
lastStartPoint=$curPoint;
echo $pathClosedTag'" />';
fi;
firstPath="TRUE";
echo '</g>';
fi
id=`echo $tmp | awk {'print $4'}`
echo '<g id="'$id'">';
fillCols=();
lineCols=();
lineWidth=();
fi
tmp=`echo $line | egrep -o "($FILLREGEX)?((\s*)$LINEREGEX)?"`;
if [ "$tmp" != "" ]
then
fillInx=`echo $tmp | egrep -o "$FILLREGEX" | awk {'print $1'}`;
fillCol=`echo $tmp | egrep -o "$FILLREGEX" | awk {'print $3'}`;
if [ "$fillCol" != "" ]
then
fillCols[$fillInx]=$fillCol;
fi
lineInx=`echo $tmp | egrep -o "$LINEREGEX" | awk {'print $1'}`;
lineWth=`echo $tmp | egrep -o "$LINEREGEX" | awk {'print $2'}`;
lineCol=`echo $tmp | egrep -o "$LINEREGEX" | awk {'print $3'}`;
if [ "$lineCol" != "" ]
then
lineCols[$lineInx]=$lineCol;
lineWidth[$lineInx]=$lineWth;
fi
fi
tmp=`echo $line | awk {'print $6'}`;
if [ "$tmp" == "lineTo" ]
then
echo $line | awk {'print "L"$7" "$8'}
fi
if [ "$tmp" == "moveTo" ]
then
curPoint=`echo $line | awk {'print $9" "$10'}`;
if [ "$lastStartPoint" != "" ]
then
if [ "$lastStartPoint" == "$curPoint" ]
then
pathClosedTag="Z";
fi
fi
lastStartPoint=$curPoint;
if [ "$firstPath" == "TRUE" ]
then
firstPath="FALSE";
else
echo $pathClosedTag'" />';
fi;
#Remove : and /
line=`echo $line | sed "s/[:/]/ /g"`
fInx=`echo $line | awk '{printf "%d", $4}'`;
lInx=`echo $line | awk '{printf "%d", $6}'`;
stl="";
val=${fillCols[$fInx]:0:6};
if [ $fInx -gt 0 -a "$val" != "" ]
then
stl="fill:#$val;";
fi
val=${lineCols[$lInx]:0:6};
if [ $lInx -gt 0 -a "$val" != "" ]
then
stl=$stl"stroke:#$val;";
val=${lineWidth[$lInx]};
if [ "$val" != "" ]
then
stl=$stl"stroke-width:$val;";
fi
fi
echo '<path style="'$stl'" d="';
echo $line | awk {'print "M"$9" "$10'}
fi
if [ "$tmp" == "splineTo" ]
then
echo $line | awk {'print "Q"$7" "$8" "$9" "$10'}
fi
done
echo 'Z" />';
echo '</g>';
echo '</svg>';
If anybody else using a more recent version of Flash (like CS4 or CS5) reads this, there is a Flash 2 SVG extension available.

Categories