We Keep Coding

TypeError: 'JavaPackage' object is not callable on google collab [duplicate]

This question already has an answer here:
spark-nlp 'JavaPackage' object is not callable
(1 answer)
Closed 1 year ago.
I am learning apache spark and I ran below code on google colab.
#installed based upon https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/quick_start_google_colab.ipynb#scrollTo=lNu3meQKEXdu
import os
# Install java
!apt-get install -y openjdk-8-jdk-headless -qq > /dev/null
!wget -q "https://downloads.apache.org/spark/spark-3.1.1/spark-3.1.1-bin-hadoop2.7.tgz" > /dev/null
!tar -xvf spark-3.1.1-bin-hadoop2.7.tgz > /dev/null
!pip install -q findspark
os.environ["SPARK_HOME"] = "/content/spark-3.1.1-bin-hadoop2.7"
os.environ["JAVA_HOME"] = "/usr/lib/jvm/java-8-openjdk-amd64"
os.environ["PATH"] = os.environ["JAVA_HOME"] + "/bin:" + os.environ["PATH"]
! java -version
# Install spark-nlp and pyspark
! pip install spark-nlp==3.0.0 pyspark==3.1.1
import sparknlp
spark = sparknlp.start()
from sparknlp.base import DocumentAssembler
documentAssembler = DocumentAssembler().setInputCol(text_col).setOutputCol('document')
I get the below error. How could i resolve it
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-48-535b177b526b> in <module>()
4
5 from sparknlp.base import DocumentAssembler
----> 6 documentAssembler = DocumentAssembler().setInputCol(text_col).setOutputCol('document')
4 frames
/usr/local/lib/python3.7/dist-packages/pyspark/ml/wrapper.py in _new_java_obj(java_class, *args)
64 java_obj = getattr(java_obj, name)
65 java_args = [_py2java(sc, arg) for arg in args]
---> 66 return java_obj(*java_args)
67
68 #staticmethod
TypeError: 'JavaPackage' object is not callable

As mentioneed in my last comment :
Change the text_col by the name of the column in your spark dataframe that you have , document by the its name
You can add .setCleanupMode("clean_mode")
For more details you can refer to this link: https://spark.apache.org/docs/latest/ml-features
documentAssembler = DocumentAssembler \
.setInputCol("text_col") \
.setOutputCol("document")

How do i display linux command result in html [duplicate]

This question already has an answer here:
Formatting issues when unix file is sent to mail
(1 answer)
Closed 1 year ago.
I want to run linux command(ps aux --sort -rss | head -n 10) and display the result in table format in mail.
can somebody please suggest.
here is my script.
#!/usr/bin/ksh
current_dir=$(pwd)
script_dir=$(dirname $0)
if [ $script_dir = '.' ]; then
script_dir="$current_dir"
fi
source_dir=$PWD
cd $script_dir
FWK_TIMESTAMP=`date +%Y%m%d_%H:%M:%S`
REPORT_TS=`date -d '1 day ago' +'%m/%d/%Y'`
TILL_DATE=`date +%m/%d/%Y`
#javac -classpath mail1.4.4.jar:activation.jar SendEmail.java
if [ $? -ne 0 ];then
echo javac failed $?
exit 1
fi
outfile=test.txt
rm test.txt
echo "<h1> Top 10 Memory Consumed Processes from $REPORT_TS to $TILL_DATE</h1>" >> $outfile
ps aux --sort -rss | head -n 10 >> $outfile
to='abc#abc.com'
mail_server='-mail.smtp.host abc.com'
param="-to $to $mail_server -from abc#abc.com"
#-mail.debug
java -cp mail1.4.4.jar:activation.jar:. SendEmail $param -subject "Top Memory Consumed Processes" -file test.txt

As I am able to see in your code,
you are saving the results of
ps aux --sort -rss | head -n 10 >> $outfile
in the $outfile.
You can use <iframe src="your_filename_goes_here" width=200 height=400 frameborder=0 ></iframe>

"Too late for "-C" option" error With Perl and Shell scripts

I have a jar application that has several functions, one of which is to convert from HTML to XML. When I try to run a simple command such as:
java -jar lt4el-cmd.jar send -l en "l2:https://en.wikipedia.org/wiki/Personal_computer"
I get the following errors:
ERROR [Thread-1]: html2base/html2base-wrapper.sh: Too late for "-C" option at html2base/html2xml.pl line 1.
/tmp/lpc.30872.html: failed
cat: /tmp/lpc.30872.xml: No such file or directory
(LpcControl.java:229)
ERROR [Thread-1]: ana2ont/ana2ont.sh ${lang}: -:1: parser error : Document is empty
-:1: parser error : Start tag expected, '<' not found
Tokenization/tagging failed
^
-:1: parser error : Document is empty
unable to parse -
-:1: parser error : Document is empty
unable to parse -
(LpcControl.java:229)
ERROR [Thread-1]: Error in conversion: Error running conversion script (ana2ont/ana2ont.sh ${lang}): 6 (AppInterface.java:159)
This is the html2base-wrapper.sh script which seems to be where the first error occurs.
#!/bin/bash
if [ "$1" == "check" ]; then
. common.sh
check_binary perl || exit 1
check_perl_module HTML::TreeBuilder || exit 1
check_perl_module XML::LibXML || exit 1
check_binary tidy || exit 1
check_binary xmllint || exit 1
check_binary xsltproc || exit 1
exit
fi
cat >"$TMPDIR/lpc.$$.html"
html2base/html2base.sh -d html2base/LT4ELBase.dtd -x html2base/LT4ELBase.xslt -t "$TMPDIR/lpc.$$.html" >&2
cat "$TMPDIR/lpc.$$.xml";
rm -f "$TMPDIR"/lpc.$$.{ht,x}ml
And the html2base.sh script:
#!/bin/bash
#
# Sample script for automated HTML -> XML conversion
#
# Miroslav Spousta <spousta#ufal.mff.cuni.cz>
# $Id: html2base.sh 462 2008-03-17 08:37:14Z qiq $
basedir=`dirname $0`;
# constants
HTML2XML_BIN=${basedir}/html2xml.pl
ICONV_BIN=iconv
TIDY_BIN=tidy
XMLLINT_BIN=xmllint
XSLTPROC_BIN=xsltproc
DTDPARSE_BIN=dtdparse
TMPDIR=/tmp
# default values
VERBOSE=0
ENCODING=
TIDY=0
VALIDATE=0
DTD=${basedir}/LT4ELBase.dtd
XSLT=${basedir}/LT4ELBase.xslt
usage()
{
echo "usage: html2base.sh [options] file(s)"
echo "XML -> HTML conversion script."
echo
echo " -e, --encoding=charset Convert input files from encoding to UTF-8 (none)"
echo " -d, --dtd=file DTD to be used for conversion and validation ($DTD)"
echo " -x, --xslt=file XSLT to be applied after conversion ($XSLT)"
echo " -t, --tidy Run HTMLTidy on input HTML files"
echo " -a, --validate Validate output XML files"
echo " -v, --verbose Be verbose"
echo " -h, --help Print this usage"
exit 1;
}
OPTIONS=`getopt -o e:d:x:tahv -l encoding:,dtd:,xlst,tidy,validate,verbose,help -n 'convert.sh' -- "$#"`
if [ $? != 0 ]; then
usage;
fi
eval set -- "$OPTIONS"
while true ; do
case "$1" in
-e | --encoding) ENCODING=$2; shift 2 ;;
-d | --dtd) DTD=$2; shift 2 ;;
-x | --xslt) XSLT=$2; shift 2 ;;
-t | --tidy) TIDY=1; shift 1;;
-a | --validate) VALIDATE=1; shift 1;;
-v | --verbose) VERBOSE=1; shift 1 ;;
-h | --help) usage; shift 1 ;;
--) shift ; break ;;
*) echo "Internal error!" ; echo $1; exit 1 ;;
esac
done
if [ $# -eq 0 ]; then
usage;
fi
DTD_XML=`echo "$DTD"|sed -e 's/\.dtd/.xml/'`
if [ "$VERBOSE" -eq 1 ]; then
VERBOSE=--verbose
else
VERBOSE=
fi
# create $DTD_XML if necessary
if [ ! -f "$DTD_XML" ]; then
if ! $DTDPARSE_BIN $DTD -o $DTD_XML 2>/dev/null; then
echo "cannot run dtdparse, cannot create $DTD_XML";
exit 1;
fi;
fi
# process file by file
total=0
nok=0
while [ -n "$1" ]; do
file=$1;
if [ -n "$VERBOSE" ]; then
echo "Processing $file..."
fi
f="$file";
result=0;
if [ -n "$ENCODING" ]; then
$ICONV_BIN -f "$ENCODING" -t utf-8 "$f" -o "$file.xtmp"
result=$?
error="encoding error"
f=$file.xtmp
fi
if [ "$result" -eq 0 ]; then
if [ "$TIDY" = '1' ]; then
$TIDY_BIN --force-output 1 -q -utf8 >"$file.xtmp2" "$f" 2>/dev/null
f=$file.xtmp2
fi
out=`echo $file|sed -e 's/\.x\?html\?$/.xml/'`
if [ "$out" = "$file" ]; then
out="$out.xml"
fi
$HTML2XML_BIN --simplify-ws $VERBOSE $DTD_XML -o "$out" "$f"
result=$?
error="failed"
fi
if [ "$result" -eq 0 ]; then
$XSLTPROC_BIN --path `dirname $DTD` $XSLT "$out" |$XMLLINT_BIN --noblanks --format -o "$out.tmp1" -
result=$?
error="failed"
mv "$out.tmp1" "$out"
if [ "$result" -eq 0 -a "$VALIDATE" = '1' ]; then
tmp=`dirname $file`/$DTD
delete=0
if [ ! -f $tmp ]; then
cp $DTD $tmp
delete=1
fi
$XMLLINT_BIN --path `dirname $DTD` --valid --noout "$out"
result=$?
error="validation error"
if [ "$delete" -eq 1 ]; then
rm -f $tmp
fi
fi
fi
if [ "$result" -eq 0 ]; then
if [ -n "$VERBOSE" ]; then
echo "OK"
fi
else
echo "$file: $error "
nok=`expr $nok + 1`
fi
total=`expr $total + 1`
rm -f $file.xtmp $file.xtmp2
shift;
done
if [ -n "$VERBOSE" ]; then
echo
echo "Total: $total, failed: $nok"
fi
And the beginning part of the html2xml.pl file:
#!/usr/bin/perl -W -C
# Simple HTML to XML (subset of XHTML) conversion tool. Should always produce a
# valid XML file according to the output DTD file specified.
#
# Miroslav Spousta <spousta#ufal.mff.cuni.cz>
# $Id: html2xml.pl 461 2008-03-09 09:49:42Z qiq $
use HTML::TreeBuilder;
use HTML::Element;
use HTML::Entities;
use XML::LibXML;
use Getopt::Long;
use Data::Dumper;
use strict;
I can't seem to figure where the problem is. And what exactly does ERROR [Thread-1] mean?
Thanks

The error comes from having -C on the shebang (#!) line of a Perl script, but not passing the -C to perl. This type of error happens when someone does
perl html2base/html2xml.pl ...
instead of
html2base/html2xml.pl ...

The error was from the the html2xml.pl script as other users rightly mentioned. I'm running ubuntu 16.04.2 system which comes with a default perl 5.22 version. And as this post mentions, using the -C option (as from perl 5.10.1) on the #! line requires you to also specify it on the command line at execution time, which I wasn't sure how to do because I was running a jar file. I installed perlbrew, instead, which I used to get an earlier version of perl and modified my perl script to:
#!/usr/bin/path/to/perlbrew/perl -W -C
# Simple HTML to XML (subset of XHTML) conversion tool. Should always produce a
# valid XML file according to the output DTD file specified.
#
# Miroslav Spousta <spousta#ufal.mff.cuni.cz>
# $Id: html2xml.pl 461 2008-03-09 09:49:42Z qiq $
This might also come in handy in setting up shell scripts when using perlbrew.
Thanks for the efforts in contribution.

Perl strings inside system call to Java

I define array #ID at the beginning of the code. I do different things with this list of ids, and therefore I define my $id; at the beginning only (I use strict; warnings).
Maybe I simplified too much to ask the question, so it end up being unclear.
Both $cv and $smo are $id dependent and they are all perl strings. Like this:
....
for $id (#ID) {
$cv = $htotal{$id};
$smo = $hsmote{$id};
system('java -Djava.util.Arrays.useLegacyMergeSort=true weka.classifiers.meta.FilteredClassifier -t Projects/proteins/$id_MSA/$id_.arff -x $cv -s 0 -p 1,2 -distribution \
-F "weka.filters.MultiFilter -F \" weka.filters.unsupervised.attribute.Remove -R 7,9\" -F \" weka.filters.unsupervised.attribute.RemoveType -T string\" \
-F \"weka.filters.supervised.instance.SMOTE -C 0 -K 5 -P $smo -S 1\"" -W weka.classifiers.functions.MultilayerPerceptron -- -L 0.3 -M 0.2 -N 500 -V 0 -S 0 -E 20 -H 0 \
> Projects/rbfs/$id.rbf');
}
...
To make sure there were no Weka errors, I tried, e.g. cv = 2, smo =100, id=P12345; and it worked fine, so it was an interpolation issue, as some of you mentioned.
From the solutions you've mentioned I tried double quotes + {} as #nlu suggested:
system('java [...] -t Projects/proteins/"${id}"_MSA/"${id}"_.arff -x "${cv}" -s 0 -p 1,2 -distribution [...] -F \"weka.filters.supervised.instance.SMOTE -C 0 -K 5 -P "${smo}" -S 1\"" [...] > Projects/rbfs/"${id}".rbf');
But didn't worked, did I write it wrong?
Same with this (replacing system() for `` ) didn't work:
java [...] -t Projects/150400_GSupdate/proteins/${id}_MSA/${id}.arff -x ${cv} -s 0 -p 1,2 [...] -F \"weka.filters.supervised.instance.SMOTE -C 0 -K 5 -P ${smo} -S [...] > Projects/150400_GSupdate/rbfs/${id}.rbf;
What finally has worked for me is string concatenation (as #Matt suggested) but I'm sure that all other options should be fine too, although don't know why not.

It's not clear if the other variables are shell variables other than $id? If you want just $id to be substituted the easiest way is to just use string concatenation:
for $id (#ID) {
system('java -Djava.util.Arrays.useLegacyMergeSort=true weka.classifiers.meta.FilteredClassifier -t $PATH/proteins/$id_MSA/$id_.arff -x $cv -s 0 -p 1,2 -distribution \
-F "weka.filters.MultiFilter -F \" weka.filters.unsupervised.attribute.Remove -R 7,9\" -F \" weka.filters.unsupervised.attribute.RemoveType -T string\" \
-F \"weka.filters.supervised.instance.SMOTE -C 0 -K 5 -P $smo -S 1\"" -W weka.classifiers.functions.MultilayerPerceptron -- -L 0.3 -M 0.2 -N 500 -V 0 -S 0 -E 20 -H 0 \
> $PATH/rbfs/' . $id . '.rbf');
}
If you wanted all the $something to be treated as if they were perl variables probably the easiest way given you need to use " and \" inside the system string is to replace system() with the `` operator which basically does the same thing as system.
for $id (#ID) {
`java -Djava.util.Arrays.useLegacyMergeSort=true weka.classifiers.meta.FilteredClassifier -t $PATH/proteins/$id_MSA/$id_.arff -x $cv -s 0 -p 1,2 -distribution \
-F "weka.filters.MultiFilter -F \" weka.filters.unsupervised.attribute.Remove -R 7,9\" -F \" weka.filters.unsupervised.attribute.RemoveType -T string\" \
-F \"weka.filters.supervised.instance.SMOTE -C 0 -K 5 -P $smo -S 1\"" -W weka.classifiers.functions.MultilayerPerceptron -- -L 0.3 -M 0.2 -N 500 -V 0 -S 0 -E 20 -H 0 \
> $PATH/rbfs/$id.rbf`;
}

When you call a function in Perl, like system in your case, you can use interpolation to have variables inside the strings substituted.
You need to pass the string in double quotes for this to happen, so first you have to replace your surrounding single quotes with double quotes.
Besides, for disambiguation of any variable names appearing in the string, you should write them like this:
"${id}"
to avoid confusing variable names like i.e. id and idsomething.
Documented here:
http://perldoc.perl.org/perldata.html#Scalar-value-constructors

Uninstalling Java 6 and Reinstalling Java 7 using Powershell

I am using the following Powershell script. The first half (uninstall) works flawlessly. The second half (install) only works if I allow user input. Can anyone provide some assistance? Here is the script: (sorry for poor formatting)
#uninstall
$java = Get-WmiObject -Class win32_product | where { $_.Name -like "*Java*"}
$msiexec = "C:\Windows\system32\msiexec.exe";
$msiexecargs = '/x "$($app.IdentifyingNumber)" /qn /norestart'
if ($java -ne $null)
{
foreach ($app in $java)
{
write-host $app.LocalPackage
write-host $app.IdentifyingNumber
C:\Windows\system32\cmd.exe /c "C:\Windows\system32\msiexec.exe /x $($app.IdentifyingNumber) /qn"
Start-Process -FilePath $msiexec -Arg $msiexecargs -Wait -Passthru
[Diagnostics.Process]::Start($msiexec, $msiexecargs);
}
}
if ($java -ne $null)
{
foreach ($app in $java)
{
write-host $app.LocalPackage
write-host $app.IdentifyingNumber
C:\Windows\system32\cmd.exe /c
"C:\Windows\system32\msiexec.exe /x $($app.IdentifyingNumber) /qn"
Start-Process -FilePath $msiexec -Arg $msiexecargs -Wait -Passthru
[Diagnostics.Process]::Start($msiexec, $msiexecargs);
}
}
function Get-ScriptDirectory{
$Invocation = (Get-Variable MyInvocation -Scope 1).Value
try {
Split-Path $Invocation.MyCommand.Path -ea 0
}
catch {
Write-Warning 'You need to call this function from within a saved script.'
}
}
function Get-Architecture{
return $(gwmi win32_operatingsystem).OSArchitecture
}
$Path = Get-ScriptDirectory
#Close all instances of IE, Firefox, & Chrome
Get-Process | where {$_.ProcessName -match "iexplore"} | Stop-Process -Force
Get-Process | where {$_.ProcessName -match "chrome"} | Stop-Process -Force
Get-Process | where {$_.ProcessName -match "firefox"} | Stop-Process -Force
#Install
Start-Process -FilePath "msiexec.exe" -ArgumentList "/i "C:\temp\jre1.7.0_17.msi" ""/log "c:\temp\javainst.log " -Credential $cred -wait
#Also Install the 64-bit JRE if on a 64 workstation
if(Get-Architecture -match "64")
{
$cred = Get-Credential
Start-Process -FilePath "msiexec.exe" -ArgumentList "/i "C:\temp\jre1.7.0_17 (x64).msi" ""/log c:\temp\javainst.log " -Credential $cred -wait
}
#Import reg keys to disable auto updating
reg import "C:\temp\JavaUpdate.reg"{
}

#uninstall everything Java
Get-WmiObject -Class win32_product | ? {$_.Name -like "*Java*"} | % {msiexec /x "$($_.IdentifyingNumber)" /qn | Out-Null}
#The Out-Null waits for the command to finish
#If you have made a Java MSI use this
msiexec /i $pathtomsi /qn
#If you only have the exe you'll need to look up the Command Line Interface (CLI) for Java
$cmd = "$pathtoexe /s"
cmd /c $cmd
As for your script change the #Install line to:
Start-Process -FilePath 'msiexec.exe' -ArgumentList '/i "C:\temp\jre1.7.0_17.msi" /log "c:\temp\javainst.log" /qn' -Credential $cred -wait
Best Practice use mostly single quotes