rawGSODData = LOAD '/usr/local/Cellar/pig/0.12.0/gsod_2016/999999-93816-2016.op.gz' USING org.apache.pig.piggybank.storage.FixedWidthLoader('
1-6,
8-12,
15-18,
19-22,
25-30,
32-33,
36-41,
43-44,
47-52,
54-55,
58-63,
65-66,
69-73,
75-76,
79-83,
85-86,
89-93,
96-100,
103-108,
109-109,
111-116,
117-117,
119-123,
124-124,
126-130,
133-138',
'SKIP_HEADER');
When I try to run this code I will get an error saying
ERROR 1070: Could not resolve org.apache.pig.piggybank.storage.FixedWidthLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
I have the file FixedWidthLoader.java file in the directory
/usr/local/Cellar/pig/0.12.0/build/classes/org/apache/pig/piggybank/storage
Please help me with this error
Where is the piggybank.jar located ? Ensure you have registered piggybank.jar in your Pig script.If not, add this to the top of your pig script.Ensure the path to piggybank.jar is correct.Below statement registers the jar file i.e. piggybank located in /usr/local/
REGISTER '/usr/local/piggybank.jar';
Related
I am trying to run an implementation a jason code that is using some Internal Actions. The interpreter is showing that it was not possible to find the "java" code of the internal action, as showed:
Server running on http://191.36.8.42:3272
[aslparser] [peleus.asl:29] warning: The internal action class for 'org.soton.peleus.act.plan(Goals)' was not loaded! Error:
java.lang.ClassNotFoundException: org.soton.peleus.act.plan
[aslparser] [peleus.asl:42] warning: The internal action class for 'org.soton.peleus.act.isTrue(H)' was not loaded! Error:
java.lang.ClassNotFoundException: org.soton.peleus.act.isTrue
[peleus] Could not finish intention: intention 1: +des([on(b3,table),on(b2,b3),on(b1,b2)])[source(self)] <- ... org.soton.peleus.act.plan(Goals); !checkGoals(Goals); .print("Goals ",Goals," were satisfied") /
{Goals=[on(b3,table),on(b2,b3),on(b1,b2)]}Trigger: +des([on(b3,table),on(b2,b3),on(b1,b2)])[noenv,code(org.soton.peleus.act.plan([on(b3,table),on(b2,b3),on(b1,b2)])),code_line(29),code_src("peleus.asl"),error(action_failed),error_msg("no environment configured!"),source(self)]
[peleus] Adding belief clear(table)
This mas2j file is as following:
MAS peleus {
infrastructure: Centralised
agents:
peleus;
}
Part of agent code (written by Felipe Meneguzzi) is showed bellow:
//The next line is line 28
+des(Goals) : true
<- org.soton.peleus.act.plan(Goals);
!checkGoals(Goals);
.print("Goals ",Goals," were satisfied").
+!checkGoals([]) : true <- true.
//The next line is line 40
+!checkGoals([H|T]) : true
<- .print("Checking ", H);
org.soton.peleus.act.isTrue(H);
!checkGoals(T).
I guess it is about the folder structure, how to set up Jason to search for java files in specific locations?
The folders structure is like this:
Peleus\src\org\soton\peleus for java files
Peleus\examples for mas2j and asl tested project
It all depends on how you are executing the application.
If you are using java, the CLASSPATH should be defined to include the missing classes.
if you are using jason script (that uses Ant), the .mas2j file should include the class path as well.
More on that in the FAQ. Notice that CLASSPATH is where .class files are found, not .java source code files. The error regards a missing class, not a missing source code.
I'm running the following Ansible task to change permission of a directory and its content.
- name: Change ownership of everything below /opt/as2/app-server
file: path=/opt/as2/app-server state=directory recurse=yes owner=adrt group=adrt
When running it I get the following issue:
TASK [appserver : Change ownership of everything below /opt/as2/app-server] ****
fatal: [192.168.1.182]: FAILED! => {"changed": false, "failed": true, "module_stderr": "", "module_stdout": "Traceback (most recent call last):\r\n File \"/tmp/ansible_UrBo6x/ansible_module_file.py\", line 451, in \r\n main()\r\n File \"/tmp/ansible_UrBo6x/ansible_module_file.py\", line 335, in main\r\n changed |= recursive_set_attributes(module, to_bytes(file_args['path'], errors='surrogate_or_strict'), follow, file_args)\r\n File \"/tmp/ansible_UrBo6x/ansible_module_file.py\", line 146, in recursive_set_attributes\r\n changed |= module.set_fs_attributes_if_different(tmp_file_args, changed)\r\n File \"/tmp/ansible_UrBo6x/ansible_modlib.zip/ansible/module_utils/basic.py\", line 1163, in set_fs_attributes_if_different\r\n File \"/tmp/ansible_UrBo6x/ansible_modlib.zip/ansible/module_utils/basic.py\", line 929, in set_owner_if_different\r\n File \"/tmp/ansible_UrBo6x/ansible_modlib.zip/ansible/module_utils/basic.py\", line 842, in user_and_group\r\nOSError: [Errno 2] No such file or directory: '/opt/as2/app-server-1.0.0/apps/station/WEB-INF/classes/org/adroitlogic/isuite/metrics/As2MetricsService/usr/bin/python$tt__collectStats_closure14.class'\r\n", "msg": "MODULE FAILURE"}
Basically it says there is no such file or directory as,
/opt/as2/app-server-1.0.0/apps/station/WEB-INF/classes/org/adroitlogic/isuite/metrics/As2MetricsService/usr/bin/python$tt__collectStats_closure14.class
The content of the directory, /opt/as2/app-server/apps/station/WEB-INF/classes/org/adroitlogic/isuite/metrics/ is,
As2MetricsService$_$tt__CountStatisticsLists_closure3.class
As2MetricsService$_$tt__collectStats_closure10.class
As2MetricsService$_$tt__collectStats_closure11.class
As2MetricsService$_$tt__collectStats_closure12.class
As2MetricsService$_$tt__collectStats_closure13.class
As2MetricsService$_$tt__collectStats_closure14.class
As2MetricsService$_$tt__collectStats_closure15.class
As2MetricsService$_$tt__collectStats_closure4.class
As2MetricsService$_$tt__collectStats_closure5.class
As2MetricsService$_$tt__collectStats_closure6.class
As2MetricsService$_$tt__collectStats_closure7.class
As2MetricsService$_$tt__collectStats_closure8.class
As2MetricsService$_$tt__collectStats_closure9.class
As2MetricsService$_CountStatisticsLists_closure1.class
As2MetricsService$_collectStats_closure2.class
As2MetricsService.class
There are no subdirectories.
Also when I run the command chown -R adrt:adrt . inside the directory /opt/as2/app-server it executes without any issue.
Help me to understand what is happening here.
Help me to understand what is happening here.
You have just found a bug in Ansible which causes modules to fail when the names of files it processes contain $_ sequence.
The name is passed without escaping the $ character (or rather with an explicit conversion request os.path.expandvars(filename)) and the sequence $_ is processed as an built-in variable resolving to the path of the current process (/usr/bin/python in this case, as Ansible uses Python to run its modules).
In result the file name:
As2MetricsService$_$tt__collectStats_closure14.class
is interpreted as:
As2MetricsService/usr/bin/python$tt__collectStats_closure14.class
and the system throws an error that the file does not exist (which is true).
Until it is fixed, I guess you have to call chown with the command module
I have a class in Java which looks like this:
package com.charandeepmatta.keywords;
import org.robotframework.javalib.annotation.RobotKeyword;
import org.robotframework.javalib.annotation.RobotKeywords;
#RobotKeywords
public class SampleKeywords {
#RobotKeyword
public void printToErrorStream() {
System.err.println("!!! Hello from keyword developed in java ...");
}
}
And my test case looks like this
*** Settings ***
Library org.robotframework.javalib.library.AnnotationLibrary /**.class
*** Test Cases ***
Keyword defined in java class can print to error stream
Print To Error Stream
When I try to run it on RIDE it gives me the following error
[ ERROR ] Error in file 'C:\Users\BFerreira\git\robotframework-maven-project\src\main\robot\suite\OwnDevelopedKeywordTestCase.txt':
Importing test library 'org.robotframework.javalib.library.AnnotationLibrary' failed:
ImportError: No module named org.robotframework.javalib.library
Traceback (most recent call last):
None
PYTHONPATH:
C:\Python27\lib\site-packages\robot\libraries
C:\Python27\lib\site-packages
C:\Windows\system32\python27.zip
C:\Python27\DLLs
C:\Python27\lib
C:\Python27\lib\plat-win
C:\Python27\lib\lib-tk
C:\Python27
C:\Python27\lib\site-packages\wx-2.8-msw-unicode
.
C:\Users\user1\git\robotframework-maven-project\src\main\robot\suite
Everything is in the same classpath, can anyone help?
From the looks of your output, you are not executing with jybot/Jython. Jython is required to load Java classes in a Python interpreter. Here is what the output would look like if you were:
PYTHONPATH:
C:\apps\Python27\Lib\site-packages
C:\apps\jython2.5.3\Lib\site-packages\setuptools-0.6c11-py2.5.egg
C:\apps\jython2.5.3\Lib\site-packages\pip-1.2.1-py2.5.egg
C:\apps\jython2.5.3\Lib
__classpath__
__pyclasspath__/
C:\apps\jython2.5.3\Lib\site-packages
.
c:\ws\local
CLASSPATH:
C:\apps\jython2.5.3\jython.jar
A word of caution: if you run the Robot Framework jar (e.g. java -jar robotframework-2.5.3.jar ...) as some examples suggest, all classpath settings are ignored. You would have to put all your dependencies in one jar for that way to work...
I am new to pig. I wrote a UDF in pig and used it in my pig script. But it gives following error
ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve UserDefined.PartsOfSpeech using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Here is my UDF code
public String exec(Tuple input) throws IOException {
//my code here
}
Here is my pig script
REGISTER /home/bigdata/NetBeansProjects/UserDefined/dist/UserDefined.jar
a = load '/user/bigdata/json' using TextLoader() as (input:chararray);
b = foreach a GENERATE UserDefined.PartsOfSpeech(input);
In the above code UserDefined is my package name and PartsOfSpeech is my class name
The error message says that Pig cannot find UserDefined.PartsOfSpeech.
What package declaration does PartsOfSpeech.java have at the top of the file?
If the package declaration is package com.my.company; try this instead:
REGISTER /home/bigdata/NetBeansProjects/UserDefined/dist/UserDefined.jar
a = load '/user/bigdata/json' using TextLoader() as (input:chararray);
b = foreach a GENERATE com.my.company.PartsOfSpeech(input);
That is, replace UserDefined.PartsOfSpeech(input) with com.my.company.PartsOfSpeech(input) since the UDF is located in the package com.my.company.
Also, consider using the DEFINE keyword in your Pig script so you don't need to repeat com.my.company every time you use PartsOfSpeech.
DEFINE PartsOfSpeech UserDefined.dist.PartsOfSpeech();
REGISTER /home/bigdata/NetBeansProjects/UserDefined/dist/UserDefined.jar
a = load '/user/bigdata/json' using TextLoader() as (input:chararray);
b = foreach a GENERATE PartsOfSpeech(input);
There is more information about DEFINE in Chapter 5 of Alan Gates' Programming Pig: http://chimera.labs.oreilly.com/books/1234000001811/ch05.html#udf_define.
Here is an example of DEFINE from Gates' book:
--define.pig
register 'your_path_to_piggybank/piggybank.jar';
define reverse org.apache.pig.piggybank.evaluation.string.Reverse();
divs = load 'NYSE_dividends' as (exchange:chararray, symbol:chararray,
date:chararray, dividends:float);
backwards = foreach divs generate reverse(symbol);
Before compiling your UDF(java class) make sure you have mentioned package name properly. for example if you have mentioned package name-
package com.pig.udf;
It means you need to take care of directory in your linux box as well.
you can follow below mentioned steps to create jar -
Create directory using
mkdir -p com/pig/udf
Create your java class with package com.pig.udf
Compile your java source code using command
javac -cp /usr/lib/pig-0.12.0.2.0.6.0-76.jar YourClass.java
Then go to the directory where you want to create jar for now -
cd ../../..
Now create jar using below command
jar -cvf yourJarName.jar com/
Register the jar in your script using keyword "register" followed by path of the jar
Now use your jar with keyword com.pig.udf.YourJavaClassName
for your scenerio -
REGISTER /home/bigdata/NetBeansProjects/UserDefined/dist/UserDefined.jar
a = load '/user/bigdata/json' using TextLoader() as (input:chararray);
b = foreach a GENERATE com.pig.udf.PartsOfSpeech(input);
In continuation of dbpedia spotlight dataset, I followed the instructions for updating my dataset but got the following error:
INFO 2012-06-19 11:58:04,300 main [MergedOccurrencesContextSearcher] - Using index at: org.apache.lucene.store.MMapDirectory#/home/user_name/new/spotlight/index lockFactory=org.apache.lucene.store.NativeFSLockFactory#671381e7
Exception in thread "main" java.io.FileNotFoundException: /home/user_name/new/spotlight/index/segments_bp (No such file or directory)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)
at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:219)
at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:345)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:265)
at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:76)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:709)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:72)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:273)
at org.dbpedia.spotlight.lucene.search.BaseSearcher.<init>(BaseSearcher.java:67)
at org.dbpedia.spotlight.lucene.search.MergedOccurrencesContextSearcher.<init>(MergedOccurrencesContextSearcher.java:64)
at org.dbpedia.spotlight.model.SpotlightFactory.<init>(SpotlightFactory.scala:71)
at org.dbpedia.spotlight.web.rest.Server.main(Server.java:86)
I had been able to successfully use the spotter dictionary but couldn't use the index files.
Can you please help me?
Hard to help you without sufficient information. The message complains that a file is not there. Is the file there? Is the directory there?
Please paste the result of the command
ls -lah /home/user_name/new/spotlight/index