We Keep Coding

Error while trying to write on parquet file in datastage 11.7 (File_Connector_20,0: java.lang.NoClassDefFoundError: org.apache.hadoop.fs.FileSystem)

we have recently upgraded the DataStage from 9.1 to 11.7 on Server AIX 7.1 .
and i'm trying to use the new connector "File Connector" to write on parquet file. i created simple job takes from teradata as a source and write on the parquet file as a target.
Image of the job
but facing below error :
> File_Connector_20,0: java.lang.NoClassDefFoundError: org.apache.hadoop.fs.FileSystem
at java.lang.J9VMInternals.prepareClassImpl (J9VMInternals.java)
at java.lang.J9VMInternals.prepare (J9VMInternals.java: 304)
at java.lang.Class.getConstructor (Class.java: 594)
at com.ibm.iis.jis.utilities.dochandler.impl.OutputBuilder.<init> (OutputBuilder.java: 80)
at com.ibm.iis.jis.utilities.dochandler.impl.Registrar.getBuilder (Registrar.java: 340)
at com.ibm.iis.jis.utilities.dochandler.impl.Registrar.getBuilder (Registrar.java: 302)
at com.ibm.iis.cc.filesystem.FileSystem.getBuilder (FileSystem.java: 2586)
at com.ibm.iis.cc.filesystem.FileSystem.writeFile (FileSystem.java: 1063)
at com.ibm.iis.cc.filesystem.FileSystem.process (FileSystem.java: 935)
at com.ibm.is.cc.javastage.connector.CC_JavaAdapter.run (CC_JavaAdapter.java: 444)
i followed the steps in below link :
https://www.ibm.com/support/knowledgecenter/SSZJPZ_11.7.0/com.ibm.swg.im.iis.conn.s3.usage.doc/topics/amaze_file_formats.html
1- i uploaded the jar files into "/ds9/IBM/InformationServer/Server/DSComponents/jars"
2- added them to CLASSPATH in agent.sh then restarted the datastage.
3- i have set The environment variable CC_USE_LATEST_FILECC_JARS to the value parquet-1.9.0.jar:orc-2.1.jar.
i tried also to add the CLASSPATH as an environment variable in the job but not worked.
noting that i'm using Local in File System.
so any hint is appreciated as i'm searching a lot time ago.
Thanks in advance,

Which File System mode you are using ? If you are using Native HDFS as File System mode, then you would need to configure CLASSPATH to include some third party jars.
Perhaps these links should provide you with some guidance.
https://www.ibm.com/support/pages/node/301847
https://www.ibm.com/support/pages/steps-required-configure-file-connector-use-parquet-or-orc-file-format
Note : Based on the hadoop distribution and version you are using, the version of the jars could be different.
If the above information does not help in resolving the issue, then you may have to reach out to IBM Support to get this addressed.

TO use File Connector, there is no need to add CLASSPATH in agent.sh unless you want to import HDFS files from IMAM.
If your requirement is reading Parquet files, then set
$CC_USE_LATEST_FILECC_JARS=parquet-1.9.0.jar
$FILECC_PARQUET_AVRO_COMPAT_MODE=TRUE
If you are still seeing issue, then run job with $CC_MSG_LEVEL=2 and open IBM support case along with job design, FULL job log and Version.xml file from Engine tier.

Why won't Palantir QS 3.8 recognise its own XML export?

Hello Palantir community on StackOverflow - if you exist?
I'm having a problem with pXML and PXZ files on a QuickStart instance (see below for details). If I export from Graph (even if only a few relatively small Objects), then try and reimport that file, I get the error message,
Error: The file [file path & name] is not a valid .pxz file:
com.palantir.exceptionPalantirUserMessageException. Unexpected error
while validating PalantirXML; please see the log for details.
The log will then give some version of,
Value " with length = '0' is not faced-valid with respect to minLength
'1' for type #AnonType_namedataSource;.
Multiple Java error references will then follow (195, 131, 384, 318, etc).
So this seems to be an issue with Palantir writing an XML file badly, then not recognising it when you try to reimport.
The XML file itself seems ok - it's not very small, all the XML tags close off, etc. But clearly there's a value somewhere that's meant to be a positive, and it's not being populated in the correct way. The errant tag isn't obvious, if that's the case.
Weirdly, I can usually export a single Object (or maybe two or three) - but not if the Object is too complex (eg has lots of Properties).
I'm using an installation of Palantir Quick Start 3.8 (3.8.2.8.603030, Java Version: 1.6.0_30 Sun Microsystems Inc. - Java HotSpot(TM) 64-Bit Server VM build 20.5-b03 64-bit).
I've tried various configurations of Java updates (6.3 32 & 64, 7.25 32 & 64, no Java update (Pal 3.8 comes with 6.3).
The computer is an Intel, 2.7 Ghz with 16 GB of RAM, running Windows 7 (SP1), 64bt.
I tried disabling the AV (McAfee) and Windows firewall - no difference.
I'll leave it there for now - very grateful for any advice / suggestions.
R

That's an old version of Palantir! I worked on the code you're seeing errors with through many versions of Gotham. The problem is that the first step in the import process is validating the pXML against its .xsd file. While writing, the library makes sure the XML is syntactically valid, but doesn't verify it against the schema.
That error makes it sound like a DataSource is missing some value that's required by the schema. Exporting from a new Investigation may work, but this bug would need to be fixed by a Palantir developer.
You could also try it later version where it may be fixed already.

Talend Open Studio for Data Integration show error when creating metadata

I have a problem when using Talend Open Studio for Data Integration 6.0.
My environment is Windows 7 64-bit, Java 1.8. When I create Metadata from a delimited file or an Excel file it always shows an error like this in the preview section:
錯誤: 找不到或無法載入主要類別 b.shadowfileinputtodelimitedoutput_0_1.ShadowFileInputToDelimitedOutput
Preview error. Some settings must be changed.
Note: Preview errors are generally due to a wrong encoding setting.
org.talend.designer.runprocess.shadow.ShadowFilePreview.preview(ShadowFilePreview.java:90)
org.talend.metadata.managment.ui.utils.ShadowProcessHelper.getCsvArray(ShadowProcessHelper.java:383)
org.talend.repository.ui.wizards.metadata.connection.files.delimited.DelimitedFileStep2Form$PreviewProcessor.nonUIProcessInThread(DelimitedFileStep2Form.java:1309)
org.talend.commons.ui.swt.thread.SWTUIThreadProcessor$1.run(SWTUIThreadProcessor.java:74)
and when I try to run the job the same error occurs:
Starting job ds at 16:21 28/08/2015.
錯誤: 找不到或無法載入主要類別 b.ds_0_1.ds
Job ds ended at 16:21 28/08/2015. [exit code=1]
Can anyone help me solve this problem?

May be Talend has problem ton interpret asiatic character, have tried with only "occidendal" caracters in your delimited file or an Excel file (and saved in UTF-8) ?

Service will not start: error 1067: the process terminated unexpectedly

We have a custom service that we install with our application. The only problem is that after it is installed, it will not start, generating the error above. I have tried to diagnose what the problem is, but can't seem to find any useful information as to why it is quitting. I have tried the same service on a non "R2" 2008 server, and manual it worked fine.
service simple java file running using batch file. Deamon service.
Has anyone had any experience troubleshooting this type of problem, where there are so few clues?

Goto:
Registry-> HKEY_LOCAL‌​_MACHINE-> System-> Cur‌​rentControlSet-> Servi‌​ces.
Find the concerned service & delete it. Close regedit. Reboot the PC & Re-install the concerned service. Now the error should be gone.

This is a problem related permission.
Make sure that the current user has access to the folder which contains installation files.

I resolved the problem.This is for EAServer Windows Service
Resolution is -->
Open Regedit in Run prompt
Under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\EAServer
In parameters, give SERVERNAME entry as EAServer.
[It is sometime overwritten with Envirnoment variable : Path value]

This error message appears if the Windows service launcher has quit immediately after being started.
This problem usually happens because the license key has not been correctly deployed(license.txt file in the license folder).
If service is not strtign with correct key, just put incorrect key and try to start. Once started, place the correct key, it will work.

I had this error, I looked into a log file C:\...\mysql\data\VM-IIS-Server.err and found this
2016-06-07 17:56:07 160c InnoDB: Error: unable to create temporary file; errno: 2
2016-06-07 17:56:07 3392 [ERROR] Plugin 'InnoDB' init function returned error.
2016-06-07 17:56:07 3392 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2016-06-07 17:56:07 3392 [ERROR] Unknown/unsupported storage engine: InnoDB
2016-06-07 17:56:07 3392 [ERROR] Aborting
The first line says "unable to create temporary file", it sounds like "insufficient privileges", first I tried to give access to mysql folder for my current user - no effect, then after some wandering around I came up to control panel->Administration->Services->Right Clicked MysqlService->Properties->Log On, switched to "This account", entered my username/password, clicked OK, and it woked!

In my case the error 1067 was caused with a specific version of Tomcat 7.0.96 32-bit in combination with AdoptOpenJDK. Spent two hours on it, un-installing, re-installing and trying different Java settings but Tomcat would not start. See...
ASF Bugzilla – Bug 63625
seems to point at the issue though they refer to seeing a different error.
I tried 7.0.99 32-bit and it started straight away with the same AdoptOpenJDK 32-bit binary install.

I solved this issue using Monitor Tomcat application. I ran it and after a few seconds its icon appeared in my system tray. I right clicked on the icon and clicked the start button and after a few seconds Apache Tomcat started.

Where can i obtain symbols table for JavaService?

I have a crash dump (Windows) for JavaService (it's basically a wrapper to expose JBoss app server as a service).
I see this error on opening the dump file using windbg :
*** ERROR: Symbol file could not be found.
Where can i find the symbols table file?

Please send your crash dump along with your wrapper.conf and wrapper.log files to Tanuki Software support at support#tanukisoftware.com We would be happy to look into the problem for you.
Cheers,
Leif