Flink Job Execution Failed on run - java

I am trying to run the data artisans examples available at github. I read the tutorial and added the needed SDKs and downloaded the files for NYCFares and Rides. Whenever i am running the RideCount.java example i get a Job Execution Failed.
Here is the link to the git repo for the RideCount class file.
Github repo RideCount.java
here is the error

It appears that the nycTaxiRides.gz file has somehow been corrupted. The line that is shown in your screenshot should have these contents
914757,START,2013-01-03 10:00:57,1970-01-01 00:00:00,-73.982796,40.781418,-73.97155,40.761257,1,2013002770,2013017291
The code that parses the input is complaining that not all of the fields are present. I suggest you re-download the data file, being careful not to decompress or otherwise alter it. Chrome can do this, or wget, or curl, but some browsers will introduce problems.

Related

Sample IBM MFP Java adapter causes an error on import. FWLSE2302E Malformed ziped content

After downloading the sample java adapter from the MobileFirst Operations console I tried to import it and received the following error:
FWLSE3051E: Invalid payload. See additional messages for details.
FWLSE2302E: Can't read the content: failed to extract data from the zip format. Malformed ziped content.
Steps to reproduce:
Navigate to your-mfpserver:9080/mfpconsole/index.html#/downloads#samples and click on the "Hello World" link under "Java Adapter Samples".
Save javaAdapter.zip to disk.
Navigate to your-mfpserver:9080/mfpconsole/index.html#/mfp/browseAdapters
Click "Actions"->"Import Adapter"
Click browse, choose the javaAdapter.zip you saved to your disk.
Click deploy.
We're using IBM MobileFirst Platform Foundation Operations Console
Product version: 8.0.0.00-20170710-1834
I'm just trying to get the simplest adapter to work, so I thought I'd start with "Hello World", then I discovered this doesn't even work. I was developing java adapters 6 months ago just fine and using maven to deploy or create an .adapter file. In the meantime, we upgraded versions of MFP and now it appears only a .zip file is acceptable, but I can't find a working example.
How can I resolve this issue? I wasn't responsible for the configuration or installation of MFP and don't know too much about it.
You can not upload zipped adapter file,you should deploy the .adapter file .
Follow below steps
1.Unzip adapter and open in a terminal
2.run command 'mfpdev adapter build'
3.run 'mfpdev adapter deploy ' or you can deploy it via console Click "Actions"->"Import Adapter choose .adapter file .
Please see here for more details
https://mobilefirstplatform.ibmcloud.com/tutorials/en/foundation/8.0/quick-start/android/
The sample app needs to be built (a detail I overlooked when asking the above question) and a .adapter file needs to be created.
The larger issue is the Malformed ziped content.(sic) error.
Solve this by creating a directory named adapter containing a single directory named adapterPackage that contains only the .adapter file in it. Then zip the contents and you can utilize the MobileFirst Operations Console to import this zip (Actions->Import Adapter).
e.g. myAdapter.zip should contain adapter->adapterPakcage->myAdapter.adapter

Not found exception while doing svn update

I'm having the following situation:
A configuration file (config.cfg) that gets accessed a lot by
different processes.
Config.cfg is under version control - SVN.
I develop and test on a staging environment, when everything is working I go to the server and execute svn up on the config.cfg.
The problem is: During svn up I get an exception by the processes accessing config.cfg: "config.cfg" not found.
It seems that svn causes a short period where the file is beeing replaced and therefore not accessible for my processes.
Any input on how to solve this issue is very much appreciated.
As suggested by ThisSuitIsBlackNot the way to go is to use a semaphor file.
Another solution which just came to my mind is to cache the config file in the process. If it is not there the cached version of the config file is used. As "svn update" doesn't take very long the process will work with cached version until it needs to use the config file the next time.

org.postgresql.*; class not found error

I am using Java with eclipse.
I downloaded postgresql-9.1-901.jdbc4.jar and added it. (Properties-> Java Build Path-> Libraries -> Add External JARs...
My Database connection is working just fine I can create tables, execute queries ect.
Now I want to use the CopyManager. And therefore added to my code:
import org.postgresql.copy.CopyManager;
...
CopyManager cManager = new CopyManager(XXX);
unfortunately I get an error Message saying: "The source attachment does not contain the source for the file CopyManager.class"
Exception in thread "main" java.lang.NullPointerException
at org.postgresql.copy.CopyManager.<init>(CopyManager.java:46)
at Table.insert_data_in_tb(Table.java:73)
at DB.main(DB.java:69)
I'd be very thankfull for any help...
This is a message you get while running your code and clicking methods in debug mode. To correct the immediate problem (being able to view the source code) do this:
Download PostgreSQL 9.1 source
code
Copy the file to some appropriate folder and untar it
tar xvf postgresql-jdbc-9.1-901.src.tar.gz
Add the source folder to your debug configuration in Eclipse
(Run->Debug Configurations->select Java Application->Click Source
tab->Add->File System Directory->add ), where directory is
where you expanded the tarball too.
However, you really don't need to view the source code in order to determine whats going wrong in your program. The documentation for CopyManager is available in online Javadocs. The constructor for the object requires a valid connection, which I am guessing is null in your program.

JGit checkout vs `git checkout` problems

tl;dr JGit's checkout throws exceptions while command line git checkout works fine
I'm currently trying to use JGit to check out certain revisions from an online Git repository, in Java (for work). My current approach is (and I'm very new to Git, coming from a SVN background, so this may be wrong):
clone the repository to a temporary location on my hard drive
figure out which revision I want, (I have tried using the SHA-1 hash as well as the name of a branch)
checkout that revision
from there, I would be using the checked out files as inputs to a later part of the program.
checkout a different revision
use those files as inputs to another part of the program
Essentially, I want to be able to swap the contents of my temp folder with whichever revision. With the command line interface I've been able to do this with git checkout master and git checkout dylanbranch (where dylanbranch is a branch I made on my own clone with an arbitrarily chosen revision), but my Java code attempting to do the same thing fails:
Git git = Git.open(new File("checkout")); //checkout is the folder with .git
git.checkout().setName("master").call(); //succeeds
git.checkout().setName("dylanbranch").call(); //fails
And the exceptions printed to the console:
Exception in thread "main" org.eclipse.jgit.api.errors.JGitInternalException: Checkout conflict with files:
src/sizzle
test/qunit
at org.eclipse.jgit.api.CheckoutCommand.call(CheckoutCommand.java:211)
at com.avi.scm.git.BranchCheckout.main(BranchCheckout.java:30)
Caused by: org.eclipse.jgit.errors.CheckoutConflictException: Checkout conflict with files:
src/sizzle
test/qunit
at org.eclipse.jgit.dircache.DirCacheCheckout.checkout(DirCacheCheckout.java:387)
at org.eclipse.jgit.api.CheckoutCommand.call(CheckoutCommand.java:162)
... 1 more
I can verify that the files in question are marked as deleted and not staged for commit by using git status though I'm not sure why those changes are there, and they come back any time I switch back to the master branch. Despite that, I can still successfully change the working tree with command line Git.
So my question: Why won't JGit work for this application when command line git will?
Any other helpful information is appreciated- educate me :)
Update I've been testing with the jQuery repository, and have noticed a few more problems with JGit: When I am working with the "master" branch, git status temms me that I'm #On branch master and that there is nothing to commit (working directory clean), but using JGit's status command I see that test/qunit and src/sizzle are marked as Missing. JGit's reset seems to do nothing.
The two directories mentioned in the stack trace are Git submodules (test/qunit and src/sizzle) which is most likely the cause of the problem since JGit does not have full submodule support yet.
This may function differently in the 1.1 JGit release due out this month based on this commit.
You can read more about the current state of JGit submodule support here.
I know this doesn't directly answer your question, but I've also had problems with the Java implementations of Git. What worked best for me was to simply ditch the java implementations, and execute command-line calls to git from within the application. It may not be ideal, but it'll do exactly what you want since you'll fully control the command.
Simply call
Runtime.getRuntime().exec(...)
Link to Javadoc for Runtime class
I had a similar problem with Checkout. I think that the fact that you can switch branch with unstaged content in Git is actually a tolerance, not a feature.
JGit is globally not as tolerant as Git, so you should usually test many cases.
It seems it is not directly your problem (which is related to the submodules), but for more general cases you would want to commit your changes before using checkout to switch to another branch.
Note that the CheckoutCommand works perfectly for me to start a new branch from an old revision (you have to set the name of the branch and the start revision).

Why do I keep getting 'SVN: Working Copy XXXX locked; try performing 'cleanup'?

If you have worked with SVN tools in Eclipse (Subversion, subversive) before, then you are likely familiar with the 'working copy 'XXX' locked..." error.
I found a very useful post with a workaround for this problem at: Working copy XXX locked and cleanup failed in SVN
As great as the workaround is, it is a pain to do it over and over again. Does anyone know why I keep getting this error and what steps I could take to prevent it?
Context: I am creating an Eclipse plugin that involves listening for SVN events, so in testing this plugin, I am constantly opening and closing the workspace. I usually do 1 or 2 commits each time I open the workspace. Every so often the commit will fail and I get the 'working copy locked' error. I would love for this error to not happen anymore, so any advice is appreciated.
Thanks!
Select the project
Right click on the selected Project
Team -> Cleanup
Problem Solved.
Note: The Above steps will work only Eclipse(Indigo package)
Generally a .lock file is created and it decides lock/unlock state checking the existince of this file. I think if you delete this .lock file only, then the problem will go away.
I've had a lot of issues with SVN before and one thing that has definitely caused me problems is modifying files outside of Eclipse or manually deleting folders (which contains the .svn folders), that has probably given me the most trouble.
edit
You should also be careful not to interrupt SVN operations, though sometimes a bug may occur and this could cause the .lock file to not be removed, and hence your error.
Make sure you exactly cleanup what the console says. For example if a subfolder (a package) is locked:
svn: E155004: Commit failed (details follow):
svn: E155004: Working copy 'C:\Users\laura\workspace\tparser\src\de\test\order' locked
svn: E155004: 'C:\Users\laura\workspace\tparser\src\de\test\order' is already locked.
cleanup C:/Users/liparulol/workspace/tparser/src/de/mc/etn/parsers/order
Then you need to cleanup the specified folder and not the whole project. If you are in eclipse right click on the package, not on the project folder and execute the clean up.
After more exploration and testing, it appears that this issue was being caused by debugging the plugin and using breakpoints. SVN/Subclipse apparently didn't like having breakpoints midway through their execution and as a result this lock files were being created. As soon as I started just running the plugin, this issue disappeared.
This will happen when something went wrong in one of your folders in you project.
You need to find out the exact folder that locked and execute svn cleanup under the specific folder.
You can solve this as follows:
run svn commit command to find out which folder went wrong.
change directory to that folder and run svn cleanup. Then it's done.
The following should unlock a locked working copy (tested on svn client version 1.6.11 and elipse version: Mars.2 Release (4.5.2))
step 1: (go to working copy directory) $cd working_copy_dir
step 2: (connect to svn sqlite database) $sqlite3 .svn/wc.db
step 3: (delete all records from table WC_LOCK) sqlite> delete from WC_LOCK;
step 4: (disconnect from sqlite 3 database) sqlite>ctrl + d
step 5: (from eclipse) right click on your working copy, then click Team -> Refresh/Cleanup
I had the same problem using the com.xxx.service.model package.
To fix it, I first made a backup of the code changes in the model package. Then deleted model package and synchronized with the repository. It will show incoming the entire folder/package. Then updated my code.
Finally, paste the old code commit to the SVN Repository. It works fine.
This happened to me when I copied a directory from another subversion project and tried to commit. The soluction was to delete the .svn director inside the directory I wanted to commit.
This type of problem can happen when you delete/move files around - in essence making changes to your directory structure. Subversion only checks for changes made in files already added to subversion, not changes made to the directory structure. Instead of using your OS's copy etc commands rather use svn copy etc. Please see http://svnbook.red-bean.com/en/1.7/svn.tour.cycle.html
Further, upon committing changes svn first stores a "summary" of changes in a todo list. Upon performing the svn operations in this todo list it locks the file to prevent other changes while these svn actions are performed. If the svn action is interrupted midway, say by a crash, the file will remain locked until svn could complete the actions in the todo list. This can be "reactivated" by using the svn cleanup command. Please see http://svnbook.red-bean.com/en/1.7/svn.tour.cleanup.html
Solution:
Step1: Have to remove “lock” file which present under “.svn” hidden file.
Step2: In case if there is no “lock” file then you would see “we.db” you have to open this database and need to delete content alone from the following tables
– lock
– wc_lock
Step3: Clean your project
Step4: Try to commit now.
Step5: Done.

Categories