We Keep Coding

Installing parquet-tools

I am trying to install parquet tools on a FreeBSD machine.
I cloned this repo: git clone https://github.com/apache/parquet-mr
Then I did cd parquet-mr/parquet-tools
Then I did `mvn clean package -Plocal
As specified here: https://github.com/apache/parquet-mr/tree/master/parquet-tools
This is what I got:
Why is this dependency error here? How do I get around it?

On Ubuntu 20, I install via pip:
python3 -m pip install parquet-tools
Haven't tried on FreeBSD but I'd imagine it would also work. See related answer for a caveat on using pip on FreeBSD.
And you can view a file with:
parquet-tools show filename.parquet

I know the question specifies FreeBSD, but if you're on mac, you can do
brew install parquet-tools

parquet-tools is just one module of parquet-mr. It depends on some of the other modules.
When you build from a source version that corresponds to a release, those other modules will be available to Maven, because release artifacts are published as a part of the release process.
However, when building from a snapshot version, you have to make those dependencies available yourself. There are two ways to do so:
Option 1: Build and install all modules of the parent directory:
git clone https://github.com/apache/parquet-mr
cd parquet-mr
mvn install -Plocal
This will put the snapshot artifacts in your local ~/.m2 directory. Subsequently, you can (re)build just parquet-tools like you initially tried, because now the snapshot artifacts will already be available from ~/.m2.
Option 2: Build the parquet-mr modules from the parent directory, while asking Maven to build needed modules as well along the way:
git clone https://github.com/apache/parquet-mr
cd parquet-mr
mvn package -pl parquet-tools -am -Plocal
Option 1 will build more projects than option 2, so if you only need parquet-tools, you are better off with the latter. Please note though that probably both will require installation of a thrift compiler.

Parquet tools- A utility that can be leveraged to read parquet files. Yuu can clone it from Github and run some maven command.
1. git clone https://github.com/Parquet/parquet-mr.git
2. cd parquet-mr/parquet-tools/
3. mvn clean package -Plocal
OR You can download stable release & built from local.
Downloading stable Parquet release.
https://github.com/apache/parquet-mr/archive/apache-parquet-1.8.2.tar.gz
2. Maven local install.
D:\parquet>cd parquet-tools && mvn clean package -Plocal
3. Test it (paste a parquet file under target directory):
D:\parquet\parquet-tools\target>java -jar parquet-tools-1.8.2.jar schema out.parquet
(where out.parquet is my parquet file under target directory)
// Read parquet file
D:\parquet\parquet-tools\target>java -jar parquet-tools-1.6.0.jar cat out.parquet
// Read few lines in parquet file
D:\parquet\parquet-tools\target>java -jar parquet-tools-1.6.0.jar head -n5 out.parquet

Some answers have broken link for the jar download, but you can get it from
maven central
However... this jar and others like it are built so that the hadoop dependencies are "provided" and if you build from source, you'll get that default. So you need to set -Dhadoop.scope=compile when you build, or the result will only work when run on a hadoop node using the "hadoop ..." command.
To make matters worse, this tool apparently disables System.out and System.err so that exceptions that cause main() fails are never printed and you'll be left wondering what happened.
I also found that the default settings for the maven-license-plugin caused it to fail the build when files showed up that it didn't expect (e.g. nbactions.xml if you use netbeans).

Solutions to "old code syndrome" ?

I don't know if this is going to get downvoted to oblivion as I can't provide a great deal of specific information. But in a nutshell. I have a maven project which has dependencies of other projects. One of the files coming in seems to be running on old code somehow. When de-bugging through. It stops on blank lines etc. It's like it's cached an old jar file somewhere or something. I have tried...
Deleting the contents my .m2 repository folder
Deleting all my temp files and anything that might reference the project from my tomcat directory.
reindexing the local repository
mvn clean install
mvn compile -pl service-module -am
mvn dependency:purge-local-repository
mvn dependency:purge-local-repository -DreResolve=false
mvn dependency:purge-local-repository -DactTransitively=false -DreResolve=false
setting the updatePolicy in settings.xml to 'always'
I have been at it for hours now and have made absolutely no progress.
Does anybody know any other solutions to this sort of problem?

Check the transitive dependency. Check for unexpected overrides or different packaging of the same library (e.g. something like XX-all and XX-part at the same time.)
Use an IDE (that understands Maven) to look up the class name and see how many implementations are found.
Check the actual command line of the JVM, the JRE library folder and the Tomcat library folder for rogue stuff.
Use this trick to find out the actual path the class is loaded from during debugging. (Use debugger facilities like value watch or expression window.)

How do I know which project is requesting a specific jar from Maven

I'm using Eclipse and recently upgraded all my projects to use the latest version of a library.
However in the Maven repository I can still see the old version of the library.
I've deleted manually the old library from the Maven repository, but it keeps coming back.
I am sure all the projects in Eclipse point to the new version: I've checked all my pom.xml, I've used the "Dependency Hierarchy" tool, etc.
Is there a way to know which project is telling Maven to download the old version of the library?
Many thanks!

You can use the Maven dependency plugin's tree goal:
mvn dependency:tree
and filter using the includes option which uses the pattern [groupId]:[artifactId]:[type]:[version].

Re: "and I have many". Perform the following in the topmost directory:
find . -name "pom.xml" -type f -exec mvn dependency:tree -f {} ';' | grep '^\[.*\] [-+\\\|].*'
Syntax details may vary from Bash to Bash.
Hint: Try it in a bottommost project directory first to ensure that it runs properly as intended. Since you have many projects it may take a while to finish and to recognize possible errors only then.

You can use below command to get a tree of all dependencies and then find out where the specific artifact is coming from.
You can pipe with grep to show only the related ones if you you are on linux/unix based os.
mvn dependency:tree

Thanks guys, appreciated, but it certainly is not an easy way. It looks like you have to do project by project (and I have many). Plus most of my pom reference poms in other folders and it's not able to process that either.

Setting groovysh classpath from a pom

I have a java project (not using groovy) but I'd like to interactively play with my java classes within groovysh. Is there an easy way to use the pom from my project to set the classpath of groovysh?

Just for the record, I've found a way to do it which I consider much more elegant when the project uses the gmaven-plugin: run mvn groovy:shell and you're ready to issue commands with the same classpath available to groovysh as the one available to the project in question!

MOP might help:
Scripting Goodies
Other times, you just need need the
CLASSPATH so you can use it in a
manually crafted script your running.
Try this.
mop classpath org.apache.camel:camel-example-pojo-messaging
Update: The above command can be used to output the classpath of an existing maven artefact. For example:
$ ./mop classpath org.hibernate:hibernate-core:3.3.2.GA
Prints the following output:
/home/pascal/opt/mop/repository/org/hibernate/hibernate-core/3.3.2.GA/hibernate-core-3.3.2.GA.jar:/home/pascal/opt/mop/repository/antlr/antlr/2.7.6/antlr-2.7.6.jar:/home/pascal/opt/mop/repository/commons-collections/commons-collections/3.1/commons-collections-3.1.jar:/home/pascal/opt/mop/repository/dom4j/dom4j/1.6.1/dom4j-1.6.1.jar:/home/pascal/opt/mop/repository/xml-apis/xml-apis/1.0.b2/xml-apis-1.0.b2.jar:/home/pascal/opt/mop/repository/javax/transaction/jta/1.1/jta-1.1.jar:/home/pascal/opt/mop/repository/org/slf4j/slf4j-api/1.5.8/slf4j-api-1.5.8.jar
That can be used somewhere else. As I said, it might help but I'm not 100% sure it will suit your needs (it seems the artifact needs to be deployed in a remote repo).

You can add them to classpath with -cp e.g.
groovysh -cp some.jar:another.jar

Why do my tests fail to run when migrating from maven2 to buildr?

I have a straightforward maven2 java project (JMS relaying system). After we released the first version, we found that we spent more time configuring maven than actually coding.
For the next release we wanted to clean up the build process and someone suggested migrating to builder. So I was tasked with doing just that.
I setup buildr (1.3.4) according to the documentation on their website. And then from the root of the project I typed the buildr command and then informed buildr to create the build file based upon my pom.xml. That processed fine and compiled all the code. All was gravy until buildr started running the tests. Here is the ouput:
Test framework error: taskdef class org.apache.tools.ant.taskdefs.optional.junit.JUnitTask cannot be found
Obviously the class specified isn't in my classpath. However, the buildr documentation says that all the required items needed for basic testing are included. Their documentation doesn't say that they need any specific libraries for ant or a version of ant. Although I do have ant 1.7.0 installed (not included in my classpath however).
Has anyone seen this before?
Update
I located the infamous ant-optional jar on the maven repository. Including that in my test.with options did not resolve the issue.
Running the buildr command with --trace gives this extra information...
Tests failed!
/pathtoruby/buildr-1.3.4/lib/buildr/core/test.rb:455:in `run_tests'
/pathtoruby/buildr-1.3.4/lib/buildr/core/test.rb:199:in `initialize'

Found the issue... Apparently there is an ant-junit.jar that is needed but for whatever reason in my local repository it was owned by root and not my local user account (OSX system). So it wasn't accessible to buildr. I deleted the items from my local repository and reran buildr (it downloaded the needed items).
Update
Also this caused a few other issues. It seems that a few other items in my local repository had strange permissions. I ended up just archiving my repository and letting maven reconstruct it. This resolved all my issues. I now have a nice build file that is 25 lines of code compared to my previous pom.xml file that was over 100 lines.

You get that error because JUnitTask isn't on the classpath. I'm not very familiar with Buildr so can't say if it is required for you to specify the JUnit jars or not, but if Buildr uses the system classpath, try adding JUnit to it and see what happens.
Once you've confirmed your builds will run with JUnit hacked in to the classpath, you can then try varying your configuration until it runs as you expected, or leave it as is.
Can you post the reference to the relevant part of the documentation? I didn't see anything (in my very brief reading of the site) that says required items are included.
Is it possible that you've not downloaded all the gems? If you run "gem update --system" to update Ruby, then "gem update buildr" you can ensure that the required dependencies have all been installed.