We Keep Coding

Get the whole index of Maven Central [duplicate]

I have downloaded the indexes generated for Maven Central from http://mirrors.ibiblio.org/pub/mirrors/maven2/dot-index/nexus-maven-repository-index.gz
I would like to list the artifacts information from these index files (groupId, artifactId, version for example). I have read that there is a high level API for that. It seems that I have to use the following maven dependency. However, I don't know what is the entry point to use (which class?) and how to use it to access those files:
<dependency>
<groupId>org.sonatype.nexus</groupId>
<artifactId>nexus-indexer</artifactId>
<version>3.0.4</version>
</dependency>

Take a peek at https://github.com/cstamas/maven-indexer-examples project.
In short: you dont need to download the GZ/ZIP (new/legacy format) manually, it will indexer take care of doing it for you (moreover, it will handle incremental updates for you too, if possible).
GZ is the "new" format, independent of Lucene index-format (hence, independent of Lucene version) containing data only, while the ZIP is "old" format, which is actually plain Lucene 2.4.x index zipped up. No data content change happens currently, but is planned in future.
As I said, there is no data content difference between two, but some fields (like you noticed) are Indexed but not stored on index, hence, if you consume the ZIP format, you will have them searchable, but not retrievable.

The https://github.com/cstamas/maven-indexer-examples is obsolete. And the build fails (tests do not pass).
The Nexus Indexer has moved along and included the examples too:
https://github.com/apache/maven-indexer/tree/master/indexer-examples
That builds, and the code works.
Here is a simplified version if you want to roll your own:
Maven:
<dependencies>
<dependency>
<groupId>org.apache.maven.indexer</groupId>
<artifactId>indexer-core</artifactId>
<version>6.0-SNAPSHOT</version>
<scope>compile</scope>
</dependency>
<!-- For ResourceFetcher implementation, if used -->
<dependency>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-http-lightweight</artifactId>
<version>2.3</version>
<scope>compile</scope>
</dependency>
<!-- Runtime: DI, but using Plexus Shim as we use Wagon -->
<dependency>
<groupId>org.eclipse.sisu</groupId>
<artifactId>org.eclipse.sisu.plexus</artifactId>
<version>0.2.1</version>
</dependency>
<dependency>
<groupId>org.sonatype.sisu</groupId>
<artifactId>sisu-guice</artifactId>
<version>3.2.4</version>
</dependency>
Java:
public IndexToGavMappingConverter(File dataDir, String id, String url)
throws PlexusContainerException, ComponentLookupException, IOException
{
this.dataDir = dataDir;
// Create Plexus container, the Maven default IoC container.
final DefaultContainerConfiguration config = new DefaultContainerConfiguration();
config.setClassPathScanning( PlexusConstants.SCANNING_INDEX );
this.plexusContainer = new DefaultPlexusContainer(config);
// Lookup the indexer components from plexus.
this.indexer = plexusContainer.lookup( Indexer.class );
this.indexUpdater = plexusContainer.lookup( IndexUpdater.class );
// Lookup wagon used to remotely fetch index.
this.httpWagon = plexusContainer.lookup( Wagon.class, "http" );
// Files where local cache is (if any) and Lucene Index should be located
this.centralLocalCache = new File( this.dataDir, id + "-cache" );
this.centralIndexDir = new File( this.dataDir, id + "-index" );
// Creators we want to use (search for fields it defines).
// See https://maven.apache.org/maven-indexer/indexer-core/apidocs/index.html?constant-values.html
List<IndexCreator> indexers = new ArrayList();
// https://maven.apache.org/maven-indexer/apidocs/org/apache/maven/index/creator/MinimalArtifactInfoIndexCreator.html
indexers.add( plexusContainer.lookup( IndexCreator.class, "min" ) );
// https://maven.apache.org/maven-indexer/apidocs/org/apache/maven/index/creator/JarFileContentsIndexCreator.html
//indexers.add( plexusContainer.lookup( IndexCreator.class, "jarContent" ) );
// https://maven.apache.org/maven-indexer/apidocs/org/apache/maven/index/creator/MavenPluginArtifactInfoIndexCreator.html
//indexers.add( plexusContainer.lookup( IndexCreator.class, "maven-plugin" ) );
// Create context for central repository index.
this.centralContext = this.indexer.createIndexingContext(
id + "Context", id, this.centralLocalCache, this.centralIndexDir,
url, null, true, true, indexers );
}
final IndexSearcher searcher = this.centralContext.acquireIndexSearcher();
try
{
final IndexReader ir = searcher.getIndexReader();
Bits liveDocs = MultiFields.getLiveDocs(ir);
for ( int i = 0; i < ir.maxDoc(); i++ )
{
if ( liveDocs == null || liveDocs.get( i ) )
{
final Document doc = ir.document( i );
final ArtifactInfo ai = IndexUtils.constructArtifactInfo( doc, this.centralContext );
if (ai == null)
continue;
if (ai.getSha1() == null)
continue;
if (ai.getSha1().length() != 40)
continue;
if ("javadoc".equals(ai.getClassifier()))
continue;
if ("sources".equals(ai.getClassifier()))
continue;
out.append(StringUtils.lowerCase(ai.getSha1())).append(' ');
out.append(ai.getGroupId()).append(":");
out.append(ai.getArtifactId()).append(":");
out.append(ai.getVersion()).append(":");
out.append(StringUtils.defaultString(ai.getClassifier()));
out.append('\n');
}
}
}
finally
{
this.centralContext.releaseIndexSearcher( searcher );
}
We use this in the Windup project - JBoss migration tool.

The legacy zip index is a simple lucene index. I was able to open it with Luke
and write some simple lucene code to dump out the headers of interest ("u" in this case)
import org.apache.lucene.document.Document;
import org.apache.lucene.search.IndexSearcher;
public class Dumper {
public static void main(String[] args) throws Exception {
IndexSearcher searcher = new IndexSearcher("c:/PROJECTS/Test/index");
for (int i = 0; i < searcher.maxDoc(); i++) {
Document doc = searcher.doc(i);
String metadata = doc.get("u");
if (metadata != null) {
System.out.println(metadata);
}
}
}
}
Sample output ...
org.ioke|ioke-lang-lib|P-0.4.0-p11|NA
org.jboss.weld.archetypes|jboss-javaee6-webapp|1.0.1.CR2|sources|jar
org.jboss.weld.archetypes|jboss-javaee6-webapp|1.0.1.CR2|NA
org.nutz|nutz|1.b.37|javadoc|jar
org.nutz|nutz|1.b.37|sources|jar
org.nutz|nutz|1.b.37|NA
org.openengsb.wrapped|com.google.gdata|1.41.5.w1|NA
org.openengsb.wrapped|openengsb-wrapped-parent|6|NA
There may be better ways to achieve this though ...

For the records, there is now a tool to extract and export maven indexes as text files: the Maven index exporter. It's available as a Docker image and no code is required.
It basically downloads all .gz index files, extracts the indexes using maven-indexer cli and exports them to a text file with clue. It has been tested on Maven Central and works on many other Maven repositories.

NoClassDefFoundError for Birt Class EngineException

Has anyone added the Birt Library to a project's build path successfully?
I installed Birt in Eclipse using Help-->Install New Software and copy paste the link for Birt 4.2.2 from the official website. Birt was downloaded from the internet and a new perspective appeared, namely Report Design. Apart from it no Library, no nothing. So I designed my report and started writing Java code in order to do a PDF export of my report, as I found on the internet. I wrote the following:
// Export Birt report
String format = HTMLRenderOption.OUTPUT_FORMAT_PDF;
EngineConfig config = new EngineConfig( );
config.setEngineHome( "C:\\Tools\\Eclipse\\plugins\\org.eclipse.birt.report.viewer_4.2.2.v201302041142\\birt" );
HTMLEmitterConfig hc = new HTMLEmitterConfig( );
HTMLCompleteImageHandler imageHandler = new HTMLCompleteImageHandler( );
hc.setImageHandler( imageHandler );
config.setEmitterConfiguration( HTMLRenderOption.OUTPUT_FORMAT_HTML, hc );
ReportEngine engine = new ReportEngine( config );
IReportRunnable report = null;
String reportFilepath = "C:/Workspace/reports/new_report.rptdesign";
try {
report = engine.openReportDesign( reportFilepath );
}
catch ( EngineException e ) {
System.err.println( "Report " + reportFilepath + " not found!\n" );
engine.destroy( );
return;
}
IRunAndRenderTask task = engine.createRunAndRenderTask( report );
HTMLRenderOption options = new HTMLRenderOption( );
options.setOutputFormat( format );
options.setOutputFileName( "C:/Workspace/reports/output.pdf" );
task.setRenderOption( options );
task.setParameterValues( parametersMap );
try {
task.run( );
}
catch ( EngineException e1 ) {
System.err.println( "Report " + reportFilepath + " run failed.\n" );
System.err.println( e1.toString( ) );
}
engine.destroy( );
return;
Of course no Birt Library is set in my Build Path, so all Birt objects were red. I manually added all Birt jars that existed in Eclipse's plugins folder and all the reds disappeared. It seemed that everything was going great. When I run it I get:
java.lang.NoClassDefFoundError: org/eclipse/birt/report/engine/api/EngineException
while the org.eclipse.birt.report.engine.api package exists in my Build Path
and the EngineException.class exist in the package. I am feeling that there are jar files that I am missing.
I have searched for a solution in Birt home page and found nothing. Most tutorials have instructions regarding how to build a report. But nothing regarding how to get the library up and running with Java.
Is there a standard or automatic or official way to add Birt Library to Eclipse? It is made by Eclipse. It shouldn't be so difficult. Any help will be appreciated.

Have you read this page -> http://wiki.eclipse.org/Servlet_Example_(BIRT)_2.1
Maybe you can find some hints there.

Java ProgramCall.run hangs

Busy trying to Call RPG function from Java and got this example from JamesA. But now I am having trouble, here is my code:
AS400 system = new AS400("MachineName");
ProgramCall program = new ProgramCall(system);
try
{
// Initialise the name of the program to run.
String programName = "/QSYS.LIB/LIBNAME.LIB/FUNNAME.PGM";
// Set up the 3 parameters.
ProgramParameter[] parameterList = new ProgramParameter[2];
// First parameter is to input a name.
AS400Text OperationsItemId = new AS400Text(20);
parameterList[0] = new ProgramParameter(OperationsItemId.toBytes("TestID"));
AS400Text CaseMarkingValue = new AS400Text(20);
parameterList[1] = new ProgramParameter(CaseMarkingValue.toBytes("TestData"));
// Set the program name and parameter list.
program.setProgram(programName, parameterList);
// Run the program.
if (program.run() != true)
{
// Report failure.
System.out.println("Program failed!");
// Show the messages.
AS400Message[] messagelist = program.getMessageList();
for (int i = 0; i < messagelist.length; ++i)
{
// Show each message.
System.out.println(messagelist[i]);
}
}
// Else no error, get output data.
else
{
AS400Text text = new AS400Text(50);
System.out.println(text.toObject(parameterList[1].getOutputData()));
System.out.println(text.toObject(parameterList[2].getOutputData()));
}
}
catch (Exception e)
{
//System.out.println("Program " + program.getProgram() + " issued an exception!");
e.printStackTrace();
}
// Done with the system.
system.disconnectAllServices();
The application Hangs at this lineif (program.run() != true), and I wait for about 10 minutes and then I terminate the application.
Any idea what I am doing wrong?
Edit
Here is the message on the job log:
Client request - run program QSYS/QWCRTVCA.
Client request - run program LIBNAME/FUNNAME.
File P6CASEL2 in library *LIBL not found or inline data file missing.
Error message CPF4101 appeared during OPEN.
Cannot resolve to object YOBPSSR. Type and Subtype X'0201' Authority
FUNNAME insert a row into table P6CASEPF through a view called P6CASEL2. P6CASEL2 is in a different library lets say LIBNAME2. Is there away to maybe set the JobDescription?

Are you sure FUNNAME.PGM is terminating and not hung with a MSGW? Check QSYSOPR for any messages.
Class ProgramCall:
NOTE: When the program runs within the host server job, the library list will be the initial library list specified in the job description in the user profile.

So I saw that my problem is that my library list is not setup, and for some reason, the user we are using, does not have a Job Description. So to over come this I added the following code before calling the program.run()
CommandCall command = new CommandCall(system);
command.run("ADDLIBLE LIB(LIBNAME)");
command.run("ADDLIBLE LIB(LIBNAME2)");
This simply add this LIBNAME, and LIBNAME2 to the user's library list.

Oh yes, the problem is Library list not set ... take a look at this discussion on Midrange.com, there are different work-around ...
http://archive.midrange.com/java400-l/200909/msg00032.html
...
Depe

Printing with Attributes(Tray Control, Duplex, etc...) using javax.print library

I've been trying for some time to determine a way to use the standard Java Print library to print files - specifically, PDF documents - with certain attributes - specifically, to certain trays or using duplex.
There exists plenty of documentation on how this should be done, and indeed, I've researched and tried these methods. The typical way is something like this:
public static void main (String [] args) {
try {
PrintService[] pservices = PrintServiceLookup.lookupPrintServices(null, null);
//Acquire Printer
PrintService printer = null;
for (PrintService serv: pservices) {
System.out.println(serv.toString());
if (serv.getName().equals("PRINTER_NAME_BLAH")) {
printer = serv;
}
}
if (printer != null) {
System.out.println("Found!");
//Open File
FileInputStream fis = new FileInputStream("FILENAME_BLAH_BLAH.pdf");
//Create Doc out of file, autosense filetype
Doc pdfDoc = new SimpleDoc(fis, DocFlavor.INPUT_STREAM.AUTOSENSE, null);
//Create job for printer
DocPrintJob printJob = printer.createPrintJob();
//Create AttributeSet
PrintRequestAttributeSet pset = new HashPrintRequestAttributeSet();
//Add MediaTray to AttributeSet
pset.add(MediaTray.TOP);
//Add Duplex Option to AttributeSet
pset.add(Sides.DUPLEX);
//Print using Doc and Attributes
printJob.print(pdfDoc, pset);
//Close File
fis.close();
}
}
catch (Throwable t) {
t.printStackTrace();
}
}
In short, you do the following
Find the Printer
Create a PrinterJob
Create an AttributeSet
Add Attributes to the AttributeSet, such as Tray and Duplex
Call print on the printer job using the AttributeSet
The problem here is that, despite being the documented way of doing this, as well as what I've found from several tutorials, this method... doesn't work. Now keep in mind, I know that doesn't sound very descript, but hear me out. I don't say that lightly...
The official documentation for PrinterJob actually mentions that the AttributeSet is ignored in the default implementation. Source code seen here shows this to be true - the attributes are passed in and ignored entirely.
So apparently, you need some sort of extended version of the class, which is possibly based on the specific printers and their capabilities? I attempted to write some test code that would tell me such capabilities - we have a large variety of printers set up at the office, large or small, simple or full of bells and whistles - not to mention several drivers on my computer just for pseudo-printer drivers that just create documents and simulate printers without going to any sort of hardware. The test code is as follows:
public static void main (String [] args) {
PrintService[] pservices = PrintServiceLookup.lookupPrintServices(null, null);
for (PrintService serv: pservices) {
System.out.println(serv.toString());
printFunctionality(serv, "Trays", MediaTray.class);
printFunctionality(serv, "Copies", Copies.class);
printFunctionality(serv, "Print Quality", PrintQuality.class);
printFunctionality(serv, "Color", ColorSupported.class);
printFunctionality(serv, "Media Size", MediaSize.class);
printFunctionality(serv, "Accepting Jobs", PrinterIsAcceptingJobs.class);
}
}
private static void printFunctionality(PrintService serv, String attrName, Class<? extends Attribute> attr) {
boolean isSupported = serv.isAttributeCategorySupported(attr);
System.out.println(" " + attrName + ": " + (isSupported ? "Y" : "N"));
}
The results I found were that every printer, without exception, returned that "copies" were supported, and all other attributes were not. Furthermore, every printer's capabilities were identical, regardless of how implausible that would seem.
The inevitable question is multi-layered: How does one send in attributes in a way that they are registered? Additionally, how does one properly detect the capabilities of a printer? Indeed, is the PrinterJob class actually extended in a usable way at all, or are the Attributes always ignored?
Examples I've found throughout The Internet seem to suggest to me that the answer to the latter question is "No, they are always ignored", which seems ridiculous to me (but increasingly more believable as I sift through hundreds of pages). Is this code that Sun simply set up but never got working to a completed state? If so, are there any alternatives?

The problem is that the the Java print API is a bridge between worlds. Printer manufacturers don't release drivers for the JVM. They release drivers for Windows, Macintosh, and maybe someone has a a driver for a given printer that works on one or more *nix platforms.
Along you come with some Java code running inside a JVM on some host system. When you start querying the printer features, you aren't talking to the printers -- you are talking to a bridge class in java.awt.print that hook into the JVM, which hooks to the host operating system, which hooks into whatever particular driver was installed for a given printer. So there are several places where this can fall apart... The particular JVM you are on may or may not fully implement the API for querying printer features, let alone passing those parameters along for a given job.
A few suggestions:
look into the javax.print classes as an alternative to
java.awt.print -- I've had more luck printing from there.
try using alternative print drivers for your printers -- you can define
multiple named connections to a given printer, each with a different
driver. If you've got a manufacturer provided driver, try a more generic driver, if you've got a generic driver, try to install a more specific one.
run your code under alternate JVM implementations for your platform

So, we inevitably found a way to print to different trays and with different settings, but not directly. We found it impossible to send attributes via the printJob.print method, and that much hasn't changed. However, we were able to set the name of the print job, then intercept the print job with a low-level Perl script, parse the name, and set the tray and duplex settings there. It's an extreme hack, but it works. It still remains true that Java Printer Attributes do not work, and you will need to find another way if you want to set them.

We had similar requirement to print PDF's and wanted to send some pages to Specific tray and also wanted the document to be stapled.
We used Java code + ghost script combination
First convert PDF to ghost script and then add PJL (Print job language) commands to ghost script file to select trays and staple the documents.
Then send that edited ghost script file to printer.
Here is complete example written in Java
http://reddymails.blogspot.com/2014/07/how-to-print-documents-using-java-how.html
-Ram

Here's what it looks like in javafx Tray's may vary and it will also print out all trays that are available just change the tray name
private void printImage(Node node) {
PrinterJob job = PrinterJob.createPrinterJob();
if (job != null) {
JobSettings js = job.getJobSettings();
PaperSource papersource = js.getPaperSource();
System.out.println("PaperSource=" + papersource);
PrinterAttributes pa = printer.getPrinterAttributes();
Set<PaperSource> s = pa.getSupportedPaperSources();
System.out.println("# of papersources=" + s.size());
if (s != null) {
for (PaperSource newPaperSource : s) {
System.out.println("newpapersource= " + newPaperSource);
//Here is where you would put the tray name that is appropriate
//in the contains section
if(newPaperSource.toString().contains("Tray 2"))
js.setPaperSource(newPaperSource);
}
}
job.getJobSettings().setJobName("Whatever");
ObjectProperty<PaperSource> sources = job.getJobSettings().paperSourceProperty();
System.out.println(sources.toString());
boolean success = job.printPage(node);
if (success) {
System.out.println("PRINTING FINISHED");
job.endJob();
//Stage mainStage = (Stage) root.getScene().getWindow();
//mainStage.close();
}
}
}
Here's My output:
PaperSource=Paper source : Automatic
# of papersources=6
newpapersource= Paper source :
newpapersource= Paper source : Manual Feed in Tray 1
newpapersource= Paper source : Printer auto select
newpapersource= Paper source : Tray 1
newpapersource= Paper source : Tray 2
newpapersource= Paper source : Form-Source
ObjectProperty [bean: Collation = UNCOLLATED
Copies = 1
Sides = ONE_SIDED
JobName = Whatever
Page ranges = null
Print color = COLOR
Print quality = NORMAL
Print resolution = Feed res=600dpi. Cross Feed res=600dpi.
Paper source = Paper source : Tray 2
Page layout = Paper=Paper: Letter size=8.5x11.0 INCH Orient=PORTRAIT leftMargin=54.0 rightMargin=54.0 topMargin=54.0 bottomMargin=54.0, name: paperSource, value: Paper source : Tray 2]
PRINTING FINISHED

I've found the trick for the printer trays is to iterate over the Media.class using getSupportedAttributeValues(...), match the human-readable name, and select that particular value. Tested on Windows, MacOS with several tray configurations.
String tray = "1";
// Handle human-readable names, see PRINTER_TRAY_ALIASES usage below for context. Adjust as needed.
List<String> PRINTER_TRAY_ALIASES = Arrays.asList("", "Tray ", "Paper Cassette ");
// Get default printer
PrintService printService = PrintServiceLookup.lookupDefaultPrintService();
// Attributes to be provided at print time
PrintRequestAttributeSet pset = new HashPrintRequestAttributeSet();
Media[] supported = printService.getSupportedAttributeValues(Media.class, null, null);
for(Media m : supported) {
for(String pta : PRINTER_TRAY_ALIASES) {
// Matches "1", "Tray 1", or "Paper Cassette 1"
if (m.toString().trim().equalsIgnoreCase(pta + tray)) {
attributes.add(m);
break;
}
}
}
// Print, etc
// printJob.print(pdfDoc, pset);

Configure a p2 update repository programmatically

There is an article in the Eclipse wiki how to configure user's p2 default repositories of an RCP application by adding a static conf file to your product:
Equinox/p2/Adding Self-Update to an RCP Application - Configuring the user's default repositories
I want to do the same programmatically in a Java class when the user changes some configuration details. I could not find appropriate p2 API documentation for that.

Use this solution for Eclipse 3.7 based applications:
final ProvisioningUI ui = ProvUIActivator.getDefault().getProvisioningUI();
IArtifactRepositoryManager artifactManager = ProvUI.getArtifactRepositoryManager(ui.getSession());
artifactManager.addRepository(new URI(UPDATE_SITE_URL);
IMetadataRepositoryManager metadataManager = ProvUI.getMetadataRepositoryManager(ui.getSession());
metadataManager.addRepository(new URI(UPDATE_SITE_URL);
For ProvUI and ProvisioningUI you have to import bundles org.eclipse.equinox.p2.ui and org.eclipse.equinox.p2.operations (among others).

I found a solution. It's easy - unfortunately there is no documentation...
// from bundle org.eclipse.equinox.p2.console
import org.eclipse.equinox.internal.p2.console.ProvisioningHelper;
URI repoUri = new URI(UPDATE_SITE_URL);
try {
ProvisioningHelper.addMetadataRepository(repoUri);
} catch( Exception e ) {
LOG.warn("Can not add update repository: " + repoUri);
}
try {
ProvisioningHelper.addArtifactRepository(repoUri);
} catch( Exception e ) {
LOG.warn("Can not add update repository: " + repoUri);
}

Furthermore you can add more than one repositories with ElementUtils and also you can sort them.
MetadataRepositoryElement[] element = new MetadataRepositoryElement[links.length];
for (int i = 0; i < links.length; i++) {
element[i] = new MetadataRepositoryElement(null, new URI(links[i]), true);
element[i].setNickname("Link-"+i);
}
ElementUtils.updateRepositoryUsingElements(element, null);
These links will be appeared alphabetically sorted.

This is high on the Google query for this issue, and there's still not a good way to do it published:
If anyone finds this page via Google as I did, I've solved this problem. You can use org.eclipse.equinox.internal.p2.ui.model.ElementUtils.updateRepositoryUsingElements to set the repositories programmatically. Full code can be found here.