We Keep Coding

Optimize the code which Copying AWS S3 objects into DB

I have written a code which fetched the S3 objects from AWS s3 using S3 sdk and stores the same in our DB, the only problem is the task is repeated for three different services, the only thing is changed is the instance of service class.
I have copy and pasted code in each service layer just to changes the instance for an instance.
The task is repeated for service classes VehicleImageService, MsilLayoutService and NonMsilLayoutService, every layer is having its own repository.
I am trying to identify a way to accomplish the same by placing that snippet in one place and on an runtime using Reflection API I wish to pass the correct instance and invoke the method, but I want to achieve the same using best industry practices and pattern. I.e. I want to refactor into generic methods for other services, so instance can be passed at runtime.
So kindly assist me on the same.
public void persistImageDetails() {
log.info("MsilVehicleLayoutServiceImpl::persistImageDetails::START");
String bucketKey = null; //common param
String modelCode = null;//common param
List<S3Object> objList = new ArrayList<>(); //common param
String bucketName = s3BucketDetails.getBucketName();//common param
String bucketPath = s3BucketDetails.getBucketPrefix();//common param
try {
//the layoutRepository object can be MSILRepository,NonMSILRepository and VehilceImageRepository
List<ModelCode> modelCodes = layoutRepository.findDistinctAllBy(); // this line need to take care of
List<String> modelCodePresent = modelCodes.stream().map(ModelCode::getModelCode)
.collect(Collectors.toList());
List<CommonPrefix> allKeysInDesiredBucket = listAllKeysInsideBucket(bucketName, bucketPath);//common param
synchDB(modelCodePresent, allKeysInDesiredBucket);
if (null != allKeysInDesiredBucket && !allKeysInDesiredBucket.isEmpty()) {
for (CommonPrefix commonPrefix : allKeysInDesiredBucket) {
bucketKey = commonPrefix.prefix();
modelCode = new File(bucketKey).getName();
if (modelCodePresent.contains(modelCode)) {
log.info("skipping iteration for {} model code", modelCode);
continue;
}
objList = s3Service.getBucketObjects(bucketName, bucketKey);
if (null != objList && !objList.isEmpty()) {
for (S3Object object : AppUtil.skipFirst(objList)) {
saveLayout(bucketName, modelCode, object);
}
}
}
}
log.info("MSIL Vehicle Layout entries has been successfully saved");
} catch (Exception e) {
log.error("Error occured", e);
e.printStackTrace();
}
log.info("MsilVehicleLayoutServiceImpl::persistImageDetails::END");
}
private void saveLayout(String bucketName, String modelCode, S3Object object) {
log.info("Inside saveLayout::Start preparing entity to persist");
String resourceUri = null;
MsilVehicleLayout vehicleLayout = new MsilVehicleLayout();// this can be MsilVehicleLayout. NonMsilVehicleLayout, VehicleImage
vehicleLayout.setFileName(FilenameUtils.removeExtension(FilenameUtils.getName(object.key())));
vehicleLayout.setModelCode(modelCode);
vehicleLayout.setS3BucketKey(object.key());
resourceUri = getS3ObjectURI(bucketName, object.key());
vehicleLayout.setS3ObjectUri(resourceUri);
vehicleLayout.setS3PresignedUri(null);
vehicleLayout.setS3PresignedExpDate(null);
layoutRepository.save(vehicleLayout); //the layoutRepository object can be MSILRepository,NonMSILRepository and VehilceImageRepository
log.info("Exiting saveLayout::End entity saved");
}

Using M2Doc programmatically : Error in the generated .docx document

I'm trying to use M2Doc programmatically, I managed to generate my .docx file without getting errors in the validation part but I'm getting the following Error in the generated document:
{m:self.Name} Couldn't find the 'aqlFeatureAccess(org.eclipse.emf.common.util.URI.Hierarchical,java.lang.String)' service
The "self.Name" part is what I wrote in my template.
I think I'm lacking some kind of reference to a service but I don't know how to fix it.
The self variable is a reference to a model based on a meta-model I created. But I'm not sure I imported it correctly in my code.
I based my code on the code I found on the M2Doc website + some code I found on their GitHub, especially concerning how to add a service in the queryEnvironment.
I searched in the source code of acceleo and M2Doc to see which services they add but it seems that they already import all the services I'm using.
As I said, the validation part is going well and doesn't generate a validation file.
public static void parseDocument(String templateName) throws Exception{
final URI templateURI = URI.createFileURI("Template/"+templateName+"."+M2DocUtils.DOCX_EXTENSION_FILE);
final IQueryEnvironment queryEnvironment =
org.eclipse.acceleo.query.runtime.Query.newEnvironmentWithDefaultServices(null);
final Map<String, String> options = new HashMap<>(); // can be empty
M2DocUtils.prepareEnvironmentServices(queryEnvironment, templateURI, options); // delegate to IServicesConfigurator
prepareEnvironmentServicesCustom(queryEnvironment, options);
final IClassProvider classProvider = new ClassProvider(ClassLoader.getSystemClassLoader()); // use M2DocPlugin.getClassProvider() when running inside Eclipse
try (DocumentTemplate template = M2DocUtils.parse(templateURI, queryEnvironment, classProvider)) {
ValidationMessageLevel validationLevel = validateDocument(template, queryEnvironment, templateName);
if(validationLevel == ValidationMessageLevel.OK){
generateDocument(template, queryEnvironment, templateName, "Model/ComplexKaosModel.kaos");
}
}
}
public static void prepareEnvironmentServicesCustom(IQueryEnvironment queryEnvironment, Map<String, String> options){
Set<IService> services = ServiceUtils.getServices(queryEnvironment, FilterService.class);
ServiceUtils.registerServices(queryEnvironment, services);
M2DocUtils.getConfigurators().forEach((configurator) -> {
ServiceUtils.registerServices(queryEnvironment, configurator.getServices(queryEnvironment, options));
});
}
public static void generateDocument(DocumentTemplate template, IQueryEnvironment queryEnvironment,
String templateName, String modelPath)throws Exception{
final Map<String, Object> variable = new HashMap<>();
variable.put("self", URI.createFileURI(modelPath));
final Monitor monitor = new BasicMonitor.Printing(System.out);
final URI outputURI = URI.createFileURI("Generated/"+templateName+".generated."+M2DocUtils.DOCX_EXTENSION_FILE);
M2DocUtils.generate(template, queryEnvironment, variable, outputURI, monitor);
}

The variable "self" contains an URI:
variable.put("self", URI.createFileURI(modelPath));
You have to load your model and set the value of self to an element from your model using something like:
final ResourceSet rs = new ResourceSetImpl();
final Resource r = rs.getResource(uri, true);
final EObject value = r.getContents()...;
variable.put("self", value);
You can get more details on resource loading in the EMF documentation.

How can I get properties stored in ConfigAdmin?

I've created the ConfigAdmin loaded some properties. After that I've saved them. My question is: how can I get the properties that I have stored?
I've created the ConfigAdmin in the Activator.java:
public class Activator implements BundleActivator {
private String configFile = "API.properties";
#Override
public void start(BundleContext bundleContext) throws Exception {
InputStream stream = (bundleContext.getBundle().getResource(configFile)).openStream();
Properties properties = new Properties();
properties.load(stream);
createConfigAdmin(properties);
}
#Override
public void stop(BundleContext bundleContext) throws Exception {
}
private boolean createConfigAdmin(Properties properties, BundleContext context) {
try {
Dictionary<String, String> props = new Hashtable<String, String>();
ServiceReference reference = context.getServiceReference(ConfigurationAdmin.class.getName());
ConfigurationAdmin admin = (ConfigurationAdmin) context.getService(reference);
Configuration configuration = admin.createFactoryConfiguration(pid.configAdminPID, null);
for(final String name: properties.stringPropertyNames())
props.put(name, properties.getProperty(name));
configuration.update(props);
return true;
} catch(Exception e)
{
e.printStackTrace();
return false;
}
}
}

Was your intention really to create a factory config? You only need it if you want to create several configs for the same factory pid.
If you simply want to create a simple configuration then just use admin.getConfiguration(oid) you can update the configuration in the same way you do now.
If you want to read the configuration afterwards you simply get it again. If you want to configure a bundle with this configuration you typically will create a ManagedService and publish it. See http://liquid-reality.de/display/liquid/2011/09/23/Karaf+Tutorial+Part+2+-+Using+the+Configuration+Admin+Service

Calling Existing PipeLine in GATE

I am new to Java and I want to call my saved pipeline using GATE JAVA API through Eclipse
I am not sure how I could do this although I know how to create new documents etc
FeatureMap params = Factory.newFeatureMap();
params.put(Document.DOCUMENT_URL_PARAMETER_NAME, new URL("http://www.gate.ac.uk"));
params.put(Document.DOCUMENT_ENCODING_PARAMETER_NAME, "UTF-8");
// document features
FeatureMap feats = Factory.newFeatureMap();
feats.put("date", new Date());
Factory.createResource("gate.corpora.DocumentImpl", params, feats, "This is home");
//End Solution 2
// obtain a map of all named annotation sets
Document doc = Factory.newDocument("Document text");
Map <String, AnnotationSet> namedASes = doc.getNamedAnnotationSets();
System.out.println("No. of named Annotation Sets:" + namedASes.size());
// no of annotations each set contains
for (String setName : namedASes.keySet()) {
// annotation set
AnnotationSet aSet = namedASes.get(setName);
// no of annotations
System.out.println("No. of Annotations for " +setName + ":" + aSet.size());

There is a good example of GATE usage from java. Probably it does exactly what you want. BatchProcessApp.java.
In particular:
loading pipeline is done with lines
// load the saved application
CorpusController application =
(CorpusController)PersistenceManager.loadObjectFromFile(gappFile);
pipeli executed with
// run the application
application.execute();
Code is informative, clear and could be easy changed for your particular needs. The oxygen of open source project :)

Something like this could be used(do not forget to init GATE: set GATE home and etc):
private void getProcessedText(String textToProcess) {
Document gateDocument = null;
try {
// you can use your method from above to build document
gateDocument = createGATEDocument(textToProcess);
corpusController.getCorpus().add(gateDocument);
corpusController.execute();
// put here your annotations processing
} catch (Throwable ex) {
ex.printStackTrace();
} finally {
if (corpusController.getCorpus() != null) {
corpusController.getCorpus().remove(gateDocument);
}
if (gateDocument != null) {
Factory.deleteResource(gateDocument);
}
}
}
private CorpusController initPersistentGateResources() {
try {
Corpus corpus = Factory.newCorpus("New Corpus");
corpusController = (CorpusController) PersistenceManager.loadObjectFromFile(new File("PATH-TO-YOUR-GAPP-FILE"));
corpusController.setCorpus(corpus);
} catch (Exception ex) {
ex.printStackTrace();
}
return corpusController;
}

Spring - validate that all message resources are well configured

We need to add some code to be executed on application load in order to validate that all the messages.properties elements are well defined for all languages.
Is this possible?
Steps: dynamically read on application load all the spring message codes from JSP or java classes then pass through all message resources properties files and validate that nothing is missing from them.

We ended up doing this manually but without using any library.
Steps:
have all the keys used in Java classes or JSP defined in a constant file
Read them using Java .class properties:
Field[] fields = Constants.class.getFields();
String filed[i].get(Constants.class);
Read all messageResources.properties file names from the project using:
String pathToThisClass = MessageResourcesValidator.class.getProtectionDomain().getCodeSource().getLocatin().getPath();
File filePath = new File(pathToThisClass);
String[] list = filePath.list(new DirFilter("(messages).*\\.(properties)"));
DirFilter is a normal class implementing Java's FileNameFilter
Create a class that read the properties from a file using its file name:
public class PropertiesFile{
private Properties prop;
public PropertiesFile(String fileName) throws Exception
{
init(fileName);
}
private void init(String fileName) throws Exception
{
prop = new Properties();
try (InputStream input = getClass().getClassLoader().getResourceAsStream(fileName);)
{
if(input == null)
{
throw new Exception("Enable to load properties file " + fileName);
}
prop.load(input);
}
catch(IOException e)
{
throw new Exception("Error loading properties file " + fileName);
}
}
public List<String> getPropertiesKeysList()
{
List<String> result = new ArrayList<>();
Enumeration<?> e = prop.propertyNames();
while(e.hasMoreElements())
{
result.add((String) e.nextElement());
// String value = prop.getProperty(key);
}
return result;
}
}
The part that does the comparison should be something as the following code that calls the above methods:
List<String> msgResourcesFiles = getMessageResourcesFileNames();
List<String> codeKeys = getListOfCodeMessageResources();
PropertiesFile file = null;
List<String> propKeys = null;
for(String fileName : msgResourcesFiles)
{
file = new PropertiesFile(fileName);
propKeys = file.getPropertiesKeysList();
for(String key : codeKeys)
{
if(!propKeys.contains(key))
{
throw new Exception("Missing key " + key);
}
}
}
Note: or another workaround would be to compare all message resources files to a default one and this way we minimize the code needed from the above explanation.