I am working with HashMap and don't have much experience yet.
I am trying to write a csvFile for proof of a comparison between two lists.
If the compared value is the same ok otherwise not ok.
Simple, but the values "ok" or "not ok" need to be changed automatically, so it was suggested to me to use HashMap where I put the name of the compared field which is the key and the value will be its state (ok or not ok).
So far the values are returned and the file is written, but the status does not fill in automatically.
This is my code so far, if anyone knows how I can do it or has other suggestions please let me know.
HashMap
public static Map<String, String> initMap(String status) {
Map<String, String> mapFields = new HashMap<String, String>();
mapFields.put("Book Ref", status);
mapFields.put("Trade Date", status);
mapFields.put("Start Date", status);
mapFields.put("End Date", status);
mapFields.put("Period Multiplier", status);
mapFields.put("Notional", status);
mapFields.put("Currency", status);
mapFields.put("Rate", status);
mapFields.put("Frequency Multiplier", status);
mapFields.put("Day count", status);
return mapFields;
}
Here in the same class, I created this method to compare two lists and define if it is ok or not.
public static void compareValues(List<String> inputs, List<String> outputs, XrayFields fields, TradeData data, MKTWireIRS mkt) throws ParseException, InterruptedException {
int y = 0;
int x = 0;
Map<String, String> map = new HashMap<String, String>();
// List<WriterCSV> writeCSV = new ArrayList<>();
WriterCSV cv = new WriterCSV();
try {
for (String input : inputs) {
for (String out : outputs) {
cv = new WriterCSV();
map = new HashMap<String, String>();
if (y == x) {
if (input.equals(out)) {
System.out.println("ok: " + input + " = " + out);
String comment = "All fields checked are ok";
fields.setComment(comment);
fields.setStatus("PASS");
cv.setOk("Ok");
map = initMap(cv.getOk());
} else {
System.out.println("not ok: " + input + " = " + out);
fields.setStatus("FAIL");
String comment = "The value " + input + " is not the same as " + out;
fields.setComment(comment);
cv.setOk("not Ok");
map = initMap(cv.getOk());
}
}
x = x + 1; // count of the list of output
}
y = y + 1; // count of the list of inputs
x = 0; // reset to 0 the count of outputs
}
//create evidence of comparison
cv.reportMKTWireToOutputIRS(data, mkt, map);
} catch (Error e) {
System.out.println(e);
}
}
This is the method for writing the csv.
public void reportMKTWireToOutputIRS(TradeData data2, MKTWireIRS mkt, Map<String, String> map ) throws ParseException, InterruptedException {
try {
FileWriter fw = new FileWriter(new File(CSV_MKTWire + Setup.IRScsv));
CSVWriter cw = new CSVWriter(fw);
//format values
String month = PropertyNames.deCapitalize(data2.getPeriod());
String monthReset = PropertyNames.deCapitalize(data2.getResetFrequencyPeriod());
String formatTradeDateMKT = Utils.formatDateToCompareMKTWire(data2.getTradeDateIRS());
String formatStartDateMKT = Utils.formatDateToCompareMKTWire(data2.getStart_Date());
String formatMAturityDateMKT = Utils.formatDateToCompareMKTWire(data2.getMaturity_Date());
String rateActual = Utils.roundDecimal(data2.getRateIRS());
String rateFormat = Utils.roundRateMKTwire(mkt.getRateIRS());
String notionalFormat = data2.getNotional().charAt(0) + "M";
String[] headers = { "Output Field", "Output Value", " MKTWire Field", " MKTWire Value", "Status" };
List<String[]> data = new ArrayList<String[]>();
String[] book = { "Book Ref", data2.getBookRef() + data2.getBookType(),"Book MKTWire",mkt.getBookIRS(), map.get("Book Ref")};
String[] tradeDate = { "Trade Date", formatTradeDateMKT,"Trade Date MKTWire",mkt.getTradeDateIRS(), map.get("Trade Date")};
String[] startDate = { "Start Date", formatStartDateMKT, "Start Date MKTWire",mkt.getStartDate(), map.get("Start Date") };
String[] maturity = { "End Date", formatMAturityDateMKT, "End Date MKTWire",mkt.getEndDate(), map.get("End Date") };
String[] tenor = { "Period Multiplier", data2.getPeriodMultiplier() + month, "Tenor MKTWire",mkt.getTenorIRS(), map.get("Period Multiplier") };
String[] notional = { "Notional", notionalFormat, "Notional MKTWire", mkt.getNotionalValueIRS(), map.get("Notional") };
String[] currency = { "Currency", data2.getCurrencyIRS(), "Currency MKTWire", mkt.getCurrencyIRS(), map.get("Currency") };
String[] rate = { "Rate", rateActual, "Rate MKTWire", rateFormat, map.get("Rate") };
String[] resetFrequency = { "Frequency Multiplier", data2.getResetFrequencyMultiplier() + monthReset, "Frequency Multiplier MKTWire", mkt.getResetFrequencyIRS(),map.get("Frequency Multiplier") };
String[] dayCount = { "Day Count", data2.getDayCount(), "Day Count MKTWire", mkt.getDayCountIRS(), map.get("Day count") };
data.add(headers);
data.add(book);
data.add(tradeDate);
data.add(startDate);
data.add(maturity);
data.add(tenor);
data.add(notional);
data.add(currency);
data.add(rate);
data.add(resetFrequency);
data.add(dayCount);
cw.writeAll(data);
cw.flush();
fw.flush();
cw.close();
fw.close();
} catch (IOException e) {
e.printStackTrace();
}
}
You are having one map and you are calling the initMap method which sets the value for all keys in the map within a loop, in the end it will have either "ok" or "not ok" based on your final loop validation.
Related
I'll explain the logic: I am reading a XML file which contain many request and responses in soap format then I'm storing the request and response in two Hash map. In first Hash map I'm storing transaction Id(unique) as key and values as request time,til-name. In second hash map I'm storing transaction Id(unique) as key and values as response time. In both hash map the keys are same but values are different, by using for loop iterating two loops and I need to get the time difference between response time and request time
eg:request time:2020-01-30T11:07:08.351Z and response time:2020-01-30T11:07:10.152Z
public class MapTimeDiff {
public static void main(String[] args) throws ParseException {
File file =new File("C:\\Users\\gsanaulla\\Documents\\My Received Files\\ecarewsframework.xml");
Scanner in = null;
String tilname = null;
String transactionId = null;
String requesttime = null;
String responsetime = null;
Date dateOne = null;
Date dateTwo = null;
double timeDiff;
DateFormat df = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS");
Map<String,ArrayList<String>> request=new HashMap<String,ArrayList<String>>();
ArrayList<String> req=new ArrayList<String>();
Map<String,ArrayList<String>> response=new HashMap<String,ArrayList<String>>();
ArrayList<String> res=new ArrayList<String>();
try {
in = new Scanner(file);
while(in.hasNext())
{
String line=in.nextLine();
if(line.contains("</S:Envelope>")) {
System.out.println(line);
tilname=line.split("StartRecord><")[1].split("><")[0].split(":")[1];
System.out.println("tilname :: "+tilname);
transactionId = line.split("transactionId>")[1].split("<")[0];
System.out.println("transactio id :: "+transactionId);
requesttime=line.split("sourceTimestamp>")[1].split("<")[0];
System.out.println("request time is :: "+requesttime);
dateOne = df.parse(requesttime);
}
req.add(tilname);
req.add(dateOne.toString());
System.out.println("req is==== " +req);
request.put(transactionId,req);
System.out.println("request is==== " +request.get(transactionId));
if(line.contains("</SOAP-ENV:Envelope>")) {
//System.out.println(line);
if(line.contains("transactionId"))
{
responsetime=line.split("sourceTimestamp>")[1].split("<")[0];
transactionId = line.split("transactionId>")[1].split("<")[0];
System.out.println("responsetime :: "+responsetime);
System.out.println("transaction id "+transactionId);
dateTwo = df.parse(responsetime);
}
res.add(dateTwo.toString());
System.out.println("res is===== "+res);
response.put(transactionId,res);
System.out.println("response is===== "+response.get(transactionId));
for (Entry<String, ArrayList<String>> entry : request.entrySet()) {
for (Entry<String, ArrayList<String>> entry1 : response.entrySet()) {
System.out.println("Key = " + entry.getKey() +
", Value = " + entry.getValue());
System.out.println("Key = " + entry1.getKey() +
", Value = " + entry1.getValue());
if(request.keySet().equals(response.keySet())) {
timeDiff = (dateTwo.getTime() - dateOne.getTime());
}
}
}
}
}
}
catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
I'm not sure if I understood your question correctly but maybe you can do something similiar like the following:
Map<String, List<String>> requests = Map.of("1", List.of("10,13,12"), "2", List.of("8,7,9"), "3", List.of("11"));
Map<String, List<String>> responses = Map.of("1", List.of("9,10,14"), "2", List.of("8,9,6,12"));
for(Map.Entry<String, List<String>> requestEntry : requests.entrySet()) {
String transactionId = requestEntry.getKey();
if(responses.containsKey(transactionId)) {
System.out.println("Transaction Id: " + transactionId);
for(int i = 0; i < min(requestEntry.getValue().size(), responses.get(transactionId).size()); i++) {
List<String> requestTimes = asList(requestEntry.getValue().get(i).split(","));
List<String> responseTimes = asList(responses.get(transactionId).get(i).split(","));
for(int j = 0; j < min(requestTimes.size(), responseTimes.size()); j++) {
int requestTime = parseInt(requestTimes.get(j));
int responseTime = parseInt(responseTimes.get(j));
System.out.println("Difference: " + abs(requestTime - responseTime));
}
}
}
}
As you can see there are no responses for transactionId 3 so this will be ignored.
If elements in the list for a key differ in size (transactionId 2) the surplus elements will also be ignored.
Transaction Id: 1
Difference: 1
Difference: 3
Difference: 2
Transaction Id: 2
Difference: 0
Difference: 2
Difference: 3
I have a MapReduce Program which can process Delimited, Fixed Width and Excel Files. There is no problem in reading Delimited and Fixed Width File. But Problem with Excel File is setup() and cleanup() methods are getting called,but not the map(). I tried with adding annotations to map() still it didnt work.
public class RulesDriver extends Configured implements Tool {
private static Logger LOGGER = LoggerFactory.getLogger(RulesDriver.class);
RuleValidationService aWSS3Service = new RuleValidationService();
HashMap<String, Object> dataMap = new HashMap<String, Object>();
HashMap<String, String> controlMap = new HashMap<String, String>();
public String inputPath = "";
public String outputPath = "";
private static DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd-HH-mm");
ControlFileReader ctrlReader = new ControlFileReader();
CSVToExcel csv2Excel = new CSVToExcel();
HashMap<Integer,String> countMap = new HashMap<Integer,String>();
HashMap<String,Integer> numberValueMap = new HashMap<String,Integer>();
HashMap<String,Object> rulesMap = new HashMap<String,Object>();
CharsetConvertor charsetConvertor = new CharsetConvertor();
ControlFileComparison controlFileComparison = new ControlFileComparison();
boolean isControlFileValid = false;
public static void main(String[] args) throws Exception {
int res = ToolRunner.run(new Configuration(), new RulesDriver(), args);
System.exit(res);
}
#Override
public int run(String[] args) throws Exception {
LOGGER.info("HADOOP MapReduce Driver started");
if (args.length < 3) {
LOGGER.info("Args ");
return 1;
}
int j = -1;
//Prop - Starts
String cacheBucket = args[0];
String s3accesskey = args[1];
String s3accesspass = args[2];
//Prop - ends
// file2InputPath Value is from DB
String file2InputLocation = "";
String fileComparisonInd = "";
String inputPath = "";
String outputPath = "";
String url = "";
String userId = "";
String password = "";
String fileType = "";
String ctrlCompResult = "N";
try {
Configuration conf = new Configuration();
Properties prop = new Properties();
//Prop - starts
prop.setProperty("qatool.cacheBucket", cacheBucket);
prop.setProperty("qatool.s3accesskey", s3accesskey);
prop.setProperty("qatool.s3accesspass", s3accesspass);
String propertiesFile = aWSS3Service.getObjectKey(cacheBucket, "application",prop);
if(null==propertiesFile && "".equals(propertiesFile)){
return 0;
}
S3Object s3Object = aWSS3Service.getObject(cacheBucket, propertiesFile, prop);
LOGGER.info("Loading App properties");
InputStreamReader in = new InputStreamReader(s3Object.getObjectContent());
Properties appProperties = new Properties();
try {
appProperties.load(in);
prop.putAll(appProperties);
LOGGER.info(" ",prop);
}
catch (IOException e1) {
LOGGER.error("Exception while reading properties file :" , e1);
return 0;
}
initialize(prop);
if (!(dataMap == null)) {
if (("N").equals(dataMap.get("SuccessIndicator"))) {
return 0;
}
List value = (ArrayList) dataMap.get("LookUpValList");
LOGGER.info("lookUpVallist",value);
}
if (dataMap != null) {
controlMap = (HashMap<String, String>) dataMap.get("ControlMap");
}
if (controlMap != null) {
inputPath = (prop.getProperty("qatool.rulesInputLocation") + "/").concat((String) dataMap.get("InputFileName")); //TEMP
LOGGER.info(inputPath);
fileType = (String) dataMap.get("FileType");
} else {
inputPath = (prop.getProperty("qatool.rulesInputLocation") + "/").concat((String) dataMap.get("InputFileName"));
LOGGER.info(inputPath);
fileType = (String) dataMap.get("FileType");
}
rulesMap = (HashMap<String,Object>)dataMap.get("RulesMap");
isControlFileValid = controlFileComparison.compareControlFile(controlMap, aWSS3Service, prop, rulesMap); //TEMP
LOGGER.info("isControlFileValid in driver : "+isControlFileValid);
if(isControlFileValid){
ctrlCompResult = "Y";
}
conf.set("isControlFileValid", ctrlCompResult);
// ** DATABASE Connection **/
String ctrlFileId = controlMap.get("ControlFileIdentifier");
url = prop.getProperty(QaToolRulesConstants.DB_URL);
userId = prop.getProperty(QaToolRulesConstants.DB_USER_ID);
password = prop.getProperty(QaToolRulesConstants.DB_USER_DET);
InpflPrcsSumm inpflPrcsSumm = new InpflPrcsSumm();
DBConnectivity dbConnectivity = new DBConnectivity(url, userId, password);
inpflPrcsSumm = dbConnectivity.getPreviousFileDetail(ctrlFileId);
dbConnectivity.closeConnection();
LOGGER.info( " inpflPrcsSumm.getPrevFileId() " + inpflPrcsSumm.getPrevFileId());
prop.setProperty(QaToolRulesConstants.PREV_FILE_ID, inpflPrcsSumm.getPrevFileId().toString());
file2InputLocation = inpflPrcsSumm.getPrevFileLocation();
boolean file2Available = file2InputLocation.isEmpty();
String folderPath = "";
String bucket = "";
if (!file2Available) {
String arr[] = file2InputLocation.split("/");
if(file2InputLocation.startsWith("http")){
bucket = arr[3];
}else{
bucket = arr[2];
}
folderPath = file2InputLocation.substring(file2InputLocation.lastIndexOf(bucket) + bucket.length() + 1, file2InputLocation.length());
}
// File 2 input path
prop.setProperty("qatool.file2InputPath", file2InputLocation);
if(!file2Available){
file2InputLocation = file2InputLocation + "/Success";
String file2Name = aWSS3Service.getObjectKey(bucket, folderPath,prop);
LOGGER.info("bucket->"+bucket);
LOGGER.info("folderPath->"+folderPath);
file2Name = file2Name.substring(file2Name.lastIndexOf("/")+1, file2Name.length());
prop.setProperty("file2Name", (null!=file2Name && "".equals(file2Name))?"":file2Name);
LOGGER.info(prop.getProperty("file2Name"));
}
prop.setProperty("qatool.auditPrevFolderPath", folderPath);
prop.setProperty("qatool.auditBucketPrevFolderPath", bucket);
LOGGER.info("ctrlFileId : " + ctrlFileId);
LOGGER.info("BUCKET : " + bucket);
LOGGER.info("folder : " + folderPath);
Date dateobj = new Date();
outputPath = (String) prop.getProperty("qatool.rulesOutputLocation") + "/" + dateFormat.format(dateobj); //TEMP
fileComparisonInd = controlMap.get("FileComparisonIndicator");
Gson gson = new Gson();
String propSerilzation = gson.toJson(prop);
conf.set("application.properties", propSerilzation);
Job job = Job.getInstance(conf);
job.setJarByClass(RulesDriver.class);
job.setJobName("Rule Validation and Comparison");
job.getConfiguration().set("fs.s3n.awsAccessKeyId", (String) prop.getProperty("qatool.s3accesskey"));
job.getConfiguration().set("fs.s3n.awsSecretAccessKey", (String) prop.getProperty("qatool.s3accesspass"));
job.getConfiguration().set("fs.s3.awsAccessKeyId", (String) prop.getProperty("qatool.s3accesskey"));
job.getConfiguration().set("fs.s3.awsSecretAccessKey", (String) prop.getProperty("qatool.s3accesspass"));
job.getConfiguration().set("fs.s3a.awsAccessKeyId", (String) prop.getProperty("qatool.s3accesskey"));
job.getConfiguration().set("fs.s3a.awsSecretAccessKey", (String) prop.getProperty("qatool.s3accesspass"));
job.getConfiguration().set("fs.s3n.endpoint", "s3.amazonaws.com");
job.getConfiguration().set("fs.s3.endpoint", "s3.amazonaws.com");
job.getConfiguration().set("fs.s3a.endpoint", "s3.amazonaws.com");
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(Text.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
job.setReducerClass(RulesCountReducer.class);
job.setNumReduceTasks(1);
job.setMaxMapAttempts(1);
job.setMaxReduceAttempts(1);
if("UTF-16".equalsIgnoreCase(controlMap.get("FileCodePage"))){
convertEncoding((String)dataMap.get("InputFileName"),rulesInputLocation,prop);
if (!file2Available && "Y".equals(ctrlCompResult)) {
convertEncoding(inpflPrcsSumm.getPrevFileName(),file2InputLocation,prop);
}
}
LOGGER.info("fileComparisonInd + "+ fileComparisonInd + " file2Available + " + file2Available + " ctrlCompResult + " + ctrlCompResult);
if (fileType != null && fileType.equals(QaToolRulesConstants.INPUT_FILE_TYPE_DELIMI)) {
job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);
MultipleInputs.addInputPath(job, new Path(rulesInputLocation), TextInputFormat.class, TextRulesMapper.class);
if (fileComparisonInd.equalsIgnoreCase(QaToolRulesConstants.FILE_COMP_INDICATOR) && !file2Available && "Y".equals(ctrlCompResult)) {
Path file2InputPath = new Path(file2InputLocation);
if (isInputPathAvail(file2InputPath, conf)) {
MultipleInputs.addInputPath(job, file2InputPath, TextInputFormat.class, TextRulesMapperFile2.class);
}
}
} else if (fileType != null && fileType.equals(QaToolRulesConstants.INPUT_FILE_TYPE_EXCEL)) {
job.setInputFormatClass(ExcelInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);
MultipleInputs.addInputPath(job, new Path(rulesInputLocation), ExcelInputFormat.class, ExcelMapper.class);
String inputFileName = controlMap.get("InputFileName");
String fileExtn = inputFileName.substring(inputFileName.lastIndexOf(".") + 1);
prop.setProperty("File", "Excel");
prop.setProperty("fileExtension", fileExtn);
if (fileComparisonInd.equalsIgnoreCase(QaToolRulesConstants.FILE_COMP_INDICATOR) && !file2Available && "Y".equals(ctrlCompResult)) {
Path file2InputPath = new Path(file2InputLocation);
if (isInputPathAvail(file2InputPath, conf)) {
MultipleInputs.addInputPath(job, file2InputPath, ExcelInputFormat.class, ExcelMapper2.class);
}
}
} else if (fileType != null && fileType.equals(QaToolRulesConstants.INPUT_FILE_TYPE_FIXED)) {
prop.setProperty("File", "DAT");
MultipleInputs.addInputPath(job, new Path(rulesInputLocation), TextInputFormat.class, FixedWidthMapper.class);
if (fileComparisonInd.equalsIgnoreCase(QaToolRulesConstants.FILE_COMP_INDICATOR) && !file2Available && "Y".equals(ctrlCompResult)) {
Path file2InputPath = new Path(file2InputLocation);
if (isInputPathAvail(file2InputPath, conf)) {
MultipleInputs.addInputPath(job, file2InputPath, TextInputFormat.class, FixedWidthMapper2.class);
}
}
}
MultipleOutputs.addNamedOutput(job, "error", TextOutputFormat.class, Text.class, Text.class);
MultipleOutputs.addNamedOutput(job, "success", TextOutputFormat.class, Text.class, Text.class);
MultipleOutputs.addNamedOutput(job, QaToolRulesConstants.ADDED_DELETED, TextOutputFormat.class, Text.class, Text.class );
/*MultipleOutputs.addNamedOutput(job, QaToolRulesConstants.ADDED_UPDATED, TextOutputFormat.class, Text.class, Text.class );*/ //TEMP ADDED FOR ADDED AND UPDATED
MultipleOutputs.addNamedOutput(job, QaToolRulesConstants.DETAIL, TextOutputFormat.class, Text.class, Text.class);
FileOutputFormat.setOutputPath(job, new Path(outputPath));
j = job.waitForCompletion(true) ? 0 : 1;
LOGGER.info("Program Complted with return " + j);
//Code Added for Control file Movement -- starts
String outputBucket = rulesOutputLocation;
outputBucket = outputBucket.substring(outputBucket.indexOf("//")+2, outputBucket.length());
outputBucket = outputBucket.substring(0,(outputBucket.indexOf("/")));
String controlFileNamekey = aWSS3Service.getObjectKey(outputBucket, "delivery/"+ dataMap.get("ControlFileName"),prop);
if (controlFileNamekey != null) {
controlFileNamekey = (String) controlFileNamekey.substring(controlFileNamekey.lastIndexOf("/") + 1,controlFileNamekey.length());
String outputCtrlFilePath = "delivery/"+ dateFormat.format(dateobj) +"/" + controlFileNamekey;
LOGGER.info("controlFileNamekey "+controlFileNamekey+" outputCtrlFilePath "+outputCtrlFilePath);
aWSS3Service.moveObjects(outputBucket, "delivery/"+controlFileNamekey, outputBucket, outputCtrlFilePath,prop);
}
//Code Added for Control file Movement -- Ends
if (j == 0) {
// Get counters
LOGGER.info("Transfer");
final Counters counters = job.getCounters();
long duplicates = counters.findCounter(MATCH_COUNTER.DUPLICATES).getValue();
LOGGER.info("duplicates->"+duplicates);
long groupingThreshold = counters.findCounter(MATCH_COUNTER.GROUPING_ERR_THRESHOLD).getValue();
LOGGER.info("groupingThreshold->"+groupingThreshold);
if(groupingThreshold==1 || duplicates==1){
if(duplicates==1){
writeOutputFile(folderName,dateobj,"DuplicateRecords",prop,cacheBucket);
}else{
writeOutputFile(folderName,dateobj,"GroupingThreshold",prop,cacheBucket);
}
}else{
long successCount = counters.findCounter(MATCH_COUNTER.SUCCESS_COUNT).getValue();
if (controlMap.get("ColumnHeaderPresentIndicator") != null
&& controlMap.get("ColumnHeaderPresentIndicator").equals("Y")) {
successCount = successCount-1;
}
LOGGER.info("successCount "+successCount);
LOGGER.info("TOLERANCEVALUE " + counters.findCounter(MATCH_COUNTER.TOLERANCEVALUE).getValue());
LOGGER.info("RULES_ERRORTHRESHOLD " + counters.findCounter(MATCH_COUNTER.RULES_ERRORTHRESHOLD).getValue());
long errorThreshold = counters.findCounter(MATCH_COUNTER.RULES_ERRORTHRESHOLD).getValue();
LOGGER.info("COMPARISION_ERR_THRESHOLD " + counters.findCounter(MATCH_COUNTER.COMPARISION_ERR_THRESHOLD).getValue());
writeOutputFile(folderName,dateobj, outputPath + "," + successCount + "," + counters.findCounter(MATCH_COUNTER.TOLERANCEVALUE).getValue() + "," + errorThreshold + ","
+counters.findCounter(MATCH_COUNTER.COMPARISION_ERR_THRESHOLD).getValue()+","+ctrlCompResult,prop,cacheBucket);
String auditBucketName = "";
auditBucketName = rulesOutputLocation;
auditBucketName = auditBucketName.substring(auditBucketName.indexOf("//") + 2, auditBucketName.length() - 1);
auditBucketName = auditBucketName.substring(0, (auditBucketName.indexOf("/")));
String auditFileMovementPath = "delivery/" + dateFormat.format(dateobj);
auditFile = auditFile.replace(".xlsx","");
String inputFileName = (String) dataMap.get("InputFileName");
inputFileName = inputFileName.substring(0,inputFileName.lastIndexOf(".")).concat(".xlsx");
try {
LOGGER.info("Audit Bucket Name : " + auditBucketName);
LOGGER.info("Move parameter >>> outputbucketname : auditFileLocation : auditBucketName : auditFileMovementPath auditFile ");
LOGGER.info("Move parameter " + outputbucketname + ", " + auditFileLocation + " , " + auditBucketName + " , " + auditFileMovementPath + "/" + auditFile + "_" + inputFileName);
aWSS3Service.moveObjects(outputbucketname, auditFileLocation, auditBucketName, auditFileMovementPath +"/"+ auditFile +"_"+ inputFileName, prop);
} catch (Exception e) {
LOGGER.error("Exception while moving audit file ",e);
}
}
}else{
writeOutputFile(folderName,dateobj,"DuplicateRecords",prop,cacheBucket);
}
} catch (Exception e) {
LOGGER.error("Error in RulesDriver ", e);
}
return j;
}
}
Excel Mapper :
public class ExcelMapper extends Mapper<LongWritable, Text, Text, Text> {
#Override
protected void setup(Mapper<LongWritable, Text, Text, Text>.Context context)throws InterruptedException, IOException {
LOGGER.info("Inside Mapper Setup");
}
#Override
public void map(LongWritable key, Text value, Context context) throws InterruptedException, IOException {
}
#Override
protected void cleanup(Context context) throws IOException,
InterruptedException {
}
}
I want to print the all values in the array but it just prints the last value int the array, how can I get my desired result by improving this code:
public void applyAttendence(ArrayList<String> presents, ArrayList<String> absents) {
ArrayList<String> present = new ArrayList<String>();
HashMap params = new HashMap();
// [232, 232, 12, 223]
String[] stringArray = presents.toArray(new String[0]);
if (presents.size() == 0) {
params.put("present", "");
} else {
// for(String pre:presents) {
params.put("present", stringArray);
System.out.println(" present[]" + presents);
System.out.println("hellow present man: " + params.get("present"));
// }
System.out.println("hellow present man: " + params.get("present"));
}
if (absents.size() == 0) {
params.put("absent", "");
} else {
for (String abs : absents) {
params.put("absent[]", abs);
}
// params.put("present[]", presents + "");
//
params.put("absent[]", absents + "");
}
}
That is because you are overwriting same key with different value every time
for (String abs : absents) {
params.put("absent[]", abs);
}
So your hashmap will only have last value written against the key absent[]
This is may be you have defined array as:
String[] stringArray = presents.toArray(new String[0]);
try initializing as:
String[] stringArray = new String[presents.size()];
stringArray = presents.toArray(stringArray );
Try this simplified solution to show all of the attendance
public void applyAttendence(ArrayList<String> presents, ArrayList<String> absents) {
String sPresent = "";
for (String present : presents) {
sPresent += present + ", ";
}
if (!sPresent.equals(""))
sPresent = sPresent.substring(0, sPresent.length() - 2);
String sAbsent = "";
for (String absent : absents) {
sAbsent += absent + ", ";
}
if (!sAbsent.equals(""))
sAbsent = sAbsent.substring(0, sAbsent.length() - 2);
if (presents.size() > 0) {
System.out.println("present = " + sPresent);
} else {
System.out.println("present = no one");
}
if (absents.size() > 0) {
System.out.println("absent = " + sAbsent);
} else {
System.out.println("absent = no one");
}
}
Here is the log of my map entries. That have two same key exists. How this is possible?
Map<String, Objects> map = new HashMap<String, Objects>();
addContact("+917111111111");
addContact("+919222222222");
addContact("+919222222222");
private void addContact(String number){
if(TextUtils.isEmpty(number))return;
number = number.trim();
number = number.replaceAll("-", "");
number = number.replaceAll(" ", "");
if(!map.containsKey(number)) {
map.put(number, null);
}
}
/* While debugging in android studio. I have found the map have below entry.
0 = {HashMap$HashMapEntry#3798} "+919222222222" -> "null"
1 = {HashMap$HashMapEntry#3832} "+919222222222" -> "null"
2 = {HashMap$HashMapEntry#3694} "+917111111111" -> "null"
*/
map.containsKey("+919222222222");// ==> return false
Why this is happen ?
Actual task:
private void getContacts(){
try {
Cursor cursor = null;
StringBuffer sb = new StringBuffer();
Map<String, Object> map = new HashMap<String, Object>();
try {
String strOrder = ContactsContract.CommonDataKinds.Phone.DISPLAY_NAME + " ASC";
cursor = getContentResolver().query(ContactsContract.CommonDataKinds.Phone.CONTENT_URI, null, null, null, strOrder);
int contactIdIdx = cursor.getColumnIndex(ContactsContract.CommonDataKinds.Phone._ID);
int nameIdx = cursor.getColumnIndex(ContactsContract.CommonDataKinds.Phone.DISPLAY_NAME);
int phoneNumberIdx = cursor.getColumnIndex(ContactsContract.CommonDataKinds.Phone.NUMBER);
int photoIdIdx = cursor.getColumnIndex(ContactsContract.CommonDataKinds.Phone.PHOTO_ID);
cursor.moveToFirst();
do {
String idContact = cursor.getString(contactIdIdx);
String name = cursor.getString(nameIdx);
String phoneNumber = cursor.getString(phoneNumberIdx);
//...
phoneNumber = getFormatedNumber(phoneNumber);
//as map key same phone number can not be two times
if(!map.containsKey(phoneNumber)) {
map.put(phoneNumber, null);
sb.append("\nPhone Number:--- " + phoneNumber + "\nUser Name:--- "
+ name);
sb.append("\n----------------------------------");
}
} while (cursor.moveToNext());
} catch (Exception e) {
e.printStackTrace();
} finally {
if (cursor != null) {
cursor.close();
}
}
textView.setText(sb); //As in output it shows one number showing two times
} catch (Exception e) {
e.printStackTrace();
}
}
private String getFormatedNumber(String number){
if(TextUtils.isEmpty(number))return null;
number = number.trim();
number = number.replaceAll("-", "");
number = number.replaceAll(" ", "");
return number;
}
After all discussion, I found the issue like the problem occur due to unicode character append in my string that is invisible while debugging but if we copied into notepad then it clearly visible. as like :
'\u202A\u202A+91922222222\u202A\u202C'
I have an Android application in which i am scanning PDF417 barcode image. After scanning the Barcode i am getting the result as below.
#
ANSI 636014040002DL00410477ZC05180089DLDAQD1234562 XYXYXYXYXYXYXYXYX
DCSLASTNAMEXYXYXYXYXYXYXYXYXXYXYXYXYXYXYXYX
DDEU
DACFIRSTXYXYXYXYXYXYXYXYXXYXYXYXYXYXYXYXXYX
DDFU
DADXYXYXYXYXYXYXYXYXXYXYXYXYXYXYXYXXYXYXYXY
DDGU
DCAA XYXY
DCBNONEY1XY1XY1
DCDNONEX
DBD10312009
DBB10311977
DBA10312014
DBC1
DAU068 IN
DAYBRO
DAG1234 ANY STREET XY1XY1XY1XY1XY1XY1X
DAICITY XY1XY1XY1XY1XY1
DAJCA
DAK000000000
DCF00/00/0000NNNAN/ANFD/YY X
DCGUSA
DCUSUFIX
DAW150
DAZBLK XY1XY1XY
DCKXY1XY1XY1XY1XY1XY1XY1XY1X
DDAF
DDBMMDDCCYY
DDD1
ZCZCAY
ZCBCORR LENS
ZCCBRN
ZCDXYX
ZCEXYXYXYXYXYXYXY
ZCFXY1XY1XY1XY1XY1XY1XYXYXYXYXYXYXY
I want to get details like FirstName, LastName, City, Address etc from the above String.
Can anyone please tell me how do i get the details.
Thanks.
Please see below link and generate the parser to extract the information of driver License.
http://www.dol.wa.gov/external/docs/barcodeCalibration-EDLEID.pdf
I have make this decoder for ios app
here the code :
NSString *message=barcode.barcodeString;
NSMutableArray *arrFixedData=[[NSMutableArray alloc]initWithObjects:#"DCS",#"DCT",#"DCU",#"DAG",#"DAI",#"DAJ",#"DAK",#"DCG",#"DAQ",#"DCA",#"DCB",#"DCD",#"DCF",#"DCH",#"DBA",#"DBB",#"DBC",#"DBD",#"DAU",#"DCE",#"DAY",#"ZWA",#"ZWB",#"ZWC",#"ZWD",#"ZWE",#"ZWF", nil];
NSMutableArray *arrDriverData=[[NSMutableArray alloc]initWithObjects:#"Customer Family Name",#"Customer Given Name",#"Name Suffix",#"Street Address 1",#"City",#"Jurisdction Code",#"Postal Code",#"Country Identification",#"Customer Id Number",#"Class",#"Restrictions",#"Endorsements",#"Document Discriminator",#"Vehicle Code",#"Expiration Date",#"Date Of Birth",#"Sex",#"Issue Date",#"Height",#"Weight",#"Eye Color",#"Control Number",#"Endorsements",#"Transaction Types",#"Under 18 Until",#"Under 21 Until",#"Revision Date", nil];
NSMutableDictionary *dict=[[NSMutableDictionary alloc]init];
for (int i=0; i<[arrFixedData count]; i++)
{
NSRange range = [message rangeOfString: [arrFixedData objectAtIndex:i] options: NSCaseInsensitiveSearch];
NSLog(#"found: %#", (range.location != NSNotFound) ? #"Yes" : #"No");
if (range.location != NSNotFound)
{
NSString *temp=[message substringFromIndex:range.location+range.length];
NSRange end = [temp rangeOfString:#"\n"];
if (end.location != NSNotFound)
{
temp = [temp substringToIndex:end.location];
temp =[temp stringByReplacingOccurrencesOfString:#"\n" withString:#""];
temp=[temp stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]];
}
NSLog(#"temp data : %#",temp);
[dict setObject:temp forKey:[arrDriverData objectAtIndex:i]];
}
}
NSLog(#"Dictionary : %#",dict);
The barcodestring contains the data which are scanned from pdf 417.
Thanks
Here's the Decoder for Android
Here, The Parameter "data" contains the string which you have to scanned pdf417.
==========Properties=============
HashMap<String, String> myData = new HashMap<String, String>();
public final String Customer_Family_Name = "DCS", Customer_Given_Name = "DCT", Name_Suffix = "DCU",
Street_Address_1 = "DAG", City = "DAI", Jurisdction_Code = "DAJ", Postal_Code = "DAK",
Country_Identification = "DCG", Customer_Id_Number = "DAQ", Class = "DCA", Restrictions = "DCB",
Endorsements = "DCD", Document_Discriminator = "DCF", Vehicle_Code = "DCH", Expiration_Date = "DBA",
Date_Of_Birth = "DBB", Sex = "DBC", Issue_Date = "DBD", Height = "DAU", Weight = "DCE", Eye_Color = "DAY",
Control_Number = "ZWA", WA_SPECIFIC_ENDORSMENT = "ZWB", Transaction_Types = "ZWC", Under_18_Until = "ZWD",
Under_21_Until = "ZWE", Revision_Date = "ZWF", Customer_Full_Name = "DAA", Customer_First_Name = "DAC",
Customer_Middle_Name = "DAD", Street_Address_2 = "DAH", Street_Address_1_optional = "DAL",
Street_Address_2_optional = "DAM";
ArrayList<String> allKeys = new ArrayList<String>();
============Methods after Scaning================
#Override
protected void onActivityResult(int requestCode, int resultCode, Intent data)
{
if (requestCode == SCAN_REQUEST_CODE && resultCode == Activity.RESULT_OK)
{
ArrayList<BarcodeResult> barcodes = data.getParcelableArrayListExtra(BarcodeScanActivity.RESULT_EXTRA);
Log.e("BARCODE RESULT ", "<<<>>" + barcodes.toString());
String barcodeResult = barcodes.get(0).barcodeString;
String lines[] = barcodeResult.split("\\r?\\n");
for (int i = 0; i < lines.length; i++)
{
String str = lines[i];
if (str.contains("ANSI"))
{
str = str.substring(str.indexOf("DL"));
String str1[] = str.split("DL");
if (str1.length > 1)
{
str = str1[str1.length - 1];
}
}
if (str.length() > 3)
{
String key = str.substring(0, 3);
String value = str.substring(3);
if (allKeys.contains(key))
{
if (!value.equalsIgnoreCase("None"))
{
myData.put(allKeys.get(allKeys.indexOf(key)), value);
}
}
}
Log.e("RESULT ", "<<>>" + lines[i]);
}
Log.e("TAG", "SO MAY BE FINAL RESULT");
if (myData.containsKey(Customer_Family_Name))
{
Log.v("TAG", "users family name:" + myData.get(Customer_Family_Name));
lname = myData.get(Customer_Family_Name).trim();
}
if (myData.containsKey(Customer_Given_Name))
{
Log.v("TAG", "users Given name:" + myData.get(Customer_Given_Name));
try
{
String CustomerName[] = myData.get(Customer_Given_Name).split(" ");
fname = CustomerName[0].trim();
mname = CustomerName[1].substring(0, 1).trim();
}
catch (Exception e)
{
e.printStackTrace();
}
}
if (myData.containsKey(Name_Suffix))
{
Log.v("TAG", "Surname name:" + myData.get(Name_Suffix));
}
if (myData.containsKey(Street_Address_1))
{
Log.v("TAG", "Address line 1 :" + myData.get(Street_Address_1));
try
{
address = myData.get(Street_Address_1).trim();
}
catch (Exception e)
{
e.printStackTrace();
}
}
if (TextUtils.isEmpty(address))
{
if (myData.containsKey(Street_Address_2))
{
address = myData.get(Street_Address_2).trim();
}
if (TextUtils.isEmpty(address))
{
if (myData.containsKey(Street_Address_1_optional))
{
address = myData.get(Street_Address_1_optional).trim();
}
}
if (TextUtils.isEmpty(address))
{
if (myData.containsKey(Street_Address_2_optional))
{
address = myData.get(Street_Address_2_optional).trim();
}
}
}
if (myData.containsKey(City))
{
Log.v("TAG", "City:" + myData.get(City));
city = myData.get(City).trim();
}
if (myData.containsKey(Jurisdction_Code))
{
Log.v("TAG", "State:" + myData.get(Jurisdction_Code));
State = myData.get(Jurisdction_Code).trim();
}
if (myData.containsKey(Postal_Code))
{
Log.v("TAG", "Pin Code:" + myData.get(Postal_Code));
zipcode = myData.get(Postal_Code).substring(0, 5).trim();
}
if (myData.containsKey(Date_Of_Birth))
{
Log.v("TAG", "Birth Date :" + myData.get(Date_Of_Birth));
birthday = myData.get(Date_Of_Birth).substring(0, 2) + "/" + myData.get(Date_Of_Birth).substring(2, 4)
+ "/" + myData.get(Date_Of_Birth).substring(4);
if (isThisDateValid(birthday, "MM/dd/yyyy", myData.get(Date_Of_Birth)))
Log.e("TAG", "IS VALID");
else
Log.e("TAG", "IS NOT VALID");
}
if (myData.containsKey(Sex))
{
Log.v("TAG", "Sex:" + (myData.get(Sex).toString().trim().equals("1") ? "Male" : "Female"));
}
if (myData.containsKey(Customer_Full_Name))
{
String cName = myData.get(Customer_Full_Name);
int startIndexOfComma = 0;
int endIndexOfComma = 0;
startIndexOfComma = cName.indexOf(",");
endIndexOfComma = cName.lastIndexOf(",");
if (startIndexOfComma != endIndexOfComma)
{
String CustomerName[] = myData.get(Customer_Full_Name).split(",");
lname = CustomerName[0].replace(",", "").trim();
fname = CustomerName[1].trim();
mname = CustomerName[2].substring(0, 1).trim();
}
else
{
String CustomerName[] = myData.get(Customer_Full_Name).split(" ");
lname = CustomerName[0].replace(",", "").trim();
fname = CustomerName[1].trim();
mname = CustomerName[2].substring(0, 1).trim();
}
}
if (myData.containsKey(Customer_First_Name))
{
fname = myData.get(Customer_First_Name).trim();
}
if (myData.containsKey(Customer_Middle_Name))
{
mname = myData.get(Customer_Middle_Name).substring(0, 1).trim();
}
// TODO edit here at 7/3/2014
if (myData.containsKey(Customer_Id_Number))
{
licence_number = myData.get(Customer_Id_Number).trim();
Log.e("TAG", "Licence Number is :" + licence_number);
}
if (myData.containsKey(Expiration_Date))
{
licence_expire_date = myData.get(Expiration_Date).trim();
licence_expire_date = myData.get(Expiration_Date).substring(0, 2) + "/"
+ myData.get(Expiration_Date).substring(2, 4) + "/" + myData.get(Expiration_Date).substring(4);
licence_expire_date = makeDateValid(licence_expire_date, "MM/dd/yyyy", myData.get(Expiration_Date));
Log.e("TAG", "expire date is :" + licence_expire_date);
}
etFirstName.setText(fname.trim());
etMiddleName.setText(mname.trim());
etLastName.setText(lname.trim());
etAddress.setText(address.trim());
etZipCode.setText(zipcode.trim());
etCity.setText(city.trim());
etState.setText(State.trim());
etDLNumber.setText(licence_number);
etDLExpirationDate.setText(licence_expire_date);
etBirthDay.setText(birthday.trim());
}
}
Here's the code I use to decode the PDF417 data in Swift.
private let pdf417Map: [String: String] = [
"DAA":"Full Name",
"DAB":"Family Name",
"DAC":"Given Name",
"DAD":"Middle Name",
"DAE":"Name Suffix",
"DAF":"Name Prefix",
"DAG":"Mailing Street Address1",
"DAH":"Mailing Street Address2",
"DAI":"Mailing City",
"DAJ":"Mailing Jurisdiction Code",
"DAK":"Mailing Postal Code",
"DAL":"Residence Street Address1",
"DAM":"Residence Street Address2",
"DAN":"Residence City",
"DAO":"Residence Jurisdiction Code",
"DAP":"Residence Postal Code",
"DAQ":"License or ID Number",
"DAR":"License Classification Code",
"DAS":"License Restriction Code",
"DAT":"License Endorsements Code",
"DAU":"Height in FT_IN",
"DAV":"Height in CM",
"DAW":"Weight in LBS",
"DAX":"Weight in KG",
"DAY":"Eye Color",
"DAZ":"Hair Color",
"DBA":"License Expiration Date",
"DBB":"Date of Birth",
"DBC":"Sex",
"DBD":"License or ID Document Issue Date",
"DBE":"Issue Timestamp",
"DBF":"Number of Duplicates",
"DBG":"Medical Indicator Codes",
"DBH":"Organ Donor",
"DBI":"Non-Resident Indicator",
"DBJ":"Unique Customer Identifier",
"DBK":"Social Security Number",
"DBL":"Date Of Birth",
"DBM":"Social Security Number",
"DBN":"Full Name",
"DBO":"Family Name",
"DBP":"Given Name",
"DBQ":"Middle Name or Initial",
"DBR":"Suffix",
"DBS":"Prefix",
"DCA":"Virginia Specific Class",
"DCB":"Virginia Specific Restrictions",
"DCD":"Virginia Specific Endorsements",
"DCE":"Physical Description Weight Range",
"DCF":"Document Discriminator",
"DCG":"Country territory of issuance",
"DCH":"Federal Commercial Vehicle Codes",
"DCI":"Place of birth",
"DCJ":"Audit information",
"DCK":"Inventory Control Number",
"DCL":"Race Ethnicity",
"DCM":"Standard vehicle classification",
"DCN":"Standard endorsement code",
"DCO":"Standard restriction code",
"DCP":"Jurisdiction specific vehicle classification description",
"DCQ":"Jurisdiction-specific",
"DCR":"Jurisdiction specific restriction code description",
"DCS":"Last Name",
"DCT":"First Name",
"DCU":"Suffix",
"DDA":"Compliance Type",
"DDB":"Card Revision Date",
"DDC":"HazMat Endorsement Expiry Date",
"DDD":"Limited Duration Document Indicator",
"DDE":"Family Name Truncation",
"DDF":"First Names Truncation",
"DDG":"Middle Names Truncation",
"DDH":"Under 18 Until",
"DDI":"Under 19 Until",
"DDJ":"Under 21 Until",
"DDK":"Organ Donor Indicator",
"DDL":"Veteran Indicator",
"PAA":"Permit Classification Code",
"PAB":"Permit Expiration Date",
"PAC":"Permit Identifier",
"PAD":"Permit IssueDate",
"PAE":"Permit Restriction Code",
"PAF":"Permit Endorsement Code",
"ZVA":"Court Restriction Code"
]
// metadataObj = # ANSI 63601404....
if metadataObj.stringValue != nil {
let licenseData = metadataObj.stringValue!.components(separatedBy: "\n")
var customerProfile: [[String: String]] = []
for item in licenseData {
var metaDataItem = item
if metaDataItem.count > 3 {
if let dlCodeRange = metaDataItem.range(of: "DAQ") {
let dlStart = dlCodeRange.lowerBound
let dlEnd = metaDataItem.index(metaDataItem.endIndex, offsetBy: 0)
let dlNoRange = dlStart..<dlEnd
metaDataItem = String(metaDataItem[dlNoRange])
}
// Get the 3 letter code
let pdf417Code = String(metaDataItem.prefix(3))
// See if the code exists in the map
if pdf417Map[pdf417Code] != nil {
// Code exists in map, save to profile
let start = metaDataItem.index(metaDataItem.startIndex, offsetBy: 3)
let end = metaDataItem.index(metaDataItem.endIndex, offsetBy: 0)
let range = start..<end
let extractedData = metaDataItem[range]
customerProfile.append([pdf417Map[pdf417Code]!: String(extractedData)])
}
}
}
print("customerProfile: \(customerProfile)")
}
Please look into this Link having decoder for driver license in Java. It might help.