I have tried many sample codes to parse APDU response to TLV format.
I am able to parse it properly if the response length is less but facing issue if length is more(how calculate length of a tag without any libraries)
NOTE: I am using predefined tags in Constants
code:
private HashMap<String, String> parseTLV(String apduResponse) {
HashMap<String, String> tagValue = new HashMap<>();
String remainingApdu = apduResponse.replaceAll(" ", "");
if (remainingApdu.endsWith(ResponseTags._SUCCESS_STATUSWORDS)) {
remainingApdu = remainingApdu.substring(0, remainingApdu.length() - 4);
}
while (remainingApdu != null && remainingApdu.length() > 2) {
remainingApdu = addTagValue(tagValue, remainingApdu);
}
return tagValue;
}
addTagValue method
private String addTagValue(HashMap<String, String> tagValue, String apduResponse) {
String tag = "";
String length = "";
String value = "";
int tagLen = 0;
if (tagUtils.isValidTag(apduResponse.substring(0, 2))) {
tagLen = readTagLength(apduResponse.substring(3));
// tagLen = 2;
tag = apduResponse.substring(0, 2);
} else if (tagUtils.isValidTag(apduResponse.substring(0, 4))) {
tagLen = 4;
tag = apduResponse.substring(0, 4);
} else {
return "";
}
Log.e("TAG_LEN","tag: "+tag+"taglen: "+tagLen);
if (tagUtils.shouldCheckValueFor(tag)) {
length = apduResponse.substring(tagLen, tagLen + 2);
int len = tagUtils.hexToDecimal(length);
value = apduResponse.substring(tagLen + 2, (len * 2) + tagLen + 2);
tagValue.put(tag, value);
if (ResponseTags.getRespTagsmap().containsKey(tag)) {
//logData = logData + "\nKEY:" + tag + " TAG:" + ResponseTags.getRespTagsmap().get(tag)/* + " VALUE:" + value + "\n "*/;
}
if (tagUtils.isTemplateTag(tag)) {
// logData = logData + "\n\t-->";
return addTagValue(tagValue, value) + apduResponse.substring(tag.length() + value.length() + length.length());
} else {
return apduResponse.substring(tag.length() + value.length() + length.length());
}
} else {
value = apduResponse.substring(2, 4);
tagValue.put(tag, value);
// logData = logData + "\n\t\tKEY:" + tag + " TAG:" + ResponseTags.getRespTagsmap().get(tag) /*+ " VALUE:" + value + "\n "*/;
return apduResponse.substring(tag.length() + value.length() + length.length());
}
}
readTagLength :
private int readTagLength(String apduResponse) {
int len_bytes = 0;
if (apduResponse.length() > 2) {
len_bytes = (apduResponse.length()) / 2;
}
Log.e("tlv length:", "bytes:" + len_bytes);
if (len_bytes < 128) {
return 2;
} else if (len_bytes > 127 && len_bytes < 255) {
return 4;
} else {
return 6;
}
}
I cannot able to get length properly for few cards(if apdu response is long)
Please help
First be sure the input data is proper before you go into the code. Take the full data and try it on https://www.emvlab.org/tlvutils/ .
Once its confirmed the data is proper, go through in EMV 4.3 Book 3,
Annex B Rules for BER-TLV Data Objects sections B1, B2, B3 - with utmost attention.
If you follow this precisely, then you wouldn't need to store a static list of tags; will save time in future.
Below sample has an assumption that TLV array is ending with special 0x00 tag but for sure you can ignore it.
Pojo class :
public class Tlv {
private short tag;
private byte[] value;
public Tlv(short tag) {
this.tag = tag;
}
public short getTag() {
return tag;
}
public byte[] getValue() {
return value;
}
public void setValue(byte[] valueBytes) {
this.value = valueBytes;
}
}
Utility method :
public static Map<Byte, Tlv> parse(ByteBuffer bb) throws TlvException {
Map<Byte, Tlv> tlvs = null;
tlvs = new HashMap<Byte, Tlv>();
try {
while (bb.remaining() > 0) {
byte tag = bb.get();
if(tag == 0x00)
continue;
int length = bb.get();
byte[] value = new byte[length];
bb.get(value, 0, length);
Tlv tlv = new Tlv(tag);
tlv.setValue(value);
tlvs.put(tag, tlv);
}
} catch (IndexOutOfBoundsException e) {
throw new TlvException("Malformed TLV part: " + bb.toString() + ".", e);
}
return tlvs;
}
Related
I have implemented this java app communicating with a remote 8 modem server and when I request for an image from a security camera (error free or not) the outcome is a 1 byte image (basically a small white square). Can you help me find the error?
Here is my code:
import java.io.*;
import java.util.Scanner;
import ithakimodem.Modem;
public class g {
private static Scanner scanner = new Scanner(System.in);
private static String EchoCode = "E3369";
private static String noImageErrorscode = "M2269";
private static String imageErrorsCode = "G6637";
private static String GPS_Code = "P7302";
private static String ACK_Code = "Q2591";
private static String NACK_Code = "R4510";
public static void main(String[] args) throws IOException, InterruptedException {
int timeout = 2000;
int speed = 80000;
Modem modem = new Modem(speed);
modem.setTimeout(timeout);
modem.write("ATD2310ITHAKI\r".getBytes());
getConsole(modem);
modem.write(("test\r").getBytes());
getConsole(modem);
while (true) {
System.out.println("\nChoose one of the options below :");
System.out.print("Option 0: Exit\n" + "Option 1: Echo packages\n" + "Option 2: Image with no errors\n" + "Option 3: Image with errors\n" + "Option 4: Tracks of GPS\n"
+ "Option 5: ARQ\n" );
System.out.print("Insert option : ");
int option = scanner.nextInt();
System.out.println("");
switch (option) {
case 0: {
// Exit
System.out.println("\nExiting...Goodbye stranger");
modem.close();
scanner.close();
return;
}
case 1: {
// Echo
getEcho(modem, EchoCode);
break;
}
case 2: {
// Image with no errors
getImage(modem, noImageErrorscode, "no_error_image.jpg");
break;
}
case 3: {
// Image with errors
getImage(modem, imageErrorsCode, "image_with_errors.jpg");
}
case 4: {
// GPS
getGPS(modem, GPS_Code);
break;
}
case 5: {
// ARQ
getARQ(modem, ACK_Code, NACK_Code);
break;
}
default:
System.out.println("Try again please\n");
}
}
}
public static String getConsole(Modem modem) {
int l;
StringBuilder stringBuilder= new StringBuilder();
String string = null;
while (true) {
try {
l = modem.read();
if (l == -1) {
break;
}
System.out.print((char) l);
stringBuilder.append((char) l);
string = stringBuilder.toString();
} catch (Exception exc) {
break;
}
}
System.out.println("");
return string;
}
private static void getImage(Modem modem, String password, String fileName) throws IOException {
int l;
boolean flag;
FileOutputStream writer = new FileOutputStream(fileName, false);
flag = modem.write((password + "\r").getBytes());
System.out.println("\nReceiving " + fileName + "...");
if (!flag) {
System.out.println("Code error or end of connection");
writer.close();
return;
}
while (true) {
try {
l = modem.read();
writer.write((char) l);
writer.flush();
if (l == -1) {
System.out.println("END");
break;
}
}
catch (Exception exc) {
break;
}
}
writer.close();
}
private static void getGPS(Modem modem, String password) throws IOException {
int rows = 99;
String Rcode = "";
Rcode = password + "R=10200" + rows;
System.out.println("Executing R parameter = " + Rcode + " (XPPPPLL)");
modem.write((Rcode + "\r").getBytes());
String stringConsole = getConsole(modem);
if (stringConsole == null) {
System.out.println("Code error or end of connection");
return;
}
String[] stringRows = stringConsole.split("\r\n");
if (stringRows[0].equals("n.a")) {
System.out.println("Error : Packages not received");
return;
}
System.out.println("**TRACES**\n");
float l1 = 0, l2 = 0;
int difference = 8;
difference *= 100 / 60;
int traceNumber = 7;
String[] traces = new String[traceNumber + 1];
int tracesCounter = 0, flag = 0;
for (int i = 0; i < rows; i++) {
if (stringRows[i].startsWith("$GPGGA")) {
if (flag == 0) {
String str = stringRows[i].split(",")[1];
l1 = Integer.valueOf(str.substring(0, 6)) * 100 / 60;
flag = 1;
}
String str = stringRows[i].split(",")[1];
l2 = Integer.valueOf(str.substring(0, 6)) * 100 / 60;
if (Math.abs(l2 - l1) >= difference) {
traces[tracesCounter] = stringRows[i];
if (tracesCounter == traceNumber)
break;
tracesCounter++;
l1 = l2;
}
}
}
for (int i = 0; i < traceNumber; i++) {
System.out.println(traces[i]);
}
String w = "", T_cd_fnl = password + "T=";
int p = 1;
System.out.println();
for (int i = 0; i < traceNumber; i++) {
String[] strSplit = traces[i].split(",");
System.out.print("T parameter = ");
String str1 = strSplit[4].substring(1, 3);
String str2 = strSplit[4].substring(3, 5);
String str3= String.valueOf(Integer.parseInt(strSplit[4].substring(6, 10)) * 60 / 100).substring(0, 2);
String str4= strSplit[2].substring(0, 2);
String str5= strSplit[2].substring(2, 4);
String str6= String.valueOf(Integer.parseInt(strSplit[2].substring(5, 9)) * 60 / 100).substring(0, 2);
w = str1 + str2 + str3 + str4 + str5 + str6 + "T";
p = p + 5;
System.out.println(w);
T_cd_fnl = T_cd_fnl + w + "=";
}
T_cd_fnl = T_cd_fnl.substring(0, T_cd_fnl.length() - 2);
System.out.println("\nSending code: " + T_cd_fnl);
getImage(modem, T_cd_fnl, "traces GPS.jpg");
}
private static void getEcho(Modem modem, String strl) throws InterruptedException, IOException {
FileOutputStream echo_time_writer = new FileOutputStream("echoTimes.txt", false);
FileOutputStream echo_counter_writer = new FileOutputStream("echoCounter.txt", false);
int h;
int runtime = 5, packetCounter = 0;
String str = "";
System.out.println("\nRuntime (mins) = " + runtime + "\n");
long start_time = System.currentTimeMillis();
long stop_time = start_time + 60 * 1000 * runtime;
long send_time = 0, receiveTime = 0;
String time = "", ClockTime = "";
echo_time_writer.write("Clock Time\tSystem Time\r\n".getBytes());
while (System.currentTimeMillis() <= stop_time) {
packetCounter++;
send_time = System.currentTimeMillis();
modem.write((strl + "\r").getBytes());
while (true) {
try {
h = modem.read();
System.out.print((char) h);
str += (char) h;
if ( h== -1) {
System.out.println("\nCode error or end of connection");
return;
}
if (str.endsWith("PSTOP")) {
receiveTime = System.currentTimeMillis();
ClockTime = str.substring(18, 26) + "\t";
time = String.valueOf((receiveTime - send_time) + "\r\n");
echo_time_writer.write(ClockTime.getBytes());
echo_time_writer.write(time.getBytes());
echo_time_writer.flush();
str = "";
break;
}
} catch (Exception e) {
break;
}
}
System.out.println("");
}
echo_counter_writer.write(("\r\nRuntime: " + String.valueOf(runtime)).getBytes());
echo_counter_writer.write(("\r\nPackets Received: " + String.valueOf(packetCounter)).getBytes());
echo_counter_writer.close();
echo_time_writer.close();
}
private static void getARQ(Modem modem, String strl, String nack_code) throws IOException, InterruptedException {
FileOutputStream arq_time_writer = new FileOutputStream("ARQtimes.txt", false);
FileOutputStream arq_counter_writer = new FileOutputStream("ARQcounter.txt", false);
int runtime = 5;
int xor = 1;
int f = 1;
int m;
int packageNumber = 0;
int iterationNumber = 0;
int[] nack_times_counter = new int[15];
int nack_to_package = 0;
String time = "", clock_time = "", s = "";
long start_time = System.currentTimeMillis();
long stop_time = start_time + 60 * 1000 * runtime;
long send_time = 0, receive_time = 0;
System.out.printf("Runtime (mins) = " + runtime + "\n");
arq_time_writer.write("Clock Time\tSystem Time\tPacket Resends\r\n".getBytes());
while (System.currentTimeMillis() <= stop_time) {
if (xor == f) {
packageNumber++;
nack_times_counter[nack_to_package]++;
nack_to_package = 0;
send_time = System.currentTimeMillis();
modem.write((strl + "\r").getBytes());
} else {
iterationNumber++;
nack_to_package++;
modem.write((nack_code + "\r").getBytes());
}
while (true) {
try {
m = modem.read();
System.out.print((char) m);
s += (char) m;
if (m == -1) {
System.out.println("\nCode error or end of connection");
return;
}
if (s.endsWith("PSTOP")) {
receive_time = System.currentTimeMillis();
break;
}
} catch (Exception e) {
break;
}
}
System.out.println("");
String[] string = s.split("<");
string = string[1].split(">");
f = Integer.parseInt(string[1].substring(1, 4));
xor = string[0].charAt(0) ^ string[0].charAt(1);
for (int i = 2; i < 16; i++) {
xor = xor ^ string[0].charAt(i);
}
if (xor == f) {
System.out.println("Packet ok");
receive_time = System.currentTimeMillis();
time = String.valueOf((receive_time - send_time) + "\t");
clock_time = s.substring(18, 26) + "\t";
arq_time_writer.write(clock_time.getBytes());
arq_time_writer.write(time.getBytes());
arq_time_writer.write((String.valueOf(nack_to_package) + "\r\n").getBytes());
arq_time_writer.flush();
} else {
xor = 0;
}
s = "";
}
arq_counter_writer.write(("\r\nRuntime: " + String.valueOf(runtime)).getBytes());
arq_counter_writer.write("\r\nPackets Received (ACK): ".getBytes());
arq_counter_writer.write(String.valueOf(packageNumber).getBytes());
arq_counter_writer.write("\r\nPackets Resent (NACK): ".getBytes());
arq_counter_writer.write(String.valueOf(iterationNumber).getBytes());
arq_counter_writer.write("\r\nNACK Time Details".getBytes());
for (int i = 0; i < nack_times_counter.length; i++) {
arq_counter_writer.write(("\r\n" + i + ":\t" + nack_times_counter[i]).getBytes());
}
arq_counter_writer.close();
arq_counter_writer.close();
System.out.println("Packets Received: " + packageNumber);
System.out.println("Packets Resent: " + iterationNumber);
System.out.println("\n\nFile arqTimes.txt is created.");
System.out.println("File arqCounter.txt is created.");
}
}
I know the problem is most probably in the getImage() function but I haven't figured it out yet.
Official Document for How to Read QR code data from Secure QR code of UIDAI Aadhaar card: https://uidai.gov.in/images/resource/User_manulal_QR_Code_15032019.pdf
Have used zxing scanner to scan Secure QR code,
And was able to get details of aadhar card with the help of follow project in Github:
https://github.com/dimagi/AadharUID
Some how I figured out how to extract photo and bit_indicator_value from converted byte array with the help of instructions on document
But I am unable to get hashed exact hashed value of email and mobile from Secure QR code of Aadhar card byte array to verify
I am bit confused from this line of above mentioned document in page no 6:
Post getting signature value, check the value of
Email_mobile_present_bit_indicator_value:
if its 3 then first read mobile from index (Byte array length – 1-256) and then email from index (Byte array length – 1- 256- 32) in
reverse order. Each value will be of fix size of 32 byte.
If Email_mobile_present_bit_indicator_value is 1 then only mobile
is present.
If Email_mobile_present_bit_indicator_value is 2 then only email is
present.
If Email_mobile_present_bit_indicator_value is 0 then no mobile or
email present.
Email and Mobile value will available in byte. Convert into Hexadecimal
String.
.
I have also prepared last step in document page no 6 but the users mobile/email hash value and documents mobile/email byte arrays hash value never matches
Code snippet:
public ScanResult(String input) {
rawString = input;
// copied from http://www.java-samples.com/showtutorial.php?tutorialid=152
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
Document dom;
try {
//Using factory get an instance of document builder
DocumentBuilder db = dbf.newDocumentBuilder();
// Replace </?xml... with <?xml...
if (input.startsWith("</?")) {
input = input.replaceFirst("</\\?", "<?");
}
// Replace <?xml...?"> with <?xml..."?>
input = input.replaceFirst("^<\\?xml ([^>]+)\\?\">", "<?xml $1\"?>");
//parse using builder to get DOM representation of the XML file
dom = db.parse(new ByteArrayInputStream(input.getBytes("UTF-8")));
} catch (ParserConfigurationException | SAXException | IOException e) {
dom = null;
}
if (rawString.matches("[0-9]*")) {
type = QR_CODE_TYPE_SECURE;
byte[] msgInBytes = null;
try {
msgInBytes = decompressByteArray(new BigInteger(rawString).toByteArray());
} catch (IOException e) {
e.printStackTrace();
}
if (msgInBytes != null) {
int[] delimiters = locateDelimiters(msgInBytes);
String referenceId = getValueInRange(msgInBytes, delimiters[0] + 1, delimiters[1]);
uid = referenceId.substring(0, 4);
name = getValueInRange(msgInBytes, delimiters[1] + 1, delimiters[2]);
dob = formatDate(getValueInRange(msgInBytes, delimiters[2] + 1, delimiters[3]),
new String[] {"dd-MM-yyyy", "dd/MM/yyyy"});
yob = dob.substring(0, 4);
gender = getValueInRange(msgInBytes, delimiters[3] + 1, delimiters[4]);
co = getValueInRange(msgInBytes, delimiters[4] + 1, delimiters[5]);
dist = getValueInRange(msgInBytes, delimiters[5] + 1, delimiters[6]);
lm = getValueInRange(msgInBytes, delimiters[6] + 1, delimiters[7]);
house = getValueInRange(msgInBytes, delimiters[7] + 1, delimiters[8]);
loc = getValueInRange(msgInBytes, delimiters[8] + 1, delimiters[9]);
pc = getValueInRange(msgInBytes, delimiters[9] + 1, delimiters[10]);
po = getValueInRange(msgInBytes, delimiters[10] + 1, delimiters[11]);
state = getValueInRange(msgInBytes, delimiters[11] + 1, delimiters[12]);
street = getValueInRange(msgInBytes, delimiters[12] + 1, delimiters[13]);
subdist = getValueInRange(msgInBytes, delimiters[13] + 1, delimiters[14]);
vtc = getValueInRange(msgInBytes, delimiters[14] + 1, delimiters[15]);
statusCode = STATUS_SUCCESS;
} else {
statusCode = STATUS_PARSE_ERROR;
uid = "";
name = "";
gender = "";
yob = "";
co = "";
house = "";
street = "";
lm = "";
loc = "";
vtc = "";
po = "";
dist = "";
subdist = "";
state = "";
pc = "";
dob = "";
}
} else {
type = QR_CODE_TYPE_UNKNOWN;
statusCode = STATUS_PARSE_ERROR;
uid = "";
name = "";
gender = "";
yob = "";
co = "";
house = "";
street = "";
lm = "";
loc = "";
vtc = "";
po = "";
dist = "";
subdist = "";
state = "";
pc = "";
dob = "";
}
dobGuess = getDobGuess(dob, yob);
statusText = getStatusText(statusCode);
}
private static int[] locateDelimiters(byte[] msgInBytes) {
int[] delimiters = new int[NUMBER_OF_PARAMS_IN_SECURE_QR_CODE + 1];
int index = 0;
int delimiterIndex;
for (int i = 0; i <= NUMBER_OF_PARAMS_IN_SECURE_QR_CODE; i++) {
delimiterIndex = getNextDelimiterIndex(msgInBytes, index);
delimiters[i] = delimiterIndex;
index = delimiterIndex + 1;
}
return delimiters;
}
private static String getValueInRange(byte[] msgInBytes, int start, int end) {
return new String(Arrays.copyOfRange(msgInBytes, start, end), Charset.forName("ISO-8859-1"));
}
private static int getNextDelimiterIndex(byte[] msgInBytes, int index) {
int i = index;
for (; i < msgInBytes.length; i++) {
if (msgInBytes[i] == -1) {
break;
}
}
return i;
}
private static byte[] decompressByteArray(byte[] bytes) throws IOException {
java.io.ByteArrayInputStream bytein = new ByteArrayInputStream(bytes);
java.util.zip.GZIPInputStream gzin = new GZIPInputStream(bytein);
java.io.ByteArrayOutputStream byteout = new ByteArrayOutputStream();
int res = 0;
byte buf[] = new byte[1024];
while (res >= 0) {
res = gzin.read(buf, 0, buf.length);
if (res > 0) {
byteout.write(buf, 0, res);
}
}
return byteout.toByteArray();
}
private String formatDate(String rawDateString, String[] possibleFormats) {
if (rawDateString.equals("")) {
return "";
}
SimpleDateFormat toFormat = new SimpleDateFormat("yyyy-MM-dd");
Date date = null;
ParseException parseException = null;
for (String fromFormatPattern : possibleFormats) {
try {
SimpleDateFormat fromFormat = new SimpleDateFormat(fromFormatPattern);
date = fromFormat.parse(rawDateString);
break;
} catch (ParseException e) {
parseException = e;
}
}
if (date != null) {
return toFormat.format(date);
} else if (parseException != null) {
System.err.println("Expected dob to be in dd/mm/yyyy or yyyy-mm-dd format, got " + rawDateString);
return rawDateString;
} else {
throw new AssertionError("This code is unreachable");
}
}
#NonNull
protected String formatGender(String gender) throws ParseException {
String lowercaseGender = gender.toLowerCase();
if (lowercaseGender.equals("male") || lowercaseGender.equals("m")) {
return "M";
} else if (lowercaseGender.equals("female") || lowercaseGender.equals("f")) {
return "F";
} else if (lowercaseGender.equals("other") || lowercaseGender.equals("o")) {
return "O";
} else {
throw new ParseException("404 gender not found", 0);
}
}
#NonNull
private String getStatusText(int statusCode) {
switch (statusCode) {
case ScanResult.STATUS_SUCCESS:
return "✓";
default:
return "✗";
}
}
#NonNull
private String getDobGuess(String dob, String yob) {
if (dob.equals("")) {
Integer yearInt;
try {
yearInt = Integer.parseInt(yob);
} catch (NumberFormatException e) {
return "";
}
// June 1 of the year
return Integer.toString(yearInt) + "-06-01";
} else {
return dob;
}
}
Github Code Link
Just now got the solution, fetched exact mobile and email hash value from byte array
first remove last 256 bytes from byte array then,
if only mobile present then take last 32 bytes from byte array for mobile
i.e. (array.length - 32 to array.length)
else if only email present then take last 32 bytes from byte array for email i.e. (array.length - 32 to array.length)
else if both email and mobile present then take last 32 bytes from byte array for mobile, take next last 32 bytes for email
i.e. (mobile = array.length - 32 to array.length and
email = array.length - 64 to array.length - 32)
From the byte array you need to read values from index byte_array_length - 256 - 32 to byte_array_length - 256 to get the mobile hash in bytes(you can easily convert the bytes to hexadecimal string) and similarly for email you need to read from index byte_array_length - 256 - 32 - 32 to byte_array_length - 256 - 32.
I have implemented a library for decoding the same but the implementation is in Python.
The question is a bit old but future readers might find these resources useful.
PyPI: aadhaar-py
Github: https://github.com/vishaltanwar96/aadhaar-py
I have a file that contains more than one value in one column. I was trying to read this file using java with this code:
ArrayList<String> linesList1 = new ArrayList<>();
ArrayList<String> roadlinkid = new ArrayList<>();
ArrayList<String> road_name_orignal = new ArrayList<>();
ArrayList<String> road_name_copy = new ArrayList<>();
ArrayList<String[]> networkmember_href = new ArrayList<>();
ArrayList<String> road_fid = new ArrayList<>();
// Input of file which needs to be parsed
String csvFile1 = "RoadData.csv";
BufferedReader csvReader1;
// Data split by ',' in CSV file
String csvSplitBy = ",";
try {
String line;
csvReader1 = new BufferedReader(new FileReader(csvFile1));
while ((line = csvReader1.readLine()) !=null) {
linesList1.add(line);
}
csvReader1.close();
}
catch (IOException e) { e.printStackTrace(); }
for (int i = 0; i < linesList1.size(); i++) {
String[] data = linesList1.get(i).split(csvSplitBy);
road_fid.add( data[1]);
road_name_orignal.add( data[9]);
if (data[9].contains("{")) {
String[] xy = data[9].replaceAll("\\{|\\}", "").split(",");
int leng = xy.length;
String[] networkmember = new String [leng];
for ( int n = 0 ; n < leng ; n++) {
networkmember[n] = xy [n];
}
networkmember_href.add(networkmember);
}
}
This code works well, but the problem is that the code deals with each value in the column as a separate column. Therefore, it returns wrong data.
Files:
http://s000.tinyupload.com/?file_id=47090134488569683648
The idea is Finding the road name from RoadData.csv and write it in RoadLink.csv by comparing road_fid in RoadData.csv and roadlink_fid in RoadLink.csv. Unfortunately, I could find a way to deal with a column with multi-values. Any advice, please.
Thanks in advance.
Below is some code to parse the file, you can add additional processing to parse the fields that have lists in them or to combine the lists like changedate and reasonforchange into a list of Objects containing both pieces of data. For example a List<ChangeInfo> where ChangeInfo holds both the changedate and reasonforchange.
I still would recommend using a csv parser but this code should work well enough for this specific use case. Test thoroughly..
Main:
public static void main(String[] args){
List<RoadLinkRecord> records = parse("path\\to\\RoadLink.csv");
// display all the records
for (RoadLinkRecord record : records) {
System.out.println(record);
}
}
CSV Parsing:
private static final Pattern csvFieldPattern =
Pattern.compile("(?<=[$,])(\"(\"\"|[^\"])*\"|[^,]*)");
/** This parse method requires the CSV file to have a header row */
public static List<RoadLinkRecord> parse(String csvFilePath) {
// TODO accept Reader or maybe InputStream rather than file path
File f = new File(csvFilePath);
List<RoadLinkRecord> records = new ArrayList<>();
try (BufferedReader br = new BufferedReader(new FileReader(f));) {
// get the header fields
String line = br.readLine();
List<String> headers = new ArrayList<>();
{
Matcher matcher = csvFieldPattern.matcher(line);
while (matcher.find())
headers.add(matcher.group());
}
// iterate through record fields
int recordNum = 0;
while ((line = br.readLine()) != null) {
recordNum++;
// allocate array to hold the fields
String[] fields = new String[headers.size()];
// use matcher to get each of the fields
Matcher matcher = csvFieldPattern.matcher(line);
for (int i = 0; i < headers.size(); i++) {
if (!matcher.find()) {
throw new IllegalArgumentException(
"Couldn't find field '" + headers.get(i) + "' for record " + recordNum);
}
fields[i] = matcher.group();
}
if (matcher.find()) {
throw new IllegalArgumentException("Found excess fields in record " + recordNum);
}
// add the record from this line
records.add(new RoadLinkRecord(recordNum, fields));
}
} catch (IOException e) {
// TODO trouble reading the file
} catch (IllegalArgumentException e) {
// TODO error while parsing the file
}
return records;
}
Data Container:
public class RoadLinkRecord {
private final int recordNumber;
private final String roadlink_fid;
private final String version;
private final String versiondate;
private final String changedate;
private final String reasonforchange;
private final String descriptivegroup;
private final String descriptiveterm;
private final String natureofroad;
private final String length;
private final String directednode_href;
private final String directednode_orientation;
private final String directednode_gradeseparation;
private final String referencetotopographicarea_href;
private final String theme;
private final String filename;
private final String wkb_geometry;
private final String roadnumber;
private final String dftname;
private final String fid;
private final String roadname;
public RoadLinkRecord(final int recordNumber, final String[] csvFields) {
if (csvFields.length != 20) {
throw new IllegalArgumentException(
"Wrong number of fields for a RoadLinkRecord! Expected 20, found "
+ csvFields.length);
}
this.recordNumber = recordNumber;
this.roadlink_fid = processStringField(csvFields[0]);
this.version = processStringField(csvFields[1]);
this.versiondate = processStringField(csvFields[2]);
this.changedate = processStringField(csvFields[3]);
this.reasonforchange = processStringField(csvFields[4]);
this.descriptivegroup = processStringField(csvFields[5]);
this.descriptiveterm = processStringField(csvFields[6]);
this.natureofroad = processStringField(csvFields[7]);
this.length = processStringField(csvFields[8]);
this.directednode_href = processStringField(csvFields[9]);
this.directednode_orientation = processStringField(csvFields[10]);
this.directednode_gradeseparation = processStringField(csvFields[11]);
this.referencetotopographicarea_href = processStringField(csvFields[12]);
this.theme = processStringField(csvFields[13]);
this.filename = processStringField(csvFields[14]);
this.wkb_geometry = processStringField(csvFields[15]);
this.roadnumber = processStringField(csvFields[16]);
this.dftname = processStringField(csvFields[17]);
this.fid = processStringField(csvFields[18]);
this.roadname = processStringField(csvFields[19]);
}
private static String processStringField(String field) {
// consider empty fields as null
if (field.isEmpty()) {
return null;
}
// strip double quotes and replace any escaped quotes
final int endIndex = field.length() - 1;
if (field.charAt(0) == '"' && field.charAt(endIndex) == '"') {
return field.substring(1, endIndex).replace("\"\"", "\"");
}
return field;
}
public int getRecordNumber() { return recordNumber; }
public String getRoadlink_fid() { return roadlink_fid; }
public String getVersion() { return version; }
public String getVersiondate() { return versiondate; }
public String getChangedate() { return changedate; }
public String getReasonforchange() { return reasonforchange; }
public String getDescriptivegroup() { return descriptivegroup; }
public String getDescriptiveterm() { return descriptiveterm; }
public String getNatureofroad() { return natureofroad; }
public String getLength() { return length; }
public String getDirectednode_href() { return directednode_href; }
public String getDirectednode_orientation() { return directednode_orientation; }
public String getDirectednode_gradeseparation() { return directednode_gradeseparation; }
public String getReferencetotopographicarea_href() { return referencetotopographicarea_href; }
public String getTheme() { return theme; }
public String getFilename() { return filename; }
public String getWkb_geometry() { return wkb_geometry; }
public String getRoadnumber() { return roadnumber; }
public String getDftname() { return dftname; }
public String getFid() { return fid; }
public String getRoadname() { return roadname; }
#Override
public String toString() {
return "roadlink_fid= " + roadlink_fid + "; version= " + version + "; versiondate= "
+ versiondate + "; changedate= " + changedate + "; reasonforchange= "
+ reasonforchange + "; descriptivegroup= " + descriptivegroup + "; descriptiveterm= "
+ descriptiveterm + "; natureofroad= " + natureofroad + "; length= " + length
+ "; directednode_href= " + directednode_href + "; directednode_orientation= "
+ directednode_orientation + "; directednode_gradeseparation= "
+ directednode_gradeseparation + "; referencetotopographicarea_href= "
+ referencetotopographicarea_href + "; theme= " + theme + "; filename= " + filename
+ "; wkb_geometry= " + wkb_geometry + "; roadnumber= " + roadnumber + "; dftname= "
+ dftname + "; fid= " + fid + "; roadname= " + roadname + ";";
}
}
public int getRank(int auxYear, String auxName){
//FileResource auxFr = new FileResource("/testing/yob" + auxYear + "short.csv");
String resourceName = "/Users/User/Desktop/coursera/week4_babybirths/babybirths/testing/yob" + auxYear + "short.csv";
File auxFile = new File(resourceName);
if(auxFile.exists()){
FileResource auxFr = new FileResource(auxFile);
//FileResource auxFr = new FileResource();
int auxRank = 0;
for (CSVRecord auxRec : auxFr.getCSVParser(false)){
if (auxRec.get(1).contains(auxGender)){
auxRank += 1;
String auxN = auxRec.get(0);
if (auxRec.get(0).contains(auxName)){
return auxRank;
}
}
}
}
the getName method is reading another csv file and not staring from the first row, below the code :
public String getName(int auxYear, int auxRank, String auxGender){
////FileResource auxFr = new FileResource("/testing/yob" + auxYear + "short.csv");
String resourceName = "/Users/User/Desktop/coursera/week4_babybirths/babybirths/testing/yob" + auxYear + "short.csv";
File auxFile = new File(resourceName);
if (auxFile.exists()){
FileResource auxFr = new FileResource(auxFile);
int auxCount = 0;
for (CSVRecord auxRec : auxFr.getCSVParser()){
String auxStr = auxRec.get(0);
if (auxRec.get(1).contains(auxGender)){
auxCount += 1;
String auxStr1 = auxRec.get(0);
if (auxCount == (auxRank-1)){
return auxRec.get(0);
}
}
}
}
I use also :
public String yourNameInYear(String auxName, int auxYear, int auxNewYear, String auxGender){
int auxRank = getRank(auxYear, auxName, auxGender);
return getName(auxNewYear, auxRank, auxGender);
}
public void testYourNameInYear(){
String auxName = yourNameInYear("Isabella", 2012, 2014, "F");
System.out.println("Isabella" + " born in " + 2012 + " would be " + auxName + " in " + 2014);
}
by default YourNameInYear will call getRank this one will open the yob2012short.csv then after processing, getName will be called and open yob2014short.csv. I don't know why this one is not starting from the first row ?
try to use this
if (auxRec.get(1).contains(auxGender)){
String auxStr1 = auxRec.get(0);
if (auxCount == (auxRank-1)){
return auxRec.get(0);
}
auxCount += 1;
}
as I gess auxCaunt must be your current row so if you increment it from 0 to 1 then you do call the getName with an auxRank equals to 0 (first row may be 0 not 1) so 0 is not equal to 1 so it will loop until auxRank equals 2
So I've been working on this bit of code for awhile now. It's about encrypting and decrypting a message and producing the two keys used alternatively to encrypt a message using the Caesar Cipher method of changing letters with corresponding letters from a shifted alphabet. For example "Fruit" would be "Hwwnv" according to the shifted alphabets found by implementing +2 to every other letter starting with the first letter, and implementing +5 to every other letter starting with the second letter. I've been using an instance of another class called BreakCaesarThree to find these two keys, dkey_0 and dkey_1 and a decrypted message. I would rather use my method breakCaesarTwo instead, because of the ease of having all my necessary code in one class. How would I go about doing this? How do I change it so that I'm using breakCaesarTwo method instead of BreakCaesarThree class, and still be able to print out the dkey_0 and dkey_1 and a decrypted message? I am hoping that changing to using the breakCaesarTwo method will yield the right results.
Note: Right now calling BreakCaesarThree doesn't yield a decrypted message or give the right keys (I get 0s).
Here's my TestCaesarCipherTwo code which includes the breakCaesarTwo method:
import edu.duke.*;
public class TestCaesarCipherTwo {
private String alphabetLower;
private String alphabetUpper;
private String shiftedAlphabetLower1;
private String shiftedAlphabetUpper1;
private String shiftedAlphabetLower2;
private String shiftedAlphabetUpper2;
private int mainKey1;
private int mainKey2;
private int dkey_0;
private int dkey_1;
/**
*
*/
public void simplebreaker()
{
FileResource fr = new FileResource();
String encrypted = fr.asString();
BreakCaesarThree bct = new BreakCaesarThree();
String broken = bct.decrypt(encrypted);
System.out.println("Keys found: " + bct.dkey_0 + ", " + bct.dkey_1 + "\n" + broken);
}
public String halfOfString(String message, int start) {
StringBuilder halfString = new StringBuilder();
for (int index=start;index < message.length();index += 2) {
halfString.append(message.charAt(index));
}
return halfString.toString();
}
public String decrypt(String input) {
CaesarCipherTwoKeys cctk= new CaesarCipherTwoKeys(26 - mainKey1, 26 - mainKey2);
String decrypted = cctk.encrypt(input);
return decrypted;
}
public int[] countOccurrencesOfLetters(String message) {
//snippet from lecture
String alph = "abcdefghijklmnopqrstuvwxyz";
int[] counts = new int[26];
for (int k=0; k < message.length(); k++) {
char ch = Character.toLowerCase(message.charAt(k));
int dex = alph.indexOf(ch);
if (dex != -1) {
counts[dex] += 1;
}
}
return counts;
}
public int maxIndex(int[] values) {
int maxDex = 0;
for (int k=0; k < values.length; k++) {
if (values[k] > values[maxDex]) {
maxDex = k;
}
}
return maxDex;
}
public void simpleTests()
{
int key1 = 17;
int key2 = 3;
FileResource fr = new FileResource();
String message = fr.asString();
CaesarCipherTwoKeys cctk = new CaesarCipherTwoKeys(key1, key2);
String encrypted = cctk.encrypt(message);
System.out.println(encrypted);
String decrypted = cctk.decrypt(encrypted);
System.out.println(decrypted);
BreakCaesarThree bct = new BreakCaesarThree();
String broken = bct.decrypt(encrypted);
System.out.println("Keys found: " + bct.dkey_0 + ", " + bct.dkey_1 + "\n" + broken);
}
public String breakCaesarTwo(String input) {
String in_0 = halfOfString(input, 0);
String in_1 = halfOfString(input, 1);
// Find first key
// Determine character frequencies in ciphertext
int[] freqs_0 = countOccurrencesOfLetters(in_0);
// Get the most common character
int freqDex_0 = maxIndex(freqs_0);
// Calculate key such that 'E' would be mapped to the most common ciphertext character
// since 'E' is expected to be the most common plaintext character
int dkey_0 = freqDex_0 - 4;
// Make sure our key is non-negative
if (dkey_0 < 0) {
dkey_0 = dkey_0+26;
}
// Find second key
int[] freqs_1 = countOccurrencesOfLetters(in_1);
int freqDex_1 = maxIndex(freqs_1);
int dkey_1 = freqDex_1 - 4;
if (freqDex_1 < 4) {
dkey_1 = dkey_1+26;
}
CaesarCipherTwoKeys cctk = new CaesarCipherTwoKeys(dkey_0, dkey_1);
return cctk.decrypt(input);
}
}
I'd like to implement the changes here:
public void simplebreaker()
{
FileResource fr = new FileResource();
String encrypted = fr.asString();
BreakCaesarThree bct = new BreakCaesarThree();
String broken = bct.decrypt(encrypted);
System.out.println("Keys found: " + bct.dkey_0 + ", " + bct.dkey_1 + "\n" + broken);
}
and here:
public void simpleTests()
{
int key1 = 17;
int key2 = 3;
FileResource fr = new FileResource();
String message = fr.asString();
CaesarCipherTwoKeys cctk = new CaesarCipherTwoKeys(key1, key2);
String encrypted = cctk.encrypt(message);
System.out.println(encrypted);
String decrypted = cctk.decrypt(encrypted);
System.out.println(decrypted);
BreakCaesarThree bct = new BreakCaesarThree();
String broken = bct.decrypt(encrypted);
System.out.println("Keys found: " + bct.dkey_0 + ", " + bct.dkey_1 + "\n" + broken);