My use case is reading objects from a CSV, performing some modifications on the objects, then writing them to another CSV. I've attempted to modify the supercsv examples as below, but am getting a SuperCsvConstraintViolationException that I'm not sure how to address.
public class ReadingWriting {
private static final String CSV_READ_FILENAME = "src/test/resources/customers.csv";
public static final String CSV_WRITE_FILENAME = "target/writeWithCsvBeanWriter.csv";
public static void main(String[] args) throws Exception {
readWithCsvBeanReader();
}
/**
* Sets up the processors used for the examples. There are 10 CSV columns, so 10 processors are defined. Empty
* columns are read as null (hence the NotNull() for mandatory columns).
*
* #return the cell processors
*/
private static CellProcessor[] getProcessors() {
final String emailRegex = "[a-z0-9\\._]+#[a-z0-9\\.]+"; // just an example, not very robust!
StrRegEx.registerMessage(emailRegex, "must be a valid email address");
final CellProcessor[] processors = new CellProcessor[]{
new UniqueHashCode(), // customerNo (must be unique)
new NotNull(), // firstName
new NotNull(), // lastName
new ParseDate("dd/MM/yyyy"), // birthDate
new NotNull(), // mailingAddress
new Optional(new ParseBool()), // married
new Optional(new ParseInt()), // numberOfKids
new NotNull(), // favouriteQuote
new StrRegEx(emailRegex), // email
new LMinMax(0L, LMinMax.MAX_LONG) // loyaltyPoints
};
return processors;
}
/**
* An example of reading using CsvBeanReader.
*/
private static void readWithCsvBeanReader() throws Exception {
ICsvBeanReader beanReader = null;
ICsvBeanWriter beanWriter = null;
try {
beanReader = new CsvBeanReader(new FileReader(CSV_READ_FILENAME), CsvPreference.STANDARD_PREFERENCE);
beanWriter = new CsvBeanWriter(new FileWriter(CSV_WRITE_FILENAME), CsvPreference.STANDARD_PREFERENCE);
final CellProcessor[] processors = getProcessors();
// the header elements are used to map the values to the bean (names must match)
final String[] header = beanReader.getHeader(true);
beanWriter.writeHeader(header);
CustomerBean customer;
while ((customer = beanReader.read(CustomerBean.class, header, processors)) != null) {
System.out.println(String.format("lineNo=%s, rowNo=%s, customer=%s", beanReader.getLineNumber(),
beanReader.getRowNumber(), customer));
beanWriter.write(customer, header, processors);//this line causes the below output
/*
lineNo=4, rowNo=2, customer=CustomerBean(customerNo=1, loyaltyPoints=0, mailingAddress=1600 Amphitheatre Parkway
Mountain View, CA 94043
United States)
Exception in thread "main" Disconnected from the target VM, address: '127.0.0.1:60782', transport: 'socket'
org.supercsv.exception.SuperCsvConstraintViolationException: duplicate value '1' encountered with hashcode 49
processor=org.supercsv.cellprocessor.constraint.UniqueHashCode
context={lineNo=2, rowNo=2, columnNo=1, rowSource=[1, John, Dunbar, Wed Jun 13 00:00:00 AEST 1945, 1600 Amphitheatre Parkway
Mountain View, CA 94043
United States, null, null, "May the Force be with you." - Star Wars, jdunbar#gmail.com, 0]}
at org.supercsv.cellprocessor.constraint.UniqueHashCode.execute(UniqueHashCode.java:78)
at org.supercsv.util.Util.executeCellProcessors(Util.java:93)
at org.supercsv.io.CsvBeanWriter.write(CsvBeanWriter.java:136)
at Reading.readWithCsvBeanReader(Reading.java:78)
at Reading.main(Reading.java:25)
Process finished with exit code 1
*/
}
} finally {
if (beanReader != null) {
beanReader.close();
}
if( beanWriter != null ) {
beanWriter.close();
}
}
}
}
In your cell processor you are mentioning that customer number is having unique hash code. In the file I suspect a line with the same customer number.
Related
I have 2 csv files which the same data but output of the two files are in different order.
I want to output both lists in the same order.
List csv1
System.out.println(csv1);
Employee, Address, Name, Email
System.out.println(csv2);
Output of this List looks like;
Address, Email, Employee Name
How can I sort the lists to print in the column order;
Employee, Name, Email, Address
Note: I can't use integer col(1),col(3) because column 1 in csv1 does not match col1 in csv2
data is read as follows:
List<String> ret = new ArrayList<>();
BufferedReader r = new BufferedReader(new InputStreamReader(str));
Stream lines = r.lines().skip(1);
lines.forEachOrdered(
line -> {
line= ((String) line).replace("\"", "");
ret.add((String) line);
I've assumed that you need to parse these two csv files and output in order.
You can use Apache Commons-CSV library for parsing. I've considered below examples
Solution using external library:
test1.csv
Address,Email,Employee,Name
SecondMainRoad,test2#gmail.com,Frank,Michael
test2.csv
Employee,Address,Name,Email
John,FirstMainRoad,Doe,test#gmail.com
Sample program
public static void main(String[] args) throws IOException {
try(Reader csvReader = Files.newBufferedReader(Paths.get
("test2.csv"))) {
// Initialize CSV parser and iterator.
CSVParser csvParser = new CSVParser(csvReader, CSVFormat.Builder.create()
.setRecordSeparator(System.lineSeparator())
.setHeader()
.setSkipHeaderRecord(true)
.setIgnoreEmptyLines(true)
.build());
Iterator<CSVRecord> csvRecordIterator = csvParser.iterator();
while(csvRecordIterator.hasNext())
{
final CSVRecord csvRecord = csvRecordIterator.next();
final Map<String, String> recordMap = csvRecord.toMap();
System.out.println(String.format("Employee:%s", recordMap.get("Employee")));
System.out.println(String.format("Name:%s", recordMap.get("Name")));
System.out.println(String.format("Email:%s", recordMap.get("Email")));
System.out.println(String.format("Address:%s", recordMap.get("Address")));
}
}
}
Standlone Solution:
public class CSVTesterMain {
public static void main(String[] args) {
// I have used string variables to hold csv data, In this case, you can replace with file output lines.
String csv1= "Employee,Address,Name,Email\r\n" +
"John,FirstMainRoad,Doe,test#gmail.com\r\n" +
"Henry,ThirdCrossStreet,Joseph,email#gmail.com";
String csv2 = "Address,Email,Employee,Name\r\n" +
"SecondMainRoad,test2#gmail.com,Michael,Sessner\r\n" +
"CrossRoad,test25#gmail.com,Vander,John";
// Map key - To hold header information
// Map Value - List of lines holding values to the corresponding headers.
Map<String, List<String>> dataMap = new HashMap<>();
Stream<String> csv1LineStream = csv1.lines();
Stream<String> csv2LineStream = csv2.lines();
// We are using the same method to parse different csv formats. We are maintaining reference to the headers
// in the form of Map key which will helps us to emit output later as per our format.
populateDataMap(csv1LineStream, dataMap);
populateDataMap(csv2LineStream, dataMap);
// Now we have dataMap that holds data from multiple csv files. Key of the map is responsible to
// determine the header sequence.
// Print the output as per the sequence Employee, Name, Email, Address
System.out.println("Employee,Name,Email,Address");
dataMap.forEach((header, lineList) -> {
// Logic to determine the index value for each column.
List<String> headerList = Arrays.asList(header.split(","));
int employeeIdx = headerList.indexOf("Employee");
int nameIdx = headerList.indexOf("Name");
int emailIdx = headerList.indexOf("Email");
int addressIdx = headerList.indexOf("Address");
// Now we know the index value of each of these columns that can be emitted in our format.
// You can output to a file in your case.
// Iterate through each line, split and output as per the format.
lineList.forEach(line -> {
String[] data = line.split(",");
System.out.println(String.format("%s,%s,%s,%s", data[employeeIdx],
data[nameIdx],
data[emailIdx],
data[addressIdx]
));
});
});
}
private static void populateDataMap(Stream<String> csvLineStream, Map<String, List<String>> dataMap) {
// Populate data map associating the data to respective headers.
Iterator<String> csvIterator = csvLineStream.iterator();
// Fetch header. (In my example, I am sure that my first line is always the header).
String header = csvIterator.next();
if(! dataMap.containsKey(header))
dataMap.put(header, new ArrayList<>());
// Iterate through the remaining lines and populate data map.
while(csvIterator.hasNext())
dataMap.get(header).add(csvIterator.next());
}
}
Here I am using Jackson dataformat library to parse the csv files.
Dependency
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-csv</artifactId>
<version>2.13.2</version>
</dependency>
File 1
employee, address, name, email
1, address 1, Name 1, name1#example.com
2, address 2, Name 2, name2#example.com
3, address 3, Name 3, name3#example.com
File 2
address, email, employee, name
address 4, name4#example.com, 4, Name 4
address 5, name5#example.com, 5, Name 5
address 6, name6#example.com, 6, Name 6
Java Program
Here EmployeeDetails is a POJO class. And it is expected that the location of the csv files is passed as an argument.
import com.fasterxml.jackson.databind.MappingIterator;
import com.fasterxml.jackson.databind.ObjectReader;
import com.fasterxml.jackson.dataformat.csv.CsvMapper;
import com.fasterxml.jackson.dataformat.csv.CsvSchema;
import java.io.*;
import java.util.ArrayList;
import java.util.List;
public class EmployeeDataParser {
public static void main(String[] args) {
File directoryPath = new File(args[0]);
File filesList[] = directoryPath.listFiles();
List<EmployeeDetails> employeeDetails = new ArrayList<>();
EmployeeDataParser employeeDataParser=new EmployeeDataParser();
for(File file : filesList) {
System.out.println("File path: "+file.getAbsolutePath());
employeeDataParser.readEmployeeData(employeeDetails, file.getAbsolutePath());
}
System.out.println("number of employees into list: " + employeeDetails.size());
employeeDataParser.printEmployeeDetails(employeeDetails);
}
private List<EmployeeDetails> readEmployeeData(List<EmployeeDetails> employeeDetails,
String filePath){
CsvMapper csvMapper = new CsvMapper();
CsvSchema schema = CsvSchema.emptySchema().withHeader();
ObjectReader oReader = csvMapper.readerFor(EmployeeDetails.class).with(schema);
try (Reader reader = new FileReader(filePath)) {
MappingIterator<EmployeeDetails> mi = oReader.readValues(reader);
while (mi.hasNext()) {
EmployeeDetails current = mi.next();
employeeDetails.add(current);
}
} catch (IOException e) {
System.out.println("IOException Caught !!!");
System.out.println(e.getStackTrace());
}
return employeeDetails;
}
private void printEmployeeDetails(List<EmployeeDetails> employeeDetails) {
System.out.printf("%5s %10s %15s %25s", "Employee", "Name", "Email", "Address");
System.out.println();
for(EmployeeDetails empDetail:employeeDetails){
System.out.format("%5s %15s %25s %15s", empDetail.getEmployee(),
empDetail.getName(),
empDetail.getEmail(),
empDetail.getAddress());
System.out.println();
}
}
}
How do you send emails with voting buttons using the Graph API? I know how to do this using EWS, but I can't find anything describing how to do this using the Graph API.
I've included all the code to send emails with voting buttons using the MS Graph API Java SDK. The voting button labels are sent in a SingleValueLegacyExtendedProperty.
// send an email
Message message = new Message();
message.subject = "Meet for lunch?";
ItemBody body = new ItemBody();
body.contentType = BodyType.TEXT;
body.content = "The new cafeteria is open.";
message.body = body;
EmailAddress emailAddress = new EmailAddress();
emailAddress.address = to;
Recipient toRecipient = new Recipient();
toRecipient.emailAddress = emailAddress;
message.toRecipients = List.of(toRecipient);
EmailAddress fromAddress = new EmailAddress();
fromAddress.address = sendingMailbox;
Recipient fromRecipient = new Recipient();
fromRecipient.emailAddress = fromAddress;
message.from = fromRecipient;
VotingButtonEncoder vbe = new VotingButtonEncoderImpl();
SingleValueLegacyExtendedProperty prop =
new SingleValueLegacyExtendedProperty();
prop.value = vbe.createVoteButtonsBase64String(
List.of("Yes, let's have lunch.", "No, thank you though."));
// https://learn.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-oxprops/e11cc753-cecf-4fdc-bec7-23304d12388a
prop.id = "Binary {00062008-0000-0000-C000-000000000046} Id 0x00008520";
List<com.microsoft.graph.options.Option> requestOptions =
new ArrayList<>();
String requestUrl = "https://graph.microsoft.com/v1.0/users/"
+ sendingMailbox + "/microsoft.graph.sendMail";
SingleValueLegacyExtendedPropertyCollectionRequestBuilder builder =
new SingleValueLegacyExtendedPropertyCollectionRequestBuilder(
requestUrl, graphClient, requestOptions);
List<SingleValueLegacyExtendedProperty> pageContents =
new ArrayList<>();
pageContents.add(prop);
SingleValueLegacyExtendedPropertyCollectionPage singleValueExtPropPage =
new SingleValueLegacyExtendedPropertyCollectionPage(
pageContents, builder);
message.singleValueExtendedProperties = singleValueExtPropPage;
boolean saveToSentItems = true;
graphClient.users(sendingMailbox)
.sendMail(UserSendMailParameterSet.newBuilder()
.withMessage(message)
.withSaveToSentItems(saveToSentItems).build())
.buildRequest().post();
Voting button encoder:
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
public class VotingButtonEncoderImpl implements VotingButtonEncoder {
public static final String voteRequestPropHex = "0x00008520";
#Override
public String voteRequestId() {
return voteRequestPropHex;
}
#Override
public String createVoteButtonsBase64String(
Collection<String> voteOptions) throws DecoderException {
String hex = createVoteButtonsHexString(voteOptions);
byte[] bytes = Hex.decodeHex(hex.toCharArray());
return Base64.getEncoder().encodeToString(bytes);
}
#Override
public String createVoteButtonsHexString(Collection<String> voteOptions) {
// let's build a PidLidVerbStream!
// https://msdn.microsoft.com/en-us/library/ee218541(v=exchg.80).aspx
//
// that is a bit...dense...so how about the partial solution from
// Glen'S Exchange Blog
// http://gsexdev.blogspot.com/2015/01/sending-message-with-voting-buttons.html
// don't worry, as of 2017/12/11 the content there is totally A-OK. Just
// Make sure that the vote button strings are in the VoteOptionsExtra
// section.
List<VoteButtonHexifier> options =
new ArrayList<VoteButtonHexifier>(voteOptions.size());
for (String optionString : voteOptions) {
options.add(new VoteButtonHexifier(optionString));
}
String header = "0201";
// docs say count of VoteOption stuctures plus VoteOptionExtras
// structures, but appears to actually be the count of
// VoteOption + 4. Witness, here are the start of the binary from
// emails with 10, 9, 8 ... 1 voting options as created in
// Outlook. Look at the 4th and 5th hex digits which have been
// separated from the rest with a space. (Come to think of it,
// those extra 4 bytes are probably a null terminator for the
// string.)
//
// Email Options: one;two;three;four;five;six;seven;eight;nine;ten
// 0x8520: 0201 0e 00000000000000055265706c79084...
// Email Options: one;two;three;four;five;six;seven;eight;nine
// 0x8520: 0201 0d 00000000000000055265706c79084...
// Email Options: one;two;three;four;five;six;seven;eight
// 0x8520: 0201 0c 00000000000000055265706c79084...
// Email Options: one;two;three;four;five;six;seven
// 0x8520: 0201 0b 00000000000000055265706c79084...
// Email Options: one;two;three;four;five;six
// 0x8520: 0201 0a 00000000000000055265706c79084...
// Email Options: one;two;three;four;five
// 0x8520: 0201 09 00000000000000055265706c79084...
// Email Options: gov one;two;three;four
// 0x8520: 0201 08 00000000000000055265706c79084...
// Email Options: onebutton;twobutton;threebutton
// 0x8520: 0201 07 00000000000000055265706c79084...
// Email Options: onebutton;twobutton
// 0x8520: 0201 06 00000000000000055265706c79084...
// Email Options: one button
// 0x8520: 0201 05 00000000000000055265706c79084...
String recordCnt = intToLittleEndianString(options.size() + 4);
// not documented anywhere, but seems necessary (null terminator?)
String preReplyAllPadding = "00000000";
String replyToAllHeader =
"055265706C790849504D2E4E6F7465074D657373616765025245050000000000000000";
String replyToAllFooter =
"0000000000000002000000660000000200000001000000";
String replyToHeader =
"0C5265706C7920746F20416C6C0849504D2E4E6F7465074D657373616765025245050000000000000000";
String replyToFooter = "0000000000000002000000670000000300000002000000";
String forwardHeader =
"07466F72776172640849504D2E4E6F7465074D657373616765024657050000000000000000";
String forwardFooter = "0000000000000002000000680000000400000003000000";
String replyToFolderHeader =
"0F5265706C7920746F20466F6C6465720849504D2E506F737404506F737400050000000000000000";
String replyToFolderFooter = "00000000000000020000006C00000008000000";
// Yes, each option goes in TWICE as an ANSI string with no terminator
// https://msdn.microsoft.com/en-us/library/ee218406(v=exchg.80).aspx
StringBuilder optionsAscii = new StringBuilder(2000);
int count = 0;
for (VoteButtonHexifier option : options) {
optionsAscii.append("04000000")
// option (first time)
.append(option.getAsciiHexString())
.append("0849504D2E4E6F746500")
// option (second time)
.append(option.getAsciiHexString())
// internal2 - fixed
.append("00000000"
// internal3 - fixed
+ "00"
// fUseUSHeaders:
// 00000000 = international;
// 01000000 = us
+ "00000000"
// internal4 - fixed
+ "01000000"
// send behavior (01000000 = send immediately,
// 02000000 prompt user to edit or send)
+ "01000000"
// internal5 - fixed: int 0x00000002 (little endian)
+ "02000000"
// ID: record index, 1 based. (little endian)
+ intToLittleEndianString(count++)
// internal 6 (terminator, -1)
+ "ffffffff");
}
// voting option extra bits
String VoteOptionExtras =
"0401055200650070006C00790002520045000C5200650070006C007900200074006F00200041006C006C0002520045000746006F007200770061007200640002460057000F5200650070006C007900200074006F00200046006F006C0064006500720000";
// they are in here in UTF-16LE twice and up above in ASCII twice.
// https://msdn.microsoft.com/en-us/library/ee217598(v=exchg.80).aspx
StringBuilder optionsUtf16Le = new StringBuilder(2000);
for (VoteButtonHexifier option : options) {
// UTF-16LE option (first time)
optionsUtf16Le.append(option.getUtf16LeHexString())
// UTF-16LE option (second time)
.append(option.getUtf16LeHexString());
}
String allowReplyAllVal = "00"; // false
String allowReplyVal = "00"; // false
String allowReplyToFolderVal = "00"; // false
String allowForwardVal = "00"; // false
String VerbValue = header + recordCnt + preReplyAllPadding
+ replyToAllHeader + allowReplyAllVal + replyToAllFooter
+ replyToHeader + allowReplyVal + replyToFooter + forwardHeader
+ allowForwardVal + forwardFooter + replyToFolderHeader
+ allowReplyToFolderVal + replyToFolderFooter + optionsAscii
+ VoteOptionExtras + optionsUtf16Le;
return VerbValue;
}
public static String intToLittleEndianString(int count) {
return String.format("%08x", swapEndianOrder(count));
}
public static int swapEndianOrder(int i) {
return (i & 0xff) << 24
| (i & 0xff00) << 8
| (i & 0xff0000) >> 8
| (i >> 24) & 0xff;
}
}
You need both ASCII and little-endian UTF16 for the voting button labels:
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import org.apache.commons.codec.binary.Hex;
/**
* Accepts a String and produces ASCII and little-endian UTF16
* representations of those string to embed in voting buttons.
* TODO: enhance code to accept separate strings for the ASCII and UTF16
* strings to better support I18Z.
* #author Tim Perry
*/
public class VoteButtonHexifier
implements Comparable<VoteButtonHexifier> {
private String buttonLabel;
public VoteButtonHexifier(String buttonLabel) {
if (buttonLabel == null) {
throw new NullPointerException("buttonLabel may not be null");
}
this.buttonLabel = buttonLabel;
}
/**
* This will return valid ASCII characters IFF the input can be
* represented as ASCII characters. Be careful to sanitize input.
* #return the button label as the hex of UTF_8.
*/
public String getAsciiHexString() {
String buttonLabel = UnicodeCharacterUtils
.fixQuotesElipsesAndHyphens(this.buttonLabel);
String lengthHex = String.format("%02X", buttonLabel.length());
String labelHex = Hex.encodeHexString(
buttonLabel.getBytes(StandardCharsets.UTF_8));
return lengthHex + labelHex;
}
public String getUtf16LeHexString() {
String lengthHex = String.format("%02X", buttonLabel.length());
String labelUtf16_ElHex = utf16LeHex(buttonLabel);
return lengthHex + labelUtf16_ElHex;
}
private String utf16LeHex(String var) {
Charset charset = Charset.forName("UTF-16LE");
return Hex.encodeHexString(var.getBytes(charset));
}
#Override
public boolean equals(Object o) {
if (o == null || !(o instanceof VoteButtonHexifier)) {
return false;
}
VoteButtonHexifier other = (VoteButtonHexifier) o;
return buttonLabel.equals(other.buttonLabel);
}
#Override
public int hashCode() {
return buttonLabel.hashCode();
}
#Override
public int compareTo(VoteButtonHexifier o) {
if (o == null) {
return 1;
}
return buttonLabel.compareTo(o.buttonLabel);
}
}
I am learning how to work with files in Java. I have a sample file which contains key pairs and it values. I am trying to find a key pairs and if it matches, then output file would be updated with both, key pair and it's value. I am able to get key pairs in output file but unable to get values too. Stringbuilder may work here to append strings but I don't know how.
Below are my input and output files.
Input File:
born time 9 AM London -- kingNumber 1234567890 -- address: abc/cd/ef -- birthmonth: unknown
born time 9 AM Europe -- kingNumber 1234567890 -- address: abc/cd/ef -- birthmonth: december
Expected Output File:
kingNumber 1234567890 birthmonth unknown
kingNumber 1234567890 birthmonth unkbown
Current Output File:
kingNumber birthmonth
kingNumber birthmonth
I am able to write key pair ("kingNumber" and "birthmonth" in this case) to output file but I am not sure what I can do to get it's value too.
String kn = "kingNumber:";
String bd = "birthmonth:";
try {
File f = new File("sample.txt");
Scanner sc = new Scanner(f);
FileWriter fw = new FileWriter("output.txt");
while(sc.hasNextLine()) {
String lineContains = sc.next();
if(lineContains.contains(kn)) {
fw.write(kn + "\n");
// This is where I am stuck. What
// can I do to get it's value (number in this case).
}
else if(lineContains.contains(bd)) {
fw.write(bd);
// This is where I am stuck. What
// can I do to get it's value (birthday in this case).
}
}
} catch (IOException e) {
e.printStackTrace();
}
you could use java.util.regex.Pattern & java.util.regex.Matcherwith a pattern alike:
^born\stime\s([a-zA-Z0-9\s]*)\s--\skingNumber\s(\d+)\s--\saddress:\s([a-zA-Z0-9\s/]*)\s--\sbirthmonth:\s([a-zA-Z0-9\s]*)$
write less, do more.
I have written a simple parser that it following data format from your example.
You will need to call it like this:
PairParser parser = new PairParser(lineContains);
then you can get value from the parser by pair keys
How to get value:
parser.getValue("kingNumber")
Note that keys do not have trailing column character.
The parser code is here:
package com.grenader.example;
import java.util.HashMap;
import java.util.Map;
public class PairParser {
private Map<String, String> data = new HashMap<>();
/**
* Constructor, prepare the data
* #param dataString line from the given data file
*/
public PairParser(String dataString) {
if (dataString == null || dataString.isEmpty())
throw new IllegalArgumentException("Data line cannot be empty");
// Spit the input line into array of string blocks based on '--' as a separator
String[] blocks = dataString.split("--");
for (String block : blocks)
{
if (block.startsWith("born time")) // skip this one because it doesn't looks like a key/value pair
continue;
String[] strings = block.split("\\s");
if (strings.length != 3) // has not exactly 3 items (first items is empty), skipping this one as well
continue;
String key = strings[1];
String value = strings[2];
if (key.endsWith(":"))
key = key.substring(0, key.length()-1).trim();
data.put(key.trim(), value.trim());
}
}
/**
* Return value based on key
* #param key
* #return
*/
public String getValue(String key)
{
return data.get(key);
}
/**
* Return number of key/value pairs
* #return
*/
public int size()
{
return data.size();
}
}
And here is the Unit Test to make sure that the code works
package com.grenader.example;
import com.grenader.example.PairParser;
import org.junit.Test;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.fail;
public class PairParserTest {
#Test
public void getValue_Ok() {
PairParser parser = new PairParser("born time 9 AM London -- kingNumber 1234567890 -- address: abc/cd/ef -- birthmonth: unknown");
assertEquals("1234567890", parser.getValue("kingNumber"));
assertEquals("unknown", parser.getValue("birthmonth"));
}
#Test(expected = IllegalArgumentException.class)
public void getValue_Null() {
new PairParser(null);
fail("This test should fail with Exception");
}
#Test(expected = IllegalArgumentException.class)
public void getValue_EmptyLine() {
new PairParser("");
fail("This test should fail with Exception");
}
#Test()
public void getValue_BadData() {
PairParser parser = new PairParser("bad data bad data");
assertEquals(0, parser.size());
}
}
I'm working on gridsim project in Java eclipse. I have found a network flow program, which works only for one-to-one connection between the sender and receiver. If the same user (sender) wish to send a message to any other receiver, the program does not work. Similarly, if a receiver wish to send message to two sender users, it does not work. Here, I'm including all the java files for this work. In order to run the program, we need to include external .jar file path in the project. The gridsim.jar and simjava2.jar files can be downloaded from http://sourceforge.net/projects/gridsim/
The following are the programs. The main program is FlowNetEx01.java
package network.flow.example01;
import gridsim.*;
import gridsim.net.*;
import gridsim.net.flow.*;
import java.util.*;
// Test Driver class for this example
public class FlowNetEx01
{
// Creates main() to run this example
public static void main(String[] args)
{
System.out.println("Starting network example ...");
try
{
int num_user = 4; // number of grid users
Calendar calendar = Calendar.getInstance();
boolean trace_flag = false; // mean trace GridSim events
System.out.println("Initializing GridSim package");
// It is essential to set the network type before calling GridSim.init()
GridSim.initNetworkType(GridSimTags.NET_FLOW_LEVEL);
GridSim.init(num_user, calendar, trace_flag);
// In this example, the topology is:
// user(s) --10Mb/s-- r1 --1.5Mb/s-- r2 --10Mb/s-- GridResource(s)
Router r1 = new FlowRouter("router1", trace_flag); // router 1
Router r2 = new FlowRouter("router2", trace_flag); // router 2
String sender1 = "user1";
String receipient1 = "test1";
String sender2 = "user2";
String receipient2 = "test2";
// these entities are the senders
FlowNetUser user1 = new FlowNetUser(sender1, receipient2, 5.0);
FlowNetUser user2 = new FlowNetUser(sender2, receipient1, 20.0);
// these entities are the receipients
FlowTest test1 = new FlowTest(receipient1, sender2);
FlowTest test2 = new FlowTest(receipient2, sender1);
// The schedulers are redundent and will be stripped out soon
FIFOScheduler userSched1 = new FIFOScheduler("NetUserSched_0");
r1.attachHost(user1, userSched1);
FIFOScheduler userSched2 = new FIFOScheduler("NetUserSched_1");
r1.attachHost(user2, userSched2);
FIFOScheduler testSched1 = new FIFOScheduler("FlowTestSched_0");
r2.attachHost(test1, testSched1);
FIFOScheduler testSched2 = new FIFOScheduler("FlowTestSched_1");
r2.attachHost(test2, testSched2);
//////////////////////////////////////////
// Second step: Creates a physical link
double baud_rate = 1572864; // bits/sec (baud) [1.5Mb/s]
double propDelay = 300; // propagation delay in millisecond
int mtu = Integer.MAX_VALUE;; // max. transmission unit in byte
Link link = new FlowLink("r1_r2_link", baud_rate, propDelay, mtu);
FIFOScheduler r1Sched = new FIFOScheduler("r1_Sched");
FIFOScheduler r2Sched = new FIFOScheduler("r2_Sched");
r1.attachRouter(r2, link, r1Sched, r2Sched);
//////////////////////////////////////////
// Final step: Starts the simulation
GridSim.startGridSimulation();
System.out.println("\nFinish network example ...");
}
catch (Exception e)
{
e.printStackTrace();
System.err.print(e.toString());
System.out.println("Unwanted errors happen");
}
}
} // end class
Program-2:
package network.flow.example01;
import gridsim.*;
import gridsim.net.*;
import gridsim.net.flow.*;
import eduni.simjava.*;
import java.util.*;
public class FlowNetUser extends GridSim
{
private int myID_; // my entity ID
private String name_; // my entity name
private String destName_; // destination name
private int destID_; // destination id
private double wait_; // Delay until I begin sending
public static final int SEND_MSG = 1;
public static final int ACK_MSG = 2;
public FlowNetUser(String name, String destName, Link link, double wait) throws Exception
{
super(name, link);
// get this entity name from Sim_entity
this.name_ = super.get_name();
// get this entity ID from Sim_entity
this.myID_ = super.get_id();
// get the destination entity name
this.destName_ = destName;
// get the waiting time before sending
this.wait_ = wait;
}
public FlowNetUser(String name, String destName, double wait) throws Exception
{
// 10,485,760 baud = 10Mb/s
super(name, new FlowLink(name+"_link",10485760,450,Integer.MAX_VALUE));
// get this entity name from Sim_entity
this.name_ = super.get_name();
// get this entity ID from Sim_entity
this.myID_ = super.get_id();
// get the destination entity name
destName_ = destName;
// get the waiting time before sending
this.wait_ = wait;
}
public void body()
{
int packetSize = 524288000; // packet size in bytes [5MB]
//int packetSize = 52428800; // packet size in bytes [50MB]
//int packetSize = 524288000; // packet size in bytes [500MB]
//int packetSize = 5242880000; // packet size in bytes [5000MB]
int size = 3; // number of packets sent
int i = 0;
// get the destination entity ID
this.destID_ = GridSim.getEntityId(destName_);
//super.sim_pause(this.wait_);
this.gridSimHold(this.wait_);
// sends messages over the other side of the link
for (i = 0; i < size; i++)
{
String msg = "Message_" + i;
IO_data data = new IO_data(msg, packetSize, destID_);
System.out.println(name_ + ".body(): Sending " + msg +
", at time = " + GridSim.clock() );
// sends through Output buffer of this entity
super.send(super.output, GridSimTags.SCHEDULE_NOW,
GridSimTags.FLOW_SUBMIT, data);
//super.sim_pause();
super.sim_pause(10.0);
//this.gridSimHold((Math.random()*10)+1.0);
}
// get the ack back
Object obj = null;
for (i = 0; i < size; i++)
{
// waiting for incoming event in the Input buffer
obj = super.receiveEventObject();
System.out.println(name_ + ".body(): Receives Ack for " + obj);
}
// Wait for other FlowNetUser instances to finish
this.gridSimHold(1000.0);
super.send(destID_, GridSimTags.SCHEDULE_NOW,
GridSimTags.END_OF_SIMULATION);
// shut down I/O ports
shutdownUserEntity();
terminateIOEntities();
System.out.println(this.name_ + ":%%%% Exiting body() at time " +
GridSim.clock() );
}
} // end class
Program-3:
package network.flow.example01;
import java.util.*;
import gridsim.*;
import gridsim.net.*;
import gridsim.net.flow.*;
import gridsim.util.SimReport;
import eduni.simjava.*;
public class FlowTest extends GridSim
{
private int myID_; // my entity ID
private String name_; // my entity name
private String destName_; // destination name
private int destID_; // destination id
private SimReport report_; // logs every activity
public FlowTest(String name, String destName, Link link) throws Exception
{
super(name, link);
// get this entity name from Sim_entity
this.name_ = super.get_name();
// get this entity ID from Sim_entity
this.myID_ = super.get_id();
// get the destination entity name
this.destName_ = destName;
// logs every activity. It will automatically create name.csv file
report_ = new SimReport(name);
report_.write("Creates " + name);
}
public FlowTest(String name, String destName) throws Exception
{
// 10,485,760 baud = 10Mb/s
super(name, new FlowLink(name+"_link",10485760,250,Integer.MAX_VALUE));
// get this entity name from Sim_entity
this.name_ = super.get_name();
// get this entity ID from Sim_entity
this.myID_ = super.get_id();
// get the destination entity name
this.destName_ = destName;
// logs every activity. It will automatically create name.csv file
report_ = new SimReport(name);
report_.write("Creates " + name);
}
public void body()
{
// get the destination entity ID
this.destID_ = GridSim.getEntityId(destName_);
int packetSize = 1500; // packet size in bytes
Sim_event ev = new Sim_event(); // an event
// a loop waiting for incoming events
while ( Sim_system.running() )
{
// get the next event from the Input buffer
super.sim_get_next(ev);
// if an event denotes end of simulation
if (ev.get_tag() == GridSimTags.END_OF_SIMULATION)
{
System.out.println();
write(super.get_name() + ".body(): exiting ...");
break;
}
// if an event denotes another event type
else if (ev.get_tag() == GridSimTags.FLOW_SUBMIT)
{
System.out.println();
write(super.get_name() + ".body(): receive " +
ev.get_data() + ", at time = " + GridSim.clock());
// No need for an ack, it is handled in FlowBuffer now on our behalf
// sends back an ack
IO_data data = new IO_data(ev.get_data(), packetSize, destID_);
write(name_ + ".body(): Sending back " +
ev.get_data() + ", at time = " + GridSim.clock() );
// sends through Output buffer of this entity
super.send(super.output, GridSimTags.SCHEDULE_NOW,
GridSimTags.FLOW_ACK, data);
}
else if (ev.get_tag() == GridSimTags.INFOPKT_SUBMIT)
{
processPingRequest(ev);
}
}
// shut down I/O ports
shutdownUserEntity();
terminateIOEntities();
// don't forget to close the file
if (report_ != null) {
report_.finalWrite();
}
System.out.println(this.name_ + ":%%%% Exiting body() at time " +
GridSim.clock() );
}
private void processPingRequest(Sim_event ev)
{
InfoPacket pkt = (InfoPacket) ev.get_data();
pkt.setTag(GridSimTags.INFOPKT_RETURN);
pkt.setDestID( pkt.getSrcID() );
// sends back to the sender
super.send(super.output, GridSimTags.SCHEDULE_NOW,
GridSimTags.INFOPKT_RETURN,
new IO_data(pkt,pkt.getSize(),pkt.getSrcID()) );
}
private void write(String msg)
{
System.out.println(msg);
if (report_ != null) {
report_.write(msg);
}
}
} // end class
After running these programs, someone can tell us how to extend the required functionality as I mentioned in the beginning.
Personal experience ... GridSim5.2 potentially buggy.
The examples are dated written for version < 4.0, demonstrating not very complex scenarios.
Using version 5.2. According to the API docs every simulation should have at least one TopRegionalRC. This appends a UniqueID to the filename and records the location of the file in two hashmaps one for filename and the other for fileattr. Now the event filename used for the lookup remains unchanged - the lookup fails - compares to the fileattrmap.name_. Consequently, the waiting for ack block is never executed when performing a addmaster operation.
Fix: Since the UniqueId is returned by the initial request to the CTLG this can be appended to the filename for the subsequent event requiring the lookup. Alternatively, add the filename to fileattrmap and the filename+uniqueid to fileattrmap, then test for both in the lookup.
Also, GridSimTags uses -1 to signal END_OF_SIMULATION, but this conflicts with Advanced Reservation (AR) block of tags. That also use negative numbers. GridSimTags has an optional routine to check for duplicates but its use is optional and does not apply to DataGridTags. I created a reversemap for ease of debugging adding validation to ensure no duplicates occur and deprecated the GridSimTags method.
I am now wrestling with the DataGrid user tasks that do not seem to create events, I am also concerned that the delay operations are not effective.
I am currently working on a Java project,
below are my attempts at coding so far:
public class MyZoo
{
// zoo identifier
private String zooId;
// a number used in generating a unique identifier for the next animal to be added to the zoo
private int nextAnimalIdNumber;
// zstorage for the Animal objects
private TreeMap<String, List<Animal>> animals;
/**
* Create an "empty" zoo.
*
* #param zooId an identifier for the zoo, at least three characters long.
*/
public MyZoo(String zooId)
{
this.zooId = zooId.trim().substring(0,3).toUpperCase();
nextAnimalIdNumber = 0;
animals = new TreeMap<String, List<Animal>>();
}
/**
* Returns a unique identifier, for an <tt>Animal</tt> object, based on the
* zoo identifier and the field <tt>nextAnimalIdNumber</tt> which is incremented
* ready for next time the method is called.
*
* #return a unique identifier.
*/
public String allocateId()
{
// increment nextAnimalIdNumber and then construct a six digit string from it
nextAnimalIdNumber++;
String s = Integer.toString(nextAnimalIdNumber);
while ( s.length()<6 )
s = "0" + s;
return zooId + "_" + s;
}
/**
* Adds an animal to the zoo.
*
* #param animal the Animal object to be added.
*/
public void addAnimal(Animal animal)
{
String animalName = animal.getName();
// Is there already an animal with the same name?
if (!animals.containsKey(animalName)){
// If it is not in the map, create a list
animals.put(animalName, new ArrayList<Animal>());
}
// Now add the animal to the list
animals.get(animalName).add(animal);
}
/**
* Reads <tt>Animal</tt> data from a text file and adds them to the zoo. The
* format of the data is specified in the MyZoo coursework assignment.
*
* #param animal the Animal object to be added.
*/
public void readDataFromFile()
{
int noOfAnimalsRead = 0;
// set up an owner for the FileDialog
JFrame jframe = new JFrame();
jframe.setVisible(true);
// use a Filedialog to select a file to read from
FileDialog fDialog = new FileDialog(jframe, "Read from", FileDialog.LOAD);
fDialog.setFile("import001.txt");
fDialog.setDirectory(".");
fDialog.setVisible(true);
String fname = fDialog.getFile();
jframe.dispose();
File inFile = new File(fname);
String fileName = "import002.txt";
// This will reference one line at a time
String line = null;
try {
// FileReader reads text files in the default encoding.
FileReader fileReader =
new FileReader(fileName);
// Always wrap FileReader in BufferedReader.
BufferedReader bufferedReader =
new BufferedReader(fileReader);
while((line = bufferedReader.readLine()) != null) {
System.out.println(line);
}
// Always close files.
bufferedReader.close();
}
catch(FileNotFoundException ex) {
System.out.println(
"Unable to open file '" +
fileName + "'");
}
catch(IOException ex) {
System.out.println(
"Error reading file '"
+ fileName + "'");
}
addAnimal( new Animal("golden eagle", "Eddie", this) ); //
addAnimal( new Animal("tiger", "Tommy", this) );
addAnimal( new Animal("lion", "Leo", this) );
addAnimal( new Animal("parrot", "Polly", this) );
addAnimal( new Animal("cobra", "Collin", this) );
noOfAnimalsRead = 5;
// this next line should be retained
System.out.println("no of animals read from file was " + noOfAnimalsRead + "\n");
}
/**
* Prints out details of all animal in the zoo.
*
*/
public void printAllAnimals()
{
System.out.println("\nDetails for all animals in Zoo " + zooId);
System.out.println( "==================================");
Collection<Animal> c = animals.values();
// The name of the file to open.
String fileName = "import001.txt";
// This will reference one line at a time
String line = null;
for (Map.Entry<String, List<Animal>> animalEntry : animals.entrySet())
{
List<Animal> animalsOfAName = animalEntry.getValue();
for (Animal animal: animalsOfAName){
// output here, change as appropriate, maybe add the type of animal
System.out.println(animal.getName());
}
}
}
}
however there are text names in the text file (Eddie) which are the same, how do I get the second Eddie to replace the first Eddie without losing alphabetical order, in the addAnimal() the put() uses the name field
Keys must be unique in a Map so that is why the second Eddie animal replaces the first one.
To hold multiple objects under the same key, you can declare a Map that uses a List (or another appropriate Collection) as the value in the key-value pair.
TreeMap<String, List<Animal>> animals;
Now some logic is needed to handle this when adding animals.
public void addAnimal(Animal animal)
{
String animalName = animal.getName();
// Is there already an animal with the same name?
if (!animals.containsKey(animalName){
// If it is not in the map, create a list
animals.put(animalName, new ArrayList<Animal>();
}
// Now add the animal to the list
animals.get(animalName).add(animal);
}
Now for printing, you need to iterate through the animals for each name:
for (Map.Entry<String, List<Animal>> animalEntry : animals.entrySet())
{
List<Animal> animalsOfAName = animalEntry.getValue();
for (Animal animal: animalsOfAName){
// output here, change as appropriate, maybe add the type of animal
System.out.println(animal.getName());
}
}