I want to write to a csv file ignoring object field which are null.
Currently it writes:
Test,PBAFFF,,
The 3rd and 4th values can be null in the object.
How can I configure FlatFileItemWriter with BeanWrapperFieldExtractor which would only write to file the non null fields?
I have my writer configured like this:
csvWriter.setLineAggregator(new DelimitedLineAggregator<Transaction>() {
{
setDelimiter(",");
setFieldExtractor(new BeanWrapperFieldExtractor<Transaction>() {
{
setNames(new String[] { "id", "source", "startDate", "endDate"});
}
});
}
});
There are two ways to do this:
Implement your custom LineAggregator
public class NullSafeDelimitedLineAggregator<T> extends ExtractorLineAggregator<T> {
private String delimiter;
public NullSafeDelimitedLineAggregator(String delimiter, FieldExtractor<T> fieldExtractor) {
this.delimiter = delimiter;
setFieldExtractor(fieldExtractor);
}
#Override
public String doAggregate(Object[] fields) {
return arrayToDelimitedString(fields, delimiter);
}
private String arrayToDelimitedString(#Nullable Object[] arr, String delim) {
if (ObjectUtils.isEmpty(arr)) {
return "";
}
if (arr.length == 1) {
return ObjectUtils.nullSafeToString(arr[0]);
}
StringBuilder sb = new StringBuilder();
for (int i = 0; i < arr.length; i++) {
if(arr[i] == null || "".equals(arr[i])) {
continue;
}
if (i > 0) {
sb.append(delim);
}
sb.append(arr[i]);
}
return sb.toString();
}
}
And change config in following way:
writer.setLineAggregator(new NullSafeDelimitedLineAggregator<>(
",",
new BeanWrapperFieldExtractor<Transaction>() {
{
setNames(new String[] { "id", "source", "startDate", "endDate"});
}
}));
Implement your custom FieldExtractor
public class NullSafeBeanWrapperFieldExtractor<T> implements FieldExtractor<T>, InitializingBean {
private String[] names;
public void setNames(String[] names) {
Assert.notNull(names, "Names must be non-null");
this.names = Arrays.asList(names).toArray(new String[names.length]);
}
#Override
public Object[] extract(T item) {
List<Object> values = new ArrayList<Object>();
BeanWrapper bw = new BeanWrapperImpl(item);
for (String propertyName : this.names) {
Object value = bw.getPropertyValue(propertyName);
if(value == null || "".equals(value)) {
continue;
}
values.add(value);
}
return values.toArray();
}
#Override
public void afterPropertiesSet() {
Assert.notNull(names, "The 'names' property must be set.");
}
}
And change config in following way:
writer.setLineAggregator(new DelimitedLineAggregator<Transaction>() {
{
setDelimiter(",");
setFieldExtractor(new NullSafeBeanWrapperFieldExtractor<Transaction>() {
{
setNames(new String[] { "id", "source", "startDate", "endDate"});
}
});
}
});
Important: You should chose one of two options based on one question: is it important for you to know how many values you have in the array or not? if yes - LineAggregator, no - FieldExtractor.
Related
ExtractSourceQueryOB is an object that represents queries, a query could have a master query and in this case the master query should not be removed from the list, querySet contains strings of keys that represent queries that are required in this operation but it doesnt take account of the possibility of that query having a master. So i tried to loop through all queries checking that query has a master and in that case add that object (ExtractSourceQueryOB ) to the final list.
private List<ExtractSourceQueryOB> checkRequiredQueries(List<ExtractSourceQueryOB> extractSourceQueryList, ExtractElement extractElement) {
Set<ExtractSourceQueryOB> queryList = new HashSet();
Set<String> querySet = new HashSet();
fillUsedSymbolList(querySet, extractElement);
for(ExtractSourceQueryOB extractSourceQuery : extractSourceQueryList) {
if(extractSourceQuery.getMaster() != null ) {
for(ExtractSourceQueryOB extractSourceQuery2 : extractSourceQueryList) {
if(extractSourceQuery.getMaster().equals(extractSourceQuery2.getSymbol())){
queryList.add(extractSourceQuery2);
}
}
}
}
for (ExtractSourceQueryOB extractSourceQuery : extractSourceQueryList) {
for (String s : querySet) {
if (extractSourceQuery.getSymbol().equalsIgnoreCase(s)) {
queryList.add(extractSourceQuery);
}
}
}
return new ArrayList<>(queryList);
}
How can i simplify this method ?
You could eliminate the second for by combining that code with the first. Like,
for(ExtractSourceQueryOB extractSourceQuery : extractSourceQueryList) {
if(extractSourceQuery.getMaster() != null ) {
for(ExtractSourceQueryOB extractSourceQuery2 : extractSourceQueryList) {
if(extractSourceQuery.getMaster().equals(extractSourceQuery2.getSymbol())){
queryList.add(extractSourceQuery2);
}
}
}
for (String s : querySet) {
if (extractSourceQuery.getSymbol().equalsIgnoreCase(s)) {
queryList.add(extractSourceQuery);
}
}
}
private List<ExtractSourceQueryOB> checkRequiredQueries(List<ExtractSourceQueryOB> extractSourceQueryList, ExtractElement extractElement) {
Set<ExtractSourceQueryOB> queryList = new HashSet();
Set<String> querySet = new HashSet();
fillUsedSymbolList(querySet, extractElement);
for (ExtractSourceQueryOB extractSourceQuery : extractSourceQueryList) {
if (isyUsedAsMaster(extractSourceQuery,extractSourceQueryList) || isUsed(extractSourceQuery,querySet, extractSourceQueryList)) {
queryList.add(extractSourceQuery);
}
}
return new ArrayList<>(queryList);
}
private boolean isyUsedAsMaster(ExtractSourceQueryOB extractSourceQuery, List<ExtractSourceQueryOB> extractSourceQueryList) {
if (extractSourceQuery.getMaster() != null) {
for (ExtractSourceQueryOB extractSourceQuery2 : extractSourceQueryList) {
if (extractSourceQuery.getMaster().equals(extractSourceQuery2.getSymbol())) {
return true;
}
}
}
return false;
}
private boolean isUsed(ExtractSourceQueryOB extractSourceQuery, Set<String> querySet, List<ExtractSourceQueryOB> extractSourceQueryList) {
for (String s : querySet) {
if(extractSourceQuery.getSymbol().equalsIgnoreCase(s)) {
return true;
}
}
return false;
}
Previous answer above is correct but this is with some refactoring
I was given exercise that I need to refactor several java projects.
Only those 2 left which I truly don't have an idea how to refactor.
csv.writer
public class CsvWriter {
public CsvWriter() {
}
public void write(String[][] lines) {
for (int i = 0; i < lines.length; i++)
writeLine(lines[i]);
}
private void writeLine(String[] fields) {
if (fields.length == 0)
System.out.println();
else {
writeField(fields[0]);
for (int i = 1; i < fields.length; i++) {
System.out.print(",");
writeField(fields[i]);
}
System.out.println();
}
}
private void writeField(String field) {
if (field.indexOf(',') != -1 || field.indexOf('\"') != -1)
writeQuoted(field);
else
System.out.print(field);
}
private void writeQuoted(String field) {
System.out.print('\"');
for (int i = 0; i < field.length(); i++) {
char c = field.charAt(i);
if (c == '\"')
System.out.print("\"\"");
else
System.out.print(c);
}
System.out.print('\"');
}
}
csv.writertest
public class CsvWriterTest {
#Test
public void testWriter() {
CsvWriter writer = new CsvWriter();
String[][] lines = new String[][] {
new String[] {},
new String[] { "only one field" },
new String[] { "two", "fields" },
new String[] { "", "contents", "several words included" },
new String[] { ",", "embedded , commas, included",
"trailing comma," },
new String[] { "\"", "embedded \" quotes",
"multiple \"\"\" quotes\"\"" },
new String[] { "mixed commas, and \"quotes\"", "simple field" } };
// Expected:
// -- (empty line)
// only one field
// two,fields
// ,contents,several words included
// ",","embedded , commas, included","trailing comma,"
// """","embedded "" quotes","multiple """""" quotes"""""
// "mixed commas, and ""quotes""",simple field
writer.write(lines);
}
}
test
public class Configuration {
public int interval;
public int duration;
public int departure;
public void load(Properties props) throws ConfigurationException {
String valueString;
int value;
valueString = props.getProperty("interval");
if (valueString == null) {
throw new ConfigurationException("monitor interval");
}
value = Integer.parseInt(valueString);
if (value <= 0) {
throw new ConfigurationException("monitor interval > 0");
}
interval = value;
valueString = props.getProperty("duration");
if (valueString == null) {
throw new ConfigurationException("duration");
}
value = Integer.parseInt(valueString);
if (value <= 0) {
throw new ConfigurationException("duration > 0");
}
if ((value % interval) != 0) {
throw new ConfigurationException("duration % interval");
}
duration = value;
valueString = props.getProperty("departure");
if (valueString == null) {
throw new ConfigurationException("departure offset");
}
value = Integer.parseInt(valueString);
if (value <= 0) {
throw new ConfigurationException("departure > 0");
}
if ((value % interval) != 0) {
throw new ConfigurationException("departure % interval");
}
departure = value;
}
}
public class ConfigurationException extends Exception {
private static final long serialVersionUID = 1L;
public ConfigurationException() {
super();
}
public ConfigurationException(String arg0) {
super(arg0);
}
public ConfigurationException(String arg0, Throwable arg1) {
super(arg0, arg1);
}
public ConfigurationException(Throwable arg0) {
super(arg0);
}
}
configuration.test
public class ConfigurationTest{
#Test
public void testGoodInput() throws IOException {
String data = "interval = 10\nduration = 100\ndeparture = 200\n";
Properties input = loadInput(data);
Configuration props = new Configuration();
try {
props.load(input);
} catch (ConfigurationException e) {
assertTrue(false);
return;
}
assertEquals(props.interval, 10);
assertEquals(props.duration, 100);
assertEquals(props.departure, 200);
}
#Test
public void testNegativeValues() throws IOException {
processBadInput("interval = -10\nduration = 100\ndeparture = 200\n");
processBadInput("interval = 10\nduration = -100\ndeparture = 200\n");
processBadInput("interval = 10\nduration = 100\ndeparture = -200\n");
}
#Test
public void testInvalidDuration() throws IOException {
processBadInput("interval = 10\nduration = 99\ndeparture = 200\n");
}
#Test
public void testInvalidDeparture() throws IOException {
processBadInput("interval = 10\nduration = 100\ndeparture = 199\n");
}
#Test
private void processBadInput(String data) throws IOException {
Properties input = loadInput(data);
boolean failed = false;
Configuration props = new Configuration();
try {
props.load(input);
} catch (ConfigurationException e) {
failed = true;
}
assertTrue(failed);
}
#Test
private Properties loadInput(String data) throws IOException {
InputStream is = new StringBufferInputStream(data);
Properties input = new Properties();
input.load(is);
is.close();
return input;
}
}
Ok, here some advice regarding the code.
CsvWriter
The bad thing is that you print everything to System.out. It will be hard to test without mocks. Instead I suggest you to add field PrintStream which defines where all data will go.
import java.io.PrintStream;
public class CsvWriter {
private final PrintStream printStream;
public CsvWriter() {
this.printStream = System.out;
}
public CsvWriter(PrintStream printStream) {
this.printStream = printStream;
}
...
You then write everything to this stream. This refactoring easy since you use replace function(Ctrl+R in IDEA). Here is the example how you do it.
private void writeField(String field) {
if (field.indexOf(',') != -1 || field.indexOf('\"') != -1)
writeQuoted(field);
else
printStream.print(field);
}
Others stuff seems ok in this class.
CsvWriterTest
First thing first you don't check all logic in a single method. Make small methods with different kind of tests. It's ok to keep your current test though. Sometimes it's useful to check most of the logic in a complex scenario.
Also pay attention to the names of the methods. Check this
Obviously you test doesn't check the results. That's why we need this functionality with PrintStream. We can build a PrintStream on top of the instance of ByteArrayOutputStream. We then construct a string and check if the content is valid. Here is how you can easily check what was written
public class CsvWriterTest {
private ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
private PrintStream printStream = new PrintStream(byteArrayOutputStream);
#Test
public void testWriter() {
CsvWriter writer = new CsvWriter(printStream);
... old logic here ...
writer.write(lines);
String result = new String(byteArrayOutputStream.toByteArray());
Assert.assertTrue(result.contains("two,fields"));
Configuration
Make fields private
Make messages more concise
ConfigurationException
Seems good about serialVersionUID. This thing is needed for serialization/deserialization.
ConfigurationTest
Do not use assertTrue(false/failed); Use Assert.fail(String) with some message which is understandable.
Tip: if you don't have much experience and need to refactor code like this, you may want to read some chapters of Effective Java 2nd edition by Joshua Bloch. The book is not so big so you can read it in a week and it has some rules how to write clean and understandable code.
I am currently working on a "code parser" parsing Valve Map Format (.vmf files) into a java readable Object.
In vmf files,
there are 2 types of objects: Classes and Properties.
classes have a name and can contain other classes and properties.
properties have a name and an unlimited number of values.
Therefore I created a VMFClass Object Class and a VMFProperty Object Class.
I created a List with self-created HierarchyObjects, containing the VMFClass/VMFProperty Object, an UUID and the parentUUID.
The VMFClass Object Contains 2 Lists one with sub-VMFClasses, one with properties.
My Problem is that I have no clue on how to achieve that a Class contains all of its subclasses, since I can't tell how much subclasses the subclasses have and so on...
Here is my Code (Github):
HierachyObject:
package net.minecraft.sourcecraftreloaded.utils;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class HierarchyObject {
private static Map<Long, Long> usedUUIDs = new HashMap<>();
private long parentUUID;
private long UUID;
private Object object;
/**
*
* #param Object
* #param parent -1 is maximum level
*/
public HierarchyObject(Object object, long parent) {
this.object = object;
this.parentUUID = parent;
while (true) {
long random = (long) (Math.random() * Long.MAX_VALUE);
if (usedUUIDs.containsKey(random)) {
this.UUID = random;
usedUUIDs.put(random, parent);
break;
}
}
}
public long getUUID() {
return UUID;
}
public long getParentUUID() {
return parentUUID;
}
public static long getParentUUIDbyUUID(long UUID) {
if (usedUUIDs.containsKey(UUID)) {
return usedUUIDs.get(UUID);
}
return -1;
}
public Object getObject() {
return object;
}
public static boolean hasChild(long UUID){
if(usedUUIDs.containsValue(UUID)){
return true;
}
if(UUID == -1){
return true;
}
return false;
}
public boolean hasChild(){
return hasChild(this.UUID);
}
public static long[] getChildUUIDs(long UUID){
if(hasChild(UUID)){
List<Long> cUUIDs = new ArrayList<>();
for(int i = 0; i < usedUUIDs.size(); i++){
for (Map.Entry<Long, Long> e : usedUUIDs.entrySet()) {
if(e.getValue().longValue() == UUID){
cUUIDs.add(e.getKey());
}
}
}
return ListUtils.toPrimitivebyList(cUUIDs);
}
return null;
}
}
VMFProperty:
package net.minecraft.sourcecraftreloaded.source;
public class VMFProperty{
private String name;
private String[] values;
public VMFProperty(String name, String... values) {
this.name = name;
this.values = values;
}
public String getName() {
return name;
}
public String[] getValues() {
return values;
}
#Override
public boolean equals(Object paramObject){
if(paramObject instanceof VMFProperty){
return ((VMFProperty)paramObject).name.equals(this.name) && ((VMFProperty)paramObject).values.equals(this.values);
}
return false;
}
}
VMFClass:
package net.minecraft.sourcecraftreloaded.source;
import java.util.List;
public class VMFClass{
private List<VMFClass> classes;
private List<VMFProperty> properties;
private String name;
public VMFClass(String name, List<VMFClass> classes, List<VMFProperty> properties) {
this.name = name;
this.classes = classes;
this.properties = properties;
}
public String getName() {
return name;
}
public List<VMFClass> getClasses() {
return classes;
}
public List<VMFProperty> getProperties() {
return properties;
}
public void add(VMFClass vmfclass) {
classes.add(vmfclass);
}
public void add(VMFProperty vmfproperty) {
properties.add(vmfproperty);
}
public void remove(VMFClass vmfclass) {
classes.remove(vmfclass);
}
public void remove(VMFProperty vmfproperty) {
properties.remove(vmfproperty);
}
#Override
public boolean equals(Object paramObject){
if(paramObject instanceof VMFClass){
return ((VMFClass)paramObject).properties.equals(this.properties) && ((VMFClass)paramObject).classes.equals(this.classes) && ((VMFClass)paramObject).name.equals(this.name);
}
return false;
}
}
VMFObject (the class executing all the code):
package net.minecraft.sourcecraftreloaded.source;
import java.io.File;
import java.util.ArrayList;
import java.util.List;
import net.minecraft.sourcecraftreloaded.utils.HierarchyObject;
public class VMFObject {
private String rawfile = "";
private List<VMFClass> toplevelclasses;
private static final String INVALID_CHARS = "\\*,;<>|?=`´#'+~^°!§$%&()[].:-_";
public VMFObject(List<VMFClass> toplevelclasses) {
this.toplevelclasses = toplevelclasses;
}
public VMFObject() {
this(new ArrayList<VMFClass>());
}
public void write(File file) {
VMFWriter.write(file, rawfile);
}
public VMFObject read(File file) throws VMFParsingException {
this.rawfile = VMFReader.read(file);
parse();
return this;
}
public List<VMFClass> getClasses() {
return toplevelclasses;
}
private void parse() throws VMFParsingException {
evaluate();
get();
}
private void evaluate() throws VMFParsingException {
char[] textchars = rawfile.toCharArray();
int[] c = new int[]{0, 0, 0};
int line = 0;
int linepos = 0;
for (int i : textchars) {
linepos++;
if (textchars[i] == '\n') {
line++;
linepos = 0;
c[3] = 0;
if (c[3] % 2 != 0) {
throw new VMFParsingException("Invalid quotes on line" + line + ":" + linepos);
}
}
if (textchars[i] == '{') {
c[1]++;
}
if (textchars[i] == '}') {
c[2]++;
}
if (textchars[i] == '"') {
c[3]++;
if (c[1] - c[2] == 0) {
}
}
if (textchars[i] == '/' && textchars[i + 1] == '/') {
while (true) {
i++;
if (textchars[i] == '\n') {
break;
}
}
}
if (textchars[i] == '/' && textchars[i + 1] == ' ') {
throw new VMFParsingException("Invalid Character '/' on line" + line + ":" + linepos);
}
if (INVALID_CHARS.indexOf(textchars[i]) != -1) {
throw new VMFParsingException("Invalid Character '" + textchars[i] + "' on line" + line + ":" + linepos);
}
}
if (c[1] != c[2]) {
throw new VMFParsingException("Unbalanced brackets in vmf File");
}
}
public void add(VMFClass vmfclass) {
toplevelclasses.add(vmfclass);
}
private void get() throws VMFParsingException {
List<HierarchyObject> content = new ArrayList<>();
long curparent = -1;
String[] text = rawfile.split("\n");
for (int i = 0; i < text.length; i++) {
String line = text[i].trim();
if (line.startsWith("//")) {
continue;
} else {
byte quotec = 0;
char[] linechar = line.toCharArray();
boolean readp = false;
List<String> reads = new ArrayList<>();
byte creads = 0;
for (int y = 0; y < linechar.length; y++) {
if (linechar[y] == '/' && linechar[y + 1] == '/') {
break;
}
if (linechar[y] == '"') {
quotec++;
if (quotec % 2 == 0) {
readp = false;
creads++;
} else {
readp = true;
}
}
if (readp) {
reads.set(creads, reads.get(creads) + linechar[y]);
}
if (linechar[y] == '{') {
HierarchyObject object = new HierarchyObject(new VMFClass(line.substring(line.substring(0, y).lastIndexOf(' '), y).trim(), null, null), curparent);
content.add(object);
curparent = object.getUUID();
}
if (linechar[y] == '}') {
curparent = HierarchyObject.getParentUUIDbyUUID(curparent);
}
}
content.add(new HierarchyObject(new VMFProperty(reads.remove(0), reads.toArray(new String[reads.size()])), curparent));
}
}
buildObject(content);
}
private void buildObject(List<HierarchyObject> content) {
long curUUID = -1;
for(int i = 0; i < HierarchyObject.getChildUUIDs(curUUID).length; i++){
HierarchyObject.getChildUUIDs(curUUID);
}
//TODO implement
}
}
the //TODO part is where the Hierachy Object should get "converted" to the actual object.
Overview
It seems to me that your class layout is overcomplicated.
Let's try to simplify it...
What you have described with the VMF model is essentially a linked-list Tree.
Here's what the model looks like:
[.vmf file] (root)
/ \
_____/ \ _____
/ \
/ \
(VMFClass) (VMFClass)
/ \ / \
/ \ / \
/ \ / \
(VMFClass) (VMFProperties) (VMFClass) (VMFProperties)
/ \
/ \
/ \
(VMFClass) (VMFProperties)
What you need:
A Parser class (in your case, you have VMFObject, but lets call this class VMFParser).
The VMFClass and VMFProperty classes which you have are fine.
What you don't need:
The HierarchyObject class. The VMFParser can be the main controller and container for the hierarchy (e.g. the linked-list Tree model).
All the UUIDs (parent, child, etc.) These are just complicated things, but I see why you have them. You don't need them to track the hierarchy - Java will do this for us!!
VMFClass
public class VMFClass
{
// name of the class
private String name;
// reference back up to the parent
private VMFClass parentClass = null;
// all direct children go here
private List<VMFClass> children = new ArrayList<VMFClass>();
// I don't think you need a list of properties here since your VMFProperty class holds onto an array of properties
private VMFProperty properties;
// set the parent of this class
public void setParent (VMFClass parent)
{
this.parentClass = parent;
}
// get the direct children
public List<VMFClass> getChildren()
{
return this.children;
}
// rest of methods...
}
VMFParser
class VMFParser
{
private String rawfile = "";
// this is really the container for everything - think of it as the file shell
private VMFClass root = new VMFClass("root", null, null);
// construct yourself with the file
public VMFParser (String fileName)
{
this.rawfile = fileName;
}
public void parse ()
{
// all the parsing code goes here
read();
evaluate();
get();
// now at this point your hierarchy is built and stored in the
// root object in this class.
// Use the traverse method to go through it
}
private void get() throws VMFParsingException
{
// keep a reference to the current VMFClass parent
// starts out as root
VMFClass currParentClass = root;
// main parse loop
for (...)
{
// if you find a class
VMFClass currClass = new VMFClass(/* params here */);
// add this class to the parent
currParentClass.add(currClass);
// set the parent of this class
currClass.setParent(currParentClass);
// if you find a property
// parse and add all the properties to the property
VMFProperty property = new VMFProperty (/* value params here */);
// attach this property to the last VMF class that got parsed
currClass.setPoperties(property);
// If you nest deeper into classes, then the parent becomes the current class
currParentClass = currClass;
// If you go back out of a class
currParentClass = currClass.getParent();
}
}
// Traverse the hierarchy
public void traverse ()
{
traverseTree(root);
}
private void traverseTree (VMFClass root)
{
System.out.println("Class Name: " + root.getName());
// print out any properties
VMFProperty prop = root.getProperty();
if (prop != null)
{
System.out.println("Property Name: " + prop.getName());
String [] props = prop.getValues();
for (String s: props)
{
System.out.println("Value: " + s);
}
}
// get all child classes
List<VMFClass> children = root.getChildren();
for (VMFClass c: children)
{
traverseTree(c);
}
}
}
Client Code
Example
public static void main(String[] args)
{
VMFParser vmfParser = null;
try
{
vmfParser = new VMFParser("myFile.vmf");
vmfParser.parse();
// access the vmfParser for the hierarchy
vmfParser.traverse();
}
catch (VMFParsingException vpe)
{
// do something here
vpe.printStackTrace();
}
finally
{
// clean up...
}
}
If you are just looking to find all sub classes of particular class or interface , this might help you,
How can I get a list of all the implementations of an interface programmatically in Java?
I have implemented two member functions in the same class:
private static void getRequiredTag(Context context) throws IOException
{
//repeated begin
for (Record record : context.getContext().readCacheTable("subscribe")) {
String traceId = record.get("trace_id").toString();
if (traceSet.contains(traceId) == false)
continue;
String tagId = record.get("tag_id").toString();
try {
Integer.parseInt(tagId);
} catch (NumberFormatException e) {
context.getCounter("Error", "tag_id not a number").increment(1);
continue;
}
//repeated end
tagSet.add(tagId);
}
}
private static void addTagToTraceId(Context context) throws IOException
{
//repeated begin
for (Record record : context.getContext().readCacheTable("subscribe")) {
String traceId = record.get("trace_id").toString();
if (traceSet.contains(traceId) == false)
continue;
String tagId = record.get("tag_id").toString();
try {
Integer.parseInt(tagId);
} catch (NumberFormatException e) {
context.getCounter("Error", "tag_id not a number").increment(1);
continue;
}
//repeated end
Vector<String> ret = traceListMap.get(tagId);
if (ret == null) {
ret = new Vector<String>();
}
ret.add(traceId);
traceListMap.put(tagId, ret);
}
}
I will call that two member functions in another two member functions(so I can't merge them into one function):
private static void A()
{
getRequiredTag()
}
private static void B()
{
getRequiredTag()
addTagToTraceId()
}
tagSet is java.util.Set and traceListMap is java.util.Map.
I know DRY principle and I really want to eliminate the repeat code, so I come to this code:
private static void getTraceIdAndTagIdFromRecord(Record record, String traceId, String tagId) throws IOException
{
traceId = record.get("trace_id").toString();
tagId = record.get("tag_id").toString();
}
private static boolean checkTagIdIsNumber(String tagId)
{
try {
Integer.parseInt(tagId);
} catch (NumberFormatException e) {
return false;
}
return true;
}
private static void getRequiredTag(Context context) throws IOException
{
String traceId = null, tagId = null;
for (Record record : context.getContext().readCacheTable("subscribe")) {
getTraceIdAndTagIdFromRecord(record, traceId, tagId);
if (traceSet.contains(traceId) == false)
continue;
if (!checkTagIdIsNumber(tagId))
{
context.getCounter("Error", "tag_id not a number").increment(1);
continue;
}
tagSet.add(tagId);
}
}
private static void addTagToTraceId(Context context) throws IOException
{
String traceId = null, tagId = null;
for (Record record : context.getContext().readCacheTable("subscribe")) {
getTraceIdAndTagIdFromRecord(record, traceId, tagId);
if (traceSet.contains(traceId) == false)
continue;
if (!checkTagIdIsNumber(tagId))
{
context.getCounter("Error", "tag_id not a number").increment(1);
continue;
}
Vector<String> ret = traceListMap.get(tagId);
if (ret == null) {
ret = new Vector<String>();
}
ret.add(traceId);
traceListMap.put(tagId, ret);
}
}
It seems I got an new repeat... I have no idea to eliminate repeat in that case, could anybody give me some advice?
update 2015-5-13 21:15:12:
Some guys gives a boolean argument to eliminate repeat, but I know
Robert C. Martin's Clean Code Tip #12: Eliminate Boolean Arguments.(you can google it for more details).
Could you gives some comment about that?
The parts that changes requires the values of String tagId and String traceId so we will start by extracting an interface that takes those parameters:
public static class PerformingInterface {
void accept(String tagId, String traceId);
}
Then extract the common parts into this method:
private static void doSomething(Context context, PerformingInterface perform) throws IOException
{
String traceId = null, tagId = null;
for (Record record : context.getContext().readCacheTable("subscribe")) {
getTraceIdAndTagIdFromRecord(record, traceId, tagId);
if (traceSet.contains(traceId) == false)
continue;
if (!checkTagIdIsNumber(tagId))
{
context.getCounter("Error", "tag_id not a number").increment(1);
continue;
}
perform.accept(tagId, traceId);
}
}
Then call this method in two different ways:
private static void getRequiredTag(Context context) throws IOException {
doSomething(context, new PerformingInterface() {
#Override public void accept(String tagId, String traceId) {
tagSet.add(tagId);
}
});
}
private static void addTagToTraceId(Context context) throws IOException {
doSomething(context, new PerformingInterface() {
#Override public void accept(String tagId, String traceId) {
Vector<String> ret = traceListMap.get(tagId);
if (ret == null) {
ret = new Vector<String>();
}
ret.add(traceId);
traceListMap.put(tagId, ret);
}
});
}
Note that I am using lambdas here, which is a Java 8 feature (BiConsumer is also a functional interface defined in Java 8), but it is entirely possible to accomplish the same thing in Java 7 and less, it just requires some more verbose code.
Some other issues with your code:
Way too many things is static
The Vector class is old, it is more recommended to use ArrayList (if you need synchronization, wrap it in Collections.synchronizedList)
Always use braces, even for one-liners
You could use a stream (haven't tested):
private static Stream<Record> validRecords(Context context) throws IOException {
return context.getContext().readCacheTable("subscribe").stream()
.filter(r -> {
if (!traceSet.contains(traceId(r))) {
return false;
}
try {
Integer.parseInt(tagId(r));
return true;
} catch (NumberFormatException e) {
context.getCounter("Error", "tag_id not a number").increment(1);
return false;
}
});
}
private static String traceId(Record record) {
return record.get("trace_id").toString();
}
private static String tagId(Record record) {
return record.get("tag_id").toString();
}
Then could do just:
private static void getRequiredTag(Context context) throws IOException {
validRecords(context).map(r -> tagId(r)).forEach(tagSet::add);
}
private static void addTagToTraceId(Context context) throws IOException {
validRecords(context).forEach(r -> {
String tagId = tagId(r);
Vector<String> ret = traceListMap.get(tagId);
if (ret == null) {
ret = new Vector<String>();
}
ret.add(traceId(r));
traceListMap.put(tagId, ret);
});
}
tagId seems to be always null in your second attempt.
Nevertheless, one approach would be to extract the code that collects tagIds (this seems to be the same in both methods) into its own method. Then, in each of the two methods just iterate over the collection of returned tagIds and do different operations on them.
for (String tagId : getTagIds(context)) {
// do method specific logic
}
EDIT
Now I noticed that you also use traceId in the second method. The principle remains the same, just collect Records in a separate method and iterate over them in the two methods (by taking tagId and traceId from records).
Solution with lambdas is the most elegant one, but without them it involves creation of separate interface and two anonymous classes which is too verbose for this use case (honestly, here I would rather go with a boolean argument than with a strategy without lambdas).
Try this approach
private static void imYourNewMethod(Context context,Boolean isAddTag){
String traceId = null, tagId = null;
for (Record record : context.getContext().readCacheTable("subscribe")) {
getTraceIdAndTagIdFromRecord(record, traceId, tagId);
if (traceSet.contains(traceId) == false)
continue;
if (!checkTagIdIsNumber(tagId))
{
context.getCounter("Error", "tag_id not a number").increment(1);
continue;
}
if(isAddTag){
Vector<String> ret = traceListMap.get(tagId);
if (ret == null) {
ret = new Vector<String>();
}
ret.add(traceId);
traceListMap.put(tagId, ret);
}else{
tagSet.add(tagId);
}
}
call this method and pass one more parameter boolean true if you want to add otherwise false to get it.
I'm trying to access JSON file on the internet and it has some null fields like this one:
{
"c":[
{
"v": "Week 10"
},
null,
null,
null,
null,
{
"v": null
}]
}
when I try to access them to put them on a string an exception is thrown
I've used JSONObject.isNull(""), JSONObject.toString().equals("null") and non of them work help please!!
you can read this Json use getJsonArray to get a List, and then loop this list.
String jstr = "{'c':[{'v': 'Week 10'}, null,null,null,null,{'v': null}]}";
JSONObject json = JSONObject.fromObject(jstr);
JSONArray jsonArray = json.getJSONArray("c");
for(int i=0;i < jsonArray.size();i++){
System.out.println(jsonArray.get(i).toString());
}
or you can read this json into a bean, below is the sample code,hope this can help you.
c.java
public class c {
private String v;
public String getV() {
return v;
}
public void setV(String v) {
this.v = v;
}
}
json2Bean.java
public class json2Bean {
private List<c> c;
public List<c> getC() {
return c;
}
public void setC(List<c> c) {
this.c = c;
}
}
json2String.java
public class json2String {
public static void main(String[] args) {
String jstr = "{'c':[{'v': 'Week 10'}, null,null,null,null,{'v': null}]}";
Map<String, Class<?>> m = new HashMap<String, Class<?>>();
m.put("c", c.class);
json2Bean myBean = (json2Bean) JSONObject.toBean(JSONObject.fromObject(jstr), json2Bean.class, m);
for (c e : myBean.getC()) {
if (e != null) {
System.out.println(e.getV());
}
}
}
}