We Keep Coding

Write a MessageDirect using the xxhash algorithm

I would be grateful for help with the following problem.
An application I'm working on uses the SHA-256 algorithm to produce a hashing value. i.e.
MessageDigest messageDigest = MessageDigest.getInstance("SHA-256");
This has given us performance issues, and so I have been asked to replace this with an xxhash function. (Our main concern is having a good and fast hashing algorithm.)
We have identified the following component which we understand to be a good and effective implementation.
http://mvnrepository.com/artifact/net.jpountz.lz4/lz4/1.0.0
An example is given here for how to use the artefact. https://github.com/jpountz/lz4-java
XXHashFactory factory = XXHashFactory.fastestInstance();
byte[] data = "12345345234572".getBytes("UTF-8");
ByteArrayInputStream in = new ByteArrayInputStream(data);
int seed = 0x9747b28c; // used to initialize the hash value, use whatever
// value you want, but always the same
StreamingXXHash32 hash32 = factory.newStreamingHash32(seed);
byte[] buf = new byte[8]; // for real-world usage, use a larger buffer, like 8192 bytes
for (;;) {
int read = in.read(buf);
if (read == -1) {
break;
}
hash32.update(buf, 0, read);
}
int hash = hash32.getValue();
In order to switch-out the SHA-256 MessageDigest, I think the best course of action is to extend the java.security.MessageDigest abstract class. The following shows what I think would be an appropriate implementation, with missing implementation for engineDigest(). Would anyone be able to give guidance on how to implement this method, or point out an approach which they think is better? Thanks in advance.
package com.company.functions.crypto;
import java.security.MessageDigest;
import java.util.ArrayList;
import java.util.List;
import net.jpountz.xxhash.StreamingXXHash32;
import net.jpountz.xxhash.XXHashFactory;
public class XxHashMessageDigest extends MessageDigest {
public static final String XXHASH = "xxhash";
private static final int SEED = 0x9747b28c;
private static final XXHashFactory FACTORY = XXHashFactory
.fastestInstance();
private List<Integer> values = new ArrayList<Integer>();
protected XxHashMessageDigest() {
super(XXHASH);
}
#Override
protected void engineUpdate(byte input) {
// intentionally no implementation
}
#Override
protected void engineUpdate(byte[] input, int offset, int len) {
StreamingXXHash32 hash32 = FACTORY.newStreamingHash32(SEED);
hash32.update(input, offset, len);
values.add(hash32.getValue());
}
#Override
protected byte[] engineDigest() {
/*
* TODO provide implementation to "digest" the list of values into a
* hash value
*/
engineReset();
return null;
}
#Override
protected void engineReset() {
values.clear();
}
}

Junit-Quickcheck: Generate String matching a pattern

I am using pholser's port. I have to generate strings matching a given pattern like \[a-zA-Z0-9\\.\\-\\\\;\\:\\_\\#\\[\\]\\^/\\|\\}\\{]* Length 40.
I extend the Generator class as:
public class InputGenerator extends Generator<TestData> {...}
It overloads a function:
publicTestData generate(SourceOfRandomness random, GenerationStatus status) {...}
Now, random has functions like nextDouble(), nextInt() but there is nothing for strings! How can I generate random strings matching the above pattern?

Find below snippet for a custom generator which implement the generate(..) method to return a random string matching your posted pattern.
public class MyCharacterGenerator extends Generator<String> {
private static final String LOWERCASE_CHARS = "abcdefghijklmnopqrstuvwxyz";
private static final String UPPERCASE_CHARS = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
private static final String NUMBERS = "0123456789";
private static final String SPECIAL_CHARS = ".-\\;:_#[]^/|}{";
private static final String ALL_MY_CHARS = LOWERCASE_CHARS
+ UPPERCASE_CHARS + NUMBERS + SPECIAL_CHARS;
public static final int CAPACITY = 40;
public MyCharacterGenerator () {
super(String.class);
}
#Override
public String generate(SourceOfRandomness random, GenerationStatus status) {
StringBuilder sb = new StringBuilder(CAPACITY);
for (int i = 0; i < CAPACITY; i++) {
int randomIndex = random.nextInt(ALL_MY_CHARS.length());
sb.append(ALL_MY_CHARS.charAt(randomIndex));
}
return sb.toString();
}
}
edit A simple unit test to demonstrate the usage of the MyCharacterGenerator class.
import com.pholser.junit.quickcheck.ForAll;
import com.pholser.junit.quickcheck.From;
import static org.junit.Assert.assertTrue;
import org.junit.contrib.theories.Theories;
import org.junit.contrib.theories.Theory;
import org.junit.runner.RunWith;
#RunWith(Theories.class)
public class MyCharacterGeneratorTest {
#Theory
public void shouldHold(#ForAll #From(MyCharacterGenerator.class) String s) {
// here you should add your unit test which uses the generated output
//
// assertTrue(doMyUnitTest(s) == expectedResult);
// the below lines only for demonstration and currently
// check that the generated random has the expected
// length and matches the expected pattern
System.out.println("shouldHold(): " + s);
assertTrue(s.length() == MyCharacterGenerator.CAPACITY);
assertTrue(s.matches("[a-zA-Z0-9.\\-\\\\;:_#\\[\\]^/|}{]*"));
}
}
sample output generated by shouldHold
shouldHold(): MD}o/LAkW/hbJVWPGdI;:RHpwo_T.lGs^DOFwu2.
shouldHold(): IT_O{8Umhkz{#PY:pmK6}Cb[Wc19GqGZjWVa#4li
shouldHold(): KQwpEz.CW28vy_/WJR3Lx2.tRC6uLIjOTQtYP/VR
shouldHold(): pc2_T4hLdZpK78UfcVmU\RTe9WaJBSGJ}5v#z[Z\
...

There is no random.nextString(), but there is a way to generate random strings within junit-quickcheck-generators library. You can access it when creating new generators using gen().type(String.class). However, it seems we don't have much control over it.
Here is a silly example of a StringBuilder generator to demonstrate how to use the String generator:
import com.pholser.junit.quickcheck.generator.GenerationStatus;
import com.pholser.junit.quickcheck.generator.Generator;
import com.pholser.junit.quickcheck.random.SourceOfRandomness;
public class StringBuilderGenerator extends Generator<StringBuilder> {
public StringBuilderGenerator() {
super(StringBuilder.class);
}
#Override
public StringBuilder generate(SourceOfRandomness random, GenerationStatus status) {
String s = gen().type(String.class).generate(random, status);
return new StringBuilder(s);
}
}

I just made a library that suppose to do what you want in a generic way: https://github.com/SimY4/coregex
Simple usage example:
import com.pholser.junit.quickcheck.Property;
import com.pholser.junit.quickcheck.runner.JUnitQuickcheck;
import org.junit.runner.RunWith;
import java.util.UUID;
import static org.junit.Assert.assertEquals;
#RunWith(JUnitQuickcheck.class)
public class CoregexGeneratorTest {
#Property
public void shouldGenerateMatchingUUIDString(
#Regex("[0-9a-f]{8}-[0-9a-f]{4}-[0-5][0-9a-f]{3}-[089ab][0-9a-f]{3}-[0-9a-f]{12}")
String uuid) {
assertEquals(uuid, UUID.fromString(uuid).toString());
}
}

Selecting object items with same value from a list of objects

I have a array or list which contains a timestamp, an Id and other data.
Something like this:
public class CanMessage {
public int MsgID ;
public byte length ;
public byte [] dataByte = new byte[8];
}
public class CanTrace {
public float timestamp ;
public CanMessage canMsg ;
}
Question is how can I make a list or Array of CanTrace, in which I can make selection on List items that have a certain MsgID. So that for example I can make a plot of the dataByte with one and the same MsgID.
Or is this only possible by searching with a while loop through for example a ct object create with
List ct =new ArrayList();

If MsgID is unique, then it would be useful to use a HashMap with a key,value pair of (MsgID,CanTrace)
Then you would be able to access the dataByte using -
(CanTrace)(map.get(MsgID)).dataByte

If you are using Java 8 you could directly use the filter method as listOfCanMessages.stream().filter(b -> b.MsgID == 1).
In other case you can either try Guava or Lamdaj
For example.
import static ch.lambdaj.Lambda.having;
import static ch.lambdaj.Lambda.on;
import static ch.lambdaj.Lambda.select;
import static org.hamcrest.core.IsEqual.equalTo;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;
import com.google.common.base.Predicate;
import com.google.common.collect.Collections2;
import com.google.common.collect.Lists;
public class FilterSample {
public static void main(String[] args) {
List<CanMessage> list = new ArrayList<CanMessage>();
CanMessage c = new CanMessage();
c.MsgID = 1;
CanMessage c2 = new CanMessage();
c2.MsgID = 2;
CanMessage c3 = new CanMessage();
c3.MsgID = 1;
CanMessage c4 = new CanMessage();
c4.MsgID = 4;
list.add(c);
list.add(c2);
list.add(c3);
list.add(c4);
//Java 8 example
List<CanMessage> filteredList1 = list.stream().filter(b -> b.MsgID == 1).collect(Collectors.toList());
System.out.println(filteredList1);
//Guava example
List<CanMessage> filteredList2 = Lists.newArrayList(Collections2.filter(
list, new Predicate<CanMessage>() {
#Override
public boolean apply(CanMessage input) {
if (input.MsgID == 1) {
return true;
}
return false;
}
}));
System.out.println(filteredList2);
//Lambdaj example. Please note 6 down vote accepted Lambdaj wraps your (non-final) classes with a Proxy
// and intercepts METHOD invocations on them. That means it cannot work on fields but only on methods. So
// you cannot access MsgID directly, you'll have to provide a getter method
List<CanMessage> filteredList3 = select(list, having(on(CanMessage.class).getMsgID(), equalTo(1)));
System.out.println(filteredList3);
}
}

thanks for the fast answer.
I ask a collegue and he came with this Idea:
public void OrderedMsg(int Id){
List<CanTrace> traces = new ArrayList<CanTrace>();
Comparator<CanTrace> comparator = new Comparator<CanTrace>() {
public int compare(CanTrace o1, CanTrace o2) {
if (o1.time < o2.time) {
return -1;
} else if (o1.time > o2.time) {
return 1;
} else {
return 0;
}
}
};
for (CanTrace trace : traces) {
int id = trace.canMsg.MsgID;
if (!orderedMap.containsKey(id)) {
orderedMap.put(id, new TreeSet<CanTrace>(comparator));
}
orderedMap.get(id).add(trace);
}
}

Data Structure of the Periodic Table of Elements

My goal is to use the periodic table of elements (or a list) to get information about a specific element in Java. I want to search it by atomic number and symbol (but that conversion should be simple).
I found that information in this JQuery plugin. But it is stored as a JSON file.
It seems like it would be most efficient to hardcode the information (since it doesn't change too often and due to performance reasons), but how do I convert JSON to a hardcoded enum?

Since:
the information about elements is totally static
each elemental symbol is alphanumeric
the discovery of new elements is both rare and irrelevant (because they are extremely unstable)
An enum seems a good option:
public enum Element {
H(1, "Hydrogen", 1.008, -259.1),
He(2, "Helium", 4.003, -272.2),
Li(3, "Lithium", 6.941, 180.5),
// ... 90+ others
;
private static class Holder {
static Map<Integer, Element> map = new HashMap<Integer, Element>();
}
private final int atomicNumber;
private final String fullName;
private final double atomicMass;
private final double meltingPoint;
private Element(int atomicNumber, String fullName, double atomicMass, double meltingPoint) {
this.atomicNumber = atomicNumber;
this.fullName = fullName;
this.atomicMass = atomicMass;
this.meltingPoint = meltingPoint;
Holder.map.put(atomicNumber, this);
}
public static Element forAtomicNumber(int atomicNumber) {
return Holder.map.get(atomicNumber);
}
public int getAtomicNumber() {
return atomicNumber;
}
public String getFullName() {
return fullName;
}
public double getAtomicMass() {
return atomicMass;
}
public double getMeltingPoint() {
return meltingPoint;
}
}
There's a bit of java kung fu going on here that deserves an explanation. The map is put inside a static inner (holder) class so it gets initialized before the enum instances are initialized, that way they can add themselves to it. If not in the inner static class, it would not be initialize, because the first thing initialized in the enum class must be the instances, but static inner classes are initialized before the class is initialized.
This approach means the the instances don't need to be listed in any particular order (they could be alphabetical listed, or otherwise).

Suppose you have a PeriodicTable.txt file with following format:
ATOMIC_NUMBER SYMBOL OTHER_INFO
Like:
1 H Hydrogen -> Lightest element
2 He Helium -> Second lightest element
3 Li Lithium -> Third lightest element
// and so on...
Then you can have a fairly straightforward implementation of your own PeriodicTable like following:
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import java.util.Scanner;
public class PeriodicTable
{
private List<Element> elements;
public PeriodicTable() throws IOException
{
elements = new ArrayList<>();
List<String> list = Files.readAllLines(Paths.get("PeriodicTable.txt"));
list.forEach(this::process);
}
public static void main(String[] args) throws IOException
{
final PeriodicTable periodicTable = new PeriodicTable();
System.out.println(periodicTable.getElementByNumber(1));
System.out.println(periodicTable.getElementBySymbol("Li"));
}
private void process(String line)
{
try (Scanner scanner = new Scanner(line))
{
int atomicNumber = scanner.nextInt();
String symbol = scanner.next();
String info = scanner.nextLine();
elements.add(new Element(atomicNumber, symbol, info));
}
}
public Element getElementByNumber(int atomicNumber)
{
return elements.stream().
filter(e -> e.atomicNumber == atomicNumber).
findFirst().orElse(null);
}
public Element getElementBySymbol(String symbol)
{
return elements.stream().
filter(e -> e.symbol.equals(symbol)).
findFirst().orElse(null);
}
private class Element
{
private String info;
private int atomicNumber;
private String symbol;
public Element(int atomicNumber, String symbol, String info)
{
this.atomicNumber = atomicNumber;
this.symbol = symbol;
this.info = info;
}
public String toString()
{
return "[ " + atomicNumber + " " + symbol + " " + info + " ]";
}
}
}
If you see, I have created an Element class which holds the atomic number, symbol, and info of the element, and I have a list of elements in PeriodicTable class.
I read the PeriodicTable data from the PeriodicTable.txt file and process each line of the text file by parsing it appropriately and creating element for each line and adding it to the elements.
I also add two methods for filtering the element based on atomic number and symbol properties.
The code works in Java 8, so you should have at least that to run it, or one can easily write a code for this which will run on earlier JVMs, though it won't be as compact as this one.
Since there are just limited number of elements in the PeriodicTable, I don't bother to have elements sorted by their atomic number though they will be if your PeriodicTable.txt file has elements with increasing atomic number.
Since we know the exact number of elements in the PeriodicTable and its something that doesn't change frequently, the filtering methods take constant time.
All you have to do now is create a proper PeriodicTable.txt file which can then be used by the program.
Note: The PeriodicTable class can be written in better ways as well. This is just an example. I can have it as Singleton. I can even have enum of Element with hardcoded values, but I think loading data from file will keep the code cleaner.
One can even augment the PeriodicTable class with additional properties to each Element, by changing the process() method accordingly, and changing the format of the text file based on the assumptions, and augmenting the Element class, so that it can hold even more information.
Just for fun, following is a Singleton based solution:
// PeriodicTable.java
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import java.util.Scanner;
public class PeriodicTable
{
private static PeriodicTable periodicTable = new PeriodicTable();
private List<Element> elements;
private PeriodicTable()
{
try
{
elements = new ArrayList<>();
List<String> list = Files.readAllLines(Paths.get("PeriodicTable.txt"));
list.forEach(this::process);
}
catch (IOException e)
{
e.printStackTrace();
}
}
public static Element getElementByNumber(int atomicNumber)
{
return periodicTable.elements.stream().
filter(e -> e.atomicNumber == atomicNumber).
findFirst().orElse(null);
}
public static Element getElementBySymbol(String symbol)
{
return periodicTable.elements.stream().
filter(e -> e.symbol.equals(symbol)).
findFirst().orElse(null);
}
private void process(String line)
{
try (Scanner scanner = new Scanner(line))
{
int atomicNumber = scanner.nextInt();
String symbol = scanner.next();
String info = scanner.nextLine();
elements.add(new Element(atomicNumber, symbol, info));
}
}
private class Element
{
private String info;
private int atomicNumber;
private String symbol;
public Element(int atomicNumber, String symbol, String info)
{
this.atomicNumber = atomicNumber;
this.symbol = symbol;
this.info = info;
}
public String toString()
{
return "[ " + atomicNumber + " " + symbol + " " + info + " ]";
}
}
}
// Demo.java
public class Demo
{
public static void main(String[] args)
{
System.out.println(PeriodicTable.getElementByNumber(1));
System.out.println(PeriodicTable.getElementBySymbol("Li"));
}
}
Now you can use your PeriodicTable safely and directly as shown in the Demo method.

Using a config/property file with variables

sI use a simple text-file like this
BMG-P (someLongComplicatedExpression)(.*P)
BMG T (someLongComplicatedExpression)(.*[Tt])
BMG MPA (someLongComplicatedExpression)(.*MPA)
to configure my application (Simple import with bufferedReader.readLine().split("\t")). What is bugging me is the redundance.
I am thinking about a solution like this:
%s=(someLongComplicatedExpression)
BMG-P %s(.*P)
BMG T %s(.*[Tt])
BMG MPA %s(.*MPA)
where I read the value of my variables (like %s), then replace their occurrences in the Strings after the import.
My questions are:
What alternative approaches do you know?
What is an easy way to implement the replacement of my variables in my code?
Can you point me to any frameworks that support property-files like that?

I wrote this simple extension to the Java Properties class:
import java.io.Serializable;
import java.util.Properties;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
/**
* Allows properties to contain expansions of the form ${propertyName}. This
* class makes no attempt to detect circular references, so be careful.
*/
public class ExpandingProperties extends Properties implements PropertySource {
private static final long serialVersionUID = 259782782423517925L;
private final Expander expander = new Expander();
#Override
public String getProperty(String key) {
return expander.expand(super.getProperty(key), this);
}
}
class Expander implements Serializable {
private static final long serialVersionUID = -2229337918353092460L;
private final Pattern pattern = Pattern.compile("\\$\\{([^}]+)\\}");
/**
* Expands variables of the form "${variableName}" within the
* specified string, using the property source to lookup the
* relevant value.
*/
public String expand(final String s, final PropertySource propertySource) {
if (s == null) {
return null;
}
final StringBuffer sb = new StringBuffer();
final Matcher matcher = pattern.matcher(s);
while (matcher.find()) {
final String variableName = matcher.group(1);
final String value = propertySource.getProperty(variableName);
if (value == null) {
throw new RuntimeException("No property found for: " + variableName);
}
matcher.appendReplacement(sb, value.replace("$", "\\$"));
}
matcher.appendTail(sb);
return sb.toString();
}
}
interface PropertySource {
String getProperty(String key);
}
Example usage:
public static void main(String[] args) {
Properties properties = new ExpandingProperties();
properties.put("myVar", "myLongExpression");
properties.put("foo", "${myVar}_1");
properties.put("bar", "${foo}_abc");
System.out.println(properties.getProperty("bar"));
}
Prints:
myLongExpression_1_abc
As ExpandingProperties is an extension of Properties it inherits all the load...() methods for loading values from property files.
An alternative is EProperties which does a similar thing to the above code, but goes even further and allows you to nest property files etc. I found it overkill for what I needed.