Reading/writing binary structures: how to simplify this code? - java

I'm writing a network app, which sends and receives a lot of different kinds of binary packets, and I'm trying to make adding new kinds of packets to my app as easy as possible.
For now, I created a Packet class, and I create subclasses of it for each different kind of packet. However, it isn't as clean as it seems; I've ended up with code like this:
static class ItemDesc extends Packet {
public final int item_id;
public final int desc_type;
public final String filename;
public final String buf;
public ItemDesc(Type t, int item_id, int desc_type, String filename, String buf) {
super(t); // sets type for use in packet header
this.item_id = item_id;
this.desc_type = desc_type;
this.filename = filename;
this.buf = buf;
}
public ItemDesc(InputStream i) throws IOException {
super(i); // reads packet header and sets this.input
item_id = input.readInt();
desc_type = input.readByte();
filename = input.readStringWithLength();
buf = input.readStringWithLength();
}
public void writeTo(OutputStream o) throws IOException {
MyOutputStream dataOutput = new MyOutputStream();
dataOutput.writeInt(item_id);
dataOutput.writeByte(desc_type);
dataOutput.writeStringWithLength(filename);
dataOutput.writeStringWithLength(buf);
super.write(dataOutput.toByteArray(), o);
}
}
What bothers me about this approach is the code repetition - I'm repeating the packet structure four times. I'd be glad to avoid this, but I can't see a reasonable way to simplify it.
If I was writing in Python I would create a dictionary of all possible field types, and then define new packet types like this:
ItemDesc = [('item_id', 'int'), ('desc_type', 'byte'), ...]
I suppose that I could do something similar in any functional language. However, I can't see a way to take this approach to Java.
(Maybe I'm just too pedantic, or I got used to functional programming and writing code that writes code, so I could avoid any repetition :))
Thank you in advance for any suggestions.

I agree with #silky that your current code is a good solution. A bit of repetitious (though not duplicated) code is not a bad thing, IMO.
If you wanted a more python-like solution, you could:
Replace the member attributes of ItemDesc with some kind of order-preserving map structure, do the serialization using a common writeTo method that iterates over the map. You also need to add getters for each attribute, and replace all uses of the existing fields.
Replace the member attributes with a Properties object and use Properties serialization instead of binary writes.
Write a common writeTo method that uses Java reflection to access the member attributes and their types and serialize them.
But in all 3 cases, the code will be slower, more complicated and potentially more fragile than the current "ugly" code. I wouldn't do this.

Seem okay to me. You may just want to abstract some of the 'general' parts of the packet up the inheritance chain, so you don't need to read them, but it makes sense to be repeating the format like you are, because you've got a case for reading in raw from the constructor, reading from a stream, and writing. I see nothing wrong with it.

I am not sure you can do this in java- but maybe you could reuse one of the ctors:
public ItemDesc(InputStream i) throws IOException {
super(i); // reads packet header and sets this.input
this(input.readInt(), input.readByte(), input.readStringWithLength(), input.readStringWithLength());
}
Were 'this' means a call to this classes ctor, whtever the syntax might be.

Related

JNA: Change String encoding for only one external native library

We have here an JAVA application which loads and use a lots of external librarys. The default encoding of the operating system (Windows) is "windows-1252" (or "cp-1252"). But there is one external library which want all String (incoming and outgoing) in "utf-8". How can I do that? How can I change the String encoding type for only one JNA library?
The normal JNA pattern is this:
public interface DemoLibrary extends Library {
DemoLibrary INSTANCE = Native.load("demoLibrary", DemoLibrary.class);
// abstract method declarations as interface to native library
}
However, Native#load is overloaded multiple times to support customizing the bindings. The relevant overload is: Native#load(String, Class, Map<String,?>). The third argument can be used to pass options to the native library loader. The options can be found in the com.sun.jna.Library Interface.
The relevant option here is Library.OPTION_STRING_ENCODING. That option is passed to the NativeLibrary instance loaded and will be used as the default encoding for this class.
The sample above becomes then
public interface DemoLibrary extends Library {
DemoLibrary INSTANCE = Native.load("demoLibrary", DemoLibrary.class,
Collections.singletonMap(Library.OPTION_STRING_ENCODING, "UTF-8"));
}
If you need to customize more (typemapper, calling convention), you'll need to create the option map for example in a static initializer block.
Matthias Bläsing's answer is a much better solution to this specific use case. Please read it first if you only need character encoding.
My original answer is below and is more general to a wider range of applications.
An easy way to handle it is to not directly map String fields/args at all. Just send and receive byte arrays from the library, and create a helper function to do the translation between Strings and the byte arrays. As you've pointed out, you can write those bytes to an allocated Memory block and pass the pointer.
If you want a more permanent solution to do the same thing behind the scenes, you can use a TypeMapper for that particular library.
The W32APITypeMapper is a good reference, with the stringConverter variable showing you how in unicode it maps String to the wide string WString (UTF16).
Create your own UTF8TypeMapper (or similar) and use Java's character set/encoding functions to translate your strings to a sequence of UTF-8 bytes.
This is untested, but should be close to what you need. You could do a bit more abstraction to create a new UTF8String type that handles the details.
public class UTF8TypeMapper extends DefaultTypeMapper {
public UTF8TypeMapper() {
TypeConverter stringConverter = new TypeConverter() {
#Override
public Object toNative(Object value, ToNativeContext context) {
if (value == null)
return null;
String str = (String) value;
byte[] bytes = str.getBytes(StandardCharsets.UTF_8);
// Allocate an extra byte for null terminator
Memory m = new Memory(bytes.length + 1);
// write the string's bytes
m.write(0, bytes, 0, bytes.length);
// write the terminating null
m.setByte((long) bytes.length, (byte) 0);
return m;
}
#Override
public Object fromNative(Object value, FromNativeContext context) {
if (value == null)
return null;
Pointer p = (Pointer) value;
// handles the null terminator
return p.getString(0, StandardCharsets.UTF_8.name());
}
#Override
public Class<?> nativeType() {
return Pointer.class;
}
};
addTypeConverter(String.class, stringConverter);
}
}
Then, add the type mapper to the options when loading the library:
private static final Map<String, ?> UTF8_OPTIONS =
Collections.singletonMap(Library.OPTION_TYPE_MAPPER, new UTF8TypeMapper());
TheUTF8Lib INSTANCE = Native.load("TheUTF8Lib", TheUTF8Lib.class, UTF8_OPTIONS);
The only way I have found is to use the Function class of JNA (see at https://java-native-access.github.io/jna/5.2.0/javadoc/com/sun/jna/Function.html ) like this:
public void setIp(String ip) {
Function fSetIp = Function.getFunction("myLib", "setIp", Function.C_CONVENTION, "utf-8");
Object[] args = {ip};
fSetIp.invoke(args);
}
But I must implementing this for each function I want to call. Not sure if there is a better/easier way. If so: Please answer my question.

How to refactor "stringly-typed" code?

I'm currently working on a codebase where there are a few classes of variable, like database paths, which are simply represented as Strings. Most of the operations on these (non-)types are defined in a utility class.
I have created a new class to represent a database, with operations defined as instance methods, in traditional OOP style. However it is quite laborious to go through the large codebase and refactor it to use the new types. Does anyone have any advice as to how to do this quickly and effectively?
Migrate the utility class to use your new class. Then the utility class methods should only contain two statements. One for creating your class and the other is invoking your class. After that, you can inline the utility class methods thereby eliminating the need for it.
When you are finished with that, you need to look for a way to not instantiate your new class over and over again. This should be done by refactoring the local variable to an instance field which is initialized at construction time.
Database paths sound like they should be Strings to me. What else makes sense? And they should be externalized, either in configuration files or a database. That's the least of your problems.
Persistence has been so many times over (e.g. Hibernate, Spring JDBC, iBatis, etc.) that I'd wonder how you could possibly improve on them. If you have to go to the trouble of refactoring - and you must - I'd advise using anything other than what you've done.
If you must write something, Google for "generic DAO". You'll get stuff like this:
http://www.ibm.com/developerworks/java/library/j-genericdao/index.html
If your work isn't patterned after something like that, throw it away and re-think things.
A technique I've used in C# (and just ported to Java - apologies if I've made an error, I'm new to Java) is to create StringlyTyped classes, e.g. a base class
public abstract class StringlyTyped {
private String value;
public StringlyTyped (String value){
if (value == null){
throw new IllegalArgumentException("value must not be null");
}
this.value = value;
}
public String getValue() { return value; }
#Override
public boolean equals(Object other){
if (other == this) {
return true;
}
if (other == null || !this.getClass().equals(other.getClass())){
return false;
}
StringlyTyped o = (StringlyTyped)other;
return o.getValue().equals(this.getValue());
}
#Override
public int hashCode(){ return this.getValue().hashCode(); }
#Override
public String toString() { return this.getValue(); }
}
Then derived class
public class ProviderName extends StringlyTyped {
public ProviderName(String value) {
super(value);
}
}
And usage
public void Foo(ProviderName provider) {
}
It makes sense when you have methods with many String parametrers, e.g. you can avoid
public void Foo(String username, String password, String db, String table, String constraint)
and instead have code that is strongly typed like this:
public void Foo(UserName username, Password password, DatabasePath db, TableName table...)
etc...
I generally try to isolate the strings at the limit of the application/process boundary, such as when they are retrieved from a database or received via a web operation.
At that application/process boundary is often the ideal place to map/convert/deserialize the strings into a more proper object model, as supported by whatever language you are using.
Similarly, the object model can be mapped/converted/serialized back into string form as it exits your application/process boundary.
It is worth noting that this stringly typing can be somewhat subtle. I commonly see xml intruding into application and domain layers. A similar example from the .NET space would be failing to map ADO.NET DataTables (with their string column names and untyped field values) into classes/objects pretty much as soon as they are received. I have no doubt that there are many similar equivalents in the Java world. Stringly Typing is not just limited to date values, as the joke goes.

Calculate total calls made to a method without using static

UPDATE:: OK i am putting the original problem statement here
Given the Main class create a method createPerson and call it in any other method more than once, then on the basis of the number of times the createPerson has been executed you have to initialize the objects and input names of students and output the names.
once i come to know how many objects i have to create its quite trivial to program the later part
for the prior problem of finding the number of objects to be created i have chosen the way of file handling as i come from a C, C++ background where file handling is comparatively simple.
now how should i modify the program such that i write an integer in the file, and later when i will read the file i will get the number of objects
this example forbids the use of static variable, it is a sort of brain teaser
so do Not use static
this is my Main.java file
import java.io.IOException;
public class Main {
public static void main(String[] args) throws IOException{
int i;
Student[] totalStudents = new Student[10];
Student.create3Persons();
Student.create2Persons();
}
}
and this is my Student.java file
import java.io.*;
public class Student {
private static void createPerson() throws IOException{
int number=0;
File file = new File("arg.txt", null);
FileOutputStream fos = new FileOutputStream(file);
DataOutputStream dos = new DataOutputStream(fos);
FileInputStream fis = new FileInputStream(file);
DataInputStream dis = new DataInputStream(fis);
while(dis.readInt()!= -1)
{
number++;
dos.writeInt(1);
}
}
static void create2Persons() throws IOException{
Student.createPerson();
Student.createPerson();
}
static void create3Persons() throws IOException{
Student.createPerson();
Student.createPerson();
Student.createPerson();
}
}
How should i modify this program so that i calculate how many times has been the function createPerson being called??
If you want to calculate how many times a static method has been called, you have to use a static variable - it's genuinely global state. EDIT: Yes, you can use the file system as another repository for global state, but I'd personally say that's generally a bad idea - particularly if it's just to get round a requirement in an academic question.
I guess there are some exceptions to this - if the static method is provably only called from one class, and that class is a singleton, then there could be an instance method in that singleton... but that's pretty much a corner case. In your case, create3Persons and create2Persons are both accessible from anywhere in the package, and they aren't instance methods in a singleton, so therefore there's no single context in which to keep the call count except a static variable.
Perhaps if you could explain a bit more about what you're trying to achieve, we could help you more...
EDIT: Yes, if you can rely on cooperative callers - and you can change the method signatures - then you could keep (say) an AtomicInteger, and make sure you always pass a reference to the same object into the createStudent method. Again, we'd really need to know what the rules of the question are...
i have to disagree with #Jon Skeet, i dont think you have to use a static variable. Why not simply return the number of items created, and keep the total count in a local variable in your main method ? (assuming youre allowed to modifiy method signatures of course, otherwise youre stuck with the global variable solution) you seem to have an unused counter right there already, why not use it?
You can use:
synchronized(this){
System.setProperty("methodCounter",number++);
}
instead of a file based counter. And in the main method:
System.getProperty("methodCounter");
Use AOP (Aspect-Oriented Programming).
You can use a system property to store the quantity of calls...this way you don't have to use the file system or AOP or change method signature.
Mostly when we want to retain value across method, either we go for
Static variable
or pass the variable as reference
or pass the variable and return the updated variable and use it again
int createPerson(int counter);
i got the Solution
instead of file handling i am doing this
static int createPerson(int c){
return (++c);
}
static int create2Persons(int c) {
return Student.createPerson(Student.createPerson(c));
}
static int create3Persons(int c) {
return Student.createPerson(Student.createPerson(Student.createPerson(c)));
}
public static int create5Persons(int i) {
return Student.createPerson(Student.createPerson(Student.createPerson(Student.createPerson(Student.createPerson(i)))));
}
and in the main method i have initialised an integer and i do this
c += Student.create2Persons(i)-1;
c += Student.create3Persons(i)-1;
c += Student.create5Persons(i)-1;
System.out.println(c+"c is");
this saves use of files
how to tag the thread as answered?

Java method invocation parser?

I need to parse certain method invocation calls including the whole signature out of some Java classes, e.g.
public class MyClass {
public void myMthod() {
// ... some code here
result = someInstance.someOtherMethod(param1, param2);
// ... some other code here
}
}
As a result I would like to get something like:
serviceName = someInstance
methodName = someOtherMethod
arguments = {
argument = java.lang.String,
argument = boolean
}
result = java.lang.Long
What would be the fastest way to achieve this?
I was thinking about using a RegEx parser. The problem there is there are several occurence patterns, e.g.
a)
result = someInstance.someOtherMethod(getSomething(), param);
b)
result =
getSomeInstance().someOtherMethod(param);
c)
result = getSomeInstance()
.someOtherMethod(
getSomethingElse(), null, param);
Any help would be really appreciated! Thanks!
Don't use regex! Use tools that understand java.
Use either:
A source code parser (e.g. javaparser)
Byte code analysis (e.g. ASM)
An aspect (AspectJ)
In both source parser and ASM, you will write a visitor that scans for method invocations.
For javaparser: Read this page, extend VoidVisitorAdapter and override
public void visit(MethodCallExpr n, A arg)
Sample Code:
public static void main(final String[] args) throws Exception{
parseCompilationUnit(new File("src/main/java/foo/bar/Phleem.java"));
}
public static void parseCompilationUnit(final File sourceFile)
throws ParseException, IOException{
final CompilationUnit cu = JavaParser.parse(sourceFile);
cu.accept(new VoidVisitorAdapter<Void>(){
#Override
public void visit(final MethodCallExpr n, final Void arg){
System.out.println(n);
super.visit(n, arg);
}
}, null);
}
The problem here is that you only have the object names, not the object types, so you will also have to keep a local Map of variable / field to type and that's where things get messy. Perhaps ASM is the easier choice, after all.
For ASM: read this tutorial page to get started
Realistically you can't parse a typical programming langauge like Java with regular expressions; you need a full parser. In your case, you need a parser that is capable of parsing arbitrary substrings (in this case, function call nonterminals) of the langauge, and those are much harder to find.
You also didn't say how you intended to find (and delimit) the calls of interest, before you decided to call them. You are likely to need Java symbol table data in order to just pick the method names you want; oherwise,
how will you know that "someService" resolves the actual service class that you care about, and not just some arbitrary class that happens to have what appears to be the right name?

Java: Using one function to return a few different types of value

I have a readData() function that reads files a returns a few different objects of parsed data. Right now, the return type of readData() is Object[]:
Object[] data = readData();
MyGenome genome = data[0];
Species[] breeds = data[1];
//etc
This feels awkward. Is there a better way to return this data? I don't want to have separate functions like readGenome() and readSpecies() because that would require iterating over the files twice. Also, I'd rather wrap up all the gathering of data in one function.
A similar issue: a function that returns a match of at least four characters between two strings:
public int[][] findMatch(String g0, String g1) { /* cool stuff */ }
//...
int[][] bounds = findMatch("ACOIICOCOCICOICA", "AOCCCCCCICCIIIIIOIAOCICOICOICA");
where bounds[0][0] is the left bound on the g0, bounds[0][1] is the right bound on g0, bounds[1][0] is the left bound on g1, etc. This also feels sort of awkward. It is difficult to code with the result without continuously looking up the keys.
Create a new Class:
class MyAnalysedGenome {
MyGenome genome;
Species[] species
...
}
and return that. You'll probably find you have other functionality that should go in there too. Perhaps the code that surrounds your getData() call.
How about using a strongly typed class to represent the complex return type of readData()?
public class Taxonomy
{
public MyGenome genome;
public Species[] breeds;
//etc
{
Taxonomy data = readData();
You can do the same thing for your search bounds problem
public class SearchBoundary
{
public int left;
public int right;
}
SearchBoundary resultBounds = findMatch(searchBounds);
For the first issue,couldn't you simply use an intermediate data representation ? I mean you could read your file once, which would give you the file content (that you could format the way you want), and then create two methods readGenome() and readSpecies() that would take this file content as a parameter.
You can create a class that have genome and species as fields.
...
class DataToBeRead {
MyGenome genome;
Species[] breeds;
}
...
DataToBeRead data = readData();
MyGenome genome = data.genome;
Species[] breeds = data.breeds;
You can make the class private if you do not think anybody else will used it or make it public if someone else will use it.
You can also make it static if you do not want to create a separate file for it.
Hope this helps.

Categories