We Keep Coding

QuickFIX/J Get fields and groups for customized data dictionary

How can I get fields and groups using QuickFIX/J for customized data dictionary?
I receive market data transmitted in customized MarketDataSnapshotFullRefresh (type W) FIX messages. As I understood, I can't use the crack method for this. I'm not quite familiar with Java and QuickFIX/J, but when I use QuickFIX/n and Python, I can define classes for fields and groups like that:
class CustomField(fix.StringField):
tag_number = *SomeTagNumber*
def __init__(self, data=None):
args = (self.tag_number,) if data is None else (self.tag_number, data)
super(CustomField, self).__init__(*args)
def getValue(self, message: object) -> str:
try:
if message.getField(self.tag_number):
return message.getField(self.tag_number)
except fix.FieldNotFound:
return None
else: raise
class CustomGroupField(fix.StringField):
tag_number = *SomeTagNumber*
def __init__(self, data=None):
args = (self.tag_number,) if data is None else (self.tag_number, data)
super(CustomGroupField, self).__init__(*args)
def getValue(self, message: object) -> str:
try:
if message.getField(self.tag_number):
return message.getField(self.tag_number)
except fix.FieldNotFound:
return None
else: raise
class XXXGroup(fix.Group):
def __init__(self):
order = fix.IntArray(4)
order[0] = No_XXX_UMD_Entries.tag_number # This is the NoGroup field
order[1] = XXX_UMD_Entry_ID.tag_number # This is the field in the repeating group
order[2] = CustomGroupField.tag_number
order[3] = 0
# fix.Group.__init__(self, order[0], order[1], order)
args = (order[0], order[1], order)
super(XXXGroup, self).__init__(*args)
def getValue(self, field: object) -> str:
try:
if group.getField(tag_number):
return group.getField(tag_number)
except fix.FieldNotFound:
return None
else: raise
And then I can get value inside the fromApp(self, message, sessionID) method like this:
# Get value of the field
some_custom_field = CustomField().getValue(message)
# Get value in the group
group = XXXGroup()
for idx in range(1, no_entries+1):
message.getGroup(idx,group)
custom_gr_field = group.getValue(CustomGroupField)
How can I achieve the same logic in Java using QuickFIX/J? Or maybe there is a better way to work with a custom data dictionary in Java? Maybe you can refer to some examples?

QuickFIX/J use XML files as Dictionary.
To parse message you should create dictionary and use it. The code looks like:
DataDictionary dataDictionary = new DataDictionary("FIX44.xml");
Message message = new Message();
message.fromString(text, dataDictionary, false);
Standard dictionary stored in quickfixj jars resources. For example, dictionary for FIX 4.4 stores in https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-messages/quickfixj-messages-fix44/src/main/resources/FIX44.xml
For custom dictionary you should create a XML file with all additional fields and groups.
You could copy the standard file, rename it, add group and use like: DataDictionary dataDictionary = new DataDictionary("/var/my_dict/myFIX44.xml");

Write elements of a map to a CSV correctly in a simplified way in Java 8

I have a countries Map with the following design:
England=24
Spain=21
Italy=10
etc
Then, I have a different citiesMap with the following design:
London=10
Manchester=5
Madrid=7
Barcelona=4
Roma=3
etc
Currently, I am printing these results on screen:
System.out.println("\nCountries:");
Map<String, Long> countryMap = countTotalResults(orderDataList, OrderData::getCountry);
writeResultInCsv(countryMap);
countryMap.entrySet().stream().forEach(System.out::println);
System.out.println("\nCities:\n");
Map<String, Long> citiesMap = countTotalResults(orderDataList, OrderData::getCity);
writeResultInCsv(citiesMap);
citiesMap.entrySet().stream().forEach(System.out::println);
I want to write each line of my 2 maps in the same CSV file. I have the following code:
public void writeResultInCsv(Map<String, Long> resultMap) throws Exception {
File csvOutputFile = new File(RUTA_FICHERO_RESULTADO);
try (PrintWriter pw = new PrintWriter(csvOutputFile)) {
resultMap.entrySet().stream()
.map(this::convertToCSV)
.forEach(pw::println);
}
}
public String convertToCSV(String[] data) {
return Stream.of(data)
.map(this::escapeSpecialCharacters)
.collect(Collectors.joining("="));
}
public String escapeSpecialCharacters(String data) {
String escapedData = data.replaceAll("\\R", " ");
if (data.contains(",") || data.contains("\"") || data.contains("'")) {
data = data.replace("\"", "\"\"");
escapedData = "\"" + data + "\"";
}
return escapedData;
}
But I get compilation error in writeResultInCsv method, in the following line:
.map(this::convertToCSV)
This is the compilation error I get:
reason: Incompatible types: Entry is not convertible to String[]
How can I indicate the following result in a CSV file in Java 8 in a simplified way?
This is the result and design that I want my CSV file to have:
Countries:
England=24
Spain=21
Italy=10
etc
Cities:
London=10
Manchester=5
Madrid=7
Barcelona=4
Roma=3
etc

Your resultMap.entrySet() is a Set<Map.Entry<String, Long>>. You then turn that into a Stream<Map.Entry<String, Long>>, and then run .map on this. Thus, the mapper you provide there needs to map objects of type Map.Entry<String, Long> to whatever you like. but you pass the convertToCSV method to it, which maps string arrays.
Your code tries to join on comma (Collectors.joining(",")), but your desired output contains zero commas.
It feels like one of two things is going on:
you copy/pasted this code from someplace or it was provided to you and you have no idea what any of it does. I would advise tearing this code into pieces: Take each individual piece, experiment with it until you understand it, then put it back together again and now you know what you're looking at. At that point you would know that having Collectors.joining(",") in this makes no sense whatsoever, and that you're trying to map an entry of String, Long using a mapping function that maps string arrays - which obviously doesn't work.
You would know all this stuff but you haven't bothered to actually look at your code. That seems a bit surprising, so I don't think this is it. But if it is - the code you have is so unrelated to the job you want to do, that you might as well remove your code entirely and turn this question into: "I have this. I want this. How do I do it?"
NB: A text file listing key=value pairs is not usually called a CSV file.

SpringBoot Path variable problem with slash

I have some problem with slashes in variable thats hould be send in url.
For example id could be like this:
ID: /gfdge1/DkxA8P+jYw43
URL: localhost:8080/find-user//gfdge1/DkxA8P+jYw43
"error": "Internal Server Error",
"exception": "org.springframework.security.web.firewall.RequestRejectedException",
"message": "The request was rejected because the URL was not normalized.",
because of this slash on first place it make problem
Before that i had problems with these slash in middle of ID but I've solved that with this code:
#RequestMapping(name = "Get user data by ID", value = "/find-user/{userId}/**", method = RequestMethod.GET)
#ResponseBody
public User getUserData(#PathVariable String userId, HttpServletRequest request) {
final String path =
request.getAttribute(HandlerMapping.PATH_WITHIN_HANDLER_MAPPING_ATTRIBUTE).toString();
final String bestMatchingPattern =
request.getAttribute(HandlerMapping.BEST_MATCHING_PATTERN_ATTRIBUTE).toString();
String arguments = new AntPathMatcher().extractPathWithinPattern(bestMatchingPattern, path);
String id="";
if (null != arguments && !arguments.isEmpty()) {
id = userId + '/' + arguments;
} else {
id = userId;
}
return userService.getUserData(id);
}
but this doesn't work for this case when slash is on first place.
I've also try to user RequestParam instead of PathVariable, but it have problems with some special characters for example when I user RequestParam it replace '+' with empty space,...
Does anyone can help me how to solve this problem?

Its an inherent issue with using Strings as path variables, it's not an issue with your code but how the HTTP request is interpreted so you can't do anything in your code to make this work.
You do have some options though:
Ensure the values you use cannot be created with special characters such as "/"
Avoid using Strings in path variables completely.
I lean more towards 2 as maintaining 1 for all possible problem characters/strings is pretty messy and unnecessary.
To do 2 correctly you should consider having all your REST getters finding their related entities by a numeric ID only e.g.
localhost:8080/find-user/3
If you need to add additional search parameters e.g. username in your case then you should use something like QueryDSL to create a predicate of search parameters which are passed as query parameters instead of path variables e.g.:
localhost:8080/find-user?username=/gfdge1/DkxA8P+jYw43

Spring REST Controller understanding arrays of strings when having special characters like blank spaces or commas

I am trying to write a Spring REST Controller getting an array of strings as input parameter of a HTTP GET request.
The problem arises when in the GET request, in some of the strings of the array, I use special characters like commas ,, blank spaces or forward slash /, no matter if I URL encode the query part of the URL HTTP GET request.
That means that the string "1/4 cup ricotta, yogurt" (edit which needs to be considered as a unique ingredient contained as a string element of the input array) in either this format:
http://127.0.0.1:8080/[...]/parseThis?[...]&ingredients=1/4 cup ricotta, yogurt
This format (please note the blank spaces encoded as + plus, rather than the hex code):
http://127.0.0.1:8080/[...]/parseThis?[...]&ingredients=1%2F4+cup+ricotta%2C+yogurt
Or this format (please note the blank space encoded as hex code %20):
http://127.0.0.1:8080/[...]/parseThis?[...]&ingredients=1%2F4%20cup%20ricotta%2C%20yogurt
is not rendered properly.
The system does not recognize the input string as one single element of the array.
In the 2nd and 3rd case the system splits the input string on the comma and returns an array of 2 elements rather than 1 element. I am expecting 1 element here.
The relevant code for the controller is:
#RequestMapping(
value = "/parseThis",
params = {
"language",
"ingredients"
}, method = RequestMethod.GET, headers = HttpHeaders.ACCEPT + "=" + MediaType.APPLICATION_JSON_VALUE)
#ResponseBody
public HttpEntity<CustomOutputObject> parseThis(
#RequestParam String language,
#RequestParam String[] ingredients){
try {
CustomOutputObject responseFullData = parsingService.parseThis(ingredients, language);
return new ResponseEntity<>(responseFullData, HttpStatus.OK);
} catch (Exception e) {
// TODO
}
}
I need to perform HTTP GET request against this Spring controller, that's a requirement (so no HTTP POST can be used here).
Edit 1:
If I add HttpServletRequest request to the signature of the method in the controller, then I add a log statement like log.debug("The query string is: '" + request.getQueryString() + "'"); then I am seeing in the log a line like The query string is: '&language=en&ingredients=1%2F4+cup+ricotta%2C+yogurt' (So still URL encoded).
Edit 2:
On the other hand if I add WebRequest request to the signature of the method, the the log as log.debug("The query string is: '" + request.getParameter("ingredients") + "'"); then I am getting a string in the log as The query string is: '1/4 cup ricotta, yogurt' (So URL decoded).
I am using Apache Tomcat as a server.
Is there any filter or something I need to add/review to the Spring/webapp configuration files?
Edit 3:
The main problem is in the interpretation of a comma:
#ResponseBody
#RequestMapping(value="test", method=RequestMethod.GET)
public String renderTest(#RequestParam("test") String[] test) {
return test.length + ": " + Arrays.toString(test);
// /app/test?test=foo,bar => 2: [foo, bar]
// /app/test?test=foo,bar&test=baz => 2: [foo,bar, baz]
}
Can this behavior be prevented?

The path of a request parameter to your method argument goes through parameter value extraction and then parameter value conversion. Now what happens is:
Extraction:
The parameter is extracted as a single String value. This is probably to allow simple attributes to be passed as simple string values for later value conversion.
Conversion:
Spring uses ConversionService for the value conversion. In its default setup StringToArrayConverter is used, which unfortunately handles the string as comma delimited list.
What to do:
You are pretty much screwed with the way Spring handles single valued request parameters. So I would do the binding manually:
// Method annotations
public HttpEntity<CustomOutputObject> handlerMethod(WebRequest request) {
String[] ingredients = request.getParameterValues("ingredients");
// Do other stuff
}
You can also check what Spring guys have to say about this.. and the related SO question.

Well, you could register a custom conversion service (from this SO answer), but that seems like a lot of work. :) If it were me, I would ignore the declaration the #RequestParam in the method signature and parse the value using the incoming request object.

May I suggest you try the following format:
ingredients=egg&ingredients=milk&ingredients=butter
Appending &ingredients to the end will handle the case where the array only has a single value.
ingredients=egg&ingredients=milk&ingredients=butter&ingredients
ingredients=milk,skimmed&ingredients
The extra entry would need to be removed from the array, using a List<String> would make this easier.
Alternatively if you are trying to implement a REST controller to pipe straight into a database with spring-data-jpa, you should take a look at spring-data-rest. Here is an example.
You basically annotate your repository with #RepositoryRestResource and spring does the rest :)

A solution from here
public String get(WebRequest req) {
String[] ingredients = req.getParameterValues("ingredients");
for(String ingredient:ingredients ) {
System.out.println(ingredient);
}
...
}
This works for the case when you have a single ingredient containing commas

Best Way to Append Integers to End of Username?

I'm working on a Java class that gets parameters from an "Account Request" form (like 'First Name', 'Last Name', 'email', etc.).
The Java class first gets the parameters from the form. It then initializes a user name with the first and last name strings like so:
String userName = firstName.substring(0,1).toLowerCase() + lastName.toLowerCase();
For example, the username for "Jake Smith" would be jsmith.
Afterwards, it checks to see if this 'userName' exists in the database. Obviously, "Jake Smith" and "John Smith" would create identical user names, "jsmith". To account for this, I'd just like to append numbers, starting with 1, to any matching user names.
How can I append a unique number at the end of each and every conflicting username (starting with 1 and incremented by 1 for each additional conflict)?
Obviously, a more elegant solution would be to allow the user to specify their user name, but this is how the mock client wants the project to be completed.
Thank you!

You could use String.format, something like:
String.format("%s%d", username, i++);

A "dumb" solution, but if it's just for tests...
private String getUsername(){
String userName = firstName.substring(0,1).toLowerCase() + lastName.toLowerCase();
int i=1;
while(usernameExists(userName)){
userName = userName.replaceAll("\\d+$","") + i++;
}
return userName;
}
Considering that userName don't have digits in the end.

Use a StringBuilder to construct the new string and StringBuilder#toString to obtain it.

Here's a simple method that uses a Map to keep track of ids.
class UniqueUsername {
private static Map<String, Integer> ids = new HashMap<String, Integer>();
public static String getUniqueUsername(String username) {
if (!ids.containsKey(username)) {
ids.put(username, new Integer(0));
return username;
}
ids.put(username, ids.get(username)+1);
return username + ids.get(username);
}
}

If conflicts are unlikely then you could just loop until unique, staring at 1 and adding 1 each time
For example:
i=0;
String usernameToTry=username;
while (notUnique(usernameToTry)) {
i++;
usernameToTry=username+i;
}
If conflicts are likely then you may find this approach unsatisfactory because of the performance penalty of hitting the database (to check if unique) for every iteration of the loop. For example, if you have jsmith100 already then you don't want to check 100 times before coming up with jsmith101.