We Keep Coding

How to create Junit testcases for lambda expression using Mockito Junit 5?

I want to create Junit TestCases of method, in which we are iterating List<Map<String,Object>> using forEach loop with lambda expresion. Now I want to mock statement objectMapper.writeValueAsString(recordObj.get("value")); but I am not understanding how to use recordObj.
public String apply(MyRequestWrapper requestWrapper) {
String resultStr=null;
final Map<String, List<PubSubEvent>> packagesEventList = AppUtilities.getPackagesEventsMappedList();
try {
logger.debug("Received Record:: " + requestWrapper.getBody().toString());
List<RecordProcessedResult> results = new ArrayList<>();
List<Map<String,Object>> recordMaps= string2List(objectMapper,requestWrapper.getBody().toString());
logger.debug("Parsed received payload ::: "+ LocalDateTime.now() + " batch size is ::: "+ recordMaps.size());
if(! ObjectUtils.isEmpty(recordMaps) && !recordMaps.isEmpty() ) {
recordMaps.forEach(recordObj ->{
ConsumerRecord record=objectMapper.convertValue(recordObj, ConsumerRecord.class);
String topicName = recordObj.get("topic").toString();
String key = null;
String value = null;
String offset = null;
String xTraceabilityId = ((Map<String, String>) recordObj.get("headers")).get(IdTypeConstants.XTRACEABILITYID);
String xCorrelationId = ((Map<String, String>) recordObj.get("headers")).get(IdTypeConstants.XCORRELATIONID);
MDC.put(IdTypeConstants.XTRACEABILITYID, xTraceabilityId);
MDC.put(IdTypeConstants.XCORRELATIONID, xCorrelationId);
try {
key = objectMapper.writeValueAsString(recordObj.get("key"));
value = objectMapper.writeValueAsString(recordObj.get("value"));
offset = objectMapper.writeValueAsString(recordObj.get("offset"));
MyEvent myEvent= objectMapper.readValue(value, MyEvent.class);
subscribedPackageProcessor.setInput(input);
subscribedPackageProcessor.setOutput(output);
subscribedPackageProcessor.setPackagesEventList(packagesEventList);
subscribedPackageProcessor.setRequesterType(requesterType); subscribedPackageProcessor.processSubscribedPackage(myEvent.getPackageId());
RecordProcessedResult rpr = new RecordProcessedResult(record, true, null, xTraceabilityId, xCorrelationId, key, System.currentTimeMillis());
results.add(rpr);
}
catch(Exception e) {
RecordProcessedResult rpr = new RecordProcessedResult(record, false, ExceptionUtils.getStackTrace(e), xTraceabilityId, xCorrelationId, key, System.currentTimeMillis());
results.add(rpr);
logger.info("Exception occured while processing fund data :::out ", e);
}
MDC.clear();
});
}
resultStr = objectMapper.writeValueAsString(results);
}catch (Exception e) {
logger.debug(e.getMessage());
}
return resultStr;
}
I have tried following testcases.
#Test void applyTest() throws Exception {
MyEvent myEvent = new MyEvent();
myEvent.setPackageId("test");
MyRequestWrapper flowRequestWrapper= getMyRequestWrapper();
List<Map<String, Object>> maps = string2List(objectMapper1, flowRequestWrapper.getBody().toString());
Map<String,Object> map = new HashMap<String, Object>();
Mockito.when(objectMapper.readValue(Mockito.anyString(), Mockito.any(TypeReference.class))).thenReturn(maps);
Mockito.when(objectMapper.writeValueAsString(Mockito.anyString())).thenReturn("test");
Mockito.when(objectMapper.readValue(Mockito.anyString(), Mockito.eq(MyEvent.class))).thenReturn(myEvent);
//doNothing().when(subscribedPackageProcessor).processSubscribedPackage("");
String response = processESignCompletedEventSvcFlow.apply(flowRequestWrapper);
Assertions.assertNotNull(response);
}
Please help, Thanks

Your method is way too complex to be unit tested. For example it declares dependencies by calling methods in the same class. You cannot mock those and it makes the testing many times more complicated.
List<Map<String,Object>> recordMaps =
string2List(objectMapper,requestWrapper.getBody().toString());
You need to extract the string2List method into a standalone class (with it's own unit tests) that is injected into your class as a dependency.
Then you can just mock the string2List class and when you do that, you control the creation of recordObj instances from your unit test for this method.
Your second "sin" is abusing lambdas by creating one that is longer than two lines. Lambdas should be short. If it spans more than a few lines, it must be extracted into a standalone class that can be unit tested separately. And again, when you have extracted this lambda into a standalone class and unit tested it, you can't just go "new RecordObjConsumer(results)" in your method, as that creates a hard-coded dependency that you again cannot mock. You need to design the consumer so that it can be injected into your class as an external dependency.

Creating complex BigQuery Schema in Google DataFlow (java)

I have an unbounded stream of complex objects that I want to load into BigQuery. The structure of these objects represents the schema of my destination table in BigQuery.
The problem is that since there are a lot of nested fields in the POJO, its an extremely tedious task to convert it to a TableSchema object and I'm looking for a quick/ automated way to convert my POJO to TableSchema object while writing to BigQuery.
I'm not very familiar with Apache Beam API, and any help will be appreciated.

In a pipeline, I load a list of schema from GCS. I keep them in string format because the TableSchema is not serializable. However, I load them to TableSchema for validate them.
Then I add them in string format to a map in the Option object.
String schema = new String(blob.getContent());
// Decorate list of fields for allowing a correct parsing
String targetSchema = "{\"fields\":" + schema + "}";
try {
//Preload schema to ensure validity, but then use string version
Transport.getJsonFactory().fromString(targetSchema, TableSchema.class);
String tableName = blob.getName().replace(SCHEMA_FILE_PREFIX, "").replace(SCHEMA_FILE_SUFFIX, "");
tableSchemaStringMap.put(tableName, targetSchema);
} catch (IOException e) {
logger.warn("impossible to read schema " + blob.getName() + " in bucket gs://" + options.getSchemaBucket());
}
I didn't find another solution when I developed this.

In my company I created kind of a ORM (we called OBQM) to do this. We are expecting to release it to the public. The code is quite big (specially because I created annotations and so on) but I can share with you some snippets for a quick schema generation:
public TableSchema generateTableSchema(#Nonnull final Class cls) {
final TableSchema tableSchema = new TableSchema();
tableSchema.setFields(generateFieldsSchema(cls));
return tableSchema;
}
public List<TableFieldSchema> generateFieldsSchema(#Nonnull final Class cls) {
final List<TableFieldSchema> schemaFields = new ArrayList<>();
final Field[] clsFields = cls.getFields();
for (final Field field : clsFields) {
schemaFields.add(fromFieldToSchemaField(field));
}
return schemaFields;
}
This code takes all the fields from the POJO class and creates a TableSchema object (the one that BigQueryIO uses in ApacheBeam). You can see a method that I created called fromFieldToSchemaField. This method identifies each field type and setup the field name, mode, description and type. In this case to keep it simple I'm going to focus on the type and name:
public static TableFieldSchema fromFieldToSchemaField(#Nonnull final Field field) {
return fromFieldToSchemaField(field, 0);
}
public static TableFieldSchema fromFieldToSchemaField(
#Nonnull final Field field,
final int iteration) {
final TableFieldSchema schemaField = new TableFieldSchema();
final Type customType = field.getGenericType().getTypeName()
schemaField.setName(field.getName());
schemaField.setMode("NULLABLE"); // You can add better logic here, we use annotations to override this value
schemaField.setType(getFieldTypeString(field));
schemaField.setDescription("Optional"); // Optional
if (iteration < MAX_RECURSION
&& (isStruct(schemaField.getType())
|| isRecord(schemaField.getType()))) {
final List<TableFieldSchema> schemaFields = new ArrayList<>();
final Field[] fields = getFieldsFromComplexObjectField(field);
for (final Field subField : fields) {
schemaFields.add(
fromFieldToSchemaField(
subField, iteration + 1));
}
schemaField.setFields(schemaFields.isEmpty() ? null : schemaFields);
}
return schemaField;
}
And now the method that returns the BigQuery field type.
public static String getFieldTypeString(#Nonnull final Field field) {
// On my side this code is much complex but this is a short version of that
final Class<?> cls = (Class<?>) field.getGenericType()
if (cls.isAssignableFrom(String.class)) {
return "STRING";
} else if (cls.isAssignableFrom(Integer.class) || cls.isAssignableFrom(Short.class)) {
return "INT64";
} else if (cls.isAssignableFrom(Double.class)) {
return "NUMERIC";
} else if (cls.isAssignableFrom(Float.class)) {
return "FLOAT64";
} else if (cls.isAssignableFrom(Boolean.class)) {
return "BOOLEAN";
} else if (cls.isAssignableFrom(Double.class)) {
return "BYTES";
} else if (cls.isAssignableFrom(Date.class)
|| cls.isAssignableFrom(DateTime.class)) {
return "TIMESTAMP";
} else {
return "STRUCT";
}
}
Keep in mind that I'm not showing how to identify primitive types or arrays. But this is a good start for your code :). Please let me know if you need any help.

If your using JSON for the message serialization in PubSub you can make use of one of the provided templates:
PubSub To BigQuery Template
The code for that template is here:
PubSubToBigQuery.java

JPA Query Column Specification [duplicate]

I just wanted to know how to pass column name and its value to #Query annotation in Spring Data JPA.
Basically column names will be static and we used to put every column as a element in Entity class. But here I want something different, here column name will be dynamic I will be passing this value as Parameter to the method defined in repository.
Table - Calendar
Columns - id, PersonName, 1, 2, 3......31
Above is the table structure, 1,2,3,.....31 are the column names which represents calendar days and we have values in that columns. I'm using Spring Data JPA to fetch data from DB.
Here I just wanted to fetch person name for a particular day.
Below given the function defined in repository.
#Query("select c from Calendar c where :calendarDay=:value")
List<Calendar> getPersonName(#Param("calendarDay") String calendarDay, #Param("value") String value);
This is not working for me.
Any help would be appreciated.

The only dynamic parameter Spring JPA supports is #{#entityName}. Dynamic column names in #Query annotations are not supported., and that is what you are trying to accomplish.
Your only option is to construct a query manually using either QueryDSL, Specifications or Criteria API or simply by building a query string and passing it to your EntityManager. Regardless, you'll have to write code for that.
See, for instance:
https://spring.io/blog/2011/04/26/advanced-spring-data-jpa-specifications-and-querydsl/
How to add custom column name Spring Data JPA?

Take a look at sping data Specifications. You can find your solution there!
Reading the docs you can see that if Calendar is your domain (I would try to find a different name for my domain, there is a Calendar class in Java SE already), then you could use something like the above,
#Repository
public interface CalendarRepository extends JpaRepository<Calendar, Integer>, JpaSpecificationExecutor<Calendar> {
}
public class CalendarSpecification implements Specification<Calendar> {
private String randomColumnName; // A varchar column.
private String valueToSearchFor;
public CalendarSpecification(String randomColumnName, String valueToSearchFor) {
this.randomColumnName = randomColumnName;
this.valueToSearchFor = valueToSearchFor;
}
#Override
public Predicate toPredicate(Root<Calendar> root, CriteriaQuery<?> query, CriteriaBuilder builder) {
return builder.and(builder.equal(root.<String>get(this.randomColumnName), this.valueToSearchFor));
}
}
#Service
public class CalendarService {
#Autowired
private CalendarRepository calendarRepository;
public List<Calendar> findCustom(String randomColumnName, String valueToSearchFor) {
CalendarSpecification cs = new CalendarSpecification(randomColumnName, valueToSearchFor);
return calendarRepository.find(cs);
// Or using lambda expression - without the need of CalendarSpecification class.
// return calendarRepository.find((Root<ProductCategory> root, CriteriaQuery<?> query, CriteriaBuilder builder) -> {
// return builder.and(builder.equal(root.<String>get(randomColumnName), valueToSearchFor));
// });
}
}

Maybe you can use CASE, WHEN.
SELECT
Id,
PersonName,
CASE
WHEN ? = 'day_01' THEN day_01
WHEN ? = 'day_02' THEN day_02
WHEN ? = 'day_03' THEN day_03
WHEN ? = 'day_04' THEN day_04
WHEN ? = 'day_05' THEN day_05'
ELSE 0
END
AS Value FROM Calendar
Java Code
// customize entity
public interface ITask {
Long getId();
String getName();
String getValue();
}
#Repository
public interface CalendarRepository {
static final String CASE_WHEN = "\nCASE\n"
+ " WHEN :field = 'day_01' THEN day_01\n"
+ " WHEN :field = 'day_02' THEN day_02\n"
+ " WHEN :field = 'day_03' THEN day_03\n"
+ " WHEN :field = 'day_04' THEN day_04\n"
+ " WHEN :field = 'day_05' THEN day_05\n"
+ " ELSE 0\n"
+ "END\n";
#Query(nativeQuery = true, value = "SELECT Id, PersoneName, " + CASE_WHEN + " AS Value FROM Calendar WHERE field = :field")
public List<ITask> findValues(#Param(value = "field") String field);
}

Java creating an api to expose a csv instead of json

I am working on a Java api - using Spring Boot - I would like to create a controller that exports data from the db - into a csv that a user can download.
This is an example of what I have making Json responses.
// get chart
#SuppressWarnings("unchecked")
public Object getChart() {
// build clean object
JSONObject contents = new JSONObject();
//inverse bubble chart
JSONObject chart = new JSONObject();
chart.put("label", "250 applicants");
chart.put("value", 120);
contents.put("chart", chart);
contents.put("number", "0202 000 000");
JSONObject json = new JSONObject();
json.put("contents", contents);
return json;
}
I've seen this example -- but its being called from a reactjs framework - so not sure how I would fetch the HttpServletResponse?
Create and Download CSV file Java(Servlet)
would I invoke the api as usual?
//api/getMyCsv
#SuppressWarnings("unchecked")
#RequestMapping(value = {"/api/getMyC"}, method = RequestMethod.GET)
#CrossOrigin(origins = {"*"})
public ResponseEntity<?> getHome(
//HttpServletRequest request
) throws Exception {
JSONObject chart = getChart();
JSONArray data = new JSONArray();
data.add(chart);
//create empty response
JSONObject response = new JSONObject();
//create success response
response.put("data", data);
response.put("status", "success");
response.put("msg", "fetched csv");
return new ResponseEntity<>(response, HttpStatus.OK);
}
so with the react - using axios
export function fetchCsv(data) {
let url = "http://www.example.com/api/getMyC";
return function (dispatch) {
axios.get(url, {
params: data
})
.then(function (response) {
response = response.data.data;
dispatch(alertSuccess(response));
})
.catch(function (error) {
dispatch(alertFail(error));
});
}
}

CSV is just comma separated values right?
So, you can represent a row of data for example, as a class.
Take an address:
30 my street, my town, my country
if I wanted to represent that data as a class, and later as CSV data I'd make a class something like this:
public class AddressCSV{
private String street;
private String town;
private String country;
public AddressCSV(String street, String town, String country){
this.street = street;
this.town = town;
this.country = country;
}
// getters and setters here
// here you could have a method called generateCSV() for example
// or you could override the toString() method like this
#Override
public String toString(){
return street + "," + town + "," + country + "\n"; // Add the '\n' if you need a new line
}
}
Then you use it the same way as your JSONObject, except instead of returning the whole object you do return address.toString();
This is a very simple example of course. Checkout the StringBuilder class if your have a lot of things to build.
Overriding the toString() method means you can do things like pass your object like System.out.printline(address) and it will print the CSV.

The Spring way to help (un)marshaling data is to implement and register an HttpMessageConverter.
You could implement one that specifically handles MediaType text/csv, and supports whatever object types you decide to implement, e.g.
List<List<?>> - Row is a list of values, auto-converted to string.
List<Object[]> - Row is an array of values, auto-converted to string.
List<String[]> - Row is an array of string values.
List<?> - Row is an bean object, field names added as header row.
You may even implement your own field annotations to control column order and header row values.
You could also make it understand JAXB and/or JSON annotations.
With such a converter in place, it's now easy to write Controller methods returning CSV:
#GetMapping(path="/api/getMyC", produces="text/csv")
public List<String[]> getMyCAsCsv() {
List<Object[]> csv = Arrays.asList(
new Object[] { "First Name", "Last Name", "Age" },
new Object[] { "John" , "Doe" , 33 },
new Object[] { "Jane" , "Smith" , 29 }
);
return csv;
}

DO NOT try doing basic String trickery because the address, especially the street is bound to contain characters such as commas, quotes or line endings. This can easily mess up the CSV output. These characters need to be properly escaped in order to generate a parseable CSV output. Use a CSV library for that instead.
Example with univocity-parsers:
ResultSet rs = queryDataFromYourDb();
StringWriter output = new StringWriter(); //writing to a String to make things easy
CsvRoutines csvRoutine = new CsvRoutines();
csvRoutine.write(rs, output);
System.out.println(output.toString());
For performance, can write to an OutputStream directly and add more stuff to it after dumping your ResultSet into CSV. In such case you will want to keep the output open for writing. In this case call:
csvRoutine.setKeepResourcesOpen(true);
Before writing the ResultSet. You'll have to close the resultSet after writing.
Disclaimer: I'm the author of this library. It's open-source and free.

How to get category name from unique ID in Commercetools platform?

I am attempting to get the names of all categories that a product belongs to in Commercetools platform.
I can get the unique ID of each category tied to a product through the following call:
final ProductProjectionQuery query = ProductProjectionQuery.ofCurrent();
ProductProjectionQuery q = query.withLimit(450);
try {
PagedQueryResult<io.sphere.sdk.products.ProductProjection> pagedQueryResult = client.execute(q).toCompletableFuture().get();
List<io.sphere.sdk.products.ProductProjection> products = pagedQueryResult.getResults();
//createDocument(products.get(0).getMasterVariant(), request);
for (io.sphere.sdk.products.ProductProjection product : products) {
String categoryId = product.getCategories().iterator().next().getId();
//createDocument(product.getMasterVariant(), request);
}
}
Though once I have the categoryId, I am unsure of how to access the category name. I thought that the obj property might allow me to drill down into the category, but the obj variable always seems to be null.

The obj variable is null because you have not expanded the reference. Unless you explicitly request it, all references will be empty to improve performance. In order to expand it, you can use this code:
// Query products
final ProductProjectionQuery query = ProductProjectionQuery.ofCurrent()
.withExpansionPaths(product -> product.categories()) // Request to expand categories
.withLimit(450);
final List<ProductProjection> products = client.execute(query).toCompletableFuture().join().getResults();
for (ProductProjection product : products) {
final List<LocalizedString> categoryLocalizedNames = product.getCategories().stream()
.map(categoryRef -> categoryRef.getObj().getName())
.collect(toList());
// Do something with categoryLocalizedNames
}
But I strongly recommend you to cache the categories in a CategoryTree instance and grab the name from it, because otherwise the performance will be quite affected by expanding all categories of each product. Here is the code:
// Query all categories and put them in a CategoryTree
final CategoryQuery queryAllCategories = CategoryQuery.of().withLimit(500);
final List<Category> allCategories = client.execute(queryAllCategories).toCompletableFuture().join().getResults();
final CategoryTree categoryTree = CategoryTree.of(allCategories);
// Query products
final ProductProjectionQuery query = ProductProjectionQuery.ofCurrent().withLimit(500);
final List<ProductProjection> products = client.execute(query).toCompletableFuture().join().getResults();
for (ProductProjection product : products) {
final List<LocalizedString> categoryLocalizedNames = new ArrayList<>();
product.getCategories().forEach(categoryRef -> {
final Optional<Category> categoryOpt = categoryTree.findById(categoryRef.getId());
if (categoryOpt.isPresent()) {
categoryLocalizedNames.add(categoryOpt.get().getName());
}
});
// Do something with categoryLocalizedNames
}
Of course that means you also have to find a solution to invalidate the cached categories when they change.