We Keep Coding

Apache Beam create timeseries from event stream

I am trying to create a timeseries of the count of events that happened over a given time.
The events are encoded as
PCollection<KV<String, Long>> events;
Where the String is the id of the event source, and long is the timestamp of the event.
What I want out is a PCollection<Timeseries> of timeseries that have the form
class Timeseries {
String id;
List<TimeseriesWindow> windows;
}
class TimeseriesWindow {
long timestamp;
long count;
}
Toy example with a fixed window size (is this the correct term?) of 10 seconds, and an total timeseries duration of 60 seconds:
Input:
[("one", 1), ("one", 13), ("one", 2), ("one", 43), ("two", 3)]
Output:
[
{
id: "one"
windows: [
{
timestamp: 0,
count: 2
},
{
timestamp: 10,
count: 1
},
{
timestamp: 20,
count: 0
},
{
timestamp: 30,
count: 0
},
{
timestamp: 40,
count: 1
},
{
timestamp: 50,
count: 0
}
]
},
{
id: "two"
windows: [
{
timestamp: 0,
count: 1
},
{
timestamp: 10,
count: 0
},
{
timestamp: 20,
count: 0
},
{
timestamp: 30,
count: 0
},
{
timestamp: 40,
count: 0
},
{
timestamp: 50,
count: 0
}
]
}
]
I hope this makes sense :)

You can do a GroupByKey to transform your input into
[
("one", [1, 13, 2, 43]),
("two", [3]),
]
at which point you can apply a DoFn to convert the list of integers into a Timeseries object (e.g. by creating the list of TimeseriesWindow at the appropriate times, and then iterating over the values incrementing the counts.)
You may also look into the builtin windowing capabilities to see if that will meet your needs.

Angular - *ngFor only returning one record from the array, despite 12 being available

I am using HttpClient in Angular to retrieve data from an API endpoint I have created using SpringBoot.
I have created a service in Angular to consume the data. I am printing the data to the console, so I know the data is coming back.
I am trying to pass all of the outputs to a dropdown list, but despite using *ngFor, only 1 result is ever being returned:
Here is the code in the HTML file for the dropdown:
<mat-form-field>
<mat-select
placeholder="Select Competition">
<mat-option *ngFor="let comp of competitionsList; let i = index" [value]="comp">
{{ comp[i].name }}
</mat-option>
</mat-select>
</mat-form-field>
When I change the value of [i] to or [11] (the last element in my array) for testing purposes, all the data always displays as expected, but when I use comp[i].name I would have expected all elements in the array to be returned, but it is only returning the first element.
I'm sure it is something quite simple, but I have been looking at this for a long time and still cannot quite see why this is happening so any help will be greatly appreciated!
JSON Response:
[{…}]
0:
0: {id: 2013, name: "Série A"}
1: {id: 2021, name: "Premier League"}
2: {id: 2016, name: "Championship"}
3: {id: 2001, name: "UEFA Champions League"}
4: {id: 2018, name: "European Championship"}
5: {id: 2015, name: "Ligue 1"}
6: {id: 2002, name: "Bundesliga"}
7: {id: 2019, name: "Serie A"}
8: {id: 2003, name: "Eredivisie"}
9: {id: 2017, name: "Primeira Liga"}
10: {id: 2014, name: "Primera Division"}
11: {id: 2000, name: "FIFA World Cup"}
ident: "competitions"
name: "com"
**Adding additional info:
I have defined an interface for the values which will be returned by the API:
export interface Competitions {
id: number;
name: string;
}
This is my service:
retrieveAvailableCompetitions() {
return this.http.get<Competitions>(this.competitionsUrl);
}
And this is my method in TypeScript file:
getCompetitons() {
this.service.retrieveAvailableCompetitions().pipe(
map(responseData => {
const compsList: Competitions[] = [];
for (const key in responseData) {
if (responseData.hasOwnProperty(key)) {
compsList.push({ ...responseData[key], ident: key, name: 'com' });
}
}
return compsList;
})
).subscribe(
compsList => {
this.competitionsList = compsList;
console.log(compsList);
});
}
(I'm aware that I should move the majority of this code to my service, but I plan to refactor once I have fixed the issue)

You mentioned in the comments about undefined error. This indicates you are trying to render the view before the data is available. Use *ngIf on your element to only render it when data is available.
<mat-form-field *ngIf="competitionsList">
</mat-form-field>
When you are iterating over the array using *ngFor you don't need to use the index i.
Replace {{ comp[i].name }} with {{ comp.name }}.
Then your code will look like
<mat-form-field *ngIf="competitionsList">
<mat-select placeholder="Select Competition">
<mat-option *ngFor="let comp of competitionsList; let i = index" [value]="comp">
{{ comp.name }}
</mat-option>
</mat-select>
</mat-form-field>
Live demo on StackBlitz: https://stackblitz.com/edit/angular-material-with-angular-v5-eukdkc
EDIT: You should format your response compList to form competitionsList array.
Change your subscribe() method to
.subscribe(
compsList => {
this.competitionsList = Object.values(compList[0]);
});

Why does Rest Assured say "validation doesn't match" but excepted does match Actual?

I am trying to validate this JSON with RestAssured:
{
"valueBounds": [
{
"bound": {
"min": 1.0,
"max": 4.2
},
"date": "2019-01-04T18:40:28.204+0100"
}
],
}
with this code:
given().when().get(rootPath + "/test/").then().statusCode(200).log().body().
body("valueBounds.bound.min", hasItems(1.0));
Why do I get that they do not match:
java.lang.AssertionError: 1 expectation failed.
JSON path valueBounds.bound.min doesn't match.
Expected: (a collection containing <1.0>)
Actual: [1.0]
I have tried to Use Arrays.asList(1.0) but that is not the problem.

The solution was to add a f behind the number:
"valueBounds.bound.min", hasItems(1.0f)

Custom JSON to populate a Datatable

Im creating a DataTable from an Ajax json.
resultTable = $('#changeTable').DataTable({
"order": [[0, "desc"]],
"pageLength": 50,
"scrollX": true,
"lengthMenu":[[50,100,250, -1], [50, 100, 250, "All"]],
"dom":'<"toolbar">ltipr', //write ltfipr to show a search bar
"ajax":{
url:"api/changes",
"dataType":"json",
timeout:15000
}
});
The DataTables creates but it shows an error:
DataTables warning: table id=changeTable - Requested unknown parameter
'0' for row 0, column 0. For more information about this error, please
see http://datatables.net/tn/4
My JSON Looks like the following
{"data":
[
{"id":1,
"createdDate":"Apr 18, 2018 4:10:58 PM",
"source":"manual upload",
"emailId":"manual upload",
"attachmentId":"manual upload",
...,},
{next objet}]}
Such JSON object is created in my Java controller:
#RequestMapping(value = "/api/changes", method = RequestMethod.GET, produces = "application/json")
#ResponseBody
public String getChanges(){
Optional<List<PriceChange>> priceChangeList = pcService.findAllPriceChanges();
JsonObject result = new JsonObject();
if (priceChangeList.isPresent()) {
result.add("data", new Gson().toJsonTree(priceChangeList.get()));
return result.toString();
}
return null;
}
I don know how to use this information with the dataSrc property to make it work for the DataTable. Any Ideas?

You just need to define columns for the table. If you have
<table id="changeTable"></table>
add this to your DataTables options :
resultTable = $('#changeTable').DataTable({
...,
columns: [
{ data: 'id', title: 'id' },
{ data: 'createdDate', title: 'createdDate' },
{ data: 'source', title: 'source' },
{ data: 'emailId', title: 'emailId' },
{ data: 'attachmentId', title: 'attachmentId' }
]
})
If ypu have specifed a <thead> section you can skip the title's.

Swagger API Annotation for Java using Enum as options which break request

I have a class defined called Website Locale which lits all supported locales for my site
public enum WebsiteLocale {
AU ("au", 5, "en", "AUD"), // Australia
CA ("ca", 6, "en", "CAD"), // Canada
DE ("de", 1, "de", "EUR"), // Germany
ES ("es", 1, "es", "EUR"), // Spain
ES_US ("es-us", 2, "es", "USD"), // USA, Spanish language
EU ("eu", 4, "en", "EUR"), // Europe: English and Euros
FR ("fr", 1, "fr", "EUR"), // France
FR_US ("fr-us", 2, "fr", "USD"), // USA, French language
IT ("it", 1, "it", "EUR"), // Italy
NL ("nl", 3, "en", "EUR"), // Netherlands
UK ("uk", 1, "en", "GBP"), // UK
US ("us", 2, "en", "USD"); // USA
... getters / setters... }
In my API resource I have the following annotation
#GET
#Path("/countries")
#ApiOperation("Get countries")
#CacheControl(maxAge = 1, maxAgeUnit = TimeUnit.HOURS)
#Override
public List<CountryDto> getCountries(
#QueryParam("websiteLocale") #ApiParam(required = true, defaultValue = "uk") final WebsiteLocale websiteLocale) {
return delegate.getCountries(websiteLocale);
In the Swagger API test page for that endpoint it gives the accepted values as the Enum literals themselves
e.g. AU, CA, DE, ES, ES_US, EU, FR, FR_US
This breaks the controller (404 not found response) because it is looking for the associated string key for that enum
i.e. au, ca, de, es, es-us, eu, fr, fr-us
I had a look through swagger but could not see any specific annotation to look for a part of the enum properties.
Any ideas?
Thanks