I have a function formatted like this:
private void verifyDatatypeTables(
final DynamoDBMapper mapper,
final List<Datatype> datatypeMissingEntries) {
final List<Datatype> datatypeEntries = new ArrayList<>();
this.mapToListDatatypeTables(datatypeEntries);
final List<Datatype> datatypeEntriesInTable =
this.dbUtilityDatatype.scanRecord(new DynamoDBScanExpression(), true);
}
This creates a little reading problem. I want it to be formatted like this:
private void verifyDatatypeTables(
final DynamoDBMapper mapper,
final List<Datatype> datatypeMissingEntries
) {
final List<Datatype> datatypeEntries = new ArrayList<>();
this.mapToListDatatypeTables(datatypeEntries);
final List<Datatype> datatypeEntriesInTable =
this.dbUtilityDatatype.scanRecord(new DynamoDBScanExpression(), true);
}
How to achieve formatting like this in Eclipse?
Preferences -> Java -> Code Style -> Formatter -> edit -> Parentheses -> Method declaration -> separate lines
with
Line Wrapping -> Method declarations -> Parameters -> Line Wrapping policy -> Wrap all elements, every element on a new line
Related
Dates.forEach(date -> {
executeQuery(date, loads);
private void executeQuery(LocalDate date, ArrayList<Load> loads){
MapSqlParameterSource source = new MapSqlParameterSource();
source.addValue("date", date.toString());
Load load = namedJdbcTemplate.queryForObject(Constants.SQL_QUERY, source,
new BeanPropertyRowMapper<>(Load.class));
loads.add(load);
}
How can I use the streams concept for the above code
Something like this should work
// change your method like so
private Load executeQuery(LocalDate date){
MapSqlParameterSource source = new MapSqlParameterSource();
source.addValue("date", date.toString());
return namedJdbcTemplate.queryForObject(Constants.SQL_QUERY, source,
new BeanPropertyRowMapper<>(Load.class));
}
// load your dates from somewhere
List<LocalDate> dates = getYourDates();
// now use the streams API to collect the query results into a new list
List<Load> loads = dates.stream()
.map(this::executeQuery)
.collect(Collectors.toList());
or
List<Load> loads = getYourDates().stream()
.map(this::executeQuery)
.collect(Collectors.toList());
I am pretty new to java8 streams. I was trying to work on collection of objects using stream. But not able to achieve in precise way.
Below is the snippet which I achieved (which is giving wrong result). expected end result is List<String> of "Names email#test.com".
recordObjects is collection of object
choices = recordObjects.stream()
.filter(record -> record.getAttribute
(OneRecord.AT_RECORD_SUBMITTER_TABLE_EMAIL) != null)
.filter(record -> !record.getAttributeAsString
(OneRecord.AT_RECORD_SUBMITTER_TABLE_EMAIL).isEmpty())
.map(record -> record.getMultiValuedAttribute
(OneRecord.AT_RECORD_SUBMITTER_TABLE_EMAIL, String.class))
.flatMap(Collection::stream)
.map(email -> getFormattedEmailAddress(ATTRI_AND_RECORD_CONTACT_DEFAULT_NAME, email))
.collect(Collectors.toList());
but below is the exact logic i want to implement using streams.
for (CallerObject record : recordObjects) {
List<String> emails = record.getMultiValuedAttribute(
OneRecord.AT_RECORD_SUBMITTER_TABLE_EMAIL, String.class);
List<String> names = record.getMultiValuedAttribute(
OneRecord.AT_RECORD_SUBMITTER_TABLE_NAME, String.class);
int N = emails.size();
for (int i = 0 ; i < N ; i++) {
if(!isNullOrEmpty(emails.get(i)))
{
choices.add(getFormattedEmailAddress(isNullOrEmpty(names.get(i)) ?
ATTRI_AND_RECORD_CONTACT_DEFAULT_NAME : names.get(i) , emails.get(i)));
}
}
}
Since we don't know the getFormattedEmailAddress method, I used String.format instead to achieve the desired representation "Names email#test.com":
// the mapper function: using String.format
Function<RecordObject, String> toEmailString = r -> {
String email = record.getMultiValuedAttribute(OneRecord.AT_RECORD_SUBMITTER_TABLE_EMAIL, String.class);
String name = record.getMultiValuedAttribute(OneRecord.AT_RECORD_SUBMITTER_TABLE_NAME, String.class);
if (email != null) {
return String.format("%s %s", name, email);
} else {
return null;
}
};
choices = recordObjects.stream()
.map(toEmailString) // map to email-format or null
.filter(Objects::nonNull) // exclude null strings where no email was found
.collect(Collectors.toList());
Changed your older version code to Java 8
final Function<RecordedObject, List<String>> filteredEmail = ro -> {
final List<String> emails = ro.getMultiValuedAttribute(
OneRecord.AT_RECORD_SUBMITTER_TABLE_EMAIL, String.class);
final List<String> names = ro.getMultiValuedAttribute(
OneRecord.AT_RECORD_SUBMITTER_TABLE_NAME, String.class);
return IntStream.range(0, emails.size())
.filter(index -> !isNullOrEmpty(emails.get(index)))
.map(index -> getFormattedEmailAddress(isNullOrEmpty(names.get(index)) ?
ATTRI_AND_RECORD_CONTACT_DEFAULT_NAME : names.get(index) , emails.get(index)))
.collect(Collectors.toList());
};
recordObjects
.stream()
.map(filteredEmail)
.flatMap(Collection::stream)
.collect(Collectors.toList());
I have a use-case in Java where I need to populate one of the lists (say x) based on the id of the other list(say y) and to fetch the result from that list.
List<LightRecruiterScholarResponse> responses = eventScholarRepository.findScholarDetailsByEventId(eventId);
List<InterviewDto> interviewResults = interviewRepository.getInterviewResultByRoundIdAndScholarId();
for (LightRecruiterScholarResponse response : responses) {
String val = null;
for (InterviewDto dto : interviewResults) {
if (dto.getId().equals(response.getScholarId())) {
val = dto.getInterviewResult();
break;
}
}
response.setInterviewStatus(val);
}
You can use:
Map<String, String> map = interviewResults.stream()
.collect(Collectors.toMap(InterviewDto::getId, InterviewDto::getInterviewResult));
responses.forEach(response ->
response.setInterviewStatus(
map.getOrDefault(response.getScholarId(), null)));
The idea is to create a Map of Id for key and InterviewResult for value, and then for each element in responses you set InterviewStatus which you can find it in the map by ScholarId which can replace if (dto.getId().equals(response.getScholarId()))
This could be done straightforward:
List<LightRecruiterScholarResponse> responses =
eventScholarRepository.findScholarDetailsByEventId(eventId);
List<InterviewDto> interviewResults =
interviewRepository.getInterviewResultByRoundIdAndScholarId();
responses.forEach(response -> response.setInterviewStatus(
interviewResults.stream()
.filter(dto -> dto.getId().equals(response.getScholarId()))
.map(InterviewDto::getInterviewResult)
.findFirst().orElse(null)));
This is not very efficient, because you iterate over interviewResults for every response. To fix it, you can build Map and use it:
List<LightRecruiterScholarResponse> responses =
eventScholarRepository.findScholarDetailsByEventId(eventId);
Map<String, String> interviewResults =
interviewRepository.getInterviewResultByRoundIdAndScholarId().stream()
.collect(Collectors.toMap(InterviewDto::getId,
InterviewDto::getInterviewResult));
responses.forEach(response ->
response.setInterviewStatus(interviewResults.get(response.getScholarId())));
I'm brand new to Akka and I'm just trying to get the hang of it.
As an experiment, I want to read from a Kinesis stream and collect n messages and stop.
The only one I found that would stop reading records was Sink.head(). But that only returns one record, I'd like to get more than that.
I can't quite figure out how to stop reading from the stream after receiving the n messages though.
Here's the code I have tried so far
#Test
public void testReadingFromKinesisNRecords() throws ExecutionException, InterruptedException {
final ActorSystem system = ActorSystem.create("foo");
final Materializer materializer = ActorMaterializer.create(system);
ProfileCredentialsProvider profileCredentialsProvider = ProfileCredentialsProvider.create();
final KinesisAsyncClient kinesisClient = KinesisAsyncClient.builder()
.credentialsProvider(profileCredentialsProvider)
.region(Region.US_WEST_2)
.httpClient(AkkaHttpClient.builder()
.withActorSystem(system).build())
.build();
system.registerOnTermination(kinesisClient::close);
String streamName = "akka-test-stream";
String shardId = "shardId-000000000000";
int numberOfRecordsToRead = 3;
final ShardSettings settings = ShardSettings.create(streamName, shardId)
.withRefreshInterval(Duration.ofSeconds(1))
.withLimit(numberOfRecordsToRead) // return a maximum of n records (and quit?!)
.withShardIterator(ShardIterators.latest());
final Source<Record, NotUsed> sourceKinesisBasic = KinesisSource.basic(settings, kinesisClient);
Flow<Record, String, NotUsed> flowMapRecordToString = Flow.of(Record.class).map(record -> extractDataFromRecord(record));
Flow<String, String, NotUsed> flowPrinter = Flow.of(String.class).map(s -> debugPrint(s));
// Flow<String, List<String>, NotUsed> flowGroupedWithinMinute =
// Flow.of(String.class).groupedWithin(
// numberOfRecordsToRead, // group size
// Duration.ofSeconds(60) // group time
// );
Source<String, NotUsed> sourceStringsFromKinesisRecords = sourceKinesisBasic
.via(flowMapRecordToString)
.via(flowPrinter);
// .via(flowGroupedWithinMinute); // nope
// sink to list of strings
// Sink<String, CompletionStage<List<String>>> sinkToList = Sink.seq();
Sink<String, CompletionStage<List<String>>> sink10 = Sink.takeLast(10);
// Sink<String, CompletionStage<String>> sinkHead = Sink.head(); // only gives you one message
CompletionStage<List<String>> streamCompletion = sourceStringsFromKinesisRecords
.runWith(sink10, materializer);
CompletableFuture<List<String>> completableFuture = streamCompletion.toCompletableFuture();
completableFuture.join(); // never stops running...
List<String> result = completableFuture.get();
int foo = 1;
}
private String extractDataFromRecord(Record record) {
String encType = record.encryptionTypeAsString();
Instant arrivalTimestamp = record.approximateArrivalTimestamp();
String data = record.data().asString(StandardCharsets.UTF_8);
return data;
}
private String debugPrint(String s) {
System.out.println(s);
return s;
}
Thank you for any clues
I found out the answer is to use a takeN at the flow level
...
Flow<String, String, NotUsed> flowTakeN = Flow.of(String.class).take(numberOfRecordsToRead);
Source<String, NotUsed> sourceStringsFromKinesisRecords = sourceKinesisBasic
.via(flowMapRecordToString)
.via(flowPrinter)
.via(flowTakeN);
...
Just to add on to the answer you found, it is also possible to express things more directly without via:
Source<String, NotUsed> sourceStringsFromKinesisRecords = sourceKinesisBasic
.map(record -> extractDataFromRecord(record))
.map(s -> debugPrint(s))
.take(10)
i'm trying to use SessionWindows in my aggregation function in Kafka (0.11) but can not comprehend, why i get errors.
Here is my code-snippet:
// defining some values:
public static final Integer SESSION_TIMEOUT_MS = 6000000;
public static final String INTOPIC = "input";
public static final String HOST = "host";
// setting up serdes:
final Serializer<JsonNode> jsonSerializer = new JsonSerializer();
final Deserializer<JsonNode> jsonDeserializer = new JsonDeserializer();
final Serde<JsonNode> jsonSerde = Serdes.serdeFrom(jsonSerializer, jsonDeserializer);
// some more code to build up the streams
KStreamBuilder builder = new KStreamBuilder();
KStream<String, JsonNode> dataStream = builder.stream(Serdes.String(), jsonSerde, INTOPIC);
// constructing the initalMessage ObjectNode:
ObjectNode initialMessage = JsonNodeFactory.instance.objectNode();
initialMessage.put("count", 0);
initialMessage.put("endTime", "");
// transforming data to KGroupedStream<String,JsonNode>
KGroupedStream<String, JsonNode> data = dataStream.map((key, value) ->{return new KeyValue<>(value.get(HOST).asText(), value); }).groupByKey(Serdes.String(), jsonSerde);
// finally aggregate the data usind SessionWindows
KTable<Windowed<String>, JsonNode> aggregatedData = data.aggregate(
() -> initialMessage,
(key, incomingMessage, initialMessage) -> countData(incomingMessage, initialMessage),
SessionWindows.with(SESSION_TIMEOUT_MS),
jsonSerde,
"aggregated-data");
private static JsonNode countData(JsonNode incomingMessage, JsonNode initialMessage){
// some dataprocessing
}
When i change
KTable<Windowed<String>,JsonNode>
to
KTable<String, JsonNode>
and remove
SessionWindows.with(SESSION_TIMEOUT_MS)
from the aggregate function, everything is ok.
If i don't, eclipse tells me for line
KTable<Windowed<String>, JsonNode> aggregatedData = data.aggregate( [...])
The method aggregate(Initializer, Aggregator, Windows, Serde, String) in the type KGroupedStream is not applicable for the arguments (() -> {}, ( key, incomingMessage, initialMessage) -> {}, SessionWindows, Serde, String)
and for the line
() -> initialMessage
Type mismatch: cannot convert from ObjectNode to VR
and:
(key, incomingMessage, initialMessage) -> countData(incomingMessage, initialMessage),
The method countData(JsonNode, JsonNode) in the type DataWindowed is not applicable for the arguments (JsonNode, VR)
I realy don't see, where the types get lost!
Any hint would be great!
Thx :D
I realy needed to implement a Merger:
Merger<? super String, JsonNode>tmpMerger = new MergerClass<String, JsonNode>();
and add it to the aggregate function:
KTable<Windowed<String>, JsonNode> aggregatedData = data.aggregate(
() -> initialMessage,
(key, incomingMessage, initialMessage) -> countData(incomingMessage, initialMessage),
tmpMerger,
SessionWindows.with(SESSION_TIMEOUT_MS),
jsonSerde,
"aggregated-data");