We Keep Coding

Why does the paging library stop loading data after several calls?

I implemented a paging library and it does not work quite as it should. I request data from the github and paginate the list of repositories. The code works well, but after several changes to the search query, it stops loading data. In the debug, the data always loads well. I guess the problem is asynchrony, but I can’t figure out where to look.
My code:
RepoDataSource
public class RepoDataSource extends PageKeyedDataSource<Integer, Repo> {
#Override
public void loadInitial(#NonNull LoadInitialParams<Integer> params, #NonNull LoadInitialCallback<Integer, Repo> callback) {
Timber.d("Initial RepoDataSource");
try {
Response<RepoSearchResponse> response = githubService.searchRepos(query, firstNumberPage).execute();
RepoSearchResponse repoSearchResponse = response.body();
if (repoSearchResponse != null) {
List<Repo> items = repoSearchResponse.getItems();
callback.onResult(items, 1, 2);
}
} catch (IOException e) {
Timber.i(e);
}
}
#Override
public void loadBefore(#NonNull LoadParams<Integer> params, #NonNull LoadCallback<Integer, Repo> callback) {
}
#Override
public void loadAfter(#NonNull LoadParams<Integer> params, #NonNull LoadCallback<Integer, Repo> callback) {
Timber.d("Fetching next page: %s", params.key);
try {
Response<RepoSearchResponse> response = githubService.searchRepos(query, params.key).execute();
if (response.isSuccessful()) {
RepoSearchResponse repoSearchResponse = response.body();
if (repoSearchResponse != null) {
List<Repo> items = repoSearchResponse.getItems();
callback.onResult(items, params.key + 1);
}
}
} catch (IOException e) {
Timber.i(e);
}
}
}
GithubApiCall
#GET("search/repositories")
Call<RepoSearchResponse> searchRepos(#Query("q") String query, #Query("page") Integer page);
RepoDataSourceFactory
public class RepoDataSourceFactory extends DataSource.Factory<Integer, Repo> {
private GithubService githubService;
private String query;
public RepoDataSourceFactory(GithubService githubService, String query) {
this.githubService = githubService;
this.query = query;
}
#NonNull
#Override
public DataSource<Integer, Repo> create() {
return new RepoDataSource(githubService, query);
}
}
Repository method
public class RepoRepository {
...
...
public RepoDataSourceFactory getRepoPagedFactory(String query) {
return new RepoDataSourceFactory(githubService, query);
}
}
ViewModel
public final class MyViewModel {
...
public MutableLiveData<String> searchQuery = new MutableLiveData<>();
...
public LiveData<PagedList<Repo>> getRepos() {
return Transformations.switchMap(searchQuery, query -> {
RepoDataSourceFactory factory = repository.getRepoPagedFactory(query);
return new LivePagedListBuilder<>(factory, pagedListConfig).build();
});
}
...
public SearchView.OnQueryTextListener listener = new SearchView.OnQueryTextListener() {
#Override
public boolean onQueryTextSubmit(String query) {
if (query != null && !query.trim().equals("")) {
searchQuery.postValue(query);
}
return true;
}
#Override
public boolean onQueryTextChange(String newText) {
return true;
}
};
...
}
And in my activity
viewModel.getRepos().observe(this, adapter::submitList);

There is nothing wrong with your code. I was on a GitHub project, and was stuck in the same problem until I realize GitHub has a Rate Limit of 10 requests per minute for unauthenticated requests. But if it is an authenticated one, you can make up to 30 requests per minute.
I assume you also send request for every changes in the search query, just like I did, where typing/changing 5 characters equals 5 requests. So the real cause is the very limited request rate from GitHub, not your code.
Check this out: https://developer.github.com/v3/search/#rate-limit

Observable onNext() not called for Observable.zip()

I have multiple requests (upload files) into Observable and I want to execute them in parallel. The code is:
private void myMethod(List<String> filePathsList) {
List<Observable<String>> observables = new ArrayList<>();
for (String filePath : filePathsList) {
MultipartBody.Part multipartFile = getMultipartFile("some_file_name", filePath);
//here I'm just creating request from Retrofit restclient - the problem can't be here ;)
Observable<String> fileUploadObservable = UploadsRestClient.get().sendFile(multipartFile, "another_post_param");
observables.add(fileUploadObservable);
}
Observable<String> combinedObservable = Observable.zip(observables, new FuncN<String>() {
#Override
public String call(Object... args) {
return null;
}
});
combinedObservable.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io())
.subscribe(new Subscriber<String>() {
#Override
public void onCompleted() {
//called at the end
}
#Override
public void onError(Throwable throwable) {
//called if error occurs
}
#Override
public void onNext(String string) {
//should be called foreach request, but it's called only after the last one
}
});
}
The problem is that onNext() is called only after the last call is done, before onCompleted(),how can I get triggered after each request?

Best way to use retrofit response in several activies

I have a function searchForTrips() which sends an API request and fetch some response in following way.
private void searchForTrips(){
int departurePortId = PORT_ID_LIST.get(departurePort);
int returnPortId = PORT_ID_LIST.get(returnPort);
int pax= Integer.parseInt(noOfPassengers);
String departureDatePARSED = DEPARTURE_DATE_VALUES.get(departureDate);
String returnDatePARSED = RETURN_DATE_VALUES.get(departureDate);
Call<TripSearchResponse> call = apiService.searchAvailableTrips(TripType,departurePortId,returnPortId,departureDatePARSED,returnDatePARSED,pax);
call.enqueue(new Callback<TripSearchResponse>() {
#Override
public void onResponse(Call<TripSearchResponse> call, Response<TripSearchResponse> response) {
int statusCode = response.code();
switch(statusCode){
case 200:
default:
Snackbar.make(findViewById(android.R.id.content),"Error loading data. Network Error.", Snackbar.LENGTH_LONG).show();
break;
}
}
#Override
public void onFailure(Call<TripSearchResponse> call, Throwable t) {
Log.i(TAG, t.getMessage());
Snackbar.make(findViewById(android.R.id.content),"Error loading data. Network Error.", Snackbar.LENGTH_LONG).show();
}
});
}
The purpose is to make this callback function reusable so I can call it from several activities and get requested data as I need. What is the best way to implement this?

try this way, its dynamic way and easy to use:
Create Retforit Interface:
public interface ApiEndpointInterface {
#Headers("Content-Type: application/json")
#POST(Constants.SERVICE_SEARCH_TRIP)
Call<JsonObject> searchForTrip(#Body TripRequest objTripRequest);
}
Create Retrofit Class:
public class AppEndPoint {
private static Retrofit objRetrofit;
public static ApiEndpointInterface getClient() {
if (objRetrofit == null){
objRetrofit = new Retrofit.Builder()
.baseUrl(Constants.SERVER_URL)
.addConverterFactory(GsonConverterFactory.create())
.build();
}
return objRetrofit.create(ApiEndpointInterface.class);
}
}
Create this helper Classes/Interfaces to hold web service callback:
public enum ResponseState {
SUCCESS,
FAILURE,
NO_CONNECTION
}
public enum RequestType {
SEARCH_FOR_TRIP // add name for each web service
}
public class Response {
public ResponseState state;
public boolean hasError;
public RequestType requestType;
public JsonObject result;
}
public interface RestRequestInterface {
void Response(Response response);
Context getContext();
}
public class ResponseHolder { used to hold the Json response could be changed as your response
#SerializedName("is_successful")
#Expose
private boolean isSuccessful;
#SerializedName("error_message")
#Expose
private String errorMessage;
public boolean isSuccessful() {
return isSuccessful;
}
public void setSuccessful(boolean successful) {
isSuccessful = successful;
}
public String getErrorMessage() {
return errorMessage;
}
public void setErrorMessage(String errorMessage) {
this.errorMessage = errorMessage;
}
}
public class AppClient {
private static ApiEndpointInterface objApiEndpointInterface;
private static Response objResponse;
private static Call<JsonObject> objCall;
// implement new method like below for each new web service
public static void searchForTrip(TripRequest objTripRequest, RestRequestInterface objRestRequestInterface) {
objResponse = new Response();
objResponse.state = ResponseState.FAILURE;
objResponse.hasError = true;
objResponse.requestType = RequestType.SEARCH_FOR_TRIP; // set type of the service from helper interface
objApiEndpointInterface = AppEndPoint.getClient();
objCall = objApiEndpointInterface.searchForTrip(objTripRequest);
handleCallBack(objRestRequestInterface);
}
private static void handleCallBack(final RestRequestInterface objRestRequestInterface) {
objCall.enqueue(new Callback<JsonObject>() {
#Override
public void onResponse(Call<JsonObject> call, retrofit2.Response<JsonObject> response) {
try {
ResponseHolder objResponseHolder = new Gson().fromJson(response.body(), ResponseHolder.class);
if (objResponseHolder.isSuccessful()) {
objResponse.state = ResponseState.SUCCESS;
objResponse.hasError = false;
objResponse.result = response.body();
} else {
objResponse.errorMessage = objResponseHolder.getErrorMessage();
}
objRestRequestInterface.Response(objResponse);
} catch (Exception objException) {
objResponse.errorMessage = objRestRequestInterface.getContext().getString(R.string.server_error);
objRestRequestInterface.Response(objResponse);
}
}
#Override
public void onFailure(Call<JsonObject> call, Throwable objThrowable) {
String errorMessage = "";
if (objThrowable instanceof IOException) {
errorMessage = objRestRequestInterface.getContext().getString(R.string.no_connection_error);
} else {
errorMessage = objRestRequestInterface.getContext().getString(R.string.server_error);
}
objResponse.errorMessage = errorMessage;
objRestRequestInterface.Response(objResponse);
}
});
}
}
then go to your activity of fragment and make the call like this:
public class MainActivity extends AppCompatActivity implements RestRequestInterface {
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
// initialize ids
// prepare to call web service
// 1.Initialize your object to be sent over web service
TripRequest objTripRequest = new TripRequest();
objTripRequest.id = 1;
// 2.Show loader
// 3.Make the call
AppClient.searchForTrip(objTripRequest, this);
}
#Override
public void Response(Response response) {
// hide loader
try {
if (response.state == ResponseState.SUCCESS && !response.hasError) {
// check the type of web service
if (response.requestType == RequestType.SEARCH_FOR_TRIP) {
// acces the return here from response.result
}
} else {
String errorMsg = response.hasError ? response.errorMessage : getString(R.string.no_connection_error);
// show the error to the user
}
} catch (Exception objException) {
// show the error to the user
}
}
#Override
public Context getContext() {
// do not forgit set the context here
// if fragment replace with getAcitvity();
return this;
}
}

data structures in java to make changes apply from one process to other synchronously

I am trying to make two methods generator and consumer to run synchronously.I have tried queue but unable to achieve.
I am using the below code
#Override
protected List<ConfigIssue> init() {
client = new RtmClientBuilder(endpoint, appkey)
.setListener(new RtmClientAdapter() {
})
.build();
SubscriptionAdapter listener = new SubscriptionAdapter() {
#Override
public void onSubscriptionData(SubscriptionData data) {
for (AnyJson json : data.getMessages()) {
queue.add(json);
}
}
};
client.createSubscription(channel, SubscriptionMode.SIMPLE, listener);
client.start();
return super.init();
}
#Override
public String produce(String lastSourceOffset, int maxBatchSize, BatchMaker batchMaker) throws StageException {
long nextSourceOffset = 0;
if (lastSourceOffset != null) {
nextSourceOffset = Long.parseLong(lastSourceOffset);
}
if (messageQueue.size() != 0) {
for (AnyJson json : queue) {
Record record = getContext().createRecord("some-id::" + nextSourceOffset);
Map<String, Field> map = new HashMap<>();
map.put("fieldName", Field.create(json.toString()));
record.set(Field.create(map));
batchMaker.addRecord(record);
++nextSourceOffset;
messageQueue.remove();
}
}
return String.valueOf(nextSourceOffset);
}
I am trying to Sync both init method and produce method.Since init method start first and produce method start later they are not in sync.Can it be achieved by multi threading?How?

how to store handler result into JsonArray vertx?

I am facing a problem,I can't store the handler result into Json Array.Every time the array is empty . I tried to use Future but it still the same problem, here is my code :
static void getCaraReferTypeDocBin(RoutingContext routingContext){
String ids = routingContext.request().getParam("ids");
logger.debug(ids);
String[] idsArray = ids.split(",");
JsonArray caraRefTypeDocBin = new JsonArray();
for (int i = 0; i <idsArray.length ; i++) {
GET.getCaraTypeDocBin(Integer.parseInt(idsArray[i]), res->{
if (res.succeeded()){
logger.debug(res.result());
caraRefTypeDocBin.add(res.result());
}else{
logger.debug(res.cause().getMessage());
}
});
}
logger.debug(caraRefTypeDocBin);
}
this is getCaraReferTypeDocBin implementation :
public static void getCaraTypeDocBin(int id ,Handler<AsyncResult<JsonArray>> resultHandler) {
JsonArray pIn = new JsonArray();
pIn.add(new JsonObject().put("pos", 2).put("type", OracleTypes.NUMBER).put("val", id));
JsonArray pOut = new JsonArray().add(new JsonObject().put("pos", 1).put("type", OracleTypes.CURSOR));
DB.cursor(SQL.LIST_CARA_TYPE_DOC_BIN,pIn,pOut, res -> {
if (res.succeeded()) {
try {
resultHandler.handle(Future.succeededFuture(res.result().getJsonArray("1")));
}catch (Exception e){
logger.error(e);
resultHandler.handle(Future.failedFuture(Error.ERROR_OCCURED.toString()));
}
} else {
resultHandler.handle(Future.failedFuture(res.cause().getMessage()));
}
});
}

In async systems api with futures should be written something like this:
private Future<String> loadFromDb() {
Future<String> f = Future.future();
//some internal loading from db
String result = "fromDb";
//when loading completes, pass it to future result
f.complete(result);
return f;
}
And how it uses:
private void handleSo(RoutingContext routingContext) {
loadFromDb()
.map(new Function<String, JsonArray>() {
#Override
public JsonArray apply(String fromDb) {
//map to json
return new JsonArray(...);
}
})
.setHandler(
new Handler<AsyncResult<JsonArray>>() {
#Override
public void handle(AsyncResult<JsonArray> result) {
routingContext.response().end(result.result().toString());
}
}
);
}
You are using futures wrong. You example simple and you haven't async chains (where result calculates based on previous result etc), so instead futures, you can simple use callback:
private void loadFromDb(Handler<String> handler) {
String result = "fromDb";
handler.handle(result);
}
private void handleSo(RoutingContext routingContext) {
loadFromDb(new Handler<String>() {
#Override
public void handle(String fromDb) {
routingContext.response().end(new JsonArray(...).toString());
}
});
}
Upd You need collect results from multiple async calls doc. Don't know how to implement it with you callback style api. But with futures it's not problem:
private void handleSo(RoutingContext routingContext) {
List<Future> futures = new ArrayList<>();
for (int i = 0; i < 10; i++) {
//make 10 async calls
futures.add(loadFromDb()
.map(new Function<String, JsonObject>() {
#Override
public JsonObject apply(String fromDb) {
return new JsonObject().put("data", fromDb);
}
}));
}
CompositeFuture.all(futures)
.map(new Function<CompositeFuture, JsonArray>() {
#Override
public JsonArray apply(CompositeFuture compositeFuture) {
JsonArray array = new JsonArray();
List<JsonObject> jsons = compositeFuture.result().list();
jsons.forEach(jsonObject -> array.add(jsonObject));
return array;
}
})
.setHandler(new Handler<AsyncResult<JsonArray>>() {
#Override
public void handle(AsyncResult<JsonArray> res) {
routingContext.response().end(res.result().toString());
}
});
}

GET.getCaraTypeDocBin() runs asynchronously, right? And I assume there's something time-consuming in there like hitting a remote API? So the loop will run in milliseconds, kicking off all the requests, and then the "logger.debugs" will happen, and only then will the callbacks start to do caraRefTypeDocBin.add(res.result());
If I'm right, you should see the results logged before the empty array.