Is it possible to auto evict the record with value Optional.absent()? In some applications the Optional.absent() may not be the value for some keys. For example, if an application contains http calls where the key can be some string and the value is response returned from the http call, the http call can return some invalid values (for example null) because of network issues or authentication failures, then the invalid can be saved as Optional.absent() with the key in cache. At a later point, if the network and authentication problems are fixed, the key->invalidValue still remains in the cache. What is the best to fix this problem?
For example, if an application contains http calls where the key can be some string and the value is response returned from the http call, the http call can return some invalid values (for example null) because of network issues or authentication failures
If possible, I would change this behavior to throw an exception when the request fails or the response is invalid - that's what exceptions are for. See Effective Java: Item 57 for more.
then the invalid can be saved as Optional.absent() with the key in cache. At a later point, if the network and authentication problems are fixed, the key->invalidValue still remains in the cache. What is the best to fix this problem?
Is there a reason you need to save the invalid result in the cache? If you don't care about the absent case and simply want it excluded from the cache the easiest option would be to just not cache it in the first place. Throwing an exception on bad results would make that easy.
If you do need to keep the invalid results in the cache temporarily you can clear them once you're ready with a simple for loop:
ConcurrentMap<K, V> cacheAsMap = cache.asMap();
for (Entry<K, V> e : cacheAsMap .entrySet()) {
if (!e.getValue().isPresent()) {
cacheAsMap.remove(e.getKey(), e.getValue());
}
}
By using ConcurrentMap.remove() you avoid a possible race condition where the entry is updated after e.getValue().isPresent() is called but before the entry is actually invalidated.
Related
I'm running an HL Fabric private network and submitting transactions to the ledger from a Java Application using Fabric-Java-Sdk.
Occasionally, like 1/10000 of the times, the Java application throws an exception when I'm submitting the transaction to the ledger, like the message below:
ERROR 196664 --- [ Thread-4] org.hyperledger.fabric.sdk.Channel
: Future completed exceptionally: sendTransaction
java.lang.IllegalArgumentException: The proposal responses have 2
inconsistent groups with 0 that are invalid. Expected all to be
consistent and none to be invalid. at
org.hyperledger.fabric.sdk.Channel.doSendTransaction(Channel.java:5574)
~[fabric-sdk-java-2.1.1.jar:na] at
org.hyperledger.fabric.sdk.Channel.sendTransaction(Channel.java:5533)
~[fabric-sdk-java-2.1.1.jar:na] at
org.hyperledger.fabric.gateway.impl.TransactionImpl.commitTransaction(TransactionImpl.java:138)
~[fabric-gateway-java-2.1.1.jar:na] at
org.hyperledger.fabric.gateway.impl.TransactionImpl.submit(TransactionImpl.java:96)
~[fabric-gateway-java-2.1.1.jar:na] at
org.hyperledger.fabric.gateway.impl.ContractImpl.submitTransaction(ContractImpl.java:50)
~[fabric-gateway-java-2.1.1.jar:na] at
com.apidemoblockchain.RepositoryDao.BaseFunctions.Implementations.PairTrustBaseFunction.sendTrustTransactionMessage(PairTrustBaseFunction.java:165)
~[classes/:na] at
com.apidemoblockchain.RepositoryDao.Implementations.PairTrustDataAccessRepository.run(PairTrustDataAccessRepository.java:79)
~[classes/:na] at java.base/java.lang.Thread.run(Thread.java:834)
~[na:na]
While my submitting method goes like this:
public void sendTrustTransactionMessage(Gateway gateway, Contract trustContract, String payload) throws TimeoutException, InterruptedException, InvalidArgumentException, TransactionException, ContractException {
// Prepare
checkIfChannelIsReady(gateway);
// Execute
trustContract.submitTransaction(getCreateTrustMethod(), payload);
}
I'm using a 4 org network with 2 peers each and I am using 3 channels, one for each chaincode DataType, in order to keep the things clean.
I think that the error coming from the Channel doesn't make sense because I am using the Contract to submit it...
Like I'm opening the gateway and then I keep it open for continuously submit the txs.
try (Gateway gateway = getBuilder(getTrustPeer()).connect()) {
Contract trustContract = gateway.getNetwork(getTrustChaincodeChannelName()).getContract(getTrustChaincodeId(), getTrustChaincodeName());
while (!terminateLoop) {
if (message) {
String payload = preparePayload();
sendTrustTransactionMessage(gateway, trustContract, payload);
}
...
wait();
}
...
}
EDIT:
After reading #bestbeforetoday advice, I've managed to catch the ContractException and analyze the logs. Still, I don't fully understand where might be the bug and, therefore, how to fix it.
I'll add 3 prints that I've taken to the ProposalResponses received in the exception and a comment after it.
ProposalResponses-1
ProposalResponses-2
ProposalResponses-3
So, in the first picture, I can see that 3 proposal responses were received at the exception and the exception cause message says:
"The proposal responses have 2 inconsistent groups with 0 that are invalid. Expected all to be consistent and none to be invalid."
In pictures, 2/3 is represented the content of those responses and I notice that there are 2 fields saving null value, namely "ProposalRespondePayload" and "timestamp_", however, I don't know if those are the "two groups" referred at the message cause of the exception.
Thanks in advance...
It seems that, while the endorsing peers all successfully endorsed your transaction proposal, those peer responses were not all byte-for-byte identical.
There are several things that might differ, including read/write sets or the value returned from the transaction function invocation. There are several reasons why differences might occur, including non-deterministic transaction function implementation, different transaction function behaviour between peers, or different ledger state at different peers.
To figure out what caused this specific failure you probably need to look at the peer responses to identify how they differ. You should be getting a ContractException thrown back from your transaction submit call, and this should allow you to access the proposal responses by calling e.getProposalResponses():
https://hyperledger.github.io/fabric-gateway-java/release-2.2/org/hyperledger/fabric/gateway/ContractException.html#getProposalResponses()
Consider /v1/api/people/{id} endpoint handling GET, PUT and DELETE operations, where id is an integer.
I am wondering about how the below two cases should typically be handled?
a). when passed id does not exists in database, e.g. id = 100 but there is no entity of such id
b). when passed id is of wrong type e.g. it is a string "oops"
Imagine that both of above errors never actually happen as far as the real application is concerned (because e.g. application has "correct" workflow and some client-side validation).
However, I can still cause the above errors in let's say Postman or if something changes in the future, right? I want to prevent those errors in the future.
Should it be left as HTTP 500 or perhaps should it be handled as HTTP 400 or HTTP 404? Is HTTP 500 ever acceptable?
HTTP 500 - should be used as Internal Server Error meaning something unexpected happened in server side .. above errors should be part of validation ideally since they are known scenarios
a). when passed id does not exists in database, e.g. id = 100 but there is no entity of such id
here you should return 404(Not Found) with error msg like 'resource with id 100 not present for deletion/update etc' to make it user friendly
b). when passed id is of wrong type e.g. it is a string "oops"
here you should show 400 (BadRequest) since user is not passing value as per agreement that is resource id.
I am trying to determine if its possible to find out the cause of an exception we occasionally get when using Google App Engine's Memcache.
The documentation states that if the key doesn't exist it returns false, so I believe the error indicates a different issue.
This is the stack trace of the error we are getting:
com.google.appengine.api.memcache.LogAndContinueErrorHandler handleServiceError: Service error in memcache (LogAndContinueErrorHandler.java:49)
com.google.appengine.api.memcache.MemcacheServiceException: Memcache delete: Unknown exception deleting key: ahBzfmlkZXh4LXNlcnZpY2Vzch0LEgdQYXRpZW50GICA6ODn6-ELDAsSA0pvYhgBDA
at com.google.appengine.api.memcache.MemcacheServiceApiHelper$RpcResponseHandler.handleApiProxyException(MemcacheServiceApiHelper.java:69)
at com.google.appengine.api.memcache.MemcacheServiceApiHelper$1.absorbParentException(MemcacheServiceApiHelper.java:111)
at com.google.appengine.api.utils.FutureWrapper.handleParentException(FutureWrapper.java:52)
at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:91)
at com.google.appengine.api.memcache.MemcacheServiceImpl.quietGet(MemcacheServiceImpl.java:26)
at com.google.appengine.api.memcache.MemcacheServiceImpl.delete(MemcacheServiceImpl.java:125)
Here is some relevant code:
boolean putValue = memcacheService.put(syncKey, syncKey, Expiration.byDeltaMillis(60000), MemcacheService.SetPolicy.ADD_ONLY_IF_NOT_PRESENT);
memcacheService.delete(syncKey, 100L);
Later requests are unable to put the same key indicating that the object still exists. Given that I am specifying a 60 second expiration, I'd expect that it'd be available after that time, but it's not the case. Right now, the only way we've found to fix this is to clear the entire Memcache.
I'm looking for simple way of verifying an arbitrary Azure Table connection string that uses a SAS such as the one below using the Azure Storage's Java SDK:
https://example.table.core.windows.net/example?sig=aaabbbcccdddeeefffggghhh%3D&se=2020-01-01T00%3A00%3A00Z&sv=2015-04-05&tn=example&sp=raud
I tried a bunch of different methods exposed by the CloudTable api, but none of them works.
CloudTable.exists() throws a StorageException, regardless of whether the credentials are valid
getName(), getStorageUri(), getUri(), and other getters - all work locally, regardless of the credentials
getServiceClient().downloadServiceProperties() and getServiceClient().getServiceStats() also throw various exceptions, while getServiceClient().getEndpoint() and getServiceClient().getCredentials() and others always work locally.
Why don't I just query the Table for a row or two? Well, in many cases I need to verify a SAS that gives only write or update premissions (without delete or read permissions), and I do not want to execute a statement that changes something in the table just to check the credentials.
To answer your questions:
CloudTable.exists() throws a StorageException, regardless of whether
the credentials are valid
I believe there's a bug with the SDK when using this method with SAS Token. I remember running into the same issue some time back.
getName(), getStorageUri(), getUri(), and other getters - all work
locally, regardless of the credentials
These will work as they don't make network call. They simply use the data available to them in the different instance variables and return the data.
getServiceClient().downloadServiceProperties() and
getServiceClient().getServiceStats() also throw various exceptions,
while getServiceClient().getEndpoint() and
getServiceClient().getCredentials() and others always work locally.
In order for getServiceClient().someMethod() to work using SAS, you would need Account SAS instead of Service SAS (which you're using right now).
Why don't I just query the Table for a row or two? Well, in many cases
I need to verify a SAS that gives only write or update premissions
(without delete or read permissions), and I do not want to execute a
statement that changes something in the table just to check the
credentials.
One possible way to check the validity of a SAS Token for write operation is to perform a write operation which you know will fail with an error. For example, you can try to insert an entity which is already there. In this case, you should get a Conflict (409) error. Other thing you could try to do is perform an optimistic write by specifying some random Etag value and check for Precondition Failed (412) error. If you get a 403 error or 404 error, that would indicate there's something wrong with your SAS token.
I am new to grails and trying to create a form which allows a user to change the email address associated with his/her account for a site I am creating.
It asks for the user for their current password and also for the new email address they want to use.
If the user enters the wrong password or an invalid email address then it should reject them with an appropriate error message.
Now the email validation can be done through constraints in grails, but the password change has to match their current password. I have implemented this check as a method on a service class.
See code below:
def saveEmail =
{
def client = ClientUser.get(session.clientUserID)
client.email = params.email
if(clientUserService.checkPassword(session.clientUserID , params.password) ==false)
{
flash.message = "Incorrect Password"
client.discard()
redirect(action:'changeEmail')
}
else if(!client.validate())
{
flash.message = "Invalid Email Address"
redirect(action:'changeEmail')
}
else
{
client.save();
session.clientUserID = null;
flash.message = "Your email address has been changed, please login again"
redirect(controller: 'clientLogin' , action:'index')
}
}
Now what I noticed that was odd was that if I entered an invalid email then it would not save the changes (as expected) BUT if I entered the wrong password and a valid email then it would save the changes and even write them back into the database even though it would give the correct "invalid password" error message.
I was puzzled so set break points in all the if/else if/else blocks and found that it was hitting the first if statement as expected and not hitting the others , so it would never come accross a call to the save() method, yet it was saved anyway.
After a little research I came accross documentation for the discard() method which you can see used in the code above. So I added this but still no avail. I even tried using discard then reloading the client object from the DB again but still no dice.
This is very frustrating and I would be grateful for any help, since I think that this should surely not be a complicated requirement!
Grails closes your Hibernate session at the end of the web request, which will flush out the changed object. The object's connected to your Hibernate session because you got hold of it via Hibernate (get()). If you want to avoid having the change flushed, you need to use discard().
This is done automatically by a failing validator, which is why you're not having to do it for a validation fail.
You'd simplify the code, however, by either moving this logic to a custom validator on one of your ClientUser fields, which would automatically discard the object on failure, or by using a Grails command object, which could also encapsulate the verification logic. Then you'd just check for errors on the command object.
Using get() method retrieve data in read-write mode, so if you change anything it will be persist.Use read() method that can retrieve data in read-only mode so if you change anything then need to use save() method.That will resolve your problem.
Take a look at this http://grails.org/doc/latest/guide/GORM.html#basicCRUD
Use read method that retrieves an instance of the domain class for the specified id in a read-only state.
See the Grails documentation about read.
You are just doing flash which has nothing to do with rolling back transaction.
In the event of error throw RuntimeException and it will roll back your changes to the database.
After an R & D, I figure out that we need to clear a whole session like:
session.clear()
According to the documentation :
Completely clear the session. Evict all loaded instances and cancel
all pending saves, updates and deletions. Do not close open iterators
or instances of ScrollableResults.
We need to properly cleanup the hibernate session. In result the session is return to usable state and for future use it will not impact by the earlier error.