Here is a non Singleton class which is used to send a payload to an API...
class MyApiClient {
String url = "http://www.yankeeService.com"
int playerId = 99
String playerFirstName = "Aaron"
String playerLastName = "Judge"
public void sendPayload(String content) {
CloseableHttpClient client = HttpClients.createDefault();
HttpPost httpPost = new HttpPost();
String jsonPayload = """ "{"id":"$playerId","name":"$playerLastName","dailyReport":"$content"}" """ ;
StringEntity entity = new StringEntity(jsonPayload);
httpPost.setEntity(entity);
CloseableHttpResponse response = client.execute(httpPost);
assertThat(response.getStatusLine().getStatusCode(), equalTo(200));
client.close();
}
}
Would there be any problem if multiple threads were to enter that sendPayload method?
I think it would be fine because none of the global variables are modified in any way (they are read only and used to facilitate the API call).
Also the jsonPayload is a local variable so each thread would get their own version of it and there would be no chance for one thread to grab the payload content of another right?
Multi-threading issues come to play when threads are using and writing to shared data in an uncontrolled manner.
Meaning:
when all your threads are only invoking the send method, then you do not have a problem - because all threads are reading and using the same data
but when these threads change the content of any of the fields - then all bets are off.
And thing is: your fields have package visibility - this means it is very simple to update them from "outside". An object of MyApiClient would not even notice if field content is changed.
Thus:
first of all, make these fields private to hide such details from the outside
consider making them final as well
Yes it is thread safe. You are trying to post some thing to the remote location. It seems you are not worried about people overwriting the content at the remote location ( if you are then even thread safe logic will not be your help)
Your logic "I think it would be fine because none of the global variables are modified in any way (they are read only and used to facilitate the API call)."
is correct.
For readability and convention purpose I would suggest to use final construct with the attributes.
While there are multiple problems outside the scope of the question with the method you've proposed (what are """, do you really want to be crafting JSON objects by hand, and you don't handle exceptions) your assessment of concurrency appears to be correct.
You may want to ensure that, though, perhaps by making your variables final if they aren't supposed to ever be changed. This way if a future code modification does cause them to be changed, you'll know at compile time that there's a mistake. Or maybe it's not a mistake and those variables need to change... but you'll know you have to revisit your concurrency issue.
You can use http-request built on apache http api. Documentation here.
class MyApiClient {
private static final HttpRequest<?> HTTP_REQUEST =
HttpRequestBuilder.createGet("http://www.yankeeService.com")
.addContentType(ContentType.APPLICATION_JSON)
.build();
int playerId = 99
String playerFirstName = "Aaron"
String playerLastName = "Judge"
public void sendPayload(String content) {
String jsonPayload = """ "{"id":"$playerId","name":"$playerLastName","dailyReport":"$content"}" """ ;
assertThat(HTTP_REQUEST.executeWithBody(jsonPayload).getStatusCode(), equalTo(200));
}
}
HTTP_REQUEST is Thread Safe
Related
I have a Spring Boot application that uses a CredentialsService class to store credentials as GuardedStrings and return them when requested by other classes.
Where the problem arises is in the fact that we use Checkmarx to scan our code and catch potential issues. Where storage of the usernames/passwords are not a problem anymore, I still have to use a String variable to return the plain text credentials. Checkmarx doesn't like that - especially for passwords.
This is the abbreviated view of the CredentialsService:
#Component
public class CredentialsService {
final ExecutorService executor = Executors.newSingleThreadExecutor();
private GuardedString customerApiPassword;
. . .
private StringBuilder clearCustomerApiPassword;
public CredentialsService( . . .
#Value("${customerapi.pwd}") String customerApiPassword,. . .) {
setCustomerApiPassword(customerApiPassword);
. . .
}
private void setCustomerApiPassword(String customerApiPasswordString) {
this.customerApiPassword = new GuardedString(customerApiPasswordString.toCharArray());
this.customerApiPassword.makeReadOnly();
}
public String getCustomerApiPasswordNo() {
clearCustomerApiPassword = new StringBuilder();
customerApiPassword.access(new GuardedString.Accessor() {
#Override
public void access(final char[] clearChars) {
clearCustomerApiPassword.append(clearChars);
}
});
customerApiPassword.dispose();
System.out.println("DGC: clearCustomerApiPassword is " + clearCustomerApiPassword);
Runnable clearFromMemory = () -> {
clearCustomerApiPassword = null;
System.out.println("DGC: clearCustomerApiPassword is " + clearCustomerApiPassword);
};
executor.execute(clearFromMemory);
return clearCustomerApiPassword.toString();
}
And then a requester accesses the values it needs with:
IntegrationApiUtil.setBasicAuthKey(headers, credentialsService.getCustomerApiUsername(), credentialsService.getCustomerApiPassword());
However Checkmarx is still not happy. I use the same approach for storing the GuardedString usernames and passwords and the exact same approach to clearing the Strings that are returned. Checkmarx is fine with the usernames, but it still complains about the passwords:
Method clearCustomerApiPassword; at line 24 of
src/main/java/com/.../service/CredentialsService.java
defines clearCustomerApiPassword, which is designated to contain user passwords. However, while plaintext
passwords are later assigned to clearCustomerApiPassword, this variable is never cleared from memory.
I have tried all sorts of things - a finalize method to destroy the service after it is last used, a disposeAll method to explicitly set all variables to null and call the garbage collector. With the code above I am creating a separate thread in each get method to set the 'clear' variables to null as I return the value to the requester. While I can confirm that this latest approach does provide the requester with the correct values and also sets the variables to null, nothing seems to satisfy Checkmarx.
Does anyone have any ideas?
Thanks in advance.
D
Once you put sensitive data into an immutable object like String, the data will remain in the memory for a long time. You can release the variable, but even without a physical reference the value will still sit in the memory. You can run GC, it will still be there. The only thing that would help would be a creation of another variable using the same memory space and overriding the value.
Long story short: As long as you put your password in a String, Checkmarx will complain.
You have two things you can do:
you either rely on char[] only and clear the array once used,
or use a String value if you are forced to and request a special exception for your case.
Well, you are kinda throwing away all the value of storing passwords in GuardedString by returning/transporting them as regular String.
Don't know a lot about Checkmarx, but it's just a code scanning tool, so it's easy to fool. I suggest actually fixing the problems, instead of trying to sweep them under the rug.
Notice that GuardedString constructor accepts char[], not a String. That's the first problem - you should carry your password from the source up to this point as a char[] - more about it here.
Don't return String to your consumer - return the GuardedString or at least a char[].
Depends on what consumers you are targeting with this class/library, but try to provide a way for them to access the actual passwords for as short time as possible, and clearning the char[] after usage (in a way that consumer will not have to do that himself, since he can forget)
A while ago, I asked how to get the body from a Result in Play 2.5.0 Java.
The answer was basically to use play.core.j.JavaResultExtractor. I am now upgrading to 2.6, and JavaResultExtractor no longer exists (or at least is not public).
How does one do this in Play 2.6?
I did find Result.body().consumeData which seems like it might work, but also comes with the worrisome warning:
This method should be used carefully, since if the source represents an ephemeral stream, then the entity may not be usable after this method is invoked.
I suppose that, since I am doing this in an action, I could call consumeData to get all of the data into a local variable, process that and then return a new result with the stored data. That only fails in the case where the data is too big to fit into memory, which is not something I am currently expecting.
In in Play 2.6 it is still possible to re-implement 2.5 functionality. Please have a look at the example that get Json body from Result:
public static JsonNode resultToJsonNode(final Result result, final long timeout, final Materializer mat)
throws Exception {
FiniteDuration finiteDuration = Duration.create(timeout, TimeUnit.MILLISECONDS);
byte[] body = Await.result(
FutureConverters.toScala(result.body().consumeData(mat)), finiteDuration).toArray();
ObjectMapper om = new ObjectMapper();
final ObjectReader reader = om.reader();
return reader.readTree(new ByteArrayInputStream(body));
}
I have a webservice call to get an authorization token and use it for subsequent webservice calls. Now what we had done earlier was whenever we make any web service call, we first make the token web service and then make the call for actual web service.
Method to get the token is as shown below. Basically what this code does is call the webservice to get the token and using GSON parse the response and get the token.
public static String getAuthTicket() {
String authTicket = null;
HttpResponse httpResponse = getAuthResponse();
String body;
if (httpResponse.getStatusLine().getStatusCode() == 200) {
try {
body = IOUtils.toString(httpResponse.getEntity().getContent());
Gson gson = new GsonBuilder().disableHtmlEscaping().create();
ResponseTicket responseTicket = gson.fromJson(body, ResponseTicket.class);
authTicket = responseTicket.getTicket();
} catch (UnsupportedOperationException e) {
LOGGER.error("UnsupportedOperationException : ",e);
} catch (IOException e) {
LOGGER.error("IO Exception : ",e);
}
}
return authTicket;
}
This has obviously led to performance issue. Hence the party who is providing the webservice to get the token has made the token valid for 30 minutes.
So in the above method what we are thinking is to put the token in cache along with the time and check if the current time - cache time is less than 30. If time is greater than 30 we will make service call to get token and update the token with timestamp in cache.
The only thing is I am fearing is about synchronization, so that I dont get corrupt authtoken due to race condition.
I am thinking to make this static method as synchronized. Do you think is there any other better way.
The answer is: it depends.
Race conditions occur when more than one thread is accessing shared data at the same point in time. So, when you would have code such as:
private final Map<X, Y> sharedCache = new HashMap<>();
public static getAuthTicket() {
if (! sharedCache.containsKey...) {
sharedCache.put(...
...
You would be subject to a race conditions - two threads could come in at the same time, and update that shared map at the very same time; leading to all kinds of problems.
When I get your code right - you would have something similar:
private static String cachedToken = null;
public static getAuthTicket() {
if (cachedToken == null || isTooOld(cachedToken)) {
cachedToken = getAuthTicketForReal();
}
return cachedToken;
}
You probably do not want that two threads call getAuthTicketForReal() in parallel.
So, yes, making that method synchronized is a valid approach.
Where: the real question is: is it sufficient to add that keyword? Given my code - the answer is yes. You simply want to avoid that this cache is setup "in parallel" by more than one thread.
Finally: in case you are worried about the performance impact of using synchronized here - simply forget about that. You are talking about a multi-second "network based" operation; so you absolutely do not worry about the milli second of overhead that synchronized might have (making up this number - the key thing: it is so small that it doesn't matter in the context of the operation you are doing).
Regarding your comment: of course, using synchronized means that the JVM will serialize calls to that method. This means when this method needs 1 minute to return - any other calls to that method will block for that 1 minute.
In that sense; it might be a good exercise to look into ways of writing up this method in a way that does not require synchronized on method level. For example by using data structures that can deal with multiple threads manipulating them.
I'm writing an HTTP servlet (Hosted on Amazon Elastic Beanstalk) that serves as an application server for my Android app. My application will request data from my servlet, which will in turn pull from the database (SimpleDB).
Since my client requests may be frequent, I wanted to implement a cache on my servlet that will cache the requested data so as to cut down on database reads. Currently, I just initialize a "ServerCache" object as a member variable of my servlet. ServerCache contains lists for cached data, and I populate these as I go.
Looks something like this.
public class ServerCache {
/**
* ServerCache is responsible for caching data on the server.
* Create several data structures on the server to cache buy/sell listings as listing objects.
* For now, we will be able to cache the entirety of the database contents.
* -ES
*
* Only one ServerCache should ever be made, at the init() of this server
*/
private List<BuyListing> listbl; //What is the danger of having this public vs. private
private List<SellListing> listsl;
public String BLlastError;
public String SLlastError;
public ServerCache()
{
this.listbl = new ArrayList<BuyListing>();
this.listsl = new ArrayList<SellListing>();
this.BLlastError = "Initialized";
this.SLlastError = "Initialized";
} //Setters and getters ommitted
And it is initialized here
public class HelloWorld extends HttpServlet {
private SimpleDBConnect SDB; // contains functions for accessing database
private ServerCache Cache;
//Various Constants
private static final String BUY_LISTINGS_DOMAIN = "BuyListings";
private static final String SELL_LISTINGS_DOMAIN = "SellListings";
private static final String USER_ID_DOMAIN = "UserIDs";
private static final String MESSAGES_DOMAIN = "Messages";
public HelloWorld() {
super();
SDB = new SimpleDBConnect();
SDB.createConnection();
Cache = new ServerCache();
Cache.setListbl(SDB.getAllBL());
Cache.setListsl(SDB.getAllSL());
updateSDBStatus();
updateServletStatus("Initialized");
updateCacheStatus();
}
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException
{
response.getOutputStream().println("Servlet Works \n SimpleDB Status:" + SDB.dbConnectStatus + "\n \n SDB log: " + this.SDBStatus + "\n \n Servlet log: " + this.ServletStatus
+ "\n \n Buy Cache Status: " + this.BLCacheStatus + "\n \n Sell Cache Status: " + this.SLCacheStatus);
}
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
// ... Some code ommitted, the following is a relevant part of how I handle request for data
if (packet.getHeader() == Constants.BL_REQUEST || packet.getHeader() == Constants.SL_REQUEST)
{
MsgStruct temp = new MsgStruct();
if (packet.getHeader() == Constants.BL_REQUEST){
temp.setHeader(Constants.BL_REQUEST);
Type listOfTestObject = new TypeToken<List<BuyListing>>(){}.getType();
String s = gson.toJson(Cache.getListbl(), listOfTestObject);
temp.setPayload(s);
receivedString = gson.toJson(temp);
}
Is there any reason why caching a version of the data on my servlet will be a bad idea? This is just my stab in the dark, as nobody has ever taught me what to do in this case. I'm aware that concurrency is an issue, but I dont think it should matter in this case?
Thanks
Adding a cache into the program will complicate it a little bit, specifically when you want to modify the data. In this case, the database and the cache should always have the same data, no matter what GET or POST requests are received that may or may not write to the database.
A problem you may run into is how to keep the cache in sync with the database. It will be crucial to always update the cache when writing to the database, and make sure that when you fulfill a request with the cache, that you return the same data as is contained in the database. You did say that you're only caching data requested, but if data is ever written back to the database, you will have to check that cached data is also updated.
You mentioned that you'll only instantiate one instance of the class. It might make more sense to follow the singleton pattern for the cache, see the Wikipedia page http://en.wikipedia.org/wiki/Singleton_pattern for more information. Basically, this causes the cache to initialize itself and make sure no other code can make another instance. The constructor will be private, and you would write a getInstance() method to have the cache return the instantiated ServerCache object.
The reason I'm going this route is because there's always a possibility that you will write another servlet that needs to access the database, and it will be difficult to give it cache access without making the cache a singleton (not to mention multithreading issues associated with multiple reads).
Another thing to remember, your cache will take up RAM on the server. Caching trades of RAM usage for access time, and depending on your plan, you may or may not have a lot of that to spare. It's also a real possibility that you'd just replicate your entire database in RAM, defeating its purpose. You would spend a bunch of processing time making sure you update the database but never actually use it. So again, if you see an improvement in responsiveness, I see no problem with it, as long as you manage it well.
To effectively use a cache, you'd have to keep a strict limit on the number of cached objects, getting rid of objects that aren't used much, keeping only the ones that are accessed very frequently. A final point I want to make is that I noticed your cache is using ArrayLists. Those aren't the most efficient data structures for searching unsorted data. A database is made for data storage, so the database most likely has a more efficient storage system anyway, which in your case would make a cache only useful for very small amounts of frequently accessed but seldom modified data.
I'm looking for some code that will return the current instance state regardless of whether the instance is currently running or not. I found some code that will return the desired result on a running instance, but when trying to find one that will work on stopped instances, I was overwhelmed by a number of similar looking classes that appeared to perform a similar operation, but in the end, did not work or were poorly documented.
Anyway, the running instance version of the code is below:
public Integer getInstanceStatus(String instanceId) {
DescribeInstanceStatusRequest describeInstanceRequest = new DescribeInstanceStatusRequest().withInstanceIds(instanceId);
DescribeInstanceStatusResult describeInstanceResult = ec2.describeInstanceStatus(describeInstanceRequest);
List<InstanceStatus> state = describeInstanceResult.getInstanceStatuses();
return state.get(0).getInstanceState().getCode();
}
So I'm basically looking for the equivalent that does not have the DescribeInstanceStatus's restriction that requires the instance to be running. I'd assume this is possible since the getCode() documentation shows it as being able to return the value 80 which denotes a stopped instance.
And once again, I answer my own question... Though I am open to other suggestions.
public Integer getInstanceStatus(String instanceId) {
DescribeInstancesRequest describeInstanceRequest = new DescribeInstancesRequest().withInstanceIds(instanceId);
DescribeInstancesResult describeInstanceResult = ec2.describeInstances(describeInstanceRequest);
InstanceState state = describeInstanceResult.getReservations().get(0).getInstances().get(0).getState();
return state.getCode();
}
If you set withIncludeAllInstances(true) on your request then you can get states for the not running instances.
DescribeInstanceStatusResult result = ec2.describeInstanceStatus(new DescribeInstanceStatusRequest()
.withInstanceIds(instanceIds).withIncludeAllInstances(true));