I am trying create one application from where I can get the all the information related to user/Tracks/Album.
I have used below code for authentication using clientid and client secret id.
private static String clintId = "generated id";
private static String clientSecretId = "generated secret id";
private static final URI redirectUri = SpotifyHttpManager.makeUri("http://localhost:8080/api/get-user-code/");
public static final SpotifyApi spotifyApi = new SpotifyApi.Builder()
.setClientId(clintId)
.setClientSecret(clientSecretId)
.setRedirectUri(redirectUri)
.build();
#GetMapping("login")
#ResponseBody
public String spotifyLogin(){
AuthorizationCodeUriRequest authorizationCodeUriRequest = spotifyApi.authorizationCodeUri()
.scope("user-read-private user-read-email user-top-read user-library-read user-library-modify")
.show_dialog(true)
.build();
final URI uri = authorizationCodeUriRequest.execute();
return uri.toString();
}
#GetMapping(value="get-user-code")
public void getSpotifyUserCode(#RequestParam("code") String userCode, HttpServletResponse response) throws IOException {
AuthorizationCodeRequest authorizationCodeRequest = spotifyApi.authorizationCode(userCode)
.build();
try {
final AuthorizationCodeCredentials authorizationCodeCredentials = authorizationCodeRequest.execute();
// Set access and refresh token for further "spotifyApi" object usage
System.out.println("Access token::"+authorizationCodeCredentials.getAccessToken());
spotifyApi.setAccessToken(authorizationCodeCredentials.getAccessToken());
spotifyApi.setRefreshToken(authorizationCodeCredentials.getRefreshToken());
System.out.println("Expires in: " + authorizationCodeCredentials.getExpiresIn());
} catch (IOException | SpotifyWebApiException | org.apache.hc.core5.http.ParseException e){
System.out.println("Error: " + e.getMessage());
}
response.sendRedirect("http://localhost:3000/home");
}
Now take one example: Suppose I have created above client id and secret id from User A (From development dashboard) so for User A everything is working fine like I am able to get user profile, Track etc.
But suppose if I try to get same details for User B like user profile/tracks etc with User A generated client id and secret id then not able to get those information for User B its showing "forbidden".
So my question is that how to get those information for user B (with user A client id and secret details)?
I am using hbase client 2.1.7 to connect to my server(same version 2.1.7).
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>2.1.7</version>
Now there is an user who have permission to read/write on the table in the server.
User = LTzm#yA$U
For this my code looks like this:
String hadoop_user_key = "HADOOP_USER_NAME";
String user = "LTzm#yA$U";
System.setProperty(hadoop_user_key, token);
Now when I am trying to read the key from the table i am getting following error:
error.log:! Causing:
org.apache.hadoop.hbase.security.AccessDeniedException:
org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient
permissions for user 'LTzm' (table=table_name, action=READ)
Weird part is writes are working fine. To validate that whether right user is getting passed for write, i removed the user and try rerun the code and the write fails with the error:
error.log:! org.apache.hadoop.hbase.ipc.RemoteWithExtrasException:
org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient
permissions (user=LTzm#yA$U,
scope=table_name, family=d:visitId,
params=[table=table_name,family=d:visitId],action=WRITE)
Again read was also failing with:
error.log:! org.apache.hadoop.hbase.ipc.RemoteWithExtrasException:
org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient
permissions for user 'LTzm'
(table=table_name, action=READ)
Somehow Ltzm is getting passed with read call and LTzm#yA$U is getting passed for write.
Does anyone help me what is the issue here, Is # or special symbol not allowed in the user for hbase(then how is it working for write calls).
Edit 1:
Here is the function to create connection:
public static Connection createConnection() {
String hadoop_user_key = "HADOOP_USER_NAME";
String user = "LTzm#yA$U";
Map<String, String> configMap = new HashMap<>();
configMap.put("hbase.rootdir", "hdfs://session/apps/hbase/data"));
configMap.put("hbase.zookeeper.quorum", "ip1, ip2");
configMap.put("zookeeper.znode.parent", "/hbase");
configMap.put("hbase.rpc.timeout", "400");
configMap.put("hbase.rpc.shortoperation.timeout", "400");
configMap.put("hbase.client.meta.operation.timeout", "5000");
configMap.put("hbase.rpc.engine", "org.apache.hadoop.hbase.ipc.SecureRpcEngine");
configMap.put("hbase.client.retries.number", "3");
configMap.put("hbase.client.operation.timeout", "3000"));
configMap.put(HConstants.HBASE_CLIENT_IPC_POOL_SIZE, "30"));
configMap.put("hbase.client.pause", "50"));
configMap.put("hbase.client.pause.cqtbe", "1000"));
configMap.put("hbase.client.max.total.tasks", "500"));
configMap.put("hbase.client.max.perserver.tasks", "50"));
configMap.put("hbase.client.max.perregion.tasks", "10"));
configMap.put("hbase.client.ipc.pool.type", "RoundRobinPool");
configMap.put("hbase.rpc.read.timeout", "200"));
configMap.put("hbase.rpc.write.timeout", "200"));
configMap.put("hbase.client.write.buffer", "20971520"));
System.setProperty(hadoop_user_key, token);
Configuration hConfig = HBaseConfiguration.create();
for (String key : configMap.keySet())
hConfig.set(key, configMap.get(key));
UserGroupInformation.setConfiguration(hConfig);
Connection hbaseConnection;
hbaseConnection = ConnectionFactory.createConnection(config);
return connection;
}
Here are the read and write calls:
protected Result read(String tableName, String rowKey) throws IOException {
Get get = new Get(Bytes.toBytes(rowKey));
get.addFamily(COLUMN_FAMILY_BYTES);
Result res;
Table hTable = null;
try {
hTable = getHbaseTable(tableName);
res = hTable.get(get);
} finally {
if (hTable != null) {
releaseHbaseTable(hTable);
}
}
return res;
}
protected void writeRow(String tableName, String rowKey, Map<String, byte[]> columnData) throws IOException {
Put cellPut = new Put(Bytes.toBytes(rowKey));
for (String qualifier : columnData.keySet()) {
cellPut.addColumn(COLUMN_FAMILY_BYTES, Bytes.toBytes(qualifier), columnData.get(qualifier));
}
Table hTable = null;
try {
hTable = getHbaseTable(tableName);
if (hTable != null) {
hTable.put(cellPut);
}
} finally {
if (hTable != null) {
releaseHbaseTable(hTable);
}
}
}
private Table getTable(String tableName) {
try {
Table table = hbaseConnection.getTable(TableName.valueOf(tableName));
} catch (IOException e) {
LOGGER.error("Exception while adding table in factory.", e);
}
}
I'm currently new to the Spring Boot Java framework and I'm building a simple application. When my service starts, I want to be able to read a raw file from a URL, parse that data, and upload it into my mongodb database of atlas. So far this is what I have:
#Service
public class CoronaVirusDataService {
private List<LocationStats> allConfirmedStats = new ArrayList<>();
MongoOperations mongoOperations;
#PostConstruct // run this method as soon as the application runs
#Scheduled(cron = "* * 1 * * *") // execute this method every day
public void fetchVirusData() {
List<LocationStats> newStats = new ArrayList<>(); // to hold the stats of each state
HttpClient client = HttpClient.newHttpClient();
// creating a new http request
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create(ConstantsUtil.VIRUS_CONFIRMED_DATA_URL))
.build();
// get a response by having the client send the request
try {
HttpResponse<String> httpResponse = client.send(request, HttpResponse.BodyHandlers.ofString());
// parse the body of the request from csv format to readable format
StringReader csvBodyReader = new StringReader(httpResponse.body());
Iterable<CSVRecord> records = CSVFormat.DEFAULT.withFirstRecordAsHeader().parse(csvBodyReader);
for (CSVRecord record: records) {
// create a model with the parsed data
LocationStats stats = new LocationStats();
stats.setState(record.get("Province/State"));
stats.setCountry(record.get("Country/Region"));
// the latest day
int latestCases = Integer.parseInt(record.get(record.size() - 1));
int prevDayCases = Integer.parseInt(record.get(record.size() - 2));
stats.setLatestTotalCases(latestCases);
stats.setDiffFromPreviousDay(prevDayCases);
mongoOperations.save(LocationStats);
// add to new stats
newStats.add(stats);
}
// assign to class array -> we use this array to display the data
this.allConfirmedStats = newStats;
} catch (IOException | InterruptedException e) {
e.printStackTrace();
}
}
}
So the main issue with this is the data is not saving to the mongoDB once I call mongoOperations.save(). Also, I've learned that it is bad practice to maintain some type of state in a Service. What is the best practice for this? Will inserting the data into MongoDB take care of that since we are not managing state.
Here is my model class that I want to save to mongodb
#Document(collection = "LocationStats")
public class LocationStats {
/** Location model to show corona virus statistics in each state*/
#Id
private String state;
private String country;
private int latestTotalCases;
private int diffFromPreviousDay;
public String getState() {
return state;
}
public void setState(String state) {
this.state = state;
}
public String getCountry() {
return country;
}
public void setCountry(String country) {
this.country = country;
}
public int getLatestTotalCases() {
return latestTotalCases;
}
public void setLatestTotalCases(int latestTotalCases) {
this.latestTotalCases = latestTotalCases;
}
public int getDiffFromPreviousDay() {
return diffFromPreviousDay;
}
public void setDiffFromPreviousDay(int diffFromPreviousDay) {
this.diffFromPreviousDay = diffFromPreviousDay;
}
#Override
public String toString() {
return "LocationStats{" +
"state='" + state + '\'' +
", country='" + country + '\'' +
", latestTotalCases=" + latestTotalCases +
'}';
}
}
once I have my models saved into mongoDB, I want to read from the database and get all the data from each collection and display it on the webpage. I'm thinking I'd fetch that data within the controller class and pass it to the frontend, is this good practice? here is my controller class.
#Controller
public class HomeController {
/** Controller class to generate/render the html UI */
#Autowired
CoronaVirusDataService coronaVirusDataService;
#Autowired
MongoOperations mongoOperations;
#GetMapping("/") // map this to the root template
public String home(Model model) {
List<LocationStats> allStats = coronaVirusDataService.getAllConfirmedStats();
// instead of above getter method, have a method call that fetches all data from mongoDB and return it as a List<LocationStats>
// get the total confirmed cases
int totalConfirmedCases = allStats.stream().mapToInt(LocationStats::getLatestTotalCases).sum();
int totalNewCases = allStats.stream().mapToInt(LocationStats::getDiffFromPreviousDay).sum();
// send the models to the view
model.addAttribute("locationStats", allStats);
model.addAttribute("totalReportedCases", totalConfirmedCases);
model.addAttribute("totalNewCases", totalNewCases);
return "home";
}
}
I have a Java Spring based web application and I want to insert a record to a table only if the table does not contain any rows that are "similar" (according to some specific, irrelevant criteria) to the new row.
Because this is a multi-threaded environment, I cannot use a SELECT+INSERT two-step combination as it would expose me to a race condition.
The same question was first asked and answered here and here several years ago. Unfortunately, the questions have got only a little attention and the provided answer is not sufficient to my needs.
Here's the code I currently have and it's not working:
#Component("userActionsManager")
#Transactional
public class UserActionsManager implements UserActionsManagerInterface {
#PersistenceContext(unitName = "itsadDB")
private EntityManager manager;
#Resource(name = "databaseManager")
private DB db;
...
#SuppressWarnings("unchecked")
#Override
#PreAuthorize("hasRole('ROLE_USER') && #username == authentication.name")
public String giveAnswer(String username, String courseCode, String missionName, String taskCode, String answer) {
...
List<Submission> submissions = getAllCorrectSubmissions(newSubmission);
List<Result> results = getAllCorrectResults(result);
if (submissions.size() > 0
|| results.size() > 0) throw new SessionAuthenticationException("foo");
manager.persist(newSubmission);
manager.persist(result);
submissions = getAllCorrectSubmissions(newSubmission);
results = getAllCorrectResults(result);
for (Submission s : submissions) manager.lock(s, LockModeType.OPTIMISTIC_FORCE_INCREMENT);
for (Result r : results ) manager.lock(r, LockModeType.OPTIMISTIC_FORCE_INCREMENT);
manager.flush();
...
}
#SuppressWarnings("unchecked")
private List<Submission> getAllCorrectSubmissions(Submission newSubmission) {
Query q = manager.createQuery("SELECT s FROM Submission AS s WHERE s.missionTask = ?1 AND s.course = ?2 AND s.user = ?3 AND s.correct = true");
q.setParameter(1, newSubmission.getMissionTask());
q.setParameter(2, newSubmission.getCourse());
q.setParameter(3, newSubmission.getUser());
return (List<Submission>) q.getResultList();
}
#SuppressWarnings("unchecked")
private List<Result> getAllCorrectResults(Result result) {
Query q = manager.createQuery("SELECT r FROM Result AS r WHERE r.missionTask = ?1 AND r.course = ?2 AND r.user = ?3");
q.setParameter(1, result.getMissionTask());
q.setParameter(2, result.getCourse());
q.setParameter(3, result.getUser());
return (List<Result>) q.getResultList();
}
...
}
According to the answer provided here I am supposed to somehow use OPTIMISTIC_FORCE_INCREMENT but it's not working. I suspect that the provided answer is erroneous so I need a better one.
edit:
Added more context related code. Right now this code still has a race condition. When I make 10 simultaneous HTTP POST requests approximately 5 rows will get erroneously inserted. Other 5 requests are rejected with HTTP error code 409 (conflict). The correct code would guarantee that only 1 row would get inserted to the database no matter how many concurrent requests I make. Making the method synchronous is not a solution since the race condition still manifests for some unknown reason (I tested it).
Unfortunately after several days of research I was unable to find a short and simple solution to my problem. Since my time budget is not unlimited I had to come up with a workaround. Call it a kludge if you may.
Since the whole HTTP request is a transaction, it will be rolled back at the sight of any conflicts. I am using this for my advantage by locking a special entity within the context of the whole HTTP request. Should multiple HTTP requests be received at the same time, all but one will result in some PersistenceException.
In the beginning of the transaction I am checking whether no other correct answers have been submitted yet. During that check the lock is already effective so no race condition could happen. The lock is effective until the answer is submitted. This basically simulates a critical section as a SELECT+INSERT two step query on the application level (in pure MySQL I would have used the INSERT IF NOT EXISTS construct).
This approach has some drawbacks. Whenever two students submit an answer at the same time, one of them will be thrown an exception. This is sort of bad for performance and bandwidth because the student who received HTTP STATUS 409 has to resubmit their answer.
To compensate the latter, I am automatically retrying to submit the answer on the server side a couple of times between randomly chosen time intervals. See the according HTTP request controller code is below:
#Controller
#RequestMapping("/users")
public class UserActionsController {
#Autowired
private SessionRegistry sessionRegistry;
#Autowired
#Qualifier("authenticationManager")
private AuthenticationManager authenticationManager;
#Resource(name = "userActionsManager")
private UserActionsManagerInterface userManager;
#Resource(name = "databaseManager")
private DB db;
.
.
.
#RequestMapping(value = "/{username}/{courseCode}/missions/{missionName}/tasks/{taskCode}/submitAnswer", method = RequestMethod.POST)
public #ResponseBody
Map<String, Object> giveAnswer(#PathVariable String username,
#PathVariable String courseCode, #PathVariable String missionName,
#PathVariable String taskCode, #RequestParam("answer") String answer, HttpServletRequest request) {
init(request);
db.log("Submitting an answer to task `"+taskCode+"` of mission `"+missionName+
"` in course `"+courseCode+"` as student `"+username+"`.");
String str = null;
boolean conflict = true;
for (int i=0; i<10; i++) {
Random rand = new Random();
int ms = rand.nextInt(1000);
try {
str = userManager.giveAnswer(username, courseCode, missionName, taskCode, answer);
conflict = false;
break;
}
catch (EntityExistsException e) {throw new EntityExistsException();}
catch (PersistenceException e) {}
catch (UnexpectedRollbackException e) {}
try {
Thread.sleep(ms);
} catch(InterruptedException ex) {
Thread.currentThread().interrupt();
}
}
if (conflict) str = userManager.giveAnswer(username, courseCode, missionName, taskCode, answer);
if (str == null) db.log("Answer accepted: `"+answer+"`.");
else db.log("Answer rejected: `"+answer+"`.");
Map<String, Object> hm = new HashMap<String, Object>();
hm.put("success", str == null);
hm.put("message", str);
return hm;
}
}
If for some reason the controller is unable to commit the transaction 10 times in a row then it will try one more time but will not attempt to catch the possible exceptions. When an exception is thrown on the 11th try then it will be processed by the global exception controller and the client will receive HTTP STATUS 409. The global exception controller is defined below.
#ControllerAdvice
public class GlobalExceptionController {
#Resource(name = "staticDatabaseManager")
private StaticDB db;
#ExceptionHandler(SessionAuthenticationException.class)
#ResponseStatus(value=HttpStatus.FORBIDDEN, reason="session has expired") //403
public ModelAndView expiredException(HttpServletRequest request, Exception e) {
ModelAndView mav = new ModelAndView("exception");
mav.addObject("name", e.getClass().getSimpleName());
mav.addObject("message", e.getMessage());
return mav;
}
#ExceptionHandler({UnexpectedRollbackException.class,
EntityExistsException.class,
OptimisticLockException.class,
PersistenceException.class})
#ResponseStatus(value=HttpStatus.CONFLICT, reason="conflicting requests") //409
public ModelAndView conflictException(HttpServletRequest request, Exception e) {
ModelAndView mav = new ModelAndView("exception");
mav.addObject("name", e.getClass().getSimpleName());
mav.addObject("message", e.getMessage());
synchronized (db) {
db.setUserInfo(request);
db.log("Conflicting "+request.getMethod()+" request to "+request.getRequestURI()+" ("+e.getClass().getSimpleName()+").", Log.LVL_SECURITY);
}
return mav;
}
//ResponseEntity<String> customHandler(Exception ex) {
// return new ResponseEntity<String>("Conflicting requests, try again.", HttpStatus.CONFLICT);
//}
}
Finally, the giveAnswer method itself utilizes a special entity with a primary key lock_addCorrectAnswer. I lock that special entity with the OPTIMISTIC_FORCE_INCREMENT flag which makes sure that no two transactions can have overlapping execution times for the giveAnswer method. The respective code can be seen below:
#Component("userActionsManager")
#Transactional
public class UserActionsManager implements UserActionsManagerInterface {
#PersistenceContext(unitName = "itsadDB")
private EntityManager manager;
#Resource(name = "databaseManager")
private DB db;
.
.
.
#SuppressWarnings("unchecked")
#Override
#PreAuthorize("hasRole('ROLE_USER') && #username == authentication.name")
public String giveAnswer(String username, String courseCode, String missionName, String taskCode, String answer) {
.
.
.
if (!userCanGiveAnswer(user, course, missionTask)) {
error = "It is forbidden to submit an answer to this task.";
db.log(error, Log.LVL_MAJOR);
return error;
}
.
.
.
if (correctAnswer) {
.
.
.
addCorrectAnswer(newSubmission, result);
return null;
}
newSubmission = new Submission(user, course, missionTask, answer, false);
manager.persist(newSubmission);
return error;
}
private void addCorrectAnswer(Submission submission, Result result) {
String var = "lock_addCorrectAnswer";
Global global = manager.find(Global.class, var);
if (global == null) {
global = new Global(var, 0);
manager.persist(global);
manager.flush();
}
manager.lock(global, LockModeType.OPTIMISTIC_FORCE_INCREMENT);
manager.persist(submission);
manager.persist(result);
manager.flush();
long submissions = getCorrectSubmissionCount(submission);
long results = getResultCount(result);
if (submissions > 1 || results > 1) throw new EntityExistsException();
}
private long getCorrectSubmissionCount(Submission newSubmission) {
Query q = manager.createQuery("SELECT count(s) FROM Submission AS s WHERE s.missionTask = ?1 AND s.course = ?2 AND s.user = ?3 AND s.correct = true");
q.setParameter(1, newSubmission.getMissionTask());
q.setParameter(2, newSubmission.getCourse());
q.setParameter(3, newSubmission.getUser());
return (Long) q.getSingleResult();
}
private long getResultCount(Result result) {
Query q = manager.createQuery("SELECT count(r) FROM Result AS r WHERE r.missionTask = ?1 AND r.course = ?2 AND r.user = ?3");
q.setParameter(1, result.getMissionTask());
q.setParameter(2, result.getCourse());
q.setParameter(3, result.getUser());
return (Long) q.getSingleResult();
}
}
It is important to note that the entity Global has to have a version annotation in its class for the OPTIMISTIC_FORCE_INCREMENT to work (see code below).
#Entity
#Table(name = "GLOBALS")
public class Global implements Serializable {
.
.
.
#Id
#Column(name = "NAME", length = 32)
private String key;
#Column(name = "INTVAL")
private int intVal;
#Column(name = "STRVAL", length = 4096)
private String strVal;
#Version
private Long version;
.
.
.
}
Such an approach can be optimized even further. Instead of using the same lock name lock_addCorrectAnswer for all giveAnswer calls, I could generate the lock name deterministically from the name of the submitting user. For example, if the student's username is Hyena then the primary key for the lock entity would be lock_Hyena_addCorrectAnswer. That way multiple students could submit answers at the same time without receiving any conflicts. However, if a malicious user spams the HTTP POST method for submitAnswer 10x in parallel they will be prevented by the this locking mechanism.
I have a simple entity class and it is supposed to include unique names on it.
#Entity
class Package {
#PrimaryKey(sequence = "ID")
public Long id;
#SecondaryKey(relate = Relationship.ONE_TO_ONE)
public String name;
private Package() {}
public Package(String name) { this.name = name; }
#Override
public String toString() { return id + " : " + name; }
}
I want to use deferred writing option because of extensive modification. Here is the test i tried and its output.
final String dbfilename = "test01";
new File(dbfilename).mkdirs();
EnvironmentConfig config = new EnvironmentConfig().setAllowCreate(true);
Environment environment = new Environment(new File(dbfilename), config);
StoreConfig storeConfig = new StoreConfig().setAllowCreate(true).setDeferredWrite(true);
EntityStore store = new EntityStore(environment, "", storeConfig);
PrimaryIndex<Long, Package> primaryIndex = store.getPrimaryIndex(Long.class, Package.class);
try {
primaryIndex.put(new Package("package01")); // will be put.
primaryIndex.put(new Package("package01")); // throws exception.
} catch (UniqueConstraintException ex) {
System.out.println(ex.getMessage());
}
store.sync(); // flush them all
// expecting to find one element
SortedMap<Long,Package> sortedMap = primaryIndex.sortedMap();
for (Package entity : sortedMap.values()) {
System.out.println(entity);
}
Output
(JE 5.0.73) Unique secondary key is already present
1 : package01
2 : package01
So my question is that even if it throws exception while putting second package, why does it lists two packages. Any way to avoid this without using transactions?
Thanks.