Is there a way to automatically capture and store data from a tableau dashboard without access to the server.
I'm trying to automate a lot of our weekly audits (i.e. look at the dashboard, count the number of rows on a table, sum up the values in the last column, send out an email to everyone in the same template every week). I need basic functionality such as counting rows and pulling values from certain cells.
I do not have access to tabcmd (corporate blah blah).
I've previously automated Salesforce stuff in Selenium, however, the tableau dashboard is a visualization and not a web element.
I have investigated a few potential routes and was wondering if anyone else could chime in on the best way to approach this (or suggest another).
Use an OCR package to try and recognize values and sum up rows
I somehow automate the report downloading process so I can turn all the data into an excel/CSV file which I can play within Java
Use RPA tool such as UIPath (don't have much experience with the functionality but it seems like it could do it?)
Related
I'm creating a simple real-time stock market java desktop application that shows updated information of stocks. This is my first attempt to create an app that takes in real-time data. My idea to tackle this project is to:
Consume JSON rest API from Alpha Vantage.
Parse JSON into objects that are stored into a DB
Read the objects into observable lists and place into a Tableview
on JavaFX gui.
Since the stocks will be updated every few minutes, I feel like there is no need to have a database. Should I just send a GET request to the rest API every time I need to refresh the data or is there a better approach?
Thanks in advance.
I would say it depends on your needs.
Do you just want to view current stock information in a nicely formatted manner? If so, I would say you do not need to do anything with a database.
If you now, or in the future, would like to use past data to do some sort of statistical analysis, you may need to use a database. If you can use the API to find stock price at every hour for the past 24 hours, you may not need a database for some small analysis.
Step 1: Model the data
Most of our service provide data in either XML or JSON format. Unless the responses are really trivial (e.g. lists of strings), you generally don’t save time by accessing them directly.
Step 2: Fetch the data
The actual usage of the JAX-RS client API is dead simple. Still, I wrapped that part into a service class to separate that from the actual UI code. Your UI code doesn’t necessarily need to know whether it is accessing a REST service, a Web Service or a DB. The public API reveals just the reverse engineered WeatherResponse class for the actual UI code.
Step 3: Consume the data
Using POJOs with pretty much any Java technology is so basic stuff that I’ll skip covering that part thoroughly. To complete the example as a runnable application I hooked the service into a Vaadin UI. From VaadinUI you can select three predefined cities and the ForecastDisplay[ForecastDisplay] loops through the daily forecasts and displays the data in a human-readable form.
We have a requirement to incorporate an excel based tool in java web application. This excel tool has set of master data and couple of result outputs using formula calculations on master data.
Master data can be captured in database with relational tables. We are looking for the best way to provide capability to capture, validate and evaluate. formulas.
So far looked at using scripting engines nashorn and provide formula support using eval. We would like to know how people are doing in other places.
I've searched and found two possible libraries that could be useful for you please have a look.
http://mathparser.org/
http://mathparser.org/mxparser-hello-world/mxparser-hello-world-java/
https://lallafa.objecthunter.net/exp4j/
https://lallafa.objecthunter.net/exp4j/#Evaluating_an_expression_asynchronously
Depends on how big your data is and what your required SLA is. Also on what kind of formulas/other functions that you want to support.
For example, consider a function like sum or max. Now, the master data is in some relation table containing 10k rows. You could pull in all this data inside a java app and do a sum (or run any function). However, imagine if the table contained 500K rows. This would take some time to stream all 500K rows to Java app but consumes lot of cpu and network bandwidth (database resources, local cpu resources). A better optimized scenario in that case would be index that column in the database and let database do all the hard work for you.
Personally, I don't like using eval. I would rather parse the user input to determine what actions to take.
I am assuming that data is not big to use big data tools.
I want to achieve something like this
Given a document say a txt file with an id, I need to process it, do stemming on the words, and generate a index table out of it. But this index table is distributed over 3 systems probably on the basis of the criteria that words beginning with letters from [a-h] are indexed on 1st system, next one third on second and last one third on 3rd system. But i have no idea what technology should i use to achieve this? The index table data structure in ought to be in the RAM so that the search queries can be answered quickly(supposing we are able to index it in this way and have a user searching for a word or sentence from different system). Can this purpose be fulfilled by use of JAVA Sockets?
Actually we(group of 5) are trying to make a small but distributed search engine. Supposing the crawling has been done and the page(the document i was talking about) is saved somewhere and i extract it, do the processing , stemming etc, I would like to finally make a distributed Index data structure based on scheme mentioned above. Would it be possible? I just want to know what technology to use to achieve something like this. Like modifying a data structure inside some program running on some other machine(but in the same network).
Secondly, since we actually don't know if this approach is feasible, if thats the case I would be keen to know the correct way I should look at a distributed index table.
Have the index information saved as you crawl the documents. Have a head node which presents the search user interface. The head node then distributes the search to the index nodes, and collects the results to present to the user.
There are a number of available frameworks, such as Mapreduce, which will help you solve this problem.
I have to create a java program that builds a "tidy" daily schedule for a sports arena.
The program takes in an excel file with a list of "untidy" daily bookings with the following information:
Start Time
End Time
Court
Client
I can read in this information and store it in "BookingObjects".
My question is this:
What is an elegant way to create a printable schedule output, similar to schedules displayed by Microsoft Outlook for example, with time as the rows and which court the booking is on in the columns. The schedule does not need to be interactive, just needs to be printable through the program's GUI and if it can be displayed within the GUI this is also a bonus?
We've recent made the move to Jasper Reports, the main reason was we were producing the reports in PDF, Excel and to the printer, all of which had to execute through different code and different libraries. Jasper Reports has allowed us to go through a single engine.
While not perfect, it does an outstanding job across different export options.
It does have a small learning curve, but the basics allow you to deliver a List of objects which you can then access from within the report and populate your view.
Equally, you can simply connect a SQL data source to it directly
Printing is one of the sore spots in Java. What you can try is create a UI with a table (see the Swing trail for examples) and then print that table.
The problems will start when the table doesn't fit on a single page of paper. If it's longer, then some lines will be printed on the margin (half of it will appear on the previous page and the lower half on the next page). If it's wider, ... let's not go there.
Alternatively, you can try to create a PDF using tools like JasperReports or BIRT. These work better but it will take some time to learn how they work.
Is there a way to speed up the population of a page with GWT's UI elements which are generated from data loaded from the datastore? Can I avoid making the unnecessary RPC call when the page is loaded?
More details about the problem I am experiencing: There is a page on which I generate a table with names and buttons for a list of entities loaded from the datastore. There is an EntryPoint for the page and in its onModuleLoad() I do something like this:
final FlexTable table = new FlexTable();
rpcAsyncService.getAllCandidates(new AsyncCallback<List<Candidate>>() {
public void onSuccess(List<Candidate> candidates) {
int row = 0;
for (Candidate person : candidates) {
table.setText(row, 0, person.getName());
table.setWidget(row, 1, new ToggleButton("Yes"));
table.setWidget(row, 2, new ToggleButton("No"));
row++;
}
}
...
});
This works, but takes more than 30 seconds to load the page with buttons for 300 candidates. This is unacceptable.
The app is running on Google App Engine and using the app engine's datastore.
You could do a lot of things, I will just list them in order that will give you the best impact.
FlexTable is not meant for 300 rows. Since your table is so simple, you should consider generating the HTML by hand, and then using simple HTML widget. Also, 300 rows is a lot of information - consider using pagination. The DynaTable sample app shows you how to do this.
It looks like you are using one GWT module per page. That is the wrong approach to GWT. Loading a GWT module has some non-trivial cost. To understand what I mean, compare browser refresh on gmail v/s the refresh link that gmail provides. That is the same cost you pay when every page in your website has a distinct GWT module.
If the list of candidates is needed across views, you can send it along with the HTML as a JSON object, and then use the Dictionary class in GWT to read it. This saves you the RPC call that you are making. This approach is only recommended if the data is going to be useful across multiple views/screens (like logged in users info)
Check how long your RPC method call is taking. You can enable stats in GWT to figure out where your application is taking time.
You can also run Speed Tracer to identify where the bottleneck is. This is last only because it is obvious FlexTable is performing a lot of DOM manipulations. In general, if you don't know where to start, Speed Tracer is a great tool.
The significant thing here is how you're retrieving the list of candidates, which you haven't shown. 30 seconds is extremely high, and it's unlikely that it's due to the datastore alone.
Have you tried using appstats to profile your app?
Like sri suggested - pagination is easiest and (I think) best solution (along with switching to Grid or just <table>). But in case you wanted for some reason to show/render many rows at once, the GWT Incubator project has a nice wiki page about it - along with some benchmarks showing how FlexTable sucks at large row count. Check out their other tables too ;)
Your problem is that everytime you add something to the FlexTable it has to re-render the whole page and repaint. Try creating a new FlexTable, populating it, when it is fully populated, get rid of the old one and put the new one there.