Forgive me for the beginner question. I'm trying to build a web scraper . I made a helper class Selectors to store CSS selectors of different type and the class has a getter method that returns a optional field value:
public Optional<String> getName() {
return Optional.ofNullable(this.name);
}
In my main Scraper class I have a method to extract the name from the HTML using the Selectors object like this:
public String extractName(Element building) {
if (this.selectors.getName().isPresent()) {
String cssSelector = this.selectors.getName().get();
Elements buildingNameElement = building.select(cssSelector);
return buildingNameElement.text();
}
return "N/A";
}
As much as I've read this isn't a very nice way to use the Optional class. But I'm struggling to come up with a better solution. My first thought was to use the ifPresent method but it does not work in this manner:
public String extractName(Element building) {
this.selectors.getName().ifPresent(() -> {
String cssSelector = this.selectors.getName().get();
Elements buildingNameElement = building.select(cssSelector);
return buildingNameElement.text();
});
return "N/A";
}
I'd like that the Elements.select() would execute only if there's a name field present in the Selectors object. Could anyone help me make the code a bit more functional?
Thanks!
public String extractName(Element building) {
return this.selectors
.getName()
.map(cssSelector -> {
Elements buildingNameElement = building.select(cssSelector);
return buildingNameElement.text();
})
.orElse("N/A");
}
This is what Optional.map is for. When you do return inside a lambda, you are only returning from the lambda, not from the outer method. So the above uses the text of the building name element if getName returned a name/selector. And returns N/A if not.
If you’re fine with a more condensed syntax and fewer named variables, you may go with the following:
return this.selectors
.getName()
.map(cssSelector -> building.select(cssSelector).text())
.orElse("N/A");
Disclaimer: I haven’t got JSoup on my computer, so I haven’t tested. Please forgive if there’s a typo, and report back.
Related
I'm using https://github.com/leangen/graphql-spqr with spring-boot java application. I can reach to alias name easily but how can I reach to original fieldName?
class Food {
#GraphQLQuery(name = "aliasNameX", description = "A food's name")
private String originalName;
...
}
....
#GraphQLQuery(name = "foods") // READ ALL
#Override
public List<Food> getFoods(#GraphQLEnvironment ResolutionEnvironment env) {
DataFetchingFieldSelectionSet selectionSet = env.dataFetchingEnvironment.getSelectionSet();
List<SelectedField> fields = selectionSet.getFields();
for (SelectedField f: fields)
{
System.out.println(f.getName());
}
return foodRepository.findAll();
}
When I run this code, Output looks like with alias fields: "aliasNameX", ..., but I need original name like "originalName". Is there a way to do it?
Solved, according to:
https://github.com/leangen/graphql-spqr/issues/381
Posting my original answer here as well.
You want the underlying field names, but from a level above. Still possible, but ugly :(
for (SelectedField selectedField : env.dataFetchingEnvironment.getSelectionSet().getImmediateFields()) {
Optional<Operation> operation = Directives.getMappedOperation(selectedField.getFieldDefinition());
String javaName = operation.map(op -> ((Member) op.getTypedElement().getElement()).getName()).orElse(null);
}
Be very careful though. If there's more than one Java element exposed per GraphQL field, getTypedElement().getElement() will explode. So to be sure you'd have to call getTypedElement().getElements() (plural) instead and decide what to do. ClassUtils#getPropertyMembers might also be useful, or the ClassUtils.findXXX family of methods.
You'd basically have to do this:
List<AnnotatedElement> elements = getTypedElement().getElements();
//Look for a field and use its name
Optional<String> field = Utils.extractInstances(elements, Field.class).findFirst().map(Field::getName);
//Look for a getter and find its associated field name
Optional<String> getter = Utils.extractInstances(elements, Method.class).findFirst().map(ClassUtils::getFieldNameFromGetter);
This API might have to change in future, as SDL-based tools are proliferating, so complex directives like the ones SPQR is using are causing problems...
I have automated a new customer form for work, but there are a lot of options and based on how questions are answered , different fields need to be filled out. Rather than just make a copy of the code and make a different script for each option, I'd like to do this by passing values to a class that determines what options are chosen based on what is passed in. I'm trying to figure most of this out myself and I'm somewhat of a n00b, but if someone can get me past the first hurdle, I'd like to tackle the rest of the hurdles myself.
So I want to start by just doing one line of the script this way, and eventually I will do more. Up front, it is going to seem like a lot of code just to do this, but here is the line:
driver.findElement(By.id("OrganizationName")).sendKeys("The Rolling Stones");
Here is what I have so far:
ncformPage1 skifootz = new ncformPage1("Rolling Stones");
skifootz.getOrgname();
That is the part that is in the script. Here is the class I wrote:
public class ncformPage1 {
private String orgName;
public ncformPage1(String on) {
orgName = on;
}
public String getOrgname() { return "driver.findElement(By.id(\"OrganizationName\")).sendKeys(\""
+ orgName + "\");";
}
}
So when I run this, it goes right past that organizationName element and leaves it blank, does all the other elements, and then fails because organization name is a required field. So I added this bit of code here to see what it prints out to the console:
System.out.println( skifootz.getOrgname());
Sure enough, it prints out
driver.findElement(By.id("OrganizationName")).sendKeys("Rolling Stones");
Which is exactly what I want returned. (I think the last semicolon is extraneous in this case, but at least it returned what I wanted!) But it doesn't execute that. I've tried all kinds of stuff to get it to execute, such as removing driver from what is returned and appending it here instead:
driver.skifootz.getOrgname();
but that gives me skifootz cannot be resolved or is not a field. I tried this:
String a = skifootz.getOrgname();
driver.a();
But that just made a get underlined in red saying method a() is undefined for the type Webdriver. So then I changed String a to Webdriver a:
WebDriver a = skifootz.getOrgname();
driver.a();
But now skifootz.getOrgname(); is underlined saying "type mismatch: cannot convert from String to WebDriver." I've been messing around with it for a few days now, and I haven't gotten any closer. Maybe this is an easy solution, but if I can just get this part working then perhaps I can move on to the next phase? This n00b thanks everyone in advance for any help anyone can give.
The method is returning a String type and you expect it to act like a driver object. That part is incorrect.
I think you can write methods to be more like
public WebElement getOrgname(WebDriver driver, String OrganizationName) {
return driver.findElement(By.id(OrganizationName));
}
WebElement a = skifootz.getOrgname(driver);
a.sendKeys("Rolling Stones");
OR
public void TypeText(WebDriver driver, String OrganizationName, String TextToType) {
driver.findElement(By.id(OrganizationName)).sendKeys(TextToType);;
}
in your context this should probably work.
ncformPage1 skifootz = new ncformPage1();
skifootz.getOrgname(skifootz.driver, "OrganizationName");
skifootz.sendKeys("Rolling Stones");
public class ncformPage1 {
private String orgName;
public WebDriver driver = new ChromeDriver(); // I'm assuming this part.
public ncformPage1(String on) {
orgName = on;
}
public WebElement getOrgname(WebDriver driver, String OrganizationName) {
return driver.findElement(By.id(OrganizationName));
}
}
We're in the process of trying to identify everywhere that a specific type of object is used only to get a specific property from it, and pass that property into the method instead.
I'm thinking IntelliJ IDEA's "Structural Search" might be a good tool for this, but I'm not sure how to formulate the search template.
A concrete example:
public class MyClass {
public Long getId() {...}
public void setSomethingElse(int se) {...}
}
public class SomeOtherClasses {
public void shouldBeMatched(MyClass mc) {
doSomething();
mc.getId();
doSomethingElse();
}
public void shouldNotBeMatched(MyClass mc) {
doSomething();
mc.getId();
mc.setSomethingElse(14);
doSomethingElse();
}
public void alsoShouldNotBeMatched(MyClass mc) {
shouldBeMatched(mc);
}
}
In the above example, if I'm looking for methods that only use getId, then I should find shouldBeMatched, but not be bothered with shoudNotBeMatched and alsoShouldNotBeMatched, because they do something with the mc object other than call getId().
I'm thinking IntelliJ IDEA's "Structural Search" might be a good tool for this
And it is indeed. The documentation can be tough though.
Let's check Search templates, filters, and script constraints page. It goes as follows.
Let's say, you have a variable that matches a method, a toString()
method. Then this variable is actually a PsiMethod node. Retrieving
variable.parent will produce a PsiClass node, and so forth.
variable.text then will give you the entire text of the method. If you
just need the name of the method, you can use variable.name.
It seems that the task can be done by choosing the right template and writing a corresponding Groovy script.
The template is called methods of the class and can be found under Existing templates. They provide __context__variable to be used with a script.
We have to be sure matched methods have parameters. It is simple enough, just put a count filter on a $Parameter$ variable.
Then we need to extract the name of a parameter of desired type and see if it is called in the body of the method. The following script will do.
def parameters = __context__.getParameterList().getParameters();
def parameter = parameters.find { p -> p.getType().getName().equals('MyClass') };
if (parameter == null) return false;
String parameterName = parameter.getName();
String methodText = __context__.getText();
String occurrence = "${parameterName}.";
String methodCall = "${parameterName}.getId()";
return methodText.count(occurrence) > 0 && methodText.count(occurrence) == methodText.count(methodCall);
Put it in the $Method$ variable filter and verify the results.
I currently have a list of Site objects as my Sites object:
List<Site> sites = new LinkedList();
Site is made up of:
String url;
String id;
Host host;
I want to write a method that tests to see if my any of the 'Site' values in site contains the url String or the id String.
I can't use the contains method for this, since I don't care about the host value.
Is there a easier way to do it via Collections besides writing something like:
public boolean checkValues(String url, String id) {
for (Site : this.sites) {
// Check if the url, id is in the site
}
No there isn't any such method in Collections yet. I am not sure if there are any 3rd party solutions.
Alternatively, the following can be done (if usecase permits ofcourse)
public class Site {
String url;
String id;
String host;
boolean contains(String str) {
if(url.contains(str) || id.contains(str)) return true;
return false;
}
}
=====================
for (Site site : sites) {
if(site.contains(s)) {
//TODO
}
}
Use Guava collection filter predicate
Iterable<Person> filtered = Iterables.filter(allPersons, new Predicate<Person>() {
#Override
public boolean apply(Person p) {
return acceptedNames.contains(p.getName());
}
});
There is same question: Filtering a list of JavaBeans with Google Guava
You can do it with either Guava, as explained in another answer, or natively within Java 8. This makes use of a lambda function and a Stream object.
Suppose we only care about IDs that contain the string 4 or URLs that contain the string 6. Here's one approach to that.
This is the more verbose approach, combining Java 8 lambdas and old-school iteration.
final Stream<Site> filter =
siteList.stream().filter(site -> site.getId().contains("4") || site.getUrl().contains("6"));
for(Iterator<Site> filterIter = filter.iterator(); filterIter.hasNext(); ) {
Site next = filterIter.next();
System.out.println(next);
}
Here's a more succinct way, using the forEach Consumer:
siteList.stream().filter(
site -> site.getId().contains("4") ||
site.getUrl().contains("6")).forEach((site) -> System.out.println(site));
Here's the most terse approach, using the System.out::println method reference.
siteList.stream().filter(
site -> site.getId().contains("4") ||
site.getUrl().contains("6"))
.forEach(System.out::println);
I'm writing a plug-in for Jira and I need to add custom calculated column to issue navigator. That's column should show last comment to issue. But in issue navigator values in this column are something like "ClassName#123456", not comment's body. What should I do to return comment's body to this column?
Code so far:
public class LastCommentField extends CalculatedCFType {
private CommentManager commentManager = null;
public LastCommentField(CommentManager commentManager) {
this.commentManager=commentManager;
}
public Object getValueFromIssue(CustomField field, Issue issue) {
Comment lastComment=null;
List<Comment> comments = commentManager.getComments(issue);
if(comments != null && !comments.isEmpty()) {
lastComment = (Comment)comments.get(comments.size() - 1);
}
return lastComment;
}
public String getStringFromSingularObject (Object object) {
return object.toString();
}
public Object getSingularObjectFromString(String value) {
return value;
}
}
This functionality already exists in at least two plugins, e.g. https://marketplace.atlassian.com/plugins/net.customware.jira.utils.customware-jira-utilities
But in the code above, the singular object being used is a Comment object as documented at http://docs.atlassian.com/jira/4.4/com/atlassian/jira/issue/comments/Comment.html
but you probably just want a String, so try
return lastComment.getBody();
Unfortunately I don't know JIRA from the coding side of things, but from the Java side, that sounds a hell of a lot like the object behind the column doesn't have ToString() overriden. What you are seeing is the name of the class followed by the address in memory.
If you could show us the code behind the column, I might be able to make a little more sense of it.