We Keep Coding

Add https to missing strings of an array?

I'm writing an app for a client who doesn't have an official API but wants the app to extract video links from his website so I wrote a logic using jsoup. Everything seems to work fine except some of the links don't start with https so I'm trying to add it before the URL.
Here's my code:
new Thread(() -> {
final StringBuilder jsoupStr = new StringBuilder();
String URL = "https://example.com" +titleString
.replaceAll(":", "")
.replaceAll(",", "")
.replaceAll(" ", "-")
.toLowerCase();
Log.d("CALLING_URL", " " +URL);
try {
Document doc = Jsoup.connect(URL).get();
Element content = doc.getElementById("list-eps");
Elements links = content.getElementsByTag("a");
for (Element link : links) {
jsoupStr.append("\n").append(link.attr("player-data"));
}
} catch (IOException e) {
e.getMessage();
}
String linksStr = jsoupStr.toString().trim();
if (!linksStr.startsWith("https://")) {
linksStr = "https:" + linksStr;
}
String[] links_array = linksStr.split("\n");
arrayList.addAll(Arrays.asList(links_array));
}).start();
The website contains about 10 links per video but some links start like "//" instead of https.
This code adds the https but only for the first link it finds missing.
if (!linksStr.startsWith("https://")) {
linksStr = "https:" + linksStr;
}

You need to iterate over your final array to apply your function to all links.
String[] links_array = linksStr.split("\n");
for(int i = 0; i < length; i++)
if(!links_array[i].startsWith("https://"))
links_array[i] = "https:" + links_array[i];

If this code working just for first missing link:
if (!linksStr.startsWith("https://")) {
linksStr = "https:" + linksStr;
}
I believe you can use loop for control every link.

Scraping multiple pages with jsoup

I am trying to scrap links in pagination of GitHub repositories
I have scraped them separately but what Now I want is to optimize it using some loop. Any idea how can i do it? here is code
ComitUrl= "http://github.com/apple/turicreate/commits/master";
Document document2 = Jsoup.connect(ComitUrl ).get();
Element pagination = document2.select("div.pagination a").get(0);
String Url1 = pagination.attr("href");
System.out.println("pagination-link1 = " + Url1);
Document document3 = Jsoup.connect(Url1).get();
Element pagination2 = document3.select("div.pagination a").get(1);
String Url2 = pagination2.attr("href");
System.out.println("pagination-link2 = " + Url2);
Document document4 = Jsoup.connect(Url2).get();
Element check = document4.select("span.disabled").first();
if (check.text().equals("Older")) {
System.out.println("No pagination link more");
}
else { Element pagination3 = document4.select("div.pagination a").get(1);
String Url3 = pagination3.attr("href");
System.out.println("pagination-link3 = " + Url3);
}

Try something like given below:
public static void main(String[] args) throws IOException{
String url = "http://github.com/apple/turicreate/commits/master";
//get first link
String link = Jsoup.connect(url).get().select("div.pagination a").get(0).attr("href");
//an int just to count up links
int i = 1;
System.out.println("pagination-link_"+ i + "\t" + link);
//parse next page using link
//check if the div on next page has more than one link in it
while(Jsoup.connect(link).get().select("div.pagination a").size() >1){
link = Jsoup.connect(link).get().select("div.pagination a").get(1).attr("href");
System.out.println("pagination-link_"+ (++i) +"\t" + link);
}
}

URL is getting displayed as String in table

I wanted to display one URL in a table.
Value I had was : www.yahoo.com
I converted into format as below and returning back to table :
www.yahoo.com</font>"> Link
I want it to be displayed in URL form. But it is displaying as a string like as it is above.
Below is the method used for conversion :
private static String validateString(String header) throws Exception
{
System.out.println("&&&&&&&&&& inside validateString ");
String displayString = "";
String colorValue = "";
try
{
if((header) != null && !"null".equalsIgnoreCase(header))
{
displayString = header;
colorValue = "red";
displayString = "<font color=\""+colorValue+"\">"+displayString+"</font>";
}
}
catch(Exception e)
{
displayString = " ";
}
if(!"".equals(displayString))
{
displayString = " Link ";
}
if ("".equals(displayString))
{
displayString = " ";
}
return displayString;
}
Please let me know how to display as URL.
Thanks.

In the attribute href for the tag A, you only should use the proper URL not ant other tag. Change it to something like this:
Link

JAVA - How to access variables inside a FOR loop, from outside the for loop

My code is to add RSS feeds to a list - and the code originally was only to pull one feed from the first position in a list, and add this object to another list.
This was the original code:
public static List<Feed> getFeedsFromXml(String xml) {
Pattern feedPattern = Pattern.compile("<feed>\\s*<name>\\s*([^<]*)</name>\\s*<uri>\\s*([^<]*)</uri>\\s*</feed>");
Matcher feedMatch = feedPattern.matcher(xml);
while (feedMatch.find()) {
String feedName = feedMatch.group(1);
String feedURI = feedMatch.group(2);
feeds.add(new Feed(feedName, feedURI));
}
return feeds;
}
#POST
#Consumes(MediaType.APPLICATION_XML)
#Produces(MediaType.APPLICATION_XML)
public String addXmlFeed() throws IOException
{
int i = 0;
String stringXml = "<feed><name>SMH Top Headlines</name><uri>http://feeds.smh.com.au/rssheadlines/top.xml</uri></feed><feed><name>UTS Library News</name>";
getFeedsFromXml(stringXml);
Feed f = (Feed) feeds.get(0);
feedList.add(f);
String handler = "You have successfully added: \n";
String xmlStringReply = "" + f + "\n";
feedList.save(feedFile);
return handler + xmlStringReply;
}
Everything was going well, and then I decided to implement a for loop for handling the adding of more than one feed to the list, and I tried the following (only the code for the second method in question):
#POST
#Consumes(MediaType.APPLICATION_XML)
#Produces(MediaType.APPLICATION_XML)
public String addXmlFeed() throws IOException
{
int i = 0;
String stringXml = "<feed><name>SMH Top Headlines</name><uri>http://feeds.smh.com.au/rssheadlines/top.xml</uri></feed><feed><name>UTS Library News</name>";
getFeedsFromXml(stringXml);
for (Feed feed: feeds)
{
Feed f = (Feed) feeds.get(i++);
feedList.add(f);
String handler = "You have successfully added: \n";
String xmlStringReply = "" + f + "\n";
}
feedList.save(feedFile);
return handler + xmlStringReply;
}
Now I'm sure this is a basic problem, but now in the line:
return handler + xmlStringReply;
handler and xmlStringReply cannot be resolved to a variable as they are within the FOR LOOP.
Is there any easy way around this?

The scope of those 2 variables is limited to the for loop. To access them outside the loop, you need to increase their scope by declaring them before the loop:
String handler = "";
String xmlStringReply = "";
for (Feed f: feeds) {
feedList.add(f);
handler = "You have successfully added: \n";
xmlStringReply = "" + f + "\n";
}
feedList.save(feedFile);
return handler + xmlStringReply;
Also, your current code overwrites the value of your strings at each loop, whereas you probably meant to concatenate the values. In that case, you could use a StringBuilder instead of string concatenation:
StringBuilder xmlStringReply = new StringBuilder("You have successfully added: \n");
for (Feed f: feeds) {
feedList.add(f);
xmlStringReply.append(f + "\n");
}
feedList.save(feedFile);
return xmlStringReply.toString();

The question you need to answer is "what do I want to return if I add several feeds ?".
Maybe you'd like to return "You have successfully added : feed1 feed2 feed3\n"
In that case, the code is :
StringBuilder response = new StringBuilder( "You have successfully added: ");
for (Feed feed: feeds)
{
feedList.add(feed);
response.append(f.toString()).append(" ");
}
feedList.save(feedFile);
return response.toString();
By the way, your feedand fvariables are just the same and redondant !
Don't write :
int i = 0;
for (Feed feed: feeds)
{
Feed f = (Feed) feeds.get(i++);
feedList.add(f);
}
but
for (Feed feed: feeds)
{
feedList.add(feed);
}

You need to accumulate the result into a variable. I am using StringBuilder because it makes string concatenation efficient.
#POST
#Consumes(MediaType.APPLICATION_XML)
#Produces(MediaType.APPLICATION_XML)
public String addXmlFeed() throws IOException
{
String stringXml = "<feed><name>SMH Top Headlines</name><uri>http://feeds.smh.com.au/rssheadlines/top.xml</uri></feed><feed><name>UTS Library News</name>";
getFeedsFromXml(stringXml);
StringBuilder replyBuilder = new StringBuilder("You have successfully added: \n");
for (Feed feed : feeds)
{
feedList.add(feed);
String xmlStringReply = feed + "\n";
reployBuilder.append(xmlStringReply);
}
feedList.save(feedFile);
return replyBuilder.toString();
}

Because, now they became out of scope.
Beside the original error -- you can easily fix that using other suggestions, I would like to suggest that you should not make feeds as instance variable. I can see your method getFeedsFromXml() is returning the list. So, I think it would have been better if you define that variable inside that method. And then, call the method like,
List<Feed> feeds = getFeedsFromXml(stringXml);
Or in case, this doesn't give you the desired behaviour, then you should rename the method to something, loadFeedsFromXml(). Making that as instance variable may result in threading issues.
Now, trying to improve on your looping,
StringBuilder xmlStringReply = new StringBuilder("You have successfully added: \n");
for (Feed feed: feeds) {
feedList.add(feed);
xmlStringReply.append(f + "\n");
}
feedList.save(feedFile);
return xmlStringReply.toString();
Moreover, I found that your feedList is also a instance variable. And this again can cause threading issues, as it doesn't sound immutable or stateless. Synchronising the methods will give you performance issues. See if you can make it local to this method. A rule of thumb is to keep variable scope as narrow as possible.

A good rule of thumb is to view scope like this:
{ //This is a constructor
int i;
} // This is a deconstructor
anything that is created / instantiated between the curlies only lives inside the curlies. Whenever your working with variables and loops:
for(int i = 0; i < 10; i++){
//some code here
} // after this curly i is no longer in scope or accessible.

Use JDT to get full method name

I am new to eclipse plugin development and I am trying to convert a IMethod to a string representation of the full method name. I.E.
my.full.package.ClassName.methodName(int param, String string)
so far I have had to hand roll my own solution. Is there a better way?
private static String getMethodFullName(IMethod iMethod)
{
String packageString = "[Default Package]";
try {
IPackageDeclaration[] declarations = iMethod.getCompilationUnit().getPackageDeclarations();
if(declarations.length > 0)
{
packageString = declarations[0].getElementName();
}
} catch (JavaModelException e) {
}
String classString = iMethod.getCompilationUnit().getElementName();
classString = classString.replaceAll(".java", "");
String methodString = iMethod.getElementName() + "(";
for (String type : iMethod.getParameterTypes()) {
methodString += type + ",";
}
methodString += ")";
return packageString + "." + classString + "." + methodString;
}

You can get the Fully qualified name for the type using
method.getDeclaringType().getFullyQualifiedName();
This is probably easier than accessing the package from the compilation unit. The rest of you function looks correct.
One small point: you should use StringBuilder to build up the string instead of adding to a standard String. Strings are immutable so addition creates loads of unrecesary temparary objects.
private static String getMethodFullName(IMethod iMethod)
{
StringBuilder name = new StringBuilder();
name.append(iMethod.getDeclaringType().getFullyQualifiedName());
name.append(".");
name.append(iMethod.getElementName());
name.append("(");
String comma = "";
for (String type : iMethod.getParameterTypes()) {
name.append(comma);
comma = ", ";
name.append(type);
}
name.append(")");
return name.toString();
}

Thanks to iain and some more research I have come up with this solution. It seems like something like this should be built into the JDT....
import org.eclipse.jdt.core.Signature;
private static String getMethodFullName(IMethod iMethod)
{
StringBuilder name = new StringBuilder();
name.append(iMethod.getDeclaringType().getFullyQualifiedName());
name.append(".");
name.append(iMethod.getElementName());
name.append("(");
String comma = "";
String[] parameterTypes = iMethod.getParameterTypes();
try {
String[] parameterNames = iMethod.getParameterNames();
for (int i=0; i<iMethod.getParameterTypes().length; ++i) {
name.append(comma);
name.append(Signature.toString(parameterTypes[i]));
name.append(" ");
name.append(parameterNames[i]);
comma = ", ";
}
} catch (JavaModelException e) {
}
name.append(")");
return name.toString();
}

I am not sure it would take into account all cases (method within an internal class, an anonymous class, with generic parameters...)
When it comes to methods signatures, the classes to look into are:
org.eclipse.jdt.internal.corext.codemanipulation.AddUnimplementedMethodsOperation
org.eclipse.jdt.internal.corext.codemanipulation.StubUtility2
You need to get the jdt.core.dom.IMethodBinding, from which you can extract all what you need.
If you have a MethodInvocation, you can:
//MethodInvocation node
ITypeBinding type = node.getExpression().resolveTypeBinding();
IMethodBinding method=node.resolveMethodBinding();