I have a static file server at "localhost:8888/fileserver".
I am trying to write a program in java to download the files from the server. The file server consists of three folders, therefore I am trying to write a script that automatically goes through the directory and copies it to my computer.
I know there is a wget function for linux that accomplishes this recursively. Is there a way to do this in Java?
Please could you advise on how I should go about doing this or proceed.
Thank you
Below is a code to go through an online directory and return back all the links needed to download.
After that I just need to download each individual link.
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
public class WebCrawler {
//Created a global list variable to save the links
static List<String> createList = new ArrayList<String>();
public static void main(String[] args) throws IOException {
String url = "http://localhost:8888";
System.out.println(myCrawler(url)+"\n"+"Size: "+myCrawler(url).size());
}
public static List<String> myCrawler(String url) throws IOException{
//Creates an open connection to a link
Document doc = Jsoup.connect(url).ignoreContentType(true).get();
Elements links = doc.select("a[href]");
//Recursively iterates through all the links provided on the initial url
for (Element i : links) {
String link = print("%s", i.attr("abs:href"));
if (link.endsWith("/")){myCrawler(link);} //Recursive part, calls back on itself
else {createList.add(link);}
}
return createList;
}
//Translates the link into a readable string object
private static String print(String msg, Object... args){return String.format(msg, args);}
}
Related
I am making a stock market simulator app in java, and there is an issue in the deleteHistoryFiles() method. It says that array is null. However, I have no idea what array this error is talking about.
Here's the code (I've deleted some methods to save space):
package stock.market.simulator;
import java.util.Random;
import java.text.DecimalFormat;
import java.util.Timer;
import java.util.TimerTask;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
public class StockMarketSimulator {
// Path to where the files are stored for rate history
// USE WHEN RUNNING PROJECT IN NETBEANS
//public static final String HISTORYFILEPATH = "src/stock/market/simulator/history/";
// Path to history files to be used when executing program through jar file
public static final String HISTORYFILEPATH = "history/";
public static void main(String[] args) throws IOException {
accountProfile accProfile = accountCreation();
stockProfile[][] stockProfile = createAllStocks();
deleteHistoryFiles(new File(HISTORYFILEPATH));
createHistoryFiles(stockProfile);
mainWindow window = new mainWindow(accProfile, stockProfile);
recalculationLoop(stockProfile, window);
}
// Procedure to create the history files
public static void createHistoryFiles(stockProfile[][] stocks) throws IOException {
String fileName;
FileWriter fileWriter;
for (stockProfile[] stockArray : stocks) {
for (stockProfile stock : stockArray) {
fileName = stock.getProfileName() + ".csv";
fileWriter = new FileWriter(HISTORYFILEPATH + fileName);
}
}
}
// Procedure to delete the history files
public static void deleteHistoryFiles(File directory) {
for (File file : directory.listFiles()) {
if (!file.isDirectory()) {
file.delete();
}
}
}
}
I got the same exception in exactly the same scenario. I tried to create an array of files by calling File.listFiles() and then iterating the array.
Got exception Cannot read the array length because "<local3>" is null.
Problem is that the path to the directory simply does not exist (my code was copied from another machine with a different folder structure).
I don't understand where is <local1> (sometimes it is <local3>) comes from and what does it mean?
It should be just like this: Cannot read the array length because the array is null.
Edit (answering comment) The sole interesting question in this question is what is a <local1>
My answer answers this question: <local1> is just an array created by File.listFiles() method. And an array is null because of the wrong path.
I tried to add a read permission for a given file to a user on windows 10 using the AclEntry.add () method in java, the process runs normally. However, this change does not occur in windows.
I want to know if this method should assign permission permanently or only during the execution of the program.
NOTE: I am using the AclFileAttributeView interface of the Nio.File package.
In addition I packaged this jar using Launch4j to be able to run it as an administrator.
below the code I'm using.
import java.io.IOException;
import java.nio.file.FileSystems;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.attribute.*;
import java.util.List;
public class Inicial {
public Inicial() throws IOException {
Path path = Paths.get("C:\\Windows\\System32\\config\\journal");
UserPrincipal usercop =
FileSystems.getDefault().getUserPrincipalLookupService().lookupPrincipalByName("Gilderlei");
AclFileAttributeView aclFileAttributeView = Files.getFileAttributeView(path,
AclFileAttributeView.class);
List<AclEntry> aclEntries = aclFileAttributeView.getAcl();
AclEntry.Builder aclEntry = AclEntry.newBuilder()
.setType(AclEntryType.ALLOW)
.setPrincipal(usercop)
.setPermissions(AclEntryPermission.READ_DATA);
AclEntry entry = aclEntry.build();
aclEntries.clear();
aclEntries.add(entry);
}
public static void main(String[] args) throws IOException {
new Inicial();
}
}
You just add the created permission to the aclEntries list. Java is not an ORM mapped system that automatically writes back any data changes.
Therefore you have to call at least setAcl
aclFileAttributeView.setAcl(aclEntries);
at the end of your code.
I'm working with NEO4J and Java to create a prototype for an application that integrates a graph database that holds patient information (with fake, made up data).
I've created a simple two class program, and created nodes (haven't assigned relationships yet). However, I want to be able to view the nodes that I've created in order to make sure that my application is working properly, and to be able to see the results in the NEO4J Browser / Community Server.
How can I get the nodes to appear visually? I know I could test the fact that they're being created by querying them, but I also want to know how to visually display them.
What I've tried to do is go into the Neo4j.conf file, and change the active database from the default "graph.db" to "Users/andrew/eclipse-workspace/patients-database/target/patient-db", since in the Java class I've created, I use this line of code to set my database:
private static final File Patient_DB = new File("target/patient-db");
However, whenever I open the NEO4J browser at localhost:7474, after running my code there are no nodes visible.
Below, I'll paste the code to my PatientGraph class (the other class is just a Patient class that creates the Patients and their attributes)
package com.andrewhe.neo4j.Patients_Database;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import org.neo4j.graphdb.Direction;
import org.neo4j.graphdb.GraphDatabaseService;
import org.neo4j.graphdb.Node;
import org.neo4j.graphdb.Path;
import org.neo4j.graphdb.Relationship;
import org.neo4j.graphdb.RelationshipType;
import org.neo4j.graphdb.Transaction;
import org.neo4j.graphdb.factory.GraphDatabaseFactory;
import org.neo4j.graphdb.traversal.Evaluators;
import org.neo4j.graphdb.traversal.TraversalDescription;
import org.neo4j.graphdb.traversal.Traverser;
import org.neo4j.io.fs.FileUtils;
public class PatientGraph {
private static final File Patient_DB = new File("target/patient-db");
private ArrayList<Patient> patients = new ArrayList<Patient>();
private long patientZero;
private GraphDatabaseService graphDb;
public ArrayList<Patient> getPatients() { return patients; }
public void manualPatientSetUp() throws IOException {
Patient homeNode = new Patient("");
Patient jan = new Patient("Jan");
patients.add(jan);
Patient kim = new Patient("Kim");
patients.add(kim);
Patient ahmad = new Patient("Ahmad");
patients.add(ahmad);
Patient andrew = new Patient("Andrew");
patients.add(andrew);
}
public void createPatientNodes() throws IOException {
FileUtils.deleteRecursively(Patient_DB);
graphDb = new GraphDatabaseFactory().newEmbeddedDatabase(Patient_DB);
registerShutdownHook();
try (Transaction tx = graphDb.beginTx()) {
for (Patient patient : patients) {
Node patientNode = graphDb.createNode();
System.out.println("Node created");
setProperties(patientNode, patient);
}
tx.success();
}
}
//Method to create and set properties for node instead of using 5 set properties each time.
public void setProperties(Node node, Patient patient) {
node.setProperty("name", patient.getName());
node.setProperty("weight", patient.getWeight());
node.setProperty("pat_id", new String(patient.getPatientID()));
node.setProperty("age", patient.getAge());
//Don't worry about diagnoses yet;
//To get it to work, just turn the diagnoses ArrayList into a String separated by commas.
}
public void setUp() throws IOException {
//reads in patients using a file
}
public void shutdown()
{
graphDb.shutdown();
}
private void registerShutdownHook()
{
// Registers a shutdown hook for the Neo4j instance so that it
// shuts down nicely when the VM exits (even if you "Ctrl-C" the
// running example before it's completed)
Runtime.getRuntime().addShutdownHook(new Thread(() -> graphDb.shutdown()));
}
public static void main(String[] args) throws IOException {
PatientGraph pg = new PatientGraph();
pg.manualPatientSetUp();
pg.createPatientNodes();
for (int i = 0; i < pg.getPatients().size(); i++) {
pg.getPatients().get(i).printAllData();
}
pg.shutdown();
}
}
You did not provide sufficient information about how you are querying the nodes. You did not even elaborate as to what you are classes do in a brief detail before actually pasting the entire class contents. What is the relationship between these classes? Expecting someone to decode this for you from the code, is asking for too much.
You could use Neo4J Browser (comes built-in) or Neo4J Bloom (commercial tool) to visualize the graph nodes and their interconnections (relations).
To query a Neo4j database you can use Cypher, a pictorial graph query language, that represents patterns as Ascii Art.
A detailed Hands-On procedure for querying and visualizing the Graph Nodes is given in the article below.
https://medium.com/neo4j/hands-on-graph-data-visualization-bd1f055a492d
Trying to make a super simple bit of code using jsoup to see if a webpage contains a specific word (checks to see the availability of a class to take in-residence). jsoup seems to be catching lots of the webpage specified, but won't capture all of it - specifically the area I'm interested in. Is there a reason for this? Am I doing something wrong?
import org.jsoup.Jsoup;
import java.io.IOException;
import org.jsoup.nodes.Document;
public class main {
public static void main(String[] args) throws IOException{
String html = Jsoup.connect("https://www.afit.edu/CE/Course_Desc.cfm?p=WENG%20481").maxBodySize(0).get().html();
System.out.print(html);
if (html.contains("Resident")) {
System.out.print("\nAVAILABLE!");
}
}
}
I cannot figure out what I am doing wrong here. I have tried all sorts of things, including absolute paths, relative, enabling logging (which also doesnt seem to be working, using Main, using DefaultCamelContext, adding threadsleep, but I cannot get camel to move a file from one folder to another.
Here is my code:
package scratchpad;
import org.apache.camel.CamelContext;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.dataformat.beanio.BeanIODataFormat;
import org.apache.camel.impl.DefaultCamelContext;
import org.apache.camel.main.Main;
import org.apache.camel.spi.DataFormat;
public class CamelMain {
private static Main main;
public static void main(String[] args) throws Exception {
main = new Main();
main.addRouteBuilder( new RouteBuilder() {
#Override
public void configure() throws Exception {
// DataFormat format = new BeanIODataFormat(
// "org/apache/camel/dataformat/beanio/mappings.xml",
// "orderFile");
System.out.println("starting route");
// a route which uses the bean io data format to format a CSV data
// to java objects
from("file://input?noop=true&startingDirectoryMustExist=true")
.to("file://output");
}
});
//main.run();
main.start();
Thread.sleep(5000);
main.stop();
}
}
Can someone spot anything wrong with the above?
Thanks
You can for example read from the free chapter 1 for the Camel in Action book, as it has a file copied example it covers from top to bottom.
The pdf can be downloaded here: http://manning.com/ibsen/