currently I'm facing an issue where the URL that are outputted in the console is different from the actual URL display in the nav bar. I look at the value that I had regex, it was correct but when put into IE, it is different. The URL in the nav bar would be only a=test instead of a=test&c=import&f=&uid=user1 which is what is shown in the console. May I know how to solve this? Thanks.
Here is the codes for it.
UserID user = new UserID();
public static void main(String[] args) {
String newUrl = replaceUserID(url);
System.out.print("cmd.exe /c start iexplore -k " + newUrl);
try{
Process p = Runtime.getRuntime().exec("cmd.exe /c start iexplore " + newUrl);
try{
p.waitFor();
}
catch( InterruptedException ie ){
System.out.println("InterruptedException " + ie.getMessage());
}
InputStream err = p.getErrorStream();
int ctr = 0;
if ( (ctr = err.available()) > 0 ){
byte[] buf = new byte[ctr];
System.out.println("Process failed with error:\n" + new String(buf, 0, ctr));
}
}
catch(IOException ioe)
{
System.out.println("InterruptedException " + ioe.getMessage());
}
}
public static String checkUserID(){
String username = System.getenv("USERNAME"); //grab user login id for windows
return username;
}
public static String replaceUserID(String oldUrl) {
String params = "http://example.com?a=test&c=import&f=&uid=userid";
String username = checkUserID();
try {
Pattern p = Pattern.compile("uid=([^&%]+)");
Matcher m = p.matcher(params);
while (m.find()) {
// System.out.println(m.group(1).toString());
if(m.group(1).toString() != username) {
//replace url paremeters with correct userID.
String newUrl = params.replace(m.group(1).toString(), username);
return newUrl;
}
else {
System.out.println("The username is the same");
return params;
}
}
} catch (PatternSyntaxException ex) {
// error handling
ex.printStackTrace();
}
return oldUrl;
}
& has a special meaning to the command line (concat commands, see here: How do I run two commands in one line in Windows CMD?).
You need to escape the url or maybe the entire command. If I remember correctly you need to put double quotes around it: "...iexplore -k \"" + newUrl + "\""
I have two models in my program, Bus and Learner.
Each is stored in a txt file, name Busses.txt and Learners.txt, respectively.
I am experiencing an issue where the method to delete a learner entry works, but the method to delete a bus entry does not, even though the code is practically identical.
Learner delete method:
public void deleteLearner(String ID) {
removeBlankLines("Learners.txt");
File oldFile = new File("Learners.txt");
File tempFile = new File("tempFile.txt");
String removeKey = ID;
String LearnerID;
String nameSurname;
boolean status;
String busOfLearner;
String line;
String lineToKeep;
try {
Scanner scFile = new Scanner(new File("Learners.txt"));
while (scFile.hasNext()) {
line = scFile.nextLine();
Scanner scLine = new Scanner(line).useDelimiter("#");
LearnerID = scLine.next();
nameSurname = scLine.next();
status = scLine.nextBoolean();
if (scLine.hasNext()) {
busOfLearner = scLine.next();
} else {
busOfLearner = "";
}
if (!LearnerID.equalsIgnoreCase(removeKey)) {
lineToKeep = LearnerID + "#" + nameSurname + "#" + status + "#" + busOfLearner + "\n";
FileWriter fWriter = new FileWriter(tempFile,true);
BufferedWriter bWriter = new BufferedWriter(fWriter);
bWriter.write(lineToKeep);
bWriter.close();
fWriter.close();
}
scLine.close();
}
scFile.close();
boolean successfulDelete = oldFile.delete();
File transfer = new File("Learners.txt");
boolean successfulRename = tempFile.renameTo(transfer);
}
catch (Exception e) {
System.out.println("An error has occured deleting a learner record " + e);
}
}
Delete bus method:
public void deleteBus(String removeBusName) {
removeBlankLinesBus("Busses.txt");
File oldFile = new File("Busses.txt");
File newFile = new File("NewBusFile.txt");
String deleteKey = removeBusName;
String currentBusName;
int currentNumSeats;
String currentPickLocation;
String currentDropLocation;
String currentPickTime;
String currentDropTime;
String line;
String lineToKeep;
try {
Scanner scFile = new Scanner(new File("Busses.txt"));
while (scFile.hasNext()) {
line = scFile.nextLine();
Scanner scLine = new Scanner(line).useDelimiter("#");
currentBusName = scLine.next();
currentNumSeats = scLine.nextInt();
currentPickLocation = scLine.next();
currentDropLocation = scLine.next();
currentPickTime = scLine.next();
currentDropTime = scLine.next();
if (!currentBusName.equalsIgnoreCase(deleteKey)) {
lineToKeep = currentBusName + "#" + currentNumSeats + "#" + currentPickLocation + "#" + currentDropLocation + "#" + currentPickTime + "#" + currentDropTime + "\n";
FileWriter fWriter = new FileWriter(newFile,true);
BufferedWriter bWriter = new BufferedWriter(fWriter);
bWriter.write(lineToKeep);
bWriter.close();
fWriter.close();
}
scLine.close();
}
scFile.close();
boolean successfulDelete = oldFile.delete();
File transfer = new File("Busses.txt");
boolean successfulRename = newFile.renameTo(transfer);
}
catch (Exception e) {
System.out.println("An error has occured deleting " + removeBusName + " from the file: " + e);
}
}
Problem:
With the delete bus method, the old file doesn't get deleted and the temporary or new file doesn't get renamed to the original file.
I am very confident that all files, streams, scanners, etc. are closed, as it is exactly the same as I did in the delete learner method, which does work and the files are deleted and renamed in the learner delete method as it should.
Assistance would be greatly appreciated.
EDIT: Implementation of methods:
Learner:
System.out.println(myController.PrintLearnerArr(myController.LoadLearner("Learners.txt")));
String delete = "0210045112055";
myController.deleteLearner(delete);
System.out.println(myController.PrintLearnerArr(myController.LoadLearner("Learners.txt")));
Bus:
System.out.println(myController.PrintBusArr(myController.LoadBus("Busses.txt")));
String deleteKey = "deleteme";
myController.deleteBus(deleteKey);
System.out.println(myController.PrintBusArr(myController.LoadBus("Busses.txt")));
I am facing a problem in the following code. I am trying to run the program and it terminates when it hits empty space in my input. How else I should approach this.
try {
BufferedReader sc = new BufferedReader(new FileReader(text.txt);
ArrayList<String> name = new ArrayList<>();
ArrayList<String> id = new ArrayList<>();
ArrayList<String> place = new ArrayList<>();
ArrayList<String> details = new ArrayList<>();
String line = null;
while ((line = sc.readLine()) !=null) {
if (!line.trim().equals("")) {
System.out.println(line);
if (line.toLowerCase().contains("name")) {
name.add(line.split("=")[1].trim());
}
if (line.toLowerCase().contains("id")) {
id.add(line.split("=")[1].trim());
}
if (line.toLowerCase().contains("location")) {
place.add(line.split("=")[1].trim());
}
if (line.toLowerCase().contains("details")) {
details.add(line.split("=")[1].trim());
}
}
}
PrintWriter pr = new PrintWriter(new File(text.csv));
pr.println("Name;Id;;Location;Details");
for (int i = 0; i < name.size(); i++) {
pr.println(name.get(i) + ";" + id.get(i) + ";" + place.get(i) + ";" + details.get(i));
}
pr.close();
sc.close();
} catch (Exception e) {
e.printStackTrace();
} }
My Input looks like
name = abc
id = 123
place = xyz
details = hsdyhuslkjaldhaadj
name = ert
id = 7872
place =
details = shahkjdhksdhsala
name = sfd
id = 4343
place = ksjks
Details = kljhaljs
when im trying to execute then above text my program terminates at place = "null" because of no value there.I need the output as an empty space created in place ="null" and print the rest as follows in a .csv file
If you process the location, line.split("=")[1] could result in an ArrayIndexOutOfBoundException and line.split("=")[1].trim() could result in a NullPointerException.
You can avoid this by testing your parsed result.
Instead of place.add(line.split("=")[1].trim());, do place.add(parseContentDefaultEmpty(line));, with:
private String parseContentDefaultEmpty(final String line) {
final String[] result = line.split("=");
if(result.length <= 1) {
return "";
}
final String content = line.split("=")[1];
return content != null ? content.trim() : "";
}
First there is a issue,your input file contains key as "place" but your are trying for word "location"
if (line.toLowerCase().contains("location")) { //this must be changed to place
place.add(line.split("=")[1].trim());
}
Modified the code snippet as below.check it
while ((line = sc.readLine()) != null) {
if (!line.trim().equals("")) {
System.out.println(line);
if (line.toLowerCase().contains("name")) {
name.add(line.split("=")[1].trim());
}
if (line.toLowerCase().contains("id")) {
id.add(line.split("=")[1].trim());
}
if (line.toLowerCase().contains("place")) {
// change done here to add space if no value
place.add(line.split("=").length > 1 ? line.split("=")[1]
.trim() : " ");
}
if (line.toLowerCase().contains("details")) {
details.add(line.split("=")[1].trim());
}
}
}
Setting question to line doesn't appear to change what line is read later (if you're wanting the line to advance before it hits the while loop).
I want to read specific columns from a multi column csv file and print those columns in other csv file using Java. Any help please? Following is my code to print each token line by line..But I am looking to print only few columns out of the multi column csv.
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.FileWriter;
import java.util.StringTokenizer;
public class ParseCSV {
public static void main(String[] args) {
try
{
//csv file containing data
String strFile = "C:\\Users\\rsaluja\\CMS_Evaluation\\Drupal_12_08_27.csv";
//create BufferedReader to read csv file
BufferedReader br = new BufferedReader( new FileReader(strFile));
String strLine = "";
StringTokenizer st = null;
int lineNumber = 0, tokenNumber = 0;
//read comma separated file line by line
while( (strLine = br.readLine()) != null)
{
lineNumber++;
//break comma separated line using ","
st = new StringTokenizer(strLine, ",");
while(st.hasMoreTokens())
{
//display csv values
tokenNumber++;
System.out.println("Line # " + lineNumber +
", Token # " + tokenNumber
+ ", Token : "+ st.nextToken());
System.out.println(cols[4]);
You should use the excellent OpenCSV for reading and writing CSV files. To adapt your example to use the library it would look like this:
public class ParseCSV {
public static void main(String[] args) {
try {
//csv file containing data
String strFile = "C:/Users/rsaluja/CMS_Evaluation/Drupal_12_08_27.csv";
CSVReader reader = new CSVReader(new FileReader(strFile));
String [] nextLine;
int lineNumber = 0;
while ((nextLine = reader.readNext()) != null) {
lineNumber++;
System.out.println("Line # " + lineNumber);
// nextLine[] is an array of values from the line
System.out.println(nextLine[4] + "etc...");
}
}
}
}
Reading a CSV file in very simple and common in Java. You actually don't require to load any extra third party library to do this for you. CSV (comma separated value) file is just a normal plain-text file, store data in column by column, and split it by a separator (e.g comma ",").
In order to read specific columns from the CSV file, there are several ways. Simplest of all is as below:
Code to read CSV without any 3rd party library
BufferedReader br = new BufferedReader(new FileReader(csvFile));
while ((line = br.readLine()) != null) {
// use comma as separator
String[] cols = line.split(cvsSplitBy);
System.out.println("Coulmn 4= " + cols[4] + " , Column 5=" + cols[5]);
}
If you notice, nothing special is performed here. It is just reading a text file, and spitting it by a separator – ",".
Consider an extract from legacy country CSV data at GeoLite Free Downloadable Databases
"1.0.0.0","1.0.0.255","16777216","16777471","AU","Australia"
"1.0.1.0","1.0.3.255","16777472","16778239","CN","China"
"1.0.4.0","1.0.7.255","16778240","16779263","AU","Australia"
"1.0.8.0","1.0.15.255","16779264","16781311","CN","China"
"1.0.16.0","1.0.31.255","16781312","16785407","JP","Japan"
"1.0.32.0","1.0.63.255","16785408","16793599","CN","China"
"1.0.64.0","1.0.127.255","16793600","16809983","JP","Japan"
"1.0.128.0","1.0.255.255","16809984","16842751","TH","Thailand"
Above code will output as below:
Column 4= "AU" , Column 5="Australia"
Column 4= "CN" , Column 5="China"
Column 4= "AU" , Column 5="Australia"
Column 4= "CN" , Column 5="China"
Column 4= "JP" , Column 5="Japan"
Column 4= "CN" , Column 5="China"
Column 4= "JP" , Column 5="Japan"
Column 4= "TH" , Column 5="Thailand"
You can, in fact, put the columns in a Map and then get the values simply by using the key.
Shishir
I am sorry, but none of these answers provide an optimal solution. If you use a library such as OpenCSV you will have to write a lot of code to handle special cases to extract information from specific columns.
For example, if you have rows with less columns than what you're after, you'll have to write a lot of code to handle it. Using the OpenCSV example:
CSVReader reader = new CSVReader(new FileReader(strFile));
String [] nextLine;
while ((nextLine = reader.readNext()) != null) {
//let's say you are interested in getting columns 20, 30, and 40
String[] outputRow = new String[3];
if(parsedRow.length < 40){
outputRow[2] = null;
} else {
outputRow[2] = parsedRow[40]
}
if(parsedRow.length < 30){
outputRow[1] = null;
} else {
outputRow[1] = parsedRow[30]
}
if(parsedRow.length < 20){
outputRow[0] = null;
} else {
outputRow[0] = parsedRow[20]
}
}
This is a lot of code for a simple requirement. It gets worse if you are trying to get values of columns by name. You should use a more modern parser such as the one provided by uniVocity-parsers.
To reliably and easily get the columns you want, simply write:
CsvParserSettings settings = new CsvParserSettings();
parserSettings.selectIndexes(20, 30, 40);
CsvParser parser = new CsvParser(settings);
List<String[]> allRows = parser.parseAll(new FileReader(yourFile));
Disclosure: I am the author of this library. It's open-source and free (Apache V2.0 license).
To read some specific column
I did something like this:
dpkcs.csv content:
FN,LN,EMAIL,CC
Name1,Lname1,email1#gmail.com,CC1
Nmae2,Lname2,email2r#gmail.com,CC2
The function to read it:
private void getEMailRecepientList() {
List<EmailRecepientData> emailList = null;// Blank list of POJO class
Scanner scanner = null;
BufferedReader reader = null;
try {
reader = new BufferedReader(new FileReader("dpkcs.csv"));
Map<String, Integer> mailHeader = new HashMap<String, Integer>();
// read file line by line
String line = null;
int index = 0;
line = reader.readLine();
// Get header from 1st row of csv
if (line != null) {
StringTokenizer str = new StringTokenizer(line, ",");
int headerCount = str.countTokens();
for (int i = 0; i < headerCount; i++) {
String headerKey = str.nextToken();
mailHeader.put(headerKey.toUpperCase(), new Integer(i));
}
}
emailList = new ArrayList<EmailRecepientData>();
while ((line = reader.readLine()) != null) {
// POJO class for getter and setters
EmailRecepientData email = new EmailRecepientData();
scanner = new Scanner(line);
scanner.useDelimiter(",");
//Use Specific key to get value what u want
while (scanner.hasNext()) {
String data = scanner.next();
if (index == mailHeader.get("EMAIL"))
email.setEmailId(data);
else if (index == mailHeader.get("FN"))
email.setFirstName(data);
else if (index == mailHeader.get("LN"))
email.setLastName(data);
else if (index == mailHeader.get("CC"))
email.setCouponCode(data);
index++;
}
index = 0;
emailList.add(email);
}
reader.close();
} catch (Exception e) {
StringWriter stack = new StringWriter();
e.printStackTrace(new PrintWriter(stack));
} finally {
scanner.close();
}
System.out.println("list--" + emailList);
}
The POJO Class:
public class EmailRecepientData {
private String emailId;
private String firstName;
private String lastName;
private String couponCode;
public String getEmailId() {
return emailId;
}
public void setEmailId(String emailId) {
this.emailId = emailId;
}
public String getFirstName() {
return firstName;
}
public void setFirstName(String firstName) {
this.firstName = firstName;
}
public String getLastName() {
return lastName;
}
public void setLastName(String lastName) {
this.lastName = lastName;
}
public String getCouponCode() {
return couponCode;
}
public void setCouponCode(String couponCode) {
this.couponCode = couponCode;
}
#Override
public String toString() {
return "Email Id=" + emailId + ", First Name=" + firstName + " ,"
+ " Last Name=" + lastName + ", Coupon Code=" + couponCode + "";
}
}
I sugges to use the Apache Commons CSV https://commons.apache.org/proper/commons-csv/
Here is one example:
Path currentRelativePath = Paths.get("");
String currentPath = currentRelativePath.toAbsolutePath().toString();
String csvFile = currentPath + "/pathInYourProject/test.csv";
Reader in;
Iterable<CSVRecord> records = null;
try
{
in = new FileReader(csvFile);
records = CSVFormat.EXCEL.withHeader().parse(in); // header will be ignored
}
catch (IOException e)
{
e.printStackTrace();
}
for (CSVRecord record : records) {
String line = "";
for ( int i=0; i < record.size(); i++)
{
if ( line == "" )
line = line.concat(record.get(i));
else
line = line.concat("," + record.get(i));
}
System.out.println("read line: " + line);
}
It automaticly recognize , and " but not ; (maybe it can be configured...).
My example file is:
col1,col2,col3
val1,"val2",val3
"val4",val5
val6;val7;"val8"
And output is:
read line: val1,val2,val3
read line: val4,val5
read line: val6;val7;"val8"
Last line is considered like one value.
We can use the core java stuff alone to read the CVS file column by column. Here is the sample code I have wrote for my requirement. I believe that it will help for some one.
BufferedReader br = new BufferedReader(new FileReader(csvFile));
String line = EMPTY;
int lineNumber = 0;
int productURIIndex = -1;
int marketURIIndex = -1;
int ingredientURIIndex = -1;
int companyURIIndex = -1;
// read comma separated file line by line
while ((line = br.readLine()) != null) {
lineNumber++;
// use comma as line separator
String[] splitStr = line.split(COMMA);
int splittedStringLen = splitStr.length;
// get the product title and uri column index by reading csv header
// line
if (lineNumber == 1) {
for (int i = 0; i < splittedStringLen; i++) {
if (splitStr[i].equals(PRODUCTURI_TITLE)) {
productURIIndex = i;
System.out.println("product_uri index:" + productURIIndex);
}
if (splitStr[i].equals(MARKETURI_TITLE)) {
marketURIIndex = i;
System.out.println("marketURIIndex:" + marketURIIndex);
}
if (splitStr[i].equals(COMPANYURI_TITLE)) {
companyURIIndex = i;
System.out.println("companyURIIndex:" + companyURIIndex);
}
if (splitStr[i].equals(INGREDIENTURI_TITLE)) {
ingredientURIIndex = i;
System.out.println("ingredientURIIndex:" + ingredientURIIndex);
}
}
} else {
if (splitStr != null) {
String conditionString = EMPTY;
// avoiding arrayindexoutboundexception when the line
// contains only ,,,,,,,,,,,,,
for (String s : splitStr) {
conditionString = s;
}
if (!conditionString.equals(EMPTY)) {
if (productURIIndex != -1) {
productCVSUriList.add(splitStr[productURIIndex]);
}
if (companyURIIndex != -1) {
companyCVSUriList.add(splitStr[companyURIIndex]);
}
if (marketURIIndex != -1) {
marketCVSUriList.add(splitStr[marketURIIndex]);
}
if (ingredientURIIndex != -1) {
ingredientCVSUriList.add(splitStr[ingredientURIIndex]);
}
}
}
}
Finds all files in folder and write that data to ArrayList row.
Initialize
ArrayList<ArrayList<String>> row=new ArrayList<ArrayList<String>>();
BufferedReader br=null;
For Accessing row
for(ArrayList<String> data:row){
data.get(col no);
}
or row.get(0).get(0) // getting first row first col
Functions that reads all files from folders and concatenate them row.
static void readData(){
String path="C:\\Users\\Galaxy Computers\\Desktop\\Java project\\Nasdaq\\";
File files=new File(path);
String[] list=files.list();
try {
String sCurrentLine;
char check;
for(String filename:list){
br = new BufferedReader(new FileReader(path+filename));
br.readLine();//If file contains uneccessary first line.
while ((sCurrentLine = br.readLine()) != null) {
row.add(splitLine(sCurrentLine));
}
}
}
} catch (IOException e) {
e.printStackTrace();
}
try {
if (br != null)br.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
static ArrayList<String> splitLine(String line){
String[] ar=line.split(",");
ArrayList<String> d=new ArrayList<String>();
for(String data:ar){
d.add(data);
}
return d;
}
Well, how about this !!
This code calculates both row and column count in a csv file. Try this out !!
static int[] getRowsColsNo() {
Scanner scanIn = null;
int rows = 0;
int cols = 0;
String InputLine = "";
try {
scanIn = new Scanner(new BufferedReader(
new FileReader("filename.csv")));
scanIn.useDelimiter(",");
while (scanIn.hasNextLine()) {
InputLine = scanIn.nextLine();
String[] InArray = InputLine.split(",");
rows++;
cols = InArray.length;
}
} catch (Exception e) {
System.out.println(e);
}
return new int[] { rows, cols };
}
I am wondering how to find the maven repository close to my home?
Actually, the listing at http://docs.codehaus.org/display/MAVENUSER/Mirrors+Repositories is problematic. Is Central in San Francisco, California or in St. Louis, Missouri? (I guess this is an inadvertent lesson on why code, or even xml, should not be commented). To make things more confusing, the owner of the domain (according to http://whois.net/whois/maven.org) is in Fulton, MD.
Maybe the question was not "Where are the repositories?" (which esaj answered), but really:
"How can I find the maven repository closest to my home?"
That question bugged me, too.
On the other hand, does it really matter, given how fast packets travel across the world?
But it still bugged me.
There are two ways to find out which host is the closest:
Open a browser, go to http://freegeoip.net, and type in the hostname. If you know where you are, then you can go to GoogleMaps, get directions, and check the distance.
Open a terminal window, and ping the hostname to find the average round trip transit time in milliseconds. This will give you the electronic distance, which is really more important for file transfer times.
Repeat for each hostname (i.e. about 40 times, if you're looking at the download sites for Maven itself)
Sort by the distance measured by the GoogleMaps directions, or by the transit time.
For 40 hosts you could probably do that manually in 20 tedious minutes, but a true professional would rather spend a few hours writing the Selenium and java code that could do it in 5 minutes.
In my copious amounts of spare time, I threw together only half the job (i.e. no Selenium):
public class ClosestUrlFinder {
/**
* Find out where a hostname is.
* Adapted from http://www.mkyong.com/java/how-to-send-http-request-getpost-in-java/
* Send HTTP GET requests to: freegeoip.net/{format}/{ip_or_hostname}
* The API supports both HTTP and HTTPS.
* Supported formats are csv, xml or json.
* E.G. http://freegeoip.net/xml/www.poolsaboveground.com
* returns <Response><Ip>207.58.184.93</Ip><CountryCode>US</CountryCode>
* <CountryName>United States</CountryName><RegionCode>VA</RegionCode><RegionName>Virginia</RegionName>
* <City>Mclean</City><ZipCode>22101</ZipCode>
* <Latitude>38.9358</Latitude><Longitude>-77.1621</Longitude>
* <MetroCode>511</MetroCode><AreaCode>703</AreaCode></Response>
* #param urlOrHostname
* #return
*/
public String getUrlLocation(String urlOrHostname) {
String USER_AGENT = "Mozilla/5.0";
BufferedReader in = null;
URL urlObject;
StringBuilder response = null;
try {
urlObject = new URL("http://freegeoip.net/xml/" + urlOrHostname);
HttpURLConnection connection = (HttpURLConnection) urlObject.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("User-Agent", USER_AGENT);
int responseCode = connection.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + urlOrHostname);
System.out.println("Response Code : " + responseCode);
in = new BufferedReader( new InputStreamReader(connection.getInputStream()));
String inputLine;
response = new StringBuilder();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (in != null) {
try { in.close(); }
catch (IOException e) {
e.printStackTrace();
}
}
}
//System.out.println(response.toString());
return response.toString();
}
/*
* Fixed version of code from http://stackoverflow.com/questions/11506321/java-code-to-ping-an-ip-address
*/
public String runDosCommand(List<String> command)
{
String string = "", result = "";
ProcessBuilder pBuilder = new ProcessBuilder(command);
Process process;
try {
process = pBuilder.start();
BufferedReader stdInput = new BufferedReader(new InputStreamReader(process.getInputStream()));
BufferedReader stdError = new BufferedReader(new InputStreamReader(process.getErrorStream()));
//System.out.println("Standard output of the command:");
while ((string = stdInput.readLine()) != null)
{
System.out.println(string);
result = result + "\n" + string;
}
while ((string = stdError.readLine()) != null)
{
System.out.println("stdError: "+ string);
result = result + "\nstdError: " + string;
}
} catch (IOException e) {
System.out.println("Error with command "+command);
e.printStackTrace();
}
return result;
}
public String pingUrl(String url) {
List<String> command = new ArrayList<String>();
String averagePingTime = "0", geoText = "", loss = "";
command.add("ping");
command.add("-n");
command.add("2");
url = url.replace("ftp://", "");
url = Utils.first(url.replace("/", " "));
url = Utils.first(url.replace(":", " "));
command.add(url);
System.out.println("command is: "+command);
String pingResult = runDosCommand(command);
String timeoutString = "Request timed out";
String timeout = Utils.grep(timeoutString, pingResult);
String noHostString = "Ping request could not find host";
String noHost = Utils.grep(noHostString, pingResult);
String unreachableString = "Destination net unreachable";
String unreachable = Utils.grep(unreachableString, pingResult);
if (Utils.isNullOrEmptyString(timeout) && Utils.isNullOrEmptyString(noHost)
&& Utils.isNullOrEmptyString(unreachable)) {
String lostString = "Lost =";
loss = Utils.grep(lostString, pingResult);
int index = loss.indexOf(lostString);
loss = loss.substring(index);
if (!loss.equals("Lost = 0 (0% loss),")) {
System.out.println("Non-zero loss for " + url);
averagePingTime = "0";
} else {
String replyString = "Reply from";
String replyFrom = Utils.grep(replyString, pingResult);
String ipAddress = Utils.nth(replyFrom, 3);
System.out.println("reply Ip="+ipAddress.replace(":", ""));
String averageString = "Average =";
averagePingTime = Utils.grep(averageString, pingResult);
index = averagePingTime.indexOf(averageString);
averagePingTime = averagePingTime.substring(index);
averagePingTime = Utils.last(averagePingTime).replace("ms", "");
String xml = getUrlLocation(url);
//System.out.println("xml="+xml);
geoText = extractTextFromXML(xml);
System.out.println("geoText="+geoText);
}
} else {
averagePingTime = "0";
System.out.println("Error. Either could not find host, Request timed out, or unreachable network for " + url);;
}
System.out.println("Results for " + url + " are: " + loss + " " + averagePingTime + " miliseconds.");
return url + " " + averagePingTime + " " + geoText ;
}
public ArrayList<Entry<String,Integer>> pingManyUrls() {
ArrayList<Entry<String,Integer>> mirrorTimeList = new ArrayList<>();
String[] urls = Utils.readTextFile("resources/MavenUrls.txt").split("\\s+");
System.out.println("Start pingManyUrls with "+urls.length + " urls.");
for (String url : urls) {
url = url.trim();
System.out.println("************************************\npingManyUrls: Check Url " + url);
if (arrayListContainsKey(url, mirrorTimeList)) {
System.out.println("Key " + url + " already in array.");
} else {
String pingInfo = pingUrl(url);
String averagePingString = Utils.nth(pingInfo, 2);
if (!averagePingString.equals("0")) {
int averagePingMilliseconds = Integer.parseInt(averagePingString);
pingInfo = Utils.rest(pingInfo); //chop the first term (the url)
pingInfo = Utils.rest(pingInfo); // chop the 2nd term (the time)
url = url + " " + pingInfo;
System.out.println(" Adding Key " + url + " " + averagePingMilliseconds + " to array.");
Entry<String,Integer> urlTimePair = new java.util.AbstractMap.SimpleEntry<>(url, averagePingMilliseconds);
mirrorTimeList.add(urlTimePair);
} else { System.out.println("Url " + url + " has a problem and therefore will be not be included."); }
}
}
Collections.sort(mirrorTimeList, new Comparator<Entry<String,Integer>>() {
#Override
public int compare (Entry<String,Integer> pair1, Entry<String,Integer> pair2) {
return pair1.getValue().compareTo(pair2.getValue());
}
});
return mirrorTimeList;
}
public boolean arrayListContainsKey(String key, ArrayList<Entry<String,Integer>> arrayList) {
for (Entry<String,Integer> keyValuePair : arrayList) {
//System.out.println(keyValuePair.getKey() + " =? " + key);
if (key.equalsIgnoreCase(keyValuePair.getKey())) { return true; }
}
return false;
}
public String extractFirstTagContents(Element parentElement, String tagname) {
NodeList list = parentElement.getElementsByTagName(tagname);
if (list != null && list.getLength()>0) {
String contents = list.item(0).getTextContent();
System.out.println("Tagname=" + tagname + " contents="+contents);
return contents;
}
return "";
}
public String extractTextFromXML(String xml) {
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder;
String content = "";
try {
builder = factory.newDocumentBuilder();
Document document = builder.parse(new InputSource(new StringReader(xml)));
Element rootElement = document.getDocumentElement();
content = rootElement.getTextContent();
//System.out.println("content="+content);
// String city = extractFirstTagContents(rootElement, "City");
// String state = extractFirstTagContents(rootElement, "RegionName");
// String zipCode = extractFirstTagContents(rootElement, "ZipCode");
// String country = extractFirstTagContents(rootElement, "CountryName");
// String latitude = extractFirstTagContents(rootElement, "Latitude");
// String longitude = extractFirstTagContents(rootElement, "Longitude");
} catch (ParserConfigurationException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (SAXException saxex) {
// TODO Auto-generated catch block
saxex.printStackTrace();
} catch (IOException ioex) {
// TODO Auto-generated catch block
ioex.printStackTrace();
}
return content;
}
public static void main(String[] args) {
System.out.println("Starting ClosestUrlFinder");
ClosestUrlFinder ping = new ClosestUrlFinder();
ArrayList<Entry<String,Integer>> mirrorList = ping.pingManyUrls();
System.out.println("Final Results, sorted by trip travel time (out of "+mirrorList.size()+" successful pings).");
for (Entry<String,Integer> urlTime : mirrorList) {
System.out.println(urlTime.getKey() + " " + urlTime.getValue());
}
}
} // End of Class
Where the data in MavenUrls.txt is:
apache.claz.org
apache.cs.utah.edu
apache.mesi.com.ar
apache.mirrors.hoobly.com
apache.mirrors.lucidnetworks.net
apache.mirrors.pair.com
apache.mirrors.tds.net
apache.osuosl.org
apache.petsads.us
apache.spinellicreations.com
apache.tradebit.com
download.nextag.com
ftp.osuosl.org
ftp://mirror.reverse.net/pub/apache/
mirror.cc.columbia.edu/pub/software/apache/
mirror.cogentco.com/pub/apache/
mirror.metrocast.net/apache/
mirror.nexcess.net/apache/
mirror.olnevhost.net/pub/apache/
mirror.reverse.net/pub/apache/
mirror.sdunix.com/apache/
mirror.symnds.com/software/Apache/
mirror.tcpdiag.net/apache/
mirrors.gigenet.com/apache/
mirrors.ibiblio.org/apache/
mirrors.sonic.net/apache/
psg.mtu.edu/pub/apache/
supergsego.com/apache/
www.bizdirusa.com
www.bizdirusa.com/mirrors/apache/
www.carfab.com/apachesoftware/
www.dsgnwrld.com/am/
www.eng.lsu.edu/mirrors/apache/
www.gtlib.gatech.edu/pub/apache/
www.interior-dsgn.com/apache/
www.motorlogy.com/apache/
www.picotopia.org/apache/
www.poolsaboveground.com/apache/
www.trieuvan.com/apache/
www.webhostingjams.com/mirror/apache/
And my Utils class includes the following:
/**
* Given a string of words separated by spaces, returns the first word.
* #param string
* #return
*/
public static String first(String string) {
if (isNullOrEmptyString(string)) { return ""; }
string = string.trim();
int index = string.indexOf(" "); //TODO: shouldn't this be "\\s+" to handle double spaces and tabs? That means regexp version of indexOf
if (index<0) { return string; }
return string.substring(0, index);
}
/**
* Given a string of words separated by spaces, returns the rest of the string after the first word.
* #param string
* #return
*/
public static String rest(String string) {
if (isNullOrEmptyString(string)) { return ""; }
string = string.trim();
int index = string.indexOf(" "); //TODO: shouldn't this be "\\s+" to handle double spaces and tabs? That means regexp version of indexOf
if (index<0) { return ""; }
return string.substring(index+1);
}
public static String grep(String regexp, String multiLineStringToSearch) {
String result = "";
//System.out.println("grep for '"+ regexp + "'");
String[] lines = multiLineStringToSearch.split("\\n");
//System.out.println("grep input string contains "+ lines.length + " lines.");
Pattern pattern = Pattern.compile(regexp);
for (String line : lines) {
Matcher matcher = pattern.matcher(line);
if (matcher.find()) {
//System.out.println("grep found match="+ line);
result = result + "\n" + line;
}
}
return result.trim();
}
/**
* Given the path and name of a plain text (ascii) file,
* reads it and returns the contents as a string.
* #param filePath
* #return
*/
public static String readTextFile(String filePath) {
BufferedReader reader;
String result = "";
int counter = 0;
try {
reader = new BufferedReader(new FileReader(filePath));
String line = "";
StringBuilder stringBuilder = new StringBuilder();
String lineSeparator = System.getProperty("line.separator");
while ((line = reader.readLine()) != null) {
counter = counter + 1;
stringBuilder.append(line);
stringBuilder.append(lineSeparator);
}
reader.close();
result = stringBuilder.toString();
logger.info("readTextFile: Read "+filePath+" with "+ counter + " lines, "+result.length()+" characters.");
return result;
} catch (FileNotFoundException e) {
logger.fatal("readTextFile: Could not find file "+filePath+".");
e.printStackTrace();
} catch (IOException e) {
logger.fatal("readTextFile: Could not read file "+filePath+".");
e.printStackTrace();
}
return "";
}
public static boolean isNullOrEmptyString(String s) {
return (s==null || s.equals(""));
}
/**
* Given a string of words separated by one or more whitespaces, returns the Nth word.
* #param string
* #return
*/
public static String nth(String string, int n) {
string = string.trim();
String[] splitstring = string.split("\\s+");
String result = "";
if (splitstring.length<n) { return ""; }
else {
result = splitstring[n - 1];
}
return result.trim();
}
/**
* Given a string of words separated by spaces, returns the last word.
* #param string
* #return
*/
public static String last(String string) {
return string.substring(string.trim().lastIndexOf(" ")+1, string.trim().length());
}
Enjoy!
Here's there official central repository mirror listing (as xml-file) and here's another list of central repository mirrors.