I wrote a program that will fetch text data from http://worldtimeapi.org/api/ip.txt, and extract X, where X is the value next to "unixtime". This is what I got so far.
public class GetDataService implements DataService{
#Override
public ArrayList<String> getData() {
ArrayList<String> lines = new ArrayList<>();
try {
URL url = new URL("http://worldtimeapi.org/api/ip.txt");
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(url.openStream()));
String line;
while ((line = bufferedReader.readLine()) != null) {
String a = line;
lines.add(a);
}
bufferedReader.close();
} catch (IOException ex) {
throw new RuntimeException("Can not making the request to the URL.");
}
return lines;
}
public interface DataService {
ArrayList<String> getData() throws IOException;
}
public class UnixTimeExtractor {
private GetDataService getDataService;
public String unixTimeExtractor() {
ArrayList<String> lines = getDataService.getData();
//how to extract the value next to "unixtime"
I don't know how to extract value next to "unixtime". And how can I test NetWork Error for GetDataService Class.
I don't know how to extract value next to "unixtime".
To extract values from a list, you can iterate over the list,
do some check on each value as needed,
and return the value when you found a match, for example:
for (String line : lines) {
if (line.startsWith("unixtime: ")) {
return line;
}
}
To extract the value that comes after "unixtime: " in a string, you could use several strategies:
line.substring("unixtime: ".length())
line.replaceAll("^unixtime: ", "")
line.split(": ")[1]
...
Btw do you really need the list of lines?
If not, then you can save memory and reduce input processing if you perform this check while reading the input stream from the URL,
and stop reading immediately after you found just what you needed.
And how can I test NetWork Error for GetDataService Class.
To test that network errors are handled correctly,
you would need to make the parts of code that can throw network errors injectable.
Then in your test cases you could inject replacement code that will throw exception,
and verify that the program handles the exceptions correctly.
One technique is "extract and extend".
That is, extract the url.openStream() call to a dedicated method:
InputStream getInputStream(URL url) throws IOException {
return url.openStream();
}
And replace in your code url.openStream() with a call to getInputStream(url).
Then in your test method, you can override this method with throwing an exception,
and verify what happens. Using fluent assertions with AssertJ:
#Test
public void test_unixtime() {
UnixTimeExtractor extractor = new UnixTimeExtractor() {
#Override
InputStream getInputStream(URL url) throws IOException {
throw new IOException();
}
};
assertThatThrownBy(extractor::unixtime)
.isInstanceOf(RuntimeException.class)
.hasMessage("Error while reading from stream");
}
You could do similarly for reading from the input stream.
You can iterate over ArrayList using indexOf and get the next value
public String unixTimeExtractor() {
List<String> lines = getDataService.getData();
int i = lines.indexOf(unixTime);
if (i != -1 && ++i < lines.size()) {
return lines.get(i);
}
return null;
}
You can use java-8 to achieve the same. Change your method to the following :
public String unixTimeExtractor() {
ArrayList<String> lines = getDataService.getData();
return lines.stream().filter(s -> s.contains("unixtime"))
.map(s -> s.substring("unixtime: ".length()))
.findFirst()
.orElse("Not found");
}
Here we stream over the list lines to check if the String unixtime is found. If it is found then we return its value using sub-string else we return Not found.
For the test case you can refer to janos' answer.
Related
I am a begginer(recently began learning) at programming in Java and I need help.
I have to read from a file, which contains numbers. I would like to make a method for reading from a file. Then I need to analyze this data and write it in another file.
What I strugle with is if I make a method just to read from a file or do I also have to save this read data into a variable. Where should this variable be declared inside a method (if is inside, how do I use it outside), if is outside how do I use it inside a method and also outside. Can anyone help me clarify this? What am I doing wrong?
My code of what I wrote until now. File from which I had to read has houndreds of numbers.
public class Test1 {
public static void main(String[] args) {
String nameFile = "numbers.txt";
File file = new File(nameFile);
String contentFile ="";
}
//Method for reading a .txt file
private static String readFromFile(String nameFile, String contentFile) {
String line = "";
try {
BufferedReader read = new BufferedReader(new FileReader(nameFile));
while((line = read.readLine()) != null) {
line = contentFIle;
}
read.close();
} catch (IOException e) {
System.out.println("There was an error reading from a file");
}
return line;
}
}
Theoretically speaking: mathematical functions get input variables, they preform some transformation on the variables and output the result of the transformation.
For example: f(x) = x - 1, g(x) = x * 2
You can chain functions in a way that one functions output will be the other function input: g(f(2)). In this case, the number 2 is used as an input for function f(x) and the output of f(x) is the input of g(x).
Functions and methods in programming can work in a similar way, but It may be more readable to save function output into meaningful variable names, and then to apply these variables to the next function.
Instead of doing: outputText(processText(readText(someFilename)))
You can write (pseudocode):
someFilename = 'foo'
text = readText(someFilename)
processed = processText(text)
outputText(processed)
In java and in your context this would look like the following:
public class Test1 {
public static void main(String[] args) {
String nameFile = "numbers.txt";
String contentFile = readFromFileByName(nameFile);
String altered = processText(contentFile);
saveToFile(altered, "processed.txt");
}
private static String readFromFileByName(String nameFile) {
String fullRead = "";
try {
File file = new File(nameFile);
BufferedReader read = new BufferedReader(new FileReader(file));
String line; // define line variable
while((line = read.readLine()) != null) {
fullRead += line; // pay attention for the altered code
}
read.close();
} catch (IOException e) {
System.out.println("There was an error reading from a file");
} finally {
return fullRead;
}
}
private static List<Integer> stringToIntList(String string) {
return Arrays
.stream(text.split(", "))
.map(Integer::parseInt)
.collect(Collectors.toList());
}
private static String processText(String text) {
String processed = text.replace('H', 'h'); // Some heavy processing :)
return processed;
}
private static void saveToFile(String text, String fileName) {
// save <text> to file with filename <filename>
}
}
1) Line is the variable that you have read to. So you shouldn't change its value.
line = contentFIle;
if you need only first line this method should look like:
private static String readFromFile(String nameFile) {
String line = "";
try {
BufferedReader read = new BufferedReader(new FileReader(nameFile));
line = read.readLine();
read.close();
} catch (IOException e) {
System.out.println("There was an error reading from a file");
}
return line;
}
if you need a list of this:
List<String> lines = Collections.emptyList();
try {
Files.readAllLines(Paths.get(fileName), StandardCharsets.UTF_8);
} catch (IOException e) {
// do something
e.printStackTrace();
}
return lines;
2) Also you don't call readFromFile function. So you need to change the main method:
public static void main(String[] args) {
String nameFile = "numbers.txt";
String contentFile = readFromFile(nameFile);
}
3)For your particular case, there's no sense to call readFromFile with String contentFile because you don't use this variable.
Forgive the beginners question but I'm a tester first and foremost and am struggling to comprehend how to work through this. I'm simply trying to populate some test data by reading a partial website url and its corresponding syndicator login ID by using a buffered input stream reader and returning two values from each line, split by a comma.
Here's my csv:
website1.uk, website1syndicator
website2.uk, website2syndicator
website3.uk, website3syndicator
Here's my class to read the csv and populate List with one String element:
public class AbstractTestAllSites extends PageBase {
private static Logger log = LoggerFactory.getLogger(AbstractTestAllSites.class);
private static List<String> allWebsiteNames;
static {
try (InputStream websiteListInputStream = AbstractTestAllSites.class.getResourceAsStream("/websites/my_sites.csv")) {
readAllWebsiteNamesFrom(websiteListInputStream);
} catch (IOException e) {
log.error("Failed to read websitelist!", e);
}
}
private static void readAllWebsiteNamesFrom(InputStream input) throws IOException {
BufferedReader reader = new BufferedReader(new InputStreamReader(input, StandardCharsets.UTF_8));
List<String> websites = new ArrayList<String>();
String listLine;
while ((listLine = reader.readLine()) != null) {
listLine = listLine.trim();
if (!(listLine.startsWith("#") || isBlank(listLine))) {
websites.add(listLine);
}
}
allWebsiteNames = unmodifiableList(websites);
}
#Parameterized.Parameters
public static final List<String> data() {
return allWebsiteNames;
}
}
I can then pass the website endpoints into my test like so:
private static final String url = "http://mydomain.";
private String website;
private String syndicator;
public static WebDriver driver;
public TestAllSitesTest(String website, String syndicator){
this.website = website;
this.syndicator = syndicator;
}
#Before
public void getNextWebsite(){
driver.get(url + this.website);
}
//run my tests here...
...and iterate over them until done. But how can I pass two params in so I can access the syndicator variable - probably need a HashMap or similar and then split on the comma but struggling a bit.
In case you want to know how to split each line of your csv file in order to create an object of class TestAllSitesTest (which has a constructor taking a website and a syndicator), you can do it as follows (at the desired position in your code, this is just a main method showing an example):
public static void main(String[] args) {
// create two ArrayLists, first one containing lines, second containing desired objects
List<String> websites = new ArrayList<String>();
List<TestAllSitesTest> testAllSitesTests = new ArrayList<TestAllSitesTest>();
// add csv lines to the first ArrayList
websites.add("website1.uk, website1syndicator");
websites.add("website2.uk, website2syndicator");
websites.add("website3.uk, website3syndicator");
// iterate the list containing the csv lines
websites.forEach((String website) -> {
// split one line into the desired two parts, eliminating comma and space
String[] splitWebsite = website.split(", ");
// create a new object passing the parts of the split line as constructor parameters
TestAllSitesTest test = new TestAllSitesTest(splitWebsite[0], splitWebsite[1]);
testAllSitesTests.add(test);
});
// print the resulting objects
testAllSitesTests.forEach((TestAllSitesTest t) -> {
System.out.println("Website: " + t.getWebsite()
+ ", Syndicator: " + t.getSyndicator());
});
}
I hope this helps…
I'm building a small application in Java, small game mechanics but nothing serious. I have a class which purpose is to fetch data from a file. But when I declare the two classes to read from it the program justs ignore everything and continues. As a result, when I try to access the respective lists it gives me null pointer exception. Code of the method that fetches data below:
public void getData(int l, player tmp, level le) {
String[] dataPlayer;
String[] dataLevel;
try {
//FileReader f = new FileReader(this.levelPath.concat(Integer.toString(l)));
File f = new File(this.levelPath.concat(Integer.toString(l)));
BufferedReader buff = new BufferedReader(new FileReader(f));
System.out.println("Reached");
boolean eof = false;
while (!eof) {
String b = buff.readLine();
if (b == null)
eof = true;
else {
if (b.contains("player")) {
dataPlayer = b.split("-");
for (int i = 0; i < dataPlayer.length; i++) {
if (i == 0)
continue;
items it = new items(dataPlayer[i]);
tmp.setInventory1(it);
}
}else if (b.contains("level")) {
dataLevel = b.split("-");
for (int i = 0; i < dataLevel.length; i++) {
if (i == 0)
continue;
items it = new items(dataLevel[i]);
le.setSpecific(it);
}
}
}
}
}catch (IOException i) {
i.getMessage();
}
}
File contents of the file "levelData1":
player-hat
player-flashlight
level-flower
level-rock
player-adz
The problem with this particular problem was the path, it needed the absolute like that /home/toomlg4u/IdeaProjects/javaProject/src/Data/levelData.
You're doing a lot of things inside that try/catch that may not throw an IOException. If you get any other exception, it's not going to be caught. Depending on what other exception handling you have in place, that may cause weird behavior. For debugging, you could catch all exceptions, and see if you're getting something else.
If you want to remain to your loop code then you can refactor your code to look like this one:
public void getData(int l, player tmp, level le) {
try (BufferedReader buff = new BufferedReader(new FileReader(new File(this.levelPath + l)))) {
String b;
while ((b = buff.readLine()) != null) {
if (b.contains("player")) {
String[] dataPlayer = b.split("-");
items it = new items(dataPlayer[1]); //because you know that you will have an array with only 2 elements
tmp.setInventory1(it);
}else if (b.contains("level")) {
String[] dataLevel = b.split("-");
items it = new items(dataLevel[1]); //because you know that you will have an array with only 2 elements
le.setSpecific(it);
}
}
}catch (IOException e) {
e.printStackTrace();
}
}
It is a little bit better than that you have, easier to debug and to read. I advice you to read about try with resources.
As a rule of thumb, each time when you open a stream you have to close it. When you don't open it yourself then don't close it.
This is how it should look like a decent program in Java:
private Stream<Items> asStreamOfItems(String line){
return Stream.of(line.split("-")).skip(1).map(Items::new);
}
public void parseFile(String pathToTheFile) throws IOException {
List<String> lines = Files.readAllLines(Paths.get(pathToTheFile));
List<Items> players = lines.stream().filter(line -> line.contains("player")).flatMap(this::asStreamOfItems).collect(Collectors.toList());
List<Items> levels = lines.stream().filter(line -> line.contains("level")).flatMap(this::asStreamOfItems).collect(Collectors.toList());
........
}
In this case all your weird errors will vanish.
After you edited the post I saw your file content. In this case the code should look like this one:
class Items {
private final String name;
public Items(String name) {
this.name = name;
}
public String getName() {
return name;
}
public static Items parse(String line) {
return new Items(line.split("-")[1]);
}
}
public void parseFile(String pathToTheFile) throws IOException {
List<String> lines = Files.readAllLines(Paths.get(pathToTheFile));
List<Items> players = lines.stream().filter(line -> line.contains("player")).map(Items::parse).collect(Collectors.toList());
List<Items> levels = lines.stream().filter(line -> line.contains("level")).map(Items::parse).collect(Collectors.toList());
..............
}
Btw, you broke a lot of Java and general programming rules like:
using continue is a bad practice. It should be used only in extreme cases because it makes the code difficult to read.
the class name in Java should be in the CamelCase notation
one method should have only one responsibility
DON'T mutate the object inside of a method (example: tmp.setInventory1(it);) very very very bad practice
when you work with streams use try with resource or try/catch/finally to close your stream after you finish the reading.
Before jumping to write code explore the JAVA IO SDK to look for better methods to read from files
im implementing external merge sort using Java.
So given a file I split it into smaller ones , then sort the smaller portions and finally merge the sorted (smaller) files.
So , the last step is what im having trouble with.
I have a list of files and I want at each step , take the minimum value of the first rows of each file and then remove that line.
So , it is supposed to be something like this:
public static void mergeSortedFiles(List<File> sorted, File output) throws IOException {
BufferedWriter wf = new BufferedWriter(new FileWriter(output));
String curLine = "";
while(!sorted.isEmpty()) {
curLine = findMinLine(sorted);
wf.write(curLine);
}
}
public static String findMinLine(List<File> sorted) throws IOException {
List<BufferedReader> brs = new ArrayList<>();
for(int i =0; i<sorted.size() ; i++) {
brs.add(new BufferedReader(new FileReader(sorted.get(i))));
}
List<String> lines = new ArrayList<>();
for(BufferedReader br : brs) {
lines.add(br.readLine());
}
Collections.sort(lines);
return lines.get(0);
}
Im not sure how to update the files, anyone can help with that?
Thanks for helping!
You can create a Comparable wrapper around each file and then place the wrappers in a heap (for example a PriorityQueue).
public class ComparableFile<T extends Comparable<T>> implements Comparable<ComparableFile<T>> {
private final Deserializer<T> deserializer;
private final Iterator<String> lines;
private T buffered;
public ComparableFile(File file, Deserializer<T> deserializer) {
this.deserializer = deserializer;
try {
this.lines = Files.newBufferedReader(file.toPath()).lines().iterator();
} catch (IOException e) {
// deal with it differently if you want, I'm just providing a working example
// and wanted to use the constructor in a lambda function
throw new UncheckedIOException(e);
}
}
#Override
public int compareTo(ComparableFile<T> that) {
T mine = peek();
T theirs = that.peek();
if (mine == null) return theirs == null ? 0 : -1;
if (theirs == null) return 1;
return mine.compareTo(theirs);
}
public T pop() {
T tmp = peek();
if (tmp != null) {
buffered = null;
return tmp;
}
throw new NoSuchElementException();
}
public boolean isEmpty() {
return peek() == null;
}
private T peek() {
if (buffered != null) return buffered;
if (!lines.hasNext()) return null;
return buffered = deserializer.deserialize(lines.next());
}
}
Then, you can merge them this way:
public class MergeFiles<T extends Comparable<T>> {
private final PriorityQueue<ComparableFile<T>> files;
public MergeFiles(List<File> files, Deserializer<T> deserializer) {
this.files = new PriorityQueue<>(files.stream()
.map(file -> new ComparableFile<>(file, deserializer))
.filter(comparableFile -> !comparableFile.isEmpty())
.collect(toList()));
}
public Iterator<T> getSortedElements() {
return new Iterator<T>() {
#Override
public boolean hasNext() {
return !files.isEmpty();
}
#Override
public T next() {
if (!hasNext()) throw new NoSuchElementException();
ComparableFile<T> head = files.poll();
T next = head.pop();
if (!head.isEmpty()) files.add(head);
return next;
}
};
}
}
And here's some code to demonstrate it works:
public static void main(String[] args) throws IOException {
List<File> files = Arrays.asList(
newTempFile(Arrays.asList("hello", "world")),
newTempFile(Arrays.asList("english", "java", "programming")),
newTempFile(Arrays.asList("american", "scala", "stackoverflow"))
);
Iterator<String> sortedElements = new MergeFiles<>(files, line -> line).getSortedElements();
while (sortedElements.hasNext()) {
System.out.println(sortedElements.next());
}
}
private static File newTempFile(List<String> words) throws IOException {
File tempFile = File.createTempFile("sorted-", ".txt");
Files.write(tempFile.toPath(), words);
tempFile.deleteOnExit();
return tempFile;
}
Output:
american
english
hello
java
programming
scala
stackoverflow
world
So what you want to do is to swap two lines in a text file? You can do it by using a RandomAccessFile however this will be horrible slow since everytime when you swap two lines you have to wait for the next IO burst.
So i highly recommend you to use the following code to be able to do the merge sort on the heap:
List<String> lines1 = Files.readAllLines(youFile1);
List<String> lines2 = Files.readAllLines(youFile2);
//use merge sort on theese lines
List<String> merged;
FileWriter writer = new FileWriter(yourOutputFile);
for(String str: merged) {
writer.write(str + System.lineSeparator());
}
writer.close();
The standard merge technique between a fixed number of files (say, 2) is :
have a variable for the value of the ordering key of the current record of each file (for java, make that variable Comparable).
start the process by reading the first record of each file (and fill in the corresponding variable)
loop (until end-of-file on both) through a code block that says essentially
if (key_1.compareTo(key_2) == 0) { process both files ; then read both files}
else if (key_1.compareTo(key_2) == -1) { process file 1 ; then read file 1}
else { process file 2 ; then read file 2}
Note how this code does essentially nothing more than determine the file with the lowest key, and process that.
If your number of files is variable, then your number of key variables is variable too, and "determining the file with the lowest current key" cannot be done as per above. Instead, have as many current_key_value objects as there are files, and store them all in a TreeSet. Now, the first element of the TreeSet will be the lowest current key value of all the files and if you make sure that you maintain a link between your key variable and the file number you just process that file (and delete the just processed key value from the TreeSet and read a new record from the processed file and add its key value to the TreeSet).
I am currently learning Java and I have faced this problem where I want to load a file that consists a huge number of lines (I am reading the file line by line ) and the thing I want to do is skip certain lines (pseudo-code).
the line thats starts with (specific word such as "ABC")
I have tried to use
if(line.startwith("abc"))
But that didn't work. I am not sure if I am doing it wrong, that's why I am here asking for a help, below part of the load function:
public String loadfile(.........){
//here goes the variables
try {
File data= new File(dataFile);
if (data.exists()) {
br = new BufferedReader(new FileReader(dataFile));
while ((thisLine = br.readLine()) != null) {
if (thisLine.length() > 0) {
tmpLine = thisLine.toString();
tmpLine2 = tmpLine.split(......);
[...]
Try
if (line.toUpperCase().startsWith("ABC")){
//skip line
} else {
//do something
}
This will converts the line to all the Upper Characters by using function toUpperCase() and will check whether the string starts with ABC .
And if it is true then it will do nothing(skip the line) and go into the else part.
You can also use startsWithIgnoreCase which is a function provided by the Apache Commons . It takes the two string arguments.
public static boolean startsWithIgnoreCase(String str,
String prefix)
This function return boolean.
And checks whether a String starts with a specified prefix.
It return true if the String starts with the prefix , case insensitive.
If the case isn't important try using the StringUtils.startsWithIgnoreCase(String str,
String prefix) of Apache Commons
This function return boolean.
See javadoc here
Usage:
if (StringUtils.startsWithIgnoreCase(line, "abc")){
//skip line
} else {
//do something
}
If you have large a input File, you code will create a OutOfMemoryError. there is nothing you can do against it without editing te code (adding more memory will fail, if the file gets bigger).
I beleave you store the selected lines in memory. If the file gets lager (2GB or so) you'll have 4GB in memory. (The old Value of the String and the new one).
You have to work with streams to solve this.
Create a FileOutpuStream, and write the selcted line into that Stream.
Your method must be changed. For a large input yo cannot return a String:
public String loadfile(...){
You can return a Stream or a file.
public MyDeletingLineBufferedReader loadFile(...)
you can use:
BufferedReader br = new BufferedReader(new FileReader("file.txt"));
String lineString;
try{
while((lineString = br.readLine()) != null) {
if (lineString.toUpperCase().startsWith("abc")){
//skip
} else {
//do something
}
}
}
or
static boolean startsWithIgnoreCase(String str, String prefix) method in org.apache.commons.lang.StringUtils like below.
BufferedReader br = new BufferedReader(new FileReader("file.txt"));
String lineString;
try{
while((lineString = br.readLine()) != null) {
if (StringUtils.startsWithIgnoreCase(lineString, "abc")){
//skip
} else {
//do something
}
}
}