I've written some java code that creates a CSV file at the mount point of a disk I have attached to a Google Compute instance. I run the script in the form of a SQL stored procedure from the instance that the disk is attached to. The issue is that at the mount point, a "lost+found" folder is created where I would expect to find my CSV file. What am I doing wrong? Thank you for your time! The code is similar to as follows:
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.FileOutputStream;
public class file_write {
public static void main(String[] args) throws IOException{
String filePath = "/mnt/point/file.csv";
// Creates file in mount point
File myFile = new File(filePath);
myFile.getParentFile().mkdirs();
myFile.createNewFile();
FileWriter stringWriter = new FileWriter(myFile);
for(int i = 0; i < 100000; i++) {
stringWriter.write(i + ", ");
stringWriter.write("something");
stringWriter.write(System.lineSeparator());
}
stringWriter.close();
}
}
Related
import java.io.BufferedInputStream;
import java.io.FileInputStream;
import java.io.InputStream;
import java.io.OutputStream;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;
import org.apache.hadoop.util.Tool;
import org.apache.hadoop.util.ToolRunner;
public class HdfsWriter extends Configured implements Tool {
public int run(String[] args) throws Exception {
//String localInputPath = args[0];
Path outputPath = new Path(args[0]); // ARGUMENT FOR OUTPUT_LOCATION
Configuration conf = getConf();
FileSystem fs = FileSystem.get(conf);
OutputStream os = fs.create(outputPath);
InputStream is = new BufferedInputStream(new FileInputStream("/home/acadgild/acadgild.txt")); //Data set is getting copied into input stream through buffer mechanism.
IOUtils.copyBytes(is, os, conf); // Copying the dataset from input stream to output stream
return 0;
}
public static void main(String[] args) throws Exception {
int returnCode = ToolRunner.run(new HdfsWriter(), args);
System.exit(returnCode);
}
}
Need to Move the data from Local to HDFS.
The above code I got from another blog , it's not working. can anyone help me on this.
Also i need to parse the Json using MR and group by DateTime and move to HDFS
Map Reduce is a distributed job processing framework
for each mapper local means the local filesytem on the node on which that mapper is running.
What you want is reading from local on a given node to be put on to HDFS and then processing it via MapReduce.
There are multiple tools available for copying from Local of one node to HDFS
hdfs put localPath HdfsPath (Shell script)
flume
I'm trying to write some text to an html file as an output using PrintWriter, and the text isn't saving to the file.
import java.util.Random;
import java.util.*;
import java.io.PrintWriter;
import java.io.File;
import java.io.FileNotFoundException;
public class Creator
{
static ArrayList<Character> grid = new ArrayList<Character>();
public static void main(String[]args) throws FileNotFoundException
{
char[] alphabet={'a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z'};
for(int row=0;row<625;row++)
{
grid.add(alphabet[RandGen(0,25)]);
//System.out.print(grid.get(out));
}
Creator.Output();
System.out.println("Executed.");
}
public static int RandGen(int min, int max)
{
Random ran = new Random();
int randomNum = ran.nextInt(max) + min;
return randomNum;
}
public static void Output()throws FileNotFoundException
{
//File file=new File("wsm.html");
//File.createNewFile();
PrintWriter writer = new PrintWriter("wordsearchmaker.html");
writer.println("<html>");
writer.println("<table>");
writer.println("tr");
for(int j=0;j<25;j++)
{
//for(int k=0;k<25;k++)
// {
System.out.println("<th>"+grid.get(j));
writer.println("<th>"+grid.get(j));
// }
}
writer.flush();
writer.close();
System.out.println("Outputting...");
}
}
So I've checked that the methods are all running (hence the "outputting..."), and I system.out.printed the content that I'm intending to write to the file, which is outputting exactly what I want it to. It's supposed to output html code into a html file (named wordsearchmaker.html), but nothing is saving to the file. Everywhere I looked online just said to make sure I'm closing the writer, which I did.
Note: I am working in eclipse, which has always been kind of finicky with me, so I may be messing something up there? I don't usually work in eclipse so that's totally possible.
It looks like you've opened the PrintWriter, which enables you to send data to the file. But you haven't actually opened or created a file.
Try first creating a new file and modifying it:
import java.io.File;
File newFile = new File ("LOCATION OF FILE");
Then, set your PrintWriter to use newFile.
I have to move files from one directory to other directory.
Am using property file. So the source and destination path is stored in property file.
Am haivng property reader class also.
In my source directory am having lots of files. One file should move to other directory if its complete the operation.
File size is more than 500MB.
import java.io.File;
import java.nio.file.Files;
import java.nio.file.StandardCopyOption;
import static java.nio.file.StandardCopyOption.*;
public class Main1
{
public static String primarydir="";
public static String secondarydir="";
public static void main(String[] argv)
throws Exception
{
primarydir=PropertyReader.getProperty("primarydir");
System.out.println(primarydir);
secondarydir=PropertyReader.getProperty("secondarydir");
File dir = new File(primarydir);
secondarydir=PropertyReader.getProperty("secondarydir");
String[] children = dir.list();
if (children == null)
{
System.out.println("does not exist or is not a directory");
}
else
{
for (int i = 0; i < children.length; i++)
{
String filename = children[i];
System.out.println(filename);
try
{
File oldFile = new File(primarydir,children[i]);
System.out.println( "Before Moving"+oldFile.getName());
if (oldFile.renameTo(new File(secondarydir+oldFile.getName())))
{
System.out.println("The file was moved successfully to the new folder");
}
else
{
System.out.println("The File was not moved.");
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
System.out.println("ok");
}
}
}
My code is not moving the file into the correct path.
This is my property file
primarydir=C:/Desktop/A
secondarydir=D:/B
enter code here
Files should be in B drive. How to do? Any one can help me..!!
Change this:
oldFile.renameTo(new File(secondarydir+oldFile.getName()))
To this:
oldFile.renameTo(new File(secondarydir, oldFile.getName()))
It's best not to use string concatenation to join path segments, as the proper way to do it may be platform-dependent.
Edit: If you can use JDK 1.7 APIs, you can use Files.move() instead of File.renameTo()
Code - a java method:
/**
* copy by transfer, use this for cross partition copy,
* #param sFile source file,
* #param tFile target file,
* #throws IOException
*/
public static void copyByTransfer(File sFile, File tFile) throws IOException {
FileInputStream fInput = new FileInputStream(sFile);
FileOutputStream fOutput = new FileOutputStream(tFile);
FileChannel fReadChannel = fInput.getChannel();
FileChannel fWriteChannel = fOutput.getChannel();
fReadChannel.transferTo(0, fReadChannel.size(), fWriteChannel);
fReadChannel.close();
fWriteChannel.close();
fInput.close();
fOutput.close();
}
The method use nio, it make use os underling operation to improve performance.
Here is the import code:
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
If you are in eclipse, just use ctrl + shift + o.
I've tried directly linking using the entire path but that hasn't solved it either.
package eliza;
import java.io.*;
public class Eliza {
public static void main(String[] args) throws IOException {
String inputDatabase = "src/eliza/inputDataBase.txt";
String outputDatabase = "src/eliza/outputDataBase.txt";
Reader database = new Reader();
String[][] inputDB = database.Reader(inputDatabase);
String[][] outputDB = database.Reader(outputDatabase);
}
}
Here is the reader class:
package eliza;
import java.io.FileReader;
import java.io.BufferedReader;
import java.io.IOException;
public class Reader {
public String[][] Reader(String name) throws IOException {
int length = 0;
String sizeLine;
FileReader sizeReader = new FileReader(name);
BufferedReader sizeBuffer = new BufferedReader(sizeReader);
while((sizeLine = sizeBuffer.readLine()) != null) {
length++;
}
String[][] database = new String[length][1];
return (database);
}
}
Here's a photo of my directory. I even put these text files in the "eliza" root folder: here
Any ideas?
Since you are using an IDE, you need to give the complete canonical path. It should be
String inputDatabase = "C:\\Users\\Tommy\\Desktop\\Eliza\\src\\eliza\\inputDataBase.txt";
String outputDatabase = "C:\\Users\\Tommy\\Desktop\\Eliza\\src\\eliza\\outputDataBase.txt";
The IDE is probably executing the bytecode from its bin folder and cannot find the relative reference.
give the exact path like
String inputDatabase = "c:/java/src/eliza/inputDataBase.txt";
you have not given the correct path, Please re check
try
{BASE_PATH}+ "Eliza/src/inputDataBase.txt"
The source directory tree isn't generally present during execution, so files that are required at runtime shouldn't be put there ... unless you're going to use them as resources, in which case their pathname is relative to the package root, and does not begin with 'src', and the data is accessed by a getResourceXXX() method, not via a FileInputStream.
How we can take input from the file in the Eclipse?
Just like we direct the I/O from the file from the command line.
java MyProgram < input.txt >output.txt
I am unable to direct the input.
but output directing is easy.
Just go->Run->Run->Configurations->Common
Why don't you use File instead of redirection?
Your program will have a fileName as input and then write the result in a file.
If you need necessarily use the default in you can do something like this:
System.setIn(new FileInputStream("testFile.txt"));
a sample of how it works follows:
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
public class TestSystemIn {
public static void main(final String[] args) throws IOException {
// prepare test
FileOutputStream fos = new FileOutputStream("testFile.txt");
fos.write("testToken".getBytes());
// configure env
System.setIn(new FileInputStream("testFile.txt"));
// perform read test
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
System.out.println("read: " + br.readLine());
}
}
The output could be done in the same way using:
System.setOut(new PrintStream("testFile.txt"));