I have a mysql table that has a column with the type longtext, this column containt the content of a csv file. Is there a way to create from the content of this column a csv file and extract data from it by using talend ?
You could extract the String from the DB using the proper Input component. Then use tExtractDelimitedField to split this long String string against a separator character (the comma, I guess). Don't forget to specify carefully your output schema
Finally, use tFileDelimitedOut to write the delimited file with data from the outgoing connection on the file system.
This could help: Validate a csv file
Sure, request the string out of the database, open a file and write the string to it - where's your actual problem?
Related
Created csv file with name text.csv (Microsoft Excel Comma Separated Values File (.csv) type) with column data in the column as 55,666,666,7777,9999,7777,999,777 through java code. If I try to open the file by 5.56666667777999E+25. Even by changing the format of the cell, the data is displaying in the same format. But if I use Text Import Wizard and selecting the Column data format as 'Text' I am able to see the data in the way I created while writing the CSV file.
Is that something I can do through code, or is this related to Excel formatting. As when we provide the same to client, they won't change the formatter the column in excel. They just double click and open the file.
I want to split and save my file as xml on the basis of column name, but below code is not working.
File are getting split automatically.
df.write().partitionBy(columnName).format("xml").save(filePath);
I am working on a java application which inserts data from a CSV file to mysql database table using below query
LOAD DATA LOCAL INFILE 'test.csv' INTO TABLE Test FIELDS TERMINATED BY ',' LINES TERMINATED BY ', ' IGNORE 1 LINES (id, name, address)
I have a issue when there is a comma between words for the a field like for address = pune, india. It is pushing the word after comma to next column.
I have found some workaround, such as escaping comma in Java code like
strAddress = strAddressWithComma.replaceAll(",", "\\"+"\\,");
This works but after data is inserted in DB table it looks like “pune, india” (double quotes around the original string)
Another workaround is adding ENCLOSED BY ‘\”’ clause in LOAD DATA LOCAL INFILE and using below java code.
strAddress ="\"" + strAddressWithComma+ "\"";
But with this workaround also we have string with double quotes around the original string in DB when we use LOAD DATA LOCAL INFILE.
I do not need double quotes around original value in DB after
insertion with Load Data statement . DB entry should be same as that
of data in original csv file.
I know we may have a DB trigger which can strip double quotes from the new string but we want to handle this is application only.
Any solution or suggestion will be appreciated
I need to bind a group of csv file in the format "YYYY-MM-DD hh:mm:ss.csv" that are present in the same folder with a unique table that contains all the data present in all the files.
I need to read the data from a Java EE application thus I would like to create a connection pool inside the application server. I found the CsvJdbc driver that allows the reading of multiple files as a single entity. A good starting point was this page in the section with this paragraph:
To read several files (for example, daily log files) as a single table, set the database connection property indexedFiles. The following example demonstrates how to do this.
The example could be fine for me but the problem is that I do not have a header word in the filename string. So the corresponding table becames an empty string that makes obviously impossible to query the table.
How can I tell the driver to map the pattern to a table that hasn't a header part?
P.S. I already tried to use hsqldb as a frontend to the csv files but it does not support multiple files.
Setup CsvJdbc to read several files as described in http://csvjdbc.sourceforge.net/doc.html and then use an empty table name in the SQL query because your CSV filenames do not have any header before the fileTailPattern regular expression. For example:
props.put("fileTailPattern", "(\\d+)-(\\d+)-(\\d+) (\\d+):(\\d+):(\\d+)");
props.put("fileTailParts", "Year,Month,Day,Hour,Minutes,Seconds");
...
ResultSet results = stmt.executeQuery("SELECT * FROM \"\" AS T1");
I want to insert the Xml file data into MySQL table ,, by choosing which column to insert into ,, using Java How will this be done ?
It really depends on the format of your XML file. If your XML file is a direct export from the MySQL file, please refer to this question.
If your XML is in some other format, then I would probably be using JAXB to parse XML into POJO, then write some logic to map the POJO into the database table.