I am running this command via JAVA program builder
plpgsql = "\"path_to_psql_executable\psql.exe" -U myuser -w -h myhost -d mydb -a -f "some_path\copy_7133.sql" 2> "log_path\plsql_7133.log\"";
ProcessBuilder pb = new ProcessBuilder("C:\\Windows\\System32\\cmd.exe", "/c", plpgsql);
Process p = pb.start();
p.getOutputStream().close();
p.waitFor();
This is returning me the following error:
ERROR: invalid byte sequence for encoding "UTF8": 0xbd CONTEXT: COPY
copy_7133, line 4892
The catch is if I the run the SQL command manually in cmd, then it is copying all of the data successfully giving me the number of rows inserted. Not able to figure out the reason
NOTE: The code is causing problem only for one particular file, for rest working fine.
EDIT:
Copy command being run:
\copy s_m_asset_7140 FROM 'C:\ER\ETL\Unzip_7140\asset.csv' csv HEADER QUOTE '"' ENCODING 'UTF8';
The last error the command gave:
psql:C:/ER/ETL/Unzip_7140/copy_s_m_asset_7140.sql:1: ERROR: invalid
byte sequence for encoding "UTF8": 0xa0 CONTEXT: COPY s_m_asset_7140,
line 10282
But there doesn't seem to be any special character except a '-'. Not sure what it is not able read.
Few more details abt DB:
show client_encoding;
"UNICODE"
show server_encoding;
"UTF8"
Worked. But still not understand why UTF8 did not work.
I changed the encoding to LATIN1 and it worked
\copy s_m_asset_7140 FROM 'C:\ER\ETL\Unzip_7140\asset.csv' csv HEADER QUOTE '"' ENCODING 'LATIN1';
Can somebody pls explain why UTF8 did not work?
I have registered a windows service but when trying to start it says,
[SC] StartService FAILED 2:
The system cannot find the file specified.
I checked the regedit where iamgepath is not getting set properly.
path should be D:\abc\Windows.exe but it is D:abcWindows.exe
I am using java to do above things.
Please help.....
Replace slash \ with double slash \\ like below:
"D:\\abc\\Windows.exe"
and it should work
In Java '\' is a character which define escape sequences like \n means new line character, So if you want to use \ as a character in your input string place one more \ before your .
So if you want to write a\b\c write a\b\c
System.out.println("a\b\c"); will print a\b\c
On Windows, using System.out.println() prints out \n\r while on a Unix system you would get \n.
Is there any way to tell java what new-line characters you want to use?
As already stated by others, the system property line.separator contains the actual line separator. Strangely, the other answers missed the simple conclusion: you can override that separator by changing that system property at startup time.
E.g. if you run your program with the option -Dline.separator=X at the command line you will get the funny behavior of System.out.println(…); ending the line with an X.
The tricky part is how to specify characters like \n or \r at the command line. But that’s system/environment specific and not a Java question anymore.
Yes, there is a way and I've just tried it.
There is a system property line.separator. You can set it using System.setProperty("line.separator", whatever)
To be sure that it indeed causes JVM to use other separator I implemented the following exercise:
PrintWriter writer = new PrintWriter(new FileWriter("c:/temp/mytest.txt"));
writer.println("hello");
writer.println("world");
writer.close();
I am running on windows now, so the result was 14 bytes long file:
03/27/2014 10:13 AM 14 mytest.txt
1 File(s) 14 bytes
0 Dir(s) 409,157,980,160 bytes free
However when I added the following line to the beginning of my code:
System.setProperty("line.separator", "\n");
I got 14 bytes long file:
03/27/2014 10:13 AM 14 mytest.txt
1 File(s) 14 bytes
0 Dir(s) 409,157,980,160 bytes free
I opened this file with notepad that does not recognize single \n as a new line and saw one-line text helloworld instead of 2 separate lines. So, this works.
Because the accepted answer simply does not work, as others pointed out before me, and the JDK only initialises the value once and then never reads the property anymore, only an internal static field, it became clear that the clean way to change the property is to set it on the command line when starting the JVM. So far, so good.
The reason I am writing yet another answer is that I want to present a reflective way to change the field, which really works with streams and writers relying on System.lineSeparator(). It does not hurt to update the system property, too, but the field is more important.
I know that reflection is ugly, as of Java 16+ needs an extra JVM command line parameter in order to allow it, and only works as long as the internals of System do not change in OpenJDK. But FWIW, here is my solution - don't do this at home, kids:
import java.io.*;
import java.lang.reflect.Field;
import java.nio.file.Files;
/**
* 'assert' requires VM parameter '-ea' (enable assert)
* 'Field.setAccessible' on System requires '--add-opens java.base/java.lang=ALL-UNNAMED' on Java 16+
*/
public class ChangeLineSeparator {
public static void main(String[] args) throws IOException, NoSuchFieldException, IllegalAccessException {
assert System.lineSeparator().equals("\r\n") : "default Windows line separator should be CRLF";
Field lineSeparator = System.class.getDeclaredField("lineSeparator");
lineSeparator.setAccessible(true);
lineSeparator.set(null, "\n");
assert System.lineSeparator().equals("\n") : "modified separator should be LF";
File tempFile = Files.createTempFile(null, null).toFile();
tempFile.deleteOnExit();
try (PrintWriter out = new PrintWriter(new FileWriter(tempFile))) {
out.println("foo");
out.println("bar");
}
assert tempFile.length() == "foo\nbar\n".length() : "unexpected file size";
}
}
You may try with:
String str = "\n\r";
System.out.print("yourString"+str);
but you can instead use this:-
System.getProperty("line.separator");
to get the line seperator
Returns the system-dependent line separator string. It always returns
the same value - the initial value of the system property
line.separator.
On UNIX systems, it returns "\n"; on Microsoft Windows systems it
returns "\r\n".
As stated in the Java SE tutorial:
To modify the existing set of system properties, use
System.setProperties. This method takes a Properties object that has
been initialized to contain the properties to be set. This method
replaces the entire set of system properties with the new set
represented by the Properties object.
Warning: Changing system
properties is potentially dangerous and should be done with
discretion. Many system properties are not reread after start-up and
are there for informational purposes. Changing some properties may
have unexpected side-effects.
In the case of System.out.println(), the line separator that existed on system startup will be used. This is probably because System.lineSeparator() is used to terminate the line. From the documentation:
Returns the system-dependent line separator string. It always returns
the same value - the initial value of the system property
line.separator.
On UNIX systems, it returns "\n"; on Microsoft Windows systems it
returns "\r\n".
As Holger pointed out, you need to overwrite this property at startup of the JVM.
Windows cmd: Credit jeb at https://superuser.com/a/1519790 for a technique to specify a line-feed character in a parameter using a cmd variable. This technique can be used to specify the java line.separator.
Here's a sample javalf.cmd file
#echo off
REM define variable %\n% to be the linefeed character
(set \n=^^^
^
)
REM Start java using the value of %\n% as the line.separator System property
java -Dline.separator=%\n% %1 %2 %3 %4 %5 %6 %7 %8 %9
Here's a short test progam.
public class ReadLineSeparator {
public static void main(String... ignore) {
System.out.println(System.lineSeparator().chars()
.mapToObj(c -> "{"+Character.getName(c)+"}")
.collect(java.util.stream.Collectors.joining()));
}
}
On Windows,
java ReadLineSeparator produces
{CARRIAGE RETURN (CR)}{LINE FEED (LF)}
.\javalf.cmd ReadLineSeparator produces
{LINE FEED (LF)}
The method System.lineSeparator() returns the line separator used by the system. From the documentation it specifies that it uses the system property line.separator.
Checked Java 11 implementation:
private static void initPhase1() {
lineSeparator = props.getProperty("line.separator");
}
public static String lineSeparator() {
return lineSeparator;
}
So altering system property at runtime doesn't change System.lineSeparator().
For this reason some projects re-read system property directly, see my answer: How to avoid CRLF (Carriage Return and Line Feed) in Logback - CWE 117
The only viable option is to set system property during app startup.
For Bash it is as simple as: java -Dline.separator=$'\n' -jar my.jar.
For POSIX shell it is better to save that character in some variable first:
LF='
'
java -Dline.separator="$LF" -jar my.jar
If you are not sure debug it:
printf %s "$LF" | wc -c
1
printf %s "$LF" | od -x
0000000 000a
For Gradle I use:
tasks.withType(Test).configureEach {
systemProperty 'line.separator', '\n'
}
bootRun {
systemProperty 'line.separator', '\n'
}
I need your assistance to chop a single continuous line from a log file of HL7 messages, into individual lines such as the following in using Unix commands / Java or Both:
Timestamp : 20130221 001805
Code : OUT031
Severity : F
Component : cmHL7OutboundBuild_jcdOutboundBuild1/cmHL7OutboundBuild/dpPRODHL7Outbound
Description : Last HL7 Message read <MSH|^~\&|target application|hostname|source application|A212|201302210016||ORM^O01|62877102|D|2.2|||ER
PID||.|0625176^^^00040||JOHN^SMITH^ ||19390112|F||4|address||^^^^^^93271081||||||41253603422
PV1|1||ED^^^40||||||name of physician||||||||physician operator id
ORC|SC||13-4529701-TFT-0|13-4529701|P|||||||name of physician
OBR|1||13-4529701-TFT-0|0360000^THYROID FUNCTION TESTS^1||201302212108|201302212102|||||||201302212108||name of physician||.|||13-4529701|201302210016||department|P||^^^201302212108
> Exception:com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot add or update a child row: a foreign key constraint fails (`schema/table`, CONSTRAINT `table_ibfk_request_order_panel` FOREIGN KEY (`fk_gp_latest_request_order_panel`) REFERENCES `gp_request_order_panel` (`gp_request_order_panel_seq`))
SendEmail : Y
The issue is that all 11 lines are wrapped around in one continuous line which made up a single record, followed by the next record of another 11 lines and so on Solaris 10 Unix server. Yet
the same log would display properly when ftped over to a desktop Windows side. As a result, I am looking for a solution, possibly a combination of Unix & Java to break down each record of
11 lines on their own and the same thing for the next record....
Below is a break down on what I believe need to be done on the Solaris 10 server:
( i ) Would be great if this log could be converted to Windows ASCII format but staying on the Solaris 10 server, like the one being ftped to Windows. When the same file is ftped from
Windows back to Solaris 10 server, it reverted back to its original 1 continuous line format.
( ii ) Otherwise, break it down line by line or insert and end of line but where to?
I have tried various Unix string manipulated commands such as sed, awk, grep, fold, cut, unix2dos including from
Bash: how to find and break up long lines by inserting continuation character and newline? without success.
Your assistance would be much appreciated.
Thanks,
Jack
This is a windows text file (windows carriage control) since it displays ok in Windows but not UNIX.
Solaris has a dos2unix command to handle this issue.
dos2unix filename > newfilename
Try that. To see what I mean use the vi editor on "filename" in the example above. You will see ^M characters in the file. UNIX does not use those for formatting lines.
If you have the vim editor it will read "long lines". Or try od -c < filename | more to see the ^M or ascii 13 characters. od will show them as \r I'm reasonably sure the issue is that the carriage control on the file prevents UNIX from seeing the lines. Try the od approach.
I'm facing a problem with calling java from php on a linux server with popen.
$java = '/usr/bin/java';
$cmd = "$java -jar javafiles/register.jar < $tmpFile";
What does the < before $tmpFile mean? Because apparently it is loading the content of $tmpFile from disk and inputting it directly on the console of the register.jar execution. Is that the case? Because the content of $tmpFile has special characters and these aren't being encoded in the right charset.
Thats exactly what it does. Specifically, it runs the program and sends the contents of $tmpFile into the standard input (System.in) of the program being executed.