We Keep Coding

How to encode properly the plus sign (+) when making a request with webflux webclient?

I am trying to send an international formatted phone number using spring Webflux Webclient and to read this phone number via another application also using webflux.
My code looks like this :
webClient = WebClient.builder()
.baseUrl(baseUrl)
.build();
return webClient
.get()
.uri(uriBuilder -> uriBuilder
.path("/endpoint")
.queryParam("phone-number", "+33612345678")
.build()
)
.retrieve()
.bodyToMono(String.class);
Unfortunately, somewhere between this call and the receiver, the plus sign is replaced by a space.
The endpoint receives : " 33612345678" as a String.
The netty debug log of the request shows this :
+--------+-------------------------------------------------+----------------+
|00000000| 47 45 54 20 2f 63 75 73 74 6f 6d 65 72 73 3f 70 |GET /endpoint?p|
|00000010| 68 6f 6e 65 2d 6e 75 6d 62 65 72 3d 2b 33 33 36 |hone-number=+336|
|00000020| 31 32 33 34 35 36 37 38 26 6f 6e 6c 79 2d 72 65 |12345678
I tried to encode the phone-number by myself like this :
.queryParam("phone-number", UriUtils.encode("+34612345678", StandardCharsets.UTF_8))
And netty's log shows :
+--------+-------------------------------------------------+----------------+
|00000000| 47 45 54 20 2f 63 75 73 74 6f 6d 65 72 73 3f 70 |GET /endpoint?p|
|00000010| 68 6f 6e 65 2d 6e 75 6d 62 65 72 3d 25 32 35 32 |hone-number=%252|
|00000020| 42 33 34 36 31 32 33 34 35 36 37 38 20 48 54 54 |B34612345678 HTT|
It seems that the phone number has been encoded twice.
+ -> %2B -> %252B
the plus sign has been encoded by UriUtils.encode then uriBuilder has encoded the %.
The only way I found to make it work is by disabling the encoding of the UriBuilder :
DefaultUriBuilderFactory factory = new DefaultUriBuilderFactory(baseUrl);
factory.setEncodingMode(DefaultUriBuilderFactory.EncodingMode.NONE);
this.webClient = WebClient.builder()
.baseUrl(baseUrl)
.uriBuilderFactory(factory)
.build();
and having my custom encoding UriUtils.encode("+34612345678", StandardCharsets.UTF_8)
in which case the netty's logs looks like expected :
+--------+-------------------------------------------------+----------------+
|00000000| 47 45 54 20 2f 63 75 73 74 6f 6d 65 72 73 3f 70 |GET /endpoint?p|
|00000010| 68 6f 6e 65 2d 6e 75 6d 62 65 72 3d 25 32 42 33 |hone-number=%2B3|
|00000020| 34 36 31 32 33 34 35 36 37 38 20 48 54 54 50 2f |4612345678 HTTP/|
And of course, the endpoint receiving the phone number get : "+33612345678"
To sum it up, it looks like the UriBuilder is encoding certain sign like "%" but does not encode the "+" sign.
Spring reference : https://docs.spring.io/spring-framework/docs/current/reference/html/web.html#web-uri-encoding

I struggled with same issue but found a workaround from the Spring reference you linked.
This should work for you:
return webClient
.get()
.uri(uriBuilder -> UriComponentsBuilder.fromUri(uriBuilder.build())
.path("/endpoint")
.queryParam("phone-number", "{phone-number}")
.encode()
.buildAndExpand("+33612345678")
.toUri()
)
.retrieve()
.bodyToMono(String.class);

I had the same issue and was able to resolve it in my Kotlin application using:
webClient.get()
.uri { uriBuilder ->
val queryParams = mapOf(
"phone-number" to "+33612345678",
"mobile-number" to "+61432111222"
)
uriBuilder.path("/endpoint")
.apply {
queryParams.keys.forEach { key ->
queryParam(key, "{$key}")
}
}
.build(queryParams)
}
.retrieve()
.bodyToMono<String>()

A solution provided here, you can try this way
DefaultUriBuilderFactory factory = new DefaultUriBuilderFactory();
factory.setEncodingMode(DefaultUriBuilderFactory.EncodingMode.VALUES_ONLY);
URI uri = factory.uriString("https://spring.io/")
.queryParam("query", "{query}")
.build("spring+framework");
// http://spring.io/?query=spring%2Bframework

What is the parseBase64Binary function in Python?

In Java, I use:
String str = “%EF!c&WrDwCCTe<fX$,#8L<YTs?G5d>F])ub.63G=Xn<cdef2R{47JQexxN”;
byte[] result = DatatypeConverter.parseBase64Binary(str);
for(byte i : result){
System.out.print(i);
System.out.print(" ");
}
to decode str.
Output:
16 87 22 -84 60 2 9 55 -97 95 -62 -40 78 -63 -71 116 91 -101 -21 113 94 119 29 121 -3 -111 -29 -78 80 123 28 77
Now I need to decode str with Base64 in python but I don't know which lib and function should I choose.
I've tried base64.b64decode but its result is different from that in Java.
str = '%EF!c&WrDwCCTe<fX$,#8L<YTs?G5d>F])ub.63G=Xn<cdef2R{47JQexxN'
result = base64.b64decode(str)
print(result)
for i in range(0, len(decode_secret)):
print(decode_secret[i], end=" ")
Output:
b'\x10W\x16\xac<\x02\t7\x9f_\xc2\xd8N\xc1\xb9t[\x9b\xebq'
16 87 22 172 60 2 9 55 159 95 194 216 78 193 185 116 91 155 235 113

Different data when printing and writing to file

There is a stream of data which is sent from server. I need to store this byte stream into a file. The problem is the data which I output to console and the one which I store in a file are different. Seems like there is a change in format of data when I stored in a file.
Here is the program:
try
{
System.out.println("My Address is "+serverSocket.getLocalSocketAddress());
Socket server = serverSocket.accept(); // return a new socket
System.out.println("Connected to client "+server.getRemoteSocketAddress());
inputStream = server.getInputStream();
in = new DataInputStream(inputStream);
out = new FileOutputStream("output.txt");
ArrayList<Byte> bytes = new ArrayList<Byte>();
int curi;
byte cur;
byte[] curBytes = null;
int length = 0;
System.out.println("Before while loop");
while((curi = in.read())!=-1 && count!=500)
{
System.out.println(count+" Reading some data");
//out.write(curi);
cur = (byte)curi;
bytes.add(cur);
curBytes = getPrimativeArray(bytes);
String curBytesString = new String(curBytes, "UTF-8");
count++;
}
int i=0;
for(byte b : bytes)
{
System.out.print(b+" ");
curBytes[i] = b;
i++;
}
out.write(curBytes);
server.close();
}
catch(IOException e)
{
e.printStackTrace();
}
What I print using System.out.print(b+" "); and the one I store in curBytes[] are the same thing. But when I compare the console and file output, they are different.
Console output: 0 0 113 -100 -126 -54 0 32 14 1 0 0 1 -58 60 54 0 3 63 -2 85 74 -81 -88 0 9 1 24 85 74 -81 -48 0 13 65 -113 85 74 -81 -88 0 12 125 -126 85 74 -81 -88 0 13 21 97 85 74 -81 -88 0 13 31 19 85 74 -81 -48 0 13 42 24 0 6 0 0 0 0 0 0 0 0 0 0 32 0 7 -100 0 -5 6 -128 0 -56 29 -127 23 112 -1 -1 0 0 64 0 1 -121 28 115 105 112 58 43 49 52 50 50 50 48 57 57 57 49 53 64 111 110 101 46 97 116 116 46 110 101 116 28 115 105 112 58 43 49 52 50 50 50 48 57 57 57 54 53 64 111 110 101 46 97 116 116 46 110 101 116 37 50 57 54 53 45 49 53 48 53 48 54 50 51 50 55 48 50 45 50 48 53 48 54 54 50 55 54 54 64 48 48 55 56 48 48 49 49 16 32 1 5 6 64 0 0 0 32 16 0 0 0 120 0 17 16 32 1 24 -112 16 1 46 2 0 0 0 0 0 0 0 6 1 -113 0 4 0 33 -64 -42 0 91 5 8 0 9 0 -56 0 0 0 15 3 85 74 -81 -88 0 12 -120 -28 8 0 9 0 -56 0 0 0 15 3 85 74 -81 -88 0 12 -44 -39 8 0 4 0 -56 0 0 1 11 3 85 74 -81 -88 0 9 1 24 8 0 5 0 0 0 0 0 0 3 85 74 -81 -88 0 13 31 19 8 0 1 0 -56 0 0 0 6 3 85 74 -81 -48 0 13 42 24 -64 34 4 24 9 89 83 73 80 47 50 46 48 47 84 67 80 32 91 50 48 48 49 58 53 48 54 58 52 48 48 48 58 48 58 50 48 49 48 58 48 58 55 56 58 49 49 93 58 49 51 55 48 59 98 114 97 110 99 104 61 122 57 104 71 52 98 75 50 57 48 45 48 48 55 56 48 48 49 49 45 48 48 48 102 45 52 52 49 57 55 49 52 48 51 3 85 74 -81 -88 0 12 -120 -28 127 83 73 80 47 50 46 48 47 84 67 80 32 91 50 48 48 49 58 53 48 54 58 52 48 48 48 58 48 58 50 48 49 48 58 48 58 55 56 58 49 49 93 58 49 51 55 48 59 114 101 99 101 105 118 101 100 61 50 48 48 49
File Output: ^#^#q<9c><82>Ê^# ^N^A^#^#^AÆ<6^#^C?þUJ¯¨^# ^A^XUJ¯Ð^#^MA<8f>UJ¯¨^#^L}<82>UJ¯¨^#^M^UaUJ¯¨^#^M^_^SUJ¯Ð^#^M*^X^#^F^#^#^#^#^#^#^#^#^#^# ^#^G<9c>^#û^F<80>^#È^]<81>^Wpÿÿ^#^##^#^A<87>^\sip:+14222099915#one.att.net^\sip:+14222099965#one.att.net%2965-150506232702-2050662766#00780011^P ^A^E^F#^#^#^# ^P^#^#^#x^#^Q^P ^A^X<90>^P^A.^B^#^#^#^#^#^#^#^F^A<8f>^#^D^#!ÀÖ^#[^E^H^# ^#È^#^#^#^O^CUJ¯¨^#^L<88>ä^H^# ^#È^#^#^#^O^CUJ¯¨^#^LÔÙ^H^#^D^#È^#^#^A^K^CUJ¯¨^# ^A^X^H^#^E^#^#^#^#^#^#^CUJ¯¨^#^M^_^S^H^#^A^#È^#^#^#^F^CUJ¯Ð^#^M*^XÀ"^D^X YSIP/2.0/TCP [2001:506:4000:0:2010:0:78:11]:1370;branch=z9hG4bK290-00780011-000f-441971403^CUJ¯¨^#^L<88>ä^?SIP/2.0/TCP [2001:506:4000:0:2010:0:78:11]:1370;received=2001
Please let me know at what step I'm making a mistake.

The other answers here tell you to use a PrintWriter or a FileWriter instead of the FileOutputStream but I'm fairly sure that this is not what you want.
Your problem is that you're writing raw bytes to a file and then reading it back as characters and comparing that to byte values represented as characters and then printed with System.out.
Let's take a look at what happens when you print a byte with the value 65 (or 01000001 in binary).
When you use System.out.print you will invoke PrintStream.print(int) with the integer value of 65 which will in turn print the characters 6 and 5 to the terminal.
When you use out.write you will invoke FileOutputStream.write(byte[]) which will write the bits 01000001 to the file.
Later, when you check the contents of the file your tool will try to interpret this byte as a character and it will most likely use the ASCII encoding to do so (even if you're using Unicode as your default encoding this is likely what will happen since Unicode is a superset of ASCII). This results in the character A being printed.
If you want to view the output file in a way similar to what you've printed with System.out.print you can use the following command on linux:
$ hexdump -e '/1 "%i "' <file>
Example:
$ cat /etc/issue
Ubuntu 12.04.5 LTS \n \l
$ hexdump -e '/1 "%i "' /etc/issue
85 98 117 110 116 117 32 49 50 46 48 52 46 53 32 76 84 83 32 92 110 32
92 108 10 *

My first answer was wrong, so I am editing this because I made the assumption that you could write out a string to the FileOutputStream, but I don't think that is the case. FileOutputStream is only used for byte streams, so you've got to stick to that format when writing out to the file.
If you hold the data in a buffer[array], and then write those bytes out to a file that you have created using the output stream, it should work. I found this document that might be helpful.
The main idea is that somewhere in your code, the byte array isn't getting written to the file correctly. Perhaps its just a matter of adding the close() method.
out.close();
server.close();
reading and writing files in java
Here is the section I found useful.
import java.io.*;
public class Test {
public static void main(String [] args) {
// The name of the file to create.
String fileName = "temp.txt";
try {
// Put some bytes in a buffer so we can
// write them. Usually this would be
// image data or something. Or it might
// be unicode text.
String bytes = "Hello theren";
byte[] buffer = bytes.getBytes();
FileOutputStream outputStream =
new FileOutputStream(fileName);
// write() writes as many bytes from the buffer
// as the length of the buffer. You can also
// use
// write(buffer, offset, length)
// if you want to write a specific number of
// bytes, or only part of the buffer.
outputStream.write(buffer);
// Always close files.
outputStream.close();
System.out.println("Wrote " + buffer.length +
" bytes");
}
catch(IOException ex) {
System.out.println(
"Error writing file '"
+ fileName + "'");
// Or we could just do this:
// ex.printStackTrace();
}
}
}

The console (System.out) is a PrintWriter, while the file output is a FileOutputStream.
The basic difference between Stream and Writer: Streams are supposed to manipulate "raw data", like numbers taken directly from binary format, while writers are used to manipulate "human-readable data", transforming all the data you write.
For example, the 6 int is different from the 6 character. When you use a stream, you write directly the int, while with a writer, the data wrriten is transformed into the character.
Then, if you want your file output to be the same as your console output, do not use FileOutputStream, but instead, use FileWriter, and it's method write(String).
How to make this work:
1 - replace out = new FileOutputStream("output.txt"); by out = new FileWriter("output.txt");
2 - replace out.write(curBytes); by:
for (byte b : curBytes) {
out.write(b + " ");
}

I would suggest you use IOUtils.copy and use a BufferedReader
to wrap your InputStream.
The output stream should obviously be FileOutputStream
I hope this helps

compile-time error: illegal Character: \8279

When I try to compile my servlet I get folowing exception:
illegal character: \8279
And it's pointing to &
msg.setContent("<a href=\"" + server +
":8080/myApp/ResetPasswordPage.jsp?randNum=" + randNum + ‌​
"&practiceName=" + practiceName+"\" Click Here </a>",
"text/html" );
I can't find a whole lot on the net about it...

I tried to copy this String to a java file in Eclipse. When I tried to save it I got :
There are 2 problematic invisible characters just after randNum +.
Remove them.

This is a dump of a copy-and-paste of your code:
00000010 3c 61 20 68 72 65 66 3d 5c 22 22 20 2b 20 73 65 |<a href=\"" + se|
00000020 72 76 65 72 20 2b 20 0a 20 20 20 20 20 20 20 20 |rver + . |
00000030 20 20 20 20 20 20 20 22 3a 38 30 38 30 2f 6d 79 | ":8080/my|
00000040 41 70 70 2f 52 65 73 65 74 50 61 73 73 77 6f 72 |App/ResetPasswor|
00000050 64 50 61 67 65 2e 6a 73 70 3f 72 61 6e 64 4e 75 |dPage.jsp?randNu|
00000060 6d 3d 22 20 2b 20 72 61 6e 64 4e 75 6d 20 2b 20 |m=" + randNum + |
00000070 e2 80 8c e2 80 8b 0a 20 20 20 20 20 20 20 20 20 |....... |
00000080 20 20 20 20 20 20 22 26 70 72 61 63 74 69 63 65 | "&practice|
00000090 4e 61 6d 65 3d 22 20 2b 20 70 72 61 63 74 69 63 |Name=" + practic|
000000a0 65 4e 61 6d 65 2b 22 5c 22 20 43 6c 69 63 6b 20 |eName+"\" Click |
Note the e2 80 8c and e2 80 8b between randNum + and the next line. You need to remove those.

Read Java in as Hex

I have tried to solve this but I keep coming up with stuff that is no help I'm sure this is easy (when you know how of course ;) )
What I would like to do is read in a file using a byte stream like below:
while((read = in.read()) != -1){
//code removed to save space
Integer.toHexString(read);
System.out.println(read);
}
When it prints out the Hex to the screen it will print out numbers fine e.g
31
13
12
0
but when it comes to a hex code that should be 01 31 it will print 0 131. I want to read it in to a variable like you would see in a hex editor i.e 00 11 21 31 no single numbers as i need to scan the whole file and look for patterns which I know how to do I'm just stuck on this :/
so in short i need a variabe to contain the two hex characters i.e int temp = 01 not int temp = 0 , I hope this all makes sense, I'm a little confused as it's 3am!
If anyone knows how to do this I would be most greatful, p.s thanks for the help in advance this site has saved me loads of research and have learnt a lot!
Many thanks.

This method :
public static void printHexStream(final InputStream inputStream, final int numberOfColumns) throws IOException{
long streamPtr=0;
while (inputStream.available() > 0) {
final long col = streamPtr++ % numberOfColumns;
System.out.printf("%02x ",inputStream.read());
if (col == (numberOfColumns-1)) {
System.out.printf("\n");
}
}
}
will output something like this :
40 32 38 00 5f 57 69 64 65 43
68 61 72 54 6f 4d 75 6c 74 69
42 79 74 65 40 33 32 00 5f 5f
69 6d 70 5f 5f 44 65 6c 65 74
65 46 69 6c 65 41 40 34 00 5f
53 65 74 46 69 6c 65 50 6f 69
6e 74 65 72 40 31 36 00 5f 5f
69 6d 70 5f 5f 47 65 74 54 65
6d 70 50 61 74 68 41 40 38 00
Is it what you are looking for?

I think what you're looking for is a formatter. Try:
Formatter formatter = new Formatter();
formatter.format("%02x", your_int);
System.out.println(formatter.toString());
Does that do what you're looking for? Your question wasn't all that clear (and I think maybe you deleted too much code from your snippet).

import org.apache.commons.io.IOUtils;
import org.apache.commons.codec.binary.Hex;
InputStream is = new FileInputStream(new File("c:/file.txt"));
String hexString = Hex.encodeHexString(IOUtils.toByteArray(is));
In java 7 you can read byte array directly from file as below :
import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.Path;
Path path = Paths.get("path/to/file");
byte[] data = Files.readAllBytes(path)

Hi everyone one posted, thanks for the reply but the way I eneded up doing it was:
hexIn = in.read();
s = Integer.toHexString(hexIn);
if(s.length() < 2){
s = "0" + Integer.toHexString(hexIn);
}
Just thought I would post they way I did it for anyone else in future, thank you soo much for your help though!