We Keep Coding

Best way to get Amazon page and product information

I want to get Amazon page and product information from their website so I work on a future project. I have no experience with APIs but also saw that I would need to pay in order to use Amazon's. My current plan was to use a WebRequest class which basically takes down the page's raw text and then parse through it to get what I need. It pulls down HTML from all the websites I have tried except amazon. When I try and use it for amazon I get text like this...
??èv~-1?½d!Yä90û?¡òk6??ªó?l}L??A?{í??j?ì??ñF Oü?ª[D ú7W¢!?É?L?]â  v??ÇJ???t?ñ?j?^,Y£>O?|?I`OöN??Q?»bÇJPy1·¬Ç??RtâU??Q%vB??^íè|??ª?
Can someone explain to me why this happens? Or even better if you could point me towards a better way of doing this? Any help is appreciated.
This is the class I mentioned...
public class WebRequest {
protected String url;
protected ArrayList<String> pageText;
public WebRequest() {
url = "";
pageText = new ArrayList<String>();
}
public WebRequest(String url) {
this.url = url;
pageText = new ArrayList<String>();
load();
}
public boolean load() {
boolean returnValue = true;
try {
URL thisURL = new URL(url);
BufferedReader reader = new BufferedReader(new InputStreamReader(thisURL.openStream()));
String line;
while ((line = reader.readLine()) != null) {
pageText.add(line);
}
reader.close();
}
catch (Exception e) {
returnValue = false;
System.out.println("peepee");
}
return returnValue;
}
public boolean load(String url) {
this.url = url;
return load();
}
public String toString() {
String returnString = "";
for (String s : pageText) {
returnString += s + "\n";
}
return returnString;
}
}

It could be that the page is returned using a different character encoding than your platform default. If that's the case, you should specify the appropriate encoding, e.g:
new InputStreamReader(thisURL.openStream(), "UTF-8")
But that data doesn't look like character data at all to me. It's too random. It looks like binary data. Are you sure you're not downloading an image by mistake?
If you want to make more sophisticated HTTP requests, there are quite a few Java libraries, e.g. OkHttp and AsyncHttpClient.
But it's worth bearing in mind that Amazon probably doesn't like people scraping its site, and will have built in detection of malicious or unwanted activity. It might be sending you gibberish on purpose to deter you from continuing. You should be careful because some big sites may block your IP temporarily or permanently.
My advice would be to learn how to use the Amazon APIs. They're pretty powerful—and you won't get yourself banned.

How do I send nested json POST request in java using jersey?

I am using a document converter api called cloudconvert. They don't have an official java library, but a third party java option. I needed a little customization so I cloned the github project and added it to my project. I am sending cloudconvert a .epub file and getting a .pdf file in return. If I use the default settings it works without issue and properly converts my .epub to a .pdf. Here is the code that makes it happen.
Here is what triggers the conversion:
// Create service object
CloudConvertService service = new CloudConvertService("api-key");
// Create conversion process
ConvertProcess process = service.startProcess(convertFrom, convertTo);
// Perform conversion
//convertFromFile is a File object with a .epub extension
process.startConversion(convertFromFile);
// Wait for result
ProcessStatus status;
waitLoop:
while (true) {
status = process.getStatus();
switch (status.step) {
case FINISHED:
break waitLoop;
case ERROR:
throw new RuntimeException(status.message);
}
// Be gentle
Thread.sleep(200);
}
//Download result
service.download(status.output.url, convertToFile);
//lean up
process.delete();
startConversion() calls:
public void startConversion(File file) throws ParseException, FileNotFoundException, IOException {
if (!file.exists()) {
throw new FileNotFoundException("File not found: " + file);
}
startConversion(new FileDataBodyPart("file", file));
}
Which calls this to actually send the POST request using jersey:
private void startConversion(BodyPart bodyPart) {
if (args == null) {
throw new IllegalStateException("No conversion arguments set.");
}
MultiPart multipart = new FormDataMultiPart()
.field("input", "upload")
.field("outputformat", args.outputformat)
.bodyPart(bodyPart);
//root is a class level WebTarget object
root.request(MediaType.APPLICATION_JSON).post(Entity.entity(multipart, multipart.getMediaType()));
}
Up to this point everything is working. My problem is that the when the conversion happens the .pdf that returns has very small margins. cloudconvert provides a way to change those margins. You can send in an optional json param converteroptions and set the margins manually. I have tested this out using postman and it works without issue, I was able to get a properly formatted margin document. So know this is possible. Here is the POSTMAN info I used:
#POST : https://host123d1qo.cloudconvert.com/process/WDK9Yq0z1xso6ETgvpVQ
Headers: 'Content-Type' : 'application/json'
Body:
{
"input": "base64",
"file": "0AwAAIhMAAAAA", //base64 file string that is much longer than this
"outputformat": "pdf",
"converteroptions": {
"margin_bottom": 75,
"margin_top": 75,
"margin_right": 50,
"margin_left": 50
}
}
Here are my attempts at getting the POST request formatted properly, I'm just not very experienced with jersey and the couple of answers I did find on stackoverflow didn't work for me.
Attempt 1, I tried adding the json string as a Multipart.field. It didn't give me any errors and still returned a converted .pdf file, but the margins didn't get changed so I must not be sending it back right.
private void startConversion(BodyPart bodyPart) {
String jsonString = "{\"margin_bottom\":75,\"margin_top\":75,\"margin_right\":50,\"margin_left\":50}";
MultiPart multipart = new FormDataMultiPart()
.field("input", "upload")
.field("outputformat", args.outputformat)
.field("converteroptions", jsonString)
.bodyPart(bodyPart);
root.request(MediaType.APPLICATION_JSON).post(Entity.entity(multipart, multipart.getMediaType()));
}
Attempt 2, when I had it working in POSTMAN it was using the 'input' type as 'base64' so I tried changing it to that but it this time it doesn't return anything at all, no request errors, just a timeout error at the 5 minute mark.
//I pass in a File object rather than the bodypart object.
private void startConversion(File file) {
byte[] encoded1 = Base64.getEncoder().encode(FileUtils.readFileToByteArray(file));
String encoded64 = new String(encoded1, StandardCharsets.US_ASCII);
String jsonString = "{\"margin_bottom\":75,\"margin_top\":75,\"margin_right\":50,\"margin_left\":50}";
MultiPart multipart = new FormDataMultiPart()
.field("input", "base64")
.field("outputformat", args.outputformat)
.field("file", encoded64)
.field("converteroptions", jsonString);
root.request(MediaType.APPLICATION_JSON).post(Entity.entity(multipart, multipart.getMediaType()));
}
Attempt 3, after some googling on how to properly send jersey json post requests I changed the format. This time it returned a 400 bad request error.
private void startConversionPDF(File file) throws IOException {
byte[] encoded1 = Base64.getEncoder().encode(FileUtils.readFileToByteArray(file));
String encoded64 = new String(encoded1, StandardCharsets.US_ASCII);
String jsonString = "{\"input\":\"base64\",\"file\":\"" + encoded64 + "\",\"outputformat\":\"pdf\",\"converteroptions\":{\"margin_bottom\":75,\"margin_top\":75,\"margin_right\":50,\"margin_left\":50}}";
root.request(MediaType.APPLICATION_JSON).post(Entity.json(jsonString));
}
Attempt 4, Someone said you don't need to manually use a jsonString you should use serializable java beans. So I created the corresponding classes and made the request like shown below. Same 400 bad request error.
#XmlRootElement
public class PDFConvert implements Serializable {
private String input;
private String file;
private String outputformat;
private ConverterOptions converteroptions;
//with the a default constructor and getters/setters for all
}
#XmlRootElement
public class ConverterOptions implements Serializable {
private int margin_bottom;
private int margin_top;
private int margin_left;
private int margin_right;
//with the a default constructor and getters/setters for all
}
private void startConversionPDF(File file) throws IOException {
byte[] encoded1 = Base64.getEncoder().encode(FileUtils.readFileToByteArray(file));
String encoded64 = new String(encoded1, StandardCharsets.US_ASCII);
PDFConvert data = new PDFConvert();
data.setInput("base64");
data.setFile(encoded64);
data.setOutputformat("pdf");
ConverterOptions converteroptions = new ConverterOptions();
converteroptions.setMargin_top(75);
converteroptions.setMargin_bottom(75);
converteroptions.setMargin_left(50);
converteroptions.setMargin_right(50);
data.setConverteroptions(converteroptions);
root.request(MediaType.APPLICATION_JSON).post(Entity.json(data));
}
I know this is quite the wall of text, but I wanted to show all the different things I tried so that I wouldn't waste anyone's time. Thank you for any help or ideas you might have to make this work. I really want to make it work with jersey because I have several other conversions I do that work perfectly, they just don't need any converteroptions. Also I know its possible because it works when manually running the process through POSTMAN.
Cloudconvert api documentation for starting a conversion
Github repo with the recommended 3rd party java library I am using/modifying

I finally figured it out. Hours of trial and error. Here is the code that did it:
private void startConversionPDF(File file) throws IOException {
if (args == null) {
throw new IllegalStateException("No conversion arguments set.");
}
PDFConvert data = new PDFConvert();
data.setInput("upload");
data.setOutputformat("pdf");
ConverterOptions converteroptions = new ConverterOptions();
converteroptions.setMargin_top(60);
converteroptions.setMargin_bottom(60);
converteroptions.setMargin_left(30);
converteroptions.setMargin_right(30);
data.setConverteroptions(converteroptions);
MultiPart multipart = new FormDataMultiPart()
.bodyPart(new FormDataBodyPart("json", data, MediaType.APPLICATION_JSON_TYPE))
.bodyPart(new FileDataBodyPart("file", file));
root.request(MediaType.APPLICATION_JSON).post(Entity.entity(multipart, multipart.getMediaType()));
}

FTPS storeFile return always false Java

im trying to send files to FTPS server
connection method: FTPS, ACTIVE, EXPLICIT
setFileType(FTP.BINARY_FILE_TYPE);
setFileTransferMode(FTP.BLOCK_TRANSFER_MODE);
Checking the reply string right after connect i got:
234 AUTH command ok. Expecting TLS Negotiation.
from here
234 Specifies that the server accepts the authentication mechanism specified by the client, and the exchange of security data is complete. A higher level nonstandard code created by Microsoft.
while trying to send file with storeFile or storeUniqeFile i get false
checking the reply string right after store file i got: 501 Server cannot accept argument.
what is weird i was able creating a directory to this client without any issues
with makeDirectory("test1");
i was trying both this links : link1 , link2
FOR EXAMPLE when i was trying to use ftp.enterLocalPassiveMode(); before ftp.storeFile(destinationfile, in);
i got time out error .
Does anyone have any idea how to solve it ?
public static void main(String[] args) throws Exception {
FTPSProvider ftps = new FTPSProvider();
String json = "connection details";
DeliveryDetailsFTPS details = gson.fromJson(json, DeliveryDetailsFTPS .class);
File file = File.createTempFile("test", ".txt");
FileUtils.write(file, " some test", true);
try (FileInputStream stream = new FileInputStream(file)) {
ftps.sendInternal(ftps.getClient(details), details, stream, file.getName());
}
}
protected void sendInternal(FTPClient client, DeliveryDetailsFTPS details, InputStream stream, String filename) throws Exception {
try {
// release the enc
DeliveryDetailsFTPS ftpDetails = (DeliveryDetailsFTPS) details;
setClient(client, ftpDetails);
boolean isSaved = false;
try (BufferedInputStream bis = new BufferedInputStream(stream)) {
isSaved = client.storeFile(filename, bis);
}
client.makeDirectory("test1");
client.logout();
if (!isSaved) {
throw new IOException("Unable to upload file to FTP");
}
} catch (Exception ex) {
LOG.debug("Unable to send to FTP", ex);
throw ex;
} finally {
client.disconnect();
}
}
#Override
protected FTPClient getClient(DeliveryDetails details) {
return new FTPSClient(isImplicitSSL((DeliveryDetailsFTPS ) details));
}
protected void setClient(FTPClient client, DeliveryDetailsFTPS details) throws Exception {
DeliveryDetailsFTPS ftpDetails = (DeliveryDetailsFTPS ) details;
client.setConnectTimeout(100000);
client.setDefaultTimeout(10000 * 60 * 2);
client.setControlKeepAliveReplyTimeout(300);
client.setControlKeepAliveTimeout(300);
client.setDataTimeout(15000);
client.connect(ftpDetails.host, ftpDetails.port);
client.setBufferSize(1024 * 1024);
client.login(ftpDetails.username, ftpDetails.getSensitiveData());
client.setControlEncoding("UTF-8");
int code = client.getReplyCode();
if (code == 530) {
throw new IOException(client.getReplyString());
}
// Set binary file transfer
client.setFileType(FTP.BINARY_FILE_TYPE);
client.setFileTransferMode(FTP.BLOCK_TRANSFER_MODE);
if (ftpDetails.ftpMode == FtpMode.PASSIVE) {
client.enterLocalPassiveMode();
}
client.changeWorkingDirectory(ftpDetails.path);
}
I have tried this solution as well didn't solve the problem:
they only way i was able send file is with FileZilla and it is using FTPES .
But i need my Java code to do it . can anyone give me a clue

I have tried almost any possible solution offered on different websites could not make it work with Apache FTPS CLIENT ,
had to use a different class which worked like a charm here is a snippet:
com.jscape.inet.ftps Link
private Ftps sendWithFtpsJSCAPE(ConnDetails details, InputStream stream, String filename) throws FtpException, IOException {
Ftps ftp;
FtpConnectionDetails ftpDetails = FtpConnectionDetails details;
ftp = new Ftps(ftpDetails.getHost(), ftpDetails.getUsername(), ftpDetails.getPassword());
if (ftpDetails.getSecurityMode().equals(FtpConnectionDetails.SecurityMode.EXPLICIT)) {
ftp.setConnectionType(Ftps.AUTH_TLS);
} else {
ftp.setConnectionType(Ftps.IMPLICIT_SSL);
}
ftp.setPort(ftpDetails.getPort());
if (!ftpDetails.getFtpMode().equals(FtpMode.ACTIVE)) {
ftp.setPassive(true);
}
ftp.setTimeout(FTPS_JSCAPE_TIME_OUT);
ftp.connect();
ftp.setBinary();
ftp.setDir(ftpDetails.getPath());
ftp.upload(stream, filename);
return ftp;
}

HttpURLConnection Invalid HTTP method: PATCH

When I try to use a non-standard HTTP Method like PATCH with URLConnection:
HttpURLConnection conn = (HttpURLConnection) new URL("http://example.com").openConnection();
conn.setRequestMethod("PATCH");
I get an exception:
java.net.ProtocolException: Invalid HTTP method: PATCH
at java.net.HttpURLConnection.setRequestMethod(HttpURLConnection.java:440)
Using a higher level API like Jersey generates the same error. Is there a workaround to issue a PATCH HTTP request?

There are a lot of good answers, so here is mine (not work in jdk12):
import java.io.IOException;
import java.lang.reflect.Field;
import java.lang.reflect.Modifier;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.Arrays;
import java.util.LinkedHashSet;
import java.util.Set;
public class SupportPatch {
public static void main(String... args) throws IOException {
allowMethods("PATCH");
HttpURLConnection conn = (HttpURLConnection) new URL("http://example.com").openConnection();
conn.setRequestMethod("PATCH");
}
private static void allowMethods(String... methods) {
try {
Field methodsField = HttpURLConnection.class.getDeclaredField("methods");
Field modifiersField = Field.class.getDeclaredField("modifiers");
modifiersField.setAccessible(true);
modifiersField.setInt(methodsField, methodsField.getModifiers() & ~Modifier.FINAL);
methodsField.setAccessible(true);
String[] oldMethods = (String[]) methodsField.get(null);
Set<String> methodsSet = new LinkedHashSet<>(Arrays.asList(oldMethods));
methodsSet.addAll(Arrays.asList(methods));
String[] newMethods = methodsSet.toArray(new String[0]);
methodsField.set(null/*static field*/, newMethods);
} catch (NoSuchFieldException | IllegalAccessException e) {
throw new IllegalStateException(e);
}
}
}
It also uses reflection, but instead of hacking into every connection object we're hacking HttpURLConnection#methods static field which is used in the checks internally.

Yes there is workaround for this. Use
X-HTTP-Method-Override
. This header can be used in a POST request to “fake” other HTTP methods. Simply set the value of the X-HTTP-Method-Override header to the HTTP method you would like to actually perform.
So use following code.
conn.setRequestProperty("X-HTTP-Method-Override", "PATCH");
conn.setRequestMethod("POST");

There is a Won't Fix bug in OpenJDK for this: https://bugs.openjdk.java.net/browse/JDK-7016595
However, with Apache Http-Components Client 4.2+ this is possible. It has a custom networking implementation, thus using all standard HTTP methods like PATCH is possible. It even has a HttpPatch class supporting the patch method.
CloseableHttpClient httpClient = HttpClients.createDefault();
HttpPatch httpPatch = new HttpPatch(new URI("http://example.com"));
CloseableHttpResponse response = httpClient.execute(httpPatch);
Maven Coordinates:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.2+</version>
</dependency>

If the project is on Spring/Gradle; the following solution will workout.
For the build.gradle, add the following dependency;
compile('org.apache.httpcomponents:httpclient:4.5.2')
And define the following bean in your #SpringBootApplication class inside the
com.company.project;
#Bean
public RestTemplate restTemplate() {
HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory();
requestFactory.setReadTimeout(600000);
requestFactory.setConnectTimeout(600000);
return new RestTemplate(requestFactory);
}
This solutions worked for me.

In java 11+ you can use the HttpRequest class to do what you want:
import java.net.http.HttpRequest;
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create(uri))
.method("PATCH", HttpRequest.BodyPublishers.ofString(message))
.header("Content-Type", "text/xml")
.build();

Reflection as described in this post and a related post does not work if you are using a HttpsURLConnection on Oracle's JRE, becausesun.net.www.protocol.https.HttpsURLConnectionImpl is using the method field from the java.net.HttpURLConnection of its DelegateHttpsURLConnection!
So a complete working solution is:
private void setRequestMethod(final HttpURLConnection c, final String value) {
try {
final Object target;
if (c instanceof HttpsURLConnectionImpl) {
final Field delegate = HttpsURLConnectionImpl.class.getDeclaredField("delegate");
delegate.setAccessible(true);
target = delegate.get(c);
} else {
target = c;
}
final Field f = HttpURLConnection.class.getDeclaredField("method");
f.setAccessible(true);
f.set(target, value);
} catch (IllegalAccessException | NoSuchFieldException ex) {
throw new AssertionError(ex);
}
}

Using the answer:
HttpURLConnection Invalid HTTP method: PATCH
I'm created a sample request and work like a charm:
public void request(String requestURL, String authorization, JsonObject json) {
try {
URL url = new URL(requestURL);
httpConn = (HttpURLConnection) url.openConnection();
httpConn.setRequestMethod("POST");
httpConn.setRequestProperty("X-HTTP-Method-Override", "PATCH");
httpConn.setRequestProperty("Content-Type", "application/json");
httpConn.setRequestProperty("Authorization", authorization);
httpConn.setRequestProperty("charset", "utf-8");
DataOutputStream wr = new DataOutputStream(httpConn.getOutputStream());
wr.writeBytes(json.toString());
wr.flush();
wr.close();
httpConn.connect();
String response = finish();
if (response != null && !response.equals("")) {
created = true;
}
}
catch (Exception e) {
e.printStackTrace();
}
}
public String finish() throws IOException {
String response = "";
int status = httpConn.getResponseCode();
if (status == HttpURLConnection.HTTP_OK || status == HttpURLConnection.HTTP_CREATED) {
BufferedReader reader = new BufferedReader(new InputStreamReader(
httpConn.getInputStream()));
String line = null;
while ((line = reader.readLine()) != null) {
response += line;
}
reader.close();
httpConn.disconnect();
} else {
throw new IOException("Server returned non-OK status: " + status);
}
return response;
}
I hope it help you.

I had the same exception and wrote sockets solution (in Groovy) but I translate in the answer form to Java for you:
String doInvalidHttpMethod(String method, String resource){
Socket s = new Socket(InetAddress.getByName("google.com"), 80);
PrintWriter pw = new PrintWriter(s.getOutputStream());
pw.println(method +" "+resource+" HTTP/1.1");
pw.println("User-Agent: my own");
pw.println("Host: google.com:80");
pw.println("Content-Type: */*");
pw.println("Accept: */*");
pw.println("");
pw.flush();
BufferedReader br = new BufferedReader(new InputStreamReader(s.getInputStream()));
String t = null;
String response = "";
while((t = br.readLine()) != null){
response += t;
}
br.close();
return response;
}
I think it works in Java. You have to change the server and port number remember change the Host header too and maybe you have to catch some exception.

For anyone using Spring restTemplate looking for a detailed answer.
You will face the problem if you are using SimpleClientHttpRequestFactory as your restTemplate's ClientHttpRequestFactory.
From java.net.HttpURLConnection:
/* valid HTTP methods */
private static final String[] methods = {
"GET", "POST", "HEAD", "OPTIONS", "PUT", "DELETE", "TRACE"
};
As PATCH is not a supported operation, this line of code from the same class will execute:
throw new ProtocolException("Invalid HTTP method: " + method);
I ended up using the same as what #hirosht suggested in his answer.

Another dirty hack solution is reflexion:
private void setVerb(HttpURLConnection cn, String verb) throws IOException {
switch (verb) {
case "GET":
case "POST":
case "HEAD":
case "OPTIONS":
case "PUT":
case "DELETE":
case "TRACE":
cn.setRequestMethod(verb);
break;
default:
// set a dummy POST verb
cn.setRequestMethod("POST");
try {
// Change protected field called "method" of public class HttpURLConnection
setProtectedFieldValue(HttpURLConnection.class, "method", cn, verb);
} catch (Exception ex) {
throw new IOException(ex);
}
break;
}
}
public static <T> void setProtectedFieldValue(Class<T> clazz, String fieldName, T object, Object newValue) throws Exception {
Field field = clazz.getDeclaredField(fieldName);
field.setAccessible(true);
field.set(object, newValue);
}

You can find a detailed solution that can work even if you don't have direct access to the HttpUrlConnection (like when working with Jersey Client here: PATCH request using Jersey Client

If your server is using ASP.NET Core, you can simply add the following code to specify the HTTP method using the header X-HTTP-Method-Override, as described in the accepted answer.
app.Use((context, next) => {
var headers = context.Request.Headers["X-HTTP-Method-Override"];
if(headers.Count == 1) {
context.Request.Method = headers.First();
}
return next();
});
Simply add this code in Startup.Configure before your call to app.UseMvc().

In emulator of API 16 I received an exception: java.net.ProtocolException: Unknown method 'PATCH'; must be one of [OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE].
While an accepted answer works, I want to add one detail. In new APIs PATCH works well, so in conjunction with https://github.com/OneDrive/onedrive-sdk-android/issues/16 you should write:
if (method.equals("PATCH") && Build.VERSION.SDK_INT <= Build.VERSION_CODES.KITKAT) {
httpConnection.setRequestProperty("X-HTTP-Method-Override", "PATCH");
httpConnection.setRequestMethod("POST");
} else {
httpConnection.setRequestMethod(method);
}
I changed JELLY_BEAN_MR2 to KITKAT after testing in API 16, 19, 21.

I got mine with Jersey client.
The workaround was:
Client client = ClientBuilder.newClient();
client.property(HttpUrlConnectorProvider.SET_METHOD_WORKAROUND, true);

We have faced the same problem with slightly different behavior. We were using apache cxf library for making the rest calls.
For us, PATCH was working fine till we were talking to our fake services which were working over http.
The moment we integrated with actual systems (which were over https) we started facing the same issue with following stack trace.
java.net.ProtocolException: Invalid HTTP method: PATCH at java.net.HttpURLConnection.setRequestMethod(HttpURLConnection.java:428) ~[na:1.7.0_51] at sun.net.www.protocol.https.HttpsURLConnectionImpl.setRequestMethod(HttpsURLConnectionImpl.java:374) ~[na:1.7.0_51] at org.apache.cxf.transport.http.URLConnectionHTTPConduit.setupConnection(URLConnectionHTTPConduit.java:149) ~[cxf-rt-transports-http-3.1.14.jar:3.1.14]
Issue was happening in this line of code
connection.setRequestMethod(httpRequestMethod); in URLConnectionHTTPConduit class of cxf library
Now the real reason for the failure is that
java.net.HttpURLConnection contains a methods variable which looks like below
/* valid HTTP methods */
private static final String[] methods = {
"GET", "POST", "HEAD", "OPTIONS", "PUT", "DELETE", "TRACE"
};
And we can see that there is no PATCH method defined hence the error made sense.
We tried lots of different thing and looked over stack overflow. The only reasonable answer was to use reflection to modify the methods variable to inject another value "PATCH". But somehow we were not convinced to use that as the solution was kind of hack and is too much work and might have impact as we had common library to make all connection and performing these REST calls.
But then we realized that cxf library itself is handling the exception and there is code written in the catch block to add the missing method using reflection.
try {
connection.setRequestMethod(httpRequestMethod);
} catch (java.net.ProtocolException ex) {
Object o = message.getContextualProperty(HTTPURL_CONNECTION_METHOD_REFLECTION);
boolean b = DEFAULT_USE_REFLECTION;
if (o != null) {
b = MessageUtils.isTrue(o);
}
if (b) {
try {
java.lang.reflect.Field f = ReflectionUtil.getDeclaredField(HttpURLConnection.class, "method");
if (connection instanceof HttpsURLConnection) {
try {
java.lang.reflect.Field f2 = ReflectionUtil.getDeclaredField(connection.getClass(),
"delegate");
Object c = ReflectionUtil.setAccessible(f2).get(connection);
if (c instanceof HttpURLConnection) {
ReflectionUtil.setAccessible(f).set(c, httpRequestMethod);
}
f2 = ReflectionUtil.getDeclaredField(c.getClass(), "httpsURLConnection");
HttpsURLConnection c2 = (HttpsURLConnection)ReflectionUtil.setAccessible(f2)
.get(c);
ReflectionUtil.setAccessible(f).set(c2, httpRequestMethod);
} catch (Throwable t) {
//ignore
logStackTrace(t);
}
}
ReflectionUtil.setAccessible(f).set(connection, httpRequestMethod);
message.put(HTTPURL_CONNECTION_METHOD_REFLECTION, true);
} catch (Throwable t) {
logStackTrace(t);
throw ex;
}
}
Now this gave us some hopes, so we spent some time in reading the code and found that if we provide a property for URLConnectionHTTPConduit.HTTPURL_CONNECTION_METHOD_REFLECTION then we can make cxf to execute the exception handler and our work is done as by default the variable will be assigned to false due to below code
DEFAULT_USE_REFLECTION =
Boolean.valueOf(SystemPropertyAction.getProperty(HTTPURL_CONNECTION_METHOD_REFLECTION, "false"));
So here is what we had to do to make this work
WebClient.getConfig(client).getRequestContext().put("use.httpurlconnection.method.reflection", true);
or
WebClient.getConfig(client).getRequestContext().put(HTTPURL_CONNECTION_METHOD_REFLECTION, true);
Where WebClient is from cxf library itself.
Hope this answer helps some one.

**CloseableHttpClient http = HttpClientBuilder.create().build();
HttpPatch updateRequest = new HttpPatch("URL");
updateRequest.setEntity(new StringEntity("inputjsonString", ContentType.APPLICATION_JSON));
updateRequest.setHeader("Bearer", "auth");
HttpResponse response = http.execute(updateRequest);
JSONObject result = new JSONObject(IOUtils.toString(response.getEntity().getContent()));**
maven plugin
> <dependency>
> <groupId>org.apache.httpcomponents</groupId>
> <artifactId>httpclient</artifactId>
> <version>4.3.4</version>
> <!-- Exclude Commons Logging in favor of SLF4j -->
> <exclusions>
> <exclusion>
> <groupId>commons-logging</groupId>
> <artifactId>commons-logging</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
use this really it would helps you

PayPal REST API java sdk - custom config file

Good afternoon all!
I use PayPal REST API java sdk and I want to have different configurations for different environments of my application. Here is how I'm trying to do so:
private static boolean IS_PRODUCTION = false;
private static String PAYPAL_ACCESS_TOKEN;
private static void initPayPal() {
InputStream is = null;
try {
is = ApplicationConfig.class.getResourceAsStream(
IS_PRODUCTION? "/my_paypal_sdk_config.properties" : "/my_paypal_sdk_config_test.properties");
PayPalResource.initConfig(is);
String clientID = ConfigManager.getInstance().getConfigurationMap().get("clientID");
String clientSecret = ConfigManager.getInstance().getConfigurationMap().get("clientSecret");
PAYPAL_ACCESS_TOKEN = new OAuthTokenCredential(clientID, clientSecret).getAccessToken();
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
IOUtils.closeQuietly(is);
}
}
and while trying to get the clientID I have
java.io.IOException: Resource 'sdk_config.properties' could not be found
Strange behavior - I thought I've just configured the sdk to use my own properties file.
Please advice how could I set up those settings properly!

So here is the solution I found:
Create an empty sdk_config.properties file in default location
Load your own properties:
private static void initPayPal() {
InputStream is = null;
try {
is = ApplicationConfig.class.getResourceAsStream(
IS_PRODUCTION ? "/my_paypal_sdk_config.properties" : "/my_paypal_sdk_config_test.properties");
Properties props = new Properties();
props.load(is);
PayPalResource.initConfig(props);
ConfigManager.getInstance().load(props);
String clientID = ConfigManager.getInstance().getConfigurationMap().get("clientID");
String clientSecret = ConfigManager.getInstance().getConfigurationMap().get("clientSecret");
PAYPAL_ACCESS_TOKEN = new OAuthTokenCredential(clientID, clientSecret).getAccessToken();
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
IOUtils.closeQuietly(is);
}
}

We have made some good improvements to the PayPal Java SDK on integration steps. We are removing the need for sdk_config.properties file as they do not work as well, specially for multi-configuration settings.
Now, all you do is create an APIContext instance with clientId, clientSecret, and mode. You pass that context object for any API operation from there on.
Here is how the code would look like for different configurations:
APIContext defaultContext = new APIContext(clientId1, clientSecret1, "sandbox");
APIContext sandboxContext = new APIContext(clientId2, clientSecret2, "sandbox");
APIContext someOtherContext = new APIContext(clientId3, clientSecret3, "live");
APIContext liveContext = new APIContext(clientId, clientSecret, "live");
// Now pass any of the above context in these calls, and it would use those configurations.
Payment payment = new Payment();
// Fill in all the details.
payment.create(defaultContext);
// Replace that defaultContext with any of those other contexts.
Here is the wiki page explaining that: https://github.com/paypal/PayPal-Java-SDK/wiki/Making-First-Call

I had the same error with SDK 0.11 version. I use my own properties file, but code still looked for "sdk_config.properties". I put it into root in my CLASSPATH, but still got the same error. Then I made obvious and horrible solution: put empty "sdk_config.properties" into "rest-api-sdk-0.11.0.jar" library. This street magic solved my problem.