I am trying to put file on S3 and read files from S3 buckets using JAVA but encounter Unable to execute HTTP request.
while i tried to list the buckets then it works fine.
to me it look like only listBuckets() method is working while all other methods like putObject(), listObjects(), createBucket() etc are throwing same error Unable to execute HTTP request
here is my code which i am working on to fix i have used many methods but none works other than listBuckets() --
package test;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.Bucket;
import com.amazonaws.services.s3.model.ObjectListing;
import com.amazonaws.services.s3.model.S3ObjectSummary;
import com.amazonaws.services.s3.model.PutObjectRequest;
import com.amazonaws.services.s3.model.ObjectMetadata;
import com.amazonaws.AmazonServiceException;
import com.amazonaws.SdkClientException;
public class JavaS3Conn {
//private static Logger log = Logger.getLogger(JavaS3Conn.class);
public static void main(String[] args) {
String access_key = "access_key";
String secret_key = "secret_key";
String end_point = "http://end_point:port/";
String bucketName = "bucketName";
String stringObjKeyName = "Hello_S3.txt";
String fileObjKeyName = "Hello_S3.txt";
String fileName = "C:\\Users\\Desktop\\hello_S3.txt";
try {
BasicAWSCredentials creds = new BasicAWSCredentials(access_key, secret_key);
ClientConfiguration clientConfig = new ClientConfiguration()
.withProxyHost("wdctestlab-ecs1-node1.systems.uk.hsbc").withProxyPort(9020)
.withNonProxyHosts("");
AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withClientConfiguration(clientConfig).withCredentials(new AWSStaticCredentialsProvider(creds)).withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(end_point,Regions.DEFAULT_REGION.getName())).build();
/*
if(s3Client.doesBucketExistV2(bucketName)) {
System.out.println("Bucket name is not available."
+ " Try again with a different Bucket name.");
return;
}*/
//s3Client.createBucket(bucketName);
ObjectListing objectListing = s3Client.listObjects(bucketName);
for(S3ObjectSummary os : objectListing.getObjectSummaries()) {
System.out.println(os.getKey());
}
/*
s3Client.putObject(
bucketName,
stringObjKeyName,
"Uploaded String Object"
);
*/
PutObjectRequest request = new PutObjectRequest(bucketName, fileObjKeyName, new File(fileName));
ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentType("plain/text");
metadata.addUserMetadata("x-amz-meta-title", "someTitle");
request.setMetadata(metadata);
//s3Client.putObject(request);
//s3Client.listObjects(bucketName);
for (Bucket bucket : s3Client.listBuckets()) {
System.out.println(" - " + bucket.getName());
}
} catch (AmazonServiceException e) {
e.printStackTrace();
} catch (SdkClientException e) {
e.printStackTrace();
}
}
}
Error --
com.amazonaws.SdkClientException: Unable to execute HTTP request: bucket.endpoint
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1175
)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1121)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4914)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4860)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4854)
at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:880)
at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:848)
at test.JavaS3Conn.main(JavaS3Conn.java:51)
Caused by: java.net.UnknownHostException: bucket.endpoint
at java.net.InetAddress.getAllByName0(InetAddress.java:1280)
at java.net.InetAddress.getAllByName(InetAddress.java:1192)
at java.net.InetAddress.getAllByName(InetAddress.java:1126)
at com.amazonaws.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:27)
at com.amazonaws.http.DelegatingDnsResolver.resolve(DelegatingDnsResolver.java:38)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:111)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76)
at com.amazonaws.http.conn.$Proxy3.connect(Unknown Source)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1297)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)
It seems like i have to add client config which resolve my the issue of unable to execute HTTP
ClientConfiguration clientConfig = new ClientConfiguration()
.withProxyHost("wdctestlab-ecs1-node1.systems.uk.hsbc").withProxyPort(9020)
.withNonProxyHosts("");
I am running s3 on a localstack (a devlocal s3 emulator) docker container on a Mac and got this error ("SdkClientException: Unable to execute HTTP request:bucketname.localhost", "UnknownHostException"). And listBuckets works.
Putting this in /etc/hosts fixed it for me:
127.0.0.1 bucketname.localhost
The following alternative solution also fixed it for me but this path-style-access will not work with real s3 buckets (as opposed to localstack s3 buckets) created after Sep 2020:
.withPathStyleAccessEnabled(true)
(added on the AmazonS3ClientBuilder setup line)
Why:
Note the UnknownHostException on bucket.endpoint - why is it trying to put the bucket in the DNS name?:
com.amazonaws.SdkClientException: Unable to execute HTTP request: bucket.endpoint
...
Caused by: java.net.UnknownHostException: bucket.endpoint
For me it was bucketname.localhost since I was trying to connect to a localstack S3 end-point running on localhost.
The new S3 bucket naming uses the virtual hosted style naming where the bucket goes on the front of the path. bucketname.s3.amazonaws.com not s3.amazonaws.com/bucketname
Virtual hosted style is the default.
AWS has deprecated path-style naming. All buckets created after September 2020 can't use path-style. More info:
https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story
localstack has implemented support for virtual host style but if you are running it locally in a docker container, you have to create host aliases for each bucket name. https://github.com/localstack/localstack/issues/2631
When running docker in linux (and using systemd-resolved - Fedora/Ubuntu do), it works. I think this is because the following line works in linux and not on mac: ping abc.localhost (where abc can be anything). This is called localhost subdomains.
For mac (or linux not using systemd-resolved), another option is adding lines to /etc/hosts or using brew install dnsmasq (dnsmasq is also on linux) - https://serverfault.com/questions/118378/in-my-etc-hosts-file-on-linux-osx-how-do-i-do-a-wildcard-subdomain#118589.
Linux systemd-resolved localhost-subdomain information source: https://unix.stackexchange.com/questions/401966/how-does-every-subdomain-of-localhost-point-to-localhost-on-fedora
I traced the virtual-host hostname addition to this spot: (sdk version 1.11)
AmazonS3Client.doesObjectExist
getObjectMetadata
createRequest
resolveRequestEndpoint
new S3RequestEndpointResolver(..)
S3RequestEndpointResolver.resolveRequestEndpoint
shouldUseVirtualAddressing
BucketNameUtils.isDNSBucketName
isValidV2BucketName
isValidV2BucketName
request.setEndpoint(convertToVirtualHostEndpoint(endpoint, bucketName));
Related
I have some code that unit tests fine using https.proxyHost and https.proxyPort to access Google Cloud Storage Buckets on on a corporate network which needs to go via a proxy:
log.info("resolving service....");
Storage storage = StorageOptions.newBuilder()
.setCredentials(
ServiceAccountCredentials.fromStream(
new FileInputStream(fullPath)))
.build()
.getService();
log.info("resolving bucket....");
bucket = storage.get(bucketName);
Yet when I run it in a larger app that starts a lot of other internal services (e.g., RMI) the proxy settings stop working.
Running as:
java -Dhttps.proxyHost=googleapis-dev.gcp.cloud.corporate -Dhttps.proxyPort=3128 ...
When it tries to resolve a bucket with the unit tested code it hangs for ages then throws:
com.google.cloud.storage.StorageException: Error getting access token for service account: oauth2.googleapis.com
at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:231) ~[htu-gcs-plugin.jar:?]
...
Caused by: java.io.IOException: Error getting access token for service account: oauth2.googleapis.com
at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:444) ~[?:?]
at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:157) ~[?:?]
...
Caused by: java.net.UnknownHostException: oauth2.googleapis.com
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184) ~[?:1.8.0_231]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[?:1.8.0_231]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.8.0_231]
at java.net.Socket.connect(Socket.java:606) ~[?:1.8.0_231]
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:666) ~[?:1.8.0_231]
I can get exactly the same error without proxy settings by adding this to the top of the main method:
String hostname = "https://oauth2.googleapis.com";
HttpURLConnection con = (HttpURLConnection) new URL(hostname).openConnection();
int code = con.getResponseCode();
System.out.println("> https://oauth2.googleapis.com returned: "+code);
Yet if the proxy settings are pass that runs fine then later throws the java.net.UnknownHostException as through the proxy settings are cleared.
To make things a bit more complex a custom URLClassLoader is used to load the code in question. Yet I have made a standalone app that uses the classloader and runs the code fine with the proxy settings passed as normal.
So it appears that something in the larger app is messing with the proxy system settings. Searching the codebase I can see no trace of that.
I have looked at https://googleapis.github.io/google-http-java-client/http-transport.html to see if there is way to plugin in a transport that has a proxy but cannot find a clear example.
Is there a way to coerce the use of a proxy when using google-cloud-storage?
To explicitly force a proxy to not rely upon the standard java System properties add the client libraries:
<!-- https://mvnrepository.com/artifact/com.google.http-client/google-http-client-apache-v2 -->
<dependency>
<groupId>com.google.http-client</groupId>
<artifactId>google-http-client-apache-v2</artifactId>
<version>1.37.0</version>
</dependency>
Then you can create a custom HttpTransportFactory with something like:
public class ProxyAwareTransportFactory implements HttpTransportFactory {
public static SSLContext trustAllSSLContext() throws Exception {
SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
sslContext.init(null, null, null);
return sslContext;
}
#Override
public HttpTransport create() {
InetSocketAddress socketAddress = new InetSocketAddress(this.host,this.port);
Proxy proxy = new Proxy(Proxy.Type.HTTP, socketAddress);
try {
return new NetHttpTransport.Builder()
.setProxy(proxy)
.setConnectionFactory(new DefaultConnectionFactory(proxy) )
.setSslSocketFactory(trustAllSSLContext().getSocketFactory())
.setHostnameVerifier(new DefaultHostnameVerifier())
.build();
} catch (Exception e) {
final String msg = "Could not build HttpTransport due to; " + e.getMessage();
log.error(msg, e);
throw new RuntimeException(e);
}
}
}
You can then use it using something like:
StorageOptions.Builder builder = StorageOptions.newBuilder();
if( useProxy ) {
HttpTransportFactory httpTransportFactory = new ProxyAwareTransportFactory(proxyHost, proxyPort);
HttpTransportOptions options = HttpTransportOptions.newBuilder().setHttpTransportFactory(httpTransportFactory).build();
builder.setTransportOptions(options);
}
Storage storage = builder
.setCredentials(
ServiceAccountCredentials.fromStream(
new FileInputStream(fullPath)))
.build()
.getService();
Also I have port number, but i didn't find where to set it. How can I fix this problem? Through python's "boto" client I successfully connected to my service, but on Scala it always returns forbidden message.
object a extends App{
import com.amazonaws.auth.BasicAWSCredentials
val accessKey = "myAccessKey"
val secretKey = "mySecretKey"
val credentials = new BasicAWSCredentials(accessKey, secretKey)
import com.amazonaws.services.s3.AmazonS3Client
val s3client = new AmazonS3Client(credentials)
s3client.setEndpoint("address")
val s3object = s3client.getObject("myBucket", "file.txt")
println(s3object.getKey, s3object.getObjectMetadata.getLastModified)
}
Here is error log:
Exception in thread "main" com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: null; S3 Extended Request ID: null), S3 Extended Request ID: null
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1639)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4319)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4266)
at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1406)
at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1267)
at a$.delayedEndpoint$a$1(a.scala:17)
at a$delayedInit$body.apply(a.scala:3)
at scala.Function0.apply$mcV$sp(Function0.scala:34)
at scala.Function0.apply$mcV$sp$(Function0.scala:34)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App.$anonfun$main$1$adapted(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:389)
at scala.App.main(App.scala:76)
at scala.App.main$(App.scala:74)
at a$.main(a.scala:3)
at a.main(a.scala)
The AWS SDK somehow discourages using the endpoint (JavaDoc):
Only use this if using a non-standard service endpoint - the recommended approach for configuring a client is to use {#link #withRegion(String)}
Instead, use the withRegion:
import com.amazonaws.auth.{AWSStaticCredentialsProvider, BasicAWSCredentials}
import com.amazonaws.regions.Regions
import com.amazonaws.services.s3.AmazonS3Client
AmazonS3Client.builder()
.withCredentials(new AWSStaticCredentialsProvider(
new BasicAWSCredentials("accessKey", "secretKey")))
.withRegion(Regions.EU_WEST_1).build()
I am using the JClouds-Chef API to bootstrap a Linux VM (myapp01.me.example.com) with Chef Client, and to then run the client and configure the VM with its app stack via a typicalapp role:
package com.me.myorg.chef;
import org.jclouds.Constants
import org.jclouds.ContextBuilder
import org.jclouds.chef.ChefContext
import org.jclouds.chef.ChefService
import org.jclouds.chef.config.ChefProperties
import org.jclouds.chef.domain.BootstrapConfig
import org.jclouds.chef.util.RunListBuilder
import org.jclouds.compute.domain.ExecResponse
import org.jclouds.compute.domain.OsFamily
import org.jclouds.domain.LoginCredentials
import org.jclouds.scriptbuilder.domain.Statement
import org.jclouds.ssh.SshClient
import org.jclouds.sshj.config.SshjSshClientModule
import com.google.common.base.Charsets
import com.google.common.collect.ImmutableSet
import com.google.common.io.Files
import com.google.common.net.HostAndPort
import com.google.inject.Key
import com.google.inject.TypeLiteral
public class ChefProvisioner {
public static void main(String[] args) {
ChefProvisioner.provision()
}
public static provision() {
String vmIp = "myapp01.me.example.com"; // A Linux VM living in our local vCenter
String vmSshUsername = "admin";
String vmSshPassword = "12345";
String endpoint = "https://mychefserver";
String client = "myuser";
String validator = "chef-validator";
String clientCredential = Files.toString(new File("C:\\Users\\myuser\\sandbox\\chef\\myuser.pem"), Charsets.UTF_8);
String validatorCredential = Files.toString(new File("C:\\Users\\myuser\\sandbox\\chef\\chef-validator.pem"), Charsets.UTF_8);
Properties props = new Properties();
props.put(ChefProperties.CHEF_VALIDATOR_NAME, validator);
props.put(ChefProperties.CHEF_VALIDATOR_CREDENTIAL, validatorCredential);
props.put(Constants.PROPERTY_RELAX_HOSTNAME, "true");
props.put(Constants.PROPERTY_TRUST_ALL_CERTS, "true");
ChefContext ctx = ContextBuilder.newBuilder("chef")
.endpoint(endpoint)
.credentials(client, clientCredential)
.overrides(props)
.modules(ImmutableSet.of(new SshjSshClientModule())) //
.buildView(ChefContext.class);
ChefService chef = ctx.getChefService();
List<String> runlist = new RunListBuilder().addRole("typicalapp").build();
BootstrapConfig bootstrapConfig = BootstrapConfig.builder().runlist(runlist).build();
chef.updateBootstrapConfigForGroup("jclouds-chef", bootstrapConfig);
Statement bootstrap = chef.createBootstrapScriptForGroup("jclouds-chef");
SshClient.Factory sshFactory = ctx.unwrap().utils()
.injector().getInstance(Key.get(new TypeLiteral<SshClient.Factory>() {}));
SshClient ssh = sshFactory.create(HostAndPort.fromParts(vmIp, 22),
LoginCredentials.builder().user(vmSshUsername).password(vmSshPassword).build());
ssh.connect();
try {
String rawScript = bootstrap.render(OsFamily.UNIX);
ExecResponse result = ssh.exec(rawScript);
} finally {
ssh.disconnect();
}
}
}
When I run this I get:
Exception in thread "main" java.util.ServiceConfigurationError: org.jclouds.apis.ApiMetadata: Provider org.jclouds.openstack.swift.SwiftApiMetadata could not be instantiated: java.lang.IllegalAccessError: tried to access class com.google.common.reflect.TypeResolver from class org.jclouds.util.TypeToken2
at java.util.ServiceLoader.fail(ServiceLoader.java:224)
at java.util.ServiceLoader.access$100(ServiceLoader.java:181)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:377)
at java.util.ServiceLoader$1.next(ServiceLoader.java:445)
at com.google.common.collect.ImmutableCollection$Builder.addAll(ImmutableCollection.java:323)
at com.google.common.collect.ImmutableSet$Builder.addAll(ImmutableSet.java:633)
at org.jclouds.apis.Apis.all(Apis.java:72)
at org.jclouds.apis.Apis.withId(Apis.java:89)
at org.jclouds.ContextBuilder.newBuilder(ContextBuilder.java:168)
at org.jclouds.ContextBuilder$newBuilder.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at com.me.myorg.chef.ChefProvisioner.provision(ChefProvisioner.groovy:51)
at com.me.myorg.chef.ChefProvisioner$provision.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
at com.me.myorg.chef.ChefProvisioner.main(ChefProvisioner.groovy:27)
Caused by: java.lang.IllegalAccessError: tried to access class com.google.common.reflect.TypeResolver from class org.jclouds.util.TypeToken2
at org.jclouds.util.TypeToken2.where(TypeToken2.java:47)
at org.jclouds.rest.internal.BaseRestApiMetadata.contextToken(BaseRestApiMetadata.java:60)
at org.jclouds.rest.internal.BaseRestApiMetadata$Builder.<init>(BaseRestApiMetadata.java:74)
at org.jclouds.openstack.swift.SwiftApiMetadata$Builder.<init>(SwiftApiMetadata.java:85)
at org.jclouds.openstack.swift.SwiftApiMetadata$Builder.<init>(SwiftApiMetadata.java:81)
at org.jclouds.openstack.swift.SwiftApiMetadata$ConcreteBuilder.<init>(SwiftApiMetadata.java:108)
at org.jclouds.openstack.swift.SwiftApiMetadata$ConcreteBuilder.<init>(SwiftApiMetadata.java:108)
at org.jclouds.openstack.swift.SwiftApiMetadata.<init>(SwiftApiMetadata.java:60)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at java.lang.Class.newInstance(Class.java:374)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:373)
... 16 more
Any ideas as to what is going on here?
You will need the following, in order to be able to bootstrap your VM:
The ip address of the VM. It must be reachable, have the port 22 open (as the bootstrap will be done via SSH) and have access to the Internet so it can install Ruby and Chef.
A client in the Chef Server and its corresponding private key. That client is used by jclouds to perform the operations against the Chef Server REST API.
The validator certificate. That certificate will be uploaded to the node so it can self-register in the Chef Server.
In this case, as the jclouds ComputeService is not being used, you'll have to manually instantiate an SSH client to connect to the virtual machine, but it should be pretty straightforward.
In the example you mention, the Git repo is cloned when using ChefSolo, but since you have a Chef Server the only thing you need to do is to configure the connection and the desired run list and attributes.
A minimal program to do that would require the following dependencies:
<!-- Required dependencies -->
<dependency>
<groupId>org.apache.jclouds.driver</groupId>
<artifactId>jclouds-sshj</artifactId>
<version>${jclouds.version}</version>
</dependency>
<dependency>
<groupId>org.apache.jclouds.api</groupId>
<artifactId>chef</artifactId>
<version>${jclouds.version}</version>
</dependency>
And it could be something like the following:
Update: I've changed the configuration to match the provided knife.rb configuration, and also added a couple properties to avoid SSL errors, as your Chef endpoint is https.
// Configuration
String vmIp = "vm-ip";
String vmSshUsername = "root";
String vmSshPassword = "foo";
String endpoint = "https://mychefserver.example.com";
String client = "myuser";
String validator = "chef-validator";
String clientCredential = Files.toString(new File("/home/myuser/.chef/myuser.pem"), Charsets.UTF_8);
String validatorCredential = Files.toString(new File("/home/myuser/.chef/chef-validator.pem"), Charsets.UTF_8);
Properties props = new Properties();
props.put(ChefProperties.CHEF_VALIDATOR_NAME, validator);
props.put(ChefProperties.CHEF_VALIDATOR_CREDENTIAL, validatorCredential);
props.put(Constants.PROPERTY_RELAX_HOSTNAME, "true");
props.put(Constants.PROPERTY_TRUST_ALL_CERTS, "true");
/* *** First, create the context to connect to the Chef Server *** */
// Create the context and configure the SSH driver to use. sshj in this example
ChefContext ctx = ContextBuilder.newBuilder("chef")
.endpoint(endpoint)
.credentials(client, clientCredential)
.overrides(props)
.modules(ImmutableSet.of(new SshjSshClientModule())) //
.buildView(ChefContext.class);
CherService chef = ctx.getChefService();
/* *** Second, generate the bootstrap script *** */
// Generate the bootsrap configuration
List<String> runlist = new RunListBuilder().addRole("typicalapp").build();
BootstrapConfig bootstrapConfig = BootstrapConfig.builder().runlist(runlist).build();
// Generate the bootstrap script to be executed in the VM (this will persist
// the configuration in a data bag under the key 'jclouds-chef' so it can be reused
// and then build the bootstrap script with the information in the configuration data bag)
chef.updateBootstrapConfigForGroup("jclouds-chef", bootstrapConfig);
Statement bootstrap = chef.createBootstrapScriptForGroup("jclouds-chef");
/* *** Finally create an SSH connection manually and run the script on the VM *** */
SshClient.Factory sshFactory = ctx.unwrap().utils()
.injector().getInstance(Key.get(new TypeLiteral<SshClient.Factory>() {}));
SshClient ssh = sshFactory.create(HostAndPort.fromParts(vmIp, 22),
LoginCredentials.builder().user(vmSshUsername).password(vmSshPassword).build());
ssh.connect();
try {
String rawScript = bootstrap.render(OsFamily.UNIX);
ExecResponse result = ssh.exec(rawScript);
} finally {
ssh.disconnect();
}
I am trying to connect to Salesforce from a java class (on my local machine). I have used the WSC-22.jar (webservice connector) and used the same as the library in eclipse. I have also parsed the enterprise wsdl to a jar and uploaded the library in the eclipse. I am running the below java class which is error out.
package wsc;
import com.sforce.soap.enterprise.Connector;
import com.sforce.soap.enterprise.DeleteResult;
import com.sforce.soap.enterprise.EnterpriseConnection;
import com.sforce.soap.enterprise.Error;
import com.sforce.soap.enterprise.QueryResult;
import com.sforce.soap.enterprise.SaveResult;
import com.sforce.soap.enterprise.sobject.Account;
import com.sforce.soap.enterprise.sobject.Contact;
import com.sforce.ws.ConnectionException;
import com.sforce.ws.ConnectorConfig;
public class main {
static final String USERNAME = "username";
static final String PASSWORD = "password + sec token";
static EnterpriseConnection connection;
public static void main(String[] args) {
ConnectorConfig config = new ConnectorConfig();
config.setUsername(USERNAME);
config.setPassword(PASSWORD);
//config.setTraceMessage(true);
try {
connection = com.sforce.soap.enterprise.Connector.newConnection(config);
// display some current settings
System.out.println("Auth EndPoint: "+config.getAuthEndpoint());
System.out.println("Service EndPoint: "+config.getServiceEndpoint());
System.out.println("Username: "+config.getUsername());
System.out.println("SessionId: "+config.getSessionId());
// run the different examples
queryContacts();
createAccounts();
updateAccounts();
deleteAccounts();
} catch (ConnectionException e1) {
System.out.println("hello world");
e1.printStackTrace();
}
}
//somemore code----
ERROR MESSAGE:
com.sforce.ws.ConnectionException: Failed to send request to
https://login.salesforce.com/services/Soap/c/26.0
at com.sforce.ws.transport.SoapConnection.send(SoapConnection.java:120)
at com.sforce.soap.enterprise.EnterpriseConnection.login(EnterpriseConnection.java:1)
at com.sforce.soap.enterprise.EnterpriseConnection.<init>(EnterpriseConnection.java:1)
at com.sforce.soap.enterprise.Connector.newConnection(Connector.java:1)
at wsc.main.main(main.java:32)
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)hello world
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:570)
at com.sun.net.ssl.internal.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:141)
at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:411)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:525)
at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:272)
at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:329)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:172)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:966)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:158)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1031)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:230)
at com.sforce.ws.transport.JdkHttpTransport.connectRaw(JdkHttpTransport.java:133)
at com.sforce.ws.transport.JdkHttpTransport.connectLocal(JdkHttpTransport.java:97)
at com.sforce.ws.transport.JdkHttpTransport.connectLocal(JdkHttpTransport.java:92)
at com.sforce.ws.transport.JdkHttpTransport.connect(JdkHttpTransport.java:88)
at com.sforce.ws.transport.SoapConnection.send(SoapConnection.java:94)
I am unable to figure out how to solve this issue. As the error message says "Falied to connect to http://login.salesforce.com./...", should I enable some setting in Eclipse??
Regards
Sam
Are you behind a proxy ?
If so, enable proxy settings in eclipse preferences general → network
...
String proxyHost = getUserInput("Enter proxy host:");
String proxyPort = getUserInput("Enter proxy port:");
try {
...
if(proxyHost!=null && !proxyHost.trim().equals("")){
config.setProxy(proxyHost, Integer.valueOf(proxyPort));
}
...
I want to upload files using the java API for DropBox. The following code gets me the oauth_token and oauth_secret. but when ever I try to upload a file I get a exception.
Java Class
package com.dropbox.client;
import com.dropbox.client.DropboxAPI.Config;
import java.io.File;
import java.net.URL;
import java.util.HashMap;
import java.util.Map;
/**
*
* #author Charan
*/
public class DBmain {
public DBmain() {
System.setProperty("java.net.useSystemProxies", "true");
Map configuration= new HashMap();
configuration.put("consumer_key", "XXXXXXXXXXXXXXXX");
configuration.put("consumer_secret", "XXXXXXXXXXXXXXXX");
configuration.put("request_token_url", "http://api.dropbox.com/0/oauth/request_token");
configuration.put("access_token_url", "http://api.dropbox.com/0/oauth/access_token");
configuration.put("authorization_url", "http://api.dropbox.com/0/oauth/authorize");
configuration.put("port",80);
//configuration.put("trusted_access_token_url","http://api.getdropbox.com/0/token");
configuration.put("server","api.getdropbox.com");
configuration.put("content_server","api-content.dropbox.com");
String username="myMailAddress#domain.com";
String password="myPassword";
try {
Authenticator auth = new Authenticator(configuration);
String url = auth.retrieveRequestToken("");
String access_key = auth.getTokenKey();
String access_secret = auth.getTokenSecret();
System.out.println(access_key);
System.out.println(access_secret);
DropboxAPI api = new DropboxAPI();
DropboxAPI.Config conf = api.new Config(configuration);
api.authenticateToken("XXXXXXXXXXXX", "XXXXXXXXXXX", conf);
System.out.println(api.isAuthenticated());
URL resource = this.getClass().getResource("/config/testing.json");
File f= new File(resource.toURI());
api.putFile("dropbox", "/Project", f);
//api.accountInfo(); //even this method gives me a exception
} catch (Exception e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
new DBmain();
}
}
Exception
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.http.protocol.BasicHttpContext: method <init>()V not found
at org.apache.http.impl.client.DefaultHttpClient.createHttpContext(DefaultHttpClient.java:205)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:532)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:487)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:465)
at com.dropbox.client.DropboxClient.putFile(DropboxClient.java:299)
at com.dropbox.client.DropboxAPI.putFile(DropboxAPI.java:463)
at com.dropbox.client.DBmain.<init>(DBmain.java:58)
at com.dropbox.client.DBmain.main(DBmain.java:70)
Java Result: 1
Some extra info
Edit Date:18-6-2011
I changed the httpclient-4.0-beta1.jar and httpcore-4.0-alpha6.jar to httpclient-4.0.jar and httpcore-4.0.1.jar respectively and I no longer get the above exception [ java.lang.NoSuchMethodError ]
This stackoverflow question helped me in solving this : java.lang.NoSuchMethodError: org.apache.http.protocol.BasicHttpContext: method <init>()V not found
But Now I get UnknownhostException on execution of any methods of the API
com.dropbox.client.DropboxException: java.net.UnknownHostException: api.getdropbox.com:80
at com.dropbox.client.RESTUtility.request(RESTUtility.java:250)
at dump.DropboxClient.accountInfo(DropboxClient.java:121)
at com.charan.client.DBmain.<init>(DBmain.java:57)
at com.charan.client.DBmain.main(DBmain.java:65)
Caused by: java.net.UnknownHostException: api.getdropbox.com:80
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:436)
at java.net.Socket.connect(Socket.java:525)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:123)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:123)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:147)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:101)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:381)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)
at com.dropbox.client.RESTUtility.executeRequest(RESTUtility.java:172)
at com.dropbox.client.RESTUtility.request(RESTUtility.java:248)
... 3 more
You should use http-client 4.0.3 jar
Your class should NOT be in package com.dropbox.client. Try moving it to a different (i.e. com.yourname.client). It looks like you might be creating a name clash and inadvertently overriding something in the Dropbox client API.
You have specified api.getdropbox.com:80 as the host name. Try using api.getdropbox.com instead.