I wrote the below controller to generate PreSigned S3 upload links.
case class S3Controller(private val s3Config: S3Config, private val awsConfig: AwsConfig) {
val URL_TIMEOUT_IN_MILLIS: Long = 60 * 5 * 1000
def getPreSignedURLForUpload(keyName: String): String = {
val preSigner: S3Presigner = DefaultS3Presigner.builder()
.serviceConfiguration(s3Config.s3Configuration)
.credentialsProvider(awsConfig.awsCredentialsProvider).build()
val objectRequest: PutObjectRequest = PutObjectRequest.builder()
.bucket(s3Config.bucketName)
.key(keyName)
.contentType("text/plain")
.build()
val preSignRequest: PutObjectPresignRequest = PutObjectPresignRequest.builder()
.signatureDuration(Duration.ofMinutes(10))
.putObjectRequest(objectRequest)
.build()
val preSignedRequest: PresignedPutObjectRequest = preSigner.presignPutObject(preSignRequest)
val myURL: String = preSignedRequest.url().toString
myURL
}
}
Config objects that I used
case class S3Config (
bucketName: String,
s3Configuration: S3Configuration
)
case class AwsConfig (
awsCredentialsProvider: AwsCredentialsProvider
)
I tried to test it with the following code
test("S3 Controller"){
val s3Configuration: S3Configuration = S3Configuration.builder()
.pathStyleAccessEnabled(true).build()
val s3Config: S3Config = S3Config(
bucketName = "ccc",
s3Configuration = s3Configuration
)
val awsCredentials: AwsCredentials = AwsBasicCredentials.create("aaa", "bbb")
val awsCredentialsProvider: AwsCredentialsProvider = AwsCredentialsProviderChain.of(StaticCredentialsProvider.create(awsCredentials))
val awsConfig: AwsConfig = AwsConfig(awsCredentialsProvider = awsCredentialsProvider)
val s3Controller: S3Controller = S3Controller(s3Config, awsConfig)
s3Controller.getPreSignedURLForUpload("ab")
}
This test fails the with the messsage
Unable to load region from any of the providers in the chain software.amazon.awssdk.regions.providers.DefaultAwsRegionProviderChain#4e5ed836: [software.amazon.awssdk.regions.providers.SystemSettingsRegionProvider#5f8edcc5: Unable to load region from system settings. Region must be specified either via environment variable (AWS_REGION) or system property (aws.region)., software.amazon.awssdk.regions.providers.AwsProfileRegionProvider#60015ef5: No region provided in profile: default, software.amazon.awssdk.regions.providers.InstanceProfileRegionProvider#2ab4bc72: Unable to contact EC2 metadata service.]
software.amazon.awssdk.core.exception.SdkClientException: Unable to load region from any of the providers in the chain software.amazon.awssdk.regions.providers.DefaultAwsRegionProviderChain#4e5ed836: [software.amazon.awssdk.regions.providers.SystemSettingsRegionProvider#5f8edcc5: Unable to load region from system settings. Region must be specified either via environment variable (AWS_REGION) or system property (aws.region)., software.amazon.awssdk.regions.providers.AwsProfileRegionProvider#60015ef5: No region provided in profile: default, software.amazon.awssdk.regions.providers.InstanceProfileRegionProvider#2ab4bc72: Unable to contact EC2 metadata service.]
I understand this is happening since I've not configured the Region anywhere.
All the ways to configure the Region are either through environment variables or config files.
Is there a way to programmatically configure the Region?
YOu set a Region when you declare a Service Object using a builder():
Region region = Region.US_EAST_1;
S3Presigner presigner = S3Presigner.builder()
.region(region)
.build();
All the Java V2 code examples show this way of setting a Region:
https://github.com/awsdocs/aws-doc-sdk-examples/tree/master/javav2/example_code/s3/src/main/java/com/example/s3
Related
The client object has methods for registering and deregistering targets. I cannot find how to get a list of Targets or TargetDescriptions for a TargetGroup. The AWS console shows the list of targets for any given TargetGroup so I assume it is available somewhere.
Region region = Region.US_EAST_1;
AwsCredentialsProvider provider = ProfileCredentialsProvider.create("...");
ElasticLoadBalancingV2Client client = ElasticLoadBalancingV2Client.builder().region(region).credentialsProvider(provider).build();
String targetGroupArn = "...";
DescribeTargetGroupsRequest req = DescribeTargetGroupsRequest.builder().targetGroupArns(targetGroupArn).build();
DescribeTargetGroupsResponse tgResp = client.describeTargetGroups(req);
for (TargetGroup targetGroup : tgResp.targetGroups()) {
???
}
I am integrating Stripe for Java in a Kotlin app.
This is the code that I wrote to create a Charge
createCharge function:
fun createCharge(charge: Charge, testApiKey: String): Charge? {
//met
// On your server, grab the Stripe token in the POST parameters submitted by your form. From there, it’s one simple API call to charge the card
Stripe.apiKey = testApiKey
try {
val chargeParams = mutableMapOf<String, Any?>()
chargeParams["amount"] = charge.amount
chargeParams["currency"] = charge.currency
chargeParams["description"] = charge.description
chargeParams["source"] = charge.source
chargeParams["customer"] = charge.customer
chargeParams["receipt_email"] = charge.receiptEmail
val requestOptions = idempotencyKeySetter()
val initialMetadata = mutableMapOf<String, String?>()
initialMetadata["start_date"] = charge.metadata["start_date"]
initialMetadata["end_date"] = charge.metadata["end_date"]
chargeParams["metadata"] = initialMetadata
return Charge.create(chargeParams, requestOptions)
} catch (e: StripeException) {
e.printStackTrace()
return null
}
}
and the function calling the createCharge function:
var formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd")
fun checkCreateCharge() {
val chargeParams: Charge = Charge()
chargeParams.amount = 2000
chargeParams.currency = "usd"
chargeParams.description = description
chargeParams.source = PaymentSource { "" }
chargeParams.customer = customerId
chargeParams.receiptEmail = testEmail
chargeParams.metadata["start_date"] = LocalDate.parse("2018-12-31", formatter).toString()
chargeParams.metadata["end_date"] = LocalDate.parse("2019-03-31", formatter).toString()
val newCharge: Charge? = createCharge(chargeParams, testApiKey)
}
When I the function checkCreateCharge runs, it sends the following error:
java.lang.IllegalStateException: chargeParams.metadata must not be null
Does anyone know why this is the case?
So while this is "possible" you absolutely should not do this. It's a very large security vulnerability. Anyone with your App would be able to sniff your Stripe Secret API Key and do any number of bad things. They could, for instance, test credit cards or even take money from your account.
When implementing Stripe on Android you should only use your Publishable API Key and only use the key to create Tokens/Sources. All other operations should leverage your Secret API Key on a secured backend.
Best I can tell from the API documentation, you're accessing getMetadata() which is returning null (as you've never set a value on it). Basically, the line:
chargeParams.metadata["start_date"] = LocalDate.parse("2018-12-31", formatter).toString()
is equivalent to the Java code:
chargeParams.getMetadata().put("start_date", LocalDate.parse("2018-12-31", formatter).toString())
and in your situation getMetadata() is null. I believe changing this to be:
chargeParams.setMetadata(
mapOf(
"start_date" to LocalDate.parse("2018-12-31", formatter).toString(),
"end_date" to LocalDate.parse("2019-03-31", formatter).toString()
)
)
will resolve the issue.
I am attempting to implement a relatively simple ETL pipeline that iterates through files in a google cloud bucket. The bucket has two folders: /input and /output.
What I'm trying to do is write a Java/Scala script to iterate through files in /input, and have the transformation applied to those that are not present in /output or those that have a timestamp later than that in /output. I've been looking through the Java API doc for a function I can leverage (as opposed to just calling gsutil ls ...), but haven't had any luck so far. Any recommendations on where to look in the doc?
Edit: There is a better way to do this than using data transfer objects:
public Page<Blob> listBlobs() {
// [START listBlobs]
Page<Blob> blobs = bucket.list();
for (Blob blob : blobs.iterateAll()) {
// do something with the blob
}
// [END listBlobs]
return blobs;
}
Old method:
def getBucketFolderContents(
bucketName: String
) = {
val credential = getCredential
val httpTransport = GoogleNetHttpTransport.newTrustedTransport()
val requestFactory = httpTransport.createRequestFactory(credential)
val uri = "https://www.googleapis.com/storage/v1/b/" + URLEncoder.encode(
bucketName,
"UTF-8") +
"o/raw%2f"
val url = new GenericUrl(uri)
val request = requestFactory.buildGetRequest(uri)
val response = request.execute()
response
}
}
You can list objects under a folder by setting the prefix string on the object listing API: https://cloud.google.com/storage/docs/json_api/v1/objects/list
The results of listing are sorted, so you should be able to list both folders and then walk through both in order and generate the diff list.
I want to setup Gatling so that in one setup, I can send 3000 requests, of which 95% will use one test file, and 5% another test file. These files are retrieved as json files (called 'userFeeder' in below code. Can Gatling support the above scenario?
The code is currently as follows, which works for a request per second approach, but needs amending.
class AddUserSimulation extends Simulation {
private val conf = ConfigFactory.load() //loads a setup file of parameters
private val TOKEN_VALUE = "tokenvalue"
private val userFeeder = jsonFile("/" + conf.getString("environment") + "/testaddUser.json")
val httpConf = http
.baseURL(conf.getString("api.gateway.url")) // Here is the root for all relative URLs
.header("Referer", conf.getString("referer"))
.header("Cache-Control", "no-cache")
.contentTypeHeader("application/json")
val Login = new ADFSAuthentication Login
val scnAPI = scenario("test add User") // A scenario is a chain of requests and pauses
.feed(userFeeder.circular)
.exec(Login.process)
.repeat(conf.getInt("repeat.count")) {
exec(http("test add User")
.post("/" + conf.getString("environment") + "/users/")
.body(StringBody("${payload}")).asJSON
.header("Authorization", "Bearer ${"+TOKEN_VALUE+"}")
.check(status.is(200)))
.pause(conf.getInt("execution.pause"))
}
setUp(scnAPI.inject(constantUsersPerSec(11) during(30 minutes)).protocols(httpConf))
}
Any help is greatly appreciated.
Sure! First set up your two feeders. Then set up two scenarios, one using the first feeder, one using the second feeder. Finally setUp both with the number of users you wish (corresponding to your distribution).
The code could look something like this:
private val firstFeeder = jsonFile("/" + conf.getString("environment") + "/testaddUser.json")
private val secondFeeder = jsonFile("/" + conf.getString("environment") + "/testaddUser2.json")
val scnAPI = scenario("test add User")
.feed(firstFeeder)
//...
val scnAPI2 = scenario("second test add User")
.feed(secondFeeder)
//...
setUp(scnAPI.inject(constantUsersPerSec(95) during(30 minutes)).protocols(httpConf),
scnAPI2.inject(constantUsersPerSec(5) during(30 minutes)).protocols(httpConf))
Note: This will not create exactly 3000 requests but I think you get the idea.
I'm trying to change the base URL of the the AWS SDK for Java.
It is possible with the SDK for PHP like this:
require 'vendor/autoload.php';
use Aws\DynamoDb\DynamoDbClient;
// Create a client that that contacts a completely customized base URL
$client = DynamoDbClient::factory(array(
'endpoint' => 'http://my-custom-url',
'region' => 'my-region-1',
'credentials' => array(
'key' => 'abc',
'secret' => '123'
)
));
It's also possible to set this up for s3cmd in .s3conf:
host_base = s3.mylocalaws.com
host_bucket = %(bucket)s.s3.mylocalaws.com
I can't figure out how this works for the Java SDK.
I've tried this but the result is not https://s3.mylocalaws.com/bucketName/key as I expected but https://bucketName.s3.mylocalaws.com
AmazonS3 s3Client = new AmazonS3Client(new AWSTestCredentialsImpl());
s3Client.setEndpoint("https://s3.mylocalaws.com");
S3Object resource = s3Client.getObject(
new GetObjectRequest(bucketName, key));
Look at Choosing a Specific Endpoint in the documentation.
AmazonEC2 ec2 = new AmazonEC2(myCredentials);
ec2.setEndpoint("https://ec2.eu-west-1.amazonaws.com");