We Keep Coding

Azure blob services - AuthorizationFailed - /Microsoft.Storage/storageAccounts/<ACCOUNT_NAME>/blobServices/default - JAVA sdk

I am trying to obtain the "Versioning" status of my Storage account using azure-sdk-for-java
// Azure environment URL is ".core.windows.net" hence used "AzureEnvironment.AZURE"
// this.clientSecretCredential ->is built using AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID
AzureProfile profile = new AzureProfile("<TENANT_ID", "<SUBSCRIPTION_ID>", AzureEnvironment.AZURE);
StorageManager manager = StorageManager.authenticate(this.clientSecretCredential, profile);
BlobServicesClient blobServicesClient = manager.serviceClient().getBlobServices();
// Exception is thrown at the following line
BlobServicePropertiesInner blobServicePropertiesInner = blobServicesClient.getServiceProperties("<RESOURCE_GROUP_NAME>", "<ACCOUNT_NAME>");
boolean versionFlag = blobServicePropertiesInner.isVersioningEnabled();
Azure configuration details:
Subscrition: "<SUBSCRIPTION_ID>" is created in the subscriptions.
Resource Group: "<RESOURCE_GROUP_NAME>" is configured with the "<SUBSCRIPTION_ID>".
Storage Accouunt: "<ACCOUNT_NAME>" is configured with the "<SUBSCRIPTION_ID>".
App Registartion: "<APP_REGISTARTION>" is created to provide the "<AZURE_CLIENT_ID>", "<AZURE_CLIENT_SECRET>", "<AZURE_TENANT_ID>"
Role Assignments: "DEVELOPER" has "Reader" access across subscriptions, resource groups, storage accounts but still I still have no idea on how the App registration is configured into the subscription
Error Message:
{
"code": "ERROR",
"message": "Status code 403, "{"error":{"code":"AuthorizationFailed","message":"The client '<APP_REGISTARTION_OBJECT_ID>' with object id '<APP_REGISTARTION_OBJECT_ID>' does not have authorization to perform action 'Microsoft.Storage/storageAccounts/blobServices/read' over scope '/subscriptions/{"<SUBSCRIPTION_ID>"}/resourceGroups/Titaniam-Sandbox/providers/Microsoft.Storage/storageAccounts/sandboxtestaccount/blobServices/default' or the scope is invalid. If access was recently granted, please refresh your credentials."}}""
}
Kinldy let me know what is the mistake i am making whether it is a code issue or configuration issue.
Things i am need of clarification:
Is there any other way to get the "is Versioning enabled" value using azure-sdk-for-java?
How App registration is connected with subscription.
How are roles connected with App registration as well as subscription.
How to set the scope
How to add the application and how to identify the application.
Thanks in advance.

Microsoft exchange - IMAP OAuth2 With client credentials

I am writing application that need to read mailbox using IMAP, but as daemon, without user interaction. I need to use OAuth2 to get access.
Because I need it without user interaction, I need to use client credentials flow. This was added this June.
I have done everything from official documentation. Registered application, added permissions, added mailbox permission using PowerShell.
When I get request access token with scope https://outlook.office365.com/.default, the one that I receive has role IMAP.AccessAsApp, so I believe that is correct. I used https://jwt.ms/ to parse JWT.
The problem is when I try to authenticate using this access token in Java, for example
Properties props = new Properties();
props.put("mail.imap.ssl.enable", "true");
props.put("mail.imap.auth.mechanisms", "XOAUTH2");
props.put("mail.debug", "true");
Session session = Session.getInstance(props);
Store store = session.getStore("imap");
store.connect("outlook.office365.com", 993, "testing#mydomain.com", "accessToken");
I receive AUTHENTICATE failed. I tried same code with access token received using authorization code flow, which requires user interaction. Using that access code I was able to connect to mailbox. So the code is correct.
I even tried using client id and service id instead of email address as username, but without success.
I am not sure where I made the mistake and if I am using correct username. Any help is appreciated.

I wrote same answer here, so I am coping it here.
I think I made some progress.
I read documentation few times, tried few times from the start with same error. I even have tried using client and object ids instead of email as username, in lack of better ideas.
So this is where I think I have made mistake previous times.
On the part where it is needed to register service principal, I needed to execute
New-ServicePrincipal -AppId <APPLICATION_ID> -ServiceId <OBJECT_ID> [-Organization <ORGANIZATION_ID>]
Here I have put enterprise application object id as ServiceId argument. And that is ok.
But on
Add-MailboxPermission -Identity "email address removed for privacy reasons" -User
<SERVICE_PRINCIPAL_ID> -AccessRights FullAccess
I have put my registered application object id as User argument. I also tried setting object id of enterprise application, but it did not have success.
When I executed
Get-ServicePrincipal -Organization <ORGANIZATION_ID> | fl
I did not pay attention to ServiceId property, even with documentation specifying it and saying it will be different.
Now I cleared everything and started fresh.
I have executed all the steps again, but on the step for creating new service principal I used data from enterprise application view. When I need to add mail permission, I list service principals, and then use ServiceId value from the output, as argument for user.
With that, I was able to authorise.

Thanks everyone for sharing your experience. This has proved to be a little confusing. :)
To sum everything up, to access a mailbox with IMAPS and OAuth2 (as opposed to using Graph API which is another method Microsoft recommends):
Create an Azure App Registration
Add API permission Office 365 Exchange Online - IMAP.AccessAsApp and grant admin consent
Create a service principal, which will be used to grant mailbox permissions to in Exchange Online
Connect-AzureAD
Connect-ExchangeOnline
$azapp = Get-AzureADApplication -SearchString 'App Registration Name'
$azsp = Get-AzureADServicePrincipal -SearchString $azapp.DisplayName
# GOTCHA: You need the ObjectId from 'Enterprise applications' (Get-AzureADServicePrincipal), not 'Application registrations' (Get-AzureADApplication) for ServiceId (thanks #[jamie][1])
$sp = New-ServicePrincipal -AppId $azapp.AppId -ServiceId $azsp.ObjectId -DisplayName "EXO Service Principal for $($azapp.DisplayName)"
Grant access rights to mailboxes for the service principal
$mbxs = 'mymbx1#yourdomain.tld',`
'mymbx2#yourdomain.tld',`
'mymbx3#yourdomain.tld'
$mbxs | %{ Add-MailboxPermission -Identity $_ -User $sp.ServiceId -AccessRights FullAccess } | fl *
Get-MailboxPermission $mbxs[-1] | ft -a
You can use Get-IMAPAccessToken.ps1 to test your setup
.\Get-IMAPAccessToken.ps1 -TenantID $TenantId -ClientId $ClientId -ClientSecret $ClientSecret -TargetMailbox $TargetMailbox
Other parameters you may need:
Authority: https://login.microsoftonline.com/<YourTenantId>/
Scope: https://outlook.office365.com/.default

Couchbase Analytics Java SDK connection creation + Security Roles

I am using the Couchbase Java SDK to query a couchbase analytics service. The process is covered in this tutorial: https://docs.couchbase.com/java-sdk/2.7/analytics-using-sdk.html
The Java SDK provides a Bucket object as the means of accessing couchbase. However a bucket is a separate entity from the analytics data-set. For example, my bucket is called data, and I have an analytics data-set that I want to query called requests.
I cannot find a means of connecting to just the requests data-set. The SDK will only connect to the data bucket. From there I can query the requests data-set by writing some N1QL. This work-around means that the user credentials I'm using to run analytics queries must also have access to my main production data bucket, which I'd rather prevent.
Is there a way to connect to simply the analytics data-set using the SDK?
The code I have currently creating the connection looks like this:
public class CouchbaseConfig {
#Bean
public Bucket bucket(CouchbaseProperties properties) {
return cluster().openBucket("data"); // Changing this to the data-set name returns error
}
private Cluster cluster() {
Cluster cluster = CouchbaseCluster.create("localhost");
cluster.authenticate("Administrator", "password");
return cluster;
}
}
Using the requests data-set name in the bucket name results in this error:
Failed to instantiate [com.couchbase.client.java.Bucket]: Factory method 'bucket' threw exception; nested exception is com.couchbase.client.java.error.BucketDoesNotExistException: Bucket "requests" does not exist.
Using the data bucket name, but authentication username / password "analytics-reader" / "password" (with only Analytics Reader) roles results in this error:
Could not load bucket configuration: FAILURE({"message":"Forbidden. User needs one of the following permissions","permissions":["cluster.bucket[data].settings!read"]})
The only work around I have found is to give the analytics-reader user 'Application Access' to the data` bucket 😢

Connecting directly to analytics is possible with SDK3 and Couchbase 6.5 . In all the previous versions (SDK 2.7 included), the only way to query analytics is to connect to a bucket first.

Azure list Azure Database for PostgreSQL servers in Resource group using Azure Java SDK

What is the best and correct way to list Azure Database for PostgreSQL servers present in my Resource Group using Azure Java SDK?
Currently, we have deployments that happen using ARM templates and once the resources have been deployed we want to be able to get the information about those resources from Azure itself.
I have tried doing in the following way:
PagedList<SqlServer> azureSqlServers = azure1.sqlServers().listByResourceGroup("resourceGrpName");
//PagedList<SqlServer> azureSqlServers = azure1.sqlServers().list();
for(SqlServer azureSqlServer : azureSqlServers) {
System.out.println(azureSqlServer.fullyQualifiedDomainName());
}
System.out.println(azureSqlServers.size());
But the list size returned is 0.
However, for virtual machines, I am able to get the information in the following way:
PagedList<VirtualMachine> vms = azure1.virtualMachines().listByResourceGroup("resourceGrpName");
for (VirtualMachine vm : vms) {
System.out.println(vm.name());
System.out.println(vm.powerState());
System.out.println(vm.size());
System.out.println(vm.tags());
}
So, what is the right way of getting the information about the Azure Database for PostgreSQL using Azure Java SDK?
P.S.
Once I get the information regarding Azure Database for PostgreSQL, I would need similar information about the Azure Database for MySQL Servers.
Edit: I have seen this question which was asked 2 years back and would like to know if Azure added Support for Azure Database for PostgreSQL/MySQL servers or not.
Azure Java SDK for MySQL/PostgreSQL databases?

So, I kind of implemented it in the following way and it can be treated as an alternative way...
Looking at the Azure SDK for java repo on Github (https://github.com/Azure/azure-sdk-for-java/tree/master/sdk/postgresql), looks like they have it in beta so I searched for the pom in mvnrepository. I imported the following pom in my project (azure-mgmt-postgresql is still in beta):
<!-- https://mvnrepository.com/artifact/com.microsoft.azure.postgresql.v2017_12_01/azure-mgmt-postgresql -->
<dependency>
<groupId>com.microsoft.azure.postgresql.v2017_12_01</groupId>
<artifactId>azure-mgmt-postgresql</artifactId>
<version>1.0.0-beta-5</version>
</dependency>
In the code, Following is the gist of how I did it:
I already have a service principal created and have its information with me.
But, anyone trying this will require clientId, tenantId, clientSecret, and subscriptionId with them, the way #Jim Xu explained.
// create the credentials object
ApplicationTokenCredentials credentials = new ApplicationTokenCredentials(clientId, tenantId, clientSecret, AzureEnvironment.AZURE);
// build a rest client object configured with the credentials created above
RestClient restClient = new RestClient.Builder()
.withBaseUrl(credentials.environment(), AzureEnvironment.Endpoint.RESOURCE_MANAGER)
.withCredentials(credentials)
.withSerializerAdapter(new AzureJacksonAdapter())
.withResponseBuilderFactory(new AzureResponseBuilder.Factory())
.withInterceptor(new ProviderRegistrationInterceptor(credentials))
.withInterceptor(new ResourceManagerThrottlingInterceptor())
.build();
// use the PostgreSQLManager
PostgreSQLManager psqlManager = PostgreSQLManager.authenticate(restClient, subscriptionId);
PagedList<Server> azurePsqlServers = psqlManager.servers().listByResourceGroup(resourceGrpName);
for(Server azurePsqlServer : azurePsqlServers) {
System.out.println(azurePsqlServer.fullyQualifiedDomainName());
System.out.println(azurePsqlServer.userVisibleState().toString());
System.out.println(azurePsqlServer.sku().name());
}
Note: Server class refers to com.microsoft.azure.management.postgresql.v2017_12_01.Server
Also, if you take a look at the Azure class, you will notice this is how they do it internally.
For reference, you can use SqlServerManager sqlServerManager in the Azure class and look at how they have used it and created an authenticated manager in case you want to use some services that are still in preview or beta.

According to my test, we can use java sdk azure-mgmt-resources to implement your need. For example
Create a service principal
az login
# it will create a service pricipal and assign a contributor rolen to the sp
az ad sp create-for-rbac -n "MyApp" --scope "/subscriptions/<subscription id>" --sdk-auth
code
String tenantId = "<the tenantId you copy >";
String clientId = "<the clientId you copy>";
String clientSecret= "<the clientSecre you copy>";
String subscriptionId = "<the subscription id you copy>";
ApplicationTokenCredentials creds = new
ApplicationTokenCredentials(clientId,domain,secret,AzureEnvironment.AZURE);
RestClient restClient =new RestClient.Builder()
.withBaseUrl(AzureEnvironment.AZURE, AzureEnvironment.Endpoint.RESOURCE_MANAGER)
.withSerializerAdapter(new AzureJacksonAdapter())
.withReadTimeout(150, TimeUnit.SECONDS)
.withLogLevel(LogLevel.BODY)
.withResponseBuilderFactory(new AzureResponseBuilder.Factory())
.withCredentials(creds)
.build();
ResourceManager resourceClient= ResourceManager.authenticate(restClient).withSubscription(subscriptionId);
ResourceManagementClientImpl client= resourceClient.inner();
String filter="resourceType eq 'Microsoft.DBforPostgreSQL/servers'"; //The filter to apply on the operation
String expand=null;//The $expand query parameter. You can expand createdTime and changedTime.For example, to expand both properties, use $expand=changedTime,createdTime
Integer top =null;// The number of results to return. If null is passed, returns all resource groups.
PagedList<GenericResourceInner> results= client.resources().list(filter, null,top);
while (true) {
for (GenericResourceInner resource : results.currentPage().items()) {
System.out.println(resource.id());
System.out.println(resource.name());
System.out.println(resource.type());
System.out.println(resource.location());
System.out.println(resource.sku().name());
System.out.println("------------------------------");
}
if (results.hasNextPage()) {
results.loadNextPage();
} else {
break;
}
}
Besides, you also can use Azure REST API to implement your need. For more details, please refer to https://learn.microsoft.com/en-us/rest/api/resources/resources

Querying Views in Couchbase, Java Client

I'm using the 1.4.3 version of the java client and am attempting to connect to the Couchbase server I have running locally but I'm getting auth errors. After looking through the code (isn't open source great?) of how their client library is using the variables amongst their classes I've come to the conclusion that if I want to be able to connect to a "bucket" that I have to create a user for each "bucket" with the same user name as that bucket. This makes no sense to me. I have to be wrong. Aren't I? There has to be another way. What is that way?
For reference, here is what I'm using to create a connection (it's Scala but would look nearly identical in Java):
val cf = new CouchbaseConnectionFactoryBuilder()
.setViewTimeout(opTimeout)
.setViewWorkerSize(workerSize)
.setViewConnsPerNode(conPerNode)
.buildCouchbaseConnection(nodes, bucket, password)
new CouchbaseClient(cf)
which follows directly from their examples.
Their Code
If I look into the code in which they're connecting to the "view" itself I see the following:
public ViewConnection createViewConnection(
List<InetSocketAddress> addrs) throws IOException {
return new ViewConnection(this, addrs, bucket, pass);
}
which is then passed to a constructor:
public ViewConnection(final CouchbaseConnectionFactory cf,
final List<InetSocketAddress> seedAddrs, final String user,
final String password) //more code...
and that user variable is actually used in the HTTP Basic Auth to form the Authentication header. That user variable being, of course, equivalent to the bucket variable in the CouchbaseConnectionFactory.

You are correct - each bucket should be authenticated with the bucket name as the user. However, there aren't any users to 'create' - you're just using whatever (bucket) name and password you setup when you created the bucket on the Cluster UI.
Note that people usually use one bucket per application (don't think bucket == table, think bucket == database) and so you wouldn't typically need more than a couple of buckets for most applications.