We Keep Coding

How to explicitly point my service account file in code

In other Google Services such as Storage, BigQuery you can define what service account you are going to use in the JAVA code:
// You can specify a credential file by providing a path to GoogleCredentials.
// Otherwise credentials are read from the GOOGLE_APPLICATION_CREDENTIALS environment variable.
GoogleCredentials credentials = GoogleCredentials.fromStream(new FileInputStream(jsonPath))
.createScoped(Lists.newArrayList("https://www.googleapis.com/auth/cloud-platform"));
Storage storage = StorageOptions.newBuilder().setCredentials(credentials).build().getService();
Using Google Secret Manager it seems not possibile. Why?
The only way is to set an environment variable on VM?
I tried as suggested to use Credentials Provider
GoogleCredentials credentials = ServiceAccountCredentials.fromStream(credentialsInputStream);
CredentialsProvider credentialsProvider = FixedCredentialsProvider.create(credentials);
SecretManagerServiceSettings settings = SecretManagerServiceSettings.newBuilder().setCredentialsProvider(credentialsProvider).build();
client = SecretManagerServiceClient.create(settings);
but it doesn't work
Caused by: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;CLjava/lang/Object;)V
at io.grpc.Metadata$Key.validateName(Metadata.java:742)
at io.grpc.Metadata$Key.<init>(Metadata.java:750)
at io.grpc.Metadata$Key.<init>(Metadata.java:668)
at io.grpc.Metadata$AsciiKey.<init>(Metadata.java:959)
at io.grpc.Metadata$AsciiKey.<init>(Metadata.java:954)
at io.grpc.Metadata$Key.of(Metadata.java:705)
at io.grpc.Metadata$Key.of(Metadata.java:701)
at com.google.api.gax.grpc.GrpcHeaderInterceptor.<init>(GrpcHeaderInterceptor.java:60)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:239)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.access$1600(InstantiatingGrpcChannelProvider.java:71)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider$1.createSingleChannel(InstantiatingGrpcChannelProvider.java:210)
at com.google.api.gax.grpc.ChannelPool.create(ChannelPool.java:72)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:217)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:200)
at com.google.api.gax.rpc.ClientContext.create(ClientContext.java:156)
at com.google.cloud.secretmanager.v1.stub.GrpcSecretManagerServiceStub.create(GrpcSecretManagerServiceStub.java:237)
at com.google.cloud.secretmanager.v1.stub.SecretManagerServiceStubSettings.createStub(SecretManagerServiceStubSettings.java:226)
at com.google.cloud.secretmanager.v1.SecretManagerServiceClient.<init>(SecretManagerServiceClient.java:154)
at com.google.cloud.secretmanager.v1.SecretManagerServiceClient.create(SecretManagerServiceClient.java:135)
because of an Exception in the class com.google.cloud.secretmanager.v1.SecretManagerServiceClient
/**
* Constructs an instance of SecretManagerServiceClient, using the given settings. This is
* protected so that it is easy to make a subclass, but otherwise, the static factory methods
* should be preferred.
*/
protected SecretManagerServiceClient(SecretManagerServiceSettings settings) throws IOException {
this.settings = settings;
this.stub = ((SecretManagerServiceStubSettings) settings.getStubSettings()).createStub();
}

To customize credentials, you can create a custom secretManagerServiceSettings:
SecretManagerServiceSettings secretManagerServiceSettings =
SecretManagerServiceSettings.newBuilder()
.setCredentialsProvider(FixedCredentialsProvider.create(credentials))
.build();
SecretManagerServiceClient client =
SecretManagerServiceClient.create(secretManagerServiceSettings);

Where to store .pem file (private key) android java

I'm trying to create a JWT using a private key stored in a .PEM file. I understand private keys shouldn't be stored in the code and you can add keys to the global gradle.properties file to avoid exposing them but if I have a .PEM file how do I access it, in my code. from my local drive.
The yodlee documentation tells me to add the path below but how does this work?
public class ConfigurationParams {
public static final String iss = "135143514315321";
public static final String privateKeyFile = "/Users/bob/token/priv.pem";
}
The privateKeyFile path is passed through the file variable.
public class TokenManagement {
#RequiresApi(api = Build.VERSION_CODES.O)
static private String getKey(String file) {
String privKey;
try {
privKey = new String(Files.readAllBytes((Paths.get(file))));
} catch (IOException ex) {
privKey = null;
}
return privKey;
}
}

It seems more of an android related question. But if you are planning to have the key on the user's device then you should not do that.
All Yodlee APIs are IP restricted in production environment, and all API calls should be made from your server to Yodlee. Hence, you should store it at your server side not at the client side, if that's what you are trying to do.

Using ACL with Curator

Using CuratorFramework, could someone explain how I can:
Create a new path
Set data for this path
Get this path
Using username foo and password bar? Those that don't know this user/pass would not be able to do anything.
I don't care about SSL or passwords being sent via plaintext for the purpose of this question.

ACL in Apache Curator are for access control. Therefore, ZooKeeper do not provide any authentication mechanism like, clients who don't have correct password cannot connect to ZooKeeper or cannot create ZNodes. What it can do is, preventing unauthorized clients from accessing particular Znode/ZNodes. In order to do that, you have to setup CuratorFramework instance as I have described below. Remember, this will guarantee that, a ZNode create with a given ACL, can be again accessed by the same client or by a client presenting the same authentication information.
First you should build the CuratorFramework instane as follows. Here, the connectString means a comma separated list of ip and port combinations of the zookeeper servers in your ensemble.
CuratorFrameworkFactory.Builder builder = CuratorFrameworkFactory.builder()
.connectString(connectString)
.retryPolicy(new ExponentialBackoffRetry(retryInitialWaitMs, maxRetryCount))
.connectionTimeoutMs(connectionTimeoutMs)
.sessionTimeoutMs(sessionTimeoutMs);
/*
* If authorization information is available, those will be added to the client. NOTE: These auth info are
* for access control, therefore no authentication will happen when the client is being started. These
* info will only be required whenever a client is accessing an already create ZNode. For another client of
* another node to make use of a ZNode created by this node, it should also provide the same auth info.
*/
if (zkUsername != null && zkPassword != null) {
String authenticationString = zkUsername + ":" + zkPassword;
builder.authorization("digest", authenticationString.getBytes())
.aclProvider(new ACLProvider() {
#Override
public List<ACL> getDefaultAcl() {
return ZooDefs.Ids.CREATOR_ALL_ACL;
}
#Override
public List<ACL> getAclForPath(String path) {
return ZooDefs.Ids.CREATOR_ALL_ACL;
}
});
}
CuratorFramework client = builder.build();
Now you have to start it.
client.start();
Creating a path.
client.create().withMode(CreateMode.PERSISTENT).forPath("/your/ZNode/path");
Here, the CreateMode specify what type of a node you want to create. Available types are PERSISTENT,EPHEMERAL,EPHEMERAL_SEQUENTIAL,PERSISTENT_SEQUENTIAL,CONTAINER. Java Docs
If you are not sure whether the path up to /your/ZNode already exists, you can create them as well.
client.create().creatingParentsIfNeeded().withMode(CreateMode.PERSISTENT).forPath("/your/ZNode/path");
Set Data
You can either set data when you are creating the ZNode or later. If you are setting data at the creation time, pass the data as a byte array as the second parameter to the forPath() method.
client.create().withMode(CreateMode.PERSISTENT).forPath("/your/ZNode/path","your data as String".getBytes());
If you are doing it later, (data should be given as a byte array)
client.setData().forPath("/your/ZNode/path",data);
Finally
I don't understand what you mean by get this path. Apache Curator is a java client (more than that with Curator Recipes) which use Apache Zookeeper in the background and hides edge cases and complexities of Zookeeper. In Zookeeper, they use the concept of ZNodes to store data. You can consider it as the Linux directory structure. All ZNodePaths should start with / (root) and you can go on specifying directory like ZNodePaths as you like. Ex: /someName/another/test/sample.
As shown in the above diagram, ZNode are organized in a tree structure. Every ZNode can store up to 1MB of data. Therefore, if you want to retrieve data stored in a ZNode, you need to know the path to that ZNode. (Just like you should know the table and column of a database in order to retrive data).
If you want to retrive data in a given path,
client.getData().forPath("/path/to/ZNode");
That's all you have to know when you want to work with Curator.
One more thing
ACL in Apache Curator are for access control. That is, if you set ACLProvider as follows,
new ACLProvider() {
#Override
public List<ACL> getDefaultAcl () {
return ZooDefs.Ids.CREATOR_ALL_ACL;
}
#Override
public List<ACL> getAclForPath (String path){
return ZooDefs.Ids.CREATOR_ALL_ACL;
}
}
only the client with the credentials identical to the creator will be given access to the corresponding ZNode later on. Autherization details are set as follows (See the client building example). There are other modes of ACL availble, like OPEN_ACL_UNSAFE which do not do any access control if you set it as the ACLProvider.
authorization("digest", authorizationString.getBytes())
they will be used later to control access to a given ZNode.
In short, if you want to prevent others from interfering your ZNodes, you can set the ACLProvider to return CREATOR_ALL_ACL and set the authorization to digest as shown above. Only the CuratorFramework instances using the same authorization string ("username:password") will be able to access those ZNodes. But it will not prevent others from creating ZNodes in paths which are not interfering with yours.
Hope you found what you want :-)

It wasn't part of the original question, but I thought I would share a solution I came up with in which the credentials used determine the access level.
I didn't have much luck finding any examples and kept ending up on this page so maybe it will help someone else. I dug through the source code of Curator Framework and luckily the org.apache.curator.framework.recipes.leader.TestLeaderAcls class was there to point me in the right direction.
So in this example:
One generic client used across multiple apps which only needs to read data from ZK.
Another admin client has the ability to read, delete, and update nodes in ZK.
Read-only or admin access is determined by the credentials used.
FULL-CONTROL ADMIN CLIENT
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.List;
import org.apache.curator.RetryPolicy;
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.curator.framework.api.ACLProvider;
import org.apache.curator.retry.ExponentialBackoffRetry;
import org.apache.zookeeper.ZooDefs;
import org.apache.zookeeper.data.ACL;
import org.apache.zookeeper.data.Id;
import org.apache.zookeeper.server.auth.DigestAuthenticationProvider;
public class AdminClient {
protected static CuratorFramework client = null;
public void initializeClient() throws NoSuchAlgorithmException {
String zkConnectString = "127.0.0.1:2181";
RetryPolicy retryPolicy = new ExponentialBackoffRetry(1000, 3);
final List<ACL> acls = new ArrayList<>();
//full-control ACL
String zkUsername = "adminuser";
String zkPassword = "adminpass";
String fullControlAuth = zkUsername + ":" + zkPassword;
String fullControlDigest = DigestAuthenticationProvider.generateDigest(fullControlAuth);
ACL fullControlAcl = new ACL(ZooDefs.Perms.ALL, new Id("digest", fullControlDigest));
acls.add(fullControlAcl);
//read-only ACL
String zkReadOnlyUsername = "readuser";
String zkReadOnlyPassword = "readpass";
String readOnlyAuth = zkReadOnlyUsername + ":" + zkReadOnlyPassword;
String readOnlyDigest = DigestAuthenticationProvider.generateDigest(readOnlyAuth);
ACL readOnlyAcl = new ACL(ZooDefs.Perms.READ, new Id("digest", readOnlyDigest));
acls.add(readOnlyAcl);
//create the client with full-control access
client = CuratorFrameworkFactory.builder()
.connectString(zkConnectString)
.retryPolicy(retryPolicy)
.authorization("digest", fullControlAuth.getBytes())
.aclProvider(new ACLProvider() {
#Override
public List<ACL> getDefaultAcl() {
return acls;
}
#Override
public List<ACL> getAclForPath(String string) {
return acls;
}
})
.build();
client.start();
//Now create, read, delete ZK nodes
}
}
READ-ONLY CLIENT
import java.security.NoSuchAlgorithmException;
import org.apache.curator.RetryPolicy;
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.curator.retry.ExponentialBackoffRetry;
public class ReadOnlyClient {
protected static CuratorFramework client = null;
public void initializeClient() throws NoSuchAlgorithmException {
String zkConnectString = "127.0.0.1:2181";
RetryPolicy retryPolicy = new ExponentialBackoffRetry(1000, 3);
String zkReadOnlyUsername = "readuser";
String zkReadOnlyPassword = "readpass";
String readOnlyAuth = zkReadOnlyUsername + ":" + zkReadOnlyPassword;
client = CuratorFrameworkFactory.builder()
.connectString(zkConnectString)
.retryPolicy(retryPolicy)
.authorization("digest", readOnlyAuth.getBytes())
.build();
client.start();
//Now read ZK nodes
}
}

azure java sdk authentication

I would like to list available IP VM's in the new Azure portal using Java SDK.
Couple of years back in the good old classic portal, I had followed the usual management certificate procedure to access vm's,create vm's and work with Azure Endpoints.
Fast fwd now I see that they have used a new portal and new mechanisms to interact with Java SDK. I read somewhere in the above link that with the old way with certificates, I can manage only the class portal resources.
I'm trying to code a simple program which authenticates and lists the vm's of the new portal as a start. Seems like they have complicated it a lot.
I followed the below link to "Create service principal with password"
https://azure.microsoft.com/en-us/documentation/articles/resource-group-authenticate-service-principal/
Then I went to this link
https://azure.microsoft.com/en-us/documentation/samples/resources-java-manage-resource-group/
which asked me go the "See how to create an Auth file" link in above page
(mine is not a webapp and when I try to create the AD as a native client application, it is not allowing me to save keys in configure tab, so I had to create a web app)
After doing all this, I got stuck with this below error
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
'authority' Uri should have at least one segment in the path (i.e.https://<host>/<path>/...)
java.lang.IllegalArgumentException: 'authority' Uri should have at least one segment in the path (i.e. https://<host>/<path>/...)
at com.microsoft.aad.adal4j.AuthenticationAuthority.detectAuthorityType(AuthenticationAuthority.java:190)
at com.microsoft.aad.adal4j.AuthenticationAuthority.<init>(AuthenticationAuthority.java:73)
When I checked it says that the error is because I don't have a valid client application id in your Azure Active Directory.
Is there any simple way to authenticate and start using the API's?

#Vikram, I suggest that you can try to refer to the article to create an application on AAD.
Then you can follow the code below to get the access token for authentication.
// The parameters include clientId, clientSecret, tenantId, subscriptionId and resourceGroupName.
private static final String clientId = "<client-id>";
private static final String clientSecret = "<key>";
private static final String tenantId = "<tenant-id>";
private static final String subscriptionId = "<subscription-id>";
// The function for getting the access token via Class AuthenticationResult
private static AuthenticationResult getAccessTokenFromServicePrincipalCredentials()
throws ServiceUnavailableException, MalformedURLException, ExecutionException, InterruptedException {
AuthenticationContext context;
AuthenticationResult result = null;
ExecutorService service = null;
try {
service = Executors.newFixedThreadPool(1);
// TODO: add your tenant id
context = new AuthenticationContext("https://login.microsoftonline.com/" + tenantId, false, service);
// TODO: add your client id and client secret
ClientCredential cred = new ClientCredential(clientId, clientSecret);
Future<AuthenticationResult> future = context.acquireToken("https://management.azure.com/", cred, null);
result = future.get();
} finally {
service.shutdown();
}
if (result == null) {
throw new ServiceUnavailableException("authentication result was null");
}
return result;
}
String accessToken = getAccessTokenFromServicePrincipalCredentials().getAccessToken();
If you want to list the VMs on new portal, you can try to use the REST API List the resources in a subscription to get all resources and filter the VMs via the resource type Microsoft.Compute/virtualMachines.
Hope it helps.

SoundCloud API wrapper, how to find a users favorites in JAVA?

I have written the code in java that creates an instance of the wrapper and verifies the user's email and password for their account, however with discontinued support for the java SoundCloud API I can't seem to find a way to get a users' likes from this point and I've looked up all the documentation and examples but none seem to work when implemented.
PS. I changed the client id, client secret, and username and password for security so ignore that in the code.
import com.soundcloud.api.ApiWrapper;
import com.soundcloud.api.Token;
import java.io.File;
/**
* Creates an API wrapper instance, obtains an access token and serializes the
* wrapper to disk. The serialized wrapper can then be used for subsequent
* access to resources without re-authenticating
*
* #see GetResource
*/
public final class CreateWrapper {
public static final File WRAPPER_SER = new File("wrapper.ser");
public static void main(String[] args) throws Exception {
final ApiWrapper soundCloud = new ApiWrapper(
"client_id", /*client id*/
"client_secret" /* client_secret */,
null /* redirect URI */,
null /* token */);
Token token;
token = soundCloud.login("username#username.com" /* login */, "password" /* password */);
System.out.println("got token from server: " + token);
// in this example the whole wrapper is serialised to disk -
// in a real application you would just save the tokens and usually have the client_id/client_secret
// hardcoded in the application, as they rarely change
soundCloud.toFile(WRAPPER_SER);
System.out.println("wrapper serialised to " + WRAPPER_SER);
}
}

Look at the developers documentation for the me endpoint
All subresources that are documented in /users are also available via /me. For example /me/tracks will return the tracks owned by the user. Additionally you can access the users dashboard activities through /me/activities and his or her connected external accounts through /me/connections.
https://developers.soundcloud.com/docs/api/reference#me
GET /users/{id}/favorites list of tracks favorited by the user