AWS Marketplace Dshop container does not comply with policy

[franckc]

The AWS Marketplace team informed us that the dshop container needs to be updated to comply with their new policies:

Application use requires customer Access Keys or Secret Keys as the primary means of accessing AWS resources. AMIs must not request or use access or secret keys from users to access AWS resources. If your AMI application requires access to the user account, it must be achieved through an AWS Identity and Access Management (IAM) role instantiated through AWS CloudFormation, which creates the instance and associates the appropriate role https://docs.aws.amazon.com/marketplace/latest/userguide/product-and-ami-policies.html

When pressed for more details, they added:

AMIs must not request or use access or secret keys from users to access AWS resources. If your AMI application requires access to the user account, it must be achieved through an AWS Identity and Access Management (IAM) role instantiated through AWS CloudFormation, which creates the instance and associates the appropriate role. Upon testing your product we discovered a request for the customers AWS Access Key ID and AWS Secret Access Key. Please note that this goes against policy.

It is not clear to me exactly what changes need to be made to the AWS dshop container to comply... Anyone has an idea?

[phyninja]

I'm just now learning about AWS CloudFormation, so I will refrain from commenting on

If your AMI application requires access to the user account, it must be achieved through an AWS Identity and Access Management (IAM) role instantiated through AWS CloudFormation, which creates the instance and associates the appropriate role

I noticed, however, that DShop asks for one's AWS Access Key ID and AWS Secret Key when setting up the super-admin account. This information is used to configure and deploy shops to an Amazon S3 "bucket". I think the first step to comply with AWS' security policies is to move away from collecting that information from DShop merchants [i.e. "AWS Marketplace Buyer"].

[phyninja]

@franckc

I'm still studying the documentation on AWS CloudFormation. Here's what I know:
CloudFormation is a service provided by Amazon to "simplify infrastructure management". It is supposed to help AWS users bring different (AWS) resources under a single management framework called a stack. In the case of DShop, "resources" may translate into:

• The AWS dshop container
• Service objects linked with Amazon S3, AWS SES, Amazon Route 53 etc.

I will need more time to identify

exactly what changes need to be made

Sincerely,
Rajath

[phyninja]

While it may be possible to release a new version of the DShop AMI from Origin's AWS Marketplace Seller account (linked below), it seems more likely that we will be required to create a new AMI altogether, with one of the following two licensing models:

License name	Description
Single AMI with CloudFormation Stack	A single AMI that uses an AWS CloudFormation template with pay-as-you go pricing.
AMI with contract pricing	A single AMI product or single AMI with CloudFormation stack that the buyer pays an upfront fee for.

Do you know whose AWS Marketplace Seller account we will use to make and submit changes?

Link: https://aws.amazon.com/marketplace/pp/prodview-6vq37prq3244e

[phyninja]

@franckc,

As detailed here, launching an updated Amazon Marketplace product for DShop will require that I co-ordinate with a support team at Amazon.

Could you respond to the question above, please?

[phyninja]

Hi @franckc

We know that the problem with the DShop solution on AWS marketplace is that it "asks" for a user's AWS credentials to "access AWS resources". To work around this issue, we have been asked by AWS Marketplace to update our source code to use AWS CloudFormation - an automation tool - to create an "Identity and Access Management (IAM) role" associated with a customer of the DShop solution on AWS Marketplace, and use that role to access AWS resources instead.

You previously asked what changes need to be made (to the DShop source code) to comply with AWS' security policies. Here are my thoughts:

• We must establish a policy that all shop admins who need to leverage AWS resources such as Amazon S3, AWS SES, etc., create an IAM role manually from their AWS Management console. We can include a guide to help them: https://github.com/OriginProtocol/dshop/blob/aws-container-compliance/docs/dshop/deployment/aws-iam-role.md

• We should alter our source code to use an AWS SDK method called ChainableTemporaryCredentials to create a set of temporary user credentials (accessKeyId and secretAccessKey) associated with the IAM role. These credentials can take the place of ones DShop currently uses to access AWS resources.
  For example, here's a code snippet I used to upload DShop data to Amazon S3:

async function configure({ networkConfig }) {
  //Creds of the IAM user
  AWS.config = new AWS.Config({
    accessKeyId: networkConfig.awsAccessKeyId,
    secretAccessKey: networkConfig.awsSecretAccessKey
  })

  tempCredentials = new AWS.ChainableTemporaryCredentials({
    params: {
      RoleArn: networkConfig.awsRoleARN,
      RoleSessionName: `DShop-aws-s3-session-${new Date().getTime()}`
    }
  })

  await tempCredentials.getPromise().then(() => {
    cachedClient = new S3({
      apiVersion: '2006-03-01',
      credentials: tempCredentials,
      region: networkConfig.awsRegion // Optional
    })
  })
}

Source code:

dshop/backend/logic/deploy/bucket/aws.js

Lines 37 to 44 in 43904f9

	function configure({ networkConfig }) {
	cachedClient = new S3({
	apiVersion: '2006-03-01',
	accessKeyId: networkConfig.awsAccessKeyId,
	secretAccessKey: networkConfig.awsSecretAccessKey,
	region: networkConfig.awsRegion // Optional
	})
	}

• Calling the ChainableTemporaryCredentials method is impossible without the request being "signed" by a set of "master (AWS) credentials". While we can continue to use networkConfig.awsAccessKeyId and networkConfig.awsSecretAccessKey to sign these requests for DShop instances launched outside of AWS Marketplace, AWS CloudFormation will need to serve as our delegate to perform the same task on instances launched from the AWS Marketplace AMI.
  The DShop source code therefore needs a way to identify whether an instance is launched from the AWS Marketplace AMI (or outside of it). With that information, we can hide (or show) the
  AWS Access Key Id
AWS Secret Access Key
  fields on the Network Configuration page.
  Update [4th Jan, 2022]:
  I've learned that using the CloudFormation service via the AWS SDK also requires that we sign requests with a set of master credentials.

[phyninja]

@franck @franckc,
I'm in need of advice to solve a problem relating to the creation of environment variables that can be used by DShop's FE.

Context:

The AWS Marketplace instance we're familiar with is generated using a script stored in DShop's repository. In the script, an ENV var called AWS_MARKETPLACE_DEPLOYMENT is first created with a value of true, before being stored in dshop/backend/.env

I'm not able to use this environment variable to show or hide the
AWS Access Key Id
AWS Secret Access Key
fields on the Network Configuration page of the FE:

dshop/shop/src/pages/super-admin/networks/_Form.js

Lines 206 to 225 in 249b52d

	const conditionallyRequestAWSCreds = () => {
	if (process.env.AWS_MARKETPLACE_DEPLOYMENT) {
	return
	} else {
	return (
	<div className="form-row">
	<div className="form-group col-md-6">
	<label>AWS Access Key ID</label>
	<input {...input('awsAccessKeyId')} />
	{Feedback('awsAccessKeyId')}
	</div>
	<div className="form-group col-md-6">
	<label>AWS Secret Access Key</label>
	<PasswordField field="awsSecretAccessKey" input={input} />
	{Feedback('awsSecretAccessKey')}
	</div>
	</div>
	)
	}
	}

Console-logging process.env resulted in the return of an empty object.

[franck]

Careful guys. @franck is my github account and I'm not the Franck you're trying to reach. Cheers Franck

…

On Mon, Jan 17, 2022 at 12:16 PM Rajath ***@***.***> wrote: @franck <https://github.com/franck>, I'm in need of advice to solve a problem relating to the creation of environment variables that can be used by DShop's FE. Context: The AWS Marketplace instance we're familiar with is generated using a script stored in DShop's repository. In the script <https://github.com/OriginProtocol/dshop/blob/f65491f003fa96271f9b6f7c6554d05ae039e737/backend/scripts/aws/marketplace/startup_script.sh#L67-L68>, an ENV var called AWS_MARKETPLACE_DEPLOYMENT is first created with a value of true, before being stored in dshop/backend/.env I'm not able to use this environment variable to show or hide the AWS Access Key Id AWS Secret Access Key fields on the Network Configuration page of the FE: https://github.com/OriginProtocol/dshop/blob/249b52df46139dbfa95979559e281d7fc02f23c7/shop/src/pages/super-admin/networks/_Form.js#L206-L225 Console-logging process.env resulted in the return of an empty object. — Reply to this email directly, view it on GitHub <#1011 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAABHW2SESHEHBN5VZNVH5TUWP3A7ANCNFSM5HULPALQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[phyninja]

@franckc,

I've edited the source code to make DShop's Amazon S3 integration compatible with AWS Marketplace's security policy. I've also had the chance to test those changes on a 'production' version of DShop running on AWS EC2.
Could you take a look at the PR, please?

[phyninja]

As of this writing, I've implemented changes that bring two (2) of the four (4) AWS services integrated with DShop in close compliance with AWS Marketplace's security policy. The two services are:

1. AWS S3
2. AWS SES

The principle on which both sets of changes work is the same:
An application running on an AWS EC2 instance can programatically obtain a set of temporary credentials using the "EC2 instance metadata service", before using those credentials to create clients for other "AWS resources" like S3, SES, etc. For this to work, however, the shop admin will have to

• Create an IAM role that allows access to the desired AWS resource
• Associate that IAM role with the EC2 instance