Skip to content

chore: Updates Project CFN resource handler to accept Lambda Proxy ARN as input & use as Http transport in Atlas SDK client #1300

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cfn-resources/project/cmd/resource/model.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 5 additions & 2 deletions cfn-resources/project/cmd/resource/resource.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,15 @@
"fmt"
"reflect"

"go.mongodb.org/atlas-sdk/v20231115014/admin"

"github.com/aws-cloudformation/cloudformation-cli-go-plugin/cfn/handler"
"github.com/aws/aws-sdk-go/service/cloudformation"

"github.com/mongodb/mongodbatlas-cloudformation-resources/util"
"github.com/mongodb/mongodbatlas-cloudformation-resources/util/constants"
"github.com/mongodb/mongodbatlas-cloudformation-resources/util/progressevent"
"github.com/mongodb/mongodbatlas-cloudformation-resources/util/validator"
"go.mongodb.org/atlas-sdk/v20231115014/admin"
)

var CreateRequiredFields = []string{constants.OrgID, constants.Name}
Expand All @@ -45,7 +47,8 @@
return nil, errEvent
}

client, peErr := util.NewAtlasClient(&req, currentModel.Profile)
// client, peErr := util.NewAtlasClient(&req, currentModel.Profile)

Check failure on line 50 in cfn-resources/project/cmd/resource/resource.go

View workflow job for this annotation

GitHub Actions / lint

commentedOutCode: may want to remove commented-out code (gocritic)
client, peErr := util.NewAtlasClientWithLambdaProxySupport(&req, currentModel.Profile, currentModel.LambdaProxyArn)
if peErr != nil {
return nil, peErr
}
Expand Down
12 changes: 12 additions & 0 deletions cfn-resources/project/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ To declare this entity in your AWS CloudFormation template, use the following sy
"<a href="#withdefaultalertssettings" title="WithDefaultAlertsSettings">WithDefaultAlertsSettings</a>" : <i>Boolean</i>,
"<a href="#projectsettings" title="ProjectSettings">ProjectSettings</a>" : <i><a href="projectsettings.md">projectSettings</a></i>,
"<a href="#profile" title="Profile">Profile</a>" : <i>String</i>,
"<a href="#lambdaproxyarn" title="LambdaProxyArn">LambdaProxyArn</a>" : <i>String</i>,
"<a href="#projectteams" title="ProjectTeams">ProjectTeams</a>" : <i>[ <a href="projectteam.md">projectTeam</a>, ... ]</i>,
"<a href="#projectapikeys" title="ProjectApiKeys">ProjectApiKeys</a>" : <i>[ <a href="projectapikey.md">projectApiKey</a>, ... ]</i>,
"<a href="#regionusagerestrictions" title="RegionUsageRestrictions">RegionUsageRestrictions</a>" : <i>String</i>,
Expand All @@ -37,6 +38,7 @@ Properties:
<a href="#withdefaultalertssettings" title="WithDefaultAlertsSettings">WithDefaultAlertsSettings</a>: <i>Boolean</i>
<a href="#projectsettings" title="ProjectSettings">ProjectSettings</a>: <i><a href="projectsettings.md">projectSettings</a></i>
<a href="#profile" title="Profile">Profile</a>: <i>String</i>
<a href="#lambdaproxyarn" title="LambdaProxyArn">LambdaProxyArn</a>: <i>String</i>
<a href="#projectteams" title="ProjectTeams">ProjectTeams</a>: <i>
- <a href="projectteam.md">projectTeam</a></i>
<a href="#projectapikeys" title="ProjectApiKeys">ProjectApiKeys</a>: <i>
Expand Down Expand Up @@ -105,6 +107,16 @@ _Type_: String

_Update requires_: [Replacement](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-updating-stacks-update-behaviors.html#update-replacement)

#### LambdaProxyArn

lambda arn

_Required_: No

_Type_: String

_Update requires_: [No interruption](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-updating-stacks-update-behaviors.html#update-no-interrupt)

#### ProjectTeams

Teams to which the authenticated user has access in the project specified using its unique 24-hexadecimal digit identifier.
Expand Down
51 changes: 51 additions & 0 deletions cfn-resources/project/lambdaproxy/ec2proxy.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
from flask import Flask, request, Response
import requests
import logging

app = Flask(__name__)

logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)

# NOTE: additional configuration would be required to also support Realm
TARGET_SERVER = "https://cloud-dev.mongodb.com"
logger.debug(f"EC2 Proxy configured with TARGET_SERVER: {TARGET_SERVER}")

@app.route('/', defaults={'path': ''}, methods=["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD"])
@app.route('/<path:path>', methods=["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD"])
def proxy(path):
logger.debug(f"Received request for path: {path}")
# Build the target URL
target_url = f"{TARGET_SERVER}/{path}"
logger.debug(f"Target URL: {target_url}")

# Copy the incoming headers
headers = {key: value for key, value in request.headers if key.lower() != 'host'}
logger.debug(f"Request headers: {headers}")

# Forward the request to Atlas
try:
resp = requests.request(
method=request.method,
url=target_url,
headers=headers,
data=request.get_data(),
cookies=request.cookies,
allow_redirects=False
)
# logger.debug(f"Received response from target server - Status: {resp.status_code}, Headers: {resp.headers}")
except Exception as e:
logger.exception("Error forwarding the request to the target server:")
return Response("Error forwarding request", status=500)

excluded_headers = ['content-encoding', 'content-length', 'transfer-encoding', 'connection']
response_headers = [(name, value) for (name, value) in resp.raw.headers.items()
if name.lower() not in excluded_headers]

response = Response(resp.content, resp.status_code, response_headers)
logger.debug(f"Returning proxied response with status: {resp.status_code}")
return response

if __name__ == '__main__':
logger.debug("Starting EC2 Proxy Flask app on 0.0.0.0:80")
app.run(host='0.0.0.0', port=80)
73 changes: 73 additions & 0 deletions cfn-resources/project/lambdaproxy/lambda.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
import json
import logging
import requests # this requires adding request layer to lambda function
from urllib.parse import urlparse, urlunparse

logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger()

EC2_PROXY_ENDPOINT = "http://XX.X.X.XX" # Replace with private IP of EC2 proxy running Python/Flask
if not EC2_PROXY_ENDPOINT:
logger.error("EC2_PROXY_ENDPOINT is not set")
raise ValueError("EC2_PROXY_ENDPOINT is not set")
logger.debug(f"EC2_PROXY_ENDPOINT: {EC2_PROXY_ENDPOINT}")

def lambda_handler(event, context):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is how customer should define the lambda function in their own AWS Account. right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

"""
Expected event example:
{
"method": "GET",
"url": "https://cloud-dev.mongodb.com/api/atlas/v2/groups",
"headers": {"Header-Key": "value", ...},
"body": "request body as string"
}

This Lambda function should be deployed in a private subnet. It's corresponding SG
only allows traffic to EC2 proxy running Python/Flask
"""
logger.debug(f"Received event: {json.dumps(event)}") # TODO: remove
try:
method = event.get("method")
url = event.get("url")
headers = event.get("headers", {})
body = event.get("body", None)

if method is None or url is None:
msg = "Missing 'method' or 'url' in the event payload"
logger.error(msg)
raise ValueError(msg)

logger.debug(f"Forwarding request - Method: {method}, URL: {url}, Headers: {headers}, Body: {body}")

parsed_url = urlparse(url)
if parsed_url.scheme and parsed_url.netloc:
# Extract the path and query string because incoming URL will be in format:
# https://www.cloud.mongodb.com/api/atlas/v2/groups?param=value
path = parsed_url.path or ""
query = f"?{parsed_url.query}" if parsed_url.query else ""
new_url = path + query
logger.debug(f"Extracted relative URL: {new_url} from absolute URL: {url}")
else:
new_url = url
logger.debug(f"URL is relative: {new_url}")

# Construct URL for the EC2 proxy
full_url = EC2_PROXY_ENDPOINT.rstrip("/") + new_url
logger.debug(f"Constructed full URL for proxy: {full_url}")

# Forward request to the EC2 proxy
response = requests.request(method, full_url, headers=headers, data=body)
logger.debug(f"Response from EC2 proxy - Status: {response.status_code}, Headers: {dict(response.headers)}, Body: {response.text}")

return {
"statusCode": response.status_code,
"headers": dict(response.headers),
"body": response.text
}
except Exception as e:
logger.exception("Error processing the request:")
return {
"statusCode": 500,
"headers": {},
"body": json.dumps({"error": str(e)})
}
20 changes: 16 additions & 4 deletions cfn-resources/project/mongodb-atlas-project.json
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,10 @@
"description": "Profile used to provide credentials information, (a secret with the cfn/atlas/profile/{Profile}, is required), if not provided default is used",
"default": "default"
},
"LambdaProxyArn": {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, LambdaArn may be taken as an input in the Profile

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would definitely have it in the Profile so we don't need to change every single resource

"type": "string",
"description": "lambda arn"
},
"ProjectTeams": {
"items": {
"$ref": "#/definitions/projectTeam"
Expand Down Expand Up @@ -177,22 +181,30 @@
"handlers": {
"create": {
"permissions": [
"secretsmanager:GetSecretValue"
"secretsmanager:GetSecretValue",
"lambda:InvokeFunction",
"lambda:GetFunction"
]
},
"read": {
"permissions": [
"secretsmanager:GetSecretValue"
"secretsmanager:GetSecretValue",
"lambda:InvokeFunction",
"lambda:GetFunction"
]
},
"update": {
"permissions": [
"secretsmanager:GetSecretValue"
"secretsmanager:GetSecretValue",
"lambda:InvokeFunction",
"lambda:GetFunction"
]
},
"delete": {
"permissions": [
"secretsmanager:GetSecretValue"
"secretsmanager:GetSecretValue",
"lambda:InvokeFunction",
"lambda:GetFunction"
]
}
},
Expand Down
2 changes: 2 additions & 0 deletions cfn-resources/project/resource-role.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ Resources:
Statement:
- Effect: Allow
Action:
- "lambda:GetFunction"
- "lambda:InvokeFunction"
- "secretsmanager:GetSecretValue"
Resource: "*"
Outputs:
Expand Down
Loading
Loading