S3 Integration Guide
Last updated Oct 9th, 2024
Overview
Common Room supports using Amazon S3 for both importing data into and exporting data out of Common Room. Importing data via S3 may be suitable for you if you're interested in setting up a recurring import of data from your warehouse into Common Room. Exporting data via S3 can enable you to sync data into your warehouse for any custom reporting or analysis you may wish to run.
Setup
When setting up an integration via S3, we recommend that you create a new S3 bucket with appropriate permissions and roles so you can have full control over the life cycle of the data. Setting up the connection involves the following steps:
- [Common Room] Provide an `externalId` to be used during bucket setup.
- [Customer] Create the S3 bucket using the provided `externalId`.
- [Customer] Provide your Common Room contact with the S3 bucket's `prefix`, `bucket name`, `role arn`, and `region`.
- [Common Room] Validate that data can be read from and / or written to the bucket.
- [Common Room + Customer] Set up the recurring imports / exports.
To help make the bucket setup as easy as possible, here is a sample Pulumi snippet that can be used to set up the S3 bucket. This sample code illustrates how you can use a single bucket for both imports and exports but you can also choose to set up separate buckets if needed.
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
const EXTERNAL_ID = "COMMON_ROOM_TO_PROVIDE_EXTERNAL_ID"; // to be provided by Common Room team
const READ_DATA_PREFIX: string | undefined = "DEFINE_YOUR_READ_PREFIX"; // this can be be undefined if no data imports are configured
const WRITE_DATA_PREFIX: string | undefined = "DEFINE_YOUR_WRITE_PREFIX"; // this can be be undefined if no data exports are configured
// Create an S3 bucket, choose your own bucket name
const bucket = new aws.s3.Bucket("common-room-shared-bucket", {
bucket: "common-room-shared-bucket",
});
// Create an IAM role that will be assumable by Common Room
const role = new aws.iam.Role("common-room-access-role", {
assumeRolePolicy: {
Version: "2012-10-17",
Statement:
[{
Effect: "Allow",
Principal: {
AWS: "arn:aws:iam::322919613312:root",
},
Action: "sts:AssumeRole",
Condition: {
StringEquals: {
"sts:ExternalId": EXTERNAL_ID,
},
},
},
],
},
});
if (READ_DATA_PREFIX != null) {
// Allows Common Room to read and write to the prefix
const policy = new aws.iam.RolePolicy(
`common-room-access-${READ_DATA_PREFIX}`,
{
role: role.id,
policy: bucket.arn.apply((arn) =>
JSON.stringify({
Version: "2012-10-17",
Statement: [
{
Sid: "AllowListingOfPrefix",
Action: ["s3:ListBucket"],
Effect: "Allow",
Resource: arn,
Condition: {
StringLike: { "s3:prefix": `${READ_DATA_PREFIX}/*` },
},
},
{
Sid: "AllowAllActionsInPrefix",
Effect: "Allow",
Action: ["s3:GetObject", "s3:PutObject"],
Resource: [`${arn}/${READ_DATA_PREFIX}/*`],
},
],
})
),
}
);
}
if (WRITE_DATA_PREFIX != null) {
// Ensures that the objects written by Common Room are readable
new aws.s3.BucketOwnershipControls("common-room-s3-object-ownership", {
bucket: bucket.id,
rule: {
objectOwnership: "BucketOwnerEnforced",
},
});
// Allows Common Room to read and write to the prefix
const policy = new aws.iam.RolePolicy(
`common-room-access-${WRITE_DATA_PREFIX}`,
{
role: role.id,
policy: bucket.arn.apply((arn) =>
JSON.stringify({
Version: "2012-10-17",
Statement: [
{
Sid: "AllowListingOfPrefix",
Action: ["s3:ListBucket"],
Effect: "Allow",
Resource: arn,
Condition: {
StringLike: { "s3:prefix": `${WRITE_DATA_PREFIX}/*` },
},
},
{
Sid: "AllowAllActionsInPrefix",
Effect: "Allow",
Action: ["s3:GetObject", "s3:PutObject"],
Resource: [`${arn}/${WRITE_DATA_PREFIX}/*`],
},
],
})
),
}
);
}
export const bucketArn = bucket.arn;
export const bucketRegion = bucket.region;
export const roleArn = role.arn;
Please contact us if you're interested in exploring this option and have any questions!
Availability
Our S3 integration is included on the Enterprise plan and is available on the Team plan as an add-on. Please work with your Common Room contact for more information.