AWS S3 Integration Guide

Last updated Nov 28th, 2024

Overview

Common Room supports importing data from Amazon S3. Importing data via S3 may be suitable for you if you're interested in setting up a recurring import of data from your warehouse into Common Room.

Setup

When setting up an integration via S3, we recommend that you create a new S3 bucket with appropriate permissions and roles so you can have full control over the life cycle of the data. Setting up the connection involves the following steps:

[Common Room] Provide an `externalId` to be used during bucket setup.
[Customer] Create the S3 bucket using the provided `externalId`.
[Customer] Provide your Common Room contact with the S3 bucket's `prefix`, `bucket name`, `role arn`, and `region`.
[Common Room] Validate that data can be read from and / or written to the bucket.
[Common Room + Customer] Set up the recurring imports by following our general configuration instructions.

To help make the bucket setup as easy as possible, here is a sample Pulumi snippet that can be used to set up the S3 bucket. This sample code illustrates how you can use a single bucket for both imports and exports (in case you choose to also export data to S3) but you can also choose to set up separate buckets if needed.

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const EXTERNAL_ID = "COMMON_ROOM_TO_PROVIDE_EXTERNAL_ID"; // to be provided by Common Room team

const READ_DATA_PREFIX: string | undefined = "DEFINE_YOUR_READ_PREFIX"; // this can be be undefined if no data imports are configured
const WRITE_DATA_PREFIX: string | undefined = "DEFINE_YOUR_WRITE_PREFIX"; // this can be be undefined if no data exports are configured

// Create an S3 bucket, choose your own bucket name
const bucket = new aws.s3.Bucket("common-room-shared-bucket", {
  bucket: "common-room-shared-bucket",
});

// Create an IAM role that will be assumable by Common Room
const role = new aws.iam.Role("common-room-access-role", {
  assumeRolePolicy: {
    Version: "2012-10-17",
    Statement:
      [{
        Effect: "Allow",
        Principal: {
          AWS: "arn:aws:iam::322919613312:root",
        },
        Action: "sts:AssumeRole",
        Condition: {
          StringEquals: {
            "sts:ExternalId": EXTERNAL_ID,
          },
        },
      },
    ],
  },
});

if (READ_DATA_PREFIX != null) {
  // Allows Common Room to read and write to the prefix
  const policy = new aws.iam.RolePolicy(
    `common-room-access-${READ_DATA_PREFIX}`,
    {
      role: role.id,
      policy: bucket.arn.apply((arn) =>
        JSON.stringify({
          Version: "2012-10-17",
          Statement: [
            {
              Sid: "AllowListingOfPrefix",
              Action: ["s3:ListBucket"],
              Effect: "Allow",
              Resource: arn,
              Condition: {
                StringLike: { "s3:prefix": `${READ_DATA_PREFIX}/*`               },
              },
            },
            {
              Sid: "AllowAllActionsInPrefix",
              Effect: "Allow",
              Action: ["s3:GetObject", "s3:PutObject"],
              Resource: [`${arn}/${READ_DATA_PREFIX}/*`],
            },
          ],
        })
      ),
    }
  );
}

if (WRITE_DATA_PREFIX != null) {
  // Ensures that the objects written by Common Room are readable
  new aws.s3.BucketOwnershipControls("common-room-s3-object-ownership", {
    bucket: bucket.id,
    rule: {
      objectOwnership: "BucketOwnerEnforced",
    },
  });

  // Allows Common Room to read and write to the prefix
  const policy = new aws.iam.RolePolicy(
    `common-room-access-${WRITE_DATA_PREFIX}`,
    {
      role: role.id,
      policy: bucket.arn.apply((arn) =>
        JSON.stringify({
          Version: "2012-10-17",
          Statement: [
            {
              Sid: "AllowListingOfPrefix",
              Action: ["s3:ListBucket"],
              Effect: "Allow",
              Resource: arn,
              Condition: {
                StringLike: { "s3:prefix": `${WRITE_DATA_PREFIX}/*`  },
              },
            },
            {
              Sid: "AllowAllActionsInPrefix",
              Effect: "Allow",
              Action: ["s3:GetObject", "s3:PutObject"],
              Resource: [`${arn}/${WRITE_DATA_PREFIX}/*`],
            },
          ],
        })
      ),
    }
  );
}

export const bucketArn = bucket.arn;
export const bucketRegion = bucket.region;
export const roleArn = role.arn;

Please contact us if you're interested in exploring this option and have any questions!

Availability

Our S3 integration is included on the Enterprise plan and is available on the Team plan as an add-on. Please work with your Common Room contact for more information.

Didn't find your answer?Get in touch· Visit ourhelp center

On this page