S3 Integration Guide logoS3 Integration Guide

Last updated Aug 28th, 2024

Overview

Common Room supports using Amazon S3 for both importing data into and exporting data out of Common Room. Importing data via S3 may be suitable for you if you're interested in setting up a recurring import of data from your warehouse into Common Room. Exporting data via S3 can enable you to sync data into your warehouse for any custom reporting or analysis you may wish to run.

Setup

When setting up an integration via S3, we recommend that you create a new S3 bucket with appropriate permissions and roles so you can have full control over the life cycle of the data. Setting up the connection involves the following steps:

  1. [Common Room] Provide an `externalId` to be used during bucket setup.
  2. [Customer] Create the S3 bucket using the provided `externalId`.
  3. [Customer] Provide your Common Room contact with the S3 bucket's `prefix`, `bucket name`, `role arn`, and `region`.
  4. [Common Room] Validate that data can be read from and / or written to the bucket.
  5. [Common Room + Customer] Set up the recurring imports / exports.

To help make the bucket setup as easy as possible, here is a sample Pulumi snippet that can be used to set up the S3 bucket. This sample code illustrates how you can use a single bucket for both imports and exports but you can also choose to set up separate buckets if needed. 

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const EXTERNAL_ID = "COMMON_ROOM_TO_PROVIDE_EXTERNAL_ID"; // to be provided by Common Room team

const READ_DATA_PREFIX: string | undefined = "DEFINE_YOUR_READ_PREFIX"; // this can be be undefined if no data imports are configured
const WRITE_DATA_PREFIX: string | undefined = "DEFINE_YOUR_WRITE_PREFIX"; // this can be be undefined if no data exports are configured

// Create an S3 bucket, choose your own bucket name
const bucket = new aws.s3.Bucket("common-room-shared-bucket");

// Create an IAM role that will be assumable by Common Room
const role = new aws.iam.Role("common-room-access-role", {
  assumeRolePolicy: {
    Version: "2012-10-17",
    Statement:
      [{
        Effect: "Allow",
        Principal: {
          AWS: "arn:aws:iam::322919613312:root",
        },
        Action: "sts:AssumeRole",
        Condition: {
          StringEquals: {
            "sts:ExternalId": EXTERNAL_ID,
          },
        },
      },
    ],
  },
});

if (READ_DATA_PREFIX != null) {
  // Allows Common Room to read and write to the prefix
  const policy = new aws.iam.RolePolicy(
    `common-room-access-${READ_DATA_PREFIX}`,
    {
      role: role.id,
      policy: bucket.arn.apply((arn) =>
        JSON.stringify({
          Version: "2012-10-17",
          Statement: [
            {
              Sid: "AllowListingOfPrefix",
              Action: ["s3:ListBucket"],
              Effect: "Allow",
              Resource: arn,
              Condition: {
                StringLike: { "s3:prefix": `${READ_DATA_PREFIX}/*`               },
              },
            },
            {
              Sid: "AllowAllActionsInPrefix",
              Effect: "Allow",
              Action: ["s3:GetObject", "s3:PutObject"],
              Resource: [`${arn}/${READ_DATA_PREFIX}/*`],
            },
          ],
        })
      ),
    }
  );
}

if (WRITE_DATA_PREFIX != null) {
  // Ensures that the objects written by Common Room are readable
  new aws.s3.BucketOwnershipControls("common-room-s3-object-ownership", {
    bucket: bucket.id,
    rule: {
      objectOwnership: "BucketOwnerEnforced",
    },
  });

  // Allows Common Room to read and write to the prefix
  const policy = new aws.iam.RolePolicy(
    `common-room-access-${WRITE_DATA_PREFIX}`,
    {
      role: role.id,
      policy: bucket.arn.apply((arn) =>
        JSON.stringify({
          Version: "2012-10-17",
          Statement: [
            {
              Sid: "AllowListingOfPrefix",
              Action: ["s3:ListBucket"],
              Effect: "Allow",
              Resource: arn,
              Condition: {
                StringLike: { "s3:prefix": `${WRITE_DATA_PREFIX}/*`  },
              },
            },
            {
              Sid: "AllowAllActionsInPrefix",
              Effect: "Allow",
              Action: ["s3:GetObject", "s3:PutObject"],
              Resource: [`${arn}/${WRITE_DATA_PREFIX}/*`],
            },
          ],
        })
      ),
    }
  );
}

export const bucketArn = bucket.arn;
export const bucketRegion = bucket.region;
export const roleArn = role.arn;

Please contact us if you're interested in exploring this option and have any questions!

Requirements

Our S3 integration is available on an Enterprise plan. Please work with your Common Room contact for more information.

Didn't find your answer?Get in touch· Visit ourhelp center