Repository

Looks good to me!

User Tools

Site Tools


kb:internet:services:s3:start

S3

Changelog

  • 2026-01-12: Init

2026-03-24 Tuesday

As it turns out, creating a budget is extremely straightforward:

Screenshot

Storage class comparison (updated from the old comparison), for us-east-1, as of 24 March (to two significant figures) and excluding tiered pricing:

Storage class Min storage (days) Storage (¢/GB/mth) PUTs (¢/'000) GETs (¢/'000) Lifecycle transfer (¢/'000) Transfer out (¢/GB)
Standard 0 2.3 0.5 0.04 0 9.0
Standard-IA 30 1.3 1.0 0.1 1.0
Glacier IR 90 0.4 2.0 1.0 2.0
Glacier Deep Archive 180 0.1 5.0 0.04 5.0
  • Glacier Deep Archive charges an additional 40KB per archived object (8KB at Standard and 32KB at Deep Archive) for metadata (e.g. low latency object listing). Glacier Deep Archive files require data retrieval charges as well, for bulk requests at 2.5¢/'000 requests AND 0.25¢/GB. Expensive, yeesh.
  • Minimum billable object size is 128KB.
  • PUT requests (file uploads via API) are charged similarly to COPY (e.g., between regions), POST (file uploads via browser), LIST (enumerate contents of bucket)
  • GET requests (file content retrieval) are charged similarly to SELECT requests (filter query on file content).
  • All data transfer in from internet into S3 is free, as well as between buckets in the same region. First 100GB/mth data transfer out also free, across all regions aggregated.
  • Minimum storage duration applies only upon lifecycle transition into said storage class (and not based on the total object storage duration).

Calculator to put all these into action: https://calculator.aws/#/createCalculator/S3

Suppose 1TB/mth worth of data split across approximately 20MB blobs.

  • This requires 50k/mth PUT requests. For 1% data retrieval daily for integrity tests, this yields 0.5k/day GET requests, or 15k/mth.
    • At Glacier IR prices, $1/mth in PUT and $0.15/mth in GET. Trivial amounts.
    • Data transfers are more costly, with 10GB/day * 30days = 300GB/mth total retrieval, or $27/mth.
    • 1TB corresponds to $4/mth storage.
  • Increasing the blob size to 50MB results in 20k PUT requests, or 40% of original cost for requests.
  • For Glacier Deep Archive, 50k objects = 0.4GB in Standard + 1.6GB in Glacier.
  • Total costs for 20MB blobs would correspond to (with 300GB retrieval), in $/TB/mth:
Storage class Total (excluding transfer out) Storage PUTs GETs Lifecycle transfer Transfer out
Standard 23 23 0.25 0.01 0 27
Standard-IA 14 13 0.5 0.02 0.5
Glacier IR 6 4 1 0.15 1
Glacier Deep Archive 6 1 + 0.01 (metadata) 2.5 0.01 2.5 27 + 0.38 (requests) + 0.75 (retrieval)

2026-03-23 Monday

Looks like the deploy keyword is intended more for serverless deploy. Use create-stack instead, with the --parameters argument to supply parameters.

# Create stack
aws cloudformation create-stack \
    --stack-name teststack \
    --template-body file://template.yaml \
    --capabilities CAPABILITY_IAM \
    --parameters ParameterKey=EnableVersioning,ParameterValue=Enabled
 
# Monitor stack status and view parameters
aws cloudformation describe-stacks --stack-name teststack \
    | yq '.Stacks[0].StackStatus'
 
# Delete stack
aws cloudformation delete-stack --stack-name teststack

CloudFormation policy so far

2026-03-22 Sunday

Going through the cfn best-practices page now, some hints:

  • Linting with cfn-lint: uvx cfn-lint -t [TEMPLATE]
    • A more basic template check is already available in AWS CLI with aws cloudformation validate-template --template-body file://[TEMPLATE], but is more limited in scope and cannot check more than 51kB templates (requires uploading and checking against a URL).
  • Migration of existing AWS resources to IaC using IaC generator
  • Sensitive parameters with AWS Secrets Manager
  • IAM least privilege with IAM Access Analyzer

Other miscellaneous things:

  • Multi-stack / Multi-region:
    • TaskCat for cfn template testing wrt multi-region deployment.
    • Use parameter exports to share variables between stacks.
    • Consider Rain for interactive CloudFormation workflows.
  • Organizational policies:

See templates at sample templates.

---

cfn templates allow the following root sections:

  • Resources: Specifies stack resources and their properties (e.g. unique logical ID, type, configuration).
  • Parameters: Allow users to pass values at runtime, and can be referenced in Resources and Outputs.
  • Outputs: Defines values returned when viewing a stack's properties (e.g. resource ID, URLs).

Other sections are intended for policy checks, i.e. Metadata, Rules, Conditions, Transform. The template reference is here.

---

Relevant resource policies for S3 are DeletionPolicy (whether resource should be retained when stack is deleted) and UpdateReplacePolicy (whether resource should be retained during replacement as part of stack update). Both are Delete by default. Can be assigned to Retain.

Pseudo-parameters: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html

---

Outputs:

Outputs:
  BucketName:
    Description: Name of S3 Bucket
    Value: !Ref S3Bucket  # defaults to bucket name
    Export:
      Name: !Sub '${AWS::StackName}-BucketName'
  • Describe stacks: aws cloudformation describe-stacks
  • Exports listed: aws cloudformation list-exports

2026-03-14 Saturday

Using AWS CloudFormation for IaC. Documentation here. Summary of key points:

  • Configuration comes as a template usually in YaML format, which creates a stack that bundles services together. changesets marking modifications between templates can be generated.

CloudFormation policies

Specific to S3 buckets: here is a 'Hello World' policy:

# helloworld.yaml
AWSTemplateFormatVersion: "2010-09-09"
Description: "Hello World"
Resources:
  S3Bucket:
    Type: "AWS::S3::Bucket"

Configure AWS CLI to reference the target region:

user:~$ aws configure
AWS Access Key ID [********************]: 
AWS Secret Access Key [********************]: 
Default region name [ap-southeast-1]: 
Default output format [yaml]:

Run a deploy:

user:~$ aws cloudformation deploy --stack-name teststack --template-file helloworld.yaml

Then find the stack initialized within the region in CloudFormation, e.g. ap-southeast-1. As well as in the respective service page.

Screenshots

Delete stack:

user:~$ aws cloudformation delete-stack --stack-name teststack

2026-01-12 Monday

Creating an S3 bucket... a couple of TODOs:

  1. IAM:
    • Create a user group: Name, Users, Permission policies.
    • Create a user: Name, Permissions.
    • A specific permission policy can be created as well, under Policies.

Some comments:

  • Role-based access is available from IAM: authentication using PKI, by creating CA from within AWS and (probably) issuing certificates to devices which can be used to request for access.
kb/internet/services/s3/start.txt · Last modified: 7 days ago (25 March 2026) by justin