System Design

S3 File Upload Pipeline

This is a fairly typical system found in most companies. It is common enough and provides a good introduction to both system design and cloud based pipelines.

Beginner/Intermidiate4 videosCloud, File Upload, S3
Video 1

Part 1 - Introduction and constraint dicovery

In this video we define the initial constraints of the system

Video available

Constraint discovery
Video 2

Part 2 - Introduction and basic S3 signed url architecture

In this video we cover the basic design of the application in the subtubes dojo.

Video available

Video 3

Part 3 - File upload process system design

In this video I guide you through the design of the actual S3 upload pipeline. We create a queue and a quarantine bucket.

Video available

S3 file upload pipeline
Video 4

Part 4 - Design Dojo wrap up and lambda processing

In this video I guide you through the last major hurdle in the design where we create the lambda and move the image from the quarantine bucket to the final bucket.

Video available

AWS Lambda
S3 File Upload Pipeline | System Design | Subtubes