Aws Cloud Disk (serverless media gallery)
Prologue
In this article, I would like to describe an application from AWS Serverless repository. This is a fully serverless application that requires no constant expenses and can be deployed and forget. To be precise: all expenses for deployed applications should fit AWS Free Tier. The expenses might go up only in case you start extensively upload data on the deployed S3 bucket. What does it mean “Extensively”. Well, 100G of data uploaded will result in ~ 20$. And most expensive part is image content recognition with AWS Rekognition. For uploads less than 1G of data — all resources will use AWS Free Tier only.
Application overview
The application is a serverless implementation of the online media gallery. The features that are supported:
- serving loaded images and videos as a gallery via Web Browser
- providing a preview for images (which is created based on original images) and videos
- serving a dry list of not media files (with options to download)
- images content detection and providing search feature based on detected content
In general, the application appears to have features like Google Photos. However, you can imagine that I spend fewer man/hours on my application than Google on Google Photos. Therefore do not expect my app to be superior to Google Photos🙂.
What is under the hood
The application is built fully on serverless services. That means that no expenses occur unless you use the application. In the overview, I mentioned that for playing around — you should encounter no fees according to the current AWS Free Tier. However, for extensive usage, some expenses might occur.
Services used:
- S3 for storing content itself and serving it via WEB
- SNS as a glue between events on S3 and Computing logic
- Lambdas for computing logic of the service
- DynamoDB for storing detected image content labels
- AWS Rekognition for detecting content of the images
- AWS CloudFront for providing REST interface for Search Lamdba
The solution, step by step
Now it is time to take close look at how it works under the good.
The flow starts from the point where you upload images/files on S3.
- Everything starts from the point where a user uploads media files (2) in S3 bucket (1).
- After that, SNS event (3) is triggered about the new S3 object.
- A new thumbnail image is generated (4) out of the original media file and stored in S3.
- index.html is generated (5) for each directory of the image gallery.
- index.html is be served via WEB (6) and uses generated thumbnail.
This is the base flow. In parallel with that, the image content recognition flow is started:
- List of all media files are stored in DynamoDb (7).
- Content of the image is recognized (8)(9). Then it is saved as a list of labels in DynamoDD.
- Now a user can use search.html which call CloudFront which proxy request to search lambda (10). The lambda serves as a server-side rendering for search.html. It fetches data from DynamoDB and renders search.html with results.
Deploying the sample solution
Since Stack name is used as a suffix in names of almost all created resources, therefore it should meet requirements: :
- it consists of lower case Latin symbols or digits
- no special symbols (except for dash)
Otherwise, you can get exceptions during stack creation like: “Bucket name should not contain uppercase characters.”
After the successful deployment of the template, there is one more step to perform manually: copy init files.
So first check that you still on the “Overview” tab,
scroll down to the “Resources” section:
Pick the name of the s3 bucket. After that: just execute copying init files to your newly created S3 bucket (do not forget to put your own bucket name in the destination):
aws s3 cp s3://ruholnikov-artifacts-us-east-1/init-data/ s3://serverlessrepo-aws-cloud-disk1-bucket/ — acl=public-read — recursive
After that, you can go to the deployment itself to check the output (it has the link to your newly created media gallery:
Congrats! You got your media gallery. As of the start, you have one image file in your gallery. Also, you have Search form that has a list of labels for all recognized content. Only one should be detected for the beginning. Check it.
How to use it
The gallery is created, what is next? The next step — start using it. The weak side of this app is the upload interface — I have not come up with anything :). So just upload files using AWS console via awscli into data folder (under the data key) inside your S3 bucket. You will see those files will automatically appear on the WEB UI.
Links:
Aws Serveress repo:
us-east-1 region (N. Virginia):
eu-central-1 (Frankfurt):
Gitlab repository (source code):
