반응형
블로그 이미지
개발자로서 현장에서 일하면서 새로 접하는 기술들이나 알게된 정보 등을 정리하기 위한 블로그입니다. 운 좋게 미국에서 큰 회사들의 프로젝트에서 컬설턴트로 일하고 있어서 새로운 기술들을 접할 기회가 많이 있습니다. 미국의 IT 프로젝트에서 사용되는 툴들에 대해 많은 분들과 정보를 공유하고 싶습니다.
솔웅

최근에 받은 트랙백

글 보관함


S3 Summary



* Remember that S3 is Object based i.e. allows you to upload files.

* Files can be from 0 Bytes to 5TB

* There is unlimited storage

* Files are stored Buckets

* S3 is a universal namespace, that is, names must be unique globally

* name - i.e. https://s3-eu-west-1.amazonaws.com/acloudgutu


* Read after Write consistency for PUTs of new Objects

* Eventual Consistency for overwrite PUTS and DELETES (can take some time to propagate)

* S3 Storage Classes/Tiers

  : S3 (durable, immediately available, frequently accessed)

  : S3 - IA (durable, immediately available, infrequently accessed)

  : Reduced Redundancy Storage (data that is easily reproducible, such as thumb nails etc).

  : Glacier - Archived data, where you can wait 3 - 5 hours before accessing


* Remember the core fundamentals of S3

  : Key (name)

  : Value (data)

  : Version ID

  : Metadata

  : Access Control lists

  

* Object based storage only (for files)

* Not suitable to install an operating system on (***)

 


Versioning





* Stores all versions of an object (including all writes and even if you delete an object)

* Great backup tool

* Once enabled, Versioning cannot be disabled, only suspended.

* Integrates with Lifecycle rules

* Versioning's MFA Delete capability, which uses multi-factor authentication, can be used to provide an additional layer of security

* Cross Region Replication, requires versioning enabled on the source bucket

  


Lifecycle Management





* Can be used in conjunction with versioning

* Can be applied to current versions and previous versions

* Following actions can now be done

  : Transition to the Standard-Infrequent Access Storage Class (128Kb and 30 days after the creation date)

  : Archive to the Glacier Storage Class (30 days after IA, if relevant)

  : Permanently Delete

  


CloudFront



* Edge Location - This is the location where content will be cached. This is separate to an AWS Region/AZ

* Origin - This is the origin of all the files that the CDN will distribute. This can be either an S3 Bucket, an EC2 Instance, an Elastic Load Balancer or Route53

* Distribution - This is the name given the CDN which consists of a collection of Edge Locations.

  : Web Distribution - Typically used for Websites

  : RTMP - Used for Media Streaming

* Edge locations are not just READ only, you can write to them too. (i.e. put an object on to them)

* Objects are cached for the life of the TTL (Time To Live)

* You can clear cached objects, but you will be charged.



Securing your buckets



* By default, all newly created buckets are PRIVATE

* You can setup access control to your buckets using

  : Bucket Policies

  : Access Control Lists

* S3 buckets can be configured to create access logs which log all requests made to the S3 bucket. This can be done to another bucket.



Encryption



* In Transit

  : SSL/TLS

* At Rest

  : Server Side Encryption

    - S3 Managed Keys - SSE-S3 (***)

    - AWS Key Management Service, Managed Keys - SSE-KMS (***)

    - Server Side Encryption With Customer Provided Keys - SSE-C (***)

* Client Side Encryption



Storage Gateway



* File Gateway - For flat files, stored directly on S3

* Volume Gateway

  : Stored Volumes - Entire Dataset is stored on site and is asynchronously backed up to S3.

  : Cached Volumes - Entire Dataset is stored on S3 and the most frequently accessed data is cached on site

* Gateway Virtual Tape Library (VTL)

  : Used for backup and uses popular backup applications like NetBackup, Backup Exec, Veam etc.






Snowball



* Snowball

* Snowball Edge

* Snowmobile


* Understand what Snowball is

* Understand what Import Export is

* Snowball Can

  : Import to S3

  : Export from S3

  


S3 Transfer Acceleration



* You can speed up transfers to S3 using S3 transfer acceleration. This costs extra, and has the greatest impact on people who are in far away location.



S3 Static Websites



* You can use S3 to host static websites

* Serverless

* Very cheap, scales automatically

* STATIC only, cannot host dynamic sites



CORS



* Cross Origin Resource Sharing

* Need to enable it on the resources bucket and state the URL for the origin that will be calling the bucket.

i.e. 

http://mybucketname.s3-website.eu-west-2.amazonaws.com - S3 Website

https://s3.eu-west-2.amazonaws/mybucketname      - Bucket



Last few tips



* Write to S3 - HTTP 200 code for a successful write

* You can load files to S3 much faster by enabling multipart upload

* Read the S3 FAQ before taking the exam. It comes up A LOT!





=====================================



S3 Quiz



* The minimum file size allowed on S3 is 0 bytes? True

* If you encrypt a bucket on S3 what encryption does AWS use? 

  ==> Advanced Encryption Standard (AES) 256

* You create a static hosting website in a bucket called "acloudguru" in Japan using S3. What would the new URL End Point be? 

  ==> http://acloudguru.s3-website-ap-northeast-1.amazonaws.com

* You are hosting a static website in an S3 bucket which uses Java script to reference assets in another S3 bucket. For some reason however these assets are not displaying when users browse to the site. What could be the problem?

  ==> You haven't enabled Cross Origin Resource Sharing (CORS) on the bucket where the assets are stored

* What is the HTTP code you would see if once you successfully place a file in an S3 bucket? ==> 200


* S3 provides unlimited storage. ==> True

* What is the maximum file size that can be stored on S3? ==> 5Tb

* What is the largest size file you can transfer to S3 using a PUT operation? 

  ==> The correct answer is 5Gb. After that you must use a multipart upload. This can be an exam question. Please remember this before going in to your exam. Correct! http://docs.aws.amazon.com/AmazonS3/latest/dev/UploadingObjects.html

* If you want to enable a user to download your private data directly from S3, you can insert a pre-signed URL into a web page before giving it to your user. ==> True

* When you first create an S3 bucket, this bucket is publicly accessible by default. ==> False





반응형

Comment

  1. leebywan 2020.11.25 09:14

    재미있는 글 매우 잘 배우고 가여