This assumes you're the owner of the s3 website bucket (you have full access and control of it) and already have set up a Cloudfront distribution for it.
s3
- Create a 'logs' bucket eg. mybucket.logs with acls ENABLED (under Object Ownership), leave all other defaults as they are, eg. block all public access, no bucket policy, no cors, etc.
- Create a bucket for athena results eg. mybucket.athena.Leave all defaults as they are, eg. block all public access, no bucket policy, no cors, etc.
- Create a folder in the athena bucket named results.
Cloudfront
- Click on the distribution id for the site you want to set up logging for.
- You should be under the default General tab. Click on Edit.
- Under Standard logging, set it to On. 'Select S3 bucket' will appear. Enter the .log bucket you made above.
- Hit Save.
- The distribution will be set to 'deploying'.
Athena
- Under Settings - click on 'Manage'.
- Enter the S3 athena bucket you just made and the folder. eg. s3://mybucket.athena/results.
- Enter your aws account id.
In the following query, replace s3://mybucket.logs/ with the logs bucket you just made:
CREATE EXTERNAL TABLE IF NOT EXISTS default.cloudfront_logs (
`date` DATE,
time STRING,
location STRING,
bytes BIGINT,
request_ip STRING,
method STRING,
host STRING,
uri STRING,
status INT,
referrer STRING,
user_agent STRING,
query_string STRING,
cookie STRING,
result_type STRING,
request_id STRING,
host_header STRING,
request_protocol STRING,
request_bytes BIGINT,
time_taken FLOAT,
xforwarded_for STRING,
ssl_protocol STRING,
ssl_cipher STRING,
response_result_type STRING,
http_version STRING,
fle_status STRING,
fle_encrypted_fields INT,
c_port INT,
time_to_first_byte FLOAT,
x_edge_detailed_result_type STRING,
sc_content_type STRING,
sc_content_len BIGINT,
sc_range_start BIGINT,
sc_range_end BIGINT
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
LOCATION 's3://mybucket.logs/'
TBLPROPERTIES ( 'skip.header.line.count'='2' )Go to the Athena query editor and paste the query you just modified above into it.
Click on Run.
Click on your website to generate traffic. It may take a few seconds for the logs to be generated. Try this query in the query editor:
SELECT DISTINCT * FROM cloudfront_logs LIMIT 10;
