AWS 查询 ALB access log
AWS 查询 ALB access log
开启AWS访问日志
创建S3桶并进行policy授权
S3->Permissions
其中 iam id 分区域不同，具体参考。
Enable access logs for your Application Load Balancer - Elastic Load Balancing为 Application Load Balancer 启用访问日志 - Elastic Load Balancing
-
中国（北京）– 638102146993
-
中国（宁夏）– 037604701340
aws --region cn-northwest-1 s3api put-bucket-policy --bucket prod-alb-access-log --policy "{\"Version\": \"2012-10-17\",\"Statement\": [{\"Sid\": \"Statement1\",\"Effect\": \"Allow\",\"Principal\": { \"AWS\": \"arn:aws-cn:iam::037604701340:root\" },\"Action\": \"s3:PutObject\",\"Resource\": \"arn:aws-cn:s3:::prod-alb-access-log/*\"}]
}"ALB开启访问日志
打开 Amazon EC2 控制台，网址为https://console.aws.amazon.com/ec2/。
在导航窗格中，选择负载均衡器。
选择您的负载均衡器的名称以打开其详细信息页面。
在属性选项卡上，选择编辑。
对于监控，打开访问日志。
对于 S3 URI，输入日志文件的 S3 URI。您指定的 URI 取决于您是否使用前缀。
带有前缀的 URI: s3://
amzn-s3-demo-logging-bucket/logging-prefix不带前缀的 URI: s3://
amzn-s3-demo-logging-bucket选择保存更改。
基于Athena查询S3中ALB log
CREATE EXTERNAL TABLE IF NOT EXISTS pmt_alb_logs(type string,time string,elb string,client_ip string,client_port int,target_ip string,target_port int,request_processing_time double,target_processing_time double,response_processing_time double,elb_status_code int,target_status_code string,received_bytes bigint,sent_bytes bigint,request_verb string,request_url string,request_proto string,user_agent string,ssl_cipher string,ssl_protocol string,target_group_arn string,trace_id string,domain_name string,chosen_cert_arn string,matched_rule_priority string,request_creation_time string,actions_executed string,redirect_url string,lambda_error_reason string,target_port_list string,target_status_code_list string,classification string,classification_reason string,conn_trace_id string)PARTITIONED BY(day STRING)ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.RegexSerDe'WITH SERDEPROPERTIES('serialization.format' = '1','input.regex' = '([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*):([0-9]*) ([^ ]*)[:-]([0-9]*) ([-.0-9]*) ([-.0-9]*) ([-.0-9]*) (|[-0-9]*) (-|[-0-9]*) ([-0-9]*) ([-0-9]*) \"([^ ]*) (.*) (- |[^ ]*)\" \"([^\"]*)\" ([A-Z0-9-_]+) ([A-Za-z0-9.-]*) ([^ ]*) \"([^\"]*)\" \"([^\"]*)\" \"([^\"]*)\" ([-.0-9]*) ([^ ]*) \"([^\"]*)\" \"([^\"]*)\" \"([^ ]*)\" \"([^\s]+?)\" \"([^\s]+)\" \"([^ ]*)\" \"([^ ]*)\" ?([^ ]*)?( .*)?')LOCATION 's3://prod-alb-access-log/adtech_gcpmt_alb_logs/AWSLogs/871252328373/elasticloadbalancing/cn-northwest-1/'TBLPROPERTIES("projection.enabled" = "true","projection.day.type" = "date","projection.day.range" = "2022/01/01,NOW","projection.day.format" = "yyyy/MM/dd","projection.day.interval" = "1","projection.day.interval.unit" = "DAYS","storage.location.template" = "s3://prod-alb-access-log/adtech_gcpmt_alb_logs/AWSLogs/871252328373/elasticloadbalancing/cn-northwest-1/${day}")pmt_alb_logs SELECT * FROM "AwsDataCatalog"."adtech_pmt"."pmt_alb_logs" where domain_name = 'domain' and request_url like '%/ads/landing%' limit 10;