Amazon CloudFront 장애 발생 시 필요 정보

https://github.com/leedoing/cdn_checker

GitHub - leedoing/cdn_checker

 

1. CF Resolver DNS Query

2. curl 등 다운로드 테스트

3. Traceroute 정보

log 확인 명령어

aws s3 cp s3://'path'/ . --recursive --exclude "*" --include "EXXXXX.2018-07-11*.gz"
zcat * |awk '{ if ( $9 != 200 && $9 != 301 && $9 != 302 && $9 != 303 && $9 != 304 && $9 != 404 && $9 != 000) print ($0) }'

 

 

 

[Case Open 시 AWS에서 필요 정보 요청]

1) What latency do you expect 
2) Please provide the Request ID of a request that appears to experience high latency 
3) Do "dig resolver-identity.cloudfront.net"; or "nslookup resolver-identity.cloudfront.net" 
4) Please send us the http headers from when you had the problem retrieval (you can do that from the command line using curl or wget, or from the browser using HAR - https://code.google.com/p/harviewer/) 
 Example: a) curl -v -k http://mydistributionid.cloudfront.net/object.html > /dev/null 
 b) wget --server-response http://mydistributionid.cloudfront.net/object.html 
 c) From a browser https://toolbox.googleapps.com/apps/har_analyzer/ or https://confluence.atlassian.com/kb/generating-har-files-and-analysing-web-requests-720420612.html 
5) Where (physically) is the computer that is running the test or downloading content? 
6) Do "mtr dfafdfadfffdfafd.cloudfront.net" and if you cannot do that, run "traceroute -T dfdfafdfa.cloudfront.net"

; or "tracert dfadfafd4zlj.cloudfront.net" I look forward to your response. Please feel free to reach out to me with any additional questions or concerns. Best regards, Olamide O. Amazon Web Services Check out the AWS Support Knowledge Center, a knowledge base of articles and videos that answer customer questions about AWS services: https://aws.amazon.com/premiumsupport/knowledge-center/?icmpid=support_email_category We value your feedback. Please rate my response using the link below. ==============================

====To contact us again about this case, please return to the AWS Support Center using the following

 

아래 쉘을 참고해서 장애 발생 시 정보를 수집. 필요한 항목만 주기적으로 데이터 저장 권장

#!/bin/bash

##CDN performance check
#INSTALL CURL, MUUT PACKEGE
echo "Packages Install(curl, mutt)"
mutt=$(cat /etc/issue)
os=`echo $mutt | cut -f1 -d" "`
if [ "$os" = "Amazon" -o "$os" = "CentOS" -o "$os" = "Red" ]
then
        mutt=$(yum -y install mutt)
        echo "$mutt"
        curl=$(yum -y install curl-devel)
        echo "$curl"
else
        mutt=$(apt-get install mutt)
        echo "$mutt"
        curl=$(apt-get install curl)
        echo "$curl"
fi
echo "Installed curl, mutt Packages"

#HOST NAME
host_name=$(hostname)
dns_resolv=$(cat /etc/resolv.conf)
mutt=$(yum -y install mutt)
curl=$(yum -y install curl-devel)

echo -n "Input URL(ex. https://media.amazonwebservices.com/urchin.js): "
read URL
echo -n "Input mail Address(ex. user@hostname.com): "
read MAIL

pwd=$(pwd)
path=$pwd/cf_issue_$host_name.log
echo Save $path
HOST=`echo $URL | cut -f3 -d"/"`

#Host Infomation
exec 3<> cf_issue_$host_name.log
echo "Host_name: " $host_name >&3
echo "DNS_resolv" >&3
echo "$dns_resolv" >&3
echo >&3

##1.DNS RESOLVER
echo "[1. DNS RESOLVER]" >&3
dns_query=$(dig resolver-identity.cloudfront.net +trace)
echo "$dns_query" >&3
echo >&3

##2. CURL SPEED
#curl_speed=$(curl -w "%{time_namelookup}/%{time_connect}/%{time_starttransfer}/%{time_total}" -tlsv1.2 -o /dev/null -s "https://images-na.ssl-images-amazon.com/images/G/01/awssignin/static/aws_logo_smile.png")
curl_speed=$(curl -w "%{time_namelookup}/%{time_connect}/%{time_starttransfer}/%{time_total}" -tlsv1.2 -o /dev/null -s "$URL")
        time_namelookup=`echo $curl_speed | cut -f1 -d"/"`
        time_connect=`echo $curl_speed | cut -f2 -d"/"`
        time_starttransfer=`echo $curl_speed | cut -f3 -d"/"`
        time_total=`echo $curl_speed | cut -f4 -d"/"`
        time_connect_r=`echo "$time_connect - $time_namelookup"|bc`
        time_starttransfer_r=`echo "$time_starttransfer - $time_connect"|bc`
echo "[2. CURL SPEED]" >&3
echo $(date) time_namelookup=$time_namelookup time_connect=$time_connect_r time_starttransfer=$time_starttransfer_r time_total=$time_total >&3
echo >&3

##3. CURL RESPONSE HEADER
#curl_header=$(curl -v -tlsv1.2 -o /dev/null -s "https://images-na.ssl-images-amazon.com/images/G/01/awssignin/static/aws_logo_smile.png")
curl_header=$(curl -I -tlsv1.2 -s "$URL")
echo "[3. CF RESPONSE HEADER]" >&3
echo "$curl_header" >&3
echo >&3

##4. NETWORK TRACE
echo "[4. NETWORK TRACEROUTE]" >&3
network_trace=$(traceroute -T "$HOST")
echo "$network_trace" >&3
echo >&3

#Send mail
mutt -s "CloudFront Issue Check_$host_name" $MAIL < $path

 

 

해당 정보를 통해 Case Open

 

윈도우의 경우 내부 직원이나 Client에게 첨부 파일 던져주고 정보가 취합된 text 파일 받아서 확인

 

main.exe

다운로드

 

추가 Python 2.7

import pycurl

c = pycurl.Curl()

c.setopt(c.URL, 'https://www.google.com/')
c.setopt(c.NOBODY, True)
try:
    c.perform()
    print('NAMELOOKUP_TIME: {}' .format(c.getinfo(c.NAMELOOKUP_TIME)))
    print('CONNECT TIME: {}' .format(c.getinfo(c.CONNECT_TIME)))
    print('PRETRANSFER_TIME: {}' .format(c.getinfo(c.PRETRANSFER_TIME)))
    print('STARTTRANSFER_TIME: {}' .format(c.getinfo(c.STARTTRANSFER_TIME)))
    print('TOTAL TIME: {}' .format(c.getinfo(c.TOTAL_TIME)))
    print('HTTP CODE: {}' .format(c.getinfo(c.RESPONSE_CODE)))
except pycurl.error as e:
    print('Check Error: {}' .format(e))
finally:
    c.close()