目 錄
第１章 負(fù)載均衡概述 ....................................................................................................... 1
1.1 從數(shù)據(jù)中心說起 ........................................................................................... 1
1.1.1 數(shù)據(jù)中心的發(fā)展歷史 ......................................................................... 2
1.1.2 超融合數(shù)據(jù)中心架構(gòu)和大型互聯(lián)網(wǎng)架構(gòu) ............................................ 3
1.2 負(fù)載均衡必要性分析 .................................................................................... 4
1.2.1 負(fù)載均衡的作用 ................................................................................ 5
1.2.2 場(chǎng)景需求 ........................................................................................... 7
1.3 負(fù)載均衡的產(chǎn)生 ........................................................................................... 8
1.3.1 負(fù)載均衡的歷史 ................................................................................ 9
1.3.2 負(fù)載均衡的現(xiàn)狀 .............................................................................. 10
1.3.3 負(fù)載均衡面臨的挑戰(zhàn) ....................................................................... 10
1.4 負(fù)載均衡的原理 ......................................................................................... 11
1.4.1 四層負(fù)載均衡的原理 ....................................................................... 11
1.4.2 七層負(fù)載均衡的原理 ....................................................................... 12
1.5 典型的負(fù)載均衡器介紹 .............................................................................. 13
1.5.1 LVS ................................................................................................. 13
1.5.2 Nginx .............................................................................................. 15
第 2 章 負(fù)載均衡詳細(xì)介紹.............................................................................................. 18
2.1 四層負(fù)載均衡技術(shù) ...................................................................................... 18
2.1.1 經(jīng)典四層負(fù)載均衡器 LVS 的相關(guān)術(shù)語 ........................................... 19
2.1.2 LVS 轉(zhuǎn)發(fā)模式及配置實(shí)例 ............................................................... 20
2.1.3 不同協(xié)議下的負(fù)載均衡 ................................................................... 27
2.1.4 獲取真實(shí)的客戶端 IP 地址和端口信息 ............................................ 31
2.2 七層負(fù)載均衡技術(shù) ...................................................................................... 33
2.2.1 經(jīng)典七層負(fù)載均衡器 Nginx 的部署架構(gòu) ........................................ 33
2.2.2 Nginx 轉(zhuǎn)發(fā)粒度控制 ........................................................................ 34
2.2.3 獲取真實(shí)的客戶端 IP 地址和端口信息 ............................................ 35
2.3 Real Server 調(diào)度算法 ................................................................................. 35
2.3.1 輪詢（RR） .................................................................................... 35
2.3.2 加權(quán)輪詢（WRR） .......................................................................... 37
2.3.3 最少連接調(diào)度（LC） ...................................................................... 39
2.3.4 加權(quán)最少連接調(diào)度（WLC） ........................................................... 39
2.3.5 一致性哈希（CONNHASH） .......................................................... 41
2.3.6 連接模板 ......................................................................................... 43
第 3 章 負(fù)載均衡功能 ..................................................................................................... 45
3.1 基本功能特性 ............................................................................................. 46
3.1.1 負(fù)載均衡器的網(wǎng)絡(luò)結(jié)構(gòu) ................................................................... 47
3.1.2 提供外網(wǎng) VIP 和流量均衡 ............................................................... 50
3.1.3 提供內(nèi)網(wǎng) VIP 和 IDC 內(nèi)部服務(wù) ....................................................... 54
3.1.4 使用 SNAT 集群提供外網(wǎng)訪問 ......................................................... 56
3.1.5 使用 SNAT 隧道服務(wù)無外網(wǎng)出口的 IDC .......................................... 58
3.1.6 服務(wù)隔離和安全控制 ....................................................................... 59
3.2 高可用性 .................................................................................................... 60
3.2.1 使用 Keepalived 做健康檢查 ............................................................ 60
3.2.2 使用 VRRP 實(shí)現(xiàn)主備 ....................................................................... 62
3.2.3 使用 ECMP 實(shí)現(xiàn)集群化 ................................................................... 64
3.2.4 使用網(wǎng)卡綁定擴(kuò)展單網(wǎng)卡流量 ......................................................... 65
3.3 高可擴(kuò)展性 ................................................................................................ 65
3.3.1 擴(kuò)展負(fù)載均衡器 .............................................................................. 66
3.3.2 擴(kuò)展后端服務(wù)器 .............................................................................. 69
3.4 使用 BGP Anycast 實(shí)現(xiàn)多個(gè) IDC 負(fù)載均衡和機(jī)房災(zāi)備 ............................... 72
第 4 章 現(xiàn)有負(fù)載均衡器比較 .......................................................................................... 74
4.1 四層負(fù)載均衡器 ......................................................................................... 74
4.1.1 硬件實(shí)現(xiàn) ......................................................................................... 74
4.1.2 軟件實(shí)現(xiàn)：Linux Virtual Server（LVS） .......................................... 75
4.1.3 軟件實(shí)現(xiàn)方案對(duì)比 ........................................................................... 75
4.2 七層負(fù)載均衡器 ......................................................................................... 76
4.2.1 硬件實(shí)現(xiàn) ......................................................................................... 76
4.2.2 軟件實(shí)現(xiàn)：HAProxy ........................................................................ 77
4.2.3 實(shí)現(xiàn)方案對(duì)比：F-Stack 與 Nginx ..................................................... 77
第 5 章 負(fù)載均衡與云計(jì)算.............................................................................................. 79
5.1 負(fù)載均衡與彈性計(jì)算 .................................................................................. 79
5.2 跨區(qū)域調(diào)度與容災(zāi) ...................................................................................... 80
5.3 API Gateway ............................................................................................... 82
第 6 章 網(wǎng)絡(luò)協(xié)議優(yōu)化 ..................................................................................................... 84
6.1 TCP 協(xié)議優(yōu)化............................................................................................. 84
6.2 TLS/HTTPS 協(xié)議優(yōu)化 ................................................................................. 87
6.2.1 TLS/HTTPS 協(xié)議的性能問題 ........................................................... 87
6.2.2 Session ID 及 Session Ticket ............................................................. 89
6.2.3 False-Start ........................................................................................ 93
6.2.4 TLS1.3 協(xié)議 .................................................................................... 94
6.2.5 硬件加速卡和計(jì)算分離 ................................................................... 98
6.2.6 自動(dòng)化數(shù)字證書管理 ..................................................................... 100
6.3 HTTP 協(xié)議優(yōu)化和 HTTP2.0 ...................................................................... 101
6.4 基于 UDP 的傳輸協(xié)議優(yōu)化 ....................................................................... 104
6.4.1 基于 UDP 的傳輸協(xié)議簡(jiǎn)介 ............................................................ 104
6.4.2 QUIC 協(xié)議優(yōu)化 ............................................................................. 106
6.5 DNS 協(xié)議優(yōu)化 .......................................................................................... 115
第 7 章 性能優(yōu)化 .......................................................................................................... 117
7.1 主要性能指標(biāo) ........................................................................................... 117
7.2 性能挑戰(zhàn)與分析 ....................................................................................... 120
7.2.1 C10K 問題及 C10M 問題 ............................................................... 120
7.2.2 LVS 性能瓶頸分析 ........................................................................ 121
7.2.3 內(nèi)核成為瓶頸的原因 ..................................................................... 123
7.3 高性能四層負(fù)載均衡關(guān)鍵技術(shù) .................................................................. 126
7.3.1 Kernel Bypass 技術(shù)與 DPDK .......................................................... 127
7.3.2 Share-Nothing 思想 ........................................................................ 129
7.3.3 避免上下文切換 ............................................................................ 130
7.3.4 使用輪詢而非中斷 ......................................................................... 131
7.3.5 避免數(shù)據(jù)復(fù)制 ................................................................................ 131
7.3.6 其他技術(shù) ....................................................................................... 131
7.4 使用 DPDK 加速四層負(fù)載均衡 ................................................................. 132
7.4.1 高性能負(fù)載均衡器的架構(gòu) .............................................................. 133
7.4.2 高性能負(fù)載均衡器功能模塊 .......................................................... 136
7.4.3 數(shù)據(jù)流大圖 ................................................................................... 144
7.4.4 項(xiàng)目開源的緣由和一些經(jīng)驗(yàn) .......................................................... 147
7.5 七層負(fù)載均衡性能優(yōu)化 ............................................................................ 149
7.5.1 Nginx 調(diào)優(yōu) .................................................................................... 149
7.5.2 內(nèi)核參數(shù)調(diào)優(yōu) ................................................................................ 152
7.5.3 利用最新內(nèi)核的特性 ..................................................................... 154
7.5.4 利用硬件卸載 ................................................................................ 156
7.6 性能測(cè)試環(huán)境與數(shù)據(jù) ................................................................................ 158
7.6.1 大并發(fā)測(cè)試環(huán)境 ............................................................................ 158
7.6.2 四層負(fù)載均衡性能數(shù)據(jù) ................................................................. 160
7.6.3 七層負(fù)載均衡性能數(shù)據(jù) ................................................................. 161
7.7 大容量網(wǎng)卡 .............................................................................................. 163
第 8 章 安全設(shè)計(jì) .......................................................................................................... 165
8.1 數(shù)據(jù)中心面臨的安全問題 ......................................................................... 165
8.2 Syn-flood 攻擊與防御 ............................................................................... 165
8.3 DDoS 與流量清洗 ..................................................................................... 167
8.4 黑名單與 ACL .......................................................................................... 167
8.5 WAF ......................................................................................................... 168
第 9 章 負(fù)載均衡實(shí)踐 ................................................................................................... 171
9.1 服務(wù)配置部署 ........................................................................................... 171
9.1.1 主備模式與集群模式 ..................................................................... 171
9.1.2 部署 FullNAT 外網(wǎng)集群 ................................................................. 172
9.1.3 部署 FullNAT 內(nèi)網(wǎng)集群 ................................................................. 173
9.1.4 部署 DR 集群 ................................................................................ 175
9.1.5 部署 SNAT 集群 ............................................................................ 175
9.1.6 部署 SNAT-GRE 集群 .................................................................... 177
9.1.7 部署 Nginx 集群 ............................................................................ 178
9.2 監(jiān)控與故障分析 ....................................................................................... 179
9.2.1 負(fù)載均衡監(jiān)控指標(biāo)及基本監(jiān)控 ....................................................... 179
9.2.2 監(jiān)控?cái)?shù)據(jù)成圖 ................................................................................ 181
9.2.3 線上故障及排查實(shí)例 ..................................................................... 182
9.3 SLA 簡(jiǎn)介.................................................................................................. 182
9.3.1 服務(wù)自助化 ................................................................................... 183
9.3.2 服務(wù)告警 ....................................................................................... 183
9.3.3 流量異常檢測(cè) ................................................................................ 185
9.4 集群性能分析 ........................................................................................... 186
9.5 負(fù)載均衡與 Kubernetes 云原生 ................................................................ 188
9.5.1 負(fù)載均衡服務(wù) ................................................................................ 189
9.5.2 Ingress ........................................................................................... 190
9.5.3 SNAT 出網(wǎng)訪問 ............................................................................ 192
9.6 邊緣計(jì)算 .................................................................................................. 193
第 10 章 展望 ............................................................................................................... 195