關(guān)于CentOS 8 搭建MongoDB4.4分片集群的問(wèn)題
一,簡(jiǎn)介
1.分片
在MongoDB里面存在另一種集群,就是分片技術(shù),可以滿(mǎn)足MongoDB數(shù)據(jù)量大量增長(zhǎng)的需求。
在MongoDB存儲(chǔ)海量數(shù)據(jù)時(shí),一臺(tái)機(jī)器可能不足以存儲(chǔ)數(shù)據(jù),也可能不足以提供可接受的讀寫(xiě)吞吐量。這時(shí),我們就可以通過(guò)在多臺(tái)機(jī)器上分割數(shù)據(jù),使得數(shù)據(jù)庫(kù)系統(tǒng)能存儲(chǔ)和處理更多的數(shù)據(jù)。
2.為什么使用分片
- 復(fù)制所有的寫(xiě)入操作到主節(jié)點(diǎn)
- 延遲的敏感數(shù)據(jù)會(huì)在主節(jié)點(diǎn)查詢(xún)
- 單個(gè)副本集限制在12個(gè)節(jié)點(diǎn)
- 當(dāng)請(qǐng)求量巨大時(shí)會(huì)出現(xiàn)內(nèi)存不足
- 本地磁盤(pán)不足
- 垂直擴(kuò)展價(jià)格昂貴
3.分片原理概述
分片就是把數(shù)據(jù)分成塊,再把塊存儲(chǔ)到不同的服務(wù)器上,MongoDB的分片是自動(dòng)分片的,當(dāng)用戶(hù)發(fā)送讀寫(xiě)數(shù)據(jù)請(qǐng)求的時(shí)候,先經(jīng)過(guò)mongos這個(gè)路由層,mongos路由層去配置服務(wù)器請(qǐng)求分片的信息,再來(lái)判斷這個(gè)請(qǐng)求應(yīng)該去哪一臺(tái)服務(wù)器上讀寫(xiě)數(shù)據(jù)。

二,準(zhǔn)備環(huán)境
- 操作系統(tǒng):CentOS Linux release 8.2.2004 (Core)
- MongoDB版本:v4.4.10
- IP:10.0.0.56 實(shí)例:mongos(30000) config(27017) shard1主節(jié)點(diǎn)(40001) shard2仲裁節(jié)點(diǎn)(40002) shard3副節(jié)點(diǎn)(40003)
- IP:10.0.0.57 實(shí)例:mongos(30000) config(27017) shard1副節(jié)點(diǎn)(40001) shard2主節(jié)點(diǎn)(40002) shard3仲裁節(jié)點(diǎn)(40003)
- IP:10.0.0.58 實(shí)例:mongos(30000) config(27017) shard1仲裁節(jié)點(diǎn)(40001) shard3副節(jié)點(diǎn)(40002) shard3主節(jié)點(diǎn)(40003)
三,集群配置部署
1.創(chuàng)建相應(yīng)目錄(三臺(tái)服務(wù)器執(zhí)行相同操作)
mkdir -p /mongo/{data,logs,apps,run}
mkdir -p /mongo/data/shard{1,2,3}
mkdir -p /mongo/data/config
mkdir -p /mongo/apps/conf
2.安裝MongoDB修改創(chuàng)建配置文件(三臺(tái)執(zhí)行相同操作)
安裝教程
安裝可以通過(guò)下載MongoDB安裝包,再進(jìn)行配置環(huán)境變量。這里是直接配置yum源,通過(guò)yum源安裝的MongoDB,后面直接執(zhí)行mongod加所需配置文件路徑運(yùn)行即可。
(1)mongo-config配置文件
vim /mongo/apps/conf/mongo-config.yml
systemLog:
destination: file
#日志路徑
path: "/mongo/logs/mongo-config.log"
logAppend: true
storage:
journal:
enabled: true
#數(shù)據(jù)存儲(chǔ)路徑
dbPath: "/mongo/data/config"
engine: wiredTiger
wiredTiger:
engineConfig:
cacheSizeGB: 12
processManagement:
fork: true
pidFilePath: "/mongo/run/mongo-config.pid"
net:
#這里ip可以設(shè)置為對(duì)應(yīng)主機(jī)ip
bindIp: 0.0.0.0
#端口
port: 27017
setParameter:
enableLocalhostAuthBypass: true
replication:
#復(fù)制集名稱(chēng)
replSetName: "mgconfig"
sharding:
#作為配置服務(wù)
clusterRole: configsvr
(2)mongo-shard1配置文件
vim /mongo/apps/conf/mongo-shard1.yml
systemLog:
destination: file
path: "/mongo/logs/mongo-shard1.log"
logAppend: true
storage:
journal:
enabled: true
dbPath: "/mongo/data/shard1"
processManagement:
fork: true
pidFilePath: "/mongo/run/mongo-shard1.pid"
net:
bindIp: 0.0.0.0
#注意修改端口
port: 40001
setParameter:
enableLocalhostAuthBypass: true
replication:
#復(fù)制集名稱(chēng)
replSetName: "shard1"
sharding:
#作為分片服務(wù)
clusterRole: shardsvr
(3)mongo-shard2配置文件
vim /mongo/apps/conf/mongo-shard2.yml
systemLog:
destination: file
path: "/mongo/logs/mongo-shard2.log"
logAppend: true
storage:
journal:
enabled: true
dbPath: "/mongo/data/shard2"
processManagement:
fork: true
pidFilePath: "/mongo/run/mongo-shard2.pid"
net:
bindIp: 0.0.0.0
#注意修改端口
port: 40002
setParameter:
enableLocalhostAuthBypass: true
replication:
#復(fù)制集名稱(chēng)
replSetName: "shard2"
sharding:
#作為分片服務(wù)
clusterRole: shardsvr
(4)mongo-shard3配置文件
vim /mongo/apps/conf/mongo-shard3.yml
systemLog:
destination: file
path: "/mongo/logs/mongo-shard3.log"
logAppend: true
storage:
journal:
enabled: true
dbPath: "/mongo/data/shard3"
processManagement:
fork: true
pidFilePath: "/mongo/run/mongo-shard3.pid"
net:
bindIp: 0.0.0.0
#注意修改端口
port: 40003
setParameter:
enableLocalhostAuthBypass: true
replication:
#復(fù)制集名稱(chēng)
replSetName: "shard3"
sharding:
#作為分片服務(wù)
clusterRole: shardsvr
(5)mongo-route配置文件
vim /mongo/apps/conf/mongo-route.yml systemLog: destination: file #注意修改路徑 path: "/mongo/logs/mongo-route.log" logAppend: true processManagement: fork: true pidFilePath: "/mongo/run/mongo-route.pid" net: bindIp: 0.0.0.0 #注意修改端口 port: 30000 setParameter: enableLocalhostAuthBypass: true replication: localPingThresholdMs: 15 sharding: #關(guān)聯(lián)配置服務(wù) configDB: mgconfig/10.0.0.56:27017,10.0.0.57:27017,10.0.0.58:27018
3.啟動(dòng)mongo-config服務(wù)(三臺(tái)服務(wù)器執(zhí)行相同操作)
#關(guān)閉之前yum安裝的MongoDB systemctl stop mongod cd /mongo/apps/conf/ mongod --config mongo-config.yml #查看端口27017是否啟動(dòng) netstat -ntpl Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1129/sshd tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 1131/cupsd tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN 2514/sshd: root@pts tcp 0 0 127.0.0.1:6011 0.0.0.0:* LISTEN 4384/sshd: root@pts tcp 0 0 0.0.0.0:27017 0.0.0.0:* LISTEN 4905/mongod tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd tcp6 0 0 :::22 :::* LISTEN 1129/sshd tcp6 0 0 ::1:631 :::* LISTEN 1131/cupsd tcp6 0 0 ::1:6010 :::* LISTEN 2514/sshd: root@pts tcp6 0 0 ::1:6011 :::* LISTEN 4384/sshd: root@pts tcp6 0 0 :::111 :::* LISTEN 1/systemd
4.連接一臺(tái)實(shí)例,創(chuàng)建初始化復(fù)制集
#連接mongo
mongo 10.0.0.56:27017
#配置初始化復(fù)制集,這里的mgconfig要和配置文件里的replSet的名稱(chēng)一致
config={_id:"mgconfig",members:[
{_id:0,host:"10.0.0.56:27017"},
{_id:1,host:"10.0.0.57:27017"},
{_id:2,host:"10.0.0.58:27017"},
]}
rs.initiate(config)
#ok返回1便是初始化成功
{
"ok" : 1,
"$gleStats" : {
"lastOpTime" : Timestamp(1634710950, 1),
"electionId" : ObjectId("000000000000000000000000")
},
"lastCommittedOpTime" : Timestamp(0, 0)
}
#檢查狀態(tài)
rs.status()
{
"set" : "mgconfig",
"date" : ISODate("2021-10-20T06:24:24.277Z"),
"myState" : 1,
"term" : NumberLong(1),
"syncSourceHost" : "",
"syncSourceId" : -1,
"configsvr" : true,
"heartbeatIntervalMillis" : NumberLong(2000),
"majorityVoteCou
5.配置部署shard1分片集群,啟動(dòng)shard1實(shí)例(三臺(tái)執(zhí)行同樣操作)
cd /mongo/apps/conf mongod --config mongo-shard1.yml #查看端口40001是否啟動(dòng) netstat -ntpl Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:40001 0.0.0.0:* LISTEN 5742/mongod tcp 0 0 0.0.0.0:27017 0.0.0.0:* LISTEN 5443/mongod tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1139/sshd tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 1133/cupsd tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN 2490/sshd: root@pts tcp 0 0 127.0.0.1:6011 0.0.0.0:* LISTEN 5189/sshd: root@pts tcp6 0 0 :::111 :::* LISTEN 1/systemd tcp6 0 0 :::22 :::* LISTEN 1139/sshd tcp6 0 0 ::1:631 :::* LISTEN 1133/cupsd tcp6 0 0 ::1:6010 :::* LISTEN 2490/sshd: root@pts tcp6 0 0 ::1:6011 :::* LISTEN 5189/sshd: root@pts
6.連接一臺(tái)實(shí)例,創(chuàng)建復(fù)制集
#連接mongo
mongo 10.0.0.56:40001
#配置初始化復(fù)制集
config={_id:"shard1",members:[
{_id:0,host:"10.0.0.56:40001",priority:2},
{_id:1,host:"10.0.0.57:40001",priority:1},
{_id:2,host:"10.0.0.58:40001",arbiterOnly:true},
]}
rs.initiate(config)
#檢查狀態(tài)
rs.status()
7.配置部署shard2分片集群,啟動(dòng)shard1實(shí)例(三臺(tái)執(zhí)行同樣操作)
cd /mongo/apps/conf mongod --config mongo-shard2.yml #查看端口40002是否啟動(dòng) netstat -ntpl Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:40001 0.0.0.0:* LISTEN 5742/mongod tcp 0 0 0.0.0.0:40002 0.0.0.0:* LISTEN 5982/mongod tcp 0 0 0.0.0.0:27017 0.0.0.0:* LISTEN 5443/mongod tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1139/sshd tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 1133/cupsd tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN 2490/sshd: root@pts tcp 0 0 127.0.0.1:6011 0.0.0.0:* LISTEN 5189/sshd: root@pts tcp6 0 0 :::111 :::* LISTEN 1/systemd tcp6 0 0 :::22 :::* LISTEN 1139/sshd tcp6 0 0 ::1:631 :::* LISTEN 1133/cupsd tcp6 0 0 ::1:6010 :::* LISTEN 2490/sshd: root@pts tcp6 0 0 ::1:6011 :::* LISTEN 5189/sshd: root@pts
8.連接第二個(gè)節(jié)點(diǎn)創(chuàng)建復(fù)制集
因?yàn)槲覀円?guī)劃的shard2的主節(jié)點(diǎn)是10.0.0.57:40002,仲裁節(jié)點(diǎn)不能寫(xiě)數(shù)據(jù),所以要連接10.0.0.57主機(jī)
#連接mongo
mongo 10.0.0.57:40002
#創(chuàng)建初始化復(fù)制集
config={_id:"shard2",members:[
{_id:0,host:"10.0.0.56:40002",arbiterOnly:true},
{_id:1,host:"10.0.0.57:40002",priority:2},
{_id:2,host:"10.0.0.58:40002",priority:1},
]}
rs.initiate(config)
#查看狀態(tài)
rs.status()
9.配置部署shard3分片集群,啟動(dòng)shard3實(shí)例(三臺(tái)執(zhí)行同樣操作)
cd /mongo/apps/conf/ mongod --config mongo-shard3.yml ##查看端口40003是否啟動(dòng) netstat -ntpl Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:40001 0.0.0.0:* LISTEN 5742/mongod tcp 0 0 0.0.0.0:40002 0.0.0.0:* LISTEN 5982/mongod tcp 0 0 0.0.0.0:40003 0.0.0.0:* LISTEN 6454/mongod tcp 0 0 0.0.0.0:27017 0.0.0.0:* LISTEN 5443/mongod tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1139/sshd tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 1133/cupsd tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN 2490/sshd: root@pts tcp 0 0 127.0.0.1:6011 0.0.0.0:* LISTEN 5189/sshd: root@pts tcp6 0 0 :::111 :::* LISTEN 1/systemd tcp6 0 0 :::22 :::* LISTEN 1139/sshd tcp6 0 0 ::1:631 :::* LISTEN 1133/cupsd tcp6 0 0 ::1:6010 :::* LISTEN 2490/sshd: root@pts tcp6 0 0 ::1:6011 :::* LISTEN 5189/sshd: root@pts
10.連接第三個(gè)節(jié)點(diǎn)(10.0.0.58:40003)創(chuàng)建復(fù)制集
#連接mongo
mongo 10.0.0.58:40003
#創(chuàng)建初始化復(fù)制集
config={_id:"shard3",members:[
{_id:0,host:"10.0.0.56:40003",priority:1},
{_id:1,host:"10.0.0.57:40003",arbiterOnly:true},
{_id:2,host:"10.0.0.58:40003",priority:2},
]}
rs.initiate(config)
#查看狀態(tài)
rs.status()
11.配置部署路由節(jié)點(diǎn)
#路由節(jié)點(diǎn)啟動(dòng)登錄用mongos
mongos --config mongo-route.yml
#連接添加分片到集群中
mongo 10.0.0.56:30000
sh.addShard("shard1/10.0.0.56:40001,10.0.0.57:40001,10.0.0.58:40001")
sh.addShard("shard2/10.0.0.56:40002,10.0.0.57:40002,10.0.0.58:40002")
sh.addShard("shard3/10.0.0.56:40003,10.0.0.57:40003,10.0.0.58:40003")
#查看分片狀態(tài)
sh.status()
四,測(cè)試服務(wù)器分片功能
#查看所有庫(kù)
mongos> show dbs
admin 0.000GB
config 0.003GB
#進(jìn)入config
use config
#這里默認(rèn)的chunk大小是64M,db.settings.find()可以看到這個(gè)值,這里為了測(cè)試看的清楚,把chunk調(diào)整為1M
db.settings.save({"_id":"chunksize","value":1})
模擬寫(xiě)入數(shù)據(jù)
#在tydb庫(kù)的tyuser表中循環(huán)寫(xiě)入6萬(wàn)條數(shù)據(jù)
mongos> use tydb
mongos> show tables
mongos> for(i=1;i<=60000;i++){db.tyuser.insert({"id":i,"name":"ty"+i})}
啟用數(shù)據(jù)庫(kù)分片
mongos> sh.enableSharding("tydb")
#ok返回1
{
"ok" : 1,
"operationTime" : Timestamp(1634716737, 2),
"$clusterTime" : {
"clusterTime" : Timestamp(1634716737, 2),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
啟用表分片
mongos> sh.shardCollection(”tydb.tyuser",{"id":1})
查看分片情況
mongos> sh.status()
查看開(kāi)啟關(guān)閉平衡器
#開(kāi)啟 mongos> sh.startBalancer() #或者sh.startBalancer(true) #關(guān)閉 mongos> sh.stopBalancer() #或者sh.stopBalancer(false) #查看是否關(guān)閉 mongos> sh.getBalancerState() #返回flase表示關(guān)閉
到此這篇關(guān)于CentOS 8 搭建MongoDB4.4分片集群的文章就介紹到這了,更多相關(guān)MongoDB分片集群內(nèi)容請(qǐng)搜索本站以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持本站!
版權(quán)聲明:本站文章來(lái)源標(biāo)注為YINGSOO的內(nèi)容版權(quán)均為本站所有,歡迎引用、轉(zhuǎn)載,請(qǐng)保持原文完整并注明來(lái)源及原文鏈接。禁止復(fù)制或仿造本網(wǎng)站,禁止在非maisonbaluchon.cn所屬的服務(wù)器上建立鏡像,否則將依法追究法律責(zé)任。本站部分內(nèi)容來(lái)源于網(wǎng)友推薦、互聯(lián)網(wǎng)收集整理而來(lái),僅供學(xué)習(xí)參考,不代表本站立場(chǎng),如有內(nèi)容涉嫌侵權(quán),請(qǐng)聯(lián)系alex-e#qq.com處理。
關(guān)注官方微信