LVM缩容

LVM可动态扩容,也可以缩容,我们直接讲缩容部分,缩容前,肯定是有环境配置成了LVM,我们看下环境的目前现状。

现状

看看当前的磁盘信息:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
host ~ # lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 119.2G 0 disk
|-sda1 8:1 0 500M 0 part /boot
|-sda2 8:2 0 50G 0 part /home/myvg
|-sda3 8:3 0 25G 0 part /
|-sda4 8:4 0 1K 0 part
|-sda5 8:5 0 20G 0 part /home/myvg/var/logs
|-sda6 8:6 0 10G 0 part /home/myvg/var/tmp
|-sda7 8:7 0 9.9G 0 part /ac
`-sda8 8:8 0 3.9G 0 part
sdb 8:16 0 931.5G 0 disk
`-myvg-data 253:0 0 8.2T 0 lvm /data
sdc 8:32 0 1.8T 0 disk
`-myvg-data 253:0 0 8.2T 0 lvm /data
sdd 8:48 0 3.7T 0 disk
`-myvg-data 253:0 0 8.2T 0 lvm /data
sde 8:64 0 1.8T 0 disk
`-myvg-data 253:0 0 8.2T 0 lvm /data

可以看出sdb, sdc, sdd, sde 组成了一个lvm 挂载到/data上。

再看volume groups信息:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
host ~ # vgdisplay -v
Using volume group(s) on command line.
Finding all volume groups.
--- Volume group ---
VG Name myvg # 只有一个卷组,名叫myvg
System ID
Format lvm2
Metadata Areas 4
Metadata Sequence No 2
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 4
Act PV 4
VG Size 8.19 TiB
PE Size 4.00 MiB
Total PE 2146192
Alloc PE / Size 2146192 / 8.19 TiB
Free PE / Size 0 / 0
VG UUID podu2f-xyGh-ZerU-cVNa-fNeL-5MFh-wzO3ye
# 在myvg卷组下,有一个逻辑分区,叫data
--- Logical volume ---
LV Path /dev/myvg/data
LV Name data
VG Name myvg
LV UUID 4YcW3O-NbZD-3zrR-olkO-CFQ9-xcJz-0vzYEZ
LV Write Access read/write
LV Creation host, time localhost.localdomain, 2017-08-09 14:32:48 +0800
LV Status available
# open 1
LV Size 8.19 TiB
Current LE 2146192
Segments 4
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0
# 有4个物理分区,现在受LVM管理
--- Physical volumes ---
PV Name /dev/sdb
PV UUID 1YfPS2-8OR8-LFEP-WDyk-kLAw-wBL6-0NPDyf
PV Status allocatable
Total PE / Free PE 238467 / 0
PV Name /dev/sdc
PV UUID f2VZdK-8hRF-QJlF-BreW-VeEO-Ctrb-U9zujp
PV Status allocatable
Total PE / Free PE 476932 / 0
PV Name /dev/sdd
PV UUID H4n9ta-bYRW-TD88-MYL2-H9Nv-vuhT-jNaUbq
PV Status allocatable
Total PE / Free PE 953861 / 0
PV Name /dev/sde
PV UUID Xv9I0x-09Bk-Q5J3-2YKs-9ACP-nJUg-eSEhiE
PV Status allocatable
Total PE / Free PE 476932 / 0

看看逻辑分区/dev/myvg/data,他实际上是挂载到了/data目录:

1
2
3
4
5
6
host ~ # df
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/myvg-data 8.2T 784G 7.0T 10% /data
host ~ # lh /dev/myvg/data
lrwxrwxrwx 1 root root 7 Oct 19 20:15 /dev/myvg/data -> ../dm-0

我们上面知道了,有4块物理分区加入了LVM,我们看看其详细信息:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
host ~ # pvdisplay -m
--- Physical volume ---
PV Name /dev/sdb
VG Name myvg
PV Size 931.51 GiB / not usable 1.71 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 238467
Free PE 0
Allocated PE 238467
PV UUID 1YfPS2-8OR8-LFEP-WDyk-kLAw-wBL6-0NPDyf
--- Physical Segments ---
Physical extent 0 to 238466:
Logical volume /dev/myvg/data
Logical extents 1907725 to 2146191 # 第4块逻辑盘sdb
--- Physical volume ---
PV Name /dev/sdc
VG Name myvg
PV Size 1.82 TiB / not usable 1.09 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 476932
Free PE 0
Allocated PE 476932
PV UUID f2VZdK-8hRF-QJlF-BreW-VeEO-Ctrb-U9zujp
--- Physical Segments ---
Physical extent 0 to 476931:
Logical volume /dev/myvg/data
Logical extents 953861 to 1430792 # 第2块逻辑盘sdc
--- Physical volume ---
PV Name /dev/sdd
VG Name myvg
PV Size 3.64 TiB / not usable 3.84 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 953861
Free PE 0
Allocated PE 953861
PV UUID H4n9ta-bYRW-TD88-MYL2-H9Nv-vuhT-jNaUbq
--- Physical Segments ---
Physical extent 0 to 953860:
Logical volume /dev/myvg/data
Logical extents 0 to 953860 # 第1块逻辑盘sdd
--- Physical volume ---
PV Name /dev/sde
VG Name myvg
PV Size 1.82 TiB / not usable 1.09 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 476932
Free PE 0
Allocated PE 476932
PV UUID Xv9I0x-09Bk-Q5J3-2YKs-9ACP-nJUg-eSEhiE
--- Physical Segments ---
Physical extent 0 to 476931:
Logical volume /dev/myvg/data
Logical extents 1430793 to 1907724 # 第3块逻辑盘sde

缩减

我的缩容需求为,将4块物理磁盘的内容,放到1块里面,释放其他3块。

尝试pvmove数据

1
2
3
4
5
6
7
host ~ # pvmove /dev/sdb
No extents available for allocation
host ~ # pvmove /dev/sdb --alloc anywhere
No extents available for allocation
host ~ # pvmove /dev/sdb /dev/sdd
No free extents on physical volume "/dev/sdd".
No specified PVs have space available.

直接将一个物理分区的数据移动到另一个物理分区,看样子都失败了,原因应该是上面我们的pv信息里显示的full:

1
Allocatable yes (but full)

但是,通过df看,我的实际占用大小是 784G,第一块盘是 sdd 是 3.64T,数据量远远不足一块盘。维护不能pvmove?
原因是,文件系统占满了分区,我们需要先缩减文件系统。

缩减文件系统

  1. 卸载分区

    1
    host ~ # umount /dev/mapper/myvg-data
  2. 检查文件系统是否损坏

    缩减文件系统大小前,会强制要求检测分区是否损坏,执行如下命令:

    1
    2
    3
    4
    5
    6
    7
    8
    host ~ # e2fsck -f /dev/mapper/myvg-data
    e2fsck 1.42.9 (28-Dec-2013)
    Pass 1: Checking inodes, blocks, and sizes
    Pass 2: Checking directory structure
    Pass 3: Checking directory connectivity
    Pass 4: Checking reference counts
    Pass 5: Checking group summary information
    /dev/mapper/myvg-data: 90068/274714624 files (5.4% non-contiguous), 222667189/2197700608 blocks
  3. 缩减文件系统

    使用resize2fs来缩减,因为我只想将数据放到第一块盘,第一块盘是sdd 是 3.64T,我将文件系统缩小为3T,小于第一块盘即可:

    1
    2
    3
    4
    host ~ # resize2fs /dev/mapper/myvg-data 3T
    resize2fs 1.42.9 (28-Dec-2013)
    Resizing the filesystem on /dev/mapper/myvg-data to 805306368 (4k) blocks.
    The filesystem on /dev/mapper/myvg-data is now 805306368 blocks long.

缩减逻辑卷

上面,我们将文件系统缩减了,接着就可以缩减LV大小了:

1
2
3
4
5
6
host ~ # lvreduce -L 3T /dev/mapper/myvg-data
WARNING: Reducing active logical volume to 3.00 TiB
THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce data? [y/n]: y
Size of logical volume myvg/data changed from 8.19 TiB (2146192 extents) to 3.00 TiB (786432 extents).
Logical volume data successfully resized

缩减完成再看看lv信息:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
host ~ # lvdisplay -m
--- Logical volume ---
LV Path /dev/myvg/data
LV Name data
VG Name myvg
LV UUID 4YcW3O-NbZD-3zrR-olkO-CFQ9-xcJz-0vzYEZ
LV Write Access read/write
LV Creation host, time localhost.localdomain, 2017-08-09 14:32:48 +0800
LV Status available
# open 0
LV Size 3.00 TiB # 看这里,变成了3T
Current LE 786432
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0
--- Segments ---
Logical extents 0 to 786431:
Type linear
Physical volume /dev/sdd
Physical extents 0 to 786431

再看看pv信息,此时的Allocatable后面已经不是yes (but full)了,而是 yes:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
host ~ # pvdisplay -m
--- Physical volume ---
PV Name /dev/sdb
VG Name myvg
PV Size 931.51 GiB / not usable 1.71 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 238467
Free PE 238467
Allocated PE 0
PV UUID 1YfPS2-8OR8-LFEP-WDyk-kLAw-wBL6-0NPDyf
--- Physical Segments ---
Physical extent 0 to 238466:
FREE
--- Physical volume ---
PV Name /dev/sdc
VG Name myvg
PV Size 1.82 TiB / not usable 1.09 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 476932
Free PE 476932
Allocated PE 0
PV UUID f2VZdK-8hRF-QJlF-BreW-VeEO-Ctrb-U9zujp
--- Physical Segments ---
Physical extent 0 to 476931:
FREE
--- Physical volume ---
PV Name /dev/sdd
VG Name myvg
PV Size 3.64 TiB / not usable 3.84 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 953861
Free PE 167429
Allocated PE 786432
PV UUID H4n9ta-bYRW-TD88-MYL2-H9Nv-vuhT-jNaUbq
--- Physical Segments ---
Physical extent 0 to 786431:
Logical volume /dev/myvg/data
Logical extents 0 to 786431
Physical extent 786432 to 953860:
FREE
--- Physical volume ---
PV Name /dev/sde
VG Name myvg
PV Size 1.82 TiB / not usable 1.09 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 476932
Free PE 476932
Allocated PE 0
PV UUID Xv9I0x-09Bk-Q5J3-2YKs-9ACP-nJUg-eSEhiE
--- Physical Segments ---
Physical extent 0 to 476931:
FREE

再次pvmove数据

可以看出,现在除了第一块盘sdd有数据,其他的Free PE都等于Total PE,完全没有数据,试试move数据,提示没有数据可以移动:

1
2
3
4
5
6
host ~ # pvmove /dev/sdb
No data to move for myvg
host ~ # pvmove /dev/sde
No data to move for myvg
host ~ # pvmove /dev/sdc
No data to move for myvg

pvchange属性为不可用

修改几块pv的属性,让其不能再分配数据,pv修改为Allocatable为no

1
2
3
4
5
6
7
8
9
host ~ # pvchange -xn /dev/sdb
Physical volume "/dev/sdb" changed
1 physical volume changed / 0 physical volumes not changed
host ~ # pvchange -xn /dev/sde
Physical volume "/dev/sde" changed
1 physical volume changed / 0 physical volumes not changed
host ~ # pvchange -xn /dev/sdc
Physical volume "/dev/sdc" changed
1 physical volume changed / 0 physical volumes not changed

此时再用pvdisplay查看,Allocatable 字段已经设置为 NO 了。

vgreduce和pvremove删除pv

接下来,咱们可以安全地将 几块pv从vg里面删除了:

1
2
3
4
5
6
host ~ # vgreduce myvg /dev/sdb
Removed "/dev/sdb" from volume group "myvg"
host ~ # vgreduce myvg /dev/sde
Removed "/dev/sde" from volume group "myvg"
host ~ # vgreduce myvg /dev/sdc
Removed "/dev/sdc" from volume group "myvg"

此时的pv列表里面还有这几个磁盘的信息,但是状态是NEW Physical volume:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
host ~ # pvdisplay -m
--- Physical volume ---
PV Name /dev/sdd
VG Name myvg
PV Size 3.64 TiB / not usable 3.84 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 953861
Free PE 167429
Allocated PE 786432
PV UUID H4n9ta-bYRW-TD88-MYL2-H9Nv-vuhT-jNaUbq
--- Physical Segments ---
Physical extent 0 to 786431:
Logical volume /dev/myvg/data
Logical extents 0 to 786431
Physical extent 786432 to 953860:
FREE
"/dev/sdb" is a new physical volume of "931.51 GiB"
--- NEW Physical volume ---
PV Name /dev/sdb
VG Name
PV Size 931.51 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID 1YfPS2-8OR8-LFEP-WDyk-kLAw-wBL6-0NPDyf
"/dev/sde" is a new physical volume of "1.82 TiB"
--- NEW Physical volume ---
PV Name /dev/sde
VG Name
PV Size 1.82 TiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID Xv9I0x-09Bk-Q5J3-2YKs-9ACP-nJUg-eSEhiE
"/dev/sdc" is a new physical volume of "1.82 TiB"
--- NEW Physical volume ---
PV Name /dev/sdc
VG Name
PV Size 1.82 TiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID f2VZdK-8hRF-QJlF-BreW-VeEO-Ctrb-U9zujp

彻底从pv列表里面移除:

1
2
3
4
5
6
host ~ # pvremove /dev/sdb
Labels on physical volume "/dev/sdb" successfully wiped
host ~ # pvremove /dev/sde
Labels on physical volume "/dev/sde" successfully wiped
host ~ # pvremove /dev/sdc
Labels on physical volume "/dev/sdc" successfully wiped

现在在pv列表里面已经看不到了:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
host ~ # pvdisplay -m
--- Physical volume ---
PV Name /dev/sdd
VG Name myvg
PV Size 3.64 TiB / not usable 3.84 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 953861
Free PE 167429
Allocated PE 786432
PV UUID H4n9ta-bYRW-TD88-MYL2-H9Nv-vuhT-jNaUbq
--- Physical Segments ---
Physical extent 0 to 786431:
Logical volume /dev/myvg/data
Logical extents 0 to 786431
Physical extent 786432 to 953860:
FREE

大功告成,现在可以恢复lvm之前的挂载点了:

1
host ~ # mount /dev/mapper/myvg-data /data

缩减后

现在的三块是否后的硬盘 sdb sdc sde,就像刚插入的硬盘一样,想怎么处理,都可以(可以对比看一下,和最开始的lsblk,有什么差别):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
host ~ # lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 119.2G 0 disk
|-sda1 8:1 0 500M 0 part /boot
|-sda2 8:2 0 50G 0 part /home/myvg
|-sda3 8:3 0 25G 0 part /
|-sda4 8:4 0 1K 0 part
|-sda5 8:5 0 20G 0 part /home/myvg/var/logs
|-sda6 8:6 0 10G 0 part /home/myvg/var/tmp
|-sda7 8:7 0 9.9G 0 part /ac
`-sda8 8:8 0 3.9G 0 part
sdb 8:16 0 931.5G 0 disk
sdc 8:32 0 1.8T 0 disk
sdd 8:48 0 3.7T 0 disk
`-myvg-data 253:0 0 3T 0 lvm /data
sde 8:64 0 1.8T 0 disk