Tag: more

Posted on by — Leave a comment

Add storage to your Fedora system with LVM

Sometimes there is a need to add another disk to your system. This is where Logical Volume Management (LVM) comes in handy. The cool thing about LVM is that it’s fairly flexible. There are several ways to add a disk. This article describes one way to do it.

Heads up!

This article does not cover the process of physically installing a new disk drive into your system. Consult your system and disk documentation on how to do that properly.

Important: Always make sure you have backups of important data. The steps described in this article will destroy data if it already exists on the new disk.

Good to know

This article doesn’t cover every LVM feature deeply; the focus is on adding a disk. But basically, LVM has volume groups, made up of one or more partitions and/or disks. You add the partitions or disks as physical volumes. A volume group can be broken down into many logical volumes. Logical volumes can be used as any other storage for filesystems, ramdisks, etc. More information can be found here.

Think of the physical volumes as forming a pool of storage (a volume group) from which you then carve out logical volumes for your system to use directly.

Preparation

Make sure you can see the disk you want to add. Use lsblk prior to adding the disk to see what storage is already available or in use.

$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
zram0 251:0 0 989M 0 disk [SWAP]
vda 252:0 0 20G 0 disk
├─vda1 252:1 0 1G 0 part /boot
└─vda2 252:2 0 19G 0 part
└─fedora_fedora-root 253:0 0 19G 0 lvm /

This article uses a virtual machine with virtual storage. Therefore the device names start with vda for the first disk, vdb for the second, and so on. The name of your device may be different. Many systems will see physical disks as sda for the first disk, sdb for the second, and so on.

Once the new disk has been connected and your system is back up and running, use lsblk again to see the new block device.

$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
zram0 251:0 0 989M 0 disk [SWAP]
vda 252:0 0 20G 0 disk
├─vda1 252:1 0 1G 0 part /boot
└─vda2 252:2 0 19G 0 part
└─fedora_fedora-root 253:0 0 19G 0 lvm /
vdb 252:16 0 10G 0 disk

There is now a new device named vdb. The location for the device is /dev/vdb.

$ ls -l /dev/vdb
brw-rw----. 1 root disk 252, 16 Nov 24 12:56 /dev/vdb

We can see the disk, but we cannot use it with LVM yet. If you run blkid you should not see it listed. For this and following commands, you’ll need to ensure your system is configured so you can use sudo:

$ sudo blkid
/dev/vda1: UUID="4847cb4d-6666-47e3-9e3b-12d83b2d2448" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="830679b8-01"
/dev/vda2: UUID="k5eWpP-6MXw-foh5-Vbgg-JMZ1-VEf9-ARaGNd" TYPE="LVM2_member" PARTUUID="830679b8-02"
/dev/mapper/fedora_fedora-root: UUID="f8ab802f-8c5f-4766-af33-90e78573f3cc" BLOCK_SIZE="4096" TYPE="ext4"
/dev/zram0: UUID="fc6d7a48-2bd5-4066-9bcf-f062b61f6a60" TYPE="swap"

Add the disk to LVM

Initialize the disk using pvcreate. You need to pass the full path to the device. In this example it is /dev/vdb; on your system it may be /dev/sdb or another device name.

$ sudo pvcreate /dev/vdb
Physical volume "/dev/vdb" successfully created.

You should see the disk has been initialized as an LVM2_member when you run blkid:

$ sudo blkid
/dev/vda1: UUID="4847cb4d-6666-47e3-9e3b-12d83b2d2448" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="830679b8-01"
/dev/vda2: UUID="k5eWpP-6MXw-foh5-Vbgg-JMZ1-VEf9-ARaGNd" TYPE="LVM2_member" PARTUUID="830679b8-02"
/dev/mapper/fedora_fedora-root: UUID="f8ab802f-8c5f-4766-af33-90e78573f3cc" BLOCK_SIZE="4096" TYPE="ext4"
/dev/zram0: UUID="fc6d7a48-2bd5-4066-9bcf-f062b61f6a60" TYPE="swap"
/dev/vdb: UUID="4uUUuI-lMQY-WyS5-lo0W-lqjW-Qvqw-RqeroE" TYPE="LVM2_member"

You can list all physical volumes currently available using pvs:

$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/vda2 fedora_fedora lvm2 a-- <19.00g 0
/dev/vdb lvm2 --- 10.00g 10.00g

/dev/vdb is listed as a PV (phsyical volume), but it isn’t assigned to a VG (Volume Group) yet.

Add the pysical volume to a volume group

You can find a list of available volume groups using vgs:

$ sudo vgs
VG #PV #LV #SN Attr VSize VFree
fedora_fedora 1 1 0 wz--n- 19.00g 0

In this example, there is only one volume group available. Next, add the physical volume to fedora_fedora:

$ sudo vgextend fedora_fedora /dev/vdb
Volume group "fedora_fedora" successfully extended

You should now see the physical volume is added to the volume group:

$ sudo pvs PV VG Fmt Attr PSize PFree
/dev/vda2 fedora_fedora lvm2 a– <19.00g 0
/dev/vdb fedora_fedora lvm2 a– <10.00g <10.00g

Look at the volume groups:

$ sudo vgs
VG #PV #LV #SN Attr VSize VFree
fedora_fedora 2 1 0 wz–n- 28.99g <10.00g

You can get a detailed list of the specific volume group and physical volumes as well:

$ sudo vgdisplay fedora_fedora
--- Volume group ---
VG Name fedora_fedora
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 3
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 2
Act PV 2
VG Size 28.99 GiB
PE Size 4.00 MiB
Total PE 7422
Alloc PE / Size 4863 / 19.00 GiB
Free PE / Size 2559 / 10.00 GiB
VG UUID C5dL2s-dirA-SQ15-TfQU-T3yt-l83E-oI6pkp

Look at the PV:

$ sudo pvdisplay /dev/vdb --- Physical volume --- PV Name /dev/vdb VG Name fedora_fedora PV Size 10.00 GiB / not usable 4.00 MiB Allocatable yes PE Size 4.00 MiB Total PE 2559 Free PE 2559 Allocated PE 0 PV UUID 4uUUuI-lMQY-WyS5-lo0W-lqjW-Qvqw-RqeroE

Now that we have added the disk, we can allocate space to logical volumes (LVs):

$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root fedora_fedora -wi-ao---- 19.00g

Look at the logical volumes. Here’s a detailed look at the root LV:

$ sudo lvdisplay fedora_fedora/root
--- Logical volume ---
LV Path /dev/fedora_fedora/root
LV Name root
VG Name fedora_fedora
LV UUID yqc9cw-AvOw-G1Ni-bCT3-3HAa-qnw3-qUSHGM
LV Write Access read/write
LV Creation host, time fedora, 2020-11-24 11:44:36 -0500
LV Status available
LV Size 19.00 GiB
Current LE 4863
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0

Look at the size of the root filesystem and compare it to the logical volume size.

$ df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/fedora_fedora-root 19G 1.4G 17G 8% /

The logical volume and the filesystem both agree the size is 19G. Let’s add 5G to the root logical volume:

$ sudo lvresize -L +5G fedora_fedora/root
Size of logical volume fedora_fedora/root changed from 19.00 GiB (4863 extents) to 24.00 GiB (6143 extents).
Logical volume fedora_fedora/root successfully resized.

We now have 24G available to the logical volume. Look at the / filesystem.

$ df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/fedora_fedora-root 19G 1.4G 17G 8% /

We are still showing only 19G free. This is because the logical volume is not the same as the filesytem. To use the new space added to the logical volume, resize the filesystem.

$ sudo resize2fs /dev/fedora_fedora/root
resize2fs 1.45.6 (20-Mar-2020)
Filesystem at /dev/fedora_fedora/root is mounted on /; on-line resizing required
old_desc_blocks = 3, new_desc_blocks = 3
The filesystem on /dev/fedora_fedora/root is now 6290432 (4k) blocks long.

Look at the size of the filesystem.

$ df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/fedora_fedora-root 24G 1.4G 21G 7% /

As you can see, the root file system (/) has taken all of the space available on the logical volume and no reboot was needed.

You have now initialized a disk as a physical volume, and extended the volume group with the new physical volume. After that you increased the size of the logical volume, and resized the filesystem to use the new space from the logical volume.

Posted on by — Leave a comment

5 cool terminal pagers in Fedora

Large files like logs or source code can run into the thousands of lines. That makes navigating them difficult, particularly from the terminal. Additionally, most terminal emulators have a scrollback buffer of only a few hundred lines. That can make it impossible to browse large files in the terminal using utilities which print to standard output like cat, head and tail. In the early days of computing, programmers solved these problems by developing utilities for displaying text in the form of virtual “pages” — utilities imaginatively described as pagers.

Pagers offer a number of features which make text file navigation much simpler, including scrolling, search functions, and the ability to feature as part of a pipeline of commands. In contrast to most text editors, some terminal pagers do not require loading the entire file for viewing, which makes them faster, especially for very large files.

In the modern era of Linux computing, terminal emulators are more sophisticated than ever. They offer support for a kaleidoscope of colors, terminal resizing, as well as a host of other features to make parsing text on screen easier and more efficient. Terminal pagers have undergone a similar evolution, from extremely simple UNIX utilities like pg and more, to sophisticated programs with a wide range of features, covering any number of use cases. With this in mind, we’ve put together a list of some of the most popular terminal paging utilities — more or less.

More

more is one of the earliest pagers, initially featured in version 3.0 BSD. The first implementation of more was written in 1978 by Daniel Halbert. Since then, more has become a ubiquitous feature of many operating systems, including Windows, OS/2, MacOS and most linux distributions.

more is a very lightweight utility. The version featured in util-linux runs to just under 2100 lines of C. However, this small footprint comes at a price. Most versions of more feature relatively limited functionality, with no support for backwards scroll or search. Commands are similarly stripped back: press enter to scroll one line, or space to scroll one page. Some other useful commands include:

  • Press v while reading to open the current file in your default terminal editor.
  • ‘/pattern‘ let’s you search for the next occurrence of pattern.
  • :n and :p will open the next and previous files respectively when more is called with more than one file as arguments

Less

less was initially conceived as a successor to more, addressing some of its limitations. Building on the functionality of more, less adds a number of useful features including backwards scroll, backwards search. It is also more amenable to window resizing.

Navigation in less is similar to more, though less borrows a few useful commands from the vi editor as well. Users can navigate the document using the familiar home row navigational keys. A glance at the man page for less reveals a fairly rich repertoire of available commands. Some particularly useful examples include:

  • ?pattern lets you search backwards in the file for pattern
  • &pattern shows only lines which feature pattern. This is particularly useful for those who find themselves issuing $ grep pattern | less regularly.
  • Calling less with the -s (–sqeueeze-blank-lines) flag allows you to view text files with large gaps. Multiple newline characters are reduced to single breaks.
  • s filename, called from within the program, saves input to filename (if input is a pipe).
  • Alternatively, calling less with the -o filename flag will save the input of less to filename.

With this enhanced functionality comes a little extra weight. The version of less that ships with Fedora at the time of writing clocks in at around 25000 lines of source code. Granted, for all but the most storage constrained systems, this is a non-issue. Besides, less is more than more.

Most

While less aims to expand on the existing capabilities of more, most takes a different approach. Rather than expanding on the traditional single file view, most gives users the ability to split their view into “windows.” Each window contains different files in different viewing modes.
Significantly, most takes into account the width of its input text. The default viewing mode doesn’t wrap text (-S in less), a feature particularly useful when dealing with “wide” files. While these design decisions might represent a significant departure from tradition for some users, the end result is very powerful.

In addition to the navigation commands offered by more, most uses intuitive mnemonics for file navigation. For example, t moves to the top of a file, and b moves to the bottom. As a result, users unfamiliar with vi and its descendants will find most to be refreshingly simple.

The distinguishing feature of most is its ability to split windows and contexts quickly and easily. For example, one could open two distinct text files using the following:

$ most textFile1.txt textFile2.txt

In order to split the screen horizontally, use the key combos Ctrl+x, 2 or Ctrl+w, 2. The command :n will open the next file argument in a given window, offering a split screen view of two files:

If you turn wrap off in one window, it does not affect the behavior of other windows. The \ character indicates a wrap or fold, while the $ character indicates that the file extends past the limitations of the current window.

pspg

Those who work with SQL databases often need to be able to examine the contents of our databases at a glance. The command line interfaces for many popular open source DBMS’s, such as MySQL and PostGreSQL, use the system default pager to view outputs that don’t fit on a single screen. Utilities like more and less are designed around the idea of presenting text files, but for more structured data, leave something to be desired. Naive text paginating programs have no concept of broad, tabular data, which can be frustrating when dealing with large queries.

pspg attempts to address this by offering users the ability to freeze columns while viewing, sort data in situ, and colourize output. While pspg was intended initially to serve as a pager replacement for psql specifically, the program also supports the viewing of CSV data, and is a suitable drop-in replacement for mysql and pgcli.

Vim

In a modern, technicolor terminal, the idea of endless pages of drab grey on black text can feel like something of an anachronism. The syntax highlighting options offered by powerful text editors like vim can be useful for browsing source code. Furthermore, the search functions offered by vim vastly outclass the competition. With this in mind, vim ships with a shell script less.sh that lets vim serve as a replacement for conventional pagers.

To set vim as the default pager for man pages, add the following to your shell’s config (such as ~/.bashrc if using the default bash shell):

export MANPAGER="/bin/sh -c \"col -b | vim -c 'set ft=man ts=8 nomod nolist nonu noma' -\""

Alternatively, to set vim as the default pager system-wide, locate the less.sh script. (You can find it at /usr/share/vim/vim81/macros/ on current Fedora systems.) Export this location as the variable PAGER to set it as default, or under an alias to invoke it explicitly.

Photo by Cathy Mü on Unsplash.