Create an account


Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Fedora - Using mergerfs to increase your virtual storage

#1
Using mergerfs to increase your virtual storage

What happens if you have multiple disks or partitions that you’d like to use for a media project and you don’t want to lose any of your existing data, but you’d like to have everything located or mounted under one drive. That’s where mergerfs can come to your rescue!

mergerfs is a union filesystem geared towards simplifying storage and management of files across numerous commodity storage devices.

You will need to grab the latest RPM from their github page here. The releases for Fedora have fc and the version number in the name. For example here is the version for Fedora 31:

mergerfs-2.29.0-1.fc31.x86_64.rpm

Installing and configuring mergerfs


Install the mergerfs package that you’ve downloaded using sudo:

$ sudo dnf install mergerfs-2.29.0-1.fc31.x86_64.rpm

You will now be able to mount multiple disks as one drive. This comes in handy if you have a media server and you’d like all of your media files to show up under one location. If you upload new files to your system, you can copy them to your mergerfs directory and mergerfs will automatically copy them to which ever drive has enough free space available.

Here is an example to make it easier to understand:

$ df -hT | grep disk
/dev/sdb1 ext4 23M 386K 21M 2% /disk1
/dev/sdc1 ext4 44M 1.1M 40M 3% /disk2 $ ls -l /disk1/Videos/
total 1
-rw-r--r--. 1 curt curt 0 Mar 8 17:17 Our Wedding.mkv $ ls -l /disk2/Videos/
total 2
-rw-r--r--. 1 curt curt 0 Mar 8 17:17 Baby's first Xmas.mkv
-rw-rw-r--. 1 curt curt 0 Mar 8 17:21 Halloween hijinks.mkv

In this example there are two disks mounted as disk1 and disk2. Both drives have a Videos directory with existing files.

Now we’re going to mount those drives using mergerfs to make them appear as one larger drive.

$ sudo mergerfs -o defaults,allow_other,use_ino,category.create=mfs,moveonenospc=true,minfreespace=1M /disk1:/disk2 /media

The mergerfs man page is quite extensive and complex so we’ll break down the options that were specified.

  • defaults: This will use the default settings unless specified.
  • allow_other: allows users besides sudo or root to see the filesystem.
  • use_ino: Causes mergerfs to supply file/directory inodes rather than libfuse. While not a default it is recommended it be enabled so that linked files share the same inode value.
  • category.create=mfs: Spreads files out across your drives based on available space.
  • moveonenospc=true: If enabled, if writing fails, a scan will be done looking for the drive with the most free space.
  • minfreespace=1M: The minimum space value used.
  • disk1: First hard drive.
  • disk2: Second hard drive.
  • /media: The directory folder where the drives are mounted.

Here is what it looks like:

$ df -hT | grep disk /dev/sdb1 ext4 23M 386K 21M 2% /disk1 /dev/sdc1 ext4 44M 1.1M 40M 3% /disk2 $ df -hT | grep media 1:2 fuse.mergerfs 66M 1.4M 60M 3% /media 

You can see that the mergerfs mount now shows a total capacity of 66M which is the combined total of the two hard drives.

Continuing with the example:

There is a 30Mb video called Baby’s second Xmas.mkv. Let’s copy it to the /media folder which is the mergerfs mount.

$ ls -lh "Baby's second Xmas.mkv"
-rw-rw-r--. 1 curt curt 30M Apr 20 08:45 Baby's second Xmas.mkv
$ cp "Baby's second Xmas.mkv" /media/Videos/

Here is the end result:

$ df -hT | grep disk
/dev/sdb1 ext4 23M 386K 21M 2% /disk1
/dev/sdc1 ext4 44M 31M 9.8M 76% /disk2 $ df -hT | grep media
1:2 fuse.mergerfs 66M 31M 30M 51% /media

You can see from the disk space utilization that mergerfs automatically copied the file to disk2 because disk1 did not have enough free space.

Here is a breakdown of all of the files:

$ ls -l /disk1/Videos/
total 1
-rw-r--r--. 1 curt curt 0 Mar 8 17:17 Our Wedding.mkv $ ls -l /disk2/Videos/
total 30003
-rw-r--r--. 1 curt curt 0 Mar 8 17:17 Baby's first Xmas.mkv
-rw-rw-r--. 1 curt curt 30720000 Apr 20 08:47 Baby's second Xmas.mkv
-rw-rw-r--. 1 curt curt 0 Mar 8 17:21 Halloween hijinks.mkv $ ls -l /media/Videos/
total 30004
-rw-r--r--. 1 curt curt 0 Mar 8 17:17 Baby's first Xmas.mkv
-rw-rw-r--. 1 curt curt 30720000 Apr 20 08:47 Baby's second Xmas.mkv
-rw-rw-r--. 1 curt curt 0 Mar 8 17:21 Halloween hijinks.mkv
-rw-r--r--. 1 curt curt 0 Mar 8 17:17 Our Wedding.mkv

When you copy files to your mergerfs mount, it will always copy the files to the hard disk that has enough free space. If none of the drives in the pool have enough free space, then you won’t be able to copy them.



https://www.sickgaming.net/blog/2020/05/...l-storage/
Reply



Forum Jump:


Users browsing this thread:
1 Guest(s)

[-]
Discord

[-]
Active Threads
Работа со свободным графиком. Нужны все
Last Post: Ronaldfloon
Today 04:59 AM
» Replies: 0
» Views: 1
Быстрое трудоустройство, высокая ЗП!
Last Post: Ronaldfloon
Today 02:32 AM
» Replies: 0
» Views: 8
Частная организация ищет работников по н...
Last Post: Ronaldfloon
Today 12:06 AM
» Replies: 0
» Views: 8
[Tut] Python __ne__ Magic Method
Last Post: xSicKxBot
Yesterday 01:27 PM
» Replies: 0
» Views: 28
(Indie Deal) Tales of Arise, Paradox, Ko...
Last Post: xSicKxBot
Yesterday 01:27 PM
» Replies: 0
» Views: 4
AppleInsider - These are the Mac feature...
Last Post: xSicKxBot
Yesterday 01:26 PM
» Replies: 0
» Views: 4
Microsoft - Microsoft Turing-NLRv5 achie...
Last Post: xSicKxBot
Yesterday 01:26 PM
» Replies: 0
» Views: 4
Xbox Wire - Crazy Athletics Is Available...
Last Post: xSicKxBot
Yesterday 01:26 PM
» Replies: 0
» Views: 4
News - CopperCube 6.5 Released
Last Post: xSicKxBot
Yesterday 01:26 PM
» Replies: 0
» Views: 4
Steam - Team Fortress 2 Update Released
Last Post: GilbertBib
Yesterday 08:34 AM
» Replies: 1
» Views: 131

[-]
Twitter

[-]
Sponsored
Get the Deal of the Week at RefurBees.com



Discord Server © SickGaming.net 2012-2021