06-19-2018, 05:29 PM
Systemd Services: Monitoring Files and Directories
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/06/systemd-services-monitoring-files-and-directories.png" width="1500" height="877" title="" alt="" /></div><div><div><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/06/systemd-services-monitoring-files-and-directories.png" class="ff-og-image-inserted" /></div>
<p>So far in this systemd multi-part tutorial, we’ve covered <a href="https://www.linux.com/blog/learn/intro-to-linux/2018/5/writing-systemd-services-fun-and-profit">how to start and stop a service by hand</a>, <a href="https://www.linux.com/blog/learn/2018/5/systemd-services-beyond-starting-and-stopping">how to start a service when booting your OS and have it stop on power down</a>, and <a href="https://www.linux.com/blog/intro-to-linux/2018/6/systemd-services-reacting-change">how to boot a service when a certain device is detected</a>. This installment does something different yet again and covers how to create a unit that starts a service when something changes in the filesystem. For the practical example, you’ll see how you can use one of these units to extend the <a href="https://www.linux.com/blog/learn/intro-to-linux/2018/6/systemd-services-monitoring-files-and-directories">surveillance system we talked about last time</a>.</p>
<h3>Where we left off</h3>
<p><a href="https://www.linux.com/blog/intro-to-linux/2018/6/systemd-services-reacting-change">Last time we saw how the surveillance system took pictures, but it did nothing with them</a>. In fact, it even overwrote the last picture it took when it detected movement so as not to fill the storage of the device.</p>
<p>Does that mean the system is useless? Not by a long shot. Because, you see, systemd offers yet another type of units, <i>paths</i>, that can help you out. <i>Path</i> units allow you to trigger a service when an event happens in the filesystem, say, when a file gets deleted or a directory accessed. And, overwriting an image is exactly the kind of event we are talking about here.</p>
<h3>Anatomy of a Path Unit</h3>
<p>A systemd path unit takes the extension <i>.path</i>, and it monitors a file or directory. A <i>.path</i> unit calls another unit (usually a <i>.service</i> unit with the same name) when something happens to the monitored file or directory. For example, if you have a <i>picchanged.path</i> unit to monitor the snapshot from your webcam, you will also have a <i>picchanged.service</i> that will execute a script when the snapshot is overwritten.</p>
<p>Path units contain a new section, <code>[Path]</code>, with few more directives. First, you have the what-to-watch-for directives:</p>
<ul>
<li><strong><code>PathExists=</code></strong> monitors whether the file or directory exists. If it does, the associated unit gets triggered. <code>PathExistsGlob=</code> works in a similar fashion, but lets you use globbing, like when you use <code>ls *.jpg</code> to search for all the JPEG images in a directory. This lets you check, for example, whether a file with a certain extension exists.</li>
<li><strong><code>PathChanged=</code></strong> watches a file or directory and activates the configured unit whenever it changes. It is not activated on every write to the watched file but only when a monitored file open for for writing is changed and then closed. The associated unit is executed when the file is closed.</li>
<li><strong><code>PathModified=</code></strong>, on the other hand, does activate the unit when anything is changed in the file you are monitoring, even before you close the file.</li>
<li><strong><code>DirectoryNotEmpty=</code></strong> does what it says on the box, that is, it activates the associated unit if the monitored directory contains files or subdirectories.</li>
</ul>
<p>Then, we have <code>Unit=</code> that tells the <i>.path</i> which <i>.service</i> unit to activate, in case you want to give it a different name to that of your <i>.path</i> unit; <code>MakeDirectory=</code> can be <code>true</code> or <code>false</code> (or <code>0</code> or <code>1</code>, or <code>yes</code> or <code>no</code>) and creates the directory you want to monitor before monitoring starts. Obviously, using <code>MakeDirectory=</code> in combination with <code>PathExists=</code> does not make sense. However, <code>MakeDirectory=</code> can be used in combination with <code>DirectoryMode=</code>, which you use to set the the mode (permissions) of the new directory. If you don’t use <code>DirectoryMode=</code>, the default permissions for the new directory are <code>0755</code>.</p>
<h3>Building <i>picchanged.path</i></h3>
<p>All these directives are very useful, but you will be just looking for changes made to one single file, so your <i>.path</i> unit is very simple:</p>
<pre>
#picchanged.path
[Unit] Wants= webcam.service [Path] PathChanged= /home/[<i>user name</i>]/monitor/monitor.jpg </pre>
<p>In the <code>Unit=</code> section the line that says</p>
<pre>
Wants= webcam.service </pre>
<p>The <code>Wants=</code> directive is the preferred way of starting up a unit the current unit needs to work properly. <a href="https://www.linux.com/blog/intro-to-linux/2018/6/systemd-services-reacting-change"><code>webcam.service</code> is the name you gave the surveillance service that you saw in the previous article</a> and is the service that actually controls the webcam and makes it take a snap every half second. This means it’s <i>picchanged.path</i> that is going to start up <i>webcam.service</i> now, and not the <a href="https://www.linux.com/blog/intro-to-linux/2018/6/systemd-services-reacting-change">Udev rule you saw in the prior article</a>. You will use the Udev rule to start <i>picchanged.path</i> instead.</p>
<p>To summarize: the Udev rule pulls in your new <i>picchanged.path</i> unit, which, in turn pulls in the <i>webcam.service</i> as a requirement for everything to work perfectly.</p>
<p>The “thing” that <i>picchanged.path</i> monitors is the <i>monitor.jpg</i> file in the <i>monitor/</i> directory in your home directory. As you saw last time, <i>webcam.service</i> called a script, <i>checkimage.sh</i>, took a picture at the beginning of its execution and stored it in <i>monitor/temp.jpg</i>. <i>checkimage.sh</i> then took another pic, <i>temp.jpg</i>, and compared it with <i>monitor.jpg</i>. If it found significant differences (like when somebody walks into frame) the script overwrote <i>monitor.jpg</i> with the <i>temp.jpg</i>. That is when <i>picchanged.path</i> fires.</p>
<p>As you haven’t included a <code>Unit=</code> directive in your <i>.path</i>, the unit systemd expects a matching <i>picchanged.service</i> unit which it will trigger when <i>/home/[<i>user name</i>]/monitor/monitor.jpg</i> gets modified:</p>
<pre>
#picchanged.service
[Service] Type= simple ExecStart= /home/[<i>user name</i>]/bin/picmonitor.sh
</pre>
<p>For the time being, let’s make <i>picmonitor.sh</i> save a time-stamped copy of <i>monitor.jpg</i> every time changes get detected:</p>
<pre>
#!/bin/bash
# This is the pcmonitor.sh script cp /home/[<i>user name</i>]/monitor/monitor.jpg /home/[<i>user name</i>]/monitor/"`date`.jpg"
</pre>
<h3>Udev Changes</h3>
<p>You have to change the custom Udev rule you wrote in <a href="https://www.linux.com/blog/intro-to-linux/2018/6/systemd-services-reacting-change">the previous installment</a> so everything works. Edit <i>/etc/udev/rules.d/01-webcam.rules</i> so instead of looking like this:</p>
<pre>
ACTION=="add", SUBSYSTEM=="video4linux", ATTRS{idVendor}=="03f0", ATTRS{idProduct}=="e207", SYMLINK+="mywebcam", TAG+="systemd", MODE="0666", ENV{SYSTEMD_WANTS}="webcam.service"
</pre>
<p>It looks like this:</p>
<pre>
ACTION=="add", SUBSYSTEM=="video4linux", ATTRS{idVendor}=="03f0", ATTRS{idProduct}=="e207", SYMLINK+="mywebcam", TAG+="systemd", MODE="0666", ENV{SYSTEMD_WANTS}="picchanged.path"
</pre>
<p>The new rule, instead of calling <i>webcam.service</i>, now calls <i>picchanged.path</i> when your webcam gets detected. (Note that you will have to change the <code>idVendor</code> and <code>IdProduct</code> to those of your own webcam — you saw how to find these out previously).</p>
<p>For the record, I also changed <i>checkimage.sh</i> from using PNG to JPEG images. I did this because I found some dependency problems with PNG images when working with <i>mplayer</i> on some versions of Debian. <i>checkimage.sh</i> now looks like this:</p>
<pre>
#!/bin/bash mplayer -vo jpeg -frames 1 tv:// -tv driver=v4l2:width=640:height=480:device=
/dev/mywebcam &>/dev/null mv 00000001.jpg /home/paul/monitor/monitor.jpg while true do mplayer -vo jpeg -frames 1 tv:// -tv driver=v4l2:width=640:height=480:device=
/dev/mywebcam &>/dev/null mv 00000001.jpg /home/paul/monitor/temp.jpg imagediff=`compare -metric mae /home/paul/monitor/monitor.jpg /home/paul/monitor/temp.jpg /home/paul/monitor/diff.png 2>&1 > /dev/null | cut -f 1 -d " "` if [ `echo "$imagediff > 700.0" | bc` -eq 1 ] then mv /home/paul/monitor/temp.jpg /home/paul/monitor/monitor.jpg fi sleep 0.5 done
</pre>
<h3>Firing up</h3>
<p>This is a multi-unit service that, when all its bits and pieces are in place, you don’t have to worry much about: you plug in the designated webcam (or boot the machine with the webcam already connected), <i>picchanged.path</i> gets started thanks to the Udev rule and takes over, bringing up the <i>webcam.service</i> and starting to check on the snaps. There is nothing else you need to do.</p>
<h3>Conclusion</h3>
<p>Having the process split into two doesn’t only help explain how path units work, but it’s also very useful for debugging. One service does not “touch” the other in any way, which means that you could, for example, improve the “motion detection” part, and it would be very easy to roll back if things didn’t work as expected.</p>
<p>Admittedly, the example is a bit goofy, as there are definitely <a href="https://www.linux.com/learn/how-operate-linux-spycams-motion">better ways of monitoring movement using a webcam</a>. But remember: the main aim of these articles is to help you learn how systemd units work within a context.</p>
<p>Next time, we’ll finish up with systemd units by looking at some of the other types of units available and show how to improve your home-monitoring system further by setting up service that sends images to another machine.</p>
<p><em>Learn more about Linux through the free <a href="https://training.linuxfoundation.org/linux-courses/system-administration-training/introduction-to-linux">“Introduction to Linux” </a>course from The Linux Foundation and edX.</em></p>
</div>
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/06/systemd-services-monitoring-files-and-directories.png" width="1500" height="877" title="" alt="" /></div><div><div><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/06/systemd-services-monitoring-files-and-directories.png" class="ff-og-image-inserted" /></div>
<p>So far in this systemd multi-part tutorial, we’ve covered <a href="https://www.linux.com/blog/learn/intro-to-linux/2018/5/writing-systemd-services-fun-and-profit">how to start and stop a service by hand</a>, <a href="https://www.linux.com/blog/learn/2018/5/systemd-services-beyond-starting-and-stopping">how to start a service when booting your OS and have it stop on power down</a>, and <a href="https://www.linux.com/blog/intro-to-linux/2018/6/systemd-services-reacting-change">how to boot a service when a certain device is detected</a>. This installment does something different yet again and covers how to create a unit that starts a service when something changes in the filesystem. For the practical example, you’ll see how you can use one of these units to extend the <a href="https://www.linux.com/blog/learn/intro-to-linux/2018/6/systemd-services-monitoring-files-and-directories">surveillance system we talked about last time</a>.</p>
<h3>Where we left off</h3>
<p><a href="https://www.linux.com/blog/intro-to-linux/2018/6/systemd-services-reacting-change">Last time we saw how the surveillance system took pictures, but it did nothing with them</a>. In fact, it even overwrote the last picture it took when it detected movement so as not to fill the storage of the device.</p>
<p>Does that mean the system is useless? Not by a long shot. Because, you see, systemd offers yet another type of units, <i>paths</i>, that can help you out. <i>Path</i> units allow you to trigger a service when an event happens in the filesystem, say, when a file gets deleted or a directory accessed. And, overwriting an image is exactly the kind of event we are talking about here.</p>
<h3>Anatomy of a Path Unit</h3>
<p>A systemd path unit takes the extension <i>.path</i>, and it monitors a file or directory. A <i>.path</i> unit calls another unit (usually a <i>.service</i> unit with the same name) when something happens to the monitored file or directory. For example, if you have a <i>picchanged.path</i> unit to monitor the snapshot from your webcam, you will also have a <i>picchanged.service</i> that will execute a script when the snapshot is overwritten.</p>
<p>Path units contain a new section, <code>[Path]</code>, with few more directives. First, you have the what-to-watch-for directives:</p>
<ul>
<li><strong><code>PathExists=</code></strong> monitors whether the file or directory exists. If it does, the associated unit gets triggered. <code>PathExistsGlob=</code> works in a similar fashion, but lets you use globbing, like when you use <code>ls *.jpg</code> to search for all the JPEG images in a directory. This lets you check, for example, whether a file with a certain extension exists.</li>
<li><strong><code>PathChanged=</code></strong> watches a file or directory and activates the configured unit whenever it changes. It is not activated on every write to the watched file but only when a monitored file open for for writing is changed and then closed. The associated unit is executed when the file is closed.</li>
<li><strong><code>PathModified=</code></strong>, on the other hand, does activate the unit when anything is changed in the file you are monitoring, even before you close the file.</li>
<li><strong><code>DirectoryNotEmpty=</code></strong> does what it says on the box, that is, it activates the associated unit if the monitored directory contains files or subdirectories.</li>
</ul>
<p>Then, we have <code>Unit=</code> that tells the <i>.path</i> which <i>.service</i> unit to activate, in case you want to give it a different name to that of your <i>.path</i> unit; <code>MakeDirectory=</code> can be <code>true</code> or <code>false</code> (or <code>0</code> or <code>1</code>, or <code>yes</code> or <code>no</code>) and creates the directory you want to monitor before monitoring starts. Obviously, using <code>MakeDirectory=</code> in combination with <code>PathExists=</code> does not make sense. However, <code>MakeDirectory=</code> can be used in combination with <code>DirectoryMode=</code>, which you use to set the the mode (permissions) of the new directory. If you don’t use <code>DirectoryMode=</code>, the default permissions for the new directory are <code>0755</code>.</p>
<h3>Building <i>picchanged.path</i></h3>
<p>All these directives are very useful, but you will be just looking for changes made to one single file, so your <i>.path</i> unit is very simple:</p>
<pre>
#picchanged.path
[Unit] Wants= webcam.service [Path] PathChanged= /home/[<i>user name</i>]/monitor/monitor.jpg </pre>
<p>In the <code>Unit=</code> section the line that says</p>
<pre>
Wants= webcam.service </pre>
<p>The <code>Wants=</code> directive is the preferred way of starting up a unit the current unit needs to work properly. <a href="https://www.linux.com/blog/intro-to-linux/2018/6/systemd-services-reacting-change"><code>webcam.service</code> is the name you gave the surveillance service that you saw in the previous article</a> and is the service that actually controls the webcam and makes it take a snap every half second. This means it’s <i>picchanged.path</i> that is going to start up <i>webcam.service</i> now, and not the <a href="https://www.linux.com/blog/intro-to-linux/2018/6/systemd-services-reacting-change">Udev rule you saw in the prior article</a>. You will use the Udev rule to start <i>picchanged.path</i> instead.</p>
<p>To summarize: the Udev rule pulls in your new <i>picchanged.path</i> unit, which, in turn pulls in the <i>webcam.service</i> as a requirement for everything to work perfectly.</p>
<p>The “thing” that <i>picchanged.path</i> monitors is the <i>monitor.jpg</i> file in the <i>monitor/</i> directory in your home directory. As you saw last time, <i>webcam.service</i> called a script, <i>checkimage.sh</i>, took a picture at the beginning of its execution and stored it in <i>monitor/temp.jpg</i>. <i>checkimage.sh</i> then took another pic, <i>temp.jpg</i>, and compared it with <i>monitor.jpg</i>. If it found significant differences (like when somebody walks into frame) the script overwrote <i>monitor.jpg</i> with the <i>temp.jpg</i>. That is when <i>picchanged.path</i> fires.</p>
<p>As you haven’t included a <code>Unit=</code> directive in your <i>.path</i>, the unit systemd expects a matching <i>picchanged.service</i> unit which it will trigger when <i>/home/[<i>user name</i>]/monitor/monitor.jpg</i> gets modified:</p>
<pre>
#picchanged.service
[Service] Type= simple ExecStart= /home/[<i>user name</i>]/bin/picmonitor.sh
</pre>
<p>For the time being, let’s make <i>picmonitor.sh</i> save a time-stamped copy of <i>monitor.jpg</i> every time changes get detected:</p>
<pre>
#!/bin/bash
# This is the pcmonitor.sh script cp /home/[<i>user name</i>]/monitor/monitor.jpg /home/[<i>user name</i>]/monitor/"`date`.jpg"
</pre>
<h3>Udev Changes</h3>
<p>You have to change the custom Udev rule you wrote in <a href="https://www.linux.com/blog/intro-to-linux/2018/6/systemd-services-reacting-change">the previous installment</a> so everything works. Edit <i>/etc/udev/rules.d/01-webcam.rules</i> so instead of looking like this:</p>
<pre>
ACTION=="add", SUBSYSTEM=="video4linux", ATTRS{idVendor}=="03f0", ATTRS{idProduct}=="e207", SYMLINK+="mywebcam", TAG+="systemd", MODE="0666", ENV{SYSTEMD_WANTS}="webcam.service"
</pre>
<p>It looks like this:</p>
<pre>
ACTION=="add", SUBSYSTEM=="video4linux", ATTRS{idVendor}=="03f0", ATTRS{idProduct}=="e207", SYMLINK+="mywebcam", TAG+="systemd", MODE="0666", ENV{SYSTEMD_WANTS}="picchanged.path"
</pre>
<p>The new rule, instead of calling <i>webcam.service</i>, now calls <i>picchanged.path</i> when your webcam gets detected. (Note that you will have to change the <code>idVendor</code> and <code>IdProduct</code> to those of your own webcam — you saw how to find these out previously).</p>
<p>For the record, I also changed <i>checkimage.sh</i> from using PNG to JPEG images. I did this because I found some dependency problems with PNG images when working with <i>mplayer</i> on some versions of Debian. <i>checkimage.sh</i> now looks like this:</p>
<pre>
#!/bin/bash mplayer -vo jpeg -frames 1 tv:// -tv driver=v4l2:width=640:height=480:device=
/dev/mywebcam &>/dev/null mv 00000001.jpg /home/paul/monitor/monitor.jpg while true do mplayer -vo jpeg -frames 1 tv:// -tv driver=v4l2:width=640:height=480:device=
/dev/mywebcam &>/dev/null mv 00000001.jpg /home/paul/monitor/temp.jpg imagediff=`compare -metric mae /home/paul/monitor/monitor.jpg /home/paul/monitor/temp.jpg /home/paul/monitor/diff.png 2>&1 > /dev/null | cut -f 1 -d " "` if [ `echo "$imagediff > 700.0" | bc` -eq 1 ] then mv /home/paul/monitor/temp.jpg /home/paul/monitor/monitor.jpg fi sleep 0.5 done
</pre>
<h3>Firing up</h3>
<p>This is a multi-unit service that, when all its bits and pieces are in place, you don’t have to worry much about: you plug in the designated webcam (or boot the machine with the webcam already connected), <i>picchanged.path</i> gets started thanks to the Udev rule and takes over, bringing up the <i>webcam.service</i> and starting to check on the snaps. There is nothing else you need to do.</p>
<h3>Conclusion</h3>
<p>Having the process split into two doesn’t only help explain how path units work, but it’s also very useful for debugging. One service does not “touch” the other in any way, which means that you could, for example, improve the “motion detection” part, and it would be very easy to roll back if things didn’t work as expected.</p>
<p>Admittedly, the example is a bit goofy, as there are definitely <a href="https://www.linux.com/learn/how-operate-linux-spycams-motion">better ways of monitoring movement using a webcam</a>. But remember: the main aim of these articles is to help you learn how systemd units work within a context.</p>
<p>Next time, we’ll finish up with systemd units by looking at some of the other types of units available and show how to improve your home-monitoring system further by setting up service that sends images to another machine.</p>
<p><em>Learn more about Linux through the free <a href="https://training.linuxfoundation.org/linux-courses/system-administration-training/introduction-to-linux">“Introduction to Linux” </a>course from The Linux Foundation and edX.</em></p>
</div>