Modifying Windows local accounts with Fedora and chntpw

I recently encountered a problem at work where a client’s Windows 10 PC lost trust to the domain. The user is an executive and the hindrance of his computer can affect real-time mission-critical tasks. He gave me 30 minutes to resolve the issue while he attended a meeting.

Needless to say, I’ve encountered this issue many times in my career. It’s an easy fix using the Windows 7/8/10 installation media to reset the Administrator password, remove the PC off the domain and rejoin it. Unfortunately it didn’t work this time. After 20 minutes of scouring the net and scanning through the Microsoft Docs with no success, I turned to my development machine running Fedora with hopes of finding a solution.

With dnf search I found a utility called chntpw:

$ dnf search windows | grep password

According to the summary, chntpw will “change passwords in Windows SAM files.”

Little did I know at the time there was more to this utility than explained in the summary. Hence, this article will go through the steps I used to successfully reset a Windows local user password using chntpw and a Fedora Workstation Live boot USB. The article will also cover some of the features of chntpw used for basic user administration.

Installation and setup

If the PC can connect to the internet after booting the live media, install chntpw from the official Fedora repository with:

$ sudo dnf install chntpw

If you’re unable to access the internet, no sweat! Fedora Workstation Live boot media has all the dependencies installed out-of-the-box, so all we need is the package. You can find the builds for your Fedora version from the Fedora Project’s Koji site. You can use another computer to download the utility and use a USB thumb drive, or other form of media to copy the package.

First and foremost we need to create the Fedora Live USB stick. If you need instructions, the article on How to make a Fedora USB stick is a great reference.

Once the key is created shut-down the Windows PC, insert the thumb drive if the USB key was created on another computer, and turn on the PC — be sure to boot from the USB drive. Once the live media boots, select “Try Fedora” and open the Terminal application.

Also, we need to mount the Windows drive to access the files. Enter the following command to view all drive partitions with an NTFS filesystem:

$ sudo blkid | grep ntfs

Most hard drives are assigned to /dev/sdaX where X is the partition number — virtual drives may be assigned to /dev/vdX, and some newer drives (like SSDs) use /dev/nvmeX. For this example the Windows C drive is assigned to /dev/sda2. To mount the drive enter:

$ sudo mount /dev/sda2 /mnt

Fedora Workstation contains the ntfs-3g and ntfsprogs packages out-of-the-box. If you’re using a spin that does not have NTFS working out of the box, you can install these two packages from the official Fedora repository with:

$ sudo dnf install ntfs-3g ntfsprogs

Once the drive is mounted, navigate to the location of the SAM file and verify that it’s there:

$ cd /mnt/Windows/System32/config
$ ls | grep SAM
SAM
SAM.LOG1
SAM.LOG2

Clearing or resetting a password

Now it’s time to get to work. The help flag -h provides everything we need to know about this utility and how to use it:

$ chntpw -h
chntpw: change password of a user in a Windows SAM file,
or invoke registry editor. Should handle both 32 and 64 bit windows and
all version from NT3.x to Win8.1
chntpw [OPTIONS] [systemfile] [securityfile] [otherreghive] […]
-h This message
-u Username or RID (0x3e9 for example) to interactively edit
-l list all users in SAM file and exit
-i Interactive Menu system
-e Registry editor. Now with full write support!
-d Enter buffer debugger instead (hex editor),
-v Be a little more verbose (for debuging)
-L For scripts, write names of changed files to /tmp/changed
-N No allocation mode. Only same length overwrites possible (very safe mode)
-E No expand mode, do not expand hive file (safe mode)

Usernames can be given as name or RID (in hex with 0x first)
See readme file on how to get to the registry files, and what they are.
Source/binary freely distributable under GPL v2 license. See README for details.
NOTE: This program is somewhat hackish! You are on your own!

Use the -l parameter to display a list of users it reads from the SAM file:

$ sudo chntpw -l SAM
chntpw version 1.00 140201, (c) Petter N Hagen
Hive name (from header): <\SystemRoot\System32\Config\SAM>
ROOT KEY at offset: 0x001020 * Subkey indexing type is: 686c
File size 65536 [10000] bytes, containing 7 pages (+ 1 headerpage)
Used for data: 346/37816 blocks/bytes, unused: 23/7016 blocks/bytes.

| RID -|---------- Username ------------| Admin? |- Lock? --|
| 01f4 | Administrator | ADMIN | dis/lock |
| 01f7 | DefaultAccount | | dis/lock |
| 03e8 | defaultuser0 | | dis/lock |
| 01f5 | Guest | | dis/lock |
| 03ea | sysadm | ADMIN | |
| 01f8 | WDAGUtilityAccount | | dis/lock |
| 03e9 | WinUser | | |

Now that we have a list of Windows users we can edit the account. Use the -u parameter followed by the username and the name of the SAM file. For this example, edit the sysadm account:

$ sudo chntpw -u sysadm SAM
chntpw version 1.00 140201, (c) Petter N Hagen
Hive name (from header): <\SystemRoot\System32\Config\SAM>
ROOT KEY at offset: 0x001020 * Subkey indexing type is: 686c
File size 65536 [10000] bytes, containing 7 pages (+ 1 headerpage)
Used for data: 346/37816 blocks/bytes, unused: 23/7016 blocks/bytes.

================= USER EDIT ====================

RID : 1002 [03ea]
Username: sysadm
fullname: SysADM
comment :
homedir :

00000220 = Administrators (which has 2 members)

Account bits: 0x0010 =
[ ] Disabled | [ ] Homedir req. | [ ] Passwd not req. |
[ ] Temp. duplicate | [X] Normal account | [ ] NMS account |
[ ] Domain trust ac | [ ] Wks trust act. | [ ] Srv trust act |
[ ] Pwd don't expir | [ ] Auto lockout | [ ] (unknown 0x08) |
[ ] (unknown 0x10) | [ ] (unknown 0x20) | [ ] (unknown 0x40) |

Failed login count: 0, while max tries is: 0
Total login count: 0

- - - User Edit Menu:
1 - Clear (blank) user password
(2 - Unlock and enable user account) [seems unlocked already]
3 - Promote user (make user an administrator)
4 - Add user to a group
5 - Remove user from a group
q - Quit editing user, back to user select
Select: [q] >

To clear the password press 1 and ENTER. If successful you will see the following message:

...
Select: [q] > 1
Password cleared!
================= USER EDIT ====================

RID : 1002 [03ea]
Username: sysadm
fullname: SysADM
comment :
homedir :

00000220 = Administrators (which has 2 members)

Account bits: 0x0010 =
[ ] Disabled | [ ] Homedir req. | [ ] Passwd not req. |
[ ] Temp. duplicate | [X] Normal account | [ ] NMS account |
[ ] Domain trust ac | [ ] Wks trust act. | [ ] Srv trust act |
[ ] Pwd don't expir | [ ] Auto lockout | [ ] (unknown 0x08) |
[ ] (unknown 0x10) | [ ] (unknown 0x20) | [ ] (unknown 0x40) |

Failed login count: 0, while max tries is: 0
Total login count: 0
** No NT MD4 hash found. This user probably has a BLANK password!
** No LANMAN hash found either. Try login with no password!
...

Verify the change by repeating:

$ sudo chntpw -l SAM
chntpw version 1.00 140201, (c) Petter N Hagen
Hive name (from header): <\SystemRoot\System32\Config\SAM>
ROOT KEY at offset: 0x001020 * Subkey indexing type is: 686c
File size 65536 [10000] bytes, containing 7 pages (+ 1 headerpage)
Used for data: 346/37816 blocks/bytes, unused: 23/7016 blocks/bytes.

| RID -|---------- Username ------------| Admin? |- Lock? --|
| 01f4 | Administrator | ADMIN | dis/lock |
| 01f7 | DefaultAccount | | dis/lock |
| 03e8 | defaultuser0 | | dis/lock |
| 01f5 | Guest | | dis/lock |
| 03ea | sysadm | ADMIN | *BLANK* |
| 01f8 | WDAGUtilityAccount | | dis/lock |
| 03e9 | WinUser | | |

...

The “Lock?” column now shows BLANK for the sysadm user. Type q to exit and y to write the changes to the SAM file. Reboot the machine into Windows and login using the account (in this case sysadm) without a password.

Features

Furthermore, chntpw can perform basic Windows user administrative tasks. It has the ability to promote the user to the administrators group, unlock accounts, view and modify group memberships, and edit the registry.

The interactive menu

chntpw has an easy-to-use interactive menu to guide you through the process. Use the -i parameter to launch the interactive menu:

$ chntpw -i SAM
chntpw version 1.00 140201, (c) Petter N Hagen
Hive name (from header): <\SystemRoot\System32\Config\SAM>
ROOT KEY at offset: 0x001020 * Subkey indexing type is: 686c
File size 65536 [10000] bytes, containing 7 pages (+ 1 headerpage)
Used for data: 346/37816 blocks/bytes, unused: 23/7016 blocks/bytes.

<>========<> chntpw Main Interactive Menu <>========<>
Loaded hives:
1 - Edit user data and passwords
2 - List groups
- - -
9 - Registry editor, now with full write support!
q - Quit (you will be asked if there is something to save)

Groups and account membership

To display a list of groups and view its members, select option 2 from the interactive menu:

...
What to do? [1] -> 2
Also list group members? [n] y
=== Group # 220 : Administrators
0 | 01f4 | Administrator |
1 | 03ea | sysadm |
=== Group # 221 : Users
0 | 0004 | NT AUTHORITY\INTERACTIVE |
1 | 000b | NT AUTHORITY\Authenticated Users |
2 | 03e8 | defaultuser0 |
3 | 03e9 | WinUser |
=== Group # 222 : Guests
0 | 01f5 | Guest |
=== Group # 223 : Power Users
...
=== Group # 247 : Device Owners

Adding the user to the administrators group

To elevate the user with administrative privileges press 1 to edit the account, then 3 to promote the user:

...
Select: [q] > 3

=== PROMOTE USER
Will add the user to the administrator group (0x220)
and to the users group (0x221). That should usually be
what is needed to log in and get administrator rights.
Also, remove the user from the guest group (0x222), since
it may forbid logins.

(To add or remove user from other groups, please other menu selections)

Note: You may get some errors if the user is already member of some
of these groups, but that is no problem.

Do it? (y/n) [n] : y

Adding to 0x220 (Administrators) …
sam_put_user_grpids: success exit
Adding to 0x221 (Users) …
sam_put_user_grpids: success exit
Removing from 0x222 (Guests) …
remove_user_from_grp: NOTE: group not in users list of groups, may mean user not member at all. Safe. Continuing.
remove_user_from_grp: NOTE: user not in groups list of users, may mean user was not member at all. Does not matter, continuing.
sam_put_user_grpids: success exit

Promotion DONE!

Editing the Windows registry

Certainly the most noteworthy, as well as the most powerful, feature of chntpw is the ability to edit the registry and write to it. Select 9 from the interactive menu:

...
What to do? [1] -> 9
Simple registry editor. ? for help.

> ?
Simple registry editor:
hive [] - list loaded hives or switch to hive number
cd - change current key
ls | dir [] - show subkeys & values,
cat | type - show key value
dpi - show decoded DigitalProductId value
hex - hexdump of value data
ck [] - Show keys class data, if it has any
nk - add key
dk - delete key (must be empty)
ed - Edit value
nv - Add value
dv - Delete value
delallv - Delete all values in current key
rdel - Recursively delete key & subkeys
ek - export key to (Windows .reg file format)
debug - enter buffer hexeditor
st [] - debug function: show struct info
q - quit

Finding help

As we saw earlier, the -h parameter allows us to quickly access a reference guide to the options available with chntpw. The man page contains detailed information and can be accessed with:

$ man chntpw

Also, if you’re interested in a more hands-on approach, spin up a virtual machine. Windows Server 2019 has an evaluation period of 180 days, and Windows Hyper-V Server 2019 is unlimited. Creating a Windows guest VM will provide the basics to modify the Administrator account for testing and learning. For help with quickly creating a guest VM refer to the article Getting started with virtualization in Gnome Boxes.

Conclusion

chntpw is a hidden gem for Linux administrators and IT professionals alike. While a nifty tool to quickly reset Windows account passwords, it can also be used to troubleshoot and modify local Windows accounts with a no-nonsense feel that delivers. This is perhaps only one such tool for solving the problem, though. If you’ve experienced this issue and have an alternative solution, feel free to put it in the comments below.

This tool, like many other “hacking” tools, holds with it an ethical responsibility. Even chntpw states:

NOTE: This program is somewhat hackish! You are on your own!

When using such programs, we should remember the three edicts outlined in the message displayed when running sudo for the first time:

  1. Respect the privacy of others.
  2. Think before you type.
  3. With great power comes great responsibility.

Photo by Silas Köhler on Unsplash,

Bond WiFi and Ethernet for easier networking mobility

Sometimes one network interface isn’t enough. Network bonding allows multiple network connections to act together with a single logical interface. You might do this because you want more bandwidth than a single connection can handle. Or maybe you want to switch back and forth between your wired and wireless networks without losing your network connection.

The latter applies to me. One of the benefits to working from home is that when the weather is nice, it’s enjoyable to work from a sunny deck instead of inside. But every time I did that, I lost my network connections. IRC, SSH, VPN — everything goes away, at least for a moment while some clients reconnect. This article describes how I set up network bonding on my Fedora 30 laptop to seamlessly move from the wired connection my laptop dock to a WiFi connection.

In Linux, interface bonding is handled by the bonding kernel module. Fedora does not ship with this enabled by default, but it is included in the kernel-core package. This means that enabling interface bonding is only a command away:

sudo modprobe bonding

Note that this will only have effect until you reboot. To permanently enable interface bonding, create a file called bonding.conf in the /etc/modules-load.d directory that contains only the word “bonding”.

Now that you have bonding enabled, it’s time to create the bonded interface. First, you must get the names of the interfaces you want to bond. To list the available interfaces, run:

sudo nmcli device status

You will see output that looks like this:

DEVICE          TYPE      STATE         CONNECTION         
enp12s0u1       ethernet  connected     Wired connection 1
tun0            tun       connected     tun0               
virbr0          bridge    connected     virbr0             
wlp2s0          wifi      disconnected  --      
p2p-dev-wlp2s0  wifi-p2p disconnected  --      
enp0s31f6       ethernet  unavailable   --      
lo              loopback  unmanaged     --                 
virbr0-nic      tun       unmanaged     --       

In this case, there are two (wired) Ethernet interfaces available. enp12s0u1 is on a laptop docking station, and you can tell that it’s connected from the STATE column. The other, enp0s31f6, is the built-in port in the laptop. There is also a WiFi connection called wlp2s0. enp12s0u1 and wlp2s0 are the two interfaces we’re interested in here. (Note that it’s not necessary for this exercise to understand how network devices are named, but if you’re interested you can see the systemd.net-naming-scheme man page.)

The first step is to create the bonded interface:

sudo nmcli connection add type bond ifname bond0 con-name bond0

In this example, the bonded interface is named bond0. The “con-name bond0” sets the connection name to bond0; leaving this off would result in a connection named bond-bond0. You can also set the connection name to something more human-friendly, like “Docking station bond” or “Ben”

The next step is to add the interfaces to the bonded interface:

sudo nmcli connection add type ethernet ifname enp12s0u1 master bond0 con-name bond-ethernet
sudo nmcli connection add type wifi ifname wlp2s0 master bond0 ssid Cotton con-name bond-wifi

As above, the connection name is specified to be more descriptive. Be sure to replace enp12s0u1 and wlp2s0 with the appropriate interface names on your system. For the WiFi interface, use your own network name (SSID) where I use “Cotton”. If your WiFi connection has a password (and of course it does!), you’ll need to add that to the configuration, too. The following assumes you’re using WPA2-PSK authentication

sudo nmcli connection modify bond-wifi wifi-sec.key-mgmt wpa-psk
sudo nmcli connection edit bond-wif

The second command will bring you into the interactive editor where you can enter your password without it being logged in your shell history. Enter the following, replacing password with your actual password

set wifi-sec.psk password
save
quit

Now you’re ready to start your bonded interface and the secondary interfaces you created

sudo nmcli connection up bond0
sudo nmcli connection up bond-ethernet
sudo nmcli connection up bond-wifi

You should now be able to disconnect your wired or wireless connections without losing your network connections.

A caveat: using other WiFi networks

This configuration works well when moving around on the specified WiFi network, but when away from this network, the SSID used in the bond is not available. Theoretically, one could add an interface to the bond for every WiFi connection used, but that doesn’t seem reasonable. Instead, you can disable the bonded interface:

sudo nmcli connection down bond0

When back on the defined WiFi network, simply start the bonded interface as above.

Fine-tuning your bond

By default, the bonded interface uses the “load balancing (round-robin)” mode. This spreads the load equally across the interfaces. But if you have a wired and a wireless connection, you may want to prefer the wired connection. The “active-backup” mode enables this. You can specify the mode and primary interface when you are creating the interface, or afterward using this command (the bonded interface should be down):

sudo nmcli connection modify bond0 +bond.options "mode=active-backup,primary=enp12s0u1"

The kernel documentation has much more information about bonding options.

What is Silverblue?

Fedora Silverblue is becoming more and more popular inside and outside the Fedora world. So based on feedback from the community, here are answers to some interesting questions about the project. If you do have any other Silverblue related questions, please leave it in the comments section and we will try to answer them in a future article.

What is Silverblue?

Silverblue is a codename for the new generation of the desktop operating system, previously known as Atomic Workstation. The operating system is delivered in images that are created by utilizing the rpm-ostree project. The main benefits of the system are speed, security, atomic updates and immutability.

What does “Silverblue” actually mean?

“Team Silverblue” or “Silverblue” in short doesn’t have any hidden meaning. It was chosen after roughly two months when the project, previously known as Atomic Workstation was rebranded. There were over 150 words or word combinations reviewed in the process. In the end Silverblue was chosen because it had an available domain as well as the social network accounts. One could think of it as a new take on Fedora’s blue branding, and could be used in phrases like “Go, Team Silverblue!” or “Want to join the team and improve Silverblue?”.

What is ostree?

OSTree or libostree is a project that combines a “git-like” model for committing and downloading bootable filesystem trees, together with a layer to deploy them and manage the bootloader configuration. OSTree is used by rpm-ostree, a hybrid package/image based system that Silverblue uses. It atomically replicates a base OS and allows the user to “layer” the traditional RPM on top of the base OS if needed.

Why use Silverblue?

Because it allows you to concentrate on your work and not on the operating system you’re running. It’s more robust as the updates of the system are atomic. The only thing you need to do is to restart into the new image. Also, if there’s anything wrong with the currently booted image, you can easily reboot/rollback to the previous working one, if available. If it isn’t, you can download and boot any other image that was generated in the past, using the ostree command.

Another advantage is the possibility of an easy switch between branches (or, in an old context, Fedora releases). You can easily try the Rawhide or updates-testing branch and then return back to the one that contains the current stable release. Also, you should consider Silverblue if you want to try something new and unusual.

What are the benefits of an immutable OS?

One of the main benefits is security. The base operating system is mounted as read-only, and thus cannot be modified by malicious software. The only way to alter the system is through the rpm-ostree utility.

Another benefit is robustness. It’s nearly impossible for a regular user to get the OS to the state when it doesn’t boot or doesn’t work properly after accidentally or unintentionally removing some system library. Try to think about these kind of experiences from your past, and imagine how Silverblue could help you there.

How does one manage applications and packages in Silverblue?

For graphical user interface applications, Flatpak is recommended, if the application is available as a flatpak. Users can choose between Flatpaks from either Fedora and built from Fedora packages and in Fedora-owned infrastructure, or Flathub that currently has a wider offering. Users can install them easily through GNOME Software, which already supports Fedora Silverblue.

One of the first things users find out is there is no dnf preinstalled in the OS. The main reason is that it wouldn’t work on Silverblue — and part of its functionality was replaced by the rpm-ostree command. Users can overlay the traditional packages by using the rpm-ostree install PACKAGE. But it should only be used when there is no other way. This is because when the new system images are pulled from the repository, the system image must be rebuilt every time it is altered to accommodate the layered packages, or packages that were removed from the base OS or replaced with a different version.

Fedora Silverblue comes with the default set of GUI applications that are part of the base OS. The team is working on porting them to Flatpaks so they can be distributed that way. As a benefit, the base OS will become smaller and easier to maintain and test, and users can modify their default installation more easily. If you want to look at how it’s done or help, take a look at the official documentation.

What is Toolbox?

Toolbox is a project to make containers easily consumable for regular users. It does that by using podman’s rootless containers. Toolbox lets you easily and quickly create a container with a regular Fedora installation that you can play with or develop on, separated from your OS.

Is there any Silverblue roadmap?

Formally there isn’t any, as we’re focusing on problems we discover during our testing and from community feedback. We’re currently using Fedora’s Taiga to do our planning.

What’s the release life cycle of the Silverblue?

It’s the same as regular Fedora Workstation. A new release comes every 6 months and is supported for 13 months. The team plans to release updates for the OS bi-weekly (or longer) instead of daily as they currently do. That way the updates can be more thoroughly tested by QA and community volunteers before they are sent to the rest of the users.

What is the future of the immutable OS?

From our point of view the future of the desktop involves the immutable OS. It’s safest for the user, and Android, ChromeOS, and the last macOS Catalina all use this method under the hood. For the Linux desktop there are still problems with some third party software that expects to write to the OS. HP printer drivers are a good example.

Another issue is how parts of the system are distributed and installed. Fonts are a good example. Currently in Fedora they’re distributed in RPM packages. If you want to use them, you have to overlay them and then restart to the newly created image that contains them.

What is the future of standard Workstation?

There is a possibility that the Silverblue will replace the regular Workstation. But there’s still a long way to go for Silverblue to provide the same functionality and user experience as the Workstation. In the meantime both desktop offerings will be delivered at the same time.

How does Atomic Workstation or Fedora CoreOS relate to any of this?

Atomic Workstation was the name of the project before it was renamed to Fedora Silverblue.

Fedora CoreOS is a different, but similar project. It shares some fundamental technologies with Silverblue, such as rpm-ostree, toolbox and others. Nevertheless, CoreOS is a more minimal, container-focused and automatically updating OS.

Firefox 68 available now in Fedora

Earlier this week, Mozilla released version 68 of the Firefox web browser. Firefox is the default web browser in Fedora, and this update is now available in the official Fedora repositories.

This Firefox release provides a range of bug fixes and enhancements, including:

  • Better handling when using dark GTK themes (like Adwaita Dark). Previously, running a dark theme may have caused issues where user interface elements on a rendered webpage (like forms) are rendered in the dark theme, on a white background. Firefox 68 resolves these issues. Refer to these two Mozilla bugzilla tickets for more information.
  • The about:addons special page has two new features to keep you safer when installing extensions and themes in Firefox. First is the ability to report security and stability issues with addons directly in the about:addons page. Additionally, about:addons now has a list of secure and stable extensions and themes that have been vetted by the Recommended Extensions program.

Updating Firefox in Fedora

Firefox 68 has already been pushed to the stable Fedora repositories. The security fix will be applied to your system with your next update. You can also update the firefox package only by running the following command:

$ sudo dnf update --refresh firefox

This command requires you to have sudo setup on your system. Additionally, note that not every Fedora mirrors syncs at the same rate. Community sites graciously donate space and bandwidth these mirrors to carry Fedora content. You may need to try again later if your selected mirror is still awaiting the latest update.

Fedora job opening: Fedora Community Action and Impact Coordinator (FCAIC)

I’ve decided to move on from my role as the Fedora Community Action and Impact Coordinator (FCAIC).  This was not an easy decision to make. I am proud of the work I have done in Fedora over the last three years and I think I have helped the community move past many challenges.  I could NEVER have done all of this without the support and assistance of the community!

As some of you know, I have been covering for some other roles in Red Hat for almost the last year.  Some of these tasks have led to some opportunities to take my career in a different direction. I am going to remain at Red Hat and on the same team with the same manager, but with a slightly expanded scope of duties.  I will no longer be day-to-day on Fedora and will instead be in a consultative role as a Community Architect at Large. This is a fancy way of saying that I will be tackling helping lots of projects with various issues while also working on some specific strategic objectives.

I think this is a great opportunity for the Fedora community.  The Fedora I became FCAIC in three years ago is a very different place from the Fedora of today.  While I could easily continue to help shape and grow this community, I think that I can do more by letting some new ideas come in.  The new person will hopefully be able to approach challenges differently. I’ll also be here to offer my advice and feedback as others who have moved on in the past have done.  Additionally, I will work with Matthew Miller and Red Hat to help hire and onboard the new Fedora Community and Impact Coordinator. During this time I will continue as FCAIC.

This means that we are looking for a new FCAIC. Love Fedora? Want to work with Fedora full-time to help support and grow the Fedora community? This is the core of what the FCAIC does. The job description (also below), has a list of some of the primary job responsibilities and required skills – but that’s just a sample of the duties required, and the day to day life working full-time with the Fedora community.

Day to day work includes working with Mindshare, managing the Fedora Budget, and being part of many other teams, including the Fedora Council.  You should be ready to write frequently about Fedora’s achievements, policies and decisions, and to draft and generate ideas and strategies. And, of course, planning Flock and Fedora’s presence at other events. It’s hard work, but also a great deal of fun.

Are you good at setting long-term priorities and hacking away at problems with the big picture in mind? Do you enjoy working with people all around the world, with a variety of skills and interests, to build not just a successful Linux distribution, but a healthy project? Can you set priorities, follow through, and know when to say “no” in order to focus on the most important tasks for success? Is Fedora’s mission deeply important to you?

If you said “yes” to those questions, you might be a great candidate for the FCAIC role. If you think you’re a great fit apply online, or contact Matthew Miller, Brian Exelbierd, or Stormy Peters.


Fedora Community Manager

Location: CZ-Remote – prefer Europe but can be North America

Company Description

At Red Hat, we connect an innovative community of customers, partners, and contributors to deliver an open source stack of trusted, high-performing solutions. We offer cloud, Linux, middleware, storage, and virtualization technologies, together with award-winning global customer support, consulting, and implementation services. Red Hat is a rapidly growing company supporting more than 90% of Fortune 500 companies.

Job summary

Red Hat’s Open Source Programs Office (OSPO) team is looking for the next Fedora Community Action and Impact Lead. In this role, you will join the Fedora Council and guide initiatives to grow the Fedora user and developer communities, as well as make Red Hat and Fedora interactions even more transparent and positive. The Council is responsible for stewardship of the Fedora Project as a whole, and supports the health and growth of the Fedora community.

As a the Fedora Community Action and Impact Lead, you’ll facilitate decision making on how to best focus the Fedora community budget to meet our collective objectives, work with other council members to identify the short, medium, and long-term goals of the Fedora community, and organize and enable the project.

You will also help make decisions about trademark use, project structure, community disputes or complaints, and other issues. You’ll hold a full council membership, not an auxiliary or advisory role.

Primary job responsibilities

  • Identify opportunities to engage new contributors and community members; align project around supporting those opportunities.
  • Improve on-boarding materials and processes for new contributors.
  • Participate in user and developer discussions and identify barriers to success for contributors and users.
  • Use metrics to evaluate the success of open source initiatives.
  • Regularly report on community metrics and developments, both internally and externally.  
  • Represent Red Hat’s stake in the Fedora community’s success.
  • Work with internal stakeholders to understand their goals and develop strategies for working effectively with the community.
  • Improve onboarding materials and presentation of Fedora to new hires; develop standardized materials on Fedora that can be used globally at Red Hat.
  • Work with the Fedora Council to determine the annual Fedora budget.
  • Assist in planning and organizing Fedora’s flagship events each year.
  • Create and carry out community promotion strategies; create media content like blog posts, podcasts, and videos and facilitate the creation of media by other members of the community

Required skills

  • Extensive experience with the Fedora Project or a comparable open source community.
  • Exceptional writing and speaking skills
  • Experience with software development and open source developer communities; understanding of development processes.
  • Outstanding organizational skills; ability to prioritize tasks matching short and long-term goals and focus on the tasks of high priority
  • Ability to manage a project budget.
  • Ability to lead teams and participate in multiple cross-organizational teams that span the globe.
  • Experience motivating volunteers and staff across departments and companies

Red Hat is proud to be an equal opportunity workplace and an affirmative action employer. We review applications for employment without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, ancestry, citizenship, age, veteran status, genetic information, physical or mental disability, medical condition, marital status, or any other basis prohibited by law.

Red Hat does not seek or accept unsolicited resumes or CVs from recruitment agencies. We are not responsible for, and will not pay, any fees, commissions, or any other payment related to unsolicited resumes or CVs except as required in a written contract between Red Hat and the recruitment agency or party requesting payment of a fee.


Photo by Deva Williamson on Unsplash.

Red Hat, IBM, and Fedora

Today marks a new day in the 26-year history of Red Hat. IBM has finalized its acquisition of Red Hat, which will operate as a distinct unit within IBM.

What does this mean for Red Hat’s participation in the Fedora Project?

In short, nothing.

Red Hat will continue to be a champion for open source, just as it always has, and valued projects like Fedora that will continue to play a role in driving innovation in open source technology. IBM is committed to Red Hat’s independence and role in open source software communities. We will continue this work and, as always, we will continue to help upstream projects be successful and contribute to welcoming new members and maintaining the project.

In Fedora, our mission, governance, and objectives remain the same. Red Hat associates will continue to contribute to the upstream in the same ways they have been.

We will do this together, with the community, as we always have.

If you have questions or would like to learn more about today’s news, I encourage you to review the materials below. For any questions not answered here, please feel free to contact us. Red Hat CTO Chris Wright will host an online Q&A session in the coming days where you can ask questions you may have about what the acquisition means for Red Hat and our involvement in open source communities. Details will be announced on the Red Hat blog.

Regards,

Matthew Miller, Fedora Project Leader
Brian Exelbierd, Fedora Community Action and Impact Coordinator

Command line quick tips: Permissions

Fedora, like all Linux based systems, comes with a powerful set of security features. One of the basic features is permissions on files and folders. These permissions allow files and folders to be secured from unauthorized access. This article explains a bit about these permissions, and shows you how to share access to a folder using them.

Permission basics

Fedora is by nature a multi-user operating system. It also has groups, which users can be members of. But imagine for a moment a multi-user system with no concept of permissions. Different logged in users could read each other’s content at will. This isn’t very good for privacy or security, as you can imagine.

Any file or folder on Fedora has three sets of permissions assigned. The first set is for the user who owns the file or folder. The second is for the group that owns it. The third set is for everyone else who’s not the user who owns the file, or in the group that owns the file. Sometimes this is called the world.

What permissions mean

Each set of permissions comes in three flavors — read, write, and execute. Each of these has an initial that stands for the permission, thus r, w, and x.

File permissions

For files, here’s what these permissions mean:

  • Read (r): the file content can be read
  • Write (w): the file content can be changed
  • Execute (x): the file can be executed — this is used primarily for programs or scripts that are meant to be run directly

You can see the three sets of these permissions when you do a long listing of any file. Try this with the /etc/services file on your system:

$ ls -l /etc/services
-rw-r--r--. 1 root root 692241 Apr 9 03:47 /etc/services

Notice the groups of permissions at the left side of the listing. These are provided in three sets, as mentioned above — for the user who owns the file, for the group that owns the file, and for everyone else. The user owner is root and the group owner is the root group. The user owner has read and write access to the file. Anyone in the group root can only read the file. And finally, anyone else can also only read the file. (The dash at the far left shows this is a regular file.)

By the way, you’ll commonly find this set of permissions on many (but not all) system configuration files. They are only meant to be changed by the system administrator, not regular users. Often regular users need to read the content as well.

Folder (directory) permissions

For folders, the permissions have slightly different meaning:

  • Read (r): the folder contents can be read (such as the ls command)
  • Write (w): the folder contents can be changed (files can be created or erased in this folder)
  • Execute (x): the folder can be searched, although its contents cannot be read. (This may sound strange, but the explanation requires more complex details of file systems outside the scope of this article. So just roll with it for now.)

Take a look at the /etc/grub.d folder for example:

$ ls -ld /etc/grub.d
drwx------. 2 root root 4096 May 23 16:28 /etc/grub.d

Note the d at the far left. It shows this is a directory, or folder. The permissions show the user owner (root) can read, change, and cd into this folder. However, no one else can do so — whether they’re a member of the root group or not. Notice you can’t cd into the folder, either:

$ cd /etc/grub.d
bash: cd: /etc/grub.d: Permission denied

Notice how your own home directory is setup:

$ ls -ld $HOME
drwx------. 221 paul paul 28672 Jul 3 14:03 /home/paul

Now, notice how no one, other than you as the owner, can access anything in this folder. This is intentional! You wouldn’t want others to be able to read your private content on a shared system.

Making a shared folder

You can exploit this permissions capability to easily make a folder to share within a group. Imagine you have a group called finance with several members who need to share documents. Because these are user documents, it’s a good idea to store them within the /home folder hierarchy.

To get started, use sudo to make a folder for sharing, and set it to be owned by the finance group:

$ sudo mkdir -p /home/shared/finance
$ sudo chgrp finance /home/shared/finance

By default the new folder has these permissions. Notice how it can be read or searched by anyone, even if they can’t create or erase files in it:

drwxr-xr-x. 2 root root 4096 Jul 6 15:35 finance

That doesn’t seem like a good idea for financial data. Next, use the chmod command to change the mode (permissions) of the shared folder. Note the use of g to change the owning group’s permissions, and o to change other users’ permissions. Similarly, u would change the user owner’s permissions:

$ sudo chmod g+w,o-rx /home/shared/finance

The resulting permissions look better. Now, anyone in the finance group (or the user owner root) have total access to the folder and its contents:

drwxrwx---. 2 root root 4096 Jul 6 15:35 finance

If any other user tries to access the shared folder, they won’t be able to do so. Great! Now our finance group can put documents in a shared place.

Other notes

There are additional ways to manipulate these permissions. For example, you may want any files in this folder to be set as owned by the group finance. This requires additional settings not covered in this article, but stay tuned to the Magazine for more on that topic soon.

Manage your shell environment

Some time ago, the Fedora Magazine has published an article introducing ZSH — an alternative shell to Fedora’s default, bash. This time, we’re going to look into customizing it to use it in a more effective way. All of the concepts shown in this article also work in other shells such as bash.

Alias

Aliases are shortcuts for commands. This is useful for creating short commands for actions that are performed often, but require a long command that would take too much time to type. The syntax is:

$ alias yourAlias='complex command with arguments'

They don’t always need to be used for shortening long commands. Important is that you use them for tasks that you do often. An example could be:

$ alias dnfUpgrade='dnf -y upgrade'

That way, to do a system upgrade, I just type dnfUpgrade instead of the whole dnf command.

The problem of setting aliases right in the console is that once the terminal session is closed, the alias would be lost. To set them permanently, resource files are used.

Resource Files

Resource files (or rc files) are configuration files that are loaded per user in the beginning of a session or a process (when a new terminal window is opened, or a new program like vim is started). In the case of ZSH, the resource file is .zshrc, and for bash it’s .bashrc.

To make the aliases permanent, you can either put them in your resource. You can edit your resource file with a text editor of your choice. This example uses vim:

$ vim $HOME/.zshrc

Or for bash:

$ vim $HOME/.bashrc

Note that the location of the resource file is specified relatively to a home directory — and that’s where ZSH (or bash) are going to look for the file by default for each user.

Other option is to put your configuration in any other file, and then source it:

$ source /path/to/your/rc/file

Again, sourcing it right in your session will only apply it to the session, so to make it permanent, add the source command to your resource file. The advantage of having your source file in a different location is that you can source it any time. Or anywhere which is especially useful in shared environments.

Environment Variables

Environment variables are values assigned to a specific name which can be then called in scripts and commands. They start with the $ dollar sign. One of the most common is $HOME that references the home directory.

As the name suggests, environment variables are a part of your environment. Set a variable using the following syntax:

$ http_proxy="http://your.proxy"

And to make it an environment variable, export it with the following command:

$ export $http_proxy

To see all the environment variables that are currently set, use the env command:

$ env

The command outputs all the variables available in your session. To demonstrate how to use them in a command, try running the following echo commands:

$ echo $PWD
/home/fedora
$ echo $USER
fedora

What happens here is variable expansion — the value stored in the variable is used in your command.

Another useful variable is $PATH, that defines directories that your shell uses to look for binaries.

The $PATH variable

There are many directories, or folders (the way they are called in graphical environments) that are important to the OS. Some directories are set to hold binaries you can use directly in your shell. And these directories are defined in the $PATH variable.

$ echo $PATH
/usr/lib64/qt-3.3/bin:/usr/share/Modules/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/usr/libexec/sdcc:/usr/libexec/sdcc:/usr/bin:/bin:/sbin:/usr/sbin:/opt/FortiClient

This will help you when you want to have your own binaries (or scripts) accessible in the shell.

post

Jupyter and data science in Fedora

In the past, kings and leaders used oracles and magicians to help them predict the future — or at least get some good advice due to their supposed power to perceive hidden information. Nowadays, we live in a society obsessed with quantifying everything. So we have data scientists to do this job.

Data scientists use statistical models, numerical techniques and advanced algorithms that didn’t come from statistical disciplines, along with the data that exist on databases, to find, to infer, to predict data that doesn’t exist yet. Sometimes this data is about the future. That is why we do a lot of predictive analytics and prescriptive analytics.

Here are some questions to which data scientists help find answers:

  1. Who are the students with high propensity to abandon the class? For each one, what are the reasons for leaving?
  2. Which house has a price above or below the fair price? What is the fair price for a certain house?
  3. What are the hidden groups that my clients classify themselves?
  4. Which future problems this premature child will develop?
  5. How many calls will I get in my call center tomorrow 11:43 AM?
  6. My bank should or should not lend money to this customer?

Note how the answer to all these question is not sitting in any database waiting to be queried. These are all data that still doesn’t exist and has to be calculated. That is part of the job we data scientists do.

Throughout this article you’ll learn how to prepare a Fedora system as a Data Scientist’s development environment and also a production system. Most of the basic software is RPM-packaged, but the most advanced parts can only be installed, nowadays, with Python’s pip tool.

Jupyter — the IDE

Most modern data scientists use Python. And an important part of their work is EDA (exploratory data analysis). EDA is a manual and interactive process that retrieves data, explores its features, searches for correlations, and uses plotted graphics to visualize and understand how data is shaped and prototypes predictive models.

Jupyter is a web application perfect for this task. Jupyter works with Notebooks, documents that mix rich text including beautifully rendered math formulas (thanks to mathjax), blocks of code and code output, including graphics.

Notebook files have extension .ipynb, which means Interactive Python Notebook.

Setting up and running Jupyter

First, install essential packages for Jupyter (using sudo):

$ sudo dnf install python3-notebook mathjax sscg

You might want to install additional and optional Python modules commonly used by data scientists:

$ sudo dnf install python3-seaborn python3-lxml python3-basemap python3-scikit-image python3-scikit-learn python3-sympy python3-dask+dataframe python3-nltk

Set a password to log into Notebook web interface and avoid those long tokens. Run the following command anywhere on your terminal:

$ mkdir -p $HOME/.jupyter
$ jupyter notebook password

Now, type a password for yourself. This will create the file $HOME/.jupyter/jupyter_notebook_config.json with your encrypted password.

Next, prepare for SSLby generating a self-signed HTTPS certificate for Jupyter’s web server:

$ cd $HOME/.jupyter; sscg

Finish configuring Jupyter by editing your $HOME/.jupyter/jupyter_notebook_config.json file. Make it look like this:

{
"NotebookApp": {
"password": "sha1:abf58...87b",
"ip": "*",
"allow_origin": "*",
"allow_remote_access": true,
"open_browser": false,
"websocket_compression_options": {},
"certfile": "/home/aviram/.jupyter/service.pem",
"keyfile": "/home/aviram/.jupyter/service-key.pem",
"notebook_dir": "/home/aviram/Notebooks"
}
}

The parts in red must be changed to match your folders. Parts in blue were already there after you created your password. Parts in green are the crypto-related files generated by sscg.

Create a folder for your notebook files, as configured in the notebook_dir setting above:

$ mkdir $HOME/Notebooks

Now you are all set. Just run Jupyter Notebook from anywhere on your system by typing:

$ jupyter notebook

Or add this line to your $HOME/.bashrc file to create a shortcut command called jn:

alias jn='jupyter notebook'

After running the command jn, access https://your-fedora-host.com:8888 from any browser on the network to see the Jupyter user interface. You’ll need to use the password you set up earlier. Start typing some Python code and markup text. This is how it looks:

Jupyter with a simple notebook

In addition to the IPython environment, you’ll also get a web-based Unix terminal provided by terminado. Some people might find this useful, while others find this insecure. You can disable this feature in the config file.

JupyterLab — the next generation of Jupyter

JupyterLab is the next generation of Jupyter, with a better interface and more control over your workspace. It’s currently not RPM-packaged for Fedora at the time of writing, but you can use pip to get it installed easily:

$ pip3 install jupyterlab --user
$ jupyter serverextension enable --py jupyterlab

Then run your regular jupiter notebook command or jn alias. JupyterLab will be accessible from http://your-linux-host.com:8888/lab.

Tools used by data scientists

In this section you can get to know some of these tools, and how to install them. Unless noted otherwise, the module is already packaged for Fedora and was installed as prerequisites for previous components.

Numpy

Numpy is an advanced and C-optimized math library designed to work with large in-memory datasets. It provides advanced multidimensional matrix support and operations, including math functions as log(), exp(), trigonometry etc.

Pandas

In this author’s opinion, Python is THE platform for data science mostly because of Pandas. Built on top of numpy, Pandas makes easy the work of preparing and displaying data. You can think of it as a no-UI spreadsheet, but ready to work with much larger datasets. Pandas helps with data retrieval from a SQL database, CSV or other types of files, columns and rows manipulation, data filtering and, to some extent, data visualization with matplotlib.

Matplotlib

Matplotlib is a library to plot 2D and 3D data. It has great support for notations in graphics, labels and overlays

matplotlib pair of graphics showing a cost function searching its optimal value through a gradient descent algorithm

Seaborn

Built on top of matplotlib, Seaborn’s graphics are optimized for a more statistical comprehension of data. It automatically displays regression lines or Gauss curve approximations of plotted data.

Linear regression visualised with SeaBorn

StatsModels

StatsModels provides algorithms for statistical and econometrics data analysis such as linear and logistic regressions. Statsmodel is also home for the classical family of time series algorithms known as ARIMA.

Normalized number of passengers across time (blue) and ARIMA-predicted number of passengers (red)

Scikit-learn

The central piece of the machine-learning ecosystem, scikit provides predictor algorithms for regression (Elasticnet, Gradient Boosting, Random Forest etc) and classification and clustering (K-means, DBSCAN etc). It features a very well designed API. Scikit also has classes for advanced data manipulation, dataset split into train and test parts, dimensionality reduction and data pipeline preparation.

XGBoost

XGBoost is the most advanced regressor and classifier used nowadays. It’s not part of scikit-learn, but it adheres to scikit’s API. XGBoost is not packaged for Fedora and should be installed with pip. XGBoost can be accelerated with your nVidia GPU, but not through its pip package. You can get this if you compile it yourself against CUDA. Get it with:

$ pip3 install xgboost --user

Imbalanced Learn

imbalanced-learn provides ways for under-sampling and over-sampling data. It is useful in fraud detection scenarios where known fraud data is very small when compared to non-fraud data. In these cases data augmentation is needed for the known fraud data, to make it more relevant to train predictors. Install it with pip:

$ pip3 install imblearn --user

NLTK

The Natural Language toolkit, or NLTK, helps you work with human language data for the purpose of building chatbots (just to cite an example).

SHAP

Machine learning algorithms are very good on predicting, but aren’t good at explaining why they made a prediction. SHAP solves that, by analyzing trained models.

Where SHAP fits into the data analysis process

Install it with pip:

$ pip3 install shap --user

Keras

Keras is a library for deep learning and neural networks. Install it with pip:

$ sudo dnf instal python3-h5py
$ pip3 install keras --user

TensorFlow

TensorFlow is a popular neural networks builder. Install it with pip:

$ pip3 install tensorflow --user

Photo courtesy of FolsomNatural on Flickr (CC BY-SA 2.0).

Upcoming features in Fedora 31 Workstation

The Fedora Workstation edition is a fabulous operating system that includes everything a developer needs. But it’s also a perfect solution for anyone who wants to be productive online with their desktop or laptop computer. It features a sleek interface and an enormous catalog of ready-to-install software. Recently, Christian Schaller shared information about what’s coming in the Workstation for Fedora 31.

Fedora 31 is currently scheduled for release in late October 2019. With it, as usual, will come an assortment of new and refreshed free and open source software. This includes the GNOME desktop which is planned to be updated to the latest 3.34.

Under the hood of the desktop, many intrepid open source developers have been toiling away. They’ve been working on things like:

  • The Wayland desktop compositor
  • Working with NVidia to provide better driver support
  • PipeWire, for better audio and video handling
  • Expanded Flatpak support and features
  • A container toolbox
  • …and much more!

Long-time and keen readers of the Magazine probably know that Christian is deeply involved in the Workstation effort. He heads up the desktop engineering groups at Red Hat. But he’s also involved heavily in the community Workstation Working Group, which guides these efforts as well. As an experienced developer himself, he brings his expertise to the open source community every day to build a better desktop.

For all the details, check out Christian’s detailed and informative blog post on Fedora 31 Workstation. And stay tuned to the Magazine for more about the upcoming release in the next few months!