Create an account


Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Demystifying Kubernetes Operators with the Operator SDK: Part 2

#1
Demystifying Kubernetes Operators with the Operator SDK: Part 2

<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/demystifying-kubernetes-operators-with-the-operator-sdk-part-2.png" width="256" height="256" title="" alt="" /></div><div><p><span><span>In the <a href="https://www.linux.com/blog/2018/12/demystifying-kubernetes-operators-operator-sdk-part-1">previous article</a>, we started building the foundation for</span></span> building a custom operator that can be applied to real-world use cases. <span><span> In this part of our tutorial series, we are going to create a generic </span><strong><span>example-operator</span></strong><span> that manages our apps of </span><strong><span>Examplekind</span></strong><span>. We have already used the operator-sdk to build it out and implement the custom code in a repo </span><a href="https://github.com/kenzanlabs/operator"><span>here</span></a><span>. For the tutorial, we will rebuild what is in this repo. </span></span></p>
<p><span><span>The example-operator will manage our Examplekind apps with the following behavior:  </span></span></p>
<ul>
<li>
<p><span><span>Create an Examplekind deployment if it doesn’t exist using an </span><a href="https://github.com/kenzanlabs/operator/blob/master/deploy/crds/example_v1alpha1_examplekind_cr.yaml"><span>Examplekind CR spec</span></a><span> (for this example, we will use an </span><strong><span>nginx</span></strong><span> </span><em><span>image</span></em><span> running on </span><em><span>port</span></em><span> 80). </span></span></p>
</li>
<li>
<p><span><span>Ensure that the pod </span><span>count</span><span> is the same as specified in the </span><a href="https://github.com/kenzanlabs/operator/blob/master/deploy/crds/example_v1alpha1_examplekind_cr.yaml"><span>Examplekind CR spec</span></a><span>. </span></span></p>
</li>
<li>
<p><span><span>Update the Examplekind CR status with: </span></span></p>
</li>
</ul>
<h3><span><span>Prerequisites</span></span></h3>
<p><span><span>You’ll want to have the following prerequisites installed or set up before running through the tutorial. These are prerequisites to install operator-sdk, as well as a a few extras you’ll need. </span></span></p>
<h3><span><span>Initialize your Environment</span></span></h3>
<p><span><span>​1. </span></span><span><span>Make sure you’ve got your Kubernetes cluster running by spinning up minikube. </span></span><span> <span>minikube start</span></span></p>
<p><span><span>2. Create a new folder for your example operator within your Go path.</span></span></p>
<pre>
mkdir -p $GOPATH/src/github.com/linux-blog-demo</pre>
<pre>
cd $GOPATH/src/github.com/linux-blog-demo</pre>
<p><span><span>3. Initialize a new example-operator project within the folder you created using the operator-sdk. </span></span></p>
<pre>
operator-sdk new example-operator</pre>
<pre>
cd example-operator
</pre>
<div>
<table>
<tbody>
<tr>
<td><span><img alt="JMgJ3Q24igREHRCmKiX0UAhZVXH-arLxyyXWfKAm" height="20" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/demystifying-kubernetes-operators-with-the-operator-sdk-part-2.png" width="20" /></span></td>
<td>
<p><strong><span><span>What just got created?</span></span></strong></p>
<p><span><span>By running the operator-sdk </span><span>new</span><span> command, we scaffolded out a number of files and directories for our defined project. See the </span><a href="https://github.com/operator-framework/operator-sdk/blob/master/doc/project_layout.md"><span>project layout</span></a><span> for a complete description; for now, here are some important directories to note: </span></span></p>
<ul>
<li><strong>pkg/apis</strong><span> – contains the APIs for our CR. Right now this is relatively empty; the commands that follow will create our specific API and CR for Examplekind. </span></li>
<li><strong>pkg/controller</strong><span> – contains the Controller implementations for our Operator, and specifically the custom code for how we reconcile our CR (currently this is somewhat empty as well). </span></li>
<li><strong>deploy/</strong><span> – contains generated K8s yaml deployments for our operator and its RBAC objects. The folder will also contain deployments for our CR and CRD, once they are generated in the steps that follow.  </span></li>
</ul>
</td>
</tr>
</tbody>
</table>
</div>
<h3><span><span>Create a Custom Resource and Modify it</span></span></h3>
<p><span><span>4. Create the Custom Resource and it’s API using the operator-sdk. </span></span></p>
<pre>
operator-sdk add api --api-version=example.kenzan.com/v1alpha1 --kind=Examplekind
</pre>
<div>
<table>
<tbody>
<tr>
<td><span><img alt="JMgJ3Q24igREHRCmKiX0UAhZVXH-arLxyyXWfKAm" height="20" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/demystifying-kubernetes-operators-with-the-operator-sdk-part-2.png" width="20" /></span></td>
<td>
<p><strong><span><span>What just got created?</span><span> </span></span></strong></p>
<p><span><span>Under </span><strong><span>pkg/ais/example/v1alpha</span></strong><span>, a new generic API was created for Examplekind in the file </span><strong><span>examplekind_types.go</span></strong><span>. </span></span></p>
<p><span>Under </span><strong><span>deploy/crds</span></strong><span>, two new K8s yamls were generated: </span></p>
<ul>
<li><strong>examplekind_crd.yaml</strong><span> – a new </span><a href="https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/"><span>CustomResourceDefinition</span></a><span> defining our Examplekind object so Kubernetes knows about it. </span></li>
<li><strong>examplekind_cr.yaml</strong><span><strong> </strong>– a general manifest for deploying apps of type Examplekind</span></li>
</ul>
<p><span>A DeepCopy methods library is generated for copying the Examplekind object</span></p>
</td>
</tr>
</tbody>
</table>
</div>
<p>5. We need to modify the API in <em>pkg/apis/example/v1alpha1/examplekind_types.go</em> with some custom fields for our CR. Open this file in a text editor. Add the following custom variables to <em>ExamplekindSpec</em> and <em>ExamplekindStatus</em> structs.</p>
<div>
<table>
<tbody>
<tr>
<td><span><img alt="JMgJ3Q24igREHRCmKiX0UAhZVXH-arLxyyXWfKAm" height="20" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/demystifying-kubernetes-operators-with-the-operator-sdk-part-2.png" width="20" /></span></td>
<td>
<p><span><span>The variables in these structs are used to generate the data structures in the yaml </span><span>spec</span><span> for the Custom Resource, as well as variables we can later display in getting the </span><span>status</span><span> of the Custom Resource.  </span></span></p>
</td>
</tr>
</tbody>
</table>
</div>
<p><span><span>6. After modifying the </span><em><span>examplekind_types.go</span></em><span>, regenerate the code. </span></span></p>
<pre>
operator-sdk generate k8s
</pre>
<div>
<table>
<tbody>
<tr>
<td><span><img alt="JMgJ3Q24igREHRCmKiX0UAhZVXH-arLxyyXWfKAm" height="20" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/demystifying-kubernetes-operators-with-the-operator-sdk-part-2.png" width="20" /></span></td>
<td>
<p><strong><span><span>What just got created?</span></span></strong><strong><span><span> </span></span></strong></p>
<p><span><span>You always want to run the operator-sdk </span><em><span>generate</span></em><span> command after modifying the API in the <em>_</em></span><em><span>types.go</span></em><span> file. This will regenerate the DeepCopy methods. </span></span></p>
</td>
</tr>
</tbody>
</table>
</div>
<h3><span><span>Create a New Controller and Write Custom Code for it</span></span></h3>
<p><span><span>7. Now add a controller to your operator.  </span></span></p>
<pre>
operator-sdk add controller --api-version=example.kenzan.com/v1alpha1 --kind=Examplekind</pre>
<div>
<table>
<tbody>
<tr>
<td><span><img alt="JMgJ3Q24igREHRCmKiX0UAhZVXH-arLxyyXWfKAm" height="20" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/demystifying-kubernetes-operators-with-the-operator-sdk-part-2.png" width="20" /></span></td>
<td>
<p><strong><span><span>What just got created?</span><span> </span></span></strong><br class="kix-line-break" /><span><span>Among other code, a <em> </em></span><em><span>pkg/controller/examplekind/examplekind_controller.go</span></em><span> file was generated. This is the primary code running our controller; it contains a Reconcile loop where custom code can be implemented to reconcile the Custom Resource against its spec.  </span></span></p>
</td>
</tr>
</tbody>
</table>
</div>
<p><span><span>8. Replace the </span><em><span>examplekind_controller.go</span></em><span> file with the </span><a href="https://github.com/kenzanlabs/operator/blob/master/pkg/controller/examplekind/examplekind_controller.go"><span>one in our completed repo</span></a><span>. The new file contains the custom code that we’ve added to the generated skeleton. </span></span></p>
<h3><span><span>Wait, what was in the custom code we just added?  </span></span></h3>
<p><span><span>If you want to know what is happening in the code we just added, read on. If not, you can skip to the next section to continue the tutorial. </span></span></p>
<p><span><span>To break down what we are doing in our </span><em><span>examplekind_controller.go</span></em><span>, lets first go back to what we are trying to accomplish: </span></span></p>
<ol>
<li>
<p><span><span>Create an <em>Examplekind</em> deployment if it doesn’t exist </span></span></p>
</li>
<li>
<p><span><span>Make sure our count matches what we defined in our manifest </span></span></p>
</li>
<li>
<p><span><span><span><span>Update the status with our group and podnames. </span></span></span></span></p>
</li>
</ol>
<p><span><span>To achieve these things, we’ve created three methods: one to get pod names, one to create labels for us, and last to create a deployment.</span></span></p>
<p>In <em><span>getPodNames()</span></em><span>, we are using the </span><a href="https://godoc.org/k8s.io/api/core/v1#Pod"><span>core/v1 API</span></a><span> to get the names of pods and appending them to a slice.</span></p>
<p><span><span>In </span><em><span>labelsForExampleKind()</span></em><span>, we are creating a label to be used later in our deployment. The operator name will be passed into this as a name value.</span></span></p>
<p><span><span>In </span><em><span>newDeploymentForCR()</span></em><span>, we are creating a deployment using the </span><a href="https://godoc.org/k8s.io/api/apps/v1#Deployment"><span>apps/v1 API</span></a><span>. The label method is used here to pass in a label. It uses whatever image we specify in our manifest as you can see below in </span><em><span>Image: m.Spec.Image</span></em><span>. Replicas for this deployment will also use the<em> </em></span><em><span>count</span></em><span> field we specified in our manifest.</span></span></p>
<p><span><span>Then in our main </span><em><span>Reconcile()</span></em><span> method, we check to see if our deployment exists. If it does not, we create a new one using the </span><span>new<em>DeploymentForCR()</em></span><span><em> </em>method. If for whatever reason it cannot create a deployment, print an error to the logs.</span></span></p>
<p><span><span>In the same </span><em><span>Reconcile()</span></em><span> method, we are also making sure that the deployment </span><em><span>replica</span></em><span> field is set to our </span><em><span>count</span></em><span><em> </em>field in the spec of our manifest.</span></span></p>
<p><span><span>And we are getting a list of our pods that matches the label we created.</span></span></p>
<p><span><span>We are then passing the pod list into the </span><em><span>getPodNames()</span></em><span> method. We are making sure that the </span><em><span>podNames</span></em><span> field in our ExamplekindStatus ( in examplekind_types.go) is set to the podNames list. </span></span></p>
<p><span><span>Finally, we are making sure the </span><em><span>AppGroup</span></em><span> in our ExamplekindStatus (in examplekind_types.go) is set to the </span><em><span>Group</span></em><span> field in our Examplekind spec (also in examplekind_types.go). </span></span></p>
<h3><span><span>Deploy your Operator and Custom Resource</span></span></h3>
<p><span><span>We could run the </span><em><span>example-operator</span></em><span> as Go code locally outside the cluster, but here we are going to run it inside the cluster as its own </span><em><span>Deployment</span></em><span>, alongside the </span><em><span>Examplekind</span></em><span> apps it will watch and reconcile. </span></span></p>
<p><span><span>9. Kubernetes needs to know about your </span><span>Examplekind</span><span> Custom Resource Definition before creating instances, so go ahead and apply it to the cluster. </span></span></p>
<pre>
kubectl create -f deploy/crds/example_v1alpha1_examplekind_crd.yaml</pre>
<p><span><span>10. Check to see that the custom resource definition is deployed.</span></span></p>
<pre>
kubectl get crd</pre>
<p>11. <span>We will need to build the example-operator as an image and push it to a repository. For simplicity, we’ll create a public repository on your account on dockerhub.com.</span></p>
<p><span><span>  a. Go to </span><a href="https://hub.docker.com/"><span>https://hub.docker.com/</span></a><span> and login</span></span></p>
<p><span><span>  b. Click </span><strong><span>Create Repository</span></strong></span></p>
<p><span><span>  c. Leave the namespace as your username</span></span></p>
<p><span><span>  d. Enter the repository as “example-operator”</span></span></p>
<p><span><span>  e. Leave the visibility as Public. </span></span></p>
<p><span>  f. Click <strong>Create</strong></span></p>
<p><span>12. Build the example-operator.</span><span> </span></p>
<pre>
<span><span>operator-sdk build [Dockerhub username]/example-operator:v0.0.1</span></span></pre>
<p><span><span>13. Push the image to your repository on Dockerhub (this command may require logging in with your credentials). </span></span></p>
<pre>
docker push [Dockerhub username]/example-operator:v0.0.1</pre>
<p><span><span>14. Open up the </span><em><span>deploy/operator.yaml</span></em><span><em> </em>file that was generated during the build. This is a manifest that will run your example-operator as a Deployment in Kubernetes. We need to change the image so it is the same as the one we just pushed. </span></span></p>
<p><span><span>  a. </span></span><span><span>Find </span><em><span>image: REPLACE_IMAGE</span><span> </span></em></span></p>
<p><span><em><span><span><span> </span></span></span></em><span><span><span> b.<em><span><span><span> </span></span></span></em>Replace with </span></span></span></span><span><em><span>image: [Dockerhub username]/example-operator:v0.0.1</span></em></span></p>
<p><span><span>15. Set up Role-based Authentication for the example-operator by applying the RBAC manifests that were previously generated. </span></span></p>
<pre>
kubectl create -f deploy/service_account.yaml</pre>
<p><span>kubectl create -f deploy/role.yaml</span></p>
<p><span>kubectl create -f deploy/role_binding.yaml</span><span> </span></p>
<p><span><span>16. Deploy the example-operator. </span></span></p>
<pre>
kubectl create -f deploy/operator.yaml</pre>
<p><span><span>17. Check to see that the example-operator is up and running.</span></span></p>
<pre>
kubectl get deploy</pre>
<p><span><span>18. Now we’ll deploy several instances of the Examplekind app for our operator to watch. Open up the </span><em><span>deploy/crds/example_v1alpha1_examplekind_cr.yaml</span></em><span><em> </em>deployment manifest. Update fields so they appear as below, with </span><em><span>name</span><span>, </span><span>count</span><span>, </span><span>group</span><span>, </span><span>image</span></em><span> and </span><em><span>port</span></em><span>. Notice we are adding fields that we defined in the spec struct of our </span><em><span>pkg/apis/example/v1alpha1/examplekind_types.go</span></em><span>.</span></span></p>
<pre>
<span>apiVersion: "example.kenzan.com/v1alpha1"</span>
<span>kind: "Examplekind"</span>
<span>metadata:</span>
<strong><span> name: "kenzan-example"</span></strong>
<span>spec:</span>
<strong><span> count: 3</span>
<span> group: Demo-App</span>
<span> image: nginx</span>
<span> port: 80</span></strong></pre>
<p><span><span>19. Apply the Examplekind app deployment. </span></span></p>
<pre>
kubectl apply -f deploy/crds/example_v1alpha1_examplekind_cr.yaml</pre>
<p><span><span>20. Check that an instance of the Examplekind object exists in Kubernetes.  </span> </span></p>
<pre>
kubectl get Examplekind</pre>
<p><span><span>21. Let’s describe the Examplekind object to see if our status now shows as expected. </span></span></p>
<pre>
kubectl describe Examplekind kenzan-example</pre>
<p>Note that the Status describes the <em><span>AppGroup</span></em><span> the instances are a part of (“Demo-App”), as well as enumerates the </span><em><span>Podnames</span></em><span>.</span></p>
<p><span><span>22. Within the </span><em><span>deploy/crds/example_v1alpha1_examplekind_cr.yaml</span></em><span><em>,</em> change the </span><span>count</span><span> to be 1 pod. Apply the deployment again. </span></span></p>
<pre>
kubectl apply -f deploy/crds/example_v1alpha1_examplekind_cr.yaml</pre>
<p><span><span>23. Based on the operator reconciling against the spec, you should now have one instance of kenzan-example. </span></span></p>
<pre>
<span>kubectl describe Examplekind kenzan-example</span></pre>
<p><span><span><span>Well done. You’ve successfully created an example-operator, and become familiar with all the pieces and parts needed in the process. You may even have a few ideas in your head about which stateful applications you could potentially automate the management of for your organization, getting away from manual intervention. Take a look at the following links to build on your Operator knowledge:</span></span></span></p>
<p><em>Toye Idowu is a Platform Engineer at Kenzan Media.</em></p>
</div>
Reply



Forum Jump:


Users browsing this thread:
1 Guest(s)

Forum software by © MyBB Theme © iAndrew 2016