Create an account


Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
[Tut] Check for NaN Values in Python

#1
Check for NaN Values in Python

<div><h2><strong>Overview</strong></h2>
<p><strong>Problem</strong>: How to check if a given value is <code>NaN</code>?</p>
<p>Here’s a quick look at the solutions to follow:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import math
import numpy as np
import pandas as pd x = float('nan')
print(math.isnan(x))
print(x != x)
print(np.isnan(x))
print(pd.isna(x))
print(not(float('-inf') &lt; x &lt; float('inf')))</pre>
<p>So, what is a <code>NaN</code> value?</p>
<p><code>NaN</code> is a constant value that indicates that the given value is Not a Number. It’s a floating-point value, hence cannot be converted to any other type other than float. We should know that <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code> and Null are two different things in Python. The Null values indicate something which does not exist, i.e. is empty. But that is not the case with <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code>.</p>
<p>We have to deal with <code>NaN</code> values frequently in Python especially when we deal with array objects or DataFrames. So, without further delay, let us dive into our mission critical question and have a look at the different methods to solve our problem.</p>
<h2><strong>Method 1: Using math.isnan()</strong></h2>
<p class="has-global-color-8-background-color has-background">The simplest solution to check for NaN values in Python is to use the mathematical function <code>math.isnan()</code>.</p>
<p><code>math.isnan()</code> is a function of the math module in Python that checks for <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code> constants in float objects and returns True for every NaN value encountered and returns False otherwise. </p>
<p><strong>Example: </strong></p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Importing the math module
import math # Function to check for NaN values
def isNaN(a): # Using math.isnan() if math.isnan(a): print("NaN value encountered!") else: print("Type of Given Value: ", type(a)) # NaN value
x = float('NaN')
isNaN(x)
# Floating value
y = float("5.78")
isNaN(y)</pre>
<p><strong>Output:</strong></p>
<pre class="wp-block-code"><code>NaN value encountered!
Type of Given Value: &lt;class 'float'&gt;</code></pre>
<p>In the above example, since <code>x</code> represents a NaN value, hence, the <code>isNaN</code> method returns <code>True</code> but in case of <code>y</code> , <code>isNan</code> returns <code>False</code> and prints the type of the variable <code>y</code> as an output. </p>
<h2><strong>Method 2: Hack NaN Using != Operator</strong></h2>
<p class="has-global-color-8-background-color has-background">The most unique thing about <code>NaN</code> values is that they are constantly shapeshifting. This means we cannot compare the <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code> value even against itself. Hence, we can use the <code>!=</code> (not equal to) operator to check for the <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code> values. Thus, the idea is to check if the given variable is equal to itself. If we consider any object other than <code>NaN</code>, the expression <code>(x == x)</code> will always return <code data-enlighter-language="generic" class="EnlighterJSRAW">True</code>. If it’s not equal, then it is a <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code> value. </p>
<p><strong>Example 1:</strong></p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">print(5 == 5)
# True
print(['a', 'b'] == ['a', 'b'])
# True
print([] == [])
# True
print(float("nan") == float("nan"))
# False
print(float("nan") != float("nan"))
# True</pre>
<p><strong>Example 2:</strong></p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Function to check for NaN values
def not_a_number(x): # Using != operator if x != x: print("Not a Number!") else: print(f'Type of {x} is {type(x)}') # Floating value
x = float("7.8")
not_a_number(x)
# NaN value
y = float("NaN")
not_a_number(y)</pre>
<p><strong>Output:</strong></p>
<pre class="wp-block-code"><code>Type of 7.8 is &lt;class 'float'&gt;
Not a Number!</code></pre>
<h2><strong>Method 3: Using <a href="https://blog.finxter.com/numpy-tutorial/" target="_blank" rel="noreferrer noopener">numpy</a>.isnan()</strong></h2>
<p>We can also use the <code>NumPy</code> library to check whether the given value is <code>NaN</code> or not. We just need to ensure that we import the library at the start of the program and then use its <code>np.isnan(x)</code> method. </p>
<p class="has-global-color-8-background-color has-background">The <code>np.isnan(number)</code> function checks whether the element in a Numpy array is <code>NaN</code> or not. It then returns the result as a boolean array.</p>
<p><strong>Example: </strong>In the following example we have a Numpy Array and then we will check the type of each value. We will also check if it is a <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code> value or not.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import numpy as np arr = np.array([10, 20, np.nan, 40, np.nan])
for x in arr: if np.isnan(x): print("Not a Number!") else: print(x, ":", type(x))</pre>
<p><strong>Output:</strong></p>
<pre class="wp-block-code"><code>10.0 : &lt;class 'numpy.float64'&gt;
20.0 : &lt;class 'numpy.float64'&gt;
Not a Number!
40.0 : &lt;class 'numpy.float64'&gt;
Not a Number!</code></pre>
<p><strong><img src="https://s.w.org/images/core/emoji/13.1.0/72x72/1f4a1.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" />TRIVIA</strong></p>
<p>Let us try to perform some basic functions on an numpy array that involves <code>NaN</code> values and find out what happens to it.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import numpy as np arr = np.array([10, 20, np.nan, 40, np.nan])
print(arr.sum())
print(arr.max())</pre>
<p><strong>Output:</strong></p>
<pre class="wp-block-code"><code>nan
nan</code></pre>
<p>Now this can be a problem in many cases. So, do we have a way to eliminate the NaN values from our array object and then perform the mathematical operations upon the array elements? <strong>Yes! </strong>Numpy facilitates us with methods like <code>np.nansum()</code> and <code>np.nanmax()</code> that help us to calculate the sum and maximum values in the array by ignoring the presence of <code>NaN</code> values in the array.</p>
<p><strong>Example:</strong></p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import numpy as np arr = np.array([10, 20, np.nan, 40, np.nan])
print(np.nansum(arr))
print(np.nanmax(arr))</pre>
<p><strong>Output:</strong></p>
<pre class="wp-block-code"><code>70.0
40.0</code></pre>
<h2><strong><strong>Method 4: Using <a href="https://blog.finxter.com/pandas-quickstart/" target="_blank" rel="noreferrer noopener">pandas</a>.isna()</strong></strong></h2>
<p>Another way to solve our problem is to use the <code>isna()</code> method of the Pandas module. <code>pandas.isna()</code> is a function that detects missing values in an array-like object. It returns True if any <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code> value is encountered. </p>
<p><strong>Example 1:</strong></p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import pandas as pd x = float("nan")
y = 25.75
print(pd.isna(x))
print(pd.isna(y))</pre>
<p><strong>Output:</strong></p>
<pre class="wp-block-code"><code>True
False</code></pre>
<p><strong>Example 2: </strong>In the following example we will have a look at a Pandas DataFrame and detect the presence of NaN values in the DataFrame.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import pandas as pd df = pd.DataFrame([['Mercury', 'Venus', 'Earth'], ['1', float('nan'), '2']])
print(pd.isna(df))</pre>
<p><strong>Output:</strong></p>
<pre class="wp-block-code"><code> 0 1 2
0 False False False
1 False True False</code></pre>
<h2><strong>Method 5: By Checking The Range</strong></h2>
<p>We can check for the <code>NaN</code> values by using another NaN special property: limited range. The range of all the floating-point values falls within negative infinity to infinity. However, <code>NaN</code> values do not fall within this range. </p>
<p class="has-global-color-8-background-color has-background">Hence, the idea is to check whether a given value lies within the range of <code>-inf</code> and <code>inf</code>. If yes , then it is not a <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN </code>value else it is a <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN </code>value. </p>
<p><strong>Example:</strong></p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">li = [25.87, float('nan')]
for i in li: if float('-inf') &lt; float(i) &lt; float('inf'): print(i) else: print("Not a Number!")</pre>
<p><strong>Output:</strong></p>
<pre class="wp-block-code"><code>25.87
Not a Number!</code></pre>
<p class="has-background" style="background-color:#fdfeb7"><strong>Recommended read: <a href="https://blog.finxter.com/python-infinity/" target="_blank" rel="noreferrer noopener">Python Infinity</a></strong></p>
<h2><strong>Conclusion</strong></h2>
<p>In this article, we learned how we can use the various methods and modules (<code>pandas</code>, <code>NumPy</code>, and <code>math</code>) in Python to check for the <code>NaN</code> values. I hope this article was able to answer your queries. Please <strong><a rel="noreferrer noopener" href="https://blog.finxter.com" target="_blank">stay tuned</a></strong> and <strong><a rel="noreferrer noopener" href="https://blog.finxter.com/subscribe/" target="_blank">subscribe</a></strong> for more such articles. </p>
<p class="has-text-align-center has-contrast-3-background-color has-background has-small-font-size"><strong>Authors: SHUBHAM SAYON and RASHI AGARWAL</strong></p>
<hr class="wp-block-separator" />
<p><strong>Do you want to become a NumPy master?</strong> Check out our interactive puzzle book <a href="https://amzn.to/39dEykm" target="_blank" rel="noreferrer noopener" title="https://amzn.to/39dEykm"><strong>Coffee Break NumPy</strong></a> and boost your data science skills! <em>(Amazon link opens in new tab.)</em></p>
<div class="wp-block-image">
<figure class="aligncenter size-medium"><a href="https://amzn.to/39dEykm" target="_blank" rel="noopener noreferrer"><img loading="lazy" width="200" height="300" src="https://blog.finxter.com/wp-content/uploads/2019/04/Cover_Coffee_Break_NumPy-200x300.jpg" alt="Coffee Break NumPy" class="wp-image-2766"/></a></figure>
</div>
</div>


https://www.sickgaming.net/blog/2022/03/...in-python/
Reply



Forum Jump:


Users browsing this thread:
1 Guest(s)

Forum software by © MyBB Theme © iAndrew 2016