09-12-2020, 02:00 PM
What Is Python Output Buffering and How to Disable It?
<div><p><strong>Summary:</strong> <em>Python output buffering</em> is the process of storing the output of your code in buffer memory. Once the buffer is full, the output gets displayed on the standard output screen. Buffering is enabled by default and you can use one of the following methods to disable it: </p>
<ul>
<li>Execute the Python code with the <code>-u</code> command line switch, for example <code>python -u code.py</code></li>
<li>Use the <code>flush</code> <a href="https://blog.finxter.com/python-cheat-sheet/" target="_blank" rel="noreferrer noopener">keyword</a></li>
<li>Use <code>sys.stdout.flush()</code></li>
<li>Wrap <code>sys.stdout</code> in <code>TextIOWrapper</code> and set the buffered size to 0</li>
<li>Set <code>PYTHONUNBUFFERED</code> to a non-empty string</li>
</ul>
<p><strong>Problem: </strong>Given a Python program; how to disable output buffering?</p>
<h2>Overview Of Python Output Buffering</h2>
<figure class="wp-block-image size-large is-resized"><img loading="lazy" src="https://blog.finxter.com/wp-content/uploads/2020/09/buffer.gif" alt="" class="wp-image-12938" width="652" height="337"/></figure>
<p>Before we <a href="https://finxter-coding-university.teachable.com/courses/coffee-break-python/lectures/6277038">learn</a> buffering in the context of Python, we should ponder upon what buffer means in real life? Let me answer this question with another question. When you are in a stressful situation, whom or what do you look up to as an option to lessen your stress? On a personal level, I look up to my friends and family. So they are my “<strong>buffer</strong>” against stress. </p>
<p>According to the <a href="https://blog.finxter.com/python-dictionary/" target="_blank" rel="noreferrer noopener">dictionary</a> definition, a buffer is a person or thing that reduces a shock or that forms a barrier between incompatible or antagonistic people or things. This is what exactly a buffer does when it comes to a programming language like <a href="https://blog.finxter.com/python-crash-course/" target="_blank" rel="noreferrer noopener" title="Python Programming Tutorial [+Cheat Sheets]">Python </a>(though it is not a person in this case).</p>
<p>In terms of computer science, you may consider buffer as a <strong><em>middle layer which ensures a more efficient way for data to be transferred from one location to another in the system</em></strong>. For example, if you are downloading a 100 MB file and 1 MB gets written at a time to the hard disk would mean that it would require a total of 100 disk writes. However, a buffer could ensure that 10 MB gets written at a time to the disk, and this way only 10 disk writes will be needed. </p>
<p><strong>Advantages and Disadvantages:</strong></p>
<p>From the above scenario, we learned how buffer can act as an intermediate resource and reduce the number of disk writes thereby enhancing write efficiency and memory management. However, when it comes to a programming language, buffering has a downside too. We cannot see the output in real-time. This does not make much of a difference in case of simple or short programs but in <a href="https://blog.finxter.com/complexity-of-python-operations/">complex</a> and long programs that require a lot of computation, this might become a major drawback since we might need a more granular output rather than the entire output being displayed at once for debugging and testing purposes.</p>
<p><strong>Example:</strong> Consider the following program for which we shall compare the buffered and unbuffered outputs:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">for buffer in range(20): print(buffer, end=" ")</pre>
<p>We know that the Output is a range of numbers from 0 to 19, that is</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 </pre>
<p>But we are more interested in understanding the <em>nature </em>of the output. If buffering is enabled you will get the output at once on the screen after waiting for 19 seconds. However, when buffering is disabled the numbers will be displayed one by one every 2 seconds.</p>
<p>You can try this in our interactive shell:</p>
<p> <iframe height="600px" width="100%" src="https://repl.it/@finxter/buffering?lite=true" scrolling="no" frameborder="no" allowtransparency="true" allowfullscreen="true" sandbox="allow-forms allow-pointer-lock allow-popups allow-same-origin allow-scripts allow-modals"></iframe> </p>
<p><em><strong>Exercise</strong>: Which behavior do you observe?</em></p>
<h2>How To Disable Output Buffering In Python?</h2>
<p>By default output buffering is enabled in Python. Let us have a look at the different ways in which we can disable output buffering in python.</p>
<h2>Method 1: Running Python With -u Command Line Arguement</h2>
<p>If you want to skip buffering for the entire program then the simplest solution to do this is to run your program using the command line interface and use the <code>-u</code> switch to bypass buffering. The syntax to disable buffering through the command line is: </p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">python -u filename.py</pre>
<p><strong>Output</strong></p>
<figure class="wp-block-image size-large"><img src="https://blog.finxter.com/wp-content/uploads/2020/09/filename-1.gif" alt="" class="wp-image-12933"/></figure>
<h2>Method 2: Using The Flush Keyword</h2>
<p>If you want to skip buffering for specific lines of your code, passing the <code>flush</code> <a href="https://blog.finxter.com/python-cheat-sheet/">keyword</a> as an argument to the print statement is a good option. By default, the flush argument is set as <code>False</code>. To ensure that output buffering is disabled, the flush argument should be set to <code>True</code>.</p>
<p>Let us have a look at the following example to understand the syntax and usage of the <code>flush</code> keyword:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time
for buffer in range(20): print(buffer, end=" ", flush=True) time.sleep(2)</pre>
<p><strong>Output</strong></p>
<figure class="wp-block-image is-resized"><img loading="lazy" src="https://blog.finxter.com/wp-content/uploads/2020/09/Untitled1.gif" alt="This image has an empty alt attribute; its file name is Untitled1.gif" width="453" height="153"/></figure>
<h2>Method 3: Using sys.stdout.flush()</h2>
<p>Using the <code>sys.stdout.flush()</code> forces the program to flush the buffer. Therefore everything in the buffer will be displayed on the standard output without delay. </p>
<p>Let us have a look at how <code>sys.stdout.flush()</code> works in the following program:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time
import sys for buffer in range(20): print(buffer, end=" ") sys.stdout.flush() time.sleep(2)</pre>
<p><strong>Output</strong></p>
<figure class="wp-block-image is-resized"><img loading="lazy" src="https://blog.finxter.com/wp-content/uploads/2020/09/Untitled1.gif" alt="This image has an empty alt attribute; its file name is Untitled1.gif" width="436" height="147"/></figure>
<h2>Method 4: Wrapping sys.stdout Using TextIOWrapper And Setting Buffered Size = 0</h2>
<p>Another approach to disable the output buffering in python is to open up the <code>stdout</code> in write mode with buffer size set to 0 (unbuffered) and then wrapping it using TextIOWrapper to ensure that you get a TextIO stream and not a binary <a href="https://blog.finxter.com/python-one-liner-write-string-to-file/">file object</a>. Also, you must ensure to enable the write_through to eliminate all buffers. </p>
<p>Let us have a look at the following code to get a better grip on the above explanation:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># open the stdout file descriptor in write mode and set 0 as the buffer size: unbuffered
import io
import os
import sys
import time try: # open stdout in binary mode, then wrap it in a TextIOWrapper and enable write_through sys.stdout = io.TextIOWrapper(open(sys.stdout.fileno(), 'wb', 0), write_through=True) # for flushing on newlines only use : # sys.stdout.reconfigure(line_buffering=True)
except TypeError: # In case you are on Python 2 sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0) print("Enter a Line: ")
line = sys.stdin.readline()
for i in line: print(i, end=" ") time.sleep(2)</pre>
<p><strong>Output</strong></p>
<figure class="wp-block-image size-large"><img src="https://blog.finxter.com/wp-content/uploads/2020/09/Untitled.gif" alt="" class="wp-image-12930"/></figure>
<h2>Method 5: Setting Python Environment Variable PYTHONUNBUFFERED</h2>
<p>When the <code><a href="https://docs.python.org/3/using/cmdline.html#envvar-PYTHONUNBUFFERED">PYTHONUNBUFFERED</a></code> variable is set to a non-empty string, it is equivalent to the <code>-u</code> command line option.</p>
<h2>Conclusion</h2>
<p>I hope this article helped you to get an overview of Python output buffering and the methods to disable it. <a href="https://blog.finxter.com/subscribe/">Please stay tuned for more interesting articles.</a></p>
<h2>Where to Go From Here?</h2>
<p>Enough theory, let’s get some practice!</p>
<p>To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?</p>
<p><strong>Practice projects is how you sharpen your saw in coding!</strong></p>
<p>Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?</p>
<p>Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.</p>
<p>Join my free webinar <a rel="noreferrer noopener" href="https://blog.finxter.com/webinar-freelancer/" target="_blank">“How to Build Your High-Income Skill Python”</a> and watch how I grew my coding business online and how you can, too—from the comfort of your own home.</p>
<p><a href="https://blog.finxter.com/webinar-freelancer/" target="_blank" rel="noreferrer noopener">Join the free webinar now!</a></p>
</div>
https://www.sickgaming.net/blog/2020/09/...isable-it/
<div><p><strong>Summary:</strong> <em>Python output buffering</em> is the process of storing the output of your code in buffer memory. Once the buffer is full, the output gets displayed on the standard output screen. Buffering is enabled by default and you can use one of the following methods to disable it: </p>
<ul>
<li>Execute the Python code with the <code>-u</code> command line switch, for example <code>python -u code.py</code></li>
<li>Use the <code>flush</code> <a href="https://blog.finxter.com/python-cheat-sheet/" target="_blank" rel="noreferrer noopener">keyword</a></li>
<li>Use <code>sys.stdout.flush()</code></li>
<li>Wrap <code>sys.stdout</code> in <code>TextIOWrapper</code> and set the buffered size to 0</li>
<li>Set <code>PYTHONUNBUFFERED</code> to a non-empty string</li>
</ul>
<p><strong>Problem: </strong>Given a Python program; how to disable output buffering?</p>
<h2>Overview Of Python Output Buffering</h2>
<figure class="wp-block-image size-large is-resized"><img loading="lazy" src="https://blog.finxter.com/wp-content/uploads/2020/09/buffer.gif" alt="" class="wp-image-12938" width="652" height="337"/></figure>
<p>Before we <a href="https://finxter-coding-university.teachable.com/courses/coffee-break-python/lectures/6277038">learn</a> buffering in the context of Python, we should ponder upon what buffer means in real life? Let me answer this question with another question. When you are in a stressful situation, whom or what do you look up to as an option to lessen your stress? On a personal level, I look up to my friends and family. So they are my “<strong>buffer</strong>” against stress. </p>
<p>According to the <a href="https://blog.finxter.com/python-dictionary/" target="_blank" rel="noreferrer noopener">dictionary</a> definition, a buffer is a person or thing that reduces a shock or that forms a barrier between incompatible or antagonistic people or things. This is what exactly a buffer does when it comes to a programming language like <a href="https://blog.finxter.com/python-crash-course/" target="_blank" rel="noreferrer noopener" title="Python Programming Tutorial [+Cheat Sheets]">Python </a>(though it is not a person in this case).</p>
<p>In terms of computer science, you may consider buffer as a <strong><em>middle layer which ensures a more efficient way for data to be transferred from one location to another in the system</em></strong>. For example, if you are downloading a 100 MB file and 1 MB gets written at a time to the hard disk would mean that it would require a total of 100 disk writes. However, a buffer could ensure that 10 MB gets written at a time to the disk, and this way only 10 disk writes will be needed. </p>
<p><strong>Advantages and Disadvantages:</strong></p>
<p>From the above scenario, we learned how buffer can act as an intermediate resource and reduce the number of disk writes thereby enhancing write efficiency and memory management. However, when it comes to a programming language, buffering has a downside too. We cannot see the output in real-time. This does not make much of a difference in case of simple or short programs but in <a href="https://blog.finxter.com/complexity-of-python-operations/">complex</a> and long programs that require a lot of computation, this might become a major drawback since we might need a more granular output rather than the entire output being displayed at once for debugging and testing purposes.</p>
<p><strong>Example:</strong> Consider the following program for which we shall compare the buffered and unbuffered outputs:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">for buffer in range(20): print(buffer, end=" ")</pre>
<p>We know that the Output is a range of numbers from 0 to 19, that is</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 </pre>
<p>But we are more interested in understanding the <em>nature </em>of the output. If buffering is enabled you will get the output at once on the screen after waiting for 19 seconds. However, when buffering is disabled the numbers will be displayed one by one every 2 seconds.</p>
<p>You can try this in our interactive shell:</p>
<p> <iframe height="600px" width="100%" src="https://repl.it/@finxter/buffering?lite=true" scrolling="no" frameborder="no" allowtransparency="true" allowfullscreen="true" sandbox="allow-forms allow-pointer-lock allow-popups allow-same-origin allow-scripts allow-modals"></iframe> </p>
<p><em><strong>Exercise</strong>: Which behavior do you observe?</em></p>
<h2>How To Disable Output Buffering In Python?</h2>
<p>By default output buffering is enabled in Python. Let us have a look at the different ways in which we can disable output buffering in python.</p>
<h2>Method 1: Running Python With -u Command Line Arguement</h2>
<p>If you want to skip buffering for the entire program then the simplest solution to do this is to run your program using the command line interface and use the <code>-u</code> switch to bypass buffering. The syntax to disable buffering through the command line is: </p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">python -u filename.py</pre>
<p><strong>Output</strong></p>
<figure class="wp-block-image size-large"><img src="https://blog.finxter.com/wp-content/uploads/2020/09/filename-1.gif" alt="" class="wp-image-12933"/></figure>
<h2>Method 2: Using The Flush Keyword</h2>
<p>If you want to skip buffering for specific lines of your code, passing the <code>flush</code> <a href="https://blog.finxter.com/python-cheat-sheet/">keyword</a> as an argument to the print statement is a good option. By default, the flush argument is set as <code>False</code>. To ensure that output buffering is disabled, the flush argument should be set to <code>True</code>.</p>
<p>Let us have a look at the following example to understand the syntax and usage of the <code>flush</code> keyword:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time
for buffer in range(20): print(buffer, end=" ", flush=True) time.sleep(2)</pre>
<p><strong>Output</strong></p>
<figure class="wp-block-image is-resized"><img loading="lazy" src="https://blog.finxter.com/wp-content/uploads/2020/09/Untitled1.gif" alt="This image has an empty alt attribute; its file name is Untitled1.gif" width="453" height="153"/></figure>
<h2>Method 3: Using sys.stdout.flush()</h2>
<p>Using the <code>sys.stdout.flush()</code> forces the program to flush the buffer. Therefore everything in the buffer will be displayed on the standard output without delay. </p>
<p>Let us have a look at how <code>sys.stdout.flush()</code> works in the following program:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import time
import sys for buffer in range(20): print(buffer, end=" ") sys.stdout.flush() time.sleep(2)</pre>
<p><strong>Output</strong></p>
<figure class="wp-block-image is-resized"><img loading="lazy" src="https://blog.finxter.com/wp-content/uploads/2020/09/Untitled1.gif" alt="This image has an empty alt attribute; its file name is Untitled1.gif" width="436" height="147"/></figure>
<h2>Method 4: Wrapping sys.stdout Using TextIOWrapper And Setting Buffered Size = 0</h2>
<p>Another approach to disable the output buffering in python is to open up the <code>stdout</code> in write mode with buffer size set to 0 (unbuffered) and then wrapping it using TextIOWrapper to ensure that you get a TextIO stream and not a binary <a href="https://blog.finxter.com/python-one-liner-write-string-to-file/">file object</a>. Also, you must ensure to enable the write_through to eliminate all buffers. </p>
<p>Let us have a look at the following code to get a better grip on the above explanation:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># open the stdout file descriptor in write mode and set 0 as the buffer size: unbuffered
import io
import os
import sys
import time try: # open stdout in binary mode, then wrap it in a TextIOWrapper and enable write_through sys.stdout = io.TextIOWrapper(open(sys.stdout.fileno(), 'wb', 0), write_through=True) # for flushing on newlines only use : # sys.stdout.reconfigure(line_buffering=True)
except TypeError: # In case you are on Python 2 sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0) print("Enter a Line: ")
line = sys.stdin.readline()
for i in line: print(i, end=" ") time.sleep(2)</pre>
<p><strong>Output</strong></p>
<figure class="wp-block-image size-large"><img src="https://blog.finxter.com/wp-content/uploads/2020/09/Untitled.gif" alt="" class="wp-image-12930"/></figure>
<h2>Method 5: Setting Python Environment Variable PYTHONUNBUFFERED</h2>
<p>When the <code><a href="https://docs.python.org/3/using/cmdline.html#envvar-PYTHONUNBUFFERED">PYTHONUNBUFFERED</a></code> variable is set to a non-empty string, it is equivalent to the <code>-u</code> command line option.</p>
<h2>Conclusion</h2>
<p>I hope this article helped you to get an overview of Python output buffering and the methods to disable it. <a href="https://blog.finxter.com/subscribe/">Please stay tuned for more interesting articles.</a></p>
<h2>Where to Go From Here?</h2>
<p>Enough theory, let’s get some practice!</p>
<p>To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?</p>
<p><strong>Practice projects is how you sharpen your saw in coding!</strong></p>
<p>Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?</p>
<p>Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.</p>
<p>Join my free webinar <a rel="noreferrer noopener" href="https://blog.finxter.com/webinar-freelancer/" target="_blank">“How to Build Your High-Income Skill Python”</a> and watch how I grew my coding business online and how you can, too—from the comfort of your own home.</p>
<p><a href="https://blog.finxter.com/webinar-freelancer/" target="_blank" rel="noreferrer noopener">Join the free webinar now!</a></p>
</div>
https://www.sickgaming.net/blog/2020/09/...isable-it/