To speed up my code, I just decided to (finally) dive into Python’s async with statement. In this article, you’ll find some of my learnings – let’s go!

What Is Python Async With?

Python’s async with statement is a way to work with asynchronous context managers, which can be really useful when dealing with I/O-bound tasks, such as reading or writing files, making HTTP requests, or interacting with databases. These tasks normally block your program’s execution, involving waiting for external resources. But using async with, you can perform multiple tasks concurrently!

Let’s see some code. Picture this scenario: you’re using asyncio and aiohttp to fetch some content over HTTP. If you were to use a regular with statement, your code would look like this:

import aiohttp

import asyncio async def fetch(url): with aiohttp.ClientSession() as session: response = await session.get(url) content = await response.read() print(asyncio.run(fetch("https://example.com")))

But see the problem? This would block the event loop, making your app slower .

The solution is using async with alongside a context manager that supports it:

import aiohttp

import asyncio async def fetch(url): async with aiohttp.ClientSession() as session: response = await session.get(url) content = await response.read() print(asyncio.run(fetch("https://example.com")))

Thanks to async with, your code won’t block the event loop while working with context managers, making your program more efficient and responsive!

No worries if you didn’t quite get it yet. Keep reading!

Python Async With Examples

The async with statement is used when you want to run a certain operation concurrently and need to manage resources effectively, such as when dealing with I/O-bound tasks like fetching a web page .

Let’s jump into an example using an asyncio-based library called aiohttp.

Here’s how you can make an HTTP GET request using an async with statement and aiohttp’s ClientSession class:

import aiohttp

import asyncio async def fetch_page(session, url): async with session.get(url) as response: return await response.text() async def main(): async with aiohttp.ClientSession() as session: content = await fetch_page(session, 'https://example.com') print(content) loop = asyncio.get_event_loop()

loop.run_until_complete(main())

In the example above, you use an async with statement in the fetch_page function to ensure that the response is properly acquired and released. You can see a similar pattern in the main function, where the aiohttp.ClientSession is managed using an async with statement. This ensures that resources such as network connections are handled properly.

Now, let’s discuss some key entities in this example:

• session: An instance of the aiohttp.ClientSession class, used to manage HTTP requests.
• url: A variable representing the URL of the web page you want to fetch.
• response: The HTTP response object returned by the server.
• clientsession: A class provided by the aiohttp library to manage HTTP requests and responses.
• text(): A method provided by the aiohttp library to read the response body as text.
• async with statement: A special construct to manage resources within an asynchronous context.

Async With Await

In Python, the await keyword is used with asynchronous functions, which are defined using the async def syntax. Asynchronous functions, or coroutines, enable non-blocking execution and allow you to run multiple tasks concurrently without the need for threading or multiprocessing.

In the context of the async with statement, await is used to wait for the asynchronous context manager to complete its tasks. The async with statement is used in conjunction with an asynchronous context manager, which is an object that defines __aenter__() and __aexit__() asynchronous methods. These methods are used to set up and tear down a context for a block of code that will be executed asynchronously.

Here’s an example of how async with and await are used together:

import aiohttp

import asyncio async def fetch(url): async with aiohttp.ClientSession() as session: async with session.get(url) as response: return await response.text() async def main(): url = 'https://example.com/' html = await fetch(url) print(html) loop = asyncio.get_event_loop()

loop.run_until_complete(main())

In this example, the fetch function is an asynchronous function that retrieves the content of a given URL using the aiohttp library.

The async with statement is used to create an asynchronous context manager for the aiohttp.ClientSession() and session.get(url) objects.

The await keyword is then used to wait for the response from the session.get() call to be available, and to retrieve the text content of the response.

Async With Open

By using “async with open“, you can open files in your asynchronous code without blocking the execution of other coroutines.

When working with async code, it’s crucial to avoid blocking operations. To tackle this, some libraries provide asynchronous equivalents of “open“, allowing you to seamlessly read and write files in your asynchronous code.

For example:

import aiofiles async def read_file(file_path): async with aiofiles.open(file_path, 'r') as file: contents = await file.read() return contents

Here, we use the aiofiles library, which provides an asynchronous file I/O implementation. With “async with“, you can open the file, perform the desired operations (like reading or writing), and the file will automatically close when it’s no longer needed – all without blocking your other async tasks. Neat, huh?

Remember, it’s essential to use an asynchronous file I/O library, like aiofiles, when working with async with open. This ensures that your file operations won’t block the rest of your coroutines and keep your async code running smoothly.

Async With Yield

When working with Python’s async functions, you might wonder how to use the yield keyword within an async with statement. In this section, you’ll learn how to effectively combine these concepts for efficient and readable code.

Recommended: Understanding Python’s yield Keyword

First, it’s essential to understand that you cannot use the standard yield with async functions. Instead, you need to work with asynchronous generators, introduced in Python 3.6 and PEP 525. Asynchronous generators allow you to yield values concurrently using the async def keyword and help you avoid blocking operations.

To create an asynchronous generator, you can define a function with the async def keyword and use the yield statement inside it, like this:

import asyncio async def asyncgen(): yield 1 yield 2

To consume the generator, use async for like this:

async def main(): async for i in asyncgen(): print(i) asyncio.run(main())

This code will create a generator that yields values asynchronously and print them using an async context manager. This approach allows you to wrapp an asynchronous generator in a friendly and readable manner.

Now you know how to use the yield keyword within an async with statement in Python. It’s time to leverage the power of asynchronous generators in your code!

Async With Return

The async with statement is used to simplify your interactions with asynchronous context managers in your code, and yes, it can return a value as well!

When working with async with, the value returned by the __enter__() method of an asynchronous context manager gets bound to a target variable. This helps you manage resources effectively in your async code.

For instance:

async def fetch_data(url): async with aiohttp.ClientSession() as session: async with session.get(url) as response: data = await response.json() return data

In this example, the response object is the value returned by the context manager’s .__enter__() method. Once the async with block is executed, the response.json() method is awaited to deserialize the JSON data, which is then returned to the caller .

Here are a few key takeaway points about returning values with async with:

• async with simplifies handling of resources in asynchronous code.
• The value returned by the .__enter__() method is automatically bound to a target variable within the async with block.
• Returning values from your asynchronous context managers easily integrates with your async code.

Advanced Usage of Async With

As you become more familiar with Python’s async with statement, you’ll discover advanced techniques that can greatly enhance your code’s efficiency and readability. This section will cover four techniques:

• Async With Timeout,
• Async With Multiple,
• Async With Lock, and
• Async With Context Manager

Async With Timeout

When working with concurrency, it’s essential to manage timeouts effectively. In async with statements, you can ensure a block of code doesn’t run forever by implementing an async timeout. This can help you handle scenarios where network requests or other I/O operations take too long to complete.

Here’s an example of how you can define an async with statement with a timeout:

import asyncio

import aiohttp async def fetch_page(session, url): async with aiohttp.Timeout(10): async with session.get(url) as response: assert response.status == 200 return await response.read()

This code sets a 10-second timeout to fetch a web page using the aiohttp library.

Async With Multiple

You can use multiple async with statements simultaneously to work with different resources, like file access or database connections. Combining multiple async with statements enables better resource management and cleaner code:

async def two_resources(resource_a, resource_b): async with aquire_resource_a(resource_a) as a, aquire_resource_b(resource_b) as b: await do_something(a, b)

This example acquires both resources asynchronously and then performs an operation using them.

Async With Lock

Concurrency can cause issues when multiple tasks access shared resources. To protect these resources and prevent race conditions, you can use async with along with Python’s asyncio.Lock() class:

import asyncio lock = asyncio.Lock() async def my_function(): async with lock: # Section of code that must be executed atomically.

This code snippet ensures that the section within the async with block is executed atomically, protecting shared resources from being accessed concurrently.

Async With Context Manager

Creating your own context managers can make it easier to manage resources in asynchronous code. By defining an async def __aenter__() and an async def __aexit__() method in your class, you can use it within an async with statement:

class AsyncContextManager: async def __aenter__(self): # Code to initialize resource return resource async def __aexit__(self, exc_type, exc, tb): # Code to release resource async def demo(): async with AsyncContextManager() as my_resource: # Code using my_resource

This custom context manager initializes and releases a resource within the async with block, simplifying your asynchronous code and making it more Pythonic.

Error Handling and Exceptions

When working with Python’s Async With Statement, handling errors and exceptions properly is essential to ensure your asynchronous code runs smoothly. Using try-except blocks and well-structured clauses can help you manage errors effectively.

Be aware of syntax errors, which may occur when using async with statements. To avoid SyntaxError, make sure your Python code is properly formatted and follows PEP 343’s guidelines. If you come across an error while performing an IO operation or dealing with external resources, it’s a good idea to handle it with an except clause.

In the case of exceptions, you might want to apply cleanup code to handle any necessary actions before your program closes or moves on to the next task. One way to do this is by wrapping your async with statement in a try-except block, and then including a finally clause for the cleanup code.

Here’s an example:

try: async with some_resource() as resource: # Perform your IO operation or other tasks here

except YourException as e: # Handle the specific exception here

finally: # Add cleanup code here

Remember, you need to handle exceptions explicitly in the parent coroutine if you want to prevent them from canceling the entire task. In the case of multiple exceptions or errors, using asyncio.gather can help manage them effectively.

Recommended: Python Beginner Cheat Sheet: 19 Keywords Every Coder Must Know

If you are interested in natural language processing (NLP) and computer vision, you may have heard about MiniGPT-4.

This neural network model has been developed to improve vision-language comprehension by incorporating a frozen visual encoder and a frozen large language model (LLM) with a single projection layer.

MiniGPT-4 has demonstrated numerous capabilities similar to GPT-4, like generating detailed image descriptions and creating websites from handwritten drafts.

One of the most impressive features of MiniGPT-4 is its computation efficiency. Despite its advanced capabilities, this model is designed to be lightweight and easy to use. This makes it an ideal choice for developers who need to generate natural language descriptions of images but don’t want to spend hours training a complex neural network.

Image source: https://github.com/Vision-CAIR/MiniGPT-4

Additionally, MiniGPT-4 has been shown to have high generation reliability, meaning that it consistently produces accurate and relevant descriptions of images.

What is MiniGPT-4?

If you’re looking for a computationally efficient large language model that can generate reliable text, MiniGPT-4 might be the solution you’re looking for.

MiniGPT-4 is a language model architecture that combines a frozen visual encoder with a frozen large language model (LLM) using just one linear projection layer. The model is designed to align the visual features with the language model, making it capable of processing images alongside language.

Image source: https://github.com/Vision-CAIR/MiniGPT-4

MiniGPT-4 is an open-source model that can be fine-tuned to perform complex vision-language tasks like GPT-4. The model architecture consists of a vision encoder with a pre-trained ViT and Q-Former, a single linear projection layer, and an advanced Vicuna large language model. The trained checkpoint can be used for transfer learning, and the model can be fine-tuned on specific tasks with additional data.

MiniGPT-4 has many capabilities similar to those exhibited by GPT-4, including detailed image description generation and website creation from hand-written drafts.

Image Source: https://minigpt-4.github.io/

The model is computationally efficient and can be trained on a single GPU, making it accessible to researchers and developers who don’t have access to large-scale computing resources.

Video Example of Using MiniGPT

MiniGPT-4 Demo

If you’re interested in trying out MiniGPT-4, you’ll be pleased to know that a demo is available for you to test:

Demo Link: https://minigpt-4.github.io/

The demo allows you to see the capabilities of MiniGPT-4 in action and provides a glimpse of what you can expect if you decide to use it in your own projects.

User-Friendly Demo: The MiniGPT-4 demo is user-friendly and easy to use, even if you’re unfamiliar with this technology. The interface is simple and straightforward, allowing you to input text or images and see how MiniGPT-4 processes them. The demo is intuitive, so you can start immediately without prior knowledge or experience.

Generate Websites From Hand-Written Text: One of the most impressive features of the MiniGPT-4 demo is its ability to generate websites from handwritten text. This means you can input a piece of text, and MiniGPT-4 will create a website based on that text. The websites generated by MiniGPT-4 are professional-looking and can be used for various purposes.

Create Image Descriptions: MiniGPT-4 can also create detailed image descriptions in addition to generating websites. This is particularly useful for those who work in fields such as art or photography, where providing detailed descriptions of images is essential. With MiniGPT-4, you can input an image and receive a detailed description that accurately captures the essence of the image.

Image Source: https://minigpt-4.github.io/

MiniGPT-4 for Image-Text Pairs

Let’s explore how MiniGPT-4 can help you with image-text pairs.

Aligned Image-Text Pairs

MiniGPT-4 uses aligned image-text pairs to learn how to generate accurate descriptions of images. MiniGPT-4 aligns a frozen visual encoder with a frozen language model called Vicuna using just one projection layer during training.

This allows MiniGPT-4 to learn how to generate natural language descriptions of images aligned with the image’s visual features.

Raw Image-Text Pairs

MiniGPT-4 can also work with raw image-text pairs. However, the quality of the dataset is crucial for the performance of MiniGPT-4.

To achieve high accuracy, you need a high-quality dataset of image-text pairs. MiniGPT-4 requires a large and diverse dataset of high-quality image-text pairs to learn how to generate accurate descriptions of images.

Image Descriptions

MiniGPT-4 can generate accurate descriptions of images, write texts based on images, provide solutions to problems depicted in pictures, and even teach users how to do certain things based on photos. MiniGPT-4’s ability to generate accurate descriptions of images is due to its powerful visual encoder and ability to align the visual features with natural language descriptions.

Multi-Modal Abilities

MiniGPT-4 has demonstrated extraordinary multi-modal abilities, such as directly generating websites from handwritten text and identifying humorous elements within images. These features are rarely observed in previous vision-language models.

Image Source: https://minigpt-4.github.io/

Let’s take a closer look at some of MiniGPT-4’s multi-modal abilities:

Image Description Generation

MiniGPT-4 can generate descriptions of images.

For example, if you have an image of a product you want to sell online, you can use MiniGPT-4 to generate a description of the product you can use in your online store.

MiniGPT-4 can also be used to generate descriptions of images for people who are visually impaired. This can be particularly helpful for people who rely on screen readers to access information online.

Conversation Template

MiniGPT-4 can generate conversational templates. MiniGPT-4 can generate a template to use as a starting point for your conversation.

Examples:

• If you need to have a conversation with your boss about a difficult topic, you can use MiniGPT-4 to generate a template that you can use to start the conversation.
• MiniGPT-4 can also generate conversational templates for people struggling to express themselves verbally or with hand-written drafts.

Recommended: Free OpenAI Terminology Cheat Sheet (PDF)

MiniGPT-4 Implementation

Installation

You can install the code from the Vision-CAIR/MiniGPT-4 GitHub repository. The code is available under the BSD 3-Clause License. To install MiniGPT-4, clone the repository and install the required packages.

The installation instructions are provided in the README file of the repository:

git clone https://github.com/Vision-CAIR/MiniGPT-4.git

cd MiniGPT-4

conda env create -f environment.yml

conda activate minigpt4

Dataset Preparation

MiniGPT-4 requires aligned image-text pairs for training. The authors of MiniGPT-4 used the Laion and CC datasets for the first pretraining stage.

To prepare the datasets, download and preprocess them using the provided scripts. The instructions for dataset preparation are also available in the repository’s README file.

Model Config File

The model configuration file contains the hyperparameters and settings for the MiniGPT-4 model.

You can modify the configuration file to adjust the model settings according to your needs. The configuration file is provided in the repository and is named config.yaml.

The configuration file contains settings for the vision encoder, language model, training, and evaluation parameters.

Evaluation Config File

The evaluation configuration file contains the settings for evaluating the MiniGPT-4 model. You can modify the evaluation configuration file to adjust the evaluation settings according to your needs.

The evaluation configuration file is provided in the repository and is named eval.yaml. The evaluation configuration file contains settings for the evaluation dataset, the evaluation metrics, and the evaluation batch size.

MiniGPT-4 aligns a frozen visual encoder from BLIP-2 with a frozen LLM, Vicuna, using just one projection layer. The first traditional pretraining stage is trained using roughly 5 million aligned image-text pairs in 10 hours using 4 A100s.

After the first stage, Vicuna can understand the image. MiniGPT-4 is an implementation of the GPT architecture that enhances vision-language understanding by combining a frozen visual encoder with a frozen large language model (LLM) using just one projection layer.

The implementation is lightweight and requires training only the linear layer to align the visual features with the Vicuna.

Research Paper Citation

If you want to use this in your own research, use the following Latex template for citation:

@misc{zhu2022minigpt4, title={MiniGPT-4: Enhancing Vision-language Understanding with Advanced Large Language Models}, author={Deyao Zhu and Jun Chen and Xiaoqian Shen and Xiang Li and Mohamed Elhoseiny}, journal={arXiv preprint arXiv:2304.10592}, year={2023},
}

Recommended: Free ChatGPT Prompting Cheat Sheet (PDF)

Download Link: https://blog.finxter.com/wp-content/uploads/2023/04/Finxter_OpenAI_Python_API.pdf

This code creates a simple Snake game using the Pygame library in Python.

The game starts with a small snake moving around the screen, controlled by arrow keys, eating randomly placed food items. Each time the snake eats food, it grows longer, and the game gets faster.

The game ends if the snake collides with itself or the screen borders, and the player’s goal is to keep the snake growing as long as possible.

import pygame

import sys

import random # Initialize pygame

pygame.init() # Set screen dimensions

WIDTH = 640

HEIGHT = 480

CELL_SIZE = 20 # Colors

WHITE = (255, 255, 255)

GREEN = (0, 255, 0)

RED = (255, 0, 0) # Create the game screen

screen = pygame.display.set_mode((WIDTH, HEIGHT)) # Set the game clock

clock = pygame.time.Clock() def random_food_position(): return (random.randint(0, (WIDTH-CELL_SIZE)//CELL_SIZE) * CELL_SIZE, random.randint(0, (HEIGHT-CELL_SIZE)//CELL_SIZE) * CELL_SIZE) def draw_snake(snake_positions): for pos in snake_positions: pygame.draw.rect(screen, GREEN, pygame.Rect(pos[0], pos[1], CELL_SIZE, CELL_SIZE)) def draw_food(food_position): pygame.draw.rect(screen, RED, pygame.Rect(food_position[0], food_position[1], CELL_SIZE, CELL_SIZE)) def main(): snake_positions = [(100, 100), (80, 100), (60, 100)] snake_direction = (20, 0) food_position = random_food_position() game_speed = 10 while True: for event in pygame.event.get(): if event.type == pygame.QUIT: pygame.quit() sys.exit() if event.type == pygame.KEYDOWN: if event.key == pygame.K_UP and snake_direction != (0, 20): snake_direction = (0, -20) elif event.key == pygame.K_DOWN and snake_direction != (0, -20): snake_direction = (0, 20) elif event.key == pygame.K_LEFT and snake_direction != (20, 0): snake_direction = (-20, 0) elif event.key == pygame.K_RIGHT and snake_direction != (-20, 0): snake_direction = (20, 0) new_head = (snake_positions[0][0] + snake_direction[0], snake_positions[0][1] + snake_direction[1]) if new_head in snake_positions or new_head[0] < 0 or new_head[0] >= WIDTH or new_head[1] < 0 or new_head[1] >= HEIGHT: break snake_positions.insert(0, new_head) if new_head == food_position: food_position = random_food_position() game_speed += 1 else: snake_positions.pop() screen.fill(WHITE) draw_snake(snake_positions) draw_food(food_position) pygame.display.update() clock.tick(game_speed) if __name__ == "__main__": main()

This simple implementation of the classic Snake game uses the Pygame library in Python — here’s how I quickly lost in the game:

Here’s a brief explanation of each part of the code:

1. Import libraries:
   
   • pygame for creating the game window and handling events,
   • sys for system-specific parameters and functions, and
   • random for generating random numbers.
2. Initialize pygame with pygame.init().
3. Set screen dimensions and cell size. The WIDTH, HEIGHT, and CELL_SIZE constants define the game window size and the size of each cell (both snake segments and food).
4. Define color constants WHITE, GREEN, and RED as RGB tuples.
5. Create the game screen with pygame.display.set_mode((WIDTH, HEIGHT)).
6. Set the game clock with pygame.time.Clock() to control the game’s frame rate.
7. Define the random_food_position() function to generate a random food position on the grid, ensuring it’s aligned with the CELL_SIZE.
8. Define draw_snake(snake_positions) function to draw the snake’s body segments at their respective positions using green rectangles.
9. Define draw_food(food_position) function to draw the food as a red rectangle at its position.
10. Define the main() function, which contains the main game loop and logic.
    
    • Initialize the snake’s starting position, direction, food position, and game speed.
    • The while True loop is the main game loop that runs indefinitely until the snake collides with itself or the screen borders.
    • Handle events like closing the game window or changing the snake’s direction using arrow keys.
    • Update the snake’s position based on its current direction.
    • Check for collisions with itself or the screen borders, breaking the loop if a collision occurs.
    • If the snake’s head is at the food position, generate a new food position and increase the game speed. Otherwise, remove the last snake segment.
    • Update the game display by filling the screen with a white background, drawing the snake and food, and updating the display.
    • Control the game speed using clock.tick(game_speed).
11. Run the main() function when the script is executed by checking if __name__ == "__main__".