Create an account


Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
[-]
Welcome
You have to register before you can post on our site.

Username:


Password:





[-]
Discord Widget

[-]
Twitter: SickGaming

[-]
Latest Threads
[Oracle Blog] What’s New in Java Managem...
Last Post: xSicKxBot
Today 06:51 AM
» Replies: 0
» Views: 7
[Tut] How to Flush Your Cache on Windows...
Last Post: xSicKxBot
Today 06:51 AM
» Replies: 0
» Views: 5
(Indie Deal) Match3 Love Bundle & MechWa...
Last Post: xSicKxBot
Today 06:51 AM
» Replies: 0
» Views: 5
PC - Backfirewall_
Last Post: xSicKxBot
Today 06:51 AM
» Replies: 0
» Views: 5
[Oracle Blog] JavaOne is Back!
Last Post: xSicKxBot
Yesterday 12:34 PM
» Replies: 0
» Views: 10
[Tut] How I Built a House Price Predicti...
Last Post: xSicKxBot
Yesterday 12:33 PM
» Replies: 0
» Views: 8
[Tut] How to Install phpMyAdmin on a Win...
Last Post: xSicKxBot
Yesterday 12:33 PM
» Replies: 0
» Views: 8
(Indie Deal) PlayStation Sale: God of Wa...
Last Post: xSicKxBot
Yesterday 12:33 PM
» Replies: 0
» Views: 7
News - New Marvel Book Details The MCU's...
Last Post: xSicKxBot
Yesterday 12:33 PM
» Replies: 0
» Views: 7
PC - Marvel's Midnight Suns - The Good, ...
Last Post: xSicKxBot
Yesterday 12:33 PM
» Replies: 0
» Views: 7

[Tut] Python Video to Text – Speech Recognition

#1
Python Video to Text – Speech Recognition

5/5 – (1 vote)

A good friend and his wife recently founded an AI startup in the lifestyle niche that uses machine learning to discover specific real-world patterns from videos.

For their business system, they need a pipeline that takes a video file, converts it to audio, and transcribes the audio to standard text that is then used for further processing. I couldn’t help but work on a basic solution to help fix their business problem.

Project Overview



I finished the project in three steps:

  • First, install the necessary libraries.
  • Second, convert the video to an audio file (.mp4 to .wav)
  • Third, convert the audio file to a speech file (.wav to .txt). We first break the large audio file into smaller chunks and convert each of them separately due to the size restrictions of the used API.

Let’s get started!

Step 1: Install Libraries


We need the following import statements in our code:

# Import libraries
import speech_recognition as sr
import os
from pydub import AudioSegment
from pydub.silence import split_on_silence
import moviepy.editor as mp

Consequently, you need to pip install the following three libraries in your shell — assuming you run Python version 3.9:

pip3.9 install pydub
pip3.9 install SpeechRecognition
pip3.9 install moviepy

The os module is already preinstalled as a Python Standard Library.

If you need an additional guide on how to install Python libraries, check out this tutorial:

👉 Recommended: Python Install Library Guide

Step 2: Video to Audio



Before you can do speech recognition on the video, we need to extract the audio as a .wav file using the moviepy.editor.VideoFileClip().audio.write_audiofile() method.

Here’s the code:

def video_to_audio(in_path, out_path): """Convert video file to audio file""" video = mp.VideoFileClip(in_path) video.audio.write_audiofile(out_path)

👉 Recommended: Python Video to Audio

Step 3: Audio to Text


After extracting the audio file, we can start transcribing the speech from the .wav file using Google’s powerful speech recognition library on chunks of the potentially large audio file.

Using chunks instead of passing the whole audio file avoids an error for large audio files — Google has some restrictions on the audio file size.

However, you can play around with the splitting thresholds of 700ms silence—it can be more or less, depending on your concrete file.

Here’s the audio to text code function that worked for me:

def large_audio_to_text(path): """Split audio into chunks and apply speech recognition""" # Open audio file with pydub sound = AudioSegment.from_wav(path) # Split audio where silence is 700ms or greater and get chunks chunks = split_on_silence(sound, min_silence_len=700, silence_thresh=sound.dBFS-14, keep_silence=700) # Create folder to store audio chunks folder_name = "audio-chunks" if not os.path.isdir(folder_name): os.mkdir(folder_name) whole_text = "" # Process each chunk for i, audio_chunk in enumerate(chunks, start=1): # Export chunk and save in folder chunk_filename = os.path.join(folder_name, f"chunk{i}.wav") audio_chunk.export(chunk_filename, format="wav") # Recognize chunk with sr.AudioFile(chunk_filename) as source: audio_listened = r.record(source) # Convert to text try: text = r.recognize_google(audio_listened) except sr.UnknownValueError as e: print("Error:", str(e)) else: text = f"{text.capitalize()}. " print(chunk_filename, ":", text) whole_text += text # Return text for all chunks return whole_text

Need more info? Check out the following deep dive:

👉 Recommended: Large Audio to Text? Here’s My Speech Recognition Solution in Python

Step 4: Putting It Together



Finally, we can combine our functions. First, we extract the audio from the video. Second, we chunk the audio into smaller files and recognize speech independently on each chunk using Google’s speech recognition module.

I added comments to annotate the most important parts of this code:

# Import libraries
import speech_recognition as sr
import os
from pydub import AudioSegment
from pydub.silence import split_on_silence
import moviepy.editor as mp def video_to_audio(in_path, out_path): """Convert video file to audio file""" video = mp.VideoFileClip(in_path) video.audio.write_audiofile(out_path) def large_audio_to_text(path): """Split audio into chunks and apply speech recognition""" # Open audio file with pydub sound = AudioSegment.from_wav(path) # Split audio where silence is 700ms or greater and get chunks chunks = split_on_silence(sound, min_silence_len=700, silence_thresh=sound.dBFS-14, keep_silence=700) # Create folder to store audio chunks folder_name = "audio-chunks" if not os.path.isdir(folder_name): os.mkdir(folder_name) whole_text = "" # Process each chunk for i, audio_chunk in enumerate(chunks, start=1): # Export chunk and save in folder chunk_filename = os.path.join(folder_name, f"chunk{i}.wav") audio_chunk.export(chunk_filename, format="wav") # Recognize chunk with sr.AudioFile(chunk_filename) as source: audio_listened = r.record(source) # Convert to text try: text = r.recognize_google(audio_listened) except sr.UnknownValueError as e: print("Error:", str(e)) else: text = f"{text.capitalize()}. " print(chunk_filename, ":", text) whole_text += text # Return text for all chunks return whole_text # Create a speech recognition object
r = sr.Recognizer() # Video to audio to text
video_to_audio('sample_video.mp4', 'sample_audio.wav')
result = large_audio_to_text('sample_audio.wav') # Print to shell and file
print(result)
print(result, file=open('result.txt', 'w'))

Store this code in a folder next to your video file 'sample_video.mp4' and run it. It will create an audio file 'sample_audio.wav' and chunk the audio and print the result to the shell, as well as to a file called 'result.txt'. This contains the transcription of the video file.



https://www.sickgaming.net/blog/2023/01/...cognition/
Reply





Users browsing this thread:
1 Guest(s)

𝚂̷𝚒̷𝚌̷𝚔̷𝙶̷𝚊̷𝚖̷𝚒̷𝚗̷𝚐̷ 𝙷̷𝚘̷𝚖̷𝚎̷

Copyright © SickGaming.net 2022