Understanding divmod() in Python

divmod() is a useful built-in function in Python that takes two arguments and returns a tuple containing the quotient and the remainder. The function’s syntax is quite simple: divmod(x, y), where x is the dividend, and y is the divisor.

The divmod() function is particularly handy when you need both the quotient and the remainder for two numbers. In Python, you can typically compute the quotient using the // operator and the remainder using the % operator. Using divmod() is more concise and efficient because it avoids redundant work.

Here’s a basic example to illustrate how divmod() works:

x, y = 10, 3

result = divmod(x, y)

print(result) # Output: (3, 1)

In this example, divmod() returns a tuple (3, 1) – the quotient is 3, and the remainder is 1.

divmod() can be particularly useful in various applications, such as solving mathematical problems or performing operations on date and time values. Note that the function will only work with non-complex numbers as input.

Here’s another example demonstrating divmod() with larger numbers:

x, y = 2050, 100

result = divmod(x, y)

print(result) # Output: (20, 50)

In this case, the quotient is 20, and the remainder is 50.

To summarize, the divmod() function in Python is an efficient way to obtain both the quotient and the remainder when dividing two non-complex numbers.

I created an explainer video on the function here:

Divmod’s Parameters and Syntax

The divmod() function in Python is a helpful built-in method used to obtain the quotient and remainder of two numbers. To fully understand its use, let’s discuss the function’s parameters and syntax.

This function accepts two non-complex parameters, number1 and number2.

• The first parameter, number1, represents the dividend (the number being divided), while
• the second parameter, number2, denotes the divisor (the number dividing) or the denominator.

The syntax for using divmod() is straightforward:

divmod(number1, number2)

Note that both parameters must be non-complex numbers. When the function is executed, it returns a tuple containing two values – the quotient and the remainder.

Here’s an example to make it clear:

result = divmod(8, 3)

print("Quotient and Remainder =", result)

This code snippet would output:

Quotient and Remainder = (2, 2)

This indicates that when 8 is divided by 3, the quotient is 2 and the remainder is 2. Similarly, you can apply divmod() with different numbers or variables representing numbers.

Return Value of Divmod

The divmod() function in Python is a convenient way to calculate both the quotient and remainder of two numbers simultaneously. This function accepts two arguments, which are the numerator and denominator, and returns a tuple containing the quotient and remainder as its elements.

The syntax for divmod() is as follows:

quotient, remainder = divmod(number1, number2)

Here is an example of how divmod() can be used:

result = divmod(8, 3)

print('Quotient and Remainder =', result)

In this example, divmod() returns the tuple (2, 2) representing the quotient (8 // 3 = 2) and the remainder (8 % 3 = 2). The function is useful in situations where you need to calculate both values at once, as it can save computation time by avoiding redundant work.

When working with arrays, you can use NumPy’s divmod() function to perform element-wise quotient and remainder calculations.

Here is an example using NumPy:

import numpy as np x = np.array([10, 20, 30])

y = np.array([3, 5, 7]) quotient, remainder = np.divmod(x, y)

print('Quotient:', quotient)

print('Remainder:', remainder)

In this case, the output will be two arrays, one for the quotients and one for the remainders of the element-wise divisions.

Working with Numbers

In Python, working with numbers, specifically integers, is a common task that every programmer will encounter. The divmod() function is a built-in method that simplifies the process of obtaining both the quotient and the remainder when dividing two numbers. This function is especially useful when working with large datasets or complex calculations that involve integers.

The divmod() function takes two arguments, the dividend and the divisor, and returns a tuple containing the quotient and remainder. The syntax for using this function is as follows:

result = divmod(number1, number2)

Here’s a simple example that demonstrates how to use divmod():

dividend = 10

divisor = 3

result = divmod(dividend, divisor)

print(result) # Output: (3, 1)

In this example, we divide 10 by 3, and the function returns the tuple (3, 1), representing the quotient and remainder, respectively.

An alternative approach to finding the quotient and remainder without using divmod() is to employ the floor division // and modulus % operators. Here’s how you can do that:

quotient = dividend // divisor

remainder = dividend % divisor

print(quotient, remainder) # Output: 3 1

While both methods yield the same result, the divmod() function offers the advantage of calculating the quotient and remainder simultaneously, which can be more efficient in certain situations.

When working with floating-point numbers, the divmod() function can still be applied. However, keep in mind that the results may be less precise due to inherent limitations in representing floating-point values in computers:

dividend_float = 10.0

divisor_float = 3.0

result_float = divmod(dividend_float, divisor_float)

print(result_float) # Output: (3.0, 1.0)

Divmod in Action: Examples

The divmod() function in Python makes it easy to simultaneously obtain the quotient and remainder when dividing two numbers. It returns a tuple that includes both values. Let’s dive into several examples to see how it works.

Consider dividing 25 by 7. Using divmod(), we can quickly obtain the quotient and remainder:

result = divmod(25, 7)

print(result) # Output: (3, 4)

In this case, the quotient is 3, and the remainder is 4.

Now, let’s look at a scenario involving floating-point numbers. The divmod() function can also handle them:

result = divmod(8.5, 2.5)

print(result) # Output: (3.0, 0.5)

Here, we can see that the quotient is 3.0, and the remainder is 0.5.

Another example would be dividing a negative number by a positive number:

result = divmod(-15, 4)

print(result) # Output: (-4, 1)

The quotient is -4, and the remainder is 1.

It’s essential to remember that divmod() does not support complex numbers as input:

result = divmod(3+2j, 2)

# Output: TypeError: can't take floor or mod of complex number.

The Division and Modulo Operators

In Python programming, division and modulo operators are commonly used to perform arithmetic operations on numbers. The division operator (//) calculates the quotient, while the modulo operator (%) computes the remainder of a division operation. Both these operators are an essential part of Python’s numeric toolkit and are often used in mathematical calculations and problem-solving.

The division operator is represented by // and can be used as follows:

quotient = a // b

Here, a is the dividend, and b is the divisor. This operation will return the quotient obtained after dividing a by b.

On the other hand, the modulo operator is represented by % and helps in determining the remainder when a number is divided by another:

remainder = a % b

Here, a is the dividend, and b is the divisor. This operation will return the remainder obtained after dividing a by b.

Let’s take a look at an example:

a = 10

b = 3

quotient = a // b # Result: 3

remainder = a % b # Result: 1

print("Quotient:", quotient, "Remainder:", remainder)

This code snippet computes the quotient and remainder when 10 is divided by 3. The output of this code will be:

Quotient: 3 Remainder: 1

Python also provides a built-in function divmod() for simultaneously computing the quotient and remainder. The divmod() function takes two arguments – the dividend and the divisor – and returns a tuple containing the quotient and the remainder:

result = divmod(10, 3)

print(result) # Output: (3, 1)

Alternative Methods to Divmod

In Python, the divmod() method allows you to easily compute the quotient and remainder of a division operation. However, it’s also worth knowing a few alternatives to the divmod() method for computing these values.

One of the simplest ways to find the quotient and remainder of a division operation without using divmod() is by using the floor division (//) and modulus (%) operators. Here’s an example:

dividend = 10

divisor = 3

quotient = dividend // divisor

remainder = dividend % divisor

print(quotient, remainder) # Output: 3 1

If you want to avoid using the floor division and modulus operators and only use basic arithmetic operations, such as addition and subtraction, you can achieve the quotient and remainder through a while loop. Here’s an example:

dividend = 10

divisor = 3

quotient = 0

temp_dividend = dividend while temp_dividend >= divisor: temp_dividend -= divisor quotient += 1 remainder = temp_dividend

print(quotient, remainder) # Output: 3 1

For finding the quotient and remainder of non-integer values, you may consider using the math module, which provides math.floor() and math.fmod() functions that work with floating-point numbers:

import math dividend = 10.5

divisor = 3.5

quotient = math.floor(dividend / divisor)

remainder = math.fmod(dividend, divisor)

print(quotient, remainder) # Output: 2 3.5

Implementing Divmod in Programs

The divmod() function in Python is a convenient way to obtain both the quotient and the remainder of two numbers. It takes two numbers as arguments and returns a tuple containing the quotient and the remainder.

Here’s a basic example that demonstrates how to use the divmod() function:

numerator = 10

denominator = 3

quotient, remainder = divmod(numerator, denominator)

print("Quotient:", quotient)

print("Remainder:", remainder)

In this example, the divmod() function receives two arguments, numerator and denominator, and returns the tuple (quotient, remainder). The output will be:

Quotient: 3

Remainder: 1

You can also use divmod() in a program that iterates through a range of numbers. For example, if you want to find the quotient and remainder of dividing each number in a range by a specific denominator, you can do the following:

denominator = 3

for num in range(1, 11): quotient, remainder = divmod(num, denominator) print(f"{num} // {denominator} = {quotient}, {num} % {denominator} = {remainder}")

This program will print the quotient and remainder for each number in the range 1 to 10 inclusive, when divided by 3.

When writing functions that require a variable number of arguments, you can use the *args syntax to pass a tuple of numbers to divmod().

Here’s an example:

def custom_divmod(*args): results = [] for num_pair in zip(args[::2], args[1::2]): results.append(divmod(*num_pair)) return results quotients_remainders = custom_divmod(10, 3, 99, 5, 8, 3)

print(quotients_remainders)

In this example, the custom_divmod() function receives a variable number of arguments. The zip() function is used to create pairs of numerators and denominators by slicing the input arguments. The resulting list of quotient-remainder tuples is then returned.

By utilizing the divmod() function in your programs, you can efficiently obtain both the quotient and remainder of two numbers in a single call, making your code more concise and easier to read.

Frequently Asked Questions

How to use divmod function in Python?

The divmod() function in Python is a built-in function that takes two numbers as arguments and returns a tuple containing the quotient and the remainder of the division operation. Here’s an example:

result = divmod(10, 3)

print(result) # Output: (3, 1)

How to find quotient and remainder using divmod?

To find the quotient and remainder of two numbers using divmod(), simply pass the dividend and divisor as arguments to the function. The function will return a tuple where the first element is the quotient and the second element is the remainder:

q, r = divmod(10, 3)

print("Quotient:", q) # Output: 3

print("Remainder:", r) # Output: 1

How does divmod work with negative numbers?

When using divmod() with negative numbers, the function will return the quotient and remainder following the same rules as for positive numbers. However, if either the dividend or the divisor is negative, the result’s remainder will have the same sign as the divisor:

result = divmod(-10, 3)

print(result) # Output: (-4, 2)

How to perform division and modulo operations simultaneously?

By using the divmod() function, you can perform both division and modulo operations in a single step, as it returns a tuple containing the quotient and the remainder:

result = divmod(10, 3)

print("Quotient and Remainder:", result) # Output: (3, 1)

Is there a divmod equivalent in other languages?

While not all programming languages have a function named “divmod,” most languages provide a way to perform integer division and modulo operations. For example, in JavaScript, you can use the following code to obtain similar results:

let dividend = 10;

let divisor = 3; let quotient = Math.floor(dividend / divisor);

let remainder = dividend % divisor;

console.log(`Quotient: ${quotient}, Remainder: ${remainder}`); // Output: Quotient: 3, Remainder: 1

What are the differences between divmod and using // and %?

Using divmod() is more efficient when you need both the quotient and remainder, as it performs the calculation in a single step. However, if you only need the quotient or the remainder, you can use the floor division // operator for the quotient and the modulo % operator for the remainder:

q = 10 // 3

r = 10 % 3

print("Quotient:", q) # Output: 3

print("Remainder:", r) # Output: 1

Recommended: Python Programming Tutorial [+Cheat Sheets]

The post Python – Get Quotient and Remainder with divmod() appeared first on Be on the Right Side of Change.

If you want to upload a file using AJAX and also need to show a progress bar during the upload, you have landed on the right page.

This article has an example code for JavaScript AJAX file upload with a progress bar.

An AJAX-based file upload is a repeatedly needed requirement for a web application.

It is for providing an inline editing feature with the uploaded file content. For example, the following tasks can be achieved using the AJAX file upload method.

1. Photo or banner update on the profile page.
2. Import CSV or Excel files to load content to the data tables.

View demo

HTML upload form

This HTML shows the input to choose a file. This form has a button that maps its click event with an AJAX handler.

In a previous tutorial, we have seen a jQuery example for uploading form data with a chosen file binary.

But in this example, the HTML doesn’t have any form container. Instead, the form data is created by JavaScript before processing the AJAX.

This HTML has a container to show the progress bar. Once the progress is 100% complete, a success message is added to the UI without page refresh.

<div class="phppot-container tile-container text-center"> <h2>AJAX File Upload with Progress Bar using JavaScript</h2> <input type="file" id="fileUpload" /> <br> <br> <button on‌click="uploadFile()">Upload</button> <div class="progress"> <div class="progress-bar" id="progressBar"></div> </div> <br> <div id="uploadStatus"></div>

</div>

AJAX file upload request with progress bar

This section is the core of this example code. This example’s HTML and PHP files are prevalent, as seen in other file upload examples.

The script below follows the steps to achieve the AJAX file upload.

1. It reads the file binary chosen in the file input field.
2. It instantiates JavaScript FormData and appends the file binary into it.
3. It creates an XMLHttpRequest handle.
4. This handle uses the ‘upload’ property to get XMLHttpRequestUpload object.
5. This XMLHttpRequestUpload object tracks the upload progress in percentage.
6. It creates event listeners to update the progressing percentage and the upload status.
7. Then finally, it posts the file to the PHP endpoint like usual AJAX programming.

function uploadFile() { var fileInput = document.getElementById('fileUpload'); var file = fileInput.files[0]; if (file) { var formData = new FormData(); formData.append('file', file); var xhr = new XMLHttpRequest(); xhr.upload.addEventListener('progress', function (event) { if (event.lengthComputable) { var percent = Math.round((event.loaded / event.total) * 100); var progressBar = document.getElementById('progressBar'); progressBar.style.width = percent + '%'; progressBar.innerHTML = percent + '%'; } }); xhr.addEventListener('load', function (event) { var uploadStatus = document.getElementById('uploadStatus'); uploadStatus.innerHTML = event.target.responseText; }); xhr.open('POST', 'upload.php', true); xhr.send(formData); }

}

PHP endpoint to move the uploaded file into a directory

This PHP  has a standard code to store the uploaded file in a folder using the PHP move_uploaded_file(). The link has the code if you want to store the uploaded file and save the path to the database.

This endpoint creates a unique name for the filename before upload. It is a good programming practice, but the code will work without it, also.

It is for stopping file overwriting in case of uploading different files in the same name.

Note: Create a folder named “uploads” in the project root. Give sufficient write permissions.

<?php if ($_SERVER['REQUEST_METHOD'] === 'POST' && isset($_FILES['file'])) { $file = $_FILES['file']; // file will be uploaded to the following folder // you should give sufficient file permissions $uploadDir = 'uploads/'; // unique file name generated $fileName = uniqid() . '_' . $file['name']; // moving the uploaded file from temp location to our target location if (move_uploaded_file($file['tmp_name'], $uploadDir . $fileName)) { echo 'File uploaded successfully.'; } else { echo 'Failed to upload file.'; }

}

View demo Download

↑ Back to Top

Understanding Collections.Counter

Python’s Collections Module

The collections module in Python contains various high-performance container datatypes that extend the functionality of built-in types such as list, dict, and tuple. These datatypes offer more specialized tools to efficiently handle data in memory. One of the useful data structures from this module is Counter.

Counter Class in Collections

Counter is a subclass of the dictionary that allows you to count the frequency of elements in an iterable. Its primary purpose is to track the number of occurrences of each element in the iterable. This class simplifies the process of counting items in tuples, lists, dictionaries, sets, strings, and more.

Here’s a basic example of using the Counter class:

from collections import Counter count = Counter("hello")

print(count)

This example would output:

Counter({'l': 2, 'h': 1, 'e': 1, 'o': 1})

The Counter class provides a clear and pythonic way to perform element counting tasks. You can easily access the count of any element using the standard dictionary syntax:

print(count['l'])

This would return:

2

In conclusion, the collections.Counter class is an invaluable tool for handling and analyzing data in Python. It offers a straightforward and efficient solution to count elements within iterables, enhancing the standard functionality provided by the base library.

Working with Collections.Counter

The collections.Counter class helps maintain a dictionary-like structure that keeps track of the frequency of items in a list, tuple, or string. In this section, we will discuss how to create and update a collections.Counter object.

Creating a Counter

To get started with collections.Counter, you need to import the class from the collections module. You can create a counter object by passing an iterable as an argument:

from collections import Counter my_list = ['a', 'b', 'a', 'c', 'a', 'b']

counter = Counter(my_list) print(counter)

Output:

Counter({'a': 3, 'b': 2, 'c': 1})

In this example, the Counter object, counter, counts the occurrences of each element in my_list. The result is a dict-like structure where the keys represent the elements and the values represent their counts.

Updating a Counter

You can update the counts in a Counter object by using the update() method. This method accepts an iterable or a dictionary as its argument:

counter.update(['a', 'c', 'd'])

print(counter)

Output:

Counter({'a': 4, 'b': 2, 'c': 2, 'd': 1})

In the above example, we updated the existing Counter object with a new list of elements. The resulting counts now include the updated elements.

You can also update a Counter with a dictionary whose keys are elements and values are the desired updated counts:

counter.update({'a': 1, 'd': 5})

print(counter)

Output:

Counter({'a': 5, 'd': 6, 'b': 2, 'c': 2})

In this case, we provided a dictionary to the update() method, and the counts for the ‘a’ and ‘d’ elements were increased accordingly.

Working with collections.Counter is efficient and convenient for counting items in an iterable while maintaining a clear and concise code structure. By leveraging the methods to create and update Counter objects, you can effectively manage frequencies in a dict-like format for various data processing tasks.

Counting Elements in a List

There are several ways to count the occurrences of elements in a list. One of the most efficient approaches is to use the collections.Counter class from the collections module. This section will explore two main methods: using list comprehensions and utilizing Counter class methods.

List Comprehensions

List comprehensions offer a concise and readable way to count elements in a list. For example, let’s assume we have a list of numbers and want to count how many times each number appears in the list:

numbers = [1, 2, 3, 2, 1, 3, 1, 1, 2, 3]

unique_numbers = set(numbers)

count_dict = {num: numbers.count(num) for num in unique_numbers}

print(count_dict)

The output would be: {1: 4, 2: 3, 3: 3}

In this example, we first create a set of unique numbers and then use a list comprehension to create a dictionary with the counts of each unique element in the list.

Using Counter Class Methods

The collections.Counter class provides an even more efficient way to count elements in a list. The Counter class has a constructor that accepts an iterable and returns a dictionary-like object containing the counts of each element.

Here’s an example using the same list of numbers:

from collections import Counter numbers = [1, 2, 3, 2, 1, 3, 1, 1, 2, 3]

counter = Counter(numbers)

print(counter)

The output would be: Counter({1: 4, 2: 3, 3: 3})

In addition to the constructor, the Counter class also provides other methods to work with counts, such as most_common() which returns a list of the n most common elements and their counts:

most_common_elements = counter.most_common(2)

print(most_common_elements)

The output would be: [(1, 4), (2, 3)]

In summary, counting elements in a list can be achieved using list comprehensions or by employing the collections.Counter class methods. Both techniques can provide efficient and concise solutions to count elements in a Python list.

Counter Methods and Usage

In this section, we will explore the various methods and usage of the collections.Counter class in Python. This class is a versatile tool for counting the occurrences of elements in an iterable, such as lists, strings, and tuples.

Keys and Values

Counter class in Python is a subclass of the built-in dict class, which means it provides methods like keys() and values() to access the elements and their counts. To obtain the keys (unique elements) and their respective values (counts), use the keys() and values() methods, respectively.

from collections import Counter my_list = ['a', 'b', 'a', 'c', 'c', 'c']

counter = Counter(my_list) # Access keys and values

print(counter.keys()) # Output: dict_keys(['a', 'b', 'c'])

print(counter.values()) # Output: dict_values([2, 1, 3])

Most Common Elements

The most_common() method returns a list of tuples containing the elements and their counts in descending order. This is useful when you need to identify the most frequent elements in your data.

from collections import Counter my_list = ['a', 'b', 'a', 'c', 'c', 'c']

counter = Counter(my_list) # Get most common elements

print(counter.most_common()) # Output: [('c', 3), ('a', 2), ('b', 1)]

You can also pass an argument to most_common() to return only a specific number of top elements.

print(counter.most_common(2)) # Output: [('c', 3), ('a', 2)]

Subtracting Elements

The subtract() method allows you to subtract the counts of elements in another iterable from the current Counter. This can be helpful in comparing and analyzing different datasets.

from collections import Counter my_list1 = ['a', 'b', 'a', 'c', 'c', 'c']

my_list2 = ['a', 'c', 'c']

counter1 = Counter(my_list1)

counter2 = Counter(my_list2) # Subtract elements

counter1.subtract(counter2)

print(counter1) # Output: Counter({'c': 1, 'a': 1, 'b': 1})

In this example, the counts of elements in my_list2 are subtracted from the counts of elements in my_list1. The counter1 object is updated to reflect the new counts after subtraction.

Working with Different DataTypes

In this section, we’ll explore how to use Python’s collections.Counter to count elements in various data types, such as strings and tuples.

Counting Words in a String

Using collections.Counter, we can easily count the occurrence of words in a given string. First, we need to split the string into words, and then pass the list of words to the Counter object.

Here’s an example:

from collections import Counter text = "This is a sample text. This text is just a sample."

words = text.split() word_count = Counter(words)

print(word_count)

This would output a Counter object showing the frequency of each word in the input string:

Counter({'This': 2, 'is': 2, 'a': 2, 'sample': 2, 'text.': 1, 'text': 1, 'just': 1})

Counting Elements in Tuples

collections.Counter is also handy when you want to count the occurrences of elements in a tuple. Here’s a simple example:

from collections import Counter my_tuple = (1, 5, 3, 2, 1, 5, 3, 1, 1, 2, 2, 3, 3, 3) element_count = Counter(my_tuple)

print(element_count)

This code would output a Counter object showing the count of each element in the tuple:

Counter({3: 5, 1: 4, 2: 3, 5: 2})

As you can see, using collections.Counter makes it easy to count elements in different data types like strings and tuples in Python. Remember to import the Counter class from the collections module.

Advanced Topics and Additional Functions

Negative Values in Counter

The collections.Counter can handle negative values as well. It means that the count of elements can be negative, zero, or positive integers.

Let’s see an example with negative values:

from collections import Counter count_1 = Counter(a=3, b=2, c=0, d=-1)

print(count_1)

Output:

Counter({'a': 3, 'b': 2, 'c': 0, 'd': -1})

As you can see, the Counter object shows negative, zero, and non-negative occurrences of elements. Remember that negative counts do not affect the total number of elements.

Working with Unordered Collections

When you use collections.Counter to count elements in a list, tuple, or string, you don’t need to worry about the order of the elements. With Counter, you can count the occurrences of elements in any iterable without considering the sequence of the contained items.

Here’s an example for counting elements in an unordered collection:

from collections import Counter unordered_list = [12, 5, 3, 5, 3, 7, 12, 3] count_result = Counter(unordered_list)

print(count_result)

Output:

Counter({3: 3, 12: 2, 5: 2, 7: 1})

As demonstrated, the Counter function efficiently calculates element occurrences, regardless of the input order. This behavior makes it a practical tool when working with unordered collections, thus simplifying element frequency analysis.

Frequently Asked Questions

How do you use Collections.Counter to count elements in a list?

To use collections.Counter to count elements in a list, you first need to import the Counter class from the collections module. Then, create a Counter object by passing the list as an argument to the Counter() function. Here’s an example:

from collections import Counter my_list = ['apple', 'banana', 'apple', 'orange', 'banana', 'apple']

counter = Counter(my_list)

print(counter)

This will output:

Counter({'apple': 3, 'banana': 2, 'orange': 1})

What is the syntax for importing collections.Counter in Python?

To import the Counter class from the collections module in Python, use the following syntax:

from collections import Counter

How can you sort a Counter object by key?

To sort a Counter object by its keys, you can use the sorted() function in combination with the items() method:

sorted_counter = dict(sorted(counter.items()))

print(sorted_counter)

How do you convert a Python Counter object to a dictionary?

Converting a Counter object to a regular Python dictionary is as simple as passing the Counter object to the dict() function:

counter_dict = dict(counter)

print(counter_dict)

What are some examples of using collections.Counter in Python?

collections.Counter is versatile and can be used with different types of iterables, including lists, tuples, and strings. Here are some examples:

# Count characters in a string

text = "hello world"

char_counter = Counter(text)

print(char_counter) # Count items in a tuple

my_tuple = (1, 2, 3, 2, 1, 3, 1, 1, 2)

tuple_counter = Counter(my_tuple)

print(tuple_counter)

How can you update a Counter object in Python?

You can update a Counter object with new elements by using the update() method. Here’s an example:

counter = Counter({'apple': 3, 'banana': 2, 'orange': 1})

new_items = ['apple', 'banana', 'grape']

counter.update(new_items)

print(counter)

This will update the counts for the existing elements and add new elements if they were not already present:

Counter({'apple': 4, 'banana': 3, 'orange': 1, 'grape': 1})

Recommended: Python Container Types: A Quick Guide

The post Collections.Counter: How to Count List Elements (Python) appeared first on Be on the Right Side of Change.

So, you wrote your Quarkus application, and now you want to deploy it to a Kubernetes cluster. Good news: Deploying a Quarkus application to a Kubernetes cluster is easy. Before you do this, though, you need to straighten out your application’s properties. After all, your app probably has to connect with a database, call other services, and so on. These settings are already defined in your application.properties file, but the values match the ones for your local environment and won’t work once deployed onto your cluster.

So, how do you easily solve this problem? Let’s walk through an example.

Create the example Quarkus application

Instead of using a complex example, let’s take a simple use case that explains the concept well. Start by creating a new Quarkus app:

$ mvn io.quarkus:quarkus-maven-plugin:1.1.1.Final:create

You can keep all of the default values while creating the new application. In this example, the application is named hello-app. Now, open the HelloResource.java file and refactor it to look like this:

@Path("/hello") public class HelloResource { @ConfigProperty(name = "greeting.message") String message; @GET @Produces(MediaType.TEXT_PLAIN) public String hello() { return "hello " + message; } } 

In your application.properties file, now add greeting.message=localhost. The @ConfigProperty annotation is not in the scope of this article, but here we can see how easy it is to inject properties inside our code using this annotation.

Now, let’s start our application to see if it works as expected:

$ mvn compile quarkus:dev

Browse to http://localhost:8080/hello, which should output hello localhost. That’s it for the Quarkus app. It’s ready to go.

Deploy the application to the Kubernetes cluster

The idea here is to deploy this application to our Kubernetes cluster and replace the value of our greeting property with one that will work on the cluster. It is important to know here that all of the properties from application.properties are exposed, and thus can be overridden with environment variables. The convention is to convert the name of the property to uppercase and replace every dot (.) with an underscore (_). So, for instance, our greeting.message will become GREETING_MESSAGE.

At this point, we are almost ready to deploy our app to Kubernetes, but we need to do three more things:

1. Create a Docker image of your application and push it to a repository that your cluster can access.
2. Define a ConfgMap resource.
3. Generate the Kubernetes resources for our application.

To create the Docker image, simply execute this command:

$ docker build -f src/main/docker/Dockerfile.jvm -t quarkus/hello-app .

Be sure to set the right Docker username and to also push to an image registry, like docker-hub or quay. If you are not able to push an image, you can use sebi2706/hello-app:latest.

Next, create the file config-hello.yml:

apiVersion: v1 data: greeting: "Kubernetes" kind: ConfigMap metadata: name: hello-config 

Make sure that you are connected to a cluster and apply this file:

$ kubectl apply -f config-hello.yml

Quarkus comes with a useful extension, quarkus-kubernetes, that generates the Kubernetes resources for you. You can even tweak the generated resources by providing extra properties—for more details, check out this guide.

After installing the extension, add these properties to our application.properties file so it generates extra configuration arguments for our containers specification:

kubernetes.group=yourDockerUsername kubernetes.env-vars[0].name=GREETING_MESSAGE kubernetes.env-vars[0].value=greeting kubernetes.env-vars[0].configmap=hello-config

Run mvn package and view the generated resources in target/kubernetes. The interesting part is in spec.containers.env:

- name: "GREETING_MESSAGE"   valueFrom:   configMapKeyRef:     key: "greeting"    name: "hello-config"

Here, we see how to pass an environment variable to our container with a value coming from a ConfigMap. Now, simply apply the resources:

$ kubectl apply -f target/kubernetes/kubernetes.yml

Expose your service:

kubectl expose deployment hello --type=NodePort

Then, browse to the public URL or do a curl. For instance, with Minikube:

$ curl $(minikube service hello-app --url)/hello

This command should output: hello Kubernetes.

Conclusion

Now you know how to use a ConfigMap in combination with environment variables and your Quarkus’s application.properties. As we said in the introduction, this technique is particularly useful when defining a DB connection’s URL (like QUARKUS_DATASOURCE_URL) or when using the quarkus-rest-client (ORG_SEBI_OTHERSERVICE_MP_REST_URL).

The post Using Kubernetes ConfigMaps to define your Quarkus application’s properties appeared first on Red Hat Developer.

What is a LLM?

From a technical standpoint, a large language model (LLM) can be seen as a massive file on a computer, containing billions or even trillions of numerical values, known as parameters. These parameters are fine-tuned through extensive training on diverse datasets, capturing the statistical properties of human language.

However, such a dry description hardly does justice to the magic of LLMs. From another perspective, they function almost like an oracle. You call upon them with a query, such as llm("What is the purpose of life"), and they may respond with something witty, insightful, or enigmatic, like "42" (a humorous nod to Douglas Adams’ The Hitchhiker’s Guide to the Galaxy).

By the way, you can check out my article on using LLMs like this in the command line here:

Recommended: How to Run Large Language Models (LLMs) in Your Command Line?

Isn’t it wild to think about how Large Language Models (LLMs) can turn math into something almost magical? It’s like they’re blending computer smarts with human creativity, and the possibilities are just getting started.

Now, here’s where it gets really cool.

These LLMs take all kinds of complex patterns and knowledge and pack them into binary files full of numbers. We don’t really understand what these numbers represent but together they encode a deep understanding of the world. LLMs are densely compressed human wisdom, knowledge, and intelligence. Now imagine having these files and being able to copy them millions of times, running them all at once.

It’s like having a huge team of super-smart people, but they’re all in your computer.

So picture this: Millions of brainy helpers in your pocket, working day and night on anything you want.

You know how doctors are always trying to figure out the best way to treat illnesses? Imagine having millions of super-smart helpers to quickly find the answers.

Or think about your savings and investments; what if you had a team of top financial experts guiding you 24/7 to make the smartest choices with your money?

And for kids in school, picture having a personal tutor for every subject, making sure they understand everything perfectly. LLMs is like having an army of geniuses at your service for anything you need.

LLMs, what Willison calls alien technology, have brought us closer to solving the riddle of intelligence itself, turning what was once the exclusive domain of human cognition into something that can be copied, transferred, and harnessed like never before.

I’d go as far as to say that the age-old process of reproducing human intelligence has been transcended. Intelligence is solved. LLMs will only become smarter from now on. Like the Internet, LLMs will stay and proliferate and penetrate every single sector of our economy.

How Do LLMs Work?

The underlying mechanism of Large Language Models (LLMs) might seem almost counterintuitive when you delve into how they operate. At their core, LLMs are essentially word-prediction machines, fine-tuned to anticipate the most likely next word (more precisely: token) in a sequence.

For example consider ChatGPT’s LLM chat interface that has reached product market fit and is used by hundreds of millions of users. The ingenious “hack” that allows LLMs to participate in a chat interface is all about how the input is framed. In essence, the model isn’t inherently conversing with a user; it’s continuing a text, based on a conversational pattern it has learned from vast amounts of data.

Consider this simplified example:

You are a helpful assistant User: What is the purpose of life?
Assistant: 42
User: Can you elaborate?
Assistant:

Here’s what’s happening under the hood:

1. Setting the Scene: The introductory line, "You are a helpful assistant" sets a context for the LLM. It provides an instruction to guide its responses, influencing its persona.
2. User Input: The following lines are framed as a dialogue, but to the LLM, it’s all part of a text it’s trying to continue. When the user asks, "What is the purpose of life?" the LLM looks at this as the next part of a story, or a scene in a play, and attempts to predict the next word or phrase that makes the most sense.
3. Assistant Response: The assistant’s response, "42" is the model’s guess for the next word, given the text it has seen so far. It’s a clever completion, reflecting the model’s training on diverse and creative texts. In the second run, however, the whole conversation is used as input and the LLM just completes the conversation.
4. Continuing the Conversation: When the user follows up with "Can you elaborate?" the LLM is once again seeking to continue the text. It’s not consciously leading a conversation but following the statistical patterns it has learned, which, in this context, would typically lead to an elaboration.

The magic is in how all these elements come together to create an illusion of a conversation. In reality, the LLM doesn’t understand the conversation or its participants. It’s merely predicting the next word, based on an intricately crafted pattern.

This “dirty little hack” transforms a word-prediction engine into something that feels interactive and engaging, demonstrating the creative application of technology and the power of large-scale pattern recognition. It’s a testament to human ingenuity in leveraging statistical learning to craft experiences that resonate on a very human level.

Prompt Engineering is a clever technique used to guide the behavior of Large Language Models (LLMs) by crafting specific inputs, or prompts, that steer the model’s responses. It’s akin to creatively “hacking” the model to generate desired outputs.

For example, if you want the LLM to act like a Shakespearean character, you might begin with a prompt like "Thou art a poet from the Elizabethan era". The model, recognizing the pattern and language style, will respond in kind, embracing a Shakespearean tone.

This trickery through carefully designed prompts transforms a word-prediction machine into a versatile and interactive tool that can mimic various styles and tones, all based on how you engineer the initial prompt.

Prompt Engineering with Python and OpenAI

You can check out the whole course on OpenAI Prompt Engineering using Python on the Finxter academy. We cover topics such as:

• Embeddings
• Semantic search
• Web scraping
• Query embeddings
• Movie recommendation
• Sentiment analysis

Academy: Prompt Engineering with Python and OpenAI

What’s the Secret of LLMs?

The secret to the magical capabilities of Large Language Models (LLMs) seems to lie in a simple and perhaps surprising element: scale.

The colossal nature of these models is both their defining characteristic and the key to their unprecedented performance.

Tech giants like Meta, Google, and Microsoft have dedicated immense resources to developing LLMs. How immense? We’re talking about millions of dollars spent on cutting-edge computing power and terabytes of textual data to train these models. It’s a gargantuan effort that converges in a matrix of numbers — the model’s parameters — that represent the learned patterns of human language.

The scale here isn’t just large; it’s virtually unprecedented in computational history. These models consist of billions or even trillions of parameters, fine-tuned across diverse and extensive textual datasets. By throwing such vast computational resources at the problem, these corporations have been able to capture intricate nuances and create models that understand and generate human-like text.

However, this scale comes with challenges, including the enormous energy consumption of training such models, the potential biases embedded in large-scale data, and the barrier to entry for smaller players who can’t match the mega corporations’ resources.

The story of LLMs is a testament to the “bigger is better” philosophy in the world of artificial intelligence. It’s a strategy that seems almost brute-force in nature but has led to a qualitative leap in machine understanding of human language. It illustrates the power of scale, paired with ingenuity and extensive resources, to transform a concept into a reality that pushes the boundaries of what machines can achieve.

Attention Is All You Need

The 2017 paper by Google “Attention is All You Need” marked a significant turning point in the world of artificial intelligence. It introduced the concept of transformers, a novel architecture that is uniquely scalable, allowing training to be run across many computers in parallel both efficiently and easily.

This was not just a theoretical breakthrough but a practical realization that the model could continually improve with more and more compute and data.

Key Insight: By using unprecedented amount of compute on unprecedented amount of data on a simple neural network architecture (transformers), intelligence seems to emerge as a natural phenomenon.

Unlike other algorithms that may plateau in performance, transformers seemed to exhibit emerging properties that nobody fully understood at the time. They could understand intricate language patterns, even developing coding-like abilities. The more data and computational power thrown at them, the better they seemed to perform. They didn’t converge or flatten out in effectiveness with increased scale, a behavior that was both fascinating and mysterious.

OpenAI, under the guidance of Sam Altman, recognized the immense potential in this architecture and decided to push it farther than anyone else. The result was a series of models, culminating in state-of-the-art transformers, trained on an unprecedented scale. By investing in massive computational resources and extensive data training, OpenAI helped usher in a new era where large language models could perform tasks once thought to be exclusively human domains.

This story highlights the surprising and yet profound nature of innovation in AI.

A simple concept, scaled to extraordinary levels, led to unexpected and groundbreaking capabilities. It’s a reminder that sometimes, the path to technological advancement isn’t about complexity but about embracing a fundamental idea and scaling it beyond conventional boundaries. In the case of transformers, scale was not just a means to an end but a continually unfolding frontier, opening doors to capabilities that continue to astonish and inspire.

Ten Tips to Use LLMs Effectively

As powerful and versatile as Large Language Models (LLMs) are, harnessing their full potential can be a complex endeavor.

Here’s a series of tricks and insights to help tech enthusiasts like you use them effectively:

1. Accept that No Manual Exists: There’s no step-by-step guide to mastering LLMs. The field is still relatively new, and best practices are continually evolving. Flexibility and a willingness to experiment are essential.
2. Iterate and Refine: Don’t reject the model’s output too early. Your first output might not be perfect, but keep iterating. Anyone can get an answer from an LLM, but extracting good answers requires persistence and refinement. You can join our prompt engineering beginner and expert courses to push your own understanding to the next level.
3. Leverage Your Domain Knowledge: If you know coding, use LLMs to assist with coding tasks. If you’re a marketer, apply them for content generation. Your expertise in a particular area will allow you to maximize the model’s capabilities.
4. Understand How the Model Works: A rough understanding of the underlying mechanics can be immensely beneficial. Following tech news, like our daily Finxter emails, can keep you informed and enhance your ability to work with LLMs.
5. Gain Intuition by Experimenting: Play around with different prompts and settings. Daily hands-on practice can lead to an intuitive feel for what works and what doesn’t.
6. Know the Training Cut-off Date: Different models have different cut-off dates. For example, OpenAI’s GPT-3.5 models were trained until September 2021, while Claude 2 Anthropic and Google PaLM 2 are more recent. This can affect the accuracy and relevance of the information they provide.
7. Understand Context Length: Models have limitations on the number of tokens (words, characters, spaces) they can handle. It’s 4000 tokens for GPT-3, 8000 for GPT-4, and 100k for Claude 2. Tailoring your input to these constraints will yield better results.
8. Develop a “Sixth Sense” for Hallucinations: Sometimes, LLMs may generate information that seems plausible but is incorrect or hallucinated. Developing an intuition for recognizing and avoiding these instances is key to reliable usage.
9. Stay Engaged with the Community: Collaborate with others, join forums, and stay abreast of the latest developments. The collective wisdom of the community is a powerful asset in mastering these technologies.
10. Be Creative: Prompt the model for creative ideas (e.g., "Give me 20 ideas on X"). The first answers might be obvious, but further down the list, you might find a spark of brilliance.

Retrieval Augmented Generation

Retrieval Augmented Generation (RAG) represents an intriguing intersection between the vast capabilities of Large Language Models (LLMs) and the power of information retrieval. It’s a technique that marries the best of both worlds, offering a compelling approach to generating information and insights.

Here’s how it works and why it’s making waves in the tech community:

What is Retrieval Augmented Generation?

RAG is a method that, instead of directly training a model on specific data or documents, leverages the vast information already available on the internet. By searching for relevant content, it pulls this information together and uses it as a foundation for asking an LLM to generate an answer.

How Does RAG Work?

1. Search for Information: First, a search is conducted for content relevant to the query or task at hand. This could involve scouring databases, the web, or specialized repositories.
2. Prepend the Retrieved Data: The content found is then prepended to the original query or prompt. Essentially, it’s added to the beginning of the question or task you’re posing to the LLM.
3. Ask the Model to Answer: With this combined prompt, the LLM is then asked to generate an answer or complete the task. The prepended information guides the model’s response, grounding it in the specific content retrieved.

Why is RAG Valuable?

• Customization: It allows for tailored responses based on real-world data, not just the general patterns an LLM has learned from its training corpus.
• Efficiency: Rather than training a specialized model, which can be costly and time-consuming, RAG leverages existing models and augments them with relevant information.
• Flexibility: It can be applied to various domains, from coding to medical inquiries, by merely adapting the retrieval component to the area of interest.
• Quality: By guiding the model with actual content related to the query, it often results in more precise and contextually accurate responses.

Retrieval Augmented Generation represents an elegant solution to some of the challenges in working with LLMs. It acknowledges that no model, no matter how large, can encapsulate the entirety of human knowledge. By dynamically integrating real-time information retrieval, RAG opens new horizons for LLMs, making them even more versatile and responsive to specific and nuanced inquiries.

In a world awash with information, the fusion of search and generation through RAG offers a sophisticated tool for navigating and extracting value. Here’s my simple formula for RAG:

USEFULNESS ~ LLM_CAPABILITY * CONTEXT_DATA or more simply: 

USEFULNESS ~ Intelligence * Information

Let’s examine an advanced and extremely powerful technique to provide helpful context to LLMs and, thereby, get the most out of it:

Embeddings and Vector Search: A Special Case of Retrieval Augmented Generation (RAG)

In the broader context of RAG, a specialized technique called “Embeddings and Vector Search” takes text-based exploration to a new level, allowing for the construction of semantic search engines that leverage the capabilities of LLMs.

Here’s how it works:

Transforming Text into Embeddings

1. Text to Vector Conversion: Any string of text, be it a sentence, paragraph, or document, can be transformed into an array of floating-point numbers, or an “embedding”. This embedding encapsulates the semantic meaning of the text based on the LLM’s mathematical model of human language.
2. Dimensionality: These embeddings are positioned in a high-dimensional space, e.g., 1,536 dimensions. Each dimension represents a specific aspect of the text’s semantic content, allowing for a nuanced representation.

Example: Building a Semantic Search Engine

1. Cosine Similarity Distance: To find the closest matches to a given query, the cosine similarity distance between vectors is calculated. This metric measures how closely the semantic meanings align between the query and the existing embeddings.
2. Combining the Brain (LLM) with Application Data (Embedding): By pairing the vast understanding of language embedded in LLMs with specific application data through embeddings, you create a bridge between generalized knowledge and specific contexts.
3. Retrieval and Augmentation: The closest matching embeddings are retrieved, and the corresponding text data is prepended to the original query. This process guides the LLM’s response, just as in standard RAG.

Why is this Technique Important?

You can use embeddings as input to LLM prompts to provide context in a highly condensed and efficient form. This solves one half of the problem of using LLMs effectively!

• Precision: It offers a finely-tuned mechanism for retrieving content that semantically resonates with a given query.
• Scalability: The method can be applied to vast collections of text, enabling large-scale semantic search engines.
• Customization: By building embeddings from specific data sources, the search process can be tailored to the unique needs and contexts of different applications.

Embeddings are a powerful extension of the RAG paradigm, enabling a deep, semantic understanding of text. By translating text into numerical vectors and leveraging cosine similarity, this technique builds bridges between the abstract mathematical understanding of language within LLMs and the real-world applications that demand precise, context-aware responses.

Using embeddings in OpenAI is as simple as running the following code:

response = openai.Embedding.create( input="Your text string goes here", model="text-embedding-ada-002"

)

embeddings = response['data'][0]['embedding']

Possible output:

{ "data": [ { "embedding": [ -0.006929283495992422, -0.005336422007530928, ... -4.547132266452536e-05, -0.024047505110502243 ], "index": 0, "object": "embedding" } ], "model": "text-embedding-ada-002", "object": "list", "usage": { "prompt_tokens": 5, "total_tokens": 5 }
}

If you want to dive deeper into embeddings, I recommend checking out our blog post and the detailed OpenAI guide!

Recommended: What Are Embeddings in OpenAI?

ChatGPT Plugins

OpenAI has recently announced the initial support for plugins in ChatGPT. As part of the gradual rollout of these tools, the intention is to augment language models with capabilities that extend far beyond their existing functionalities.

ChatGPT plugins are tools specifically designed for language models to access up-to-date information, run computations, or use third-party services such as Expedia, Instacart, Shopify, Slack, Wolfram, and more.

The implementation of plugins opens up a vast range of possible use cases. From giving parents superpowers with Milo Family AI to enabling restaurant bookings through OpenTable, the potential applications are expansive. Examples like searching for flights with KAYAK or ordering groceries from local stores via Instacart highlight the practical and innovative utilization of these plugins.

OpenAI is also hosting two plugins, a web browser and a code interpreter (see below) to broaden the model’s reach and increase its functionality. An experimental browsing model will allow ChatGPT to access recent information from the internet, further expanding the content it can discuss with users.

Recommended: Top 5 LLM Python Libraries Like OpenAI, LangChain, Pinecone

ChatGPT Code Interpreter: What Is It and How Does It Work?

The ChatGPT Code Interpreter is a revolutionary feature added to OpenAI’s GPT-4 model, enabling users to execute Python code within the ChatGPT environment.

It functions as a sandboxed Python environment where tasks ranging from PDF conversion using OCR to video trimming and mathematical problem-solving can be carried out.

Users can upload local files in various formats, including TXT, PDF, JPEG, and more, as the Code Interpreter offers temporary disk space and supports over 300 preinstalled Python packages.

Whether it’s data analysis, visualization, or simple file manipulations, the Code Interpreter facilitates these actions within a secure, firewalled environment, transforming the chatbot into a versatile computing interface.

Accessible to ChatGPT Plus subscribers, this feature amplifies the range of possibilities for both coders and general users, blending natural language interaction with direct code execution.

Here’s a list of tasks that can be solved by Code Interpreter that were previously solved by specialized data scientists:

1. Explore Your Data: You can upload various data files and look into them. It’s a handy way to see what’s going on with your numbers.
2. Clean Up Your Data: If your data’s a little messy, you can tidy it up by removing duplicates or filling in missing parts.
3. Create Charts and Graphs: Visualize your data by making different types of charts or graphs. It’s a straightforward way to make sense of complex information.
4. Try Out Machine Learning: Build your own machine learning models to predict outcomes or categorize information. It’s a step into the more advanced side of data handling.
5. Work with Text: Analyze texts to find out what’s being said or how it’s being expressed. It’s an interesting dive into natural language processing.
6. Convert and Edit Files: Whether it’s PDFs, images, or videos, you can convert or modify them as needed. It’s quite a practical feature.
7. Gather Data from Websites: You can pull data directly from web pages, saving time on collecting information manually.
8. Solve Mathematical Problems: If you have mathematical equations or problems, you can solve them here. It’s like having a calculator that can handle more complex tasks.
9. Experiment with Algorithms: Write and test your algorithms for various purposes. It’s a useful way to develop custom solutions.
10. Automate Tasks: If you have repetitive or routine tasks, you can write scripts to handle them automatically.
11. Edit Images and Videos: Basic editing of images and videos is possible, allowing for some creative applications.
12. Analyze IoT Device Data: If you’re working with Internet of Things (IoT) devices, you can analyze their data in this environment.

Here’s an example run in my ChatGPT environment:

Yay you can now run Python code and plot scripts in your ChatGPT environment!

If you click on the “Show work” button above, it toggles the code that was executed:

A simple feature but powerful — using ChatGPT has now become even more convincing for coders like you and me.

To keep learning about OpenAI and Python, you can download our cheat sheet here:

Recommended: Python OpenAI API Cheat Sheet (Free)

Resources:

• https://openai.com/blog/chatgpt-plugins
• https://simonwillison.net/2023/Aug/3/weird-world-of-llms/
• https://beebom.com/chatgpt-code-interpreter/
• https://chat.openai.com

The post Alien Technology: Catching Up on LLMs, Prompting, ChatGPT Plugins & Embeddings appeared first on Be on the Right Side of Change.