6 min read 

How to read data from a .csv file and add its column or row to the list? There are three main ways: 

• Option 1 (the quickest): use the standard library 
• Option 2 (the most preferred): use pandas.read_csv() 
• Option 3 (optional): use csv.reader() 

Short answer 

The simplest option to read a .csv file into a list is to use it with open(“file”) as f: and apply the actions you need. You should also remember that such an approach has its limitations, as you’ll see in the tutorial. 

Prerequisites 

To read the .csv file, I used the following tools: 

• Python 3.8
• PyCharm IDE for convenient coding experience 
• Sublime Text Editor for a manual check of the .csv file 

By default, you can read CSV files using other tools or even default, pre-installed programs you have on your machine, so it is just a matter of choice what tools to use. The codebase can be executed anywhere with the same results.

What is the CSV Format? 

Nowadays, three main data formats are used for passing data from one machine to another: CSV, XML, and JSON.

The abbreviation CSV stands for “comma-separated values”. As the name implies, it is just a list of elements separated by commas. It is the most straightforward format to transfer data and should be used if

1. you need the most compact file size, or 
2. you have a flat data structure. 

Keep in mind that CSV files do not give you such flexibility in presenting the data as the other two options. 

Related articles:

• Pandas DataFrame to CSV
• How to Convert a List of Lists to a CSV File in Python?

Example Task 

This is a real-world task in a simplified form. The goal is to read data from CSV file (70 KB) and form a list of all series codes present in the second line of it. 

The provided data is an open statistical data from the European Central Bank (ECB) in CSV format and present financial flows during the period. The file consist of three main fields: 

1. series code 
2. observed date (period, e.g., 2019Q4, 2020Q1, etc.) 
3. observed value (data point, float number) 

Direct download link.

Data Preparation 

To focus on the parsing option, I suggest you download and extract a file beforehand. In the examples, the file will be placed on the Desktop, but you can put it anywhere you like. 

Script: 

import os import wget link = "http://sdw.ecb.europa.eu/export.do? mergeFilter=&removeItem=L&REF_AREA.252=I8&COUNTERPART_AREA.252=W0 &rc=&ec=&legendPub=published&oc=&df=true&DATASET=0&dc=&ACCOUNTING _ENTRY.252=A&node=9689710&showHide=&removedItemList=&pb=&legendNo r=&activeTab=&STO.252=F&STO.252=K7&STO.252=KA&STO.252=LE&legendRe f=reference&REF_SECTOR.252=S1&exportType=csv&ajaxTab=true" path = f"C:{os.environ['HOMEPATH']}\\Desktop\\data.csv" wget.download(link, path) 

Script breakdown:

import os import wget

Import statements are used to install code base which was written by someone else before and is ready to use just by referring to it. Some them (e.g. wget) should be additionally installed using similar command: 

The following command will install the latest version of a module and its dependencies from the Python Packaging Index:

python -m pip install SomePackage 

os package is used to perform basic operation with files and folders in your operating system. 

wget package is used to download files from websites. 

link = "http://sdw.ecb.europa.eu/export.do? mergeFilter=&removeItem=L&REF_AREA.252=I8&COUNTERPART_AREA.252=W0 &rc=&ec=&legendPub=published&oc=&df=true&DATASET=0&dc=&ACCOUNTING _ENTRY.252=A&node=9689710&showHide=&removedItemList=&pb=&legendNo r=&activeTab=&STO.252=F&STO.252=K7&STO.252=KA&STO.252=LE&legendRe f=reference&REF_SECTOR.252=S1&exportType=csv&ajaxTab=true" 

The string variable link is created which represents a direct download link. This link can be easily tested in any web-browser. 

path = f"C:{os.environ['HOMEPATH']}\\Desktop\\data.csv" 

string variable path is created which represents a path in your system where files will be downloaded later. 

The prefix “f” before the string makes it an “f-string” which means that you can use other variables in the string by using {placeholders}. In this case, variable os.environ[‘HOMEPATH’] refers to system variable (declared in the Windows system by default, not in your python script) and puts it into a string we just created. By default, HOMEPATH refers to the current user by C:\Users\%user% (you). 

wget.download(link, path)

The function call wget.download() triggers the file download from previously specified link and saves it by previously specified path. 

The result of this step is a ready-to-use CSV file on your Desktop. Now we can parse data from CSV file and extract series codes to list.

Data Exploration 

It is a good practice to explore data before you start parsing it. In this case, you can see that series codes are present in the second row of data.csv. 

Option 1 (the Fastest): Use the Standard Library 

This is the fastest option of reading a file using the standard library. Assuming the file is prepared and located on your Desktop, you can use the script below. This is the easiest way of getting data on the list. However, it has its drawbacks.

Input: 

import os path = f"C:{os.environ['HOMEPATH']}\\Desktop\\data.csv" with open(path, "r") as f: print(list(f.readlines()[1].split(","))[1:]) 

Output:

[‘QSA.Q.N.I8.W0.S1.S1.N.A.F.F._Z._Z.XDC._T.S.V.N._T’,... ‘QSA.Q.N.I8.W0.S1.S1.N.A.LE.F89.T._Z.XDC._T.S.V.N._T’]

Script breakdown: 

with open(path, "r") as f: print(list(f.readlines()[1].split(","))[1:]) 

The import statement and variable assignment is skipped as it was described previously and the main attention is given to the last statements. 

This is a combined statement of three parts:

1. The “with” statement, in the general meaning, allows us to define what code block (actions) we want to do with the object while it is “active”. In this case, we want to tell python that it has to do some actions while the file is open, and when all statements are completed, close it. 
2. The “open” statement allows us to open a file and place it is into Python memory. In this case, we open the previously given file (“path” variable) in “r” mode, which stands for “read”-only mode. 
3. The “print” statement allows you to see output on your screen. In this case we
   
   1. take file object f with open(path, 'r') as f,
   2. read second line with f.readlines()[1],
   3. split the line by the , separator in f.readlines()[1].split(“,”),
   4. convert the to list list(f.readlines()[1].split(“,”)),
   5. return the list starting from second element as the first one is empty in list(f.readlines()[1].split(“,”))[1:], and
   6. print the result in print(list(f.readlines()[1].split(“,”))[1:]).

There is no specific documentation as this code base uses standard library which is built-in in Python. 

Pros/Cons: Such an approach allows the user to get an instant view of the CSV file and select the required data. You can use this for spot checks and simple transformations. It is important to remember that such an approach has the lowest amount of adjustable settings, and it requires lots of workarounds when transformations are complicated. 

Option 2 (the Most Preferred): Use pandas.read_csv() 

The most preferred option of reading .csv file is using the Pandas library (Pandas cheat sheets here). Pandas is a fast, powerful, flexible, and easy to use open-source data analysis and manipulation tool, built on top of the Python programming language. 

Pandas is usually used for more advanced data analysis where data is stored in a “DataFrame” which is basically like a table in excel. A DataFrame has a header row and an index column so that you can refer to table values by “column x row” intersection. 

Script: 

import os import pandas as pd path = f”C:{os.environ[‘HOMEPATH’]}\\Desktop\\data.csv” df = pd.read_csv(path, delimiter=”,”, skiprows=[0]) list = df.columns.to_list()[1:] print(list) 

Output:

[‘QSA.Q.N.I8.W0.S1.S1.N.A.F.F._Z._Z.XDC._T.S.V.N._T’,... ‘QSA.Q.N.I8.W0.S1.S1.N.A.LE.F89.T._Z.XDC._T.S.V.N._T’] 

Script breakdown:

df = pd.read_csv(path, delimiter=”,”, skiprows=[0])

In this dataframe, variable df is created from the .csv file by executing pandas method read_csv. In this case the method requires several arguments: file, delimiter and skiprows. 

The file is the same as used before. The delimiter is “,” which is a default option for .csv files, and it might be skipped. But it’s good to know that you can use any other delimiter.

list = df.columns.to_list()[1:] print(list)

This line selects the column headers and puts them into a list starting from the second element going forward. The result is printed.

Pros/Cons: Such an approach is relatively fast, visually appealing to the reader, and is fully adjustable using a consistent approach. Comparing to the first option when standard libraries are used, it requires additional packages to be installed. I personally believe that it is not a problem, and such a drawback can be neglected. But another point should not be skipped — the amount of data. This approach is inefficient when you need lots of “side” data, which is useless for your purpose. 

Full documentation is available here with more guides and instructions on how to use it. 

Option 3 (optional): use csv.reader() 

There is also another way how to read .csv files, which might be useful in certain circumstances. The csv module implements classes to read and write tabular data in CSV format. It allows programmers to say, “write this data in the format preferred by Excel,” or “read data from this file which was generated by Excel,” without knowing the precise details of the CSV format used by Excel. Programmers can also describe the CSV formats understood by other applications or define their own special-purpose CSV formats. 

Script: 

import os import csv path = f"C:{os.environ['HOMEPATH']}\\Desktop\\data.csv" with open(path, 'r') as f: wines = list(csv.reader(f, delimiter=","))[1][1:] 

Output:

[‘QSA.Q.N.I8.W0.S1.S1.N.A.F.F._Z._Z.XDC._T.S.V.N._T’,... ‘QSA.Q.N.I8.W0.S1.S1.N.A.LE.F89.T._Z.XDC._T.S.V.N._T’]

Script breakdown: 

with open(path, 'r') as f: wines = list(csv.reader(f, delimiter=","))[1][1:] 

csv.reader() is a method which allows you to parse .csv file with specified delimiter. 

After that, we select the second row using first brackets “[1]” and after that, select all elements from that list starting from second “[1:]” using slicing. 

As this is a standard package, there is documentation at the official python website:

Pros/Cons: Such an approach is relatively simple and has just a few lines of code. On the other hand, it requires an additional package to be installed. 

Summary 

You should remember that there are different ways of reading data from CSV files. Select the one which suits your needs most or has the best performance and runtime. 

Where to Go From Here?

Enough theory, let’s get some practice!

To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

Practice projects is how you sharpen your saw in coding!

Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?

Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

Join my free webinar “How to Build Your High-Income Skill Python” and watch how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

Today, I’m excited to announce the preview of the next generation of Azure VMware Solution, designed, built, and supported by Microsoft and endorsed by VMware.

With the current economic environment, many organizations face new challenges to find rapid and cost-effective solutions that enable business stability, continuity, and resiliency. The new Azure VMware Solution empowers customers to seamlessly extend or completely migrate their existing on-premises VMware applications to Azure without the cost, effort, or risk of re-architecting applications or retooling operations. This helps our customers gain cloud efficiency and enables them to innovate at their own pace with Azure services across security, data, and artificial intelligence, as well as unified management capabilities. Customers can also save money with Windows Server and SQL Server workloads running on Azure VMware by taking advantage of Azure Hybrid Benefits.

Microsoft first party service

The new Azure VMware Solution is a first party service from Microsoft. By launching a new service that is directly owned, operated, and supported by Microsoft, we can ensure greater quality, reliability, and direct access to Azure innovation for our customers while providing you with a single point of contact for all your needs. With today’s announcement and our continued collaboration with VMware, the new Azure VMware Solution lays the foundation for our customers’ success in the future.

Sanjay Poonen, Chief Operating Officer at VMware commented, “VMware and Microsoft have a long-standing partnership and a shared heritage in supporting our customers. Now more than ever it is important we come together and help them create stability and efficiency for their businesses. The new Azure VMware Solution gives customers the ability to use the same VMware foundation in Azure as they use in their private data centers. It provides a consistent operating model that can increase business agility and resiliency, reduces costs, and enable a native developer experience for all types of applications.”

These comments were echoed by Jason Zander, Executive Vice President at Microsoft, who said, “This is an amazing milestone for Microsoft and VMware to meet our customers where they are today on their cloud journey. Azure VMware Solution is a great example of how we design Azure services to support a broad range of customer workloads. Through close collaboration with the VMware team, I’m excited that customers running VMware on-premises will be able to benefit from Azure’s highly reliable infrastructure sooner.”

The new solution is built on Azure, delivering the speed, scale, and high availability of our global infrastructure. You can provision a full VMware Cloud Foundation environment on Azure and gain compute and storage elasticity as your business needs change. Azure VMware Solution is VMware Cloud Verified, giving customers confidence they’re using the complete set of VMware capabilities, with consistency, performance, and interoperability for their VMware workloads.

Access to VMware technology and experiences

Azure VMware Solution allows you to leverage your existing investments, in VMWare skills and tools. Customers can maintain operational consistency as they accelerate a move to the cloud with the use of familiar VMware technology including VMWare vSphere, HCX, NSX-T, and vSAN. Additionally, the new Azure VMware Solution has an option to add VMware HCX Enterprise, which will enable customers to further simplify their migration efforts to Azure including support for bulk live migrations. HCX also enables customers running older versions of vSphere on-premises to move to newer versions of vSphere seamlessly running on Azure VMware Solution.

Seamless Azure integration

Through integration with Azure management, security, and services, Azure VMware Solution provides the opportunity for customers to continue to build cloud competencies and modernize overtime. Customers maintain the choice to use the native VMware tools and management experiences they are familiar with, and incrementally leverage Azure capabilities as required.

As we look to meet customers where they are today, we are deeply investing in support for hybrid management scenarios, and automation that can streamline the journey. We are excited to announce more about future hybrid capabilities as they relate to Azure VMware Solution, soon.

Leverage Azure Hybrid Benefit pricing for Microsoft workloads

Take advantage of Azure as the best cloud for your Microsoft workloads running in Azure VMware Solution with unmatched pricing benefits for Windows Server and SQL Server. Azure Hybrid Benefit extends to Azure VMware Solution allowing customers with software assurance to maximize the value of existing on-premises Windows Server and SQL Server license investments when migrating or extending to Azure. In addition, Azure VMware Solution customers are also eligible for three years of free Extended Security Updates on 2008 versions of Windows Server and SQL Server. The combination of these unmatched pricing benefits on Azure ensures customers can simplify cloud adoption with cost efficiencies across their VMware environments.

In addition, at general availability Reserved Instances will also be available for Azure VMware Solution customers, with one-year and three-year options on dedicated hosts.

Global availability and expansion

The Azure VMware Solution preview is initially available in US East and West Europe Azure regions. We expect the new Azure VMware Solution to be generally available in the second half of 2020 and at that time, availability will be extended across more regions. Plans on regional availability for Azure VMware Solution will be made available here as they are disclosed.

To register your interest in taking part in the Azure VMware Solution preview, please contact your Microsoft Account Representative or contact our sales team.

Learn more about Azure VMware Solution on the Azure website.

Microsoft gaming VP Phil Spencer dropped by CNBC’s “Squawk Alley” with a bit of good and bad news yesterday. The good news is that the Xbox Series X appears to be on track for its late 2020 release, despite COVID-19’s impact on the global supply chain.

The bad news is that some planned launch titles may not join the platform’s launch after all. Spencer didn’t confirm any specific delays, but explained to CNBC viewers that game production is “the bigger unknown” in how COVID-19 might impact the Xbox Series X’s launch.

“Game production is a large scale entertainment activity now, you have hundreds of people coming together, building assets, working through creative,” Spencer explained. He said that Microsoft is comitted to developers’ safety and security, and it won’t just “push when things aren’t ready.”

Spencer’s tone appeared to be optimistic, but it’s the first public acknowledgment we’ve heard from Sony or Microsoft that a once-in-a-century global pandemic could impact their planned product launches.

Developers everywhere are still trying to adapt to COVID-19’s impact on their process. Though many say the remote work isn’t the problem, the tiny changes in the game development process appear to be impacting some developers already.

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.

---

Hi, my name is Ovidiu Mantoc, a passionate level designer with +3 years working on professional and personal projects. I am currently working as a level designer at Gameloft.

In the past month after E3 2018, I started my breakdown study of the The Last of Us Part II gameplay trailer level design. This is my personal study analysis, of the level design that is in the video trailer. Here it goes:

 

Studying the masters! 

Naughty Dog designers are one of the level design masters in the industry. Therefore,  it’s logical to take the matter into your own hands and try to crack the “code”, to understand how they achieve that awesome level from the trailer and try to figure out all the bits and building blocks of the level design behind it.

I took this as a study challenge and practice, for my level design blockout skills, just like a concept artist is looking at reference images, using all the things around them to enhance and practice their craft. Why can’t Level Designers do the same for level design or any other discipline?

Drawing the top-down map 

First, I started to do a mental map of the environment and draw a top-down sketch to see how the map flows, where the edges of the environment are, as well as obstacles and different paths the player can take.

This part took a lot of replaying the trailer over and over, drawing every bit of the environment on paper, but in the end, it paid off in understanding the overall structure of the level.

References and study 

Knowing the action takes place in Seattle, I started searching for the area where the gameplay is located and found some key buildings I think were used as an inspiration and starting point for the level.

I found that in-game buildings are not at the same scale as the ones in real life but the shape and structure are very similar. This is for gameplay purposes (real life is boring), no need to walk 10 minutes in the game just to get to the next area.

I think this is a fine balance between making the gamespace feel “real” and not losing the player inside a real-life size space.

 

Blockout the level 

After the paper-map, I started a blockmesh level layout in Maya.

I started with the parking lot and spread out from there with the surrounding buildings. This was one of the most iconic buildings in the level and it was a great starting point to get the “right” size of the level and props. The odd shape of the parking lot was a little tricky to get right but I think I’m pretty close to the one in the demo.

Seeing the level from many angles gives you the feeling of space, options, and different paths the player can take. This way you can see how all combined elements click together and support all gameplay mechanics in the environment.

This process of making levels also helps in improving your level blockouts skills. It is really interesting doing this type of analysis, creating and studying levels.

Leading lines 

The level is rich in guiding the player’s eyes in the right direction, with the subtle help from leading lines that are present through the environment.

At first, they can go unnoticed but when looking closely, they do the job of guiding the player subconsciously in the correct direction.

  

Playtesting in engine 

I imported the blockout of the map in Unity engine, added a 3rd person character controller and did a lot of playtests in the environment, trying new paths and setting the right scale and size for all the props and environment.

Doing this in the engine made it easier to establish a relationship between objects and get the sense of scale and distances between them.

Getting the right scale of the space, compositions, leading lines, and readability, helped a lot in defining how the player is guided through the environment.

Conclusion 

As far as I can see in the gameplay trailer and in my rebuilt level, the Naughty Dog team did a great job making the flow of the level feel natural, with lots of options and multiple paths the player can take and elements that contribute to the overall structure in creating a non-linear level.

Some people may think that in that small video trailer there is so little info, but actually there is a lot, from spatial arrangement, sizes, and distances, enemy types, placement, and behaviors, even spawning enemies, everything, you just need to know where to look and how to analyze each area pixel by pixel.

Analyzing levels in any form (taking notes when you play, rebuilding levels you like, build from memory, even top-down drawings and videos analysis) is a good practice to boost up your level design skills, helping you learn something new each time.

This was a pretty fun challenge, I really enjoyed it. I will try to do more of this in the future.

Feel free to poke me on twitter or facebook and chat about level design stuff.
If interested, you can find my online portfolio here.

Thanks and have fun making your levels!

Note: The Last of Us Part II is the property of Sony Interactive Entertainment America LLC ©2018. Created and developed by Naughty Dog, Inc.

First launched in 2003, the open source cross platform vector graphics application Inkscape just hit the major 1.0 milestone!  Inkscape is open source with the source code available on GitLab.  Details of the release from the Inkscape news page:

After a little over three years in development, the team is excited to launch the long awaited Inkscape 1.0 into the world.

Built with the power of a team of volunteers, this open source vector editor represents the work of many hearts and hands from around the world, ensuring that Inkscape remains available free for everyone to download and enjoy.

In fact, translations for over 20 languages were updated for version 1.0, making the software more accessible to people from all over the world.

A major milestone was achieved in enabling Inkscape to use a more recent version of the software used to build the editor’s user interface (namely GTK+3). Users with HiDPI (high resolution) screens can thank teamwork that took place during the 2018 Boston Hackfest for setting the updated-GTK wheels in motion.

Smoother performance & first native macOS application

This latest version is available for Linux, Windows and macOS. All macOS users will notice that this latest version is labelled as ‘preview’, which means that additional improvements are scheduled for the next versions. Overall, 1.0 delivers a smoother, higher performance experience on Linux and Windows, and a better system integration (no more XQuartz!) on macOS.

So many new bells and whistles

One of the first things users will notice is a reorganized tool box, with a more logical order. There are many new and improved Live Path Effect (LPE) features. The new searchable LPE selection dialog now features a very polished interface, descriptions and even the possibility of marking favorite LPEs. Performance improvements are most noticeable when editing node-heavy objects, using the Objects dialog, and when grouping/ungrouping.

You may encounter some challenges downloading today.  I was unable to download the Win64 version, but the 32bit version worked fine.  Hopefully these download issues are fixed soon.  You can learn more in the video below.

Art GameDev News Applications

---

COPR is a collection of personal repositories for software that isn’t carried in Fedora. Some software doesn’t conform to standards that allow easy packaging. Or it may not meet other Fedora standards, despite being free and open source. COPR can offer these projects outside the Fedora set of packages. Software in COPR isn’t supported by Fedora infrastructure or signed by the project. However, it can be a neat way to try new or experimental software.

This article presents a few new and interesting projects in COPR. If you’re new to using COPR, see the COPR User Documentation for how to get started.

Ytop

Ytop is a command-line system monitor similar to htop. The main difference between them is that ytop, on top of showing processes and their CPU and memory usage, shows graphs of system CPU, memory, and network usage over time. Additionally, ytop shows disk usage and temperatures of the machine. Finally, ytop supports multiple color schemes as well as an option to create new ones.

Installation instructions

The repo currently provides ytop for Fedora 30, 31, 32, and Rawhide, as well as EPEL 7. To install ytop, use these commands with sudo:

sudo dnf copr enable atim/ytop

sudo dnf install ytop

Ctop

Ctop is yet another command-line system monitor. However, unlike htop and ytop, ctop focuses on showing resource usage of containers. Ctop shows both an overview of CPU, memory, network and disk usage of all containers running on your machine, and more comprehensive information about a single container, including graphs of resource usage over time. Currently, ctop has support for Docker and runc containers.

Installation instructions

The repo currently provides ctop for Fedora 31, 32 and Rawhide, EPEL 7, as well as for other distributions. To install ctop, use these commands:

sudo dnf copr enable fuhrmann/ctop

sudo dnf install ctop

Shortwave

Shortwave is a program for listening to radio stations. Shortwave uses a community database of radio stations www.radio-browser.info. In this database, you can discover or search for radio stations, add them to your library, and listen to them. Additionally, Shortwave provides information about currently playing song and can record the songs as well.

Installation instructions

The repo currently provides Shortwave for Fedora 31, 32, and Rawhide. To install Shortwave, use these commands:

sudo dnf copr enable atim/shortwave

sudo dnf install shortwave

Setzer

Setzer is a LaTeX editor that can build pdf documents and view them as well. It provides templates for various types of documents, such as articles or presentation slides. Additionally, Setzer has buttons for a lot of special symbols, math symbols and greek letters.

Installation instructions

The repo currently provides Setzer for Fedora 30, 31, 32, and Rawhide. To install Setzer, use these commands:

sudo dnf copr enable lyessaadi/setzer

sudo dnf install setzer

If you’re into your lightsaber wielding, Wookie-filled sci-fi epics, May 4th is a pretty special occasion. Borrowing the saga’s iconic phrase of ‘May the force be with you’, the date is recognised all over the world as Star Wars day and this time around, we’re also being treated to a special Star Wars-themed sale on Nintendo Switch.

Three Star Wars games have been discounted in total across both Europe and North America. We’ve got the games, discounts, and new prices for you below.

These discounts are available on the Nintendo Switch eShop from now until 6th May, so if you’re after a Star Wars gaming fix, make sure to get on these offers while you still can. Fortnite fans can also join in the celebrations with the limited-time return of Star Wars skins and lightsabers.

Will you be grabbing any of these deals today? Share your passion for all things Star Wars in the comments below.

---

Further Reading: