One of these days, while setting up a couple of Linux virtual machines to test my Python applications, I was faced with a series of obstacles on CentOS 7 that I needed to overcome. Since this is the kind of issues that many other Python developers may encounter when setting up a Linux machine, I will share what I learned with this experience.
I usually use macOS as my main personal development environment. Although I have come to enjoy PyCharm as a very valuable IDE, I still like to do a lot of things in the Terminal. Apples’s macOS includes a wonderful shell and many well known Unix utilities, and many other are available through the Homebrew package manager. I tend to create and activate virtual environments with Pipenv, usually using the most current Python version, and sometimes in some of the previous ones. For instance, I may want to check how one of my Python applications is behaving under Python 3.6, 3.7 and 3.8, or even the latest Python 3.9 alpha. Or I may want to check if updating a specific third-party dependency will break something.
From time to time, I also want to check for platform specific issues and make sure my code is working as expected on Windows, Linux or Haiku. For that kind of work, in the past I have used both VMware Fusion and VirtualBox, but currently I am using Parallels Desktop.
So, this week, I decided to test one of my applications on CentOS 7, a version that has been released a few years ago, but has excellent support in Parallels Desktop and is still officially maintained.
Installing Python 3
The first disappointment with CentOS 7 was that it didn’t include Python 3. Even after updating the system software, all I got was Python 2.7.5 (which was released 7 years ago!), and in the default
yum repositories all I could find was an older Python 3.6.8 version. I wanted to use the current stable version (Python 3.8.2), so I had to find out how to get it and install it.
At the official Python website, when searching for Linux downloads, we are directed to the Python Source Releases page, which lists many source code downloads for both stable releases and pre-releases. So, jump right to “Python 3.8.2 - Feb. 24, 2020” and copy the link from “Download Gzipped source tarball”. You can use that URL to download the Python source code right from the Terminal using
wget. In my case, I already had
wget installed, so I could move on.
1. Download Python source code (gzipped tarball):
2. Extract source code:
3. Go inside the extracted
Python 3.8.2 folder, build and install Python:
1 2 3
Whenever you need to run a command with administrative privileges in Unix-like systems, you may use the
sudo prefix. It will ask you the password in order to proceed.
make commands in step 3 will take some time and generate a lot of console output, but if there are no errors it should be ok.
Unfortunately, I got one error, with the following traceback:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
The solution was to install
zlib-devel and then rebuild Python using the same commands:
1 2 3
This time I had no errors. But…
Configuring additional builtin packages
…I would soon discover that my freshly installed Python 3.8.2 was missing some modules I have used in my projects, like
Once again, I needed to install a few support packages using
yum and then rebuild Python using
1 2 3
So, you can see a pattern here.
- Python misses something.
- You find out which development package provides that dependency.
- You install it using a package manager (for CentOS 7, it’s
yum, but for other versions or other Linux distributions it could be
apt-getor even other package managers).
- Then you re-run the configuration and
makesteps to recompile and reinstall Python.
In fact, I also needed to install other dependencies later, when I tried to use
pip and got an SSLError.
Once again, after identifying the missing packages, the process was the same:
1 2 3
Then, to upgrade
pip for the current user:
This time, it worked just fine.
Managing virtual environments and third party packages
In order to be able to set up different Python environments for each application, which allow us to use more than one Python version on the same computer and different sets of third-party packages, there are multiple ways. You could stick with the builtin
venvas described in the Python documentation, or you could use Virtualenv, or Poetry. Since I am used to Pipenv and it has served me well for a few years, that’s what I use when I am able to do so. After that last step in the previous section, I was able to install Pipenv by running the following command:
Then I was finally able to run the usual commands
pipenv --python 3.8,
pip instal -r requirements.txt and so on.
As a side note, to be able to easily clone my git repositories, I just needed to install
Setting up a Python version that does not come preinstalled can be an headache and it surely can discourage people from using Python on Linux. But once you figure out how to do it, it gets easier.
I hope this article can help those of you that encounter a similar difficulty to find their way, but please remember that there are many Linux distributions and versions. Many of them may require a different set of steps to achieve the same result, and some even include Python versions that are much more recent than the one in CentOS 7. On the other hand, having available a Python version that is separate from the one used internally by the operating system may be a good idea, as it helps to keep the system sane and working properly.
Finally, just a brief word to remember that dependency hell can be a really ugly place. So, if you are starting to learn Python, please take some time to understand virtual environments and to find out the virtual environment management tool that works best for you. It will save you from a lot of trouble and make your Python developer life much more enjoyable.