Maximization

This article describe how I maximize my Python experience.

Virtual Environment

Virtualenv is one of the best things in Python world. She’s set your Python free from bloody civil war. No matter what, you must learn virtualenv. If you don’t learn it by now, do your research now. I you can’t understand it yourself after research, find yourself a mentor to teach you virtualenv. Believe me, you won’t regret it.

After some times using virtual environment, I realize one thing. Otherwise this will lead to a frustrating annoyance. That is:

“We should always name our virtual environment!”

It’s so easy to do, you’re a fool if you don’t do this.

> python -m venv --prompt (MY_PROJECT_NAME) .venv

Packaging vs Executable

Because of Ms Visual Basic 6, it seems that my world view of computer programming is greatly skewed. When I come to Python, first thing I want to do is how to make the app executable! Stand alone executable to be exact. Which later I found out that it’s really differ from other python programmer.

Well, there is a few ways to do it. Even though in a lot of case you may not really need it. Of course there will be a case where you absolutely need to create exe file. The problem is how it’s handled differently in every platform. While Python is multi platform by nature.

After working with Python for years, I began to think that I should revise my world view. So lately I try not to create a stand alone executable like app created by VB. I just package my Python app as described in Python official here. To my amazement, I’m completely fine. And as a happy side effect, my app now easily become multi platform!

Quirks

One of the most confusing things about packaging is about data file. Setuptools have two kind of data type which is inside and outside of package. If your data is outside of package, you should use data_file, but if your data is inside the package, you should use package_data. It make sense, yes. But things is not as simple as it seems. It’s tricky. Even a trailing backslash is having a tremendous effect. Using asterisk leads to unexpected behavior too.

After a lot of trial and error, for package_data we simply need to state the directory. No need for backslash or asterisk at all. This way, all files and subdirectory will be automatically included. But strangely, if the subdirectory contains python modules, the modules is ignored.

CLI vs GUI

As a new generation of programmer myself, I only spent a little time in console word before moving to Windows. This is why for a very long time I always try to create GUI app. Which is a big mistake in the spirit of automation.

By creating GUI app, I find myself at the mercy of my own app. Every time I need to use the app, I have to do it myself. Start the app, set things one way or another, click here and there, before finally start the app to work. Even if the things needs to be set is the same every single time. There’s nothing automatic here. I needs to set things every time before starting the work. Well okay, after tiring myself out by setting things, I save the setting. But still, I need to start it myself every single time!

In contrast, the workflow is really different when I create CLI app. Though sadly in a lot of computer school, I was taught how to bring GUI app into CLI environment. Which in my opinion now is really silly. Come on… We are not in the 80’s anymore. We should not try to emulate GUI app in CLI environment. Nor should we bring CLI app into GUI. If you do that, you will soon disappointed by the result.

In general, you should create app in GUI if your app needs user interaction. More specifically, your app user won’t do it over and over again with the same setting. In other words, you can’t make the process automatic, because it needs the user judgement each time.

Now, if your app only needs to do the same things periodically, that’s when you need to create your app in CLI environment. But, you may argue, CLI app is hard to use. Especially for general public. Well, that’s where you’re wrong.

Remember, your app needs to do the same things periodically. Do you expect the user run your app periodically? That’s just simply torture for your user. Which in this case the user is mostly I, me and myself. Enough is enough, and I’m not an M too. Let’s push all the works to the computer.

Scheduling

As embarrassing as it is, I must admit that for a very long time, I don’t even know Windows has Task Scheduler. It’s when I starts working in Linux and use Cron extensively that I begin to wonder whether Windows has something similar. To my surprise, lo and behold, there is Task Scheduler sitting silently in the corner.

Yes, Task Scheduler is your all powerful ally to free your user from torture. Specify your task there, set the period how your app needs to run. Then Windows will take care of the rest. Windows won’t even forget to run your app periodically, in contrast to your user that highly likely will forget from times to times.

So, if now you still don’t get your hand dirty in Cron or Task Scheduler, learn it now. A simple google search will take you to a lot of resource to learn.

Conclusion

Inspired by Brandon Rhodes who named all his tool scripts started with comma and make his home directory a version control repo, I too begin to think how to make my python scripts more convenience. But I don’t quite like the idea of naming my Python script started with comma. Not to mention that in Windows this naming convention is simply didn’t work. I consider to use other prefix, but it’s still doesn’t fit me.

Then one day I got a flash of inspiration. Rather than treat my python script like bash script, why not just package them then install them to virtualenv as usual and add virtualenv Scripts directory to path. Package can also automatically create the platform executable and solves dependency problem.

Then if I named the entry point with the same as the package name, I can simply view the requirement file to see the executable name. But in case I use different name for the executable, a little comment in requirements file will do the job. Though I find a little comment of what the package do is a great help. But of course there is nothing to stop you to create a comprehensive comment. You can add as many comment as you like.

This is how I do it:

1. Make home directory in Windows and Linux a version control repo.

2. Create a file requirements.txt as usual in home directory.

3. Add your package to requirements file.

4. Add requirements.txt to version control.

5. Create virtualenv venv or .venv in home directory as you like.

6. Activate and install the package to virtualenv using pip as usual.

7. Add virtualenv Scripts directory to path.

8. Create task in Task Scheduler or Cron.

Now you don’t have to do anything and the computer will do the job for you.

Computer is automation, let’s do more of those!

Templating

After creating a package again and again and again…

I realize that this procedure is amazingly annoying.

There must be a better way!

—Raymond Hettinger (Python Core Developer)

True enough, others also think that this process is obnoxious. There are many ways to ease this process. Cookiecutter is my favorite.

Post Script

After some times, I end up with a lot of packages and I’m starting to overwhelmed.

So I starts to tinkering with my package. After a lot of grueling trial and error, I finally able to combine my package but break it into manageable modules. Here it is.

Now I only have one entry point, but with virtually unlimited sub commands. But the most important part is my script now separated into small modules. My main click script is so messy in the past

By making every command into module and every sub command into package, I isolate my code into the most easy to managed state.