A Generic Terminal in Python

More than once I have found myself wanting an easy-to-use interactive shell for custom tasks. I often open a Python interactive session to perform quick tasks like string manipulation, rather than having to deal with the likes of sed/awk (which I admittedly know next to none of). With a little creativity, the same style of interaction session can be a custom UI that is easy to use and modify. Normally I would assign quick functions such as:

>>> def f(s):
...     with open("a.txt", "w") as f:
...         f.write(s)
>>> def g(s):
...     return "\n".join([s.replace("jan", a) for a in 
...         "jan,feb,mar,apr,may,jun,jul,aug,sep,oct,nov,dec".split(",")])
>>> f(g("jan_avg_high_temp_degf"))

But if I closed the Python session and then later wanted to do the same thing, I would be repeating work, so I thought of ways to accomplish this kind of thing more extensibly.

Recently I was creating a simple program to output the Morse Code equivalent of a string (either to file or to a playback device) and found myself wanting to test it quickly by passing it strings on the fly and listening to the output. Sure, I could have done something like a while loop asking for input and translating it, but I wanted a bit more functionality. I wanted to copy-paste text from the internet and save their translation to file for listening practice, and the ability to turn saving on and off at will. I wanted to change the words per minute of playback. I’m sure there are plenty of other simple tasks that could be added here.

I decided to implement this to look like a terminal, with a $ shell prompt and special commands beginning with : (and the ability for these to take arguments). All other input is interpreted as a string to be translated, and the values of internal options (WPM, whether to save to file, other aspects of the waveform generated) are persistent during any session.

I implemented the terminal as a simple Terminal class that includes all the basic functionality, so I could create a MorseCodeTerminal inheriting from it and adding its own specifics. The basic Terminal specifies a shell prompt, the commands :q (quit), :h (help), and the run() method that begins a while loop processing the user’s input. It is easy to add new commands using self.add_command(), specifying the command name, the callable to which args (separated by spaces) are passed, and the help string for inclusion in :h output.

Several terminal options (e.g. whether to save to file, whether to play aloud) are boolean, and they make use of a change_binary_attribute() method for extensible use. If an arg is given, the option is set to it; otherwise, the current value is printed. The same usage holds for non-binary attributes, although a general function for these is trickier due to different kinds of validation, so I still consume new input in a custom method for each of them. Below is an example from the MorseCodeTerminal:

$ :save
$ :save 0
$ :save
$ :wpm
$ :wpm asdf
Invalid input for int: asdf
$ :wpm 25
$ :wpm
$ :asdf
Unrecognized command

I found it important to keep the terminal from crashing due to user error, because it was pretty frustrating to set up the options I wanted (if different from defaults) and try to begin doing real work, only to make a silly typo and have to start over because I didn’t handle the error. Ideally also the terminal will not quit on ^C, but will just catch the KeyboardInterrupt, stop any existing task, and await the next input. Only :q should cause the program to exit completely.

This ability of the terminal to keep going in the case of bad input, or exceptions thrown during a task, became more important once I made my second one, which allows the user to construct and edit a Pandas DataFrame manually. Similarly to the Morse Code use case, this could easily be accomplished in a normal interactive Python session by someone who is used to the task. For example:

>>> import pandas as pd
>>> df = pd.DataFrame()
>>> df.loc["H", "atomic_number"] = 1
>>> df.loc["He", "atomic_number"] = 2
>>> df.loc["U", "atomic_number"] = 92
>>> df.loc["H", "boiling_point_K"] = 20.27
>>> df.loc["He", "boiling_point_K"] = 4.22
>>> df.loc["U", "boiling_point_K"] = 4404.00
>>> df
    atomic_number  boiling_point_K
H             1.0            20.27
He            2.0             4.22
U            92.0          4404.00
>>> df = df.set_index(df.index.rename("Index"))
>>> df
       atomic_number  boiling_point_K
H                1.0            20.27
He               2.0             4.22
U               92.0          4404.00
>>> df.to_csv("data.csv")
>>> # done

This could also be accomplished in Excel, but I wanted to see how well I could streamline it this way. The terminal approach:

$ :filepath data.csv
$ :index H
$ atomic_number = 1
$ boiling_point_K = 20.27
$ :index He
$ atomic_number = 2
$ boiling_point_K = 4.22
$ :index U
$ atomic_number = 92
$ boiling_point_K = 4404.00
$ :save

A lot of the repetition has been cut out here. Also, the terminal is configured to re-save the CSV on each change automatically, so we just check that :save is True and we’re good! As with Morse Code, it’s easy to add new functionality here, such as :nan (tell me which columns are missing (NaN) in the current row). Clearly with large amounts of data, the constant saving would be inefficient, but it can be turned off, and the current df would just be stored as an attribute of the current Terminal. This makes it clear that quitting the terminal on any bad input is a bad idea, as all of the unsaved changes would be lost.

I’m not saying this is the best way to edit a DataFrame at all, but for quick-and-dirty changes like one might make to a CSV in Excel, or for tasks that I find myself repeating all the time, it’s pretty nice.

I’m interested to see what other uses there are for this kind of terminal. Right now I’m just playing with it for personal projects, but I think there’s a lot of potential.