Documentation + Sphinx

Lecture Video

On May 19, 2021 Prof Wirthlin discussed how to set up Sphinx and how to document code. The video is embedded below.

Timestamps

0:00 Introduction and documentation discussion
6:56 Creating documentation from source code
12:08 Introduction to Sphinx
13:17 The Python Docstring
18:52 Installation and setup of Sphinx
25:34 Customizing your Sphinx output
34:34 Example project
41:35 Read the Docs
44:10 Alternatives to restructured text
48:10 Pylint

Introduction and Installing Sphinx

Sphinx is a very powerful tool that allows you to automatically generate documentation for your code. It works with a variety of different coding languages (not just python) and also allows you to generate html websites from restructured text.

You can install sphinx using either the system package manager or pip. You can also try installing sphinx within a conda env. The commands for each of these options can be found bellow:

System Package Manager

sudo apt-get install python3-sphinx

Pip

pip install sphinx

Conda

conda install sphinx

After installing sphinx, create an empty directory. You can name your directory whatever you want but for demonstration purposes we will call our directory sphinx-docs.

mkdir sphinx-docs

To prepare the directory to be used by sphinx run the following command:

sphinx-quickstart <path to the directory you created>

You will be asked several questions about how you want your documentation to be set up. You can chose the default option for most of these questions by just hitting enter, but you will want to specify the name of your project and the name of the author. Also be sure to type y if you are asked if you want to use autodoc and if you want sphinx to automatically generate a Makefile.

After sphinx-quickstart completes, move into the empty directory you created before. Take a look at the files and folders that sphinx has automatically created. Of special note are the following:

index.rst This file is the root of all the other documentation files we are going to create. It serves as the landing page of the sphinx generated website. You can customize this file to your heart’s content just don’t remove the toctree directive from the file. We will talk more about toctrees latter.

conf.py This is a python script containing all of the settings used by sphinx to configure itself. You can modify this file to include advanced features such as different highlighting for code snippets or changing your default webpage layout. We will modify this file latter to include the autodoc feature of sphinx.

_build directory This directory will contain all of your html generated code.

Makefile If you haven’t changed the setting from its default, you should also see that sphinx has created a Makefile. This file is used to build an html version of our documentation from RST.

Building your documentation

Run make html inside of your sphinx-docs directory. This will build a website version of your documentation and dump it inside of _build. Take a look at the html file generated by make by opening index.html in a web browser:

firefox _build/html/index.html

RST basics and index.rst

Now open the index.rst folder created automatically by sphinx and compare it to the html version on your browser. Notice anything? Sphinx uses a documentation formate called re-structured text (RST) by default to generate html pages. Take a look at the title of the page in index.rst:

   Welcome to <Authors name> documentation!
   ==========================================

In rst, the tittle of a page is indicated by placing a line of “=” underneath the name of a tittle.

headings or subheadings can be created by using a line of “-“ or “~” as follows:

   this is a heading 
   ------------------
   
   this is a subheading
   ~~~~~~~~~~~~~~~~~~~~

Your tittle/heading markers must be at least as long as the text above them. You also must have a blank line between your section tittle and the body of your text. For instance, these tittles will generate errors from sphinx or will produce unintended renders:

   Don't do this
   =======

   Or this
   =======
   This text needs to be put after a newline.

Things to Try

Experiment by putting a few headings and subheadings inside your index.rst file and re-run make html. What changes were made to the website?

Directives in Sphinx

Before we move on, we need to talk about directives in sphinx. Directives are special functions that allow us to do things we cant usually do very easily with plain text. The general formate of a directive is as follows:

.. <name of directive>::
   :<directive option1>:
   :<directive option2>:
   ...

   Contents

The name of your directive should be preceded by two dots “..” and a space. After the name you should have two colons “::”. Each option for the directive as well as every line of content in the directive should be indented exactly three spaces. Also note that there must be a blank line both preceding and following the contents of the directive.

The Toctree Directive

You may have noticed these lines of code in your index.rst file:

.. toctree::
   :maxdepth: 2 
   :caption: Contents:

These lines define a table of contents tree (a toctree). A toctree is a directive that gives links to other files within your documentation. This toctree is pretty boring right now as it doesn’t have any contents. The maxdepth option of a toctree specifies the number of subheadings (not headings) to include from each file within the toctree’s contents.

Every rst file within the sphinx-docs directory or any subdirectories of it must be included in at least one toctree. If you neglect to do this you will get a warning from sphinx. Additionally, every .rst file included in a toctree must have a tittle.

Things to Try

Create two new rst files in the directory you have been working with. Give each rst file a title, a few headings, and a few sub-headings. Then add your files to the toctree. If I had created two new files named sphinx-test1.rst and sphinx-test2.rst then my toctree would look like this:

.. toctree::
   :maxdepth: 2
   :caption: Contents:

   sphinx-test1
   sphinx-test2.rst

Notice that the .rst file extension is optional. Also pay attention to the blank line between :caption: Contents and sphinx-test1.

Re-run make and take a look at how your new toctree rendered.

Using tags and Automodule in Sphinx

On top of creating your own documentation using RST, sphinx also has a very useful feature called autodoc that finds special comments in your code and generates documentation from them. To enable autodoc you will need to modify your conf.py script. Under the #-- General Configuration --- section of the conf.py file, you should see an array called extensions. Add the autodoc sphinx extension to the array:

extensions = [
    'sphinx.ext.autodoc'
]

Now we can use the automodule directive to generate documentation for our code within an RST file. Open one of the RST files you have created and add these lines to it:

.. automodule:: <name of your python file (file extension is optional)>
   :members:

Now lets test out autodoc on some python code. Copy the python csv application you have been working on for the past few days into your sphinx directory and add some docstrings to your code. Docstrings in python are added directly after a function definition and are enclosed in three–yes three–quotation marks. Here is an example of a docstring for main:

def main():
    """
    Put your documentation here
    ...
    """

    Code for main ...

After adding a few docstrings to your code re-run Make html and see what changes have been made to your website. If you run into any warnings or errors checkout the notes bellow:

Note 1

If the code you wish to document lives outside of the directory where your sphinx documentation is, you will need to uncomment these three lines at the top of conf.py to avoid warnings about sphinx not knowing which module to import for autodocumenting:

import os
import sys
sys.path.insert(0, os.path.abspath('<path to the directory where your code is stored>'))

Note 2

Automodule works by actually importing the python scripts you ask it to document. This can cause problems if your scripts rely on command line arguments. To avoid problems with autodoc your main function should always be protected by adding the following lines directly after your main function:

if __name__ == "__main__":
    main()

This will protect your main function from being run unless it is being called by the python interpreter.

This concludes our module about Documentation and Sphinx. For more information checkout some of the activities and resources bellow.

More things to try

• Go through the different RST functions in the sphinx guide and pick a few that interest you. Incorporate this functionality into your documentation.
• Go to Symbiflow.readthedocs.io and look at the RST that the webpage is written in by clicking on Show Source in the top right hand corner of each page. Try to incorporate a few of the design techniques into your own documentation.
• Sphinx uses alabaster as the default theme of its webpage. Learn how to change the theme of your webpage by following this link.
• Learn about doctest and try adding a few tests to your code using sphinx.

Additional resources

• We have barely scratched the surface of rst and sphinx in this module. A very helpful resource is the sublime and sphinx guide. If you are working with rst at all I highly recommend reading at least these sections from that resource. I personally use these methods nearly every time I write a new document:
  • Showing example code
  • Adding links
  • Adding Images
• This online tool is invaluable for building complex tables in RST or Markdown format.

• Visit this website for the official Sphinx guide.

• If you want RST examples the SpyDrNet documentation is written in RST. See how each page is written by clicking on View Page Source at the top of the page. As previously mentioned Symbiflow is also written in RST.