henson.contrib.sphinx broken on versions of Python older than 3.5.2

[dirn]

A change was introduced in Sphinx 1.7.1 that breaks Henson's Sphinx extension for anyone generating documentation using a version of Python older than 3.5.2 and Sphinx 1.7.1 (or, presumably, newer). This was addressed for Henson itself, but this fix won't work for anyone installing Henson[sphinx]¹. sphinx<1.7.1 should be included in setup.py as well.

$ python -V
Python 3.5.1
$ tox -e docs
GLOB sdist-make: /path/to/Henson-Database/setup.py
docs create: /path/to/Henson-Database/.tox/docs
docs installdeps: -rdocs-requirements.txt
docs inst: /path/to/Henson-Database/.tox/dist/Henson-Database-0.5.0.zip
docs installed: alabaster==0.7.10,alembic==0.9.8,argh==0.26.2,Babel==2.5.3,certifi==2018.1.18,chardet==3.0.4,doc8==0.8.0,docutils==0.14,Henson==1.1.0,Henson-Database==0.5.0,idna==2.6,imagesize==1.0.0,Jinja2==2.10,Mako==1.0.7,MarkupSafe==1.0,packaging==17.1,pathtools==0.1.2,pbr==3.1.1,Pygments==2.2.0,pyparsing==2.2.0,python-dateutil==2.7.0,python-editor==1.0.3,pytz==2018.3,PyYAML==3.12,requests==2.18.4,restructuredtext-lint==1.1.3,six==1.11.0,snowballstemmer==1.2.1,Sphinx==1.7.1,sphinx-rtd-theme==0.2.4,sphinxcontrib-autoprogram==0.1.4,sphinxcontrib-websupport==1.0.1,SQLAlchemy==1.2.5,stevedore==1.28.0,urllib3==1.22,watchdog==0.8.3
docs runtests: PYTHONHASHSEED='4183452930'
docs runtests: commands[0] | sphinx-build -W -b html -d /path/to/Henson-Database/.tox/docs/tmp/doctrees docs docs/_build/html
Traceback (most recent call last):
  File "/path/to/Henson-Database/.tox/docs/bin/sphinx-build", line 7, in <module>
    from sphinx.cmd.build import main
  File "/path/to/Henson-Database/.tox/docs/lib/python3.5/site-packages/sphinx/cmd/build.py", line 13, in <module>
    from typing import TYPE_CHECKING
ImportError: cannot import name 'TYPE_CHECKING'
ERROR: InvocationError: '/path/to/Henson-Database/.tox/docs/bin/sphinx-build -W -b html -d /path/to/Henson-Database/.tox/docs/tmp/doctrees docs docs/_build/html'
___________________________________ summary ____________________________________
ERROR:   docs: commands failed

¹ Henson-Database uses 3.6 to generate its Sphinx docs, but there's no guarantee other extensions will do this. There's some merit to generating docs using the lowest supported version of Python. If such a change were introduced to Henson-Database, that would mean the docs would be generated using 3.4.x.

[lbedner]

@dirn Pull Request #151 has been submitted to address this.

I noticed something else while running my tests, something that may be better suited for discussion somewhere else, and if so, let me know.

If left as is, tox -e docs for Henson-Database won't work with this, because of the contents of its docs-requirements.txt:

alembic
doc8
-e git+https://github.com/iheartradio/Henson.git@9d1558f03025232688f3c85e9383925aa7169842#egg=Henson[sphinx]
sphinx>=1.3 <--------------------------
sphinx_rtd_theme

sphinx==1.7.1 is still installed because of the sphinx>=1.3 line.

To test this further, I created a new virtual environment, and did a pip install -r docs-requirements.txt, with the following results:

Leonard-Bedner-MacBook-Pro:~ leonardbedner$ mkvirtualenv test
Using base prefix '/Users/leonardbedner/.pyenv/versions/3.6.0'
New python executable in /Users/leonardbedner/.virtualenvs/test/bin/python3.6
Also creating executable in /Users/leonardbedner/.virtualenvs/test/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /Users/leonardbedner/.virtualenvs/test/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/leonardbedner/.virtualenvs/test/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/leonardbedner/.virtualenvs/test/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/leonardbedner/.virtualenvs/test/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/leonardbedner/.virtualenvs/test/bin/get_env_details
(test) Leonard-Bedner-MacBook-Pro:~ leonardbedner$ fish
leonardbedner@Leonard-Bedner-MacBook-Pro ~>  pip install -r docs-requirements.txt
Collecting alembic (from -r docs-requirements.txt (line 1))
  Downloading alembic-0.9.8.tar.gz (1.0MB)
    100% |████████████████████████████████| 1.0MB 1.2MB/s
Collecting doc8 (from -r docs-requirements.txt (line 2))
  Downloading doc8-0.8.0-py2.py3-none-any.whl
Obtaining Henson[sphinx] from git+https://github.com/iheartradio/Henson.git@9d1558f03025232688f3c85e9383925aa7169842#egg=Henson[sphinx] (from -r docs-requirements.txt (line 3))
  Cloning https://github.com/iheartradio/Henson.git (to 9d1558f03025232688f3c85e9383925aa7169842) to ./.virtualenvs/test/src/henson
  Could not find a tag or branch '9d1558f03025232688f3c85e9383925aa7169842', assuming commit.
Collecting sphinx>=1.3 (from -r docs-requirements.txt (line 5))
  Downloading Sphinx-1.7.1-py2.py3-none-any.whl (1.9MB)
    100% |████████████████████████████████| 1.9MB 639kB/s
Collecting sphinx_rtd_theme (from -r docs-requirements.txt (line 6))
  Downloading sphinx_rtd_theme-0.2.4-py2.py3-none-any.whl (1.4MB)
    100% |████████████████████████████████| 1.4MB 935kB/s
Collecting SQLAlchemy>=0.7.6 (from alembic->-r docs-requirements.txt (line 1))
  Downloading SQLAlchemy-1.2.5.tar.gz (5.6MB)
    100% |████████████████████████████████| 5.6MB 247kB/s
Collecting Mako (from alembic->-r docs-requirements.txt (line 1))
  Downloading Mako-1.0.7.tar.gz (564kB)
    100% |████████████████████████████████| 573kB 2.1MB/s
Collecting python-editor>=0.3 (from alembic->-r docs-requirements.txt (line 1))
  Downloading python-editor-1.0.3.tar.gz
Collecting python-dateutil (from alembic->-r docs-requirements.txt (line 1))
  Downloading python_dateutil-2.7.0-py2.py3-none-any.whl (207kB)
    100% |████████████████████████████████| 215kB 1.7MB/s
Collecting chardet (from doc8->-r docs-requirements.txt (line 2))
  Downloading chardet-3.0.4-py2.py3-none-any.whl (133kB)
    100% |████████████████████████████████| 143kB 4.4MB/s
Collecting docutils (from doc8->-r docs-requirements.txt (line 2))
  Downloading docutils-0.14-py3-none-any.whl (543kB)
    100% |████████████████████████████████| 552kB 1.9MB/s
Collecting six (from doc8->-r docs-requirements.txt (line 2))
  Downloading six-1.11.0-py2.py3-none-any.whl
Collecting stevedore (from doc8->-r docs-requirements.txt (line 2))
  Downloading stevedore-1.28.0-py2.py3-none-any.whl
Collecting restructuredtext-lint>=0.7 (from doc8->-r docs-requirements.txt (line 2))
  Downloading restructuredtext_lint-1.1.3.tar.gz
Collecting argh (from Henson[sphinx]->-r docs-requirements.txt (line 3))
  Downloading argh-0.26.2-py2.py3-none-any.whl
Collecting watchdog>=0.8.3 (from Henson[sphinx]->-r docs-requirements.txt (line 3))
  Downloading watchdog-0.8.3.tar.gz (83kB)
    100% |████████████████████████████████| 92kB 5.1MB/s
Collecting sphinxcontrib-autoprogram>=0.1.3 (from Henson[sphinx]->-r docs-requirements.txt (line 3))
  Downloading sphinxcontrib_autoprogram-0.1.4-py2.py3-none-any.whl
Collecting packaging (from sphinx>=1.3->-r docs-requirements.txt (line 5))
  Downloading packaging-17.1-py2.py3-none-any.whl
Collecting alabaster<0.8,>=0.7 (from sphinx>=1.3->-r docs-requirements.txt (line 5))
  Downloading alabaster-0.7.10-py2.py3-none-any.whl
Collecting imagesize (from sphinx>=1.3->-r docs-requirements.txt (line 5))
  Downloading imagesize-1.0.0-py2.py3-none-any.whl
Collecting Jinja2>=2.3 (from sphinx>=1.3->-r docs-requirements.txt (line 5))
  Downloading Jinja2-2.10-py2.py3-none-any.whl (126kB)
    100% |████████████████████████████████| 133kB 4.7MB/s
Collecting babel!=2.0,>=1.3 (from sphinx>=1.3->-r docs-requirements.txt (line 5))
  Downloading Babel-2.5.3-py2.py3-none-any.whl (6.8MB)
    100% |████████████████████████████████| 6.8MB 196kB/s
Collecting sphinxcontrib-websupport (from sphinx>=1.3->-r docs-requirements.txt (line 5))
  Downloading sphinxcontrib_websupport-1.0.1-py2.py3-none-any.whl
Collecting requests>=2.0.0 (from sphinx>=1.3->-r docs-requirements.txt (line 5))
  Downloading requests-2.18.4-py2.py3-none-any.whl (88kB)
    100% |████████████████████████████████| 92kB 5.3MB/s
Requirement already satisfied: setuptools in ./.virtualenvs/test/lib/python3.6/site-packages (from sphinx>=1.3->-r docs-requirements.txt (line 5))
Collecting snowballstemmer>=1.1 (from sphinx>=1.3->-r docs-requirements.txt (line 5))
  Downloading snowballstemmer-1.2.1-py2.py3-none-any.whl (64kB)
    100% |████████████████████████████████| 71kB 4.1MB/s
Collecting Pygments>=2.0 (from sphinx>=1.3->-r docs-requirements.txt (line 5))
  Downloading Pygments-2.2.0-py2.py3-none-any.whl (841kB)
    100% |████████████████████████████████| 849kB 1.4MB/s
Collecting MarkupSafe>=0.9.2 (from Mako->alembic->-r docs-requirements.txt (line 1))
  Downloading MarkupSafe-1.0.tar.gz
Collecting pbr!=2.1.0,>=2.0.0 (from stevedore->doc8->-r docs-requirements.txt (line 2))
  Downloading pbr-3.1.1-py2.py3-none-any.whl (99kB)
    100% |████████████████████████████████| 102kB 5.6MB/s
Collecting PyYAML>=3.10 (from watchdog>=0.8.3->Henson[sphinx]->-r docs-requirements.txt (line 3))
  Downloading PyYAML-3.12.tar.gz (253kB)
    100% |████████████████████████████████| 256kB 2.7MB/s
Collecting pathtools>=0.1.1 (from watchdog>=0.8.3->Henson[sphinx]->-r docs-requirements.txt (line 3))
  Downloading pathtools-0.1.2.tar.gz
Collecting pyparsing>=2.0.2 (from packaging->sphinx>=1.3->-r docs-requirements.txt (line 5))
  Downloading pyparsing-2.2.0-py2.py3-none-any.whl (56kB)
    100% |████████████████████████████████| 61kB 5.4MB/s
Collecting pytz>=0a (from babel!=2.0,>=1.3->sphinx>=1.3->-r docs-requirements.txt (line 5))
  Downloading pytz-2018.3-py2.py3-none-any.whl (509kB)
    100% |████████████████████████████████| 512kB 1.1MB/s
Collecting certifi>=2017.4.17 (from requests>=2.0.0->sphinx>=1.3->-r docs-requirements.txt (line 5))
  Downloading certifi-2018.1.18-py2.py3-none-any.whl (151kB)
    100% |████████████████████████████████| 153kB 338kB/s
Collecting idna<2.7,>=2.5 (from requests>=2.0.0->sphinx>=1.3->-r docs-requirements.txt (line 5))
  Downloading idna-2.6-py2.py3-none-any.whl (56kB)
    100% |████████████████████████████████| 61kB 5.5MB/s
Collecting urllib3<1.23,>=1.21.1 (from requests>=2.0.0->sphinx>=1.3->-r docs-requirements.txt (line 5))
  Downloading urllib3-1.22-py2.py3-none-any.whl (132kB)
    100% |████████████████████████████████| 133kB 4.5MB/s
Building wheels for collected packages: alembic, SQLAlchemy, Mako, python-editor, restructuredtext-lint, watchdog, MarkupSafe, PyYAML, pathtools
  Running setup.py bdist_wheel for alembic ... done
  Stored in directory: /Users/leonardbedner/Library/Caches/pip/wheels/29/f4/c3/96037a3f2dcc2b8b59eff64746ea71bb5957f189c5a0877364
  Running setup.py bdist_wheel for SQLAlchemy ... done
  Stored in directory: /Users/leonardbedner/Library/Caches/pip/wheels/61/22/7c/7f7a00946677df88d7072eb5c225108fc5c1e7e68b56a72689
  Running setup.py bdist_wheel for Mako ... done
  Stored in directory: /Users/leonardbedner/Library/Caches/pip/wheels/33/bf/8f/036f36c35e0e3c63a4685e306bce6b00b6349fec5b0947586e
  Running setup.py bdist_wheel for python-editor ... done
  Stored in directory: /Users/leonardbedner/Library/Caches/pip/wheels/84/d6/b8/082dc3b5cd7763f17f5500a193b6b248102217cbaa3f0a24ca
  Running setup.py bdist_wheel for restructuredtext-lint ... done
  Stored in directory: /Users/leonardbedner/Library/Caches/pip/wheels/e5/ee/23/ed9eeb56acf6d0002865629097a1224db2998cf1d5a77550fe
  Running setup.py bdist_wheel for watchdog ... done
  Stored in directory: /Users/leonardbedner/Library/Caches/pip/wheels/3c/9c/be/e82ae5a37c19baf8abe88623d1f47d2d502bed7b54d4f34740
  Running setup.py bdist_wheel for MarkupSafe ... done
  Stored in directory: /Users/leonardbedner/Library/Caches/pip/wheels/88/a7/30/e39a54a87bcbe25308fa3ca64e8ddc75d9b3e5afa21ee32d57
  Running setup.py bdist_wheel for PyYAML ... done
  Stored in directory: /Users/leonardbedner/Library/Caches/pip/wheels/2c/f7/79/13f3a12cd723892437c0cfbde1230ab4d82947ff7b3839a4fc
  Running setup.py bdist_wheel for pathtools ... done
  Stored in directory: /Users/leonardbedner/Library/Caches/pip/wheels/60/0d/2a/ffe065d190b580be6af9a862c68ed6a8a89c1778bd5c7ba7b8
Successfully built alembic SQLAlchemy Mako python-editor restructuredtext-lint watchdog MarkupSafe PyYAML pathtools
Installing collected packages: SQLAlchemy, MarkupSafe, Mako, python-editor, six, python-dateutil, alembic, chardet, docutils, pbr, stevedore, restructuredtext-lint, doc8, argh, PyYAML, pathtools, watchdog, pyparsing, packaging, alabaster, imagesize, Jinja2, pytz, babel, sphinxcontrib-websupport, certifi, idna, urllib3, requests, snowballstemmer, Pygments, sphinx, sphinxcontrib-autoprogram, Henson, sphinx-rtd-theme
  Running setup.py develop for Henson
Successfully installed Henson Jinja2-2.10 Mako-1.0.7 MarkupSafe-1.0 PyYAML-3.12 Pygments-2.2.0 SQLAlchemy-1.2.5 alabaster-0.7.10 alembic-0.9.8 argh-0.26.2 babel-2.5.3 certifi-2018.1.18 chardet-3.0.4 doc8-0.8.0 docutils-0.14 idna-2.6 imagesize-1.0.0 packaging-17.1 pathtools-0.1.2 pbr-3.1.1 pyparsing-2.2.0 python-dateutil-2.7.0 python-editor-1.0.3 pytz-2018.3 requests-2.18.4 restructuredtext-lint-1.1.3 six-1.11.0 snowballstemmer-1.2.1 sphinx-1.7.1 sphinx-rtd-theme-0.2.4 sphinxcontrib-autoprogram-0.1.4 sphinxcontrib-websupport-1.0.1 stevedore-1.28.0 urllib3-1.22 watchdog-0.8.3

Oddly enough, if I create requirements.in from docs-requirements.txt and do a pip-compile --upgrade requirements.in, the resulting requirements.txt has sphinx==1.7.0:

leonardbedner@Leonard-Bedner-MacBook-Pro ~>  cp docs-requirements.txt requirements.in
leonardbedner@Leonard-Bedner-MacBook-Pro ~>  pip-compile --upgrade requirements.in
Could not find a tag or branch '9d1558f03025232688f3c85e9383925aa7169842', assuming commit.
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile --output-file requirements.txt requirements.in
#
-e git+https://github.com/iheartradio/Henson.git@9d1558f03025232688f3c85e9383925aa7169842#egg=Henson[sphinx]
alabaster==0.7.10         # via sphinx
alembic==0.9.8
argh==0.26.2              # via watchdog
babel==2.5.3              # via sphinx
certifi==2018.1.18        # via requests
chardet==3.0.4            # via doc8, requests
doc8==0.8.0
docutils==0.14            # via doc8, restructuredtext-lint, sphinx
idna==2.6                 # via requests
imagesize==1.0.0          # via sphinx
jinja2==2.10              # via sphinx
mako==1.0.7               # via alembic
markupsafe==1.0           # via jinja2, mako
packaging==17.1           # via sphinx
pathtools==0.1.2          # via watchdog
pbr==3.1.1                # via stevedore
pygments==2.2.0           # via sphinx
pyparsing==2.2.0          # via packaging
python-dateutil==2.7.0    # via alembic
python-editor==1.0.3      # via alembic
pytz==2018.3              # via babel
pyyaml==3.12              # via watchdog
requests==2.18.4          # via sphinx
restructuredtext-lint==1.1.3  # via doc8
six==1.11.0               # via doc8, packaging, python-dateutil, sphinx, sphinxcontrib-autoprogram, stevedore
snowballstemmer==1.2.1    # via sphinx
sphinx-rtd-theme==0.2.4
sphinx==1.7.0
sphinxcontrib-autoprogram==0.1.4
sphinxcontrib-websupport==1.0.1  # via sphinx
sqlalchemy==1.2.5         # via alembic
stevedore==1.28.0         # via doc8
urllib3==1.22             # via requests
watchdog==0.8.3

This obviously goes away if sphinx>1.3 is removed from docs-requirements.txt, though I suspect that was in there for a reason. Perhaps it's something that should be moved into setup.py as well, and subsequently removed from Henson-Database and anything else that uses it?

[dirn]

The 1.3 requirement was added in d961e92. It’s needed for Henson’s docs; it won’t necessarily be needed for people writing their own extensions.

As for restricting 1.7.1, you can chain multiple requirements.

sphinx>=1.3,<1.7.1