All releases BROKEN due to json-table-schema name change

[davidread]

json-table-schema is a broken dependency as of yesterday. This affects current and previous releases on pypi.

To fix this at this end we've changed the dep #143 and now messytables installs from source again, but it needs a release to pypi. I don't have permission for this.

(test)co@precise64:/tmp$ pip install messytables
Downloading/unpacking messytables
  Downloading messytables-0.15.0.tar.gz
  Running setup.py egg_info for package messytables

Downloading/unpacking xlrd>=0.8.0 (from messytables)
  Downloading xlrd-0.9.4.tar.gz (322Kb): 322Kb downloaded
  Running setup.py egg_info for package xlrd

Downloading/unpacking python-magic>=0.4.6 (from messytables)
  Downloading python-magic-0.4.10.tar.gz
  Running setup.py egg_info for package python-magic

    no previously-included directories found matching 'test'
Downloading/unpacking chardet>=2.3.0 (from messytables)
  Downloading chardet-2.3.0.tar.gz (164Kb): 164Kb downloaded
  Running setup.py egg_info for package chardet

    warning: no files found matching 'COPYING'
    warning: no files found matching '*.html' under directory 'docs'
    warning: no files found matching '*.css' under directory 'docs'
    warning: no files found matching '*.png' under directory 'docs'
    warning: no files found matching '*.gif' under directory 'docs'
Downloading/unpacking python-dateutil>=2.4.2 (from messytables)
  Downloading python-dateutil-2.4.2.tar.gz (209Kb): 209Kb downloaded
  Running setup.py egg_info for package python-dateutil

Downloading/unpacking lxml>=3.2 (from messytables)
  Downloading lxml-3.5.0b1.tar.gz (3.8Mb): 3.8Mb downloaded
  Running setup.py egg_info for package lxml
    Building lxml version 3.5.0b1.
    Building without Cython.
    Using build configuration of libxslt 1.1.26
    Building against libxml2/libxslt in the following directory: /usr/lib/x86_64-linux-gnu

    warning: no previously-included files found matching '*.py'
Downloading/unpacking requests (from messytables)
  Downloading requests-2.8.1.tar.gz (480Kb): 480Kb downloaded
  Running setup.py egg_info for package requests

Downloading/unpacking html5lib (from messytables)
  Downloading html5lib-1.0b8.tar.gz (889Kb): 889Kb downloaded
  Running setup.py egg_info for package html5lib

Downloading/unpacking json-table-schema>=0.2 (from messytables)
  Downloading json-table-schema-0.5.0.tar.gz
  Running setup.py egg_info for package json-table-schema
    json-table-schema has been replaced by jsontableschema. See https://github.com/okfn/json-table-schema-py-old for details.
    Traceback (most recent call last):
      File "<string>", line 14, in <module>
      File "/tmp/test/build/json-table-schema/setup.py", line 16, in <module>
        with io.open(README_PATH, mode='r+t', encoding='utf-8') as stream:
    IOError: [Errno 2] No such file or directory: '/tmp/test/build/json-table-schema/README.md'
    Complete output from command python setup.py egg_info:
    json-table-schema has been replaced by jsontableschema. See https://github.com/okfn/json-table-schema-py-old for details.

Traceback (most recent call last):

  File "<string>", line 14, in <module>

  File "/tmp/test/build/json-table-schema/setup.py", line 16, in <module>

    with io.open(README_PATH, mode='r+t', encoding='utf-8') as stream:

IOError: [Errno 2] No such file or directory: '/tmp/test/build/json-table-schema/README.md'

----------------------------------------
Command python setup.py egg_info failed with error code 1 in /tmp/test/build/json-table-schema
Storing complete log in /home/co/.pip/pip.log

[pwalsh]

@davidread the json-table-schema dependency in messy tables should be pinned to the last version of json-table-schema that worked. It is an old, unsupported library and I announced changes here a few months ago:

#135

I did not look not at the specific problem here, but, pinning dependencies should have prevented suprises.

[pwalsh]

After looking at this and seeing what happened in #143 I urge anyone who is committing to messytables to:

1. Pin dependencies in order to ensure deterministic builds (or, be flexible when things will break)
2. Have a look at the API of http://github.com/okfn/jsontableschema-py which is up to date with the JTS spec, and maintained, and talk with myself or @brew about integrating it into messy tables, as I suggested a few months back in JSON Table Schema in MessyTables #135

[brew]

Okay - the new version of json-table-schema, deprecating it, has been removed from Pypi. The most recent version available there is 0.2.1.

[davidread]

Great, thanks @brew for removing the problematic json-tables-schema version 0.5 from PyPI.

Looks like there is a fix in the json-table-schema source too, described as version 0.5.1: okfn/json-table-schema-py-old@0be548e

[davidread]

pip install messytables now works, closing. Discussion about pinning json-table-schema can continue on #135. Thanks all.

[davidread]

Ok, #135 isn't really about pinning after all.

@pwalsh said:

I did not look not at the specific problem here, but, pinning dependencies should have prevented suprises.

However the article he links to reminds us:

WARNING: Don’t pin by default when you’re building libraries! Only use pinning for end products.

which is the concern that I have.

[pwalsh]

@davidread I think you are misreading the quote. To me, the quote says: "when you are building your stuff (local development), don't pin. Only pin when you publish your stuff (the product)". A MessyTables release is a product.

[davidread]

The reasoning is because a product uses lots of libraries with lots of dependencies. If two libraries have a common dependency, then it is clearly helpful for them to both specify the range of versions that they are compatible with. If they both pin to a particular version, purely so their own tests pass, then chances are they won't agree and then setuptools prevents you from running your software until you hack the libraries' setup.py to agree and then reinstall them.

[pwalsh]

@davidread yes I agree on ranges, but not on open ended deps. as we have here in messytables. Again, it is just a risk calculation. There is simply risk in unpinned (or, unranged) deps. In any event, this case is quite particular due to us taking over abandoned code.

[davidread]

Cool with ranges then.

[brew]

The Python Packaging Guide recommends the flexible pinning style for install_requires, and pinning specific versions in requirements.txt: https://python-packaging-user-guide.readthedocs.org/en/latest/requirements/#install-requires-vs-requirements-files

So requirements.txt represents the list of specific versions for a repeatable installation, whereas install_requires is a list of minimally required abstract dependencies.

I'm not sure that would help much in this case, as install_requires is what's used by python setup.py install, unless the dependencies was capped at the upper end.