Python3 and DM

RFC-60Tim Jenness

2

Python 3

• Version 3.0 released in 2008• 3.5 to be released in September• 2.7 was released in 2010– Critical bug fixes only.– Supported (nominally) until 2020• This date is not within LSST operations.

• All the new features are turning up in Python3

3

What has Python3 ever given us?• Strings are Unicode characters and not a collection of

bytes 👍✔︎��Ευχαριστώ• Consistent use of iterators• Improved package handling (namespace packages)• Matrix operator (@)• Improved handling of super() and metaclasses• Many improvements to standard library: asyncio, single-

dispatch generic functions.• Parser improvements• Not all of these features are available on PyPI or using __future__

4

To lead or to follow?

From HipChat, May 2015:“[LSST Python 3 adoption] depends on the rest of the community

[migrating].”

From the Astropy mailing list, June 2015:“My highly subjective experience has been that in astronomy, the biggest sticking point is at facilities. E.g., tools provided by some observatories to prepare for proposals or observations, or single-use scripts like specialized finder charts and the like. Often that code gets written once and then left alone (the author may even be long gone), so if it's not 3.x compatible it probably never will be without a specific concerted effort. In contrast, “projects” (like astropy) are most there in terms of Py 3.x support.”

5

Community uptake

• 2015 Python survey by Robitaille indicates the community is (slowly) migrating to Python3

• Python books have begun defaulting to Python 3 (e.g. Fluent Python, Effective Python)

• Python 3 is becoming the default for new users.

• LSST should not impede this migration.

6

Proposed Migration Plan

• Support 2.7 and 3.4+ simultaneously.• Deprecate 2.7 when compelling need to adopt

native 3.x features.• Jenkins CI system running with Python3 (EUPS and

Scons excepted)• Generate test coverage report• Adopt future module to handle the 2.7/3.x bridge.– future lets you write code using Python3 style– Provides a futurize script to modernize current code

base.

7

Concerns

• Poor test coverage so migration may break working code. – If we aren’t testing it the code could be broken

anyhow.• Big bang conversion versus incremental.– Big bang is less confusing as people understand

the switch and start coding in Python3 by default.– Incremental conversion risks back-sliding if the

code has never been tested on Python3 in CI.