bonus_importing_modules.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Modules"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Most of the functionality in Python is provided by *modules*. The Python Standard Library is a large collection of modules that provides *cross-platform* implementations of common facilities such as access to the operating system, file I/O, string management, network communication, and much more."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## References"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " * The Python Language Reference: http://docs.python.org/3/reference/index.html\n",
    " * The Python Standard Library: http://docs.python.org/3/library/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "To use a module in a Python program it first has to be imported. A module can be imported using the `import` statement."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "For example, to import the module `math`, which contains many standard mathematical functions, we can do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import math"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This includes the whole module and makes it available for use later in the program. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "For example, we can do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n"
     ]
    }
   ],
   "source": [
    "import math\n",
    "\n",
    "x = math.cos(2 * math.pi)\n",
    "\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Alternatively, we can chose to import all symbols (functions and variables) in a module to the current namespace (so that we don't need to use the prefix \"`math.`\" every time we use something from the `math` module:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n"
     ]
    }
   ],
   "source": [
    "from math import cos, pi\n",
    "\n",
    "x = cos(2 * pi)\n",
    "\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "This is called **selective Import**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "This pattern can be very convenient, but in large programs that include many modules it is often a good idea to keep the symbols from each module in their own namespaces, by using the `import math` pattern. This would elminate potentially confusing problems with name space collisions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Btw, in case of `namespace collisions` (or to avoid `namespace pollution`) we may use the `as` **keyword**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "from math import cos as cosine  # Now the `cos` function can be referenced as `cosine`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.123233995736766e-17"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cosine(pi/2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Finally, if we want to import **everything** from a module, we may the `*` character:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "from math import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cosine Function:  -1.0\n",
      "Sin Function:  1.2246467991473532e-16\n",
      "Logarithm:  1.0\n",
      "Power function:  27.0\n"
     ]
    }
   ],
   "source": [
    "print(\"Cosine Function: \", cos(pi))\n",
    "print(\"Sin Function: \", sin(pi))\n",
    "print(\"Logarithm: \", log(e))\n",
    "print(\"Power function: \", pow(3, 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Looking at what a module contains, and its documentation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Once a module is imported, we can list the symbols it provides using the `dir` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'copysign', 'cos', 'cosh', 'degrees', 'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'hypot', 'isfinite', 'isinf', 'isnan', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'log2', 'modf', 'pi', 'pow', 'radians', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'trunc']\n"
     ]
    }
   ],
   "source": [
    "import math\n",
    "\n",
    "print(dir(math))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "And using the function `help` we can get a description of each function (almost .. not all functions have docstrings, as they are technically called, but the vast majority of functions are documented this way). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on built-in function log in module math:\n",
      "\n",
      "log(...)\n",
      "    log(x[, base])\n",
      "    \n",
      "    Return the logarithm of x to the given base.\n",
      "    If the base not specified, returns the natural logarithm (base e) of x.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(math.log)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We can also use the `help` function directly on modules: Try\n",
    "\n",
    "    help(math) \n",
    "\n",
    "Some very useful modules form the Python standard library are `os`, `sys`, `math`, `shutil`, `re`, `subprocess`, `multiprocessing`, `threading`. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How Import Works"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because imports are at the heart of program structure in **Python**, this section goes into more formal detail on the import operation to make this process less abstract.\n",
    "\n",
    "In more details, this is particularly important as in Python, the import process is **not** just a textual insertions of one file into another. \n",
    "\n",
    "It really consists of a set of runtime operations that perform three distinct steps the first time a program imports a given file:\n",
    "\n",
    "1. *Find* the module file\n",
    "2. *Compile* it to byte code\n",
    "3. *Run* the module's code to build the objects it defines\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Find It"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, Python must locate the (module) file the code is trying to `import` (omitting extension and directory paths).\n",
    "\n",
    "To do so, Python applies a proper *module search path* algorithm. In particular, Python looks for the module to import in the following directories, and in the following order:\n",
    "\n",
    "1. The Home Directory of the program\n",
    "2. `PYTHONPATH` directories\n",
    "3. Standard library directories\n",
    "4. The contents of any `.pth` file (if present)\n",
    "5. The `site-packages` home of third-party extensions (e.g., `/usr/lib/python3/site-packages/`)\n",
    "\n",
    "(**More on this, later**)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Compile it"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After finding a source code file that matches an import statement by traversing the module search path, Python next compiles it to byte code, if necessary.\n",
    "\n",
    "In particular, in this step, the two choices are:\n",
    "\n",
    "* **Compile**: \n",
    "\n",
    "If the byte code file is older than the source file (i.e., if you’ve changed the source) or was created by a different Python version, Python automatically regenerates the byte code when the program is run.\n",
    "\n",
    "This model is modified somewhat in Python 3.2+ where byte code files are segregated in a `__pycache__` subdirectory and named with their Python version to avoid contention and recompiles when multiple Pythons are installed. \n",
    "\n",
    "This obviates the need to check version numbers in the byte code, but the timestamp check is still used to detect changes in the source.\n",
    "\n",
    "* **Don't Compile**:\n",
    "\n",
    "If, on the other hand, Python finds a `.pyc` byte code file that is not older than the corresponding `.py` source file and was created by the same Python version, it skips the source-to-byte-code compile step.\n",
    "\n",
    "In addition, if Python finds only a byte code file on the search path and no source, **it simply loads the byte code directly**.\n",
    "\n",
    "This means you can ship a program **as just byte code files** and avoid sending source. \n",
    "\n",
    "In other words, the compile step is *by-passed* if possible to speed program startup."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Run it"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The final step of an import operation executes the byte code of the module.\n",
    "\n",
    "All statements in the file are run in turn, from top to bottom, and any assignments made to names during this step generate attributes of the resulting module object. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modules & Packages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Modules** are probably best understood as simply packages of names - i.e., *places to define names you want to make visible to the rest of a system*. \n",
    "\n",
    "Technically, modules usually correspond to **files**, and Python creates a **module object** to contain all the names assigned in a module file. \n",
    "\n",
    "But in simple terms, modules are just namespaces (places where names are created), and the names that live in a module are called its *attributes*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to a module name, an import can name a *directory path*. \n",
    "\n",
    "A directory of Python code is said to be a **package**, so such imports are known as package imports. \n",
    "\n",
    "In effect, a package import turns a directory on your computer into another Python namespace, with *attributes* corresponding to the *subdirectories* and *module files* that the directory contains."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Package Relative Imports"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The coverage of package imports so far has focused mostly on importing package files from *outside* the package.\n",
    "\n",
    "Within the package itself, imports of same-package files can use the same full path syntax as imports from outside the package. However, package files can also make use of special *intrapackage* search rules to simplify import statements. \n",
    "\n",
    "That is, rather than listing package import paths, imports within the package can be **relative** to the package."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The way this works is version-dependent: \n",
    "\n",
    "**Python 2.X** implicitly searches package directories first on imports, while **Python 3.X** requires explicit relative import syntax in order to import from the package directory. \n",
    "\n",
    "This 3.X change can enhance code readability by making same-package imports more obvious, but it’s also incompatible with 2.X and may break some programs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Python 3.3 changes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For imports in packages, though, Python 3.X introduces two changes:\n",
    "\n",
    "* It modifies the module import search path semantics to skip the package’s own directory by default. Imports check only paths on the sys.path search path. These are known as **absolute imports**.\n",
    "\n",
    "* It extends the syntax of `from` statements to allow them to explicitly request that imports search the package’s directory only, with leading dots. This is known as **relative import** syntax."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Examples:\n",
    "\n",
    "```python\n",
    "\n",
    "from . import spam\n",
    "\n",
    "from .spam import name\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Why Relative Imports?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consider the following package directory:\n",
    "\n",
    "```\n",
    "    mypkg\\ \n",
    "        __init__.py\n",
    "        main.py \n",
    "        string.py\n",
    "```\n",
    "\n",
    "This defines a package named `mypkg` containing modules named `mypkg.main` and `mypkg.string`. \n",
    "\n",
    "Now, suppose that the main module tries to import a module named `string`. \n",
    "\n",
    "In **Python 2.X** and earlier, Python will first look in the mypkg directory to perform a relative import. \n",
    "\n",
    "It will find and import the `string.py` file located there, assigning it to the name string in the `mypkg.main`\n",
    "module’s namespace.\n",
    "It could be, though, that the intent of this import was to load the Python standard library’s **string module** instead.\n",
    "\n",
    "Unfortunately, in these versions of Python, there’s **no straightforward** way to ignore `mypkg.string` and look for the standard library’s string module located on the module search path.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On the other hand, in Python 3.3+, you could:\n",
    "\n",
    "```python\n",
    "\n",
    "from string import punctuations  # standard library module\n",
    "from .string import *            # relative import (intrapackage)\n",
    "\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}