Python Programming Language

Code
Python
Published

October 2, 2016

Modified

July 12, 2023

Language

print

…is a build in function in Python 3

print('s')              # s
print('s',end='')       # disable newline
print(1,2,3)            # 1 2 3
print(1,2,3,sep=',')    # 1,2,3
print('s','r',sep='/')  # s/r
print([1,2,3])          # [1, 2, 3]
print(*[1,2,3])         # 1 2 3

Variables

The assignment operation name = object references values to variables:

  • Assignment declares and initializes a variable with a value
  • Most suitable data type for assignment is select automatically by the interpreter
  • A cascading assignment modifies an object referenced by a variable
x = 1.2 + 8     # assign right expression
x = y = z = 0   # assign a single value to multiple variables
x,y,z = 1,2,3   # multiple assignments
x,y = y,x       # variable swap
a,*b = [1,2,3]  # unpacking a sequence
del x           # remove variable
x = None        # undefined value
id(x)           # return object memory address for variable x
hex(id(x))      # ...hexadecimal memory address
globals()       # dictionary of global variables

Naming conventions:

var_with_sep    # underscore separated downcase letters
_               # unused variable (i.e. within a loop)
var             # public
_var            # internal
var_            # convention to avoid conflict keyword
__var           # private use in class
_var_           # protected use in class
__var__         # "magic: method or attribute
  • Names are case sensitive
  • Python keywords can not be used as names

Scope and binding:

  • Variables are always assigned to local scope current code block (i.e. within a function body)
  • nonlocal assigns to variables in an outer scope (but not global)
  • global assigns to a variable in the module’s top level aka global scope
globals().keys()         # prints all variable names in global scope
locals().keys()          # prints all variable names in local scope

Numbers

Type Description
int Integer number are of arbitrary size
float Floating point number precision depends on the implementation and system architecture
complex Complex number

Literals

123             # integer
-123            # negative signed integer
9.23            # float 
-1.7e-6         # scientific notation
2 + 1j          # complex number
0b010           # 2         binary notation
0o642           # 418       octal
0xf3            # 243       hexadecimal
None            # indefined value

Types, and type-casting:

type(1)                  # <class 'int'>
type(1.2)                # <class 'float'>
type(None)               # <class 'NoneType'>
## string to numerical
int('123')               # 123
float('1.23')            # 1.23
## Type check ##
isinstance(10,int)       # True
isinstance(1.234,float)  # True

Arithmetic

1+2                       # 3
1-2                       # -1
1*2                       # 2
1/2                       # 0.5
5%3                       # 2 (remainder)
2**3                      # 8
## Built-in functions ##
abs(-2.3)                 # 2.3
round(1.6666,2)           # 1.67
pow(2,3)                  # 8
sum((1,2,3,4,5))          # 15
max([1,2,3,4,5])          # 5
min([1,2,3,4,5])          # 1

Logic

## Booleans
True               # logic true
False              # logic fales
not True           # False
True and True      # True
True and False     # False
True or False      # True
False or False     # False
1                  # True
0                  # False
## Comparison with boolean results
1 < 2              # True
1 > 2              # False
1 <= 1             # True
2 >= 3             # False
1 == 1             # True
1 != 1             # False
## Ternary conditional statement
1 if True else 2   # 1
1 if False else 2  # 2

Strings

Escape sequences interpreted according to rules similar to those used by Standard C

''                                 # empty line
'\n'                               # blank line

# Double quote (escape with \)
"a\"bc"                            # 'a"bc'

# Single quote
'a\'bc'                            # "a'bc"

## Raw strings ##
r"\t\n\\"                          # '\\t\\n\\\\'
R"\"\n\""                          # '\\"\\n\\"'

Raw-string prefixed with r or R use different rules for backslash escape sequences

# Built-in function to get ASCII codes
ord('a')                           # 97

# convert to sequence pr collection types
list('abc')                        # ['a', 'b', 'c']
set('abc')                         # {'b', 'a', 'c'}
tuple('abc')                       # ('a', 'b', 'c')

Escape sequences:

\\         Backslash (\)
\'         Single-quote (')
\"         Double-quote (")
\a         ASCII bell (BEL)
\b         ASCII backspace (BS)
\f         ASCII formfeed (FF)
\n         ASCII linefeed (LF)
\N{name}   Character named name in the Unicode database (Unicode only)
\r         ASCII carriage return (CR)
\t         ASCII horizontal tab (TAB)
\uxxxx     Character with 16- bit hex value xxxx (Unicode only)
\Uxxxxxxxx Character with 32- bit hex value xxxxxxxx (Unicode only)
\v         ASCII vertical tab (VT)
\ooo       Character with octal value oo
\xhh       Character with hex value hh

Format

Legacy format operator…

"%s, %s" % ('s','t')                           # 's, t'

Global build in function

format(10.0,"7.3g")                            # '     10'

Use of the str.format() function…

"{}|{}".format(1,2)                            # '1|2'

# ...positional index
"{1},{0},{2}".format('r','s','t')              # 's,r,t'

# ...parameter names
"{b}{a}".format(a='s',b='t')                   # 'ts'

# ...nested data structures
'{d[2]},{d[0]}'.format(d=['r','s','t'])        # 't,r'
'{d[b]},{d[a]}'.format(d={'a':1,'b':2})        # '2,1'

Padding…

"{:4d}".format(123)                            # ' 123'
'{:06.2f}'.format(3.14159)                     # '003.14'
"{:>3}".format('s')                            # '  s'
"{:.<4}".format('s')                           # 's...'
"{:^5}".format('s')                            # '  s  '

# ...parametrized format
'{:{a}{w}}'.format('s',a='>',w=5)              # '    s'

# ...positional arguments
'{:{}{}{}.{}}'.format(2.7182,'>','+',10,3)     # '     +2.72'

Since Python 3.6 f-strings…

  • …formatted string literals …less verbose then str.format()
  • …f as string literal prefix …{} curly braces containing expressions …will be replaced with their values
  • …evaluated at runtime …allow any valid Python expressions
v = 's'
f'text{v}'                                     # 'texts'
f"{1 + 1}"                                     # '2'
F'{v.upper()}'                                 # 'S'

Multi line f-strings…

# ...place an f in front of each line
v = f'line' \
    f'line'

# ...easier...
f"""
   multi lines text
"""

Manipulation

# ...concatenation
"s" + "t"                          # 'st'
"s " + str(123)                    # 's 123'

# ...leading, trailing white-space management 
" s ".strip()                      # 's'
' s \n'.rstrip()                   # ' s'
' s'.lstrip()                      # 's'

# ...cut by separator
"s\nt\nr\n".splitlines()           # ['s', 't', 'r']
"s|t".split("|")                   # ['s', 't']
'p:q:r:s'.rsplit(':',2)            # ['p:q', 'r', 's']

# ...return tuple, preserve the delimiter
"s:r:t".partition(':')             # ('s', ':', 'r:t')
"s:r:t".rpartition(':')            # ('s:r', ':', 't')

# ...join by separator
':'.join('123')                    # '1:2:3'

# ...matching
't' in 'str'                       # True
"st".startswith('s')               # True
'str'.endswith('r')                # True
'   '.isspace()                    # True
'12'.isdecimal()                   # True
'1.2'.isdecimal()                  # False
'strts'.find('r')                  # 2

# ...replacement
'srtr'.replace('r','R')            # 'sRtR'
'srtr'.replace('r','R',1)          # 'sRtr'

Control & Loop Constructs

Block indentation:

  • Blocks of code begin with a colon (:) and contain indented lines below (no ending identifier)
  • Always use 4 spaces for indentation
  • Single line statements may be put on the same line (considered bad style)
  • Empty blocks cause an IndentationError, use pass (a command that does nothing)`

Conditional Statements

  • Can go with multiple elif followed by a single else
  • First true condition is executed
if x < 2:
    ...
elif x > 2:
    ...
else:
    ...

Python has no direct analogous to a switch-case statement.

def switch(case):
    return {
        'a': 'A',
        'b': 'B'
    }.get(case,None)

print(switch('a'))                   # A
def a():
    return 'A'

def b():
    return 'B'

switch = { 'a': a(), 'b': b() }

print(switch['a'])                   # A

Loops

Code blocks repeated through a number of loops (iterations):

  • The break statement executed within a loop breaks the control flow out of the loop immediatly
  • A continue statement skips to the next interation of a loop

A while loop executed a code block until the loop condition is false:

x = 0
while True:                    # endless loop
    print(x)
    if x == 3:                 # break condition
        break                  # break the loop
    x += 1                     # increment

for loops iterate over a collection of items, and execute a code block for each element:

for x in (0,1,2,3):
    if x == 2:                 # skip condition
        continue               # skip the rest of the code block
    print(x)

range is a function returning a series of numbers in an iterable from, commenly use with for loops:

for x in range(3):
    print(x)
else:
    print('no break')

Loops can optionally have an else clause executed if the iteration is completed successfull.

# couple of range examples
list(range(5))           # [0, 1, 2, 3, 4]
tuple(range(4,12))       # (4, 5, 6, 7, 8, 9, 10, 11)
tuple(range(0,10,2))     # (0, 2, 4, 6, 8)
tuple(range(100,0,-10))  # (100, 90, 80, 70, 60, 50, 40, 30, 20, 10)

Sequences & Collections

Tuple

A tuple is a immutable sequences of elements with an index number and a value.

# literals for tuples
()
(1,2,3)
('a','b','c')
type((1,2))             # tuple
# assign tuple to a variable t
t = ('a','b','c')
t                       # ('a', 'b', 'c')
# does a value exists in a tuple t 
'a' in t                # True
'a' not in t            # False
# access elements in a tuple t
t[0]                    # 'a'
t[-2]                   # 'b'
t[:2]                   # ('a', 'b')
t[2:]                   # ('c',)
# find index of value
t.index('b')            # 1
# number of occurrences of a given value
t.count('b')            # 1
# concatenation return a new tuple
(1, 2, 3) + (4, 5, 6)   # (1, 2, 3, 4, 5, 6)
# number of elements in a tuple
len(('a','b'))          # 2
# sort a tuple
sorted(('b','a'))       # ['a', 'b']
# smallest/largest value in tuple
min(34,23,45)           # 23'
max(34,23,45)           # 45
# convert a string to tuple
tuple('abcd')           # ('a', 'b', 'c', 'd')
# convert a list into a tuple
tuple([1,2,3,4,5])      # (1, 2, 3, 4, 5)

List

A list is a sequences (with dynamic length) of elements with an index number and a value.

type([1,'a',2])         # list
[]                      # empty list
l = [1,2,'a',3,'b']     # list assignment to variable
# append value to the end of the list list
l.append(4)             
# append multiple values to list
l.extend([5,'c'])
# print a list
l                       # [1, 2, 'a', 3, 'b', 4, 5, 'c']
# does a value exists in a list
'a' in l                # True
# find index of a value
l.index('a')            # 2
# number of elements in list
len(l)                  # 8
# access values by index
l[:]                    # [1, 2, 'a', 3, 'b', 4, 5, 'c']
l[:4]                   # [1, 2, 'a', 3] 
l[-4]                   # 'b'
l[4:-1]                 # ['b', 4, 5]
# delete element by value 
l.remove(3)             
l                       # [1, 2, 'a', 'b', 4, 5, 'c']
# remove index from list and return its value
l.pop(-1)               # 'c' 
l                       # [1, 2, 'a', 'b', 4, 5]
# concatenation return a new list
l + [6,7,'c']           # [1, 2, 'a', 'b', 4, 5, 6, 7, 'c']
l                       # [1, 2, 'a', 'b', 4, 5]
# delete element by index
del l[3]
l                       # [1, 2, 'a', 4, 5]
# number of occurrences of a given value
l.count('a')            # 1
# shallow copy of the list
m = l.copy()
m[0] = 'z'
l                       # [1, 2, 'a', 4, 5]
m                       # ['z', 2, 'a', 4, 5]
# iteration
for i in l: print(l.index(i),i)
# convert a string into a list
list("abcde")           # ['a', 'b', 'c', 'd', 'e']
# convert tuple into a list
list((1,2,3,4,5,6))     # [1, 2, 3, 4, 5, 6]

Set

A set is a unordered collections of unique elements:

{}
{1,2,'a'}               # {1, 2, 'a'}
{1,2,1,2,1}             # {1, 2}
type({1,2})             # set
# union
{1,2} | {2,3,4}         # {1, 2, 3, 4}
# intersection
{1,2} & {2,3,4}         # {2}
# difference
{1,2} - {2,3,4}         # {1}
# symmetric difference
{1,2} ^ {2,3,4}         # {1, 3, 4}
# does a value exists in set
1 in {1,2,3}            # True
'a' in {1,2,3}          # False
# iterator
for v in {1,2,3}:

Dictionary

A dictionary is a associative list with defined keys and values.

{}
{1:'a','b':2}          # {1: 'a', 'b': 2} 
# add/change key/value
d = {}
d['a'] = 1             
d['b'] = 2             
d                      # {'a': 1, 'b': 2}
d['b']                 # 2
# remove element by key
del d['a']
d                      # {'b': 2}
# merge 
d.update({'c':3, 'd':4})
d                      # {'b': 2, 'c': 3, 'd': 4}
# remove element, return value
d.pop('c')             # 3
d                      # {'b': 2, 'd': 4}
# get a value
d.get('b')             # 2
d                      # {'b': 2, 'd': 4}
# remove all elements
d.clear()
d                      # {}
# iterators
for k in d.keys()
for v in d.values()
for k,v in d.items()
# conversion
dict(a=1,b=2,c=3)                  # {'a': 1, 'b': 2, 'c': 3}
dict(zip(['a','b'],[1,2]))         # {'a': 1, 'b': 2}

Subprocess

Simplest method to execute a command and iterate over the output by line…

import subprocess
text = subprocess.getoutput('ls -l')
for line in text.splitlines():
    # ...

File I/O

Use open() to store data in a file and read it back:

  • The path to the file is the first argument.
  • Followed by the access mode: r (read), w (write), a (append)
  • Encoding: ‘ascii’, ‘utf8’
txt = "1st line\n2nd line\n3rd line\n4th line"
path = '/tmp/file.txt'
## write into a file
f = open(path,'w',encoding='utf8')
f.write(txt)                   # write into the file
f.flush()                      # write cache
f.close()                      # close when finished
f = open(path,'r')
f.name                         # path ot the file '/tmp/file.txt'
f.read()                       # read entire file
# iterate over file content
for l in f.readlines()         # by line
for l in iter(f): 
for l in f.read().split('\n')  # by seperator

Using a context manager:

with open('/etc/hosts') as f:
    for _ in f.readlines():
        print(_)

Modules

A module is a files containing Python definitions and statements.

dir(__builtins__)        # list build-in functions
help(type)               # help text for a given modul/function

Import a module to use its functions:

import math
math.pi          # 3.141592653589793
math.sqrt(81)    # 9.0

Load module, and allow direct access to functions:

from math import pi,e,sin,log
sin(pi/4)       # 0.7071067811865475
log(e**2)       # 2.0

Define an alias for a module:

import math as m
m.pi           # 3.141592653589793

Cf. Python Module Index

Functions

Functions are defined using the def keyword

  • Followed by the function name (identifier), i.e. f
  • Arguments given between parentheses followed by : (colon)
  • The function body (blocks) must be indented
  • The return keyword passes values from the function
# includes argument with default value
def f(x,y,z=3):
    """documentation"""
    return (x,y,z)
f(1,2)                        # (1, 2, 3)
# variable positional arguments as tuple
def g(x,*y):
    return (x,y)
g(1,2,3,4,5,6)                # (1, (2, 3, 4, 5, 6))
# variable named arguments as dict
def h(x,**y):
    return [x,y]
h(1,a=1,b=2)                  # [1, {'a': 1, 'b': 2}]

Lambda

The Lambda expression (anonymous function) creates a function objects with following notation:

lambda: <<args,...>> : <<expression>>

Semantically lambda is a shorthand for a function definition:

  • Lambda functions can be used wherever function objects are required.
  • It can have any number of arguments before the colon.
  • The function body is syntactically restricted to a single expression.
  • Typically used as nameless function as argument to a higher-order function.
f = lambda x,y : x+y
f(1,1)                        # 2
f = lambda x: x**2 + 2*x - 5
f(2)                          # 3
# Fahrenheit to Celsius conversion
f2c = lambda c: float('{:.2f}'.format((5.0 / 9) * ( c - 32 )))
f2c(32)                       # 0

Lambda functions are used along with build-in function like map(), or filter().

Map

The map(<<func>>,<<sequence>>) function applies a function to every item in an sequence. It returns a list containing all the function call results.

def sqr(x): return x ** 2
list(map(sqr, [1, 2, 3, 4, 5]))                            # [1, 4, 9, 16, 25]
# with a lambda expression
list(map(lambda x: x+1, [1,2,3,4,5,6]))                    # [2, 3, 4, 5, 6, 7]
list(map(lambda x: x**2, range(0,12,2)))                   # [0, 4, 16, 36, 64, 100]

Filter

The filter(<<func>>,<<sequence>>) function extracts each element in a sequence for which a function returns True.

list(filter(lambda x: x<0,range(-5,5)))                     # [-5, -4, -3, -2, -1]
list(filter(lambda x: (x%2==0), [1,5,4,6,8,11,3,12]))       # [4, 6, 8, 12]
## intersection
a,b = [1,2,3,5,7,9],[2,3,5,6,7,8]
list(filter(lambda x: x in a,b))                            # [2, 3, 5, 7]

Reduce

The reduce() function reduces a sequence to a single value by combining all elements via a defined function.

reduce(<<func>>,<<sequence>>[,<<initializer>>])

By default, the first item in the sequence initialized the starting value.

from functools import reduce
reduce(lambda x,y: x+y, [1,2,3,4])                          # 10
reduce(lambda x,y: x*y, [2,3],2)                            # 12
import operator
reduce(operator.sub,[50,3,4,6])                             # 37
## flatten a list
reduce(list.__add__, [[1, 2, 3], [4, 5], [6, 7, 8]], [])    # [1, 2, 3, 4, 5, 6, 7, 8]
## union of a list of sets
reduce(operator.or_, ({1},{1,2},{1,3}))                     # {1, 2, 3}
## intersection of a list of sets
reduce(operator.and_, ({1},{1,2},{1,3}))                    # {1}

Classes

Classes, instances, and data attributes:

  • A class is defined with the keyword class followed by a name (capitalized) and colon.
  • Class instantiation uses function notation to assign a class object (instance”) to a variable.
  • Class attributes are referenced with the dot notation <object>.<attribute>.
  • Object data attributes (instance variables) need not be declared, they are assigned on first used.
class Human():      # define a class called Human
    pass            # Use pass for a class without attributes/methods
# create two instances of the class
alice = Human()
bob = Human()
# set data attributes of both instances
alice.age = 25
bob.age = 31
# print instance attributes
print(bob.age,alice.age) # 31 25

Class Constructor & Instance Methods

  • Methods automatically pass a class object self (by convention) as first argument.
  • The method __init__() (the constructor) is automatically invoked on newly-created class instances.
  • Instance objects can use attribute references to data attributes and methods.
class Human():
    # constructor
    def __init__(self, name, age):
        self.name = name
        self.age = age
    # method
    def who(self):
        return '{} age {}'.format(self.name, self.age)
# Iterate over two class objects
for _ in (Human('alice',25),Human('bob',31)):
    # call the method of an object
    print(_.who())
    # pass an object to a method
    print(Human.who(_))

Class Variables & Class Methods

Class variables:

  • Shared among all instances of a class
  • Accessible as <class>.<attribute> or as <object>.<attribute>

Class methods:

  • Declared with a decorator @classmethod
  • Automatically pass a class as first argument called cls (by convention)
  • Typically use to build alternative constructors
class Human():
    num = 0 # define a class variable
    # constructor
    def __init__(self, name, age):
        self.name = name
        self.age = age
        # increment the number of humans
        Human.num += 1
    # decorator to identify a class method
    @classmethod
    def from_str(cls, string):
        name, age =  string.split(':')
        return Human(name,age)
    # method
    def who(self):
        # use self to access a class variable
        return '{} age {} [of {}]'.format(self.name, self.age, self.num)
humans = [
  Human('alice',25),
  Human('bob',31),
  Human.from_str('joe:19')
]
print(humans[2].who())       # joe age 19 [of 3]

Class Properties

A method used to get a value is decorated with @property before its definition.

A method used to set a value is decorated with @<<name>>.setter before its definition.

class C:
    def __init__(self,v):
        self._v = v
    # getter
    @property
    def v(self):
        return self._v
    # setter
    @v.setter
    def v(self,__):
        self._v = __
c = C(123)
c.v = 321         # set a value
print(c.v)        # get a value

Command-Line

Use following libraries:

  • argparse to parse command-line options, arguments and sub-commands
  • logging for multi-level application logging

Runtime services in the sys library:

import sys
sys.argv                        # list of command line arguments
sys.stdout.write(s)             # write string s to standard output
sys.stderr.write(s)             # write string s to standard error
sys.exit(i)                     # exit with error code i

Python modules executable as main program also:

  • Source files executed as the main program have the variable __name__ set to __main__
  • Call an optional main() function if not loaded by an import <module>
def main():
    pass

if __name__ == '__main__':
    main()

Get input from the user with the input function:

  • The argument will be printed followed by a prompt to wait for user input
  • The function returns the input provided by the user
  • Note that the return value is always of type str
input('Give me a string: ')
int(input('Give me a number'))

Consume data from the input pipe STDIN:

# read input from STDIN
if not sys.stdin.isatty():
     stdin = io.StringIO(sys.stdin.read())
else:
     raise Exception('No input data specified, STDIN is empty')

venv

venv

  • …creation of virtual environments
  • …requires Python 3.3 or newer.
  • …a virtual environment includes:
    • …isolates Python (interpreter,libraries) into a dedicated directory
    • …easy_install and pip work es expected
python -m venv $path          # create new virtual environment
source $path/bin/activate     # activate environment
deactivate                    # exit virtual environment

conda

Conda…

  • …package manager for Python
  • …open-source …BSD-3 License …hosted on github.com/conda
  • Partly owned and controlled by Anaconda Inc…
    • …Anaconda is a commercial distribution of Python …not free and not open-source
    • …note that anaconda channel is prohibiting for commercial usage
conda info                                          # basic information
conda info --envs                                   # display a list of all environments
conda search --full-name python                     # list available Python versions
conda create --name <env> python=<ver> [<pkg>]      # install global environment
conda create --prefix ~/<path> python=<ver> [<pkg>] # install use specific version
source activate <env>                               # load global environment
source activate ~/<path>                            # load user specific environment
source deactivate                                   # unload environment
conda remove --name <env> --all                     # delete an environment

Managing packages…

  • …make sure to only use free and open-source Conda package channels
  • …check the active channels by running conda config --show channels
  • …use ~/.condarc to configure the channels to use
conda list                                      # list packages
conda search $pkg                               # search for a package
conda install $pkg                              # install into current environment
conda update $pkg                               # update package in current environment
conda remove $pkg                               # delete package fro current environment

Miniconda…

  • …free minimal installer for conda
  • …includes only conda and Python

ipython

Interactive Python

Keyboard shortcuts:

ctrl-l                 clear terminal
ctrl-c                 interrupt command
ctrl-d                 exit session
ctrl-r                 search command history
ctrl-a                 cursor to the beginning of the line
ctrl-e                 cursor to the end of the linr
ctrl-k                 cut text from cursor to end of line
ctrl-u                 cut text from beginning of line to cursor
ctrl-y                 yank (paste) text cut before

Access Documentation

Access the documentation with the build-in help(<obj>[.<method>]) function.

Alternatively use the short hand <obj>[.<method>]? with a question mark:

In [1]: tuple?
Init signature: tuple(self, /, *args, **kwargs)
Docstring:
tuple() -> empty tuple
tuple(iterable) -> tuple initialized from iterable's items

If the argument is a tuple, the return value is the same object.
Type:           type
In [2]: list.index?
Docstring:
L.index(value, [start, [stop]]) -> integer -- return first index of value.
Raises ValueError if the value is not present.
Type:      method_descriptor

It support objects methods, and includes user defined code.

In [3]: def square(a):
   ...:     """Return square of argument."""
   ...:     return a**2
   ...:
In [4]: square?
Signature: square(a)
Docstring: Return square of argument.
Type:      function

In [5]: square??
Signature: square(a)
Source:
def square(a):
    """Return square of argument."""
    return a**2
Type:      function

Display source code with the short hand <obj>[.<method>]?? (two questions marks), unless it is a build-in implemented in C.

Help support wild card completion?

In [6]: C*Error*?
ChildProcessError
ConnectionAbortedError
ConnectionError
ConnectionRefusedError
ConnectionResetError
In [7]: dict.*__g*?
dict.__ge__
dict.__getattribute__
dict.__getitem__
dict.__gt__

Magic Commands

The %lsmagic command lists all magic commands:

  • Single magic commands are prefixed with % (percent)
  • Multi-line expressions start with %% (double percent)
  • Prefix the magic command with ? (question mark) to see the help text
In [1]: ?%lsmagic
Docstring: List currently available magic functions.
File:      /usr/lib/python3/dist-packages/IPython/core/magics/basic.py
In [2]: %cat square.py
def square(a):
    """Return square of argument."""
    return a**2

print(square(2))
print(square(10))

In [2]: %run square.py
4
100

General description of magic functions is available with %magic.

Shell Commands

Shell commands are prefixed with !, which can be omitted if auto-magic is on:

In [1]: %automagic 1

Automagic is ON, % prefix IS NOT needed for line magics.
In [2]: ls
ipython.md  jupyter.ipynb  mathplot.ipynb  numpy.md  README.md

Store output of a shell command into a variables by assignment. (Here the exclamation mark is required)

In [3]: files = !ls
In [4]: print(files)
['ipython.md', 'jupyter.ipynb', 'mathplot.ipynb', 'numpy.md', 'README.md']
In [5]: s = "cheers"
In [6]: !echo {s}

Interpolate the contents of a variable with {<var>}.

Subshell started with ! are non-interactive non-login instance of the user’s default shell:

In [1]: !echo $SHELL
/bin/zsh
In [2]: !ps -p $$ && :
  PID TTY          TIME CMD
  19295 pts/5    00:00:00 zsh

ZSH user need to load their custom environment with ~/.zshenv

Library

Numpy

The Nympy library adds support for large homogeneous multi-dimensional arrays and matrices, along with a large collection of high-level mathematical functions to operate on these arrays.

  • Homogeneously typed (all elements have the same type).
  • Numpy ndarray class for n-dimensional array is a more efficient implementation of a python list.
  • Numerical operations with ndarray run on full compiled code speed.
# recommended convention to import numpy
import numpy as np                 # with abbr. np

Create

Dimensions are called axes, and the number of axes is rank

# crate an ndarray using a tuple or list
np.array((1,2,3,4,5))
np.array([1,2,3,4,5])                         # [1 2 3 4 5]
# explicit data type
np.array((1.1,2,3))                           # [ 1.1  2.   3. ]
np.array((1,2),float)                         # [ 1.  2.]
np.array([1,2],dtype=complex)                 # [ 1.+0.j  2.+0.j]
# nested tuples/lists result in multi-dimensional arrays
np.array(([1,2,3],[4,5,6],[7,8,9]))           # [[1 2 3] [4 5 6] [7 8 9]]
np.array([range(i, i+3) for i in [1,2,3]])    # [[1 2 3] [2 3 4] [3 4 5]]
# initial with placeholder content
np.zeros((3,2))                      # [[ 0.  0.] [ 0.  0.] [ 0.  0.]]
np.ones((2,2,2))                     # [[[ 1.  1.] [ 1.  1.]] [[ 1.  1.] [ 1.  1.]]]
np.full((2,2),5)                     # [[5 5] [5 5]]
np.diag(np.array([1, 2, 3]))         # [[1 0 0] [0 2 0] [0 0 3]]
# create sequences of numbers with args. [start,]stop[,step] (cf. range)
np.arange(5)                         # [0 1 2 3 4]
np.arange(5,10)                      # [5 6 7 8 9]
np.arange(0,30,10)                   # [0 10 20]
# linearly spaced sequence
np.linspace(2,3,5)                   # [2. 2.25 2.5 2.75 3.]
# log spaced sequence 
np.logspace(0,1,5)                   # [1. 1.77827941 3.16227766 5.62341325 10. ]

Random values:

# float values between 0 and 1 in defined dimension
np.random.rand(1,3)                  # [0.19544155  0.389351  0.09039669]
np.random.rand(2,2)                  # [[ 0.62209577  0.55083187] [ 0.31431768  0.98404029]]
# 5 integers between 0 and 10
np.random.randint(0,10,5)            # [7 2 9 4 8]
# in defined dimension
np.random.randint(0,10,(2,3))        # [[6 3 6] [8 2 7]]

Shape

# print dimensions
a = np.array(((1,2),(3,4)))
a.ndim                               # 2
a,size                               # 4
a.shape                              # (2, 2)
a.dtype                              # dtype('int64')
# size in bytes per element
a.itemsize                           # 8
# size of all elements (size x itemsize)
a.nbytes                             # 32 
# flatten
np.array(((1,2),(3,4))).flatten()
np.array(((1,2),(3,4))).ravel()      # [1 2 3 4]
# change dimensions
np.arange(6).reshape(2,3)            # [[0 1 2] [3 4 5]]
# transpose
np.zeros((2,3)).T                    # [[ 0.  0.] [ 0.  0.] [ 0.  0.]]
np.zeros((2,3)).transpose()
## stacking together
# vertical
np.vstack((np.zeros((2,2)),np.ones((2,2))))       # [[ 0.  0.] [ 0.  0.] [ 1.  1.] [ 1.  1.]]
# horizontal
np.hstack((np.zeros((2,2)),np.ones((2,2))))       # [[ 0.  0.  1.  1.] [ 0.  0.  1.  1.]]
np.hsplit(np.arange(10),2)                        # [[0 1 2 3 4] [5 6 7 8 9]]

Operations

np.array((10,20,30)) + np.array((1,2,3))                # [11 22 33]
np.array((10,20,30)) - np.array((1,2,3))                # [ 9 18 27]
np.array((2,4,8)) ** 2                                  # [ 4 16 64]                                             
np.array((2,4,8)) >= 4                                  # [False  True  True]
np.array((2,2,2)).sum()                                 # 6
np.array(((1,2),(4,5))).sum(axis=1)                     # [3 9]
np.array((1,2,3)).min()                                 # 1
np.array((1,2,3)).max()                                 # 3
np.array(((1,2),(4,5))).cumsum(axis=1)                  # [[1 3] [4 9]]
# vector products
np.array([1,2]).dot(np.array([3, 4]))                   # 11
# multiply a vector by a matrix
np.array([1,2]).dot(np.array([[3,4],[5,6]]))            # [13 16]
# multiply matrices
np.array([[1,2],[3,4]]).dot(np.array([[5,6],[7,8]]))    # [[19 22] [43 50]]

Indexing, Slicing and Iterating

a = np.array([1,2,3,4])            
# access elements in array a
a[3]                               # 4
a[-2]                              # 3 
a[:2]                              # [1 2] last index not included!
a[3] = 5                           # [1 2 3 5]
## multiple dimensions
a = np.array([[1,2,3],[4,5,6]], float)
# ':' all elements in dimension
a[1,:]                             # [ 4.  5.  6.]
a[:,2]                             # [ 3.  6.]
a = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])
a[:2,1:3]                          # [[2 3] [6 7]]
# reversing a sequence
np.arange(10)[::-1]                # [9 8 7 6 5 4 3 2 1 0]
# slice sequences [start:end:step]
np.arange(10)[2:9:3]               # [2 5 8]
np.arange(10)[::3]                 # [0 3 6 9]
# ... (dots) represent as many colons as needed to produce a complete indexing tuple
np.arange(12).reshape(3,4)[1,...]  # [4 5 6 7]
np.arange(12).reshape(3,4)[...,2]  # [2 6 10]

Slicing operation creates a view on the original array.

a = np.arange(10)                  # [0 1 2 3 4 5 6 7 8 9] 
b = a[::2]                         # [0 2 4 6 8]
np.may_share_memory(a,b)           # True
c = b.copy() # force a copy
np.may_share_memory(a,c)           # False

IO

Store data into a file:

# write a binary file
>>> np.array([1,2,3],float).tofile('f.bin')
>>> !file f.bin
f.bin: dBase III DBT, next free block index 1
>>> print(np.fromfile('f.bin'))
[ 1.  2.  3.]
# write a clear text file
>>> np.savetxt('f.txt',np.array((1,2,3)))
>>> !cat f.txt
1.000000000000000000e+00
2.000000000000000000e+00
3.000000000000000000e+00
>>> print(np.loadtxt('f.txt'))
[ 1.  2.  3.]

Embedded Python

What is MicroPython?

  • Reimplementation of Python 3 for MCUs
    • Efficient with resources, runs on bare metal
    • Written in C++, includes compiler, run-time
    • REPL (read, evaluate, print loop)
  • Compiler emitters…
    • Byte code for a virtual machine
    • Native machine code (x86, x64, ARM…)
    • Supports inline assembler

Micro:bit uses a pre-compiles run-time…

  • Runtime .hex flashed to the micro:bit
  • .hex contains complete MicroPython language

CicruitPython open source derivative of MicroPython

Install uflash:

sudo apt install -y python3-pip
pip3 install uflash
# mount the device
pmount /dev/sdb MICROBIT
uflash $source

References

Editors & IDEs…

Workshops & Tutorials…

References

Lectures…

Books…