Posts

Showing posts with the label Python

Finally block always runs in python, avoid returning values in this block

>>> def foo(): ...    try: ...        return 1 ...    except: ...        return 2 ...    finally: ...        return 3 ...  >>> foo() 3 >>> def foo(): ...     try: ...        raise ValueError ...     except: ...        return 2 ...     finally: ...        return 3 ...  >>> foo() 3

Avoid using global/class-level mutable datatypes like list/dicts

Class-level variables that are assigned with mutable datatypes are like global static variables, they refer to the same list or dict and can be modified/accessed via multiple instance or through the class itself, >>> class Foo: bar = [] >>> a, b = A(), A() >>> a.bar.append(10) >>> a.bar [10] >>> b.bar [10] >>> Foo.bar [10] >>> class A: def __init__(self): self.bar = [] >>> a, b = A(), A() >>> a.bar.append(10) >>> a.bar [10] >>> b.bar [] >>> Foo.bar Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: type object 'Foo' has no attribute 'bar' To avoid using the same global list/dict across multiple instances we should avoid initializing class/global variables with a mutable datatypes like list/dict

Avoid initializing kwargs in signature as they might not work as expected when you need to pass kwargs from other methods

$ cat test_kwargs . py def m1 ( a , b = 2 ) : print a , b def m2 ( a , b = None ) : ​m1 ( a , b = b ) m1 ( 1 ) m2 ( 1 ) m1 ( 1 , 2 ) m2 ( 1 , 2 ) $ python test_kwargs . py 1 2 1 None 1 2 1 2 A better version of the above methods would be, ​ $ cat test_kwargs . py def m1 ( a , b = None ) : if b is None : b = 2 print a , b def m2 ( a , b = None ) : m1 ( a , b = b ) m1 ( 1 ) m2 ( 1 ) m1 ( 1 , 2 ) m2 ( 1 , 2 ) $ python test_kwargs . py 1 2 1 2 1 2 1 2​

Understanding multiple inheritance and method resolution order in Python

# cat multiple_inheritance.py import inspect class A ( object ): pass class B (A): pass class C (A): pass class D (B): pass class E (C): pass class F (D, E): pass print inspect.getmro(A) print inspect.getmro(B) print inspect.getmro(C) print inspect.getmro(D) print inspect.getmro(E) print inspect.getmro(F) # python multiple_inheritance.py (< class ' __main__ .A '>, <type ' object '>) (< class ' __main__ .B '>, <class ' __main__.A '>, <type ' object '>) (< class ' __main__ .C '>, <class ' __main__.A '>, <type ' object '>) (< class ' __main__ .D '>, <class ' __main__.B '>, <class ' __main__.A '>, <type ' object '>) (< class ' __main__ .E '>, <class ' __main__.C '>, <class ' __main__.A '>, <type ' object '>) (< class ...

eventlet.GreenPool.waitall() just logs the exception but doesn't raise them

waitall() method in eventlet.GreenPool doesn't raise exceptions but just logs them, whereas in case of wait on the greenthread returned by spawn from evenlet.GreenPool raises the exception. # cat foo . py import eventlet pool = eventlet . GreenPool ( 1 ) def foo ( ) : raise ValueError gt = pool . spawn ( foo ) try: pool . waitall ( ) except Exception , e : print " pool.waitall Exception raised: %r " % e try: gt . wait ( ) except Exception , e : print " gt.wait() Exception raised: %r " % e # python foo . py Traceback ( most recent call last ) : File " /usr/lib/python2.7/site-packages/eventlet/hubs/hub.py " , line 346 , in fire_timers timer ( ) File " /usr/lib/python2.7/site-packages/eventlet/hubs/timer.py " , line 56 , in __call__ cb ( * args , * * kw ) File " /usr/lib/python2.7/site-packages/eventlet/greenthread.py " , line 194 , in main result = functi...

Using repr instead of str for parameters (not objects) for disambiguity

>>> for e in ( 0 , "0" , None , "None" , 1.23 , "1.23" , [ 1 , 2 ], { "foo" : "bar" }): ... print " %r " % e ... 0 '0' None 'None' 1.23 '1.23' [ 1 , 2 ] { 'foo' : 'bar' } When %s is used to interpolate to string there could be a bit of ambiguity introduced. >>> for e in ( 0 , "0" , None , "None" , 1.23 , "1.23" , [ 1 , 2 ], { "foo" : "bar" }): ... print " %s " % e ... 0 0 None None 1.23 1.23 [ 1 , 2 ] { 'foo' : 'bar' }

Tuples are expanded as multiple arguments when passed to print statement after %

Below the tuple (1, 2, 3) is getting expanded as 3 arguments and the format representation is expecting to have 3 %s or %r to be printed, > > > print " %r " % ( 1 , 2 , 3 ) Traceback ( most recent call last ) : File " <stdin> " , line 1 , in < module > TypeError: not all arguments converted during string formatting > > > print " %s " % ( 1 , 2 , 3 ) Traceback ( most recent call last ) : File " <stdin> " , line 1 , in < module > TypeError: not all arguments converted during string formatting > > > print " %s " % str ( ( 1 , 2 , 3 ) ) ( 1 , 2 , 3 )

How to avoid adding duplicate keys to dictionary by mistake ...

There would be cases where large dictionaries are being embedded in the code (multiple pages in length) and users might append something to end of the dictionary, just assuming that its not already in the dictionary.  This could be easily avoided by following dict() instead of {} notation for dictionaries. > > > a = { . . . 'a' : 1 , . . . 'b' : 2 , . . . 'c' : 3 , . . . 'd' : 'nextpage' , . . . 'a' : 0 . . . } > > > print a { 'a' : 0 , 'c' : 3 , 'b' : 2 , 'd' : 'nextpage' } > > > a = dict ( . . . a = 1 , . . . b = 2 , . . . c = 3 , . . . d = 'nextpage' , . . . a = 0 . . . ) File " <stdin> " , line 6 SyntaxError : keyword argument repeated

Multiple repeat error when the regex has multiple wildcards in python re module

When the regex being compiled in python which has multiple wild cards like plus or asterisks sequentially repeated, you will need to escape or else run into "multiple repeat error". This can happen with any of re module methods like search and sub etc, where the regex is compiled. > > > import re > > > re . compile ( " r++ " ) Traceback ( most recent call last ) : File " <stdin> " , line 1 , in < module > File " /usr/lib/python2.7/re.py " , line 190 , in compile return _compile ( pattern , flags ) File " /usr/lib/python2.7/re.py " , line 242 , in _compile raise error , v # invalid expression sre_constants . error : multiple repeat > > > re . compile ( " r** " ) Traceback ( most recent call last ) : File " <stdin> " , line 1 , in < module > File " /usr/lib/python2.7/re.py " , line 190 , in compile return _...

Splitting python path using os.path module

Avoid using "/" at the end of the directory paths as they would generate empty filename when used with functions like os.path.split() > > > import os > > > os . path . split ( '/tmp/foo' ) ( '/tmp' , 'foo' ) > > > os . path . split ( '/tmp/foo/' ) ( '/tmp/foo' , '' ) Reference: https://docs.python.org/2/library/os.path.html

Class/Global variables vs Instance Variables ...

Class/Global variables are like CoW, only when the instances assign them values they have their own memory location allocated. For example in this case a.bar is assigned a value and hence it has different memory location allocated, but the same is not the case with b.bar >> > class Foo ( object ) : bar = None >> > repr ( Foo . bar ) 'None' >> > a = Foo ( ) >> > a . bar = 1 >> > repr ( Foo . bar ) 'None' >> > a . bar 1 >> > b = Foo ( ) >> > b . bar >> > repr ( b . bar ) 'None' >> > a . bar 1 >> > id ( Foo . bar ) 135796184 >> > id ( a . bar ) 151257264 >> > id ( b . bar ) 135796184

Weakref proxy is for instance only ...

Proxy cannot be created for basic datatypes like list, dict, str and int. Proxy can be create for class instances only. Also proxy cannot be created for proxy of another instance. >>> import weakref >>> a = 'str' >>> b = weakref.proxy(a) Traceback (most recent call last):   File "<stdin>", line 1, in <module> TypeError: cannot create weak reference to 'str' object >>> a = 1 >>> b = weakref.proxy(a) Traceback (most recent call last):   File "<stdin>", line 1, in <module> TypeError: cannot create weak reference to 'int' object >>> a = [] >>> b = weakref.proxy(a) Traceback (most recent call last):   File "<stdin>", line 1, in <module> TypeError: cannot create weak reference to 'list' object >>> a = {} >>> b = weakref.proxy(a) Traceback (most recent call last):   File "<stdin>",...

Raising vs Reraising exceptions in python

>>> def foo(x): ...    raise Exception('%s is invalid input' % x) ... >>> try: ...     foo(1) ... except Exception, e: ...     raise e                          # RE-RAISING EXCEPTIONS ... Traceback (most recent call last):   File "<stdin>", line 4, in <module>    # LOSES foo Exception: 1 is invalid input >>> try: ...     foo(1) ... except Exception: ...     raise                             # RAISING EXCEPTIONS ... Traceback (most recent call last):   File "<stdin>", line 2, in <module>   File "<stdin>", line 2, in foo          # STORES foo too Exception: 1 is invalid input >>>

Invalid positonal args cannot be passed over as kwargs using **kwargs

Input: def a(x, y=None):    print x, y def b(x, **kw):     a(x, **kw) def c(x, y):     print x, y c(1, y=1) b(1, y=2) b(1, 3) Output: 1 1 1 2 Traceback (most recent call last):   File "a.py", line 12, in <module>     b(1, 3) TypeError: b() takes exactly 1 argument (2 given)

Try/Except vs Exception handling

try:   foo() except:   pass or try:   foo() except Exception:   pass Difference is that the first will catch everything including OsError, KeboardInterrupt, SystemExit which are derived from BaseException instead of Exception class. More documentation for details: try statement — http://docs.python.org/reference/compound_stmts.html#try exceptions — http://docs.python.org/library/exceptions

Empty strings "" and zero 0 are treated as False in python

>>> any ([0, 0]) False >>> any ([1, 0]) True >>> any ([None, 0, ""]) False >>> bool(0) False >>> bool("") False >>> bool(1) True

Python code: TypeError: 'str' object is not callable

Missing % can cause chaos with string expansions, Sometime the print/logging statement become too huge to verify where the string is coming from. For example in this case you might be looking for string that is used as callable forever, if you dont look at the actual string thats being expanded. Input: def foo1(x):     return str(x) foo2 = lambda x: None if x is None else str(x) def myprint(x):     print x x = 5 myprint("x=%s, foo1(x)=%s, foo2(x)=%s" %         (str(x), foo1(x), foo2(x))) myprint("x=%s, foo1(x)=%s, foo2(x)=%s"         (str(x), foo1(x), foo2(x))) Output: x=5, foo1(x)=5, foo2(x)=5 Traceback (most recent call last):   File "a.py", line 15, in <module>     (x, foo1(x), foo2(x))) TypeError: 'str' object is not callable

Python madness in comparisons with None

In python, everything is greater than None, >>> 0 > None True >>> 0 < None False >>> 1 > None True >>> 1 < None False >>> -1 > None True >>> -1 < None False >>> 'a' > None True >>> 'a' < None False >>> 1.0 > None True >>> 1.0 < None False

Overriding class variables and destroying overriden references in python classes

Input: class foo(object):     ref = None     def setup(self):         self.ref = 1234     def clear(self):         del self.ref obj = foo() print obj.ref obj.setup() print obj.ref obj.clear() print obj.ref O utput: None 1234 None

Exceptions and Finally in python

>>> Input def foo():    try:        print "I am in foo"        return    except:        raise    finally:        print "finally of foo" def foo_exec():    try:        print "I am in foo_exec"        raise Exception("Exception in foo_exec")    except:        raise    finally:        print "finally of foo_exec" foo() foo_exec() >>> Output I am in foo finally of foo I am in foo_exec finally of foo_exec Traceback (most recent call last):   File "a.py", line 20, in <module>     foo_exec()   File "a.py", line 13, in foo_exec     raise Exception("Exception in foo_exec") Exception: Exception in foo_exec