Brogramming - Python, Bash for Data Processing, and Git
0 likes•1,136 views
The document discusses Python coding conventions and best practices including PEP 8. It covers topics like formatting, spacing, imports, and other stylistic guidelines to make Python code more readable and consistent across projects. The Zen of Python philosophy emphasizes simple and explicit code over complex solutions. Git concepts like merge vs rebase and Git flow are also briefly mentioned.
![This
is
OK:
my_list
=
[
1,
2,
3,
4,
5,
6,
]
result
=
some_function_that_takes_arguments(
'a',
'b',
'c',
'd',
'e',
'f',
)
This
is
OK
too
(use
this
at
Crosswise):
my_list
=
[
1,
2,
3,
4,
5,
6,
]
result
=
some_function_that_takes_arguments(
'a',
'b',
'c',
'd',
'e',
'f',
)](https://image.slidesharecdn.com/brogramming-150415003745-conversion-gate01/85/Brogramming-Python-Bash-for-Data-Processing-and-Git-15-320.jpg)
![¡ Use
blank
lines
in
functions,
sparingly,
to
indicate
logical
sections.
#
do
something
something
=
[x
for
x
in
list]
#
now
do
something
different
something_different
=
do_it(something)](https://image.slidesharecdn.com/brogramming-150415003745-conversion-gate01/85/Brogramming-Python-Bash-for-Data-Processing-and-Git-19-320.jpg)
![¡ Immediately
inside
parentheses,
brackets
or
braces.
Yes:
spam(ham[1],
{eggs:
2})
No:
spam(
ham[
1
],
{
eggs:
2
}
)](https://image.slidesharecdn.com/brogramming-150415003745-conversion-gate01/85/Brogramming-Python-Bash-for-Data-Processing-and-Git-22-320.jpg)
![¡ Immediately
before
the
open
parenthesis
that
starts
an
indexing
or
slicing:
Yes:
dict['key']
=
list[index]
No:
dict
['key']
=
list
[index]](https://image.slidesharecdn.com/brogramming-150415003745-conversion-gate01/85/Brogramming-Python-Bash-for-Data-Processing-and-Git-26-320.jpg)
![¡ Use
list
comprehensions
instead
of
map
with
lambdas.
¡ If
you
want
to
apply
a
function,
use
map.
¡ Generally,
map
and
filter
are
less
readable.
Yes:
result
=
[x
+
1
for
x
in
xrange(10)
if
x
%
2
==
0]
No:
result
=
filter(lambda
x:
x
%
2
==
0,
map(
lambda
x:
x
+
1,
xrange(10)))
Yes:
result
=
map(get_data,
items)](https://image.slidesharecdn.com/brogramming-150415003745-conversion-gate01/85/Brogramming-Python-Bash-for-Data-Processing-and-Git-29-320.jpg)
Brogramming - Python, Bash for Data Processing, and Git
- 1. Crosswise
- 3. ¡ Python § The Zen of Python § Conventions and PEP8 § Tips & Tricks, Do’s and don’t’s ¡ Bash for Data Processing ¡ Git § Merge vs rebase § Git flow
- 5. >>> import this The Zen of Python, by Tim Peters Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-‐-‐ and preferably only one -‐-‐obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -‐-‐ let's do more of those!
- 6. There should be one and preferably only one obvious way to do it. (Although that way may not be obvious at first unless you're Dutch.)
- 7. ¡ PEP 8 defines the standard coding convention for the global Python community. ¡ It does not leave much room for different code conventions ¡ Makes code more readable ¡ Gets people to easily go in and out of code which they did not write
- 8. ¡ Easy, just use PyCharm. ¡ It will emit warnings if your code is not PEP 8 compliant.
- 11. Spaces. (4 of them)
- 12. Yes: # Aligned with opening delimiter. foo = long_function_name(var_one, var_two, var_three, var_four)
- 13. Yes (please use hanging indents at Crosswise): # More indentation included to distinguish this from the rest. def long_function_name( var_one, var_two, var_three, var_four): print(var_one) # Hanging indents should add a level. foo = long_function_name( var_one, var_two, var_three, var_four)
- 14. No: # Arguments on first line forbidden when not using vertical alignment. foo = long_function_name(var_one, var_two, var_three, var_four) # Further indentation required as indentation is not distinguishable. def long_function_name( var_one, var_two, var_three, var_four): print(var_one)
- 15. This is OK: my_list = [ 1, 2, 3, 4, 5, 6, ] result = some_function_that_takes_arguments( 'a', 'b', 'c', 'd', 'e', 'f', ) This is OK too (use this at Crosswise): my_list = [ 1, 2, 3, 4, 5, 6, ] result = some_function_that_takes_arguments( 'a', 'b', 'c', 'd', 'e', 'f', )
- 16. ¡ PEP 8 -‐ 79 characters. ¡ But it’s too short, so we use 120 (PyCharm).
- 17. ¡ Top level functions and classes – 2 blank lines: Class Foo(object): pass Class Bar(object): pass
- 18. ¡ Method definitions inside a class, 1 blank line Class Foo(object): def foo1(self): pass def foo2(self): pass
- 19. ¡ Use blank lines in functions, sparingly, to indicate logical sections. # do something something = [x for x in list] # now do something different something_different = do_it(something)
- 20. Yes: import os import sys No: import os, sys Yes: from subprocess import Popen, PIPE
- 21. ¡ PEP 8 recommends absolute imports, so use them. ¡ They avoid confusion and duplicate imports. from foo.bar.yourclass import YourClass
- 22. ¡ Immediately inside parentheses, brackets or braces. Yes: spam(ham[1], {eggs: 2}) No: spam( ham[ 1 ], { eggs: 2 } )
- 23. ¡ Immediately before a comma, semicolon, or colon: Yes: if x == 4: print x, y; x, y = y, x No: if x == 4 : print x , y ; x , y = y , x
- 24. ¡ Immediately before the open parenthesis that starts the argument list of a function call: Yes: spam(1) No: spam (1)
- 25. ¡ More than one space around an assignment (or other) operator to align it with another. Yes: x = 1 y = 2 long_variable = 3 No: x = 1 y = 2 long_variable = 3
- 26. ¡ Immediately before the open parenthesis that starts an indexing or slicing: Yes: dict['key'] = list[index] No: dict ['key'] = list [index]
- 27. ¡ Don't use spaces around the = sign when used to indicate a keyword argument or a default parameter value. Yes: def complex(real, imag=0.0): return magic(r=real, i=imag) No: def complex(real, imag = 0.0): return magic(r = real, i = imag)
- 28. ¡ Use inline comments sparingly. ¡ Don’t document obvious things – if you write Python expressive enough, you might need even need documentation at all.
- 29. ¡ Use list comprehensions instead of map with lambdas. ¡ If you want to apply a function, use map. ¡ Generally, map and filter are less readable. Yes: result = [x + 1 for x in xrange(10) if x % 2 == 0] No: result = filter(lambda x: x % 2 == 0, map( lambda x: x + 1, xrange(10))) Yes: result = map(get_data, items)
- 30. ¡ Python Facts – a website I wrote: § http://thingsyoumust.appspot.com/ ¡ Quora § http://www.quora.com/Python-‐programming-‐ language-‐1/What-‐are-‐some-‐cool-‐Python-‐tricks
- 32. ¡ Pipe contents of file into another file cat file.txt > newfile.txt cat file1.txt file2.txt > newfile.txt ¡ Pipe contents of a compressed file gzcat file.txt.gz > newfile.txt ¡ Pipe contents of a compressed file and write it to a compressed file gzcat file.txt.gz | gzip > newfile.txt.gz
- 33. echo "1t2t3" | cut -‐f 2 2 echo "1,2,3" | cut -‐f 2 -‐d"," 2
- 34. ¡ awk is a mini programming language which is very convenient mostly for outputting a column/line according to a different condition echo "1t2t3n4t5t6t" | awk '{ if ($1 % 2 == 0) print $2 }' 5
- 35. ¡ awk can also do cool stuff like sampling a file. echo "1n2n3n4n5n6" | awk 'BEGIN {srand()} {if (rand() < 0.1) print $0}'
- 36. echo "2n1n3n10" | sort 1 10 2 3 echo "2n1n3n10" | sort -‐n 1 2 3 10
- 37. ¡ uniq returns unique values, assuming the input is sorted. echo "1n2n1" | uniq 1 2 1 echo "1n2n1" | sort | uniq 1 2 ¡ uniq also knows to return value counts for each value using –c. echo "anbna" | sort | uniq -‐c 2 a 1 b
- 38. echo "a b cn1 2 3" | wc -‐l 2 echo "a b cn1 2 3" | wc -‐w 6
- 39. ¡ Counting unique values in a column: cat foo.txt | cut –f 2 | sort | uniq | wc –l ¡ Showing the value counts cat foo.txt | cut –f 2 | sort | uniq -‐c
- 40. ¡ Git is a distributed version control system (DVCS) ¡ Written by Linus Torvalds, the author of Linux, in order to maintain the open source project
- 42. ¡ Advantages of rebase over merge: § Simplifies history – merge commits are ugly because they have two parents § Each commit is an addition over the existing code ¡ Advantages of merge over rebase: § Preserves the commits as they were in the developer’s computer
- 43. ¡ HOWTO http://danielkummer.github.io/git-‐flow-‐cheatsheet/ http://nvie.com/posts/a-‐successful-‐git-‐branching-‐model/ ¡ Install brew install git-‐flow-‐avh
- 44. ¡ Git flow defines a universal standard for developing features and maintaining releases over git. ¡ There are 3 commands in git flow: § feature – for implementing new features § release – for adjusting the release according to feedback from QA and such § hotfix – for creating a new release on top of a previous release
- 45. ¡ Features are branched in and out of develop ¡ Releases are branched in and out of develop and master ¡ Hotfixes are branched in and out of master
- 46. ¡ There are 3 operations: § start – start a feature/release/hotfix by branching out § publish – push to the remote repository § finish – Merge back and remove the branch ▪ When finishing, please use the -‐r feature to rebase the changes back instead of merging them, so we would have a cleaner history to look at.