Python 3 Scripting for System Administrators:-Part:1


#1

History and Overview of Python

1)First appearance: 1991
2) Creator: Guido van Rossum
3) Key Points About Python:
3.1.Object-Oriented Scripting Language.
3.2. Dynamic & strong typing system.
3.3. Dynamic types are checked at runtime, Strong types don’t change implicitly, can’t add 1 and “something”.
3.4. Supports functional concepts like map, reduce, filter, and list comprehension.
3.5. Extremely popular language used across many different disciplines (academia, data science, scripting, web development, etc.).

Required Software Packages, Tools, and Files
• git
• wget
• which
• words (need file at /usr/share/dict/words)
• lsof
• text editor of your choice
• python 3.6.5

Python Keywords:-

print
print to console
while
controlling the flow of the program
for
iterate over items of a collection in order that they appear
break
interrupt the (loop) cycle, if needed
continue
used to interrupt the current cycle, without jumping out of the whole cycle.
New cycle will begin.
if
used to determine, which statements are going to be executed.
elif
stands for else if.If the first test evaluates to False, then it continues with the next one is optional.
else
The statement after the else keyword is executed, unless the condition is True
is
tests for object identity
not
negates a boolean value
and
all conditions in a boolean expression must be met
or
at least one condition must be met.
import
import other modules into a Python script
as
if we want to give a module a different alias
from
for importing a specific variable, class or a function from a module
def
used to create a new user defined function
return
exits the function and returns a value
lambda
creates a new anonymous function
global
access variables defined outside functions
try
specifies exception handlers
except
catches the exception and executes codes
finally
is always executed in the end. Used to clean up resources.
raise
create a user defined exception
del
deletes objects
pass
does nothing
assert
used for debugging purposes
class
used to create new user defined objects
exec
executes Python code dynamically
yield
is used with generators

What is a REPL?
REPL stands for: R ead, E valuate, P rint, L oop
Each line is read, evaluated, the return value is then printed to the screen, and then the process repeats.
Python ships with a REPL, and you can access it by running python3.6 from your terminal.

$ python3.6

Python 3.6.4 (default, Jan 5 2018, 20:24:27)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.

>>>

The >>> indicates that you can type on that line. Later on, you’ll also see a … which means that you are currently in a scoped area and will need to enter a blank line (no spaces) before it evaluates the entire code block.

The simplest use of this would be to do some math:

>>> 1 + 1

2

>>>

‘2’ is the return value of the expression, and it is then printed to the screen. If something doesn’t have a return value, then nothing will be printed to the screen and you’ll see the next prompt immediately. We’ll cover this later, but an example would be None:

>>> None

>>>

Creating Our First Python Script

Let’s create our first script to write our obligatory “Hello, World!” program:

$ vim hello.py

From inside this file, we can enter the lines of Python that we need. For the “Hello, World!” example we only need:

print("Hello, World!")

There are a few different ways that we can run this file. The first is by passing it to the python3.6 CLI:

$ python3.6 hello.py

Hello, World!

Setting a Shebang

You’ll most likely want your scripts to be:
1. Executable from anywhere (in our $PATH).
2. Executable without explicitly using the python3.6 CLI.
Thankfully, we can set the process to interpret our scripts by setting a shebang at the top of the file:

hello.py

#!/usr/bin/env python3.6
print(“Hello, World”)

We’re not quite done; now we need to make the file executable using chmod:

$ chmod u+x hello.py
Run the script now by using ./hello.py and we’ll see the same result. If we’d rather not have a file extension on our script, we can now remove that since we’ve put a shebang in the file mv hello.py hello, and running ./hello will still result in the same thing.

Adding Scripts to Our $PATH
Now we need to make sure that we can put this in our $PATH. For this course, we’ll be using a bin directory in our $HOME folder to store our custom scripts, but scripts can go into any directory that is in your $PATH.

Let’s create a bin directory and move our script:
$ mkdir ~/bin
$ mv hello ~/bin/

Here’s how we add this directory to the $PATH in our .bashrc (the .bashrc for this course already contains this):
$ export PATH=$HOME/bin:$PATH
Finally, run the hello script from our $PATH:
$ hello
Hello, World!

Single Line Comment

We can comment out a whole line:

This is a full like comment

or we can comment at the end of a line:

2 + 2 # This will add the numbers

What About Block Comments?

Python does not have the concept of block commenting that you may have encountered in other languages. Many people mistake a triple-quoted string as being a comment, but it is not, it’s a multi-line string. That being said, multi-line strings can functionally work like comments, but they will still be allocated into memory.

"""

This is not a block comment,

but it will still work when you really need

for some lines of code to not execute.

"""

Strings

Open a REPL to start exploring Python strings:

$ python3.6

We’ve already worked with a string when we created our “Hello, World!” program. We create strings using either single quotes (’), double quotes ("), or triple single or double quotes for a multi-line string:

>>> ‘single quoted string’

‘single quoted string’

>>> "double quoted string"

‘double quoted string’

>>> ‘’’

… this is a triple

… quoted string

… ‘’’

Numbers

There are two main types of numbers that we’ll use in Python, int and float. For the most part, we won’t be calling methods on number types, and we will instead be using a variety of operators.

>>> 2 + 2 # Addition

4

>>> 10 - 4 # Subtraction

6

>>> 3 * 9 # Multiplication

27

>>> 5 / 3 # Division

1.66666666666667

>>> 5 // 3 # Floor division, always returns a number without a remainder

1

>>> 8 % 3 # Modulo division, returns the remainder

2

>>> 2 ** 3 # Exponent

8

If either of the numbers in a mathematical operation in Python is a float, then the other will be converted before carrying out the operation, and the result will always be a float.

Converting Strings and Numbers

Conversion is not uncommon since we need to convert from one type to another when writing a script and Python provides built-in functions for doing that with the built-in types. For strings and numbers, we can use the str, int, and float functions to convert from one type to another (within reason).

>>> str(1.1)

‘1.1’

>>> int("10")

10

>>> int(5.99999)

5

>>> float("5.6")

5.6

>>> float(5)

5.0

You’ll run into issues trying to convert strings to other types if they aren’t present in the string

>>> float("1.1 things")

Traceback (most recent call last):

File "", line 1, in

ValueError: could not convert string to float: ‘1.1 things’

Working with Variables

We can assign a value to a variable by using a single = and we don’t need to (nor can we) specify the type of the variable.

>>> my_str = "This is a simple string"

Now we can print the value of that string by using my_var later on:

>>> print(my_str)

This is a simple string

Before, we talked about how we can’t change a string because it’s immutable. This is easier to see now that we have variables.

>>> my_str += " testing"

>>> my_str

‘This is a simple string testing’

That didn’t change the string; it reassigned the variable. The original string of "This is a simple string"was unchanged.

An important thing to realize is that the contents of a variable can be changed and we don’t need to maintain the same type:

>>> my_str = 1

>>> print(my_str)

1

Ideally, we wouldn’t change the contents of a variable called my_str to be an int, but it is something that python would let use do.

One last thing to remember is that if we assign a variable with another variable, it will be assigned to the result of the variable and not whatever that variable points to later.

>>> my_str = 1

>>> my_int = my_str

>>> my_str = "testing"

>>> print(my_int)

1

>>> print(my_str)

testing

Lists

A list is created in Python by using the square brackets ([, and ]) and separating the values by commas. Here’s an example list:

>>> my_list = [1, 2, 3, 4, 5]

There’s really not a limit to how long our list can be (there is, but it’s very unlikely that we’ll hit it while scripting).

Reading from Lists

To access an individual element of a list, you can use the index and Python uses a zero-based index system:

>>> my_list[0]

1

>>> my_list[1]

2

If we try to access an index that is too high (or too low) then we’ll receive an error:

>>> my_list[5]

Traceback (most recent call last):

File "", line 1, in

IndexError: list index out of range

To make sure that we’re not trying to get an index that is out of range, we can test the length using the lenfunction (and then subtract 1):

>>> len(my_list)

5

Additionally, we can access subsections of a list by “slicing” it. We provide the starting index and the ending index (the object at that index won’t be included).

>>> my_list[0:2]

[1, 2]

>>> my_list[1:0]

[2, 3, 4, 5]

>>> my_list[:3]

[1, 2, 3]

>>> my_list[0::1]

[1, 2, 3, 4, 5]

>>> my_list[0::2]

[1, 3, 5]

Modifying a List

Unlike strings which can’t be modified (you can’t change a character in a string), you can change a value in a list using the subscript equals operation:

>>> my_list[0] = "a"

>>> my_list

[‘a’, 2, 3, 4, 5]

If we want to add to a list we can use the .append method. This is an example of a method that modifies the object that is calling the method:

>>> my_list.append(6)

>>> my_list.append(7)

>>> my_list

[‘a’, 2, 3, 4, 5, 6, 7]

Lists can be added together (concatenated):

>>> my_list + [8, 9, 10]

[‘a’, 2, 3, 4, 5, 6, 7, 8, 9, 10]

>>> my_list += [8, 9, 10]

>>> my_list

[‘a’, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Items in lists can be set using slices also:

>>> my_list[1:3] = [‘b’, ‘c’]

>>> my_list

[‘a’, ‘b’, ‘c’, 4, 5, 6, 7, 8, 9, 10]

Replacing 2 sized slice with length 3 list inserts new element

my_list[3:5] = [‘d’, ‘e’, ‘f’]

print(my_list)

We can remove a section of a list by assigning an empty list to the slice:

>>> my_list = [‘a’, ‘b’, ‘c’, ‘d’, 5, 6, 7]

>>> my_list[4:] = []

>>> my_list

[‘a’, ‘b’, ‘c’, ‘d’]

Removing items from a list based on value can be done using the .remove method:

>>> my_list.remove(‘b’)

>>> my_list

[‘a’, ‘c’, ‘d’]

Attempting to remove and item that isn’t in the list will result in an error:

>>> my_list.remove(‘f’)

Traceback (most recent call last):

File "", line 1, in

ValueError: list.remove(x): x not in list

Items can also be removed from the end of a list using the pop method:

>>> my_list = [‘a’, ‘c’, ‘d’]

>>> my_list.pop()

‘d’

>>> my_list

[‘a’, ‘c’]

We can also use the pop method to remove items at a specific index:

>>> my_list.pop(0)

‘a’

>>> my_list

[‘c’]

>>> my_list.pop(1)

Traceback (most recent call last):

File "", line 1, in

IndexError: pop index out of range

>>> [ ].pop()

Traceback (most recent call last):

File "", line 1, in

IndexError: pop from empty list

Tuples

Tuples are a fixed width, immutable sequence type. We create tuples using parenthesis (( and )) and at least one comma (,):

>>> point = (2.0, 3.0)

Since tuples are immutable, we don’t have access to the same methods that we do on a list. We can use tuples in some operations like concatenation, but we can’t change the original tuple that we created.

>>> point_3d = point + (4.0,)

>>> point_3d

(2.0, 3.0, 4.0)

One interesting characterist of tuples is that we can unpack them into multiple variables at the same time:

>>> x, y, z = point_3d

>>> x

2.0

>>> y

3.0

>>> z

4.0

The time you’re most likely to see tuples will be when looking at a format string that’s compatible with Python 2:

>>> print("My name is: %s %s" % ("Keith", "Thompson"))

Dictionaries

Dictionaries are the main mapping type that we’ll use in Python. This object is comparable to a Hash or “associative array” in other languages.
Things to note about dictionaries:
1. Unlike Python 2 dictionaries, as of Python 3.6, keys are ordered in dictionaries. You’ll need OrderedDict if you want this to work on another version of Python.
2. You can set the key to any IMMUTABLE TYPE (no lists).
3. Avoid using things other than simple objects as keys.
4. Each key can only have one value (so don’t have duplicates when creating a dict).
We create dictionary literals by using curly braces ({ and }), separating keys from values using colons (:), and separating key/value pairs using commas (,). Here’s an example dictionary:

>>> ages = { ‘kevin’: 59, ‘alex’: 29, ‘bob’: 40 }

>>> ages

{‘kevin’: 59, ‘alex’: 29, ‘bob’: 40}

We can read a value from a dictionary by subscripting using the key:

>>> ages[‘kevin’]

59

>>> ages[‘billy’]

Traceback (most recent call last):

File "", line 1, in

KeyError: ‘billy’

Keys can be added or changed using subscripting and assignment:

>>> ages[‘kayla’] = 21

>>> ages

{‘kevin’: 59, ‘alex’: 29, ‘bob’: 40, ‘kayla’: 21}

Items can be removed from a dictionary using the del statement or by using the pop method:

>>> del ages[‘kevin’]

>>> ages

{‘alex’: 29, ‘bob’: 40, ‘kayla’: 21}

>>> del ages

>>> ages

Traceback (most recent call last):

File "", line 1, in

NameError: name ‘ages’ is not defined

>>> ages = { ‘kevin’: 59, ‘alex’: 29, ‘bob’: 40 }

>>> ages.pop(‘alex’)

29

>>> ages

{‘kevin’: 59, ‘bob’: 40}

It’s not uncommon to want to know what keys or values we have without caring about the pairings. For that situation we have the values and keys methods:

>>> ages = {‘kevin’: 59, ‘bob’: 40}

>>> ages.keys()

dict_keys([‘kevin’, ‘bob’])

>>> list(ages.keys())

[‘kevin’, ‘bob’]

>>> ages.values()

dict_values([59, 40])

>>> list(ages.values())

[59, 40]

Alternative Ways to Create a dict Using Keyword Arguments

There are a few other ways to create dictionaries that we might see, those being those that use the dictconstructor with key/value arguments and a list of tuples:

>>> weights = dict(kevin=160, bob=240, kayla=135)

>>> weights

{‘kevin’: 160, ‘bob’: 240, ‘kayla’: 135}

>>> colors = dict([(‘kevin’, ‘blue’), (‘bob’, ‘green’), (‘kayla’, ‘red’)])

>>> colors

{‘kevin’: ‘blue’, ‘bob’: ‘green’, ‘kayla’: ‘red’}

Comparisons

There are some standard comparison operators that we’ll use that match pretty closely to those used in mathematical equations. Let’s take a look at them:

>>> 1 < 2

True

>>> 0 > 2

False

>>> 2 == 1

False

>>> 2 != 1

True

>>> 3.0 >= 3.0

True

>>> 3.1 <= 3.0

False

If we try to make comparisons of types that don’t match up, we will run into errors:

>>> 3.1 <= "this"

Traceback (most recent call last):

File "", line 1, in

TypeError: ‘<=’ not supported between instances of ‘float’ and ‘str’

>>> 3 <= 3.1

True

>>> 1.1 == "1.1"

False

>>> 1.1 == float("1.1")

True

We can compare more than just numbers. Here’s what it looks like when we compare strings:

>>> "this" == "this"

True

>>> "this" == "This"

False

>>> "b" > "a"

True

>>> "abc" < "b"

True

Notice that the string ‘b’ is considered greater than the strings ‘a’ and ‘abc’. The characters are compared one at a time alphabetically to determine which is greater. This concept is used to sort strings alphabetically.

The in Check

We often get lists of information that we need to ensure contains (or doesn’t contain) a specific item. To make this check in Python, we’ll use the in and not in operations.

>>> 2 in [1, 2, 3]

True

>>> 4 in [1, 2, 3]

False

>>> 2 not in [1, 2, 3]

False

>>> 4 not in [1, 2, 3]

True

if/elif/else

With a grasp on comparisons, we can now look at how we can run different pieces of logic based on the values that we’re working with using conditionals. The keywords for conditionals in Python are if, elif, and else. Conditionals are the first language feature that we’re using that requires us to utilize whitespace to separate our code blocks. We will always use indentation of 4 spaces. The basic shape of an if statement is this:

if CONDITION:

pass

The CONDITION portion can be anything that evaluates to True or False, and if the value isn’t explicitly a boolean, then it will be converted to determine how to carry out proceed past the conditional (basically using the bool constructor).

>>> if True:

… print("Was True")

Was True

>>> if False:

… print("Was True")

>>>

To add an alternative code path, we’ll use the else keyword, followed by a colon (:), and indenting the code underneath:

>>> if False:

… print("Was True")

… else:

… print("Was False")

Was False

In the even that we want to check multiple potential conditions we can use the elif CONDITION: statement. Here’s a more robust example:

>>> name = "Kevin"

>>> if len(name) >= 6:

… print("name is long")

… elif len(name) == 5:

… print("name is 5 characters")

… elif len(name) >= 4:

… print("name is 4 or more")

… else:

… print("name is short")

name is 5 characters

Notice that we fell into the first elif statement’s block and then the second elif block was never executed even though it was true. We can only exercise one branch in an if statement.

The while Loop

The most basic type of loop that we have at our disposal is the while loop. This type of loop repeats itself based on a condition that we pass to it. Here’s the general structure of a while loop:

while CONDITION:

pass

The CONDITION in this statement works the same way that it does for an if statement. When we demonstrated the if statement, we first tried it by simply passing in True as the condition. Let’s see when we try that same condition with a while loop:

>>> while True:

… print("looping")

looping

looping

looping

looping

That loop will continue forever, we’ve created an infinite loop. To stop the loop, press Ctrl-C. Infinite loops are one of the potential problems with while loops if we don’t use a condition that we can change from within the loop then it will continue forever if initially true. Here’s how we’ll normally approach using a while loop where we modify something about the condition on each iteration:

>>> count = 1

>>> while count <= 4:

… print("looping")

… count += 1

looping

looping

looping

looping

>>>

We can use other loops or conditions inside of our loops; we need only remember to indent four more spaces for each context. If in a nested context, we want to continue to the next iteration or stop the loop entirely. We also have access to the continue and break keywords:

>>> count = 0

>>> while count < 10:

… if count % 2 == 0:

… count += 1

… continue

… print(f"We’re counting odd numbers: {count}")

… count += 1

We’re counting odd numbers: 1

We’re counting odd numbers: 3

We’re counting odd numbers: 5

We’re counting odd numbers: 7

We’re counting odd numbers: 9

>>>

In that example, we also show off how to “string interpolation” in Python 3 by prefixing a string literal with an fand then using curly braces to substitute in variables or expressions (in this case the count value).

Here’s an example using the break statement:

>>> count = 1

>>> while count < 10:

… if count % 2 == 0:

… break

… print(f"We’re counting odd numbers: {count}")

… count += 1

We’re counting odd numbers: 1

The for Loop

The most common use we have looping is when we want to execute some code for each item in a sequence. For this type of looping or iteration, we’ll use the for loop. The general structure for a for loop is:

for TEMP_VAR in SEQUENCE:

pass

The TEMP_VAR will be populated with each item as we iterate through the SEQUENCE and it will be available to us in the context of the loop. After the loop finishes one iteration, then the TEMP_VAR will be populated with the next item in the SEQUENCE, and the loop’s body will execute again. This process continues until we either hit a break statement or we’ve iterated over every item in the SEQUENCE. Here’s an example looping over a list of colors:

>>> colors = [‘blue’, ‘green’, ‘red’, ‘purple’]

>>> for color in colors:

… print(color)

Other Iterable Types

Lists will be the most common type that we iterate over using a for loop, but we can also iterate over other sequence types. Of the types we already know, we can iterate over strings, dictionaries, and tuples.

Here’s a tuple example:

>>> point = (2.1, 3.2, 7.6)

>>> for value in point:

… print(value)

2.1

3.2

7.6

>>>

A dictionary example:

>>> ages = {‘kevin’: 59, ‘bob’: 40, ‘kayla’: 21}

>>> for key in ages:

… print(key)

kevin

bob

kayla A string example:

>>> for letter in "my_string":

… print(letter)

m

y

_

s

t

r

i

n

g

>>>

Unpacking Multiple Items in a for Loop

We discussed in the tuples how you can separate a tuple into multiple variables by “unpacking” the values. Unpacking works in the context of a loop definition, and you’ll need to know this to most effectively iterate over dictionaries because you’ll usually want the key and the value. Let’s iterate of a list of “points” to test this out:

>>> list_of_points = [(1, 2), (2, 3), (3, 4)]

>>> for x, y in list_of_points:

… print(f"x: {x}, y: {y}")

x: 1, y: 2

x: 2, y: 3

x: 3, y: 4

Seeing how this unpacking works, let’s use the items method on our ages dictionary to list out the names and ages:

>>> for name, age in ages.items():

… print(f"Person Named: {name}")

… print(f"Age of: {age}")

Person Named: kevin

Age of: 59

Person Named: bob

Age of: 40

Person Named: kayla

Age of: 21