Chapter 2. The Basics of Python Scripting

Before diving into writing your first Python script, a few concepts should be understood. Learning these items now will help you develop code quicker in the future. This will improve your abilities as a penetration tester or in understanding what an assessor is doing when they are creating real-time custom code and what questions you should be asking. You should also understand how to create the scripts and the goal you are trying to achieve. You will often find out that your scripts will morph over time and the purpose may change. This may happen because you realize that the real need for the script may not be there or that there is an existing tool for the particular capability.

Many scripters find this discouraging, as a project that they may have been working on for a great deal of time you may find that the tool has duplicate features of more advanced tools. Instead of looking at this as a failed project, look at the activity as an experience wherein you learned new concepts and techniques that you did not initially know. Additionally, keep it at the back of your mind at all times when you are developing code snippets that can be used for other projects in the future.

To this end, try and build your code cleanly, comment it with what you are doing, and make it modular so that once you learn how to build functions, they can be cut and pasted into other scripts in the future. The first step in this journey is to describe the computer science glossary at a high level so that you can understand future chapters or other tutorials. Without understanding these basic concepts, you may misunderstand how to achieve your desired results.

Note

Before running any of the scripts in this module, I recommend that you run the setup script on the git repository, which will configure your Kali instance with all the necessary libraries. The script can be found at https://raw.githubusercontent.com/funkandwagnalls/pythonpentest/master/setup.sh.

Understanding the difference between interpreted and compiled languages

Python, like Ruby and Perl, is an interpreted language, which means that the code is turned into a machine language and run as the script is executed. A language that needs to be compiled prior to running, such as Cobol, C, or C++, can be more efficient and faster, as it is compiled prior to execution, but it also means that the code is typically less portable. As compiled code is generated for specific environments, it may not be as useful when you have to move through heterogeneous environments.

Note

A heterogeneous environment is an environment that has multiple system types and different distributions. So, there may be multiple Unix/Linux distributions, Mac OS, and Windows systems.

Interpreted code usually has the benefit of being portable to different locations as long as the interpreter is available. So for Python scripts, as long as the script is not developed for an operating system, the interpreter is installed, and the libraries are natively available, the Python script should work. Always keep in mind that there will be idiosyncrasies in an environment, and before scripts are used, they should be thoroughly tested in similar test beds.

So why should you learn Python over other scripting languages? I am not making this argument here, and the reason is that the best assessors use the tools available in the environment that they are assessing. You will build scripts that are useful for assessing environments, and Python is fantastic for doing this, but when you gain access to a system, it is best to use what is available to you.

Highly secure environments may prevent you from using exploitation frameworks, or the assessment rules may do the same. When this happens, you have to look at what is available on the system to take advantage of and move forward. Today, newer generation Windows systems are compromised with PowerShell. Often in current Mac, Linux, Unix, and Windows Operating System (OS), you can find a version of Python, especially in development environments. On web servers, you will find Ruby, Python, or Perl. On all forms of operating systems, you will find native shell languages. They provide many capabilities, but typically, they have archaic language structures that require more lines of code than other scripting languages to accomplish the same task. Examples of these shell languages would include Bourne-again Shell (BASH), Korn Shell (KSH), Windows Command Shell, and equivalents.

In most exploitation systems, you will find all the languages, as most hacking laptops, or HackTops, use multiple Virtual Machines (VMs) with many operating systems. Older assessment tools were coded in Perl, as the language provided multiple capabilities that other interpreted languages could not provide at that time. Newer tools are typically created in Ruby and Python. In fact, many libraries that are being created today are for improving the capabilities of these languages, specifically for assessing the potential viability an organization has for being compromised by a malicious actor.

Tip

Keep in mind that your HackTop has multiple VMs to provide you with not only attack tools but also a test bed to test your scripts safely. Reverting to a snapshot of a VM on your HackTop is easy, but telling a customer why you damaged their business-critical component with an untested script is not.

Compiled languages are not without value; many tools have been created in C, C++, and Java. Examples of these types of tools include Burp Suite, Cain & Abel, DirBuster, Zed Attack Proxy (ZAP), CSRFtester, and so on. You might notice that most of these tools were generated originally in the early days of assessing environments. As systems have gotten more powerful and interpreters have become more efficient, we have seen additional tools move to languages that are interpreted as against compiled.

So what is the lesson here? Learn as much as you can to operate in as many environments as possible. In this way, when you encounter an obstacle, you can return to the code and script your way to the level of access necessary.

Python – the good and the bad

Python is one of the easiest languages for creating a working piece of code that accomplishes tangible results. In fact, Python has a native interactive interpreter through which you can test code directly by just executing the word python at the CLI. This will bring up an interface in which concepts of code can be tested prior to trying to write a script. Additionally, this interface allows a tester to not only test new concepts, but also to import modules or other scripts as modules and use them to create powerful tools.

Not only does this testing capability of Python allow assessors to verify concepts, but they can also avoid dealing with extensive debuggers and test cases to quickly prototype attack code. This is especially important when on an engagement and when determining whether a particular exploit train will net useful results in a timely manner. Most importantly, the use of Python and the importing of specific libraries usually do not break entire tool suites, and uninstalling a specific library is very easy.

Note

To maintain the integrity of the customer environment, you should avoid installing libraries on client systems. If there is a need to do so, make sure that you work with your point of contact, because there may be unintended consequences. It could also be considered a violation of the organization's System Development Life cycle (SDLC) and its change control process. The end result is that you could be creating more risk for the client than the original assessment's intention.

The language structure for Python, though different from many other forms of coding, is very simple. Reading Python is similar to reading a module, but with some slight caveats. There are basically two different forms of Python development trees at the time of writing this module—Python 2.X and Python 3.X. Most assessment tools run on the 2.X version, which is what we will be focusing on, but improvements in the language versions for all intents and purposes has stopped. You can write code that works for both versions, but it will take some effort.

In essence, Python version 3.X has been developed to be more Object-oriented (OO), which means that coding for it means focusing on OO methods and attributes. This is not to say that 2.X is not OO; it's just that it is not as well developed as version 3.X. Most importantly, some libraries are not compatible with both versions.

Believe it or not, the most common reason a Python script is not completely version compatible is the built-in print function.

Note

In Python 2.X, print is a statement, and in 3.X, it is a function, as you will see next. Throughout this module, the use of the word statement and function may be used interchangeably, but understanding the difference is the key to building version-agnostic scripts.

Attempting to print something on the screen with print can be done in two ways. One is by using wrapped-in parameters, and the other is without using them. If it is with wrapped-in parameters, it is compatible with both 2.X and 3.X; if not, then it will work with 2.X only.

The following example shows what a 2.X-only print function looks like:

print "You have been hacked!"

This is an example of a print function that is compatible with both 2.X and 3.X Python interpreters:

print("You have been hacked!")

After you have started creating scripts, you will notice how often you will be using the print function in your scripts. As such, large-scale text replacements in big scripts can be laborious and error-prone, even with automated methods. Examples include the use of sed, awk, and other data manipulation tools.

As you become a better assessor, you should endeavor to write your scripts so that they would run in either version. The reason is that if you compromise an environment and you need a custom script to complete some post-exploitation activity, you would not want to be slowed down because it is version incompatible. The best way to start is to make sure that you use print functions that are compatible with both versions of Python.

Note

OO programming means that the language supports objects that can be created and destroyed as necessary to complete tasks. Entire training classes have been developed on explaining and expanding on OO concepts. Deep explanations of these concepts are beyond the scope of this module, but further study is always recommended.

In addition to the OO thought process and construction of OO supported code, there is also creating scripts "Pythonically," or "Pythonic scripts". This is not made up; instead, it is a way of defining the proper method of creating and writing a Python script. There are many ways you can write a Python script, and over the years, best practices have evolved. This is called Pythonic, and as such, we should always endeavor to write in this fashion. The reason is that when we, as contributors, provide scripts to the community, they are easier to read, maintain, and use.

Note

Pythonic is a great concept as it deals with some of the biggest things that have impacted the adoption of other languages and bad practices among the community.

A Python interactive interpreter versus a script

There are two ways in which the Python language can be used. One is through an interactive interpreter, that allows quick testing of functions, code snippets, and ideas. The other is through a full-fledged script that can be saved and transported between systems. If you want to try out an interactive interpreter, just type python in your command-line shell.

Note

An interactive interpreter will function the same way in different operating systems, but the libraries and called functions that interact with a system may not. If specific locations are referenced or if commands and/or libraries use operating-system-specific capabilities, the functionality will be different. As such, referencing these details in a script will impact its portability substantially, so it is not considered a leading practice.

Environmental variables and PATH

These variables are important for executing scripts written in Python, not for writing them. If they are not configured, the location of the Python binary has to be referenced by its fully qualified path location. As an example, here is the execution of a Python script without the environmental variable being declared in Windows:

C:\Python27\python wargames_print.py

The following is the equivalent in Linux or Unix if the reference to the proper interpreter is not listed at the top of the script and the file is in your current directory:

/usr/bin/python ./wargames_print.py

In Windows, if the environmental variable is set, you can simply execute the script by typing python and the script name. In Linux and Unix, we add a line at the top of the script to make it more portable. A benefit to us (penetration testers) is that this makes the script useful on many different types of systems, including Windows. This line is ignored by the Windows operating system natively, as it is treated as a comment. The following referenced line should be included at the top of all Python scripts:

#!/usr/bin/env python

This line lets the operating system determine the correct interpreter to run based on what is set in the PATH environmental variable. In many script examples on the Internet, you may see a direct reference to an interpreter, such as /usr/bin/python. This not considered good practice as it makes the code less portable and more prone to errors with potential system changes.

Tip

Setting up and dealing with PATH and environmental variables will be different for each operating system. Refer to details can be found at https://docs.python.org/2/using/unix.html#python-related-paths-and-files. Additionally, if you need to create specialty environmental variables for a specific tool someday, you can find the details at https://docs.python.org/2/using/cmdline.html.

Understanding dynamically typed languages

Python is a dynamically typed language, which means many things, but the most crucial aspect is how variables or objects are handled. Dynamically typed languages are usually synonymous with scripting languages, but this is not always the case, just to be clear. What this means to you when you write your script is that variables are interpreted at runtime, so they do not have to defined in size or by content.

The first Python script

Now that you have a basic idea of what Python is, let's create a script. Instead of the famous Hello World! introduction, we are going to use a cult film example. The scripts will define a function, which will print a famous quote from the 1983 cult classic WarGames. There are two ways of doing this, as mentioned previously; the first is through the interactive interpreter, and the second is through a script. Open an interactive interpreter and execute the following line:

print("Shall we play a game?\n")

The preceding print statement will show that the code execution worked. To exit the interactive interpreter, either type exit() or use Ctrl + Z in Windows or Ctrl + D in Linux. Now, create a script in your preferred editing tool, such as vi, vim, emacs, or gedit. Then save the file in /root/Desktop as wargames_print.py:

#!/usr/bin/env python
print("Shall we play a game?\n")

After saving the file, run it with the following command:

python /root/Desktop/wargames_print.py

You will again see the script execute with the same results. Be aware of a few items in this example. The python script is run by referencing the fully qualified path so as to ensure that the correct script is called, no matter what the location is. If the script resided in the current location, it could, instead, be executed in the following manner:

python ./wargames_print.py

Tip

Kali does not natively require ./ to execute these scripts, but it is a good habit to be in, as most other Linux and Unix operating systems do. If you are out of the habit and slightly sleep deprived on an assessment, you may not realize why your script is not executing initially. This technique can save you a little embarrassment on multimember team engagements.

Developing scripts and identifying errors

Before we jump into creating large-scale scripts, you need to understand the errors that can be produced. If you start creating scripts and generating a bunch of errors, you may get discouraged. Keep in mind that Python does a pretty good job at directing you to what you need to look at. Often, however, the producer of the error is either right before the line referenced or the function called. This in turn can be misleading, so to prevent discouragement, you should understand the definitions that Python may reference in the errors.

Reserved words, keywords, and built-in functions

Reserved words, keywords, and built-in functions are also known as prohibited, which means that the name cannot be used as a variable or function. If the word or function is reused, an error will be shown. There are set words and built-in functions natively within Python, and depending on the version you are using, they can change. You should not worry too much about this now, but if you see errors related to the definitions of variables or values, consider the fact that you may be using a keyword or built-in function.

Note

More details about keywords and built-in functions can be found at https://docs.python.org/2/library/keyword.html.

Here are some examples of Python keywords and some brief definitions. These are described in detail throughout the rest of the chapter:

If you want to confirm a name as a keyword, fire up the interactive interpreter and set a variable to the specific keyword name. Then, run it through the function of keyword. If it returns true, then you know it is a keyword; if it returns false, you know it is not. Refer to the following screenshot to better understand this concept:

Global and local variables

Global variables are defined outside of functions, and local variables are defined within a specific function. This is important because if the name is reused within a function, its value will remain only within that function—typically. If you wished to change the value of a global variable, you could call the global version with the global keyword and set a new value. This practice should be avoided, if at all possible. As an example of local and global variable usage, see this code:

#!/usr/bin/env python

hacker = "me"

def local_variable_example():
    hacker = "you"
    print("The local variable is %s") % (hacker)

local_variable_example()
print("The global variable is %s") % (hacker)

The following output of this script shows the printing of the local variable hacker within the local_variable_example function example. Then, we have the printing of the global variable hacker after the function has been executed.

Note

The preceding example shows how to insert a value into a string through a variable. Further along in this chapter, several methods of doing this are provided.

Understanding a namespace

The basic idea of a variable in Python is a name; these names reside in a bucket. Every module or script receives its own global namespace, and the names reside in this bucket, which is called the namespace. This means that when a name is used, it is reserved for a specific purpose. If you use it again, it is going to result in one of two things: either you are going to overwrite the value or you are going to see an error.

Modules and imports

Within Python, a library or module can be imported to execute a specific task or supplement functionality. When you have written your own script, you can import a script as a module to be used within a new script. There are a couple of ways of doing this, and each way has its benefits and disadvantages:

import module

This allows you to import a module and use it and functions by referencing them similar to a function. As an example, you could reference the module and the function within the module as module.function(). This means that your namespace is kept simple and you do not have to worry about overwrites and collisions, unlike the following method:

from module import *

This is very commonly seen in Python scripts and examples on the Internet. The danger is that all functions or functions within the module are brought in directly. This means that if you defined a function within your script named hacker_tool and hacker_tool (the imported module contains a module with the same name), you could get a namespace collision and produce multiple errors. At runtime, when the script is interpreted, it will take up a larger memory footprint because unnecessary functions are imported. The benefit, however, is that you will not have to identify the necessary function, nor will you have to the method of module.function(). You can instead just directly call function().

The next two methods are ways of referencing a module or function as a different name. This allows you to shorten statements that need reuse and can often improve readability. The same namespace conflicts are present, so your imports and references should be defined carefully. The first is the declaration of a module as a different name:

import module as a

The second is the declaration of a function as a different name:

from module import function as a

There are other methods of executing these tasks, but this is enough to read the majority of the scripts produced and create useful tools.

Tip

Did you know that Python modules are scripts themselves? You can take a look at how these products work by checking out the Lib directory within the Python installation of Windows, which defaults to C:\Python27\Lib for Python 2.7. In Kali Linux, it can be found at /usr/lib/python2.7.

Python formatting

This language's greatest selling feature for me is its formatting. It takes very little work to put a script together, and because of its simplistic formatting requirements, you reduce chances of errors. For experienced programmers, the loathsome ; and {} signs will no longer impact your development time due to syntax errors.

Indentation

The most important thing to remember in Python is indentation. Python uses indents to show where logic blocks are changed. So, if you are writing a simple print script as mentioned earlier, you are not necessarily going to see this, but if you are writing an if statement, you will. See the following example, which prints the statement previously mentioned here:

#!/usr/bin/env python
execute=True
if execute != False:
    print("Do you want to play a game?\n")

More details on how this script operates and executes can be found in the Compound statements section of this chapter. The following example prints the statement to the screen if execute is not False. This indentation signifies that the function separates it from the rest of the global code.

There are two ways of creating an indent: either through spaces or through tabs. Four spaces are equivalent to one tab; the indentation in the preceding code signifies the separation of the codes logic from the rest of the global code. The reason for this is that spaces translate better when moved from one system type to another, which again makes your code more portable.

Python variables

The Python scripting language has five types of variables: numbers, strings, lists, dictionaries, and tuples. These variables have different intended purposes, reasons for use, and methods of declaration. Before seeing how these variable types work, you need to understand how to debug your variables and ensure that your scripts are working.

Note

Lists, tuples, and dictionaries fall under a variable category know as data structures. This chapter covers enough details to get you off the ground and running, but most of the questions you notice about Python in help forums are related to proper use and handling of data structures. Keep this in mind when you start venturing on your own projects outside of the details given in this module. Additional information about data structures and how to use them can be found at https://docs.python.org/2/tutorial/datastructures.html.

Debugging variable values

The simple solution for debugging variable values is to make sure that the expected data is passed to a variable. This is especially important if you need to convert a value in a variable from one type to another, which will be covered later in this chapter. So, you need to know what the value in the variable is, and often what type it is. This means that you will have to debug your scripts as you build them; this is usually done through the use of print statements. You will often see initial scripts sprinkled with print statements throughout the code. To help you clean these at a later point in time, I recommend adding a comment to them. I typically use a simple #DEBUG comment, as shown here:

print(variable_name) #DEBUG

This will allow you to quickly search for and delete the #DEBUG line. In vi or vim, this is very simple—by first pressing Esc, then pressing :, and then executing the following command, which searches for and deletes the entire line:

g/.*DEBUG/d

If you wanted to temporarily comment out all of the #DEBUG lines and delete them later, you can use the following:

%s/.*DEBUG/#&

String variables

Variables that hold strings are basically words, statements, or sentences placed in a reference. This item allows easy reuse of values as needed throughout a script. Additionally, these variables can be manipulated to produce different values over the course of the script. To pass a value to the variable, the equal to sign is used after the word has been selected to assign a value. In a string, the value is enclosed in either quotes or double quotes. The following example shows how to assign a value using double quotes:

variable_name = "This is the sentence passed"

The following example shows single quotes assigned to a variable:

variable_name = 'This is the sentence passed'

The reason for allowing both single and double quotes is to grant a programmer the means to insert one or the other into a variable as a part of a sentence. See the following example to highlight the differences:

variable_name = 'This is the "sentence" passed'

In addition to passing strings or printing values in this method, you can use the same type of quote to escape the special character. This is done by preceding any special character with a \ sign, which effectively escapes the special capability. The following example highlights this:

variable_name = "This is the \"sentence\" passed"

The important thing about declaring strings is to pick a type of quote to use—either single or double—and use it consistently through the script. Additionally, as you can see in Python, variable sizes do not have to be declared initially. This is because they are interpreted at runtime. Now you know how to create variables with strings in them. The next step is to create variables with numbers in them.

Number variables

Creating variables that hold numbers is very straight forward. You define a variable name and then assign it a value by placing a number on the right-hand side of an equal to sign, as shown here:

variable_name = 5

Once a variable has been defined, it holds a reference to the value it was passed. These variables can be overwritten, can have mathematical operations executed against them, and can even be changed in the middle of the program. The following example shows variables of the same type being added together and printed. First, we show the same variable added and printed, and then we show two different variables. Finally, the two variables are added together, assigned to a new variable, and printed.

Notice that the numerical values passed to the variables do not have quotes. If they did, the Python interpreter would consider them as strings, and the results would be significantly different. Refer to the following screenshot, which shows the same method prescribed to numeric variables with string equivalents:

As you can see, the values are—instead—merged into a single string verses adding them together. Python has built-in functions that allow us to interpret strings as numbers and numbers as strings. Additionally, you can determine what a variable is using the type function. This screenshot shows the declaration of two variables, one as a string and one as an integer:

Had the variable been declared with a decimal value in it, it would have been declared as a floating-point number or a float for short. This is still a numeric variable, but it requires a different method of storage, and as you can see, the interpreter has determined that for you. The following screenshot shows an example of this:

Converting string and number variables

As mentioned in the number variables section, Python has functions that are built-in in a manner that allows you to convert one variable type to another. As a simple example, we are going to convert a number into a string and string into a number. When using the interactive interpreter, the variable value will be printed immediately if it is not passed to a new variable; however, in a script, it will not. This method of manipulation is extremely useful if data is passed by the Command-line Interface (CLI) and you want to ensure the method that the data will be handled.

This is executed using the following three functions: int(), str(), and float(). These functions do exactly what you think they would; int() changes the applicable variables of other types to integers, str() turns other applicable variable types to strings, and float() turns applicable variables to floating-point numbers. It is important to keep in mind that if the variable cannot be converted to the desired type, you will receive a ValueError exception, as shown in this screenshot:

As an example, let's take a string and an integer and try to add them together. If the two values are not of the same type, you will receive a TypeError exception. This is demonstrated in the following screenshot:

This is where you will have to determine what type the variable is and choose one of them to convert to the same type. Which one you choose to convert will depend on the expected outcome. If you want a variable that contains the total value of two numbers, then you need to convert string variables into number type variables. If you want the values to be combined together, then you would convert the non-string variable into a string. This example shows the definition of two values: one of a string and one of an integer. The string will be converted into an integer to allow the mathematical operation to continue, as follows:

Now that you can see how easy this is, consider what would happen if a string variable was the representative of a float value and was converted to an integer. The decimal portion of the number will be lost. This does not round the value up or down; it just strips the decimal part and gives a whole number. Refer to the following screenshot to understand an example of this:

So be sure to change the numeric variable to the appropriate type. Otherwise, some data will be lost.

List variables

Lists are data structures that hold values in a method that can be organized, adjusted, and easily manipulated. An easy way to identify a list in Python is by [], which denotes where the values will reside. The manipulation of these lists is based on adjusting the values by position, typically. To create a list, define a variable name, and on the right-hand side of the equal to sign, place brackets with comma-separated values. This simple script counts the length of a predefined list and iterates and prints the position and value of the list. It is important to remember that a list starts at position 0, not 1. Since a list can contain different types of variables in order to include other lists, we are going to print the values as strings to be safe:

#!/usr/bin/env python

list_example = [100,222,333,444,"string value"]
list_example_length = len(list_example)
for iteration in list_example:
 index_value = list_example.index(iteration)
 print("The length of list list_example is %s, the value at position %s is %s") % (str(list_example_length), str(index_value), str(iteration).strip('[]'))

print("Script finished")

The following screenshot shows the successful execution of this script:

As you can see, extracting values from a list and converting them into numerical or string values are important concepts. Lists are used to hold multiple values, and extracting these values so that they can be represented is often necessary. The following code shows you how to do this for a string:

#!/usr/bin/env python

list_example = [100,222,333,444]
list_value = list_example[2]
string_value_from_list = str(list_value)
print("String value from list: %s") % (str(list_value))

It is important to note that a list cannot be printed as an integer, so it has to be either converted to a string or iterated through and printed. To show only the simple differences, the following code demonstrates how to extract an integer value from the list and print both it and a string:

#!/usr/bin/env python

list_example = [100,222,333,444]
list_value = list_example[2]
int_value_from_list = int(list_value))
print("String value from list: %s") % (str(list_value))
print("Integer value from list: %d") % (int_value_from_list)

List values can be manipulated further with list-specific functions. All you have to do is call the name of the list and then add .function(x) to the list, where function is the name of the specific activity you want to accomplish and x is the position or data you want to manipulate. Some common functions used include adding values to the end of a list, such as the number 555, which would be accomplished like this: list_example.append(555). You can even combine lists; this is done using the extend function, which adds the relevant items at the end of the list. This is accomplished by executing the function as follows: list_example.extend(list_example2). If you want to remove the value of 555, you can simply execute list_example.remove(555). Values can be inserted in specific locations using the appropriately named insert function like this: list_example.insert(0, 555). The last function that will be described here is the pop function, which allows you to either remove the value at a specific location by passing a positional value, or remove the last entry in the list by specifying no value.

Tuple variables

Tuples are similar to lists, but unlike lists, they are defined using (). Also, they are immutable; that is, they cannot be changed. The motive behind this is to provide a means of controlling data in complex operations that will not destroy it during the process. A tuples can be deleted, and a new tuple can be created to hold portions of a different tuple's data and show as if the data has changed. The simple rule with tuples is as follows: if you want data to be unaltered, use tuples; otherwise, use lists.

Dictionary variables

Dictionaries are a means of associating a key with a value. If you see curly brackets, it means that you are looking at a dictionary. The key represents a reference to a specific value stored in an unsorted data structure. You may be asking yourself why you would do this when standard variables already do something similar. Dictionaries provide you with the means to store other variables and variable types as values. They also allow quick and easy referencing as necessary. You will see detailed examples of dictionaries in later chapters; for now, check out the following example:

#!/usr/bin/env python
dictionary_example = {'james':123,'jack':456}
print(dictionary_example['james'])

This example will print the numbers related to the 'james' key, as shown in the following screenshot:

Adding data to dictionaries is extremely simple; you just have to assign a new key to the dictionary and a value for that key. For example, to add the value of 789 to a 'john' key, you can execute the following: dictionary_example['john'] = 789. This will assign the new value and key to the dictionary. More details about dictionaries will be covered later, but this is enough to gain an understanding of them.

Understanding default values and constructors

People who have programmed or scripted previously are probably used to declaring a variable with a default value or setting up constructors.

In Python, this is not necessary to get started, but it is a good habit to set a default value in a variable prior to its use. Besides being good practice, it will also mitigate some of the reasons for your scripts to have unexpected errors and crashes. This will also add traceability if a value is passed to a variable that was unexpected.

Tip

In Python, constructor methods are handled by __init__ and __new__ when a new object is instantiated. When creating new classes, however, it is only required to use the __init__ function to act as the constructor for the class. This will not be needed until much later, but keep it in mind; it is important if you want to develop a multithreaded application.

Passing a variable to a string

Let's say that you want to produce a string with a dynamic value, or include a variable in the string as it is printed and interpret the value in real time. With Python, you can do it in a number of ways. You can either combine the data using arithmetic symbols, such as +, or insert values using special character combinations.

The first example will use a combination of two strings and a variable joined with the statement to create a dynamic statement, as shown here:

#!/usr/bin/env python
name = "Hacker"
print("My profession is "+name+", what is yours?")

This produces the following output:

After creating the first script, you can improve it by inserting a value directly into the string. This is done by using the % special character and appending s for a string or d for a digit to produce the intended result. The print statement then has the % sign appended to it, with parameters wrapped around the requisite variable or variables. This allows you to control data quickly and easily and clean up your details as you prototype or create your scripts.

The variables in the parameters are passed to replace the keyed symbol in the statement. Here is an example of this type of script:

#!/usr/bin/env python
name = "Hacker"
print("My profession is %s, what is yours?") % (name)

The following image shows the code being executed:

An added benefit is that you can insert multiple values into this script without drastically altering it, as shown in the following example:

#!/usr/bin/env python

name = "Hacker"
name2 = "Penetration Tester"

print("My profession is %s, what is yours? %s") % (name, name2)

This form of insertion can be done with digits as mentioned in the preceding lines and by changing %s to %d:

#!/usr/bin/env python

name = "Hacker"
name2 = "Penetration Tester"
years = 15

print("My profession is %s, what is yours? %s, with %d years experience!") % (name, name2, years)

The output can be seen in this screenshot:

Instead of using variables, statements can be passed directly. There is usually little reason to do such things, as variables provide you with a means to change code and have it applied to the entire script. When possible, variables should be used to define statements as necessary. This is very important when you start writing statements that will be passed to systems. Use a combination of joined variables to create commands that will be executed in your Python scripts. If you do so, you can change the content provided to the system by simply changing a specific value. More examples on this will be covered later.

Operators

Operators in Python are symbols that represent functional executions.

Note

More details about this can be found at https://docs.python.org/2/library/operator.html.

The important thing to remember is that Python has extensive capabilities that allow complex mathematical and comparative operations. Only a few of them will be covered here to prepare you for more detailed work.

Comparison operators

A comparison operator checks whether a condition is true or false based on the method of evaluation. In simpler terms, we try to determine whether one value equals, does not equal, is greater than, is less than, is greater than or equal to, or is less than or equal to another value. Interestingly enough, the Python comparison operators are very straightforward.

The following table will help define the details of operators:

Assignment operators

Assignment operators confuse most people when they transition from a different language. The reason for this is that AND assignment operators are different from most languages. People who are used to writing incrementors short hands of variable = variable + 1 from in other languages using the format variable++, they are often confused to see the exact operation is not done in Python.

The functional equivalent of a variable incrementor in Python is variable=+1, which is the same as variable = variable + 1. You might notice something here, however; you can define what is added to the variable in this expression. So, instead of the double addition sign, which means, "add 1 to this variable," the AND expression allows you to add anything you want to it.

This is important when you write exploits, because you can append multiple hexadecimal values to the same string with this operator, as shown in the previous string concatenation example, where two strings were added together. exploit. Until then, consider this table to see the different assignment operators and what they are used for:

Arithmetic operators

Arithmetic operators are extremely simple overall and are what you would expect. Addition executions use the + symbol, subtraction executions use -, multiplication executions use *, and division executions use /. There are also additional items that can be used, but these four cover the majority of cases you are going to see.

Logical and membership operators

Logical and membership operators utilize words instead of symbols. Generally, Python's most confusing operators are membership operators, because new script writers think of them as logical operators. So let's take a look at what a logical operator really is.

A logical operator helps a statement or a compound statement determine whether multiple conditions are met so as to prove a true or false condition. So what does this mean in layman terms? Look at the following script, which helps determine whether two variables contain the values required to continue the execution:

#!/usr/bin/env python

a = 10
b = 5
if a == 10 and b == 5:
 print("The condition has been met")
else:
 print("the condition has not been met")

Logical operators include and, or, and not, which can be combined with more complex statements. The not operator here can be confused with not in, which is part of a membership operator. A not test reverses the combined condition test. The following example highlights this specifically; if both values or False or not equal to each other, then the condition is met; otherwise, the test fails. The reason for this is that the test checks whether it is both. Examples similar to this do surface, but they are not common, and this type of code can be avoided if you are not feeling comfortable with the logic flow yet:

#!/usr/bin/env python

a = False
b = False
if not(a and b):
 print("The condition has been met")
else:
 print("The condition has not been met")

Membership operators, instead, test for the value being part of a variable. There are two of these types of operators, in and not in. Here is an example of their usage:

#!/usr/bin/env python

variable = "X-Team"

if "Team" in variable:
 print("The value of Team is in the variable")
else:
 print("The value of Team is not in the variable")

The logic of this code will cause the statement to return as True and the first conditional message will be printed to screen.

Compound statements

Compound statements contain other statements. This means a test or execution while true or false executes the statements within itself. The trick is to write statements so that they are efficient and effective. Examples of this include if then statements, loops, and exception handling.

The if statements

An if statement tests for a specific condition, and if that condition is met (or not met), then the statement is executed. The if statement can include a simple check to see whether a variable is true or false, and then print the details, as shown in the following example:

x = 1
if x == 1:
 print("The variable x has a value of 1")

The if statement can even be used to check for multiple conditions at the same time. Keep in mind that it will execute the first portion of the compound statement that meets the condition and skip the rest. Here is an example that builds on the previous one, using else and elif statements. The else statement is a catch all if none of the if or elif statements are met. An elif test is a follow-on if test. Its condition can be tested after if and before else. Refer to the following example to understand this better:

#!/usr/bin/env python
x=1
if x == 3:
 print("The variable x has a value of 3")
elif x == 2:
 print("The variable x has a value of 2")
elif x == 1:
 print("The variable x has a value of 1")
else:
 print("The variable x does not have a value of 1, 2, or 3")

As you can see from these statements, the second elif statement will process the results. Change the value of x to something else and see how the script flow really works.

Keep one thing in mind: testing for conditions requires thinking through the results of your test. The following is an example of an if test that may not provide the expected results depending on the variable value:

#!/usr/bin/env python

execute=True
if execute != False:
 print("Do you want to play a game?\n")

This script sets the execute variable to True. Then, if is the script with the print statement. If the variable had not been set to True and had not been set to False either, the statement would have still been printed. The reason for this is that we are simply testing for the execute variable not being equal to False. Only if execute had been set to False would nothing be printed.

Python loops

A loop is a statement that is executed over and over until a condition is either met or not met. If a loop is created within another loop, it is known as an embedded loop. In penetration testing, having multiple loops within each other is typically not considered best practice. This is because it can create situations of memory exhaustion if they are not properly controlled. There are two primary forms of loops: while loops and for loops.

The while loop

The while loops are useful when a situation is true or false and you want the test to be executed as long as the condition is valid. As an example, this while loop checks whether the value of x is greater than 0, and if it is, the loop continues to process the data:

x=5
while x > 0:
print("Your current count is: %d") % (x)
 x -= 1

The for loop

The for loop is executed with the idea that a defined situation has been established and it is going to be tested. As a simple example, you can create a script that counts a range of numbers between 1 and 15, one number at a time, and then prints the results. The following example of a for loop statement does this:

for iteration in range(1,15,1):
 print("Your current count is: %d") % (iteration)

The break condition

A break condition is used to exit a loop and continue processing the script from the next statement. Breaks are used to control loops when a specific situation occurs within the loop instead of the next iteration of a loop. Even though breaks can be used to control loops, you should consider writing your code in such a way that you don't need breaks. The following loop with a break condition will stop executing if the variable value equals 5:

#!/usr/bin/
numeric = 15
while numeric > 0:
    print("Your current count is: %d") %(numeric)
    numeric -= 1
    if numeric == 5:
        break
print("Your count is finished!")

The output of this script is as follows:

Though this works, the same results can be achieved with a better designed script, as shown in the following code:

#!/usr/bin/env python

numeric = 15
for iteration in range(numeric,5,-1):
 print("Your current count is: %d") % (iteration)

print("Your count is finished!")

As you can see here, the same results are produced with cleaner and more manageable code:

Conditional handlers

Python, like many other languages, has the ability to handle situations where exceptions or relatively unexpected things occur. In such situations, a catch will occur and capture the error and the follow-on activity. This is completed with the try and except clauses, which handle the condition. As an example, I often use conditional handlers to determine whether the necessary library is installed, and if it is not, it tells you how and where to get it. This is a simple, but effective, example:

try:
 import docx
 from docx.shared import Inches
except:
 sys.exit("[!] Install the docx writer library as root or through sudo: pip install python-docx")

Functions

Python functions allow a scripter to create a repeatable task and have it called frequently throughout the script. When a function is part of a class or module, it means that a certain portion of the script can be called specifically from another script, also known as a module, once imported to execute a task. An additional benefit in using Python functions is the reduction of script size. An often unexpected benefit is the ability to copy functions from one script to another, speeding up development.

The impact of dynamically typed languages on functions on functions

Remember that variables hold references to objects, so as the script is written, you are executing tests with variables that reference the value. One fact about this is that the variable can change and can still point to the original value. When a variable is passed to a function through a parameter, it is done as an alias of the original object. So, when you are writing a function, the variable name within the function will often be different—and it should be. This allows easier troubleshooting, cleaner scripts, and more accurate error control.

Curly brackets

If you have ever written in another language, the one thing that will surprise you is that there are no curly brackets like these: {}. This is usually done to delineate where the code for a logic test or compound statement stops and begins, such as a loop, an if statement, a function, or even an entire class. Instead, Python uses the aforementioned indentation method, and the deeper the indent, the more nested the statement.

Note

A nested statement or function means that within a logic test or compound statement, another an additional logic test is being performed. An example would be an if statement within another if statement. More examples of this type will be seen later in this chapter.

To see a difference between logic tests in Python and other languages, an example of a Perl function known as a subroutine will be shown. An equivalent Python function will also be demonstrated to showcase the differences. This will highlight how Python controls logic flows throughout a script. Feel free to try both of these scripts and see how they work.

Note

The following Python script is slightly longer than the Perl one due to the fact that a return statement was included. This is not necessary for this script, but it is a habit many scripters get into. Additionally, the print statement has been modified, as you can see, to support both version 2.X and version 3.X of Python.

Here is an example of the Perl function:

#!/usr/bin/env perl

# Function in Perl
sub wargames{
 print "Do you want to play a game?\n";
print "In Perl\n";
}

# Function call
wargames();

The following function is the equivalent in Python:

#!/usr/bin/env python

# Function in Python
def wargames():
 print("Do you want to play a game?")
print("In Python")
return

# Function call
wargames()

The output of both of these scripts can be seen in this screenshot:

Instead, in Python, curly brackets are used for dictionaries, as previously described in the Python variable section of this chapter.

How to comment your code

In a scripting language, a comment is useful for blocking code and/or describing what it is trying to achieve. There are two types of comments in Python: single-line and multiline. Single-line comments make everything from the # sign to the end of the line a comment; it will not be interpreted. If you place code on the line and then follow it up with a comment at the end of the line, the code will still be processed. Here is an example of effective single-line comment usage:

#!/usr/bin/env python
#Author: Chris Duffy
#Date: 2015
x = 5 #This defines the value of the x followed by a comment

This works, but it may be easier to do the same thing using a multiline comment, as there are two lines within the preceding code are comments. Multiline comments are created by placing three quotes in each line that begins and ends the comment block. The following code shows an example of this:

"""
Author: Chris Duffy
Date: 2015
"""

The Python style guide

When writing your scripts, there are a few naming conventions to observe that are common to scripting and programming. These conventions are more of guidelines and best practices than hard rules, which means that you will hear opinions on both sides. As scripting is a form of art, you will see examples that rebut these suggestions, but following them will improve readability.

Note

Most of the suggestions here were borrowed from the style guide for Python, which can be found at http://legacy.python.org/dev/peps/pep-0008/, and follow-on style guides.

If you see specifics here that do not directly match this guide, keep in mind that all assessors develop habits and styles that differ. The trick is to incorporate as many of the best practices as possible while not impacting the speed and quality of development.

Classes

Classes typically begin with an uppercase letter, and the rest of the first word is lowercase. Each word after that starts with an uppercase letter as well. As such, if you see a defined reference being used and it begins with an uppercase letter, it is likely a class or module name. No spaces or underscores should be used between the words used to define a class, though people typically forget or break this rule.

Functions

When you are developing functions, remember that the words should be lowercase and separated by underscores.

Variables and instance names

Variables and instances should be lowercase with underscores separating the words, and if they are private, they must lead with two underscores. Public and Private variables are common in major programming languages, but in Python, they are not truly necessary. If you would like to emulate the functionality of a private variable in Python, you can lead the variable with __ to define it as private. A private member's major benefit in Python is the prevention of namespace clashing.

Arguments and options

There are multiple ways in which arguments can be passed to scripts; we will cover more on this in future chapters, as they are applicable to specific scripts. The simplest way to take arguments is to pass them without options. Arguments are the values passed to scripts to give them some dynamic capability.

Options are flags that represent specific calls to the script, stating the arguments that are going to be provided. In other words, if you want to get the help or usage instructions for a script, you typically pass the -h option. If you write a script that accepts both IP addresses and MAC addresses, you could configure it to use different options to signify the data that is about to be presented to it.

Writing scripts to take options is significantly more detailed, but it is not as hard as people make it out to be. For now, let's just look at basic argument passing. Arguments can be made natively with the sys library and the argv function. When arguments are passed, a list containing them is created in sys.argv, which starts at position 0.

The first argument provided to argv is the name of the script run, and each argument provided thereafter represents the other argument values:

#!/usr/bin/env python

import sys

arguments = sys.argv
print("The number of arguments passed was: %s") % (str(len(arguments)))
i=0
for x in arguments:
 print("The %d argument is %s") % (i,x)
 i+=1

The output of this script produces the following result:

Your first assessor script

Now that you have understood the basics of creating scripts in Python, let's create a script that will actually be useful to you. In later chapters, you will need to know your local and public IP addresses for each interface, hostname, Media Access Control (MAC) addresses, and Fully Qualified Domain Name (FQDN). The script that follows here demonstrates how to execute all of these. A few of the concepts here may still seem foreign, especially how IP and MAC addresses are extracted from interfaces. Do not worry about that; this is not the script you are going to write. You can use this script if you like, but it is here to show you that you can salvage components of scripts—even seemingly complex ones—to develop your own simple scripts.

Note

This script uses a technique to extract IP addresses for Linux/Unix systems by querying the details based on an interface that has been used in several Python modules and examples. The specific recipe for this technique can be found in many places, but the best documented reference to this technique can be found at http://code.activestate.com/recipes/439094-get-the-ip-address-associated-with-a-network-inter/.

Let's break down the script into its components. This script uses a few functions that make execution cleaner and repeatable. The first function is called get_ip. It takes an interface name and then tries to identify an IP address for that interface:

def get_ip(inter):
 s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
 ip_addr = socket.inet_ntoa(fcntl.ioctl(s.fileno(), 0x8915, struct.pack('256s', inter[:15]))[20:24])
 return ip_addr

The second function, called get_mac_address, identifies the MAC address of a specific interface:

def get_mac_address(inter):
 s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
 info = fcntl.ioctl(s.fileno(), 0x8927, struct.pack('256s', inter[:15]))
 mac_address = ''.join(['%02x:' % ord(char) for char in info[18:24]])[:-1]
 return mac_address

As you can see, these functions rely on the low-level network interface language of the socket library. Your concentration should not be on understanding every detail about this function, but more on the flow of information, the types of variables being used, and how the libraries are integrated. The reason for this is that you are going to generate a script later that requires fewer components and replicates the activity of grabbing a public IP address later.

The third function gets the details of the host and returns them to the main part of the script. It determines whether the host is Windows or not so that the correct functions are called. The function accepts two lists, one for Ethernet interfaces and the wireless interfaces typical in Linux/Unix. These interfaces are processed through the previous functions called in this bigger function. This allows the decision-making to be handled by the get_localhost_details function, and then returns the values for the host that will be represented by the print statements at the end of the script:

def get_localhost_details(interfaces_eth, interfaces_wlan):
    hostdata = "None"
    hostname = "None"
    windows_ip = "None"
    eth_ip = "None"
    wlan_ip = "None"
    host_fqdn = "None"
    eth_mac = "None"
    wlan_mac = "None"
    windows_mac = "None"
    hostname = socket.gethostbyname(socket.gethostname())
    if hostname.startswith("127.") and os.name != "nt":
        hostdata = socket.gethostbyaddr(socket.gethostname())
        hostname = str(hostdata[1]).strip('[]')
        host_fqdn = socket.getfqdn()
        for interface in interfaces_eth:
            try:
                eth_ip = get_ip(interface)
                if not "None" in eth_ip:
                    eth_mac = get_mac_address(interface)
                break
            except IOError:
                pass
        for interface in interfaces_wlan:
            try:
                wlan_ip = get_ip(interface)
                if not "None" in wlan_ip:
                    wlan_mac = get_mac_address(interface)
                break
            except IOError:
                pass
    else:
        windows_ip = socket.gethostbyname(socket.gethostname())
        windows_mac = hex(getnode()).lstrip('0x')
        windows_mac = ':'.join(pos1+pos2 for pos1,pos2 in zip(windows_mac[::2],windows_mac[1::2]))
        hostdata = socket.gethostbyaddr(socket.gethostname())
        hostname = str(hostdata[1]).strip("[]\'")
        host_fqdn = socket.getfqdn()
    return hostdata, hostname, windows_ip, eth_ip, wlan_ip, host_fqdn, eth_mac, wlan_mac, windows_mac

The final function in this script is called get_public_ip, which queries a known website for the IP address that is connected to it. This IP address is returned to the web page in a simple, raw format. There are a number of sites against which this can be done, but make sure you know the acceptable use and terms of service authorized. The function accepts one input, which is the website you are executing the query against:

def get_public_ip(request_target):
    grabber = urllib2.build_opener()
    grabber.addheaders = [('User-agent','Mozilla/5.0')]
    try:
        public_ip_address = grabber.open(target_url).read()
    except urllib2.HTTPError, error:
        print("There was an error trying to get your Public IP: %s") % (error)
    except urllib2.URLError, error:
        print("There was an error trying to get your Public IP: %s") % (error)
    return public_ip_address

For Windows systems, this script utilizes the simple socket.gethostbyname(socket.gethostname()) function request. This does work for Linux, but it relies on the /etc/hosts file to have the correct information for all interfaces. Much of this script can be replaced by the netifaces library, as pointed out by the previous reference. This would greatly simplify the script, and examples of its use will be shown in the following Chapter. The netifaces library is not installed by default, and so you will have to install it on every host on which you want to run this script. Since you typically do not want to make any impact on a host's integrity, this specific script is designed to avoid that conflict.

Tip

The final version of this script can be found at https://raw.githubusercontent.com/funkandwagnalls/pythonpentest/master/hostdetails.py.

The following screenshot shows the output of running this script. Components of this script will be used in later chapters, and they allow the automated development of exploit configurations and reconnaissance of networks.

So your useful script is going take components of this script and only find the public IP address of the system you are on. I recommend that you try doing this prior to looking at the following code (which shows what the actual script looks like). If you want to skip this step, the solution can be seen here:

import urllib2

def get_public_ip(request_target):
    grabber = urllib2.build_opener()
    grabber.addheaders = [('User-agent','Mozilla/5.0')]
    try:
        public_ip_address = grabber.open(target_url).read()
    except urllib2.HTTPError, error:
        print("There was an error trying to get your Public IP: %s") % (error)
    except urllib2.URLError, error:
        print("There was an error trying to get your Public IP: %s") % (error)
    return public_ip_address
public_ip = "None"
target_url = "http://ip.42.pl/raw"
public_ip = get_public_ip(target_url)
if not "None" in public_ip:
    print("Your Public IP address is: %s") % (str(public_ip))
else:
    print("Your Public IP address was not found")

The output of your script should look similar to this:

Tip

This script can be found at https://raw.githubusercontent.com/funkandwagnalls/pythonpentest/master/publicip.py.

Summary

This chapter focused on taking you through the basics of how the Python scripting language works and developing your own code by example. It also pointed out the common pitfalls related to creating scripts for assessments. The final section of this chapter focused on how to create useful scripts, even by simply piecing together components of already generated examples.

In the following chapter, we are going to dive even deeper into this subject with a proper reconnaissance of an environment, using nmap, scapy, and automation with Python.