Python - Advanced Wrangling With Pandas - Practice

QTM 350: Data Science Computing

Davi Moreira

Introduction


No description has been provided for this image

This topic material is based on the [Python Programming for Data Science](https://www.tomasbeuzen.com/python-programming-for-data-science/README.html) book and adapted for our purposes in the course.

Exercise

In this set of practice exercises we'll be looking at a cool dataset of real passwords (made available from actual data breaches) sourced and compiled from Information is Beautiful and contributed to R's Tidy Tuesday project. These passwords are common ("bad") passwords that you should avoid using! But we're going to use this dataset to practice some regex skills.

Let's start by importing pandas with the alias pd.

In [1]:
# Your answer here.

Exercise

The dataset has the following columns:

variable class description
rank int popularity in their database of released passwords
password str Actual text of the password
category str What category does the password fall in to?
value float Time to crack by online guessing
time_unit str Time unit to match with value
offline_crack_sec float Time to crack offline in seconds
rank_alt int Rank 2
strength int Strength = quality of password where 10 is highest, 1 is lowest, please note that these are relative to these generally bad passwords
font_size int Used to create the graphic for KIB

In these exercises, we're only interested in the password, value and time_unit columns so import only these two columns as a dataframe named df from this url: https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-01-14/passwords.csv

In [2]:
# Your answer here.

Exercise

An online password attack is when someone tries to hack your account by simply trying a very large number of username/password combinations to access your account. For each password in our dataset, the value column shows the amount of time it is estimated to take an "online password attack" to hack your account. The column time_unit shows the units of that time value (e.g., hours, days, years, etc.)

It would be much nicer if our values were of the same units so we can more easily compare the "online password guessing time" for each password. So your first task is to convert all of the values to units of hours (assume the conversion units I've provided below, e.g., 1 day is 24 hours, 1 week is 168 hours, etc).

In [3]:
units = {
    "seconds": 1 / 3600,
    "minutes": 1 / 60,
    "days": 24,
    "weeks": 168,
    "months": 720,
    "years": 8760,
}
In [4]:
# Your answer here.

Exercise

How many password begin with the sequence 123?

In [5]:
# Your answer here.

Exercise

What is the average time in hours needed to crack these passwords that begin with 123? How does this compare to the average of all passwords in the dataset?

In [6]:
# Your answer here.

Exercise

How many passwords do not contain a number?

In [7]:
# Your answer here.

Exercise

How many passwords contain at least one number?

In [8]:
# Your answer here.

Exercise

Is there an obvious difference in online cracking time between passwords that don't contain a number vs passwords that contain at least one number?

In [9]:
# Your answer here.

Exercise

How many passwords contain at least one of the following punctuations: [.!?\\-] (hint: remember this dataset contains weak passwords...)?

In [10]:
# Your answer here.

Exercise

Which password(s) in the datasets took the shortest time to crack by online guessing? Which took the longest?

In [11]:
# Your answer here.
In [ ]:
!jupyter nbconvert _09-py-wrangling-advanced-practice.ipynb --to html --template classic --output 09-py-wrangling-advanced-practice.html

Have fun!