Mini Project 3: Pythonifier

This assignment is due on Friday, Apr. 26th, at 11:30 PM.

This assignment covers the material up to Lecture 9, focusing on file io, dictionaries and tuples, and program decomposition.

You will be handing in the following completed files, all of which are started for you in mp3_starter.zip:

• analyzer.py
• fixer.py
• generator.py

Similar to MP 2, make sure you write a docstring for every function we ask you to write. It should describe what the function does, what the arguments represent, and what the return value represents, as we’ve discussed in class and in the readings (refer to Lecture 4 Slides and recent lecture code for some examples). Do not include implementation details. For very short or very simple functions you can just write a one-line description of what the function does; this is in line with the guidelines in PEP 257.

Note: We have provided you both docstrings in Part 1's analyzer.py which you can leave as-is and reference for examples of documentation for file-processing functions.

There are a number of supporting files you need to download for this assignment, which you can find in mp3_starter.zip. They include:

As you'll learn, tabs are a very annoying culprit in codebases leading to some serious (╯°□°)╯︵ ┻━┻. Let's write a function to quickly swap out those subtle nuances. ┬─┬ノ(^_ ^ノ)

Write a function tabs_to_spaces which takes a filename string and a number of tab-spaces and converts each tab ('\t') character in that file to the given number of spaces per tab. The file with the given filename should not change, but the results should be written to a new file in the format spaced_<filename> (e.g. spaced_test.py for test.py) which should be identical to the original file only replacing the tabs with spaces as specified (hint: you can use the string's replace(old, new) method to help, but make sure you aren't making any assumptions about the number/location of tab characters in a string).

Your function should not return anything, but should print out the results as number of lines found with tabs and the number of tab characters replaced in the new file. An example is given below, testing the program on the provided math_fns.py program which has 31 tab characters (if you opened that file in VSCode you may need to replace it with the one you downloaded again if your editor automatically replaces tabs for any opened file).

Hint: You should expect this function to take the most time in the assignment, so if you would find it helpful to jump to Part 3, you can. The Case Study El went over is very similar to this problem, and a video is posted for this review session which goes over strategies, pitfalls, and common questions for this similar problem. If you find yourself getting stuck on this problem, we encourage you to review the video/substitution.py code!

>>> tabs_to_spaces('math_fns.py', 4)
Lines with tabs: 29
Tabs replaced: 34
>>> tabs_to_spaces('two_tab_test.txt', 4)
Lines with tabs: 1
Tabs replaced: 2
>>> tabs_to_spaces('four_tab_test.txt', 4)
Lines with tabs: 2
Tabs replaced: 4

From the above code, you can see that the provided math_fns.py file has 34 '\t' characters spanning 29 lines, and the function call should result in a copy called spaced_math_fns.py with any '\t' characters replaced with spaces as described above.

You can also test your results with the following bash commands as a sanity check (you of course are not expected to know the magical world of grep, but it essentially counts all occurrences of a character in a file and then pipes the result to wc to count the number of lines that were changed :))

$ grep -o '\t' math_fns.py | wc -l
34
$ grep -o '\t' spaced_math_fns.py | wc -l
0
$ grep -o ' ' math_fns.py | wc -l
137
$ grep -o ' ' spaced_math_fns.py | wc -l
273
$ grep -o '\t' two_tab_test.txt | wc -l
2
$ grep -o ' ' two_tab_test.txt | wc -l
0
$ grep -o '\t' spaced_two_tab_test.txt | wc -l
0
$ grep -o ' ' spaced_two_tab_test.txt wc -l
8
$ grep -o '\t' four_tab_test.txt | wc -l
4
$ grep -o ' ' four_tab_test.txt | wc -l
3
$ grep -o '\t' spaced_four_tab_test.txt | wc -l
0
$ grep -o ' ' spaced_four_tab_test.txt wc -l
19

You can test changing space argument as well:

$ python3
>>> from fixer import tabs_to_spaces
>>> tabs_to_spaces('math_fns.py', 2) 
Lines with tabs: 29
Tabs replaced: 34
>>> quit()
$ grep -o ' ' spaced_math_fns.py | wc -l
205

Requirements: Your function should not loop through any line more than once, and do not use read() or readlines(). Your function should have exactly one (un-nested) loop to process the input file. You can use the writelines method to write a list of strings to a writable file object. For reference, our solution is 20 lines (not including comments or blank lines).

Hint: First, try to represent a space with '_' (underscore) when using replace since you won't be able to necessarily see the differences between tabs and spaces in the output file. Remember that the tab-space number parameter will determine how many space characters should replace a single '\t' tab character. Make sure to fix this to ' ' (space character) before your submission and try the above bash commands if you'd like to do a final check.

Write a function named to_snake_case that takes a string as an argument and returns a new string converting the argument to Python snake_case conventions (all lower-case, words separated with _ and numbers allowed, _ must be between characters not at end of strings). In particular, this function should handle strings following camelCasing (multiple words capitalized after first word) and PascalCasing (all words capitalized) such that any capital letter is replaced with _ followed by the lowercased version of the letter unless the capital letter is the first or last letter in the string, in which case it should just be lowercased.

Write a function named format_fn_header which returns a Python function header string using a given function name (which should be converted to snake_case) and a string list of arg names. This is a list of strings, not a dictionary but when this function is called, that list could be one generated with args.keys() (in that case, it's technically a dict_keys object but it works the same as a list in this case as long as you don't index it since dict_keys don't support index access (for a good reason)), where args is the dictionary passed to build_fn_str (we'll use this function in build_fn_str like so). You can assume the list of argument name strings are in snake_case already.

Note: While you see a use of keys() here for the purpose of an example, the only use of keys() should be in the build_fn_str function that will use format_fn_header when passing the argument names from its args dictionary as a list structure.

Do not use any dictionaries in the format_fn_header function.

These last few functions will be related to user input and dictionaries to complete a provided utility function (fn_generator) that generates Python function stubs (with docstrings!) for us (and yes, you can use that function on your assignments/this midterm!). This problem and generate_return_data are started for you, with a few TODOs to fill in to practice reading and understanding given code to collect data with tuples and dictionaries. The more tedious input-handling is provided for you, but we expect you to understand the logic to complete the TODOs.

generate_args_data takes no arguments but prompts a user for as many arguments as they'd like to specify. Your task in this exercise is to finish the 3 TODOs to finish the returning of a dict with each argument as a key mapping to a 2-string tuple containing (arg type, description).

Each argument will be assigned a desired name, type, and description (optional) using the prompts as follows:

The given argument name should be converted to snake_case as soon as it is entered (and this snake_cased version should be the name entered for the argument in the result dictionary). Remember that if a string is in snake_case already, your to_snake_case shouldn't return a different that differs from that passed string.

If no type is given in the second line, 'unspecified' should be assigned as argument's type string. If no description is given, '' should be assigned for that argument's description. A blank line should be printed after each "Description of ..." prompt. A user can specify as many arguments as they want, stopping when they press the <return> ("Enter") key on the prompt for an argument name.

If no arguments are added in the prompt, the function should return {}.

Next, you'll finish a function generate_return_data to get the type and description of a function's return. You only need to replace the single TODO to return a tuple as described, but we have provided a summary of this function here for reference.

This function takes no arguments, but prompts a user for a type and a description for a return value. If no type is given, the type is assigned the string 'unspecified' similar to generate_args_data (the description is also optional, '' if not provided). A blank line should be printed after each "Description of ..." prompt. The function should return a tuple containing the inputted type string and description (in this order).

We're almost there! The last function you will implement in this project is called save_program, which takes a program template string and saves to a new file. The three given functions in generator.py are implemented for you, as they factor out a lot of the tedious UI management to prompt the user for program information using the functions you've implemented so far. We've provided a brief summary of the function-generation functions here that you can test in your Python interpreter to make everything is ready to go at this point before you move on to the final save_program function.

These functions should not be changed, and are documented for you. They require that you have correctly implemented Exercises 3.1-3.4 at this point.

generate_fn takes a function name string as an argument (which is converted to snake casing using your to_snake_case function), and then prompts the user for arguments and any return for that function (using your generate_args_data and generate_return_data, respectively).

After collecting this data from the user, it generates the function stub using the generate_fn_stub function we've provided. Note that we could have merged both of these functions into one, but factoring out the function-stub-string-generation demonstrates good practice in program decomposition. If all of your functions above are correctly implemented, the following example runthroughs should produce the output shown below:

Finally, you will implement the final function to save your generated programs! Note that the final block of code in the provided generate_program function calls save_program with the generated program body string (everything is ready to go!).

Finish the save_program function in generator.py, which takes a single string argument representing the body of the program, with each line separated by '\n' (assuming all functions are working as expected at this point). This function should prompt the user for a file name to save to, reprompting until they give a non-empty file name.

Note: While we will not require you to handle this, it is not uncommon for new programmers to accidentally overwrite a file. To help prevent this without adding more tasks in this assignment, we have provided you a check_rewrite(filename) function which you may use to confirm with a user that they want to overwrite an existing file. To use it, update your break condition (which should at minimum be on a condition for a non-empty filename) to break only when passing the non-empty filename string to check_rewrite returns True (when the file doesn't yet exist or when the user has confirmed they want to overwrite it).

Once a valid filename is found, save the body to a new file (hint: use f.writelines) and print a success message in the same format as shown in the examples below. Note: We have added an example of confirmation if you choose to implement this feature.

For reference, our solution is 9 lines, most of which is the re-prompting logic for empty filenames.

>>> from generator import save_program
>>> # This is a test "body" string
>>> body = '"""TODO: File header"""\n\ndef foo():\n    pass\n\n'
>>> print(body)
"""TODO: File header"""

def foo():
    pass


>>> save_program(body)
What is the name of your file? foo_test.py
Successfully wrote to foo_test.py!
>>> save_program(body)
What is the name of your file?
Please provide a non-empty file name, e.g. test.py
What is the name of your file? test.py
Successfully wrote to test.py!
>>> # Optional handling of existing files
>>> save_program(body)
What is the name of your file? foo_test.py
A file is already found with that name. Are you sure you want to overwrite? n
What is the name of your file? foo_test.py
A file is already found with that name. Are you sure you want to overwrite? Y
Successfully wrote to foo_test.py!