Study Notes formatter

This is my CS50P project for formatting study notes.

The problem

A lot of the times, when I'm studying, I find text on study note websites that I want to kind of manipulate to get it into a format that I can put into my flashcards app. This usually also involves removing a lot of the explanation text and only going for the headings. You may ask why I want to get it into this format? The reason is that I want to just memorise the key points on one flashcard, and leave the explanations for each point on their own different flashcards, combined with making each main point an li item in an html ol tag if your flashcard editor supports that. An example of the format conversion:

Before text:

Introduction
This section explains the basic concepts you need to understand before moving forward. It covers the foundational ideas and sets up the rest of the material.
Core Principles
There are several key principles that guide the entire process. Each one builds on the previous, so it helps to read them in order and take notes as you go.
Summary
In short, the ideas presented here form a cohesive framework that you can apply to many different situations.

After text:

Introduction
Core Principles
Summary

Description:

This program takes an input through a file, formats the input, and puts it into an output file. There are different options for the formatting depending on how your study notes are formatted, but all lead to the following format:

Taking the titles/headings/main points from the text and places on different lines separated by newline characters.

As mentioned above, the resulting format is good for putting on flashcards.

Formatting options

While talking about the formatting options in this file, a line is defined as a section of the text between two \n characters, or two \Z characters, or a \n and \Z character.

1. Keep short lines

This option keeps lines that are shorter than the 1st quantile given n number of quantiles.

For example, if n = 2, lines that are longer than the mean line length are deleted and the lines that are shorter than the mean length are kept. Line length is defined as the number of characters in a line (defined above)

When is this a good option?

When your text is irregularly formatted with main points and explanations, and if every main point has a different number of explanations, and the explanations are pretty much longer than the main points.

Example

Main point 1
Lorem ipsum dolor sit amet consectetur adipiscing elit quisque faucibus ex sapien vitae pellentesque sem.

Main point 2
Lorem ipsum dolor sit amet consectetur adipiscing elit quisque faucibus ex sapien vitae pellentesque sem placerat in id cursus mi pretium
tellus duis convallis tempus leo eu aenean sed diam urna tempor pulvinar vivamus fringilla lacus nec metus bibendum egestas iaculis
massa nisl malesuada lacinia integer nunc posuere ut hendrerit.

The above text will be formatted into:

Main point 1
Main point 2

Quantiles input for option 1

If you choose option 1 following the steps in the usage section of this file, you should be prompted for the number of quantiles, with the following being displayed to you:

How many quantiles would you like? (Try bigger values if the output isn't accurate, max. 100)
If you don't know what to enter here, 3 is a good option.
>

An integer between 1 and 100 is expected here. Simply type the integer and press enter when you're done.

Refer to the first line of the description of option 1 in this file to see how the number of quantiles affects the result.

• If you're not sure what to put here, 3 quantiles is a good option.
• If the result is inaccurate, you are advised to try a smaller number of quantiles.
• May also be innaccurate if the number of explanation lines (longer lines) is similar to the number of heading lines. Consider using option 2 in that case if appropriate.

Limitations of option 1

• May not be accurate if Main points are similar in length to explanations. In this case, Option 1 may keep unwanted lines and/or delete wanted lines.

2. Remove every other line

This option keeps lines that are evenly indexed and removes lines are are oddly indexed.

A line's index is defined as the line's index in an array if the entire input file is a string and this string was split by \n characters.

When is this a good option?

When each main point is followed by exactly one explanation.

Example

Take this input text, for example:

This is the zeroeth line, and is even numbered. This line will be kept if this example is formatting through option 2.
This is the first line, and is odd numbered. This line won't be kept if this example is formatting through option 2.
This is the second line, and is even numbered. This line will be kept
This is the third line, and is odd numbered. This line won't be kept

This will give an ouput of the following when formatted with option 2:

This is the zeroeth line, and is even numbered. This line will be kept if this example is formatting through option 2.
This is the second line, and is even numbered. This line will be kept

3. Format numbered headings

This option formats text that contains numbered headings in the format

{digit}. {heading}\n

And keeps the {heading}.

Example

Take this input text, for example:

1. Introduction
This section explains the basic concepts you need to understand before moving forward. It covers the foundational ideas and sets up the rest of the material.
2. Core Principles
There are several key principles that guide the entire process. Each one builds on the previous, so it helps to read them in order and take notes as you go.
3. Summary
In short, the ideas presented here form a cohesive framework that you can apply to many different situations.

This will give an ouput of the following when formatted with option 3:

Introduction
Core Principles
Summary

Limitations of option 3

• Doesn't support Numbered headings that use dashes instead of periods, eg.

1 - Heading 1
2 - Heading 2
3 - Heading 3

• Doesn't keep the numbers before the headings nor the periods, only the heading text itself.

4. Remove duplicate capital letters

Removes duplicate consecutive capital letters from input text. May seem like a weird option but sometimes notes are formatted like this in study notes online.

Example

Take this input text, for example:

IIntroduction
CCore Principles
SSummary

This will give an ouput of the following when formatted with option 3:

Introduction
Core Principles
Summary

Limitations of option 4

• Doesn't turn triplets of letters into 1 letter; turns triplets of letters into 2 letters. Perhaps run the program 2 times if you have triplets of letters.

Usage

How to run:

1. python project.py input-filename output-filename In terminal

This reads the input from input-filename and overwrites output-filename with the output.

2. python project.py input-filename output-filename In terminal

This reads the input from input-filename and overwrites the same input-filename with the output. Think of it as... modifying the file in place?

3. This project contains a Makefile. Type make run into your terminal to execute python project.py data/input.txt data/output.txt

Pick an option

The program will output the following to you:

Welcome to my study notes formatter!
Here are the formatting options:
1. Keep short lines (A good option to keep headings but not long notes)
2. Remove every other line (Good if every main point/heading has one line of explanation under it.)
3. Format numbered headings
4. Remove duplicate capital letters
Please choose an option by entering the corresponding number!
>

Enter the number of the option you would like to use. eg. enter '1' for 'Keep short lines' option.

Result

The program should then output the following message:

Succesfully processed filename!

You should then check your output file for the result.

FileNotFoundError

In the case of a FileNotFoundError while trying to read or write to your file/s, the program will output Couldn't find filename.

Program design, code, and limitations

Code features:

• This program is implemented in Python and I generally included type hints.

1. Option selection

   • Each formatting option has a function that takes an argument contents: str and returns the processed string. This is shown in the formatting_options list that maps each option to its handling function.
   • The options are dynamically presented to the user by looping through a list of all the option descriptions, which is generated by taking the value of the "option" key from each dictionary in the formatting_options list
     • Then displaying the index + 1 of each option's dict, so that the options start at 1 rather than 0.
     • The user inputs a number, 1 is subtracted from this number and the value of the function key of the dictionary at this index of the formatting_options list is passed into the process_file_with function.

2. Processing

   • The process_file_with function depends on a function given, and this function is one of the option handler functions.
   • The process_file_with function reads the input file, passes the contents into the function provided to be processed.
   • The dependency function then returns the processed contents, and the process_file_with function write this into the output file.
   • A file not found error is caught and a message is displayed to the user if the file name isn't found.

3. Dependency functions

   • The dependency functions rely on splitting by \n, list logic and sometimes regular expressions.
   • The dependency functions are the following:
     • keep_short_lines
     • every_other_line
     • remove_numbered_headings
     • remove_duplicate_capital_letters

Testing

• A test script is also included, made with pytest
• To run, execute pytest test_project.py in terminal.

Current Limitations:

• No support for passing in a flag for the option in the running command for the program.
• No GUI
• The output file is always overwritten, and not appended to
• Limited formatting options
• Comments in the code may not be thorough
• Code isn't organised into multiple files/modules which may have been more appropriate.
• Testing can be more extensive