ANALYSIS TOOL BOX

Data Collection

ExtractTextFromPDF

Extract text from a PDF file, clean it, and save to a text file.

data-collectionscrapingapiingestion

Introduction

Reports, filings, and studies rarely arrive as clean text — they arrive as PDFs, with line breaks, hyphenation, and whitespace that break downstream analysis before it starts. ExtractTextFromPDF answers "how do I turn this document into text I can actually search, summarize, or feed to a model?" by pulling text page by page and normalizing the layout artifacts (hyphenated line breaks, duplicate spaces, stray line breaks mid-sentence) that would otherwise corrupt word counts, embeddings, or keyword search. Reach for it as the first step of any workflow that needs clean text out of a PDF corpus, rather than hand-rolling extraction and cleanup each time.

Teaching Note

Text cleaning operations include:

  • Stripping leading and trailing whitespace
  • Removing duplicate spaces
  • Handling hyphenation at line breaks
  • Removing line breaks between continuous words

Parameters

ParameterTypeDefaultDescription
filepath_to_pdfrequired—Path to the PDF file to extract text from. Must be a string ending in '.pdf'.
filepath_for_exported_textrequired—Path where the extracted text will be saved. Must be a string ending in '.txt'.
start_page1Page number to start extracting text from (1-indexed).
end_pageNonePage number to stop extracting text from. If None, extracts text from all pages in the PDF file.
show_word_countFalseIf True, displays the word count of the extracted text.
show_estimated_token_countFalseIf True, displays an estimated token count based on word count.

Example

python
from analysistoolbox.data_collection import ExtractTextFromPDF

# Extract all text from a PDF and save to a text file
ExtractTextFromPDF(
    filepath_to_pdf='document.pdf',
    filepath_for_exported_text='output.txt',
    show_word_count=True
)