ANALYSIS TOOL BOX

Data Collection

GetGoogleSearchResults

Fetch Google search results programmatically using the Serper API.

data-collectionscrapingapiingestion

Introduction

Answering "what does Google actually say about this topic right now?" at any scale rules out manual searching and risks IP blocks or CAPTCHAs if you scrape the results page directly. GetGoogleSearchResults answers the question cleanly through the Serper API, returning structured JSON — titles, snippets, URLs, ranking position — instead of raw HTML to parse. Reach for it for SEO research, brand monitoring, or automated fact-checking, where you need reliable, repeatable search data rather than a one-off manual query.

Teaching Note

Unlike direct web scraping, using the Serper API provides:

  • Reliable, structured data without HTML parsing
  • No risk of IP blocking or CAPTCHA challenges
  • Consistent data format across queries
  • Access to Google's autocorrect and spell-check features
  • Support for large-scale search operations

Parameters

ParameterTypeDefaultDescription
queryrequiredThe search query string to submit to Google. Supports all standard Google search operators (e.g., quotes for exact match, site: for domain filtering).
serper_api_keyNoneAPI key for the Serper service. If not provided, the function attempts to load it from environment variables (typically in a .env file).
number_of_results10Number of search results to return. Maximum depends on API subscription tier.
apply_autocorrectFalseIf True, applies Google's autocorrect feature to fix misspelled queries. If False, searches for the exact query as provided.
display_resultsFalseIf True, displays the organic search results in a pandas DataFrame for immediate inspection in notebook environments.

Returns

A dict — the JSON response from the Serper API containing search results. Key fields include organic (list of search results), searchParameters (query metadata), and optionally relatedSearches, knowledgeGraph, etc.

Example

python
from analysistoolbox.data_collection import GetGoogleSearchResults
import os

# Search with autocorrect enabled and display results
api_key = os.getenv('SERPER_API_KEY')
results = GetGoogleSearchResults(
    query='artifical inteligence',
    serper_api_key=api_key,
    apply_autocorrect=True,
    display_results=True
)