AffordableHousingScraper

A Python tool for scraping rental listing data from AffordableHousing.com using the ScrapingAnt web scraping API. Built for educational purposes to demonstrate how to work with JSON-based search APIs, proxy-based web scraping, and affordable housing data extraction.

Disclaimer: This project is intended for educational and research purposes only. Please review AffordableHousing.com's Terms of Service before use. The authors are not responsible for any misuse of this tool.

How It Works

1. Sends POST requests with JSON body to AffordableHousing's AJAX search endpoint (/v4/AjaxHandler?message=SearchListings)
2. Routes requests through ScrapingAnt proxy (browser=false, proxy_country=US) for reliable access
3. Parses the JSON response from the listings array — configurable items per page (default 20)
4. Handles page-based pagination via the page field in the JSON request body
5. Deduplicates listings by community_id and exports to CSV/JSON

Setup

pip install -r requirements.txt

Sign up for a ScrapingAnt API key to get started.

Usage

python main.py \
  --center "-87.6437,41.9306" \
  --bounds 41.523632720169296 42.3375672798307 -88.04794002874596 -87.23945997125404 \
  --max-pages 3 \
  --api-key "YOUR_SCRAPINGANT_API_KEY" \
  --verbose

Required Arguments

Argument	Description
--center	Search center as "longitude,latitude"
--bounds MIN_LAT MAX_LAT MIN_LON MAX_LON	Geographic bounding box (4 values)

Optional Arguments

Argument	Default	Description
--page	1	Starting page number
--max-pages	0 (all)	Max pages to fetch
--items-per-page	20	Results per page
--radius	45	Search radius in miles
--sort	LastUpdate Desc	Sort expression
--property-types	all	Property type filter (e.g. Apartment, House)
--output / -o	output/affordable_housing_rentals.csv	Output CSV path
--json	false	Also export as JSON
--api-key	env SCRAPINGANT_API_KEY	ScrapingAnt API key
--verbose / -v	false	Enable verbose logging

Output Fields

Field	Description
community_id	AffordableHousing property identifier
title	Listing title
property_type	Apartment, House, Townhouse, etc.
address	Full formatted address
city, county, state, zip_code	Location components
rent_min, rent_max	Rent price range (USD/month)
bedrooms_min, bedrooms_max	Bedroom count range
bathrooms_min, bathrooms_max	Bathroom count (full + 0.5*half)
sqft_min, sqft_max	Living area range
year_built	Year of construction
unit_count	Number of available units
lat, lon	Geographic coordinates
url	Full AffordableHousing listing URL
photo_url	Primary listing photo URL
description	Listing description (truncated to 500 chars)
availability	Availability status text
section8	Accepts Section 8 vouchers
income_restricted	Income-restricted property
pets_allowed	Allows pets
premium_listing	Premium/featured listing
date_scraped	Timestamp of scrape

Example

Scrape affordable rental listings in the Chicago, IL area:

python main.py \
  --center "-87.6437,41.9306" \
  --bounds 41.523632720169296 42.3375672798307 -88.04794002874596 -87.23945997125404 \
  --radius 45 \
  --items-per-page 20 \
  --max-pages 5 \
  --json \
  --api-key "YOUR_KEY" \
  --verbose

Learning Objectives

This project demonstrates several key concepts for educational purposes:

• JSON POST API Interaction: How to send structured JSON request bodies to search APIs through a proxy service
• Proxy-Based Scraping: Using ScrapingAnt to route POST requests with JSON payloads for reliable data access
• Geospatial Search: Working with center-point and bounding-box geographic search parameters
• Affordable Housing Data: Understanding Section 8, income-restricted, and subsidized housing listing structures
• Data Normalization: Transforming complex nested API responses (including SpotlightData amenities) into flat, structured datasets
• Pagination via Request Body: Handling page-based pagination where the page number is part of the JSON payload rather than URL parameters