01 etl demo
In [1]:
Copied!
# Uncomment the line below to install the package if running in Google Colab
# %pip install geocrime-stl -q
# Uncomment the line below to install the package if running in Google Colab
# %pip install geocrime-stl -q
In [2]:
Copied!
import os
import geocrime_stl as gc
import os
import geocrime_stl as gc
Run the ETL pipeline (using September 2026 data as a test case)¶
In [3]:
Copied!
data_pkg = gc.run_pipeline(9, 2026)
data_pkg = gc.run_pipeline(9, 2026)
Verify the raw CSV came through the ETL pipeline and was delivered as a CrimeDataPackage¶
In [4]:
Copied!
print(f"Package Type: {type(data_pkg)}")
print(f"Package Tags: Month={data_pkg.month}, Year={data_pkg.year}")
print(f"Data Shape: {data_pkg.df.shape}")
print(f"Package Type: {type(data_pkg)}")
print(f"Package Tags: Month={data_pkg.month}, Year={data_pkg.year}")
print(f"Data Shape: {data_pkg.df.shape}")
Package Type: <class 'geocrime_stl.etl.transform.CrimeDataPackage'> Package Tags: Month=9, Year=2026 Data Shape: (3247, 15)
Output Summary Statistics of the Monthly Data¶
In [5]:
Copied!
gc.generate_monthly_metrics(data_pkg)
gc.generate_monthly_metrics(data_pkg)
================================================== 📊 ST. LOUIS CRIME DATA SUMMARY REPORT 📅 September 2026 ================================================== 🔹 Total Incident Count: 3247 🚨 TOP 5 CRIME CATEGORIES: - Destruction/Damage/Vandalism of Property: 401 incidents - Weapons Law Violations: 300 incidents - Simple Assault: 240 incidents - Theft From Motor Vehicle: 237 incidents - Motor Vehicle Theft: 215 incidents ⭐ INCIDENTS BY POLICE DISTRICT: - District 1: 528 incidents - District 2: 418 incidents - District 3: 498 incidents - District 4: 797 incidents - District 5: 501 incidents - District 6: 505 incidents 🔫 FIREARM INVOLVEMENT: - Firearm Involved: 428 (13.2%) 🏘️ NEIGHBORHOOD SUMMARY: Offense Type: Person ------------------------------ * Downtown: 40 incidents * Dutchtown: 35 incidents * Downtown West: 29 incidents Offense Type: Property ------------------------------ * Downtown: 98 incidents * Downtown West: 94 incidents * Dutchtown: 83 incidents Offense Type: Society ------------------------------ * Downtown: 62 incidents * Dutchtown: 51 incidents * Downtown West: 39 incidents Offense Type: Other ------------------------------ * Downtown: 34 incidents * Dutchtown: 30 incidents * Baden: 22 incidents ==================================================
Output Crime Distribution Maps of the Monthly Data¶
In [6]:
Copied!
gc.plot_monthly_maps(data_pkg)
gc.plot_monthly_maps(data_pkg)
findfont: Failed to find font weight semibold for DejaVu Sans, now using 700.
Run the exporters found in utils/export.py¶
Create a dedicated test directory to hold output files¶
In [7]:
Copied!
TEST_DIR = "./test_exports"
os.makedirs(TEST_DIR, exist_ok=True)
print(f"✅ Ready to test. Output directory created at: {TEST_DIR}")
TEST_DIR = "./test_exports"
os.makedirs(TEST_DIR, exist_ok=True)
print(f"✅ Ready to test. Output directory created at: {TEST_DIR}")
✅ Ready to test. Output directory created at: ./test_exports
Export the dataframe to a CSV, a GeoJSON and a GPKG (QGIS)¶
In [8]:
Copied!
print("🚀 Executing file export tests...")
# Test Flat CSV Export
csv_success = gc.export_to_csv(data_pkg, output_dir=TEST_DIR)
print(f" - CSV Export: {'SUCCESS 💾' if csv_success else 'FAILED ❌'}")
# Test GeoJSON Export (Triggers on-the-fly spatial reconstruction)
geojson_success = gc.export_to_geojson(data_pkg, output_dir=TEST_DIR)
print(f" - GeoJSON Export: {'SUCCESS 🌍' if geojson_success else 'FAILED ❌'}")
# Test GeoPackage Export (Triggers spatial reconstruction + SQLite Layer storage)
gpkg_success = gc.export_to_gpkg(data_pkg, output_dir=TEST_DIR)
print(f" - GeoPackage Export:{'SUCCESS 📦' if gpkg_success else 'FAILED ❌'}")
print("🚀 Executing file export tests...")
# Test Flat CSV Export
csv_success = gc.export_to_csv(data_pkg, output_dir=TEST_DIR)
print(f" - CSV Export: {'SUCCESS 💾' if csv_success else 'FAILED ❌'}")
# Test GeoJSON Export (Triggers on-the-fly spatial reconstruction)
geojson_success = gc.export_to_geojson(data_pkg, output_dir=TEST_DIR)
print(f" - GeoJSON Export: {'SUCCESS 🌍' if geojson_success else 'FAILED ❌'}")
# Test GeoPackage Export (Triggers spatial reconstruction + SQLite Layer storage)
gpkg_success = gc.export_to_gpkg(data_pkg, output_dir=TEST_DIR)
print(f" - GeoPackage Export:{'SUCCESS 📦' if gpkg_success else 'FAILED ❌'}")
🚀 Executing file export tests... - CSV Export: SUCCESS 💾 - GeoJSON Export: SUCCESS 🌍 - GeoPackage Export:SUCCESS 📦