EchoReel: On-Device Photo Memories
Private photo memory compilation with Apple Vision face detection and strict local-only storage.
WHERE IT STANDS
- PhotoKit local asset scanning with 0 network permissions requested or granted
- Apple Vision face detection pipeline generating on-device face bounding boxes
- Review Queue clustering requiring explicit human tap to link identities
- Local SQLite relationship graph persisting confirmed face clusters
- AVFoundation foreground video export with pan/zoom Ken Burns effects
- Automated face embedding candidate discarded after failing recall gate (0.585 measured vs 0.850 target)
- Background rendering constrained by iOS memory limits; requires foreground export
- Zero external or App Store adoption; tested solely on private library fixtures (1,420 assets)
01 · Problem & Privacy Goals
Commercial cloud photo apps typically upload your full photo library to remote servers to run facial recognition and assemble memory albums. This creates privacy concerns: personal biometric data is stored remotely, and incorrect face groupings are permanently saved without easy ways to review them.
EchoReel was built as a local-first iOS app. The goal was to group photos and generate smooth memory videos entirely on the phone, keeping memory usage low (under 0.25 MiB per photo) while guaranteeing that no photo data or face coordinates ever leave the device.
02 · System Constraints
The app sandbox does not request network capabilities. No telemetry, no third-party analytics, no remote API calls.
Scanning large libraries of 48MP photos must not cause iOS Jetsam out-of-memory kills. Memory usage is budgeted at ≤0.250 MiB per photo.
Face matching can suggest candidate groups, but cannot write confirmed relationships into SQLite without user approval.
03 · Architecture & Boundaries
Swift actors separate photo loading, background Apple Vision analysis, tentative candidate queues, and the user review UI.
On-Device Photo & Face Processing Pipeline
01. Input: PhotoKit loads photos directly from the local library into device memory with no network access.
02. Face Detection: Apple Vision finds face locations and extracts lightweight visual features locally.
03. Candidate Groups: Groups faces into tentative suggestions only. The system cannot confirm matches on its own.
04. Manual Review: The user confirms or rejects suggested face groups in a SwiftUI review screen.
05. Storage: SQLite saves confirmed identities and memory collections within a 0.215 MiB/photo memory budget.
06. Video Export: AVFoundation compiles and encodes photos into an animated video file locally.
04 · Three Architecture Decisions
1. Native Apple Vision instead of custom CoreML models
ZERO EXTRA DEPSContext: Tested bundling a custom CoreML face model against using iOS built-in Apple Vision framework.
Decision: Use Apple Vision. The operating system shares Neural Engine weights across apps, keeping our binary at 14MB instead of ~85MB while ensuring hardware acceleration.
Outcome: Minor API differences across iOS versions, managed through version-specific adapters and test fixtures.
2. Downsampled thumbnail pipeline with autoreleasepool drainage
MEMORY EFFICIENCYContext: Scanning full 48MP ProRAW photos in batches exhausted available RAM after ~30 photos, causing app termination.
Decision: Read photos with CGImageSourceCreateThumbnailAtIndex at a max size of 512px before full decompression, enclosed in explicit autoreleasepool blocks.
Outcome: Kept steady memory usage at 0.215 MiB per photo during tests of 1,000+ images, safely under the 0.250 MiB ceiling.
3. Review queue for ambiguous face matches
DATA INTEGRITYContext: Many apps auto-merge faces above an arbitrary similarity threshold, creating incorrect albums.
Decision: Any face match with intermediate similarity is routed to a review queue. The app never auto-merges ambiguous faces into confirmed database records.
Outcome: Zero false merges during testing; the user stays in control of their photo library.
05 · Hardest Technical Problem
Fixing memory buildup during video compilation
Root Cause: Generating video montages with pan/zoom Ken Burns effects using AVVideoCompositionCoreAnimationTool created intermediate CoreAnimation layers and CVPixelBuffers faster than memory was reclaimed.
Engineering Approach: Rewrote the export loop with a custom video export coordinator using a 2-frame lookahead queue and buffer recycling. Replaced CALayer animations with direct transforms inside a custom AVVideoCompositing implementation.
Verification: Instruments Allocations and Leaks runs showed zero persistent leaks across 20 consecutive 60-second 1080p60 video exports, keeping memory below 68 MB.
06 · Discarded Prototype: Embedding Model
Documenting negative results is standard engineering practice. When a prototype fails its target metrics, it is discarded rather than shipped anyway.
07 · Test Suite & Verification
Full unit and integration suite covering photo library scanning, face normalization, and SQLite queries.
Automated UI tests verifying onboarding, library scanning, and video export.
Checked against strict SwiftLint rules and Swift 5.9/6.0 concurrency checks.
All source files match local milestone archives.
08 · Retrospective & Learnings
Engineering Takeaway: Takeaway: On-device mobile engineering is mostly about memory budgeting and OS constraints, not complex model design. A model is useless if it runs out of memory during background indexing or locks the UI during video export. Swift actor boundaries and careful memory pooling mattered far more than model size.
What I Would Do Differently: What I Would Do Differently: Build a diverse offline test set of face angles and lighting conditions earlier in the project to benchmark Apple Vision edge cases before testing on a real device.