ADSB Flight Data
ADSB Flight Data Processing and GeoJSON Visualisation Pipeline
A comprehensive data pipeline that downloads, processes, and visualizes ADS-B (Automatic Dependent Surveillance-Broadcast) flight tracking data from June 2024. The system extracts flight traces from GitHub releases, processes them into a DuckDB database, and generates regional GeoJSON files for flight path visualisation.Libraries Needed
Core Libraries
- os - File system operations
- requests - HTTP requests for downloading data
- tarfile - Archive extraction
- gzip - Compressed file handling
- json - JSON data parsing
Database & Data Processing
- duckdb - High-performance analytical database
- pandas - Data manipulation and analysis
- numpy - Numerical computing
Utilities
- datetime / time - Date/time operations
- pathlib.Path - Modern path handling
- collections.defaultdict - Data structures
- multiprocessing.Pool - Parallel processing
- concurrent.futures - Asynchronous execution
- tempfile - Temporary file management
- shutil - High-level file operations
- urllib3.util.retry - HTTP retry strategies
- requests.adapters.HTTPAdapter - HTTP session configuration
Input Files
GitHub Release Archives
- v2024.06.01-planes-readsb-staging-0 through v2024.06.30-planes-readsb-staging-0
Compressed JSON Files
- .json.gz, .json files containing flight trace data
Split TAR Archives
- .tar.aa, .tar.ab, .tar.gz files
Flight Trace JSON Structure
- icao - Aircraft identifier
- registration - Aircraft registration
- trace - Array of [timestamp, latitude, longitude, altitude, heading, speed, vertical_rate]
- aircraft_type - Aircraft model
- description - Aircraft description
Output Files
DuckDB Database
- adsb_june.duckdb, adsb_dat_main.duckdb - Structured flight data storage
Temporary Parquet Files
Intermediate data storage for parallel processingRegional GeoJSON Files
- north_america_flights.geojson
- europe_flights.geojson
- asia_flights.geojson
- africa_flights.geojson
- australia_flights.geojson
- oceania_flights.geojson
- And other regional flight path visualizations
Flight Traces Table Schema
- Aircraft metadata (ICAO, registration, type)
- Positional data (latitude, longitude, altitude)
- Movement data (speed, heading, vertical rate)
- Temporal data (timestamps, sequences)
🚀 Ready to implement this solution?
Access the complete code, step-by-step instructions, and interactive notebook in Nika Hub.
View Full Solution →Get Expert Help
Talk to a Geospatial Expert
Need help with your geospatial projects? Our team of experts is here to assist you with implementation, best practices, and technical support.
Other ways to get help:
- Guides: Use the /guides tab for detailed tutorials
- Community: Ask questions in our community forum
- Support: Send us a support request