Amanullah Naseer
Embedded AI EngineerAutonomous UAVs • C# (.NET) / PyQt6 • Edge Vision
Autonomous UAVs • C# (.NET) / PyQt6 • Edge Vision
I build end-to-end intelligent flight and perception systems operating under rigorous physical constraints. My work bridges PX4/ArduPilot autopilot integration, NVIDIA Isaac Sim digital twins, custom RF-DETR aerial detection models trained and validated on Roboflow, and high-performance C# (.NET) & PyQt6 ground control stations consuming 50+ Hz telemetry feeds without event loop stutter.
// Real-time MAVLink State Decoupling
[PUB/SUB] ZMQ Sub port 5556 active (55.4 Hz)
[ESTIMATOR] State Vector Est: X=12.44m Y=-4.18m Z=28.50m
[VISION] Subject: Vehicle
[SERVO] Offboard Yaw Rate Cmd: +0.14 rad/s ($dx/dy$ Align)
[UI-AGG] C# TPL / PyQt6 Ring Buffer: 20.0 FPS (0 dropped msgs)
Actual recordings validating closed-loop Software-in-the-Loop (SITL) flight control and aerial vehicle detection pipelines.
One-click launcher that starts NVIDIA Isaac Sim with a PX4-controlled drone (Pegasus Simulator) in the Baylands park with a car and a pedestrian, opens QGroundControl for take-off and joystick flight, and shows the drone's 2-axis gimbal camera with live YOLOv8s (VisDrone) detection. Recording: Holybro X500 take-off from QGC, gimbal steered from the app in flight, car and pedestrian detected, return to launch.
Split-screen architecture: Left window executes the Autonomous Aerial Tracking & Visual Servoing GCS with real-time pixel offset calculation ($dx/dy$) and MAVLink manual override. Right window runs NVIDIA Isaac Sim physics to validate autonomous following and visual navigation prior to physical field hardware tests.
RF-DETR Nano, fine-tuned on Roboflow (PyTorch-based) for 50 epochs on a custom-annotated aerial dataset (1,781 images, 384x384). The model detects ground vehicles and their components (Class 0-4); achieved 89.4% mAP@50, 86.8% precision, and 89.2% recall. Results are on the validation set only.
Does simulator-rendered data improve a drone-view detector on real images? A Replicator generator renders cars and people in the Baylands park with randomized altitude (8-120 m), camera angle, sun, paint and ground; labels come from the renderer. Four YOLOv8s variants were scored on the 548 real VisDrone2019 validation images.
Instead of treating objects as single bounding boxes, the custom RF-DETR Nano model detects granular vehicle components to enable precise aerial vehicle detection.
● Class 0: 100% AP | ● Class 1: 69% AP | ● Class 2: 97% AP | ● Class 3: 87% AP | ● Class 4: 94% AP
Outputs normalized JSON telemetry stream for companion computer pipelines.
Smooth convergence across Box Location Loss, Class Loss, and Overlap Loss.
Custom PyTorch-based model trained via Roboflow, utilizing data augmentations (scaling, rotation, brightness) for outdoor robustness.
Each project repository includes technical documentation, architecture diagrams, and implementation code.
Desktop launcher for NVIDIA Isaac Sim + Pegasus + PX4 SITL + QGroundControl with a selectable drone (Holybro X500 built from PX4's model, 5-inch FPV racer, Pegasus quad, Iris), a gimbal camera steerable in flight, and live YOLOv8s detection of people and vehicles in the Baylands park.
Software-in-the-loop simulation platform coupling NVIDIA Isaac Sim with PX4 firmware and offboard ROS 2 Humble nodes to validate autonomous visual tracking before hardware tests. Bidirectional MAVLink bridge streaming 50 Hz IMU and state telemetry.
RF-DETR Nano, fine-tuned on Roboflow (PyTorch-based) for 50 epochs on a custom-annotated aerial dataset (1,781 images, 384x384). Detects ground vehicles and their components (Class 0-4). Achieved 89.4% mAP@50, 86.8% precision, and 89.2% recall on the validation set only.
Pilot-in-the-loop follow system running on Raspberry Pi 5 with a Google Coral Edge TPU (SSD-MobileNetV2, 24-30 FPS) and an IMX415 camera. The pilot selects the subject with an RC switch; Kalman-filtered detections drive yaw and forward-velocity commands over MAVLink (Guided mode) to hold a 5 m standoff, returning to hover if the subject is lost. Live video goes to FPV goggles via HDMI and an analogue converter. Flight-tested on a real drone.
Isaac Sim Replicator generator that renders labelled drone-view images of cars and people (visibility-aware placement via PhysX ray casts, randomized camera, lighting and paint), and a controlled experiment testing whether the data helps a YOLOv8s detector on real VisDrone images. Reported as measured, including the null result.
Multi-threaded PyQt6 ground control station for MAVLink drones. Separate threads handle the drone link (serial/TCP/UDP), the RTSP video stream and a Leaflet offline-tile map, keeping the UI responsive. Runs a pretrained YOLOv8n (Ultralytics) in a worker thread for person detection on the live feed. Waypoint missions are created on the offline map and were flight-tested on a real drone in Guided mode with set altitude. Controls for altitude change and flight modes (RTL, Land, Guided). Live telemetry: position, flight time, battery, speed, altitude, roll/pitch/yaw, bearing.
Spatial mapping pipeline toward GPS-degraded UAV navigation. Integrated an RPLIDAR S2 with ROS 2 Humble on Ubuntu 22.04, generating occupancy grids in Gazebo and RViz for waypoint navigation without GPS. Tested on the bench, in simulation, and on a small ground robot.
How hardware-software decoupling solves event loop starvation and latency jitter in deployed systems.
When raw MAVLink telemetry packets are emitted at 50–60 Hz across multiple UAV channels, streaming them directly into the GUI thread causes the desktop event loop (WPF/Avalonia or PyQt6) to choke on repetitive paint calls, producing visual lag and delaying critical operator inputs.
[ UAV MAVLink Stream (50-60 Hz) ]
│
▼
┌──────────────────────────────────────────────┐
│ Network Worker (C# Channel / Python QThread)│
│ - Non-blocking ZMQ socket poll & CRC check │
└──────────────────────┬───────────────────────┘
│
┌───────────────┴───────────────┐
▼ ▼
[ Telemetry Ring Buffer ] [ High-Priority Bypass ]
(Atomic State @ 50 Hz) (Emergency / Safety Alarms)
│ │
▼ (Aggregated @ 20 Hz) ▼ (Instant Event/Signal)
┌──────────────────────────────────────────────┐
│ Desktop UI (WPF/.NET or PyQt6) │
│ - 20 Hz Instrument & Map Paint (Consistent) │
│ - Instant Emergency Motor Cutoff Handler │
└──────────────────────────────────────────────┘
Standard OpenCV image decoding copies pixels through the CPU, introducing 15-25ms latency per frame before inference even starts. On companion computers like Jetson Orin, this exhausts thermal budgets.
[ Camera Stream ] ────(MIPI-CSI Raw Data)──► [ Hardware ISP ]
│
▼
[(Shared RAM Buffer)]
│
(Direct Zero-Copy Pointer)
▼
[ NPU Accelerator ]
│
(BBox X, Y Coordinates)
▼
[ Pixhawk FCU ] ◄──(MAVLink UART)── [ CPU: PID Math & Vector ]
Physical drone flight tests for autonomous visual tracking carry severe hardware crash risks and unpredictable wind/lighting variables. Developing algorithms directly on hardware slows iteration cycles.
┌─────────────────────────────────────────────────────────────┐
│ NVIDIA Isaac Sim 4.0.1 (RTX Physics & Camera Simulation) │
└──────────────┬───────────────────────────────▲──────────────┘
│ (Synthetic Image) │ (Velocity Cmd)
▼ │
┌──────────────────────────────┐ ┌─────────────┴──────────────┐
│ Perception Node (ROS/YOLO) │ │ PX4 SITL Flight Controller │
└──────────────┬───────────────┘ └─────────────▲──────────────┘
│ (Detection BBox) │ (MAVLink Offboard)
▼ │
┌──────────────────────────────────────────────┴──────────────┐
│ Visual Servoing Node (PID) │
└─────────────────────────────────────────────────────────────┘
Technologies, protocols, and embedded platforms utilized across commercial UAV and vision projects.