Publications

2025

1. VG-SSL: Benchmarking Self-Supervised Representation Learning Approaches for Visual Geo-Localization

   Jiuhong Xiao, Gao Zhu, and Giuseppe Loianno

   2025

   Abs

   Visual Geo-localization (VG) is a critical research area for identifying geo-locations from visual inputs, particularly in autonomous navigation for robotics and vehicles. Current VG methods often learn feature extractors from geo-labeled images to create dense, geographically relevant representations. Recent advances in Self-Supervised Learning (SSL) have demonstrated its capability to achieve performance on par with supervised techniques with unlabeled images. This study presents a novel VG-SSL framework, designed for versatile integration and benchmarking of diverse SSL methods for representation learning in VG, featuring a unique geo-related pair strategy, GeoPair. Through extensive performance analysis, we adapt SSL techniques to improve VG on datasets from hand-held and car-mounted cameras used in robotics and autonomous vehicles. Our results show that contrastive learning and information …

2. Smooth Games of Configuration in the Linear-Quadratic Setting

   Jesse Milzman, Jeffrey Mao, and Giuseppe Loianno

   arXiv preprint arXiv:2507.16611, 2025

   Abs

   Dynamic game theory offers a toolbox for formalizing and solving for both cooperative and non-cooperative strategies in multi-agent scenarios. However, the optimal configuration of such games remains largely unexplored. While there is existing literature on the parametrization of dynamic games, little research examines this parametrization from a strategic perspective where each agent’s configuration choice is influenced by the decisions of others. In this work, we introduce the concept of a game of configuration, providing a framework for the strategic fine-tuning of differential games. We define a game of configuration as a two-stage game within the setting of finite-horizon, affine-quadratic, AQ, differential games. In the first stage, each player chooses their corresponding configuration parameter, which will impact their dynamics and costs in the second stage. We provide the subgame perfect solution concept and a method for computing first stage cost gradients over the configuration space. This then allows us to formulate a gradient-based method for searching for local solutions to the configuration game, as well as provide necessary conditions for equilibrium configurations over their downstream (second stage) trajectories. We conclude by demonstrating the effectiveness of our approach in example AQ systems, both zero-sum and general-sum.

3. Query-Based Adaptive Aggregation for Multi-Dataset Joint Training Toward Universal Visual Place Recognition

   Jiuhong Xiao, Yang Zhou, and Giuseppe Loianno

   arXiv preprint arXiv:2507.03831, 2025

   Abs

   Deep learning methods for Visual Place Recognition (VPR) have advanced significantly, largely driven by large-scale datasets. However, most existing approaches are trained on a single dataset, which can introduce dataset-specific inductive biases and limit model generalization. While multi-dataset joint training offers a promising solution for developing universal VPR models, divergences among training datasets can saturate limited information capacity in feature aggregation layers, leading to suboptimal performance. To address these challenges, we propose Query-based Adaptive Aggregation (QAA), a novel feature aggregation technique that leverages learned queries as reference codebooks to effectively enhance information capacity without significant computational or parameter complexity. We show that computing the Cross-query Similarity (CS) between query-level image features and reference codebooks provides a simple yet effective way to generate robust descriptors. Our results demonstrate that QAA outperforms state-of-the-art models, achieving balanced generalization across diverse datasets while maintaining peak performance comparable to dataset-specific models. Ablation studies further explore QAA’s mechanisms and scalability. Visualizations reveal that the learned queries exhibit diverse attention patterns across datasets. Code will be publicly released.

4. NOVA: Navigation via Object-Centric Visual Autonomy for High-Speed Target Tracking in Unstructured GPS-Denied Environments

   Alessandro Saviolo and Giuseppe Loianno

   arXiv preprint arXiv:2506.18689, 2025

   Abs

   Autonomous aerial target tracking in unstructured and GPS-denied environments remains a fundamental challenge in robotics. Many existing methods rely on motion capture systems, pre-mapped scenes, or feature-based localization to ensure safety and control, limiting their deployment in real-world conditions. We introduce NOVA, a fully onboard, object-centric framework that enables robust target tracking and collision-aware navigation using only a stereo camera and an IMU. Rather than constructing a global map or relying on absolute localization, NOVA formulates perception, estimation, and control entirely in the target’s reference frame. A tightly integrated stack combines a lightweight object detector with stereo depth completion, followed by histogram-based filtering to infer robust target distances under occlusion and noise. These measurements feed a visual-inertial state estimator that recovers the full 6-DoF pose of the robot relative to the target. A nonlinear model predictive controller (NMPC) plans dynamically feasible trajectories in the target frame. To ensure safety, high-order control barrier functions are constructed online from a compact set of high-risk collision points extracted from depth, enabling real-time obstacle avoidance without maps or dense representations. We validate NOVA across challenging real-world scenarios, including urban mazes, forest trails, and repeated transitions through buildings with intermittent GPS loss and severe lighting changes that disrupt feature-based localization. Each experiment is repeated multiple times under similar conditions to assess resilience, showing consistent and reliable performance …

5. Time-Optimized Safe Navigation in Unstructured Environments through Learning Based Depth Completion

   Jeffrey Mao, Raghuram Cauligi Srinivas, Steven Nogar, and 1 more author

   arXiv preprint arXiv:2506.14975, 2025

   Abs

   Quadrotors hold significant promise for several applications such as agriculture, search and rescue, and infrastructure inspection. Achieving autonomous operation requires systems to navigate safely through complex and unfamiliar environments. This level of autonomy is particularly challenging due to the complexity of such environments and the need for real-time decision making especially for platforms constrained by size, weight, and power (SWaP), which limits flight time and precludes the use of bulky sensors like Light Detection and Ranging (LiDAR) for mapping. Furthermore, computing globally optimal, collision-free paths and translating them into time-optimized, safe trajectories in real time adds significant computational complexity. To address these challenges, we present a fully onboard, real-time navigation system that relies solely on lightweight onboard sensors. Our system constructs a dense 3D map of the environment using a novel visual depth estimation approach that fuses stereo and monocular learning-based depth, yielding longer-range, denser, and less noisy depth maps than conventional stereo methods. Building on this map, we introduce a novel planning and trajectory generation framework capable of rapidly computing time-optimal global trajectories. As the map is incrementally updated with new depth information, our system continuously refines the trajectory to maintain safety and optimality. Both our planner and trajectory generator outperforms state-of-the-art methods in terms of computational efficiency and guarantee obstacle-free trajectories. We validate our system through robust autonomous flight experiments in …

6. Collision Detection and Reaction for Quadrotors Using Encoder-Integrated Tombo Propellers

   Hung Tien Pham, Quang Ngoc Pham, Quan Khanh Luu, and 2 more authors

   IEEE Access, 2025

   Abs

   This study presents a novel collision detection and reaction strategy for quadrotors equipped with deformable propellers (Tombo propellers). By integrating rotary encoders, we achieve real-time monitoring of individual propeller speeds, enabling rapid and reliable collision detection based on sudden velocity drops. The propeller speed variation during collisions is analyzed through simulations of the BLDC motor-Tombo propeller collision model. In addition, experimental validation is performed on a quadrotor to reinforce the correlation between propeller velocity changes and collisions. A collision detection algorithm based on this phenomenon successfully identifies eight out of twelve collisions during hover flights, while the remaining four minor collisions, which were undetected, had negligible effects on quadrotor stability. Subsequently, a collision response strategy that takes advantage of this algorithm for a …

7. Intuitive Human-Drone Collaborative Navigation in Unknown Environments Through Mixed Reality

   Sanket A Salunkhe, Pranav Nedunghat, Luca Morando, and 3 more authors

   2025

   Abs

   The widespread use of aerial robots in inspection, search and rescue, and monitoring has created a growing need for intuitive human-drone interfaces. These aim to streamline and enhance the user interaction and collaboration process during drone navigation, ultimately expediting mission success and accommodating users’ inputs. In this paper, we present a novel human-drone mixed reality interface that aims to (a) increase human-drone spatial awareness by sharing relevant spatial information and representations between the human equipped with a Head Mounted Display (HMD) and the robot and (b) enable safer and intuitive human-drone interactive and collaborative navigation in unknown environments beyond the simple command and control or teleoperation paradigm. Our framework is validated through extensive user studies and experiments conducted in simulated post-disaster scenarios, with …

8. ES-HPC-MPC: Exponentially Stable Hybrid Perception Constrained MPC for Quadrotor with Suspended Payloads

   Luis F Recalde, Mrunal Sarvaiya, Giuseppe Loianno, and 1 more author

   arXiv preprint arXiv:2504.08841, 2025

   Abs

   Aerial transportation using quadrotors with cable-suspended payloads holds great potential for applications in disaster response, logistics, and infrastructure maintenance. However, their hybrid and underactuated dynamics pose significant control and perception challenges. Traditional approaches often assume a taut cable condition, limiting their effectiveness in real-world applications where slack-to-taut transitions occur due to disturbances. We introduce ES-HPC-MPC, a model predictive control framework that enforces exponential stability and perception-constrained control under hybrid dynamics. Our method leverages Exponentially Stabilizing Control Lyapunov Functions (ES-CLFs) to enforce stability during the tasks and Control Barrier Functions (CBFs) to maintain the payload within the onboard camera’s field of view (FoV). We validate our method through both simulation and real-world experiments, demonstrating stable trajectory tracking and reliable payload perception. We validate that our method maintains stability and satisfies perception constraints while tracking dynamically infeasible trajectories and when the system is subjected to hybrid mode transitions caused by unexpected disturbances.

9. Optimal Trajectory Planning for Cooperative Manipulation with Multiple Quadrotors Using Control Barrier Functions

   Arpan Pallar, Guanrui Li, Mrunal Sarvaiya, and 1 more author

   arXiv preprint arXiv:2503.01096, 2025

   Abs

   In this paper, we present a novel trajectory planning algorithm for cooperative manipulation with multiple quadrotors using control barrier functions (CBFs). Our approach addresses the complex dynamics of a system in which a team of quadrotors transports and manipulates a cable-suspended rigid-body payload in environments cluttered with obstacles. The proposed algorithm ensures obstacle avoidance for the entire system, including the quadrotors, cables, and the payload in all six degrees of freedom (DoF). We introduce the use of CBFs to enable safe and smooth maneuvers, effectively navigating through cluttered environments while accommodating the system’s nonlinear dynamics. To simplify complex constraints, the system components are modeled as convex polytopes, and the Duality theorem is employed to reduce the computational complexity of the optimization problem. We validate the performance of our planning approach both in simulation and real-world environments using multiple quadrotors. The results demonstrate the effectiveness of the proposed approach in achieving obstacle avoidance and safe trajectory generation for cooperative transportation tasks.

10. UASTHN: Uncertainty-Aware Deep Homography Estimation for UAV Satellite-Thermal Geo-localization

    Jiuhong Xiao and Giuseppe Loianno

    arXiv preprint arXiv:2502.01035, 2025

    Abs

    Geo-localization is an essential component of Unmanned Aerial Vehicle (UAV) navigation systems to ensure precise absolute self-localization in outdoor environments. To address the challenges of GPS signal interruptions or low illumination, Thermal Geo-localization (TG) employs aerial thermal imagery to align with reference satellite maps to accurately determine the UAV’s location. However, existing TG methods lack uncertainty measurement in their outputs, compromising system robustness in the presence of textureless or corrupted thermal images, self-similar or outdated satellite maps, geometric noises, or thermal images exceeding satellite maps. To overcome these limitations, this paper presents UASTHN, a novel approach for Uncertainty Estimation (UE) in Deep Homography Estimation (DHE) tasks for TG applications. Specifically, we introduce a novel Crop-based Test-Time Augmentation (CropTTA) strategy, which leverages the homography consensus of cropped image views to effectively measure data uncertainty. This approach is complemented by Deep Ensembles (DE) employed for model uncertainty, offering comparable performance with improved efficiency and seamless integration with any DHE model. Extensive experiments across multiple DHE models demonstrate the effectiveness and efficiency of CropTTA in TG applications. Analysis of detected failure cases underscores the improved reliability of CropTTA under challenging conditions. Finally, we demonstrate the capability of combining CropTTA and DE for a comprehensive assessment of both data and model uncertainty. Our research provides profound insights into …

11. Trajectory Planning and Control for Differentially Flat Fixed-Wing Aerial Systems

    Luca Morando, Sanket A Salunkhe, Nishanth Bobbili, and 5 more authors

    arXiv preprint arXiv:2502.00581, 2025

    Abs

    Efficient real-time trajectory planning and control for fixed-wing unmanned aerial vehicles is challenging due to their non-holonomic nature, complex dynamics, and the additional uncertainties introduced by unknown aerodynamic effects. In this paper, we present a fast and efficient real-time trajectory planning and control approach for fixed-wing unmanned aerial vehicles, leveraging the differential flatness property of fixed-wing aircraft in coordinated flight conditions to generate dynamically feasible trajectories. The approach provides the ability to continuously replan trajectories, which we show is useful to dynamically account for the curvature constraint as the aircraft advances along its path. Extensive simulations and real-world experiments validate our approach, showcasing its effectiveness in generating trajectories even in challenging conditions for small FW such as wind disturbances.

2024

1. Learning to fly in seconds

   Jonas Eschmann, Dario Albani, and Giuseppe Loianno

   IEEE Robotics and Automation Letters, 2024

   Abs

   Learning-based methods, particularly Reinforcement Learning (RL), hold great promise for streamlining deployment, enhancing performance, and achieving generalization in the control of autonomous multirotor aerial vehicles. Deep RL has been able to control complex systems with impressive fidelity and agility in simulation but the simulation-to-reality transfer often brings a hard-to-bridge reality gap. Moreover, RL is commonly plagued by prohibitively long training times. In this work, we propose a novel asymmetric actor-critic-based architecture coupled with a highly reliable RL-based training paradigm for end-to-end quadrotor control. We show how curriculum learning and a highly optimized simulator enhance sample complexity and lead to fast training times. To precisely discuss the challenges related to low-level/end-to-end multirotor control, we also introduce a taxonomy that classifies the existing levels of …

2. Human-aware physical human-robot collaborative transportation and manipulation with multiple aerial robots

   Guanrui Li, Xinyang Liu, and Giuseppe Loianno

   IEEE Transactions on Robotics, 2024

   Abs

   Human–robot interaction will play an essential role in various industries and daily tasks, enabling robots to effectively collaborate with humans and reduce physical workload. Most existing approaches for physical human–robot interaction focus on collaboration between a human and a single ground or aerial robot. In recent years, very little progress has been made in this research area when considering multiple aerial robots, which offer increased versatility and mobility. This article presents a novel approach for physical human–robot collaborative transportation and manipulation of a cable-suspended payload with multiple aerial robots. The proposed method enables smooth and intuitive interaction between the transported objects and a human worker. We address the inter-robots and inter-robot–human separation during the operations by exploiting the internal redundancy of the multirobot transportation system …

3. From propeller damage estimation and adaptation to fault tolerant control: Enhancing quadrotor resilience

   Jeffrey Mao, Jennifer Yeom, Suraj Nair, and 1 more author

   IEEE Robotics and Automation Letters, 2024

   Abs

   Aerial robots are required to remain operational even in the event of system disturbances, damages, or failures to ensure resilient and robust task completion and safety. One common failure case is propeller damage, which presents a significant challenge in both quantification and compensation. In this letter, we propose a novel adaptive control scheme capable of detecting and compensating for multi-rotor propeller damages, ensuring safe and robust flight performances. Our solution combines an L1 adaptive controller with an optimization routine for damage inference and compensation of single or dual propellers, with the capability to seamlessly transition to a fault-tolerant solution in case the damage becomes severe. We experimentally identify the conditions under which the L1 adaptive solution remains preferable over a fault-tolerant alternative. Experimental results validate the proposed approach …

4. Graph neural network for decentralized multi-robot goal assignment

   Manohari Goarin and Giuseppe Loianno

   IEEE Robotics and Automation Letters, 2024

   Abs

   The problem of assigning a set of spatial goals to a team of robots plays a crucial role in various multi-robot planning applications including, but not limited to exploration, search and rescue, or surveillance. The Linear Sum Assignment Problem (LSAP) is a common way of formulating and resolving this problem. This optimization problem aims at assigning the tasks to the robots minimizing the sum of costs while respecting a one-to-one matching constraint. However, communication restrictions in real-world scenarios pose significant challenges. Existing decentralized solutions often rely on numerous communication interactions to converge to a conflict-free and optimal solution or assume a prior conflict-free random assignment. In this paper, we propose a novel Decentralized Graph Neural Network approach for multi-robot Goal Assignment (DGNN-GA). We leverage a heterogeneous graph representation to model …

5. Unifying foundation models with quadrotor control for visual tracking beyond object categories

   Alessandro Saviolo, Pratyaksh Rao, Vivek Radhakrishnan, and 2 more authors

   2024

   Abs

   Visual control enables quadrotors to adaptively navigate using real-time sensory data, bridging perception with action. Yet, challenges persist, including generalization across scenarios, maintaining reliability, and ensuring real-time responsiveness. This paper introduces a perception framework grounded in foundation models for universal object detection and tracking, moving beyond specific training categories. Integral to our approach is a multi-layered tracker integrated with the foundation detector, ensuring continuous target visibility, even when faced with motion blur, abrupt light shifts, and occlusions. Complementing this, we introduce a model-free controller tailored for resilient quadrotor visual tracking. Our system operates efficiently on limited hardware, relying solely on an onboard camera and an inertial measurement unit. Through extensive validation in diverse challenging indoor and outdoor …

6. Sthn: Deep homography estimation for uav thermal geo-localization with satellite imagery

   Jiuhong Xiao, Ning Zhang, Daniel Tortei, and 1 more author

   IEEE Robotics and Automation Letters, 2024

   Abs

   Accurate geo-localization of Unmanned Aerial Vehicles (UAVs) is crucial for outdoor applications including search and rescue operations, power line inspections, and environmental monitoring. The vulnerability of Global Navigation Satellite Systems (GNSS) signals to interference and spoofing necessitates the development of additional robust localization methods for autonomous navigation. Visual Geo-localization (VG), leveraging onboard cameras and reference satellite maps, offers a promising solution for absolute localization. Specifically, Thermal Geo-localization (TG), which relies on image-based matching between thermal imagery with satellite databases, stands out by utilizing infrared cameras for effective nighttime localization. However, the efficiency and effectiveness of current TG approaches, are hindered by dense sampling on satellite maps and geometric noises in thermal query images. To …

7. Decentralized nonlinear model predictive control for safe collision avoidance in quadrotor teams with limited detection range

   Manohari Goarin, Guanrui Li, Alessandro Saviolo, and 1 more author

   arXiv preprint arXiv:2409.17379, 2024

   Abs

   Multi-quadrotor systems face significant challenges in decentralized control, particularly with safety and coordination under sensing and communication limitations. State-of-the-art methods leverage Control Barrier Functions (CBFs) to provide safety guarantees but often neglect actuation constraints and limited detection range. To address these gaps, we propose a novel decentralized Nonlinear Model Predictive Control (NMPC) that integrates Exponential CBFs (ECBFs) to enhance safety and optimality in multi-quadrotor systems. We provide both conservative and practical minimum bounds of the range that preserve the safety guarantees of the ECBFs. We validate our approach through extensive simulations with up to 10 quadrotors and 20 obstacles, as well as real-world experiments with 3 quadrotors. Results demonstrate the effectiveness of the proposed framework in realistic settings, highlighting its potential for reliable quadrotor teams operations.

8. Coped-advancing multi-robot collaborative perception: A comprehensive dataset in real-world environments

   Yang Zhou, Long Quang, Carlos Nieto-Granda, and 1 more author

   IEEE Robotics and Automation Letters, 2024

   Abs

   In the past decade, although single-robot perception has made significant advancements, the exploration of multi-robot collaborative perception remains largely unexplored. This involves fusing compressed, intermittent, limited, heterogeneous, and asynchronous environmental information across multiple robots to enhance overall perception, despite challenges like sensor noise, occlusions, and sensor failures. One major hurdle has been the lack of real-world datasets. This letter presents a pioneering and comprehensive real-world multi-robot collaborative perception dataset to boost research in this area. Our dataset leverages the untapped potential of air-ground robot collaboration featuring distinct spatial viewpoints, complementary robot mobilities, coverage ranges, and sensor modalities. It features raw sensor inputs, pose estimation, and optional high-level perception annotation, thus accommodating diverse …

9. Spatial assisted human-drone collaborative navigation and interaction through immersive mixed reality

   Luca Morando and Giuseppe Loianno

   2024

   Abs

   Aerial robots have the potential to play a crucial role in assisting humans with complex and dangerous tasks. Nevertheless, the future industry demands innovative solutions to streamline the interaction process between humans and drones to enable seamless collaboration and efficient coworking. In this paper, we present a novel tele-immersive framework that promotes cognitive and physical collaboration between humans and robots through Mixed Reality (MR). This framework incorporates a novel bi-directional spatial awareness and a multi-modal virtual-physical interaction approaches. The former seamlessly integrates the physical and virtual worlds, offering bidirectional egocentric and exocentric environmental representations. The latter, leveraging the proposed spatial representation, further enhances the collaboration combining a robot planning algorithm for obstacle avoidance with a variable admittance …

10. The power of input: Benchmarking zero-shot sim-to-real transfer of reinforcement learning control policies for quadrotor control

    Alberto Dionigi, Gabriele Costante, and Giuseppe Loianno

    2024

    Abs

    In the last decade, data-driven approaches have become popular choices for quadrotor control, thanks to their ability to facilitate the adaptation to unknown or uncertain flight conditions. Among the different data-driven paradigms, Deep Reinforcement Learning (DRL) is currently one of the most explored. However, the design of DRL agents for Micro Aerial Vehicles (MAVs) remains an open challenge. While some works have studied the output configuration of these agents (i.e., what kind of control to compute), there is no general consensus on the type of input data these approaches should employ. Multiple works simply provide the DRL agent with full state information, without questioning if this might be redundant and unnecessarily complicate the learning process, or pose superfluous constraints on the availability of such information in real platforms. In this work, we provide an in-depth benchmark analysis of …

11. Directed graph topology preservation in multi-robot systems with limited field of view using control barrier functions

    Filippo Bertoncelli, Vivek Radhakrishnan, Mattia Catellani, and 2 more authors

    IEEE Access, 2024

    Abs

    This paper addresses the crucial challenge of maintaining the directed graph topology in multi-robot systems, particularly when operating under limited field-of-view constraints and with a lack of communication among robots. Traditional methods for multi-robot coordination rely heavily on inter-robot communication, which may not always be feasible, particularly in constrained or hostile environments. Our work presents a novel distributed control algorithm that leverages Control Barrier Functions (CBFs) to maintain the graph topology of a multi-robot system based solely on local, onboard sensor data. This approach is particularly beneficial in situations where external communication channels are disrupted or unavailable. The key contributions of this research are threefold: First, we design a novel control algorithm that efficiently maintains the graph topology in multi-robot systems using CBFs, which operate on …

12. Data-Driven System Identification of Quadrotors Subject to Motor Delays

    Jonas Eschmann, Dario Albani, and Giuseppe Loianno

    2024

    Abs

    Recently non-linear control methods like Model Predictive Control (MPC) and Reinforcement Learning (RL) have attracted increased interest in the quadrotor control community. In contrast to classic control methods like cascaded PID controllers, MPC and RL heavily rely on an accurate model of the system dynamics. The process of quadrotor system identification is notoriously tedious and is often pursued with additional equipment like a thrust stand. Furthermore, low-level details like motor delays which are crucial for accurate end-to-end control are often neglected. In this work, we introduce a data-driven method to identify a quadrotor’s inertia parameters, thrust curves, torque coefficients, and first-order motor delay purely based on proprioceptive data. The estimation of the motor delay is particularly challenging as usually, the RPMs can not be measured. We derive a Maximum A Posteriori (MAP)-based method to …

13. Reactive collision avoidance for safe agile navigation

    Alessandro Saviolo, Niko Picello, Jeffrey Mao, and 2 more authors

    arXiv preprint arXiv:2409.11962, 2024

    Abs

    Reactive collision avoidance is essential for agile robots navigating complex and dynamic environments, enabling real-time obstacle response. However, this task is inherently challenging because it requires a tight integration of perception, planning, and control, which traditional methods often handle separately, resulting in compounded errors and delays. This paper introduces a novel approach that unifies these tasks into a single reactive framework using solely onboard sensing and computing. Our method combines nonlinear model predictive control with adaptive control barrier functions, directly linking perception-driven constraints to real-time planning and control. Constraints are determined by using a neural network to refine noisy RGB-D data, enhancing depth accuracy, and selecting points with the minimum time-to-collision to prioritize the most immediate threats. To maintain a balance between safety and agility, a heuristic dynamically adjusts the optimization process, preventing overconstraints in real time. Extensive experiments with an agile quadrotor demonstrate effective collision avoidance across diverse indoor and outdoor environments, without requiring environment-specific tuning or explicit mapping.

14. Experimental system design of an active fault-tolerant quadrotor

    Jennifer Yeom, Guanrui Li, and Giuseppe Loianno

    arXiv preprint arXiv:2404.06340, 2024

    Abs

    Quadrotors have gained popularity over the last decade, aiding humans in complex tasks such as search and rescue, mapping and exploration. Despite their mechanical simplicity and versatility compared to other types of aerial vehicles, they remain vulnerable to rotor failures. In this paper, we propose an algorithmic and mechanical approach to addressing the quadrotor fault-tolerant problem in case of rotor failures. First, we present a fault-tolerant detection and control scheme that includes various attitude error metrics. The scheme transitions to a fault-tolerant control mode by surrendering the yaw control. Subsequently, to ensure compatibility with platform sensing constraints, we investigate the relationship between variations in robot rotational drag, achieved through a modular mechanical design appendage, resulting in yaw rates within sensor limits. This analysis offers a platform-agnostic framework for designing more reliable and robust quadrotors in the event of rotor failures. Extensive experimental results validate the proposed approach providing insights into successfully designing a cost-effective quadrotor capable of fault-tolerant control. The overall design enhances safety in scenarios of faulty rotors, without the need for additional sensors or computational resources.

15. Robust upper limb kinematic reconstruction using a rgb-d camera

    Salvatore Maria Li Gioi, Giuseppe Loianno, and Francesca Cordella

    IEEE Robotics and Automation Letters, 2024

    Abs

    In this letter, we propose a new approach for human motion reconstruction based on Gaussian Mixture Probability Hypothesis Density (GM-PHD) Filter applied to human joint positions extracted from RGB-D camera (e.g. Kinect). Existing inference approaches require a proper association between measurements and joints, which cannot be maintained in case of the multi-tracking occlusion problem. The proposed GM-PHD recursively estimates the number and states of each group of targets. Furthermore, we embed kinematic constraints in the inference process to guarantee robustness to occlusions. We evaluate the accuracy of both the proposed approach and the default one obtained through a Kinect device by comparing them with a motion analysis system (i.e. Vicon optoelectronic system) even in presence of occlusions of one or more body joints. Experimental results show that the filter outperforms the solution …

16. HPA-MPC: Hybrid Perception-Aware Nonlinear Model Predictive Control for Quadrotors with Suspended Loads

    Mrunal Sarvaiya, Guanrui Li, and Giuseppe Loianno

    IEEE Robotics and Automation Letters, 2024

    Abs

    Quadrotors equipped with cable-suspended loads represent a versatile, low-cost, and energy efficient solution for aerial transportation, construction, and manipulation tasks. However, their real-world deployment is hindered by several challenges. The system is difficult to control because it is nonlinear, underactuated, involves hybrid dynamics due to slack-taut cable modes, and evolves on complex configuration spaces. Additionally, it is crucial to estimate the full state and the cable’s mode transitions in real-time using on-board sensors and computation. To address these challenges, we present a novel Hybrid Perception-Aware Nonlinear Model Predictive Control (HPA-MPC) control approach for quadrotors with suspended loads. Our method considers the complete hybrid system dynamics and includes a perception-aware cost to ensure the payload remains visible in the robot’s camera during navigation …

17. Learning Long-Horizon Predictions for Quadrotor Dynamics

    Pratyaksh Prabhav Rao, Alessandro Saviolo, Tommaso Castiglione Ferrari, and 1 more author

    2024

    Abs

    Accurate modeling of system dynamics is crucial for achieving high-performance planning and control of robotic systems. Although existing data-driven approaches represent a promising approach for modeling dynamics, their accuracy is limited to a short prediction horizon, overlooking the impact of compounding prediction errors over longer prediction horizons. Strategies to mitigate these cumulative errors remain underexplored. To bridge this gap, in this paper, we study the key design choices for efficiently learning long-horizon prediction dynamics for quadrotors. Specifically, we analyze the impact of multiple architectures, historical data, and multi-step loss formulation. We show that sequential modeling techniques showcase their advantage in minimizing compounding errors compared to other types of solutions. Furthermore, we propose a novel decoupled dynamics learning approach, which further simplifies …

18. QuadFormer: Real-time unsupervised power line segmentation with transformer-based domain adaptation

    Pratyaksh Prabhav Rao, Feng Qiao, Weide Zhang, and 5 more authors

    2024

    Abs

    Accurately identifying Power Lines (PLs) is crucial for ensuring the safety of aerial vehicles. Despite the potential of recent deep learning approaches, obtaining high-quality ground truth annotations remains a challenging and labor-intensive task. Unsupervised Domain Adaptation (UDA) emerges as a promising solution, leveraging knowledge from labeled synthetic data to improve performance on unlabeled real images. However, existing UDA methods often suffer of huge computation costs, limiting their deployment on real-time embedded systems commonly utilized on aerial vehicles. To mitigate this problem, this paper introduces QuadFormer, a real-time framework designed for unsupervised semantic segmentation within the UDA paradigm. QuadFormer integrates a lightweight transformer-based segmentation model with a cross-attention mechanism to narrow the gap between a labelled synthetic domain and …

19. Collision Dynamics of Motorized Deformable Propellers for Drones

    Hung Tien Pham, Dinh Quang Nguyen, Son Tien Bui, and 2 more authors

    2024

    Abs

    This paper investigates and analyzes the behavior of a deformable propeller during and after collisions. The experimental setup includes a deformable propeller, a BLDC motor, and a collision initiated while the propeller is rotating steadily. Here, we examine the changes in propeller’s angular velocity over time from the start of the collision until it fully recovers its initial velocity. This variation will be compared between the experimentally measured wing velocity using an encoder and the calculated propeller’s angular velocity in the simulation. The constructed model describes the relationship between propeller’s angular velocity and the input voltage supplied to the motor based on the Lagrange method. The study confirmed the shape transformation process and full restoration of the propeller’s original shape following collisions through high-speed video analysis. The results demonstrate consistent monitoring of …

20. System and method for autonomous charging

    Giuseppe Loianno, Alessandro Saviolo, Jeffrey Mao, and 2 more authors

    2024

    Abs

    2023-09-26 Assigned to NEW YORK UNIVERSITY reassignment NEW YORK UNIVERSITY ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: RADHAKRISHNAN, VIVEK, LOIANNO, GIUSEPPE, MAO, JEFFREY, MARGABANDU BALAKRISHNAN, ROSHAN BALU, SAVIOLO, ALESSANDRO

21. Rising stars in field robotics: 2022

    Dimitrios Kanoulas, Shehryar Khattak, and Giuseppe Loianno

    2024

    Abs

    It’s important to note that the selected papers are recognitions of talented early career researchers that might shape the trajectory of technological evolution. As we stand on the cusp of future innovations in the field of robotics, which remain largely undiscovered, this Research Topic offers a glimpse into who might lead these advancements. This initiative is more than just an academic exercise; it’s a beacon for the scientific community and industry alike to identify and follow the luminaries of tomorrow.In conclusion, while the full scope of future innovations in field robotics is yet to unfold, this collection is a guiding light, pointing us towards the brilliant minds who will drive the next wave of technological revolution.Maani Ghaffari et al., present advancements in the field of robot perception and control, focusing on the integration of symmetry in problem-solving. Drawing inspiration from mathematical techniques used to analyze symmetry in geometric spaces, this research explores how geometric sensor registration, state estimation, and control methods can offer crucial understanding in formulating robotics algorithms. These algorithms are particularly tailored for complex, unexplored environments. Furthermore, the incorporation of computational techniques for determining difficult-to-quantify factors enhances the efficacy of these symmetrybased methods. The paper substantiates its assertions by demonstrating experimental outcomes in real-world settings, covering aspects of robot perception, state estimation, and control.Manoni et al., address the challenge of real-time adjustment in swarm flocking behaviors for aerial drones, a task complicated by fast …

22. ON UNMANNED AIRCRAFT SYSTEMS

    Nikos Tsourveloudis, Didier Theilliol, H Jin Kim, and 9 more authors

    IEEE CONTROL SYSTEMS, 2024

    Abs

    February 2024 Conference Calendar [Conference Calendar] Page 1 » CONFERENCE CALENDAR 88 IEEE CONTROL SYSTEMS » FEBRUARY 2024 » 2024 ● 2024 AUSTRALIAN AND NEW ZEALAND CONTROL CONFERECE (ANZCC 2024) 1–2 February Gold Coast, Australia General Chair: Ljubo Vlacic Program Chair: Victor Sreeram https://anzcc.org.au/ANZCC2024/index. php ● 2024 INTERNATIONAL CONFERENCE ON UNMANNED AIRCRAFT SYSTEMS 4–7 June Crete, Greece General Chairs: Nikos Tsourveloudis and Didier Theilliol Program Chairs: Nikos Vitzilaios, H. Jin Kim, and Giuseppe Loianno https://uasconferences.com/2024_icuas/ ● ECC 2024 22ND EUROPEAN CONTROL CONFERENCE 25–28 June Stockholm, Sweden General Chairs: Karl H. Johansson and Anders Rantzer Program Chairs: Henrik Sandberg and Antonis Papachristodoulou https://ecc24.euca-ecc.org/ ▲ AMERICAN …

2023

1. Learning quadrotor dynamics for precise, safe, and agile flight control

   Alessandro Saviolo and Giuseppe Loianno

   2023

   Abs

   This article reviews the state-of-the-art modeling and control techniques for aerial robots such as quadrotor systems and presents several future research directions in this area. The review starts by introducing the benefits and drawbacks of classic physic-based dynamic modeling and control techniques. Subsequently, the manuscript presents the key challenges to augment or replace classic techniques with data-driven approaches that can offer several key benefits in terms of flight precision, safety, adaptation, and agility.

2. Active learning of discrete-time dynamics for uncertainty-aware model predictive control

   Alessandro Saviolo, Jonathan Frey, Abhishek Rathod, and 2 more authors

   IEEE Transactions on Robotics, 2023

   Abs

   Model-based control requires an accurate model of the system dynamics for precisely and safely controlling the robot in complex and dynamic environments. Moreover, in presence of variations in the operating conditions, the model should be continuously refined to compensate for dynamics changes. In this article, we present a self-supervised learning approach that actively models the dynamics of nonlinear robotic systems. We combine offline learning from past experience and online learning from current robot interaction with the unknown environment. These two ingredients enable a highly sample-efficient and adaptive learning process, capable of accurately inferring model dynamics in real-time even in operating regimes that greatly differ from the training distribution. Moreover, we design an uncertainty-aware model predictive controller that is heuristically conditioned to the aleatoric (data) uncertainty of the …

3. Nonlinear model predictive control for cooperative transportation and manipulation of cable suspended payloads with multiple quadrotors

   Guanrui Li and Giuseppe Loianno

   2023

   Abs

   Autonomous Micro Aerial Vehicles (MAVs) such as quadrotors equipped with manipulation mechanisms have the potential to assist humans in tasks such as construction and package delivery. Cables are a promising option for manipulation mechanisms due to their low weight, low cost, and simple design. However, designing control and planning strategies for cable mechanisms presents challenges due to indirect load actuation, nonlinear configuration space, and highly coupled system dynamics. In this paper, we propose a novel Nonlinear Model Predictive Control (NMPC) method that enables a team of quadrotors to manipulate a rigid-body payload in all 6 degrees of freedom via suspended cables. Our approach can concurrently exploit, as part of the receding horizon optimization, the available mechanical system redundancies to perform additional tasks such as inter-robot separation and obstacle avoidance …

4. RotorTM: A flexible simulator for aerial transportation and manipulation

   Guanrui Li, Xinyang Liu, and Giuseppe Loianno

   IEEE Transactions on Robotics, 2023

   Abs

   Low-cost autonomous micro aerial vehicles have great potential to help humans by simplifying and speeding up complex tasks, such as construction, package delivery, and search and rescue. These systems, which may consist of single or multiple vehicles, can be equipped with passive connection mechanisms, such as rigid links or cables for transportation and manipulation tasks. However, these systems are inherently complex. They are often underactuated and evolve in nonlinear manifold configuration spaces. In addition, the complexity escalates for systems with cable-suspended load due to the hybrid dynamics that vary with the cables’ tension conditions. This article presents the first aerial transportation and manipulation simulator incorporating different payloads and passive connection mechanisms with full system dynamics, planning, and control algorithms. Furthermore, it includes a novel general model …

5. Robust active visual perching with quadrotors on inclined surfaces

   Jeffrey Mao, Stephen Nogar, Christopher M Kroninger, and 1 more author

   IEEE Transactions on Robotics, 2023

   Abs

   Autonomous micro aerial vehicles are deployed for a variety of tasks including surveillance and monitoring. Perching and staring allow the vehicle to monitor targets without flying, saving battery power and increasing the overall mission time without the need to frequently replace batteries. This article addresses the active visual perching (AVP) control problem to autonomously perch on inclined surfaces up to . Our approach generates dynamically feasible trajectories to navigate and perch on a desired target location while taking into account actuator and field-of-view constraints. By replanning in midflight, we take advantage of more accurate target localization increasing the perching maneuver’s robustness to target localization or control errors. We leverage the Karush–Kuhn–Tucker (KKT) conditions to identify the compatibility between planning objectives and the visual sensing constraint during the planned maneuver …

6. Long-range uav thermal geo-localization with satellite imagery

   Jiuhong Xiao, Daniel Tortei, Eloy Roura, and 1 more author

   2023

   Abs

   Onboard sensors, such as cameras and thermal sensors, have emerged as effective alternatives to Global Positioning System (GPS) for geo-Iocalization in Unmanned Aerial Vehicle (UAV) navigation. Since GPS can suffer from signal loss and spoofing problems, researchers have explored camera-based techniques such as Visual Geo-Iocalization (VG) using satellite RGB imagery. Additionally, thermal geo-Iocalization (TG) has become crucial for long-range UAV flights in low-illumination environments. This paper proposes a novel thermal geo-Iocalization framework using satellite RGB imagery, which includes multiple domain adaptation methods to address the limited availability of paired thermal and satellite images. The experimental results demonstrate the effectiveness of the proposed approach in achieving reliable thermal geo-Iocalization performance, even in thermal images with indistinct self-similar …

7. Autocharge: Autonomous charging for perpetual quadrotor missions

   Alessandro Saviolo, Jeffrey Mao, Vivek Radhakrishnan, and 1 more author

   IEEE International Conference on Robotics and Automation (ICRA), 2023

   Abs

   Battery endurance represents a key challenge for long-term autonomy and long-range operations, especially in the case of aerial robots. In this paper, we propose AutoCharge, an autonomous charging solution for quadrotors that combines a portable ground station with a flexible, lightweight charging tether and is capable of universal, highly efficient, and robust charging. We design and manufacture a pair of circular magnetic connectors to ensure a precise orientation-agnostic electrical connection between the ground station and the charging tether. Moreover, we supply the ground station with an electromagnet that largely increases the tolerance to localization and control errors during the docking maneuver, while still guaranteeing smooth un-docking once the charging process is completed. We demonstrate AutoCharge on a perpetual 10 hours quadrotor flight experiment and show that the docking and un-docking performance is solidly repeatable, enabling perpetual quadrotor flight missions.

8. GaPT: Gaussian Process Toolkit for Online Regression with Application to Learning Quadrotor Dynamics

   Francesco Crocetti, Jeffrey Mao, Alessandro Saviolo, and 2 more authors

   2023

   Abs

   Gaussian Processes (GPs) are expressive models for capturing signal statistics and expressing prediction uncertainty. As a result, the robotics community has gathered interest in leveraging these methods for inference, planning, and control. Unfortunately, despite providing a closed-form inference solution, GPs are non-parametric models that typically scale cubically with the dataset size, hence making them difficult to be used especially on onboard Size, Weight, and Power (SWaP) constrained aerial robots. In addition, the integration of popular libraries with GPs for different kernels is not trivial. In this paper, we propose GaPT, a novel toolkit that converts GPs to their state space form and performs regression in linear time. GaPT is designed to be highly compatible with several optimizers popular in robotics. We thoroughly validate the proposed approach for learning quadrotor dynamics on both single and multiple input GP settings. GaPT accurately captures the system behavior in multiple flight regimes and operating conditions, including those producing highly nonlinear effects such as aerodynamic forces and rotor interactions. Moreover, the results demonstrate the superior computational performance of GaPT compared to a classical GP inference approach on both single and multi-input settings especially when considering large number of data points, enabling real-time regression speed on embedded platforms used on SWaP-constrained aerial robots.

9. Geometric fault-tolerant control of quadrotors in case of rotor failures: An attitude based comparative study

   Jennifer Yeom, Guanrui Li, and Giuseppe Loianno

   2023

   Abs

   The ability of aerial robots to operate in the presence of failures is crucial in various applications that demand continuous operations, such as surveillance, monitoring, and inspection. In this paper, we propose a fault-tolerant control strategy for quadrotors that can adapt to single and dual complete rotor failures. Our approach augments a classic geometric tracking controller on to accommodate the effects of rotor failures. We provide an in-depth analysis of several attitude error metrics to identify the most appropriate design choice for fault-tolerant control strategies. To assess the effectiveness of these metrics, we evaluate trajectory tracking accuracies. Simulation results demonstrate the performance of the proposed approach.

10. RLtools: A fast, portable deep reinforcement learning library for continuous control

    Jonas Eschmann, Dario Albani, and Giuseppe Loianno

    arXiv preprint arXiv:2306.03530, 2023

    Abs

    Deep Reinforcement Learning (RL) can yield capable agents and control policies in several, domains but is commonly plagued by prohibitively long training times. Additionally, in the case of continuous control problems, the applicability of learned policies on real-world embedded devices is limited due to the lack of real-time guarantees and portability of existing libraries. To address these challenges, we present RLtools, a dependency-free, header-only, pure C++ library for deep supervised and reinforcement learning. Its novel architecture allows RLtools to be used on a wide variety of platforms, from HPC clusters over workstations and laptops to smartphones, smartwatches, and microcontrollers. Specifically, due to the tight integration of the RL algorithms with simulation environments, RLtools can solve popular RL problems up to 76 times faster than other popular RL frameworks. We also benchmark the inference on a diverse set of microcontrollers and show that in most cases our optimized implementation is by far the fastest. Finally, RLtools enables the first-ever demonstration of training a deep RL algorithm directly on a microcontroller, giving rise to the field of Tiny Reinforcement Learning (TinyRL). The source code as well as documentation and live demos are available through our project page at https://rl.tools.

11. Exploring deep reinforcement learning for robust target tracking using micro aerial vehicles

    Alberto Dionigi, Mirko Leomanni, Alessandro Saviolo, and 2 more authors

    2023

    Abs

    The capability to autonomously track a non-cooperative target is a key technological requirement for micro aerial vehicles. In this paper, we propose an output feedback control scheme based on deep reinforcement learning for controlling a micro aerial vehicle to persistently track a flying target while maintaining visual contact. The proposed method leverages relative position data for control, relaxing the assumption of having access to full state information which is typical of related approaches in literature. Moreover, we exploit classical robustness indicators in the learning process through domain randomization to increase the robustness of the learned policy. Experimental results validate the proposed approach for target tracking, demonstrating high performance and robustness with respect to mass mismatches and control delays. The resulting nonlinear controller significantly outperforms a standard model-based …

12. Visual geo-localization with self-supervised representation learning

    Jiuhong Xiao, Gao Zhu, and Giuseppe Loianno

    arXiv preprint arXiv:2308.00090, 2023

    Abs

    Visual Geo-localization (VG) has emerged as a significant research area, aiming to identify geolocation based on visual features. Most VG approaches use learnable feature extractors for representation learning. Recently, Self-Supervised Learning (SSL) methods have also demonstrated comparable performance to supervised methods by using numerous unlabeled images for representation learning. In this work, we present a novel unified VG-SSL framework with the goal to enhance performance and training efficiency on a large VG dataset by SSL methods. Our work incorporates multiple SSL methods tailored for VG: SimCLR, MoCov2, BYOL, SimSiam, Barlow Twins, and VICReg. We systematically analyze the performance of different training strategies and study the optimal parameter settings for the adaptation of SSL methods for the VG task. The results demonstrate that our method, without the significant computation and memory usage associated with Hard Negative Mining (HNM), can match or even surpass the VG performance of the baseline that employs HNM. The code is available at https://github.com/arplaboratory/VG_SSL.

13. Learning heuristics for efficient environment exploration using graph neural networks

    Edwin P Herrera-Alarcón, Gabriele Baris, Massimo Satler, and 2 more authors

    2023

    Abs

    The robot exploration problem focuses on maximizing the volumetric map of a previously unknown environment. This is a relevant problem in several applications, such as search and rescue and monitoring, which require autonomous robots to examine the surroundings efficiently. Graph-based planning approaches embed the exploration information into a graph describing the global map while the robot incrementally builds it. Nevertheless, even if graph-based representations are computational and memory-efficient, the exploration decision-making problem complexity increases according to the graph size that grows at each iteration. In this paper, we propose a novel Graph Neural Network (GNN) approach trained with Reinforcement Learning (RL) that solves the decision-making problem for autonomous exploration. The learned policy represents the exploration expansion criterion, solving the decision-making …

14. Visual Environment Assessment for Safe Autonomous Quadrotor Landing

    Mattia Secchiero, Nishanth Bobbili, Yang Zhou, and 1 more author

    arXiv preprint arXiv:2311.10065, 2023

    Abs

    Autonomous identification and evaluation of safe landing zones are of paramount importance for ensuring the safety and effectiveness of aerial robots in the event of system failures, low battery, or the successful completion of specific tasks. In this paper, we present a novel approach for detection and assessment of potential landing sites for safe quadrotor landing. Our solution efficiently integrates 2D and 3D environmental information, eliminating the need for external aids such as GPS and computationally intensive elevation maps. The proposed pipeline combines semantic data derived from a Neural Network (NN), to extract environmental features, with geometric data obtained from a disparity map, to extract critical geometric attributes such as slope, flatness, and roughness. We define several cost metrics based on these attributes to evaluate safety, stability, and suitability of regions in the environments and identify the most suitable landing area. Our approach runs in real-time on quadrotors equipped with limited computational capabilities. Experimental results conducted in diverse environments demonstrate that the proposed method can effectively assess and identify suitable landing areas, enabling the safe and autonomous landing of a quadrotor.

15. PENet: A Joint Panoptic Edge Detection Network

    Yang Zhou and Giuseppe Loianno

    arXiv preprint arXiv:2303.08848, 2023

    Abs

    In recent years, compact and efficient scene understanding representations have gained popularity in increasing situational awareness and autonomy of robotic systems. In this work, we illustrate the concept of a panoptic edge segmentation and propose PENet, a novel detection network called that combines semantic edge detection and instance-level perception into a compact panoptic edge representation. This is obtained through a joint network by multi-task learning that concurrently predicts semantic edges, instance centers and offset flow map without bounding box predictions exploiting the cross-task correlations among the tasks. The proposed approach allows extending semantic edge detection to panoptic edge detection which encapsulates both category-aware and instance-aware segmentation. We validate the proposed panoptic edge segmentation method and demonstrate its effectiveness on the real-world Cityscapes dataset.

2022

1. Multi-robot collaborative perception with graph neural networks

   Yang Zhou, Jiuhong Xiao, Yue Zhou, and 1 more author

   IEEE Robotics and Automation Letters, 2022

   Abs

   Multi-robot systems such as swarms of aerial robots are naturally suited to offer additional flexibility, resilience, and robustness in several tasks compared to a single robot by enabling cooperation among the agents. To enhance the autonomous robot decision-making process and situational awareness, multi-robot systems have to coordinate their perception capabilities to collect, share, and fuse environment information among the agents efficiently to obtain context-appropriate information or gain resilience to sensor noise or failures. In this letter, we propose a general-purpose Graph Neural Network (GNN) with the main goal to increase, in multi-robot perception tasks, single robots’ inference perception accuracy as well as resilience to sensor failures and disturbances. We show that the proposed framework can address multi-view visual perception problems such as monocular depth estimation and semantic …

2. Physics-inspired temporal learning of quadrotor dynamics for accurate model predictive trajectory tracking

   Alessandro Saviolo, Guanrui Li, and Giuseppe Loianno

   IEEE Robotics and Automation Letters, 2022

   Abs

   Accurately modeling quadrotor’s system dynamics is critical for guaranteeing agile, safe, and stable navigation. The model needs to capture the system behavior in multiple flight regimes and operating conditions, including those producing highly nonlinear effects such as aerodynamic forces and torques, rotor interactions, or possible system configuration modifications. Classical approaches rely on handcrafted models and struggle to generalize and scale to capture these effects. In this letter, we present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor’s system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions. In addition, physics constraints are embedded in the training process to facilitate the network’s generalization …

3. Generative neural network channel modeling for millimeter-wave UAV communication

   William Xia, Sundeep Rangan, Marco Mezzavilla, and 4 more authors

   IEEE Transactions on Wireless Communications, 2022

   Abs

   The millimeter wave bands are being increasingly considered for wireless communication to unmanned aerial vehicles (UAVs). Critical to this undertaking are statistical channel models that describe the distribution of constituent parameters in scenarios of interest. This paper presents a general modeling methodology based on data-training a generative neural network. The proposed generative model has a two-stage structure that first predicts the link state (line-of-sight, non-line-of-sight, or outage), and subsequently feeds this state into a conditional variational autoencoder (VAE) that generates the path losses, delays, and angles of arrival and departure for all the propagation paths. The methodology is demonstrated for air-to-ground channels between UAVs and a cellular system in representative urban environments, with training datasets produced through ray tracing. The demonstration extends to both …

4. Autonomous single-image drone exploration with deep reinforcement learning and mixed reality

   Alessandro Devo, Jeffrey Mao, Gabriele Costante, and 1 more author

   IEEE Robotics and Automation Letters, 2022

   Abs

   Autonomous exploration is a longstanding goal of the robotics community. Aerial drone navigation has proven to be especially challenging. The stringent requirements on cost, weight, maneuverability, and power consumption do not allow exploration approaches to easily be employed or adapted to different types of environments. End-to-End Deep Reinforcement Learning (DRL) techniques based on Convolutional Networks approximators, which grant constant-time computation, predefined memory usage, and deliver high visual perception capabilities, represent a very promising alternative to current state of the art solutions relying on metric environment reconstruction. In this work, we address the autonomous exploration problem with aerial robots with a monocular camera based on DRL. Specifically, we propose a novel asymmetric actor-critic model for drone exploration that efficiently leverages ground truth …

5. Tombo Propeller: Bioinspired Deformable Structure Toward Collision-Accommodated Control for Drones

   Son Tien Bui, Quan Khanh Luu, Dinh Quang Nguyen, and 3 more authors

   IEEE Transactions on Robotics, 2022

   Abs

   There is a growing need for vertical takeoff and landing vehicles, including drones, which are safe to use and can adapt to collisions. The risks of damage by collision, to humans, obstacles in the environment, and drones themselves, are significant. This has prompted a search into nature for a highly resilient structure that can inform a design of propellers to reduce those risks and enhance safety. Inspired by the flexibility and resilience of dragonfly wings, we propose a novel design for a biomimetic drone propeller called Tombo propeller. Here, we report on the design and fabrication process of this biomimetic propeller that can accommodate collisions and recover quickly, while maintaining sufficient thrust force to hover and fly. We describe the development of an aerodynamic model and experiments conducted to investigate performance characteristics for various configurations of the propeller morphology and …

6. Vision-based relative detection and tracking for teams of micro aerial vehicles

   Rundong Ge, Moonyoung Lee, Vivek Radhakrishnan, and 3 more authors

   2022

   Abs

   In this paper, we address the vision-based detection and tracking problems of multiple aerial vehicles using a single camera and Inertial Measurement Unit (IMU) as well as the corresponding perception consensus problem (i.e., uniqueness and identical IDs across all observing agents). We design several vision-based decentralized Bayesian multi-tracking filtering strategies to resolve the association between the incoming unsorted measurements obtained by a visual detector algorithm and the tracked agents. We compare their accuracy in different operating conditions as well as their scalability according to the number of agents in the team. This analysis provides useful insights about the most appropriate design choice for the given task. We further show that the proposed perception and inference pipeline which includes a Deep Neural Network (DNN) as visual target detector is lightweight and capable of …

7. Learning model predictive control for quadrotors

   Guanrui Li, Alex Tunchez, and Giuseppe Loianno

   2022

   Abs

   Aerial robots can enhance their safe and agile navigation in complex and cluttered environments by efficiently exploiting the information collected during a given task. In this paper, we address the learning model predictive control problem for quadrotors. We design a learning receding-horizon nonlinear control strategy directly formulated on the system nonlinear manifold configuration space SO(3)×R3. The proposed approach exploits past successful task iterations to improve the system performance over time while respecting system dynamics and actuator constraints. We further relax its computational complexity making it compatible with real-time quadrotor control requirements. We show the effectiveness of the proposed approach in learning a minimum time control task, respecting dynamics, actuators, and environment constraints. Several experiments in simulation and real-world set-up validate the proposed …

8. Coexistence of UAVs and terrestrial users in millimeter-wave urban networks

   Seongjoon Kang, Marco Mezzavilla, Angel Lozano, and 5 more authors

   2022

   Abs

   5G millimeter-wave (mmWave) cellular networks are in the early phase of commercial deployments and present a unique opportunity for robust, high-data-rate communication to unmanned aerial vehicles (UAVs). A fundamental question is whether and how mmWave networks designed for terrestrial users should be modified to serve UAVs. The paper invokes realistic cell layouts, antenna patterns, and channel models trained from extensive ray tracing data to assess the performance of various network alternatives. Importantly, the study considers the addition of dedicated uptilted rooftop-mounted cells for aerial coverage, as well as novel spectrum sharing modes between terrestrial and aerial network operators. The effect of power control and of multiuser multiple-input multiple-output are also studied.

9. Vision-based perimeter defense via multiview pose estimation

   Elijah S Lee, Giuseppe Loianno, Dinesh Jayaraman, and 1 more author

   arXiv preprint arXiv:2209.12136, 2022

   Abs

   Previous studies in the perimeter defense game have largely focused on the fully observable setting where the true player states are known to all players. However, this is unrealistic for practical implementation since defenders may have to perceive the intruders and estimate their states. In this work, we study the perimeter defense game in a photo-realistic simulator and the real world, requiring defenders to estimate intruder states from vision. We train a deep machine learning-based system for intruder pose detection with domain randomization that aggregates multiple views to reduce state estimation errors and adapt the defensive strategy to account for this. We newly introduce performance metrics to evaluate the vision-based perimeter defense. Through extensive experiments, we show that our approach improves state estimation, and eventually, perimeter defense performance in both 1-defender-vs-1-intruder games, and 2-defenders-vs-1-intruder games.

10. Towards new frontiers in mobile manipulation: Team CTU-UPenn-NYU at MBZIRC 2020

    Petr Stibinger, George Broughton, Filip Majer, and 8 more authors

    Field Robotics, 2022

    Abs

    Towards new frontiers in mobile manipulation: Team CTU-UPenn-NYU at MBZIRC 2020 | OpenReview OpenReview.net Login back arrow Go to DBLP homepage Towards new frontiers in mobile manipulation: Team CTU-UPenn-NYU at MBZIRC 2020 Open Webpage Petr Stibinger, George Broughton, Filip Majer, Zdenek Rozsypálek, Anthony Wang, Kshitij Jindal, Alex Zhou, Dinesh Thakur, Giuseppe Loianno, Tomás Krajník, Martin Saska Published: 01 Jan 2022, Last Modified: 13 Nov 2024Field Robotics 2022EveryoneRevisionsBibTeXCC BY-SA 4.0 Loading About OpenReview Hosting a Venue All Venues Contact Feedback Sponsors Join the Team Frequently Asked Questions Terms of Use Privacy Policy About OpenReview Hosting a Venue All Venues Sponsors Join the Team Frequently Asked Questions Contact Feedback Terms of Use Privacy Policy OpenReview is a long-term project to advance science …

11. Towards New Frontiers in Mobile Manipulation: Team CTU-UPenn-NYU at MBZIRC 2020

    Petr Štibinger, George Broughton, Filip Majer, and 8 more authors

    Field Robotics, 2022

    Abs

    In this paper we present an autonomous robotic system for picking, transporting, and precisely placing magnetically graspable objects. Such a system would be especially beneficial for construction tasks where human presence is not possible, e.g. due to chemical or radioactive pollution. The system comprises of two primary components — a wheeled, mobile platform and a manipulator arm. Both are interconnected through an onboard computer and utilize various onboard sensors for estimating the state of the robot and its surroundings. By using efficient processing algorithms, data from the onboard sensors can be used in a feedback loop during all critical operational sections, resulting in a robust system capable of operating on uneven terrain and in environments without access to satellite navigation. System functionality has been proven in Challenge II of the MBZIRC 2020 competition. The Challenge required a …

2021

1. Mobile manipulator for autonomous localization, grasping and precise placement of construction material in a semi-structured environment

   Petr Štibinger, George Broughton, Filip Majer, and 8 more authors

   IEEE Robotics and Automation Letters, 2021

   Abs

   Mobile manipulators have the potential to revolutionize modern agriculture, logistics and manufacturing. In this work, we present the design of a ground-based mobile manipulator for automated structure assembly. The proposed system is capable of autonomous localization, grasping, transportation and deployment of construction material in a semi-structured environment. Special effort was put into making the system invariant to lighting changes, and not reliant on external positioning systems. Therefore, the presented system is self-contained and capable of operating in outdoor and indoor conditions alike. Finally, we present means to extend the perceptive radius of the vehicle by using it in cooperation with an autonomous drone, which provides aerial reconnaissance. Performance of the proposed system has been evaluated in a series of experiments conducted in real-world conditions.

2. Cooperative transportation of cable suspended payloads with mavs using monocular vision and inertial sensing

   Guanrui Li, Rundong Ge, and Giuseppe Loianno

   IEEE Robotics and Automation Letters, 2021

   Abs

   Micro Aerial Vehicles (MAVs) have the great potential to be deployed in commercial or health care services such as e-commerce package delivery, transportation of medicines, same-day food delivery, and other time-sensitive transportation tasks. A team of MAVs can cooperatively transport objects to overcome the physical limitations of a single vehicle, while concurrently increasing the system’s resilience to vehicles’ failures. In this letter, we address the state estimation, control and trajectory tracking problems of cooperative transportation of cable suspended rigid body payloads with MAVs using monocular vision and inertial sensing. The key contributions are (a) a distributed vision-based coordinated control of the cable-suspended rigid body payload on SE(3), (b) a distributed estimation approach that allows each agent to estimate its cable direction and velocity independently, and (c) a new cooperative estimation …

3. Tracking and relative localization of drone swarms with a vision-based headset

   Maxim Pavliv, Fabrizio Schiano, Christopher Reardon, and 2 more authors

   IEEE Robotics and Automation Letters, 2021

   Abs

   We address the detection, tracking, and relative localization of the agents of a drone swarm from a human perspective using a headset equipped with a single camera and an Inertial Measurement Unit (IMU). We train and deploy a deep neural network detector on image data to detect the drones. A joint probabilistic data association filter resolves the detection problems and couples this information with the headset IMU data to track the agents. In order to estimate the drones’ relative poses in 3D space with respect to the human, we use an additional deep neural network that processes image regions of the drones provided by the tracker. Finally, to speed up the deep neural networks’ training, we introduce an automated labeling process relying on a motion capture system. Several experimental results validate the effectiveness of the proposed approach. The approach is real-time, does not rely on any …

4. Lightweight UAV-based measurement system for air-to-ground channels at 28 GHz

   Vasilii Semkin, Seongjoon Kang, Jaakko Haarla, and 7 more authors

   2021

   Abs

   Wireless communication at millimeter wave frequencies is an attractive option for high-bit-rate connectivity to unmanned aerial vehicles (UAVs). However, conducting the channel measurements necessary to assess the communication performance at these frequencies has been challenging due to the severe payload and power restrictions in commercial UAVs. This work presents a novel lightweight (approximately 1.3kg) channel measurement system at 28GHz installed on a commercially available UAV. A ground transmitter equipped with a horn antenna conveys sounding signals to a UAV equipped with a lightweight spectrum analyzer. We demonstrate that the measurements can be highly influenced by the onboard antenna pattern as shaped by the UAV’s frame. A calibration procedure is presented to correct for the resulting angular variations in antenna gain. The measurement setup is then validated on real …

5. PCMPC: Perception-constrained model predictive control for quadrotors with suspended loads using a single camera and IMU

   Guanrui Li, Alex Tunchez, and Giuseppe Loianno

   2021

   Abs

   In this paper, we address the Perception– Constrained Model Predictive Control (PCMPC) and state estimation problems for quadrotors with cable suspended payloads using a single camera and Inertial Measurement Unit (IMU). We design a receding–horizon control strategy for cable suspended payloads directly formulated on the system manifold configuration space SE (3) ×S2. The approach considers the system dynamics, actuator limits and the camera’s Field Of View (FOV) constraint to guarantee the payload’s visibility during motion. The monocular camera, IMU, and vehicle’s motor speeds are combined to provide estimation of the vehicle’s states in 3D space, the payload’s states, the cable’s direction and velocity. The proposed control and state estimation solution runs in real-time at 500 Hz on a small quadrotor equipped with a limited computational unit. The approach is validated through experimental …

6. Millimeter-wave UAV coverage in urban environments

   Seongjoon Kang, Marco Mezzavilla, Angel Lozano, and 5 more authors

   2021

   Abs

   With growing interest in mmWave connectivity for unmanned aerial vehicles (UAVs), a basic question is whether networks intended for terrestrial service can provide sufficient aerial coverage as well. To assess this possibility in the context of urban environments, extensive system-level simulations are conducted using a generative channel model recently proposed by the authors. It is found that standard downtilted base stations at street level, deployed with typical microcellular densities, can indeed provide satisfactory UAV coverage. Interestingly, this coverage is made possible by a conjunction of antenna sidelobes and strong reflections. As the deployments become sparser, the coverage is only guaranteed at progressively higher UAV altitudes. The incorporation of base stations dedicated to UAV communication, rooftop-mounted and uptilted, would strengthen the coverage provided their density is comparable to …

7. Aggressive visual perching with quadrotors on inclined surfaces

   Jeffrey Mao, Guanrui Li, Stephen Nogar, and 2 more authors

   2021

   Abs

   Autonomous Micro Aerial Vehicles (MAVs) have the potential to be employed for surveillance and monitoring tasks. By perching and staring on one or multiple locations aerial robots can save energy while concurrently increasing their overall mission time without actively flying. In this paper, we address the estimation, planning, and control problems for autonomous perching on inclined surfaces with small quadrotors using visual and inertial sensing. We focus on planning and executing dynamically feasible trajectories to navigate and perch to a desired target location with on board sensing and computation. Our planner also supports certain classes of nonlinear global constraints by leveraging an efficient algorithm that we have mathematically verified. The on board cameras and IMU are concurrently used for state estimation and to infer the relative robot/target localization. The proposed solution runs in real-time …

8. Vipose: Real-time visual-inertial 6d object pose tracking

   Rundong Ge and Giuseppe Loianno

   2021

   Abs

   Estimating the 6D pose of objects is beneficial for robotics tasks such as transportation, autonomous navigation, manipulation as well as in scenarios beyond robotics like virtual and augmented reality. With respect to single image pose estimation, pose tracking takes into account the temporal information across multiple frames to overcome possible detection inconsistencies and to improve the pose estimation efficiency. In this work, we introduce a novel Deep Neural Network (DNN) called VIPose, that combines inertial and camera data to address the object pose tracking problem in real-time. The key contribution is the design of a novel DNN architecture which fuses visual and inertial features to predict the objects’ relative 6D pose between consecutive image frames. The overall 6D pose is then estimated by consecutively combining relative poses. Our approach shows remarkable pose estimation results for …

9. Defending a perimeter from a ground intruder using an aerial defender: Theory and practice

   Elijah S Lee, Daigo Shishika, Giuseppe Loianno, and 1 more author

   2021

   Abs

   The perimeter defense game has received interest in recent years as a variant of the pursuit-evasion game. A number of previous works have solved this game to obtain the optimal strategies for defender and intruder, but the derived theory considers the players as point particles with first-order assumptions. In this work, we aim to apply the theory derived from the perimeter defense problem to robots with realistic models of actuation and sensing and observe performance discrepancy in relaxing the first-order assumptions. In particular, we focus on the hemisphere perimeter defense problem where a ground intruder tries to reach the base of a hemisphere while an aerial defender constrained to move on the hemisphere aims to capture the intruder. The transition from theory to practice is detailed, and the designed system is simulated in Gazebo. Two metrics for parametric analysis and comparative study are …

10. Comparative analysis of agent-oriented task assignment and path planning algorithms applied to drone swarms

    Rohith Gandhi Ganesan, Samantha Kappagoda, Giuseppe Loianno, and 1 more author

    arXiv preprint arXiv:2101.05161, 2021

    Abs

    Autonomous drone swarms are a burgeoning technology with significant applications in the field of mapping, inspection, transportation and monitoring. To complete a task, each drone has to accomplish a sub-goal within the context of the overall task at hand and navigate through the environment by avoiding collision with obstacles and with other agents in the environment. In this work, we choose the task of optimal coverage of an environment with drone swarms where the global knowledge of the goal states and its positions are known but not of the obstacles. The drones have to choose the Points of Interest (PoI) present in the environment to visit, along with the order to be visited to ensure fast coverage. We model this task in a simulation and use an agent-oriented approach to solve the problem. We evaluate different policy networks trained with reinforcement learning algorithms based on their effectiveness, i.e. time taken to map the area and efficiency, i.e. computational requirements. We couple the task assignment with path planning in an unique way for performing collision avoidance during navigation and compare a grid-based global planning algorithm, i.e. Wavefront and a gradient-based local planning algorithm, i.e. Potential Field. We also evaluate the Potential Field planning algorithm with different cost functions, propose a method to adaptively modify the velocity of the drone when using the Huber loss function to perform collision avoidance and observe its effect on the trajectory of the drones. We demonstrate our experiments in 2D and 3D simulations.

11. Design and deployment of an autonomous unmanned ground vehicle for urban firefighting scenarios

    Kshitij Jindal, Anthony Wang, Dinesh Thakur, and 7 more authors

    Field Robotics, 2021

    Abs

    Autonomous mobile robots have the potential to execute missions that are either too complex or too dangerous for humans. In this paper, we address the design and deployment of an autonomous ground vehicle equipped with a robotic arm for urban firefighting scenarios. We describe hardware and algorithm designs for autonomous navigation, planning, fire source identification and abatement in unstructured urban scenarios. Our approach employs on-board sensors for autonomous navigation and thermal camera information for source identification. A custom electro-mechanical pump is responsible to eject water for fire abatement. The proposed approach is validated through several experiments, where we show the ability to identify and abate a simulated fire source in a building. The whole system was developed and deployed during the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020 …

12. Semi-dense visual-inertial odometry and mapping for computationally constrained platforms

    Wenxin Liu, Kartik Mohta, Giuseppe Loianno, and 2 more authors

    Autonomous Robots, 2021

    Abs

    In this paper we present a direct semi-dense stereo Visual-Inertial Odometry (VIO) algorithm enabling autonomous flight for quadrotor systems with Size, Weight, and Power (SWaP) constraints. The proposed approach is validated through experiments on a 250 g, 22 cm diameter quadrotor equipped with a stereo camera and an IMU. Semi-dense methods have superior performance in low texture areas, which are often encountered in robotic tasks such as infrastructure inspection. However, due to the measurement size and iterative nonlinear optimization, these methods are computationally more expensive. As the scale of the platform shrinks down, the available computation of the on-board CPU becomes limited, making autonomous navigation using optimization-based semi-dense tracking a hard problem. We show that our direct semi-dense VIO performs comparably to other state-of-the-art methods, while taking …

13. Selective and Hierarchical Allocation of Sensing Resources for Anomalous Target Identification in Exploratory Missions

    Brigid A Blakeslee and Giuseppe Loianno

    2021

    Abs

    We present an approach for selective, hierarchical allocation of sensing resources that aims to maximize information gain in exploratory missions such as search and rescue (SAR) or surveillance in an efficient manner. Specifically, we propose a methodology for perception-enabled SAR or crowd surveillance driven by anomaly detection based on low-level statistical assessment of a region. The characterizations of previously-observed regions are used to populate a window of observations that serves as “short-term memory,” providing a contextually-appropriate characterization of proximate regions in the scene. Currently-observed regions are compared with this short-term memory window, and if sufficiently dissimilar, can be considered as candidates for the presence of a SAR target or unexpected event. We adaptively allocate additional sensing resources for subsequent exploration of anomalous regions …

2020

1. Imu-based inertia estimation for a quadrotor using newton-euler dynamics

   James Svacha, James Paulos, Giuseppe Loianno, and 1 more author

   IEEE Robotics and Automation Letters, 2020

   Abs

   In this letter, we demonstrate that a quadrotor’s tilt, angular velocity, linear velocity and the parameters shown in Table II may be estimated using only an inertial measurement unit (IMU) and motor speed feedback for sensing. Motor speed commands are used to drive the process model and the motor speed and IMU measurements are used in the measurement model of an unscented Kalman filter (UKF) containing 32 states, 14 of which are constant parameters. We analytically show the observability of this system. Furthermore, we demonstrate through experiments that a blade flapping moment term is not only significant, but necessary to include in the rotation dynamics to get a sensible moment of inertia estimate. We also model the motor torque as a function of the angular acceleration and velocity of the motors in order to obtain a more accurate moment of inertia estimate.

2. Millimeter wave channel modeling via generative neural networks

   William Xia, Sundeep Rangan, Marco Mezzavilla, and 4 more authors

   2020

   Abs

   Statistical channel models are instrumental to design and evaluate wireless communication systems. In the millimeter wave bands, such models become acutely challenging; they must capture the delay, directions, and path gains, for each link and with high resolution. This paper presents a general modeling methodology based on training generative neural networks from data. The proposed generative model consists of a two-stage structure that first predicts the state of each link (line-of-sight, non-line-of-sight, or outage), and subsequently feeds this state into a conditional variational autoencoder that generates the path losses, delays, and angles of arrival and departure for all its propagation paths. Importantly, minimal prior assumptions are made, enabling the model to capture complex relationships within the data. The methodology is demonstrated for 28GHz air-to-ground channels in an urban environment, with …

3. Special issue on future challenges and opportunities in vision‐based drone navigation

   Giuseppe Loianno and Davide Scaramuzza

   2020

   Abs

   During the last decade, the research in the area of autonomous navigation for micro aerial vehicles (MAVs) often called drones, has fueled the rapid growth of solutions for recreational/commercial purposes. Autonomous flying robots are starting to help humans in tasks like search and rescue, environment monitoring, security surveillance, transportation, and inspection. Small‐scale size vehicles represent a viable solution to solve tasks in narrow outdoor and indoor environments and because of their size and weight, they represent only a limited risk for people. However, the creation of autonomous drones, able to accomplish complex missions without requiring human intervention, is still far to be a reality and poses several requirements and research challenges. First, navigation in GPS‐denied environment and the vehicles’ limitations in size, power, computational, and sensing capabilities pose several design constraints. These vehicles can often rely exclusively on cameras and inertial measurement units (IMUs) as sensing modalities. Second, vehicles may need to operate in adverse conditions and the autonomy requirements may vary according to the level of complexity and clutter in the environment. Operations performed in urban areas require different sensing, control, and perception modalities compared with operations in underground and space environments. Third, response time is often a key requirement during missions. The low battery density of these vehicles calls for novel agile navigation approaches such to accomplish more during the vehicles’ limited battery lifetime. Navigation at high speeds and accelerations poses several …

4. Multi-array designs for mmWave and sub-THz communication to UAVs

   William Xia, Vasilii Semkin, Marco Mezzavilla, and 2 more authors

   2020

   Abs

   Unmanned Aerial Vehicles (UAVs) are steadily being considered for scenarios demanding high bandwidth, low latency communications for video and sensor data transfer as well as real-time control. UAVs may also be used as mobile aerial Base Station (BS) for maintaining the communication in emergency scenarios. The millimeter wave (mmWave) bands, including the sub-THz frequencies above 100 GHz, are an attractive technology for high data rate UAV connectivity due to the wide bandwidths available at these frequencies. This paper studies antenna and codebook design for UAV communication at both 28 and 140 GHz with realistic antenna simulations and flight patterns. The analysis shows that multi-array configurations with proper codebook design are necessary for uniform spherical coverage and become particularly important in long range applications. The paper thus proposes a four array design …

5. Towards design of a deformable propeller for drone safety

   Dinh Quang Nguyen and Giuseppe Loianno

   2020

   Abs

   Drones have brought many benefits to our lives and their use is growing at a rapid rate. Many countries have drone flight restriction rules; however, the safety of drone operators and bystanders, and the protection of drones against damage require improvement. Here, we propose a novel design of deformable propellers inspired by dragonfly wings. The structure of these propellers includes a flexible segment similar to the nodus on a dragonfly wing. This flexible segment can bend, twist and even fold upon collision, absorbing force upon impact and protecting the propeller from damage. Part of the leading edge of the propeller consists of a pliable silicone rubber surface able to absorb impact forces and reducing blade sharpness. The propeller, which is approximately 10inches long, can generate a thrust force of nearly 1.3 N at maximum velocity of about 3200rpm. Results of blade sharpness tests showed that the …

6. Experimental evaluation and characterization of radioactive source effects on robot visual localization and mapping

   Elijah S Lee, Giuseppe Loianno, Dinesh Thakur, and 1 more author

   IEEE Robotics and Automation Letters, 2020

   Abs

   Robots are ideally suited to performing simple tasks in dangerous environments. In this letter, we address the use of robots for inspection of nuclear reactors which may be contaminated by radiation. The geometry of a reactor vessel is three-dimensional with significant clutter. Accordingly, we propose the use of small-scale, flying robots that are able to localize themselves and autonomously navigate around obstacles. Because of the constraints on size, we rely on cameras which are the best low power and lightweight sensors. However, cameras perform poorly in the presence of radioactivity and the impact of radiation on robotics systems is not well understood. In this letter, we (a) analyze the effects of radioactive sources on camera sensors, affecting localization and mapping algorithms, (b) quantify these effects from a statistical viewpoint according to different source intensities; and (c) compare different solutions …

7. Observability-aware trajectories for geometric and inertial self-calibration

   Christoph Böhm, Guanrui Li, Giuseppe Loianno, and 1 more author

   Power On and Go Robots, 2020

   Abs

   In this paper, we apply an observability-aware trajectory generation method to the estimation of geometric and inertial parameters of an Unmanned Aerial System (UAS). These parameters are critical for reliable control and agile maneuvers, especially in the context of reconfiguration of the aerial vehicles during manipulation or transportation tasks. An extended observability analysis provides detailed insights on the observable and inter-state dependencies. We employ the observability-aware motion generation approach considering full system dynamics and self-calibration parameters. Improvements in the absolute error of B rBP estimates of up to 46.8% and decreases in uncertainty of up to 87% are achievable with this approach. Experiments with an autonomous quadrotor platform validate the approach.

8. Efficient trajectory library filtering for quadrotor flight in unknown environments

   Vaibhav K Viswanathan, Eric Dexheimer, Guanrui Li, and 3 more authors

   2020

   Abs

   Quadrotor flight in cluttered, unknown environments is challenging due to the limited range of perception sensors, challenging obstacles, and limited onboard computation. In this work, we directly address these challenges by proposing an efficient, reactive planning approach. We introduce the Bitwise Trajectory Elimination (BiTE) algorithm for efficiently filtering out in-collision trajectories from a trajectory library by using bitwise operations. Then, we outline a full receding-horizon planning approach for quadrotor flight in unknown environments demonstrated at up to 50 Hz on an onboard computer. This approach is evaluated extensively in simulation and shown to collision check up to 4896 trajectories in under 20μs, which is the fastest collision checking time for a MAV planner, to the best of the authors’ knowledge. Finally, we validate our planner in over 120 minutes of flights in forest-like and urban subterranean …

9. Design and experimental evaluation of distributed cooperative transportation of cable suspended payloads with micro aerial vehicles

   Guanrui Li and Giuseppe Loianno

   2020

   Abs

   Micro Aerial Vehicles (MAVs) can play an important role in helping to accomplish tasks that are physically too dangerous or complex for humans. A team of aerial robots can cooperatively transport objects that are too large or heavy to be carried by a single MAV. This paper address the design and experimental evaluation of a distributed cooperative control method for cooperative transportation of cable-suspended payloads using MAVs. The key contributions of this paper are: i) a distributed control method for cooperative transportation of cable-suspended payloads with a team of quadrotors is experimentally designed and evaluated and allows to control both the load position and orientation, ii) our solution is light-weight and can run on-board small robots equipped with computationally limited units, iii) we study and analyze the system’s performances and maneuverability based on different cable lengths.

10. FENet: Fast Real-time Semantic Edge Detection Network

    Yang Zhou, Rundong Ge, Gary McGrath, and 1 more author

    2020

    Abs

    Semantic edge is a geometric-aware semantic feature that can be leveraged in robotic perception systems for increased situational awareness and high-level environment understanding. This sparse representation nicely encapsulates the semantic information (object categories) within the geometric object boundaries. State-of-the-art semantic edge detection approaches require significant computation power and fail to approach real-time performance on embedded devices for robotic applications. In this paper, we present FENet (Fast Realtime Semantic Edge Detection Network), a semantic edge detection approach for robots with Size, Weight, and Power (SWaP) constraints. Specifically, we adopt MobileNetV2 as a lightweight backbone network, and we utilize joint pyramid upsampling to improve the system performance. We further reduce the model complexity and latency through network pruning and multiple …

11. Agile and Resilient Autonomous Flight

    Giuseppe Loianno

    2020

    Abs

    Summary form only given, as follows. The complete presentation was not made available for publication as part of the conference proceedings. Flying robots, often called drones, are starting to play a major role in several tasks such as search and rescue, interaction with the environment, inspection, patrolling and monitoring. Agile and resilient navigation of Micro Aerial Vehicles (MAVs) through unknown environments poses a number of challenges in terms of perception, state estimation, planning, and control. To achieve this, MAVs have to localize themselves and coordinate between each other in unstructured environments. In this talk, I will present some recent research results agile and resilient navigation of aerial robots for search and rescue, exploration, transportation, physical environment interaction, and human drone collaboration using a minimal on-board sensor suite composed by an IMU and camera …

12. Unmanned Aerial Vehicles Swarms

    Giuseppe Loianno, Aaron Weinstein, and Vijay Kumar

    2020

    Abs

    An unmanned aerial vehicle (UAV) swarm can be simply defined as a group aerial robotic platform, usually similar in form, coordinating and cooperating to achieve a common goal. Swarms extend robotic capabilities beyond those of a single vehicle through various methods of coordination and cooperation between the different agents. The coordination component of independent, decoupled, and identical agents enables redundancy, allowing the system to compensate for the lack of robustness of individuals. The cooperation is an advanced form of collective behavior, which provides mutual benefits to the agents when working or acting together. This behavior is essential when agents acting by themselves are not successful and do not necessarily get rewarded for their actions. Swarms are optimized around different applications and constraints with large implementation differences including the form of individual …

2019

1. Autonomous landing on a moving vehicle with an unmanned aerial vehicle

   Tomas Baca, Petr Stepan, Vojtech Spurny, and 6 more authors

   Journal of Field Robotics, 2019

   Abs

   This paper addresses the perception, control, and trajectory planning for an aerial platform to identify and land on a moving car at 15 km/hr. The hexacopter unmanned aerial vehicle (UAV), equipped with onboard sensors and a computer, detects the car using a monocular camera and predicts the car future movement using a nonlinear motion model. While following the car, the UAV lands on its roof, and it attaches itself using magnetic legs. The proposed system is fully autonomous from takeoff to landing. Numerous field tests were conducted throughout the year‐long development and preparations for the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2017 competition, for which the system was designed. We propose a novel control system in which a model predictive controller is used in real time to generate a reference trajectory for the UAV, which are then tracked by the nonlinear feedback …

2. Cooperative autonomous search, grasping, and delivering in a treasure hunt scenario by a team of unmanned aerial vehicles

   Vojtěch Spurný, Tomáš Báča, Martin Saska, and 6 more authors

   Journal of Field Robotics, 2019

   Abs

   This paper addresses the problem of autonomous cooperative localization, grasping and delivering of colored ferrous objects by a team of unmanned aerial vehicles (UAVs). In the proposed scenario, a team of UAVs is required to maximize the reward by collecting colored objects and delivering them to a predefined location. This task consists of several subtasks such as cooperative coverage path planning, object detection and state estimation, UAV self‐localization, precise motion control, trajectory tracking, aerial grasping and dropping, and decentralized team coordination. The failure recovery and synchronization job manager is used to integrate all the presented subtasks together and also to decrease the vulnerability to individual subtask failures in real‐world conditions. The whole system was developed for the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2017, where it achieved the …

3. Human gaze-driven spatial tasking of an autonomous MAV

   Liangzhe Yuan, Christopher Reardon, Garrett Warnell, and 1 more author

   IEEE Robotics and Automation Letters, 2019

   Abs

   In this letter, we address the problem of providing human-assisted quadrotor navigation using a set of eye tracking glasses. The advent of these devices (i.e., eye tracking glasses, virtual reality tools, etc.) provides the opportunity to create new, noninvasive forms of interaction between humans and robots. We show how a set of glasses equipped with gaze tracker, a camera, and an inertial measurement unit (IMU) can be used to estimate the relative position of the human with respect to a quadrotor, and decouple the gaze direction from the head orientation, which allows the human to spatially task (i.e., send new 3-D navigation waypoints to) the robot in an uninstrumented environment. We decouple the gaze direction from head motion by tracking the human’s head orientation using a combination of camera and IMU data. In order to detect the flying robot, we train and use a deep neural network. We experimentally …

4. Mavnet: An effective semantic segmentation micro-network for mav-based tasks

   Ty Nguyen, Shreyas S Shivakumar, Ian D Miller, and 9 more authors

   IEEE Robotics and Automation Letters, 2019

   Abs

   Real-time semantic image segmentation on platforms subject to size, weight, and power constraints is a key area of interest for air surveillance and inspection. In this letter, we propose MAVNet: a small, light-weight, deep neural network for real-time semantic segmentation on micro aerial vehicles (MAVs). MAVNet, inspired by ERFNet [E. Romera, J. M. lvarez, L. M. Bergasa, and R. Arroyo, “ErfNet: Efficient residual factorized convnet for real-time semantic segmentation,” IEEE Trans. Intell. Transp. Syst., vol. 19, no. 1, pp. 263–272, Jan. 2018.], features 400 times fewer parameters and achieves comparable performance with some reference models in empirical experiments. Additionally, we provide two novel datasets that represent challenges in semantic segmentation for real-time MAV tracking and infrastructure inspection tasks and verify MAVNet on these datasets. Our algorithm and datasets are made publicly …

5. Millimeter wave remote UAV control and communications for public safety scenarios

   William Xia, Michele Polese, Marco Mezzavilla, and 3 more authors

   2019

   Abs

   Communication and video capture from unmanned aerial vehicles (UAVs) offer significant potential for assisting first responders in remote public safety settings. In such uses, millimeter wave (mmWave) wireless links can provide high throughput and low latency connectivity between the UAV and a remote command center. However, maintaining reliable aerial communication in the mmWave bands is challenging due to the need to support high speed beam tracking and overcome blockage. This paper provides a simulation study aimed at assessing the feasibility of public safety UAV connectivity through a 5G link at 28 GHz. Real flight motion traces are captured during maneuvers similar to those expected in public safety settings. The motions traces are then incorporated into a detailed mmWave network simulator that models the channel, blockage, beamforming and full 3GPP protocol stack. We show that 5G …

6. Online estimation of geometric and inertia parameters for multirotor aerial vehicles

   Valentin Wüest, Vijay Kumar, and Giuseppe Loianno

   2019

   Abs

   Accurate knowledge of geometric and inertia parameters are a necessity for precise and robust control of aerial vehicles. We propose a novel filter that is able to fuse motor speed, inertia, and pose measurements to estimate the vehicle’s key dynamic properties online. The presented framework is able to estimate the multirotor’s moment of inertia, mass, center of mass and each sensor module’s relative position. Obtaining these estimates in-flight allow the multirotor to be precisely controlled even during tasks such as load transportation or after configuration changes on scene. We provide a nonlinear observability analysis, proving that the presented model is locally weakly observable. Experimental results validate the proposed approach, showing the ability to estimate the dynamic properties accurately and demonstrate its capability to do so even while additional loads are added. The framework is flexible and can …

7. Inertial yaw-independent velocity and attitude estimation for high-speed quadrotor flight

   James Svacha, Giuseppe Loianno, and Vijay Kumar

   IEEE Robotics and Automation Letters, 2019

   Abs

   This letter addresses the velocity and attitude estimation of a quadrotor using motor speeds and inertial measurement unit (IMU) data. This result is obtained through a novel filter that employs a first-order drag model, which allows the velocity to be observed through the drag forces acting on the quadrotor. These forces are measured with the IMU. Compared to our previous contribution [J. Svacha, K. Mohta, M. Watterson, G. Loianno, and V. Kumar, “Inertial velocity and attitude estimation for quadrotors,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. , Oct. 2018, pp. 1-9.], decoupling the attitude into a yaw-tilt convention results in the tilt and velocity in the body-fixed frame being independent of the yaw. Thus, the velocity and attitude estimates are no longer affected by yaw drifts obtained from dead-reckoning procedures. The camera available on the vehicle can optionally be employed to recover the yaw in a …

8. Autonomous inspection of a containment vessel using a micro aerial vehicle

   Dinesh Thakur, Giuseppe Loianno, Laura Jarin-Lipschitz, and 2 more authors

   2019

   Abs

   In this work, we address the design, estimation, planning, control, and mapping problems to allow a small scale quadrotor to autonomously inspect the interior of a Containment Vessel (CV) test fixture, using on-board lighting, computation and sensing. We demonstrate a fully autonomous, 160 cm tip-to-tip, 236 gram Micro Aerial Vehicle (MAV) performing a complex flight mission inside of a dark test fixture that was built from a scale model of a CV. The proposed solution opens up new ways to inspect nuclear power plants and has the potential to support nuclear decommissioning, which is well known to be a dangerous, long, and tedious process. Experimental results show the ability to navigate under dripping water conditions and inside a completely dark, full-scale CV test fixture while concurrently avoiding known and unknown obstacles.

9. Artificial neural network-assisted controller for fast and agile UAV flight: Onboard implementation and experimental results

   Siddharth Patel, Andriy Sarabakha, Dogan Kircali, and 2 more authors

   2019

   Abs

   In this work, we address fast and agile manoeuvre control problem of unmanned aerial vehicles (UAVs) using an artificial neural network (ANN)-assisted conventional controller. Whereas the need for having almost perfect control accuracy for UAVs pushes the operation to boundaries of the performance envelope, safety and reliability concerns enforce researchers to be more conservative in tuning their controllers. As an alternative solution to the aforementioned trade-off, a reliable yet accurate controller is designed for the trajectory tracking of UAVs by learning system dynamics online over the trajectory. What is more, the proposed online learning mechanism helps us to deal with unmodelled dynamics and operational uncertainties. Experimental results validate the proposed approach and show the superiority of our method compared to the conventional controller for fast and agile manoeuvres, at speeds as high …

2018

1. Fast, autonomous flight in GPS‐denied and cluttered environments

   Kartik Mohta, Michael Watterson, Yash Mulgaonkar, and 14 more authors

   Journal of Field Robotics, 2018

   Abs

   One of the most challenging tasks for a flying robot is to autonomously navigate between target locations quickly and reliably while avoiding obstacles in its path, and with little to no a priori knowledge of the operating environment. This challenge is addressed in the present paper. We describe the system design and software architecture of our proposed solution and showcase how all the distinct components can be integrated to enable smooth robot operation. We provide critical insight on hardware and software component selection and development and present results from extensive experimental testing in real‐world warehouse environments. Experimental testing reveals that our proposed solution can deliver fast and robust aerial robot autonomous navigation in cluttered, GPS‐denied environments.

2. Model predictive trajectory tracking and collision avoidance for reliable outdoor deployment of unmanned aerial vehicles

   Tomas Baca, Daniel Hert, Giuseppe Loianno, and 2 more authors

   2018

   Abs

   We propose a novel approach for optimal trajectory tracking for unmanned aerial vehicles (UAV), using a linear model predictive controller (MPC) in combination with non-linear state feedback. The solution relies on fast onboard simulation of the translational dynamics of the UAV, which is guided by a linear MPC. By sampling the states of the virtual UAV, we create a control command for fast non-linear feedback, which is capable of performing agile maneuvers with high precision. In addition, the proposed pipeline provides an interface for a decentralized collision avoidance system for multi-UAY scenarios. Our solution makes use of the long prediction horizon of the linear MPC and allows safe outdoors execution of multi-UAV experiments without the need for in-advance collision-free planning. The practicality of the tracking mechanism is shown in combination with priority-based collision resolution strategy, which …

3. Localization, grasping, and transportation of magnetic objects by a team of mavs in challenging desert-like environments

   Giuseppe Loianno, Vojtech Spurny, Justin Thomas, and 9 more authors

   IEEE Robotics and Automation Letters, 2018

   Abs

   Autonomous Micro Aerial Vehicles (MAVs) have the potential to assist in real-life tasks involving grasping and transportation, but not before solving several difficult research challenges. In this work, we address the design, control, estimation, and planning problems for cooperative localization, grasping, and transportation of objects in challenging outdoor scenarios. We demonstrate an autonomous team of MAVs able to plan safe trajectories for manipulation of ferrous objects, while guaranteeing interrobot collision avoidance and automatically creating a map of the objects in the environment. Our solution is predominantly distributed, allowing the team to pick and transport ferrous disks to a final destination without collisions. This result is achieved using a new magnetic gripper with a novel feedback approach, enabling the detection of successful grasping. The gripper design and all the components to build a platform …

4. Visual inertial odometry swarm: An autonomous swarm of vision-based quadrotors

   Aaron Weinstein, Adam Cho, Giuseppe Loianno, and 1 more author

   IEEE Robotics and Automation Letters, 2018

   Abs

   In this letter, we present the system infrastructure for a swarm of quadrotors, which perform all estimation on board using monocular visual inertial odometry. This is a novel system since it does not require an external motion capture system or GPS and is able to execute formation tasks without inter-robot collisions. The swarm can be deployed in nearly any indoor or outdoor scenario and is scalable to higher numbers of robots. We discuss the system architecture, estimation, planning, and control for the multirobot system. The robustness and scalability of the approach is validated in both indoor and outdoor environments with up to 12 quadrotors.

5. Autonomous navigation of micro aerial vehicles using high-rate and low-cost sensors

   Angel Santamaria-Navarro, Giuseppe Loianno, Joan Sola, and 2 more authors

   Autonomous robots, 2018

   Abs

   The combination of visual and inertial sensors for state estimation has recently found wide echo in the robotics community, especially in the aerial robotics field, due to the lightweight and complementary characteristics of the sensors data. However, most state estimation systems based on visual-inertial sensing suffer from severe processor requirements, which in many cases make them impractical. In this paper, we propose a simple, low-cost and high rate method for state estimation enabling autonomous flight of micro aerial vehicles, which presents a low computational burden. The proposed state estimator fuses observations from an inertial measurement unit, an optical flow smart camera and a time-of-flight range sensor. The smart camera provides optical flow measurements up to a rate of 200 Hz, avoiding the computational bottleneck to the main processor produced by all image processing requirements. To …

6. Systems, devices, and methods for on-board sensing and control of micro aerial vehicles

   Giuseppe Loianno, Yash Shailesh Mulgaonkar, and R Vijay Kumar

   2018

   Abs

   Systems, devices, and methods for on-board sensing and control of robotic vehicles (e. g., MAVs) using commercial off-the-shelf hand-held electronic devices as a sensing and control system are provided. In one aspect, a system for controlling a micro aerial vehicle may include one or more sensors, a state estimation module in communication with the one or more sensors, the state estimation module being configured to generate an estimated pose of the micro aerial vehicle based on inputs from the one or more sensors, and a position controller in communication with the state esti mation module and configured to communicate attitude commands to an attitude controller of the micro aerial vehicle. Each of the one or more sensors, the state estimation module, and the position controller may be contained in a commercial off-the-shelf hand-held electronic device that is configured to be coupled to the micro aerial …

7. Special issue on high-speed vision-based autonomous navigation of uavs

   Giuseppe Loianno, Davide Scaramuzza, and Vijay Kumar

   Journal of Field Robotics, 2018

   Abs

   This first special issue of the Journal of Field Robotics (JFR) on vision-based high speed autonomous navigation of UAVs aims to establish a baseline in the field of autonomous navigation of UAVs using vision and IMU as the main sensing modalities. The goal of the research reported in this special issue is to show the improvements and present the most recent state of the art to execute fast autonomous operations with MAVs. The proposed approaches will contribute to inform the community with the most recent and innovative approaches and to extend the capabilities of current and future robotic missions.

8. Nuclear environments inspection with micro aerial vehicles: Algorithms and experiments

   Dinesh Thakur, Giuseppe Loianno, Wenxin Liu, and 1 more author

   2018

   Abs

   In this work, we address the estimation, planning, control and mapping problems to allow a small quadrotor to autonomously inspect the interior of hazardous damaged nuclear sites. These algorithms run onboard on a computationally limited CPU. We investigate the effect of varying illumination on the system performance. To the best of our knowledge, this is the first fully autonomous system of this size and scale applied to inspect the interior of a full scale mock-up of a Primary Containment Vessel (PCV). The proposed solution opens up new ways to inspect nuclear reactors and to support nuclear decommissioning, which is well known to be a dangerous, long and tedious process. Experimental results with varying illumination conditions show the ability to navigate a full scale mock-up PCV pedestal and create a map of the environment, while concurrently avoiding obstacles.

9. Spatio-temporally smooth local mapping and state estimation inside generalized cylinders with micro aerial vehicles

   Tolga Özaslan, Giuseppe Loianno, James Keller, and 2 more authors

   IEEE Robotics and Automation Letters, 2018

   Abs

   In this letter, we consider state estimation and local mapping with a micro aerial vehicle inside a tunnel that can be modeled as a generalized cylinder, using a three-dimensional lidar and an inertial measurement unit. This axisymmetric environment poses unique challenges in terms of localization and mapping. The point cloud data returned by the sensor consists of indiscriminate partial cylindrical patches complicating data association. The proposed method reconstructs an egocentric local map through an optimization process on a nonlinear manifold, which is then fed into a constrained unscented Kalman filter. The proposed method easily adapts to different diameters, cross sections, and changes in center line curves. The proposed approach outperforms our previous contribution [T. Ozaslan, G. Loianno, J. Keller, C. J. Taylor, V. Kumar, J. M. Wozencraft, and T. Hood, “Autonomous navigation and mapping for …

10. U-net for mav-based penstock inspection: an investigation of focal loss in multi-class segmentation for corrosion identification

    Ty Nguyen, Tolga Ozaslan, Ian D Miller, and 7 more authors

    arXiv preprint arXiv:1809.06576, 2018

    Abs

    Periodical inspection and maintenance of critical infrastructure such as dams, penstocks, and locks are of significant importance to prevent catastrophic failures. Conventional manual inspection methods require inspectors to climb along a penstock to spot corrosion, rust and crack formation which is unsafe, labor-intensive, and requires intensive training. This work presents an alternative approach using a Micro Aerial Vehicle (MAV) that autonomously flies to collect imagery which is then fed into a pretrained deep-learning model to identify corrosion. Our simplified U-Net trained with less than 40 image samples can do inference at 12 fps on a single GPU. We analyze different loss functions to solve the class imbalance problem, followed by a discussion on choosing proper metrics and weights for object classes. Results obtained with the dataset collected from Center Hill Dam, TN show that focal loss function, combined with a proper set of class weights yield better segmentation results than the base loss, Softmax cross entropy. Our method can be used in combination with planning algorithm to offer a complete, safe and cost-efficient solution to autonomous infrastructure inspection.

11. Inertial velocity and attitude estimation for quadrotors

    James Svacha, Kartik Mohta, Michael Watterson, and 2 more authors

    2018

    Abs

    This work addresses the design and implementation of a filter that estimates the orientation of the body-fixed z axis and the velocity of a quadrotor UAV from the inertial measurement unit (IMU) given a known yaw. The velocity and attitude estimation is possible since the filter employs a linear drag model measuring the drag forces on the quadrotor through the IMU. These forces are functions of the robot’s velocity and attitude. In addition, the filter estimates the linear drag parameters and thrust coefficient for the propellers. These parameters may be fed back into a controller to improve tracking performance. Experimental results are used to validate the proposed approach.

12. Semi-dense visual-inertial odometry and mapping for quadrotors with swap constraints

    Wenxin Liu, Giuseppe Loianno, Kartik Mohta, and 2 more authors

    2018

    Abs

    Micro Aerial Vehicles have the potential to assist humans in real life tasks involving applications such as smart homes, search and rescue, and architecture construction. To enhance autonomous navigation capabilities these vehicles need to be able to create dense 3D maps of the environment, while concurrently estimating their own motion. In this paper, we are particularly interested in small vehicles that can navigate cluttered indoor environments. We address the problem of visual inertial state estimation, control and 3D mapping on platforms with Size, Weight, And Power (SWAP) constraints. The proposed approach is validated through experimental results on a 250 g, 22 cm diameter quadrotor equipped only with a stereo camera and an IMU with a computationally-limited CPU showing the ability to autonomously navigate, while concurrently creating a 3D map of the environment.

13. Autonomous flight and cooperative control for reconstruction using aerial robots powered by smartphones

    Giuseppe Loianno, Yash Mulgaonkar, Chris Brunner, and 5 more authors

    The International Journal of Robotics Research, 2018

    Abs

    Advances in consumer electronics products and the technology seen in personal computers, digital cameras, and smartphones phones have led to the price/performance ratio of sensors and processors falling dramatically over the last decade. In particular, many consumer products are packaged with small cameras, gyroscopes, and accelerometers, all sensors that are needed for autonomous robots in GPS-denied environments. The low mass and small form factor make them particularly well suited for autonomous flight with small flying robots. In this work, we present the first fully autonomous smartphone-based system for quadrotors. We show how multiple quadrotors can be stabilized and controlled to achieve autonomous flight in indoor buildings with application to smart homes, search and rescue, monitoring construction projects, and developing models for architecture design. In our work, the computation for …

14. Inertial velocity and attitude estimation for quadrotors: Supplementary material

    J Svacha, Kartik Mohta, Michael Watterson, and 2 more authors

    [Online]. Available, 2018

    Abs

    We now demonstrate that the parallel transport on S2 with the Levi-Civita connection corresponding to the metric induced by R3 is equivalent to eq.(19) of the parent document, assuming the vector is transported along the geodesic from p to q. Without loss of generality, we will assume p is the north pole (ie, the point [0 0 1] when the sphere is naturally embedded in

15. Fast, autonomous flight in GPS-denied and cluttered environments (vol 35, pg 101, 2018)

    Kartik Mohta, Michael Watterson, Yash Mulgaonkar, and 15 more authors

    JOURNAL OF FIELD ROBOTICS, 2018

2017

1. System for deployment of groups of unmanned micro aerial vehicles in GPS-denied environments using onboard visual relative localization

   Martin Saska, Tomas Baca, Justin Thomas, and 6 more authors

   Autonomous Robots, 2017

   Abs

   A complex system for control of swarms of micro aerial vehicles (MAV), in literature also called as unmanned aerial vehicles (UAV) or unmanned aerial systems (UAS), stabilized via an onboard visual relative localization is described in this paper. The main purpose of this work is to verify the possibility of self-stabilization of multi-MAV groups without an external global positioning system. This approach enables the deployment of MAV swarms outside laboratory conditions, and it may be considered an enabling technique for utilizing fleets of MAVs in real-world scenarios. The proposed visual-based stabilization approach has been designed for numerous different multi-UAV robotic applications (leader-follower UAV formation stabilization, UAV swarm stabilization and deployment in surveillance scenarios, cooperative UAV sensory measurement) in this paper. Deployment of the system in real-world scenarios …

2. Autonomous navigation and mapping for inspection of penstocks and tunnels with MAVs

   Tolga Özaslan, Giuseppe Loianno, James Keller, and 4 more authors

   IEEE Robotics and Automation Letters, 2017

   Abs

   In this paper, we address the estimation, control, navigation and mapping problems to achieve autonomous inspection of penstocks and tunnels using aerial vehicles with on-board sensing and computation. Penstocks and tunnels have the shape of a generalized cylinder. They are generally dark and featureless. State estimation is challenging because range sensors do not yield adequate information and cameras do not work in the dark. We show that the six degrees of freedom (DOF) pose and velocity can be estimated by fusing information from an inertial measurement unit (IMU), a lidar and a set of cameras. This letter discusses in detail the range-based estimation part while leaving the details of vision component to our earlier work. The proposed algorithm relies only on a model of the generalized cylinder and is robust to changes in shape of the tunnel. The approach is validated through real experiments …

3. Cooperative transportation using small quadrotors using monocular vision and inertial sensing

   Giuseppe Loianno and Vijay Kumar

   IEEE Robotics and Automation Letters, 2017

   Abs

   Micro aerial vehicles have the potential to assist humans in tasks such as manipulation and transportation for construction and humanitarian missions, beyond simply acquiring data and building maps. In this letter, we address the state estimation, control, and trajectory planning in cooperative transportation of structures, which are either too heavy or too big to be carried by small microvehicles. Specifically, we consider small quadrotors, each equipped only with a single camera and inertial measurement unit as a sensor. The key contributions are 1) a new approach to coordinated control, which allows independent control of each vehicle while guaranteeing the system’s stability and 2) a new cooperative localization scheme that allows each vehicle to benefit from measurements acquired by other vehicles. The latter relies on the vehicles exploiting the inherent rigid structure information to infer additional constraints …

4. Autonomous flight for detection, localization, and tracking of moving targets with a small quadrotor

   Justin Thomas, Jake Welde, Giuseppe Loianno, and 2 more authors

   IEEE Robotics and Automation Letters, 2017

   Abs

   In this letter, we address the autonomous flight of a small quadrotor, enabling tracking of a moving object. The 15-cm diameter, 250-g robot relies only on onboard sensors (a single camera and an inertial measurement unit) and computers, and can detect, localize, and track moving objects. Our key contributions include the relative pose estimate of a spherical target as well as the planning algorithm, which considers the dynamics of the underactuated robot, the actuator limitations, and the field of view constraints. We show simulation and experimental results to demonstrate feasibility and performance, as well as robustness to abrupt variations in target motion.

5. Estimation, control, and planning for high speed flight with a quadrotor using on-board sensing

   Kartik Mohta, S Liu, Michael Watterson, and 2 more authors

   Journal of Field Robotics, 2017

   Abs

   Estimation, control, and planning for high speed flight with a quadrotor using on-board sensing - NYU Scholars Skip to main navigation Skip to search Skip to main content NYU Scholars Home NYU Scholars Logo Help & FAQ Home Profiles Research units Research output Search by expertise, name or affiliation Estimation, control, and planning for high speed flight with a quadrotor using on-board sensing Kartik Mohta, S. Liu, Michael Watterson, Giuseppe Loianno, Vijay Kumar Electrical and Computer Engineering Research output: Contribution to journal › Article › peer-review Overview Original language English (US) Journal Journal of Field Robotics State Published - 2017 Cite this APA Standard Harvard Vancouver Author BIBTEX RIS Powered by Pure, Scopus & Elsevier Fingerprint Engine™ All content on this site: Copyright © 2025 NYU Scholars, its licensors, and contributors. All rights are reserved, including …

6. Autonomous perching and grasping for micro aerial vehicles

   Justin Thomas, Giuseppe Loianno, Kostas Daniilidis, and 1 more author

   SPIE Newsroom, 2017

   Abs

   The ability to maneuver micro aerial vehicles (MAVs) precisely relative to specific targets and to interact with the environment (ie, aerial manipulation) could benefit society by assisting with dangerous jobs, providing useful information, and improving the efficiency of many tasks. For example, precise relative positioning would allow for close inspections of bridges, cell towers, rooftops, or water towers. Aerial manipulation could improve or enable precision farming, construction, repairing structures, transportation of objects, automated recharging or battery replacement, environmental sampling, or perching to turn off motors and reduce power consumption. The prevalence of commercially available MAVs has risen rapidly, but platforms are currently limited to sensing and data collection tasks. Indeed, many manufacturers are producing aerial robots equipped with cameras. However, none are able to physically interact with objects. Thus, there is a need for solutions empowering aerial robots to closely track, grasp, perch on, and manipulate specific objects of interest. Here, we present an overview of current approaches and challenges for vision-based perching and aerial manipulation. A more extensive discussion is available elsewhere. 1Many existing perching and grasping methods assume that the states of the robot and target are known, 2–9 which is a poor assumption and motivates the search for solutions using onboard sensors. Visual-inertial approaches are appealing because the sensors are lightweight, complement each other well, and are sufficient for navigation in unknown environments. 10, 11 However, in these cases, the vehicle is …

7. Index of papers published in the IEEE Robotics and Automation Letters and presented at IEEE/RSJ Int. Conf. on Intelligent Robots and Systems 2017 (IROS’17)

   Tolga Ozaslan, Giuseppe Loianno, James Keller, and 13 more authors

   2017

   Abs

   The following topics are dealt with: MAV; Android robot; augmented reality for kilobots; ARK; quadrotor; path planning; bipedal robots; adaptive depth control; micro diving agent; shape control; modular active-cell robots; mobile mixed-reality interaction; multi-robot systems; prestressed soft gripper; target-tracking game; time-delayed control; uncertain Euler-Lagrange systems; 7! Robots; recyclable robots; robot-integrated microfluidic chip; 3d-printed tactile gripper; humanoid robot; reinforcement learning; bioinspired continuum manipulator; musculoskeletal humanoids; feature-based matching; multimodal robotic skin; serial-link robots; feedback control; legged robots; ZMP constraints; state estimators; switching control; torque-controlled series-elastic actuators; RGB-D SLAM; reachability maps; co-safe temporal logic specifications; end effector; acceleration control; human-robot collaborative minimally invasive …

2016

1. Estimation, control, and planning for aggressive flight with a small quadrotor with a single camera and IMU

   Giuseppe Loianno, Chris Brunner, Gary McGrath, and 1 more author

   IEEE Robotics and Automation Letters, 2016

   Abs

   We address the state estimation, control, and planning for aggressive flight with a 150 cm diameter, 250 g quadrotor equipped only with a single camera and an inertial measurement unit (IMU). The use of smartphone grade hardware and the small scale provides an inexpensive and practical solution for autonomous flight in indoor environments. The key contributions of this paper are: 1) robust state estimation and control using only a monocular camera and an IMU at speeds of 4.5 m/s, accelerations of over 1.5 g, roll and pitch angles of up to 90°, and angular rate of up to 800°/s without requiring any structure in the environment; 2) planning of dynamically feasible three-dimensional trajectories for slalom paths and flights through narrow windows; and 3) extensive experimental results showing aggressive flights through and around obstacles with large rotation angular excursions and accelerations.

2. Swarm distribution and deployment for cooperative surveillance by micro-aerial vehicles

   Martin Saska, Vojtěch Vonásek, Jan Chudoba, and 3 more authors

   Journal of Intelligent & Robotic Systems, 2016

   Abs

   The task of cooperative surveillance of pre-selected Areas of Interest (AoI) in outdoor environments by groups of closely cooperating Micro Aerial Vehicles (MAVs) is tackled in this paper. In the cooperative surveillance mission, finding distributions of the MAVs in the environment to properly cover the AoIs and finding feasible trajectories to reach the obtained surveillance locations from the initial depot are crucial tasks that have to be fulfilled. In addition, motion constraints of the employed MAVs, environment constraints (e.g. non-fly zones), and constraints imposed by localization of members of the groups need to be satisfied in the planning process. We formulate the task of cooperative surveillance as a single high-dimensional optimization problem to be able to integrate all these requirements. Due to numerous constraints that have to be satisfied, we propose to solve the problem using an evolutionary-based …

3. A distributed optimization framework for localization and formation control: Applications to vision-based measurements

   Roberto Tron, Justin Thomas, Giuseppe Loianno, and 2 more authors

   IEEE Control Systems Magazine, 2016

   Abs

   Multiagent systems have been a major area of research for the last 15 years. This interest has been motivated by tasks that can be executed more rapidly in a collaborative manner or that are nearly impossible to carry out otherwise. To be effective, the agents need to have the notion of a common goal shared by the entire network (for instance, a desired formation) and individual control laws to realize the goal. The common goal is typically centralized, in the sense that it involves the state of all the agents at the same time. On the other hand, it is often desirable to have individual control laws that are distributed, in the sense that the desired action of an agent depends only on the measurements and states available at the node and at a small number of neighbors. This is an attractive quality because it implies an overall system that is modular and intrinsically more robust to communication delays and node failures.

4. Visual-inertial direct SLAM

   Alejo Concha, Giuseppe Loianno, Vijay Kumar, and 1 more author

   2016

   Abs

   The so-called direct visual SLAM methods have shown a great potential in estimating a semidense or fully dense reconstruction of the scene, in contrast to the sparse reconstructions of the traditional feature-based algorithms. In this paper, we propose for the first time a direct, tightly-coupled formulation for the combination of visual and inertial data. Our algorithm runs in real-time on a standard CPU. The processing is split in three threads. The first thread runs at frame rate and estimates the camera motion by a joint non-linear optimization from visual and inertial data given a semidense map. The second one creates a semidense map of high-gradient areas only for camera tracking purposes. Finally, the third thread estimates a fully dense reconstruction of the scene at a lower frame rate. We have evaluated our algorithm in several real sequences with ground truth trajectory data, showing a state-of-the-art performance.

5. Embedded model predictive control of unmanned micro aerial vehicles

   Tomas Baca, Giuseppe Loianno, and Martin Saska

   2016

   Abs

   We propose a lightweight embedded system for stabilization and control of Unmanned Aerial Vehicles (UAVs) and particularly Micro Aerial Vehicles (MAVs). The system relies solely on onboard sensors to localize the MAV, which makes it suitable for experiments in GPS-denied environments. The system utilizes predictive controllers to find optimal control actions for the aircraft using only onboard computational resources. To show the practicality of the proposed system, we present several indoor and outdoor experiments with multiple quadrotor helicopters which demonstrate its capability of trajectory tracking and disturbance rejection.

6. Aggressive flight with quadrotors for perching on inclined surfaces

   Justin Thomas, Morgan Pope, Giuseppe Loianno, and 5 more authors

   Journal of Mechanisms and Robotics, 2016

   Abs

   Micro-aerial vehicles (MAVs) face limited flight times, which adversely impacts their efficacy for scenarios such as first response and disaster recovery, where it might be useful to deploy persistent radio relays and quadrotors for monitoring or sampling. Thus, it is important to enable micro-aerial vehicles to land and perch on different surfaces to save energy by cutting power to motors. We are motivated to use a downward-facing gripper for perching, as opposed to a side-mounted gripper, since it could also be used to carry payloads. In this paper, we predict and verify the performance of a custom gripper designed for perching on smooth surfaces. We also present control and planning algorithms, enabling an underactuated quadrotor with a downward-facing gripper to perch on inclined surfaces while satisfying constraints on actuation and sensing. Experimental results demonstrate the proposed techniques through …

7. Visual inertial odometry for quadrotors on SE (3)

   Giuseppe Loianno, Michael Watterson, and Vijay Kumar

   2016

   Abs

   The combination of on-board sensors measurements with different statistical characteristics can be employed in robotics for localization and control, especially in GPS-denied environments. In particular, most aerial vehicles are packaged with low cost sensors, important for aerial robotics, such as camera, a gyroscope, and an accelerometer. In this work, we develop a visual inertial odometry system based on the Unscented Kalman Filter (UKF) acting on the Lie group SE(3), such to obtain an unique, singularity-free representation of a rigid body pose. We model this pose with the Lie group SE(3) and model the noise on the corresponding Lie algebra. Moreover, we extend the concepts used in the standard UKF formulation, such as state uncertainty and modeling, to correctly incorporate elements that do not belong to an Euclidean space such as the Lie group members. In this analysis, we use the parallel transport …

8. Bearing-only formation control with auxiliary distance measurements, leaders, and collision avoidance

   Roberto Tron, Justin Thomas, Giuseppe Loianno, and 2 more authors

   2016

   Abs

   We address the controller synthesis problem for distributed formation control. Our solution requires only relative bearing measurements (as opposed to full translations), and is based on the exact gradient of a Lyapunov function with only global minimizers (independently from the formation topology). These properties allow a simple proof of global asymptotic convergence, and extensions for including distance measurements, leaders and collision avoidance. We validate our approach through simulations and comparison with other stateof-the-art algorithms.

9. A swarm of flying smartphones

   Giuseppe Loianno, Yash Mulgaonkar, Chris Brunner, and 5 more authors

   2016

   Abs

   In the last decade, consumer electronic devices such as smartphones, are packaged with small cameras, gyroscopes, and accelerometers, all sensors allowing autonomous deployment of aerial robots in GPS-denied environments. Our previous work [1], demonstrated the feasibility of using smartphones for autonomous flight. In many applications, there is a large interest to the use multiple autonomous aerial vehicles in a cooperative manner to speed up the operation of the mission. In this work, we present the first fully autonomous smartphone-based swarm of quadrotors. Multiple vehicles are able to plan safe trajectories avoiding inter-robot collisions, optimizing at the same time a given task and concurrently building in a cooperative manner a 3-D map of the environment. The sensing, sensor fusion, control, and planning are all done on an offthe- shelf Samsung Galaxy S5 smartphone using just the single camera …

10. Vision-based high-speed autonomous landing and cooperative objects grasping-towards the MBZIRC competition

    Martin Saska, Tomas Baca, Vojtech Spurny, and 5 more authors

    Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems-Visionbased High Speed Autonomous Navigation of UAVs (Workshop)(Daejeon), 2016

    Abs

    I. EXTENDED ABSTRACT The aim of this paper is to present a system being developed for the Mohamed Bin Zayed International Robotics Challenge 2017 (MBZIRC) by a team of Czech Technical University in Prague, University of Pennsylvania and University of Lincoln. The system designed for autonomous landing on a moving vehicle and autonomous collecting of color objects by a team of unmanned aerial vehicles-helicopters (UAVs) will be described with latest results achieved in realscale outdoor scenarios. Both of these challenges require flying in high speed and strongly rely on vision control feedback and therefore the proposed system and designed approaches of autonomous flying with visual servoing should be within interest of participants of the Vision-based High Speed Autonomous Navigation of UAVs workshop. In our workshop presentation, we would like to introduce description and results of computer vision approaches composed from a set of state-of-the-art techniques in a unique way to recognize reliably the landing pattern on the moving vehicle and color objects randomly distributed in a 100x100m workspace. This part of the presentation will be followed by a description of a unique model predictive (MPC) technique combined with a nonlinear low-level controller and an extended Kalman-filter-based state estimator of the UAV and the following vehicle. In addition to the new controller, which is designed with the aim to solve the particular tasks as soon as possible and therefore at high speed, the contribution of this work consists in an integration of various robotic approaches into a fully autonomous system for solving tasks …

11. The role of vision in perching and grasping for MAVs

    Justin Thomas, Giuseppe Loianno, Kostas Daniilidis, and 1 more author

    2016

    Abs

    In this work, we provide an overview of vision-based control for perching and grasping for Micro Aerial Vehicles. We investigate perching on at, inclined, or vertical surfaces as well as visual servoing techniques for quadrotors to enable autonomous perching by hanging from cylindrical structures using only a monocular camera and an appropriate gripper. The challenges of visual servoing are discussed, and we focus on the problems of relative pose estimation, control, and trajectory planning for maneuvering a robot with respect to an object of interest. Finally, we discuss future challenges to achieve fully autonomous perching and grasping in more realistic scenarios.

12. Vision-based fast navigation of micro aerial vehicles

    Giuseppe Loianno and Vijay Kumar

    2016

    Abs

    We address the key challenges for autonomous fast flight for Micro Aerial Vehicles (MAVs) in 3-D, cluttered environments. For complete autonomy, the system must identify the vehicle’s state at high rates, using either absolute or relative asynchronous on-board sensor measurements, use these state estimates for feedback control, and plan trajectories to the destination. State estimation requires information from different sensors to be fused, exploiting information from different, possible asynchronous sensors at different rates. In this work, we present techniques in the area of planning, control and visual-inertial state estimation for fast navigation of MAVs. We demonstrate how to solve on-board, on a small computational unit, the pose estimation, control and planning problems for MAVs, using a minimal sensor suite for autonomous navigation composed of a single camera and IMU. Additionally, we show that a …

13. 18 Distributed Control and Estimation of Robotic Vehicle Networks

    NISAR R AhMED, JORGE CORTES, SONIA MARTíNEZ, and 9 more authors

    IEEE Control Systems Magazine, 2016

    Abs

    Provides a listing of board members, committee members, editors, and society officers.

14. System for stabilization of micro aerial vehicle swarms using onboard visual relative localization

    Martin Saska, Tomáš Báča, Justin Thomas, and 6 more authors

    Autonomous Robots, 2016

2015

1. Visual servoing of quadrotors for perching by hanging from cylindrical objects

   Justin Thomas, Giuseppe Loianno, Kostas Daniilidis, and 1 more author

   IEEE robotics and automation letters, 2015

   Abs

   This letter addresses vision-based localization and servoing for quadrotors to enable autonomous perching by hanging from cylindrical structures using only a monocular camera. We focus on the problems of relative pose estimation, control, and trajectory planning for maneuvering a robot relative to cylinders with unknown orientations. We first develop a geometric model that describes the pose of the robot relative to a cylinder. Then, we derive the dynamics of the system, expressed in terms of the image features. Based on the dynamics, we present a controller, which guarantees asymptotic convergence to the desired image space coordinates. Finally, we develop an effective method to plan dynamically feasible trajectories in the image space, and we provide experimental results to demonstrate the proposed method under different operating conditions such as hovering, trajectory tracking, and perching.

2. Cooperative localization and mapping of MAVs using RGB-D sensors

   Giuseppe Loianno, Justin Thomas, and Vijay Kumar

   2015

   Abs

   The fusion of IMU and RGB-D sensors presents an interesting combination of information to achieve autonomous localization and mapping using robotic platforms such as ground robots and flying vehicles. In this paper, we present a software framework for cooperative localization and mapping while simultaneously using multiple aerial platforms. We employ a monocular visual odometry algorithm to solve the localization task, where the depth data flow associated to the RGB image is used to estimate the scale factor associated with the visual information. The current framework enables autonomous onboard control of each vehicle with cooperative localization and mapping. We present a methodology that provides both a sparse map generated by the monocular SLAM and a multiple resolution dense map generated by the associated depth. The localization algorithm and both 3D mapping algorithms work in …

3. Flying smartphones: Automated flight enabled by consumer electronics

   Giuseppe Loianno, Gareth Cross, Chao Qu, and 3 more authors

   IEEE Robotics & Automation Magazine, 2015

   Abs

   Consumer-grade technology seen in cameras and phones has led to the price-performance ratio falling dramatically over the last decade. We are seeing a similar trend in robots that leverage this technology. A recent development is the interest of companies such as Google, Apple, and Qualcomm in high-end communication devices equipped with such sensors as cameras and inertial measurement units (IMUs) and with significant computational capability. Google, for instance, is developing a customized phone equipped with conventional as well as depth cameras. This article explores the potential for the rapid integration of inexpensive consumer-grade electronics with the off-the-shelf robotics technology for automation in homes and offices. We describe how standard hardware platforms (robots, processors, and smartphones) can be integrated through simple software architecture to build autonomous …

4. Smartphones power flying robots

   Giuseppe Loianno, Yash Mulgaonkar, Chris Brunner, and 5 more authors

   2015

   Abs

   Consumer grade technology seen in cameras and phones has led to the price/performance ratio of sensors and processors falling dramatically over the last decade. In particular, most devices are packaged with a camera, a gyroscope, and an accelerometer, important sensors for aerial robotics. The low mass and small form factor make them particularly well suited for autonomous flight with small flying robots, especially in GPS-denied environments. In this work, we present the first fully autonomous smartphone-based quadrotor. All the computation, sensing and control runs on an off-the-shelf smartphone, with all the software functionality in a smartphone app.We show how quadrotors can be stabilized and controlled to achieve autonomous flight in indoor buildings with application to smart homes, search and rescue, construction and architecture. The work allows any consumer with a smartphone to autonomously …

5. Planning and control of aggressive maneuvers for perching on inclined and vertical surfaces

   Justin Thomas, Giuseppe Loianno, Morgan Pope, and 5 more authors

   2015

   Abs

   It is important to enable micro aerial vehicles to land and perch on different surfaces to save energy by cutting power to motors and to perform tasks such as persistent surveillance. In many cases, the best available surfaces may be vertical windows, walls, or inclined roof tops. In this paper, we present approaches and algorithms for aggressive maneuvering to enable perching of underactuated quadrotors on surfaces that are not horizontal. We show the design of a custom foot/gripper for perching on smooth surfaces. Then, we present control and planning algorithms for maneuvering to land on specified surfaces while satisfying constraints on actuation and sensing. Experimental results that include successful perching on vertical, glass surfaces validate the proposed techniques.

6. Aerial service vehicles for industrial inspection: task decomposition and plan execution

   Jonathan Cacace, Alberto Finzi, Vincenzo Lippiello, and 2 more authors

   Applied Intelligence, 2015

   Abs

   This work proposes a high-level control system designed for an Aerial Service Vehicle capable of performing complex tasks in close and physical interaction with the environment in an autonomous manner. We designed a hybrid control architecture which integrates task, path, motion planning/replanning, and execution monitoring. The high-level system relies on a continuous monitoring and planning cycle to suitably react to events, user interventions, and failures, communicating with the low level control layers. The system has been assessed on real-world and simulated scenarios representing an industrial environment.

2014

1. Toward image based visual servoing for aerial grasping and perching

   Justin Thomas, Giuseppe Loianno, Koushil Sreenath, and 1 more author

   2014

   Abs

   This paper addresses the dynamics, control, planning, and visual servoing for micro aerial vehicles to perform high-speed aerial grasping tasks. We draw inspiration from agile, fast-moving birds, such as raptors, that detect, locate, and execute high-speed swoop maneuvers to capture prey. Since these grasping maneuvers are predominantly in the sagittal plane, we consider the planar system and present mathematical models and algorithms for motion planning and control, required to incorporate similar capabilities in quadrotors equipped with a monocular camera. In particular, we develop a dynamical model directly in the image space, show that this is a differentially-flat system with the image features serving as flat outputs, outline a method for generating trajectories directly in the image feature space, develop a geometric visual controller that considers the second order dynamics (in contrast to most visual …

2. Autonomous deployment of swarms of micro-aerial vehicles in cooperative surveillance

   Martin Saska, Jan Chudoba, Libor Přeučil, and 5 more authors

   2014

   Abs

   An algorithm for autonomous deployment of groups of Micro Aerial Vehicles (MAVs) in the cooperative surveillance task is presented in this paper. The algorithm enables to find a proper distributions of all MAVs in surveillance locations together with feasible and collision free trajectories from their initial position. The solution of the MAV-group deployment satisfies motion constraints of MAVs, environment constraints (non-fly zones) and constraints imposed by a visual onboard relative localization. The onboard relative localization, which is used for stabilization of the group flying in a compact formation, acts as an enabling technique for utilization of MAVs in situations where an external local system is not available or lacks the sufficient precision.

3. Toward autonomous avian-inspired grasping for micro aerial vehicles

   Justin Thomas, Giuseppe Loianno, Joseph Polin, and 2 more authors

   Bioinspiration & biomimetics, 2014

   Abs

   Micro aerial vehicles, particularly quadrotors, have been used in a wide range of applications. However, the literature on aerial manipulation and grasping is limited and the work is based on quasi-static models. In this paper, we draw inspiration from agile, fast-moving birds such as raptors, that are able to capture moving prey on the ground or in water, and develop similar capabilities for quadrotors. We address dynamic grasping, an approach to prehensile grasping in which the dynamics of the robot and its gripper are significant and must be explicitly modeled and controlled for successful execution. Dynamic grasping is relevant for fast pick-and-place operations, transportation and delivery of objects, and placing or retrieving sensors. We show how this capability can be realized (a) using a motion capture system and (b) without external sensors relying only on onboard sensors. In both cases we describe the …

4. Vision-based formation control of aerial vehicles

   Roberto Tron, Justin Thomas, Giuseppe Loianno, and 3 more authors

   2014

   Abs

   We propose a general solution for the problem of distributed, vision-based formation control of aerial vehicles. Our solution is based on pure bearing measurements, optionally augmented with the corresponding distances. As opposed to the state of the art, our control law does not require auxiliary distance measurements or estimators, it can be applied to leaderless or leaderbased formations with arbitrary topologies, and it has global convergence guarantees. We validate our approach through simulations and experiments on a platform of three quadrotors.

5. The Role of Vision Algorithms for Micro Aerial Vehicles

   Giuseppe Loianno

   2014

   Abs

   The term robot derives from the term robota which means executive labour in Slav languages. As well, robotics is commonly defined as the science studying the intelligent connection between perception and action [104]. The first definition of robot was established by Asimov. He was inspired by science fiction and it was defined as the science which is based on three fundamental laws

2013

1. Robust pose estimation algorithm for wrist motion tracking

   Francesca Cordella, Francesco Di Corato, Giuseppe Loianno, and 2 more authors

   2013

   Abs

   The wrist plays a fundamental role in reaching and grasping actions, i.e. it guides the hand to the grasp position and adjusts its orientation on the basis of the grasping type and task. This paper proposes a novel, low-cost method for wrist pose estimation by using the Asus Xtion Pro Live motion sensing device and a robust marker-based tracking approach based on Unscented Kalman Filter (UKF). The hand palm kinematic model is also considered. The applicability of the approach to evaluate some interesting kinematics parameters, such as position, orientation, Range Of Motion, angular and linear velocity and trajectory has been proved. In particular, since the nature of the paper is to present a novel approach for wrist pose estimation, only initial validation for wrist kinematic measurement will be reported.

2. Aerial Service Vehicles for Industrial Inspection: Task Decomposition and Plan Execution

   Jonathan Cacace, Alberto Finzi, Vincenzo Lippiello, and 2 more authors

   2013

   Abs

   We propose an autonomous control system for Aerial Service Vehicles capable of performing inspection tasks in buildings and industrial plants. In this paper, we present the applicative domain, the high-level control architecture along with some empirical results. The system has been assessed on real-world and simulated scenarios representing an industrial environment.

3. Fast localization and 3D mapping using an RGB-D sensor

   Giuseppe Loianno, Vincenzo Lippiello, and Bruno Siciliano

   2013

   Abs

   Low-cost range sensors represent an interesting class of sensors which are increasingly used for localization and mapping purposes in robotics. The combination of depth data and visual information can be employed to develop reliable algorithms for localization and environment mapping. A real-time approach combining a monocular visual odometry algorithm and range depth data is proposed in this paper. The scale factor problem is solved combining the depth data flow and the monocular image data. Moreover, a multiple resolution approach led by the distance of the sensor from surrounding obstacles is proposed for the depth data acquisition process. The visual egomotion estimation algorithm and the 3D map generation work in parallel improving the system realtime reliability. Experimental results show how the proposed integrated framework is able to localize in real-time the device in an unknown …

4. Visual and inertial multi-rate data fusion for motion estimation via Pareto-optimization

   Giuseppe Loianno, Vincenzo Lippiello, Carlo Fischione, and 1 more author

   2013

   Abs

   Motion estimation is an open research field in control and robotic applications. Sensor fusion algorithms are generally used to achieve an accurate estimation of the vehicle motion by combining heterogeneous sensors measurements with different statistical characteristics. In this paper, a new method that combines measurements provided by an inertial sensor and a vision system is presented. Compared to classical modelbased techniques, the method relies on a Pareto optimization that trades off the statistical properties of the measurements. The proposed technique is evaluated with simulations in terms of computational requirements and estimation accuracy with respect to a classical Kalman filter approach. It is shown that the proposed method gives an improved estimation accuracy at the cost of a slightly increased computational complexity.

5. Integrated planning and execution for an aerial service vehicle

   Jonathan Cacace, Alberto Finzi, Vincenzo Lippiello, and 2 more authors

   2013

   Abs

   Integrated planning and execution for an Aerial Service Vehicle - NYU Scholars Skip to main navigation Skip to search Skip to main content NYU Scholars Home NYU Scholars Logo Help & FAQ Home Profiles Research units Research output Search by expertise, name or affiliation Integrated planning and execution for an Aerial Service Vehicle Jonathan Cacace, Alberto Finzi, Vincenzo Lippiello, Giuseppe Loianno, Dario Sanzone Electrical and Computer Engineering Research output: Chapter in Book/Report/Conference proceeding › Conference contribution Overview Original language English (US) Title of host publication 23rd International conference on automated planning and scheduling Subtitle of host publication Workshop on planning and robotics Place of Publication Rome, Italy State Published - Jun 2013 Cite this APA Standard Harvard Vancouver Author BIBTEX RIS Powered by Pure, Scopus & Elsevier …

6. Vision based navigation, grasping, and localization for micro aerial vehicles

   Giuseppe Loianno, Justin Thomas, Kartik Mohta, and 1 more author

   2013

   Abs

   Vision based navigation, grasping, and localization for micro aerial vehicles - NYU Scholars Skip to main navigation Skip to search Skip to main content NYU Scholars Home NYU Scholars Logo Help & FAQ Home Profiles Research units Research output Search by expertise, name or affiliation Vision based navigation, grasping, and localization for micro aerial vehicles Giuseppe Loianno, Justin Thomas, Kartik Mohta, Shaojie Shen Electrical and Computer Engineering Research output: Chapter in Book/Report/Conference proceeding › Conference contribution Overview Original language English (US) Title of host publication IROS 2013 Subtitle of host publication Workshop on vision based closed-loop control and navigation of Micro Helicopters in GPS-denied Environments State Published - Nov 2013 Cite this APA Standard Harvard Vancouver Author BIBTEX RIS Powered by Pure, Scopus & Elsevier Fingerprint …

7. From Autonomous Grasping and Navigation to Cooperative Localization for Micro Aerial Vehicles

   Giuseppe Loianno, Justin Thomas, Kartik Mohta, and 2 more authors

   2013

   Abs

   This talk will provide an overview of approaches to grasping, autonomous navigation and cooperative mapping by aerial robots with an emphasis on micro UAVs. It will also discuss the scientific and technological challenges and outline directions for future research in these fields.

8. Avian-Inspired Grasping for Quadrotor MAVs

   Justin Thomas, Joe Polin, Giuseppe Loianno, and 2 more authors

   Robotics: Science and Systems, Workshop on Aerial Mobile Manipulation, 2013

   Abs

   Micro Aerial Vehicles (MAVs) have been used in a wide range of applications [9, 7]. However, there are few papers addressing high-speed grasping and transportation of payloads using MAVs. We are interested in dynamic acquisition of targets using MAVs. Drawing inspiration from aerial hunting by birds of prey, we designed and equiped a quadrotor MAV with an actuated appendage which enabled grasping and object retrieval at high speeds. We developed a nonlinear dynamic model of the system, demonstrated that the system is differentially flat, planned dynamic trajectories using the flatness property, and presented experimental results with pick-up velocities at 2 m/s (6 body lengths/second) and 3 m/s (9 body lengths/second).

2011

1. MAV indoor navigation based on a closed-form solution for absolute scale velocity estimation using optical flow and inertial data

   Vincenzo Lippiello, Giuseppe Loianno, and Bruno Siciliano

   2011

   Abs

   A new vision-based obstacle avoidance technique for indoor navigation of Micro Aerial Vehicles (MAVs) is presented in this paper. The vehicle trajectory is modified according to the obstacles detected through the Depth Map of the surrounding environment, which is computed online using the Optical Flow provided by a single onboard omnidirectional camera. An existing closed-form solution for the absolute-scale velocity estimation based on visual correspondences and inertial measurements is generalized and here employed for the Depth Map estimation. Moreover, a dynamic region-of-interest for image features extraction and a self-limitation control for the navigation velocity are proposed to improve safety in view of the estimated vehicle velocity. The proposed solutions are validated by means of simulations.

2. The AIRobots (Innovative aerial service robots for remote inspections by contact) Project

   Vincenzo Lippiello, Francesco Donnarumma, Giuseppe Loianno, and 5 more authors

   2011

   Abs

   The AIRobots (Innovative aerial service robots for remote inspections by contact) Project IRIS IRIS Home Sfoglia Macrotipologie & tipologie Autore Titolo Riviste Serie IT Italiano Italiano English English LOGIN 1.IRIS 2.Catalogo Prodotti Ricerca 3.2 Contributo in volume (exArticolo su libro) 4.2.1.2 Articolo su libro con ISBN The AIRobots (Innovative aerial service robots for remote inspections by contact) Project BASILE, FRANCESCO;CHIACCHIO, Pasquale; 2011-01-01 Scheda breve Scheda completa Scheda completa (DC) Anno 2011 ISBN 9788895028811 Appare nelle tipologie: 2.1.2 Articolo su libro con ISBN File in questo prodotto: Non ci sono file associati a questo prodotto. I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione. Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11386/3035657 Attenzione Attenzione! I …