PUBLICATIONS
A 3D Game Theoretical Framework for the Evaluation of Unmanned Aircraft Systems Airspace Integration Concepts
Predicting the outcomes of integrating Unmanned Aerial System (UAS) into the National Airspace System (NAS) is a complex problem, which is required to be addressed by simulation studies before allowing the routine access of UAS into the NAS. This paper focuses on providing a 3-dimensional (3D) simulation framework using a game-theoretical methodology to evaluate integration concepts using scenarios where manned and unmanned air vehicles co-exist. In the proposed method, the human pilot interactive decision-making process is incorporated into airspace models which can fill the gap in the literature where the pilot behavior is generally assumed to be known a priori. The proposed human pilot behavior is modeled using a dynamic level-k reasoning concept and approximate reinforcement learning. The level-k reasoning concept is a notion in game theory and is based on the assumption that humans have various levels of decision making. In the conventional “static” approach, each agent makes assumptions about his or her opponents and chooses his or her actions accordingly. On the other hand, in the dynamic level-k reasoning, agents can update their beliefs about their opponents and revise their level-k rule. In this study, Neural Fitted Q Iteration, which is an approximate reinforcement learning method, is used to model time-extended decisions of pilots with 3D maneuvers. An analysis of UAS integration is conducted using an example 3D scenario in the presence of manned aircraft and fully autonomous UAS equipped with sense and avoid algorithms.
Berat Mert Albaba, Negin Musavi, Yildiray Yildiz
[arXiv]
SyNet: An Ensemble Network for Object Detection in UAV Images
Recent advances in camera equipped drone applications and their widespread use increased the demand on vision based object detection algorithms for aerial images. Object detection process is inherently a challenging task as a generic computer vision problem, however, since the use of object detection algorithms on UAVs (or on drones) is relatively a new area, it remains as a more challenging problem to detect objects in aerial images. There are several reasons for that including: (i) the lack of large drone datasets including large object variance, (ii) the large orientation and scale variance in drone images when compared to the ground images, and (iii) the difference in texture and shape features between the ground and the aerial images. Deep learning based object detection algorithms can be classified under two main categories: (a) single-stage detectors and (b) multi-stage detectors. Both single-stage and multi-stage solutions have their advantages and disadvantages over each other. However, a technique to combine the good sides of each of those solutions could yield even a stronger solution than each of those solutions individually. In this paper, we propose an ensemble network, SyNet, that combines a multi-stage method with a single-stage one with the motivation of decreasing the high false negative rate of multi-stage detectors and increasing the quality of the single-stage detector proposals. As building blocks, CenterNet and Cascade R-CNN with pretrained feature extractors are utilized along with an ensembling strategy. We report the state of the art results obtained by our proposed solution on two different datasets: namely MS-COCO and visDrone with %52.1 mAP_{IoU = 0.75} is obtained on MS-COCO val2017 dataset and %26.2 mAP_{IoU = 0.75} is obtained on VisDrone test-set. Our code is available at: https://github.com/mertalbaba/SyNet
Berat Mert Albaba, Sedat Ozer
Accepted to 25th International Conference on Pattern Recognition (ICPR2020)
[Paper] [arXiv]
Driver Modeling through Deep Reinforcement Learning and Behavioral Game Theory
In this paper, a synergistic combination of deep reinforcement learning and hierarchical game theory is proposed as a modeling framework for behavioral predictions of drivers in highway driving scenarios. The need for a modeling framework that can address multiple human-human and human-automation interactions, where all the agents can be modeled as decision makers simultaneously, is the main motivation behind this work. Such a modeling framework may be utilized for the validation and verification of autonomous vehicles: It is estimated that for an autonomous vehicle to reach the same safety level of cars with drivers, millions of miles of driving tests are required. The modeling framework presented in this paper may be used in a high-fidelity traffic simulator consisting of multiple human decision makers to reduce the time and effort spent for testing by allowing safe and quick assessment of self-driving algorithms. To demonstrate the fidelity of the proposed modeling framework, game theoretical driver models are compared with real human driver behavior patterns extracted from traffic data.
Berat Mert Albaba, Yildiray Yildiz
In Review
Modeling Cyber-Physical Human Systems via an Interplay Between Reinforcement Learning and Game Theory
Predicting the outcomes of cyber-physical systems with multiple human interactions is a challenging problem. This article reviews a game theoretical approach to address this issue, where reinforcement learning is employed to predict the time-extended interaction dynamics. We explain that the most attractive feature of the method is proposing a computationally feasible approach to simultaneously model multiple humans as decision makers, instead of determining the decision dynamics of the intelligent agent of interest and forcing the others to obey certain kinematic and dynamic constraints imposed by the environment. We present two recent exploitations of the method to model 1) unmanned aircraft integration into the National Airspace System and 2) highway traffic. We conclude the article by providing ongoing and future work about employing, improving and validating the method. We also provide related open problems and research opportunities.
Berat Mert Albaba, Yildiray Yildiz
Annual Reviews in Control, Volume 48, 2019, 1-21
Stochastic Driver Modeling and Validation with Traffic Data
This paper describes a stochastic modeling approach for predicting driver responses in highway traffic. Different from existing approaches in the literature, the proposed modeling framework allows simultaneous decision making for multiple drivers (>100), in a computationally feasible manner, instead of modeling the decisions of an ego driver and assuming a predetermined driving pattern for other drivers in a given scenario. This is achieved by a unique combination of hierarchical game theory, which is used to model strategic decision making, and stochastic reinforcement learning, which is employed to model multi-move decision making. The proposed approach can be utilized to create high fidelity traffic simulators, which can be used to facilitate the validation of autonomous driving control algorithms by providing a safe and relatively fast environment for initial assessment and tuning. What makes the proposed approach appealing especially for autonomous driving research is that the driver models are strategic, meaning that their responses are based on predicted actions of other intelligent agents in the traffic scenario, where these agents can be human drivers or autonomous vehicles. Therefore, these models can be used to create traffic models with multiple human-machine interactions. To evaluate the fidelity of the framework, created stochastic driver models are compared with real driving patterns, processed from the traffic data collected by US Federal Highway Administration on US101 (Hollywood Freeway) on June 15th, 2005.
Berat Mert Albaba, Yildiray Yildiz, Nan Li, Ilya Kolmanovsky, Anouck Girard
American Control Conference (ACC), 2019