NTRS - NASA Technical Reports Server

Back to Results
Multi-Objective Reinforcement Learning-Based Deep Neural Networks for Cognitive Space CommunicationsFuture communication subsystems of space exploration missions can potentially benefit from software-defined radios (SDRs) controlled by machine learning algorithms. In this paper, we propose a novel hybrid radio resource allocation management control algorithm that integrates multi-objective reinforcement learning and deep artificial neural networks. The objective is to efficiently manage communications system resources by monitoring performance functions with common dependent variables that result in conflicting goals. The uncertainty in the performance of thousands of different possible combinations of radio parameters makes the trade-off between exploration and exploitation in reinforcement learning (RL) much more challenging for future critical space-based missions. Thus, the system should spend as little time as possible on exploring actions, and whenever it explores an action, it should perform at acceptable levels most of the time. The proposed approach enables on-line learning by interactions with the environment and restricts poor resource allocation performance through virtual environment exploration. Improvements in the multiobjective performance can be achieved via transmitter parameter adaptation on a packet-basis, with poorly predicted performance promptly resulting in rejected decisions. Simulations presented in this work considered the DVB-S2 standard adaptive transmitter parameters and additional ones expected to be present in future adaptive radio systems. Performance results are provided by analysis of the proposed hybrid algorithm when operating across a satellite communication channel from Earth to GEO orbit during clear sky conditions. The proposed approach constitutes part of the core cognitive engine proof-of-concept to be delivered to the NASA Glenn Research Center SCaN Testbed located onboard the International Space Station.
Document ID
Document Type
Conference Paper
Ferreria, Paulo Victor R. (Worcester Polytechnic Inst. MA, United States)
Paffenroth, Randy (Worcester Polytechnic Inst. MA, United States)
Wyglinski, Alexander M. (Worcester Polytechnic Inst. MA, United States)
Hackett, Timothy M. (Pennsylvania State Univ. University Park, PA, United States)
Bilen, Sven G. (Pennsylvania State Univ. University Park, PA, United States)
Reinhart, Richard C. (NASA Glenn Research Center Cleveland, OH United States)
Mortensen, Dale J. (NASA Glenn Research Center Cleveland, OH United States)
Date Acquired
August 23, 2017
Publication Date
June 27, 2017
Subject Category
Space Communications, Spacecraft Communications, Command and Tracking
Report/Patent Number
Meeting Information
IEEE Cognitive Communications for Aerospace Applications (CCAA) Workshop(Cleveland, OH)
Funding Number(s)
WBS: WBS 553323.
Distribution Limits
Public Use Permitted.
space archtiecture
reinforcement learning
cognitive radio

Available Downloads

NameType 20170007958.pdf STI

Related Records

IDRelationTitle20170009153See AlsoMulti-Objective Reinforcement Learning-based Deep Neural Networks for Cognitive Space Communications