The effect of haptic feedback on operator control behaviour in telemanipulation

Telemanipulation systems - in 1925 a vision to remotelytreat patients, today widely adopted in a variety of applications - allow humanoperators to perform tasks which otherwise could not be performed, due to, forexample, limitations with respect to distance (e.g., space), scale (e.g.,surgery or micro-assembly) or hostile environments (e.g., subsea, nuclear).Effectively, a telemanipulation system functions as an extension to the humanoperator’s motor apparatus, in which the mapping between motor commands andhuman hand is shifted to a mapping between motor commands and slave robot.Haptic feedback, both proprioceptive and tactile, is often essential for motorcontrol and motor learning (i.e., building the `mappings'), but may bedistorted or even lost when not appropriately re-engineered.  There is, however, no consensus on how todesign haptic feedback to best enable humans to perform practicaltelemanipulated tasks, as no theory or integrated view for human-in-the-loopdesign and evaluation of haptic feedback is available. Empirically, we knowdesign guidelines `depend’ on aspects such as operator talent, training, thetype of task or application, quality of the visual feedback, or taskinstruction. As a result, the design and evaluation of a telemanipulationsystem is heuristic: for each case, the required quality of haptic feedback isdetermined by trial-and-error. This lacuna in design guidelines based onhuman-in-the-loop theory makes telemanipulation performance suboptimal, anddevelopment slow and costly.   The aim ofthis thesis is to provide an integrated, human-centered view on the design andevaluation of haptic feedback, which can serve as a basis for generalizedhaptic feedback design. More specifically, this thesis is on the one handfocused on (i) assessment of haptic feedback design requirements for positionand rate control within a uniform evaluation framework, and on the other on(ii) the development of a fundamental understanding of the role of hapticfeedback on operator (neuromuscular) control mechanisms, and moreover, togeneralize experimental findings by adapting existing motor-control paradigmsand control-theoretic models. To do so, four key human-factor experiments wereperformed. The first experiment focused on the benefit of haptic feedback forposition controlled telemanipulation scenarios and the impact of taskinstruction and availability of visual feedback for several fundamentalsubtasks. In a second experiment the efficacy of four different haptic feedbackinterface designs for rate control was determined in a similar manner; bothstudies adopted a uniform evaluation framework, providing an integrated view onrequirements for the haptic feedback.  Wefound that such a framework should incorporate at least a (abstract) tasktaxonomy, a baseline to compare against, task instruction, speed-accuracytrade-offs (i.e. what metrics to look at), performance-control efforttrade-offs, operator training, and a control on the quality of visual feedback.Furthermore, these studies showed that the best haptic feedback design toperform a given telemanipulation task predominantly depends on the requiredtask workspace and task accuracy, and the need to reflect back contacttransitions. Large workspaces are more easily (i.e. low workload) covered usingrate control, where accuracy for positions and forces is higher using positioncontrol. Also, as an increase in device (i.e. haptic feedback) quality does notalways correlate to an increase in task performance. This implies design ofhaptic feedback should be human-centered evaluation, both assessing the problemand validating the solution with the human in-the-loop. Experiments three andfour focused on the effects of haptic feedback on the human operator’s motorcontrol mechanisms when controlling a telemanipulation system in free-space. Instudy three, well-established cybernetic models were adopted to study trainedmovements, and the impact of slave dynamics and scaling of haptic feedback. Inthe final study, a reach-adaptation paradigm was used to study the role ofhaptic feedback when learning movements, and the impact of slave dynamics and bandwidthof the presented haptic feedback.  Theselatter two experiments show that haptic feedback substantially affects anoperator’s underlying motor control mechanisms (i.e. feedback and feedforwardcontrol) when controlling a slave system. The effects were observed in bothinstantaneous improvements of task execution due to feedback of environmentalforces or device dynamics, as well as also task execution improvements overlonger periods of time due to improved internal models (i.e. learning); hapticfeedback enhances the process of building ‘mappings’ between human input and asystem’s response. This suggests that improved haptic feedback quality improveslearning rates (i.e. efficacy) and control responses (i.e. efficiency). Futurestudies should uncover the potential quantitative effects and time-scales atwhich these effects occur.  Additionally,study three showed that the amplitude of haptic feedback can be scaled downwithout harming task performance: human operators are capable of adjustingtheir (neuromuscular) control parameters independently of the absolutemagnitude (i.e. gain) of the haptic feedback controller. However, when scaling,one should account for reasonable lower boundaries, that putatively may begiven by Just Noticeable Differences (JNDs) to keep cues distinguishable. Upperboundaries may be given by individual constraints on comfort. These findingswere confirmed by the second experiment.  Studies three and four illustrate thatcomputational models and paradigms from the motor control literature can beadopted to provide generalizable descriptions of human operator behavior intelemanipulation. Here, we targeted free-space motions for systems like cranesand robot arms, and the tasks are representative for activities in domestic,nuclear or subsea environments. The cybernetic models enable for an exclusiveunderstanding of the underlying operator control mechanisms (i.e. feedback andfeedforward control) by looking in the frequency domain, as such complementingand enhancing the insights gained from the time-domain data. The reachadaptation paradigm enables to determine the extent to which haptic feedbackbandwidth affects motor learning and generalization for different slavedynamics. Moreover, these model-based approaches enable extrapolation offindings and to predict outcomes when task characteristics change, such thatinformed a priori design considerations of haptic feedback interfaces and, inthe future, haptic support systems can be made.