Tactile and Slip Sensation Acquisition in Prosthetic Hands and Proprioceptive Feedback of Perception for Arm Amputees

    

    

    Figure 1:  Closed-loop control with perception feedback for prostheses (EEG:
Electroencephalography).
    
    

Several surveys have been performed on different user groups about their prior demands for prosthesis functions [8]. Biddiss et al [9] showed that a perception feedback was recognized as the design priority for motorized prosthesis by amputees. A survey on myoelectric prosthesis users by Pylatiuk et al [10] reported that more than 95% of questioned male individuals wanted force feedbacks in their prosthetic hands. In another survey specified on the demands for sensation feedback in upper extremity prostheses by Lewis et al [11], 88% of questioned persons placed different degrees of importance on acquiring perception feedback from their prostheses, where grip force was most absolutely important for amputees’ demands, followed by proprioceptive information of prosthesis movement and position; perception of first contact during object grasping and end of contact during object releasing, as well as touching without grip, were also considered as very important and useful sensation information of prosthetic hands. Almost all the reported surveys have revealed a strong requirement for touching and/or grasping sensation feedback by prosthetic hand users, indicating the fundamentality of forcerelated perceptions inhuman hands. The information of touching, slipping, stress, or even material hardness, smoothness, and texture, might be elicited from the force-related perceptions. Although a temperature perception in prosthetic hands should also be useful for amputees in doing daily activities, it may be relatively less concerned by most of prosthetic hand users.

Tactile and Slip Sensation Acquisition in Prosthetic Hands

Generally, a tactile signal could be expressed as an acting force between a hand and an object, while a slip signal might be represented by the micro-vibrations that are generated due to the friction on contact surface. It is known that for an intact hand, the tactile and slip information is perceived with nervous system by detecting the sensation signals of force and vibration through glabrous skin receptors [12-13]. With an attempt to build artificial perceptions in prosthetic hands, some sensing techniques might be utilized to acquire the sensation signals.

Force-sensing resistors are a kind of widely used sensor with simple structure and interface, which show advantages of small thickness (usually less than 0.5 mm) and low cost. In the studies of Mingrino et al [14], Kyberd et al [15], and Tura et al [16], static sensors based on force-sensing resistors were designed and mounted in prosthetic hands to measure touching and/or grip strength signals. However, force-sensing resistors normally have relatively low measurement precision, which limits their further application in prosthetic hands where accurate signal detections are required. Strain gages are another type of commonly applied approach for force sensing. Wang et al [17] used a linkage to connect the thumb and finger of a prosthetic hand, in which the grasp force could be calculated by measuring the force acting upon the linkage with strain gauges. Maeno et al [18] developed a strain-distribution sensor for elastic fingers that was made of silicone rubber, where strain gages bonded on thin plates were arranged at uniform intervals inside the curved surface of fingers. Strain gages may achieve a good detection precision, but at the cost of relatively complicated structure design. In addition, both force-sensing resistors and strain gages may be suitable only for touching but not for slipping detection because of their working principles.

Some other force-sensing methods have also been investigated for applications in prosthetic hands. Hashimoto et al [19] proposed a tactile sensor for multi-fingered robot hands. The sensor had a silicone rubber cap with a cavity full of incompressible fluid, and the contract forces were transferred by the fluid to a semiconductor pressure gage. Shen et al [20] tested a fingertip tactile sensor basedon the optical total reflection principle for applications on a fivefingered hand system. Curcie et al [21] reported the use of myopneumatic sensors for measuring three-dimensional mechanical dynamics in a biomimetic finger control. Schmidt et al [22] presented a dynamic tactile sensor that consisted of a capacitive-sensor array for dexterous grasping with applications in human-robot interaction and object exploration. All the above sensing approaches may realize an acquisition of high quality tactile signals, however, the complicated operation principle and elaborated sensing structure would result in a lower reliability and increase the fabrication cost.

Recently, several new sensing techniques have been developed and considered as appropriate candidates for sensation signal acquisition in prosthetic hands. Heo et al [23] described two kinds of force sensor arrays using fiber Bragg gratings for distributed normal force detecting, one of which was with good sensitivity and spatial resolution, similar to human finger skin. Noda et al [24] exhibited a tactile sensor with standing piezoresistive cantilever array that could detect the directions and magnitudes of shear stress applied on its surface. Wisitsoraat et al [25] showed a thin-film based piezoresistive MEMS tactile sensor with optimized sensitivity for displacement and force sensing. Wen et al [26] demonstrated a three axes polymer tactile sensor. The sensor consisted of polymer membrane and four sensing cantilevers with piezoresistors to measure in-plane and outof- plane loads.

The measurement of slip signals is more complicated compared with that of tactile signals due to the difficulty in micro-vibration detection, and thus there are less research reports on this aspect. Miniature microphone was used by Kyberd et al [15] and optical sensor was applied by Tura et al [16] to acquire vibration signals in their prosthetic hands, respectively. Shang et al [27] developed a slip sensor based on photoelectric technology to detect micro-vibrations in prosthetic hands. Yamada et al [28] illustrated an artificial elastic finger skin that had ridges on surface to divide the stick/slip area, and slippage of the ridge could be detected. These slipping sensing approaches measure the micro-vibrations indirectly by means of voice, optical or some other signal types, which would increase the complexity of system and the difficulty in device design.

Piezoelectric materials can transfer mechanical signals directly into electric signals, which have a promising potential in sensation signal acquisition in prosthetic hands. Piezoelectric polyvinylidene fluoride film is a thin and soft polymer-based sensing material that can measure forces both in normal and tangential directions. In a prosthesis-applied sensor system suggested by Mingrino et al [14], slip signals could be successfully acquired by means of a piezoelectric polyvinylidene fluoride film. Although it is possible to collect both tactile and slip signals with a single piezoelectric polyvinylidene fluoride film, the low sensitivity and high cost prevent the further development of this material in signal acquisition for prosthetic hands. Polymer-based piezoelectrets are another type of novel piezoelectric material [29-30]. They are flexible and very thin, with a thickness of a few dozen micrometers, and show a strong piezoelectric response which is more than ten times larger than piezoelectric polyvinylidene fluoride. In addition, piezoelectrets are stretchable and low-cost. The polymer-based characteristics make them very suitable for embedding on artificial skins. Combined with their promising sensing and material properties, piezoelectrets have been experimented to acquire both tactile and slip signals in prosthetic hands [31], and more researches are in progress for possible actual applications.

Proprioceptive Feedback of Perception for Arm Amputees

It would be essential that a prosthetic hand could accurately detect possible tactile and/or slip information with some elaborate sensing techniques; more importantly, the arm amputees would be desired to proprioceptively perceive sensations from their prosthetic hands. It is a big challenge to transfer the sensor-detected sensation signals into users’ nerve systems, and many efforts have been made in previous studies with an attempt to develop appropriate modalities for sensation message transmission.

For currently developed techniques, stimulations on residual limb surface, including vibration, temperature, pressure, or even more, are identified as the well-received feedback means based on amputees’ personal acceptance and sensitivity of their residual limbs [11]. Among them, the vibration feedback is mostly of interest for applications [32], because it can easily be realized by mounting a low-cost consumer vibration motor on residual limbs. However, these feedback methods would be unintuitive and indirect because a vibration or temperature feedback is a different sensation from touching or slipping. Thus the amputees have to take a long-time training to be accustomed to these indirect feedbacks. In addition, they are often described as distracting due to the possible interference on regular body activities.

A direct neural interface may provide intuitive and accurate perception feedback instead of the stimulations on body surface such as mechanical vibration. It has been reported that electrical stimulation of somatosensory cortex [33-34] may elicit reproducible perceptions for amputees. By using microneurographic technique of intraneural microstimulation, Moore et al [35] studied the functional consequences of topographic reorganization within the human somatosensory cortex. A brain-machine-brain interface was set up by O’Doherty et al [36] for active tactile exploration, which allowed the signaling of artificial tactile feedback through intracortical microstimulation of primary somatosensory cortex. Their experiments on two monkeys suggested that the intracortical microstimulation feedback might generate somatic perceptions associated with mechanical, robotic or even virtual prostheses. Berg et al [37] implemented and tested a somatosensory prosthesis with sensorized finger on rhesus macaques. By means of intracortical microstimulation, perceptions were delivered to primary somatosensory cortex through chronically implanted multi-electrode arrays, and the perception feedback magnitude was graded according to the force applied on finger.

Electrical stimulation of peripheral never [38] is another possible approach to regenerate perception feedback for limb amputees. Dhillon et al [39] implanted longitudinal intrafascicular electrodes within individual fascicles of peripheral nerve stumps in amputees. Their work demonstrated that electrical stimulation through theelectrodes could produce graded, discrete, and distally referred sensory feedback of touch or movement. In a recent work conducted by Raspopovic et al [40], transversal multichannel intrafascicular electrodes were utilized to stimulate median and ulnar nerve fascicles, according to the sensation message obtained by artificial sensors mounted on a prosthetic hand. It was shown that real-time and near-nature perceptions could be restored for an amputee during the prosthetic hand operation for various grasping tasks. In addition, the results demonstrated that the user could identify the stiffness and shape of objects by exploiting the different characteristics of restored sensations. Tan et al [41] also implanted peripheral nerve interfaces in two human subjects with upper-limb amputation. By electrical stimulation through non-penetrating peripheral nerve cuff electrodes, natural and repeatable touch perceptions of tapping, pressure, moving touch, and vibration were reproduced in the subjects and stable for more than one year. The correlation between perception types and stimulation patterns was investigated in the work, as well.

Compared with the indirect sensation feedback on residual limb surface by means of vibration or temperature, both electrical stimulation of somatosensory cortex and peripheral never could provide a proprioceptive sensation feedback for limb amputees. In addition, the feedback through stimulation of peripheral never might be more accurate than that through stimulation of somatosensory cortex, as demonstrated by the up-to-date researches reviewed above. On the other side, however, the electric stimulations of cortex and never require a second surgery, and thus the risk and cost should be taken into account. The bio-compatibility of stimulation electrodes might be another issue, which should be considered before the wide acceptance of these feedback techniques.

Summary

Natural perceptions of external environment are necessary for arm amputees to operate their prosthetic hands independently of visual and/or auditory feedback. Various developed artificial sensing techniques may be integrated in prosthetic hands to acquire sensation information of, for example, touching and slipping. Proprioceptive perception feedback for prosthesis users is very critical to achieve an intuitive control of prostheses. Feedback through the stimulation on residual limb surface such as vibration or temperature is easy to realize, but still unintuitive. Both electrical stimulation of somatosensory cortex and peripheral never may achieve a direct and intuitive perception feedback for arm amputees, as already proved by several pilot studies. But more research work and clinical verification should be performed before actual applications.

Acknowledgment

This work was partly supported by the National Key Basic Research Program of China (#2013CB329505), the National Natural Science Foundation of China (#61203209, #61135004), the Shenzhen Peacock Plan Grant (#KQCX20130628112914295), the Shenzhen Governmental Basic Research Grant (#JCYJ20120617114419018), and the Guangdong Innovation Research Team Fund for Low-cost Healthcare Technologies.

References

1. Parker PA, Scott RN. Myoelectric control of prostheses. Crit Rev Biomed Eng. 1986; 13: 283-310.
2. Hochberg LR, Serruya MD, Friehs GM, Mukand JA, Saleh M, Caplan AH, et al. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature. 2006; 442: 164-171.
3. Hoffer JA, Loeb GE. Implantable electrical and mechanical interfaces with nerve and muscle. Ann Biomed Eng. 1980; 8: 351-360.
4. Kuiken TA, Li G, Lock BA, Lipschutz RD, Miller LA, Stubblefield KA, et al. Targeted muscle reinnervation for real-time myoelectric control of multifunction artificial arms. JAMA. 2009; 301: 619-628.
5. Fang P, Wei Z, Geng Y, Yao F, Li G. Using speech for mode selection in control of multifunctional myoelectric prostheses. Proceedings of Annual International Conference of the IEEE Engineering in Medicine and Biology Society; 2013 July 3-7; Osaka, Japan. New York: IEEE, 2013.
6. Rohland TA. Sensory feedback for powered limb prostheses. Med Biol Eng. 1975; 13: 300-301.
7. Childress DS. Closed-loop control in prosthetic systems: historical perspective. Ann Biomed Eng. 1980; 8: 293-303.
8. Atkins D, Heard D, Donovan W. Epidemiologic overview of individuals with upper limb loss and their reported research priorities. J ProsthetOrthot. 1996; 8: 2-11.
9. Biddiss E, Beaton D, Chau T. Consumer design priorities for upper limb prosthetics. Disabil Rehabil Assist Technol. 2007; 2: 346-357.
10. Pylatiuk C, Schulz S, Döderlein L. Results of an Internet survey of myoelectric prosthetic hand users. Prosthet Orthot Int. 2007; 31: 362-370.
11. Lewis S, Russold MF, Dietl H, Kaniusas E. User demands for sensory feedback in upper extremity prostheses. Proceedings of 2012 IEEE International Symposium on Medical Measurements and Applications Proceedings (Me Me A); 2012 May 18-19; Budapest, Hungary. New York: IEEE, 2013.
12. Johansson RS, Westling G. Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects. Exp Brain Res. 1984; 56: 550-564.
13. Srinivasan MA, Whitehouse JM, LaMotte RH. Tactile detection of slip: surface microgeometry and peripheral neural codes. J Neurophysiol. 1990; 63: 1323-1332.
14. Mingrino A, Bucci A, Magni R, Dario P. Slippage control in hand prostheses by sensing grasping forces and sliding motion. Proceedings of IEEE/RSJ/GI International Conference on Intelligent Robots and Systems - Advanced Robotic Systems and the Real World; 1994 September 12-16; Munich, Germany. New York: IEEE, 1994.
15. Kyberd PJ, Holland OE, Chappell PH, Smith S, Tregidgo R, Bagwell PJ, et al. MARCUS: A two degree of freedom hand prosthesis with hierarchical grip control. IEEE Trans Rehabil Eng. 1995; 3: 70-76.
16. Tura A, Lamberti C, Davalli A, Sacchetti R. Experimental development of a sensory control system for an upper limb myoelectric prosthesis with cosmetic covering. J Rehabil Res Dev. 1998; 35: 14-26.
17. Wang G, Zhang X, Zhang J, Gruver WA. Gripping force sensory feedback for a myoelectrically controlled forearm prosthesis. Proceedings of 1995 IEEE International Conference on Systems, Man and Cybernetics - Intelligent Systems for the 21st-Century; 1995 October 22-25; Vancouver, Canada. New York: IEEE, 1995.
18. Maeno T, Kawai T, Kobayashi K. Analysis and design of a tactile sensor detecting strain distribution inside an elastic finger. Proceedings of 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems; 1998 October 13-17; Victoria, Canada. New York: IEEE, 1998.
19. Hashimoto H, Ogawa H, Obama M. Development of a multi-fingered robot hand with fingertip tactile sensors. Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems; 1993 July 26-30; Yokohama, Japan. New York: IEEE, 1993.
20. Shen Y, Liu Y, Li K. Haptic tactile feedback in teleoperation of a multifingered robot hand. Proceedings of 3rd World Congress on Intelligent Control and Automation; 2000 June 28-July 02; Hefei, China. New York: IEEE, 2000.
21. Curcie DJ, Flint JA, Craelius W. Biomimetic finger control by filtering of distributed forelimb pressures. IEEE Trans Neural Syst Rehabil Eng. 2001; 9: 69-75.
22. Schmidt PA, Mael E, Wurtz RP. A sensor for dynamic tactile information with applications in human-robot interaction and object exploration. Robot Auton Syst. 2006; 54: 1005-1014.
23. Heo J, Chung J, Lee J. Tactile sensor arrays using fiber Bragg grating sensors. Sens Actuator A-Phys. 2006; 126: 312-327.
24. Noda K, Hoshino K, Matsumoto K, Shimoyama I. A shear stress sensor for tactile sensing with the piezoresistive cantilever standing in elastic material. Sens Actuator A-Phys. 2006; 127: 295-301.
25. Wisitsoraat A, Patthanasetakul V, Lomas T, Tuantranont A. Low cost thin film based piezoresistive MEMS tactile sensor. Sens Actuator A-Phys. 2007; 139: 17-22.
26. Wen C, Fang W. Tuning the sensing range and sensitivity of three axes tactile sensors using the polymer composite membrane. Sens Actuator A-Phys. 2008; 145-146: 14-22.
27. Shang Z, Han H, Deng X, Xu X. Output characteristics of photoelectric slip sensor based on micro-vibration detection. Semicond Photonics Tech. 2007; 13: 230-234.
28. Yamada D, Maeno T, Yamada Y. Artificial finger skin having ridges and distributed tactile sensors used for grasp force control. Proceedings of 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2001 October 29-November 03; Hawaii, USA. New York: IEEE, 2001.
29. Gerhard R. Less can be more: Holes in polymers lead to a new paradigm of piezoelectric materials for electret transducers. IEEE Trans DielectrElectrInsul. 2002; 9: 850-859.
30. Zhang X, Zhang X, You Q, Sessler G. Low-cost, large-area, stretchable piezoelectric films based on irradiation-crosslinkedpoly(propylene).Macromol Mater Eng. 2014; 299: 290-295.
31. Fang P, Tian L, Zheng Y, Huang J, Li G. Using thin-film piezoelectret to detect tactile and slip signals for restoring sensation of prosthetic hands. Proceedings of 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society; 2014 August 26-30; Chicago, USA. New York: IEEE, 2014.
32. Pylatiuk C, Kargov A, Schulz S. Design and evaluation of a low-cost force feedback system for myoelectric prosthetic hands. J ProsthetOrthot. 2006; 18: 57-61.
33. Penfield W, Boldrey E. Somatic motor and sensory representation in the cerebral cortex of man as studied by electrical stimulation. 1937; 60: 389-443.
34. Romo R, Hernández A, Zainos A, Salinas E. Somatosensory discrimination based on cortical microstimulation. Nature. 1998; 392: 387-390.
35. Moore CE, Schady W. Investigation of the functional correlates of reorganization within the human somatosensory cortex. Brain. 2000; 123 : 1883-1895.
36. O'Doherty JE, Lebedev MA, Ifft PJ, Zhuang KZ, Shokur S, Bleuler H, et al. Active tactile exploration using a brain-machine-brain interface. Nature. 2011; 479: 228-231.
37. Berg JA, Dammann JF 3rd, Tenore FV, Tabot GA, Boback JL, Manfredi LR, et al. Behavioral demonstration of a somatosensory neuroprosthesis. IEEE Trans Neural Syst Rehabil Eng. 2013; 21: 500-507.
38. Ochoa J, Torebjörk E. Sensations evoked by intraneural microstimulation of single mechanoreceptor units innervating the human hand. J Physiol. 1983; 342: 633-654.
39. Dhillon GS, Horch KW. Direct neural sensory feedback and control of a prosthetic arm. IEEE Trans Neural Syst Rehabil Eng. 2005; 13: 468-472.
40. Raspopovic S, Capogrosso M, Petrini FM, Bonizzato M, Rigosa J, Di Pino G, et al. Restoring natural sensory feedback in real-time bidirectional hand prostheses. Sci Transl Med. 2014; 6: 222ra19.
41. Tan DW, Schiefer MA, Keith MW, Anderson JR, Tyler J, Tyler DJ. A neural interface provides long-term stable natural touch perception. Sci Transl Med. 2014; 6: 257ra138.