Microsoft Word HapticLinks CHI2018 FinalVersion


1 Haptic Links: Bimanual Haptics for Virtual Reality Using Variable Stiffness Actuation 2 1 2 2 2 Christian Holz Mike Sinclair Hrvoje Benko Eyal Ofek Evan Strasnick 1 2 Microsoft Research Stanford University Stanford, USA Redmond, USA {cholz, eyalofek, sinclair, benko} [email protected] , Chain , Layer-Hinge Figure 1. (a) Three prototype Haptic Links attached to commercial HTC VIVE controllers. From left to right: Ratchet-Hinge . (b-c) A survival game in which a Haptic Link rigidly locks handheld controllers in the shape of a two-handed gun. Overlay added in post-production for visualization. ABSTRACT ACM Classification Keywords We present Haptic Links, electro-mechanically actuated H.5.1 [Information Interfaces and Presentation]: Multime- physical connections capable of rendering variable stiffness dia Information Systems-Artificial, Augmented, and Virtual between two commodity handheld virtual reality (VR) con- Realities; H.5.2 [User Interfaces]: Haptic I/O. trollers. When attached, Haptic Links can dynamically alter the forces perceived between the user’s hands to support the INTRODUCTION haptic rendering of a variety of two-handed objects and Researchers have made significant advancements in the interactions. They can rigidly lock controllers in an arbi- design of haptic controllers for virtual reality (VR), result- trary configuration, constrain specific degrees of freedom or ing in a variety of methods for rendering tactile and kines- directions of motion, and dynamically set stiffness along a thetic sensations in the hand [3,6,10,38]. However, the ma- continuous range. We demonstrate and compare three pro- jority of this work prioritizes single-handed interactions, Ratchet- , and Layer-Hinge , Chain totype Haptic Links: despite the prevalence of bimanual interactions in our lives . We then describe interaction techniques and scenar- Hinge [14]. That is, though many haptic controllers provide local ios leveraging the capabilities of each. Our user evaluation feedback for a single hand interacting with an object, they results confirm that users can perceive many two-handed the hands. For example, when between cannot render forces objects or interactions as more realistic with Haptic Links a user drives with a virtual steering wheel or swings a virtu- than with typical unlinked VR controllers. al golf club with handheld controllers, these interactions lack the physical constraints imposed by the real object. Author Keywords Haptic Links; virtual reality; variable stiffness; haptics; In this paper, we focus not on the design of a VR haptic two-handed; controller; the design of a haptic connection between controller, but on controllers . We present Haptic Links, electro-mechanically actuated physical connections capable of rendering variable Permission to make digital or hard copies of all or part of this work for stiffness between two commodity handheld VR controllers. personal or classroom use is granted without fee provided that copies are When attached, Haptic Links can dynamically alter the not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for com- forces perceived between the user’s hands to support the ponents of this work owned by others than the author(s) must be honored. haptic rendering of a variety of two-handed objects and Abstracting with credit is permitted. To copy otherwise, or republish, to interactions. They can rigidly lock controllers in an arbi- post on servers or to redi stribute to lists, requires prior specific permission trary configuration, to, for example, make the controllers and/or a fee. Request permissions from [email protected] feel and behave like a two-handed tool or weapon (see Fig- CHI 2018, April 21–26, 2018, Montreal, QC, Canada ure 1(c)). They can constrain specific degrees of freedom or © 2018 Copyright is held by the owner/author(s). Publication rights li- directions of motion between the controllers, such as when censed to ACM. turning a crank or pulling a lever. They can even set stiff- ACM 978-1-4503-5620-6/18/04...$15.00 ness along a continuous range, to render friction, viscosity,

2 the hands are less widely studied. Typical or tension. In these ways, Haptic Links augment existing between feedback handheld controllers with realistic mechanical constraints, bimanual tasks vary greatly in the positioning, orientation, making interaction and game play in VR scenarios more and relative motion of the hands [14]. The increased dis- immersive and tangible. tances, stronger forces, and numerous degrees of freedom in bimanual scenarios make many conventional haptic solu- This paper presents the following contributions: tions infeasible. Grounded force-feedback systems, de- scribed above, can support bimanual feedback when used in 1. The design and implementation of three Haptic Link pairs by rendering forces on each manipulandum in a coor- ,” and “ Chain ,” “ Layer-Hinge Ratchet-Hinge ”) devices (“ dinated manner [32,41]. Such dual articulated arm setups enabling variable stiffness feedback between commodi- are most frequently used in cases of robotic telemanipula- ty handheld VR controllers; tion [18,21] or in surgical simulation [7,24], where precise Technical and user evaluations outlining the tradeoffs of 2. bimanual actions are crucial. However, the size, cost, and each Haptic Link design and showcasing the potential of complexity of these setups prohibit their widespread usage each in the haptic rendering of different object types; with commodity VR systems. 3. Several two-handed interaction techniques and scenarios leveraging Haptic Links to improve bimanual haptic passive controller augmenta- An alternative solution uses rendering in VR. virtual tions to improve the haptic presentation of a specific RELATED WORK object. For example, a number of commercially sold at- Haptic interfaces for VR interaction encompass a diverse tachments anchor VR controllers in the form factor of a set of devices and form factors. Wearable devices such as two-handed gun, enabling users to feel realistic handling in gloves and full-body suits present simulated tactile sensa- applications that utilize rifles or other such weapons tions to the skin, often utilizing vibrotactile or electrical [34,44]. Similar attachments exist for golf clubs [45], musi- feedback [12,20,50]. Exoskeleton-type devices are similarly cal instruments [31], and more. Though simple and compel- worn, but mechanically impart forces or constrain motion ling, these passive attachments are static, making them less around the fingers, hands, or arms to render haptic sensa- effective for rendering multiple types of objects, or objects tions on a variety of scales and resolutions [5,11,42,44]. that are deformable or customizable. Because they do not require the user to hold a device, both Variable Stiffness Feedback wearables and exoskeletons free the user’s hands and fin- We consider variable stiffness actuation as a possible gers for direct interactions with the virtual environment. mechanism by which to enable general-purpose bimanual feedback in VR. Variable stiffness as a feedback technique Another widely studied class of haptic system is the actuat- is well studied. Approaches include electromechanical ac- ed force-feedback arm, such as the PHANToM family of tuators, jamming, rheological fluids, shape memory alloys devices [16,23,26,29]. These arms typically make use of a (SMAs), and low-melting point alloys. Detailed reviews of grounded physical reference in the user’s environment. As a these techniques can be found in [25] and [43]. result, they can impart net forces on the user to render large haptic forces and collisions, but often at the cost of reduced Already, researchers have applied several of these variable mobility and operating space. In contrast, encountered-type stiffness techniques in the design of haptic interfaces. Me- devices such as robotic arms and drones [2,47] follow the chanical exoskeletons utilize belts and cables to stiffen in- user as they move through the environment, making physi- dividual joint motion [42]. Wearable interfaces have em- cal contact as needed to render haptic cues. Though they ployed jamming to restrain finger movements in a glove- offer mobility and support large interaction spaces, chal- type form factor [39], as well as in the design of full-limb lenges in predicting and reacting to the user’s movements restraints [27]. Similarly, MR fluid brakes have been used often result in issues of latency. in the design of a haptic glove for VR [4]. Other researchers have leveraged shape-memory alloys and electro- More recently, another form factor—the handheld control- mechanical approaches to enable variable stiffness interac- ler—has emerged as the dominant mode of interaction in tions with mobile devices [15,28]. The Haptic Links de- consumer VR systems, such as in the Oculus Rift [30] and scribed in this paper apply these variable stiffness actuation the HTC VIVE [17]. Though handheld controllers greatly techniques on a larger scale to dynamically brake the rela- simplify the tracking and input requirements of VR sys- tive motion between two handheld controllers. tems, their haptic capabilities are frequently limited to inte- grated vibrotactile motors. They may also include physical DESIGN AND IMPLEMENTATION inputs which provide passive feedback, such as triggers, The implementation of our Haptic Link vision began with buttons, and joysticks. Recent work has targeted the con- numerous design considerations. These included: stiffness troller form factor in enabling rich haptic interactions such when actuated, flexibility when relaxed, resolution of stiff- as touch, grasping, weight, and texture [3,9,10]. ness, weight, bulk, moment of inertia, actuation speed, power consumption, noise, and range of motion. The ideal Bimanual Haptics in VR Haptic Link would be virtually undetectable to a user prior Despite the range of technologies capable of rendering hap- tic sensations on the hands in VR, systems that provide

3 to actuation, but could stiffen quickly, strongly, and pre- cisely as needed. Of the studied approaches for rendering variable stiffness, we first ruled out both SMAs and low-melting point alloys, Linear actuator which have prohibitively long actuation periods, and ER/MR fluids, whose solutions tend to settle out over time Cable leading to decreasing effectiveness. Particle jamming inter- faces showed promise with relatively high stiffness gains, Ball-and-socket element particularly when reinforced with internal frames or skele- tons [8,27,46]. However, such reinforcements reduce the flexibility of the joint when unjammed, and the volume of matter required adds significant weight to the mechanism. Layer jamming was perhaps more promising, as researchers have successfully used it to produce light, flexible manipu- lators with significant stiffening capabilities [19,37]. How- prototype stiffens an articulated chain by Chain Figure 2. The ever, for both jamming techniques, actuation remained un- pulling tight a cable threaded through ball-and-socket ele- suitably slow, and the large pumps required ultimately led ments. us to focus on electro-mechanical stiffening approaches. actuators retract the cable, the ball-and-socket elements are Existing electromechanical techniques to produce variable compressed into one another, increasing the friction at each stiffness could easily achieve sufficiently high braking tor- joint in the chain. As a result, the entire chain stiffens, fix- ques as well as quick and precise stiffness control. Howev- ing the current spatial relationship between the two control- er, due to the size, weight, and power constraints of our lers. task, using motors and brakes to directly oppose the user’s Chain The prototype uses two Actuonix L12-R Micro Line- torque quickly became infeasible. Thus, our designs shifted ar Servos [1] each capable of 80 N maximum force at 6.5 to investigate alternative joints and mechanisms which mm/s. The ball-and-socket elements were 3D printed using could be indirectly actuated using smaller motors. a PolyJet material, and the cable is an ultra-high-molecular- Our exploration yielded three prototype Haptic Links weight polyethylene string (1 mm diameter, 160 kg break- (Figure 1(a)), all capable of allowing and halting the 6-DOF ing force, 4.8% breaking elongation). The linear servos are motion of handheld VR controllers. Each design has mounted onto laser cut Polyoxymethylene pieces and ori- tradeoffs and advantages over the others, making them best ented coaxially with the ends of the chain. Each of these suited for different applications. We envision that designers pieces then mounts onto two 3D printed conical frusta of VR experiences could choose the Haptic Link that best which clamp down on either side of the annular portion of meets their needs out of many options, allowing users to the HTC VIVE controller. quickly attach the recommended Haptic Link to their con- Chain The prototype was designed for unrestricted 6-DOF trollers prior to entering the virtual world. motion which can globally stiffen in any configuration. We chose to design our Haptic Links to fit the commercial- Though it can render non-binary stiffness, precise control ly available HTC VIVE [17] controllers, due to the ease of over the intermediate range is difficult, as stick-slip motion constructing mounting geometries either for the annular between the individual elements results in a nonuniform tracking region or the protruding base. All Haptic Links perception of stiffness. Models of the frictional forces in the affix to these controllers without additional modifications. interdependent ball-joint elements are beyond the scope of Each Haptic Link connects via ribbon cable to a regulated this paper, but details can be found in [37]. power supply and to a Teensy 3.2 microcontroller [33] Device Layer-Hinge which controls the actuators on the device. VR applications The second prototype ( Layer-Hinge , Figure 3) consists of can communicate with this controller via serial to dynami- two main components: ball joints to allow rotation of the cally update the stiffness of each joint in the Haptic Link. controllers, and a hinge controlling the distance between Both our VR experiences and our user evaluation were im- them. Each controller is connected at its base via a 3D plemented in the Unity 2017 game engine. printed mount to a ball joint capable of 360° pan and 60° Chain Device tilt rotation. A FEETECH FS5115M servo [13] (180° rota- The prototype (Figure 2), utilizes a highly articulated Chain tion in .48 s, 15.5 kg-cm torque) drives the set screw on chain composed of ball-and-socket elements. A strong cable each ball joint, locking and unlocking rotation of the con- is threaded through the length of the chain and tethered to a troller. The hinge component consists of a series of inter- linear actuator on each end. With the linear actuators ex- leaved layers of laser cut cast acrylic. An additional tended, the chain is kept loose such that the user can arbi- FS5115M servo drives a bolt threaded through the layers trarily move the controllers in 3D space. When the linear and a captive nut on the other end. As the nut cannot rotate,

4 Ball joint Servo driving set screw Servo driving Ball joint set screw Carbon fiber rod Pawl Interwoven layers Gear Servo actuating pawl Servo driving bolt Figure 4. The prototype uses locking ball joints Ratchet-Hinge and a dual-ratchet hinge. Each of the opposing pawls can be individually disengaged to control both directions of motion. prototype consists of locking ball Figure 3. The Layer-Hinge joints to control rotation and a hinge leveraging the friction ratchet mechanism capable of independently braking in- between layers to control distance between controllers. ward or outward motion. The ratchet mechanism is cut from the rotational motion of the bolt compresses the layers, in- Polyoxymethylene and features two pawls set against a creasing the overall friction of the hinge based on the num- central gear at opposite angles. A Hitec HS-35HD Nano ber of layers in contact. The hinge and ball joint pieces are Servo [36] with an attached cam disengages each pawl, connected by carbon fiber rods (21.5 cm length, 1.27 cm freeing the corresponding direction of motion. With both diameter). pawls engaged, the gear is fixed, and with both disengaged, the gear can rotate freely. When one pawl is disengaged, the Layer-Hinge With three distinct points of actuation, the gear can ratchet against the engaged pawl in one direction, prototype has the advantage of selectively locking individu- but rotation in the opposite direction further engages the al degrees of freedom in the motion of the controllers. For remaining pawl and halts the motion. example, if the hinge is locked but the ball joints remain free, the controllers can rotate at a fixed distance apart, The directionally-selective capabilities of the Ratchet- much like joysticks. Further, the friction of each joint can prototype enable unique force-feedback interactions, Hinge be controlled with relative precision, allowing the device to such as the rendering of a midair impassable surface. How- render a continuous range of stiffness values in both the ever, each ratchet can only engage or disengage in a binary hinge and ball joints. The resistive force in the hinge can be fashion, and thus the prototype trades away the intermediate modeled with the relationship Layer-Hinge stiffness capabilities of the prototype. TECHNICAL EVALUATION 퐹 = 휇푛푝퐴 Table 1 compares the technical specifications of each Hap- where 휇 is the static coefficient of friction of the cast acryl- tic Link, and Figure 5 shows experimentally obtained is the applied ic, is the number of contact layers (12), 푝 푛 torque-angle curves of the prototypes when fully stiffened. pressure, and is the contact area of each layer. 퐴 Torque-angle measurements were taken using a lathe by anchoring one end of the joint to the spindle, attaching the Ratchet-Hinge Device other end in series with a force gauge to the slide, and re- The third prototype ( , Figure 4) uses the Ratchet-Hinge cording measurements while stepping back the slide (Figure same ball joint components beneath the controllers as in the 6). Measurements were taken with the arms of the proto- Layer-Hinge prototype, but replaces the hinge with a dual-

5 Chain Layer-Hinge Ratchet-Hinge 651 Weight (g) (without controllers) 673 793 Concentrated at the base of the Weight distribution Evenly distributed Concentrated at the base of the controllers controllers and at the hinge Actuation speed (s) (0-100%) 1.0 Hinge: .65, Ball Joint: .50 Hinge: .13, Ball Joint: .50 68 54 Maximum distance between controllers (cm) 63 Stiffness control Continuous Continuous Binary 1 . Technical specifications of each Haptic Link prototype. Table 1.6 1.4 1.2 Slip Slip 1 0.8 Slip LAYER-HINGE (Hinge) 0.6 Torque (Nm) RATCHET-HINGE (Hinge) 0.4 CHAIN 0.2 Ball Joint 0 10 12 2 6 4 0 8 Angle (degrees) Figure 5. Torque-angle curves for each actuated component of the prototypes. The Ball Joint curve applies to both the Layer-Hinge Ratchet-Hinge was not driven to failure (due to its having a critical failure mode), prototypes. The hinge of the Ratchet-Hinge and whereas all other components were driven until torque remained constant. on the stiffness of the joint, but ranges from 1-5 W on each types parallel to each other and perpendicular to the applied prototype. In addition, we can instead configure the Haptic force. Both the Chain prototype and the hinge of the Layer- Hinge prototype reached maximum torques of 1.1 Nm be- Links to push the maximum stiffness further by holding at stall torque. In this case, the total power consumption de- fore slippage, whereas the ball joints used in the Layer- Hinge prototypes yielded only .7 Nm. Ratchet-Hinge and pends upon the time spent at maximum stiffness, for which prototype holds until me- The hinge of the Ratchet-Hinge the stall power consumption is around 4.8-7.2 W on all pro- totypes. The motors may grow warm if left at stall for sig- chanical failure of the ratchets. Due to our having only a single Haptic Link of each design, we did not test destruc- nificant periods, though no heat-related issues were ever detected during the course of this work. tive failure modes. USER EVALUATION The prototypes can be configured to operate in a power- We designed an evaluation to investigate the following efficient mode or to optimize for the maximum possible questions: stiffness. For most purposes, the actuators can be set such that the static back drive force is used to hold the joint when 1. Can inter-controller variable stiffness feedback con- fully stiffened. In this case, power exceeds the standby tribute to a more realistic haptic rendering for two- power (.05-.1W) only when moving the servos (i.e. chang- handed objects and interactions than that of conven- ing the stiffness). This transient consumption varies based tional (unlinked) controllers? Which of the Haptic Link prototypes are most effective 2. Haptic Link Spindle Force Gauge Slide in providing realistic haptic renderings for different types of objects? Methodology We recruited 12 participants from our organization (ages 25-49, 1 female) to help in answering these questions. Par- ticipants were asked to rate their perceptions of object real- ism using each Haptic Link and a control pair of unlinked HTC VIVE controllers. For each trial, participants first aligned their controllers into the proper positions for the object using visual indicators. Once correctly positioned, angle curves on a lathe. Figure 6. Setup for measuring Torque-

6 . Objects explored in the user evaluation. Left to right, top to bottom: RIFLE, BOW, TROMBONE, PISTOLS. Overlay Figure 7 added in post-production for visualization. the virtual controller models disappeared and were replaced from and placed into their hands. Participants wore head- by the target object. Haptic rendering then began, and the phones playing brown noise to prevent listening for any noises from motor activations. participant was allowed up to 30 seconds to freely explore the object. Once finished exploring, the participant re- Virtual Objects sponded to the following question on a 1-5 Likert scale: We presented participants with the following four virtual “How much did it feel like you were holding and handling a objects (Figure 7). These objects were selected to cover a real object?” The participant selected a response using a range of possible stiffness requirements and motion types, laser pointer tool, and then continued to the next trial. to fully explore the capabilities of each Haptic Link. Table 2 describes how stiffness was applied to each Haptic Link We presented four objects (described below) across four to render these objects. Standard vibrotactile feedback on device conditions ( Chain , Layer-Hinge , Ratchet-Hinge , and the unmodified VTVE controllers was used across all ob- unlinked VIVE controllers). Each participant explored each jects and conditions. We refer the reader to the accompany- object twice on each condition, for a total of 32 trials. Trials ing Video Figure to see these objects in greater detail: were grouped into device blocks, such that participants ex- plored all objects in one device condition before switching RIFLE: A two-handed rifle was rendered such that the  to another device. The order of object presentation was ran- participant’s right hand held the trigger, while their left domized within a device block, and device blocks were hand rested beneath the forestock. counterbalanced across participants. BOW: A recurve bow and arrow were rendered such that  After each device block, participants rated aloud qualitative the participant’s left hand held the bow at its grip while aspects of the current device by agreeing with statements on the right hand held an arrow. After setting the arrow in a 1–5 scale (1 = “Strongly disagree”, 5 = “Strongly agree”). the bow’s nock, the participant held the trigger while re- The aspects rated were comfort (“I found the controllers tracting their right hand to pull back the bowstring and comfortable to use”) and movability (“I found that I could arrow. Releasing the trigger fired the arrow. Slight vibra- move the controllers as desired or expected”). tions were felt while pulling back the bowstring. TROMBONE: A trombone was rendered such that the  To prevent the effects of visual bias on the ratings of any bell section was held in the left hand while the right device condition, participants did not see any of the control- hand held the slide. Moving the right hand towards and lers until the end of the study. The devices were hidden away from the left hand controlled the motion of the vir- prior to the arrival of the participant, and participants kept tual slide. The slide would not move past its first posi- on the HMD throughout the study as devices were taken tion (too close) or beyond the end of the tubing (too far)

7 Layer-Hinge Ratchet-Hinge Chain RIFLE Rigidly locked Rigidly locked Rigidly locked Left (bow hand) ball joint Increasing stiffness as the user Left (bow hand) ball joint BOW locked; outward motion draws bow locked; increasing stiffness braked when fully drawn as the user draws bow Ball joints locked; direc- Small baseline stiffness in TROMBONE Small baseline stiffness hinge; ball joints locked tional brake as user exceeds max/min slide Unlocked PISTOLS Unlocked Unlocked Table 2. Specific stiffness applied to each Haptic Link to render the objects in the user evaluation. to prevent unrealistic renderings or separation of the and TROMBONE conditions, and less realistic in the pieces. Slight vibrations were felt as the slide moved. PISTOLS condition. PISTOLS: Two pistols were rendered to track each of  Results and Discussion the participant’s controllers individually. Figure 8 and Table 3 summarize the ratings for each device- object combination, as well as overall comfort and movabil- Hypotheses ity responses from post-device feedback. Using Wilcoxon The RIFLE served to inform whether Haptic Links improve Signed-Ranks Tests, we assessed the comparisons between the realistic presentation of rigid two-handed objects. The each Haptic Link and the unlinked VIVE controllers for BOW primarily explored dynamically increasing stiffness each condition. We used Dunnett’s Test as a follow-up values in a continuous range (for applicable Haptic Links). measure to account for multiple comparisons against the The TROMBONE investigated constrained degrees of free- control group. dom in motion as well as directionally selective braking (for applicable Haptic Links). Finally, the PISTOLS served Participants considered all Haptic Links to be significantly as a control condition to determine whether the presence of more realistic than unlinked controllers for the RIFLE ob- the Haptic Link detracts from the haptic presentation of ject, suggesting that variable stiffness haptics can indeed otherwise disjoint objects. improve the haptic rendering of rigid two-handed objects. Thus, for each object, post-hoc comparisons were made Mean ratings were also higher for all Haptic Links with the between each Haptic Link condition and the unlinked VIVE TROMBONE but failed to reach adjusted statistical signifi- controllers. In particular, the Haptic Links were hypothe- proto- cance. Ratings were highest for the Ratchet-Hinge sized to be perceived as more realistic in the RIFLE, BOW, Chain Layer- Ratchet- VIVE χ-crit Hinge Hinge 2.75 3.54 3.57 RIFLE .67 3.83 = .017* p = .001* p = .005* p BOW 3.79 3.54 .64 3.65 3.71 p p = .528 p = .314 = .367 3.38 TROMBONE 2.75 3.21 3.08 .73 p = .098 p = .020 = .235 p 3.63 .68 PISTOLS 3.63 2.83 3.21 = .026* = .100 p p = .973 p Overall 3.3 4.7 3.7 3.2 .7 Comfort p = .003* p p = .005* = .003* 3 3.9 3.3 3.7 .9 Overall Movability p = .086 = .042* p = .594 p * = Significance persists following multiple comparisons adjustment using Dunnett’s test (minimum significant χ-crit ) distance between group means listed as Table 3. Mean ratings and p-values for object-device realism and overall device ratings. p-values represent comparisons to Figure 8. Box plots aggregating object-device realism ratings unlinked VIVE controllers within the same category. and overall device ratings.

8 type, which is expected given that it was the only prototype capable of rendering the outward stop matching the end of the range of motion of the visual slide component. The results from the BOW trials are more surprising, as ratings were similar across all conditions. Follow-up inter- views with participants indicated that they were highly sen- sitive to the nuances of each Haptic Link’s motion in the context of such a complex action. Among the issues noted were the lack of a true spring force in the bowstring, the time to fully release stiffness after firing, and the inability Unlocked Locked to pull straight back on the hinged devices while holding the bow upright. In contrast, when using the unlinked con- Figure 9 . Summoning a vehicle using gestural input (left), then driving with a rigid steering wheel (right). trollers, participants rarely critiqued the experience at all, despite the complete lack of feedback between the bow and Object Summoning bowstring. One possible conclusion is that users are more As our Haptic Links can currently only provide resistive willing to accept no feedback in the rendering of an object forces (that is, they cannot actively move controllers into a incorrect (perhaps due to familiarity) than feedback. given configuration), a key question is how to get the user’s controllers positioned appropriately to render a particular prototype had significantly lower Chain Finally, only the object. Similar to existing work leveraging gestures that ratings than the unlinked controllers for the PISTOLS ob- mimic physical input [40], our first approach—referred to ject. Participants largely attributed these sentiments to a as “Summoning”—allows the user to assume a desired ob- limited range of motion due to the chain’s length. ject at will by mimicking its shape with their controllers. Participants rated comfort as significantly higher for the We explored this technique in the context of a virtual racing unlinked controllers than for the Haptic Links. Weight was game (Figure 9), where the user can Summon either a car or the most frequently mentioned issue, as well as the feeling a motorcycle around them by placing their controllers in the that the Haptic Link could be felt protruding if the control- shape of a steering wheel or handlebars respectively. Once lers were held close and at a particular angle. However, in position, the Haptic Link rigidly locks the controllers, some participants noted that despite the impact on comfort, thus rendering the steering wheel/handlebars which the user the added weight improved the realism of the virtual ob- can rotate as if steering a real vehicle. By pressing a button jects. Others specifically mentioned that the linked control- on the controller, the Haptic Link relaxes, and the vehicle lers felt less familiar in their hands, which suggests a fol- disappears. With this technique, users can easily switch low-up evaluation with longitudinal design to explore the between objects on the fly by Summoning them through the effects of familiarity. Finally, Movability ratings were sig- correct pose of the two controllers. prototype. As high- Chain nificantly decreased only for the In a similar fashion, we also created a zombie shooter game lighted during the PISTOLS trials, the shorter length of the utilizing the rifle and dual pistol weapons featured in the Chain prototype resulted in an unsatisfactory range of mo- user study (Figure 1(b-c)). By default, players can shoot tion for scenarios requiring total freedom of each controller. pistols in each hand, but by holding their hands out as if Given these results, we can begin to consider guidelines for wielding a rifle, the pistols are replaced by the rifle weapon, designers that suggest the usage of different Haptic Links which shoots farther and faster. Upon transition, the Haptic for different objects and interactions. For example, both the Link rigidly locks the controllers in the rifle configuration. Layer-Hinge models are effective in Ratchet-Hinge and A button on the controller returns to the pistols, allowing switching between objects that require rigid locking and the user to switch between weapons at will. Through Sum- free motion, such as picking up and wielding various static moning, Haptic Links allow users to wield an indefinite device suffers from a decreased objects. While the Chain number of objects with different haptic presentations. effective length and slower actuation speed, it can be freely Object Retrieval Indicators positioned in different shapes and is safe to grab, having no With Summoning, the user can take up a given object at any exterior actuated elements along the chain. Thus, we might time—as if the object is always with them. An alternative suggest its use in wheels, levers, ropes, and other arbitrari- style of interaction is manual object acquisition: the object ly-shaped tools that users can grasp in different positions. exists in a particular location and the user must approach APPLICATIONS AND EXTENSIONS and pick it up. To ensure that the controllers are properly Along with user evaluation, we explored the potential of arranged for haptic rendering in this style of interaction, we our Haptic Link attachments for supporting richer bimanual use a set of visual indicators that represent the proper posi- interactions in VR through several techniques. tion and orientation of each controller to pick up the object.

9 Guided indicators Inward Unlocked Locked Figure 11. Using directionally-selective braking to grasp ob- jects with two hands. As the controllers meet the object, in- ward motion is braked. Overlay created using post-processing. Figure 10. Using guided indicators (top) to retrieve a wrench object (bottom). We created a plumbing scenario (Figure 10) demonstrating the use of these indicators, in which users tighten down a nut on a pipe using a wrench. One controller holds the wrench while the other holds the pipe. To begin the interac- tion, the user must approach the pipe, grab the pipe below the nut with one hand, and position their other hand at the highlighted zone in the air above the nut. When correctly positioned, the wrench appears, and the Haptic Link pro- vides increasing stiffness on the downswing, while releas- ing the stiffness as the user returns on the upswing. Grasping Virtual Objects By tracking the controllers and locking them in a direction- Figure 12. Grounding one controller at the waist (top) or on an ally-selective fashion as they move through virtual space, external surface (bottom), to enable grounded force-feedback the Ratchet-Hinge prototype can render impassable virtual interactions with the other controller via the Haptic Link. surfaces. We explored this capability in the context of two- handed grasping interactions by creating a virtual play- walls and objects by braking outward motion of the control- ground where users grasp objects of different sizes between ler as it contacts each surface. We fabricated an additional their hands (Figure 11). controller mount allowing one of the linked controllers to attach either on-body or externally (Figure 12), and then Controller Grounding designed a virtual environment where users could reach out Again leveraging the directional capabilities of the Ratchet- and explore walls in front of them. prototype, we can anchor one controller to a fixed Hinge point, and then using the Haptic Link to provide grounded LIMITATIONS AND FUTURE WORK force feedback to the other: In short, the user grounds either Despite the capabilities we have shown, the Haptic Link on-body (the user anchors a controller to their body) or ex- prototypes have limitations requiring further iteration. From ternally (the user anchors a controller to a fixed surface in a mechanical standpoint, users mentioned weight, bulk, and the room). Then, we can repurpose the variable stiffness range of motion as noticeable detractors, all of which can actuation of the Haptic Link to halt motion of the remaining be improved through modifications to the design, materials, controller in the user’s hand. For example, if the user and fabrication. As our prototypes were primarily built with grounds one controller at their waist, we can render solid laser cut or 3D printed components, switching to manufac-

10 tured components should improve not only weight and stur- pabilities of Haptic Links. For example, can a sufficiently visually compelling experience lead users to perceive diness, but precision and robustness of the friction-based mechanisms as well. Additional investigation is needed to spring forces or inertial forces that cannot be rendered by fully understand the ergonomic impacts of Haptic Links, the Haptic Link? Or, by effectively redirecting users’ focus such the added moment of inertia in different controller and movement, can we design the interaction such that the poses, motion constraints given different sizes and attach- users themselves provide these forces from the other con- troller? We find these open questions to be exciting future ment sites of the Haptic Link, as well as potential muscle fatigue from extended use. work as we continue to iterate on the design of our Haptic Links. From an actuation standpoint, the maximum stiffness of CONCLUSION most joints—though enough to present rigidity to the us- Haptic Links demonstrate the potential to improve the hap- er—can be overcome through the user’s full strength. Simi- tic rendering of two-handed objects and interactions in VR larly, some users noted that a faster speed of actuation using inter-controller variable stiffness feedback. The mul- could improve the rendering of quick transitions—for ex- tiple implementations of Haptic Links yield different capa- ample, the releasing of tension on the bowstring. We can bilities and advantages for object rendering. Our evaluation improve both the maximum braking torque and the actua- shows that Haptic Links can improve the perceived realism tion speed by using more performant motors and by iterat- of two-handed objects without significantly detracting from ing on the designs of our joint mechanisms. However, our the rendering of normal interactions requiring disjoint con- results also suggest the need for further evaluation to identi- trollers. Finally, the interaction techniques we introduce fy how much—or how little—stiffness is needed to per- leverage Haptic Links to provide more compelling haptic ceive the controllers as a unified object, and how quickly experiences in VR. actuation must occur to render compelling interactions. Virtual reality has become increasingly immersive, leaving Finally, several modifications could make Haptic Links us with a growing need for authentic haptic interactions. more convenient for use with a commodity VR setup, such Haptic Links offer designers of VR experiences a wide as making them adjustable in length, removing the need for range of new haptic tools that work seamlessly with the a wired connection, and creating an attachment mechanism handheld controllers of commodity VR systems. While our that allows them to quickly snap onto and off of the control- prototypes represent just a starting point in the design of lers. Longevity of the friction-based mechanisms is also a future Haptic Links, we find their early success encourag- concern: over the course of our development and evalua- ing for the exploration of a new class of devices which can tion, we occasionally found the need to recalibrate the mo- rapidly augment existing controllers to provide a custom- tors to adjust for wear in the frictional surfaces. ized haptic experience. More generally, the resistive nature of Haptic Links limits REFERENCES the range of possible types of force feedback that they can 1. Actuonix. 2017. L12-R Micro Linear Servos for RC & present. Specifically, Haptic Links cannot provide inertial Arduino. Retrieved January 1, 2017 from force feedback, meaning that while they can introduce forc- es between the hands, they cannot impart net forces onto Radio-Control-p/l12-r.htm them. Further, Haptic Links are incapable of rendering the spring forces found in many objects. If stiffness is applied Bruno Araujo, Ricardo Jota, Varun Perumal, Jia Xian 2. as a controller moves in one direction, there is no restoring Yao, Karan Singh, and Daniel Wigdor. 2016. Snake force in the opposite direction. Future Haptic Links could , 218–226. Proc. TEI 2016 Charmer. In include additional components such as gyroscopic mecha- nisms or springs to render such forces, and designers could 3. Hrvoje Benko, Christian Holz, Mike Sinclair, and Eyal choose these Haptic Links as needed by their VR scenarios. Proc. Ofek. 2016. NormalTouch and TextureTouch. In , 717–728. UIST 2016 Beyond improvements to the devices themselves, we aim to explore as future work how Haptic Links can influence the perception of other haptic techniques and phenomena in Jonathan Blake and Hakan B. Gurocak. 2009. Haptic 4. bimanual scenarios. For example, recent work has exam- glove with MR brakes for virtual reality. In ined illusory haptics in VR [22,35], finding that the vi- IEEE/ASME Transactions on Mechatronics 14, 5: 606– brotactile feedback in most handheld controllers can be 615. leveraged to present stimuli that seem to originate from M Bouzit, G Burdea, G Popescu, and R Boian. 2002. 5. different locations on a virtual object held between the The Rutgers Master II-ND Force Feedback Glove. In hands. As our results show that users can perceive a two- Proc. HAPTICS 2002 , 256–263. handed object more realistically with a Haptic Link, we can investigate whether the Haptic Link also improves the ro- bustness of these spatio-haptic illusions. We can also ex- 6. Tom Carter, Sue Ann Seah, Benjamin Long, Bruce plore the power of visual dominance in expanding the ca- Drinkwater, and Sriram Subramanian. 2013. UltraHap-

11 tics : Multi-Point Mid-Air Haptic Feedback for Touch lins, M Barham, C R Voolstra, K N Salama, M L’Hour, Proc. UIST 2013 , 505–514. Surfaces. In and V Creuze. 2016. Ocean One: A Robotic Avatar for IEEE Robotics Automation Oceanic Discovery. In 23, 4: 20–29. Magazine 7. Sonny Chan, François Conti, Kenneth Salisbury, and Nikolas H. Blevins. 2013. Virtual Reality Simulation in 72, January: A154– Neurosurgery Yong Jae Kim, Shanbao Cheng, Sangbae Kim, and Neurosurgery. In 19. A164. Karl Iagnemma. 2012. Design of a tubular snake-like manipulator with stiffening capability by layer jam- , 4251–4256. ming. In Proc. IROS 2012 Nadia G. Cheng, Maxim B. Lobovsky, Steven J. Keat- 8. ing, Adam M. Setapen, Katy I. Gero, Anette E. Hosoi, Yukari Konishi, Nobuhisa Hanamitsu, Kouta Mina- and Karl D. Iagnemma. 2012. Design and analysis of a 20. mizawa, Ayahiko Sato, and Tetsuya Mizuguchi. 2016. robust, low-cost, highly articulated manipulator ena- bled by jamming of granular media. In Synesthesia suit: the full body immersive experience. Proc. ICRA ACM SIGGRAPH 2016 VR Village In , 4328–4333. 2012 , 20. A. Kron, G. Schmidt, B. Petzold, M.I. Zah, P. Hinter- 21. 9. Inrak Choi, Heather Culbertson, Mark Roman Miller, seer, and E. Steinbach. 2004. Disposal of explosive Alex Olwal, and Sean Follmer. 2017. Grabity: A ordnances by use of a bimanual haptic telepresence Wearable Haptic Interface for Simulating Weight and system. In , 199- , 1968–1973. Proc. ICRA 2004 Grasping in Virtual Reality. In Proc. UIST 2017 130. Inrak Choi and Sean Follmer. 2016. Wolverine: A 22. Jaedong Lee, Youngsun Kim, and Gerard Kim. 2012. 10. Funneling and saltation effects for tactile interaction Ad- Wearable Haptic Interface for Grasping in VR. In Proc. CHI 2012 , 3141. with virtual objects. In junct Proc. UIST 2016 , 117–119. 23. Richard Q der Linde, Piet Lammertse, Erwin Freder- 11. CyberGlove Systems Inc. 2017. CyberGrasp Glove. iksen, and B Ruiter. 2002. The HapticMaster, a new Retrieved January 1, 2017 from high-performance haptic interface. In Proc. Eurohap- , 1–5. tics 2002 12. CyberTouch. 2017. CyberTouch Glove. Retrieved Jan- uary 1, 2017 from Michael A Liss and E M McDougall. 2013. Robotic 24. Cancer Journal (United States) surgical simulation. In 19, 2: 124–129. 13. FeeTech. 2013. Standard Metal Gears Analog Servo FS5115M. Retrieved January 1, 2017 from Mariangela Manti, Vito Cacucciolo, and Matteo 25. Cianchetti. 2016. Stiffening in soft robotics: A review servo/standard-15kg-cm-metal-gears-analog-servo- fs5115m/ IEEE Robotics and Automa- of the state of the art. In tion Magazine 23, 3: 93–106. 14. Yves Guiard. 1987. Asymmetric Division of Labor in Human Skilled Bimanual Action. In Journal of Motor Behavior 19, 4: 486–517. Thomas H. Massie and J. K. Salisbury. 1994. The 26. PHANTOM Haptic Interface : A Device for Probing Virtual Objects. In Proceedings of the ASME Dynamic 15. Sidhant Gupta, Tim Campbell, Jeffrey R. Hightower, , 295-301. Systems and Control Division 1994 and Shwetak N. Patel. 2010. SqueezeBlock: Using vir- tual springs in mobile devices for eyes-free interaction. Takashi Mitsuda, Sachiko Kuge, Masato Wakabayashi, 27. 101-104. , Proc. UIST 2010 In and Sadao Kawamura. 2002. Wearable haptic display Proc. by the use of a Particle Mechanical Constraint. In , 153–158. HAPTICS 2002 16. Haption SA. 2017. Haption Virtuose 6D. Retrieved January 1, 2017 from 28. Yusuke Nakagawa, Akiya Kamimura, and Yoichiro Kawaguchi. 2012. MimicTile : A Variable Stiffness menu-en/hardware-menu-en/virtuose-6d-menu-en Deformable User Interface for Mobile Devices. In HTC. 2017. Vive. Retrieved January 1, 2017 from 17. Proc. CHI 2012 , 745–748. system/ 29. Novint. Novint Falcon. Retrieved January 1, 2017 from O Khatib, X Yeh, G Brantner, B Soe, B Kim, S Gangu- 18. ly, H Stuart, S Wang, M Cutkosky, A Edsinger, P Mul-

12 30. Oculus. 2017. Oculus Rift. Retrieved from 43. Dzmitry Tsetserukou. 2011. FlexTorque, FlexTensor, and HapticEye: Exoskeleton Haptic Interfaces for Augmented Interaction. In Proceedings of the 2nd 31. Oculus. 2017. Getting Started With Your Rock Band , 33:1-- Augmented Human International Conference VR Connector. Retrieved from 33:2. Dzmitry Tsetserukou, Shotaro Hosokawa and Kazuhi- 44. 32. Angelika Peer and Martin Buss. 2008. A new admit- ko Terashima. 2014. LinkTouch: A wearable haptic tance-type haptic interface for bimanual manipulations. device with five-bar linkage mechanism for presenta- IEEE/ASME Transactions on Mechatronics In 13, 4: tion of two-DOF force feedback at the fingerpad. In 416–428. , 307-312. Proc. HAPTICS 2014 33. PJRC. 2017. Teensy USB Development Board. Re- 45. B. Vanderborght, A. Albu-Schaeffer, A. Bicchi, E. trieved January 1, 2017 from Burdet, D. G. Caldwell, R. Carloni, M. Catalano, O. Eiberger, W. Friedl, G. Ganesh, M. Garabini, M. 34. ProTubeVR. 2017. ProTubeVR. Retrieved January 1, Grebenstein, G. Grioli, S. Haddadin, H. Hoppner, A. 2017 from Jafari, M. Laffranchi, D. Lefeber, F. Petit, S. Stramigi- page/copie-de-protubevr-for-htc-vive-and-oculus-rift- oli, N. Tsagarakis, M. Van Damme, R. Van Ham, L. C. inside-ue Visser, and S. Wolf. 2013. Variable impedance actua- tors: A review. In Robotics and Autonomous Systems 35. Jukka Raisamo. 2010. Vibrotactile Movement and Its 61, 12: 1601–1614. Applications for Virtual Reality. In PIVE 2010 , 25-27. Hitec RCD. 2017. HS-35HD Ultra Nano Servo. Re- 36. 46. VR Merch. 2017. HTC Vive Dual Controller Rifle trieved January 1, 2017 from Stock. Retrieved January 1, 2017 from servos/analog-micro-and-mini-servos/hs-35hd-ultra- rifle-stock nano-servo/product VR Merch. 2017. HTC Vive Golf Club Attachment. 47. 37. R. M. Sage. 1987. The mathematical modeling of ball Retrieved January 1, 2017 from joints with friction. Thesis. University of Leicester, Leicester, United Kingdom. club 48. Ying Wei, Yonghua Chen, Yang Yang, and Yingtian Jessie Lee C. Santiago, Isuru S. Godage, Phanideep 38. Li. 2016. A soft robotic spine with tunable stiffness Gonthina, and Ian D. Walker. 2016. Soft Robots and based on integrated ball joint and particle jamming. In Kangaroo Tails: Modulating Compliance in Continuum Mechatronics 33: 84–92. Structures Through Mechanical Layer Jamming. In Soft 3, 2: 54–63. Robotics 49. Kotaro Yamaguchi, Ginga Kato, Yoshihiro Kuroda, Kiyoshi Kiyokawa, and Haruo Takemura. 2016. A Samuel B. Schorr and Allison M. Okamura. 2017. Fin- 39. Non-grounded and Encountered-type Haptic Display gertip Tactile Devices for Virtual Object Manipulation Using a Drone. In Proceedings of the 2016 Symposium , 3115–3119. Proc. CHI 2017 and Exploration. In on Spatial User Interaction , 43–46. 40. Timothy M. Simon, Ross T. Smith, and Bruce H. Vibol Yem and Hiroyuki Kajimoto. 2017. Wearable 50. Thomas. 2014. Wearable jamming mitten for virtual tactile device using mechanical and electrical stimula- Proceedings of the 2014 ACM environment haptics. In tion for fingertip interaction with virtual world. In , 67– International Symposium on Wearable Computers , 99-104. Proc. IEEE Virtual Reality 2017 70. Christian Steins, Sean Gustafson, Christian Holz, and 41. Patrick Baudisch. 2013. Imaginary devices: Gesture- based interaction mimicking traditional input devices. , 123. In Proc. MobileHCI 2013 Anthony Talvas, Maud Marchal, Gabriel Cirio, and L 42. Anatole. 2013. 3D Interaction Techniques for Bimanu- al Haptics in Virtual Environments. In Multi-finger , 31–53. Haptic Interaction 1-4471-5204-0

Related documents