Abstract
We present an eyes-free text entry technique fortouchscreen mobile phones. Our method uses Graffiti strokesentered using a finger on a touchscreen. Although visualfeedback is present, eyes-free entry is possible using auditory andtactile stimuli. In eyes-free mode, entry is guided by speech andnon-speech sounds, and by vibrations. A study with 12participants was conducted using an Apple iPhone. Entry speed,accuracy, and stroke formations were compared between eyesfreeand eyes-on modes. Entry speeds reached 7.00 wpm in theeyes-on mode and 7.60 wpm in the eyes-free mode. Text wasentered with an overall accuracy of 99.6%. KSPC was 9% higherin eyes-free mode, at 1.36, compared to 1.24 in eyes-on mode.Keywords - Eyes-free; text entry; touchscreen; finger input;auditory feedback; mobile computing; gestural input; Graffiti.I.
INTRODUCTION
A. The Rise of Touchscreen PhonesRecently, the mobile industry has seen a rise in the use oftouchscreen phones. Although touchscreen devices are notnew, interest in them has increased since the arrival of theApple iPhone. Following the iPhone’s release, a wide array ofcompeting products emerged, such as LG's Prada, Samsung'sD988, Nokia's N95 and RIM's BlackBerry Storm.Early touchscreen phones were trimmed down versions ofdesktop computers. They were operated with a stylus,demanding a high level of accuracy. Such accuracy is notalways possible in mobile contexts. As a result, current touchbasedphones use the finger for input. The devices allow directmanipulation and gesture recognition using swiping, tapping,flicking, and even pinching (for devices with multi-touch).Such novel interactions afford a naturalness that is unparalleledby indirect methods (e.g., using a joystick).Phones with physical buttons are constrained since allinteraction involves pre-configured hardware. Once built, thehardware is fixed and cannot be customized further, whichlimits the scope of interactions possible. Touchscreen phonesuse software interfaces making them highly customizable andmultipurpose. Use of screen space is more flexible and sincethere is no physical keypad, the screen size can be bigger.However, these benefits come at a cost. Without physicalkeys, a user's ability to engage the tactile, kinesthetic, andproprioceptive sensory channels during interaction is reduced.The demand on the visual channel is increased and thiscompromises the “mobile” in “mobile phone”. One goal of ourresearch is to examine ways to reduce the visual demand forinteractions with touchscreen phones.B. Interacting With Touchscreen PhonesThe primary purpose of mobile devices is communication. So,it is important to support alphanumeric entry, even if it is justto enter a phone number. With physical buttons, users developa sense of the buttons, feel them, and over time remember theirlocations. This tactile and proprioceptive feedback is priceless.Users build a spatial motor-memory map, which allows them tocarry out basic tasks eyes-free, such as making or receiving acall. With experience, many mobile phone users carry out textentry tasks (e.g., texting) eyes-free by feeling and knowingtheir way around the device.With touchscreen phones, the feedback afforded byphysical buttons is gone. As a result, touchscreen phones aremore visually demanding, with users often unable to meet thisdemand in a mobile context. The overload of the visual channelmakes it difficult to use these devices when engaged in asecondary task, such as walking, attending a meeting, orshopping. Furthermore, the inability to use these devices in aneyes-free manner affects people with visual impairments [10].C. Mobile Phone Design SpaceThe natural interaction properties of a touch-sensitive displayallow a rich set of applications for touchscreen phones, but theincreased visual attention complicates translating the addedfeatures into mobile contexts. This brings us to a gap betweenwhat was capable with a physical button phone and what iscapable with touchscreen phones. Figure 1 serves as adescriptive model to think about the design space and potentialelements to consider in bridging this gap. Combining someform of physical feedback, finger tracking, and other feedback modalities to guide the user through tasks can provide theelements required for eyes-free text entry on a touchscreenphone. Our main purpose is to explore whether eyes-free textentry is even possible on a touchscreen phone. The nextsections discuss related work and our proposed solution. This isfollowed by a description of the evaluation carried out and adiscussion of the results.D. Eyes-free Mobile Device Use1) Button-based strategiesTwiddler [5], a one-handed chording keyboard, was used toinvestigate eyes-free typing. Entry speeds for some participantsapproached as high as 67 wpm. The difficulty with Twiddler isthe steep learning curve. This can frustrate users and may resultin low acceptability of the technique. Investing substantial timeto learn the basic operating modes of a consumer product isgenerally unacceptable.Another study compared multitap with the use of anisometric joystick using EdgeWrite, a stroke-based text entrytechnique [15]. The study involved entering EdgeWrite strokesusing gestures on a joystick. The mobile device used includedtwo isometric joysticks, one on the front and one on the back.Of specific interest is the finding that the front joystick allowedeyes-free entry at 80% of the normal-use speed (7.5 wpm).In our previous work, we presented LetterScroll, a textentry technique that used a wheel to access the alphabet [14].As text was entered, each character was spoken using a speechsynthesizer. Subjective findings revealed that entering text waseasy, but slow. The average text entry speed was 3.6 wpm.2) Touch-based strategiesAlthough touchscreen devices have been extensively studied, aliterature search revealed very little research on eyes-free textentry solutions for touchscreen devices. Below we presentsome related work in this area.FreePad [1] investigated pure handwriting recognition on atouchpad. Subjective ratings found that the overall experienceof entering text was much better than predictive text entry (akaT9). Text entry speeds were not measured, however.Wobbrock et al. proposed an enhancement to EdgeWritecalled Fisch [16], which provides in-stroke word completion.After entering a stroke for a letter, users can extend the stroketo one of the four corners of the touchpad to select a word. Theauthors duly note that the mechanism provides potentialbenefits for eyes-free use. However, no investigation wascarried out to test this.Yfantidis and Evreinov [18] proposed a gesture-driven textentry technique for touchscreens. Upon contact, a pie menuappears displaying the most frequent letters. Dwelling on themenu updates the pie menu by entering a deeper layer. Usersreceive auditory feedback to signal their position in the menuhierarchy. Some participants achieved an entry speed of12 wpm after five trials.Sánchez and Aguayo [12] proposed a text entry techniquethat places nine virtual keys on a touchscreen device.Consequently, text entry is similar to multitap. The primarymode of feedback is synthesized speech. Unfortunately, theirwork did not include an evaluation.We were unable to find literature on eyes-free text entry ona touchscreen device using a stroke-based entry mechanism.While occasionally mentioned, no controlled evaluations exist.3) Alternative modalitiesSome studies explored speech as an input mechanism formobile text entry [2, 11]. General findings reveal that speech isan attractive alternative for text input. However, suchmodalities are not always appropriate in mobile contexts. Byusing speech as input for mobile devices, there is a loss ofprivacy. Furthermore, social circumstances may inhibit the useof such techniques.We present a method that relies on speech and non-speechmodalities as a form of output rather than input. This isacceptable given that most mobile devices include earphones.The next section discusses our solution for eyes-free text inputon a mobile touchscreen phone.II.
EYES-FREE TEXT ENTRY ON A TOUCHSCREEN
A. The IngredientsTechnology to enable eyes-free text entry on a touchscreenalready exists. It is just a matter of knitting it together. Thecritical requirement is to support text entry without the need tovisually monitor or verify input. Obviously, non-visualfeedback modalities are important. Desirably, the techniqueshould have a short learning curve so that it is usableseamlessly across various devices and application domains.Goldberg and Richardson [4] were the first to propose eyesfreetext entry using a stroke-based alphabet. Their alphabet,Unistrokes, was designed to be fast for experts. A follow-oncommercial instantiation, Graffiti, was designed to be easy fornovices [9]. To maintain a short learning curve, we decided touse the Graffiti alphabet (see Figure 2). The similarity of mostcharacters to the Roman alphabet allows users to build uponprevious experiences. This encourages quick learning and aidsretention.Since the device will be occluded from view, the feedbackmust be non-visual. Given the power of current mobile devices,it is easy to incorporate basic speech synthesis. For certaininteractions, such as unrecognized strokes, we used theiPhone’s built in actuator to provide a short pulse ofvibrotactile feedback.B. Design IssuesPreviously, many studies investigated the performance ofsingle-stroke text entry using a stylus or pen on touch-sensitivedevices (e.g., [1, 15, 17]). Our work differs in that the maininput “device” is the finger. Another issue is the drawingsurface. Will it be the entire screen or just a region of thetouchscreen? Also, what is the ideal approach of incorporating


download full report
http://googleurl?sa=t&source=web&cd=1&ve...IC2009.pdf&ei=0Oh-TfyXM46PccfxnP0G&usg=AFQjCNFyzO6t7mbhK4BCQu4O-c-ff8ReKw