Home | Stories | Awards | Contacts
View stories by Year: Category:
Story Map: New pen computer will aid visually impaired to learn math, science | External Links

A new digital device could revolutionize the cumbersome task of note taking — and it's as easy to use as touching pen to paper

By Kara Furlong

Published: December 3, 2007

Seeing is believing. So is touching and hearing.

When it comes to a new digital "smartpen,” simple verbal descriptions don't do the gadget justice. One must use the device to appreciate all it has to offer.

So believes Andy Van Schaack, a lecturer in Peabody College's Department of Teaching and Learning, who is a senior science adviser for Livescribe, the company that has developed the new technology.

In addition to his efforts to aid in the smartpen's development, he and Joshua Miele, a researcher at the Smith-Kettlewell Eye Research Institute who is blind, have received a $300,000 grant from the National Science Foundation. Its purpose is to apply the smartpen to help visually impaired students learn science and mathematics, subjects that rely heavily on diagrams, graphs, charts and other figures and therefore put them at a significant disadvantage.

A handheld computer disguised as a humble ink pen? Essentially — yes. The smartpen functions as a simple ballpoint, but when used to write on specially printed paper, it can do much more.

"The smartpen is nothing less than a new computer platform,” Van Schaack says. "It includes much of the same technology found in a typical laptop — CPU, memory, microphone, speaker and display — but gets its information from handwriting rather than from a keyboard and mouse.”

So how, exactly, does the smartpen work? A camera inside its tip recognizes handwritten marks made on special paper. This paper is ordinary lined notebook paper, but appears to have a slightly grayish hue. Upon closer inspection, one sees that this hue is actually thousands of tiny, faint "microdots” printed in a unique pattern. When the smartpen is touched to the paper, its camera reads the handwritten ink marks in relation to the microdots.

The paper also is printed with a "control panel” along its bottom margin. Much like the control panel of a computer audio or video player, there are buttons for "record,” "pause” and "stop,” among others. Tapping the smartpen on "record” activates the pen's audio recording function, as shown on a digital display located on the pen's barrel. And that's when the smartpen really starts to perform.

Someone taking notes during a lecture or meeting can use the smartpen to record what is being says aloud and sync it with the notes they are simultaneously writing down. The user would later touch the smartpen directly to a specific word on the page to replay the audio spoken when it was written.

"The smartpen knows exactly where it is on the page,” Van Schaack explains. "Every square inch of every page of the dot paper notebook is absolutely unique. If you were to come back a month later and tap in your notes, you would hear that part of the lecture again.”

Notes can be transferred to a PC to be backed up, replayed or even shared online. The smartpen also can search for keywords to find a desired portion of text in seconds.

Cool technology — but why is it important?

For one thing, it combats common note-taking challenges faced by students and professors alike. If the smartpen is recording complete audio of a lecture at the same time its user is taking a minimum of notes, that user can simply return later, touch the pen to the text to listen to that part of the lecture, and flesh out the notes with all the details. "Students' mental resources often go toward being transcriptionists in the classroom instead of being students,” Van Schaack says. In other words, the smartpen frees them up to pay closer attention and think more deeply about what is being said.

Van Schaack is working to develop another useful application for professors like himself. "It's the end of the semester, and I have 90 papers to grade, 10 pages apiece, and I like to give a lot of feedback,” he says. "But it takes a long time for me to write comments in the margins of every paper.

"Imagine if all I had to do was circle a portion of the text, tap it with the smartpen and say, 'I really like the point you're making here, but you need to follow up with supporting evidence ... ' And when I dock my pen, it sends the student the paper electronically with an audio layer on top of it. The student can open the file on their PC — even if they don't own a smartpen — and when they click on what I have circled, they hear my comments.”

According to Van Schaack, when professors have used audio vs. hand-written feedback in previous experiments, audio feedback allowed them to provide up to 250 percent more information while taking only one-quarter of the time, and students were three times more likely to act on their suggestions because of the tone in which they were given and the depth of information provided. Additionally, the students perceived the professors as "caring about them more.”

When Livescribe launches the smartpen in late January, it will sell for less than $200. The notebooks, which contain 100 sheets of microdot paper each, will cost about the same as a quality notebook from the Vanderbilt Bookstore, Van Schaack says.

"We happen to be focusing on college students first with the smartpen because it really meets a need that they have. They're also not afraid of new technologies, and if they like something, they're likely to start buzzing about it,” he says. "And they have some discretionary income, and we feel it's in their price range.

"But for students, reporters, doctors, attorneys — anyone who takes notes and wants verbatim information and ultimately wants quick access to it — the smartpen is particularly useful. The goal is to be able to capture information, access it quickly, and share it with others.”

Another aspect of the technology that Van Schaack helped to develop is the smartpen's three-dimensional recording ability. He explained that each smartpen comes with a pair of earbud-style headphones that not only provide great playback quality, but also have tiny microphones built within them. When these are worn inside the ears and the smartpen is recording, the natural shape of the wearer's pinnae — or outer ear — will shape and enhance the sound being recorded.

"With these ear mikes, you can actually tune out sounds around you to hear the professor in front of you,” Van Schaack says. "It's actually better than regular human hearing because it's not only stereo and three-dimensional, but also filtered and amplified.” With traditional digital recorders, audio quality drops off quickly with distance. Sound captured with the smartpen's earbuds from the back of a 600-seat lecture hall is "fabulous,” according to Van Schaack. "In many cases, the playback sounds better than what you heard in person.”

Van Schaack's enthusiasm for the smartpen is understandable given his close work with Livescribe to help refine the technology, but it also reflects his care for the academic research in which he specializes — instructional technology.

"The Livescribe team has created this great new computer,” he says. "My job is to help answer questions like, 'How can this be used in the classroom? How can this help students to learn faster and remember what they've learned longer?'

"There's been a great deal of scientific research done on effective and efficient ways to teach and learn. Unfortunately, much of what is known sits in research journals and rarely sees the light of day. I'm really interested in technology transfer — in the application of these great ideas to help students out sooner, rather than later.”

The smartpen's characteristics make it particularly interesting to a special group of students: the visually impaired. That is the reason for the NSF grant. "Specifically, the grant addresses learning among blind college students in the STEM content areas — science, technology, engineering and math,” Van Schaack says. "For instance, how can a physics student make sense of the principles of force without seeing a diagram, or a biology student understand the cell cycle without a visual representation?”

Van Schaack and Miele are using the smartpen with the Sewell Raised Line Drawing Kit — a commonly used clipboard-like tool over which thin sheets of translucent Mylar are placed. Drawing on the Mylar creates raised furrows in its surface, which blind students can then feel to distinguish diagrams, sketches, charts and other figures. When this Mylar is printed with microdots and audio information is recorded and linked to the drawn figures using the smartpen, it becomes a marriage of what the researchers call "audio tactile graphics.”

Livescribe is encouraging user-generated content creation. "We're going to release tools very soon after launch that will allow users to create applications,” Van Schaack says. "You could make your own interactive study guide, cookbook or organizer and sell it in the Livescribe online store.”

Additional reporting by David Carew

Steve Green
Andy Van Schaack holding a smartpen


Smart pen and the earbuds that contain the microphones that enable the device to record 3-D audio sitting on the specially encoded paper that the pen computer uses.

Steve Green
A student using the smartpen with a Sewell Raised Line Drawing Kit that allows the visually impaired to feel the shapes that they have drawn.

New pen computer will aid visually impaired to learn math, science | External Links
VU Home - Flash Site