Robotic arm extend authors' signatures over cyberspace
The aptly named LongPen precisely replicates the signature of the person at a remote location signing on a tablet with a special pen—with the same speed, cadence and pressure, maintaining data security.
By André Voshart | April 17, 2009
The Government of Ontario produces many official documents that require signatures, and for security reasons, these papers cannot for sent by fax or mail for signing. However, the province recently adopted a long-distance signing station that was originally developed with the aid of author Margaret Atwood.
When Margaret Atwood called Matthew Gibson from a hotel room in the spring of 2004, she was in the middle of a grueling, multi-city, multi-country book tour. Knowing she has a 6 a.m. fight to catch, she wondered if there was a better way. She imagined being able to sign books for her fans from afar, and at that moment, she had the seed of an idea that would, years later, become the LongPen Freehand Script Robot.
The company that designed the arms says “LongPen signatures are approved in both the United States and Canada as legal.” For the government's use, they chose the LongPen BusinessWriter, which allows ink-and-paper documents to remain in the province’s cabinet office at Queen’s Park in Toronto while they are signed from other locations.
But in order for the idea to take off, Atwood teamed up with Gibson—who has a history in developing ideas into reality—and created Unotchit Inc. (pronounced “you no touch it”), the firm under which they would produce the device. Senior systems engineer John Wiltshire, who has been with the project since Day 1, led the design team through multiple incarnations, starting with a simple X-Y plotter, then moving up to experimentations with a joint-articulated single appendage.
“We tried our version of that,” Gibson said of the single appendage, “and it just wasn’t rigid or strong or fast enough ... and it sort of shook itself to death and burst into flames—and fell off the table.”
But when Jacob Apkarian, founder of Quanser Consulting Inc. in Markham, Ont., stumbled on the LongPen project in the newspaper, Unotchit had moved on was developing a prototype using linear actuators. Quanser is a leader in the development of real-time control systems, having developed haptic and simulator technologies, such as remote surgery, and Apkarian thought he could help. Gibson, in turn, had been looking to refine Unotchit’s design—so he decided to invest in a dialogue with Quanser in the summer of 2006. » Learn about the design progress and see sketches and images on the next page
“They were duly impressed with where we’d taken [our design up until that point],” Gibson said of the Quanser engineers. “Now, they then took all that work and extrapolated and built it into, instead of linear control, a rotary control and made it smaller, faster.”
Unotchit had a public event scheduled for September, so Quanser’s team of control-system wizards were given a tight, eight-week timeframe to go from concept to completion. To do so, Apkarian led a team of engineers that included Don Gardner and Ryan Leslie, both electro-mechanical design engineers, and roboticist and control specialist Paul Karam.
“They delivered it on time and doing what they said it would do,” Gibson said, clearly impressed. “In all my years, nobody’s delivered in that timeline.”
The process of design
LongPen’s design takes into consideration many factors of the human arm and how we write. (“We [initially] underestimated the technical difficulty in replicating what nature has taken a great deal of time to perfect,” Gibson said.) The system is a pantograph-based device with an anti-backlash mechanism to correct for gearbox non-linearity. It has four high-quality Faulhaber micro motors to maneuver the robotic arm. One is for vertical motion to make the robot approach the book, two are the pantograph so the pen writes on the X-Y plane, and the fourth lifts the pen toward and away from the paper.
The mechanical design was done in SolidWorks, but Apkarian and Karam needed to develop control strategies for the prototype. Gibson said the human hand oscillates at 30 Hz, pulls 6 to 9 Gs and uses 40 percent of the brain to accomplish the “simple” feat of handwriting. For the whole process to look effortless, there is a tremendous amount of work going on “behind the curtain.”
Complex mathematical operations were used to solve what’s called the “joint space and task space” movement. “When you are moving your arm,” Apkarian explained, “you are not only changing the angle of your joints, but also trying to control the fingertips of your hand. The relationship between the position in space, which humans perceive as X-Y-Z, versus the joint angles of the motors, is highly non-linear.” The kinematical and dynamic equations for the robot were developed using Maplesoft Maple, software that helps integrate complex algebraic mathematics into controllers. Apkarian used Maples to solve the joint space and task space issue, and special algorithms were created to compensate for jerking. » Learn about how it was programmed on the next page
Apkarian’s team used MathWorks’ Simulink software—the controller design graphic language—to help with preliminary analysis, modelling and simulation of the device. “First we do the mechanical design in the CAD program, but that is not enough to do the actual analysis of the motion and see if our kinematical (geometry) equations are correct. So to test the geometry equations, we pass them through the simulator to see if it’s actually doing the right thing.” When finished the control design, blocks from Simulink and Maple were integrated.
WinCon, Quanser’s rapid control prototyping and hardware-in-the-loop testing system, converts the Simulink graphic control instructions into real-time code, which sends command and measurement signals to the robot via the data acquisition and control card (a Quanser Q4 or Q8). All of the controller design block files and math guide the robot’s movement, and it all runs on a real-time operating system (RTOS) called RTX, provided by Ardence. So as the author-side writer writes, the signature is scrawled on the other end at the same time. RTX provides control of interrupt requests, I/O and memory to ensure tasks are executed with proper priority.
With the prototype ready to be refined and improved, one challenge remained: the reproduction of handwriting in a coordinated fashion. Apkarian explained that people don’t write smoothly. Rather, there are quick stops and starts. “There is character in their writing and you want to maintain that.” To do this, Apkarian explained, “we had to come up with algorithms that would maintain maximum performance of the robot while maintaining those edges. The writing process can create signals that are erratic and high in acceleration. If you have a square wave and start filtering it you then have a round square wave. If someone draws a square, it has to look like a square.”
Scaling time and tricking the eyes
To make the robot respond the way the hand responds would require powerful motors and a huge device. However, the systems needed to be small and portable. “So in order to compensate for the [smaller] motor’s inabilities,” Apkarian said, “we came up with advanced control schemes that would again not require so much performance … but still result in smooth and continuous, accurate motion.” They had to “scale time” or “stretch time” by breaking down the movements, essentially tricking the eyes into thinking the robot is writing fast. » View a demo video and read arguments against its use on the next page
So it was done. Unotchit and Quanser had created a robot that would replicate the author’s signature from anywhere in the world—either in real time or after the writer has approved their message—over a TCP/IP Internet connection. Gibson called the final product “a very swoopy, elegant solution.” The author with laptop is at one end and the robot with book at the other. From the author-end, data protocols are set up—start bit, stop bit, up-down and X-Y coordinates of the position they want to write—and the pen pressure is measured by a laptop tablet. The data streams to the robot, while algorithms smooth out all the missed points. The robot completes its task as the author lifts the pen from the page.
It is the first of its kind, but not everyone is happy about LongPen. Some people want to know one thing: why? Why eliminate the one-on-one experience between author and reader? She insists the machine doesn’t alienate the fans: the machine is designed to maintain the author-reader experience. Streaming video allows the parties to talk, and users can even save their video exchange.
“For many,” Atwood writes on the company website, “it won’t be a choice between the author-in-the-flesh and the remote signing. It’ll be a choice between the remote signing and nothing.”