another Microsoft internal newsletter about some pretty cool techniques from
Microsoft Research that have been integrated into Microsoft’s new Digital
Image software. IMHO, there’s just no decent imaging software.
I was a huge Jasc Software Paint Shop Pro 6 fan. That’s version 6,
mind you. It was a fantastic editor. It had powerful techniques ala
Photoshop with a simple and elegant interface. Later versions became far
to complex with a schizophrenic move towards supporting both vectors and rasters
and layers in one confusion interface littered with toolbars and
Photoshop is too
advanced for the average user, Paint is joke. The original Microsoft
Digital Image Suite was a train-wreck called “Picture-It” that had
the unfortunate habit of crashing by disappearing. You know these special
crashes…no Doctor Watson, just -poof-, gone.
The new Adobe
PhotoAlbum suite has some pretty impressive organizational skills, most associating
event and subject tags around the time the photo was taken. But, will
Microsoft get it right this time with a little help from Microsoft Research?
Side Note: It
would be nice if Microsoft would decide what App or Applet should be used to
view JPEGS. I’m writing this post on a fairly fresh Tablet PC, and
I’m presented with this menu when right-clicking a JPEG:
Ok, so I can
open it in the “Windows Picture and Fax Viewer” – hm, I open
pictures a LOT more than Faxes. I’ve also got Paint. Hm, then
IE. Also the Microsoft Office Picture Manager which is a GREAT tool, but:
1. it is relegated to the never used “Office Tools” program group,
and 2. it doesn’t associate itself with graphic files until you run it
once…but you have to discover it before you can run it! Seems since
Microsoft is so gung-ho on Digital Photography, that someone should organize a
Unified Front within the org and work this out.
is being written in a new way. It is being written by people through the big and
the small events in their lives. We write history through personal Web sites,
discussion boards, and the legacy of photos, taken at moments that are important to us.
at Microsoft have been working on a wide range of technologies that will help
people write their personal histories through digital photography. To tell any
history, it's best to start at the beginning, and move forward.
beginning, you buy a digital camera, take a quick look at the manual, throw it
to the side, and start pushing buttons. Digital photography has made it
possible to take an almost endless number of photos. Some of these photos are
good; some get deleted before anyone else sees them. Many of them are almost
good, with a little tweaking they'd be just fine.
A common problem with digital photography is lighting. Photos turn out either
too light or too dark. Since digital cameras allow us to take lots of shots
without running out of film, we're willing to throw away a lot of the bad
shots. But if you want to take a great picture, it might be nice to be able to
control the lighting without purchasing professional lighting equipment.
the research projects at Microsoft Research is called Continuous Flash. This
technology allows you to take the same picture with flash and without flash and
later adjust the balance between light and dark. It's better than contrast
filters in photo editing tools, because it considers the reflection
characteristics of each object in your picture.
can't compensate for having one area underexposed and one area overexposed,"
said Hugues Hoppe,
one of the project researchers. "If an area is underexposed, you can't
really get the detail back by increasing brightness, because it wasn't captured
in the first place. By having two different images which both have useful
information, you can merge them together."
A similar technology that combines the best of multiple photographs is a
project called Image Stacks. Image Stacks aligns multiple images of the same subject,
allowing the user to pick and chose the best pieces from each photograph.
Cohen, Steven Drucker
and Alex Colburn
thought this would come in handy for special events, when you want to get a
picture of the entire group that's suitable for framing.
group photographs is difficult, because capturing a single image in which
everyone looks good is almost impossible. What usually happens is that in one
shot, someone has their eyes closed, but someone else has got the most adorable
smile. Check the next shot, everyone has their eyes open, but one person is picking
a poppy seed out of their teeth. The third shot, both previous people are
behaving, but grandma is yawning, tired of waiting through multiple shots.
Which shot do you pick? With Image Stacks, you can easily cut and paste to
present everyone's best face. The images are automatically registered into a
single composite image.
Print photos are still around. They're around in shoeboxes. They're shoved
under the bed and in the back of closets. Some of us have so many print photos
we don't know what to do with them. Neatnik types tediously scan all of their
print photos one-by-one, converting them to digital form. But most of us don't
want to go to all this trouble.
Herley, a researcher in the Communications,
Collaboration and Signal Processing (CCSP)
group, has developed a way to allow people to scan multiple photos at one time.
You can put as many photos as will fit on your scanner, and the software will
recognize each photo separately. It can 'read' the edges of the photos, even if
they're crowded together or tilted. When it converts them to digital photos, it
will correct for orientation and position. It's a quick way to get the family
memories out of the shoebox and onto your hard drive.
a harder problem than it looks like on the surface. Many scanner makers have
tried, but it hasn't worked before," said Herley. "But this really
works, it's not just a demo."
Out the Red
The Media Computing group at Microsoft Research Asia has developed several cool
image editing techniques to help you fix some common problems with any photograph.
One of the worst that comes to mind is the glowing red eyes that result from
the flash hitting the pupil in just the wrong way. Unless you're way into the
vampire look, this just isn't right. The red eye fix, which currently ships in
Digital Image Suite and Windows XP Media
Center Edition, is one of the best photo retouch features around. All you do is
move the 'target' over the red eye, click, and the red is out.
group has also developed a quick and effective way to fix the brightness and
contrast in a picture. Even if you took a picture that is too dark, you can use
the Levels Auto-Fix feature in Digital Image Pro to lighten up your picture.
the Digital Shoebox
The Media Computing group
has also made it easy to organize your digital photos using image recognition
algorithms. Some people do the same thing they did with the shoebox, and
scatter pictures all over their hard drive. Then they can't find the one
picture they want to use for their holiday card. The group's algorithms can
distinguish between indoor and outdoor shots, shots with people in them and
shots without, and city and non-city scenes. Combined with other technologies,
such as keyword annotation, it makes it a breeze to find any photo in your
a researcher in the CCSP group has developed another way to manage your photos
online. His image clustering algorithm helps users find their photos by one of
the most prominent markers: events. The software is effective because it
doesn't look only at timestamps, which could be misled by a faulty camera
clock. Instead, the software looks at photograph order plus color to find
pictures taken during one particular event.
only compare colors locally in time," said Platt. "So if you have a
pumpkin in one shot, and a few months later you wear an orange shirt, later,
when you're searching for the day you visited the pumpkin patch, it won't show
photographs from when you were wearing the orange shirt."
algorithm underlies several other technologies designed to help people find
their digital photos, including the Microsoft Research Media Browser. The Media
Browser, developed by researchers in the Next Media
group, takes advantage of the photo recognition research from the Media
Computing group, and Platt's algorithms to build a unique visual experience
that helps you search for and identify your photos. And it looks darn cool as
it works. The interface is an impressive, futuristic presentation of photos
that rearrange themselves before your eyes, sliding into place in a typical 2D
presentation or a 3D stack.
idea behind this is annotation of large collections of photos," said
Steven Drucker, the lead researcher on the project. "We know that if you
put annotations on photos, that it's much easier to retrieve them. But we also
know that it's tedious and difficult to do. We use the advanced techniques that
are available, such as face detection and image clustering, to make it easier
for you to interact with your photos. We also use a game graphics card for
higher visual quality."
Smart Erase is a photo editing tool found in Digital Image Pro and invented by
Perez in Cambridge.
The feature allows users to remove objects from a picture. This can come in
handy in case you want to remove your ex from the family reunion picture, or
you before you lost the 30 pounds.
algorithm looks at areas of the image to see which patch of texture can be
"stolen" to fill in the holes left behind when the unwanted object in
the image is removed.
in the hole, Smart Erase does some reasoning about texture. It views the pixels
outside the object as potential replacement material. The program has some
strategies for knowing exactly where to look to get this material. "The
algorithm constantly reviews what pieces it's got and makes comparisons very
quickly to come up with the right fit," said Andrew Blake,
Senior Researcher in Cambridge.
Another photo editing feature from the Cambridge
lab, a tool code-named Blender, appeared in Digital Image Pro this year as the
"Blending Brush." Blender is a seamless cloning tool that can take
the wrinkles out of your face, insert a new object into a scene, and combine
parts of one scene with another - all without the usual difficulties and
distortions that most photo editing techniques present.
object inserted into a new background has complex outlines, standard cloning
may not work because of the incompatibility of color and intensity between the
background and the new object. And even the best, most careful cutting and
pasting often yields poor results because the outlines are fuzzy or jagged.
Blender 'blends' pieces of the inserted object and the background together to
form a seamless whole.
Cutting out an image and putting it somewhere else has always held a lot of
fascination for photo aficionados. Blake and his team are developing a new
algorithm they call GrabCut, that's a 'no-brainer' way to do this important
task. Instead of having to carefully trace the outlines of the object you want
to cut out, all you have to do is draw a rectangle around the object. The
algorithm selects the object and eliminates the old background. You can then
paste the cut-out object onto a different background.
Doesn't everyone want to star in their own anime or Disney cartoon?
Asia researchers can't get you a Disney contract, they can turn your digital
photograph into a cartoon. Their technology, developed in cooperation with MPD
Japan, is called the Cartoon Wizard. It is currently offered in the Japanese
version of Office 2003. Westerners will have to wait, as the Cartoon Wizard is
only trained to work with Asian faces.
system is based on statistical learning techniques. The algorithm automatically
generates a cartoon from an image using face detection and alignment, and
training data generated by studying how a human artist renders a human image
into a caricature. The resulting cartoons can be used in e-cards or
personalized emoticons for chat programs.
Now that you've stepped through the process of improving and organizing your
digital photos, perhaps you'd like to share them. Microsoft Research has
developed several ways to do this, in small and large ways.
researcher Dave Vronay was working on PhotoStory, he wanted to recreate the
feeling of a family sitting around an old-fashioned photo album and telling the
stories connected to the pictures.
picture is not just a description of what is there," he said. "For
instance, if you have a picture of a hotel, and you showed it to a friend, you
probably wouldn't just say, 'and that's the hotel we stayed at.' You might
instead launch into a story about the waiter with purple hair who served you
duck soup at the hotel restaurant, even though you didn't have a picture of
him. The photo would be a reminder of the stories that surrounded that
Photostory, you can add images, music, and background narration to tell the
stories behind the pictures and send it to the people you'd love to have
sitting on the couch next to you while you share your memories.
Your Photos with Friends
Computing group is experimenting with an online blog
and photo sharing application code-named Wallop, a project designed to help
people to connect with those close to them — families and friends, and
friends of friends.
considers Wallop a "social networking" application that provides a
way for small, closely connected groups of people to share personal information
and photographs online. The beta testers can send photos to their Wallop
interface through email or instant messages to easily update their blog
Your Photos with the World
The World Wide Media Exchange (WWMX) offers users from around the world the
chance to upload and share their photos with millions. It provides MapPoint
maps and TerraServer maps so that you can view your photos by location as well
the advantages to this interface is the ability to communicate with people
across the world. If you're planning a trip to London, for instance, maybe some nice tourist
who has gone before you has posted their pictures of a trip around the city,
complete with shots of their favorite tea stops. Then other tourists or locals
can jump in and write annotations on the photographs, such as, "don't eat
here, the crumpets aren't up to the usual standards."
the contributors to the WWMX
have contributed to history by adding photos of 'news' events, such as fires in
Southern California and search and rescue
Salesin, a senior researcher in the Document
Processing and Understanding group, has inspired many digital projects at
Microsoft Research. He is also on the faculty at the University of Washington.
Salesin recently became actively involved in a large digital photography
project. A very large project. He contributed original digital photographs from
his trip to Bhutan
to the world's largest published book, a visual journey across the last
unspoiled Himalayan kingdom on the planet.
project was funded by several sources, including the Bill & Melinda Gates
Foundation and the iCampus
program at Microsoft Research. "Mike's
project seemed like an interesting, original take on how we might be able to
use technology for education," said Salesin. He helped convince the
iCampus funding committee to support the project.