[INTERVIEW] AR 3BROTHERS: Let’s have fun with AR from now on

From markerless technology to the reproduction of smells?
In a time when it is no longer acceptable to say “I don’t know”, we talk about the AR of today and the future.

Have you heard of the term “Augmented Reality“?
Before we explain what it means, please watch the following video first.

AR3Bros episode-ii | twitter & NENPYO & namecard & AR from ar3bros on Vimeo.

The character and the speech bubbles that appear on the business card look as if they have just suddenly appeared right on top of it.
This visual display is one example of AR and it goes through the following process:

1. A web camera takes a picture of a 2D bar code (marker) within a black border printed on the business card
2. The program recognizes the marker
3. The text and 3DCG animation are matched with the movements of the real image, and are then combined and displayed in real time.

In effect, the 2D bar code (marker) within the black frame acts as a trigger to run the AR system.

AR is known as a system based on a program and a device that can display images in real time, such as a web camera, that displays additional information over an image of the real world, in the form of 3DCG animation, text or sounds (for more details please refer to Wikipedia).

AR3bros is known as the leading group for the creation of augmented reality (AR). This time we will be talking to Tom Kawada, the eldest brother, and Shinji Takagi, the second brother, about the augmented reality of today.

At one point, AR used to be spoken about in relation to devices such as the Sekai cameras, but they are now becoming more mainstream as the technology to be used in advertising and entertainment. Maybe moving forwards, AR will become a “must have” within the various choices available when designing content or when doing a promotion campaign. We asked AR3bros about the status of augmented reality today.

(Author: Ouji Mori)

AR3bros Profile

Eldest brother: Tom Kawada
Second brother: Shinji Takagi
Third brother: Yu Ogasawara (unavailable this time)

An independent AR group specializing in the production of future developments, which has been in operation since 2010.
Before becoming independent, they worked at JUKI Co., Ltd. in a variety of projects such as creating patented technology for connecting sewing machines to the net, developing WebPartsList ver.2 which allows XY coordinates of CAD data to be analyzed automatically, and producing exhibition booths (including images, music and design).
They have won a number of music video awards such as the Oyamada Keigo Award, the [[[Adobe Records]]] Excellence Award for Interactive Art and the Excellence Award for Music Clips. The video that won them the [[[Adobe Records]]] award, resulted in the production of a CD, and it also instigated collaborations with others in a variety of different areas such as fashion, music and film.

ALTERNATIVE DESIGN++ http://alternativedesign.jp/ 
twitter: @ar3bros http://twitter.com/#!/ar3bros

The world’s first! A fashion show made only with AR

This is the first fashion show in the world to be held in the AR space, done by AR3bros in conjunction with fashion brand Theatre Products. When you hold a camera over a floor plan, a model appears over the AR space. A show then unfolds based on that room’s life style.

AR-FASHION SHOW_COMPLETE from ar3bros on Vimeo.

The AR sheet (floor plan) can be obtained in PDF format from the Theatre Products web site.(http://www.theatreproducts.co.jp/)

“To a certain extent, fashion shows are well established in reality and are becoming more and more boring. So, we thought maybe we could change this by using an augmented reality? With this in mind, we decided to go with this slightly new idea of implementing it with AR,” said Mr. Kawada, the eldest brother.

Shinji TakagiMr. Takagi, the second brother, in charge of visual display. In the AR fashion show, we film the objects and the models separately and then join them together, making it look as if there was a person there in the form of a 3D image.

“When we had just started working, Sekai cameras were still the boom and so people used to say things like ‘what’s AR?’, ‘And, what’s AR3bros?’. But now, having tried out a few projects so far, we have started doing collaborations with lots of people. Now that we are finally being considered as the ‘leaders in AR’, we feel that this is the time to come up with more practical ideas and spread them around,” said Mr. Takagi, the second brother.

From the stance of an “AR expressionist”, let’s have fun with AR from now on

AR3bros is a group comprised of three people who work at the design office ALTERNATIVE DESIGN++, which was formed to do research into AR technology. Each one of them has his own role, Mr. Kawada, the eldest brother, is the brains behind their conceptual ideas; Mr. Takagi, the second brother, is in charge of visual display, and Mr. Ogasawara, the third brother, is in charge of music.

“In this place that we call society, we think that our position can be described as something like this:

In the past: they just said, ‘he’s a funny guy…’

In the present: from time to time they would say ‘ah, it’s that funny guy!’

(From now on: ‘that funny guy, wonder what he’s doing now?’)

In any case, our mission has been to build our research knowledge on AR in our own way, and to express it successfully. We are fully aware that we stand in the gap that exists between people in society and researchers. And thus, we like to cherish the fact that we can show something to people, smile at them and say ‘this is really fun’, that we can all have fun together. We believe that this allows us to see the future from a different point of view compared to the researchers, and allows us to give encouragement back to the researchers as well,” said Mr. Kawada, the eldest brother.

In fact, researchers have visited AR3bros’s work base in the past, and AR3bros has also been invited to visit research labs and universities as well.

“Fundamentally, everything that we do has no precedent, so not having a pattern to follow is what makes it difficult. For example, the other day on an NHK program, they showed a project with Sachiko Kobayashi where a video from Second Life was combined with the scenery of the Shinjuku Alta building in real life. On top of this, they also added videos of the NHK’s music show Kohaku Uta Gassen. This meant that they had to skillfully combine the reality inside TV, with reality’s Shinjuku Alta building and the view of the world inside Second Life. This requires a lot of image editing across multiple dimensions. Working with images from a new perspective that no one has attempted before is of course worth doing, but at times it can be rather lonely (laugh),” said Mr. Takagi, the second brother.

It is amazing that even unprecedented, ground breaking projects like this one can basically move ahead all from just one sheet of paper written by Mr. Kawada, the eldest brother. Ultimately, you could even say that this absurd way of working is, in some ways, shaping the characters of these three brothers.

AR3bros can start creating just from these draft proposals. Close by, you can see some Pentel felt tip pens, indispensable for Mr. Kawada to do his work.

Play back smell!? the future of AR

AR3bros previously applied AR to a film called “Eden of the East”. With this system, which was named the “Eden of the East System”, people were able to submit comments about the film sent from cell phones and smartphones which were then displayed in layers. In this way, AR3bros managed to instigate changes at movie theatres where the use of cell phones was generally banned, thus creating a new view of the world.


Eden of the east vol.1 from ar3bros on Vimeo.

Original markers were usually used as the entry point into the AR space. However, in the “Eden of the East System” they did not have any obvious markers, but instead they used the characters of the film themselves (in particular their color) to provide the sensor functionality, which introduced the great concept of “markerless” technology.

AR represents the addition of a new reality to an already existing reality. Hence, markerless technology allows the augmentation of everything that exists in reality. Although on the one hand, its basis on reality imposes restrictions in terms of copyright, on the other hand, one can expect it to be useful for CDs, magazines and product labels.
Furthermore, in the future, we may not only have this type of AR based on visual senses, but apparently it may even be possible to create AR with a sense of smell as well.

“Sensors for tiny amounts of smells currently exist, which can identify 400,000 different types of smells. Once the technology to scan these smells becomes widespread, what are the cultural changes that could occur? For example, in long distance relationships, dialogues such as the following are quite likely to arise:

Woman: ‘I guess today we have to say goodbye.’
Man: ‘It’ll be another week before I see you. Hey, can you let me scan your smell?’
Woman: ‘Oh, you’re impossible! ♥’

And then the man can reproduce the scanned smell so that he can enjoy it later on his own. This future will definitely arrive. Because in my head the couple is already messing around,” said Mr. Kawada, the eldest brother.

The secret for crossing over a dimension is to learn from “the transcendental”

Last year, AR3bros announced that they would undertake ridiculous projects with objectives such as “Bringing AR into Sachiko Kobayashi” and “Firing beams from BEAMS”, and somehow, in one year they have actually managed to achieve their goals.

Starting projects so daring which, in the first instance, make you want to say “there’s no way it can be done!” but in the end, they swiftly manage to achieve them. That’s what these three brothers do. Where does this power for generating ideas and executing actions come from? Mr. Kawada explains that their motto is “to always go and see people that you are interested in meeting”. Meeting with people who show excellence in their specialist areas and go beyond into new dimensions stimulates them every day and allows them to refine their ideas and good sense.
Recently, through the “TVBros” magazine, they met and had a stimulating conversation with Dr. Nakamatsu, the inventor of the floppy disk and the karaoke machine, who told them a number of things such as “Edison was after an inventor of the town”, “people who discover logic and invent things are the genuine inventors” and so on. Also, they have had exchanges with people from different industries who are “out of the ordinary”, which have given them the incentive to continue expressing AR in a dynamic way.

AR3bros are expanding their field of activity and growing more and more; this is a group that cannot be overlooked.

By the way… if you use a framework called the FLARToolKit, you can implement AR with Flash(ActionScript3.0).
To those creative people out there who think “I could do much more fun projects!” please try challenging AR3bros!


森 オウジ フリーランス・ライター 
@MORIOG http://moribox.com/

page top

Related article

  • img

    Message from loftwork

    awrd.com Ver7.0 toward the EvolutionMitsuhiro Suwa / loftwork Inc. co-founder CEO[caption id="attachment_487" align="alignleft" width="240" caption="Mitsuhiro Suwa"][/caption]awrd.com started in 2000. Almost 11 years have passed since then (amazing!). Our website might be one of the oldest web servi…

page top