Updates

Here's a quick rundown of projects, and some of the technologies I'm working with..

The Social Computing Room at RENCI has been busy...

• I'm working on a new media projects that uses Max/MSP/Jitter to create MIDI music using tangible objects with embedded UbiSense tags as the 'instrument'.
  
• I'm working on installing a very cool Flash/Flex based media project that uses all four surfaces of the Social Computing Room. This work has been installed elsewhere, so it's an adaptation. It really looks cool! This has allowed me to learn a bit more about Flash and Flex. As a programmer, I 'get' Flex much better than I get Flash.
• I'm working on a virtual worlds project, utilizing a 360-degree Second Life client to stage mock trials.

I'm still looking at the InfoMesa technology demonstrator, and building an application based on WPF technology for the large display environment in the Social Computing Room.

I've been working mostly on the back-end, creating a services layer for storing metadata (the part I'm working on now) and for accessing arbitrary data stores based on the metadata. The metadata service layer is pluggable by interface, and my first implementation uses NHibernate to store metadata on a back end server. Once this done, that metadata layer can have pluggable modules for things like cloud databases.

For the pluggable data stores, the first stores will probably be a mounted file system, then a database, then an iRods repository. After this, in order, it will probably be http, ftp, then cloud databases.

The Social Computing Room will be integrating a multi-touch table later in the summer, and therefore I'm kicking some of the user interface stuff down the road. I want to allow folks to sit at the multi-touch table and interact with arbitrary data stores, manipulating on the touch table, and viewing on the 360-degree display. This would be soooooo cool.

Serious Games

I'm learning about the Unity3d game engine in my 'spare cycles'. We've ported a few Unity projects to the dome, and blogged about it.

Lots of things going on, as you can see. Main technologies I've worked with in the last two months:

• .Net, C#, WPF
• Java, including some Jetty work, and some socket stuff.
• Max/MSP patches
• Flex and Flash development
• Wordpress and a bit of PHP
• Quicktime Streaming Server and Quicktime Broadcaster
• Unity3D
• Second Life building and LSL scripting.

This is why I like my job...now I busted some solder joints Friday doing some testing, so I've got to play with a soldering iron.

Sitting on Top of the (virtual) world

With apologies to Howling Wolf, Here's a pic we took in the Social Computing Room yesterday. It's showing an adaptation of the Second Life client that provides a 360-degree view of the virtual world. The development work on the SL client was done by a colleague, David Borland.

The idea is to embed the Social Computing Room within a larger virtual space, so that you can look out in every direction to see what is happening in the virtual world. In the pic below, I'm in the SCR looking at my avatar...



How about looking in? We have video cameras from all angles, so I thought about building a 'fish tank' from the SL perspective so that avatars can walk around the SCR and see inside of it. The bread and butter of two-way audio and video, along with text chat is the obvious next step. Here's a shot from SL looking into the Social Computing Room...


Right now it's just a bare media object, and one interesting question I have is what to build on the virtual side? Do I want to mirror the room, or perhaps the SCR could be sitting on the bottom of the sea? Once the basics are tackled, the SCR and the 4-channel Second Life client make for a unique research space. The SCR is well suited for installation of all sorts of sensors, robotics, and input devices. This is something of a side project, but I've noted how intrigued people are when they visit the SCR and see the first prototypes!

Hands-free control of Second Life

Cool video, though this interaction style still seems a bit awkward. I think the more interesting idea is the capture of gestures, which would be great for speaking to a group via an avatar, for example.

video here...

Virtual SunSPOT controlled by a real SunSPOT

Pardon the zapruder-like quality to this film, but this shows the hack I mentioned in my last post. I'm in SecondLife, controlling a virtual SunSPOT from a real one. In this case, tapping into the 3D accellerometer to pick up the xyz rotation, sending it through my framework to rotate the virtual one. It's a bit laggy, and not 100 percent there, but enough to get the idea.

If I ever find the time, the next cool example would be to implement the ectoplasmic bouncing ball demo using one real and one virtual SPOT. Anyhow, it works. The point really is to learn about the SPOT, and why not do something interesting while testing them...?

SunSPOTS talk and demo today at Sitterson

Paul Jones sent out this note, and I'll be attending for sure:

About SunSPOTS http://www.sunspotworld.com/

Where: Sitterson 014 When: Thursday November 15th at 3:30
Who: A member from the Sun Labs, David Simmons http://blogs.sun.com/davidgs
More:

David has hands-on experience in building applications for SunSPOTs and was instrumental in its design and development, will be on hand to offer his insight into this amazing product. http://www.sunspotworld.com/

The SunSPOT (Small Programmable Object Technology) was developed in the Sun Labs and represents the future of embedded systems. Already used throughout academia, students and professors alike are finding new and interesting uses for SunSPOTs. Each SunSPOT comes equipped with a processor, memory, eight external tri-color LEDs, light sensors, temperature sensors, an accelerometer, and several digital/analog inputs and outputs; offering up seemingly countless practical uses.

At its core, a SunSPOT is an embedded system. But, unlike other embedded systems that must be programmed using a low-level language such as assembly or C, SunSPOT applications are developed in Java. By allowing Java applications to be uploaded and run on an internal Java Virtual Machine, Sun is not only opening up SunSPOTs to more users than many other embedded systems, it is also leaving the final function of each SunSPOT up to the end user. By following a simple API with which to interface the SunSPOT, developers nationwide have created unique uses for SunSPOTs - everything from animal research to rocket testing and much more!


I'm currently working with the SunSPOT developers kit, and have been going through (and hacking on) the demo apps. One of the first things I am trying is to tap into the 3D accellerometer. I took the telemetry example and added tilt to the packets coming off the SunSPOT, and have that available on the host. At the same time I've created a virtual SunSPOT in Second Life, and have scripted that to mirror the pitch, yaw, and roll coming into the LSL script. Just a few more tweaks, and the virtual SunSPOT will be controllable from a real one. This has been done before, but not to Second Life. The lag will probably be pretty bad, but I want to explore how multiple SunSPOTS, used by different people in an immersive environment, can create cool experiences.

Anyway, here's a shot of the virtual SunSPOT, when I get it hooked up, I'll shoot a video. I might have it by this afternoon, if the creek don't rise. Anyhow, see you all at the talk this afternoon!

Turning Turing Around

I was reading Irving Wladawsky-Berger's blog today when I happened upon this wonderful observation..

I was reminded of the Turing Test recently, as I have been watching the huge progress we are making in social networks, virtual worlds and personal robots. Our objective in these applications can perhaps be viewed as the flip side of the Turing Test. We are leveraging technology to enable real people to infuse virtual objects - avatars, personal robots, etc - with intelligence, - as opposed to leveraging technology to enable machines and software to behave as if they are intelligent.

What intrigues me so much about virtual worlds like Second Life is this ability of avatar-based virtual spaces to allow you to push through the barrier, and cross over. How's that for a bunch of meta-physical BS! This is a different aim then something like Looking Glass, which is trying to apply a 3D metaphor to a 2D interaction...it's about stepping through to live with the data, or the sensors, or the other distant collaborators. As the real world becomes more inhabited by pervasive computing, it only seems natural that we go and visit the virtual on its own turf. One wonders about the definition of an application interface in the future. As machines grow smarter, perhaps we'll pop into the 'living room' of our personal agent to have a chat.

At any rate, there are a couple of fun things I'll be looking at in the near future that can tie in to these ideas. First, the idea of pervasive, wireless sensors everywhere. I'm waiting for a SunSpot Developers Kit, and there will be some sensor applications coming down the pike that could involve these extremely cool sensors. The fact that they use Java is a plus in my book. Needless to say, I'll be brushing up on my J2ME.

The next thing I see coming down the pike is real time location tracking, using the UbiSense platform. This is being leveraged for an intriguing space called a Social Computing Room, and has all sorts of potential uses. Here, I'm going to be doing some .Net programming.

Like the blog quote above, I've had a unique chance to push the physical into the virtual, and with the mentioned projects, there's a chance to work in the other direction. Where these meet is getting to be a pretty interesting space!

A goldmine of information about Second Life and Education

This came to me via Kathy Kyzer at ITS-Teaching and Learning, who have done wonderful work on Second Life (visit the UNC island sometime).

These are the proceedings of 2007 Second Life Education Workshop (PDF warning), lots of information about experiments and experience using Second Life in an educational setting.

The realities of virtual worlds for corporate sites

There has been a swirl of hype, and anti-hype around the idea of 'Serious Virtual Worlds'. Of note were the Wired article, as well as a blog posting by Chris Anderson that generated a lively exchange. Even more recent was a Gartner report cautioning corporate America about the risks of doing business in Second Life.

Metaversed, yesterday, held a really nice event in the SAP space on Silicon Island on Corporate Challanges in Virtual Worlds. The panel was:

• Craig Cmehil - Community Evangelist, SAP
• Fiona Gallagher - SL Project Lead, Sun
• Jeff Barr - Web Services Evangelist, Amazon
• Jonas Karlsson - Research Team, Xerox

Here's a shot...hardly a 'ghost town' for tech firms in Second Life!



The panel seemed to be in agreement that much of the recent coverage of virtual business was uninformed, or based on misconceptions. As an example, Gartner cautioned that corporations could not control access to their virtual sites. Expectation plays a big part here too. I think it is true that corporations that expect to build a virtual commercial for a product will be disappointed because nobody showed up. Second Life seems driven by events and gatherings, and is very much a socially driven animal around small networks. Increasingly, real life blogs, and things like Twitter are playing the same role in virtual space as they do in real life, acting as an alert system for happenings, and reporting events later to a wider audience.

The virtual medium has characteristics that distinguish it from the web. Instead of an ad that is encountered for a brief few seconds by a wide audience, virtual interactions involve deeper contacts with customers or contributors in small groups. It was pointed out that surveying a corporate office and seeing it empty can be misleading. The vendors represented pointed out the fact that one must understand the point of a particular build, it could be used for various events, could host customers for private exchanges and training, and also has ripple effects in the 2D media that must be counted in any calculation.

This was alluded to by one of the presenters, but I think about trucking out to the Airport Sheraton for Oracle Tech Days, or similar, vendor-driven events. In those cases, vendors ship out their employees, arrange conference space, and put on an all-day show to a small group. How is this any different from the corporate uses described by the panelists? Not very!

Anyhow, the Metaversed podcast carries the content. I continue to be impressed with the richness of these experiences in virtual space, including all of the networking that occurs before and after these presentations, which in itself demonstrates one of the true, unique properties of the 3D web.

I took a tram into the fourth dimension
Cos I had the blues, the blues of throwing it all away
Just gimme a Tequila, I'll slam it the 4-D way
And when I got there you know it had certain similarities

Like no smoking anywhere
And hiding in the khazi to avoid paying the fare
4-D Tequila anyone
And dont think we didn't dance to records by the Fifth Dimension

-Joe Strummer

Interesting SL Event Today...

I picked this up from Metaversed...this is at 12PM SL time today. People from Sun, Amazon, Xerox, etc...

There's been a lot of negativity in the press of late over the marketing failures of corporations in the virtual world of Second Life. Analyst firm Gartner have even warned companies away from public worlds recently. With all of that in mind, Metaversed has put together a panel of active real life firms in Second Life to discuss their experiences, and lessons learned from being part of the community. The debate will no doubt prove useful to others and be of great interest to anyone involved in the business side of virtual worlds.