Let me warn you, this post is certainly written for a niche audience. But I just made a discovery that I hope can help other people who have to deal with the same issue I was having.

So here is the situation: At ACNE Production we’re currently developing a series of Rich Media banners for one of our clients. The challenge for us is that we have a lot of different clients that use different media partners to publish and host their banners, and all of these media partners have different ways of doing things. Often figuring out the way things work with this particular media partner takes up a significant part of my time as a technical director, since I’d rather have the developers focusing on writing code. We just did a couple of banners using Google’s DoubleClick Studio, and this time the mediapartner is Mediamind, who merged with Eyeblaster last year (I think). Eyeblaster have been around for some time, and it seems they’ve been developing their framework for a LONG time. It’s certainly very big and complicated.

The problem is that to develop an Eyeblaster banner you have to build it in the Eyeblaster Workshop, which is a proprietary plugin for Flash. Once you’ve installed that plugin you have access to a very sophisticated toolbox with templates and even a sandbox for previewing as well as publishing capabilities. Unfortunately everything happens behind the scenes; You don’t know where any of the code comes from, you just know that weird classes are imported and that static objects are referenced from the timeline in the template files.

The problem is that no serious Flash developer would ever user the Flash timeline to write any code, other than maybe a stop(); action inside a graphical asset. Any code beyond that should be written in an IDE, such as FDT or Flash Builder. Our tool of choice at ACNE Production is FDT, currently version 5.5. Now, since all the mysterious Eyeblaster magic happens behind the scenes, trying to develop an Eyeblaster banner using an ActionScript IDE will result in a ton of reference errors inside your project, since all the Eyeblaster code is nowhere to be found.

With DoubleClick studio it was fairly simple and worked the way you would expect it to. There is a plugin that you install using the Adobe Extension Manager, but there are also several SWC files that contain compiled versions of the code to be used in your ActionScript IDE. With Eyeblaster, there was no such thing, at least not on the surface. So I thought that I would try and break open the .mxp file that contained the Eyeblaster Workshop, hoping to find a bunch of SWC files in there.

Unfortunately, opening an MXP file was no easy task. I assumed it was just a zip-archive, but I couldn’t get it to unzip. Enter Gooogle, that told me about an ancient piece of software called MXP Lister. And this is where it gets really old school: MXP Lister is a plugin for Total Commander(!) I actually couldn’t believe my eyes. One of the developers at my old factory used Total Commander on Windows back in 2007, and even back THEN it was totally old school, although for a semi-old geek like myself it brought back fond memories of the MS DOS days and Norton Commander or even Directory Opus on the Commodore Amiga system.

So I booted up the trusty old office Dell and installed Total Commander and the MXP Lister plugin, hoping to reveal the dark secrets of the Eyeblaster MXP package. And I certainly did – the MXP (Macromedia eXtension Package) contained MXI, which is an XML metadata file that describes what to do with the contents of the package. And this revealed that it copies actual ActionScript classes into the Flash Library. Since Flash Pro intrinsically (and secretly) understands and relies on code in there, there is absolutely no reference to that directory whatsoever in the publish settings for the .fla files created by the Eyeblaster Workshop.

Anyway, I was able to find those classes and copy the code into the project we’re working on, so that we can use a decent IDE to write the code instead of having to write code in the timeline. In case you’ve forgotton where Flash stores it’s internal classes (and I had), here it is (on OS X with Flash CS6 installed):

/Users/{USER NAME}/Library/Application\ Support/Adobe/Flash\ CS6/en_US/Configuration/Classes/Eyeblaster\ ActionScript\ 3.0/eyeblaster

So there you have it – a walk down memory lane to the days of writing code in the timeline and even further to the days of File Managers. While I’m sure the Eyeblaster Workshop works out just fine for designers who don’t write any code but just rely on simple actions and don’t care HOW it works, just that it works, it just isn’t a good solution for a developer. I hope EyeBlaster realises this and provide us with SWCs instead of the “idiot”-proof template files. I also hope someone can use this information in the future, which is the reason I wrote this post.

Geek Out,
Martin

The GE Performance Machines interactive experience combines a large library of video recordings of three pro football players with interactivity in the form of 9 mini games putting man against different machines built by GE. ACNE Production created this experience in collaboration with BBDO New York and Dinahmoe and I was the technical and interactive director on the project. The site was made Site of the Day on theFWA.com March 13 2012.

Functional Requirements
The project launched around the Super Bowl and was expected to receive a lot of traffic around the time of the launch. Furthermore it was made very clear to us that the client wasn’t a big fan of loaders on websites. Since we were creating an interactive experience it was very important to find out what exactly that meant. Turns out she didn’t like having to wait a long time to get into the main part of a website, which to us is a perfectly understandable objection – our philosophy on loading is that the site should only load what it needs to get going – many interactive websites tend to load everything up front, even though it isn’t needed right away. I believe this is done because it’s easier from a development perspective than having to deal with dynamic loading later on in the interactive narrative. Well, sites aren’t built for developers, they’re built for users, so we do of course need to figure out a way to make the site feel light and the experience seamless to the user.

We were also asked to figure out how we wanted to deal with users on mobile platforms. It wasn’t a requirement that the experience should work on mobile, but in creating a brand experience it’s of course important to take all platforms into account, since lots of traffic comes from various handheld devices.

Interactive Experiences in the Cloud
These last couple of years have seen a lot of internet infrastructure being moved from dedicated hosting solutions to some kind of cloud hosting and cloud applications. The typical interactive experience has a fairly short life expectancy, but during that time it will receive a lot of traffic, making it a perfect candidate for being hosted in the cloud. This project was no exception. We were dealing with a lot of video files – in fact no less than 197 in three bandwidth versions for a total of 591 videos. The core functionality of the site is switching rapidly between these many different videos, depending on what the gameplay dictates. Furthermore several of the videos have to start playing not from the beginning, but from a specific point, again depending on the gameplay. One way of doing this is to preload all videos for a given game and storing them in memory for when they’re needed it in the game, but that would require us to load all videos up front, resulting in potentially a very long load time. There is also a limit to how much video, we can hold in memory, and that limit is quite low.

Fortunately Amazon Web Services have a very interesting product that we can use for just this kind of experience. Amazon Cloudfront is a Content Delivery Network (CDN), that among other things have Flash Media Server capabilities in it’s portfolio. The idea behind that is to allow content providers to serve video in a more traditional sense, where a user would watch videos in a video player. But it works perfectly for an interactive video experience like ours – It sits in the cloud and scales gracefully to meet our traffic needs, and the Flash Media Server lets us play any video on demand with very low latency. Cloudfront works perfectly with the industry standard for video player Open Source Media Framework (OSMF), which is the technology being the frontend video used in this project.

Creating a Seamless Experience
While the latency is very low, it’s still there – the result is that videos will occasionally take 0.2 to 0.5 seconds to start playing, which is something the human eye notices. To make this less apparent to the user we worked with our sound partner Dinahmoe on letting the music be the element that stitches the videos together. The music is not embedded in the videos but plays out as a separate element in the experience. This means that when the video stops playing for a second, the music is still playing, and that creates a very good illusion of a seamless experience.

The mobile site
One of the things we’re particularly proud of is the mobile site that supports this experience. The desktop site is the main focus of the campaign and that is what most of the effort went into, but we came up with a simple and fast solution for a mobile site that doesn’t feel watered down. As with any interactive experience we have buttons for social sharing on the desktop site that allows users to Like the site on Facebook, Tweet it on Twitter or +1 it on Google Plus. If I go through my social feed on my mobile device and decide to check out the link to the GE Performance Machines site posted by a friend of mine, I will be taken to a site tailored specifically for the mobile platform, but with the same deeplink that my friend shared. The mobile site features videos capturing the interactive action on the desktop features, but without the interactivity. In fact, the user always wins in the mobile version of the interactive features. If I later visit the same link on my desktop computer I will be taken to the full site.

Sprinting through the Waterfall
This project had a very short timeline and was completed in only 6 weeks. To accomplish this we had to work in an iterative process were all stakeholders were involved from the beginning. Traditionally interactive campaigns have taken a waterfall approach where the developers are handed a bunch of completed Photoshop files and assets and then locked into a basement for a month or two to build the experience. While most software development companies have probably buried the waterfall approach years ago, it still makes some sense in advertising, as it allows the creatives to fully visualize and finalize their design and concept before handing it over to the developers. Later when the developers are released from the basement, that design can be used as a very accurate guideline to review the finished product. Unfortunately this approach is very time consuming and expensive, and furthermore it creates a gap between the stakeholders in the project. That’s probably the reason why production and development people hate it.

Modern software development typically uses an iterative approach, SCRUM being one of the most popular and well-known of those. This approach breaks the project up into clearly defined phases with clearly defined roles, responsibilities and goals for each phase. Developers and other production people love this, because they love clearly defined goals. It’s also a lot more efficient and flexibile than the waterfall approach, since the later phases can be re-defined based on the outcomes of the phases before them. But in our experience an approach like this is too demanding on a lot of people, in particular creatives and the client, as it requires a very high level of abstraction to fully understand how the pieces come together in the end. Another thing that makes an approach like this challenging is the fact that we have a shoot very early in the project timeline. Everybody is locked into the result of this shoot, and we can’t be flexible in later phases to make up for the stuff that went wrong during the shoot or the stuff we didn’t think about at the time.

The timeline on this project demanded that we took an iterative approach, or we would have still been working on it now. Development happened alongside design, and as soon as the edits had been selected from the shoot, we started to mock up working prototypes where we could fit all the pieces together using raw edits and experiment with the gameplay. This allowed our creative people to start playing the game very early on, which again allowed them to create and tweak the design elements to support the gameplay perfectly. Furthermore our client could start playing the games very early on, which was a huge benefit to the project, as it allowed us plenty of time to set the gameplay parameters and the game difficulty to a level that everybody liked. As the CGI was added to the videos and the post work was done, we replaced the raw edits with final videos. As the game gauges were designed and rendered, they slowly replaced the green placeholder boxes in the game, and slowly everything came together and started looking like a real game.

Most of the fall in our office was spent working on the Yahoo! Bus Stop Derby, a fantastic project involving multi player games on huge touch screens. The campaign ended January 28th, but I thought I’d share my experience from being the Tech Lead at ACNE on this project.

Here is the background: ClearChannel is putting up a new line of interactive bus shelters in the streets of San Francisco. These bus shelters feature a 72″ touch screen in portrait mode and are connected to the internet over a 3G modem built into the units. The screens are supposed to be used for interactive advertising and games, and we were given the opportunity to be the first to build an interactive experience for these screens through a partnership with Goodby, Silverstein & Partners who are the advertising agency for Yahoo!

There was a lot of buzz surrounding the campaign and I encourage you to go elsewhere if you want to read what the game is all about. Instead I will write a little bit about the challenges we faced from a technical and user experience perspective.

The user experience challenge:
The form factor in this project presented a whole new set of HCI challenges for us. We have to step away from the conventions we normally rely on when creating digital experiences for the desktop and web as there is no mouse or keyboard, and people have a very short attention span with this thing. It’s more relevant to look to smartphones and tablet computers, but obviously there are major differences between the form factor on a 4″ screen with multi touch capability that you can hold in your hand and standing in front of a screen that’s taller than most people and interacting with much larger gestures.

Take a look at the image below and you will understand some of the challenges we had:

So the person on the left is an average size male adult. The person on the right is a 10 year old boy. How do we make sure both can play these games? There is a limit to how high the child can reach, and while the adult can reach down and reach the lower parts of the screen by kneeling or bending over, this isn’t a comfortable position to be in when playing a game. Another thing is the sheer size of the visual display. If you were watching a movie on a 72″ screen, you would probably want to be standing at least 8 feet away from the screen, but in our case our users literally can’t stand longer than an arms length away from the screen.

We also had to be very careful with relying to much on some of the newer touch screen conventions coming from smartphones and tablet computers. Part of this is because not all bus passengers are necessarily that touch screen savvy, but also because you don’t necessarily think “touch screen” when you walk up to a giant display like this. From previous observations I’ve made with large touch screen installations I’ve found that not a lot of people aren’t that comfortable with walking up to one of these screens and start interacting with it.

So we had to make the user experience very simple compared to what we usually do and we can’t leverage more than roughly 50% of the screen area. The goal is that everybody should be encouraged to and able to play with these things without being hardcore gamers or super tech savvy.

The technical and practical challenges:
We started development of this project before the hardware was ready, so we started by building the project based on assumptions on how it would work and on early prototypes of the actual 72″ units. We decided to build the project in Adobe AIR 2.0, since that technology gave us the opportunity to develop the project very rapidly and to share the work between a large team of developers with experience on the Flash platform. I was technical director on the project and had no less than 6 Flash developers working with me. 1 developer responsible for each game, 1 responsible for the overall UI and 1 developer to help out where ever help was needed.

All the developers had experience with building games and most had experience with touch screen devices, but I was the only one who had created AIR for touchscreen devices previously, so we had to setup a working environment where the individual developers could work and test on their own computers without having to learn too much new technologies. So I was basically responsible for setting up an architecture for them to work in where they wouldn’t have to worry about connecting to the Flash Media Server for multi player communication, how to run and launch the application and how to switch between the games and the UI. Half of the team were working in Flash Builder 4, and the other half were working in FDT 3 or 4. And 2 were on Windows and the rest on Mac. So the solution was to create an environment where we used ANT to build and test each individual element in the application to allow for rapid test and deployment without dependencies on the development platform. Most of the team didn’t have experience with such a cross platform approach, but everybody caught up just fine with the flow without having to spend too much time learning it.

One of the big challenges with this project has been dealing with 3G connection on the units. We don’t want to bother casual bus passengers with error messages about latency issues or loss of connection, so the focus in the error handling has been to make sure the impact on the user is as small as possible. So while the Internet might be crashing in the background, the goal was to make sure the user could go on playing his or her game without noticing something is wrong. Adobe AIR offers some really good solutions for dealing with this situation, as you can do offline storage using this technology. So basically every time we get an XML response back from the backend keeping track of the score, we would store the response locally. That way, next time the unit needs to check the high score, if it fails in connecting to the backend server, it will simply fall back to the local version of the high score XML stored on the local computer. Pretty neat, and the user will never notice anything is wrong.

The high score might not be completely up to date, but there is no way the user will know, and when the internet connection comes back up, the scores will be updated with the latest data, and since it was never down for very long I doubt any users would have the time to travel to one of the other bus stops to spot the inconsistency.

There were a lot of solutions like that built into the application, and all of them were built on assumptions on how we expected the units to behave in the field, so naturally we were a little anxious once the units actually hit the field, but after a little bit of going back and forth and a very solid team effort between all the stakeholders everything worked out just fine.

On the practical side we were developing for a resolution of 1080×1920, and that meant having to get monitors that could rotate to portrait mode and connecting these to our laptops. A couple of weeks into the project we were shipped a 70″ screen from Korea. One of the more unusual challenges we had to deal with was how to get this 250kg beast through our office door and how to mount it against the wall, but fortunately a couple of the developers were pretty big, so we eventually managed.

This was definitely the best looking image I’ve ever seen on a monitor. This unit could light up the room by itself, and seeing our layout on it for the first time was really cool. To my regret we couldn’t connect our PS3 to it, so we didn’t really get to test the performance of the monitor. Also it made a lot of noise when it was turned on.

Unfortunately the 70″ device wasn’t touch screen enabled, and we had to wait another couple of weeks until we got a unit that was touch screen enabled.

Unfortunately this unit was only 47″ and while this is a pretty big screen, there is still a long way up to 72″, so building the games put a lot of demand on us, as we basically had to take the game back and forth between our 24″ development monitors, where we did the development to the 47″ touch screen to test the game play and the 70″ to check out the layout. At times the developers were standing in line to get time on the big screens, but most of the time we were able to share the screens between us. For a while I even wrote code with the 70″ display as my primary monitor just to try it out, but I had to stop when my eyes felt like they were about to start bleeding. That display is very bright indeed.

All in all I’m very proud of this project, and I absolutely love getting away from the usual challenges with developing for the web and mobile and really going big. The project ended Jan 28 and the screens are now part of the ClearChannel portfolio of outdoor displays for interactive advertising.