I know not everyone is crazy about using IDEs, if it’s not your cup of tea then thanks for stopping by and you can feel free to stop reading here. If you want to learn how to set up IDEA to do fancy things with Node.JS and HTML5 development, however, keep reading!

(Note: I did this using IntelliJ IDEA 11. I do not have any idea how this will apply to previous or future versions of the IDE. If you have success or failure using this approach on other versions please let us know in the comments, as well as describing any tweaks you had to do to make it work right!)

Here’s the end goal: we want to have two separate applications, one of them a web server running on localhost in Node.JS, the other a website being served by that server. We want to be able to build, launch and debug both projects using IntelliJ IDEA, but we want to make sure that we’re not dependent upon IDEA – we want to be able to rsync our local folder to a remote server and have it simply run.

There are a variety of ways to make this happen, but I’m going to step you through the process I used.

Step 1 – Install Node

Install node.js on your machine. Go ahead, I’ll wait. It’s pretty quick now, if you’re on a Mac or PC you just run the installer – no more waiting for it to build etc.

(EDIT: In order to get code hinting from IntelliJ’s Node.JS plugin you’ll need to provide it with a path to the core node libraries. If you use the installer as I just suggested above then these libraries will not be accessible to the node plugin. In order to get code-hinting you’ll need to download the source tarball, unzip it, stick it somewhere and tell the plugin to point to that folder.)

Step 2 – Install the Node.JS plugin

Before we can set up our Node server we need to get the Node.js plugin for intelliJ. This is kind of really really easy, but it’s an extra step.

Open IntelliJ Idea and open the Plugin Manager (there should be a link in the upper-right of the default splash screen). In the popup that comes up, click “Browse Repositories” on the bottom left. Use the search box in the upper right to type “NodeJS” and you should see your options reduce to two plugins, NodeJS and NUnitJS. I’m not mucking around with testing right now, so ignore NUnitJS – right click on NodeJS and click “Download and Install”. When it finishes it’ll ask you to restart the IDE. Do so.

(Edit: to get code hinting there’s one more step. Once you’ve restarted your IDE you should see a green “Node.JS” Icon in your toolbar. It’ll be to the far right, after the Help icon. Click it. It just wants a path to the node.js source. If you installed via the official installer, follow my recommendation above and download the source as well. Either way, now that you have access to the source, tell IDEA to look at that source for the node.js plugin. It it works you’ll get a new folder linked in your project window that points to the source. As noted in the comments below, if you use this approach while having also used the installer it means that you will have to manually update the source every time you update your node install. I’m waiting for feedback from Jetbrains to see if there’s a better way.)

Now we’re ready to begin.

EDIT: As manners points out below, you also need to make sure that you enable the Remote Hosts plugin in order to debug this once we’re up and running!

Step 3 – Create the Server project

Open IntelliJ IDEA and click “Create New Project”. You’ll get a dialog box with some options. Give your project a name (“Node Server Project”), confirm that the location matches your desired location. Make sure that “create module” is checked and set the type to “Web Module”.

(This appears to be something of a compromise – technically we’d want a JavaScript module, since Node doesn’t make use of the rest, but JS is a subset of what the Web module gives us so we’ll roll with it.)

When your dialog box looks like this, click “finish”:

New Project config dialog box for a Node.JS project in IntelliJ IDEA

So in your Project panel, right-click on your “Node Server Project” module and create a new directory, call it “server”. Then right click on that directory and create a new JavaScript file, call it “server” too. Finally, right click on your “server” folder one more time and create a folder called “www” – this is where we’ll be deploying our website later.

Here is an example of a very simple Node.JS server that simply serves up files from a “www” folder. Copy this and paste it into the “server.js” file that you created.

server.js

var http = require("http");
var fs = require("fs");
var path = require("path");
var url = require("url");
 
var mimeTypes = {
    "html": "text/html",
    "jpeg": "image/jpeg",
    "jpg": "image/jpeg",
    "png": "image/png",
    "js": "text/javascript",
    "css": "text/css"
};
 
var server = http.createServer(function(request, response) {
 
    var uri = url.parse(request.url).pathname;
 
    uri = "www/" + uri;
 
    if (uri.charAt(uri.length - 1) == "/") {
        uri += "index.html";
    }
 
    if (uri.indexOf("..") != -1) {
        response.writeHead(403);
        response.end();
    }
 
    var filename = path.join(process.cwd(), uri);
 
    console.log("\tAttempting to serve: " + filename);
 
    path.exists(filename, function(exists) {
 
        if (!exists) {
            console.log("File not found: " + filename);
            response.writeHead(404);
            response.end("Sorry, the file you requested was not found. Don't let it ruin your day! :)");
            return;
        }
 
        var mimeType = mimeTypes[path.extname(filename).split(".")[1]];
        response.writeHead(200, {'Content-Type':mimeType});
 
        var fileStream = fs.createReadStream(filename);
        fileStream.pipe(response);
    });
 
 
}).listen(8080);

Now all we have to do is configure our application server. In the upper middle of your IDEA window should be a little combo box containing your run configurations. Right now it should be empty – click it and select “Edit Configurations”.

A popup will appear. Press the “+” sign in the upper left and select “NodeJS”. If “NodeJS” isn’t available then you don’t have the NodeJS plugin installed – go back to Step 2 of these instructions and make sure you’ve restarted your IDE upon completion.

Select NodeJS and name your configuration “NodeJS Server”. The path to Node is OS specific, but I know on a Mac if you use their installer it’ll stick it into /usr/local/bin/node. If you’re on a PC or if you have a custom install just set that to point to the actual “node” executable, wherever it is.

Now you have to to set “Path to Node App JS File” equal to the server.js file you created above. Then set the “Working Directory” equal to the path to your “server” folder – for me, it looks like this: “/Users/mykola/temp/Node Server Project/server”

Once you’ve done that, your Run Configuration should look like this. If it does, hit OK:

Congratulations, you’re done! To test this, click the “Debug” button next to that combo box – not the green arrow, the one after that. That’ll launch your server in debug mode. Then open a browser window and navigate to “http://localhost:8080″ and you’ll be greeted by a wonderful File Not Found! message. No worries, everything is going great!

Before we move on to the website project, take a second to play with IDEA’s debugger. Set a break point inside of your server handler – right on line 17, for instance, where “uri” gets defined. Do that by clicking on the gutter to make the red dot appear, as shown. With the breakpoint set and the server running, hit localhost:8080 again to put IDEA through it’s paces. It should hit the breakpoint at which point you can tab over to the debugger at the bottom and introspect all of the objects in scope, evaluate watch expressions, see and navigate the call stack, etc. Really, really useful and way better than just sticking random debug values into console.log(). ;)

Step 4 – Setting up your Web Application Project

Now completely forget about all of that. Shut down your server and close the project, we’ll come back to it in a few minutes.

First, go ahead and create a brand new IntelliJ project called “MyWebsite”. For its location you can put it anywhere – we are going to be deploying the files from this project to the “www” folder in our server project, but we can automate that later so we never have to think about it. Just go ahead and create your new project with a Web module ready to go.

Right click on your module in the project view and add a folder called “www”. Inside of that folder, add a folder called “script” and a folder called “style”. This is a pretty simple run-of-the-mill html file structure. Inside of “www”, create an html file called “index.html”. Inside of “script”, create a javascript file called “main.js”. Inside of “style”, create a CSS file called “style.css”.

Here is some sample content for those files:

index.html

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
        "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
    <title>Sample Index</title>
    <script type="text/javascript" src="script/main.js"></script>
    <link href="style/style.css" rel="stylesheet" />
</head>
 
<body>
    <h1 onclick="headerClicked()">This is my Index page!</h1>
</body>
</html>

script/main.js

function headerClicked() {
    var foo = 2 + 2;
 
    alert("Sample Alert: " + foo * 2);
}

style/style.css

body {
    color:#fff;
    background-color:#000;
}

So far so good, right? It’s a really simple website, when you click on the header text you’ll get an alert showing you a random number, blah blah blah. The point is, we want to be able to do two things:

1. Deploy this sucker to our Node server!
2. Set breakpoints and debug our javascript in IntelliJ IDEA

In order to achieve these two goals we need to do the following:

1. Configure our remote deployment options so that these files get magically sent to our server’s /www folder
2. Build a debug configuration that’ll access these files via our server, instead of from the file system.

So, the first thing we need to tackle is Deployment. In IntelliJ IDEA, click on “Tools” in the menu, go down to “Deployment” and click “Configuration.”

In the ensuing popup we’ll need to add a deployment target, configure the Connection, then the mappings. First, click the “+” sign in the upper left. Name your Server and set its type to “Local or Mounted Folder”.

The Connection tab tells IDEA how to connect to your deployment target, both to upload/download files and also to actually access it as a remote http endpoint. So in the Connection tab, make sure “Type” is equal to “Local or mounted folder”. Set the “Folder” path equal to the path to the folder that holds server.js that you created earlier.

In my case, folder is “/Users/mykola/temp/Node Server Project/server”. Set the “Web Server Root URL” to “http://localhost:8080/”. We’re using port 8080 because that’s what port we’re listening on in our server.js file.

When your “Connection” tab looks like this, tab over to “Mappings”.

The “Mappings” tab is where you map files and folders in your HTML project to files and folders in your remote endpoint. Go ahead and click the “use this server as default” button. If you added a “www” folder to the root of your project as I advised above, append “/www” to the “Local path” that’s populated by default. Set your Deployment Path on server ‘NodeServer’ equal to “/www”, and wet your Web Path equal to simply “/”. Your “Mappings” tab should look like the attached screenshot.

Press OK, you’re almost done! Go back to the menu and select Tools > Deployment > Options. Check the boxes that say “Create Empty Directories” and “Upload changed files automatically to the default server.” Hit OK.

In your projects panel, Right Click your “MyWebApplication” Module and go all the way down to the Deployment item in the context menu. Select “Upload to NodeServer”. This should be the only time you ever have to do this – from now on, every time you make a change in this project it will automagically push that change to your server/www folder.

Let’s stop for a second and make sure everything is wired up correctly:

1. Go to File -> Open Recent in IntelliJ and Open your Server project. It’ll ask you if you want to open it in a new window or the same window – use a New Window.
2. In the projects panel, look into your “www” folder – you should see the files from your web project. If you don’t then you did something wrong above – make sure that you deployed your files from the context menu as I outlined last paragraph!
3. If you still have a breakpoint set, go ahead and get rid of it. Just click on it in the gutter and it should go away.
4. Make sure that the run config we set up is selected, and start your server in Debug mode.
5. Point your browser to http://localhost:8080 and voila, you should see your html page!

The very last step is to set up a Run Configuration for your Web project. This is very similar to the Run Configuration we set up for our server. Click on the combo box, select “Edit Configurations”, click the plus sign in the upper left and select “JavaScript Debug –> Remote”. Name your configuration “JS Debugger”, set the URL to “http://localhost:8080″ and map your project to the following remoteURL: “/”. Your screen should look like this:

Now for the big test! Switch back to your server project and make sure it’s running. If so, then in your HTML project go ahead and press the “Debug” button – it should open your browser to your website, served up via localhost:8080.

(Note, if you have trouble with getting it to open in the right browser, you can go into IntelliJ’s settings and make sure that the “Web Browsers” settings are working the way you’d like. This can be highly specific so I didn’t cover it above.)

While your browser is open and showing your app, click your text. You should see an Alert come up with some gibberish in it, right? Click OK and then without closing anything go back to your Web project. In script/main.js, put a breakpoint on the line that generates that alert. Tab back to your browser and click again and you’ll see that you’ve just triggered that breakpoint!

Final Words

Phew! Thanks for reading all of that. This can be a big pain in the ass to configure, but once it’s working the way you want it to you have a lot more options. Debugging in IntelliJ IDEA is more powerful than debugging in a browser console, and you can make updates and edits in your code on the fly.

If this worked for you, let me know in the comments! If it didn’t, well, I’m sure that it CAN with a bit more nudging. Feel free to ask for help in the comments and I’ll see if I can’t take a look, but with this many variables in play it probably makes sense just to tinker with it and figure out what’s breaking.

I hope that helped!

Twitter has a lot to offer Apple

• Twitter has been behaving strangely lately, creating their own media hosting service and telling devs that they should think twice before investing time in the creation of new Twitter clients. These represent a major change in tone – only a few short months ago, Twitter was everyone’s favorite public API. Now they seem to be fortifying. Why?
• Apple is rumored to be integrating Twitter deeply into their next offering of iOS, which Steve Jobs will be announcing later today. As in, very deeply. But why would Apple take a neutral third-party and build them into the core of their super-tightly-controlled walled garden? Jobs isn’t the sort who spends a lot of time relying on other people/companies – it would surprise me, honestly, if iOS had twitter built in but Apple had no control over Twitter.
• Finally, there’s simply the matter of the gaping hole in Apple’s offerings. They do hardware well, they do media well, they do software well – but it’s no secret that they suck at social. Social Media is not going away, so if Apple wants to be a relevant player in that field they have two choices. They can create a working, usable and popular social media tool – or they can buy one. This one just makes sense to me – especially when you realize how perfect Twitter + iPhone are for each other.

It’s that last point that honestly makes me believe this to be true. Twitter is not just any social media service – it’s the one that’s been most embraced by the tech-savvy minimalist. It lacks the clutter of facebook, and it lacks the fragmentation of the geo/social space. It’s simple, it’s easy, and (as long as the servers are up) it just works – it’s practical an Apple offering already.

Apple has a lot to offer Twitter

The other side of the coin is, why would Twitter want to get into bed with Apple?

• Money.
• Expansion – if Twitter goes native on every iOS device and every Mac, and gets promoted by Apple, their install base is going to go through the roof. They’re already THE social media company in elite tech circles – this could be just the move they need to go truly mainstream.
• Resources – if they get built into Apple’s iCloud service, which is presumably going to have some of the most powerful (and most stable) servers currently running then we’ll see a lot less of the fail whale.

Conclusions

What do you think? Am I crazy? I guess we’ll find out in a few hours!

In the meantime, here is a brief rundown of stuff I’ve been playing with (and will hopefully soon be blogging about):

1) Living without an IDE – a lot of folks seem to have a great time coding all kinds of stuff using TextMate, without the clunkiness of an eclipse-based IDE. I’m looking to explore this option, and I’ll let you know what I think. This ties into the next few points, because my latest interests live outside of any IDEs -

2) Node.JS – if you haven’t heard about Node.JS you’ve been living under a rock. Node is server-side javascript, bringing that lovely (ech!) JS syntax to the server. Why is this cool? Because like AS3, it relies on the Observer Pattern to handle asynchronous requests. Instead of spawning a new thread and blocking, like most other server-side languages, Node uses an Event system with async callbacks to juggle lots and lots of stuff at the same time. Plus, it’s JavaScript. Say what you will about it, it’s pretty slick to be able to handle scripting, web server and front-end coding in the same language.

3) Using JavaScript with Socket.IO to replace Python as my go-to Arduino interface language. If I’m going to learn one stack inside and out, it seems like JavaScript is the stack to learn. If I’m doing everything else in JS, why keep Python around when I can use Node just as easily?

4) Back in the Flash world, a big project I’ve been working on for a few months is approaching launch-readiness. It’s my first Flex 4 application, and I have some thoughts about that. Generally speaking, I haven’t really enjoyed it – but I’m not sure yet if that’s my fault or Flex’s. Much to think about.

5) Finally, this summer I’ll be rolling out my first iOS application. Stay tuned for rants and thoughts on the subject.

As always, the best way to keep up with me is on Twitter, where you can always find me quickly as @mykola. Drop me a line!

Last thing – I’ve implemented Google’s “+1″ feature here. If a blog post is useful or interesting, please consider hitting the +1 button to let me and others know!

As the web moves forward I can only suspect that this problem is going to become more common, not less – and so I have at least a vague hint of an idea for how we can go about solving it. I’m curious for your thoughts!

The Context – Public APIs changed or revoked

• Amazon kills Lendle’s API access – Apropos of nothing, Amazon one day decided to shut down the API access of popular up-and-coming kindle lending service Lendle. Nobody is arguing that Amazon isn’t allowed to do this – it’s their API and they are within their rights to do whatever they want with it. The problem is, where does this leave a service like Lendle? They were providing enormous utility for lots and lots of users and never violated any of Amazon’s policies.
• Twitter threatens third-party client devs. This also came almost entirely out of nowhere – apparently, now that Twitter has decided they want to turn a profit they’re gunning for anyone who is creating “unofficial” twitter apps – and this presumably includes things like TweetDeck, Destroy Twitter etc. Again, Twitter is within their rights – but the general consensus seems to be that this is a dick move. I’m certainly not about to launch a startup that uses Twitter’s API – even if I had been planning to, this would kill those plans

You’re kidding yourself if you don’t think we’re going to be seeing more of this in the future. I’m not even going to mention Facebook, whose API has been in such a state of flux for so long that writing code against it requires phenomenal patience and deep pockets on the part of any client interested in doing so.

The Complication: Companies Need This Freedom

And you know what? A company should be allowed to make sweeping changes to a public API, whether these are changes that cause it to work differently than it did yesterday or changes that make it a violation of ToS for someone to continue doing something that was legal yesterday. These companies own these APIs and if we somehow insist that they never change the way they implement them then the only recourse these companies could have is to remove public APIs altogether. I totally understand that.

The Consequences: Startups Suffer

But then, look at companies like Lendle which take advantage of these APIs to do truly interesting, unique or useful things. I can only assume Lendle co-founder Jeff Croft woke up one morning and spit out his coffee as he opened his inbox. “Oh, okay, so my company that has been getting all of this press and succeeding wildly is suddenly unable to provide the service around which it’s based. Now what?”

That’s enough to scare me out of putting any serious effort into a startup that relies on a third-party API. I could wake up one morning and realize that an API I had been relying on is no longer available, and now what do I tell my users?

Here’s where a lot of people simply say “Well, them’s the ropes. You knew the risks going in and that’s just how this works.”

And that’s true, but I think we can do better.

What a Solution Could Look Like

I’d like to see a third party emerge that serves as a sort of API Hub. This third party (let’s call them the API Hub) negotiates with anyone who offers a public API. They basically provide a service whereby this API Provider can register a “stable and permanent” API. That’s not saying that they will stop working on API updates or whatever – it just means that they will have to version them moving forward, and every version that goes through the API hub needs to be supported independently.

The API Hub in turn sells stable API access to developers. Why would a developer pay for API access instead of using the free public API being offered by the initial provider? They don’t have to. They are free to build their company on whatever the most recent public API offered by the provider happens to be, and for testing and for getting off the ground this is a perfectly viable strategy. But when they are ready to launch and have enough users that they want to make a stab at stability it’ll be worth their while to pay for access.

Would this work? Would people buy into this? I think it’s worth a shot, because money talks. If every serious twitter client out there was paying for the privilege of using a stable API then that’s going to be a nontrivial source of income for Twitter, I would think, right? Or am I being naive? And if an API Provider is being paid to provide a stable API then suddenly there’s a much greater incentive for developers to use that API, so this could provide a competitive advantage.

Is that enough to get someone like Twitter or Amazon to sign on? Obviously a big part of it would depend on how the contracts were structured, where the teeth are so to speak. But it’s at least worth thinking about, right?

I’m genuinely interested in your thoughts. Does something like this already exist? Is it working? Why/Why not? What are the flaws here? Do you provide an API? Your opinion is the one I’ve been exposed to the least, how would you feel about participating in something like this? Does it ask you to sign away too much? Is it reasonable? What would make this more compelling for you?

The cool thing about Dictionary, though, is that you can use non-string keys. In fact, you can pass a reference to ANYTHING and use that as a key. Below are a few examples of how Dictionary lets you write clean, readable code. You can do all of this stuff without Dictionary, but you’re going to be writing more lines of code. I’ve got three examples below, in increasing order of abstraction. We’ll step through using interactive elements such as Buttons as Dictionary keys, then we’ll use Functions, and finally we’ll use Classes. This is really, really useful.

Metrics and Analytics – using interactive elements in your application as keys in a Dictionary

Anyone who does Flash in the advertising world will be familiar with this – the client wants every single click to be tracked, which means you need to go through and put unique tracking codes into all of your click handlers. Then the client changes their tracking mechanism and you have to go through and find all of your button handlers and change the codes, and various nightmares ensue.

Well, good news! Dictionary can really help you out here. Here’s a very simple example of what I mean:

/* note: passing "true" to the Dictionary constructor tells it to use weak keys - meaning that if the dictionary is the only reference to an object, it's available for GC. */
var trackingCodes:Dictionary = new Dictionary(true); 
trackingCodes[button1] = "this is the tracking code for button 1";
trackingCodes[button2] = "this is the tracking code for button 2, etc";
 
button1.addEventListener(MouseEvent.CLICK, buttonHandler, false, 0, true);
button2.addEventListener(MouseEvent.CLICK, buttonHandler, false, 0, true);
 
function buttonHandler(e:MouseEvent):void {
	TrackingManager.track(trackingCodes[e.currentTarget]);
	// logic here to handle click
}

Really simple example, but you see what’s going on, right? The Tracking Code gets associated with the actual button instance. Any time you have a reference to the button you can get a reference to the tracking code – and e.currentTarget counts as a reference to the button.

If your application is more complex – for instance, a full-on MVC based web application, with lots of different views – this can still help you. Set up a Tracking class and give it a static dictionary and two static functions:

protected static var trackingCodes:Dictionary = new Dictionary(true);
 
public static function registerButton(b:DisplayObject, t:String) {
	trackingCodes[b] = t;
}
 
public static function track(b:DisplayObject) {
	tracking code = trackingCodes[b];
	// logic here to do something with that tracking code.
}

Then in your views you simply register your buttons in a constructor or init function:

Tracking.registerButton(button1, "button 1 tracking code");
Tracking.registerButton(button2, "button 2 tracking code");

Finally, on click to your button you just pass a reference to e.currentTarget to Tracking.track(). Simple, effective, and it allows you to organize your tracking codes in large blocks so you can easily make changes and no longer have to waste time looking for each button handler.

Caching – using Functions as keys in a Dictionary

This one came up recently on Twitter. Here’s the problem: you have some relatively expensive functions that only have to get called once. You want to set up a system that’ll either call your expensive function (the first time) or return a stored value from that function on every subsequent call. The good news? Since Functions are first class Citizen-Objects in Flash, you can use them as Dictionary keys. Here’s an example of what I mean:

private var cache:Dictionary = new Dictionary();
 
private function calculateExpensiveNumberOne():Number {
	// lots of messy code.
}
 
private function calculateExpensiveNumberTwo():Number {
	// lots of messy code
}
 
public function get numberOne():Number {
	if (!cache[calculateExpensiveNumberOne]) {
		cache[calculateExpensiveNumberOne] = calculateExpensiveNumberOne();
	}
 
	return cache[calculateExpensiveNumberOne];
}
 
public function get numberTwo():Number {
	if (!cache[calculateExpensiveNumberTwo]) {
		cache[calculateExpensiveNumberTwo] = calculateExpensiveNumberTwo();
	}
 
	return cache[calculateExpensiveNumberTwo];
}

Simple, right? By the way, the ability to pass references to functions around is REALLY useful. Don’t be afraid of it. This is how callbacks work, and it opens a door to the wide world of functional programming.

Class Management – using Classes as keys in a Dictionary

If you’ve ever implemented the Singleton pattern, the previous example probably looked a bit familiar to you. Just to drive the point home, let’s remember that Classes are also Objects in Flash. You can build, for instance, a factory class that returns singleton instances of the requisite classes using a very similar pattern to the one we used above. For instance, let’s create a SampleSingletonManager class like this:

package testing
{
	import flash.utils.Dictionary;
 
	public class SampleSingletonManager
	{
		private static var instances:Dictionary = new Dictionary();
 
		public static function getInstance(c:Class):Object {
			if (!instances[c]) {
				instances[c] = new c();
			}
 
			return instances[c];
		}
 
	}
}

Then, in our main class, let’s do this:

package
{
	import flash.display.Sprite;
 
	import testing.SampleSingletonManager;
 
	public class TestThing extends Sprite
	{
		public function TestThing()
		{
 
			trace(SampleSingletonManager.getInstance(Sprite) == SampleSingletonManager.getInstance(Sprite)) // returns true
			trace(SampleSingletonManager.getInstance(Sprite) === SampleSingletonManager.getInstance(Sprite)) // returns true
		}
	}
}

Again, a very simple example – obviously this isn’t enforcing any kind of singleton on your instances, and I have the return type set to Object – but I hope I’ve made my point. If you want to take this further, you can structure your singletons to require the SingletonManager to pass an instance of itself, or of a private enforce that can only come from inside SingletonManager. Then you return an Interface type that your various Singletons all implement and BAM.

Dictionary is incredibly useful – being able to use non-string Keys can lead to some very interesting patterns and can go a long way towards helping to write logical, readable code. There are, of course, some downsides to using Dictionary – it’s slower than Object or Array, and you can’t strictly type the keys or values the way you can with a Vector. However, I’ve used it to great effect on a wide variety of projects and I hope these simple examples get some of your wheels turning.

Please feel free to comment if you agree/disagree/have questions with any of the examples!

Work

2010 was incredibly exciting for me. A year ago, I was working full-time at a digital advertising agency doing Flash production. It wasn’t a bad job, but I wasn’t thrilled about agency life and in April I decided to finally take the plunge and go into business for myself – and honestly, I haven’t looked back. I prefer the consulting lifestyle over the salaried lifestyle in almost every way, and it’s allowed me to cast an incredibly wide net in terms of extracurricular activities and side projects. Having a girlfriend who is also a talented designer made this something of a no-brainer for us, and it’s been great to be able to collaborate on large projects working to meet our own deadlines. Some of my consulting has been done on-site, as has some of hers – but our most exciting projects are the ones we’re working on secretly. Hopefully in 2011 we’ll have some cool stuff to share!

Activities

As far as extracurriculars and side projects go, I took over running the Columbus Digital Adobe User Group, provided technical feedback on not one but two books from O’Reilly Press (The Flex 4 Cookbook and Getting Started with Flex 4), and went to my first industry conference (RIAUnleashed). I also submitted a talk to 2011′s Flash and the City.

Additionally I’ve become much more active on Twitter (hit me up at @mykola) and I’ve started updating this blog with almost alarming regularity (this is what, like, my fifth post this year? That’s huge for me!)

Technology

2010 was going to be the year I switched to an Android phone. I mean, come on – I’m a Flash developer. I’m supposed to hate Apple and Steve Jobs, especially after that Flash stunt he pulled! So in July, I put a bullet in my trusty iPhone 3G and bought an HTC Evo 4G.

I returned it inside of a month. I know this is a highly polarized field right now, and I don’t want to make any enemies – but I hated almost everything about the Android platform as a user. Inconsistent UI, a convoluted phone OS (“Oh, no, you should download the process manager to let you know what’s running!” – really?), a terrible app store experience. You name it, going down the list I felt like Apple simply cares more about providing a great user experience – and so I’m back on an iPhone. Maybe I’ll switch in 2011 – but honestly, Android has a lot of cleaning up to do before they can begin to compete with Apple’s user experience.

I also picked up an iPad this year. I like it okay, but honestly the novelty wore off quickly. I’m just not entirely clear what I’m supposed to do with it. Right now it kinda just lives in the bathroom, which is a constant source of amusement for guests. Still, I suspect I’ll buy an iPad 2 this coming year.

Robotics

2010 is the year I bought my first (four) Arduino microcontrollers. This has been an incredible way to spend my downtime, and I’ve learned so much I can’t stand it. Currently I’m working on developing a custom stack for robotics projects – it looks something like this:

• Robot Structure – VEX frame, with VEX motors and wheels etc. It’s a bit like Lego – everything is designed to fit together nicely.
• Microcontroller – This is going to be Arduino. It doesn’t take that much effort to make the VEX components play nice with the Arduino, and the existing support out there for open-source hardware projects using the Arduino system is fantastic.
• Communications – Right now I’m using Python to handle my serial communications, because it works so nicely. I can rig up my serial connection using USB, BlueTooth or XBee and it doesn’t make any difference – Python is able to read/write with like 2 lines of code. That said, I still haven’t resolved the difficulties I’ve been having integrating python into the final layer of my stack…
• User Interface – Adobe AIR. This is a no-brainer – it’s incredibly easy to build amazingly detailed user interfaces using the Flash Platform, and I’m almost able to get complete integration between my Python layer and my AIR layer. I’m confident that with a bit more tweaking I’ll have it down.

So yeah, that’s that. Vex -> Arduino -> Python -> AIR creates a VAPA Robotics Stack. If I can iron out the kinks, I’m going to have an incredible project to show off in a few months!

Looking Forward

2010 was a great year for me – I’m expecting 2011 to be even better. I’m going to keep learning, keep playing and keep doing cool stuff. Please reach out to me on Twitter of any of this sounds interesting to you, I love to talk shop!

In 2011, my largest focus is going to be on completing my robotics stack – my big goal for the year is to present at Adobe MAX 2011, controlling a variety of cool ‘bots from a Flash-built interface. It’s much easier than you probably think, and it’s incredibly satisfying! Like I said, I have a few kinks to work out – but it’s 80% of the way there, and I’d love to get other people to join in!

So, from me – Happy New year, and I’ll see you all in 2011!

Since it’s hard-wired into me to do things the hard way, I decided to see if it was possible to control VEX components with an Arduino microcontroller. A bit of googling shows that it can be done, but I didn’t find any good step-by-step guides – so I thought I’d write one. This isn’t a comprehensive guide to powering all things VEX through an Arduino – but follow along and I’ll show you how to control a VEX motor using an Arduino, which is pretty cool.

VEX Motors are Servos!

First of all: VEX motors don’t operate like standard motors – they have three wires, not just two, so you can’t just plug ‘em into an Ardumoto and hope for the best. Instead, you have to know that these motors are servos. The black wire is ground, the orange wire is power and the white wire is designed to carry signal. When we wire this up, we’re going to power the motor independently of the Arduino board – we’ll get to why in a bit. The important thing to note here is that you control this guy by creating a Servo object in your Arduino code, and then passing it a value. If you send “90″ the motor just kinda sits there. If you pass 0 it spins one way, if you pass 180 it spins the other way – the closer to 90 you are in either direction, the slower the spin. Cool?

Powering a VEX Motor to work with your Arduino board

If you just wire this up to your USB-powered arduino unit you’ll be in for a big disappointment – it seems like the arduino is not capable of outputting enough power to drive the motor. Fortunately, the signal wire doesn’t need as much energy as the motor itself, so the solution is fairly simple. You can solder this too, but these instructions are for a breadboard:

Proper wiring: Black wire from 9v battery to both black wire on VEX and Arduino Ground. Red wire from battery to orange wire on VEX and Arduino VIN. White wire on VEX Motor to PWM output on Arduino board.

1. Get a 9v battery with one of those connectors that sits on top and has red and black wires coming out. The connectors are really cheap, it was like 3 bucks for a bag of ‘em at Radio Shack.
2. Connect the two battery cables to two different rows on your breadboard.
3. From the row that has your red wire, run a wire to the Arduino board’s “VIN” pin and run a wire to your motor’s Orange cable.
4. From the row that has your black wire, run a wire to the Arduino board’s “Ground” pin and run a wire to your motor’s Black cable.
5. Finally, connect the white pin on your motor to a PWM output on your Arduino board. I used Pin 5.

What we’re actually doing here is powering the motor independently of the board – both the motor and the board are getting energy from the battery, but the motor is getting the full 9v from the battery rather than the 5v it would get if you were powering it from the arduino board directly. From reading the various forum threads it sounds like some people actually put a capacitor between the motor’s white wire and your Arduino’s PWM output – I haven’t had any issues doing it this way, but if you get erratic behavior you can try that.

Arduino Code

This is the easiest part, once you understand the rest. You’re going to use Arduino’s native “Servo” class. You attach it to your PWM pin, then you send a value. Really, really easy. Here’s a sample sketch that’ll get your motor turning:

#include "Servo.h"
 
Servo motor;
 
void setup() {
  motor.attach(5);
  motor.write(0); 
}
 
void loop() {
 
}

Where to now?

Once you have this part down you’re in a good position to take advantage of the various equipment that VEX lets you use. Stay tuned for a post soon that’ll have me driving a VEX robot around using easy-to-make Arduino components.

It works, but it’s not pretty. You need to have access to the right ports, and you have to run a third-party application with its own config file to make it work. Additionally, I’ve never once been able to get it working on a mac – I know I’m not the only one, either. There’s some weird shit going on there.

I decided to see if it was possible to get the same results using the new NativeProcess API in Adobe AIR 2. Read on for the details!

Overview – Selecting a Language

So, before we get started let’s define the problem as specifically as possible. We have a device, in this case an Arduino Microcontroller, that’s reading and writing to a serial port on the computer. Our challenge is to allow our Flash application, running in AIR, to interact with it. Flash, however, has no direct access to serial port data – even with the expanded AIR runtime that stuff is still sandboxed away. So we know right out of the box that we’re going to need an additional language to handle transport between the serial port and AIR.

My challenge is this: I’m a flash guy. I can comfortably scan other languages, but I don’t know the Java or Python or C API’s or syntaxes as well as I probably should. So I need to pick a language that’ll make this as easy as possible to implement. I first looked at the source of SerProxy to see if I could repurpose the existing code there – but it frankly scared me a little bit. C can be an intimidating thing to look at, and when there are 15 files full of one-or-two-character-long method names and system calls I have to assume that there’s going to be an easier way. The advantage is, there are no dependencies – once you have your program written, you can compile it down to an executable on any system, and that’s pretty neat. Still, though – too scary for now, I want to be up and running quickly.

So I looked into Java – and that made a bit more sense, but there seem to be issues with the libraries required for native serial IO and the documentation in the tutorials was a bit long. I wanted something simple. Bookmark the Java pages for future reference, but keep looking.

So I looked into python, and apparently serial IO can be handled with about two lines of code, both legible and logical. The downside is that the user needs to have Python installed, and in addition needs to have the PySerial library installed. This is a minor pain in the ass, but it’s not that big a deal – anyone who is going to be using my code should be able to at least install python and pyserial, fine. Plus, Python runs on any operating system as long as you have the python interpreter installed. So, Python it is.

Setup

Really simple. Open your AIR IDE of choice, create a new Flex project. My MXML is below, it’s all one file – paste that in there. Then right click the src folder and add a new file, call it “ThreadedSerialInterface.py” – the code that will go in here is below, as well. That’s it. The nice thing about AIR is that it will actually bundle your script in with your AIR application, so there’s no need to worry about running multiple programs the way you have to now, with SerProxy.

The Python Script

I know that I want to be able to write a script which is going to remain open in the background until I tell it to close. While open, this script needs to be able to do two things – accept any data coming out of the serial port and print it to stdout, and accept any data coming in from stdin and write it out to the serial port. Simple enough, but these are two distinct processes – so I googled around a bit and decided to implement these processes as two threads within my python script. Here’s the script as I currently have it, with the caveat that I am NOT a python developer and all of this is either made up as I went along or cribbed from helpful folks on Stack Overflow. So, here’s my current script:

from sys import argv, stdin
from serial import Serial
import threading
 
try:
    baudRate = int(argv[-1])
    serialPort = argv[1:-1]
except ValueError:
    #some nice default
    baudRate = 9600
    serialPort = argv[1:]
 
#support paths with spaces. Windows maybe?
serialPort = ' '.join(serialPort)
if not serialPort:
    exit("Please specify a serial port")
 
print( "opening connection to %r@%i ..." % (serialPort, baudRate) )
ser = Serial(serialPort, baudRate, timeout=1)    
 
 
 
class readerThread(threading.Thread):
    def __init__(self, threadID, name):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
 
    def run(self):
       while True:
           if ser.inWaiting() > 0:
               print( "[R]" + ser.readline() )
 
 
 
class writerThread(threading.Thread): 
    def __init__(self, threadID, name):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
    def run(self):
        while True:
            input = raw_input("[W}Give Me Input!")
            print ("[W]You input %s\n" % input.strip())
            ser.write(input.strip())
 
 
thread1 = readerThread(1, "readerThread")
thread2 = writerThread(2, "writerThread")
 
thread1.start()
thread2.start()

A few notes: I’ve appended “[R]” and “[W]” to outputs from my Reader and Writer threads, respectively. This lets me easily separate them in my AIR app, which is helpful for knowing what I’m looking at.

So what’s going on in this script? I instantiate an instance of pySerial’s serial class, called “ser”, and I tell it to connect to the serial port and baud rate specified as arguments when I run this script. Simple enough. Then I define two classes, readerThread and writerThread, each of which extends threading.Thread. readerThread simply loops and, if there is any output from the serial port, it prints it out. writerThread is the inverse – it loops, and waits for input from stdin each time through. It takes that input, when it gets it, and writes it to the serial port. I start both threads and I’m good to go.

Let’s see how we can tie this into Flash.

AIR and NativeProcess

This is actually a lot more painless than you might expect, once you understand what’s going on. AIR 2 has exposed the NativeProcess API, meaning that we basically have access to a command line interface directly through our AIR application. Rather than stepping through every detail, I’ll just post my code below, with comments. Right now it’s all one file:

<?xml version="1.0" encoding="utf-8"?>
<s:WindowedApplication xmlns:fx="http://ns.adobe.com/mxml/2009" 
					   xmlns:s="library://ns.adobe.com/flex/spark" 
					   xmlns:mx="library://ns.adobe.com/flex/mx"
					   closing="killProc()">
	<fx:Declarations>
		<!-- Place non-visual elements (e.g., services, value objects) here -->
	</fx:Declarations>
 
	<fx:Script>
		<![CDATA[
 
			// create our native process
			private var np:NativeProcess = new NativeProcess();
 
			// create our native process startup info, which will hold things like the path to python and any arguments.
			private var npi:NativeProcessStartupInfo = new NativeProcessStartupInfo();
 
			/**
			 * 	This method gets called when the user clicks the "init" button. It basically makes sure that your python 
			 *  path is correct, then assembles the NativeProcess call, assigns listeners and starts the process.
			 */
			private function setup():void {
 
				// the file we're actually going to be executing is python itself - the rest, my script included, gets passed in as arguments.
				var exec:File = File.applicationDirectory.resolvePath(pythonLocation.text);
 
				// check to see if the python path resolved correctly.
				if (exec.exists) {
					initButton.enabled = false;
					killButton.enabled = true;
					npi.executable = exec; // assign "exec" to be the executable in our startup info.
					var args:Vector.<String> = new Vector.<String>; // create a vector to hold arguments - don't assign til the end.
					args[0] = "-i"; // python needs this to handle I/O correctly, details to follow.
					args[1] = File.applicationDirectory.resolvePath("ThreadedSerialInterface.py").nativePath; // this is the path to our python script, included with the AIR app.
					args[2] = serialPort.text; // this is the serial port, "COM3" etc on pc or "/dev/tty.usbserial-XXXXXX" on mac, etc.
					args[3] = baudRate.text; // this is our baud rate.
 
					npi.arguments = args; // assign the arguments to the startup info object.
 
 
					// add listeners for stdout, stderr and process exit.
					np.addEventListener(ProgressEvent.STANDARD_OUTPUT_DATA, stdoutHandler);
					np.addEventListener(ProgressEvent.STANDARD_ERROR_DATA, sterrHandler);
					np.addEventListener(NativeProcessExitEvent.EXIT, exitHandler);
 
					// start the NativeProcess
					np.start(npi);
				}
			}
 
			/**
			 * 	this method passes a string to our native process, which passes it to the microcontroller.
			 */
			private function writeToSerial():void {
				np.standardInput.writeMultiByte(toSerial.text + "\n", File.systemCharset);
				toSerial.text = "";
			}
 
			/**
			 * 	This method handles output from the NativeProcess. I have some basic parsing logic in here to separate output by thread.
			 */
			private function stdoutHandler(e:ProgressEvent):void {
				var output:String = np.standardOutput.readMultiByte(np.standardOutput.bytesAvailable, File.systemCharset);
				if (output.indexOf("[R]") > -1) {
					logOutputFromReaderThread(output.slice(3));
				}
 
				if (output.indexOf("[W]") > -1) {
					logOutputFromWriterThread(output.slice(3));
				}
 
 
				trace("[Output from Python] " + output);
 
			}
 
			/**
			 * 	utility function to log Reader thread output.
			 */
			private function logOutputFromReaderThread(s:String):void {
				var text:String = s += readerOut.text;
				readerOut.text = text;
			}
 
 
			/**
			 * 	utility function to log Writer thread output.
			 */
			private function logOutputFromWriterThread(s:String):void {
				var text:String = s += writerOut.text;
				writerOut.text = text;
			}
 
			/**
			 * 	This handles STDERR output, right now it's just tracing.
			 */
			private function sterrHandler(e:ProgressEvent):void {
				var output:String = np.standardError.readMultiByte(np.standardError.bytesAvailable, File.systemCharset);
				trace("[Error!] " + output);
			}
 
			/**
			 * 	This fires when the native process is terminated. 
			 */
			private function exitHandler(e:NativeProcessExitEvent):void {
				trace("Native Process has terminated!");	
				initButton.enabled = true;
				killButton.enabled = false;
			}
 
			/**
			 * 	This forces the NativeProcess to close. You must call this when you quit your program, 
			 *  or else your processes will live on.
			 * 
			 *  Notice that the Application is using this to handle the Closing event - so when you close the app,
			 *  your process is killed.
			 */
			private function killProc():void {
				trace("Killing native process");
				np.exit(true);	
			}
 
		]]>
	</fx:Script>
 
	<!-- MXML for layout, self-explanatory I hope -->
	<s:TextInput x="105" y="16" width="174" id="pythonLocation" text="C:\\Python26\\python.exe"/>
	<s:Label x="10" y="16" text="Python Location" height="22" verticalAlign="middle"/>
	<s:TextInput x="105" y="50" width="174" id="serialPort" text="COM3"/>
	<s:Label x="10" y="50" text="Serial Port" height="22" verticalAlign="middle"/>
	<s:TextInput x="105" y="80" width="174" id="baudRate" text="9600"/>
	<s:Label x="10" y="80" text="Baud Rate" height="22" verticalAlign="middle"/>
	<s:Button id="initButton" x="10" y="110" label="Init" width="269" click="setup()"/>
	<s:Button id="killButton" x="9" y="143" label="Kill Process" width="269" click="killProc()" enabled="false"/>
 
	<mx:HRule x="10" y="172" width="268"/>
	<s:Label x="11" y="189" text="Write to Serial:" height="20" verticalAlign="middle"/>
	<s:TextInput x="98" y="187" width="174" id="toSerial"/>
	<s:Button x="10" y="217" label="Send" width="262" enabled="{toSerial.text.length}" click="writeToSerial()"/>
 
	<mx:HRule x="10" y="251" width="268"/>
	<s:TextArea x="10" y="292" width="435" height="299" id="readerOut"/>
	<s:TextArea x="453" y="292" width="421" height="299" id="writerOut"/>
	<s:Label x="10" y="272" text="Output from Reader Thread:"/>
	<s:Label x="453" y="272" text="Output from Writer Thread:"/>
 
</s:WindowedApplication>

Results

So, this ALMOST works. That is to say, it does work but it’s not quite ideal. I’m having some issues with input and output into python, and I think they’re caused by the fact that I don’t have an actual terminal window. When python is waiting for raw_input(), it outputs a prompt (“>>>”). When I print serial anything out, it’s expecting a console to be able to print to. Because we’re running this as a process from inside of AIR, there is no actual terminal for interface – which brings us to the “-i” argument I’m passing in above.

Now, I’m not a python developer – I’ve made that clear, right? – but from what I can tell, “-i” is telling Python to act as through there’s a terminal even though there isn’t. It’s saying “Trust me, someone is listening even if you can’t see them.” Without that argument, the script waits until it exits to output anything. Obviously that’s no good for our purposes because we want this thing to be constantly running, listening for output. So, the “-i” argument helps us out a lot – but there’s one final facet of this puzzle that’s throwing me for a loop.

When I run this application, it works great – except that only every other command I try to write to the serial port goes through. Boot it up, run it and see. Hit the init button, wait for the “Initialized, waiting for input.” response from the Reader thread, type a number into your “Write to Serial” box and hit submit. The Writer thread should respond with “You input 1″, and if your arduino unit is doing anything in response to serial input it should happen. Now type another number and hit submit – nothing happens. Type a third and hit submit, everything is fine.

I was confused until I ran it in debugger mode, and realized that I’m getting output from STDERR – specifically, every other time I submit serial input it returns “>>>”, a prompt, via the Error stream. I have no idea why.

See comment thread for more information – this problem was solved! Specifically, I replaced the “-i” command line option with a “-u”.

Conclusion

This is obviously a work in progress, and I’m going to update this as I go. I’m writing this post today, perhaps a bit prematurely, because there’s been some noise on Twitter lately about other folks trying to do similar stuff and I thought I’d throw this out there in case it helps anyone. If you have any feedback, I’d love it in the comments below – especially any feedback on the python section. I’m not 100% understanding every part of it.

I think as Flash developers many of us tend to live in very small boxes (albeit boxes with lots of cool stuff inside). This project has helped me to realize just how little I understand about programming in general, and in that regard it’s been very helpful. I hope to continue learning as I go. :)