1. It’s ugly.
2. It requires me to remember a password, or go through the hassle of resetting the password through email verification.
3. I don’t like to hang out on forums. Or, at least, I don’t anymore.
4. I actively avoid the site’s links when they show up in search results.

vBulletin is bad. It powers a lot of forums in the world, and you can recognize it a mile away. I looked up vBulletin. Amazingly, people pay money for it. I could enumerate its sins for a long time, but I need to focus on one. In order to not receive weekly emails, you have to change your account settings. To change your account settings, you have to log into your account. If, like me, you don’t remember passwords for sites you rarely visit, you have to reset the password through email. This is a familiar song and dance to many, but it angers me immensely. It is trivially easy to generate a single, safe, instant unsubscribe link for mailing list emails. Why don’t LinuxQuestions emails have a one-click unsubscribe? Easy, they are idiots. They are stupid for using crappy, commercial forum software. They are idiots for spamming my inbox and forcing me to jump through a bunch of hoops to unsubscribe.

Sites like Stack Overflow use my OpenID login, which I know well. I can remember a single username / password for multiple sites without the fear of one site secretly stealing my password and using it to log in to another site as me. Trust me, more sites than you want to know store your password as plain text in their databases. Why am I picking on this one website? They are trying to get more visitors who provide more content, so they can extract advertising money or satisfaction that they are helping Linux users everywhere. They want to be a popular destination for a certain niche, but they completely fail to create a compelling experience in any way at all. It is so bad, I actively ignore search hits for their site.

Software matters. Usability matters. You can’t just punt on the software. Invest in it. Pay a real designer and get a real programmer on your team. Unless you are independently wealthy, you need a software developer on your team. This software developer will want to make the site awesome, since he is personally invested in the success of the site. People want to know what makes a website successful. Ultimately, its the folks writing the code who make or break a site. The software developer needs access to the code, so don’t even dream of buying a closed-source, commercial solution. Start with the best open source solution (let the software developer pick), pay for a good design (don’t do a design competition, that’s exploitative), and let the software developer merge the two. Your software developer uses social web software all the time. He knows what’s good. Force him to eat his own dogfood (use the site himself), and he’ll work harder at improving the experience. Elicit feedback from your users and implement popular features.

The tragedy is that these old, broken forums end up loaded with useful information donated by the users despite the shortcomings of the software the site rests on. Don’t post useful information on these sites. Please, don’t encourage them.

1. Transform the input into a string of discrete tokens. “10 + 3 * 4″ becomes “10″, “+”, “3″, “*”, “4″. This is achieved with a scanner/lexer like lex, or the newer flex.
2. Transfom the token stream into a parse tree using a BNF description of the grammar with a parser tool like yacc or the newer bison. Using infix notation, our stream becomes (“+”, “10″, (“*”, “3″, “4″)).
3. Reduce the tree by walking it and calculating the value of each node, depth first, until you have calculated the value of the root node which is the value of the whole tree.
   1. Evaluate left side of “+” operation. It is the constant value 10.
   2. Evaluate right side of “+” operation.
      1. Evaluate left side of “*” operation. It is the constant value 3.
      2. Evaluate right side of “*” operation. It is the constant value 4.
      3. Reduce: 3 * 4 is 12

      It is the numerical value 12.

   3. Reduce: 10 + 12 is 22

The whole thing is straight-forward and almost insultingly easy. The tools I’ve been using are Flex and Bison which generate the scanner and parser, respectively, in C. While writing these parts to generate a parse tree, which could also be called an abstract syntax tree or AST, I had a realization. Code generation, whether it is byte-code or machine-code, is simple. The code does the same tree-walk, in the same order, but instead of evaluating and reducing the elements, just generate byte-code or machine-code which will do those simple operations at runtime. The program spits out a flat sequence of code. You are merely writing code that flattens the tree.

To put it another way, the sequence of mathematical operations that reduce the parse tree to a single value is the same ordered sequence of operations that the compiled code should represent. It’s so obvious, it’s painful. I just hadn’t thought about it before. I thought there would be some deep voodoo involved in compilation, but there isn’t. Your program is likely to be a series of expressions, not just one. All you have to do is evaluate them in order, tacking the byte-code or machine code for each expression on to the end of the code.

Here is a series of transformations that does what I am talking about:

1. Input : “x = 10 + y * 3″
2. Scanner : “10″, “+”, “y”, “*”, “3″
3. Parser : (“=”, “x”, (“+”, “10″, (“*”, “y”, “3″)))
4. Tree walk :
   1. Store the location of “x” in R0
   2. Store “10″ in R1.
   3. Store the value at location “y” in R2
   4. Store “3″ in R3
   5. Store the product of R2 and R3 in R4
   6. Store the product of R1 and R4 in R5
   7. Store R5 at the location specified in R0

It’s just one more step to take the list of operations and turn them into a list of coded instructions. At that point you are done.

One of the things I wanted with my compiler is type inference, so when I get to an “=” or “assignment” node of my tree, I evaluate the right side first, which gives me a type, then I know what the type of the identifier on the left should be. The first time an identifier is used should always be an assignment, and the type of the identifier is set at that time, if another assignment to the same identifier is made of a different type, the program should fail to compile. Here is the messy / incomplete code that handles this right now:

int genAssignByteCode(simpleblock b, simplenode n)
{
    if(n->R == NULL) {
        printf("No R-value in assignment!\n");
        exit(EXIT_FAILURE);
    }
    if(n->L == NULL) {
        printf("No L-value in assignment!\n");
        exit(EXIT_FAILURE);
    }
    if(n->L->type != IDENTIFIER) {
        printf("L-value is not an identifier!\n");
        exit(EXIT_FAILURE);
    }
    int reg_num = genByteCode(b, n->R);
    if(reg_num == -1) {
        printf("The r-value has no value. Lame.\n");
        exit(EXIT_FAILURE);
    }
    enum reg_type rval_type = get_reg_type(b, reg_num);
    simplesymbol inmap = in_map(b, n->L->data.string);
    struct byte_code bc;
    if(inmap == NULL) {
        // new value with big, scary TYPE INFERENCE
        int newr = new_reg(b,rval_type);
        new_symbol(b, n->data.string, newr);
        if(rval_type == rt_int) {
            bc.t = BCI_STORE_INT;
        } else if(rval_type == rt_string) {
            bc.t = BCI_STORE_STR;
        }
        bc.reg_a = newr;
        bc.reg_b = reg_num;
    } else {
        // existing value, better do some static type checks
        enum reg_type lval_type = get_reg_type(b,inmap->reg_num);
        if(rval_type != lval_type) {
            printf("You got the wrong type, pal.\n");
            exit(EXIT_FAILURE);
        }
        if(rval_type == rt_int) {
            bc.t = BCI_STORE_INT;
        } else if (rval_type == rt_string) {
            bc.t = BCI_STORE_STR;
        }
        bc.reg_a = inmap->reg_num;
        bc.reg_b = reg_num;
    }
    new_code(b,bc);
    return bc.reg_a;
}

Basically, simpleblock is a typedef’ed pointer to a struct that represents a block of code. It keeps an array of virtual registers allocated, array of instructions, and linked list of symbols (identifiers). This code generates the byte-code for the assignment operation. It is called from the genByteCode function which it calls recursively to evaluate the right side of the statement. The line new_code(b,bc); actually appends the byte-code bc to the array of instructions in block instance b. This is C code, and as such, it is really ugly. Also, the error messages are unhelpful and quite rude. This is a rough-draft of sorts. There are a lot of parts not in the code that would help explain it, too.

While constructing this part, I realized that it is very easy to do type inference. Java and C# took years and years to acquire type inference of any kind, but I don’t see what all the fuss is about, it is quite simple to implement, really. Compilers are not scary, go write one of your own already. Go create that perfect programming language. Do it now, or else you’ll spend the rest of your life writing software in programming languages that you don’t like and you’ll only have yourself to blame.

I often am offended by programming language zealots. Just like religious zealots, they are myopic, self-important and insulting towards outsiders. I love much of the writings of Paul Graham, but I find that his statements about Lisp and all that which is not Lisp to be difficult and often insulting. In his article “Beating the Averages,” Graham talks about a hypothetical language Blub that a programmer gets attached to. This programmer sees less powerful programming languages for what they are, but doesn’t understand the abstractions and power of better programming languages. The programmer thinks he is using the best programming language, but he isn’t. He isn’t using the best programming language, because… he isn’t using Lisp. Period. Graham states explicitly, “Lisp is so great not because of some magic quality visible only to devotees, but because it is simply the most powerful language available. And the reason everyone doesn’t use it is that programming languages are not merely technologies, but habits of mind as well, and nothing changes slower.” Again, I like Paul Graham’s writings for the most part, but saying that he is correct about this because he seems to be correct about so many other things would be a fallacy as much as saying that he is wrong because everything else he says is wrong.

I agree with many points in Graham’s article, more or less, but I don’t think that most programmers are small-minded Blub programmers. I agree that Lisp macros are powerful, that it is a powerful concept to write programs that write programs etc, but Lisp is hardly the only language that does that. Boo, a language that actually fulfills my wish-list, has macros which are very similar in functionality to Lisp macros. You get access to the actual parse tree and can create or change or augment language features in the language itself. It is a powerful feature, but it is not the most powerful feature, at least not for me. It’s not that I have a small mind, either. I understand that Lisp macros are awesome, but it is not an abstraction I need in day to day programming to get things done. I’m a big fan of getting things done, and building systems to help get them done faster and better. Graham argues that he used Lisp with lots of macros in a business that did well, with software that delivered functionality not present in the competitor’s projects, and that this was all because of Lisp macros, which comprised at least 20% of his product’s code. I’m sure there was a correlation between having features and being successful, but the 20% macros thing may just be significant of the fact that he had to add a lot of things to Lisp to be able to build the software he wanted. It could also be significant of the fact that he knows how to write macros. Just because the competitors did not deliver the same features does not mean that they didn’t or couldn’t because they used some less powerful programming language. The software he wrote could likely have been written in any number of languages by any number of programmers. The programmers that toil away in so-called Blub languages, oftentimes are quite good at what they do, and can deliver features if they can think them up or are allowed to add them or whatever. Look at Facebook, they rock the world with the power of PHP, a pretty awful language by the estimation of many. I just doubt that Lisp was the reason for Paul Graham’s success.

I said all that to say this: Lisp worship aside, it is a good idea for programmers to learn things about programming languages that they don’t understand. It is healthy to stretch your mind with different perspectives. I believe that it is a universally accepted truth that looking at a problem from different mental perspectives causes us to have breakthroughs of new ideas and understanding. One excellent way to expand your mind as a programmer is to create your own programming language–not just a theoretical grammar, but an actual working compiler and/or interpreter.

I have to admit, I’ve been a little lackadaisical regarding hacking code in my free time. I think I haven’t been able to get excited about a project. I’ve announced, prematurely, some projects in the past that have been since abandoned. Perhaps it is that I spend so much time at work “getting things done” that I relish the freedom to not get things done outside work. It’s true, I like messing around with different programming languages and not committing to projects. However, since the programming language wish-list post, I’ve been drawn to my newest project over and over. I am working on a new programming language. It is really in the very earliest of toy language stages right now. I am really just learning things as I go. I am learning how to use flex and bison, the descendants of lex and yacc, what abstract syntax tree is, and many other things. I’ve read a lot of articles about different aspects of compiler / interpreter design and techniques. Some things have inspired me a great deal, like V8, the JavaScript engine that Google created. I’ve been interested and inspired by byte-code interpreters, JIT compilation, garbage collection, stack-less interpreters, coroutines, closures, and many other things.

As you can imagine, I have a laundry-list of things I would like to implement in my very own programming language, but this project is mostly about learning, not the end product. I know most folks with a BSCS have taken a course on compilers and have written a compiler of some form or other. With all the instruction on compilers that has taken place, why do we settle for such mediocre languages. Python 3, was a very small step forward from Python 2.x. It was such a small step, that I wonder what the point was. They cleaned up the syntax and some of the libraries a little bit, but for a backwards compatibility breaking change, they sure didn’t do much to fix the inadequacies of the language. The “hot” programming languages these days are incredibly old–at least in internet years. Ruby is 15 years old. Python is 19 years old. Why, after developing a science around compilers, do we still use languages that require semicolons on nearly every line? If Ruby and Python are each at least a decade and a half old, do so many companies write code in C#, Java, PHP, and so many other languages that have vestigial syntax from C? Perhaps it is the slowness of businesses. I don’t think so. I think it is laziness.

Software developers have the ability to write their own compilers. Yet, they do not. It is intellectual laziness. We are blacksmiths. We can forge our own tools, yet we use the crappy ones handed down to us from old. There are established techniques for creating new tools, yet we forge on with the tools given to us. For shame. What if every developer wrote their own programming language? Sure, most would fall by the way-side and we need developers to write libraries, too and to research other concerns like concurrency, that don’t necessarily need to be solved at the language level. There are millions of software developers in the world. Python is dead. If you have the chance, fire up the python interpreter some time and type in “import this” and hit enter. You will discover a creed, a philosophy that underpins many of the decisions surrounding the creation and maintenance of Python. It is a philosophy I disagree with on many points. Not only that, but the Python language fails to fulfill a good number of them. “In the face of ambiguity, refuse the temptation to guess.” Python does not allow type annotations, everything is guesswork. “foo = bar” could mean a lot of things, it could be changing the value referenced by “bar” or it could be creating a new reference. “bar” could be anything, a number, a string, an object. Looking at Python code is often a lot of guesswork. Duck typing is the only typing available. When you create a function, you can only name the inputs, not define the type. It seems to fail here. Here is another : “There should be one– and preferably only one –obvious way to do it.” That line gets a lot of deserved criticism. It’s not even true. Python is a very powerful language that offers a very large number of way to solve any problem, and usually several of them could be considered obvious, depending on the style of code you normally write. Regardless, even if you could write a language that only had one obvious way to solve each problem, you would have a very poor language. “If the implementation is hard to explain, it’s a bad idea.” CPython has the GIL, both hard to explain, and a very bad idea.

I encourage every developer to write their own programming language. It will break us out of the complacency that allows us to toil day-in and day-out using the same leaky, broken abstractions. We can write our own tools. I’ve seen the HTTP 1.1 RFC, it isn’t that complicated, and I think the tools we use now to implement server-side code for web development are quite broken. PHP is just awful. Python is a little better, as is Ruby. V8 / JavaScript / Node.js has some promise, but JavaScript is a pretty broken language, too. It’s got that semi-colon problem, plus wierd object / array / function rules that lead to ambiguity and confusion. At the end of the day, you can’t have everything in one language, though they are certainly trying to do that with Perl 6. We can have better choices, or at least break through the complacency.

I don’t want to sound completely critical. Change is happening. People who are stuck with certain runtimes, like the JVM or .NET have created or ported better languages for the platform. There is quite a bit of excitement about new compiler tools, runtimes, front-ends, back-end etc. The LLVM project has gained a lot of attention, lately, for being completely self-hosting.

My own project is just barely getting going. I have the code constructing a complete AST, and I am in the process of writing the part that generates byte-code. I’ve learned so much already about grammars, tree manipulation, and parsing. I have ideas about register allocation and other things. It is quite the experience to get my hands dirty writing C code again. I am taking advantage of tools I didn’t know about in the past, like the Boehm garbage collector. I’ve spent too long in garbage collected languages to have to worry about freeing every bit of memory by hand now. That and generous use of typedefs make the experience not entirely unpleasant.

I’ve been looking for a programming language with some of the feel of Python but with static, inferred typing. There are a couple, but Genie caught my eye so I wrote his algorithm in Genie. Basically, the differences between Genie and Python I noticed in porting the code are that functions need type declarations (something I have yet to see inferred in any statically typed language) and that variables are initialized with the “var” keyword. Genie does away with Python’s use of the colon, which is redundant, and instead uses it for type declarations.

Here is the new code.

[indent=4]
uses
    GLib
 
def radians(n : double) : double
    return n * 0.0174532925
 
def distance(latA : double, lngA : double, latB : double, lngB : double) : double
    var radius = 6371.0
    var latAr = radians(latA)
    var lngAr = radians(lngA)
    var latBr = radians(latB)
    var lngBr = radians(lngB)
 
    var deltaLat = latBr - latAr
    var deltaLng = lngBr - lngAr
 
    return radius * 2 * Math.asin(Math.sqrt(Math.pow(Math.sin(deltaLat/2),2.0) \
        + Math.cos(latAr) * Math.cos(latBr) * (Math.pow(Math.sin(deltaLng/2),2.0))))
 
def bench()
    var increment = 2.5
 
    var latA = -90.0
    while(latA <= 90.0)
        var lngA = -180.0
        while(lngA <= 180.0)
            var latB = -90.0
            while(latB <= 90.0)
                var lngB = -180.0
                while(lngB <= 180.0)
                    distance(latA, lngA, latB, lngB)
                    lngB = lngB + increment
                latB = latB + increment
            lngA = lngA + increment
        latA = latA + increment
 
init
    bench()

This code runs on my laptop in about 35 seconds, which as far as I can tell from the blog post, is about as fast as the optimized Lisp. Of course, Genie is translating the code into pure C, and then compiling it to an executable. Syntax is a matter of personal taste, and I like the way Genie works. Since it compiles to a pure binary, there is no runtime. It leverages existing C libraries, using GLib objects as its native object system. The C API is universal and Genie can emit C header files too, so I could see myself writing Genie code that integrated with almost any software development platform.

For comparison, here is the somewhat verbose C code generated by valac, the Genie / Vala compiler.

#include <glib.h>
#include <glib-object.h>
#include <float.h>
#include <math.h>
#include <stdlib.h>
#include <string.h>
 
double radians (double n);
double distance (double latA, double lngA, double latB, double lngB);
void bench (void);
void _main (char** args, int args_length1);
 
double radians (double n) {
    double result;
    result = n * 0.0174532925;
    return result;
}
 
double distance (double latA, double lngA, double latB, double lngB) {
    double result;
    double radius;
    double latAr;
    double lngAr;
    double latBr;
    double lngBr;
    double deltaLat;
    double deltaLng;
    radius = 6371.0;
    latAr = radians (latA);
    lngAr = radians (lngA);
    latBr = radians (latB);
    lngBr = radians (lngB);
    deltaLat = latBr - latAr;
    deltaLng = lngBr - lngAr;
    result = (radius * 2) * asin (sqrt (pow (sin (deltaLat / 2), 2.0) + ((cos (latAr) * cos (latBr)) * pow (sin (deltaLng / 2), 2.0))));
    return result;
}
 
void bench (void) {
    double increment;
    double latA;
    increment = 2.5;
    latA = -90.0;
    while (TRUE) {
        double lngA;
        if (!(latA <= 90.0)) {
            break;
        }
        lngA = -180.0;
        while (TRUE) {
            double latB;
            if (!(lngA <= 180.0)) {
                break;
            }
            latB = -90.0;
            while (TRUE) {
                double lngB;
                if (!(latB <= 90.0)) {
                    break;
                }
                lngB = -180.0;
                while (TRUE) {
                    if (!(lngB <= 180.0)) {
                        break;
                    }
                    distance (latA, lngA, latB, lngB);
                    lngB = lngB + increment;
                }
                latB = latB + increment;
            }
            lngA = lngA + increment;
        }
        latA = latA + increment;
    }
}
 
void _main (char** args, int args_length1) {
    bench ();
}
 
int main (int argc, char ** argv) {
    g_type_init ();
    _main (argv, argc);
    return 0;
}

Using the mysql command line tool helps me be a better database designer. First of all, I have to communicate with the tool in SQL with a few helper functions. It responds by printing out information in ASCII formatted tables. I only need to know a handful of basic commands to get started.

USE dbname -- sets the active database

SHOW TABLES -- lists the tables in the active database

DESC tablename -- shows the structure of the table

SHOW CREATE TABLE tablename -- shows SQL to create table

The SHOW CREATE TABLE command is ultimately more useful than the DESC command because it gives you a better idea of how the table was created, and how to create similar tables in the future. I learned quite a bit about MySQL by using this command to inspect various tables.

Using command-line tools to interact with systems like databases puts me closer to the action. After awhile I feel like I am getting my hands dirty, and I get a feel for the data. At times this feeling can be negative. Working with MyISAM tables can be painful when the database allows me to put the database into invalid states. I get a visceral sense for the inadequacy and brittleness of MyISAM when I manipulate it by hand. InnoDB tables feel a little more constricting at times, being much stricter about data integrity, but this strictness also brings confidence in the safety of an ACID compliant database.

MySQL is just one example of how native command-line interfaces, however daunting, are often more powerful tools for interacting with computing systems than the fancy GUI tools. Perhaps, with concerted effort, the visual tools could include some of the efficiencies of the command-line tools, but for the most part they feel slow and stupid. Other tools I use more on the command line than anywhere else are revision control tools and testing frameworks.

Software development doesn’t need to be slow to be high quality. I have been a slow convert to the unit testing religion, but now I am just about there. These days when a client wants a new feature, I find the test classes that correlate to the class I’m messing with, write some tests for the new functionality which fail. I type in some code, run the tests and repeat until they are all passing. I run all the unit tests for the project (which currently runs very quickly as we haven’t been doing this for a long time) to make sure I didn’t inadvertently break anything else and ship it. It’s fast and I see very few bugs slip through this way.

Develop a small set of simple, high quality tools. Most of our work is in PHP. After you are done groaning about how awful PHP is, read on. A lot of our work consists of rescue operations. Small business owners and budding entrepreneurs, bless their little hearts, don’t usually have a lot of cash to throw around, so they take the lowest bid for their website work. The lowest bid is usually a company from India or Romania or some other exotic locale where US dollars are still worth quite a bit. These thrifty folks end up with a half-working, extremely buggy piece of software which is basically a single folder with a whole bunch of files in it ending with “.php”. The internals of these files would make any software developer worth his or her salt cringe. You and I know that the best thing to do would be to throw the whole thing out, and start from scratch, preferably with a language like Python or Ruby. Well, the client doesn’t know that, they come to us to fix the code. Well, we have a tool, it is a very lightweight PHP framework that can integrate nicely with existing code. I have used it so much that I am at a point where it feels natural to use it to solve problems. Our clients do pay us to fix their code which we do gladly, and efficiently, usually replacing the broken parts completely with clean, MVC style code.

Imperative, object-oriented code is efficient for writing web software. I’ve read a bunch about heavy OO patterns and this and that, and I think most of it is crap. Nobody needs all that. What customers need is to create related hunks of data, which I call “rows” from a database perspective, or “objects” from a programming perspective. I have created my very own simple, Active Record style ORM for PHP and MySQL. Basically, you give me an existing database table. In about 2 minutes I can slap together about 20 lines of boilerplate code (mostly informing the ORM what the name and data type is of each column) which gives me the ability to easily do CRUD operations (create, read, update, delete) on your database table. The ORM generates all the SQL in the background, so the only code I have to write is PHP. The columns of your database become public members of the class instances (objects). Now, according to the rules of OO, this is a big mistake. Well, really, it’s not. You’ve got collections of data to move around and use, why hide everything behind a bunch of pointless accessors? All that extra data-hiding cruft just slows things down. What the customer wants is a bunch of data that gets dumped into the correct parts of the HTML. The customer doesn’t care about “proper OO” and neither should you. They are not paying you hourly to implement latest design pattern you read about.

Use a dynamic, “scripting” language like PHP, Python, or Ruby (or Perl if you are a masochist). I now point Apache directly at my project folder so when I’m done typing, I hit a few keys which save my work, switch to a console to run a test or switch to a browser and reload the page. I see the results of what I typed within about one second of when I finished typing. There is no deploy step, no compile step, no waiting, no distraction, just instant feedback. It’s amazing, and it has revolutionized my workflow. Remember, our clients are paying hourly rates, which when you break it down into minutes is generally some amount of money greater than $1 per minute. A five minute compile/deploy step costs the client several dollars each time. I remember writing code in Visual Studio, I believe the F5 button was the compile/run shortcut. Hitting that on a reasonably complex piece of software was like an invitation to go get a snack, start up a conversation with a coworker or get distracted with interesting articles on the internet. The big secret is of course that every programming language gets compiled, it’s just that C++ compiles really slowly, has an extra pre-processor step and has to be linked with a bunch of other code in a time consuming way. Things like Python get compiled as they are read into the interpreter into byte code which is then executed by the interpreter immediately. That just happens to be really fast, plus the byte code is cached to disk oftentimes. In the .01% of the time when the simplest possible code in your scripting language of choice isn’t fast enough for your client’s need, first check to see if you can fix it with a better algorithm, if not, rewrite the critical, innermost loop in C.

In conclusion, I may use a lame tool (PHP) and butcher “best OO practices” and make babies cry by completely ignoring “functional” programming, but I get things done. Not only that, but I get things done well, with a low defect rate, and quickly. The code I write is easily maintainable and ships with good test coverage. The “good, fast, cheap: pick two” thing is a lie. With a familiar, well-honed collection of tools any software developer can do all three.

I don’t propose that any particular language is THE ANSWER. Such a thing does not exist, at least not yet. There aren’t even any particular fantastic ones. The more I investigate other languages, the more I see features that I wish existed in the languages I use to get things done with every day. So, I’d like to compile a list. There’s no reason why these things can’t coexist in the same programming language. Most of the programming languages we use are very old and as certain ideas drift into other languages, the need for backwards compatibility causes new features to be implemented in strange ways. For example, object oriented programming being tacked onto old programming languages has created all sorts of syntactical abominations: C++, Perl 5 blessed references, and Objective-C to name a few.

Last rant before the list: I’ve been told that every Computer Science student takes a class on compilers and yet, we still write programs in PHP, Java, and C++. I’m just a lowly English major but I think we can do better. I’ve been reading Coders at Work, a collection of interviews with important software developers. Many of the interviews are with designers or people influential in the creation or design of popular or important programming languages such as JavaScript, Haskell, Erlang, Common Lisp, and Smalltalk. It’s really interesting stuff, but it shows how most of these languages have flashes of brilliance, pieces that add to the puzzle, but are also missing out on a lot of innovations. That said, here is my list:

1. Classes and namespaces. It doesn’t need to be object oriented, as that implies that everything has to be an object, even the entry point of your program. Java is object oriented, and I consider it a fault. Too many objects and too many classes. That said, classes of objects are nice abstractions that allow the programmer to arrange code and data in useful ways. Similarly, namespaces are a nice way of avoiding name collisions and organizing code. I want to be able to put stand-alone or helper functions in namespaces, and not be forced to create static functions as part of class definitions. Not everything has to be part of a class. Python gets this right (modules as namespaces).

2, Arbitrary data structures. Sometimes data enters a program from an external source like a database, or from a web browser, or a file. It is very handy to be able to take something like a string of JSON or XML and just convert it into a schema-less blob of data, native to your programming environment. JavaScript does this particularly well with dynamic objects and arrays that can nest in arbitrary ways. The literal notation for such lists and objects is so simple and straight-forward in JavaScript that it is the de facto standard for data exchange, thanks in part to Doug Crockford who codified it. Most languages we would consider “dynamically typed” do a decent job of this like Python, Perl, Ruby, and even PHP. Most languages we consider “statically typed” do not. Do I even have to name names?

3. First class functions. Functions need to be able to move around, breathe, combine and create other functions. This allows the programmer to create innumerable abstractions. C++, Java, and PHP just completely fail at this. Python does a half-assed job–sorry, but Python just kind of sucks this way. JavaScript, Perl, and many other languages do a better job.

4. Static and dynamic typing. I want it all. There are compelling reasons for both. I see no reason why I can’t have both. Some languages already have this, like Boo, C# 4.0, Delphi, and maybe PHP. Maybe what I want is called optional static typing. That’s not quite right, as it is really is a matter of perspective. It could be called optional dynamic typing, too, I suppose. In any event, it’s not voodoo. I don’t know why stubborn languages like Java and Python will not allow optional “other-style” typing. A lot of energy is spent with statically typed languages dealing with arbitrary data structures in sane ways or initializing objects with excessive typing: “Circle mycircle = new Circle()”. Conversely, people in the purely dynamic languages camp spend a lot of time agonizing over compiler and runtime optimization when it would be so simple for the programmer to just let a profiler tell him where the “hotspots” are so he or she can just go in and add a few hints or type constraints into the code which allow the compiler or runtime to generate much faster code. In Python, you generally “drop down to C” to optimize the bits of code that run too slow. That’s a shame, since it can destroy the cross-platform capability of your code and/or tie it to a single language implementation. It is also needlessly complex. Boo is really the best language I’ve seen in this space. It looks like a dynamically typed language but it actually uses type inference and is statically typed by default but allows explicit dynamic typing.

5. Good Concurrency Options. I like Erlang’s actor model. I don’t care much for the rest of the language. Much has been written about concurrency recently so I won’t say much. There needs to be a way to share data between different threads and/or processes in a safe, sane way and the ability to effectively utilize multiple processing cores efficiently.

6. Clean syntax. I normally can adapt pretty well, but some languages just drive me bonkers with their awful syntax. Perl is one of those languages. I am not a fan of Lisp and it’s many direct offspring. S-expressions are so mundane as to be mind-erasingly boring. Those languages sorely need some variation–some different operators or something. There is a healthy balance, leaning toward the simple. Python and Ruby are both decent. CoffeeScript, a mini-language that compiles to JavaScript has some neat ideas in this space. Using different symbols and operators allow languages to be expressive, and make it easier to catch the meaning of a bit of code at a glance, but are often superfluous. Most commas, semicolons, and curly braces you see are completely useless. 99% of the time, statements are separated by new lines, why do you need a semicolon, too? You don’t. Items in a list are often separated by a comma and some white space, usually one space character. Do you need a comma? You don’t. Usually when you see curly brackets, only one of them is adding any meaning to the code: the last one.

There are more things that I want from programming languages, but those are some big ones. If I could find a language that does them all, I’d be a happier programmer. Ultimately, software development is about getting things done and no profession has a perfect set of tools, but I still think that we can do much better.

Most of my family and friends consider me to be a devout Christian. Mostly they believe this because that’s what I’ve been until relatively recently. I have a few things to get off my chest about this and to set the record straight.

I was raised in a very devout and loving Christian home. These are the types of homes that create strong Christians. I was one of those for a very long time. In college I got involved in a very intense church and college group. I felt a strong ownership of my faith and my involvement in the church. I believed the Bible deeply, especially everything about Jesus. I very much believed the core Gospel message and considered myself to have a personal relationship with Jesus. I prayed a lot. I read the Bible a lot. I attended many meetings, and worshiped, and spoke in tongues and everything. I did it all, and did it all wholeheartedly–for years.

Let nobody say that I didn’t have a real experience of Christianity–that I thought I was a Christian but wasn’t. I was the real-deal, if such a thing can exist. I had spiritual epiphanies and experiences. I often confessed my sins and shortcomings to God and other believers. I truly felt remorse over sin and asked God for forgiveness on many occasions. I had little fear of death as I was utterly convinced of a very pleasant after-life. I had faith.

All I ever got in return for my faith was warm fuzzy feelings and friendships with other Christians. I believed that God healed me when I was sick in the hospital but the fact is that my internal bleed was detected, and that surgeons took care of the physical reason for the internal bleeding. I just ascribed my good fortune to God. I never had anything happen that qualified as a miracle. All I ever had was faith. Other things happened that I prayed about that seemed to have no divine intervention. God did not seem to solve things that I prayed about. Financial problems, health problems, relational problems, and other calamities just continued for me at the pace that, I believe, most people experience them. God did nothing in response to prayer.

I gave God a lot. I gave him many, many hours of my time. I gave him a lot of money. Ten percent of my gross income turned into over half my disposable income when it came to tithing. I seriously paid a lot of money. I even tithed when I was in massive debt. I was tithing to a God that I thought was the kind of God who spoke to people and healed people and responded. However, everything that I had taken as responses from God were easily explainable from a materialist perspective. The Bible shows God talking directly to people. Somehow this never happened with me. All I had were vague impressions and feelings or coincidences that I ascribed to God.

It took me a long time to realize that it is OK to give up on one-sided relationships. It is normal and healthy to stop communicating with those who never respond. Send a few emails to somebody without ever getting a response, and eventually you stop. I prayed to God tens of thousands of times. If God wants to communicate with me, the line is always open on my end. He just hasn’t called. I’ve done plenty for God and given him plenty of chances to show himself as God, but he either doesn’t exist, doesn’t care, or doesn’t want me to believe in him. Either way, it leaves me in the clear to safely ignore him.

Having spent the last three years not attending church or involving myself much in reading the Bible, praying, worshiping and the like, I can attest that it hasn’t made much of a difference in my life. I do not feel that I have become a worse person. If anything, I have relaxed a bit. In the last three years I have become happier. I’ve left behind some unhealthy addictions. Instead of looking forward to an afterlife, I can focus on the present life. Instead of tithing, I support a child in the third world. Instead of feeling guilty for not shoving the gospel down other people’s throats, I only feel some regret for the times in the now distant past when I did. I have not become unethical or amoral. I just care more for people and less for God.

Without further ado, here is Root.js in primordial form:

var sys = require("sys");
var http = require("http");
var posix = require("posix");
 
function loadfolder (name)
{
    var data = {};
    var output = sys.exec("ls " + name).wait();
    var files = output[0].split("\n");
    for(var j in files)
    {
        if(files[j].match(/\.js$/))
        {
            var modname = files[j].replace(/\.js$/, "");
            sys.puts("Loading " + name + "/" + modname);
            data[modname] = require("./" + name + "/" + modname);
        }
    }
    return data;
}
 
var models = loadfolder("models");
var views = loadfolder("views");
var controllers = loadfolder("controllers");
 
for(var x in controllers)
{
    controllers[x].load(models,views);
}
 
var routemap = 
[
    [/^\/other(.*)$/, "other"],
    [/^\/(.*)$/, "default"]
];
 
http.createServer( function (request, response) {
    for(var i in routemap)
    {
        var route = routemap[i];
        var matches = route[0].exec(request.uri.path);
        if(matches != null)
        {
            matched = true;
            var c = controllers[route[1]];
            c.handle(matches.slice(1),request,response); 
            return;
        }
    }
}).listen(8000);
 
sys.puts("Server running at http://127.0.0.1:8000/");

This file expects a few things. It expects there will be folders named models, views, and controllers. In its current configuration the routemap will expect to find files named default.js and other.js in the controllers folder. Any controllers should export a function called load that provides a way to pass in the models and views during initialization and a function called handle that is called when there is an incoming request. Here is a simple example controller (default.js):

var models = null;
var views = null;
 
exports.load = function (m,v) { models = m; views = v; };
 
exports.handle = function (matches, request, response)
{
    models.mymodel.getData( function (data) {
        views.defaultview.show(response,data);
    });
}

This controller calls the getData function exported by the mymodel model which would be a file named mymodel.js in the models folder looking something like this:

var data = 
{
    title : "Page Title",
    message : "Hello, JS World"
};
 
exports.getData = function(callback)
{
    callback(data);
}

In this case, the model does not need to do an asynchronous call to get the data, but in case it did, the controller passes in a callback that the model can pass the data to asynchronously, in typical Node.js fashion. The last thing we need is the view (views/defaultview.js):

var posix = require("posix");
var jsontemplate = require("../json-template").jsontemplate;
var templates = {};
 
loadtemplates = function()
{
    templates['basic'] = jsontemplate.Template(
        posix.cat("./templates/basic.html").wait());
}
 
exports.show = function(response, data)
{
    response.sendHeader(200, {"Content-Type":"text/html"});
    var text = templates['basic'].expand(data);
    response.sendBody(text);
    response.finish();
}
 
loadtemplates();

Oh yes, and the template (templates/basic.html):

<!DOCTYPE html>
<head>
<title>{title}</title>
</head>
<body>
<h3>{message}</h3>
</body>

This example uses JSON Template which I briefly discussed here.

The preceding code actually works. It’s missing the controllers/other.js controller, but it’s enough to get the general idea. I could sit down with this as a starting point and create a web application. In fact, I just may do this. This framework has basically zero helper functions. I’ve noticed that the best helper functions and other abstractions of useful functionality spring forth naturally from the act of programming and active refactoring. When I notice myself repeating work, or pondering the idea of copy-pasting code from one part of an application to another, that’s generally the time I do the work of building infrastructure. If I decide to use this framework to do something useful, then it will develop naturally. Right now it is a blank canvas.

I recently used Redis to solve some problems as work. There is already a node.js library for accessing Redis here, so I’m thinking about building something with that. Redis is essentially a persistent key-value store that supports simple data structures with atomic operations such as lists and sets. The “list” solved a very specific problem for us which had previously been partially solved (in an ultimately broken way) by building a queue on top of Memcached using a locking system built on the atomic increment and decrement operations. We dropped in the Redis list and everything started working perfectly.

If I decide that I need a traditional SQL backend, I would probably use DBSlayer, a http/JSON interface for MySQL, though Ryan Dahl has promised native support for some traditional databases in future versions of Node.js. Cool.

I decided to combine the “Hello, World” examples from Node.js and JSON Template. This is what I came up with.

The first thing you have to do is make JSON Template conform to the CommonJS standard which Node.js so wisely implements. To do that, add the following line to the end of the json-template.js file:

exports.jsontemplate = jsontemplate;

That was easy enough. Put that file in the same folder as your Node.js project and include it with the following line:

var jsontemplate = require("./json-template").jsontemplate;

And now for the big finale, Node.js with JSON Template:

var sys = require("sys");
var http = require("http");
var jsontemplate = require("./json-template").jsontemplate;
 
http.createServer( function (request, response) {
    response.sendHeader(200, {"Content-Type":"text/plain"});
    var t = jsontemplate.Template('Hello {name}');
    var text = t.expand({'name': 'world'});
    response.sendBody(text);
    response.finish();
}).listen(8000);
 
sys.puts("Server running at http://127.0.0.1:8000/");

Thanks to MV for referring me to CommonJS. He seems to have realized the power of an all JavaScript web development stack for some time and has some good rants on the topic.