Category > uncategorized

Create your own programming language for great justice
Posted by postfuturist on 2010-02-07 21:52:12

In an earlier post I wrote my wish-list for the perfect programming language (for me.) I believe most programmers have a perfect programming language in mind. It usually goes something like this: "I really like the object model of language W and the functional aspects of language X and the syntax of language Y. It would be great if it ran on Z runtime / virtual machine, too." Then there are the language zealots who say that language Foo is perfect, or at least better than all the other commonly used languages and with such a large user base and library of reusable code, that even if they could fix the bits of the language they don't like, it wouldn't be worth it because it would break existing code. In a sense, they've settled, put their tent pegs in the ground and encourage others to set up camp where they are, so they can all share code. There is certainly a lot of value is just spending a lot of time one language so you can go deep with it and develop a strong set of idioms and patterns. It also leads to a large number of very exclusive camps. To be fair, many programmers dabble in multiple languages and paradigms, but most do not.

I often am offended by programming language zealots. Just like religious zealots, they are myopic, self-important and insulting towards outsiders. I love much of the writings of Paul Graham, but I find that his statements about Lisp and all that which is not Lisp to be difficult and often insulting. In his article "Beating the Averages," Graham talks about a hypothetical language Blub that a programmer gets attached to. This programmer sees less powerful programming languages for what they are, but doesn't understand the abstractions and power of better programming languages. The programmer thinks he is using the best programming language, but he isn't. He isn't using the best programming language, because... he isn't using Lisp. Period. Graham states explicitly, "Lisp is so great not because of some magic quality visible only to devotees, but because it is simply the most powerful language available. And the reason everyone doesn't use it is that programming languages are not merely technologies, but habits of mind as well, and nothing changes slower." Again, I like Paul Graham's writings for the most part, but saying that he is correct about this because he seems to be correct about so many other things would be a fallacy as much as saying that he is wrong because everything else he says is wrong.

I agree with many points in Graham's article, more or less, but I don't think that most programmers are small-minded Blub programmers. I agree that Lisp macros are powerful, that it is a powerful concept to write programs that write programs etc, but Lisp is hardly the only language that does that. Boo, a language that actually fulfills my wish-list, has macros which are very similar in functionality to Lisp macros. You get access to the actual parse tree and can create or change or augment language features in the language itself. It is a powerful feature, but it is not the most powerful feature, at least not for me. It's not that I have a small mind, either. I understand that Lisp macros are awesome, but it is not an abstraction I need in day to day programming to get things done. I'm a big fan of getting things done, and building systems to help get them done faster and better. Graham argues that he used Lisp with lots of macros in a business that did well, with software that delivered functionality not present in the competitor's projects, and that this was all because of Lisp macros, which comprised at least 20% of his product's code. I'm sure there was a correlation between having features and being successful, but the 20% macros thing may just be significant of the fact that he had to add a lot of things to Lisp to be able to build the software he wanted. It could also be significant of the fact that he knows how to write macros. Just because the competitors did not deliver the same features does not mean that they didn't or couldn't because they used some less powerful programming language. The software he wrote could likely have been written in any number of languages by any number of programmers. The programmers that toil away in so-called Blub languages, oftentimes are quite good at what they do, and can deliver features if they can think them up or are allowed to add them or whatever. Look at Facebook, they rock the world with the power of PHP, a pretty awful language by the estimation of many. I just doubt that Lisp was the reason for Paul Graham's success.

I said all that to say this: Lisp worship aside, it is a good idea for programmers to learn things about programming languages that they don't understand. It is healthy to stretch your mind with different perspectives. I believe that it is a universally accepted truth that looking at a problem from different mental perspectives causes us to have breakthroughs of new ideas and understanding. One excellent way to expand your mind as a programmer is to create your own programming language--not just a theoretical grammar, but an actual working compiler and/or interpreter.

I have to admit, I've been a little lackadaisical regarding hacking code in my free time. I think I haven't been able to get excited about a project. I've announced, prematurely, some projects in the past that have been since abandoned. Perhaps it is that I spend so much time at work "getting things done" that I relish the freedom to not get things done outside work. It's true, I like messing around with different programming languages and not committing to projects. However, since the programming language wish-list post, I've been drawn to my newest project over and over. I am working on a new programming language. It is really in the very earliest of toy language stages right now. I am really just learning things as I go. I am learning how to use flex and bison, the descendants of lex and yacc, what abstract syntax tree is, and many other things. I've read a lot of articles about different aspects of compiler / interpreter design and techniques. Some things have inspired me a great deal, like V8, the JavaScript engine that Google created. I've been interested and inspired by byte-code interpreters, JIT compilation, garbage collection, stack-less interpreters, coroutines, closures, and many other things.

As you can imagine, I have a laundry-list of things I would like to implement in my very own programming language, but this project is mostly about learning, not the end product. I know most folks with a BSCS have taken a course on compilers and have written a compiler of some form or other. With all the instruction on compilers that has taken place, why do we settle for such mediocre languages. Python 3, was a very small step forward from Python 2.x. It was such a small step, that I wonder what the point was. They cleaned up the syntax and some of the libraries a little bit, but for a backwards compatibility breaking change, they sure didn't do much to fix the inadequacies of the language. The "hot" programming languages these days are incredibly old--at least in internet years. Ruby is 15 years old. Python is 19 years old. Why, after developing a science around compilers, do we still use languages that require semicolons on nearly every line? If Ruby and Python are each at least a decade and a half old, do so many companies write code in C#, Java, PHP, and so many other languages that have vestigial syntax from C? Perhaps it is the slowness of businesses. I don't think so. I think it is laziness.

Software developers have the ability to write their own compilers. Yet, they do not. It is intellectual laziness. We are blacksmiths. We can forge our own tools, yet we use the crappy ones handed down to us from old. There are established techniques for creating new tools, yet we forge on with the tools given to us. For shame. What if every developer wrote their own programming language? Sure, most would fall by the way-side and we need developers to write libraries, too and to research other concerns like concurrency, that don't necessarily need to be solved at the language level. There are millions of software developers in the world. Python is dead. If you have the chance, fire up the python interpreter some time and type in "import this" and hit enter. You will discover a creed, a philosophy that underpins many of the decisions surrounding the creation and maintenance of Python. It is a philosophy I disagree with on many points. Not only that, but the Python language fails to fulfill a good number of them. "In the face of ambiguity, refuse the temptation to guess." Python does not allow type annotations, everything is guesswork. "foo = bar" could mean a lot of things, it could be changing the value referenced by "bar" or it could be creating a new reference. "bar" could be anything, a number, a string, an object. Looking at Python code is often a lot of guesswork. Duck typing is the only typing available. When you create a function, you can only name the inputs, not define the type. It seems to fail here. Here is another : "There should be one-- and preferably only one --obvious way to do it." That line gets a lot of deserved criticism. It's not even true. Python is a very powerful language that offers a very large number of way to solve any problem, and usually several of them could be considered obvious, depending on the style of code you normally write. Regardless, even if you could write a language that only had one obvious way to solve each problem, you would have a very poor language. "If the implementation is hard to explain, it's a bad idea." CPython has the GIL, both hard to explain, and a very bad idea.

I encourage every developer to write their own programming language. It will break us out of the complacency that allows us to toil day-in and day-out using the same leaky, broken abstractions. We can write our own tools. I've seen the HTTP 1.1 RFC, it isn't that complicated, and I think the tools we use now to implement server-side code for web development are quite broken. PHP is just awful. Python is a little better, as is Ruby. V8 / JavaScript / Node.js has some promise, but JavaScript is a pretty broken language, too. It's got that semi-colon problem, plus wierd object / array / function rules that lead to ambiguity and confusion. At the end of the day, you can't have everything in one language, though they are certainly trying to do that with Perl 6. We can have better choices, or at least break through the complacency.

I don't want to sound completely critical. Change is happening. People who are stuck with certain runtimes, like the JVM or .NET have created or ported better languages for the platform. There is quite a bit of excitement about new compiler tools, runtimes, front-ends, back-end etc. The LLVM project has gained a lot of attention, lately, for being completely self-hosting.

My own project is just barely getting going. I have the code constructing a complete AST, and I am in the process of writing the part that generates byte-code. I've learned so much already about grammars, tree manipulation, and parsing. I have ideas about register allocation and other things. It is quite the experience to get my hands dirty writing C code again. I am taking advantage of tools I didn't know about in the past, like the Boehm garbage collector. I've spent too long in garbage collected languages to have to worry about freeing every bit of memory by hand now. That and generous use of typedefs make the experience not entirely unpleasant.

How My Girlfriend Saved My Life And Other Confessions
Posted by postfuturist on 2009-08-12 00:09:45

My father rode his bicycle to work almost every day, even though he didn't have to. He took a bike path along the Amazon Slough in Eugene Oregon. Every day he got up early and packed a lunch with pieces of bread, small plastic containers of jam, peanut butter, fruits, nuts, raisins, and leftovers from my mother's cooking. On November 2, 2004 he was riding his bicycle to work and the path went under a small bridge next to the water. He still had his hands on the handlebars when they found him. He had Marfan syndrome, a genetic connective tissue disorder. It can affect the aortic tissue, and can cause a spontaneous aortic dissection which is fatal in four out of five victims, usually quickly, as in the case of my father.

My father bore a striking resemblance to Abraham Lincoln, with his long, slightly crooked nose, lanky features, and occasional beard. Lincoln's life was cut short at 56 from a gunshot wound. It is speculated that Lincoln had Marfan syndrome as well. I am the same height as my father but I'm not as lanky, rather an average build.

We were attending a church at the time that was very small and new with a serious lack of leadership or direction. The church disbanded a couple years later much to the relief of most involved. I haven't attended church regularly since, and have no plans to do so now. My marriage though young at the time was also in decline. That ended at the end of 2007, nearly four years after it started. Many of my family and friends were upset that I was abandoning the marriage. Fewer were upset that I had been emotionally abandoned and put up with it for such a long time. I lost some friends, and moved to Portland almost immediately. It was a hard time. I had suffered a lot of loss. Loss of my father, church, wife, and friends. I still had a lot of friends and my family did not abandon me, but I was hurt by a lot of people who felt it was more important to tell me that I was wrong than to listen to me. I got fed up with explaining myself to person after person--close friends who were sometimes less than sympathetic, so I wrote an email and blasted it to every email address in my contact list. It was a hurtful email, and I regretted it. I withdrew from a lot of people.

In Portland, I met up with a former girlfriend, Megan. I had been in love with her at one point before I had even met my first wife, (she had left to volunteer at an orphanage in Mexico). Unbelievably, she was single. I couldn't find a programming job right away and I ended up working in a coffee shop for a few months, living in a basement apartment with my new/old girlfriend. I didn't tell anyone that I was living with her. Most of my friends and family are evangelical Christians. Having been in that place myself, I understand the feelings and the guilt and the need to uphold the old moral rules. When in that place, seeing a close friend commit carnal sins is difficult. It makes you angry as though the person has sinned against you personally. I understood the feelings, but I was hurt nonetheless.

Megan was amazing to me (she still is, by the way). She didn't judge me, didn't throw scriptures in my face. She just loved and accepted me. You see, she had also left the church, and had dealt with moral indignation and anger and misunderstanding from friends. At the time I moved in with her, I was smoking most of a pack of cigarettes a day. After several months, she encouraged me to try to quit. It wasn't easy, I failed at first, but eventually succeeded. I only succeeded in quitting because she held me to it. I remember one rainy night, I had recently quit, but had bought a pack, and smoked a cigarette. I had another cigarette in my hand, having thrown away the rest of the pack. I was crying. I wanted that last cigarette and she told me that it wasn't OK. It was no longer going to be OK with her for me to smoke. I smoked it, and it was the bitterest cigarette I'd ever tasted. That was nearly a year ago. I've had no more than a handful of cigarettes since then, total. I breathe a lot freer now than I did before when I was smoking. Chances are, I'll live a lot longer.

We are married now--not that it matters when you are in love. My life has settled down dramatically. I've had the same programming job for nearly a year and I ride a bicycle to work every day. My wife packs me a lunch and leaves it hanging from the handle inside the front door of our apartment so I don't forget to take it when I leave in the morning. In the evening we eat together. Sometimes we have misunderstandings that take a lot of talking to work through, but we always do, and most nights our improbable love leaves us sweaty and breathless on the bed, covers in a heap on the floor, the rest of the world too far away to matter.

Phone spam and what AT&T isn't doing to help
Posted by postfuturist on 2009-07-30 20:21:38

I used to be a no-cell-phone guy, but about 2 1/2 years ago I caved and got a cell phone with service from AT&T. What I didn't know was that the phone number was deeply lodged in the databases of some offshore pharmaceutical-pushing call center. Some previous owner, who we shall call "M. Vacchio" had apparently ordered quantities of Valium from these people. I began to immediately receive phone calls, sometimes multiple calls per day with operators asking for "M. Vacchio" and asking if I needed a refill on my Diazepam prescription. If I explained that I was not, indeed, M. Vacchio I was asked if I wanted to fill any prescriptions at discount rates. Occasionally I got the telephone operators to promise that I would be removed from their database, but to no avail. Day in and day out they call.

My first telephone conversation with AT&T about the state of my phone went pretty well. The operator seemed interested that was getting offshore calls from numbers with faked US caller id numbers. He said that it was a crime and asked for details. He didn't make any promises but somehow I felt better after the call, as if AT&T was going to do something about it. Nothing changed. I added my number to the Federal Do Not Call list, and nothing changed.

My second phone conversation with AT&T was less heartening. The operator suggested that I consider getting a new number. My phone number is just too important, being my primary phone contact, inscribed on my resumes and spread among my friends, work associates, and others. I couldn't give it up. It is too far spread around. Besides, the moment I change numbers some other poor schlub will have to field phone calls for the mysterious "M". The operator insisted that AT&T was unable to block specific incoming phone call numbers stating that it was technologically impossible with the way the cell phone system is set up. I believed him even though it is not true.

The next time I called AT&T, the operator suggested that I get an addition to my plan, one that parents get for their children. It allows them to do things like limit the hours that the phone works, limit minutes used, and block incoming phone calls from specific phone numbers. Yeah, they could do it, but only recently figured out a way to monetize it. I hate AT&T. I hate them for the fact that they are lying bastards. They could care less that I pay over $50 a month for a cell phone that incessantly receives calls from phone spammers. There is no service that people pay as much for as cell phone service and willingly put up with such awful service. Cell networks go down, service is spotty, customer service is a joke, and we gladly dish out $50, $75, $100 or more a month to be mistreated by cell providers.

So, since my initial two-year contract with AT&T is over and I'm essentially a free agent, I should move to another provider, right? Well, not so fast, other providers suck, too! Also, Apple has a really amazing phone that does things like browse the internet in a useful way, integrate GPS with Google maps, play music, videos, videogames, and other random applications. It's really awesome, but it requires an AT&T service. Not only that, but for the "unlimited data" plan (basically, internet access, without which the device is mostly useless) I have to pay an extra $30 a month.

The other day we walked into an AT&T store which was strangely empty except for a single sales rep. I asked him, "Do you have iPhones?" He looked slightly dejected and replied, "No, you can get those at the Apple Store downstairs." Down in the Apple store, it was bumping. The walls were lined with iPhones and the place was crowded, the 6 or 7 customer service reps were all busy with customers. I spoke to the one behind the counter, she said that phone sales had to go through one of the reps with the "pads." There were several, all working with other customers. I was directed to talk to the ringleader of the pad carriers. I told her, "I want to buy an iPhone." She told me that there were five people already waiting to talk to the sales reps, and it would be maybe a 20 or 30 minute wait. We left but that's mostly because we had movie tickets to see (500) Days of Summer. Yes, the parentheses are part of the name of the movie. Good film.

New Project: Roguelike in Python
Posted by postfuturist on 2009-03-29 23:45:58

Something I have been mulling over in my head for awhile is the idea of writing my very own roguelike game. A few years ago I was introduced to Nethack and I haven't been the same since. Many of these games are open source already. The ones that I have looked at are Nethack, SLASH'EM (a fork of Nethack), Angband (and Zangband), and Dungeon Crawl (Stone Soup ed.). Of these games Nethack/SLASH'EM is the most venerable of the group with perhaps the deepest game play, written in C. Angband has a very mature, stable source code, also C, that has invited countless variants. Crawl has a C++ code-base and has seen a lot of interest and development in recent years.
My motivation for writing a new roguelike is that the existing choices are not terribly fun to hack*. C and C++ are languages that I am perfectly comfortable using, especially if I am getting paid to do so. While being paid a salary, I wrote nearly all of the AI code for a Godzilla : Unleashed, a console video game title "unleashed" on the Playstation 2 and Wii consoles for the Christmas 2007 holiday shopping season. I wrote that code in C++. Now, when I do programming for fun, I use languages that I like to do. For the last couple years, this has mostly been Python. Even if I am not using Python, I would prefer a dynamically typed language.
Besides working in a dynamic programming language environment, I would like to build a more flexible core game engine for roguelikes. Here are my goals for this game:


  • Written in Python

  • Loosely coupled OO-style classes

  • Easy to configure as stand-alone game, client-server, or even multiplayer client-server

  • Turn-based with option for real-time play needed for multiplayer server

  • Utilize JSON for client-server communication.

  • Ability to write clients in any language

  • Game assets / data created in JSON files outside of code

  • Easily customized by non-programmers


Secondary goals:

  • Multiplayer server

  • Game customizable through simple GUI application

  • JavaScript client that runs in browser, connects to remote server

  • JavaScript client support single-player mode and multiplayer/realtime play

  • Server database backend with web interface for showing high scores, player stats, etc.

  • Multiplayer server instance supports multiple active worlds.

  • Scripting support (game customization through user created scripts)


About my decision to use JSON for game data and communication: Python is to C/C++ as JSON is to XML. There is absolutely nothing redeemable about XML other than the fact that it attempted to be a useful data interchange format that is human readable. Too many other people have spent too much energy explaining why XML is lame for me to get into the details. Whereas I don't mind coding in C or C++ for money. Having to work with XML gives me pause. I'd rather not, under any circumstances. JSON, on the other hand, is light-weight, is free of arcane requirements, and is actually pleasant to view and edit by hand. Enough about that.

At this time, I am targetting Python 2.5. It is really the gold standard right now for Python. I'm more likely to port to 2.6 in the future when the unladen-swallow project comes to fruition.

That's enough architectural day-dreaming for now. I've got some basic functionality to build. Stay tuned for more.

* I use the verb "hack" here in the same way that an auto mechanic might use the term "tinker with."

Down 71% in popularity this week.
Posted by postfuturist on 2009-03-11 20:54:59

Due to my involvement in the making of the film Coraline, I have the singular honor of having an entry in the Internet Movie Database. Unfortunately, according to IMDB's STARmeter rating, my popularity has gone down 71% this week.

The Most Important Site on the Internet
Posted by postfuturist on 2009-03-09 23:37:43

First of all, it isn't this one. Secondly, it's not Google--thank God. Google is just the most popular website. It won't be forever. Wikipedia is certainly a contender. It's really archive.org, also known as the Internet Archive. Go to the page and you'll see their goal in the upper right hand corner of the screen.

Universal access to human knowledge.

That's a tall order. Wikipedia is just trying to be a really good encyclopedia. Google is just trying to be an index for existing publicly available knowledge on the web. The Internet Archive is trying to make all human knowledge available to everyone. It has 1.2 million "texts" (books, essentially), 300,000 audio recordings, 58,000 live concert recordings, and 160,000 "moving pictures" (movies and archival footage). They are the preservers of the world's knowledge, really the essence of humanity. It is the answer to the question: "If you were stuck on a desert island with an internet connection that could only resolve a single domain name, which would it be?" I don't know how many times I've been asked that.