Terrible Programmer

My name is André and I am a terrible programmer (Confessions of a Terrible Programmer)...
I think we all are (maybe not Dijkstra, Knuth, Wirth and some few others, that think their programs mathematically and demonstrate them http://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/EWD1009.html).

So I try to mitigate my human factor by:

• testing;
• reviewing my code carefully;
• automating tasks that I'll have to do more than a couple of times (tests, gathering information for monhtly reports, ...);
• investigating new tools and techniques that allow me to do less or with better quality;

And it's the last point that I think holds more promise. Testing and reviewing code are all a double check (most of the time by the same person) that the code is correct. But it's also subject to error.

Automating tasks also buys us more confidence (especially if the automation is tested and used for a while), but sometimes hard to justify. On a recent project that I manage, I invested a couple of days developing some office automation that allows me to save excel reports received by e-mail, uploading them to a database and importing them to a word table. But I knew that, the monthly task would take me half a day every month for one year. So I got a 4 day savings (more if the contract is renewed), and confidence on the output (at least on what depends on me). Another automation that is a must (ok, I still don't implement it) is build automation. It just eliminates all those problems related to builds (multiple web configs for various environments, flags - Release and authentication, conditional code, ...).

But automating tasks like code reviews or quality assurance is more problematic. That's why I'm longing for static analysis tools powerful enough to catch most problems (code coupling, unnecessary complexity, concurrency problems, finding and eliminating code duplication) and easily extensible so that I can easily implement new rules.

I already posted about some tools (NDepend, nStatic - not released yet, FxCop, Simian), but I still think most have some limitations and don't address many problems. Maybe with .NET 3.5 (and the ability to create Expression Trees), gives way for a new generation of tools more powerful and that don't treat code as text files but as ASTs. That way it should become easiser to find duplicate code (independent of variable names), inject faults in code (Fault Simulation), analyze and compute call graphs based on the arguments of functions (and the opposite, given an exception state, what conditions and call graph could lead to that), and many other that I can't even think of.

Blog Voices

Congratulations to my friend Mario Romano for reaching 100 posts in October. The grand total of my posts from Nov/2006 doesn't reach 50, so it's an impressive figure.

That got me thinking in what I value on blogs, like the update frequency, the voice, and made me evaluate what are the blogs that I read most and that are my biggest influences.

So, here are the bloggers that I can't miss and which posts I value the most:

• Joel on Software is one of the expert voices that has been around longer. His posts show through his experience, and his opinions on Human Resources (hiring developers, ), Software Development (Process, UI), Project Management and Economics (Buy vs Build) are truly outstanding;
• Coding Horror is another great blog, with great opinions on Programming, Software Development, Project Management, and some off-topics that appeal to many programmers, that keeps an independent voice (somestimes a bit controverse);
• DevCatharsis, from my friend Mario Romano is a very good all rounder which has the same main interests that I have right now: Architecture, Functional Programming and Software Development Process;

Some more focusing on a specific area:

• On SQL, there's Kimberly L. Tripp and her great posts on index tuning and general SQL tuning. Has many code samples and comparative tests on performance;
• Paul Graham- Unofficial RSS Feed has some great insights on Lisp and recently a focus on startups;
• On Functional Programming, with a twist and focus to imperative developers, I really like defmacro.org (great articles on FP for the rest of us) and Did it with .NET (with a focus on C# 3.0 and recently F#). See also The Mechanical Bride (C# 3.0, Linq and lots of => lambdas);
• On Performance, Rico Mariani s Performance Tidbits, are a must, although it's been somewhat quiet since he became Chief Architect of Visual Studio (a huge job indeed). It has great analysis on performance from a very high level right down to the bit;
• On Microsoft, don't miss Mini-Microsoft to get a feeling on Microsoft internals, Human Resources, shares and profits;
• Creating Passionate Users from Kathy Sierra, which is shutdown due to some creeps that harassed her, has many good points on user focus and learning (she's one of the Head First series editors);
• On a fun perspective (or maybe not, as it reflects our reality), I really like Worse Than Failure and Dilbert (Ok, it's not a blog);
• There are also some other authors like Martin Fowler, Bruce Eckel that I prefer in book format, and Ted Neward on article format and not so much on blog format;

Ok, the list is getting long but I try to keep up to date on these. So what blogs am I missing and why?

What's on your blog short list? If someone reads this, post your list and the reasons why it's on your list.

Software Estimation

So why are software estimates so difficult? I don’t know, but between technical problems, little details that take forever, customers changing requirements and conflicting requirements, estimates are normally way below the real time that takes to do something (a rule of thumb is just doubling your normal estimate just to be safe).

Steve McConnell just wrote a book (Software Estimation: Demystifying the Black Art), that I recently bought (but still haven’t read –I’m planning to read it soon), that tries to ease this problem and shed some light into the black art. But what can you say, when even someone as knowledgeable as Steve McConnell, misses estimates when building a fort house by 100% (http://blogs.construx.com/blogs/stevemcc/archive/2007/09/23/building-a-fort-lessons-in-software-estimation.aspx). Well, it was just a small construction plan, that most wouldn’t even try to estimate and just plug along.


But there are great lessons to be learned, that apply to software projects:

• Numerous unplanned problems collectively added up
• Underestimation of unfamiliar tasks
• Not decomposing big tasks into smaller subtasks
• Using overly round time units
• Substituting a target for an estimate
• Sweeping numerous little tasks under the estimation rug
• Never creating a real estimate

And some differences to SW projects:

• No way to compromise quality for the sake of schedule
• Schedule overrun was free
• The estimation error didn't really matter, because the project would be done regardless of what the estimate turned out to be

Jeff Atwood analyzes it from a different perspective (http://www.codinghorror.com/blog/archives/000960.html), and concludes that it’s very different (and requires a different skillset and discipline) to build a doghouse or a skyscraper. And the same applies to software (building a toy app is very different from an enterprise application). That is a statement that most programmers would agree but something very hard to explain to a customer (that thinks software is easily changeable without causing structural problems or bugs).

More recently (http://www.codinghorror.com/blog/archives/000981.html) Jeff took another look at estimation and current practices (planning poker sounds great) and reviews the FogBugz tool from Joel Spolsky (from the Joel on Software fame). Apparently the tool has evolved from a bug tracker to a project management tool.

The tool uses a method called evidence based scheduling (a Monte Carlo simulation based on the historical estimates of developers), that predicts when will software ship (not an exact date, but the probability that software will ship on said date).

You can also see the estimation probabilities for the developers:


So how do you know if the tool is any good? Well, I haven’t tried it, but coming from Joel Spolsky's company, it has a good chance of being high quality (attention to detail, good UI). But you can see a video from the new version of the tool FogBuz 6.0 Movie Demo, try it out or read abook about it by Mike Gunderloy: Painless Project Management with FogBugz.

SQL 2008

Well, it looks like SQL 2008 is just around the corner, but it seems rather quiet for a new release. Maybe it's a small release, or MS isn't using all the marketing power.

Anyway, as the end of support for SQL 2000 is just 2 months after, looks like a comppeling reason to upgrade :)

Among the features for development that I like more are:

• Auditing data changes (new T-SQL command AUDIT that allows auditing centralization)
• Integration with Linq, Entity Framework, .NET 3.0
• MERGE statement (allows update, insert and delete in a single statement)
• Some general nice additions (that I won't use much, but look good to have), like date and time data types, file streams (uses the file system to store files pointed by the database), table valued parameters and Spatial Data

In the administration area, there are also some highlights:

• Data compression (those that worked with medium databases, know what a hassle it is to manage files with 100 GB in storage with less than 0,5 TB)
• Data encryption (columns, database)
• Resource Governor (limit resource usage by users or groups)
• Hot-plug CPUs!

To end the long post, some links about all the new features in more detail:

SQL Server 2008 Improvement Pillars

What's new in SQL 2008 Part 1
What’s new in SQL 2008 Part 2
What’s New in SQL Server 2008 Part 3

Programming Language Humor

Great post about programming language (religious) wars with exceptional humor: Language Trolling Post You'll Ever Need To Read (Hopefully). And don't miss the comments, as some of them are of the same quality :)

In the end, programming languages are just tools and the companies behind them, the supporters. Never quite got it why some people take a critic to a programming language or a company personally...

Source Code

Scott Hanselman has a new column/category (Hanselman - Weekly Source Code) based on the tag to be a better developer started by Justice Gray (Be a Better Developer in 6 Months).

Like Justice Gray committed to reading 1 book per week, Scott Hanselman started reading good source code (as it's one of the best ways to be a better programmer, with which I deeply agree - Reading to Be a Better Developer - The Coding4Fun DevKit).
In one of the recent blogs in this category, I discovered this post: LukeH and a fully LINQified RayTracer. It's a LINQified C# ray tracer (in one Linq Expression that extends over 60 lines!!):

var pixelsQuery =
from y in Enumerable.Range(0, screenHeight)
let recenterY = -(y - (screenHeight / 2.0)) / (2.0 * screenHeight)
select from x in Enumerable.Range(0, screenWidth)

...

select new { X = x, Y = y, Color = traceRay(new TraceRayArgs(ray, scene, 0)) };

He advises against this type of programming (big bang methods/expressions), but it's one heck of an example for Linq.

In the same vein, he links to a list of Programming Guiding Principles by Patrick Cauldwell (This I Believe- The Developer Edition), that start with general principles and has good advices on Unit Tests, Test coverage, Buy vs Build, Dependencies, Contracts (code), Visibility, Authentication, Tracing, Error handling and The definition of done (it means different things to every developer that I know).

Reading all this stuff about code (and the code itself), I started thinking about what are good qualities of Source Code. So, to every beginner (and as a reminder to all senior developers that practices and repetition makes perfection) go [re-]read Code Complete by Steve McConnell...

To me, the first that comes to mind is:

• stop and think for five minutes before defining the name of a method, variable;
• think early on on the design of the code (method or class), it's complexity, performance and all other important characteristics;
• DRY (Don't repeat yourself - don't copy paste code, documentation, use cases, ...)
• KISS (Keep it Simple ..., the design, the methods)
• ...

Functional Programming Examples?

Recently, Mario Romano and me did a presentation about Language Trends, focusing on Linq and the .NET framework. It's clearly a trend in .NET, and there are some great blogs about FP in .NET:

Wes Dyer

Wesner Moise nStatic

Mechanical Bride

FP Tutorial

This one focuses more on Lisp, Haskell and other FP languages, but gives an hand to us imperative programmers:

http://www.defmacro.org/rss/news.xml

 

My main problem is applying this to practical day to day examples. I've seen many examples of the Fibonnaci, Hanoi Towers and others that are great to get your hands on code, and I can relate to the advantadges of FP (composability, restriction on side effects, conciseness/terseness?).

But I think that to convert more people (or at least look) to FP, there has to be some compelling examples that clearly show the advantadges. Maybe related to Web programming (continuations), or meta-programming (get rid of code generators/copy-paste and with the aid of expression trees and higher order functions, define the basic skeleton of methods - try , validate, call some function passed in, log exception, commit, rollback).

 

Any good examples/ideas?

.NET Framework Source Code

It's all over the place, but here it goes again: Microsoft is making available the sources of much of the .NET Libraries with Visual Studio 2008. It will allow to step into framework code while debugging!

http://blogs.msdn.com/brada/archive/2007/10/03/net-framework-source-code-release.aspx

http://blogs.tedneward.com/2007/10/03/Reports+Of+Snowballs+In+Hell.aspx