Friday, November 09, 2007

SQL Tricks/Patterns 1 - Numbers Table

One pattern that I found very useful (I think the first time I saw it was on SQL Server Central), is a Numbers Table.

So what is a numbers table? It's a table pre-loaded with numbers :)
Literally 1, 2, 3, 4, 5, ..., 100.000 (the upper limit depends on the maximum estimated size of the results).

Ok. So how can it help you? How can it be used? It has many uses, beign the most common ones reporting sheets.

Suppose you have to make a report showing all the orders by day of month (or year). But with a business twist (a requirement), you must show all dates, even if there's not an order for that day. Assuming NorthWind orders table:

OrderId         Date
10270           01/08/1996
10271           01/08/1996
10272           02/08/1996
10273           05/08/1996
10274           06/08/1996
...

You want a report that shows the number of orders for every day:

Date                        #Orders
01/08/1996        2
02/08/1996        1
03/08/1996        0
04/08/1996        0
05/08/1996        1
06/08/1996        1

So a simple group by date in the orders date doesn't cut it (we have no orders for some dates).

 

Enter the Numbers table. In this case, what we want is a table with the dates we want to report (the month, the year) and left join it with the # of orders by day. But we can generalize it, and instead of a table with dates, we can use a Numbers table. Something like:

 

declare @date datetime

set @date = '1996-07-31'

select
            dateadd(day, N.number, @date) as Date, 
            isnull(O.NumOrders, 0)
from 
           Numbers N
          left outer join (
                 select OrderDate, COUNT(*) as NumOrders
                 from Orders
                 group by OrderDate
          ) O on dateadd(day, N.number, @date) =
                         O.OrderDate

where N.number <= 31
order by Date

And voilá, the report we want comes out. Note also, that with this approach, it's easy to change the granularity of the report (by hour, by month, by year). The number column (of Numbers table) is added as days to the initial date of the report, which acomplishes us a table with 1 month dates (31 days).

If you want to see some other examples (some esoteric), see also:

The Joy of Numbers

The Helper Table Workbench

 

Technorati tags: , ,

Thursday, November 08, 2007

VB Linq Cookbook

The VB Team has a great series named Linq Cookbook. In it they show some examples of Linq usage in VB that look great.

Entries so far:

linq cookbook recipe 1 change the font for all labels on a windows form

linq cookbook recipe 2 find all capitalized words in a phrase and sort by length then alphabetically

linq cookbook recipe 3 find all the prime numbers in a given range

linq cookbook recipe 4 find all complex types in a given assembly

linq cookbook recipe 5 concatenating the selected strings from a checkedlistbox

linq cookbook recipe 6 your first linq application using northwind

linq cookbook recipe 7 selecting pages of data from northwind

linq cookbook recipe 8 querying xml using linq

linq cookbook recipe 9 dynamic sort order

linq cookbook recipe 10 pre compiling queries for performance

And apparently VB support for Linq seems superior to C#. I had already read about it (VisualBasic Envy), but the XML syntax seems much terser and cleaner. Just compare:

Dim itemList1 = 
      From item In rss.<rss>.<channel>.<item> _
          Where item.<desc>.Value.Contains("LINQ") Or _
               item.<title>.Value.Contains("LINQ")

http://blogs.msdn.com/vbteam/archive/2007/10/01/linq-cookbook-recipe-8-querying-xml-using-linq.aspx

to

var query = from c in xElement.Descendants("book")
     where double.Parse(c.Element("price").Value) < 10
     select new {
            author = c.Element("author").Value,
            title =   c.Element("title").Value
    };

http://blog.adrianroman.ro/post/C-30-Querying-XML-in-C-with-LINQ-to-XML.aspx

 

Update:

Just found out an example of Linq usage inside Excel (with VSTO). The video is available at: How Do I: Use LINQ Against Excel Ranges?

Looking at the example, two things are missing (that could be implemented):

  • Turning an excel range into a list of objects (better yet, mapping the columns into fields of a supplied class)
  • "Dumping" the results starting on a specified cell 

Other than that, just another great example of Linq usage.

 

Technorati tags: , , ,

Friday, November 02, 2007

ASP.NET Configuration - "Unable to read the project. The device is not ready"

ASP.NET configuration is somewhat an hidden and ancient art. Recently, after setting up my new machine, and while changing projects (reconfiguring IIS home directory), one of my web projects refused to open from within Visual Studio (mind you this is Visual Studio 2003 and all of the projects are still ASP.NET 1.1).

The site was accessible from IE, and worked fine.

The error message isn't very helpful: "Unable to read the project webxxx. The device is not ready". As the machine was new, I thought that I forgot to apply some of the steps while configuring the solution. So I rechecked the entire process:

  • Validate that the directory configured in IIS is the correct one (very easy to get wrong when changing between multiple solutions, each with a Web project)
  • Validate that the site is configured for ASP.NET 1.1
  • Validate the authentication for the virtual directory (Integrated windows authentication)
  • Give the apropriate permissions to ASP.NET user in the file system (in despair I even tried giving full permissions to everyone, with no luck)
  • Remove the project from the solution and re-adding it from the web
  • Validating the site URL configuration in the webinfo file (local configuration)

So I resorted to ignorance and re-installed ASP.NET (aspnet_regiis.exe -i).

All to no avail. The web project refused to load. Googling it up, there weren't many answers (I found some forums, but they either wouldn't have any answers or were in an unknown language).

Until I found this: http://www.velocityreviews.com/forums/t112064-unable-to-read-the-project-file.html

    which resolved my problem. It states the problem and solution (removing the hidden suo file), which I should have tried as it's one of the things I do when the settings and synchronization with VSS start to get all messed up.

                          After that, you have to reconfigure the settings for the web project, so it's best to:
                        • Remove all the files from the local directory mapped by in IIS (and the directory itself)
                        • Accept the setting in VS (which should maintain your original directory and not get you into the _N suffix hell. I have seen developers getting to directory _23 in some projects)

                        Thursday, November 01, 2007

                        Programming Languages and People

                        Jeff Atwood posted a thought provoking (and controverse) post: Embracing Languages Inside Languages. In it he argues againts fluent interfaces (APIs that read like English), specifically two examples about SQL and regexes. I agree that those examples add complexity, verbosity and that trying to hide one's ignorance behind an API is not a solution (one of the commenters suggests replacing "a = (b + 1) * a ^ 2" with "a.valueOf(b.plus(1).times(a.pow(2))").

                        But I also see the power in fluent interfaces, specially in Domain Driven Design. Having a DSL defined on a specific domain, the use of a good fluent interface, could allow business experts to understand code (and ease the translation from the business domain to code). Redefining two clearly established DSLs - SQL and regular expressions - with a more verbose syntax, with new quirks, seems foolish. The stated advantadge of being fully OO, is just people trying to hide or cover the fact that they don't fully understand the syntax or the semantics of the language. The fact that it's more verbose, doesn't imply that it's more readable (to me it's quite the opposite, as I prefer terseness to verboseness). And if you don't understand a subject, instead of covering it up, admit that you are a terrible programmer like me, go out, buy a book (try Head First books, as they are really good to learn new & difficult topics) and master the topic.

                        People embrace ASP.NET (or JSP) programming without question, seeing it's advantadges, conciseness over hand generating HTML. The same happened with High Level Languages and Assembly or recently between VM languages (C# and Java) and native languages (C, C++).

                        So, I really don't understand people's aversion to SQL. It's one of the oldest, more popular and widely used DSLs. Yet it scares so many people, that is hidden or abstracted away. That way, it's true power is never realized (specially the set oriented paradigm, instead of row based).

                        It must be a human thing like the ones I read in the Dinosaur Programmers Know More Than You blog post. As we stick around and watch technology waves (I've been watching professionally for almost 10 years) we get to know "what programmers forget, screw up, hope won't happen, and ignore" because that's what we did. And we become "Dinosaur Programmers", because we see each new technology as it is (to us), not as it is marketed. And see it transform from the hype of the day (XML can be used everywhere, for everything is one of my favorites) to something usable, stable and mature. To end I must quote the great ending of the post, with which I deeply agree (emphasis added by me):

                        Every new technology sounds great, and is capable of great things when it's optimally applied. Which it almost never is. And nothing brings out the imperfections of a new technology like real world usage, in a mission critical environment, with money on the line.


                        So you'll forgive me when I don't get excited about your new framework/language/architecture/process that's going to change everything/usher in a new paradigm/launch the next Google. As long as people are writing code, and assembling component frameworks, and modeling the business processes, they're going to continue to forget, screw up, hope for the best, and ignore the vague and uncertain parts.


                        While a new technology might eliminate a whole species of software development errors, you'll just discover, and now have the opportunity to explore, a whole new jungle of failure.
                        Dinosaur programmers already know this, it's our home turf.