André Silva Cardoso

Online Events

2010-11-15T01:27:00.001Z

The last couple of weeks were full of events. Although I dind’t attended any of them in person, I could get some of their essence online, in between work (or after it).

The first event was PDC2010. Even though it has always been an american and the main MS developer event, this year the keynote was broadcasted live throughout the world. I attended at the MS auditory in TagusPark (Portugal), and the keynote main themes were:

Azure (PaaS, Market place, VM role, Server App Virtualization, SQL Azure reporting, Application Fabric and composition, TFS, extra small instances)

IE9 (HTML5, hardware acceleration, pinned sites)

WP7 (many demos, developer tools, and a WP7 phone for every PDC 2010 attendee!)

But besides all the clouds, demos and TLAs (or Four LAs), the main reason I had to watch PDC sessions was the future of C#. And even though the keynote didn’t have any hints on this regard, the sessions with Hejlsberg (FT09) and with Bart De Smet (FT10) were very good.

So as the theme in C# 4 was Dynamic, the theme in C# Next will be Asynchronous Programming, keeping with the trend of Declarative and Concurrent. The presentation demoed the async modifier and the await operator, resulting in an easier model of composition of asynchronous methods.

It was a bit disapointing seeing that “Compiler as a Service”, is in the makes, but apparently only for an even future version. So all the possibilities of architectural refactoring, DSL embeding, and meta-programming won’t be possible in the near future.

The presentation on Linq to everything (FT10) was very good, focusing on some advanced and possible uses of Linq, Providers, Rx and the dual of IQueryable ~ IQbservable.

The other event (of relevance to me) was the PASS Summit 2010, also available through live streaming. As it occurred during work hours, I had to watch it with interruptions, but from what I saw, the main announcements were:

SQL Server 11(?) – Denali CTP (column based query accelerator, round-trip refactoring, deployment improvements, sequences, paging, FileTable – WinFS back?, Full-Text Search improvements)

SQL 2008 R2 Parallel DataWarehouse

MS Atlanta (cloud service for configuration problems and other issues)

MCM for SQL Server changes (the program will be avaible worldwide outside Redmond, more affordable, and the training will be available through SQL Skills, with many training videos available for free)

Marcas Technorati: SQL,.NET,Events

TechEd Europe 2009

2009-11-16T14:10:00.001Z

Well TechEd Europe 2009 held in Berlim, finished yesterday. It was another great experience, where I learned some stuff that I was eager to get my hands on (VS 2010 Architecture), contacted some Portuguese fellows and MS gurus, and discovered a little (not much time available, and at 4PM it’s completely dark) of Berlim. Also received a free copy of Win7 and some kits (SQL 2008, Win 2008), which will come handy.

There were also some missteps:

On arrival, all of the bags that came from Lisbon didn’t make it to the airplane and were left in Madrid. Not very pleasant, arriving at 22:45 at Berlim and not having any more clothes when the temperature is 2ºC.
The wireless/Internet on the hotel was not free, and very expensive: 9,95 €/hour, or 19,95 €/day.
The logistics around the event had some problems, namely with the check-in, and the cloakroom (huge lines). But, for a first event (for Berlim) and with > 7.000 people attending, it sould be expected. According to some powerpoints of the sessions, there will be no DVD with the sessions recordings and powerpoints available for the attendees (but the e-mail with the TechEd offers, mentions the MMS 2009 DVDs Breakout recordings and HOLs, so I don’t know what to make of it).
Almost nobody speaks English in Berlim, so we were on our own. The younger people speak a little English (but one person from the hotel confused “no problem” with “not possible”, making for a very strange conversation).
The flight back was at 7:30 AM so waking up and checking-out between 4:30 AM and 5:30 AM was a bit tough. Also some confusions with the credit card payment of the hotel (it was just reserved, apparently), made for a tought day (not over yet).
The connection flight is only at 15:45, so we have to wait out in the aeroport for more than 4 hours and are already very tired. Also the WI-FI at the airport is not free, so no Internet.

But back to the event. Microsoft grouped the IT sessions with the Developer sessions, which has its pros and cons. On the upside, I was able to assist some very good and on-time sessions on “Virtualization and Consolidation” (Gopal Ashok), “Upgrading to SQL 2008” (Dandy Wein), and two very good sessions from Russinovich (from SysInternals fame) “Pushing the limits of Windows”, “Windows Troubleshooting”.

On the downside, the keynote was more IT oriented (Exchange 2010, a new efficiency debate – which was a bit boring, and Windows 2008 R2 and Hyper-V).

For me, the best sessions on the Architecture Track were David Chappell “IT Innovation”, Rafal Lucawiecki “Predictive Programming”, and Don Smith “Application Architecture Guide”. On the Development Track, I really enjoyed the ones around VS2010 Architecture Tools: Peter Provost “Architecture Discovery and Validation”, and Doug Seven “TFS: Become productive in under 30 minutes”.

A bit of a let down were the sessions from Tess Fernandez “Debugging ASP.NET” and Brian Keller “SW Testing in VS 2010”. Not that they were bad, on the contrary, but I was expecting more (maybe it was the expectations).

Update:

The post was written on Saturday, 14 November on the airport, so the event finished 13 November. The post was done today because there was no free wireless on the airport and the remainder of the weekend was just family time.

Marcas Technorati: Events,TechEd,Architecture,SQL

SQL Tips/Tricks 2

2009-11-16T13:24:00.001Z

Excel is an excellent tool for generating sql scripts of reference tables. Besides being easy to generate the scripts, it’s also easy for the client to manipulate and update the reference information (almost always, it’s the format used for database like information for clients).

So for an excel with the following structure

TableX
ID	Name	Date1	UserId
1	Abc	10-01-2009	Alc
2	Def	10-01-2009	Xyz

where the $A$1 is the table name and the columns name are on row 2 (ID, Name, Date1, UserId), inserting the formula on column 6 (or bigger) without newlines:

=CONCATENATE("INSERT INTO ";$A$1;"(";A$2;",";B$2;",";C$2;",";D$2;") VALUES(";A3;", '";B3;"', '";TEXT(C3;"aaaa-mm-dd");"', '";D3;"')")

yields the following result:

INSERT INTO TableX(ID,Name,Date1,UserId) VALUES(1, 'Abc', '2009-01-10', 'Alc')

What the formula is doing is concatenating “INSERT INTO”, the table name ($A$1), the parentesis, column names (A$2, B$2, …), the string VALUES, and the actual values separated by commas and delimited with ‘ when appropriate (string or date values).

So to generate the script is just a matter of copying the formula for all the rows with values (double clicking the cross symbol on the right lower corner of the cell, and it repeats itself according to the adjacent cell, or pulling down the intended number of rows):

And the end result is this:

And that’s it. A script ready for pasting in a SQL file and applying to a DB.

Marcas Technorati: SQL,Programming

Google Chrome OS

2009-07-15T00:17:00.001+01:00

Google has made the announcement of the Chrome OS and has been riding the wave of free publicity on every blog and tech site.

So what to make of it? Bruce Eckel (from Thinking in Java fame) thinks it’s another nail in Microsoft’s coffin, the timing is perfect (because of netbooks and Vista failure), and Microsoft is too big to react rapidly.

But many think that Google has lost it’s mind, operating systems are too complex, it’s Microsoft and Apple territory, and 2H 2010 is too late (Ted Neward’s Thoughts on the Chrome OS announcement and Fake Steve’s Let's all take a deep breath and get some perspective). Mini Msft already sees Microsoft turning the corner with Windows 7 and Bing, while Google is loosing steam and facing some of Miscrosoft problems.

But to me the most correct analysis is made in The Google Revenue Equation, and Why Google’s Building Chrome OS, that puts the focus on the strategy, that for Google is to center the User Experience on the Web (as it monetizes online ads), while for Microsoft is to center the User Experience on the Desktop (because of Windows and Office). So each player is reaching out to maximize, consolidate and uniformize that experience. For Google it means reaching out to Mobile, Browsers and Operating Systems (that focus on the Web). On the other hand, for Microsoft the center is the Desktop, so RIA and it’s online offers exist primarily to focus and integrate well with Windows and Office Desktop.

Marcas Technorati: Micorosoft,Google,Strategy

VS 2010

2009-06-24T17:56:00.001+01:00

Marcas Technorati: Programming,.NET,Visual Studio 2010

There’s some great stuff coming up with VS2010. First, there’s a focus on performance (probably because Rico Mariani is for some time the chief architect of Visual Studio, so startup should improve as well as the text editor), but no plans for a 64 bit version.

Another area getting a lot of attention (that is deserved) is parallel computing, with some new patterns and librarys (thanks for the post, Mário):

An area that I’m especially eager to really discover is debugging/profiling with VS2010. According to one Dev Manager, the Beta 1 already has:

Historical Debugging (collecting events, parameters, performance monitor)
Test impact Analysis
Tier profiling (DB calls)
Parallel performance analyzer
Performance rules

Which is very good, and John Robbin already dived deep and poked with the feature.

Marcas Technorati: Programming,.NET,Visual Studio 2010

Top 100 Books of SW Engineering

2009-01-27T16:25:00.001Z

I just found out a great compilation of the top 100 books of Software Engineering. The method for computing the list involves Amazon ratings, Jolt Awards and Google hits, so it's as scientific as it can get...

Looking at the list makes for a tremendous wish-list/bookshelf:

#
Author(s) / Title

1
Steve McConnell
Code Complete: A Practical Handbook of Software Construction (2nd Edition)
2
Elisabeth Freeman, etc.
Head First Design Patterns
3
Steve McConnell
Rapid Development
4
Erich Gamma
Design Patterns: Elements of Reusable Object-Oriented Software
5
Bruce Schneier
Applied Cryptography: Protocols, Algorithms, and Source Code (2nd Edition)
6
Robert C. Martin
Agile Software Development: Principles, Patterns and Practices
7
Joel Spolsky
Joel on Software
8
Tom DeMarco, Timothy Lister
Peopleware: Productive Projects and Teams (2nd Edition)
9
Frederick P. Brooks
The Mythical Man-Month, Anniversary Edition (2nd Edition)
10
Martin Fowler
Refactoring: Improving the Design of Existing Code
11
Mike Cohn
Agile Estimating and Planning
12
Alistair Cockburn
Writing Effective Use Cases
13
Bertrand Meyer
Object-Oriented Software Construction (2nd Edition)
14
Steve McConnell
Software Estimation: Demystifying the Black Art
15
Mike Cohn
User Stories Applied: For Agile Software Development
16
Donald E. Knuth
The Art of Computer Programming, Volumes 1-3 Boxed Set (2nd Edition)
17
Martin Fowler
Patterns of Enterprise Application Architecture
18
Jeffrey Friedl
Mastering Regular Expressions
19
Andrew Hunt, David Thomas
The Pragmatic Programmer: From Journeyman to Master
20
Karl E. Wiegers
Software Requirements (2nd Edition)

The full article is at Top 100 Best Software Engineering Books, Ever.

Marcas Technorati: Architecture,Programming,Project Management,Books

Architecture again

2008-12-12T14:55:00.001Z

Right after my last post (which I was on my mind because it happened to me a couple of times this year and the last year), I discovered (thanks to a post from a friend patterns & practices- App Arch Guide Pocket Guides) that MS PnP has some interesting stuff on this topic (shame for not looking recently on PnP).

Apart from the pocket guides, that I haven't looked into, what I found were some cheat sheets and diagrams.

PnP Application Architecture Frame Cheat Sheet describes architecture frames (authentication, caching, ... and 14 others), quality attributes (14) and the mapping between application types, architecture styles and the architecture frames (and the common issues). It's great stuff, and should be mandatory read for every developper. Shame that Microsoft only has teaching offerings around products (maybe that's what the industry cares about).

Another one is PnP Application Type Matrix Cheat Sheet that summarizes the main application types, their benefits, considerations, scenarios and solutions.

The last one is PnP Visio index Diagrams that MS makes available for modification and use to document our architectures. Even if only used for proposals, it's very good and allows reuse.

Marcas Technorati: Architecture,Programming

Architecture & Process

2008-12-05T18:43:00.001Z

Lately, I've been surprised by how little attention some developers pay to application architecture. By architecture here, I mean to say the high level decisions that crosscut all the application code.

Recently, during the transfer of an application from another team, I was shocked that there was no clear vision of the main decisions regarding some fundamental aspects of a web application (I don't mean documented, just a clear definition of the decision or the approaches taken in the implementation). Every web application should make explicit the major decisions (and the reasioning behind them) regarding:

Logging (policy - how, when; and tools/code)
Exception Handling
Data Acess (ADO.NET, ORM, Linq)
Session Management (asp.net memory, DB, custom)
Security (low-level - sql injection, cross scripting; high-level - isolate data between different users/entitys/geographically)
Profiles and Permissions (management of users/groups and the corresponding permissions on application functionalities)
Operation Auditing (especially in financial systems)
Composition (tiers, layers, service oriented)
Dependencies (3rd party tools, components, services)
Patterns used (MVC, Singleton, Composite)
Naming conventions
Configuration management (reference tables, configuration values, connection strings, etc)
Concurrency (synchronizatios, async callbacks, threading)
...

There's also other important stuff, more on the process/principles side, that gain by being defined:

Planning, prioritization & risk management
Organization (teams, projects)
Tools
Automation of tasks (building, testing, importing reference data)
Testing and Coverage (unit testing, integration testing, coverage of significant program states)
Refactoring
Documentation (design, architecture, glossary, major entities, workarounds to problems)
Bug/Incident tracking
Version Control (tool, policy)
Communication
DRY
Responsabilities (code & people) and Separation of concerns
Coupling & Cohesion
Done?

As with everything, there's also the risk of overdoing stuff, or doing it as an end (and not as mean to an end).

If just 1/3 of the list were implemented, maintenance would be such an easier job...

Marcas Technorati: Architecture,Programming

SQL set approach

2008-08-01T00:01:00.001+01:00

SQL is a powerful tool that can be a great help for solving some problems. You just have to think in a set oriented approach (Thinking in Sets by Joe Celko comes to mind). I just hope thinking this way doesn't transform me into some uptight person that thinks less of everyone else, just because they don't understand it initially. And this approach must not be taken as a dogma, or an end in itself. If a solution doesn't appear in some reasonable time, I resort to iterative approaches. Better not even get into primary keys discussion and business identifiers... (end of rant)

Today, during the implementation of an algorithm for shift atribution, the client asked for the implementation of some exceptions when there wasn't anyone available in the specified region. After some discussion, the algorithm should cope with this situation by looking up in the neighbour regions, then in a larger area, and finally the whole country. As the algorithm isn't just a select (it has more rules), I didn't want to replicate the whole algorithm for each universe or apply the algorithm iteratively for each region (that would also distort the ordering and equity of the algorithm).

After some brainstorming, the problem was how to change the universe in which to look by joining to the region table. And as I was to give up, the solution became evident: what was needed was filtering the universe. But how if the table doesn't have in itself all the attributes to determine the regions?

One way would be to compute the regions with a function and using a dynamic query with an in filter. Something like:

regionId IN
dbo.fnComputeRegions(regionId, coverage)

But that forced us to use dynamic querys and the in could become quite big (there are a couple hundred regions).

Then, a better solution came to my mind: the problem was in the join, so the query should join with a table valued function that returns a table with the regions to lookup. This solution didn't force the use of dynamic queries and allowed us to reuse the whole algorithm just changing the join, surrounding the algorithm with a while that changes the coverage:

while (coverageId < numberOf Coverages)

     (algorithm)

     select ...

     from

        Region R

        inner join dbo.fnComputeRegions(regionId,
              coverageId) CR on R.regionId = CR.regionId

With this approach we gain abstraction and independence of the computation of regions (it can return a table with one record, the neighbour regions, the neighbour neighbour regions, all the regions within an hierarchy, or all the regions), without changing the algorithm. And what's best is that these computation of regions is reusable in other areas and algorithms.

Now the toughest part is convince the client that this is the best solution and that to these rules there must be no exception :)

Marcas Technorati: Programming,SQL

Lang.NET Videos

2008-02-27T14:26:00.001Z

The videos (require SilverLight 1.0) from the Lang.NET conference are available at: http://langnetsymposium.com/talks.asp

[Update] The one that I was most eager to see 3-09 - Intentional - Magnus Christerson is great, and represents what future IDEs could be in some time.

Can't also wait to watch:

Static Analysis

3-04 - NStatic - Wesner Moise - SoftPerson

2-11 - CodeIt - Serge Baranovsky - submain

2-08 - Pex - Peli de Halleux

Languages and Features

2-10 - remotion Mixins - Stefan Wenig and Fabian Schmied - rubicon

2-01 - Newspeak - Gilad Braha - Cadence

3-02 - FSharp - Luke Hoban

3-07 - Modeling and Languages - Don Box

3-07 - Modeling and Languages - Don Box_1

3-08 - Cobra - Chuck Esterbrook

(read it on Lang.NET videos are now online)

Marcas Technorati: Languages,.NET

Classic Books

2008-02-14T19:13:00.001Z

I really like reading, specially blogs, because of their small format (one can read a couple of entries in a couple of minutes). Nowadays, I don't really buy that many books as much of the information is available on the web. The exception to this is some classic books or intemporal books (Patterns, Methodology, Usability, ...) that I buy and try to read (usually takes me a long time because of interruptions, priorities, certifications, ...).

But recently there's been a trend on releasing classic books and articles free on the web:

There are some great gems (Macintosh Human Interface Guidelines, Structured Programming) available, some books related to Lisp, Smalltalk, and other esoteric languages, and CMG papers (mostly performance and ITIL).

Marcas Technorati: Books,Programming

Microsoft and Yahoo

2008-02-06T18:40:00.001Z

It's not the most recent news, as it's been dissected and commented by everyone else, but nonetheless an interesting topic.

From the MS fan base Hello, Google, can you spell hypocrisy? to the MS (particulary Ballmer) bashing A Defining Test for Ballmer, the most interesting ones were the internal ones Microsoft + Yahoo! = Microsoft - $44,600,000,000 ? and Microsoft and Yahoo! -- Stay on Target?, and Bruce Eckel's Should Microsoft Buy Yahoo?.

The internal perspective focus more on the restructuring, layoffs, overlapping of products and foresee the potential problems (culture, technological - spreading MS technology within Yahoo), and possible sinergies, defining the dominant reaction as: "talk to me in a year".

Bruce Eckel focus is on the acquisitions, and the merge failings when two big software/hardware companies try it. I can't really assess if it is truly that bad, but MS and Yahoo should really be on their toes to avoid the problems. Those that don't learn from history are condemned to repeat it (or something similar).

I really think it's impossible to predict the outcome of this. MS and Yahoo should take close attention to the details of the merge (cultural, tecnological, user base), and avoid imploding themselves.

Google should take attention as MS showed in the past, they can/will use any tactic they can to get back into the game. If MS can leverage their dominant position to spread Silverlight and get control over Web 2.0 content, Google will be in deep trouble. It's not like Google's monopoly generates user lock-in (as MS Windows and Office, and other MS tactics - IE), so they better watch it carefully as changing my search engine is just some clicks away...

So, in conclusion, I guess only time will tell what the outcome will be. Whoever does the fewest mistakes in this game, will take leadership. Although, for the stake of evolution, I hope that no true winner comes out of it, and competition becomes fiercer...

Marcas Technorati: Microsoft,Yahoo,merge

SQL 2008 is late^H^H^H^H on schedule for Q3

2008-02-01T11:31:00.001Z

As Joel Spolsky puts it, Microsoft can't speak straight any more. Instead of coming forward and saying SQL 2008 is late, marketing added it's twist to the message (I don't think an engineer can talk this way), transforming a simple message in a convoluted, positive tone marketing speak.

It's on par with Dilbert material:

I think it just transformed an inocuous message of schedule slipage (pretty common in software development) into a comic situation (or patronising, depending on how you take it).

Phil Factor digged deeper and tried to explain it: Microsoft Boy announces his School Homework.

Oh, here's the original message:

The past few months have been an amazing time for the SQL Server team as we gear up for the start of the global launch wave on February 27.

...

Simply put, SQL Server 2008 is a significant release for us – one that builds on all of the great things that we were able to deliver in SQL Server 2005. We see it as a critical step forward for our data platform...

Not surprisingly, one of the top areas of focus for us is always to deliver a high quality product, and in a very predictable manner.

...

To continue in this spirit of open communication, we want to provide clarification on the roadmap for SQL Server 2008. Over the coming months, customers and partners can look forward to significant product milestones for SQL Server. Microsoft is excited to deliver a feature complete CTP during the Heroes Happen Here launch wave and a release candidate (RC) in Q2 calendar year 2008, with final Release to manufacturing (RTM) of SQL Server 2008 expected in Q3.

...

This does not in any way change our plans for the February 27 launch...

Marcas Technorati: SQL 2008,Roadmap

Dynamic and Static Languages

2008-01-24T14:38:00.001Z

Ted Neward blogged about the "Can Dynamic Languages scale" debate going on ServerSide. He discusses the two dimensions of scale that are being debated, namely Size (LOC) and capacity (reqs/sec).

Aside the flamefest that the thread became, I always get the feeling that most arguments are somewhat technically based, but very biased.

I'm not a Dynamic Language Programmer. But as I try to stay current on the evolution of Software Development and Languages, I try to read as much as I can about technology that is being touted as the future, or very promising. Not that I immediatly try to use it in production, as I admit that some corner cases could be tricky (or badly done) because of lack of knowledge or technology imaturity. Beyond that, many hot technologies, while having their merits, are pushed way too much because of the hype (CORBA, XML, EJB, SOAP, anyone?) generated by the BIG (MS, IBM, Sun) companies.

Refactoring

Anyway, one of the arguments countering dynamic languages is the ability to refactor it. Namely that the refactoring isn't fullproof and that there can be mistakes. But guess what, Java/.NET because of Reflection, Dependency Injection can also have these mistakes. I agree that tool support is better in Java/.NET, but that's it.

Conciseness

Another argument, these time by the other camp (dynamic languages) is that dynamic languages are more concise. And comparing the best known instances (Java, C#) to Ruby or Python, one would have to agree. And no, even if the compiler could generate it, it's still code left to maintain and read, that distracts from the code intent.

But even that is changing, as C# 3.0, with type inference, lambdas, automatic properties, extension methods and Linq, removes or simplifies much of this verbosity. So the conciseness argument doesn't depend on being dynamic or static.

Speed

The argument of speed has two meanings: which language/platform executes faster and which language allows the programmer to develop/test faster. The Java/"static" side sits with the faster execution (even though, I still remember when the comparisons to C++ were very bad, this argument was dismissed by the faster programming argument), while the "dynamic" couples to the faster development side.

Even stranger is the bitterness or stuborness of each camp. It's not like the languages/platforms are stagnant. As features are added, and the platforms mature, static languages will get some of the features from the other side (conciseness, interactivity) and dynamic languages will get better tool support and faster platforms. Only by it's evolution will languages stay relevant/dominant. If they don't evolve, they'll lose dominance for the next Big Language (Cobol, C, C++?).

All of this makes sense to me, because as someone said: those that don't learn from history are condmened to repeat it...

Marcas Technorati: Languages,Programming

SQL Tricks/Patterns 0

2008-01-23T00:19:00.001Z

It's been a while since my last post... These last couple of months were full of events for me. The biggest one was the birth of my second child. It changed our routines quite a lot... Right now managing to sleep more than a couple of hours with a baby and a small child is our biggest achievement. But it's full of rewards :)

Getting back to technology, and looking at my last post and some functional programming posts around C#, I came to realize the similarities between SQL and FP. They're both strongly mathematically based, declarative... And both require a change of mindset (I'm still on a very rudimentary level on FP), from what we're used to in imperative programming.

So the first and most important pattern in SQL, and that is the base for many other patterns and the correct usage, is that SQL is set based, and all row by row processing should be avoided (by principle). Let's be a little more pragmatic, and if it's a quick and dirty solution, for a small problem set, a one shot solution never to be used again, it's ok. But if it's a migration, an operation script, a script for the client, then be very careful... It will be used on increasing larger sets, multiple times and with a diminishing time window.

Only being through a data migration in which a test takes a couple of days, that must be executed multiple times (because of bad data, or bad mappings, or bugs), or having to put up with a client because some job is increasingly taking longer and risking it's time window, or even witnessing data intensive operations being done (excruciatingly slow) on the application client, one understands the problem.

Ok. But to a freshmen/rookie/intern, and unless told otherwise, the imperative solution would be the only solution. And imperative has conditionals, and many other goodies...

In SQL, we can simulate conditionals with CASE, for example, updating the salary by 1.1 or 1.2 based on the category of the employee:

UPDATE Employees
SET salary = salary *
CASE categoryId
WHEN 1 THEN 1.1
WHEN 2 THEN 1.2
END

But what if it's impossible to do in one query, or better yet to avoid turning the query overly complex?
Then we use temporary tables to hold the intermediate results. But each step of the processing is done on the set and not on a row by row basis. If some step is impossible, or too hard to implement using set operations, then only that step is done row by row, but the remaining steps are set based.
If some task is complex (reading, parsing, validating and loading a file to the DB) or the business allows for it, it can even be done on table using some sort of state identifier (step 1 is reading, or aproving...).

But what if I have to insert into other tables? Well, insert is also set based so instead of INSERT (...) VALUES (...), use INSERT (...) SELECT ...
There's another problem, to which I don't have a satisfactory answer. That's when you have a master/detail relationship and the master uses Identity values. In that case, apart from switching the identity property on and off, and defining the max value, it's difficult to insert a batch of these relations (using set based processing). Other possibilities are using triggers or in SQL 2005, the output clause, but they all seem awkward and fragile.

Another problem is generating a number or some kind of order in the set. To that, the numbers table from my last post is the answer. There are some variations and simplifications. If all you need is a sequential number for the data (to order it, or to differentiate it - very good to remove complete duplicates), a temporary table with an identity column might suffice.This is also the principle used in most solutions that paginate result sets (joining the table with itself using some computed number and filtering it).

Well, I must be forgetting a lot more, but as it's already too long, I'll end it here.

Until next time...

SQL Tricks/Patterns 1 - Numbers Table

2007-11-09T15:26:00.001Z

One pattern that I found very useful (I think the first time I saw it was on SQL Server Central), is a Numbers Table.

So what is a numbers table? It's a table pre-loaded with numbers :)
Literally 1, 2, 3, 4, 5, ..., 100.000 (the upper limit depends on the maximum estimated size of the results).

Ok. So how can it help you? How can it be used? It has many uses, beign the most common ones reporting sheets.

Suppose you have to make a report showing all the orders by day of month (or year). But with a business twist (a requirement), you must show all dates, even if there's not an order for that day. Assuming NorthWind orders table:

OrderId         Date
10270           01/08/1996
10271           01/08/1996
10272           02/08/1996
10273           05/08/1996
10274           06/08/1996
...

You want a report that shows the number of orders for every day:

Date                        #Orders
01/08/1996        2
02/08/1996        1
03/08/1996        0
04/08/1996        0
05/08/1996        1
06/08/1996        1

So a simple group by date in the orders date doesn't cut it (we have no orders for some dates).

Enter the Numbers table. In this case, what we want is a table with the dates we want to report (the month, the year) and left join it with the # of orders by day. But we can generalize it, and instead of a table with dates, we can use a Numbers table. Something like:

declare @date datetime
set @date = '1996-07-31'
select
            dateadd(day, N.number, @date) as Date,
            isnull(O.NumOrders, 0)
from
           Numbers N
          left outer join (
                 select OrderDate, COUNT(*) as NumOrders
                 from Orders
                 group by OrderDate
          ) O on dateadd(day, N.number, @date) =
                         O.OrderDate

where N.number <= 31
order by Date

And voilá, the report we want comes out. Note also, that with this approach, it's easy to change the granularity of the report (by hour, by month, by year). The number column (of Numbers table) is added as days to the initial date of the report, which acomplishes us a table with 1 month dates (31 days).

If you want to see some other examples (some esoteric), see also:

The Joy of Numbers

The Helper Table Workbench

Technorati tags: Programming, SQL, Patterns

VB Linq Cookbook

2007-11-08T11:03:00.001Z

The VB Team has a great series named Linq Cookbook. In it they show some examples of Linq usage in VB that look great.

Entries so far:

linq cookbook recipe 1 change the font for all labels on a windows form

linq cookbook recipe 2 find all capitalized words in a phrase and sort by length then alphabetically

linq cookbook recipe 3 find all the prime numbers in a given range

linq cookbook recipe 4 find all complex types in a given assembly

linq cookbook recipe 5 concatenating the selected strings from a checkedlistbox

linq cookbook recipe 6 your first linq application using northwind

linq cookbook recipe 7 selecting pages of data from northwind

linq cookbook recipe 8 querying xml using linq

linq cookbook recipe 9 dynamic sort order

linq cookbook recipe 10 pre compiling queries for performance

And apparently VB support for Linq seems superior to C#. I had already read about it (VisualBasic Envy), but the XML syntax seems much terser and cleaner. Just compare:

Dim itemList1 =
      From item In rss.<rss>.<channel>.<item> _
          Where item.<desc>.Value.Contains("LINQ") Or _
               item.<title>.Value.Contains("LINQ")

http://blogs.msdn.com/vbteam/archive/2007/10/01/linq-cookbook-recipe-8-querying-xml-using-linq.aspx

var query = from c in xElement.Descendants("book")
     where double.Parse(c.Element("price").Value) < 10
     select new {
            author = c.Element("author").Value,
            title =   c.Element("title").Value
    };

http://blog.adrianroman.ro/post/C-30-Querying-XML-in-C-with-LINQ-to-XML.aspx

Update:

Just found out an example of Linq usage inside Excel (with VSTO). The video is available at: How Do I: Use LINQ Against Excel Ranges?

Looking at the example, two things are missing (that could be implemented):

Turning an excel range into a list of objects (better yet, mapping the columns into fields of a supplied class)
"Dumping" the results starting on a specified cell

Other than that, just another great example of Linq usage.

Technorati tags: Programming, Linq, SQL, .NET

ASP.NET Configuration - "Unable to read the project. The device is not ready"

2007-11-02T17:02:00.001Z

ASP.NET configuration is somewhat an hidden and ancient art. Recently, after setting up my new machine, and while changing projects (reconfiguring IIS home directory), one of my web projects refused to open from within Visual Studio (mind you this is Visual Studio 2003 and all of the projects are still ASP.NET 1.1).

The site was accessible from IE, and worked fine.

The error message isn't very helpful: "Unable to read the project webxxx. The device is not ready". As the machine was new, I thought that I forgot to apply some of the steps while configuring the solution. So I rechecked the entire process:

Validate that the directory configured in IIS is the correct one (very easy to get wrong when changing between multiple solutions, each with a Web project)
Validate that the site is configured for ASP.NET 1.1
Validate the authentication for the virtual directory (Integrated windows authentication)
Give the apropriate permissions to ASP.NET user in the file system (in despair I even tried giving full permissions to everyone, with no luck)
Remove the project from the solution and re-adding it from the web
Validating the site URL configuration in the webinfo file (local configuration)

So I resorted to ignorance and re-installed ASP.NET (aspnet_regiis.exe -i).

All to no avail. The web project refused to load. Googling it up, there weren't many answers (I found some forums, but they either wouldn't have any answers or were in an unknown language).

Until I found this: http://www.velocityreviews.com/forums/t112064-unable-to-read-the-project-file.html

removing the hidden suo file

Remove all the files from the local directory mapped by in IIS (and the directory itself)
Accept the setting in VS (which should maintain your original directory and not get you into the _N suffix hell. I have seen developers getting to directory _23 in some projects)

Programming Languages and People

2007-11-01T00:54:00.001Z

Jeff Atwood posted a thought provoking (and controverse) post: Embracing Languages Inside Languages. In it he argues againts fluent interfaces (APIs that read like English), specifically two examples about SQL and regexes. I agree that those examples add complexity, verbosity and that trying to hide one's ignorance behind an API is not a solution (one of the commenters suggests replacing "a = (b + 1) * a ^ 2" with "a.valueOf(b.plus(1).times(a.pow(2))").

But I also see the power in fluent interfaces, specially in Domain Driven Design. Having a DSL defined on a specific domain, the use of a good fluent interface, could allow business experts to understand code (and ease the translation from the business domain to code). Redefining two clearly established DSLs - SQL and regular expressions - with a more verbose syntax, with new quirks, seems foolish. The stated advantadge of being fully OO, is just people trying to hide or cover the fact that they don't fully understand the syntax or the semantics of the language. The fact that it's more verbose, doesn't imply that it's more readable (to me it's quite the opposite, as I prefer terseness to verboseness). And if you don't understand a subject, instead of covering it up, admit that you are a terrible programmer like me, go out, buy a book (try Head First books, as they are really good to learn new & difficult topics) and master the topic.

People embrace ASP.NET (or JSP) programming without question, seeing it's advantadges, conciseness over hand generating HTML. The same happened with High Level Languages and Assembly or recently between VM languages (C# and Java) and native languages (C, C++).

So, I really don't understand people's aversion to SQL. It's one of the oldest, more popular and widely used DSLs. Yet it scares so many people, that is hidden or abstracted away. That way, it's true power is never realized (specially the set oriented paradigm, instead of row based).

It must be a human thing like the ones I read in the Dinosaur Programmers Know More Than You blog post. As we stick around and watch technology waves (I've been watching professionally for almost 10 years) we get to know "what programmers forget, screw up, hope won't happen, and ignore" because that's what we did. And we become "Dinosaur Programmers", because we see each new technology as it is (to us), not as it is marketed. And see it transform from the hype of the day (XML can be used everywhere, for everything is one of my favorites) to something usable, stable and mature. To end I must quote the great ending of the post, with which I deeply agree (emphasis added by me):

“

Every new technology sounds great, and is capable of great things when it's optimally applied. Which it almost never is. And nothing brings out the imperfections of a new technology like real world usage, in a mission critical environment, with money on the line.

So you'll forgive me when I don't get excited about your new framework/language/architecture/process that's going to change everything/usher in a new paradigm/launch the next Google. As long as people are writing code, and assembling component frameworks, and modeling the business processes, they're going to continue to forget, screw up, hope for the best, and ignore the vague and uncertain parts.

While a new technology might eliminate a whole species of software development errors, you'll just discover, and now have the opportunity to explore, a whole new jungle of failure.
Dinosaur programmers already know this, it's our home turf.

”

Terrible Programmer

2007-10-31T15:51:00.001Z

My name is André and I am a terrible programmer (Confessions of a Terrible Programmer)...
I think we all are (maybe not Dijkstra, Knuth, Wirth and some few others, that think their programs mathematically and demonstrate them http://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/EWD1009.html).

So I try to mitigate my human factor by:

testing;
reviewing my code carefully;
automating tasks that I'll have to do more than a couple of times (tests, gathering information for monhtly reports, ...);
investigating new tools and techniques that allow me to do less or with better quality;

And it's the last point that I think holds more promise. Testing and reviewing code are all a double check (most of the time by the same person) that the code is correct. But it's also subject to error.

Automating tasks also buys us more confidence (especially if the automation is tested and used for a while), but sometimes hard to justify. On a recent project that I manage, I invested a couple of days developing some office automation that allows me to save excel reports received by e-mail, uploading them to a database and importing them to a word table. But I knew that, the monthly task would take me half a day every month for one year. So I got a 4 day savings (more if the contract is renewed), and confidence on the output (at least on what depends on me). Another automation that is a must (ok, I still don't implement it) is build automation. It just eliminates all those problems related to builds (multiple web configs for various environments, flags - Release and authentication, conditional code, ...).

But automating tasks like code reviews or quality assurance is more problematic. That's why I'm longing for static analysis tools powerful enough to catch most problems (code coupling, unnecessary complexity, concurrency problems, finding and eliminating code duplication) and easily extensible so that I can easily implement new rules.

I already posted about some tools (NDepend, nStatic - not released yet, FxCop, Simian), but I still think most have some limitations and don't address many problems. Maybe with .NET 3.5 (and the ability to create Expression Trees), gives way for a new generation of tools more powerful and that don't treat code as text files but as ASTs. That way it should become easiser to find duplicate code (independent of variable names), inject faults in code (Fault Simulation), analyze and compute call graphs based on the arguments of functions (and the opposite, given an exception state, what conditions and call graph could lead to that), and many other that I can't even think of.

Blog Voices

2007-10-31T10:55:00.001Z

Congratulations to my friend Mario Romano for reaching 100 posts in October. The grand total of my posts from Nov/2006 doesn't reach 50, so it's an impressive figure.

That got me thinking in what I value on blogs, like the update frequency, the voice, and made me evaluate what are the blogs that I read most and that are my biggest influences.

So, here are the bloggers that I can't miss and which posts I value the most:

Joel on Software is one of the expert voices that has been around longer. His posts show through his experience, and his opinions on Human Resources (hiring developers, ), Software Development (Process, UI), Project Management and Economics (Buy vs Build) are truly outstanding;
Coding Horror is another great blog, with great opinions on Programming, Software Development, Project Management, and some off-topics that appeal to many programmers, that keeps an independent voice (somestimes a bit controverse);
DevCatharsis, from my friend Mario Romano is a very good all rounder which has the same main interests that I have right now: Architecture, Functional Programming and Software Development Process;

Some more focusing on a specific area:

On SQL, there's Kimberly L. Tripp and her great posts on index tuning and general SQL tuning. Has many code samples and comparative tests on performance;
Paul Graham- Unofficial RSS Feed has some great insights on Lisp and recently a focus on startups;
On Functional Programming, with a twist and focus to imperative developers, I really like defmacro.org (great articles on FP for the rest of us) and Did it with .NET (with a focus on C# 3.0 and recently F#). See also The Mechanical Bride (C# 3.0, Linq and lots of => lambdas);
On Performance, Rico Mariani s Performance Tidbits, are a must, although it's been somewhat quiet since he became Chief Architect of Visual Studio (a huge job indeed). It has great analysis on performance from a very high level right down to the bit;
On Microsoft, don't miss Mini-Microsoft to get a feeling on Microsoft internals, Human Resources, shares and profits;
Creating Passionate Users from Kathy Sierra, which is shutdown due to some creeps that harassed her, has many good points on user focus and learning (she's one of the Head First series editors);
On a fun perspective (or maybe not, as it reflects our reality), I really like Worse Than Failure and Dilbert (Ok, it's not a blog);
There are also some other authors like Martin Fowler, Bruce Eckel that I prefer in book format, and Ted Neward on article format and not so much on blog format;

Ok, the list is getting long but I try to keep up to date on these. So what blogs am I missing and why?

What's on your blog short list? If someone reads this, post your list and the reasons why it's on your list.

Software Estimation

2007-10-23T12:15:00.000+01:00

So why are software estimates so difficult? I don’t know, but between technical problems, little details that take forever, customers changing requirements and conflicting requirements, estimates are normally way below the real time that takes to do something (a rule of thumb is just doubling your normal estimate just to be safe).

Steve McConnell just wrote a book (Software Estimation: Demystifying the Black Art), that I recently bought (but still haven’t read –I’m planning to read it soon), that tries to ease this problem and shed some light into the black art. But what can you say, when even someone as knowledgeable as Steve McConnell, misses estimates when building a fort house by 100% (http://blogs.construx.com/blogs/stevemcc/archive/2007/09/23/building-a-fort-lessons-in-software-estimation.aspx). Well, it was just a small construction plan, that most wouldn’t even try to estimate and just plug along.

But there are great lessons to be learned, that apply to software projects:

Numerous unplanned problems collectively added up
Underestimation of unfamiliar tasks
Not decomposing big tasks into smaller subtasks
Using overly round time units
Substituting a target for an estimate
Sweeping numerous little tasks under the estimation rug
Never creating a real estimate

And some differences to SW projects:

No way to compromise quality for the sake of schedule
Schedule overrun was free
The estimation error didn't really matter, because the project would be done regardless of what the estimate turned out to be

Jeff Atwood analyzes it from a different perspective (http://www.codinghorror.com/blog/archives/000960.html), and concludes that it’s very different (and requires a different skillset and discipline) to build a doghouse or a skyscraper. And the same applies to software (building a toy app is very different from an enterprise application). That is a statement that most programmers would agree but something very hard to explain to a customer (that thinks software is easily changeable without causing structural problems or bugs).

More recently (http://www.codinghorror.com/blog/archives/000981.html) Jeff took another look at estimation and current practices (planning poker sounds great) and reviews the FogBugz tool from Joel Spolsky (from the Joel on Software fame). Apparently the tool has evolved from a bug tracker to a project management tool.

The tool uses a method called evidence based scheduling (a Monte Carlo simulation based on the historical estimates of developers), that predicts when will software ship (not an exact date, but the probability that software will ship on said date).

You can also see the estimation probabilities for the developers:

So how do you know if the tool is any good? Well, I haven’t tried it, but coming from Joel Spolsky's company, it has a good chance of being high quality (attention to detail, good UI). But you can see a video from the new version of the tool FogBuz 6.0 Movie Demo, try it out or read abook about it by Mike Gunderloy: Painless Project Management with FogBugz.

SQL 2008

2007-10-22T14:44:00.001+01:00

Well, it looks like SQL 2008 is just around the corner, but it seems rather quiet for a new release. Maybe it's a small release, or MS isn't using all the marketing power.

Anyway, as the end of support for SQL 2000 is just 2 months after, looks like a comppeling reason to upgrade :)

Among the features for development that I like more are:

Auditing data changes (new T-SQL command AUDIT that allows auditing centralization)
Integration with Linq, Entity Framework, .NET 3.0
MERGE statement (allows update, insert and delete in a single statement)
Some general nice additions (that I won't use much, but look good to have), like date and time data types, file streams (uses the file system to store files pointed by the database), table valued parameters and Spatial Data

In the administration area, there are also some highlights:

Data compression (those that worked with medium databases, know what a hassle it is to manage files with 100 GB in storage with less than 0,5 TB)
Data encryption (columns, database)
Resource Governor (limit resource usage by users or groups)
Hot-plug CPUs!

To end the long post, some links about all the new features in more detail:

SQL Server 2008 Improvement Pillars

What's new in SQL 2008 Part 1
What’s new in SQL 2008 Part 2
What’s New in SQL Server 2008 Part 3

Programming Language Humor

2007-10-20T16:52:00.001+01:00

Great post about programming language (religious) wars with exceptional humor: Language Trolling Post You'll Ever Need To Read (Hopefully). And don't miss the comments, as some of them are of the same quality :)

In the end, programming languages are just tools and the companies behind them, the supporters. Never quite got it why some people take a critic to a programming language or a company personally...

Source Code

2007-10-19T18:50:00.001+01:00

Scott Hanselman has a new column/category (Hanselman - Weekly Source Code) based on the tag to be a better developer started by Justice Gray (Be a Better Developer in 6 Months).

Like Justice Gray committed to reading 1 book per week, Scott Hanselman started reading good source code (as it's one of the best ways to be a better programmer, with which I deeply agree - Reading to Be a Better Developer - The Coding4Fun DevKit).
In one of the recent blogs in this category, I discovered this post: LukeH and a fully LINQified RayTracer. It's a LINQified C# ray tracer (in one Linq Expression that extends over 60 lines!!):

var pixelsQuery =
from y in Enumerable.Range(0, screenHeight)
let recenterY = -(y - (screenHeight / 2.0)) / (2.0 * screenHeight)
select from x in Enumerable.Range(0, screenWidth)
...
select new { X = x, Y = y, Color = traceRay(new TraceRayArgs(ray, scene, 0)) };

He advises against this type of programming (big bang methods/expressions), but it's one heck of an example for Linq.

In the same vein, he links to a list of Programming Guiding Principles by Patrick Cauldwell (This I Believe- The Developer Edition), that start with general principles and has good advices on Unit Tests, Test coverage, Buy vs Build, Dependencies, Contracts (code), Visibility, Authentication, Tracing, Error handling and The definition of done (it means different things to every developer that I know).

Reading all this stuff about code (and the code itself), I started thinking about what are good qualities of Source Code. So, to every beginner (and as a reminder to all senior developers that practices and repetition makes perfection) go [re-]read Code Complete by Steve McConnell...

To me, the first that comes to mind is:

stop and think for five minutes before defining the name of a method, variable;
think early on on the design of the code (method or class), it's complexity, performance and all other important characteristics;
DRY (Don't repeat yourself - don't copy paste code, documentation, use cases, ...)
KISS (Keep it Simple ..., the design, the methods)
...

Functional Programming Examples?

2007-10-04T01:31:00.001+01:00

Recently, Mario Romano and me did a presentation about Language Trends, focusing on Linq and the .NET framework. It's clearly a trend in .NET, and there are some great blogs about FP in .NET:

This one focuses more on Lisp, Haskell and other FP languages, but gives an hand to us imperative programmers:

http://www.defmacro.org/rss/news.xml

My main problem is applying this to practical day to day examples. I've seen many examples of the Fibonnaci, Hanoi Towers and others that are great to get your hands on code, and I can relate to the advantadges of FP (composability, restriction on side effects, conciseness/terseness?).

But I think that to convert more people (or at least look) to FP, there has to be some compelling examples that clearly show the advantadges. Maybe related to Web programming (continuations), or meta-programming (get rid of code generators/copy-paste and with the aid of expression trees and higher order functions, define the basic skeleton of methods - try , validate, call some function passed in, log exception, commit, rollback).

Any good examples/ideas?

.NET Framework Source Code

2007-10-04T00:47:00.001+01:00

It's all over the place, but here it goes again: Microsoft is making available the sources of much of the .NET Libraries with Visual Studio 2008. It will allow to step into framework code while debugging!

http://blogs.msdn.com/brada/archive/2007/10/03/net-framework-source-code-release.aspx

http://blogs.tedneward.com/2007/10/03/Reports+Of+Snowballs+In+Hell.aspx

The Past and the Future

2007-09-19T23:59:00.001+01:00

Just finished reading two great articles about the past, history, and the future.

The first article is from Joel Spolsky (Strategy Letter VI) and how history tends to repeat itself. Starting there, and with some extrapolation from performance considerations in the past (with Excel) and the present (with Ajax and JavaScript), some patterns become apparent (although the actual names could change). But the real kicker is from the comparison between mainframes, CICS and the evolution to microcomputers and Windows to the current state of the Web and it's problems (interoperability between Ajax applications). Very good and thought provoking. It just reminds me of how circular history is: we started with mainframes and dumb terminals, then to client PCs applications (Office, Access, fat clients), then back to big servers and thin clients (with the Web), and with smart clients we are returning the processing power to the client.

The other great post (more of a link to actual MSDN content - Finally some news on PLINQ), reveals the future after .NET 3.5 and MS solutions to the increasing cores with the same speed problem.

The MSDN article shows the primitives for concurrency and how easily concurrency can be "added" to a program. The PTasks article shows it's application to Linq (specially Linq to SQL).

Just checked the MSDN October cover, and it's mainly about concurrency/threads and multiple cores. I'll put it on my short list of "must read"s.

Marcas Technorati: Architecture , History , Programming

Yet another PK debate

2007-09-14T00:53:00.001+01:00

Recently the article in http://weblogs.sqlteam.com/jeffs/archive/2007/08/23/composite_primary_keys.aspx got me thinking again in DB design, especially the PK dilemma (natural or surrogate).

So i googled around to see the current discussion and came up with some links:

Articles defending surrogate keys:
http://www.4guysfromrolla.com/webtech/110806-1.shtml
http://searchsqlserver.techtarget.com/general/0,295582,sid87_gci1156604,00.html
http://r937.com/20020620.html
http://www.bcarter.com/intsurr1.htm

http://www.aspfaq.com/show.asp?id=2504
http://databases.aspfaq.com/database/what-should-i-choose-for-my-primary-key.html
Articles defending natural keys:
http://www.sqlteam.com/article/identity-and-primary-keys

http://www.solidqualitylearning.com/blogs/Dejan/archive/2005/02/24/434.aspx

http://groups.google.com/group/microsoft.public.sqlserver.programming/msg/6d61dbf80d6f0fb6?hl=en&lr=&ie=UTF-8&oe=UTF-8&rnum=14

But returning to the initial article, in it, it's argued that in relation tables the PK should be natural (composite of the PKs of the relations). Naturally, it seems right, and it aligns well with the relational theory. Unfortunately, business clients don't align so well ;)

The initial problem is modeling a CustomerProducts relation table, to which the following two solutions are proposed:

Create table CustomerProducts
(
    CustomerID int references Customers(CustomerID) not null,
    ProductID int references Products(ProductID) not null,
    OrderLimit int not null,
    Primary key (CustomerID, ProductID)
)

Create table CustomerProducts
(
    Customer_ProductID int identity primary key,
    CustomerID int references Customers(CustomerID) not null,
    ProductID int references Products(ProductID) not null,
    OrderLimit int not null
)

The exchange of ideas in the comments was good, and it got me thinking how I do design.

So here are my thoughts (published also in the comments of the initial article).

One of the downsides of the modeling based on natural keys is it's averse to change.

Of course with an identity column, the model, the application must be validated, and possibly changed (including migration) to acommodate the change. But the change doesn't ripple throughout the model as with natural keys.

I honestly can't see the big difference between entity tables and relation tables. I could argue (and many DB purists would) that the PK in an entity table should be a natural key (probably composite).

And it appears you can relate to the problems it introduces:

very difficult to change (I can only imagine what must have been the migration of a DB where ISBN was used as a PK, or using the bank filial as part of the PKs of bank cards - imagine changing the account);
no significant advantadge over the more flexible surrogate model.

To me the same argument holds with relation tables. It eases future changes. The disadvantages are extra complexity, but the end result is more flexible.

I haven't seen a solid example of the supposed advantages (in practice) of the composite key approach. The extra space isn't so much as stated if considering that other indexes include the PK. So the wider is the PK, bigger becomes the index.

I haven't seen any examples or papers that demonstrate in practice that it's more performant to use composite keys (in general).

Regarding the purity or correctness of the model, I can only say, that it's a model... It can be abused if one tries to be to flexible, but sometimes it's better to be flexible and pragmatic than to be rigid and theoretic.

This comment generated a response by Jeff Smith, in which he defends his article conclusions.

Until any of this is refuted, we've clearly established that, for relation tables:
* using a natural PK uses less space, less resources, since if you use an identity, you MUST also introduce a unique constraint on your true key columns
* using a natural PK clarifies your data model and makes it easier to read, understand and work with
* using a natural PK simplifies your SQL statements
* using a natural PK enforces data integrity without the need for an additional unique constraint
* using a natural PK results in true normalization, since your attributes are fully dependent on your primary key and not just a meaningless, random number
* using an identity PK actually complicates things like editing a relation, if you are relying only on the PK, since knowing that you are editing or viewing "CustomerProductID 3922" means nothing, while knowing that you are editing or vewing the relation between "CustomerID 23 and ProductID 4" tells you everything, simplifying your code and your logic.
I'd write more, but again, I feel as though I am just repeating the same points over and over and no one is specifically addressing them ....
Let's talk actuals, here, not theory ... can you please give me a simple example where using an identity as a PK for a relation table, instead of a composite PK, makes future changes "easier"?

But this got me thinking even more in the merits and shortcomings of each design technique, in primary keys and it's relation with natural keys, business keys/rules, etc.

So my response headed for evolutive maintenance, change, flexibility:

I just can't run away from a good discussion. :)

Suppose that the client comes in and wants to introduce some “minor” changes/improvements.
First, he wants to keep the history of changes in the CustomerProducts table. So you create a CustomerProductsHistory table with the timestamp of the operation.
But he also reports that querys to find associations within an OrderLimit range are running very slowly. So you add an index to the OrderLimit.

These “changes” compound and the table becomes larger and wider with more indexes. Child tables are added and because we shouldn't use a surrogate as PK, we keep the parent PK and add some other attribute. Using Ids as the PKs from Customer and Products, the effect is lessened (the size of the PK is just double of a surrogate key, but with natural keys the effect is clear), but eventually your indexes start to suffer because the PK is larger, as well as all child tables PKs and indexes.

Future changes are easier because you get more flexibility in maintenance and with the model.

In maintenance: if a mistake was detected and you MUST correct an association (the CustomerId or ProductId), you don’t have to mess with the PK. Because you didn’t expose the PK of the table to the customer (the surrogate key), you can change the data easily.

In evolutions: if there’s a need to change the CustomerProducts table (you want to turn it into a temporal table, adding a start date and end date), it’s much easier if there’s a surrogate key, especially if the table has child tables.

I think most pragmatic people, having been through similar situations, can see the benefits and flexibility of the surrogate key model. Besides that, there’s also the other advantages mentioned like:

Consistency in the model (easing mapping tools and development),

Comprehension and usage of the model (this table PK is what 4 columns? And joins this other table with what 3 columns?) because the PK is just TableNameId and joins with the FK named TableNameId

The last point I don’t understand. You say CustomerProductID 3922 means nothing. Yet CustomerId 23 has meaning? Why? Only if you look it up in the Customers table… I just hope you don’t show CustomerId 23 in your UI, because then you just exposed the PK, and you can’t ever change it easily.

I’ll concede to the argument that it’s not completely 3NF. But if you go this way, why not make all the tables 5NF mandatory?

Just my 0.02 €

And just to clear Jeff Smith's doubts:

When you create your tables, and you are adding your identity primary keys, do you also create additional unique constraints to all of your tables to ensure you have data integrity?

Got me thinking in the design of the models that I currently adhere, and why I like them:

Yes, of course.
But maybe I wasn't explicit in my last comment (or there's some ambiguity in my terms). To me TableNameId is the PK of the table. It's "private" and not exposed to the business. If the natural key (defined by the business) is just a number, I normally use TableNameNumber (eg: CustomerNumber, ProductNumber). This column is visible to the business, and follows the business rules (no gaps, prefixes, suffixes, etc).
This way the model is very consistent, predictable, flexible and usable. It may have some overhead (not much), and not be completely normalized, but to me it's a good compromise.

Marcas Technorati: SQL , Architecture

2007-06-06T11:10:00.001+01:00

Trends in Programming Languages

It’s been a while since my last post, due to work pressure and little spare time. But in between some code analysis for the maintenance projects I’m involved, the preparation of a Linq presentation and some new blogs I discovered, I’ve been thinking about program language evolution.

It’s not new that imperative languages are integrating functional characteristics. The novelty is the big push by Microsoft, mainly Anders Helsjberg and the .NET team in that direction. Right now, we have XAML, WPF, WCF, and Linq, SilverLight, Entities, VS Orcas just ahead. I still remember the backlash that Java received because it incorporated Garbage Collection, and a Virtual Machine that run bytecode. Nowadays, the main platforms (C# and Java) all include these characteristics, and the new contenders (Ruby, Python) all incorporate it.

Recently I found a great blog that explains the characteristics and advantages of functional programming, in a way that our imperative brains can associate with. In the aptly named article Functional Programming For The Rest of Us, the author presents the following advantages:

Greater Expressiveness, which equates to less lines of code and less bugs;
Meta-programming in the same language, dismissing the need to learn a new languge and displacing many code generators;
Code as Data (in Lisp, it’s possible to treat code as data, by quoting), opening the door to code manipulation, analysis and generation;
Data as Code (in Lisp, a data structure can be “executed”);
Composibility, or the possibility to change/define an application behaviour by the simple composition of functions;
No SIDE EFFECTS
- Easier testing as the result of a function depends only on it’s arguments and produces no side effects. This way a test with the same arguments has always the same behaviour;
- Debugging is simpler as there are no dependencies on Global State;
- Concurrency (with the advent of multi-cores, this is extremely important) scales much better, because locks aren’t necessary because there is no shared state;
- Hot code deployment, without the need to persist and import state (it is passed on the stack in the arguments), it becomes possible to substitute code without stopping the system;
- Machine assisted proofs and Optimizations, without state and side effects, it’s possible to reason about the functions and demonstrate some characteristics about it

From my recollections, this blog and some other blogs on Functional Programming, these are the characteristics attributed to Functional Programming Languages:

Anonymous Functions (lambda calculus)
Type Inference
Higher Order Functions (functions as a first citizen language construct. Can receive and apply functions passed as arguments)
Currying (adaptation to an interface, by fixing one or more arguments)
- int pow(int i, int j);
- square = int pow(int i, 2);
Lazy Evaluation allows:
- New types of optimization (complex calculations that are not used aren't computed)
- Creation of new abstract control structures
- Infinite data structures
Continuations (allows the storing of state and the definition of return point or the function to execute)
Pattern Matching (definition of functions based on types, values and expressions)
Closures (functions that can reference the encompassing environment, typically defined in the body of another function, allowing the association of intermediate set of data with the function: http://en.wikipedia.org/wiki/Closure_%28computer_science%29)

Why does it matter or In what way does it affect me?

Due to the natural evolution of micro-processors, the future doesn't hold increases in speed but in integration of multiple cores in the same die. Because of this, it becomes very important to not limit the concurrency, which is simpler in functional programming (because of no shared state and the expressiveness).

Also the evolution in the .NET space points in the same direction (transition from imperative to declarative), with the eminent arrival of Linq. It's clear in the .NET evolution the trend to a more declarative style with the incorporation of many characteristics of Functional Programming.

.NET evolution

C# 1.0

Function Pointers (delegate)

C# 2.0

Limited Continuations (yield)
Closures (anonymous methods)

C# 3.0

Expressiviness
- Query expressions: from x in …
- Automatic Properties (get; set;)
- Partial Methods (“lightweight” event handling)
- Extension Methods: public static void Print(this string s)
Type Inference for local variables (var)
Lambda Expressions/Anonymous functions (x => x * x)
Anonymous Types (new {c.name, c.Phone})
Code as Data (ExpressionTrees)
- Func<int, bool> nonExprLambda = x => (x & 1) == 0;
  - Initizalizes a delegate: public delegate T Func<T, A>(A a);
  - Expression<Func<int, bool>> exprLambda = x => (x & 1) == 0;
  - Is equivalent to
  - ParameterExpression xParam = Expression.Parameter(typeof(int), "x");
    Expression<Func<int, bool>> exprLambda = Expression.Lambda<Func<int, bool>>(
    Expression.EQ(
    Expression.BitAnd(xParam, Expression.Constant(1)),
    Expression.Constant(0)),
    xParam);

References

Functional Programming explained

http://www.defmacro.org/ramblings/fp.html

http://www.defmacro.org/ramblings/lisp.html

Artigo MSDN C# 3.0

http://msdn.microsoft.com/msdnmag/issues/07/06/CSharp30/default.aspx?loc=en

Expression Trees

http://www.interact-sw.co.uk/iangblog/2005/09/30/expressiontrees

http://spellcoder.com/blogs/bashmohandes/archive/2006/06/12/143.aspx

Closures

http://pluralsight.com/blogs/dbox/archive/2005/04/27/7780.aspx

http://www.claassen.net/geek/blog/2006/08/c-closures.html

Blogs

http://blogs.msdn.com/wesdyer/rss.xml

http://themechanicalbride.blogspot.com/feeds/posts/default

http://wesnerm.blogs.com/net_undocumented/index.rdf

.NET Tools

2007-01-24T23:04:00.002Z

Recently I've been looking at tools that could alleviate the burden on lead programmers reviewing code produced by junior programmers, measuring it's quality and assisting in finding errors. But to me (and others), the current tools are not up to the task. It seems that the tools I could use with C a decade ago (while studying in the university) were more powerful than the current set of tools available for .NET or Java.

Only recently I found some tools that can help in this task (and the most promising one isn't yet available) and are affordable or even free (I don't know if many of the commercial tools are better).

The first tool, known by many programmers, but (in my opinion) used by few (me included) is FxCop by Microsoft. The tool is a code analysis tool, that does some style checking and validates that software conforms to a set of rules, signaling non conformities as defects. In my experience, if the tool is used from the start of a project, it may succeed. On a project already built, it's difficult to use (on a recent project that I work in, it found over 1000 errors. I didn't check if they were false positives or errors that can be dismissed as minor inconsistencies).

The next one, is nDepend . It allows the identification of problem areas in the overall architecture of a program (solution), identifying big/complex methods, non-cohesive classes and helps in understanding the program at an higher level. It allows the definition of rules and alerts with a SQL like language. The site has some videos showing its usage.
I haven't tried it yet on a complex project, but if it easily allows to flag calls from one layer to the wrong layer (UI to data, or business to UI) it seems promising. Another feature that I'm eager to try is the generation of metrics (# lines, # methods, # classes, cyclomatic complexity, % comments).

A little on the side are the "projects" like AspectC# and other languages that allow the separation of concerns and the "injection" of code, that could be used and parametrized to catch some of these errors. But it's a hard task just to get to a point that provides an interesting return on the investment made (in setup and parameterization time).

The last tool that I read about (this one isn't out yet), is nStatic. Now if this tool does half of what's shown and talked in Wesner Moise's blog, it will be a life saver. Apparently it interprets the program not as lines of code in a text file (like most IDEs do today), but in an functional way in internal memory (maybe those ASTs - syntax trees that we learned in compiler classes). With this representation, and using an AI workbench, it computes the paths in the program that generate errors (and the assumptions for those errors).

This reminds me of the first time I heard of an internal representation that wasn't the text source file . I think it was Visual Age for Java, that stored the tree representing the program in a DB . This approach (VA failed maybe for being ahead of it's time, and some shortcomings in speed and in error tolerance) has an enormous potential as shown by the current projects by JetBrains with Metaprogramming System and Intentional Software (I haven't tried either and I don't know if Intentional Software has some tool available for test). Just imagine the refactorings possible this way. And if it's possible to plug-in in to this representation, it would be much easier to find coupling mistakes (between layers), copy-pasted code, etc..

But back to nStatic, one of the features mentioned was finding the possible exceptions for a method and it's assumptions (for the exception). That will assist in finding many subtle errors. I'm just curious to try the tool (I'm thinking in applying for the beta), and have some questions/requests:

will it allow to detect similar or duplicate code (I found Simian, but I haven't tried it on my complex project that I know has much similar code, to figure out if the match is by line or by abstract representation of the code)?
will it be extensible, or allow access to the internal representation of the code, or provide an API?
can the scan focus on one particular method (and be completely thorough with that method)?
will it allow a fine grained control over loops (limit the # of times in a particular loop by annotation or coment) and other constructs?
will it allow exporting graphs, assumptions to a XML/text format?
it could be used to track dependencies(?) between assemblies/classes?
could it generate some metrics, and integrate some features from other tools, meaning that it would be the only code analysis tool needed :)

2006-11-22T11:21:00.001Z

.NET & Office Interop

Everyone knows what a pain it is to configure it correctly. Numerous articles try to lay out the possible solutions, workarounds,

Here's what worked for me (it it ever gets there), on a Win XP Pro, Visual Studio 2003, ASP.NET 1.1 configuration:

Install Office Interop DLLs
Add the PIA DLLs to the VS Solution
Add ASP.NET to the Administrators group?
or
Create a user with Admininstrator permissions (at least Debugger group or Power User)?
Impersonate the user in Web.Config
Give permissions to the directories to be read and written by the application (in my case read the template directory and write in the directory where the word document is created)
Give permissions in the the registry to the user to the Software Office key (http://support.microsoft.com/kb/236592)
Modify COM permissions, allowing the user to launch and activate components (http://support.microsoft.com/kb/288366/)
Modify Microsoft Word Component in COM+ so the user impersonates it (http://forums.asp.net/1091578/ShowPost.aspx)
Reinstall ASP.NET (aspnet_regiis -i, http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=363330&SiteID=1)
Reboot the machine
Cross your fingers

Considerations:

Don't Use It http://support.microsoft.com/?id=257757; http://support.microsoft.com/kb/317316/
Safe Mode http://support.microsoft.com/kb/827706

Errors:

PIA not found http://support.microsoft.com/kb/823996
Rejected Safe Mode Action:

Terminal Services & Outlook http://support.microsoft.com/?kbid=319517
COM+ configuration http://support.microsoft.com/kb/288366/; http://support.microsoft.com/kb/288367/ (verify registry IDs!); http://support.microsoft.com/kb/288368/
Start Word/Excel in "safe mode", "automation" running as the specified user

Clean up: Help --> About ... --> Disabled Items --> Enable (http://support.microsoft.com/kb/813589; http://support.microsoft.com/kb/827706)

Server Execution Failed

2006-11-22T10:59:00.001Z

Running Visual Studio 2003 in Vista

Apparently, the best option to run Visual Studio 2003 inside Vista is by using Virtual PC. And the new version of Virtual PC (2007, still in Beta), is the best choice in terms of speed:

http://codebetter.com/blogs/peter.van.ooijen/archive/2006/11/10/Vista_2C00_-Visual-studio-2003_2C00_-Virtual-PC-2007-_2800_and-the-right-hardware_2900_-.aspx

2006-11-21T16:29:00.001Z

A potentially dangerous Request.Form value was detected

Some articles related to this error message, and useful for further investigation:

http://blogs.madtechnology.net/blogs/chris/archive/2004/03/20/366.aspx

http://www.asp.net/Forums/ShowPost.aspx?tabindex=1&PostID=388929
http://www.asp.net/faq/RequestValidation.aspx
http://www.aspcode.net/articles/l_en-US/t_default/ASP.NET/A-potentially-dangerous-Request.Form-value-was-detected-from-the-client_article_147.aspx
http://www.dotnetjunkies.com/HowTo/3ABCD244-CC7C-4CED-B64E-BCF05191CDAB.dcik

http://www.mikeschinkel.com/blog/APotentiallyDangerousRequestFormValueWasDetected.aspx
http://kbase.vircom.com/Novo/default.asp?id=1041&Lang=1
http://forums.microsoft.com/MSDN-BR/ShowPost.aspx?PostID=743559&SiteID=21
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=544937&SiteID=1
http://support.microsoft.com/kb/821343/
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag2/html/PAGHT000003.asp?_r=1
http://msdn.microsoft.com/msdnmag/issues/04/12/WebQA/

2006-11-21T13:02:00.001Z

Debugging JavaScript

Good article on how to debug JavaScript in Visual Studio, including all the options that need to be activated both in IE and VS.

http://weblogs.asp.net/pglavich/archive/2006/11/02/Debugging-Javascript-on-a-Live-site-_2D00_-Fun-and-Games.aspx

2006-11-20T15:32:00.000Z

Does Microsoft Certification Matter?

A while ago, I found the following discussion about certification in http://www.theserverside.net/news/thread.tss?thread_id=32963.

In my opinion, Certification does matter. Of course it doesn't replace real world experience in projects, and it doesn't assure the quality of the certified people. But it does validate some basic knowledge, and gives some new insights in some aspects that aren't often used in a day to day basis.

Another aspect is the recognition/valuation of the certification: as more people get certified and the certification spreads, the less value the certification has (even because some tricks used in the exams are widely known)

Microsoft acknowledges this situation, and launched a new generation of certifications (http://www.microsoft.com/learning/mcp/newgen/), topped by the MCA for the top Architects.

Update: A new article (http://www.infoworld.com/article/06/11/17/47OPanalysts_1.html), questions the value of certifications, as average pay for IT certifications dropped 2% between 1/Jul and 1/Oct.

2006-11-16T11:07:00.000Z

Enterprise Platforms

Thinking about Enterprise Platforms, I realized what a great platform SQL is. Despite all the hype around other servers, more aligned with the SOA and Web 2.0 trends, SQL Server has become a true Enterprise Platform.

It's a great data storage server, and has been constantly gaining ground on the so called big DB Servers (Oracle and IBM DB2), with features like Online Index Rebuild, Table Partitioning, Mirroring, and Snapshot Isolation. On the performance side, it swept the top TPC performance benchmarks.

But that's the beginning of the story. With SQL 2005, it gained:

Message Queuing with Service Broker;
bundled Reporting Services (not only the generation but also the delivery);
Notification Services (and a new mail API);
CLR integration (v2.0), and if the .NET Framework 3.0 is included, we even get Communications, Workflow and Presentation Foundation for free;
Analysis Services for BI;
A true Enterprise ETL tool/environment with Integration Services;

And it’s not just targeting the enterprise, with the Express edition available for free (http://msdn.microsoft.com/vstudio/express/sql/download/), everyone can have a top performing Database server (limited to 4 GB DB, 1 GB of RAM and 1 CPU) with Management Studio Express, Reporting Services and Full-Text Search.

2006-11-15T14:13:00.000Z

ASP.NET Web Application Projects
(For internal use&search)

http://msdn2.microsoft.com/en-us/asp.net/aa336618.aspx (HomePage)
http://msdn.microsoft.com/vstudio/default.aspx?pull=/library/en-us/dnvs05/html/WAP.asp (Introduction)
http://webproject.scottgu.com/ (Scott Guthrie)
http://webproject.scottgu.com/CSharp/Default.aspx (Tutorial C#)
http://webproject.scottgu.com/CSharp/Migration2/Migration2.aspx
(Migration)
http://weblogs.asp.net/scottgu/archive/2006/05/08/445742.aspx
(Release)
http://forums.asp.net/thread/1279716.aspx
(FAQ and Known Issues)

2006-11-15T12:34:00.000Z

SQL Unused Space problem with heaps
SQL overallocates space in heap tables, and doesn't reuses the space.
http://support.microsoft.com/default.aspx/kb/924947

2006-11-15T11:44:00.000Z

Operação
(for internal use&search)

2006-11-15T11:41:00.000Z

SQL Misc articles
(for internal use&search)

2006-11-15T11:25:00.000Z

.NET
(for internal use&search)

.NET Performance Counters
http://www.extremeexperts.com/Net/Articles/NetCounters.aspx
http://www.aspheute.com/english/20000809.asp
http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/97d0872a-96b3-41e4-8997-b174a9dbb87f.mspx?mfr=true
http://msdn2.microsoft.com/en-us/library/fxk122b4(VS.80).aspx
Debugging
http://blogs.msdn.com/tess/archive/2006/10/16/net-hang-debugging-walkthrough.aspx (Tess Blog)
http://dotnetdebug.blogspot.com/ (Advanced .NET Debugging)
http://blogs.msdn.com/vancem/ (Vance Morrison)
http://blogs.msdn.com/ricom (Rico Mariani Performance)

2006-11-14T14:57:00.000Z

Generic SQL Articles
(For internal use&search)

URL
Notes

SQL Server Database Coding Conventions, Best Practices, and Programming Guidelines
SQL Server Database Coding Conventions, Best Practices, and Programming Guidelines
database object naming conventions
database object naming conventions
http://www.microsoft.com/sql/default.mspx
SQL Server Central
Apresenta bastantes artigos relativos a SQL 2000 (T-SQL, Administração, Optimização, Reporting Services, DTS) e também alguns de SQL 2005.

Contém foruns de discussão bastante participados e permite testar os conhecimentos de SQL através de perguntas diárias enviadas pela comunidade.

É ainda possível subscrever a mailing list diária, com os artigos, discussões e pergunta desse dia.
SQL Server Performance
Site dedicado a questões de perfomance do SQL Server.

Contém fóruns de discussão bastante activos e mailing list bi-mensal.
Generate Ad-hoc Reports with Microsoft Reporting Services 2005
Generate Ad-hoc Reports with Microsoft Reporting Services 2005
Among the many new features in Microsoft Reporting Services 2005 is one that can truly help reduce the workload for developers—the ability to give end users the power to generate their own ad-hoc reports.
Best Practices Analyzer Tool for Microsoft SQL Server 2000 1.0
Microsoft SQL Server Best Practices Analyzer é uma ferramenta de gestão de base de dados que permite verificar a implementação de "Best Practices" comuns nos servidores SQL.
Worst Practices Part I - Hungarian notation for columns
Worst Practices Part II - Objects Not Owned by DBO
Worst Practices Part III - Not Using Primary Keys and Clustered Indexes
Worst Practices Part IV - Making Databases Case Sensitive (Or Anything Else)
Worst Practice - Triggering External Events
Best Practices
BEST PRACTICE 1 - Use Mnemonic Table Aliases
BEST PRACTICE 2 - Use ANSI JOIN Syntax
Dealing with Anti-Virus Programs
SQL Server 2000 Best Practices - Extreme Experts
Brian Walker's T-SQL Coding Standards
Exame 70-441 Beta (Developing Solutions SQL 2005)
Opinião de Steve Jones (SQL Server Central) sobre o exame 70-441 Beta (Developing Solutions on SQL Server 2005)
Exemplo de ROLLUP, CUBE e COMPUTE
Exemplo prático da utilização de ROLLUPs, CUBEs e COMPUTEs em querys de SQL.
Apresentações TechDays 2005
Já estão disponíveis as apresentações do evento TechDays 2005.
Row & Cell Level Security in SQL 2005
Implementing Row- and Cell-Level Security in Classified Databases Using SQL Server 2005
IntelliSense para SQL Server
Ferramenta que fornece funções de IntelliSense no SQL Server (Query Analyzer, Enterprise Manager).
SQL Server Security: Dealing with Anti-Virus Programs
Bitmap
Recommendations for how to handle SQL Server and anti-virus agents in conjunction
Guidelines for choosing antivirus software to run on the computers that are running SQL Server
Bitmap
This article provides general guidelines to help you decide which type of antivirus software to run on the computers that are running SQL Server in your environment.
Finding Gaps in a Sequential Number Sequence
Bitmap
Finding Gaps in a Sequential Number Sequence
WMI Tasks for Scripts and Applications
Bitmap
Describes various computer and network administration tasks and provide links to the WMI class or classes used to perform those tasks
Fixing broken logins and transferring passwords
Bitmap
When transferring a database to a new server using backup/restore or detaching and re-attaching database files, the link between server logins and database users almost invariably gets broken.
Migrating Logins To Another Server
Bitmap
There are a number of different ways to migrate SQL Server logins. You can manually re-enter all the existing login on the new server. You can used DTS to transfer logins. There are probably a number of other ways to transfer logins. This article will discuss one of those other ways to streamline the migration of SQL Server logins by using the stored procedure (SP) sp_help_revlogin.
Monitoring Long Running Jobs
Bitmap
Checks if your job engine is running and how to restart it.
Guia para comentários em BDs
Standards para comentários de código SQL
XP_EXECRESULTSET
Utilização do Stored Procedure não documentado "XP_EXECRESULTSET" do SQL Server 2000.
Updating a Sequential Record Number field
Actualização de um número sequencial numa tabela
Sequential Numbering/Counting of Records with SQL Server
Várias soluções para a criação de numeração/contagem de registos em SQL Server, desde cursores, tabelas temporárias a self joins
Writing Maintainable Code
Princípios básicos para escrever código SQL fácil de manter e compreender.
Administering a Development Environment
Strategies for administering a successful development environment that will cut development time and ensure properly functioning and performing code.
SQL Server 2005 Installation Adventures
Instalação automática do SQL Server 2005
Monitoring SQL Servers Availability
Monitorização e auditoria dos serviços SQL Server e SQL Agent
Single Point of Administration
good development practices and reinforce some good coding practices
The Joy of Numbers
What exactly is a numbers table and more importantly, what can it do for you?
Performance Monitoring by Internal Fragmentation Measurement
According to a research conducted by IDC, a leading consulting firm in the computers industry, the performance degradation caused by disk fragmentation alone, shortens the life of the computer/server by 30% on average. This translates into a direct increase of 30% in the yearly budget for hardware upgrades. It is possible to minimize this significant sum utilizing traditional Defrag and Reorganization utilities. The major setback remains the timing and scheduling of this extremely long and cumbersome process. We propose that this is a management task that can, and should, be undertaken.
Questões de entrevistas SQL
Read Only Tables
The Curse and Blessings of Dynamic SQL
Discussão das utilizações de SQL dinâmico, mostrando boas e más práticas
Monitoring Drive and Database Free Space
Scripts para monitorização do espaço livre em disco e nas BDs
Using the OUTPUT Clause to Capture Identity Values on Multi-Row Inserts
Controlar tempo de execução dos Jobs SQL
Exemplo para controlar o tempo de execução de Jobs SQL
Utilização de CTEs SQL 2005
Exemplos de utilização de Common Table Expressions no SQL Server 2005
Windows Utilities for the SQL Server DBA
Utilitários Windows (Resource Kit) úteis para tarefas de Administração de Bases de Dados (FOR, FORFILES,Resource Kit, WMIC, Command-Line Scripts)
Analysis of SQL Server Performance Data
Recolha e análise de performance counters em SQL
Indexes and Fragmentation in SQL Server 2000 Part 1
Introdução à análise de fragmentação de índices em SQL 2000 (DBCC SHOWCONTIG; UPDATEUSAGE)
Index Creation Guidelines
Dicas para a criação de índices (onde e quando utilizar/não utilizar)
Fine Tuning your Database Design in SQL 2005
Optimização de BDs SQL 2005 (índices, DTA, XML Statistics e DMVs)
Reporting a menu structure using a recursive UDF
Função SQL recursiva para construção de menu hierárquico
RESTORE VERIFYONLY (Transact-SQL)
Testa a validade de um backup (se é possível restaurar, se está completo, se é legível) sem fazer o RESTORE (SQL 2005)
Stress Testing SQL Server
Dicas para fazer testes de carga em SQL Server (ambientes, counters e alertas a monitorizar)
A Guide to Application Memory Tuning
Guia para optimização da memória em SQL (switches /3GB, /PAE e opção AWE)
Simple Database Monitoring in a Windows Environment
Construção de uma pequena ferramenta para monitorizar a disponibilidade de um servidor SQL (conectividade)
How to find that job is running?
T-SQL para verificar se um determinado job está em execução ou não.
SQL Server 2005 Best Practices
Dicas e questões a rever num upgrade para SQL 2005
Deploying Scripts To Multiple Instances
Instalação de Scripts para múltiplas instâncias/servidores
File Watcher Task
SQL SSIS - Tarefa para detecção de ficheiros e alterações
Manipulating Text Columns
Manipulação de Colunas do tipo TEXT (READTEXT, WRITETEXT, UPDATETEXT)
Exploring SQL Server 2000 Configuration Properties
Descrição das propriedades de configuração do SQL 2000 (Enterprise Manager’s SQL Server Configuration Properties)
Stored Procedure Naming Conventions
Convenções de nomenclatura para Stored Procedures
Ownership Chaining
Ownership Chaining
Logins, Users, and Roles - Getting Started
Formas de gestão de Logins&Roles - Enterprise Manager, T-SQL e SQL-DMO
XML Argument Protocols for SQL 2005 Stored Procedures
Utilização de argumentos do tipo XML em Stored procedures SQL 2005
Query Analyzer Extended
Definição dos formatos/templates utilizados pelo Query Analyzer para gerar scripts
"EXCEPTION_ACCESS_VIOLATION" error message when you use a RIGHT OUTER JOIN
Problemas na utilização de RIGHT OUTER JOINS em SQL 2000 (até SP4)
No Outlook for Alerts
Envio de e-mails (CDO) em SQL sem utilização de Outlook
Conducting a SQL Server Operational Audit
Auditoria Operacional a Servidores SQL
Use SQL-DMO and Excel to Quickly Create Reports for Auditors
Utilização de SQL-DMO e Excel para construir relatórios
Troubleshooting Cannot Generate SSPI Context Errors
Motivos de aparecimento de erros "Cannot Generate SSPI Context"
Scripted Server Snapshot
Script para obtenção de snapshots da actividade da BD (sp_who2) para Excel
The Enterprise Library for .NET Framework 2.0
Introdução à utilização da Ent Lib para .NET 2.0 (Stored Procedures com valores opcionais e retorno)
How do I… install SQL Server 2005 Express Edition and SQL Server Management Studio Express?
Instalação de SQL 2005 Express Edition (Full-Text, Reporting Services) e Management Studio Express.
SOA, Multi-Tier Architectures and Logic in the Database
Artigo controverso sobre utilização da BD para armazenar lógica de negócio.
The Pitfall of "Not Equal To"
Problemas com a utilização de NOT EQUAL
Calling COM From T-SQL
Invocação de Objectos COM a partir de T-SQL
Finding the Next Business Day Recursively
Função SQL para encontrar o próximo dia útil (utiliza módulo do dia).
The New ETL Paradigm
Diferenças entre o paradigma ETL do SSIS e o antigo DTS.
SQL 2005 SP2 New/Improved features
In Microsoft SQL Server 2005 Service Pack 2 (SP2), the following components have new or improved features.

Analysis Services; Database Engine; Integration Services; Reporting Services; Shared Tools
Readme for Microsoft SQL Server 2005 Service Pack 2
Readme for Microsoft SQL Server 2005 Service Pack 2
Monitoring file sizes in SQL Part I
Monitoring the SQL Server mdb and ldb file size with a threshold to produce a visual alert every time you boot you computer (VBScript)
Monitoring file sizes in SQL Part II
Monitoring the SQL Folder size with a threshold to produce a visual alert every time you boot you computer (VBScript)
Monitoring file sizes in SQL Part III
Export of the mdb and ldb file size to excel (VBScript)
Monitoring file sizes in SQL Part IV
Extracting the size information on your SQL server drives and plotting it to a excel spreadsheet (VBScript).
Performance Monitoring - Basic Counters
Process Object, % Processor Time, System Object, Processor Queue Length, Memory Object, Pages/sec, Available MBytes, PhysicalDisk Object, Avg. Disk Queue Length, % Idle Time, Network Interface Object, Bytes Total/Sec, SQL Server Access, Methods Object, Full Scans/Sec, SQL Server Database Object, Transactions/Sec, SQL Server Buffer Manager Object, Buffer Cache Hit Ratio, SQL Server General Statistics Object, User Connections, SQL Server Locks Object, Average Wait Time
Naming Conventions
Naming conventions for DBs, Backups, Users, Logins, Tables, Columns, Indexes, Constraints, Views, User-Defined Functions, Triggers
http://www.devx.com/codemag/Article/32160
Dealing with Database Concurrency Conflicts in the Real World
Overcoming quotes in dynamic queries
Troubleshooting complex queries with single quotes, using a simple technique
E-Mail Functionality in SQL Server 2005
Database Mail in SQL 2005 and migration from SQL Mail 2000
Unit Testing SQL SPs
Unit Testing de SPs utilizando TSQLUnit e CodeSmith.
Unit Testing SQL with VSTE DB
Unit Testing utilizando o Visual Studio Team Edition for Database Developers
Cursor advantages
Quando e como considerar cursores em desenvolvimento SQL?
Get all user tables with size
Script para obter o tamanho de todas as tabelas de utilizador de uma BD
Understanding SQL Server's DBCC SHOWCONTIG
Discussão da utilização do comando DBCC SHOWCONTIG e do funcionamento interno do SQL Server
Cursor disadvantages
Desvantagens da utilização de cursores em SQL
Moving Your Users with Their Databases
This article deals with moving both the database AND the affected users, along with all their information for that one particular database that you may be interested in moving to a new server.
Moving a Database to a New Server
Moving a Database to a New Server

2006-11-14T14:54:00.000Z

SQL Performance articles
(For internal use&search)

Tuning
http://www.microsoft.com/technet/prodtechnol/sql/2005/physdbstor.mspx (Physical design)
http://www.sqlmag.com/Article/ArticleID/46377/sql_server_46377.html (Performance Tuning)
http://www.sql-server-performance.com/ac_filegroup_performance.asp (Optimizing Performance Files&Filegroups)
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnsql7/html/msdn_sql7perftune.asp (SQL 7 PerfTune)
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/optimsql/odp_tun_1_536v.asp (Query tuning)
http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/rdbmspft.mspx (SQL 2000 PerfTune DW)
http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/tranrepl.mspx (Transact Replic PerfTune)
http://www.microsoft.com/technet/prodtechnol/sql/bestpractice/default.mspx (SQL Server Best Practices)
http://www.microsoft.com/technet/prodtechnol/sql/bestpractice/tempdb_capacity_planning_index.mspx (TEMPDB Cap)
http://www.microsoft.com/technet/prodtechnol/sql/bestpractice/loading_bulk_data_partitioned_table.mspx (Load Bulk)
http://www.microsoft.com/technet/prodtechnol/sql/bestpractice/dbcc_showcontig_improvements.mspx (DBCC SHOWCONTIG improv)
http://www.microsoft.com/technet/prodtechnol/sql/2005/technologies/dbm_best_pract.mspx (DB Mirroring Best Pract)
http://www.microsoft.com/technet/prodtechnol/sql/2005/tsprfprb.mspx (Troubleshotting Performance SQL 2005)
http://www.microsoft.com/technet/scriptcenter/scripts/sql/sql2005/default.mspx?mfr=true (Script repository)
http://www.microsoft.com/technet/prodtechnol/sql/2005/bestpractice/storage-top-10.mspx (Storage 10 Best Pract)
http://www.microsoft.com/technet/prodtechnol/sql/2005/bestpractice/oltp-performance-issues.mspx (Top SQL 2005 Perf OLTP)
http://www.sqlservercentral.com/columnists/jsack/2653.asp (Tuning SQL 2005)
http://www.simple-talk.com/sql/sql-server-2005/fine-tuning-your-database-design-in-sql-2005/ (Tuning SQL 2005)
http://www.sqlservercentral.com/columnists/pmu/2644.asp (SQL Application memory tuning)
Counters
http://www.sqlservercentral.com/columnists/kjambu/2677.asp (Performance Counters)
Índices e Planos de Execução
http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/ss2kidbp.mspx
http://searchsqlserver.techtarget.com/tip/1,289483,sid87_gci1112905_tax301327,00.html?adg=301324&bucket=ETA
http://www.sql-server-performance.com/sql_server_maintenance.asp
http://www.sqlmag.com/Article/ArticleID/41844/sql_server_41844.html
http://www.developersdex.com/sql/message.asp?p=580&r=5117242 (Script)
http://education.sqlfarms.com/ShowPost.aspx?PostID=55
http://vyaskn.tripod.com/sql_server_administration_best_practices.htm

2006-11-14T01:23:00.000Z

TechEd 2006 and 2005 Comparison

In retrospect, and evaluating TechEd 2006 Barcelona, I must say that it was a great event with great speakers, with the high level of quality expected from Microsoft.
But, comparing to Amesterdam 2005, there are some details that were worse:

The corridors and stairs were very narrow (at least for the amount of people present);
The CommNet had very few computers available;
No VOIP phones (that was a major loss!);
Some sessions filled up very easily while others were less than half (the full sessions should all have a repetition);
The week pass from Amesterdam was downgraded to a 10 trips ticket (to visit the city, or even, have dinner, one would have to count the tickets very closely);
There was no major announcement or new (much of the content was based on the TechEd USA);

2006-11-13T22:51:00.000Z

TechEd 2006 End

TechEd 2006 is officially over (from last friday, that is), and I'm back to work. Just a recap of the final days:

Day Four, the final day, was a bit weak. It's understandable that, with the speakers tight schedules, most are already on their way home. So I was looking forward to Ingo Rammer's presentation about Optimizing Performance and Scalability as the high point of the day. But, the presentation was very much like last's year, so I switched to SharePoint Server 2007 Search. The latter suffered from the toy example syndrome. It was easy to follow, but very limited, and looked like the most part was getting around some limitations of the API or model.

Next, I attended MSF v4.0 Core by Rafal Lucawiecki. Very good and informative. With the upgrade to MSF 4, and the additions of MSF for agile and CMMI in VSTS, MSF has become more than a Framework. The Core, is the Discipline, from which the families derive (Application Development, Infrastructure Deployment and possibly Consulting and Operations Management - MOF in the future). In it, there's only high level models, with the Team Model revolving around Advocacyand Architecture becoming a phase in the development life cycle.
The Process Model, introduces Governance and Enactement, catering for various types of projects event small teams, and allowing the start of each phase before the end of the previous one. At Core Discipline, the Project Management Discipline recommended is the PMI BOK. At methodology level, there's support for Agile SD and CMMI PI activities and work streams.
To Rafal, the best invention in Software Engineering in the last 20 years, was the "Daily Build", because with it, there is no integration (rephrasing, it's done on a daily basis, so it's easier, cheaper, more predictable, and garantees a working product very early on).

Then there was a session about SQL 2005 Cache Sync/Query notification in Depth by Bob Beauchmin. To sum it up in one sentence:
"The feature is only useful in very specific scenarios, the drop-down combo box with few items rarely updated."
Outside that, the feature doesn't scale, or can't be used. The bulk of the session was spent analyzing the limitations imposed on queries, the inner workings of the feature and its integration with ASP.NET Cache. Not very useful.

The final session of TechEd, and because there were two portuguese fellows in the draw, was Iron Architect Finals. This was a contest to define the architecture to build up communities around TechEd, linking to the feedback system. Out of the 5 chosen proposals, 2 were from portuguese people. :)
The jury was composed by Andrew Cheeseman, Ron Jacobs, Beat Schwegler,
Jürgen Pfeifer (MCA), and clearly favoured "Jota"s ideas. From a presentation point of view, the prize would go to the belgian as he was all around at ease. From a pleasing the jury point of view the prize would go the finn/dannish "alpha geek" guy that presented last and tried to score points by pleasing the jury. But judging only the ideas (tag clouds to categorize sessions and attendees), forgetting the nervous presentation, "Jota" was better. And so, congratulations to "Jota", and I expect that he may become the first portuguese MCA.
Day three, was very varied. To start off, I watched an indepth look at Linq for SQL (that's the current name of DLinq), and it was astonishing. Very easy and very powerful, and overall very well thought out solution. Even the corner cases of ORM tools were covered: identity mapping, lazy loading, optimistic concurrency, paging, batch commands, you name it.

Then another cool session about Iron Python, and it's fit within .NET. The demos were very impressive (animating Merlin, converting from Ink to text, synthetizing speech, using WPF, with every demo below 5 lines), and there is now support for Iron Python in ASP.NET (not MS PSS support, but community support) and the tools are being extended to support Python (Intelli sense wasn't prime time yet) and other scripting languages.

After that, I attended a real world usage example of WPF in the Zurich Airport Monitoring system. Very impressive the animation capabilities of WPF. In less than 15 minutes there was a button animated to follow a path (later replaced by an airplane) in the map that represents the airport. From there on, it was just a matter of refinement and linking the actions on event handlers with code from the Expression Designer, to get a fully animated monitoring system (with tracks sensing the number of planes and glowing in red), with zoom, panning, tags on airplanes.

The presentation of Team System adoption best practices focused on advanced customizations and extensibility possibilities, and that didn't please some of the crowd in the room that were expecting the basics (how to organize SS, work items, ...). But the customizations were very useful (capture the screen and pre-fill a bug with that attachment, to save testers some keystrokes; changing the type of the description field to HTML; searching all pending check-ins from all users; and because of the annoyance from some people in the audience, information about licensing conditions, the reports generated, the off-line scenario).

Last, Friz Onion compaired the differences between the Web site model and Web Application Project, the strengths and weaknesses of each, with a clear win for Web Application Projects. Very detailed knowledge of the inner workings of ASP.NET, the compilation mechanism and the class model relationships.

2006-11-09T06:54:00.000Z

TechEd 2006 Start

The first two days of TechEd 2006 Barcelona are over.

In Day two, I focused more on .NET, and enjoyed very much the presentation on C# 3.0 (or is it 3.5?) and the innovations brought by Linq, Lambda expressions, Extension methods. To me, Linq raises the abstraction level of the language, giving first class support for queries in .NET. With it, it's possible to very easily query a DB (in a strongly typed fashion), join with a XML source or any other that implements the interfaces (Vista search is expected in the future).

Also very good were the sessions about debugging outside VS 2005 and the tools available(Ingo Rammer is a very good presenter and his demos are really on topic, not some toy example, that doesn't hold a connection with reality).

Not so good were the presentations about Using VS Team System - nothing new, just a rehash of last year's examples, and very "toyish" and about Windows Vista - Catherine Heller, knows the subject, but the notion that to get the Vista native feel in the Open File Dialog one has to tinker with COM+ and it's inner stuff, makes me want to run away from it. Another annoyance comes with the security increase of Vista. From now on, applications can't programmatically determine if a user is an administrator (the security token is stripped from that info). And to run with ellevated privilleges, guess what, it's necessary to invoke a process and the user has to confirm it. I can see, what's coming, every user pressing the yes button every time that annoyance pops up. Even debugging with VS2005, has to be run with ellevated privileges.

When does MS gets the balance between security and usability right? Why doesn't Vista Home ship with a VM to surf the Net and read mail? And to try out the installation of applications, what best than installing in a VM, and after evaluating or testing, throw it away (if it's a virus or trojan), or installing in the main OS?
From the first day, the highlights for me were the sessions by Kimberly Tripp. She really knows what she talks about, and makes it sound easy, explaining stuff in non technical ways. On the downside was the session on VSTO SE (Second edition). Basically it was explained that the missing stuff in VSTO 2005 have made it in the 2nd edition, and some stuff that has to be done to get around Word and Office quirks (mainly about closing windows events).

The keynote, was divided in two parts. The first part, was about education, and was in a partenalist tone, ending up with a presentation of 10 year old girl from India that has been certified in C#. The second part was more in the spirit of TechEd, and had some cool demos with SharePoint, Excel, Workflow, Expression Designer and Visual Studio.

2006-11-06T01:21:00.001Z

HR and Interviewing

The IT job market is on the rise again. It’s been ramping up for some time, but the clear sign, is the “movement” on the companies, people leaving and new people getting in, and the blogoshpere with articles about interviews.
Employers can no longer be laxist about the employees, or they will leave. And everyone wants to find the best people.

To Joel Spolsky, from the famous Joel on Software, hiring is about getting the best people. Period. As the gap between excellent and average programmers is 10x, it pays up to find the best. The best people, are not only very smart but people that gets things done (pragmatic). And that’s what his company tries to do. They interview by phone, and the smartest are flown to New York, with everything paid, for a personal interview.
(http://www.joelonsoftware.com/articles/GuerrillaInterviewing3.html)

To Mini (from Mini-Microsoft fame), the solution to overcome the peak in the competition for talents between employers, that leads to hiring of not so good programmers, would be to create a Microsoft Academy, to teach the main technologies used inside MS. That way, people with potential and motivated would graduate and be ready to integrate the team that best fits them, while other people would wash-out (the course would be very hard and intense).
Even to switch groups inside Microsoft, people could get the training and skills from the courses teached, giving microsofties a real chance to control their careers, and Microsoft a chance to internally reorganize, getting people in their correct positions and motivated.
(http://minimsft.blogspot.com/2006/11/microsoft-academy.html)

On the employers side, HR is getting even harder.
As the demand goes up, so goes the salary. And with big payments, everyone tries their hardest to make the cut. Embelishing the resumé, or even outright lying! To counter that, employers are paying detectives to investigate references, neighbours and past employers to check CVs, references and even neighbours.
(http://www.simple-talk.com/opinion/opinion-pieces/the-cv-detectives/)

To end up, here are some links related to interviewing for DBA, questions:
http://www.sqlservercentral.com/columnists/sjones/2686.asp (interview questions)
http://www.geekinterview.com/articles/sql-interview-questions-with-answers.html

2006-11-06T01:07:00.000Z

TechEd 2006 Barcelona Session Schedule

It's the final countdowns to TechEd 2006 in Barcelona. Right now, I'm packing up, because tomorrow I'm flying to Barcelona. The conferences only start on Tuesday, but from my company we're going the day before.

My tentative schedule, is somewhat like this (arranged by timeslot, and inside timeslot sorted by preference):

Tue Nov 7 11:30 - 12:45
DEV230 Developing Rich Web Applications with ASP.NET AJAX
DEV201 Introduction to the .NET Framework 3.0

Tue Nov 7 14:15 - 15:30
SQL408 SQL Server 2005: Advanced Indexing Strategies
ARC207 Introduction to Agile Methodologies and Concepts
DEV225 Windows Presentation Foundation (WPF): Introduction

Tue Nov 7 16:00 - 17:15
SQL409 SQL Server 2005: Advanced Indexing Strategies : Part 2
DEV347 ASP.NET AJAX Tips and Tricks
DEV223 Visual Studio: The .NET Language Integrated Query (LINQ) Framework Overview
ARC305 Connected Systems - Part 2: Communication, Flow, Rules and Logic

Tue Nov 7 17:45 - 19:00
OFF303 Visual Studio Tools for Office (VSTO) - Cypress and ORCAS
ARCWD03 Understanding Service Modeling Language (SML): The SML and Platform
DEV221 A Lap Around the New Enhancements for Web Developers in Visual Studio Orcas
DEV203 ADO.NET - Present and Future

Wed Nov 8 09:00 - 10:15
DEVWD15 Hardcore .NET Production Debugging
DEV349 ASP.NET: Developing Data-Driven Web Applications with .NET Language Integrated Query (DLINQ)
DEV323 C# 3.0: Future Directions in Language Innovation from Anders Hejlsberg
SQL402 Implementing the Service-Oriented Database Architecture (SODA) with SQL Server

Wed Nov 8 10:45 - 12:00
DEVWD03 Alternative .NET Debugging Facilities
SQL312 T-SQL Querying : Tips and Techniques
DEV217 Introducing Visual Studio (VS) 2005 Team Edition for Database Professionals
ARC304 Connected Systems - Part 3: Data and Transaction Management

Wed Nov 8 13:30 - 14:45
DEVWD12 Explore C# 3.0 with Anders Hejlsberg
DEV305 Garbage Collection - How to Make it Work for You
ARC301 Connected Systems - Part 4: Presentation and Interactivity
DEV002 DEMO: 10 Undiscovered features of ASP.NET 2.0

Wed Nov 8 15:15 - 16:30
DEV219 Using the Essential Unified Process with Visual Studio Team System (VSTS)
DEV315 Scalability, Reliability and Maintainability of Internet Information Services (IIS)-hosted Workflows
ARCWD02 Enterprise Library for .NET Framework 3.0
DEVWD28 Exploring New Functionality in Internet Explorer (IE) 7

Wed Nov 8 17:00 - 18:15
DEV317 Integrating the Visual Studio (VS) Team Edition for Database Professional into your Software Development Lifecycle
DEV325 Windows Vista: Tips & Tricks for Targeting Key Native Application Program Interfaces (APIs) from Managed Code
ARC201 Patterns and Anti-Patterns for Service-Oriented Architectures (SOA)
DEVWD18 Why is Essential Unified Process (EssUp) a Next Generation Process?
Cheeseman?

Thu Nov 9 09:00 - 10:15
DEV306 Using the .NET Language Integrated Query Framework with Relational Data
DEV345 Asynchronous ASP.NET Programming
DEV321 Delving into Visual Studio 2005 Team Edition for Software Developers
ARC307 Adding Value to Message-based SOAs with Workflow
Vista?

Thu Nov 9 10:45 - 12:00
DEV206 Common Language Runtime (CLR): IronPython and .NET Scripting Languages
DEV229 ASP.NET: End-to-End - Building a Complete Web Application using ASP.NET 2.0, Visual Studio 2005, and Internet Information Services (IIS) 7 (Part 1)
SQL403 Understanding Interactions of Transactions, Batches, Nested Procedures, MARS (Multiple Active Result Sets), and SQLCLR (Common Language Runtime) in SQL Server 2005
DEV308 Proven Practices for Implementing Services
ARC302 Connected Systems - Part 5: Identity and Access Management

Thu Nov 9 13:30 - 14:45
DEV227 Windows Presentation Foundation (WPF) in the Real World: Zürich Airport Monitoring System
DEVWD27 "Scripting ASP.NET AJAX": Deep Dive into ASP.NET AJAX, Scripting & Debugging
ARC205 Connected Systems - Part 6: The Future of the Microsoft Application Server Platform
SQL406 Developing SQL Server 2005 Analysis Services Powered Applications

Thu Nov 9 15:45 - 17:00
DEV243 Implementing Continuous Integration: With and Without Team System
DEVWD17 Team System Adoption Best Practices
DEV359 .NET Hidden Treasures
SQLWD04 The Query Governor: SQL CLR in Action

Thu Nov 9 17:30 - 18:45
DEV322 Unit Testing Best Practices With Visual Studio 2005 Team System
DEVWD24 Converting Existing ASP.NET Applications to Visual Studio 2005 Web Application Projects
SQL301 Coding SQL Server 2005: Plan Guides and Plan Forcing
ARC309 Security is a Feature - Best Practices for Designing Secure Distributed .NET Applications

Fri Nov 10 09:00 - 10:15
DEV403 Optimizing Performance and Scalability of Distributed .NET Applications
DEV360 Windows Powershell - Building Next Generation Command Line And GUI Applications On Windows
DEV338 Windows Vista for Managed Developers: Beyond NetFx3
DEVWD14 Test Driven Development: Myths and Misconceptions
DEV346 Attack and Defense: Securing ASP.NET 2.0 Applications

Fri Nov 10 10:45 - 12:00
ARC209 Microsoft Solutions Framework 4.0 Core and its Family
DEV405 BizTalk Server 2006 - Internals
SQLWD06 Best Practices for architecting Reporting Services Report Models for Ad-hoc Reporting
DEV218 Developing and Designing Standard Based Web Applications with Expression Web Designer

Fri Nov 10 13:30 - 14:45
SQL404 SQL Server 2005 Cache Sync/Query Notifications in Depth
DEV411 AJAX Patterns with the Microsoft AJAX Library
DEV004 DEMO: Technology in Action! Building a Distributed Solution with .NET Framework 3.0
SQLWD01 SQL Server Integration Services (SSIS) – The Top 10 Tips
ARC206 Abstract Concepts: Architecting Applications for a Service-Oriented World

Fri Nov 10 15:15 - 16:30
DEV319 Visual Studio 2005 – Advanced Data Access Techniques
DEV358 New Cryptography: Algorithms, APIs and Architecture
DEV366 Boost Your Data-Driven Application Development Using SQL Server Centric .NET Code Generator [OlyMars Refresh]
DEVWD25 IIS 6.0 Programming Interfaces and Architectural Drill Down

Planning to learn some interesting stuff, either new or very specialized. From my schedule, it's obvious that my main interests are Architecture, SQL, Development.

2006-10-31T11:35:00.000Z

Rereading No Silver Bullet
While re-reading "No Silver Bullet" (http://www-inst.eecs.berkeley.edu/~maratb/readings/NoSilverBullet.html), I was amazed by the intemporality of the article. It was published in April 1987 (Computer, Vol. 20, No. 4) and written by Frederick P. Brooks, and as it's almost TWENTY years old it feels more actual than ever.

In the article, Brooks divides the difficulties in Software Engineering between Essence and Accidental.

On the side of the essential difficulties, are the complexity, conformity, changeability and invisibility of software. Meaning that software is inherently complex as it is an abstract entity that represents complex interactions (business rules) and interfaces with complex systems/processes/procedures. Not only that, but generally it's the software that must change to accomodate the business process and frequently the client doesn't know what he wants/needs (funny to see how something never changes J).

The past breakthroughs (and some current silver bullets, in my opinion) only solve accidental difficulties. These are difficulties not inherent to building software, but with the way it was done.
Among these are: High-level Languages (raises the expressive power of the language, but doesn’t attack the complexity of software), Time-Sharing (reduces the time gap between code, build and execution) and Unified Programming Environments (back then Unix and Interlisp, J).

Among the hopes, there is ADA, OO programming, Artificial Intelligence, Graphical Programming, Environments and Tools and Workstations, all with marginal increment.

The possible breakthroughs must come from:
Buy vs Build (if one builds a system from bought components and minimal adjustments, the cost can be much lower, because it’s already tested, the delivery is immediate)
Requirements refinement and rapid prototyping (most agile methodologies revolve around the iterative construction of a system. Considering the first iterations of a system, as a prototype that evolves to the final system, this point%2