Normalize you DB

Hi everyone,

In the last couple of days I had the opportunity to normalize a DB. :)

Let me show you a screenshot about a single table:

As you see a lot of data is repeated across the rows. Therefore, I had some bad thoughts about those who let the DB become like this in the beginning. I didn't wanted to do it. As the moments were passing away I let the challenge to excite me. At the end I did it.

Before doing anything the total number of rows in the db was above 100000 and the size of the table was 10,5 MBytes.

My first step was to extract the name of the components into another table and reference them.

This way the size of the table decreased to 8.5 MBytes. And the new table size was 1.5 Mbyte. I earned 0.5 Mbyte, not too much, but the database was much more cleaner.

My second step was to transform the check/change data represented by a string into their numerical representations, this way enabling the database to be internationalized. By doing this the table size decrease to 6.2 MBytes. That means 72% of its original size and the data can be internationalized and we have all the benefits of a normalized DB schema (that means no update problems and a joy to work with).

I also measured the speed of joins operations. Because of the normalization the same data retrieval was increased by 3 times. I put an index onto foreign keys. And the speed was just like before. :)

Design your User Interface

Hello everybody,

I have just finished the book "The Design Of Everyday Things".
It's a great book written by Donald Norman. The guy who wrote "The invisible computer", who worked for the Apple Computer's Advanced Technology Group.

In this book he talks about many-many interesting things:
-The constraints and strength of our short term & long term memory.
-The slips and mistakes our brain makes, while we do things.
-How can we make use of our cultural knowledge and knowledge in the world while design.
-The importance of natural mapping between controls and our actions.
-Interpretation of a system state, and our mental model about the system.
-The (missing) evolution of product line. The evaluation of the already selled products and the importance of debriefing then evolving the product to a better one.

He also describes how primitive were the computers at time of writing.

And he gives us some hints how he imagines the mobile phones should easy our everyday tasks. I had the impression the he was envisioning the nowadays iPhones. He also describes how a graphical user interface would help us to be more effective with the computer interaction. I often had the feeling that he was describing a vision which is very similar about nowadays iPads, iPhones or Microsoft Surface.

I was also symphatizing with him because I remember times when I needed to setup my Mouse and CD driver in the config.sys in order to use those devices. Nowadays everything is much more easier. Especially that, I am a Macintosh user I can spend my time focusing on relevant tasks, without even realizing that I am working.

I finished the book yesterday. After I put the book back onto my shelf I started to browse the new Apple products... They are just amazing. I was especially amazed by the new Siri feature in the iPhone. It remembers me to the "Computer" from Startreak. It is a good example that if we can imagine something then we can sooner or later achieve it. Another feature I imagined even myself and waited a lot is the iCloud. I tried the beta version by downloading Ethics from Aristotle on my Macbook and the book instantly appeared on my iphone too. :) Definetely it enhances our usability experience.

The next day, other news were expecting me. Sad news. The founder and main dreamer, envisioner and technology pioneer Steve Jobs died. So good journey Steve and rest in peace.

sqlite log filtering

Hi,

Presumably a lot of you are developing rails applications using sqlite.
In this case the log is spamed by some pesky sql statements every time our app accesses the db.

 SQL (0.6ms)   SELECT name
 FROM sqlite_master
 WHERE type = 'table' AND NOT name = 'sqlite_sequence'
  SQL (0.6ms)   SELECT name
 FROM sqlite_master
 WHERE type = 'table' AND NOT name = 'sqlite_sequence'
... 
 SQL (0.6ms)   SELECT name
 FROM sqlite_master
 WHERE type = 'table' AND NOT name = 'sqlite_sequence'
  SQL (0.6ms)   SELECT name
 FROM sqlite_master
 WHERE type = 'table' AND NOT name = 'sqlite_sequence'
...

These messages bother me because they have nothing to do with my application logic, they are only needed by the sqlite driver to access the db structure.


Therefore I developed a small ruby gem called sqlite_log_filter which filters these messages out. The source can be found on github


All you need to do is to put it in your gemfile

gem 'sqlite_log_filter'

Then bundle it

bundle install

Hopefully it helps :)

Test (Behaviour) Driven Development

Recently I saw a pretty nice post:

"If you see anyone pushing code to Rails without tests, ping me and I will gladly revert. In 2011, this should be unacceptable."

I remember times when I had a project which was developed test driven at my former place of work. Of course the Test Driven initiator was me, and by time the teammates embraced the methodology and it managed to be a very successful project.

The most exciting thing was that I never had to debug; the tests always showed me the exact the location of the problem -if there was any.
Another almost unbelievable fact was that days before releases we didn't have any unfinished tasks. We were down on the streets having a Budweiser or limonade and we were chatting. What I mean here is that the test batteries saved us so much time that we never had to work overtime. We were just shipping the product confidently days before the planned release. I also remember some sales meetings where this project was showed to the customers as a success story. I also remember one of the questions one customer asked: "Then why don't you write software this way every time?" and the answer was "Because some customers simply dictate to not write any test at all..." Well everyone in the meeting laughed. But we all knew that this decision would cause any customer to throw out hundreds of thousand of dollars and hundreds of hours of overtime for the developers without mentioning the moral downgrade for every involved party. Maybe we were laughing at that unintelligent stereotype.
I also remember one important fact while writing this. In the initial project schedule the testing time wasn't calculated. We simply adopted the test first and then the benefits showed themself over and over again. We couldn't run out of the schedule... :) Almost unbelievable but it was true.

After the project has been finished I have tried to introduce this methodology in as many projects as I could. Unfortunately there were some projects where the mass was so frozen that my attempts were totally ignored. Sometimes the reason for ignorance was simply the lack of infrastructure. Fortunately the infrastructure is not a problem in Ruby on Rails. The community is also mostly test-centered.

I cannot emphasize enough for every developer in any domain to embrace this Test First thinking. There are so many benefits of it that many books have been written about it. And who knows, maybe you who will embrace this approach, will have some similar success story to tell the world