Normalize you DB

Hi everyone,

In the last couple of days I had the opportunity to normalize a DB. :)

Let me show you a screenshot about a single table:

As you see a lot of data is repeated across the rows. Therefore, I had some bad thoughts about those who let the DB become like this in the beginning. I didn't wanted to do it. As the moments were passing away I let the challenge to excite me. At the end I did it.

Before doing anything the total number of rows in the db was above 100000 and the size of the table was 10,5 MBytes.

My first step was to extract the name of the components into another table and reference them.

This way the size of the table decreased to 8.5 MBytes. And the new table size was 1.5 Mbyte. I earned 0.5 Mbyte, not too much, but the database was much more cleaner.

My second step was to transform the check/change data represented by a string into their numerical representations, this way enabling the database to be internationalized. By doing this the table size decrease to 6.2 MBytes. That means 72% of its original size and the data can be internationalized and we have all the benefits of a normalized DB schema (that means no update problems and a joy to work with).

I also measured the speed of joins operations. Because of the normalization the same data retrieval was increased by 3 times. I put an index onto foreign keys. And the speed was just like before. :)

Design your User Interface

Hello everybody,

I have just finished the book "The Design Of Everyday Things".
It's a great book written by Donald Norman. The guy who wrote "The invisible computer", who worked for the Apple Computer's Advanced Technology Group.

In this book he talks about many-many interesting things:
-The constraints and strength of our short term & long term memory.
-The slips and mistakes our brain makes, while we do things.
-How can we make use of our cultural knowledge and knowledge in the world while design.
-The importance of natural mapping between controls and our actions.
-Interpretation of a system state, and our mental model about the system.
-The (missing) evolution of product line. The evaluation of the already selled products and the importance of debriefing then evolving the product to a better one.

He also describes how primitive were the computers at time of writing.

And he gives us some hints how he imagines the mobile phones should easy our everyday tasks. I had the impression the he was envisioning the nowadays iPhones. He also describes how a graphical user interface would help us to be more effective with the computer interaction. I often had the feeling that he was describing a vision which is very similar about nowadays iPads, iPhones or Microsoft Surface.

I was also symphatizing with him because I remember times when I needed to setup my Mouse and CD driver in the config.sys in order to use those devices. Nowadays everything is much more easier. Especially that, I am a Macintosh user I can spend my time focusing on relevant tasks, without even realizing that I am working.

I finished the book yesterday. After I put the book back onto my shelf I started to browse the new Apple products... They are just amazing. I was especially amazed by the new Siri feature in the iPhone. It remembers me to the "Computer" from Startreak. It is a good example that if we can imagine something then we can sooner or later achieve it. Another feature I imagined even myself and waited a lot is the iCloud. I tried the beta version by downloading Ethics from Aristotle on my Macbook and the book instantly appeared on my iphone too. :) Definetely it enhances our usability experience.

The next day, other news were expecting me. Sad news. The founder and main dreamer, envisioner and technology pioneer Steve Jobs died. So good journey Steve and rest in peace.

sqlite log filtering

Hi,

Presumably a lot of you are developing rails applications using sqlite.
In this case the log is spamed by some pesky sql statements every time our app accesses the db.

 SQL (0.6ms)   SELECT name
 FROM sqlite_master
 WHERE type = 'table' AND NOT name = 'sqlite_sequence'
  SQL (0.6ms)   SELECT name
 FROM sqlite_master
 WHERE type = 'table' AND NOT name = 'sqlite_sequence'
... 
 SQL (0.6ms)   SELECT name
 FROM sqlite_master
 WHERE type = 'table' AND NOT name = 'sqlite_sequence'
  SQL (0.6ms)   SELECT name
 FROM sqlite_master
 WHERE type = 'table' AND NOT name = 'sqlite_sequence'
...

These messages bother me because they have nothing to do with my application logic, they are only needed by the sqlite driver to access the db structure.


Therefore I developed a small ruby gem called sqlite_log_filter which filters these messages out. The source can be found on github


All you need to do is to put it in your gemfile

gem 'sqlite_log_filter'

Then bundle it

bundle install

Hopefully it helps :)

Test (Behaviour) Driven Development

Recently I saw a pretty nice post:

"If you see anyone pushing code to Rails without tests, ping me and I will gladly revert. In 2011, this should be unacceptable."

I remember times when I had a project which was developed test driven at my former place of work. Of course the Test Driven initiator was me, and by time the teammates embraced the methodology and it managed to be a very successful project.

The most exciting thing was that I never had to debug; the tests always showed me the exact the location of the problem -if there was any.
Another almost unbelievable fact was that days before releases we didn't have any unfinished tasks. We were down on the streets having a Budweiser or limonade and we were chatting. What I mean here is that the test batteries saved us so much time that we never had to work overtime. We were just shipping the product confidently days before the planned release. I also remember some sales meetings where this project was showed to the customers as a success story. I also remember one of the questions one customer asked: "Then why don't you write software this way every time?" and the answer was "Because some customers simply dictate to not write any test at all..." Well everyone in the meeting laughed. But we all knew that this decision would cause any customer to throw out hundreds of thousand of dollars and hundreds of hours of overtime for the developers without mentioning the moral downgrade for every involved party. Maybe we were laughing at that unintelligent stereotype.
I also remember one important fact while writing this. In the initial project schedule the testing time wasn't calculated. We simply adopted the test first and then the benefits showed themself over and over again. We couldn't run out of the schedule... :) Almost unbelievable but it was true.

After the project has been finished I have tried to introduce this methodology in as many projects as I could. Unfortunately there were some projects where the mass was so frozen that my attempts were totally ignored. Sometimes the reason for ignorance was simply the lack of infrastructure. Fortunately the infrastructure is not a problem in Ruby on Rails. The community is also mostly test-centered.

I cannot emphasize enough for every developer in any domain to embrace this Test First thinking. There are so many benefits of it that many books have been written about it. And who knows, maybe you who will embrace this approach, will have some similar success story to tell the world

autotest, zentest, autotest-rails, autotest-rails-pure: which gems to choose?

It was a little bit confusing for me which "autotest like" gems to choose when I want to autotest my rails application.
Well... There are some alternatives depending which testing framework I choose rspec or Test::Unit, also there are alternatives if I want to generate the tests for my controller actions automatically with zentest or I just want to write them in some other way.

Zentest was the first tool in this domain with a lot of features. Then autotest-standalone was factored out from it. There is also an empty gem called Autotest which is dependent on Zentest.

Because the main idea is to simply map the lib and test directories, in order to test rails application we need to tell the rails specific mappings. This is done by the autotest-rails and rspec-rails and autotest-rails-pure gems. So let's walk through each of the alternatives

Test::Unit

In this case the rails specific mappings are done by the autotest-rails-pure or autotest-rails gems.
The code excerpt which makes the mapping for the controllers is:

    add_mapping %r%^app/controllers/application_controller\.rb$% do |_, m|
      files_matching %r%^test/(controllers|views|functional)/.*_test\.rb$%
    end

For the core autotest we can use the autotest-standalone or the feature rich old Zentest.

Rspec

The rails specific mapping is done by the rspec-rails in case we opt for using rspec

Here is an excerpt wich does the mapping for the controllers:

    add_mapping(%r%^app/controllers/(.*)\.rb$%) { |_, m|
      if m[1] == "application"
        files_matching %r%^spec/controllers/.*_spec\.rb$%
      else
        ["spec/controllers/#{m[1]}_spec.rb"]
      end
    }

So there are a plenty of alternatives we can choose from and it is good to know which opportunities are available. We have all we need :)

Making the Factory Girl to support Paperclip

Hi Rubysts,

Paperclip and Factory Girl Rails both of them are my loved gems.
Unfortunately Factory Girl Rails can't create the model if the model had been Papercliped. Well at least not in Rails 3.0.7. But the good news is that it can be solved by adding a few lines of Ruby code to our factory.

This is my model:

class Club < ActiveRecord::Base
  has_many :dancers
  has_attached_file :photo, :styles => { :medium => "300x300>", :thumb => "100x100>" }
  validates_attachment_presence :photo
  validates_attachment_size :photo, :less_than => 1.megabytes
end

So in order to use the factory with ease we need to tell our factory to upload some pictures into our model.
Therefore let me show you an example how to write the factory which makes the two gems to co-work:

include ActionDispatch::TestProcess

class Club
  attr_accessor :photo_file_name
  attr_accessor :photo_file_size
  has_attached_file :photo, :styles => { :medium => "300x300>", :thumb => "100x100>" },
                    :url  => "/test_can_be_deleted/:attachment/:id/:style/:filename",
                    :path => ":rails_root/public/test_can_be_deleted/:attachment/:id/:style/:filename"
end

Factory.define :club do |c|
  c.title 'Limpex'
  c.sefurl 'Limpasm'
  c.name  'Limpasm'
  c.metadescription 'good etc'
  c.content '<p> a good place</p><p>the icecream is prety damn good</p>'
  c.state 0
  c.city 'Gheorgheni'
  c.email 'some@somplexx.com'
  c.photo   { fixture_file_upload( 'spec/factories/test.png', 'image/png') }
end

As you can observe I have just reopened my model and have redefined the paperclip attachment to point to a different path in order to store the dev and test mode attachments in different places.

That's it. :)

The missing event when the mouse stays in a place

Hello everybody,

I just released a jquery plugin which detects if the mouse stays in a place for more than the specified period of millisecs.

I was missing this functionality and I also needed more than twice and :) I needed also in the past so finally I decided to extract as a reusable plugin.

You can download it from here:

https://github.com/orbanbotond/mouseStay

or here:

http://plugins.jquery.com/project/mousestay

It is easy to use as any other jquery thing.

Enjoy :)