Here’s an example. We want to find the largest index i such that haystack[i] <= needle. Needle = 15, and haystack is a sorted list of the first 8 prime numbers, indexed by binary numbers:

000     2
001     3
010     5
011     7
100     11
101     13
110     17
111     19

Note first that the haystack index requires no more than 3 bits. Therefore we start with the highest order bit set: b=100. (This font denotes binary numbers here.) Is haystack[100] <= 15? Yes, 11<=15. Observe this means that the first bit of the index we are looking for is 1. We look at the next bit. Is haystack[110] <= 15? No, 17 <= 15 is false. This means the 2nd bit is 0. Finally, is haystack[101] <= 15? Yes, 13 <= 15. Therefore the index we are looking for is 101.

In essence, the main loop to find the index i is simply:

    for( ; b ; b >>= 1 )
        if( haystack[i|b] <= needle ) i |= b;

where b is the value with 1 bit set, that began with b=100 and i=0.

It is the natural structure of the bits within the index variable that automatically tracks both the upper and lower bounds of the search window for each iteration. Compare this with the less efficient and more verbose arithmetic performed in the main loops of standard binary search algorithms.

The full C++ code for the improved binary search algorithm fbsearch() is:

// Binary search revisited.
 
// Define only one of these:
// #define SETBIT_FAST
// #define SETBIT_FASTER
#define SETBIT_FASTEST
 
#ifdef SETBIT_FAST
#include <math.h>
#endif
 
// Return 1 << log_2(list_size-1), or 0 if list_size == 1.
// This sets the initial value of b in fbsearch().
inline unsigned init_bit( unsigned list_size )
{
#ifdef SETBIT_FAST
    return list_size == 1 ? 0 :
        1 << int( log(list_size-1) / M_LN2 );
#endif
#ifdef SETBIT_FASTER
    return list_size == 1 ? 0 :
        1 << ( sizeof(unsigned) << 3 ) - 1
             - __builtin_clz(list_size-1);
#endif
#ifdef SETBIT_FASTEST
    unsigned b;
    __asm__ ( "decl %%eax;"
              "je DONE;"
              "bsrl %%eax, %%ecx;" // BSR - Bit Scan Reverse (386+)
              "movl $1, %%eax;"
              "shll %%cl, %%eax;"
              "DONE:" : "=a" (b) : "a" (list_size)
    );
    return b;
#endif
}
 
// Return the greatest unsigned i where haystack[i] <= needle.
// If i does not exist (haystack is empty, or needle < haystack[0])
// then return unsigned(-1). T can be any type for which the binary
// operator <= is defined.
template <typename T>
unsigned fbsearch( const T haystack[], unsigned haystack_size,
                   const T& needle )
{
    if( haystack_size == 0 ) return unsigned(-1);
    unsigned i = 0;
    for( unsigned b = init_bit(haystack_size) ; b ; b >>= 1 )
    {
        unsigned j = i | b;
        if( haystack_size <= j ) continue;
        if( haystack[j] <= needle ) i = j;
        else
        {
            for( b >>= 1 ; b ; b >>= 1 )
                if( haystack[i|b] <= needle ) i |= b;
            break;
        }
    }
    return i || *haystack <= needle ? i : unsigned(-1);
}
 
// Example Usage
#include <iostream>
using namespace std;
 
int main()
{
    const int sorted_list[] = { 2, 3, 5, 7, 11, 13, 17, 19, 23 };
    const unsigned list_size = sizeof(sorted_list)/sizeof(int);
    int needle = 15;
    cout << "fbsearch(sorted_list,"<<list_size<<','<<needle<<") = "
         << fbsearch(sorted_list,list_size,needle) << endl;
    // fbsearch(sorted_list,9,15) = 5
    return 0;
}

Ancillary notes:

• The function init_bit() sets the initial value of b to having a single bit set in the highest position for indexing the array list. The three SETBIT_FAST, SETBIT_FASTER, and SETBIT_FASTEST code blocks within it are simply 3 different ways of calculating the initial value of b, whose speed ranks are the reverse of their cross-platform compatibility. The first method should compile everywhere, and simply calculates it from a base-2 logarithm. The second method uses the faster GNU __builtin_clz method (thanks Chao Xu for the suggestion) that counts the highest order bit from the left. The third and fastest method uses the Intel assembly instruction BSR that counts it from the right.
• The null return value of unsigned(-1) (=4294967295 for 32-bit unsigned) is the unique value that will never otherwise be returned. Therefore it can safely be checked for by the calling function without risk of misinterpretation.
• The one else block can safely be removed without changing the overall functionality. It exists to simply save having to check that the array index is in bounds on every iteration, since it is at a point in logic where that condition is guaranteed.

Share:

I plan to use NoSQL in the not-too-distance future, but not today. When I port the data over I don’t want to have to map “old ids” to “new ids”. By using a uuid from day one I can easily maintain referential integrity when I migrate to a new datastore.

I looked up how this is typically done in Rails using MySQL and all I could find was examples that store the UUID as strings (e.g. here). The UUID is natively just 16 bytes so storing this in a VARCHAR(36) seemed like a poor solution. Ideally we would store the UUID as binary and let the ORM deal with translating it to and from a UUID object.

activeuuid solves this problem. activeuuid is a Rails plugin that will let you use UUIDTools::UUID objects as primary keys for your ActiveRecords and it stores them in MySQL as binary(16).

Here’s how to use it:

Create a Migration

activeuuid adds the uuid type to your migrations. Example:

class CreateEmails < ActiveRecord::Migration
  def self.up
    create_table :emails, :id => false  do |t|
      t.uuid :id, :unique => true
      t.uuid :sender_id  # belongs_to :sender
 
      t.string :subject
      t.text :body
 
      t.timestamp :sent_at
      t.timestamps
    end
    add_index :emails, :id
  end
 
  def self.down
    drop_table :emails
  end
end

include ActiveUUID::UUID in your model

class Email < ActiveRecord::Base
  include ActiveUUID::UUID
  belongs_to :sender
end

use it

Here are some example specs:

require 'spec_helper'
 
describe Email do
 
  context "when using uuid's as keys" do
    before(:each) do
      Email.delete_all
      @guid = "1dd74dd0-d116-11e0-99c7-5ac5d975667e"
      @e = Email.new(:subject => "hello", :body => "world") {|e| e.id = UUIDTools::UUID.parse(@guid) }
      @e.save
    end
 
    it "the id guid should be equal to the uuid" do
      @e.id.to_s.should eql(@guid)
    end
 
    it "should be able to find an email by the uuid" do
      f = Email.find(UUIDTools::UUID.parse(@guid))
      f.id.to_s.should eql(@guid)
    end
 
  end
end

Benefits

There are a number of benefits to using UUIDs as primary keys.:

• no id conflict during multi-master write
• no locking due to auto-increment
• with time-based UUIDs you can store a timestamp within your UUID
• you can create natural keys (based on the SHA of model attributes)

Future work

• more transparent support for natural and composite keys
• support for MySQLs `INSERT … ON DUPLICATE KEY UPDATE` syntax
• support a primary column name other than `id`
• work on other databases (Postgres, etc)
• tests

Installation

Add this to your Gemfile

    gem "activeuuid"

Or get the code here: https://github.com/jashmenn/activeuuid

References

• http://bret.appspot.com/entry/how-friendfeed-uses-mysql
• http://kekoav.com/blog/36-computers/58-uuids-as-primary-keys-in-mysql.html
• https://gist.github.com/937739
• http://www.codinghorror.com/blog/2007/03/primary-keys-ids-versus-guids.html
• http://krow.livejournal.com/497839.html
• https://github.com/jamesgolick/friendly

Dependencies

Note that this depends on Rails 3.1 because it uses the custom column serialization added by Aaron Patterson. (See: https://github.com/rails/rails/commit/ebe485fd8ec80a1a9b86516bc6f74bc5bbba3476)

Share:

Checkout the screencast.

Share: