Archive for the 'Technical' Category

Zen Software Development

Monday, June 28th, 2010

Software development involves following various levels of rules: rules for language syntax, rules for comprehensibility, rules for testability, scalability, essentially rules for every requirement or constraint.

When we engage in test-first development, it is as if we are writing the rules in a way that cannot be forgotten. When we write code, and the tests pass, proving that the requirements and constraints have been met, the code possesses elegance and beauty.

There is a tremendous and valuable freedom in being able to make large-scale changes in code without breaking any rules. We frequently find that rules are too strong. They have sufficient exceptions that they must be weakened.

We frequently find that rules are too weak. They allow undesirable effects to occur.

There is also satisfaction in understanding that the rules (tests) are written by us. We are free to change them if requirements change, but there is value in writing them nevertheless.

max_allowed_packet on Mac OS X MySQL

Thursday, August 13th, 2009

While developing a Ruby on Rails application that supported attachments, I ran into an error to the effect, “size exceeds max_allowed_packet”. This is because the default installation of MySQL does not accept network packets exceeding 1MB in length.

The solution involves using the MySQL Administrator tool. This is a useful program available as a separate package from MySQL. Unfortunately, it is not very user friendly.

  1. Click the “Options” button in the top navigation strip
  2. Choose “Advanced Networking” from the dropdown
  3. Click the pencil icon next to “Max. packet size:”. The words will turn from (disabled) gray to (enabled) black.
  4. Adjust the size using the text box, arrows, or dropdown.
  5. Click save. By default, there is no configuration file, so you may be asked to enter administrator credentials to create or modify the file.

The interface allows you to skip 3. and do 4. and 5. without complaint, but the configuration will not be changed. The change can be verified by re-launching the MySQL Administrator and by viewing the contents of /etc/my.cnf.

MySQL Administrator

The Search for Project Management Software

Wednesday, July 22nd, 2009

I’ve been looking for project management software to help run my consulting business. I have a small number of customers and a larger number of prospects. Some of the work is paid and some is unpaid (like moving prospects through the pipeline and developing prototypes and proposals) but in either case, I needed to prevent work from slipping into the cracks. Functionally, I needed, at minimum, the following capabilities:

  • Create a list of tasks
  • Establish durations for tasks
  • Create dependencies between tasks so that when one task is delayed (and they often are), dependents are automatically shifted. I had used Basecamp before, but it does not allow creation of such dependencies between tasks.
  • Group tasks into projects
  • Display a Gantt chart of  projects

This seemed like a reasonable set of requirements, considering Microsoft Project had these functions in its first version.

I also wanted it to be

  • web-based, so that other employees could use it when I eventually hire them.
  • free or cheap because the business is relatively poor in cash

There were a couple of good reasons why it could or should be open source.

  • The business is relatively wealthy in technical expertise (mine).
  • I envisioned customizing it and linking it into other systems.

My network query (Facebook, LinkedIn, and Twitter) yielded almost nothing. Google searches yielded a wiki page with a long list but a dearth of reviews or recommendations. Which to choose?

Since so many open source projects use SourceForge, I hit on the idea of looking for the top downloads in the project management category. OpenBravo seemed most promising because it has been downloaded a million times, over five times more than the second-place software. It claims to be a full-blown ERP system with accounting as well as project management. It runs on Tomcat. Unfortunately, my hosting relationships do not make running Tomcat software easy, so I decided to look further down the list. (In a weak moment, the engineer in me succumbed to not-invented-here angst, and briefly considered the benefits of converting the code to Ruby on Rails.)

OpenGoo was fairly easy to install, but lacks the task-dependency feature. GroupOffice seemed slightly harder to install, but also lacks this feature. I tried eGroupWare, which has an even more complex install, but I couldn’t get the project management function to work as required. The documentation wasn’t very helpful, having apparently been translated from German.

Finally, I settled on OpenProj, from Serena Software.  It is desktop-based, but at least it has the familiar interface and functions of Microsoft Project.

The Marketing Funnel and the Sales Pipeline

Sunday, June 21st, 2009

Sales and marketing professionals imagine a funnel or pipeline. Leads enter the process when they are generated. They are thrown out of the pipeline as they are disqualified or otherwise deferred. They progress in the process as they are qualified, closed, and ultimately become satisfied customers.

Having built my own marketing automation system, and having seen systems built by others, I wonder if there can be an explicit mapping between the funnel/pipeline and a web site’s structure. That is, can we create interactions that result in information exchanges and that automatically move leads through the pipeline?

Portland Code Camp Presentation 1: Agile Really Can Satisfy the Suits

Saturday, May 30th, 2009

Click here to download a PDF version of my presentation at Portland Code Camp 2009.

Deep Disappointment with Drupal

Thursday, May 14th, 2009

When I first built a web site for my company, Now Interactive LLC, I chose a content management program called Drupal.

What I wanted was a site where I could feature products, services, clients, partners and contact information–the usual components of a corporate web site. I also wanted to blog about product management and web application design.

My first thought was to use an excellent blogging program called Wordpress.  It gives you a blogging site along with the ability to write static pages. However,  Wordpress is narrowly focused on blogging. On the home page, you see the blog, and available free templates all look like blogs. I would have to write and maintain my static pages separately and in their entirety. It became clear that I would have to customize Wordpress, which is written in PHP, and this was something I wanted to avoid.

My personal web site is a Ruby on Rails application that links to a Wordpress blogging site. The problem is that while the content management on the blogging site is user friendly and sophisticated, the content management on the Ruby on Rails side is a bit clunky. After all, the application was not written for content management but for a personal marketing campaign (which went off extremely well).

For about a solid half-day, I tried using a Ruby on Rails application that read from a Wordpress database.  The idea was that my corporate site would be written in Ruby on Rails, while the content management (editing, tagging, categorizing, and controlling revisions) would be handled by WordPress. But after struggling to get categories to display in the right order as tabs, I decided to try something else.

I had used Joomla briefly and found it counter-intuitive and complex with its notions of menus and content that might or might not actually appear in the site. A representative from OpenSourcery at InnoTech Oregon had recommended Drupal over Joomla. Sure enough, some large companies use Drupal for impressive sites. Word on the internet was that Drupal had a learning curve, but not being one to shrink from learning curves, I waded in.

Drupal was easy to install in my hosting service. I’d probably put it at 3X the effort of the (ridiculously easy) Wordpress installation. Installation of new templates was a simple matter of copying files to the server. Drupal automatically checked for updates to itself and any installed templates. It ran a daily cron job for housekeeping.

Unfortunately, I started encountering problems soon afterward.

  • I struggled to understand the difference between a Page and a Story. A Page seemed like a Wordpress page–an entire page that you managed yourself and that Drupal didn’t know much about. But I wanted to deal at a finer grain.  For example, I might want a page of partners but feature one of them on the home page. A Story seemed more like what I wanted, but it had funny features like automatically showing up on the front page.
  • I figured out how you need to create content and then create menu entries pointing to the content. Much later, I realized you could create menu entries at the same time as the content.
  • I put all my menu items in the Navigation menu, which gave me a very plain menu. I didn’t realize until much later that some templates treat the Primary Links and Secondary Links with special effects.
  • My triumph was figuring out how to assign stories to categories and link a category to a menu item. It involves yet another concept called a “taxonomy”. This is a hierarchy of categories to which you can assign stories. A menu item can be linked to a taxonomy entry, which displays all content assigned to that entry.
  • I figured out that you can order menu entries by dragging and dropping, but this is interconnected with the notion of “weight”. (What does “weight” mean for a horizontal menu?) Ordering stories on a page was much harder. Eventually, I figured out it assigns an editable timestamp to each story and displays them in reverse chronological order. By assigning the appropriate date stamps, I could alter the order in which the stories were displayed. (This is called a “kludge”.)
  • One problem I ran into was when you clicked on a menu item, you would see the name of that menu item all over the [BLEEP] place. So if I clicked on “Partners” then the page would display “Partners” in enormous letters, and each content item would say, “Posted in Partners on April 25, 2009 3:45 PM”. Of course “Partners” would not show up in the URL as you would expect. Instead, it would say, “http://www.now-interactive.net/node/15″ and this was considered a “clean” URL good for search engine optimization!
  • Eventually, I found a template that would not display so much [BLEEP]. Unfortunately, templates varied widely how much they could be configured, so I really just broke some less important stuff in order to fix some more important stuff.
  • My friend pointed out that the menu you just selected isn’t highlighted. Duh! It turns out there is some support for this, but it requires editing CSS—but not the CSS you originally installed, because Drupal has made a separate copy!
  • The interface is inconsistent. For example, assigning <none> to page and story titles ensures that no title appears, but assigning <none> to a menu name gives you a menu with “<none>” prominently featured.
  • There are very few good tips and hints on the internet, and what there is is sorely out-of-date.

After all this, I began to understand why people have made a business out of being Drupal consultants. Come to think of it, my last employer had run a web site on Joomla, but used a consultant for all changes. Indeed, since I couldn’t hire one, maybe I should consider being one…

Honestly, I had no real problem with any of this. You get what you pay for, and I had been amply warned of the learning curve. Furthermore, complexity is directly proportional to flexibility.

However, recently I have been approached by people who want a fairly simple web site. In many cases, they are using FrontPage and sending files to a server using FTP, but they want to do things like

  • mass e-mail campaigns to drive traffic to their site
  • sell an inventory of products through PayPal
  • have a private area for customers or members
  • do a slide show
  • allow reservation of resources
  • choose a site design consistent with logo colors

Drupal can certainly do all of these things and more, but it would take me a long time to set it up for these people. Furthermore, I don’t want them to have to turn to me every time they need a small change, and I don’t want to expose them to the counter-intuitive terminology and complexity of the administration interface.

In my next post, I’ll spell out what’s really needed.

Enabling PHP on Mac OS X 10.4 (Tiger)

Saturday, May 9th, 2009

I didn’t see a simple solution, so here it is.

  1. Start the Apache web server as follows
    1. Bring up System Preferences. It is available under the Apple menu.
    2. Click on Sharing
    3. Make sure “Personal Web Sharing” is checked on the left. On the right, it should say “Personal Web Sharing On”
    4. You should see a message below saying “View this computer’s web site at [first link] or your personal website at [second link].
    5. Click on [first link]. Safari should come up with the Apache logo and a message saying the installation of the Apache web server was successful.
    6. Go back to System Preferences and click on [second link]. Safari should come forward with a big “X Your website here.”
  2. Edit httpd.conf
    1. Bring up a terminal window. (Terminal is located in “Applications” under “Utilities”.)
    2. cd to /etc/httpd/
    3. Bring up httpd.conf in read-write mode in your favorite editor.  I did this by doing “su [username]” where [username] is an administrator and then doing “sudo emacs httpd.conf”.
    4. Find the line that says “#LoadModule php4_module” and uncomment it. (Remove the “#” from that line.)
    5. Find the line that says “#AddModule mod_php4.c” and uncomment it. (Remove the “#” from that line.)
    6. Save the file.
  3. Restart the Apache web server
    1. Bring up System Preferences. It is available under the Apple menu.
    2. Click on Sharing
    3. Uncheck “Personal Web Sharing” On the right, it will say “Personal Web Sharing On”. The message below will disappear. If you try visiting the links in Safari, you will get an error message.
    4. Check “Personal Web Sharing” again. On the right, it should say “Personal Web Sharing On”
    5. You should see a message below saying “View this computer’s web site at [first link] or your personal website at [second link].
    6. Click on [first link]. Safari should come up with the Apache logo and a message saying the installation of the Apache web server was successful.
    7. Go back to System Preferences and click on [second link]. Safari should come forward with a big “X Your website here.”
  4. Create a test.php file
    1. In the terminal window, cd to ~/Sites. You should see index.html there. This is the file that shows “X Your website here”
    2. In the same directory, create a file named test.php. The file should contain a single line as follows: <?php phpinfo(); ?>
    3. Visit [second link] in Safari, but add “test.php” to the end.
    4. You should see a nice big table containing lots of information about PHP. The PHP logo and PHP version will be visible at the top.

Changing User Agent Strings

Thursday, March 19th, 2009

An increasingly important web site requirement is to present different pages for mobile browsers. Although it’s possible to purchase a large number of mobile devices for testing purposes, some testing can be done in software only.

Typically, you’ll be detecting the kind of device accessing your site using the user agent string sent by the browser. So, you want to test that your site presents different pages based on the user string the browser sends.

Here are instructions for instructing Safari on Mac OS X to present various user agent strings. Unfortunately, there is a fixed set that doesn’t include Blackberry browser strings.

Firefox (across platforms) can be instructed to send various customizable user agent strings using the User Agent Switcher add-on.

Here is a list of user agent strings for various browsers, and here is a list of user agent strings for various mobile browsers.

Ruby on Rails: How to allow upload and download of files

Sunday, December 7th, 2008

This example shows how to attach a file to an item in the database.

Preparation

No special preparation is required.

Database

The item’s database schema should include the following columns:

Name Type
attachmentdata LONGBLOB
attachmenttype VARCHAR(64)
attachmentfilename VARCHAR(64) (or longer)

This can be achieved with the following migration:

def self.up
  create_table :items do |t|
    t.column :attachmentdata, :binary, :limit => 64 * 1024 * 1024
    t.column :attachmenttype, :string
    t.column :attachmentfilename, :string
    t.timestamps
  end
end

NOTE: If you change the column names, avoid using simply “type”. This will produce unexpected results and errors because Rails treats uses columns with this name to store object class names.

Model

The item’s model should include the following method, which allows the upload of a file to the database. Note that the method name is different from any of the column names. Think of it as a “virtual column”:

def attachment=(rhs)
    if ("" != rhs) then
      self.attachmentdata = rhs.read
      self.attachmenttype = rhs.content_type
      self.attachmentfilename = rhs.original_filename
    else
      self.attachmentdata = nil
      self.attachmenttype = nil
      self.attachmentfilename = nil
    end
end

View

The most important part of the view is to enable multipart in the form submitted when the item is created or edited note that the name of the file field matches the virtual column we created above:

<% form_for(@item, :html => {:multipart => true}) do |f| %>
<p>
  <label for="item_attachment">Attachment:</label>
  <%= f.file_field('attachment') %>
</p>
	<%= f.submit "Update" %>
<% end %>

To enable downloads, include links referencing the download action which we will define in the controller:

<%= link_to(@item.attachmentfilename, {:id => @item.id, :action => 'download'}) %>

Controller

Assuming the model is as described above, no changes are required to the controller for upload. For download, include the following action:

def download
    src = Source.find(params[:id])
    send_data(src.attachmentdata, :type => src.attachmenttype, :filename => src.attachmentfilename)
end

Improvements

Although the above is a simple way to upload and download files for storage in the database, it has one flaw. Normally, the edit view of the item displays input fields pre-filled with the current values. The user makes any changes, all values are sent to the server, and all the columns in the database can be re-written. Even if the user makes no changes, the current values are sent and used to update the columns.

However, attachments are not “pre-filled” in the edit view. If the user brings up an edit view and presses “Update” without making any changes, an empty filename is sent to the server, and the attachment=() method removes the attachment. This is unfriendly behavior.

A simple option is to display a Replace checkbox next to the file field in the edit view. By default, the checkbox is unchecked, meaning that the attachment is to remain untouched. If the user wants to replace the attachment, he must check the box as well as select a file. If the user wants to remove the attachment, he can check the box and leave the file field empty. This is accomplished with the following modifications to the view code shown above:

<% form_for(@item, :html => {:multipart => true}) do |f| %>
<p>
  <%= f.check_box(:replace_attachment) %> <label for="item_replace_attachment">Replace Attachment</label>
  <label for="item_attachment">Attachment:</label>
  <%= f.file_field('attachment') %>
</p>
	<%= f.submit "Update" %>
<% end %>

We don’t actually need to store the state of the checkbox in the database. It is purely a UI element. However, because forms are conveniently built from model objects, we should make the checkbox state a virtual attribute of the model class. As mentioned earlier, the checkbox is unchecked by default, so we add the following code to the model class:

  def replace_attachment
    false
  end

Naturally, you would think the body of attachment=() needs to check the state of the checkbox before altering the image columns. However, the attribute setter methods of model objects are called from ActiveRecord::Base.update_attributes(), and I know of no guarantee that they will be called in a specific order. It is therefore safer simply to store the state of the checkbox as well as the contents of the file field in the attribute setter methods, as follows:

  def replace_attachment=(rhs)
    @replace_attachment = rhs
  end

  def attachment=(rhs)
    @attachment = rhs
  end

The decision of whether or not to alter the attachment columns must be delayed to another method:

  def preprocess
    if @replace_attachment == '1'
      if (!@attachment.nil? && !@attachment.kind_of?(String))
        self.attachmentdata = @attachment.read
        self.attachmenttype = @attachment.content_type
        self.attachmentfilename = @attachment.original_filename
      else
        self.attachmentdata = nil
        self.attachmenttype = nil
        self.attachmentfilename = nil
      end
    end
  end

To ensure that the preprocess method is called after the model object’s attributes are updated from the form, but before the model object is saved to the database, simply add the following line to the top of your model class definition:

  before_save :preprocess

Ruby on Rails: How to provide a list of items that can be dragged and dropped on each other

Friday, December 5th, 2008

In this example, the drag-and-drop action results in submission of a form followed by rendering of a new page.

Preparation

No special preparation is required.

Database

It is assumed only that the database includes a table of items with identifiers.

Model

No modifications to models are required.

View

First, there is a JavaScript function to submit the form upon drag-and-drop:

<script>
  function merge(draggable, droppable, event) {
    document.merge.source.value = draggable.id;
    document.merge.destination.value = droppable.id;
    document.merge.submit();
  }
</script>

Then, there is the form referencing the action to perform upon drag-and-drop. The form tag requires a name for the JavaScript function to work:

<% form_tag('/items/merge', :name => 'merge') do -%>
    <input name="source" type="hidden">
    <input name="destination" type="hidden">
<% end -%>

Each item that can be dragged and dropped on should be in a container with a unique id.

<% for item in @items %>
	<div class=”items” id=”item_<%= item.id %>”>
		<%= item.name %>
	</div>
	<%= draggable_element(“item_” + item.id.to_s, :revert => true) %>
	<%= drop_receiving_element(“item_” + item.id.to_s,
	  :onDrop => “function(drag, drop, e) { merge(drag, drop, e); }”,
	  :hoverclass => “hover”) %>
<% end %>

Controller

The controller includes an action that extracts the IDs of the dragged and dropped-on elements:

def merge
    drag = Summary.find(params[:source].split('_')[1])
    target = Summary.find(params[:destination].split('_')[1])

After performing the necessary actions, the method can render or redirect to the necessary page.