Conductr

In the past I mentioned an application that I was building using Rails and the Flickr API. After building it, it turned out to be way too resource intensive for me to host with Dreamhost. Since then, I rewrote the application in PHP and it is a bit faster. It’s still a bit of a CPU hog , and Dreamhost probably isn’t too happy about that, but I wanted to put it up anyway.

The application is video.conductr. It searches Flickr for images with your search terms in their tags, text descriptions, etc. and pulls the images together into a Flash video. The videos can be viewed on site, embedded on other pages, or even downloaded as MP4. No account registration is required, however, it will help you keep track of your movies, registered users’ movies have priority in the queue, and you will get a friendly email notice when your movies have been produced. To keep down on resource use, I am currently producing about 3-6 movies per hour maximum. If the site gets heavy usage, I will put it on a better machine and this limitation will be gone.

“Urban” was a recommended tag today when I visited the Flickr homepage. Here is the video and embedded below.

TODO:

• Fix movie introduction text
• Fix view counter
• Add commenting system

Update: This Prosper API project has been discontinued, and is not functional in it’s current state. Prosper makes huge updates to their API without warning, or even notifying the developer mailing list of the updates. The most recent change totally discontinued HTTP GET/POST access to the API (which this code used) and moved forward with a SOAP only approach. If Prosper opens communication with the developers in the future, I may revisit this project. At this point, I can not invest the time only to have them radically change the API without notice once again.

While working on ProsperK, I wrote an interface to the Prosper API. There was no such project on RubyForge so I decided to put mine up there.

It’s really simple to use, refer to the RDoc for some detailed examples.

Installation

Get the source. require the prosper.rb file.

I put prosper.rb in app/models and it will be loaded with the application.

svn co http://code.conductr.com/svn/prosper/

- or -

Just grab the http://code.conductr.com/svn/prosper/prosper.rb file and require it.

Usage

cd ./prosper
rdoc

Open the doc directory with your browser (or, the index file within). The documentation has usage examples for each method.

My latest project has just been released in Beta. It’s at 9figs.com and is a goal oriented project. The goal is to receive 1 Billion pixels, uploaded from users, to display in our widget.

This is built with Ruby on Rails except 2 files. The scripts that serve up the widget content and handle the resulting click-throughs are PHP. As the project grows, these files will likely receive a very high number of requests so RoR speed issues are a concern. Initially, I wrote them as part of the Rails app and then ported them over to PHP. Since, it has been over 2 years since I’ve touched anything ending with .php, this made the transition easy. PHP still remains extremely easy to pick up. You just have to remember to end all your lines with a semi-colon.

The widget serving file was extremely slow in Ruby. Each request generated several RMagick objects and it would not have taken many requests to pull the server to a halt. Or, require a more advanced (read: expensive) hosting solution. I was impressed by how fast the same logic could execute when written in PHP. I do miss that benefit of PHP.

I was able to build the entire app in 28 hours of development time. Speaking as a freelancer who only codes a few hours a week, productivity is key. So far, I haven’t found a case where I am not more productive in Rails than I would be in PHP (and I’m not interested in learning anything else).

Go check out the project and get the widget up on your site.

While planning any application, there is one constant: I may open things up with an API in the future. Maybe I won’t go production without it, or maybe I put the API on the back burner for a future release. Either way, I try to keep that in mind while building the HTML controllers/views. Nothing new there, I’m sure everyone does this these days, but it helped me stumble upon a API method that I think would be pretty cool.

I, as the API provider, want to give the easiest use to all developers using my service. Naturally, that means encouraging the use of Ruby on Rails. Not entirely, it will be just as easy for non-RoR users to develop with the API but we can do so much more for those on Rails. For example, give them a model with the migration, tests, etc.

So here’s the plan, let’s start by looking at some code examples of how our API should be designed to help out a RoR developer.

Inside a rails app do script/generate scaffold_resource bid end.

Migration

create_table :bids do |t|
    t.column :amount, :decimal, :precision => 9, :scale=>4
    t.column :creation_date, :datetime
    t.column :key, :string
    t.column :last_modified_date, :datetime
    t.column :listing_key, :string
    t.column :member_key, :string
    t.column :minimum_rate, :decimal, :precision => 6, :scale=>5
    t.column :participation_amount, :decimal, :precision => 9, :scale=>4
    t.column :status, :integer
    # Your own stuff
end

Once the developer has this in place, then they just follow some simple logic.

# Make a Bid instance
xml = Net::HTTP.get('api-host.cm', '/bids/1.xml')
hash = Hash.from_xml( xml )
@bid = Bid.new hash['bid']

# Developer saves it
@bid.save

# Developer uses it
puts "Winning" if @bid.status == 2

Easy, right? Let’s make it a one liner out of respect for all you Rubyist ninjas… or pirates? I won’t judge.

@bid = Bid.new( Hash.from_xml( Net::HTTP.get('api-host.cm', '/bids/1.xml') )['bid'] )

Fun stuff. Back to my prospective as an API provider. Wow. I did virtually nothing and people can tap into my API literally within 20 seconds of turning on their computer. That’s right, I timed myself to get that statistic and it assumes you have rails installed and ready to go.

The API’s XML file remains easy for any developer to use. It’s important to note that all of this works because the API service runs render :xml => @bid.to_xml such as with the generated REST scaffolding controllers. Rendering the exact XML format that will be recognized by the developers app. If you are not running Rails on your API service, provide an endpoint with the XML formated for RoRs developers. I don’t see this being done anytime soon, but you never know with the growth of RoR thus far.

Thought: For all I know this is what Action Web Services is all about. I’ve never read up on it so I don’t know anything about it. Not really investigating it either shrug.

One Dollar Give Away! To the first person who correctly guess what API was used in the above examples. Just leave your guess as a comment and I’ll send you a buck via Paypal.

Sometimes, usually in development, I find it useful to have my Javascripts printed inline rather than using the <script src="/path/to"> tags that are generated by javascript_include_tag.

With javascript_inline_tag you can simply declare your Javascripts just like you would with javascript_include_tag and it will spit out the script into your view/layout.

Put it where you like, but I chose to keep with my application_helper.rb.

javascript_inline_tag

module ApplicationHelper
  # ..... your app helpers, per usual
end

module ActionView::Helpers::AssetTagHelper
  def javascript_inline_tag(*sources)
    if sources.include?(:defaults) 
      sources = sources[0..(sources.index(:defaults))] + 
      @@javascript_default_sources.dup + 
      sources[(sources.index(:defaults) + 1)..sources.length]
      sources.delete(:defaults) 
      sources << "application" if defined?(RAILS_ROOT) && File.exists?("#{RAILS_ROOT}/public/javascripts/application.js") 
    end

    sources.collect do |source|
      source = javascript_path(source).sub(/\?\d+/,'')
      contents = ''
      File.open("#{RAILS_ROOT}/public#{source}").each do |line|
        contents << line
      end
      javascript_tag(contents)
    end.join("\n")
  end
end

Sample Usage

<%= javascript_inline_tag :defaults,'niftycube' %>

Notes

1. You can not pass an options hash.
2. For production, I’ll just say, consider caching (browser&server) and page size.
3. Nifty Corners Cube is an awesome script that I recently discovered.
4. You may find render :file => '' to work for you, I just thought it wasn’t flexible enough for this situation.

This blog was started with the purpose of writing mostly about code. I have yet to do so until now. Here we go.

My current project, similar to AmieStreet, requires the uploading of files (eg mp3’s) that are linked to a record in the database. While the database stores all of the information about the song (eg Title) the mp3 itself is stored on the filesystem. If, for whatever reason, we delete the record from the database we should also remove the associated mp3 from the filesystem.

Rails makes this incredibly easy for us. As the convention goes, I will call the delete a destroy from here on out.

Version 1

def after_destroy
  if FileTest.exist?(source)
    FileUtils.rm self.source
  end
  if FileTest.exist?(clip)
    FileUtils.rm clip
  end
end

This was my original code. I have 2 mp3 files, the source (entire song) and the clip (sample from song), each file should be removed from the filesystem after the record is destroyed. Conveniently, the after_destroy callback lets me run my logic after the destroy.

Since I have 2 files, I test to see if each exists if so I remove it. Note: the source and clip methods just return full paths to the files, like RAILS_ROOT+"/path/to/file.mp3".

It could be so much better, let’s DRY things up a bit.

Version 2

def after_destroy
  removables [source, clip]
end
def removables(paths = [])
  paths.each{|p| FileUtils.rm p if FileTest.exist?p }
end

Here I wanted to make a method that I could feed multiple file paths and it would execute the logic. So I made the removables method, which steps through each path in the paths array and removes the file if it exists. This is pretty clean, but it gets better.

Version 3, Final

def after_destroy
  FileUtils.rm_f [source, clip]
end

Upon closer inspection, I noticed that the FileUtils.rm* methods actually accept a list of files to delete. And, with the force option the method does not throw errors if the file is not there. I was so used to always having to check whether a file existed, in order to avoid errors that come up if I tried to delete a nonexistent file, that I did not even think this could be done. Of course, with Ruby, the solution is simple.

Conclusion

This is a perfect example of how I am trying to think like a Rubyist. From V1 to V3, Final took me about 5 minutes so I feel that I am getting a good deal better … and oh so addicted.

Well I have finally completed my first big App with Ruby on Rails. I learned a lot, had a good bit of fun and grew even more addicted to the framework. Ohh, and you must be wondering “Where’s the APP!” It’s a bit of a sad story really. The app is built around the Flickr API and does wonderful things with photos that, as far as I can tell, have not been done before. I found out why, it is damn hard on the servers. My shared account on DreamHost certainly can not support the load and continues to kill off all my processes. So, unfortunately, I am not able to release the App at this time. I simply do not have the server to run it. Hopefully, one day this limitation will not be a problem.

Keep on truckin’ with more development projects. I’ve been trolling through my brain, reading over all the web 2.0 blogs, and generally looking for some inspiration on a project to undertake. A couple of ideas came up.

Dating sites have been done over a billion times, but that’s okay. I read a blog post mentioning the success of PlentyOfFish with their streamlined operations generating revenue from adwords ($10K a day?). POF has an awful user experience; the layout is just plain crap. But hey, you get what you pay for. For a dating service to be useful it has to capture market share which means it has to advertise (paid usually, maybe dessemenated through the masses via blogging would be a good way?) The best/stupidest idea came to me when thinking of ways to finance such an advertising blitz. Note: this is the riskiest financing method around second only to loan sharking which is harmful to your health, this method is extremely harmful to your wealth.

1. Develop a killer App and make it free. Obviously, once users visit your site they will never want to use another dating service again. That’s how good it is.
2. Sell short the stocks of your competitors (Match, Y!, eHarmony, etc … I assume these firms are public, I have not researched whether they are)
3. Use the income received from the sale of stock options to finance a massive marketing campaign
4. You’re filthy rich, you stole everyone’s market share, their earning tanked and now their stock is worthless so the option will not be executed. You can find ways to get your own $10K a day via adwords or monetize this large audience via other routes.

Like I said, this is stupid, I know it. Just a day dream while at the dog park earlier today, but I thought it was a pretty creative financing method though – from a perspective that risk is of no concern.

Okay, so if dating services are out, what is my next app? Amie.st is a great we application for fair music pricing with no DRM and it supports independent artist to a large degree. I’ve decided to make my own version of their site. Unlike my dating site discussion above, I do not want to develop a competing product. I simply want to use their domain model to enhance my knowledge of Ruby on Rails. After all, that’s what it’s all about.

I’ve wasted a pretty good amount of my coding time lately because of some oversights made while programming. To be expected when learning a new language and framework I suppose (Ruby on Rails). No, no, no—that’s the easy excuse. Most of these errors were not related to Ruby or Rails, just me being careless.

The RMagick debacle

This one cost me the better part of an entire day. You’re going to laugh. As I said in the previous post, this RMagick thing had me going mad and running in circles. I ended up trying to contact the DreamHost support to see if they could help me out. I know the run around, I used to give it when I was running my web hosting business back in the ‘03, they will tell me they can not help me debug my program and that RMagick is working for everyone else, so I must be doing something wrong. I knew that, I was hoping for help as to what I was doing wrong.

Fast-forward to the following day. I had to go to work and effectively clear my mind from everything Ruby, Rails, RMagick and DreamHost. The next day, I had class but when I came home (I just then received a reply from DH, telling me it’s my problem) I decided to give it a quick glance. This is where I acknowledge that sometimes when I get stuck on something, I like to walk away, take a break, walk the dog, eat a sandwich, etc. I can rethink the problem and come back with a new approach. So, during my 2 day hiatus I was able to clear my mind from the problem.

When I returned to the code, my first thought was; “concentrate on what was different in my local environment as opposed to the DreamHost environment and try to identify the problem”.

The Problem

Windows + require 'rmagick' => success!!!

Debian + require 'rmagick' => failure!!!

The moment I thought about it that way, it clicked. I needed to use require 'RMagick'! And of course, it worked. Windows is case insensitive while Linux is case sensitive. Go ahead, laugh, I told you that you would.

Answering my own questions

This is a story about paths. This way, that way, any way… but not there? Most of the story unfolded in a one way conversation with myself on Dmytro Shteflyuk’s awesome post about using ffmpeg/mencoder via Ruby. What that side of the story doesn’t tell is that I spent about an hour wrestling this issue before I decided to ask Dmytro for help.

For the most part, I was using his instructions to encode some video through my app. By putting the code in a model, I was trying to load/write the images/videos from my app/models directory.

This is another trend that I have noticed that helps me troubleshoot problems. The simple act of writing an explanation of the problem helps clarity it and sometimes often times I think of the solution mid-explanation and the post never even gets submitted.

In this case, I did submit the post. Afterwards, I proofread it. While proofreading, I got the idea that maybe it was a path issue. Sure enough, as mentioned in the reply to myself, I edited the code to include the images/videos absolute paths and problem solved.

Total waste of time? (What I learned)

Be more thorough in the initial draft of my code. Developing on Windows can make me sloppy in some areas. Troubleshooting should not continue into frustration. It’s simply not productive and may compound the problem. Take a deep breath, clear your mind, come back with a fresh approach. Put the problem in writing, it will give you another way to look at it and will open up a little creative right brainedness that may be needed.