Archive for the 'Ramblings' Category

The tech industry still fails to understand basic business.

Friday, September 4th, 2009

Jason Fried, founder of 37signals recently posted a reality-check for the tech industry.

This pattern — “success” based on forecasted future success instead of current success — shows up all over the tech-business press.

He goes on to questions the false measures of success that so many companies use. Things like page views, or new customers, only matter if there is a clear deliberate way to gather a return on investment. He has a very good take on the phenomenon.
Read the rest of this entry »

I know it’s a tough job market but…

Wednesday, February 25th, 2009

It seems like every day brings a new round of layoff announcements. Companies, large and small, are having to cut back, save cash, and stop growing. In this environment, any new hires need to be as effective as possible. It’s all about the bang for the buck. This is a time where generalists may have an advantage – wearing many hats makes you more cost effective.

Given this, I’m seeing several job postings that are asking for an awful lot. It is to be expected, and makes sense from the companies point of view. There are, however, times where this “hire overloading” borders on the absurd. Take the following:

*Senior Linux/Unix Developer/Test Engineer*
*Summary:*
Installs and configures clusters of Linux based application and database servers. Drafts and executes test plans of Linux related software on clusters. Experience with Linux application cluster design, administration and tuning (including san) required. Experience with virtualization technologies required.
*Details:*
Strong software development skills in multi-tiered and distributed environments using iterative development process, including 5+ years of advanced programming experience

* Application performance testing plan drafting and execution.
* Experience with usage and customization of open source application performance test tools.
* Multiple Programming Languages: C, C++, Perl, Python, Cold Fusion, JAVA
* 4 year technical degree or higher at an accredited institution.
* Linux Cluster and cluster storage design, configuration and tuning.
* Linux Kernel customization and compilation
* Databases: Mysql, Postgresql, Oracle
* Multiple Operating Systems: Linux, FreeBSD, Windows, etc.
* Experience with multiple virtualization technologies: Xen, VMWare, KVM,
* Excellent analytical and problem solving skills; with the ability to analyze business processes and create application models utilizing project-management standards
* Strong verbal and written communication skills and ability to work effectively both independently and as part of a cross functional team.
* Telecom and/or internet domain knowledge and experience with solid experience with eBusiness processes and/or back-end applications

So, here’s what I read into this request. This company is looking for a QA engineer who has been a mid/high level software engineer, with linux administration experience, fairly advanced enterprise hosting experience, and project management experience. The hiring manager is asking for a chocolate cake that has just the right amount of fish sauce and onions.

This is at least two different career tracks. Any software engineer that has professional experience setting up virtualization clusters and SAN is not likely to be very skilled in one or both areas.

Hiring managers that cram so many diverse requirements into a single job posting really bug me. If you want a generalist, then ask for a generalist. Asking for such specific and wildly different requirements does nothing to increase your chances of finding a quality candidate. It just turns away those quality generalists that are capable of learning your specific system. If you need someone who can set up test environments using a virtualization system, then either train one of your test engineers – if muddling through is acceptable – or hire a sysadmin who has experience with virtualization. If you need someone who can write test harnesses, then hire someone who knows how to write test harnesses and exercise programming interfaces. If the harnesses require an obscure language, then asking for familiarity in that language makes sense. Remember, a good engineer can learn whatever elements are specific to your system. If you want someone who can analyze your current business processes, and help to reorganize them, don’t expect the person to also be down in the code – those are two totally different disciplines. “big picture” people, who can give quality feedback on business process tend to miss the details necessary for writing a good testing suite, and vice versa.

I feel bad for any technical recruiter who sends out job requests like this because though they are just the messenger, they look foolish. The hiring manager looks just plain silly. This was a request that was likely written by a team of engineers. They all looked at what they do, and tried to “fill in the gaps”. Bad practice! Anybody who actually fits that request is either dishonest (resume padding) or probably doesn’t require the depth of knowledge that the team is hoping for. The team, or hiring manager, should request a generalist, who has experience in the main area of the job request. Then during the interview process, explore the candidates willingness and ability to learn. Perhaps mention some of the systems that are required to do the job. Focus on concepts and approaches, not on specific technologies or processes.

I know it’s a tough job market, but come on. A little sanity please.

-Joe

Prospering with ruby vs. haskell

Wednesday, March 26th, 2008

As previously mentioned, I am learning haskell. In that endeavor, I am trying to cross the chasm from “tutorial following” to actual real projects (albeit, very small projects). My latest project is a simple simulator for my prosper.com account. For those who don’t know, prosper.com allows people to make smallish loans to each other with terms of three year repayment. Money amounts range from $50 to $25,000, and interest rates are negotiated in an auction. As a lender, I want to know what return on investment I am likely to receive given various scenarios.

Now on to the show

My first stab at the simulator was done in ruby. This gives me a working model, and the ability to compare and contrast some of the design requirements that functional programming, and specifically haskell will impose.

Ruby

First I needed a function to generate random rates, simulating the auction style rate negotiation.
def get_new_rate
   return MIN_RATE + rand(RATE_WINDOW)
end

where MIN_RATE is defined as the minimum rate I am willing to lend at (8.0%), and RATE_WINDOW is defined as the spread between my minimum rate, and the highest rate I am interested in lending at (20.0%).

Second off, I needed a function to generate a number of loans given a certain account balance.
def add_loans(loans, account_balance)
  new_loan_count = account_balance / INITIAL_PRINCIPLE
  new_loan_count.to_i.times do
    rate = get_new_rate
    loans << {:principle => INITIAL_PRINCIPLE, :rate => rate, :min_payment => calc_minimum_payment(INITIAL_PRINCIPLE,rate)}
    account_balance -= INITIAL_PRINCIPLE
  end
  account_balance
end

where INITIAL_PRINCIPLE is set to the amount that I am willing to lend ($50) in each loan. (Read this for an explanation of why I only lend $50.)
This function calculates how many loans I can generate from the given account balance, then creates each one. The new loans are appended to the collection of loans that was passed in as an argument. The calc_minimum_payment function simply determines what the minimum payment will be each month.

I then needed a function that would calculate the payment on the loan – particularly at the end of the loan when the payment may be less than the minimum payment.
def calc_payment(loan, months=1)
  if loan[:principle] < loan[:min_payment]
    payment = loan[:principle]
  loan[:principle] = 0
  return payment
  end
  
  interest = loan[:principle] * loan[:rate] / 100.0 / 12
  loan[:principle] -= loan[:min_payment] - interest
  return loan[:min_payment]
end

Given these functions, I can now create the simulation
account_balance = ARGV[0].to_f if ARGV[0]
account_balance ||= 0.0
  
monthly_deposit = ARGV[1].to_f if ARGV[1]
monthly_deposit ||= 100.0
  
number_of_years = ARGV[2].to_i if ARGV[2]
number_of_years ||= 1

First grab the scenario parameters from the cmdline. monthly_deposit is how much money to add to the account balance each month (in addition to the payments from the outstanding loans)

loans = []
for i in (1..number_of_years*12)
  old_loans = loans.size
  account_balance = add_loans loans, account_balance
  print "Month #{i}\n"
  print "Number of loans: #{loans.size} (#{loans.size - old_loans})\n"
  print "Average Rate: #{calc_average(loans.collect {|i| i[:rate]})}\n"
  income= loans.inject(0) {|bal, i| bal + calc_payment(i)}
  print "Account balance: #{account_balance}\n"
  print "Income: #{income}\n"
  account_balance += income + monthly_deposit
  print "Account value: #{loans.collect {|i| i[:principle]}.inject(account_balance) {|sum, i| sum + i}}\n"
  print "\n"
  loans.delete_if {|item| item[:principle] == 0}
end

Then run the simulation, and print out various statistics for each month of the simulation.

So that’s the simulator in ruby. It is not “perfectly optimized” for ruby, because I wanted to keep it somewhat close to the structure that I would use for haskell. See the link below for full source.

Haskell

I tried to keep the architecture of the haskell version as close to the ruby approach as was possible. As a consequence, many haskell people may look at this and balk. My apologies in advance.
First I needed some constants and a struct to keep the relevant loan data in
minRate = 8.0
maxRate = 20.0
initialPrinciple = 50.0
periods = 36
  
data Loan = Loan {principle :: Double, rate :: Double, minPayment :: Double}

Then comes the function used to simulate the rate auctions
-- Generate a random rate within the "rate window"
getNewRate :: IO Double
getNewRate = do randomRIO (minRate, maxRate)

Where I calculate some random number between the minRate and maxRate. Note the type – IO Double. For all you non-haskellites, that means that the function will be using a monad inside of itself. In this case, the monad is randomRIO. The monad allows you to call randomRIO multiple times, and get different numbers each time. Useful that!

Then I have the loan creation functions

-- figure out what the minimum payment will be on a given loan
calcMinimumPayment :: Double -> Double -> Double
calcMinimumPayment p i = (r * p *(1+r)^periods) / ((1+r)^periods - 1)
                         where r = i / 12.0 / 100
 
-- create a new loan
newLoan :: Double -> IO Loan
newLoan p = do
          i <- getNewRate
          let m = calcMinimumPayment p i
          return (Loan p i m)

As in the ruby version, create a new loan, then populate the structure with the rate and minimum payment. Note the type for calcMinimumPayment doesn’t specify IO… that means this is a “clean function” and can be called anywhere. newLoan however is a monad function – because it calls getNewRate. Since newLoan uses a monad function, it has to return a monad itself.

Here’s where things had to deviate from how I did them in ruby. Since haskell has immutable values, I couldn’t modify the loans. I had to create new loans, and collect them into a new structure. Here is where the new loan is created, given the state of the provided loan.
-- Given a loan, make a payment and create a new loan with the remaining principle
calcPayment :: Loan -> Loan
calcPayment l = if principle l > minPayment l
      then Loan (principle l - p) (rate l) (minPayment l)
      else Loan 0 (rate l) (principle l) -- mark this as the last payment
      where
        i = (principle l) * (rate l / 100 / 12)
        p = minPayment l - i

Again, here’s a “pure function”. It can be called anywhere, and any time.

Now, given an account balance, create as many loans as I can, and return them as a collection of Loans.
-- Take the current account balance, and make as many loans as possible from it
makeLoans :: Double -> IO [Loan]
makeLoans bal = if bal >= initialPrinciple
      then do
        l <- newLoan initialPrinciple
        ls <- makeLoans (bal - initialPrinciple)
        return ([l] ++ ls)
      else
        return []

Note the recursive call to continue building the list. I am finding that functional programming relies on recursion a lot more than OOP.

This is another portion of code where I had to deviate. Here is where I actually parse the passed in loans, and return a new array of updated loans, and a new account balance. This is probably the most un-haskellish function of the group, and definitely needs some work.
-- make payments on the given loans, and return the updated loans, and resulting total payments
collectPayments :: [Loan] -> ([Loan], Double)
collectPayments loans = (filteredLoans, payments)
      where
        clearStaleLoans = filter (\x -> minPayment x > 0) -- remove any loans that have been fully paid back
        filteredLoans = clearStaleLoans (map calcPayment loans= sum (map minPayment filteredLoans)

Then a function that runs through each iteration of the simulation – i.e. each month. This has to be its own function so that it can recursively call itself to continue the simulation.
-- run through a loan scenario, reinvesting returns for 'term' months. Print out various statistics on the account
run :: Double -> [Loan] -> Int -> Double -> IO Double
run startingBalance loans term monthlyDeposit = if term <= 0
        then return startingBalance
      else do
        l <- makeLoans startingBalance
                              let (newLoans, newPayments) = collectPayments (loans ++ l)
        let newPrinciple = (initialPrinciple * fromIntegral (length l))
        let newBalance = (startingBalance - newPrinciple + newPayments)
        let loanValue = sum (map principle newLoans)
        let averageRate = (sum (map rate newLoans)) / fromIntegral (length newLoans)
        putStr $ unlines ["Term: " ++ show term, "Loan count: " ++ show (length newLoans), "Average Rate: " ++ show averageRate, "Loan Value: " ++ show loanValue, "New balance: " ++ show newBalance, "New Principle: " ++ show newPrinciple, "New Payments: " ++ show newPayments,"---------"]
        bal <- (run (newBalance + monthlyDeposit) newLoans (term-1) monthlyDeposit)
        return bal

Note how, although run is a monad function, a majority of its processing is non-monadic. In theory each of those ‘let’ statements could run in parallel.

Finally a “main” function to get the works rolling
main :: IO ()
main = do
        args <- getArgs
        let accountBalance = if(length args > 0) then read (args !! 0) :: Double else 300.0
        let monthlyDeposit = if(length args > 1) then read (args !! 1) :: Double else 100.0
        let term = if(length args > 2) then read (args !! 2) :: Int else 1
        putStr $ unlines ["Starting balance: " ++ show accountBalance, "Starting run", "----------"]
        endingBalance <- run accountBalance [] (term * 12) monthlyDeposit
        putStr $ unlines ["Ending Balance: " ++ show endingBalance]

Summing it up

Well, comparing and contrasting these two scripts is giving me a new appreciation for both languages. Each script could be refined to better match its underlying language, but the goal was to keep the code as close as possible to maximize comparability. If enough people ask, perhaps I’ll refine each script.
Hopefully some comparison of the two scripts will help another budding haskell developer wrap their head around this powerful, but oh so different language.

Here is the full ruby source code – prosper.rb (right click – “Save As”)
Here is the full haskell source code – prosper.hs (right click – “Save As”)

-Joe

So what’s for lunch?

Tuesday, February 26th, 2008

A couple of months ago, when ruby on rails 2.0 came out, I started a project to see if things had really changed all that much. I wanted something that was easy, but useful to me. At work, we had been discussing the pressing need for an easy to use listing of lunch places around our area. So, I decided, not only would I pick up rails 2.0, but I would see how hard it was to do a google maps mashup. Well, after a few days, I had the basics of a site up and running. Then of course, work got busy busy. (Something about launching our new website at work.) Well, in order to start benefiting from at least some of my effort, I spent the last weekend getting everything set up to deploy to my server, so without further ado, I present Lunchspot! Now, I wouldn’t call this a completed effort. In fact, it is only a faint echo of what I had hoped to have going by this time.

So why release it now? Well, because it is still quite useful. You can already submit your favorite lunch spots, comment on them, and see where they are on the map. Soon, I’ll have directions, ratings, sample menus, and “send to a friend” links. But the biggest reason to put it up, and then post about it, is to remind me to work on it, and solicit any help on things like design/layout and graphics (two of my weak areas).

So what is for lunch anyway?