One-liners: Deformalize

Recently I needed quick hack for following situation. Given an input like this:

A 1
A 2
B 3
C 4
C 5
C 6
C 7
D 8

I wanted to generate following:

text A -p 1 -p 2
text B -p 3
text C -p 4 -p 5 -p 6 -p 7
text D -p 8

After some thought I have generated it with this awk one-liner.

cat file | awk '{ if (X==$1) { printf " -p "$2 ; } else { printf "\ntext " $1 " -p "$2 ; X=$1 } }'

What would you do? Is there an easier way ?

ENV is not a Hash but Object in Ruby

Recently I hit this funny problem in ruby, when trying to merge options to ENV. Although ENV supports most of Hash functions, it's not really Hash instance:

irb(main):001:0> ENV.class
=> Object
irb(main):002:0> env = { 'X'=>1 }
=> {"X"=>1}

irb(main):003:0> env.class
=> Hash

There are some useful methods missing from ENV like merge!

irb(main):004:0> ENV.merge(env)
NoMethodError: undefined method `merge' for #<object:0xb7f54814>
       from (irb):4

Workaround is simple enough, merge! hash manualy:

irb(main):012:0> env.each { |k,v| ENV[k.to_s]=v.to_s }
=> {"X"=>1}
irb(main):013:0> ENV["X"]
=> "1"

Work around core library class mocks/stubs in ruby

Just a short post about mocking/stubbing your Ruby functions. Let's have a class like this:

class ConfigurableClass
  attr_reader :option
  def configure_from_file(file)
    json = JSON(IO.read(file))
    @option = json['my']['option']
  end
end

One way to test this would be to mock IO.read with Mocha

describe ConfigurableClass do
 it "can configure from json" do
   data =  '{ "my": { "option" : "hello" } }'
     IO.any_instance.expect(:read).with("x.json").returns(data)
     cc = ConfigurableClass.new
     cc.configure_from_file("x.json")
     cc.option.should eq "hello"
   end
end

This is tricky if your code does IO.read anywhere else which you obviously don't want to mock. However with some refactoring code is much cleaner.

class ConfigurableClass
  attr_reader :option
  def read_config_file(file)
    IO.read(file)
  end

  def configure_from_file(file)
    json = JSON(read_config_file)
    @option = json['my']['option']
  end
end

Then in the rspec it's easy to mock only function specific to ConfigurableClass.

describe ConfigurableClass do
  it "can configure from json" do
    data =  '{ "my": { "option" : "hello" } }'
    ConfigurableClass.any_instance.expect(read_config_file).with("x.json").returns(data)
    cc = ConfigurableClass.new
    cc.configure_from_file("x.json")
    cc.option.should eq "hello"
end

GZIP/BZIP2 progress indicator

Have you ever needed to figure out how long gzip/bzip compression would take for the file and found it difficult to present it to the user ? Especially for scripts dealing with large files, this would be one of the major usability issues. Well there's a solution for the problem with the use of tool called pipe viewer. You can find more details on how to use it on it's home page, but I'll show you how I've used it to show progress for bzip2 compression of large files.

filename="file_to_bzip2"
size=`stat -L "$filename" -c "%s"`
pv -s $size "$filename" | bzip2 -c > "$filename.bz2"

Generates following output:

# output
2.75GB 0:32:47 [ 1.5MB/s] [===================>     ] 61% ETA 0:20:16