Author Archives: joe

About joe

.NET developer and board game geek located in the greater Atlanta region.

Scraping and Saving Flickr Images with Ruby

I’ve been playing around a bit more with the black arts of spidering and scraping in Ruby and I’m still amazed by how easy it is to do. For fun I whipped up a little script that will spider a Flickr photostream and download all the images.

Flickr provides a wonderful api and there’s even a great Ruby interface for it, so this script is entirely futile. But it was fun and educational.

Usage

ruby init.rb yourusername /location/to/save

Download it!

Finding and Fixing Broken Images with Ruby

A family members was having a problem with some mixed up image names on a static html site. I could have fixed it manually in a few shakes, but that’s no fun. Instead I used hpricot to scrape, open-uri to test for broken-ness, Find to search and some good old fashion regex to correct.

This was my first time messing around with hpricot and I found it to be powerful and easy to use, two thumbs up. I foresee some scraping and spidering posts in the near future.

On to the code:

My final script was a bit hairy so I broke out the bit I used to find the broken images.

If you run the script it’ll print the offending paths to screen:

ruby image_scanner.rb http://site.com/busted.html

Or you can call the get_broken_images method to get an array back:

require 'image_scanner'
scanner = Image_Scanner.new
broken_images = scanner.get_broken_images "http://site.com/busted.html"

In case you’re interested, I’ve also uploaded the full code that I used to search for and correct the images although it’s implementation specific, riddled with lazy and is poorly tested. Read the disclaimer!

Just run it and be amazed!

ruby image_scanner.rb http://site.com/busted.html /media_folder /busted.html /fixed.html

Download only the broken image scanner
Download the full script

Project Euler: Problem 11 in Ruby

This one was reminiscent of problem #8. Very straight-forward. I got the wrong answer the first time because I forgot to count BOTH diagonals. Silly me.

Problem #11

What is the greatest product of four adjacent numbers in any direction (up, down, left, right, or diagonally) in the 20×20 grid?

grid = "08 02 22 97 38 15 00 40 00 75 04 05 07 78 52 12 50 77 91 08
49 49 99 40 17 81 18 57 60 87 17 40 98 43 69 48 04 56 62 00
81 49 31 73 55 79 14 29 93 71 40 67 53 88 30 03 49 13 36 65
52 70 95 23 04 60 11 42 69 24 68 56 01 32 56 71 37 02 36 91
22 31 16 71 51 67 63 89 41 92 36 54 22 40 40 28 66 33 13 80
24 47 32 60 99 03 45 02 44 75 33 53 78 36 84 20 35 17 12 50
32 98 81 28 64 23 67 10 26 38 40 67 59 54 70 66 18 38 64 70
67 26 20 68 02 62 12 20 95 63 94 39 63 08 40 91 66 49 94 21
24 55 58 05 66 73 99 26 97 17 78 78 96 83 14 88 34 89 63 72
21 36 23 09 75 00 76 44 20 45 35 14 00 61 33 97 34 31 33 95
78 17 53 28 22 75 31 67 15 94 03 80 04 62 16 14 09 53 56 92
16 39 05 42 96 35 31 47 55 58 88 24 00 17 54 24 36 29 85 57
86 56 00 48 35 71 89 07 05 44 44 37 44 60 21 58 51 54 17 58
19 80 81 68 05 94 47 69 28 73 92 13 86 52 17 77 04 89 55 40
04 52 08 83 97 35 99 16 07 97 57 32 16 26 26 79 33 27 98 66
88 36 68 87 57 62 20 72 03 46 33 67 46 55 12 32 63 93 53 69
04 42 16 73 38 25 39 11 24 94 72 18 08 46 29 32 40 62 76 36
20 69 36 41 72 30 23 88 34 62 99 69 82 67 59 85 74 04 36 16
20 73 35 29 78 31 90 01 74 31 49 71 48 86 81 16 23 57 05 54
01 70 54 71 83 51 54 69 16 92 33 48 61 43 52 01 89 19 67 48".split ' '

directions = 4
products   = Array.new grid.size * directions, 1
size       = Math.sqrt(grid.size).to_i
seq        = directions
n          = 0

def fetch grid, size, i, j
  if i >= size || j >= size
    return 0
  end
  grid[i * size + j].to_i
end

size.times do |i|
  size.times do |j|
    seq.times do |k|
      products[n + 0] *= fetch grid, size, i, j + k
      products[n + 1] *= fetch grid, size, i + k, j
      products[n + 2] *= fetch grid, size, i + k, j + k
      products[n + 3] *= fetch grid, size, i + k, j - k
    end
    n += directions
  end
end

puts products.sort.pop

Project Euler: Problem 10 in Ruby

This one was a breeze after figuring out #7. Straight-forward and fugly. Shush.

Problem #10

The sum of the primes below 10 is 2 + 3 + 5 + 7 = 17.

Find the sum of all the primes below two million.

#The sum of the primes below 10 is 2 + 3 + 5 + 7 = 17.

#Find the sum of all the primes below two million.

$max   = 2_000_000
$stack = [2,3]
$total = 5
$store = Math.sqrt($max).floor
n      = 1

def is_prime n
  max = Math.sqrt(n).floor
  $stack.each do |i|
    if i > max
      return true
    end
    if n % i == 0
      return false
    end
  end
  true
end

def store n
  if $stack.length < $store
    $stack << n
  end
  if n >= $max
    puts $total
    exit
  else
    $total += n
  end
end

while true do

  if is_prime n += 4
    store n
  end

  if is_prime n += 2
    store n
  end

end

puts $total

Project Euler: Problem 9 in Ruby

I spent way too long trying “clever” solutions that didn’t work or took twice as long as my eventual brute force solution. Screw it.

Problem #9

A Pythagorean triplet is a set of three natural numbers, a < b < c, for which, a^(2) + b^(2) = c^(2) For example, 3^(2) + 4^(2) = 9 + 16 = 25 = 5^(2). There exists exactly one Pythagorean triplet for which a + b + c = 1000. Find the product abc.

target = 1000
min    = Math.sqrt(target).to_i
max    = target / 2

(min..max).each do |a|
  ((a + 1)..max).each do |b|
    c   = Math.sqrt(a**2 + b**2)
    sum = a + b + c
    if sum > target
      break
    elsif sum == target && (a**2 + b**2 == c**2)
      puts a * b * c
      break
    end 
  end
end

Olathe’s is pretty:

 n = 1000
a = (1..n / 2).to_a.find { |a|
  (n * (n / 2 - a) % (n - a)).zero?
}
b = n * (n / 2 - a) / (n - a)
puts "Product is #{a * b * (n - a - b)}."

Project Euler: Problem 8 in Ruby

Problem 8 is to Problem 7 as Problem 6 is to Problem. My original plan involved storing the current sum of 5 numbers and then simply dividing by $big_n[i – 1] and multiplying by $big_n[i + 5], but the zeros made it a bit messy so I thought I’d just give the straight-forward an approach a go to see what I was up against. And it ran correctly, and more importantly quickly so I let it be.

Problem #8

Find the greatest product of five consecutive digits in the 1000-digit number.

73167176531330624919225119674426574742355349194934
96983520312774506326239578318016984801869478851843
85861560789112949495459501737958331952853208805511
12540698747158523863050715693290963295227443043557
66896648950445244523161731856403098711121722383113
62229893423380308135336276614282806444486645238749
30358907296290491560440772390713810515859307960866
70172427121883998797908792274921901699720888093776
65727333001053367881220235421809751254540594752243
52584907711670556013604839586446706324415722155397
53697817977846174064955149290862569321978468622482
83972241375657056057490261407972968652414535100474
82166370484403199890008895243450658541227588666881
16427171479924442928230863465674813919123162824586
17866458359124566529476545682848912883142607690042
24219022671055626321111109370544217506941658960408
07198403850962455444362981230987879927244284909188
84580156166097919133875499200524063689912560717606
05886116467109405077541002256983155200055935729725
71636269561882670428252483600823257530420752963450

Note: Global variables ahead, proceed with caution.

$big_n ="731671765313306249192251196744265747423553
491949349698352031277450632623957831801698480186947
885184385861560789112949495459501737958331952853208
805511125406987471585238630507156932909632952274430
435576689664895044524452316173185640309871112172238
311362229893423380308135336276614282806444486645238
749303589072962904915604407723907138105158593079608
667017242712188399879790879227492190169972088809377
665727333001053367881220235421809751254540594752243
525849077116705560136048395864467063244157221553975
369781797784617406495514929086256932197846862248283
972241375657056057490261407972968652414535100474821
663704844031998900088952434506585412275886668811642
717147992444292823086346567481391912316282458617866
458359124566529476545682848912883142607690042242190
226710556263211111093705442175069416589604080719840
385096245544436298123098787992724428490918884580156
166097919133875499200524063689912560717606058861164
671094050775410022569831552000559357297257163626956
1882670428252483600823257530420752963450".split('')

$stack = []
$chars = 5
max    = 9

def find_product start
  total = 1
  $chars.times do |i|
    total *= $big_n[start + i].to_i
  end
  total
end

$chars.upto($big_n.length - $chars) do |i|
  current = find_product i
  max = [current,max].max
end

puts max

Blessed Olathe had a Ruby-er Solution:

class String
  def max_digit_product length
    # Get each string of 'length' or more nonzero digits.
    strings = scan(/((?:W*[1-9A-Z]){#{length},})/i).map { |string|
      string.first.scan(/([1-9A-Z])/i).map { |digit|
        digit.first.to_i 36
      }
    }
 
    # Find the maximum product of 'length' digits.
    max = 0
    strings.each { |string|
      product = string[0, length].inject(1) { |prod, digit|
        prod * digit
      }
      max = [max, product].max
      (0...(string.length - length)).each { |index|
        product = product / string[index] * string[index + length]
        max = [max, product].max
      }
    }
    max
  end
end

Breakout Knock Off in Ruby with Gosu

I Just finished up a Breakout knock off using Gosu and some basic extension classes I’ve been tinkering around with. There’s only “level” and I’m certain the paddle / ball bounce routine is totally incorrect but I’ve had enough fun with it for tonight. The samples were lovingly provided by http://freesoundfiles.tintagel.net/Audio/, but those swanky graphics were all me.

Download the Code!

Project Euler: Problem 7 in Ruby

It took a couple hours of a couple days for me to come up with the solution to Problem #7, heck of a time. Looking over it now, the solution seems laughably obvious to me given the problems leading up to it. I’m going to optimistically take this as a sign that I’m learning…or something.

When I first began the problem I spent a lot of time trying to wheel and sieve my data set while totally ignoring my redundant method of testing primality!. Now I’m actually utilizing the data I’d already computed to help me speed up my algorithm at the piddly expense of storing a couple thousand integers.

NOTE: I realize that it isn’t actually necessary to store all 10,001 numbers, but it’s my program and I’ll do what I want!

Problem #7

By listing the first six prime numbers: 2, 3, 5, 7, 11, and 13, we can see that the 6^(th) prime is 13.

What is the 10001^(st) prime number?

target = 10001
stack  = [2,3]
n      = 1

def is_prime stack, n
  max_check = Math.sqrt(n).floor
  stack.each do |i|
    if i > max_check
      return true
    end
    if n % i == 0
      return false
    end
  end
  true
end

while true do
  
  if is_prime stack, n += 4
    stack << n
  end

  if is_prime stack, n += 2
    stack << n
  end

  if stack.length >= target
    break
  end
end

puts stack[target - 1]

I’d show you Olathe’s, but he cheated again.

Great Programming Quotes

Ran into a list of great programming quotes @ stackoverflow and thought I’d share. Some of my favorites:

The first 90% of the code accounts for the first 90% of the development time. The remaining 10% of the code accounts for the other 90% of the development time.

Tom Cargill

And of course

Hofstadter’s Law: It always takes longer than you expect, even when you take into account Hofstadter’s Law.

Douglas Hofstadter

Enjoy!

Project Euler: Problem 6 in Ruby

I thought this one was way too easy compared the the last one. There’s some mathematical trickery you can do to speed things up, but the straight-forward solution is running in .009s so I’m going to call this one “done”.

Problem #6

The sum of the squares of the first ten natural numbers is,
1^(2) + 2^(2) + … + 10^(2) = 385

The square of the sum of the first ten natural numbers is,
(1 + 2 + … + 10)^(2) = 55^(2) = 3025

Hence the difference between the sum of the squares of the first ten natural numbers and the square of the sum is 3025 − 385 = 2640.

Find the difference between the sum of the squares of the first one hundred natural numbers and the square of the sum.

max = 100

def sum_of_squares max
  total = 0
  max.times do |i|
    i += 1
    total += i ** 2
  end
  total
end

def square_of_sums max
  total = 0
  max.times do |i|
    i += 1
    total += i
  end
  total ** 2
end

puts square_of_sums(max) - sum_of_squares(max)

I think kode had a neat solution in the forums:

sum, variable = 0, 0
100.downto(1) { |x| sum+=x; variable += (x*x) }
sum_square = sum*sum
puts sum_square - variable