Posts Tagged ‘analytics’

Ruby Script to Extract Google Analytics Keywords, Landing Page, Medium Entrances to CSV/TSV format

April 19th, 2010

I’ve had versions of this script hacked together before.  In fact, if you search for “Google Analytics Data Extractor” the #1 result is a google groups posting I made in a former life for a C# program I developed: http://groups.google.com/group/analytics-help-misc/browse_thread/thread/d2ad6ddf3d73e511

There’s now a couple of ruby gems out there for handling the Google Analytics API

One of them I’ve been doing some light-contributing to, including the pagination support (automatically grab all results in 10,000 result chunks)

I pushed a couple of changes up today at: http://github.com/er1c/gattica that hopefully will get merged into the main trunk soon (http://github.com/activenetwork/gattica)

One of the more useful uses of the library is to extract your Organic Search Results Keywords from Google Analytics. We’re using this at Frugal Mechanic to see how people search for Auto Parts.

require 'rubygems'
require 'gattica'
require 'fastercsv'

ga_profile = "" #Enter your Profile Here
start_date = Date.new(2009, 1, 1)
end_date = Date.new(2009, 12, 31)
file_path = "data" # Directory Needs to exist

puts "Google Username: "
u = gets.chomp
raise "bad username" unless !u.empty?
puts "Google Password: "
system "stty -echo"
pw = gets.chomp
system "stty echo"
raise "bad password" unless !pw.empty?

class ExtractKeywords
  def initialize(email,password)
    @gs = Gattica.new({:email => email, :password => password})
  end

  def get_accounts
    results = []
    @gs.accounts.each{|account|
      profile = {}
      profile[:site_title] = account.title
      profile[:profile_id] = account.profile_id ## this is the id required for requests to the API
      profile[:account_name] = account.account_name
      profile[:account_id] = account.account_id
      profile[:web_property_id] = account.web_property_id
      results << profile     }     return results   end   def connect_profile(profile)     @gs.profile_id = profile[:profile_id]   end   def connect_profile_id(profile_id)     @gs.profile_id = profile_id   end   def get_keywords(start_date = nil, end_date = Date.today)     results = []     csv_data = @gs.get({         :start_date => (start_date || (end_date - 365)).to_s,
        :end_date => end_date.to_s,
        :dimensions => ["medium", "keyword", "landingPagePath"],
        :metrics => "entrances",
        :sort => "-entrances",
        :page => true}).to_csv(:long)

    return FasterCSV.parse(csv_data, :headers => true)
  end

end

gs = ExtractKeywords.new(u, pw)
gs.connect_profile_id(ga_profile)

(start_date .. end_date).each { |date|
  file = "#{file_path}/medium_keyword_landingpage_visits_#{date.strftime('%Y-%m-%d')}.csv"
  next if File::exists?( file )

  FasterCSV.open(file, "w") do |csv|
    csv << ["medium", "keyword", "landingPagePath", "entrances"]
    keywords = gs.get_keywords(date, date)
    keywords.each{|row|
      csv << [row["medium"], row["keyword"], row["landingPagePath"], row["entrances"]]
    }

  end
}

Rails Plugin for Google Website Optimizer for AB Tests – Dynamic Content Experiments and No Flickering!

January 19th, 2010

Note: If you don’t care how this plugin works and just want to see/get the code, it’s available at github: http://github.com/tpunder/gwo

There’s a ton of reasons to reasons to invest in conversion rate increases.  If you’re spending $1 to acquire 10 visits, and you can get 2 instead of 1 to convert its like putting an extra dollar in your pocket.  A couple days ago Rand Fishkin from SEOMoz even said “2010 is the Year of Conversion Rate Optimization

Google Website Optimizer’s Homepage will give you a number of reasons to use their product My biggest reasons is its FREE!

Last March, we met with Eric Vasilik with the Google Website Optimizer Team in the Seattle Google Office about some hang-ups we had trying to A/B Test functionality and integrate Google Website Optimizer:

  1. Speed. The out-of-the-box Google Website Optimizer code does something like this:
    1. Download/Initialize GWO Code
    2. Identify which experiment/treatment group the user should see
    3. Download each of the content sections that should change and
    4. Use javascript to dynamically update each of the content sections of the page (FLICKER!)
  2. Static Content. Since you must store the HTML fragments in GWO for the service to return back the appropriate multivariate combination to show to users, the HTML fragments must be static.
  3. Maintenance. This ties back to our #2 reason, we would have to upload each of the HTML fragments into the GWO tool so they could be tested.  Since we have to code them locally to make sure they function & display correctly anyhow, it effectively means we need to duplicate the code to GWO.

The solution?  Leveraging a few very very clever Javascript/HTML tricks this is what we’re doing for our Auto Parts Price Comparison site.

Background:

A vanilla GWO Implementation operates similar to:

Tricked out solution looks like:

Why is it clever?  Lets walk through each of the scenarios for the GWO Variations.

Scenario 1: No Javascript Support

<!– Section1 – Default content –>
<script>
var GWO_Section1 = utmx(“variation_number”, “Section1″);
if (GWO_Section1 != undefined && GWO_Section1 != 0) document.write(‘<no’ + ‘script>’);
</script>
Default content – shown by default<br>
</noscript>

Browser ignores everything inside <script> tags and ignores the </noscript>

<!– Section1 – Variation 1 –>
<script>
if (GWO_Section1 == 1) document.write(‘</noscript a=”‘);
</script><!–”>
Alternative content 1<br>
<script>document.write(‘<’+'!’+'-’+'-’)</script>–>

<!– Section1 – Variation 2 –>
<script>
if (GWO_Section1 == 2) document.write(‘</noscript a=”‘);
</script><!–”>
Alternative content 2<br>
<script>document.write(‘<’+'!’+'-’+'-’)</script>–>

Scenario 2: Browser Supports Javascript – Variation == Control/Default Treatment

<!– Section1 – Default content –>
<script>
var GWO_Section1 = utmx(“variation_number”, “Section1″);
if (GWO_Section1 != undefined && GWO_Section1 != 0) document.write(‘<no’ + ‘script>’);
</script>
Default content – shown by default<br>
</noscript>

Browser evaluates GWO_Section == 0 so the document.write(<noscript>) is NOT executed resulting HTML code is:

Default content – shown by default<br>
</noscript>

Browser ignores the </noscript>

<!– Section1 – Variation 1 –>
<script>
if (GWO_Section1 == 1) document.write(‘</noscript a=”‘);
</script><!–”>
Alternative content 1<br>
<script>document.write(‘<’+'!’+'-’+'-’)</script>–>

Since GWO_Section == 0, the document.write(‘</noscript is not executed, but however the 2nd <script>document.write IS still executed resulting in

<!–”>
Alternative content 1<br>
<!–</script>–>

The HTML comment starts with the first <!– and keeps going until –> hits (the browser doesn’t even try to interpret the DOM inside the HTML comment – making this the most efficient way to “throw away” code from even being parsed by the HTML engine.  The variation 1 never gets displayed.

Scenario 3: Browser Supports Javascript – Variation == Experiment Treatment

<!– Section1 – Default content –>
<script>
var GWO_Section1 = utmx(“variation_number”, “Section1″);
if (GWO_Section1 != undefined && GWO_Section1 != 0) document.write(‘<no’ + ‘script>’);
</script>
Default content – shown by default<br>
</noscript>

Browser evaluates GWO_Section1 == 1 so it DOES print the <noscript> resulting in:

<noscript>
Default content – show by default<br>
</noscript>

Browser starts at the first <noscript> and ignores everything until the ending </noscript> – thereby ignoring the control treatment’s HTML

<!– Section1 – Variation 1 –>
<script>
if (GWO_Section1 == 1) document.write(‘</noscript a=”‘);
</script><!–”>
Alternative content 1<br>
<script>document.write(‘<’+'!’+'-’+'-’)</script>–>

GWO_Section1 evaluates to == 1 and the document.write prints out resulting in:

</noscript a=”
<!–”>
Alternative content 1<br>
<!–</script>–>

There’s two clever things in this one. The first is the a=” starts to create an HTML attribute that “ingests” the dangling <!–. The second, is the <!–</script>–> is a complete HTML comment, so there isn’t a dangling </script> tag floating in the HTML. Poof the Web Browser is now showing the Treatment HTML.

After we met with Eric he posted a link and an explanation for this trick at http://www.gwotricks.com/2009/05/server-side-dynamic-section-variations.html

We have a Rails Plugin that encapsulates all of this logic pretty well – so you don’t have to worry about all of the javascript hacks:

http://github.com/tpunder/gwo

Happy A/B + Multivariate Testing!