0

Anyone have details steps for installing Facebook's Scribe on an EC2 Instance?

Most of the instructions found by doing a search are a bit out of date.

Hari Menon
  • 33,649
  • 14
  • 85
  • 108
Steve Wilhelm
  • 6,200
  • 2
  • 32
  • 36

4 Answers4

3

I've used the recipe here on both the Amazon Linux AMI as well as on a Fedora image in the past 6 months. As long as that recipe is, it really does require that much effort. It still shocks me how many companies have built their own scribe RPM's and yet no good ones exist in the wild.

Blake Mathman
  • 2,699
  • 1
  • 23
  • 20
1

I found these older install instructions and used them as starting points:

http://www.design-ireland.net/?http%3A//www.design-ireland.net/article/Installing_Facebook_Scribe_on_Fedora_8 http://boost.2283326.n4.nabble.com/1-42-0-Release-candidates-available-td2626376.html

Installing

# download scribe code (I opted for /local)
cd /local
git clone https://github.com/facebook/scribe.git scribe

# install libevent
yum install libevent
yum install libevent-devel

# install boost - the version installed by 'yum boost' didn't seem to be work with scribe
# had to build my own from one revision back, v1.45, as there was a bug with current version
mkdir boost
cd boost
wget http://sourceforge.net/projects/boost/files/boost/1.45.0/boost_1_45_0.tar.gz
tar zxvf boost_1_45_0.tar.gz
boostrap.sh
./bjam -sNO_COMPRESSION=1

# Install thrift and apply patch (see below)
mkdir thrift
cd thrift
yum install python-devel
wget http://mirror.olnevhost.net/pub/apache//thrift/0.6.1/thrift-0.6.1.tar.gz
tar -zxvf thrift-0.6.1.tar.gz
cd thrift-0.6.1
# Thrift patch applied: https://issues.apache.org/jira/browse/THRIFT-1060
make
make install

# Install fb303
cd contrib/fb303
./bootstrap
./configure
make 
make install

# scribe
cd /local/scribe
export LD_LIBRARY_PATH="/usr/local/lib"
/sbin/ldconfig
./bootstrap.sh
./configure
make

Testing

Modify the scribe/example/example1.conf to fit your environment

Create the following short run script

export LD_LIBRARY_PATH="/usr/local/lib"
scribed -c ./modified_example.conf

Had to open the Thrift port, 1463, on the EC2 box by adding my MAC range to the Security Group's Inbound TCP ports

I wrote a simple ruby test app using scribe and thrift_client gem

require 'scribe'

client = Scribe.new("xxx-xxx-xx-xxx-xxx.compute-1.amazonaws.com:1463", "default" ,false, {:connect_timeout => 3, :retries => 3})

(0..100).each do |x|
  n = Time.now
  begin
    client.log("#{n} #{"%10.6f" % n.to_f} - testing loop #{x}", "bucket-#{x % 10}")
  rescue Exception => e
    puts e.message
    puts e.backtrace.inspect
  end
  sleep 0.01
end
Steve Wilhelm
  • 6,200
  • 2
  • 32
  • 36
0

I have been using this gist bash script to install scribe and dependancies on EC2 ubuntu boxes.

eodgooch
  • 1,322
  • 1
  • 11
  • 17
0

I don't have enough rep to comment, I wanted to reply to the instructions laid out by @Steve Wilhelm -- please note that this step:
./bjam -sNO_COMPRESSION=1
Is taking a long time for me (a bit over an hour now) -- as I'm trying to run this on an EC2 t1.micro instance. (it's a low-powered instance, but still) I did not know this beforehand, if I did I would have run this with a nice -n 3 so it doesn't make the box slow for another developer who's trying to use it also.

edit: It finished a bit over an hour, and ended up taking a bit over 500mb space when finished.

byl83
  • 736
  • 5
  • 8