Scribe

Scribe Scribe has moved!

Scribe is now hosted on GitHub:
http://github.com/facebook/scribe

Please update your bookmarks.

facebook's scribe at master - GitHub
Source: github.com
Scribe is a server for aggregating log data streamed in real time from a large number of servers. It is designed to be scalable, extensible without client-side modification, and robust to failure of the network or any specific machine.
Daniel

Daniel It doesn't look like the scribe configuration documentation exists on the wiki anymore, since the sourceforge redesign. Is there somewhere else where this documentation can be found?

August 13 at 3:41pm · Report
Jon Schutz
Jon Schutz
In the interim, I've rescued a copy from Google's cache and made it available here: http://notes.jschutz.net/114/scribe/scribe-server-configuration
August 16 at 11:29pm
Michael
Michael
@Nathan, that says it required authorization.
August 31 at 9:23am
Mark Lin

Mark Lin How does FB deal with message with multiple destinations? For example, a message should be processed by hadoop then archived somewhere else. Since doc mentioned one category per message, does that mean the client writting the message will have the send the same message twice?

August 12 at 5:14pm · Report
Scribe
Scribe
You can configure Scribe to route all messages of a given message category to two different locations. You just need to create 2 different stores for this category in your config file.
October 2 at 4:47pm
Jon Schutz

Jon Schutz What's happened to the concise summary at http://scribeserver.wiki.sourceforge.net/Configuration? Is there a new link with the same info?

August 5 at 10:19pm · Report
John Adams
John Adams
sourceforge got rid of their old wiki system and moved to media wiki. The configuration is no where to be found.
August 7 at 11:06am
Jon Schutz
Jon Schutz
Unofficial copy http://notes.jschutz.net/114/scribe/scribe-server-configuration
August 16 at 11:30pm
Huy

Huy The HDFS Append function crashed sometimes, this is another approach to make Scribe support HDFS ( which is more stable, I think ), check it out and tell me what do ya think : http://iamhuy.com/blog/?p=89

July 3 at 7:02pm · Report
Dhruba Borthakur
Dhruba Borthakur
Here is a description of how the hdfs-scribe integration is deployed in Facebook using hadoop 0.19 + a set of patches (HADOOP-2757, HADOOP-6099, HDFS-200 among others).
July 16 at 11:40pm
Dhruba Borthakur
Dhruba Borthakur
Here is the link : http://hadoopblog.blogspot.com/2009/06/hdfs-scribe-integration.html
July 16 at 11:40pm
Tirumala Addanki

Tirumala Addanki Hi, I am newbie to Scribe. Today while installing scribe I am getting the following error. Can any one tell me how to overcome this ?

configure: error: Could not link against boost_filesystem-boost_filesystem-gcc40-mt-1_36 !

June 18 at 5:02pm · Report
Tirumala Addanki
Tirumala Addanki
I am able to compile it successfully now. I am able to build the scribe successfully.
June 23 at 4:50pm
Huy

Huy Great to hear that Scribe supports HDFS now.

Elvis Ngwa Ambe

Elvis Ngwa Ambe Can it be that Scribe does not accept logs coming from a Client using TTransport but accepts logs from Clients using TFramedTransport

May 22 at 8:04am · Report
Scribe
Scribe
Yes, you need to use TFramedTransport.
May 22 at 4:18pm
Elvis Ngwa Ambe

Elvis Ngwa Ambe Coding a J2EE Inbound connector for Scribe , any one gone that road before?

Scribe

Scribe Check out the video of the Scribe Tech Talk on Facebook Engineering Tech Talk page: http://www.facebook.com/techtalks

Scribe

Scribe Earl, see this thread about setting up an Apache CustomLog to forward to Scribe: http://sourceforge.net/forum/forum.php?thread_id=2551690&forum_id=870823

Scribe

Scribe Earl, we are currently working on hdfs support for Scribe. Stay tuned.

Earl Cahill

Earl Cahill Is there an easy way to modify an httpd.conf file and get logging to happen via Scribe?

Earl Cahill

Earl Cahill Is there an easy way to get Scribe to write to hadoop / hdfs?

Scribe

Scribe If you want to see an example of a C++ Scribe Client, you can take a look at Scribe's source itself. Search for "resendClient" in src/conn_pool.cpp.