Sean Quinlan, “Storage at Scale” (MP3 audio), O’Reilly Media Velocity Conference, IT Conversations, 2008/06/24

When massive volumes of data is expected to always be available on the Internet, how is that engineered?

A one in a million hardware fluke would happen every day at Google, so reliability must be handled by smart software. In this talk from the 2008 Velocity Conference, Sean Quinlan of Google describes the tools that Google uses to manage terabytes of data spread over millions of machines.

Because their needs are too big for a single machine, Google does not look at single machine performance. Rather, they look for the most performance bang for the buck and buy lots of it. They then layer software on top of it, which allows them to replicate data across multiple machines to compensate for the potential failure of any one machine. The two major systems they use are the Google File System (GFS) and BigTable. GFS is a cluster file system laid on top of a data center that stores chunks of data either as append-only sequences similar to log files or as read-only sorted tables of key/value pairs. These restrictions allow seamless and reliable storage at Google’s immense scale.

IT Conversations | Velocity Conference from O’Reilly Media | Sean Quinlan

[MP3 audio]

Advertisements
About

David Ing blogs at coevolving.com , photoblogs at daviding.com , and microblogs at http://ingbrief.wordpress.com . See .

Tagged with: , ,
Posted in Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Beyond this media queue
This content is syndicated to Twitter. For professional perspectives, look to Coevolving Innovations; for a photoblog, look to Reflections, Distractions.
  • Android PDF links
    On Android Oreo tablet, reference links on my PDF book don’t show up via Adobe Acrobat nor Google PDF Reader.  Links show up with qPDF Viewer, Foxit PDF Reader & Editor, and Xodo PDF Reader & Editor.  Presumption of basic functionality by bigger names is untrue. Advertisements
  • Kubuntu 17.10 LaTeX textgreek.sty
    Under Ubuntu 16.04, building a PDF under TeXstudio worked.  Under Kubuntu 17.10 (with pdfTeX 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian)), running the same input files resulted in: ! LaTeX Error: File `textgreek.sty' not found. These seemed like an error with reported in Feb. 2015, as “texlive-latex-extra: missing dependency – textgreek.sty requires l […]
  • 2017/11/07 10:15 Donna Dillenberger, “Cognitive Blockchain”, Cascon
    Plenary #cascon @DonnaExplorer IBM Fellow, IBM Research, Global leader of Enterprise Systems This digest was created in real-time during the meeting,based on the speaker’s presentation(s) and comments from the audience. The content should not be viewed as an official transcript of the meeting, but only as an interpretation by a single individual. Lapses, gra […]
  • 2017/10/25 09:30 Michael Mehaffy, “Horizons of Pattern Languages: Software, Cities, Planet”, PLoP
    Plenary @michaelwmehaffy Pattern Languages of Program PLoP2017 This digest was created in real-time during the meeting,based on the speaker’s presentation(s) and comments from the audience. The content should not be viewed as an official transcript of the meeting, but only as an interpretation by a single individual. Lapses, grammatical errors, and typing mi […]
  • 2017/10/21 08:35 Yodan Rofe, “The issue of ‘feeling’ as a criterion for choosing between different patterns or configurations (social and spatial)”, Purplsoc
    Plenary by Yodan Rofe #purplsoc Pursuit of Pattern Languages for Societal Change http://www.purplsoc.org/conference2017/ At Ben Gurion University of the Negev, Ph.D. from U.C. Berkeley This digest was created in real-time during the meeting,based on the speaker’s presentation(s) and comments from the audience. The content should not be viewed as an official […]
  • 2017/10/20 18:05 Christian Kohls, “Patterns for Creative Space”, Purplsoc
    Plenary @chrisimweb #purplsoc Pursuit of Pattern Languages for Societal Change http://www.purplsoc.org/conference2017/ This digest was created in real-time during the meeting,based on the speaker’s presentation(s) and comments from the audience. The content should not be viewed as an official transcript of the meeting, but only as an interpretation by a sing […]
Contact
I welcome your e-mail. If you don't have my address, here's a contact page.
%d bloggers like this: