2010 English 30 slides 1901 views
BRAINREPUBLIC - Powered by no-SQL
The document summarizes Andreas Jung's presentation on using MongoDB as the database for BRAINREPUBLIC. Some key points include: Andreas evaluated different "no-SQL" database options for BRAINREPUBLIC including key-value stores, MongoDB, and CouchDB. MongoDB was chosen because it offered the best performance compared to CouchDB, and its rich query API was preferable to map-reduce alone. The proposed architecture for BRAINREPUBLIC involved using MongoDB for the database, RabbitMQ for messaging, and SOLR for search, behind a load balancing layer.
Details
- Uploaded
- 4 October 2010
- Updated
- 3 December 2022
- Language
- English
- Slides
- 30
- Image size
- 2048x1576
- Views
- 1901
- SlideShare ID
- 5352579
- Uploaded format
- Collected
- 2026-09-27
The description above is SlideShare's automatically generated summary, not the author's own text.
Source & completeness
- Content
- original + reconstructed
- Validation
- passed
- Slide text
- available (30 slides)
- Original files
- yes
- original.pdf
Checksums (SHA-256)
original.pdf: 7f59626ad296838749b8ceba7665ca309fe9ce69d68956d12d591d054f10ddc1
Slide images come from SlideShare's public renderings (2048x1576 px, WebP). Any file marked as reconstructed is assembled from those images and is not the author's original source file.
Full text of every slide
Text extracted by SlideShare from the slide images — may contain OCR errors. 30 of 30 slides have text.
1
Powered by
MongoDB & no-SQL
Mongo Berlin
October, 4th 2010
Andreas Jung
www.zopyxgroup.com
Montag, 4. Oktober 2010
2
/ME
• Developer (backend) and software-analyst
• Strong background in Python, Zope and Plone
• Former Zope 2 release manager
• Co-founder and chairman of the German Speaking Zope
User Group (DZUG)
• Director of the Zope Foundation
• Member of Plone Foundation
• Author of tons of add-ons for Python, Zope and Plone
• Head of ZOPYX
Montag, 4. Oktober 2010
3
The Zope and Plone Expert Network
• German based full-Service partner network
• ZOPYX (Tübingen)
• Veit Schiele (Berlin)
• Zetwork (Oldenburg)
• Banality (Essen)
• Python, Zope, Plone & other cool stuff
Montag, 4. Oktober 2010
4
Agenda
• What is BRAINREPUBLIC?
• „no-SQL“ techologies used in the project
• Evaluation of technologies
• My view on MongoDB - pros and cons
• BRAINREPUBLIC architecture
Montag, 4. Oktober 2010
5
Montag, 4. Oktober 2010
6
Montag, 4. Oktober 2010
7
Montag, 4. Oktober 2010
8
Montag, 4. Oktober 2010
9
Montag, 4. Oktober 2010
10
Montag, 4. Oktober 2010
11
Montag, 4. Oktober 2010
12
Montag, 4. Oktober 2010
13
Montag, 4. Oktober 2010
14
Montag, 4. Oktober 2010
15
Montag, 4. Oktober 2010
16
Montag, 4. Oktober 2010
17
Choosing a database and tools for
BRAINREPUBLIC
• Criteria:
• fast
• scalable
• distributed
• Special requirement:
• having fun :-)
Montag, 4. Oktober 2010
18
Powered by
repoze.bfg
Montag, 4. Oktober 2010
19
repoze.bfg
• BFG is a "pay only for what you eat" Python web framework
• based on WSGI (Web Service Gateway Interface)
• What makes BFG special:
• It‘s tested
• Simplicity
• Minimalism
• Documentation
• Speed
Montag, 4. Oktober 2010
20
• fulltext search-engine based on Apache Lucene
• REST-style API for HTTP (XML/JSON)
• flexible field-based configuration through XML
• many plugins
• fast
• scales up/vertically (data partitioning)
• scales out/horizontally (clustering)
Montag, 4. Oktober 2010
21
• AMPQ (Advanced Message Queuing Protocol) based message queue
• Open-Source (VMWare)
• implemented in Erlang
• very fast (7500 messages/second)
• very stable
• flexible routing mechanisms for messages
• support for clustering
• implements producer & consumer pattern
Montag, 4. Oktober 2010
22
Possible database candidates
• ZODB (Zope Object DataBase)
• RDBMS
• „no-SQL“ databases
• key-value stores
• document-oriented databases:
• MongoDB
• Couchdb
Montag, 4. Oktober 2010
23
Evaluation of key-value storages
• breaking more complex data structures into key-value
pairs is a pain
• Map-reduce is brainfuck
• implementations do not provide a „traditional“ query API
Montag, 4. Oktober 2010
24
Evaluation of document-oriented storages
• schema-less databases are nice
• easy to deal w/ requirement changes
• JSON suitable for complex data structures
Montag, 4. Oktober 2010
25
MongoDB CouchDB
very fast (>10K ops/second) pretty slow
native drivers (TCP/IP) REST/HTTP API
Map-Reduce
Map-Reduce
rich query API
Master-Slave
Replica set easy replication
Sharding
Montag, 4. Oktober 2010
26
So why MongoDB (and not CouchDB)?
• Performance, performance, performance
• implementing a fast system on top of HTTP-based web-
services/APIs is a bad idea
• Rich query API (the world needs more than pure M-R)
• JSON-like queries are not my thing
(better syntax needed?)
Montag, 4. Oktober 2010
27
BRAINREPUBLIC Architecture
Varnish HA Varnish
Heartbeat
HAProxy HAProxy
Payment
App App App App App App
Billing
MongoDB RabbitMQ SOLR
Montag, 4. Oktober 2010
28
Lessons learned/Looking back
• MongoDB is kind of the „swiss knife of the no-SQL“ DBs
• very fast and reliable
• very low entry-barrier
• easy programming
• offers more than Map-Reduce
• 10gen seems to have ambitious goals with MongoDB
• good documentation (update website, books upcoming)
• very good community support (IRC, mailing list)
Montag, 4. Oktober 2010
29
My wish list...
• Poor replication performance (Master-Slave: 2.5-3 MB/sec)
• Indexes should fit completely into memory?
• A more fine-grained authentication model?
• Parallel map-reduce?
• Better usage of existing indexes (vs. compound indexes)?
• An alternative query API (not based on JSON) possible?
Montag, 4. Oktober 2010
30
www.brainrepublic.com
www.zopyxgroup.com
Montag, 4. Oktober 2010