Monday, June 28, 2010

Thinking about the structure of the book

Update: to avoid continuous posting and reposting of the outline, we have created a page where the latest version will always be available. A link to this page is located in the top right portlet on this blog.

At the start of this project we posted a suggested outline, but while the research phase is going on, the book structure will likely undergo several changes. Please remember that on this phase your comments and suggestions are specially valuable and can be more influential on the final result. We really look forward to getting more comments and ideas.

Right now the outline I propose for the book is the following (thanks to Shane Hathaway for his input on this):

Part one: getting started. This part will have an emphasis on getting an application up and running while making simple use of the ZODB. A developer who just needs to add a simple persistent layer to his application might have enough with this.

  • Introduction to ZODB.
    This will be a very short chapter, just to get things going. What is the ZODB. Maybe some bits about the NoSQL craze, how the ZODB has been doing that for more than 10 years. Why is the ZODB a nice tool to keep in your Python developer's arsenal and when is it a good fit for your apps?

  • Your first ZODB application.
    Installation and running the first app. The objective of this chapter is to let the reader do something that works immediately. Just the basics to get an app running. Not a lot of details here.

  • Transactions.
    The ZODB depends on the transaction package and understanding this package is very important to working effectively with it. This chapter introduces transactions, shows what happens when you commit or abort, describes what a conflict error is and explains why it's a good idea to avoid long running transactions.

  • A more complex application.
    A bit more involved explanation of how the ZODB works and a more useful sample application. This chapter will build on our understanding of transactions.

  • Basic indexing and searching.
    The Catalog and indexes. I propose to use repoze.catalog here, which uses zope.index.

  • Maintenance.
    Packing, backups, etc.

  • Scaling.
  • The ZODB cache, ZEO and replication services.

    Part two - advanced topics. This will be a more in-depth review of techniques and concepts for ZODB development.

  • A more in-depth look at the ZODB internals.
    A little more information about how the ZODB works. At least enough stuff to understand the later chapters about storages and debugging.

  • Advanced transaction management. How to create data managers for working with other storages in the same transaction, how to best approach the need for well behaved, long running transactions.

  • ZODB Storages.
    Details about the FS storage and discussion of RelStorage and maybe DirectoryStorage.

  • Popular third party packages.
    Some of the most important packages for the ZODB will be described here.

  • Other indexing and searching strategies.
    Other catalog implementations, third party indexes and using external indexing solutions, like Solr.

  • Advanced ZODB.
    Evolving schemas, creating custom indexes, using ZODB in an asynchronous framework like twisted.

  • The debugging FAQ: frequent problems and suggested solutions.
    General debugging strategies and then a FAQ with common problems. For example, common traps like attempting to load an object state when the connection is closed

  • Part three - ZODB API

    The official public API will be documented here. This could serve as a really quick reference for developers. We might include APIs for some other modules, like transaction.


    1. Additional suggestions:

      - persistence/transaction machinery: ghosts, invalidations, btrees/sets, conflict errors, ACID, ...
      - writing unit tests involving zodb/zeo
      - security for custom objects (ala for instance)
      - zodb/zeo configuration options and their practical consequences in detail
      - ad-hoc queries
      - best practices
      - success stories

    2. I'm -1 on:
      - Success stories (can be handled by zodb website)
      - Security (should be stated that its application level concern, maybe in revision of book)
      - Integration into twisted, cherrypy, paster, etc. The only integration that might be worthwhile talking about would be wsgi (and there are two packages for zodb/wsgi integration, jim's and tres's.)
      - SOLR or external indexing strategies