The ChessBazaar  Marc Lacrosse's chess site
visits so far
view or sign the guestbook

Playing without opening book

25.03.2007

A frequently asked question is:

What is the value of playing blitz without opening book as compared with playing with a good general opening book.

I just held a 1900 games tournament aiming at approaching an answer to this question.

I selected the four best free 32-bits engines with an established rating in the CCRL 40/4 list (as of 16.03.2007) :


R - Rybka 1.0 Beta 32-bit   2900 +/-19
T - Toga II 1.3x4           2883 +/-23
S - Spike 1.2 Turin         2825 +/-08
N - Naum 2.0 32-bit         2799 +/-10

mean Elo : 2852


Each engine entered the tournament five times with the following Polyglot books :


Books :
00 - no book
  no explanation required :-)
01 - Performance
 The 1.5 MB book which was built for Fruit 2.2.1 and is now freely available (see ChessBazaar's file archive )
02 - DClarge
 A very good 6.2 MB general book built by Dann Corbit in july 2005 (see ChessBazaar's file archive)
03 - Best-03
 A recent attempt of mine at improving over performance.bin (1.4 MB ;private)
04 - Monster
 An enormous private polyglot book (165 MB; private)


So there were 20 players in the tournament (4 engines x 5 books).

The tournament was a ten rounds all-play-all one (1900 games).
PIV 3.0 Ghz 128MB noTB noEGBB ponder off.
1 min + 1sec per move

RESULTS

Grouping the games by engine :


Rank Name    Elo    +    - games score  draws
   1 R-any    67   16   16   950   60%   33%
   2 T-any    19   16   16   950   53%   37%
   3 S-any   -27   16   16   950   46%   34%
   4 N-any   -59   16   16   950   41%   38%


This is very similar to the CCRL rating list, proving that the tournament is not biased :


                        CCRL 40/4     Present test
Rybka 1.0 Beta 32-bit   2900 +/- 19   2919 +/- 16
Toga II 1.3x4           2883 +/- 23   2871 +/- 16
Spike 1.2 Turin         2825 +/- 08   2825 +/- 16
Naum 2.0 32-bit         2799 +/- 10   2793 +/- 16


Grouping the games by opening book :


     Book          Elo    +    -  games score  draws
   1 Monster       2868  18   18   760   53%    34%
   2 Best-03       2866  18   18   760   52%    38%
   3 Performance   2864  18   18   760   52%    37%
   4 DClarge       2860  18   18   760   51%    38%
   5 No Book       2802  19   19   760   42%    30%


So here is the answer : a general opening book is worth 65 Elo points in this experiment as compared with playing without opening book at all.

The difference between No-Book and the best suiting book varies according to the chosen engine :


Engine           Book       delta vs no book :
Rybka 1.0 Beta   Monster    +82  (2947 vs 2865)
Toga II 1.3x4    Monster    +61  (2897 vs 2836)
Spike 1.2 Turin  Monster    +69  (2846 vs 2777)
Naum 2.0         DClarge    +102 (2822 vs 2720)

COMMENTS

So it seems that Naum and Rybka gain a little more from playing with an opening book than Spike and Toga.

Rybka without book is still stronger than Spike and Naum playing with their best suited opening book.

As compared with no book at all, playing with a general book is worth 65 elo points.

This is in the same order of magnitude as searching one-ply further, doubling the clock frequency or playing on dual processsor as compared with single processor : not that high but not insignificant.



Detailed results :

Rank Name   Elo   +    -   games score  draws
   1 R-04    95   41   40   190   64%    30%
   2 R-03    89   41   40   190   64%    32%
   3 R-02    73   40   39   190   61%    32%
   4 R-01    69   40   39   190   61%    35%
   5 T-04    45   40   39   190   57%    33%
   6 T-01    29   39   38   190   54%    39%
   7 T-03    28   39   39   190   54%    38%
   8 R-00    13   39   39   190   52%    35%
   9 T-02    10   38   38   190   52%    42%
  10 S-04    -6   39   39   190   48%    35%
  11 S-03   -15   39   39   190   48%    37%
  12 T-00   -16   39   40   190   48%    31%
  13 S-01   -18   39   39   190   47%    37%
  14 S-02   -20   39   39   190   47%    36%
  15 N-02   -30   38   38   190   45%    42%
  16 N-01   -33   39   39   190   45%    37%
  17 N-03   -42   38   38   190   43%    45%
  18 N-04   -62   39   40   190   41%    37%
  19 S-00   -75   40   41   190   39%    26%
  20 N-00  -132   41   42   190   31%    27%