Evaluating Quandl Data Quality

Quandl has indexed millions of time-series datasets from over 400 sources. All of Quandl’s datasets are open and free. This is great news but before performing any backtest using Quandl data, I want to compare it with a trusted source: Bloomberg for the purpose of this post. I will focus only on daily Futures data here (front month contract).

In the table below, the first few columns are self explanatory, the last two columns are:

  • Intersec. the number of common dates between Quandl and Bloomberg as of November 14, 2013
  • Mismatch the number of different closing prices among those common dates (Intersec. ≥ Mismatch).

Instrument Quandl Symbol Bloomberg Ticker Quandl StartDate Bloomberg StartDate Intersec. Mismatch
Australian Dollar OFDP/FUTURE_AD1 AD1 Curncy 13/01/1987 19/12/1989 5933 2640
DJ-UBS Commodity Index OFDP/FUTURE_AW1 DNA Index 03/10/2011 19/12/2008 532 42
Brent Crude Oil OFDP/FUTURE_B1 CO1 Comdty 17/08/1990 23/06/1988 5906 53
Soybean Oil OFDP/FUTURE_BO1 BO1 Comdty 01/07/1959 02/01/1975 8965 816
British Pound OFDP/FUTURE_BP1 BP1 Curncy 13/02/1975 20/03/1990 5879 2959
Brazilian Real OFDP/FUTURE_BR1 BR1 Curncy 08/11/1995 29/11/1995 2023 1526
Corn OFDP/FUTURE_C1 C 1 Comdty 01/07/1959 01/07/1959 13311 4407
Cocoa OFDP/FUTURE_CC1 CC1 Comdty 05/01/1970 01/07/1959 10826 903
Canadian Dollar OFDP/FUTURE_CD1 CD1 Curncy 17/01/1977 04/04/1986 6666 3055
CER Emissions OFDP/FUTURE_CER1 ICEDCER Index 18/03/2008 13/03/2009 1173 965
WTI Crude Oil OFDP/FUTURE_CL1 CL1 Comdty 30/03/1983 30/03/1983 7620 2549
Cotton OFDP/FUTURE_CT1 CT1 Comdty 01/07/1959 01/07/1959 13559 1361
Milk OFDP/FUTURE_DA1 DA1 Comdty 03/10/2011 11/01/1996 531 24
Full-size Dow Jones OFDP/FUTURE_DJ1 DJ1 Index 02/01/1998 03/04/1998 3918 683
US Dollar Index OFDP/FUTURE_DX1 DX1 Curncy 20/11/1985 07/04/1986 6872 285
Euro FX OFDP/FUTURE_EC1 EC1 Curncy 04/01/1999 16/12/1998 3735 1739
3-Month Eurodollar OFDP/FUTURE_ED1 ED1 Comdty 01/02/1982 01/04/1986 6800 2347
Ethanol OFDP/FUTURE_EH1 DL1 Comdty 03/10/2011 16/05/2005 532 0
1-Month Eurodollar OFDP/FUTURE_EM1 EM1 Comdty 01/10/2007 05/04/1990 1020 19
E-mini S&P 500 Index OFDP/FUTURE_ES1 ES1 Index 09/09/1997 09/09/1997 4112 1112
Feeder Cattle OFDP/FUTURE_FC1 FC1 Comdty 06/09/1973 30/11/1971 9995 4188
30-day Fed Funds OFDP/FUTURE_FF1 FF1 Comdty 03/10/1988 06/12/1988 5251 958
5-year Treasury Note OFDP/FUTURE_FV1 FV1 Comdty 20/05/1988 20/05/1988 6206 2565
Gold OFDP/FUTURE_GC1 GC1 Comdty 31/12/1974 02/01/1975 9214 3006
Copper OFDP/FUTURE_HG1 HG1 Comdty 01/07/1959 06/12/1988 5486 3785
Heating Oil OFDP/FUTURE_HO1 HO1 Comdty 06/03/1979 01/07/1986 6835 2806
Japanese Yen OFDP/FUTURE_JY1 JY1 Curncy 02/08/1976 19/12/1989 6012 2548
Coffee OFDP/FUTURE_KC1 KC1 Comdty 17/08/1973 16/08/1972 9891 324
Lumber OFDP/FUTURE_LB1 LB1 Comdty 16/11/1972 07/04/1986 6922 245
Live Cattle OFDP/FUTURE_LC1 LC1 Comdty 30/11/1964 30/11/1964 12314 5364
Lean Hogs OFDP/FUTURE_LN1 LH1 Comdty 25/06/1969 01/04/1986 6937 3951
UK Natural Gas OFDP/FUTURE_M1 FN1 Comdty 02/01/2007 30/01/1997 1766 3
S&P 400 Midcap Index OFDP/FUTURE_MD1 MD1 Index 13/02/1992 13/02/1992 3980 614
Mexican Peso OFDP/FUTURE_MP1 PEA Curncy 25/04/1995 11/05/2012 382 340
New Zealand Dollar OFDP/FUTURE_NE1 NV1 Curncy 02/09/2004 08/05/1997 1937 189
Natural Gas OFDP/FUTURE_NG1 NG1 Comdty 03/04/1990 03/04/1990 5682 2460
Nikkei 225 Index OFDP/FUTURE_NK1 NX1 Index 25/09/1990 02/10/1990 5643 331
E-mini Nasdaq 100 OFDP/FUTURE_NQ1 NQ1 Index 22/06/1999 21/06/1999 3635 652
Oats OFDP/FUTURE_O1 O 1 Comdty 01/07/1959 01/07/1959 12770 3600
Orange Juice OFDP/FUTURE_OJ1 JO1 Comdty 01/02/1967 01/02/1967 11684 560
Palladium OFDP/FUTURE_PA1 PA1 Comdty 05/01/1977 01/04/1986 5596 1840
Russell 1000 OFDP/FUTURE_RF1 RM1 Index 11/05/2007 20/09/2002 1657 18
Russell Growth OFDP/FUTURE_RG1 RGAA Index 03/05/2010 24/12/2012 230 192
Rough Rice OFDP/FUTURE_RR1 RR1 Comdty 20/08/1986 06/12/1988 5112 363
Russian Ruble OFDP/FUTURE_RU1 RU1 Curncy 03/10/2011 23/04/1998 533 2
Russell Value OFDP/FUTURE_RV1 RVBA Index 03/05/2010 24/12/2012 230 192
Soybeans OFDP/FUTURE_S1 S 1 Comdty 01/07/1959 01/07/1959 13401 4299
Sugar OFDP/FUTURE_SB1 SB1 Comdty 04/01/1961 03/01/1961 13105 814
Swiss Franc OFDP/FUTURE_SF1 SF1 Curncy 13/02/1975 19/12/1989 6010 2741
Silver OFDP/FUTURE_SI1 SI1 Comdty 13/06/1963 02/01/1975 8848 3058
Soybean Meal OFDP/FUTURE_SM1 SM1 Comdty 02/07/1959 02/01/1975 9234 913
Full-size S&P500 Index OFDP/FUTURE_SP1 SP1 Index 21/04/1982 21/04/1982 7580 2688
Russell Small-Cap OFDP/FUTURE_TF1 RTA1 Index 20/08/2007 17/08/2007 1613 8
2-year Treasury Note OFDP/FUTURE_TU1 TU1 Comdty 01/10/2007 25/06/1990 1521 279
10-year Treasury Note OFDP/FUTURE_TY1 TY1 Comdty 03/05/1982 03/05/1982 7817 2044
Ultra Treasury Bond OFDP/FUTURE_UL1 WN1 Comdty 30/10/2012 11/01/2010 262 0
30-year Treasury Bond OFDP/FUTURE_US1 US1 Comdty 22/08/1977 22/08/1977 9067 5198
S&P500 Vol. Index OFDP/FUTURE_VX1 UX1 Index 03/05/2004 26/03/2004 2270 610
Wheat OFDP/FUTURE_W1 W 1 Comdty 01/07/1959 01/07/1959 13407 4033
E-mini Dow Jones OFDP/FUTURE_YM1 DM1 Index 01/10/2007 20/09/2002 1526 15

Main findings

  • I managed to find Bloomberg tickers for 60 contracts out of 68 available on Quandl
  • At first glance differences seem scary but digging a bit deeper reveals a different picture.
  • For some contracts Bloomberg and Quandl don’t use the same number of decimals which inflate artificially the mismatch numbers
  • Some contracts are not expressed in the same unit between Bloomberg and Quandl. I had to apply a multiplier to make an “apple to apple” comparison.
  • Differences seem to cluster in time. It seems like there are different rules for building front month contract (or data source changes?)
  • Differences are often more pronounced at the beginning of the sample.
  • I used PX_LAST field in Bloomberg which might not be the settlement price for all contracts (I have to check this). Quandl uses a settlement price. This might explain some differences as well.

Conclusion

This post isn’t a fully-fledged analysis but it highlights potential issues and it’s up to the interested reader to solve them. I don’t think there are major issues with Quandl data but I wouldn’t use it for serious analysis until I understand better the differences with Bloomberg. This means going through each contract individually and trying to understand the source of the differences. Finally I put below my Quandl wish list:

  • Add  European Equity Indices Futures coverage (I’m not sure a free source even exists)
  • Add an ETF category to Quandl list of instruments
  • Provide more accurate availability schedule for all instruments

For anyone keen on reproducing this analysis, I saved the code used in this post on a Gist/Github repository.

Any comments welcome

6 Responses to “Evaluating Quandl Data Quality”


  • A while back I was looking at front and back month continuous fed funds futures contracts. The Quandl data contained a lot of missing values that often spanned over several days; the bloomberg data did not. I didn’t really look into the reason for the missing values.

    In the end, I sympathize with the noble goals of the Quandl project, but, at this point in time, don’t believe that anything beats the reliability of bloomberg data

  • Is it possible you can show some distribution info re the differences between the 2 data sources?

    Bloomberg data contains error all the time as well. I personally helped them fix data at least twice.

  • Thanks for this very comprehensive analysis!

    I took a slightly closer look at the data discrepancies. I suspect that a significant number of them come from different “roll rules” used to construct the front month contracts. Bloomberg rolls its front month contract on the first day of the expiry month; Quandl rolls on the last trading date of the futures. For contracts with monthly expiries, this could lead to significant discrepancies.

    We’re working on implementing different roll rules on Quandl. The ultimate goal is to allow users to specify their own roll rules (not just Bloomberg’s 3 or 4 variants, but anything at all — totally freeform).

    What I’d be really interested in is an analysis of discrepancies on individual contracts. Any such discrepancy is likely to be an error on our (Quandl’s) part, which I would want to fix asap.

    On your other requests: European Futures, ETFs, and guaranteed data update times are all on our to-do list. (In roughly that order, too). Thanks for your patience!

    • Abraham,

      Thanks for the feed back and taking care of my wish list. As mentioned to Alex, I’ll try to post a more in depth analysis soon.

  • @joe:

    Thanks for your kind words about our “noble goals” :)

    I wouldn’t claim for a second that we’re anywhere near as reliable as Bloomberg. We have a long way to go. But we’re working hard, and trying to improve our data (width of coverage, quality, reliability, timeliness and convenience) all the time. Hopefully in the coming months and years we will be able to offer data that will legitimately compete with Bloomberg’s, for free.

    (Keep in mind also that Quandl is less than a year old — so we’re just getting started!)

Leave a Reply