December 28, 2004

Q Language Widening the Appeal of Vectors

by Simon Garland

When Kx Systems introduced the kdb+ database a year and a half ago, it included Q, a new vector language from Arthur Whitney. Q is layered on K, the language that Whitney first offered in 1993 and that has been evolving ever since. While old hands at K programming can still access K when they move to kdb+ most of them are finding that they prefer Q. Q uses English words for unary operations and removes the ambivalence in K and APL. Although Whitney’s languages are known for being concise, Q raises brevity to a higher level — it can save programmers lines of code when compared with K.

The Q language is helping to attract many more new users to the kdb+ database and is making vector programming more appealing to a wider audience by offering not only the speed of languages like K, J, and APL but also easier syntax. New users unfamiliar with vector languages are finding they can start programming in a day or two using Q.

Vector readers familiar with older versions of K will find additional language enhancements in kdb+. Developers work with a single language. Ksql, the time-series querying language used alongside earlier versions of K, has been eliminated. Time-series querying is now integrated into Q to make one language that serves both programming and querying functions. Datatypes have also been expanded to include many datetime types.

Another of the enhancements to Q is the ability to work with Dictionaries and Tables as easily as we're used to working with Arrays. According to Whitney, "Q marries programming (lists) — and databases (dictionaries)."

The example below shows the elegance and ease of use of Q's date, dictionary and table datatypes.

q)/ create a dictionary
q)dict:`start`review`end!2004.12.25 2005.01.25 2005.02.25
q)show dict
start   2004.12.25
review  2005.01.25
end     2005.02.25
q)/ 10 days later
q)show dict+10
start   2005.01.04
review  2005.02.04
end     2005.03.07
q)count dict
3
q)/ 2 take dict
q)show 2#dict
start   2004.12.25
review  2005.01.25
q)/ -2 take dict
q)show -2#dict
review  2005.01.25
end     2005.02.25
q)/ iota 8
q)key 8
0 1 2 3 4 5 6 7
q)/ 8 weeks on from 2004.12.25
q)2004.12.25+7*key 8
2004.12.25 2005.01.01 2005.01.08 2005.01.15 2005.01.22 2005.01.29
2005.02.05 ..
q)/ create a table for following 8 weeks
q)tbl:dict+/:7*key 8
q)show tbl
start       review      end
----------------------------------
2004.12.25  2005.01.25  2005.02.25
2005.01.01  2005.02.01  2005.03.04
2005.01.08  2005.02.08  2005.03.11
2005.01.15  2005.02.15  2005.03.18
2005.01.22  2005.02.22  2005.03.25
2005.01.29  2005.03.01  2005.04.01
2005.02.05  2005.03.08  2005.04.08
2005.02.12  2005.03.15  2005.04.15
q)/ (+/: is + each right)
q)/ how many reviews per month?
q)show select count i by review.month from tbl
month    x
----------
2005.01  1
2005.02  4
2005.03  3
q)/ 2 take tbl
q)show 2#tbl
start       review      end
----------------------------------
2004.12.25  2005.01.25  2005.02.25
2005.01.01  2005.02.01  2005.03.04
q)/ -2 take tbl
q)show -2#tbl
start       review      end
----------------------------------
2005.02.05  2005.03.08  2005.04.08
2005.02.12  2005.03.15  2005.04.15
q)/ all those that end before the 1st of april
q)show select from tbl where end<2005.04.01
start       review      end
----------------------------------
2004.12.25  2005.01.25  2005.02.25
2005.01.01  2005.02.01  2005.03.04
2005.01.08  2005.02.08  2005.03.11
2005.01.15  2005.02.15  2005.03.18
2005.01.22  2005.02.22  2005.03.25
q)

Compare SQL and Q

To get a good sense of how much more expressive Q is than SQL, take a look at:

http://kx.com/k4/e/tpcd.s (SQL code)
http://kx.com/k4/e/tpcd.q (q code)
http://kx.com/k4/e/tpcd.txt (comments on the q code)

Kdb+ simplifies Kx product line

With the success of kdb+, Kx has placed kdb (the earlier generation product) in maintenance mode. Much of the Kx installed base is in the process of transitioning to kdb+, and all new sales are on kdb+.

Kx offers the following developer resources for additional information:

Kdb+ database white paper
http://www.kx.com/papers/kxdb-request.php

Sample code
http://kx.com/k4

Documentation
http://www.kx.com/developers/documentation.php

Join Kx forums for a variety of discussions about Kx products
http://kxforums.com

Posted by Stevan Apter at December 28, 2004 10:02 PM

Comments

Will an evaluation version of Q be available for the curious to play with?

Posted by: steve at January 31, 2005 12:58 PM

Post a comment

Thanks for signing in, . Now you can comment. (sign out)

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)


Remember me?