xref: /src/external/cddl/dtracetoolkit/dist/Docs/History

History - History of the DTraceToolkit

------------------------------------------------------------------------------
20-Apr-2005	Brendan Gregg	Idea
	For a while I had thought that a DTrace toolkit would be a nice 
	idea, but on this day it became clear. I was explaining DTrace to 
	an SSE from Sun (Canberra, Australia), who had a need for using 
	DTrace but didn't have the time to sit down and write all the
	tools he was after. It simply made sense to have a DTrace toolkit
	that people could download or carry around a copy to use. Some
	people would write DTrace tools, others would use the toolkit.
------------------------------------------------------------------------------
15-May-2005	Brendan Gregg	Version 0.30
	I had discussed the idea of a DTrace toolkit with the Sun PAE guys in 
	Adelaide, Australia. It was making more sense now. It would be much
	like the SE Toolkit, not just due to the large number of sample 
	scripts provided, but also due to the role it would play: few people
	wrote SE Toolkit programs, more people used it as a toolkit. While
	we would like a majority of Solaris users to write DTrace scripts, 
	the reality is that many would want to use a prewritten toolkit.
	Today I created the toolkit as version 0.30, with 11 main directories,
	a dozen scripts, man pages and a structure for documentation.
------------------------------------------------------------------------------
16-May-2005	Brendan Gregg	OneLiners
	I've been using the toolkit for a day now (wow!), and have noticed
	a few problems I've been fixing. One of them was the dtrace oneliners.
	I have them in two files, Docs/oneliners.txt and the examples in
	Docs/Examples/oneliners_examples.txt. The problem is that when I'm
	looking for a script, I'm looking in Docs/Commands - a list of the
	seperate script files, or I'm doing an ls or find. Ok, so I've now
	made each one liner a seperate script. This seems at first pretty
	silly since they are oneliners and shouldn't deserve an entire script
	each, but I've found having them as seperate scripts makes them far 
	easier to find and use. The scripts and man page for each script do 
	point out the fact that it's a one liner. 
------------------------------------------------------------------------------
17-May-2005	Brendan Gregg	Version 0.33
	Version 0.33 with 33 scripts. Maybe I should make the version number
	equal the script count. :) I just finished dtruss, dapptrace and
	dappprof.
------------------------------------------------------------------------------
08-Jun-2005	Brendan Gregg	Name changes.
	I've renamed Docs/Commands to Docs/Contents. I found myself typing
	"more Docs/Contents" by mistake a lot. ok, maybe it made more sense
	to call it Contents after all. I've also made a symlink to it called
	Index.
------------------------------------------------------------------------------
08-Jun-2005	Brendan Gregg	Version 0.35
	Version 0.35 with 35 scripts. Also touched up procsystime and some
	man pages. Added the CDDL version 1.0.
------------------------------------------------------------------------------
09-Jun-2005	Brendan Gregg	Version 0.42
	Added 7 more scripts.
------------------------------------------------------------------------------
14-Jun-2005	Brendan Gregg	Version 0.57
	Added heaps of new scripts. Now at 57 scripts.
------------------------------------------------------------------------------
17-Jun-2005	Brendan Gregg	Version 0.61
	Restyled many commands.
------------------------------------------------------------------------------
28-Jun-2005	Brendan Gregg	Version 0.70
	Added several commands including dexplorer. Developed a few useful
	variants of classic scripts while writing dexplorer, and have added
	them to the toolkit (I kept wanting to run them individually but 
	not have to run an entire dexplorer).
------------------------------------------------------------------------------
25-Jul-2005	Brendan Gregg	Version 0.77
	Added tcpsnoop.d, tcpsnoop, tcptop.  Because of their addition I have
	dropped tcpwbytes.d and tcpwlist.  These are complex scripts, but they
	track TCP in an accurate manner. However! also because they are 
	complex scripts, I expect they will require maintainence for newer
	versions of [Open]Solaris, as various probes may change. They will
	become much more stable once a network provider has been added to
	DTrace (which may be some time away).
	Also added iotop, and updated a bunch of scripts. A lot of work went
	into this version, although the version change doesn't reflect that
	(I'm still keeping the version number == to number of scripts).
	Also added rwsnoop, rwtop, and more.
------------------------------------------------------------------------------
26-Jul-2005	Brendan Gregg	Version 0.82
	Many new scripts added, many updates. This is a major release.
------------------------------------------------------------------------------
17-Sep-2005	Brendan Gregg	Version 0.83
	A few scripts have been updated so that they work better.
	execsnoop, iosnoop, opensnoop and rwsnoop will be more responsive
	(increased switchrate).
------------------------------------------------------------------------------
22-Sep-2005	Brendan Gregg	Version 0.84
	Some updates, fixed some bugs (cputimes, cpudists). Added cpuwalk.d.
------------------------------------------------------------------------------
15-Nov-2005	Brendan Gregg	Sys Admin Magazine
	Ryan Matteson wrote an article on the DTraceToolkit which has been
	printed in Sys Admin Magazine, December 2005. It's quite good,
	and made it as the feature article - which means it will be available
	online for some time. Thanks Matty, and Sys Admin Magazine!
	"Observing I/O Behavior with the DTraceToolkit"
	http://www.samag.com/documents/sam0512a/
------------------------------------------------------------------------------
01-Dec-2005	Brendan Gregg	Version 0.88
	Many scripts were updated. Added the Apps category. I had planned
	to add some key scripts, but they haven't made it out of testing yet.
------------------------------------------------------------------------------
03-Dec-2005	Brendan Gregg	Version 0.89
	Added nfswisard.d, fixed a minor bug with tcp* tools (see 
	dtrace-discuss mailing list).
------------------------------------------------------------------------------
12-Jan-2006	Brendan Gregg	Version 0.92
	Added a few scripts including rwbytype.d. Fixed several issues.
------------------------------------------------------------------------------
09-Apr-2006	Brendan Gregg	Solaris Internals 2nd Edition
	In the past few months I have been contributing to Solaris Internals
	2nd Edition. This book (now two volumes) is really amazing. The 2nd
	volume does use the DTraceToolkit where appropriate, and covers loads
	of useful topics. While writing and reviewing material for Solaris
	Internals, I've had numerous new ideas for DTrace scripts. Not only
	that, but a few people have managed to send me well styled, carefully
	tested, well considered DTrace scripts for inclusion in the toolkit.
------------------------------------------------------------------------------
20-Apr-2006	Brendan Gregg	TCP bug fixed
	Stefan Parvu sent me a bug for the tcp* scripts: on build 31+ they
	error'd on the symbol SS_TCP_FAST_ACCEPT. This symbol was 
	renamed to SS_DIRECT (I checked the code, they are used in the 
	same way). Ironically, when I first wrote the scripts I had hardcoded
	the value 0x00200000, then rewrote it "properly" by importing
	the header files and using the symbol name. Had I been lazy and left
	it hardcoded, the bug would never have eventuated. Not to worry,
	it has returned to being hardcoded, so that it works on all builds
	(until something else changes).
------------------------------------------------------------------------------
21-Apr-2006	Brendan Gregg	Restyled - again!
	I've been writing the "DTraceToolkit Style Guide", to document
	the style that these scripts obey. It is quite strict, and sets
	the bar fairly high. I've been warned that it may cause very few
	people to ever contribute scripts, which is fine. At some point
	I'll carefully explain the mentality behind this, but in a nutshell:
	Users on critical production servers expect the tools to be 
	accurate, carefully tested, and cause no undocumented harm. 
------------------------------------------------------------------------------
22-Apr-2006	Brendan Gregg	Docs changes
	The "Contrib" file was merged into the "Who" file. In hindsight
	it is better to keep this data together than to split it up.
------------------------------------------------------------------------------
24-Apr-2006	Brendan Gregg	Version 0.96
	The toolkit now contains 104 scripts, however I'll keep the version
	number < 1.00 until the dust has settled on these new scripts.
	There is some special significance with version 1.00, it would 
	imply that every script had been tested for some time - not that
	I've just added a few. 
	There is a new main directory, FS for file system related scripts.
	There are some interesting scripts in there, from or based on
	Solaris Internals 2nd ed, vol 2.
------------------------------------------------------------------------------
30-Sep-2007	Brendan Gregg	Version 0.99
	It's been a year and a half since the last release, and a lot has
	happneed. Firstly, the DTraceToolkit has featured in the Prentice Hall
	book,
		Solaris Performance and Tools
		   DTrace and mdb techniques for Solaris 10 and OpenSolaris

	written by Richard McDougall, Jim Mauro and myself. It is a companion
	book to "Solaris Internals 2nd edition" by Richard McDougall and
	Jim Mauro. If you are serious about becomming a DTrace guru,
	especially on Solaris, then please study both books. (Yes, I realize
	that many people are using the DTraceToolkit because they don't have
	the time or don't want to become DTrace gurus; well, so long as
	you are using DTrace anyway :). The performance book was a great relief
	to write - since we were able to put to print much performance wisdom
	and knowledge that was begging to be documented.

	Then, in late 2006 I joined an advanced products engineering team
	at Sun in San Francisco, a team which includes the three members of
	team DTrace. It's been a great opportunity to learn from such
	engineers, and to contribute more directly to DTrace. So far my work
	has included writing a JavaScript provider, integrated inet_ntoa()
	style functions into DTrace, and prototying DTrace IP, TCP and UDP
	providers.

	Working on the network providers is good news for the DTraceToolkit,
	as it will indirectly help the tcp* scripts become more stable. Yes,
	those scripts have broken a few more times during the last 18 months,
	sorry about that, and it will keep happening until we have stable
	network providers. This is why I only ever wrote three tcp* scripts,
	and not at least a dozen, which I'd really like to do.

	I did leave my pile of old SPARC and x86 development servers behind
	in Australia, and brought over a couple of laptops. That has made me
	more dependant on Stefan for testing the toolkit - especially on SPARC.

	So, it's been about 18 months since the last release, which is 
	mostly due to having less spare time due to moving countries and
	learning a new job.

	Michelle from Sun docs has been asking for a newer version of the
	DTraceToolkit for the OpenSolaris starter kit, which is why I'm
	releasing this version now and not waiting a few more weeks as	
	I complete bug fixes.

	So the good and the bad news for this release, starting with the bad,

	Bad: tcpsnoop/tcptop still don't work on some Solaris 10 releases.
	I've added versions that should work on Solaris Nevada and OpenSolaris
	for releases from around late 2007. They are likely to break again.
	The real answer, as always, is for stable nework providers to be
	integrated into Solaris.

	Many of the exciting new language provider scripts in this release
	currently require downloading, patching and compling of the language
	interpreter to get working. See the Readme file in each directory
	for pointers.

	Good: many more scripts to cover the new DTrace language providers
	that are available (the DTraceToolkit is now 227 scripts). Many
	updates to the Notes directory. Bug fixes. Some new categories
	other than for scripts: Code - for simple programs to DTrace (and
	for the example files), and Snippits - for useful lumps of DTrace
	code to copy-n-paste from. The man pages are also making room
	for documenting both stability and supported operating systems for
	each script - now that DTrace exists for MacOS X Leopard, the
	DTraceToolkit will begin supporting multiple operating systems.

	This can be thought of as a developer's release of the DTraceToolkit -
	to help people start using DTrace with Perl, Python, Ruby, Php, Java,
	JavaScript, Shell and Tcl. I've written about 15 scripts for each
	language, to cover the basics and to show the way for deeper analysis.
	The scripts are also similar from one language to another, having
	devoleped a tried-and-tested group of scripts for analyzing real world
	issues - it made sense to repeat these scripts for every language
	possible. To see what I mean, try reading,

	   Examples/j_cputime_example.txt     Examples/py_cputime_example.txt
	   Examples/js_cputime_example.txt    Examples/rb_cputime_example.txt
	   Examples/php_cputime_example.txt   Examples/sh_cputime_example.txt
	   Examples/pl_cputime_example.txt    Examples/tcl_cputime_example.txt

	You might notice that the example files are more clearly and carefully
	explained. Claire (my wife), wrote close to one hundred of them for
	this release while I focused on writing and testing the scripts.
	Claire has worked as a SysAdmin and as an IT instructor, and is well
	skilled at explaining relavent technical details. And she can spell
	much better than I can. :)

	The future: I still have many new scripts and some bug fixes in mind,
	as well as generally improving the Notes and Examples provided.
	Hopefully it won't be too many months before you see another
	release. Check here for the lastest installment,

	http://www.opensolaris.org/os/community/dtrace/dtracetoolkit
------------------------------------------------------------------------------