Sample Documentation

Included here are a few of the documents that I have written at StarRocks, ClickHouse, and Elastic over the past six years. These are representative samples, not everything that I have worked on.

Some of the links used are pointing to copies of documents in the Internet Archive Wayback Machine. This is done so that I can link to documents that have been removed from their original location, or have been modified.

This example is designed to be followed step by step to integrate the database with a specific third-party visualization tool.

When I wrote this guide, I pulled out the reusable content (the ports used by the database and where to find the connection details in the commercial UI) into reusable snippets. These are now imported wherever needed. Imports have always been available in Asciidoc, but this was missing in Docusaurus until recently.

Key features of this guide:

The rest of the visualization tool integrations at ClickHouse follow the same pattern.

Typically, integration documentation would be limited to "install the driver, add the connection string, press the test button". I deviated from this because the community often had problems using the integration once the connection was established. Deflecting issues reported in Slack and to the support desk is important to both the users and the support team.

Superset Integration

This is a guide for someone who has already gone through the basics of starting the database, creating a "Hello World" table, and loading a few rows of data. This document exemplifies one place where I combine explanation with a how-to guide. This is meant to both teach someone how to load data and explain what they should be considering while they work through the process.

In the NYPD Complaint Data documentation, I guide a new user of the ClickHouse analytical database through investigating the structure and content of an input file containing a dataset, determining the proper schema for the database table that the data will be stored in, how to transform the data while ingesting it, and how to run some interesting queries against that data.

Most guides in this product space tell the reader "Type this, click that, clean up". I find that type of guide to be boring, and I wonder if the method presented is a "good" method or the simplest for the author to write.

Database guides often use very simple datasets that are guaranteed to work. This is necessary for the very first tutorial-type content designed to get the product installed and the very first table created. Beyond that point, the reader needs to learn about how to understand their data and the database so that they can make proper decisions. When I wrote this guide I had almost no experience with the product. My mentor recommended that I "Write down everything that I learn while working through the process". This first example is the result of that advice.

Here is an example from the NYPD Complaint Data document that I believe is a good way to present a system for learning about the data, and properly configuring the database table to store the data efficiently:

The document continues to teach a few more very important techniques for analyzing and manipulating data and then finishes up with some queries and advice on what to learn next.

ClickHouse guide to analyzing NYPD complaint data

The documentation at Elastic was traditionally product-based. This meant that the documentation was split up into these separate sets:

This separation of the documentation meant that the reader had to know which tools they needed, what terminology each of the tools used to describe the same idea, and which tool to pick if there were multiple options for a specific task. This issue hit me personally when I was trying to set up a new feature. I searched for the feature and the search engine documentation came up first in the results, so I followed that guide. I had to use pages of JSON configuration to get the integration working. I was speaking with some of the other writers about how difficult this was to configure, and the writer for the visualization tool told me that there was a button to configure that. This conversation led to regular knowledge sharing among the writers and the course developers so that we could provide end-to-end scenario-based documentation that highlighted the best way to accomplish tasks. There are several solution guides, and I worked on these:

Getting Started with Observability

Monitor your Kubernetes Infrastructure

Use Elastic Security for SIEM

I am not a fan of using four lines of data to introduce the reader to a database product capable of ingesting and analyzing billions of rows of data or analyzing those billions of rows of data where they sit in cloud storage. I replaced the four line CSV file Quick Start at StarRocks with Quick Starts that align with the needs of the community:

There is some complexity to configuring the integrations with cloud storage, Apache Iceberg, and Apache Hudi. To make this easier for the reader I wrote Docker compose files to deploy MinIO, Iceberg, and Hudi. I think that this is appropriate, as the reader who wants to use external storage with StarRocks is likely familiar with the external storage. In addition to the compose files I documented the settings necessary, and in the case of the Hudi integration I submitted a pull request to the Hudi maintainers to improve their compose-based tutorial.

The "Basics" Quick Start is a step-by-step guide with no explanation until the end. There are some complicated data manipulations during loading. In the document, I ask the reader to wait until they have finished the entire process and promise to provide them with the details. Because the complex technique included in this How To guide contains a detailed section about how to deal with a common data problem, (date and time stored in non-standard formats), the content should be moved to a How To document dedicated to that problem. This would allow readers to find the content without reading a long guide about setting up the database and loading datasets. The "Basics" How To could then link to a "Reformatting date and time data" How To.

At the end of the document is a summary, and then the promised explanation of the curl parameters. It is a long explanation, so I have only included the most complex part of it here:

The rest of the Quick Starts have complex configuration files and commands explained in detail without sending the reader off to reference documentation to learn about a configuration item. I realize that there is a risk that the parameters may change in future versions of StarRocks, but these Quick Starts and several other guide-type documents are tested with every release. The short-term plan is to use snippet files to include the configuration material from the reference sources.

StarRocks Quick Starts

I love to collaborate with other people. Learning from other people, and sharing my knowledge with others is a central part of who I am. When Elastic was a start-up we were "for developers and by developers". Even though I had a marketing title, the Elastic leadership was very clear: My job was to make sure that every word on the website was truthful. I loved that. I worked on the content of the website, but most of my time was spent writing blogs, presenting on webinars, and building demos. Some of the content I produced is described in this section.