DuckDB Bag of Tricks: First Splash

Some 1.5 years ago I heard about DuckDB for the first time. I've been using it ever since, and more and more everytime. It's one of those tools that just always delivers, no matter what kind of problem I throw at it.

Sounds too good to be true? Maybe. I quickly became a fan, and I'm not ashamed to admit it. Yet despite this confession, I think my favourable opinion towards DuckDB is not merely subjective, but grounded in experience.

If you're professionally occupied with data integration, data platforms, databases, data warehouses, data lakes, etc. you get to deal with a lot of tools and products, all the time. For some, you use them because they're the only available option. In other cases, you use them because they're the agreed upon (or mandated!) standard within a particular team or community. Many tools may not be great, but you still use them, because they're affordable and they get the job done well enough.

And then, sometimes, you run into a tool that you genuinely like, or even love. Why? Because it delivered at a time when you most needed it, and it performed well beyond your expectation, without asking a lot back. And it continues to do so, the next time, and the time after that.

It's that "WOW! Could it really be that easy?"-feeling, or that "OMG, how can it be so fast?"-rush. After a few of those experiences, you enter a phase of curiosity, and become open to trying it in favor of test-and-tried tools. And then, at some point, you just enjoy working with it so much that you want to try it for anything and everything: it's fun! Finally, you treat it as a trustworthy friend. By then, it's part of your professional life, and using it is just natural.

DuckDB is such a tool for me. I don't need to convince you, but I hope my posts will inspire you to give it a try. Anyway. Done with the evangelism: let's go ahead and have some fun!

From here onwards, I will start sharing tips and tricks about DuckDB. I will label these posts as DuckDB Bag of Tricks. Jump in, share, and comment - we can all have fun together with DuckDb. Here's a few pointers to get you started:

• DuckDB Home
• Documentation / Reference Manual
• Download/Install
• Source Code & Projects on Github
• DuckDB online shell

Community Tools &Projects

• Community Extensions Hub
• Awesome DuckDB - A curated list of awesome DuckDB libraries, tools and resources.
• Huey - blazing fast pivot tables on your browser by yours truly

All tricks so far

• Reading and Transforming Chess games in PGN-format
  1. Chess and Portable Game Notation (PGN)
  2. Ingesting raw text with the CSV Reader
  3. Distinguishing the Line Type
  4. Keeping game lines together: window functions
  5. Extracting Tagpairs with Regular Expressions
  6. Rolling up each game's lines into a single game row