How Do You Query a Stream?

Originally aired:

About the Session

So you've embraced Apache Kafka as the core of your data infrastructure, embracing event-driven services that communicate with each through topics, integrate with legacy systems through an ecosystem of connectors, and respond more or less in real time to things that happen in the world outside your software. Logs of immutable events form a more robust backbone than the one-database-to-rule-them-all of your deep monolith past. Your stack is more evolvable, more responsive, and easier to reason about. There's just one problem: now that everything is a stream, how do you query things?

If this is you, you can probably name at least one or two ways off the top of your head, but have you stopped to think through how to make the choice? It is time you did. In this talk, we'll explore the solutions currently in use in the world for asking questions about the contents of a topic, including Kafka Streams, the various streaming SQL implementations, your favorite relational database, your favorite data lake, and real-time analytics databases like Apache Pinot. There is no single correct answer to the question, so as responsible builders of systems, we must understand our options and the tradeoffs they present to us. You'll leave this talk even happier that you've embraced Kafka as the heart of your system, and ready to deploy the right choice for querying the logs that hold your data.

See Highlights

Hear What Attendees Say

PwC

“Once again Saltmarch has knocked it out of the park with interesting speakers, engaging content and challenging ideas. No jetlag fog at all, which counts for how interesting the whole thing was."

Cybersecurity Lead, PwC

Intuit

“Very much looking forward to next year. I will be keeping my eye out for the date so I can make sure I lock it in my calendar."

Software Engineering Specialist, Intuit

GroupOn

“Best conference I have ever been to with lots of insights and information on next generation technologies and those that are the need of the hour."

Software Architect, GroupOn

Hear What Speakers & Sponsors Say

Scott Davis

“Happy to meet everyone who came from near and far. Glad to know you've discovered some great lessons here, and glad you joined us for all the discoveries great and small."

Web Architect & Principal Engineer, Scott Davis

Dr. Venkat Subramaniam

“Wonderful set of conferences, well organized, fantastic speakers, and an amazingly interactive set of audience. Thanks for having me at the events!"

Founder of Agile Developer Inc., Dr. Venkat Subramaniam

Oracle Corp.

“What a buzz! The events have been instrumental in bringing the whole software community together. There has been something for everyone from developers to architects to business to vendors. Thanks everyone!"

Voltaire Yap, Global Events Manager, Oracle Corp.