Build a ChatGPT RAG Data Pipeline with RisingWave Stream Processor and Vector Store

Originally aired:

About the Session

Enter the exciting brave new world of GenAI, by building a ChatGPT Data Pipeline that leverages on RisingWave's efficient stream processing jobs for real-time data that we draw from an X (or Twitter feed) that's been enriched with vector data and similarity search.

We'll explore the exciting ChatGPT world, building an efficient data pipeline that's enriched with vector embeddings as stored in a vector DB (PgVector) , and how it can pair with the performant RisingWave stream processor for its write job. We will illustrate a sample use case with live
coding, as follows:

  • Simulate a streaming data feed from X (or Twitter), we'll be using Kafka as the message broker for data ingestion
  • RisingWave will consume the data stream, and perform data analysis
  • Construct prompts based on the top 3 hashtags identified by RisingWave
  • Prompts will be used for inferencing against a RAG-based BOT built with PgVector

See Highlights

Hear What Attendees Say

PwC

“Once again Saltmarch has knocked it out of the park with interesting speakers, engaging content and challenging ideas. No jetlag fog at all, which counts for how interesting the whole thing was."

Cybersecurity Lead, PwC

Intuit

“Very much looking forward to next year. I will be keeping my eye out for the date so I can make sure I lock it in my calendar."

Software Engineering Specialist, Intuit

GroupOn

“Best conference I have ever been to with lots of insights and information on next generation technologies and those that are the need of the hour."

Software Architect, GroupOn

Hear What Speakers & Sponsors Say

Scott Davis

“Happy to meet everyone who came from near and far. Glad to know you've discovered some great lessons here, and glad you joined us for all the discoveries great and small."

Web Architect & Principal Engineer, Scott Davis

Dr. Venkat Subramaniam

“Wonderful set of conferences, well organized, fantastic speakers, and an amazingly interactive set of audience. Thanks for having me at the events!"

Founder of Agile Developer Inc., Dr. Venkat Subramaniam

Oracle Corp.

“What a buzz! The events have been instrumental in bringing the whole software community together. There has been something for everyone from developers to architects to business to vendors. Thanks everyone!"

Voltaire Yap, Global Events Manager, Oracle Corp.