Although data warehouses offer SQL support and higher performances, they require a lot of time, effort and money that many businesses don’t have. But data lakes offer a more affordable, albeit less comprehensive, data storage.
Data lakehouses, however, combine the scalability of data lakes and the advanced capabilities of warehouses into one affordable solution. Dremio Corp. is one of these data lakehouse providers, helping businesses progress their digital transformation journeys at a cost-effective price point.
“What we’ve done now as an industry with the lakehouse technologies, like Apache Iceberg, is we’ve unlocked all the capabilities of the warehouse directly on object storage like S3,” said Tomer Shiran (pictured), co-founder and chief product officer of Dremio. “You can insert and update and delete individual records. You can do transactions. You can do all the things you could do with a database directly in kind of open formats without getting locked in at a much lower cost.”
Shiran spoke with theCUBE hosts Lisa Martin and Paul Gillin at the recent AWS re:Invent conference, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed data lakehouses, how Dremio makes data lakehouses accessible and affordable, Dremio’s query engine and more. (* Disclosure below.)
A new, affordable cloud
Cloud vendors are pushing toward data lakehouses, with Amazon Web Services Inc. announcing earlier it will be eliminating the extract, transform and load stage between its relational database service and Redshift, Shiran explained.
“We’re all coming together around standard ways to represent data so that at the end of the day, companies have this benefit of being able to have their data in their S3 account in open formats and be able to use all these different engines without losing any of the functionality that they need,” he said.
Dremio has been busy the past year with announcements, such as full Apache Iceberg support and the introduction of two different “flavors” of Dremio. Dremio Cloud, the fully SaaS version, is fully hosted and integrated with Google and Azure, and Dremio software can be self-hosted in and out of the cloud.
“We’re also excited about this new idea of data as code, and so we’ve introduced a new product that’s now in preview called Dremio Arctic. And the idea there is to bring the concepts of Git or GitHub to the world of data,” Shiran said. “Things like being able to create a branch and work in isolation. If you’re a data scientist, you want to experiment on your own without impacting other people, or you’re a data engineer and you’re ingesting data, you want to transform it and test it before you expose it to others.”
Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of AWS re:Invent:
(* Disclosure: Dremio Corp. sponsored this segment of theCUBE. Neither Dremio nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)