Clik here to view.
Finding a bug with code that isn’t there
A user called us with a strange bug report. He said that the SQL ETL process inside of RavenDB was behaving badly. It would write the data from the RavenDB server to the MySQL database, but then it...
View ArticleWhen the error is byzantine
In distributed systems, the term Byzantine fault tolerance refers to working in an environment where the other nodes in the system are going to violate the invariants held by the system. Sometimes,...
View ArticleNegative feature response: Automatic attachment compression in RavenDB
Following my previous post, which mentioned that you can save significantly on disk space if you store a plain text attachment using gzip, we go a feature request:Perhaps in future attachments could...
View ArticleClik here to view.
Heisenbug: The concurrent exception in the transaction that will only occur...
Recently we had to tackle a seriously strange bug. A customer reported that under a specific set of circumstances, when loading the database with many concurrent requests, they would get an optimistic...
View ArticleClik here to view.
Looking into convergent encryption
I’ll start with saying that this is not something that is planned in any capacity, I run into this topic recently and decided to dig a little deeper. This post is mostly about results of my research.If...
View ArticleClik here to view.
RavenDB 5.3 New Features: Elasticsearch ETL
RavenDB tries to be a good neighbor in your systems. RavenDB is typically used in polyglot solutions and we are often brought in to existing ecosystems. One of the things that we do to make it easier...
View ArticleClik here to view.
RavenDB 5.3 New Features: Experimental PostgreSQL wire protocol
A really nice feature that we have in RavenDB 5.3 is support for wire protocol compatibility with PostgreSQL. That opens up RavenDB to the entire PostgreSQL ecosystem. You are now able to connect to a...
View ArticleClik here to view.
RavenDB Features 5.3: Power BI integration
RavenDB is an OLTP database, it is meant to be the backend of business applications. There are some features in RavenDB that are meant for reporting purposes, but that is quite explicitly not our main...
View ArticleClik here to view.
RavenDB 5.3 New Features: Revisions includes
Enabling RavenDB’s revisions allows you to ask RavenDB to keep immutable copies of a document. We originally envisioned this feature as a way to have easy audit trails and a time travel feature....
View ArticleRavenDB Subscriptions & Messaging patterns
RavenDB is a database, not a queue or a service bus. That said, you can make use of RavenDB subscriptions to get a very similar behavior to a service bus. Let’s see how much effort it will take us to...
View Articlere: Why IndexedDB is slow and what to use instead
I ran into this post and I thought that I would share my thinking on the matter. I don’t actually know anything about IndexedDB, but I have been working with databases for a sufficiently long time to...
View ArticleClik here to view.
Optimizing local and distributed transactions with batching
I got into a good discussion about how RavenDB implements some optimizations with transaction handling. The details got big enough (and hopefully interesting enough) that they warrant their own...
View ArticleImplementing a file pager in Zig: What do we need?
A file pager is a component in database systems that is responsible for reading and writing pages (typically 8KB blocks) from the file system. The pager is responsible for the I/O operations and is...
View ArticleImplementing a file pager in Zig: Using mmap
Now that we know what we want to implement, let’s dig a bit deeper and see how to do it. An interesting way to implement a file pager is to… not do that. Instead, we can rely on the OS’ memory mapping...
View ArticleClik here to view.
Implementing a file pager in Zig: Overall design
In the previous post, I showed how we can get a pretty nice pager (important for building a storage system) in under 100 lines of code using mmap(). If that was all of it, it would be a pretty short...
View ArticleImplementing a file pager in Zig: Managing chunk metadata
The file pager needs to know what values it has in memory and what it needs from the disk. Instead of tracking values on a per page level, we are going to do that on a chunk basis, where each chunk in...
View ArticleProduction postmortem: An error on the first act will lead to data corruption...
The topic of this post is a bug in RavenDB, a pretty serious one. The end result is that a user reported that they got an error from RavenDB that they are unable to read a stored document. In some...
View ArticleClik here to view.
Implementing a file pager in Zig: Rethinking my approach
After writing the post about handling chunk metadata, I started thinking about the overall approach. Both the method using compressed pointers and the baseline computation felt… off to me. They were...
View ArticleImplementing a file pager in Zig: Reading & Writing from the disk
After implementing the memory management in the previous post, I set out to handle the actual I/O primitives that we need. As a reminder, we are separating the concerns here. We managed memory and...
View ArticleImplementing a file pager in Zig: Managing the list of files
This is my 7th post in this series, and we are now starting to get into the really interesting bits. So far we worked with the individual components, each of them doing just one thing, which makes it...
View Article