Introduction to Non-Relational Databases

Why the relational model cracked at web scale — and the four NoSQL families that replaced it

Learning Objectives

• ✓ Explain the three pressures — scale, schema change, and developer velocity — that pushed the industry beyond relational databases
• ✓ Distinguish vertical from horizontal scaling and explain why sharding breaks SQL joins
• ✓ State the CAP theorem correctly and apply it to a real business scenario
• ✓ Compare ACID and BASE consistency models and judge which a given workload needs
• ✓ Classify the four NoSQL families and name a production system in each
• ✓ Apply a structured framework to recommend a database for a business problem
• ✓ Provision a free MongoDB Atlas cluster ready for Lecture 12

Session Timeline

Section 1: Why Relational Databases Hit a Wall

The relational model is one of the most successful ideas in computing. Edgar Codd published it in 1970; by 1990 it was the undisputed default, and it remains the correct answer for a large share of business systems. Nothing in this module argues that SQL is obsolete. The argument is narrower and more useful: between roughly 2004 and 2010, a specific set of workloads appeared that the relational model served badly, and the industry built new tools for them.

Understanding which workloads those are is the difference between a manager who picks technology on merit and one who picks it on fashion.

Pressure 1 — Data Volume Outgrew the Single Machine

A relational database is designed around one authoritative server. You make it faster by making that server bigger. That strategy works beautifully until your dataset no longer fits on the largest server money can buy — and by the mid-2000s, Google, Amazon, and Facebook had crossed that line.

Pressure 2 — Schemas Became a Release Bottleneck

A relational table requires you to declare every column, with its type, before you insert a single row. That rigidity is a feature: it guarantees structure. It becomes a liability when the business changes shape faster than the schema can be migrated.

Consider a product catalogue. A book has an author, an ISBN, and a page count. A refrigerator has a capacity in litres, an energy rating, and a door configuration. A pair of shoes has a size, a colour, and a material. In a relational schema you resolve this with one of three unattractive options:

A document database sidesteps the dilemma: each product is a document, and each document carries exactly the fields that product needs.

Pressure 3 — The Object-Relational Impedance Mismatch

Application code works with nested objects. A customer has a list of addresses, which each have a postcode. Relational storage cannot nest, so that single conceptual object is shredded across three or four tables on write and reassembled with joins on read.

// What the application wants to work with
{
  name: "Priya Sharma",
  addresses: [
    { type: "home", city: "Delhi",  pin: "110001" },
    { type: "work", city: "Gurgaon", pin: "122002" }
  ]
}

// What a relational schema requires
customers(id, name)
addresses(id, customer_id, type, city, pin)
-- plus a JOIN on every single read

Section 2: Scaling — Vertical vs Horizontal

This single distinction explains most of what follows, so it is worth getting precisely right.

Why Sharding Breaks Joins

Sharding means splitting a dataset across servers by some key — customers A–M on Server 1, N–Z on Server 2. Each server independently handles its slice, which is exactly what makes horizontal scaling work.

Now run a join. SELECT ... FROM orders JOIN customers requires matching every order against every customer. If orders live on Server 1 and the matching customers live on Server 2, the database must ship data across the network mid-query. On a dataset of any size, this is catastrophically slow.

Section 3: CAP Theorem and the ACID/BASE Trade-Off

Eric Brewer's CAP theorem (2000) is the most cited and most misquoted idea in distributed data. Stated carefully, it says that a distributed system can guarantee at most two of three properties:

Reading CAP as a Business Decision

Make it concrete. The network between two data centres fails. Each side can still serve customers, but they cannot talk to each other.

ACID vs BASE

These are the two consistency philosophies that follow from the CAP choice.

Section 4: The Four Families of NoSQL

"NoSQL" is an unhelpfully broad label — it describes what these systems are not. The useful taxonomy is by data model, and there are four.

1. Key-Value Stores

The simplest possible model: a giant dictionary. You supply a key, you get back a value, and the database does not inspect or interpret that value.

SET  user:8471:session  "{token: 'a9f3...', expires: 1735689600}"
GET  user:8471:session

Strength: extraordinary speed — sub-millisecond lookups, because there is no query planning to do. Weakness: you can only retrieve by key. There is no "find all sessions expiring today" unless you build that index yourself.

Systems: Redis, Amazon DynamoDB, Memcached, etcd. Typical uses: caching, session storage, rate limiting, leaderboards, real-time counters.

2. Document Databases

The value is no longer opaque. Each record is a structured document — typically JSON — and the database can query inside it, index nested fields, and enforce optional validation rules.

{
  "_id": "CNR3000142",
  "status": "Completed",
  "vehicle_type": "Go Sedan",
  "route": { "pickup": "Connaught Place", "drop": "Aerocity" },
  "booking_value": 412.50,
  "ratings": { "driver": 4.6, "customer": 4.8 }
}

Strength: the stored shape matches the application's shape, so the impedance mismatch disappears. Related data lives together, which removes most joins. Weakness: data duplicated across documents must be kept consistent by the application.

Systems: MongoDB, Couchbase, Firestore, DocumentDB. Typical uses: product catalogues, user profiles, content management, event and booking records.

3. Column-Family Stores

Data is stored by column rather than by row. Reading one column across a billion rows touches only that column's data on disk instead of dragging every other field along with it.

Strength: enormous write throughput and very fast aggregation over single attributes. Weakness: the data model is rigid and query patterns must be designed up front — you cannot explore ad hoc.

Systems: Apache Cassandra, HBase, ScyllaDB, Google Bigtable. Typical uses: time-series and sensor data, message histories, write-heavy telemetry at scale.

4. Graph Databases

The relationships are the data. Nodes represent entities and edges represent connections, both of which can carry properties.

(Priya)-[:PURCHASED]->(iPhone 15)
(Priya)-[:FOLLOWS]->(Rahul)-[:PURCHASED]->(AirPods)
// "What did people Priya follows buy?" — one hop, not a 5-table self-join

Strength: traversing relationships is constant-time regardless of total dataset size, which is exactly where SQL degrades. Weakness: poor at bulk aggregation; a smaller talent pool.

Systems: Neo4j, Amazon Neptune, ArangoDB, TigerGraph. Typical uses: fraud rings, recommendation engines, social networks, supply-chain and ownership mapping.

Side-by-Side Comparison

Section 5: A Manager's Decision Framework

You will rarely write the migration yourself. You will frequently be in the room when the choice is made, and the questions below are the ones that actually settle it.

In-Class Exercise — Three Scenarios

In groups, choose a family for each and defend it in one sentence.

Section 6: Five Myths Worth Killing

Section 7: Hands-On — Create Your Atlas Cluster

Everything from Lecture 12 onward needs a running MongoDB cluster. The free M0 tier is permanent, requires no credit card, and gives you 512 MB — comfortably more than this course needs.

mongodb+srv://bdma_student:<password>@cluster0.xxxxx.mongodb.net/

Key Takeaways