Module 1 · Lecture 11

Introduction to Non-Relational Databases

Why the relational model cracked at web scale — and the four NoSQL families that replaced it.

⏱ 90 minutes 🎯 CLO1 📦 MongoDB 101

Where We Are

Ten lectures of relational thinking

You can now design a normalised schema, write joins, and reason about transactions. That toolkit solves a large share of business problems — and this module is not here to take it away.

What stays true

For structured, transactional, single-server workloads, relational databases remain the correct default. Banks still run on them.

What we add

A second toolkit for workloads where the relational model is a poor fit — and the judgement to tell the two apart.

Today's question: what kind of problem made the industry build something else?

The Breaking Point

Three pressures, roughly 2004–2010

Pressure	What Happened	Why SQL Struggled
Volume	Datasets outgrew the largest single server available	The relational model assumes one authoritative machine
Schema churn	Product shape changed weekly, not yearly	Every change is an `ALTER TABLE` on a live system
Impedance mismatch	Apps work in nested objects; tables are flat	Shred on write, join on read — forever

Scale check: PostgreSQL handles terabytes comfortably. NoSQL was invented by firms managing petabytes — a thousandfold gap.

Pressure 2 · Illustrated

Model a catalogue of books, fridges and shoes

Every product type has a different set of attributes. In a relational schema you get three unattractive options:

Option	Cost
One wide table	Hundreds of mostly-`NULL` columns; new type = schema migration
Table per type	"All products under ₹2,000" becomes a `UNION` across dozens of tables
Entity-Attribute-Value	Everything becomes a string; the database can no longer validate or index

A document database sidesteps this: each document carries exactly the fields it needs.

Pressure 3 · Illustrated

The object-relational impedance mismatch

What the app wants

{
  name: "Priya Sharma",
  addresses: [
    { type: "home", city: "Delhi" },
    { type: "work", city: "Gurgaon" }
  ]
}

What SQL requires

customers(id, name)
addresses(id, customer_id,
          type, city, pin)

-- plus a JOIN on every read

Business translation: this is paid for in developer hours — mapping layers, join debugging, migrations. A direct drag on time-to-market.

The Root Cause

Vertical vs horizontal scaling

⬆️ Scale Up

Buy a bigger machine. Simple, no app changes, joins and transactions stay easy.

Limits: cost rises faster than capacity, a hard physical ceiling, single point of failure.

➡️ Scale Out

Add commodity machines, spread the data. Capacity grows near-linearly; one node can die safely.

Limits: data is now distributed — joins, transactions and consistency get genuinely hard.

Almost everything else in this module is a consequence of choosing to scale out.

The Key Insight

Why sharding breaks joins

Sharding: split the data by key — customers A–M on Server 1, N–Z on Server 2. That is what makes horizontal scale work.

Now run SELECT ... FROM orders JOIN customers. The orders are on Server 1, the matching customers on Server 2. The database must ship data across the network mid-query.

The founding trade of NoSQL

Rather than make distributed joins fast — close to impossible — restructure the data so the join is never needed. Store the order with its customer details embedded. You accept duplication to buy horizontal scale.

Remember this slide. It explains every modelling decision in Lectures 12–16.

Distributed Systems

CAP theorem — stated correctly

Property	Meaning	If You Lose It
C Consistency	Every read returns the latest write	Users briefly see different answers
A Availability	Every request gets a response	Some requests fail or time out
P Partition tolerance	Works when the network splits	A network fault downs everything

The correction most people need

You do not choose whether to have partitions — networks fail. P is mandatory. The real choice is what to do during a split: refuse requests (CP) or answer and reconcile later (AP).

CAP as a Business Decision

The network just split. What do you do?

CP — protect correctness

Bank transfer. Refuse the transaction rather than risk double-spending a balance.

Customer sees: "Temporarily unavailable."

Systems: MongoDB, HBase, Redis Cluster

AP — protect the sale

Shopping cart. Accept the item now, reconcile versions when the network heals.

Customer sees: a working site.

Systems: Cassandra, DynamoDB, CouchDB

The question is never "which is better" — it is "what does a stale read cost us?"

Consistency Models

ACID vs BASE

	ACID	BASE
Promise	All-or-nothing; never in between	Always answers; replicas converge shortly
Optimises	Correctness	Availability and scale
Right for	Payments, ledgers, inventory, bookings	Feeds, recommendations, analytics, sessions
Wrong for	Global write scale	Anything where staleness costs money

How the debate actually resolved

MongoDB has had multi-document ACID transactions since v4.0 (2018). The modern skill is not picking a camp — it is knowing which guarantee each operation needs.

The Taxonomy

"NoSQL" is four different things

🔑 Key-Value

A giant dictionary. Sub-millisecond reads, but you can only fetch by key.

Redis, DynamoDB · caching, sessions, counters

📄 Document

Structured JSON records. Query and index inside the record.

MongoDB, Couchbase · catalogues, profiles, bookings

🧱 Column-Family

Stored by column. Enormous write throughput; rigid query patterns.

Cassandra, HBase · time-series, telemetry

🕸️ Graph

Relationships are the data. Traversal is constant-time at any scale.

Neo4j, Neptune · fraud rings, recommendations

Document Model Preview

One booking, one document

{
  "_id":           "CNR3000142",
  "status":        "Completed",
  "vehicle_type":  "Go Sedan",
  "route":   { "pickup": "Connaught Place", "drop": "Aerocity" },
  "booking_value": 412.50,
  "ride_distance": 18.4,
  "ratings": { "driver": 4.6, "customer": 4.8 },
  "payment_method": "UPI"
}

No joins. No NULL columns for the fields this booking does not use. The stored shape is the shape the application already works with.

This is the actual dataset you will query in Lectures 13–14 and visualise in Lectures 15–16.

Comparison

Pick by access pattern, not popularity

Family	Query By	Best At	Avoid When
Key-Value	Key only	Raw speed, caching	You must search by value
Document	Any field	Flexible, evolving records	Heavy multi-entity joins
Column-Family	Partition key	Massive write volume	Ad-hoc exploration
Graph	Traversal	Deep relationships	Bulk aggregation
Relational	SQL	Transactions, reporting	Petabyte horizontal scale

Exercise · 10 minutes

Choose a database. Defend it in one sentence.

#	Scenario
1	A ride-hailing firm stores 150,000 bookings a year. Fields differ by product line, and analysts need ad-hoc queries on any attribute.
2	An e-commerce site shows a live "trending in the last 5 minutes" strip on every page load.
3	An insurer wants to find rings of claimants sharing addresses, phones and repair garages.

Decide on access pattern and cost of staleness — not on which name you have heard most.

Myth-Busting

Five things people get wrong

Myth	Reality
"NoSQL means no SQL"	It means not only SQL. Many NoSQL systems ship SQL-like languages.
"There is no schema"	The schema moves into the application — it still exists.
"No transactions"	MongoDB has had ACID transactions since 4.0 (2018).
"Always faster"	Only for its intended access pattern. A bad document model loses to a good index.
"It replaces relational"	Polyglot persistence — mature firms run several, each doing its best job.

Lab · 10 minutes

Create your free Atlas cluster

Everything from Lecture 12 onward needs this. The M0 tier is free forever, needs no credit card, and gives 512 MB.

Register at mongodb.com/cloud/atlas/register with your student email
Create project BDMA → Build a Cluster → M0 Free
Region: ap-south-1 (Mumbai) for lowest latency in India
Database Access → add user bdma_student with readWriteAnyDatabase
Network Access → allow 0.0.0.0/0 (teaching cluster only)
Connect → Drivers → copy and save your connection string

⚠️ Never commit a connection string to GitHub

It contains a live password. Scanners find exposed Atlas credentials within minutes of a public push. Keep it in a local .env and .gitignore that file.

Wrap-Up

Six things to carry forward

Relational databases did not fail. A new class of workload appeared that they served badly.
Horizontal scaling causes everything else. Sharding makes joins expensive, which forces new modelling.
Partition tolerance is mandatory. The real choice is CP or AP during a split.
Four families, not one NoSQL. Key-value, document, column-family, graph.
Choose on access pattern, cost of staleness, and who operates it — never on popularity.
Polyglot persistence is normal. Run several stores, each doing its best job.

Next — Lecture 12: BSON, the document model, embedding vs referencing, and loading your first collection. Bring your connection string.