MongoDB 101: The Document Model

Documents, BSON, _id, flexible schema — and the one modelling decision that determines whether your database is fast or miserable

Learning Objectives

• ✓ Describe a MongoDB document and map the deployment → database → collection → document → field hierarchy onto the relational model you already know
• ✓ Explain why MongoDB stores BSON rather than JSON, and name the types BSON adds
• ✓ Decompose an ObjectId into its three components and explain why it is safe to generate on the client
• ✓ State accurately what "flexible schema" does and does not guarantee, and apply $jsonSchema validation to re-tighten a collection
• ✓ Decide between embedding and referencing for a given relationship, citing access pattern, size bound, and change frequency
• ✓ Recognise the 16 MB document limit and the unbounded-array anti-pattern before they reach production
• ✓ Connect to Atlas with both Compass and mongosh, and import 15,000 ride documents into the bdma.rides collection

Session Timeline

Section 1: From Rows to Documents

In Lecture 11 we argued that document databases exist because applications think in nested objects and tables do not. This lecture makes that concrete. The unit of storage in MongoDB is the document: an ordered set of field-and-value pairs, where a value may itself be a document or an array of documents.

Here is one real record from the dataset you will load today — a completed booking from the Delhi NCR ride-hailing data.

{
  _id: ObjectId("65f1a2c4e8b9d4a1c2f30142"),
  booking_id: "CNR3000142",
  booked_at: ISODate("2024-03-14T18:22:07Z"),
  date: "2024-03-14",
  status: "Completed",
  customer_id: "CID482910",
  vehicle_type: "Go Sedan",
  route: { pickup: "Connaught Place", drop: "Aerocity" },
  avg_vtat: 7.4,
  avg_ctat: 31.2,
  booking_value: 412.50,
  ride_distance: 18.4,
  ratings: { driver: 4.6, customer: 4.8 },
  payment_method: "UPI"
}

Note what has happened. In a relational design this booking would be spread across a bookings table, a routes table, and a ratings table, joined on every read. Here it is one object, retrieved in one operation, in exactly the shape the application wants to display.

The Four-Level Hierarchy

Different Shapes in One Collection

Our dataset is deliberately realistic: a booking that never completed does not have a fare, a distance, or ratings, because those facts never came into existence. Compare the completed document above with a cancelled one:

{
  booking_id: "CNR3004193",
  booked_at: ISODate("2024-01-01T13:00:56Z"),
  date: "2024-01-01",
  status: "Cancelled by Driver",
  customer_id: "CID695170",
  vehicle_type: "Premier Sedan",
  route: { pickup: "New Delhi Railway Station", drop: "Anand Vihar ISBT" },
  avg_vtat: 2.84,
  cancellation: { by: "driver", reason: "Personal & Car related issues" }
}

No booking_value. No ride_distance. No ratings. No payment_method. Instead there is a cancellation sub-document that the completed booking does not have.

Section 2: BSON — Why Not Just JSON

You write JSON. MongoDB stores BSON — Binary JSON. The difference is invisible most of the time and decisive the rest of the time.

JSON is a text format designed for humans and for interchange. It has exactly six types: string, number, boolean, null, array, object. That is fine for sending data over a wire. It is inadequate for a database, for three reasons.

Reason 1 — JSON Has No Real Types

In JSON, 412.50 is just "a number". Is it a 32-bit integer? A 64-bit integer? A float? An exact decimal suitable for money? JSON does not say, so every reader has to guess — and different languages guess differently. BSON records the type explicitly on every value.

Reason 2 — BSON Is Traversable

Every BSON element is prefixed with its length. That means the database can skip over a field it does not need instead of parsing the whole document character by character. When you ask for route.pickup from a document with fourteen fields, the storage engine jumps straight to it. On 15,000 documents the saving is small; on 150 million it is the difference between a two-second query and a two-minute one.

Reason 3 — BSON Preserves Field Order

JSON object key order is formally undefined. BSON guarantees it, which is what makes compound index key ordering and certain comparison semantics deterministic.

Section 3: The _id Field and the ObjectId

Every document in every MongoDB collection has an _id field. It is the primary key, it is unique within the collection, and it is indexed automatically — you cannot drop that index. If you insert a document without an _id, the driver generates one for you, and by default it is an ObjectId.

Anatomy of an ObjectId — 12 Bytes

ObjectId("65f1a2c4  e8b9d4a1c2  f30142")
          --------  ----------  ------
          |         |           |
          |         |           +-- 3 bytes: counter, starts at a random
          |         |                value, increments per process
          |         +-- 5 bytes: random value, unique per machine+process
          +-- 4 bytes: seconds since the Unix epoch (creation time)

Why This Design Matters Commercially

A relational AUTO_INCREMENT key requires a round trip to the database: the application must insert, wait, and read back the assigned ID. That works on one server. On twenty application servers writing to a sharded cluster it becomes a global coordination bottleneck — a single counter everyone must queue behind.

An ObjectId is generated client-side, with no coordination at all, and is still unique. The application can construct a whole object graph, assign IDs, and write it in one batch. That is a throughput property, not a cosmetic one.

// The timestamp is extractable — no separate created_at needed
const id = ObjectId("65f1a2c4e8b9d4a1c2f30142")
id.getTimestamp()        // ISODate("2024-03-13T10:34:12Z")

// Which means you can range-query on _id alone
db.rides.find({ _id: { $gt: ObjectId("65f1a2c400000000 00000000") } })

You May Supply Your Own _id

_id does not have to be an ObjectId. It can be any unique value: a string, an integer, even a sub-document. In our dataset every booking already carries a natural key — booking_id — so a defensible design would be:

db.rides.insertOne({
  _id: "CNR3000142",          // natural key as the primary key
  status: "Completed",
  vehicle_type: "Go Sedan",
  route: { pickup: "Connaught Place", drop: "Aerocity" }
})

Section 4: Flexible Schema and the Discipline It Demands

"Schemaless" is the single most damaging word in the NoSQL vocabulary. MongoDB collections are not schemaless. They are schema-flexible: the schema is not declared to the database up front, but it absolutely still exists — in your application code, in your analysts' queries, and in your dashboards.

What Flexibility Genuinely Buys You

• Additive change with no downtime. Adding a surge_multiplier field to new bookings requires no ALTER TABLE and no maintenance window. Old documents simply do not have it.
• Heterogeneity that reflects reality. A cancelled booking and a completed one are genuinely different objects. Forcing them into identical columns is a modelling lie that someone eventually has to untangle.
• Faster iteration under uncertainty. When you do not yet know what the final record looks like — which is most of the time in a new product — you are not paying migration cost for every guess.

What It Costs You

Re-Tightening: $jsonSchema Validation

MongoDB lets you attach a validator to a collection. It is opt-in, it can be as strict or as loose as you like, and it can be tightened progressively as a product matures. This is the professional answer to "flexible schema is dangerous": you choose, per collection, where on the spectrum to sit.

db.createCollection("rides", {
  validator: {
    $jsonSchema: {
      bsonType: "object",
      required: ["booking_id", "booked_at", "status", "customer_id",
                 "vehicle_type", "route"],
      properties: {
        booking_id: {
          bsonType: "string",
          pattern: "^CNR[0-9]{7}$",
          description: "must match CNR followed by 7 digits"
        },
        booked_at: { bsonType: "date" },
        status: {
          enum: ["Completed", "Cancelled by Driver", "Cancelled by Customer",
                 "No Driver Found", "Incomplete"],
          description: "must be one of the five known booking outcomes"
        },
        vehicle_type: {
          enum: ["Go Mini", "Go Sedan", "Auto", "eBike", "Bike",
                 "Premier Sedan", "Uber XL"]
        },
        route: {
          bsonType: "object",
          required: ["pickup", "drop"],
          properties: {
            pickup: { bsonType: "string" },
            drop:   { bsonType: "string" }
          }
        },
        booking_value: { bsonType: ["double", "int"], minimum: 0 },
        ride_distance: { bsonType: ["double", "int"], minimum: 0 },
        ratings: {
          bsonType: "object",
          properties: {
            driver:   { bsonType: "double", minimum: 1, maximum: 5 },
            customer: { bsonType: "double", minimum: 1, maximum: 5 }
          }
        }
      }
    }
  },
  validationLevel: "moderate",
  validationAction: "error"
})

Read that validator as a business contract. It says: every booking must have an ID in the house format, a real timestamp, one of five known outcomes, one of seven vehicle types, and a route with both ends. It says nothing at all about booking_value being required — because for 5,657 of our 15,000 bookings it legitimately does not exist.

Section 5: Embedding vs Referencing — The Central Decision

Everything else in this lecture is vocabulary. This section is the actual skill. In a relational database the model is largely dictated by normalisation rules. In MongoDB you decide, for every relationship, whether the related data lives inside the parent document or in a separate collection pointed at by an identifier. Get it right and reads are single-operation and fast. Get it wrong and you have rebuilt a slow relational database with none of the safety.

Embedding

The related data is stored inside the parent document. Our route and ratings fields are embedded sub-documents.

{
  booking_id: "CNR3000142",
  status: "Completed",
  route:   { pickup: "Connaught Place", drop: "Aerocity" },
  ratings: { driver: 4.6, customer: 4.8 }
}
// One query returns everything. No join. Atomic single-document write.

Referencing

The related data lives elsewhere and the parent stores a pointer. Our customer_id is a reference to a separate customers collection.

// bdma.rides
{ booking_id: "CNR3000142", customer_id: "CID482910", status: "Completed" }

// bdma.customers
{ _id: "CID482910", name: "Priya Sharma", phone: "+91-98xxxxxx21",
  home_city: "Delhi", joined: ISODate("2022-06-11T00:00:00Z") }
// Two queries, or one $lookup in an aggregation (Lecture 14).

The Rule of Thumb

Applying It to Our Dataset

The Hard Limit: 16 MB per Document

A single BSON document may not exceed 16 megabytes. This is not a tuning parameter; it cannot be raised. It exists to stop any one document from monopolising memory and network bandwidth.

16 MB is enormous for a booking — our documents are roughly 300 bytes each, so the entire 15,000-record dataset is about 4.5 MB. You will never hit the limit with well-modelled data. You will hit it instantly with badly modelled data, and that is the point of the limit: it is a design smoke alarm.

The Unbounded Array Anti-Pattern

This is the single most common modelling mistake in document databases, so it is worth seeing explicitly.

// ANTI-PATTERN — do not do this
{
  _id: "CID482910",
  name: "Priya Sharma",
  bookings: [
    { booking_id: "CNR3000142", date: "2024-03-14", value: 412.50 },
    { booking_id: "CNR3000288", date: "2024-03-15", value: 180.00 },
    // ... and one more every single day, forever
  ]
}

When an Array Is the Right Answer

Arrays are not the problem; unbounded arrays are. These are fine because each has a natural ceiling:

{ tags: ["airport", "night", "surge"] }                   // a handful
{ saved_addresses: [ {label:"Home",...}, {label:"Work",...} ] }  // ~5 max
{ fare_breakdown: [ {component:"base", amount:60},
                    {component:"distance", amount:294},
                    {component:"surge", amount:58.50} ] }  // fixed set

If you cannot state the maximum number of elements with a straight face, reference instead.

Section 6: Connecting — mongosh, Compass, Drivers

There are three ways into a MongoDB cluster and you will use all three this term.

The Connection String

All three take the same SRV connection string you saved in Lecture 11:

mongodb+srv://bdma_student:<password>@cluster0.xxxxx.mongodb.net/

# Connect the shell to your cluster
mongosh "mongodb+srv://bdma_student:<password>@cluster0.xxxxx.mongodb.net/"

# Safer: let the shell prompt for the password instead of typing it
mongosh "mongodb+srv://cluster0.xxxxx.mongodb.net/" \
  --username bdma_student

Section 7: Hands-On — Load the Course Dataset

From here to the end of the course, every query you write runs against the same data: 15,000 Uber-style ride bookings from Delhi NCR covering calendar year 2024, in database bdma, collection rides. Load it now.

mongosh "mongodb+srv://bdma_student:<password>@cluster0.xxxxx.mongodb.net/"

use bdma
db                      // -> bdma
show collections        // -> (empty)

db.rides.insertOne({
  booking_id: "CNR9999001",
  booked_at: new Date("2024-03-14T18:22:07Z"),
  date: "2024-03-14",
  status: "Completed",
  customer_id: "CID482910",
  vehicle_type: "Go Sedan",
  route: { pickup: "Connaught Place", drop: "Aerocity" },
  avg_vtat: 7.4,
  avg_ctat: 31.2,
  booking_value: 412.50,
  ride_distance: 18.4,
  ratings: { driver: 4.6, customer: 4.8 },
  payment_method: "UPI"
})

// { acknowledged: true,
//   insertedId: ObjectId("65f1a2c4e8b9d4a1c2f30142") }

db.rides.find()
db.rides.find().pretty()           // formatted output
show collections                   // -> rides
db.rides.countDocuments()          // -> 1

db.rides.insertOne({
  booking_id: "CNR9999002",
  booked_at: new Date("2024-03-14T19:05:44Z"),
  date: "2024-03-14",
  status: "Cancelled by Driver",
  customer_id: "CID695170",
  vehicle_type: "Premier Sedan",
  route: { pickup: "Indira Gandhi International Airport",
           drop: "Gurgaon Cyber City" },
  avg_vtat: 2.84,
  cancellation: { by: "driver", reason: "Personal & Car related issues" }
})

// in mongosh
db.rides.deleteMany({})

# in your terminal, from the repo root
mongoimport --uri "mongodb+srv://bdma_student:<password>@cluster0.xxxxx.mongodb.net/bdma" \
  --collection rides --file data/uber_bookings.json --jsonArray

db.rides.countDocuments()                              // 15000
db.rides.countDocuments({ status: "Completed" })       // 9343
db.rides.countDocuments({ booking_value: { $exists: false } })  // 4756
db.rides.findOne({ booking_id: "CNR3000142" })

db.rides.updateMany(
  { booked_at: { $type: "string" } },
  [ { $set: { booked_at: { $toDate: "$booked_at" } } } ]
)

db.rides.findOne({}, { booked_at: 1, _id: 0 })   // now an ISODate

Explore in Compass Before You Leave

In Compass, open bdma.rides and click the Schema tab. Compass samples the collection and reports, for every field, its types and how often it appears. You will see booking_value present in only about 68% of documents. That is not an error — it is the dataset telling you the truth about failed bookings, and it is the hook for Lecture 13.

Key Takeaways