If you’ve been working with MongoDB for any length of time, you’ve probably encountered the dreaded E11000 duplicate key error. This error can be frustrating, especially when you’re not sure why it’s happening or how to fix it. In this guide, I’ll break down everything you need to know about this error and provide practical solutions that actually work.
What is the E11000 Duplicate Key Error?
The E11000 duplicate key error occurs when MongoDB tries to insert or update a document that violates a unique index constraint. In simple terms, you’re attempting to store a value that already exists in a field that must be unique.
Understanding the Error Message
Here’s what a typical E11000 error looks like:
E11000 duplicate key error collection: myapp.users index: email_1 dup key: { email: "john@example.com" }
Let’s break this down:
myapp.users: The database and collection where the error occurredemail_1: The name of the index that’s causing the problem{ email: "john@example.com" }: The specific duplicate value
Common Causes of E11000 Errors
1. Attempting to Insert Duplicate Values
The most straightforward cause is trying to insert a document with a value that already exists in a unique field.
// This will work fine the first time
db.users.insertOne({ email: "user@example.com", name: "John" });
// This will throw E11000 error if run again
db.users.insertOne({ email: "user@example.com", name: "Jane" });
2. Multiple NULL Values in Unique Fields
MongoDB treats null as a value, so if you have a unique index on a field, you can only have one document with a null value for that field.
// Schema with unique email field
const userSchema = new mongoose.Schema({
name: String,
email: { type: String, unique: true } // Problem: allows only one null value
});
// First user without email - works fine
db.users.insertOne({ name: "User1" }); // email is null
// Second user without email - E11000 error!
db.users.insertOne({ name: "User2" }); // email is also null, causes duplicate
3. Stale Index Constraints
Sometimes you change your schema but old indexes remain in the database, causing conflicts.
// You might have removed unique constraint from your schema
const userSchema = new mongoose.Schema({
email: String // No longer unique in schema
});
// But the old unique index still exists in MongoDB
// This will still throw E11000 errors
4. Concurrency Issues
When multiple operations try to insert the same value simultaneously, race conditions can cause E11000 errors.
Solution Methods (Step-by-Step)
Method 1: Remove Duplicate Data
When to use: You have actual duplicate data in your collection.
Step 1: Identify duplicates
// Find all duplicate emails
db.users.aggregate([
{ $group: { _id: "$email", count: { $sum: 1 }, docs: { $push: "$_id" } } },
{ $match: { count: { $gt: 1 } } }
]);
Step 2: Remove duplicates (keep the most recent)
// Remove duplicates, keeping only the latest document
db.users.aggregate([
{ $group: {
_id: "$email",
docs: { $push: { id: "$_id", createdAt: "$createdAt" } },
count: { $sum: 1 }
}},
{ $match: { count: { $gt: 1 } } }
]).forEach(function(doc) {
// Sort by creation date (newest first)
doc.docs.sort((a, b) => new Date(b.createdAt) - new Date(a.createdAt));
// Delete all but the first (newest) document
for (let i = 1; i < doc.docs.length; i++) {
db.users.deleteOne({ _id: doc.docs[i].id });
}
});
Method 2: Drop and Recreate Problematic Indexes
When to use: You have stale indexes or need to change index configuration.
Step 1: Check existing indexes
// View all indexes on the collection
db.users.getIndexes();
Step 2: Drop the problematic index
// Drop specific index by name
db.users.dropIndex("email_1");
// Or drop by specification
db.users.dropIndex({ email: 1 });
// Drop all indexes except _id (use with caution!)
db.users.dropIndexes();
Step 3: Recreate the index with proper configuration
// Create a sparse unique index (ignores null values)
db.users.createIndex(
{ email: 1 },
{ unique: true, sparse: true }
);
Method 3: Handle NULL Values with Sparse Indexes
When to use: You need unique constraints but want to allow multiple documents without the field.
The Problem: Regular unique indexes consider null as a value, allowing only one document with a missing field.
The Solution: Use sparse indexes that ignore documents where the indexed field is missing or null.
// Instead of this (allows only one null)
db.users.createIndex({ email: 1 }, { unique: true });
// Use this (ignores null/missing values)
db.users.createIndex({ email: 1 }, { unique: true, sparse: true });
In Mongoose schemas:
const userSchema = new mongoose.Schema({
name: { type: String, required: true },
email: {
type: String,
unique: true,
sparse: true // This is the key!
}
});
Method 4: Use Upsert Operations
When to use: You want to insert if the document doesn’t exist, or update if it does.
Basic upsert example:
// Instead of trying to insert directly
try {
const result = await db.users.updateOne(
{ email: "user@example.com" }, // Filter
{
$set: { name: "John Doe", email: "user@example.com" },
$setOnInsert: { createdAt: new Date() }
},
{ upsert: true } // Insert if not found, update if found
);
if (result.upsertedCount > 0) {
console.log("New user created");
} else {
console.log("Existing user updated");
}
} catch (error) {
console.error("Upsert failed:", error);
}
With Mongoose:
async function createOrUpdateUser(userData) {
try {
const user = await User.findOneAndUpdate(
{ email: userData.email },
userData,
{
upsert: true,
new: true,
setDefaultsOnInsert: true
}
);
return user;
} catch (error) {
if (error.code === 11000) {
throw new Error("Email already exists");
}
throw error;
}
}
Method 5: Pre-Insertion Validation
When to use: You want to check for duplicates before attempting insertion.
async function createUser(userData) {
// Step 1: Check if user already exists
const existingUser = await User.findOne({ email: userData.email });
if (existingUser) {
throw new Error("User with this email already exists");
}
// Step 2: Create new user
try {
const newUser = new User(userData);
return await newUser.save();
} catch (error) {
if (error.code === 11000) {
// Handle race condition where user was created between check and insert
throw new Error("Email was taken by another user");
}
throw error;
}
}
Error Handling Best Practices
Graceful Error Handling in Express.js
app.post('/users', async (req, res) => {
try {
const user = new User(req.body);
await user.save();
res.status(201).json(user);
} catch (error) {
if (error.code === 11000) {
// Extract the field name from the error
const field = Object.keys(error.keyPattern)[0];
return res.status(400).json({
error: `${field} already exists`,
field: field,
value: error.keyValue[field]
});
}
res.status(500).json({ error: "Internal server error" });
}
});
Bulk Operations with Error Handling
// For bulk inserts, use ordered: false to continue on errors
try {
const result = await db.users.insertMany(
usersArray,
{ ordered: false } // Continue inserting even if some fail
);
console.log(`${result.insertedCount} users created successfully`);
} catch (error) {
console.log(`${error.result.nInserted} users created`);
console.log(`${error.writeErrors.length} duplicates skipped`);
// Log specific errors if needed
error.writeErrors.forEach(err => {
console.log(`Duplicate: ${JSON.stringify(err.getOperation())}`);
});
}
The E11000 duplicate key error is MongoDB’s way of maintaining data integrity, but it can be challenging to handle properly. The key is understanding why it occurs and choosing the right solution for your specific situation.
Remember these key points:
- Use sparse indexes when fields are optional but should be unique when present
- Implement proper error handling to provide meaningful feedback to users
- Consider upsert operations to avoid race conditions
- Clean your data before adding unique constraints
- Test thoroughly with edge cases like null values and concurrent operations
By following these practices, you’ll be able to handle E11000 errors gracefully and build more robust MongoDB applications.