A home-defense gun has to be simple to run when you are half awake, your heart is hammering and the lights are off. Everything about the optic choice follows from that one sentence: you want fewer decisions, not more capability.
Start with a red dot
A red dot is the right default. Both eyes stay open, so you keep your peripheral vision and situational awareness — which matters enormously in a house, where what is beside and behind the target is the whole problem. There is no magnification to fumble, no eye relief to find, and the dot works from any head position, including the awkward ones you will be in while moving around furniture in the dark.
Quality units run two to five years on a single cell left permanently switched on, which is the only acceptable arrangement for a defensive optic. Shake-awake features that wake the dot on movement are useful; a manual on-off switch you have to remember to press at 3am is not.
Keep the reticle simple. A 2 MOA dot, or a 65 MOA ring with a 2 MOA center dot, is as complicated as this needs to get. Set the brightness to a mid setting that is visible in a dark room without blooming — not the maximum, which will flare badly indoors and degrade your dark adaptation.
A weapon light is not optional
This is the part people get wrong, and it is more important than the optic.
You cannot shoot what you have not identified, and you are morally and legally responsible for identifying it. In a house at night, the thing moving at the end of the hallway may be a family member, a guest, or someone who does not belong there, and you do not get to find out afterward. A weapon light is what makes that determination possible.
For indoor use, do not reach for the brightest light available. Something in the 300 to 800 lumen range with moderate beam intensity lights a room and a hallway without reflecting off a white wall and blinding you. Extreme output belongs outdoors. See how to choose a weapon light for the full breakdown of lumens, candela and runtime.
Carry a handheld light as well. A mounted light requires you to point the gun at whatever you want to look at, and most of what you will want to look at is not something you should be pointing a gun at. A handheld solves that and costs almost nothing.
Carbine or pistol?
A carbine is substantially easier to shoot well under stress, hits harder, and gives you a stable platform with three points of contact. A pistol is easier to maneuver in tight spaces and leaves a hand free for a door, a light or a phone.
If you can only build one, the carbine with a dot and a light is the more capable defensive tool for most people, and the easier one to become competent with. The pistol wins on retention in close quarters and on the practical reality that it is what many people already own and already shoot.
Either works with the same formula. If you go carbine, the best optic for an AR-15 covers dot versus LPVO, and the answer for this role is almost always the dot. Consider co-witnessing backup irons for redundancy — a lower one-third setup costs you nothing in sight picture.
What about a shotgun?
A shotgun is an effective home-defense tool and benefits from a red dot more than most people assume — bead sights are crude, and a dot makes the gun far more precise at the distances where you must not miss. Recoil is the consideration: choose an optic rated for shotgun use and mount it properly, because a dot that shakes loose after a box of buckshot is worse than no dot.
Keep it simple, then maintain it
Zero the dot — the procedure is in how to zero a red dot — and confirm it. A 25-yard zero on a pistol or a 50-yard zero on a carbine covers every distance inside a house with no thinking required.
Then put a date on the battery and change it annually whether or not it has failed, check the mount screws every few months, and shoot the gun with the light on at least occasionally, because running a light is a skill and the first time should not be the real time.
A simple, reliable dot-and-light setup you can operate without thinking beats a complicated build every time. Browse rifle dot sights, pistol dot sights and weapon lights to put one together.