Eccentric hub (White Industries).
Eccentric BB (which will mean an outboard bearing type crank which I assume from the chainring being "non-optional" that you want to keep the cranks so an eccentric BB might be out of the question).
Half link, or half link type BMX chain.
As noted above, the chain will wear and you will need to take up the slack over time. I'd run a tensioner or a derailleur set to the right chainlink and be done with it. May not look as simple as a dedicated single speed but that's what happens with a conversion of a gearie to a single speed.
Or learn brazing, sweat the dropouts out and braze in some track tips. No such thing as overkill.