After a lot of work (it's not easy to debug monte carlo simulations with millions of rays traced) I discovered the problem was I needed to treat a distance of 0 different from a distance of nil in one bit of code.
There's a more detailed description on this card on my Swift stream.