Richards was introduced just after the hour mark with the tie goalless, replacing captain Marc Guéhi, and ended up having a big impact.
His close-range finish broke the deadlock on 72 minutes, his third goal for the Eagles. The USA international then performed a brilliant last ditch challenge to deny Aidomo Emakhu at the other end.
Although Millwall scored a dramatic equaliser to force penalties, Richards stepped up and sent Steven Benda the wrong way to put the Eagles on the road to another shoot-out success.
"It wasn't easy, it wasn't pretty by any means," he told Palace TV. "You know it's Millwall, so I don't care how we beat them, as long as we beat them that's the big thing.
"I had to get a scrappy goal... took it to penalties, but I think we have a good record in penalties so I wasn't too worried about it once we got there. I stepped up and took it.
"We knew they were going to be focused on set-pieces and sadly they scored from one. But I think in the final third we had chances to put them away a few times. It didn't happen today, so I think we're going to be focused on that over the next few days going into the weekend."