Okay, I see the problem. Even logic level MOSFETS require 4.5V to saturate, and the signal is 3.3V.
The thing is, at a multiplexing rate of around 100Hz, you are not going to need fast turn on and turn off times, so a simple voltage shifter with a collector resistor should be fine. I made a...