Upgrade to 3.55, definitely. Downgrade however (from 3.56 or higher), not so much. Not without some neat soldering skills and a progskeet or teensey++ board. Chances are that a new ps3 from the store will be on a 3.56+ firmware.
Although, there is talk of a 3.66 exploit in the works from Gitbrew.