This is a nice little music player I use and have used for more than two years now. It has support for MP3, OggVorbis, And WMA. It has a skinnable interface, can normalize mp3's, cross fade, etc.
I've tested it on my own USB device and I can say it is portable. The settings and default playlist are saved in the application folder. The interface is skinable and even though the program is mostly in Japanese the interface is obvious enough to figure out and some features are explained in in skin in english.