Sterling McGarvey at GameSpy peers into my hopes and dreams for the PSP. When the PSP and DS were new, that little USB port and PC-friendly (at least to my PCs) memory stick caused me to coin a phrase: "The DS is a toy. PSP is a way of life."
I thought for sure that there'd be homebrew out the wazoo (until I foolishly went along with the firmware upgrades, cutting me off from that world, at least until they get a 7.1 downgrader), countless USB attachments, and easily downloadable content of all kinds. (Sony, try slimming down the clutter on your PSP page.)
Some stuff I'd like to see in the PSP's future:
A cellphone attachment. N-Gage had their whole approach backwards. The PSP already has a powerful (at least on the scales I'm used to) processor and a decent display. If it can be done on a cell phone, you should be able to do it better on PSP.
PDA functionality. Can't be too hard to write the software. Would go well with a USB keyboard attachment.