Many games are "not" backwards compatible on either system.
With the PS3 you need to have a "launch" PS3 that contained the PS2 chip. Trying to get one of these units is not hard, but very expensive. EB some times has them, but eBay is your best bet. Be prepared to pay big at either outlet
The XBOX 360 requires software updates to play XBOX titles and a lot of these are buggy, don't work or have not even been programed.
Some back catalog games have been released on the PSN or XBL stores and have been re-written to work correctly on the respective platform. These games are few in number, many times expensive (especially in the case of XBOX) and don't look as good as you remember
As far as I am aware there have been no PS2 games done in this manor (only PS1, which have been remade with the PSP in mind).
I will have to go with A_1 on this one. If you want to play back catalog games, get the original console.
(And with that in mind I may just head down to PSX Warehouse and see if they have a restored Killer Instinct Arcade machine in stock ... DON'T TELL MY WIFE WHERE I AM GOING!