Developer BitsNote: This bit is largely just a collection of thoughts based on your author's attempt at self-programming the PPU during his time with it, for a simple benchmark analysis and look at how developers will be using it in codes. Ignore if you're not bothered about that, and what the SDK and runtime provide.
Programming the PPU at its most basic level is timing based, the PPU providing results at set intervals for you to use in your game.
Internal to that is the generation (Ageia call that process 'cooking') of mesh data for the PPU to chew on, before the resulting mesh (usually in screen space) is fed back into the game's renderer. Ageia recommend the developer pre-processes mesh data offline for easy further processing in real-time, since the cooker processes are computationally heavy and not accelerated by the PPU (from what your author can tell).
As far as data processing goes, the API gives you access to a range of interfaces and data structures with which to execute sims with. The API is definitely game related, the basic object types and processing commands geared toward easily simulation in computer games, be that rigid or ragdoll physics against terrain or other model geometry, vehicle simulation, collisions, fluid sims and other accessible API interfaces.
The entire API isn't hardware accelerated which somewhat dictates the programming model the developer will use to take advantage of the PPU, especially if they're relying on a framerate-based analysis for their feature levels and what to turn on or off, given the hardware in a users system.
The API does offer an easy detection mechanism on the PC, though, for deciding whether hardware is there or not, and the 'software' fallback is fairly graceful.
The current SDK has definite Novodex throwbacks, although they seem to be being shaken off as the SDK and PC system software moves forward to the 2.5.x and further releases (the current release is 2.4.x which is 32-bit only on PC).
In terms of versioning, the application programmer using the PPU can instance different versions of the PhysX runtime, and detection of available runtimes (and therefore the large chunk of any offload decision work to the PPU) is simple and to instance the SDK is bare lines of setup code to make sure it's working. There's no huge caps checking mess, you just target an SDK release and instance that.
However, that relies on prior versions of the system software being left alone when a new version is installed, and that doesn't always seem to be the case. Installing a new release of the system software will remove the version that's there and install itself, but there's no guarantees that all prior revisions will be bundled with that new release.
And, if you need an old software release on there, you need to uninstall the newer one manually, since the installer works chronologically. Therefore there's good potential for a versioning nightmare to kick in, where your PhysX-supporting games won't all work with the installed version of the software. Oops.
The SDK is C++ based, and there's no 64-bit system software for XP x64, although efforts are underway at Ageia to make that so by 2.5.x.
Overall, and this is coming from a fairly noobalicious 3D developer, it seems like the best results from integrating PhysX into a 3D game on any modern supported platform would come from a ground-up point of view. Tacking it on to any number of my existing 3D apps, most of which are small, was largely an exercise in programmer frustration. Adding it in to any existing engine framework will be an entire league on top of that, difficulty wise, so finding support inside middleware, or integrating it into a new engine from the ground-up, seems the best approach for a games developer to take.
And since it likely deviates from self-engineered physics processing and Havok Physics™ and FX™, supporting dual APIs in any app will likely be a no-go. The physics API space for games is fragmented, hinting at efforts to produce DirectPhysics inside DirectX, which Ageia could attempt to accelerate using their hardware.