You would like to upgrade your rubygems version from 1.8.17 to the latest version 2.0.7 for example.
You can upgrade to a specific or the latest gem version and relate that to a specific rvm ruby that you have installed.
So if for example you have ruby-1.9.3-p327 and ruby-2.0.0-p247, you can have different gem version (or the same if you want) for each ruby version.
To upgrade to the latest one in the ruby-2.0.0-p247 for example you could do the following:
rvm list rvm use ruby-1.9.3-p327 gem -version 1.8.17 rvm use ruby-2.0.0-p247 gem --version 1.8.17 gem update --system gem --version 2.0.7 rvm use ruby-1.9.3-p327 gem --version 1.8.17