Allowing Gitlab CI Runners to use RVM is really easy:

# Login as a super user (or use sudo)
su
# Switch to gitlab CI runner user
su - gitlab_ci_runner
# Go to his home directory
cd ~
# And install RVM
\curl -sSL https://get.rvm.io | bash -s stable --ruby
# Now go back to su
exit
# And restart (just in case) gitlab and ci
/etc/init.d/gitlab restart
/etc/init.d/gitlab_ci restart

That’s all. Just keep in mind, that each time you want a new Ruby version, you will have to login as gitlab_ci_runner and install it from console.