You can turn a manually configured VirtualBox VM into a Vagrant box.
Most importantly you need to create a user with
- the name
vagrant - the not-so-secret public key for SSH login
- passwordless
sudo
There's already a project for that ;-) See top level README.