Automated VistA installation and testing using Vagrant

I've been working on an exciting project that will make fresh VistA instance creation and development environment configuration and setup as easy as typing "vagrant up" into a clone of the OSEHRA VistA repository.

NOTE: There are more complete directions available on GitHub. Please read those in addition to the overview below.Background:

Vagrant is a popular opensource tool for deploying development environments easily and in an automated way.

Vagrant starts with a vagrant box which is a base VM to get started with for your project. OSEHRA has integrated support for Ubuntu 12.04 LTS into our repository as our base box.

OSEHRA has also chosen Virtualbox as the default provisioner for Virtual Machines which works on Linux, Mac OS X, and Windows and is an open source tool, so the whole community should be able to use the tool out of the box.

The magic is a series of shell scripts that will automatically deploy GT.M, create a VistA environment, import a VistA environment (from a git repository) and run a dashboard build which runs all of the OSEHRA tests and submits them to the OSEHRA VistA dashboard which will verify that the VistA installation works.

The glue between the box and the shell scripts is the vagrantfile which is already created for the community which specifies the vagrant box to use and how to install VistA and GT.M into Ubuntu 12.04 LTS.

Using:

It is as easy as cloning the OSEHRA VistA repository "git clone https://github.com/OSEHRA/VistA.git" type "cd Scripts/Install/Ubuntu" then type "vagrant up". The build process will than start. The process takes about an hour to run through all of the tests and submit them to the dashboard.

Benefits:

For new users this will give them a simple way to see what VistA is all about and be able to get back to a "fresh" environment at any time and without a large time investment for re-installing VistA from scratch.

For developers this allows for a quick stand up of a clean VistA system for development and testing. Also, since the systems runs all of the OSEHRA tests developers can test their new software development projects easily.

Future:

I plan on using this framework to extend the OSEHRA VM so it can be used to create new environments on the fly and be instantly up-to-date.

Feedback:

I would encourage the community to try this new tool and report feedback to me or contribute and submit a gerrit review.

Thanks:

I'd like to thank Patrick Reynolds for the initial jump start on the vagrantfile, Luis Ibáñez for the initial VistA-installation-scripts that was forked to create the installations scripts used, and Joe Snyder for some ctest help.

 
like0

Comments

Awesome!

Mark Silverberg's picture

As someone who has been keeping an eye on the VistA project but hasn't had a chance to set up a test instance (yet), this is awesome! Thank you!

like0

Automated VistA installation and testing using Vagrant

Nancy Anthracite's picture

This sounds very interesting, Chris. Are these tests being run a test to make
sure the installation is working or does it run the tests that have been
developed to test code? Is there an option of where to send the test results?
I suspect there are those that do not want to submit every run to a public
OSEHRA dashboard if that is where it is being sent.

--
Nancy Anthracite

On Saturday, October 19, 2013, Christopher.Edwards wrote:
> I've been working on an exciting project that will make fresh VistA
> instance creation and development environment configuration and setup as
> easy as typing "vagrant up" into a clone of the OSEHRA VistA repository
> [1]. Background: Vagrant [2] is a popular opensource tool for deploying
> development environments easily and in an automated way. Vagrant starts
> with a vagrant box which is a base VM to get started with for your
> project. OSEHRA has integrated support for Ubuntu 12.04 LTS into our
> repository as our base box. OSEHRA has also chosen Virtualbox as the
> default provisioner for Virtual Machines which works on Linux, Mac OS X,
> and Windows and is an open source tool, so the whole community should be
> able to use the tool out of the box. The magic is a series of shell
> scripts [3] that will automatically deploy GT.M, create a VistA
> environment, import a VistA environment (from a git repository) and run a
> dashboard build which runs all of the OSEHRA tests and submits them to the
> OSEHRA VistA dashboard [4] which will verify that the VistA installation
> works. The glue between the box and the shell scripts is the vagrantfile
> [5] which is already created for the community which specifies the vagrant
> box to use and how to install VistA and GT.M into Ubuntu 12.04
> LTS. Using: It is as easy as cloning the OSEHRA VistA repository "git
> clone https://github.com/OSEHRA/VistA.git" type "cd
> Scripts/Install/Ubuntu" then type "vagrant up". The build process will
> than start. The process takes about an hour to run through all of the
> tests and submit them to the dashboard. Benefits: For new users this will
> give them a simple way to see what VistA is all about and be able to get
> back to a "fresh" environment at any time and without a large time
> investment for re-installing VistA from scratch. For developers this
> allows for a quick stand up of a clean VistA system for development and
> testing. Also, since the systems runs all of the OSEHRA tests developers
> can test their new software development projects easily. Future: I plan on
> using this framework to extend the OSEHRA VM so it can be used to create
> new environments on the fly and be instantly up-to-date. Feedback: I would
> encourage the community to try this new tool and report feedback to me
> [6] or contribute and submit a gerrit review [7]. Thanks: I'd like to
> thank Patrick Reynolds for the initial jump start on the vagrantfile, Luis
> Ibáñez for the initial VistA-installation-scripts that was forked to
> create the installations scripts used, and Joe Snyder for some ctest help.
>
> Promote content
> Group sticky:
>
> Not Sticky
>
> Featured content:
>
> Not Featured
>
> --
> Full post:
> http://www.osehra.org/blog/automated-vista-installation-and-testing-usin...
> [8]
> Manage my subscriptions: http://www.osehra.org/og_mailinglist/subscriptions
> [9]
> Stop emails for this post:
> http://www.osehra.org/og_mailinglist/unsubscribe/3280 [10]
>
> [1] https://github.com/OSEHRA/VistA
> [2] http://www.vagrantup.com/
> [3] https://github.com/OSEHRA/VistA/tree/master/Scripts/Install
> [4] http://code.osehra.org/CDash/index.php?project=Open+Source+EHR
> [5]
> https://github.com/OSEHRA/VistA/blob/master/Scripts/Install/Ubuntu/Vagrant
> file [6] mailto:Christopher.Edwards@krminc.com
> [7] http://www.osehra.org/page/contributor-git-instructions
> [8]
> http://www.osehra.org/blog/automated-vista-installation-and-testing-using-
> vagrant [9] http://www.osehra.org/og_mailinglist/subscriptions
> [10] http://www.osehra.org/og_mailinglist/unsubscribe/3280

like0

Nancy: Are these tests being

Christopher Edwards's picture

Nancy:

Are these tests being run a test to make sure the installation is working or does it run the tests that have been
developed to test code?

The answer is yes to both. However, the tests that are run have to be known to the OSEHRA test framework. There are future enhancements to make this easier to develop tests and select the tests to run.

Is there an option of where to send the test results?

Not at the moment.

I suspect there are those that do not want to submit every run to a public OSEHRA dashboard if that is where it is being sent.

Understood. The dashboard that it runs is designed for developers as they are developing tests and code, but it depends upon the developer as to if they want it public or not.

As mentioned before there are items to be enhanced such as the enabling/disabling of tests, and being able to install & configure more than just VistA/GT.m (EWD, etc.). There is a TODO in the repository and scattered TODOs throughout the code and contributions are welcome.

like0

Automated VistA installation and testing using Vagrant

Wes Turner's picture

Nancy,

The submission site is encoded in the file CTestConfig.cmake at the top of
the source tree. Right now vagrant appears to clone directly from the
OSEHRA repository, so whatever dashboard submission site is encoded in that
repository becomes the destination site. However, there are options. For
one, once you have the VM up using vagrant, changing the submission site
should be fairly trivial using a text editor. This would result in only the
first test run going out to the external site. All others could be sent to
the dashboard of your choice including a privately held dashboard.
Alternately, you can modify Scripts/Install/Ubuntu/autoInstaller.sh to
point to a personal repository instead of to the OSEHRA repository. This
would allow you to load your own private VistA configuration, including
dashboard site, rather than the OSEHRA version. There are other options we
can consider as well, including setting configuration variables to the
vagrant script.

Good job Chris!

- Wes

On Sat, Oct 19, 2013 at 1:12 PM, Christopher.Edwards <
Christopher.Edwards@krminc.com> wrote:

> Nancy:
>
> Are these tests being run a test to make sure the installation is working
> or does it run the tests that have been
> developed to test code?
>
> The answer is yes to both. However, the tests that are run have to be
> known to the OSEHRA test framework. There are future enhancements to make
> this easier to develop tests and select the tests to run.
>
> Is there an option of where to send the test results?
>
> Not at the moment.
>
> I suspect there are those that do not want to submit every run to a
> public OSEHRA dashboard if that is where it is being sent.
>
> Understood. The dashboard that it runs is designed for developers as they
> are developing tests and code, but it depends upon the developer as to if
> they want it public or not.
>
> As mentioned before there are items to be enhanced such as the
> enabling/disabling of tests, and being able to install & configure more
> than just VistA/GT.m (EWD, etc.). There is a TODO in the repository and
> scattered TODOs throughout the code and contributions are welcome.
> --
> Full post:
> http://www.osehra.org/blog/automated-vista-installation-and-testing-usin...
> Manage my subscriptions:
> http://www.osehra.org/og_mailinglist/subscriptions
> Stop emails for this post:
> http://www.osehra.org/og_mailinglist/unsubscribe/3280
>

--
Wesley D. Turner, Ph.D.
Kitware, Inc.
Technical Leader
28 Corporate Drive
Clifton Park, NY 12065-8662
Phone: 518-881-4920

like0

Dashboard no submit

Christopher Edwards's picture

Nancy:

You may be able to add "set (dashboard_no_submit TRUE)" to the test.cmake file in a future update to disable sending the tests to the dashboard.

like0

Updates!!!

Christopher Edwards's picture

So, I've been busy this weekend updating the scripts so that the VistA RPC Broker and VistAlink work out of the box.

To update to the latest VM follow these steps

  1. cd VistA/Scripts/Install/Ubuntu
  2. vagrant destroy (WARNING: This will erase any information contaned within the VM, you will need to transfer it out if you want to keep it)
  3. git pull (This will bring in any new changes from OSEHRA's VistA repository)
  4. vagrant up (Create a new VM using the changes that have been merged into the repository)

I'm also working on documentation. I'll be putting it on OSEHRA's wiki when I get a chance, but I'll post some relevant details here:

  • It uses a Ubuntu 12.04 LTS server base box (about 330MB - http://cloud-images.ubuntu.com/vagrant/precise/current/precise-server-cloudimg-i386-vagrant-disk1.box)
  • Installs GT.M using gtminstall script on FIS GTM's sourceforge page (version 0.12)
  • Clones OSEHRA's VistA-M repository (flavor: FOIA VistA)
  • Does basic configuration work for GT.M and VistA
  • Runs the tests in OSEHRA's VistA repository which runs some basic VistA configuration tasks, XINDEX, Roll-and-Scroll Functional tests, and Python tests/unit tests. MUnit testing is comming soon. (Implementation detail: runs a dashboard build using OSEHRA VistA repository's dashboard branch)
  • Submits the testing results to the public OSEHRA Dashboard. (See discussion above for more details)

Thanks!

like0

Express installation

Rafael Richards MD, MS's picture

Thank you!   This is a very important  project and will greatly accelerate the development lifecycle for VistA / M development.

I successfully installed your latest release (as of 11/26/13) and it installed without a problem.  However, this took nearly two hours on an eight processor mac server.  It seems that most of the installation time was consumed with the testing.

Are there plans to make an express instaIter that would allow us to toggle different installation configurations (express, testing, full,  EWD, etc.)?

Again, thank you. Keep up the great work.

 

like0

Installation Options

Christopher Edwards's picture

rrichards:

I'm working on creating options right now for various installation names which could easily be expanded for the other options mentioned (testing on/off, EWD, etc.).

As far as the time goes the current defaults only do minimal testing (the tests are either quick to complete on a routine basis, or they enter data into the system). Looking at my log files it appears it takes less than 10 minutes to perform the testing. The most time consuming tasks are the following: cloning VistA-M (the raw routines and globals - .m and .zwr files respectively) and the importation of the routines and globals. The VistA-M repo is a rather large repo (a quick estimate shows about 3GiB), however it should compress rather well when retrieved with git, but it still takes a while. The importation process is heavily dependant on hard drive performance - SSDs would give the best performance (More IOPS available) and spinning disk would take longer (Less IOPS available) - this one I have absolutely no control over.

Also, the VM downloads several things over the internet as it works so it is also heavily dependant on internet download speeds.

Controlling how to clone VistA-M (and VistA) is the only place I can currently see where I have control to speed up (less downloading of items) the process. I'm currently looking into this as well since if you are doing local development downloading the latest public release probably won't do you any good.

For timing comparison on my 16GB RAM/2.6GHz i7/512GB SSD MacBook Pro it took 24 minutes from a vagrant up to completed system. This includes re-downloading the vagrant box (Ubuntu 12.04) from the internet. Why it took 2 hours on your machine I don't know.

Commands used:

vagrant box remove "Official Ubuntu 12.04 current daily Cloud Image i386" virtualbox

time vagrant up

Response from time command:

real 24m2.471s

user 0m4.454s

sys 0m6.681s

I do agree 2 hours is too long, but the only thing I can think of is download time (internet bandwith) causing the increase in time, but I'm unsure of that as I've done the same process on hotel wifi and it would take around the same time or only slightly more (never more than 1 hour).

Thanks for the feedback!

like0

space

Chris Casey's picture

Chris,

on an 8gb ec2 box there isn't an awful lot of space left for other things.

Is it safe to delete the /dashboard directory tree after installation?

Chris

like0

Automated VistA installation and testing using Vagrant

Rob Tweed's picture

Chris (Casey)

Why not set up EBS storage, attach it to the EC2 VM and use that for the disc storage? The added benefit is that you’ll not lose everything if the VM terminates. You’ll need to manually shift some files around of course.

Rob

On 13 Mar 2014, at 08:02, ChrisUK <borochris9@gmail.com> wrote:

> Chris,
>
> on an 8gb ec2 box there isn't an awful lot of space left for other things.
>
> Is it safe to delete the /dashboard directory tree after installation?
>
> Chris
>
> --
> Full post: http://www.osehra.org/blog/automated-vista-installation-and-testing-usin...
> Manage my subscriptions: http://www.osehra.org/og_mailinglist/subscriptions
> Stop emails for this post: http://www.osehra.org/og_mailinglist/unsubscribe/3280

like0

should be safe

Christopher Edwards's picture

Chris:

The Dashboard directory should be safe to delete. It is used during the testing/setup process. You will lose access to running the testing environment though.

I 2nd Rob's comment of using EBS since it allows for access to your data if the VM terminates for any reason, and you'll have more space.

Thanks!

like0

What Next?

James Graham's picture

I downloaded vagrant and followed your instructions on a Win 7x64 machine.  Looks like it downloaded Ubuntu, set it up under Virtualbox, and set up and tested Vista.  It shows that it has completed but when I go into Virtualbox, Ubuntu seems to be still running.

What's up?  How do I actually run Vista now?

 

Thanks, Steve

like0

What Next?

James Graham's picture

I used your script to download Ubuntu, set it up under Virtualbox and setup/test Vista.  It's finished, but Virtualbox shows Ubuntu still running.  What's up?  How do I run/inspect Vista?

 

Thanks, Steve

like0

Automated VistA installation and testing using Vagrant

DAVID Whitten's picture

Since VistA has a VIRTUAL Operating Operating system, you have to invoke
the User Interface to the VistA system. The VistA system runs on top of a
MUMPS implementation. You need to run the command line/Read-Eval-Print-Loop
of your MUMPS system.

This will always involve running a Linux shell or a terminal emulation
program to connect to the server.

For GT.M, start by setting up some environment variables and running the
mumps -direct command
For Cache, start by using the Cube.

On Wed, Oct 1, 2014 at 9:19 AM, jsgrahamus <jsgrahamus@yahoo.com> wrote:

> I used your script to download Ubuntu, set it up under Virtualbox and
> setup/test Vista. It's finished, but Virtualbox shows Ubuntu still
> running. What's up? How do I run/inspect Vista?
>
>
>
> Thanks, Steve
> --
> Full post:
> http://www.osehra.org/blog/automated-vista-installation-and-testing-usin...
> <http://www.osehra.org/blog/automated-vista-installation-and-testing-usin...
> Manage my subscriptions:
> http://www.osehra.org/og_mailinglist/subscriptions
> Stop emails for this post:
> http://www.osehra.org/og_mailinglist/unsubscribe/3280
>

like0

I need help please with provisioning my VistA development VM

Jeff Austin's picture

I successfully followed most of the steps Chris Edwards so kindly shared about setting up an OSEHRA VistA development VM, but when I type "vagrant up" at my git-bash prompt in the directory "~/Development/VistA/Scripts/Install/Ubuntu", I'm getting this message:

sh.exe": ./vagrant: No such file or directory

I expect this means that there is a problem resolving the path to the vagrant executable, but I'm not sure how to go about resolving the matter.

In the current directory, a listing of the contents shows the following:

Vagrantfile, autoInstaller.sh, bootstrapUbuntuServer.sh, setupEC2.sh, setupRackspace.sh, and test.cmake

The host for my VM is, unfortunately, Windows Vista Home Premium SP1.

My thanks to anyone who is able to provide some guidance - Jeff.

like0

Automated VistA installation and testing using Vagrant

Wes Turner's picture

Did you install vagrant? https://www.vagrantup.com/downloads.html

- Wes

On Fri, Oct 31, 2014 at 12:02 PM, Jeff Austin <jeffery.austin@va.gov> wrote:

> I successfully followed most of the steps Chris Edwards so kindly shared
> about setting up an OSEHRA VistA development VM, but when I type "vagrant
> up" at my git-bash prompt in the directory
> "~/Development/VistA/Scripts/Install/Ubuntu", I'm getting this message:
>
> sh.exe": ./vagrant: No such file or directory
>
> I expect this means that there is a problem resolving the path to the
> vagrant executable, but I'm not sure how to go about resolving the matter.
>
> In the current directory, a listing of the contents shows the following:
>
> Vagrantfile, autoInstaller.sh, bootstrapUbuntuServer.sh, setupEC2.sh,
> setupRackspace.sh, and test.cmake
>
> The host for my VM is, unfortunately, Windows Vista Home Premium SP1.
>
> My thanks to anyone who is able to provide some guidance - Jeff.
> --
> Full post:
> http://www.osehra.org/blog/automated-vista-installation-and-testing-usin...
> <http://www.osehra.org/blog/automated-vista-installation-and-testing-usin...
> Manage my subscriptions:
> http://www.osehra.org/og_mailinglist/subscriptions
> Stop emails for this post:
> http://www.osehra.org/og_mailinglist/unsubscribe/3280
>

--
Wesley D. Turner, Ph.D.
Kitware, Inc.
Technical Leader
28 Corporate Drive
Clifton Park, NY 12065-8662
Phone: 518-881-4920

like0

Thank you for replying.

Jeff Austin's picture

Thank you for replying. Starting with the fundamentals is a good idea.  Yes. I installed the Windows Universal (32 and 64 bit) version of Vagrant.

like0

Running Vagrant Executable

Joseph Snyder's picture

Hi Jeff,

The "vagrant up" that you entered attempts to run the "vagrant" executable with the argument "up".  The command you have used assumes that the exectuable is found in one of the directories in your PATH variable.

To get around adding it to the PATH environment variable, simply use the full path to the executable.

 For my Windows machine using the GIt Bash, it would look like:

joe.snyder@PALAVEN ~/Work/OSEHRA/VistA/Scripts/Install/Ubuntu (master)
$ /c/HashiCorp/Vagrant/bin/vagrant up
 
The full path for you would depend on where you installed the program.  And FYI, "/c/HashiCorp" is the gitBash way of expressing "C:/HashiCorp".
 
Hope that helps,
 
Joe Snyder
 
like0

I'm making progress now

Jeff Austin's picture

Thank you for replying. Starting with the fundamentals is a good idea.  Yes. I installed the Windows Universal (32 and 64 bit) version of Vagrant.

I checked the PATH environment variable in git-bash and didn't see anything set there by default during the installation process that looked like it pointed to vagrant. I "Googled" for Vagrant installation path and and files to learn what gets installed and searched for those items on my hard drive. I found Vagrant was installed to C:\HashiCorp\Vagrant on my PC.  I updated my path at the git-bash prompt by entering:  export PATH="/c/HashiCorp/Vagrant/bin:$PATH"

 

I retried the "vagrant up" command and it is running now. 

 

Looks like I need to edit my PATH environment variable setting in my git-bash profle located at C:\Program Files\Git\etc\profile so I'll always have Vagrant in my git-bash PATH.

like0

Did you use the Github directions?

Christopher Edwards's picture

+1 to Wes regarding installation of vagrant.

Also, please pay attention to the note attached to this page to use the github directions as those are kept more up-to-date and have links to all dependencies.

like0