Open Source EHR Refactoring Services Master Test Plan

Please see the March 13th blog entry for more information.

File: 

Comments

Some relevant offline correspondence on 3/14

Kathleen Keating's picture

 

Chris U’s question:

I read over the MTP and it highlighted CM. 

When tests or assertions fail for specific extrinsic sub/routines (I'm assuming your using MUnit); do you copy the last known CM Cache DB to ensure a clean slate? No right or wrong; I was just curious how you ensure things are working for example when the fileman global file structure goes awry.

 

 

Afsin's response:

One of the simpler MUnit test knows the global that it changes and copies back the original global back during SHUTDOWN.  The others currently do not ensure a clean slate. 

OSEHRA is looking into this problem (going back to a clean slate after each test).  I believe the current state of art is to start with a brand new database and bring it to a known state by running automated scripts in between tests.  How practical that is with a large number of tests is to be seen. 

In our case we are working in a Cache environment and as a short-term solution we will look into copying Cache.dat back and forth in between test session.

 

Chris U's response 3/22/2012

Kathleen Keating's picture

 

... yes; this is clearly a problem. Having had time to mull over it I think I have a simple, effective and unproven solution.

Transactions. Yes MUMPS supports transaction - similar smell to relational databases except with globals. The commands - TSTART, TCOMMIT and TROLLBACK. Special variable - $TLEVEL. The feature exists in GT.M and Cache.

With regards to MUnit, the idea is to begin the transaction at STARTUP, begin a sub-transaction at SETUP, execute the MUnit test, assert, then rollback OR commit the sub-transaction at TEARDOWN and finally rollback the transaction at SHUTDOWN. 

I'm in the midst of migrating thousands of miles so taking the unproven to proven won't be an exercise for me any time soon. I thought I'd share my idea before I forget. Hope this helps.

 

Mahalo,

Chris

Afsin's response 3/23

Kathleen Keating's picture
Thanks, Chris. We tried it and it is working for our tests. We will go with TSTART/TROLLBACK pairs until someone finds a problem.

How about same for RPCs?

Van Curtis's picture

So if we could do the same thing coming through the RPC broker that would be wicked cool. Any thoughts from those inside VistA as to how we might be able to do that? E.g. clients writing notes...

 

thanks,

van.

kthlnkeating - depending on

Chris Uyehara's picture

kthlnkeating - depending on the type of assertion being made, problems may occur when multiple users execute unit tests - in that case use M locks.

van - yes. we could follow the same approach; create a couple of new rpcs that support the transaction feature and viola. recipe for success. of course making assertions via rpc would be different; in the case of the note you  "write the note to vista" using the rpc and read the note using the rpc as well. the assertion would be based on what was read vs expected value. let's work on this together.

Keeping it simple

Van Curtis's picture

Chris -- we may want to start with the assumption that it's one test harness (VistA) per user to start with. In reality a locking problem would exist if you wanted to exercise simultaneous note writing as well -- even as a single test harness hitting a VistA with multiple virtual users simultaneously. In this case I'd probably remove the extra wrapper that we're using to reset the system state and have dedicate runs where the VistA was pounded on and then reset by copying the database back over or something.

I submit that if we're only using the transactions as part of the test harness and not as part of the functionality under test, then we're fine. if what we're testing is transactional note writing (for instance) and hitting the VistA simultaneously with transactions is causing a locking problem then that's a different story.

----

Regarding the RPCs, that sounds like a good idea. Currently we stub responses back from VistA for a variety of messages in our client testing (which is faster and a true "unit" test anyway) but there are certainly times when it would be nice to set up small integration tests to run over and over again in test driven development without actually filling up a test system with notes. Once the behavior is coded correctly, then it can be stubbed out.

In terms of validation, yes you can go and pull the note you just wrote, but sometimes you're just looking for the OK response.

Hmmm...what about RPCs to execute XTMUnit...would that get us anything?

Thanks Chris!

Kathleen Keating's picture

Thanks Chris!