Non-routine Software Elements Part 1: Data and Definitions

Yes, but what is VISTA made of?
 
So at the MUMPS level, VISTA is made of two kinds of software elements, routines and globals. Routines are readily version-controlled and managed (in theory) using industry-standard tools, but globals are not. Globals need to be flattened and exported in some way that focuses on the database files we are storing in those globals.
 
As we're going to see, this second "half" of the problem (globals, as opposed to routines) is really a lot more than half the problem. Isn't that always the way with computer science? The first 90% of the problem ends up taking only 10% of the project duration, and the last 10% ends up being 90% of the work, so all IT projects either go over schedule and budget or else deliver too little needed functionality. So it is here.
 
So when we start talking more seriously about VISTA's software elements, instead of talking about routines and globals - the MUMPS perspective - we talk about routine and non-routine software elements, because we're going to find in those globals a lot more than straight-forward database files. The MUMPS world is a lot more complex than that.
 
But nevertheless, for this blog entry let's focus on the database elements. When people think about databases in a casual way, they think about data - the records and values stored in the database. When they think in a more rigorous way, they also think about data definitions - how are those records and values defined? What can they store and not store?
 
Each MUMPS-based database-management system (DBMS) follows its own patterns for how it uses the flexible capabilities of MUMPS globals to encode its databases, so what follows will only apply to VISTA's DBMS, File Manager, aka Fileman.
 
In a Fileman database, three different globals are used to store each file:
 
1) the file is listed in Fileman's file of files, which is called the File file (#1), stored in global ^DIC (an abbreviation for dictionary);
 
2) the file and all its fields and indexes are defined in Fileman's data dictionary (#0), stored in global ^DD; and
 
3) the records and field values and their indexes are stored in a data global specific to that file. Some files (like the Patient file [#2]) use an entire global to store just that one file. Most files (like the Option file [#19]) just use a subscripted part of their global - ^DIC(19) - to store their file, and share that global with other files (like the Security Key file [#19.1], stored in ^DIC(19.1)).
 
So, regardless of which global location is used to store a Fileman file's data, that's just one of three global locations that would need to be flattened somehow and exported to successfully export an entire file for version control and distribution, along with the File file entry and the data-dictionary definition of the file. Since these are in three different globals - and those globals are generally shared with other files - you can see why a mechanical MUMPS-level operation like exporting an entire global cannot suffice here. Fileman needs its own export and import tool that knows about this three-fold file structure and where to look up the details. Other EHRs would follow very different patterns, and so need their own tools.
 
Where each MUMPS implementer provides simple routine-export and -import and global-export and -import tools, Fileman provides a tool called DIFROM that knows about its own file structure and lets you export and import Fileman files. It does this by reading the data and definitions of the files you want to export and writing routines to create those data and definitions in a new environment. These routines are called Init routines.
 
To export a VISTA application, then, up until the mid-1990s, you would run DIFROM to create your Init routines, then use MUMPS's routine-export tools to export your application's routines plus the newly created Init routines as OS files. At the destination site, you would use MUMPS's routine-import tools to read those OS files and load in all the updated routines, and then run the Init routines to create the database files - both data and definitions - at that site. As for version control, since routines lend themselves to it, the problem would seem to be solved. The problem of version control for non-routine software elements sounds simple enough. And of course we think we had already solved the problem for routine software elements.
 
But, as usual, there's more going on than meets the eye, as we'll discuss next time.
like0