Module 1 calling Module 2, over.

Last week we started to look into XINDEX utility as a tool to characterize VistA Mumps codebase.

XINDEX is a Mumps routine which parses and analyzes all the routines in VistA install.  The analysis includes a variety of useful information, including:

  • Errors and warnings for the code based on Mumps language and SAC (The Department of Veterans Affairs M Programming Standards and Conventions)
  • List of all calls to other routines
  • List of calls within the routine
  • Global and naked global usage in the routine
  • Local variable usage in the routine

In addition to routines, XINDEX also looks into package specific VistA file items that store Mumps executable code termed as “faux routines” in XINDEX output.  We are not sure yet if the items that XINDEX looks into are the complete list, but looking more closely into “faux routines” will need to be part of the characterization of the Mump codebase.

The main task we concentrated on this week was listing all module to module dependencies.  XINDEX already provides a report containing the routines that other routines and faux routine call to; our task was to convert this information into a module dependency map.

 For this, we did not use XINDEX output directly.  XINDEX stores all its analysis internally in a global called ^UTILITY and this global is used to generate the actual output reports.  In our case, the global was much more useful than the reports so our first step was to modify XINDEX code so that it became “silent” (no writes) -- we specified the input parameters programmatically to XINDEX and used ^UTILITY to extract the information we needed.

To be able to specify module dependency map, we needed to know which routines belong to which modules; for now we are using Package file in VistA (#9.4).  We decided to go this route because XINDEX itself uses this file to identify faux routines.  This file appears to include all the namespaces: the first few letters of the routine names which identify the package the routine belongs to.  It is our understanding that Package file is kept up-to-date, but we need to better understand how the packages relate to modules as defined in the OSEHRA architecture document.  In fact, there have been some discussions on weekly architecture meetings on the very same topic.   It looks like some of the documented modules are not in the OSEHRA VistA release, but rather some of them are merged, etc.  We will try to clarify this for VA before characterization is complete.

We ran our modified version of XINDEX for two modules.  Oncology (ONC) and Problem List (GMPL).  The results are as follows:

ONCOLOGY depends on: REGISTRATION, VA FILEMAN, PROBLEM LIST, DRG GROUPER, UNKNOWN, MAILMAN, KIDS, KERNEL

Nothing depends on ONCOLOGY.

PROBLEM LIST depends on: REGISTRATION, VA FILEMAN, HEALTH SUMMARY, INTEGRATED BILLING, DRG GROUPER, LEXICON UTILITY, ORDER ENTRY/RESULTS REPORTING,  PAID, PCE PATIENT CARE ENCOUNTER, SCHEDULING, UNKNOWN, LIST MANAGER, MAILMAN, KIDS, KERNEL

The following depends on PROBLEM LIST: DIETETICS, HEALTH SUMMARY, NATIONAL HEALTH INFO NETWORK, ONCOLOGY, ORDER ENTRY/RESULTS REPORTING, PCE PATIENT CARE ENCOUNTER, CLINICAL CASE REGISTRIES, TEXT INTEGRATION UTILITIES

For Problem List the dependencies are similar to what is listed in the OSEHRA architecture document which has the same dependency information gathered directly from VistA documentation.  There are some differences however, and Oncology dependencies are not listed in the architecture document at all.  Once we clarify package/module namespace information, we will get back to these dependency maps and manually verify a number of them for quality assurance.

By the way, UNKNOWN refers to calls to or from routines that we could not automatically identify from Package file. There were about 500 of these routines and about 100 of them are routines that have names that start with “%” which we can classify as KERNEL for our purposes.  Here is a sample set of other routines that our automated process could not identify to belong to a module:   A1B2OSR3, A1CKC1, ALPBFRM1, AUPNPAT, AWCMCPR1, GMVBP0, VDCVAL, IMREDIT, MXMLDOM, and YTPXRM.   Again, once we clarify package/module namespace, we will get back these routines.

In the next few weeks we will continue to characterize VistA Mumps codebase.  This will also be in the context of module-to-module dependencies.  These are some of the characteristics we are after:

  • Usage of Mumps Goto from one module to another
  • Assumed parameters for Mumps Do calls to other modules (Actual List vs Formal List)
  • Direct global usage within modules and ownership of those globals
  • Usage of Mumps Xecute command
  • Usage of indirection

XINDEX provides a good foundation to explore these, but we will need to add some additional functionality.  For example: XINDEX does not differentiate between Goto and Do for its analysis on calls from one routine to others…we will have to.  Maybe XINDEX improvements can become our first contribution to codebase.   Of course the third line of XINDEX says Per VHA Directive 2004-038, this routine should not be modified”...but that is a discussion for the OSEHRA certification team.

like0

Comments

XINDEX Utilities Already Available

Wes Turner's picture

Kathleen,

It might be worthwhile to check out two different web sites that the Certification Group is pulling together.  First, we have a Dashboard which runs and reports on XINDEX based Fatals and Warnings based on the nightly repository http://code.osehra.org/CDash/index.php?project=Open+Source+EHR.  Right now, this is reporting solely on XINDEX fatals and warnings, but as the code is refactored, enhanced, and instrumented this will be the major mechanism for detecting regression changes and errors introduced by the refactoring progress. 

Second, we just put together a module relationship chart that is on line, browsable, and generated automatically from the XINDEX results http://code.osehra.org/dox/.  I think much of what you are trying to get can be directly obtained from this browsable site.  If there is something you need that is not there, it can be added either by us, or by you with our help. 

We look forward to watching and helping your progress!

 

- Wes Turner 

like0

Good looking presentation and what more is needed

Afsin Ustundag's picture

Wes,

That is a great presentation of XINDEX results.  I think all the additional information we need/add/find about the codebase should go to these pages.

We will need more code characterization and I will try to describe some of what we need below some more in case you have this information already or if you are working on these.  First however I think we will need to clarify how these packages are related to the "modules" in architecture documents; the magic number was 168 modules and I think there are about 125 packages here.  It would be helpful for us to understand
where you are getting your namespace information.  My impression is most are coming from Package file (9.4) but some appears not to be there (LIke M XML Parser) so if you can identify namespace sources that would be great. I know the routines are listed on each package page but I think it would be great to have a namespace to package look up (and vice versa).

It would be also helpful to list which packages depends on a particular package: a fan-in graph in addition to the fan-out graph you have now.  I believe architecture team was requesting this information as well. I have a simple script that uses XINDEX generated ^UTILITY global to do this and I am not sure how you are parsing XINDEX output information with but either way it should be easy to do.

Here is a list of what we need and are working on. If you have any of this information or if our understanding of XINDEX wrong please also let us know.

1) It is possible that XINDEX is reporting more dependency than there actually is.  That is because if routine A calls to routine B and routine B calls to routine C XINDEX reports that A depends on C.  That might not be correct because if A calls to a tag in B (tag^B) that tag might not be actually calling to C.  We are trying to identify these.
2) We will need to differentiate between if dependency is based on Do calls or Goto calls.  XINDEX does not make that distinction.
3) We are trying to go down to tag level (this relates to 1 as well) for dependencies instead of routine level as it is now.
4) We are trying to identify local variables as inputs, outputs and "real local variables".  If we can do this together with 3 that will be a better interface definition between packages. 
5) We also would like to list Xecute statements and indirection usage since these can possibly create dependencies between different packages.

We are planning to work next week or two on this and our current plan is to add code to XINDEX to get this information.

Afsin
  

like0

XINDEX buggy?

Robert Sax's picture

What has been your experience wtih how well XINDEX does its job? I am not a Mumps developer, so I have never used the tool.

The reason I ask is that I have been working recently on a Mumps parser (inspired from some of the needs on a ICD10 class 3 analysis project). While it is incomplete, I was able to utilize it to do a single level (no transitive closures) dependency analysis of the two packages you mentioned above as provided in the Osehra GIT repository. The result was mostly the same, but there are some differences. Not being a mumps developer, they seem like bugs in XINDEX, but it may just be a misunderstanding on my part.

ONCOLOGY appears to depend on these additional packages

AUTOMATED LAB INSTRUMENTS(LA):/Oncology/Routines/ONCOCOC.m:13: I DZ S MODZ=$G(^LAB(61.4,+DZ,0)),MODZ=$P(MODZ,U,2)_" "_$P(MODZ,U,1) LAB SERVICE(LR):/Oncology/Routines/ONCOCFL1.m:165: S INST=$G(^LRO(68,ACCIEN,1,LBYEAR,1,LBNUM,.4)) I INST="" Q RADIOLOGY/NUCLEAR MEDICINE(RA):/Oncology/Routines/ONCOCOC.m:40: S XDT=$P(O2,U,1),RAD=$P($G(^RAMIS(71,+$P(O2,U,7),0)),U) G EX:RAD="" TOOLKIT(XT):/Oncology/Routines/ONCSAPI1.m:159: L -^XTMP("ONCSAPI","TABLES","JOB",$J)  

Health summary depends on oncology

HEALTH SUMMARY(GMTS):/Health Summary/Routines/GMTSONE.m:20: S IEN=0 F S IEN=$O(^ONCO(165.5,"C",PTIEN,IEN)) Q:IEN="" D  

Problem List

I don't see any references to PAID (may be a bug in my system or it could be caused by transitive closure - do you have a file/line reference?)

Problem List also depends on Clinical Reminders (PXRM):

CLINICAL REMINDERS(PXRM):/Problem List/Routines/GMPLP37I.m:18: ....F S IEN=$O(^PXRMINDX(9000011,"PSPI",PT,STAT,0,PL,DAT,IEN)) Q:IEN'>0 D  

and clinical reminders depends on problem list

CLINICAL REMINDERS(PXRM):/Clinical Reminders/Routines/PXRMPROB.m:67: D CALL2^GMPLUTL3(DA)

 

Robert Sax

http://www.systemsmadesimple.com/

 

 

 

 

 

like0

Globals vs. Routines

Wes Turner's picture

Robert,

We have no reason to doubt XINDEX, but we welcome any issue reports that you may find with XINDEX or with our presentation of the results.  I think the problem you may be seeing is the difference between routines and globals.  Looking at your first discrepancy:

 

AUTOMATED LAB INSTRUMENTS(LA):/Oncology/Routines/ONCOCOC.m:13: I DZ S MODZ=$G(^LAB(61.4,+DZ,0)),MODZ=$P(MODZ,U,2)_" "_$P(MODZ,U,1)

I believe that "LAB(61.4" is a reference to a global variable not to a routine.  In our diagram it does not show up as a package dependency, but does show up as a "Global Variable."   If you click on ONCOCOC and then scroll down you will see what I mean.  Glancing at the other issues, I think they may be the same.  We are working on adding globals to the diagram which may make all of this more understandable. 

The one issue I am not understanding is PAID.  I do not see where we are making a link between "Problem Report" and PAID. Are you saying you have a link in your report?

- Wes

like0

Module 1 calling Module 2, over.

Christopher Edwards's picture

The other thing to check is the patch level of the routines. This is listed on the first two lines of the routine.

Christopher

like0

Problem List to PAID

Afsin Ustundag's picture

I believe Robert is referring to the list in the original blog for Problem List to PAID dependence.  I think that dependence is coming from Faux Routines which you can get from XINDEX as well if you choose a package; run d ^XINDEX and choose GMPL when prompted.  Not sure the new visual list is doing anything with Faux Routines. 

The other differences are Global related as Wes said.  Ideally application code should not  really access globals directly but use Fileman and that is one thing that we will consider during refactoring.  Listing dependencies based on the Globals used in the application code is more accurate than not listing them.  However that is not the end of the story since the same globals can be acccessed using Fileman routines and XINDEX will give them as Fileman dependence then.

      

like0

DOX tool vs Dependencies listed here

Robert Sax's picture

My questions related to the dependencies were based on the set of dependencies in the original Blog, not the visual cross reference documentation (http://code.osehra.org/dox/). The visual cross reference documentation appears to match what I have found and provides additional detail for the global vs routine call differences. 

As far as global vs call trace, from a dependency standpoint I agree with Afsin. I would consider a global reference a worse dependency than a simple call as  it indicates tighter coupling.

For the Faux routines, how does XINDEX find the fields to check for source code?

Robert Sax

http://www.systemsmadesimple.com/

like0

XINDEX Faux Routines

Afsin Ustundag's picture

It looks at the Package File record (#9.4, global ^DIC(9.4) item 4 for a package. For example for Problem List the record id is 135 so the global it looks at is ^DIC(9.4,135,4,:,0).  These store field numbers.  For example one of them for 135 is 200 (New Person).  From these field numbers it looks at various Mumps code that are stored in ^DD.  I believe one of the code piece stored in ^DD(200 gives Problem List to PAID dependence.

Faux routines can be important depending on what they do and when they are doing it and we need to keep an eye on them during refactoring.  I do not have a handle yet on exactly what the ones reported by XINDEX are doing or if XINDEX is up to date on all the Faux routines out there.

      

like0