Over the course of debugging the premature end of the routine import on a GTM environment with UTF-8 enabled, we discovered that the OSEHRA code base contains characters that UTF-8 cannot properly encode. When exporting the routines from the Cache database, it is encoded in Latin1, which causes the UTF-8 read to fail.
To create a code base that can be impored into the UTF-8 instance, there are two options:
1. Use the 'iconv' command (http://linux.die.net/man/1/iconv) to convert the local files to UTF-8 encoding from Latin1.
This method consists of looping over the entire OSEHRA code base and piping each file into the command.
iconv -f latin1 -t utf-8
The command that was used to create the files commited to the repository in the second option is as follows. From the VistA-FOIA directory run:
git ls-files -- 'Packages/*.m' | while read f; do cat "$f" | iconv -f latin1 -t utf-8 > "$f.u" && mv "$f.u" "$f"; done
This will go through all routines in the Packages directory, read and convert the to UTF-8, and then replace the current file with the newly encoded one. To do the same for the globals, simply replace 'Packages/*.m' with 'Packages/*.zwr'.
At this point, this repository can be used with the current instructions for importing the routines and globals into your UTF-8 instance.
2. Add a remote repository pointing at the github VistA-FOIA Git instance and retrieve a branch called convert_latin1_to_utf-8.
Disclaimer: This branch contains the OSEHRA Master branch as of 2/3/2012 (SHA: 308d1760e5dc813a9288d6ef92d530b3caa58579)
This branch contains the results of running the inconv command over the OSEHRA code base. To add a remote repository, go to the local repository of the OSEHRA VistA-FOIA. To add the remote repository use the command:
git remote add github git://github.com/OSEHRA/VistA-FOIA.git
This will give Git another repository to pull from. You can check that it was sucessful by running the command:
git remote -v
which will list all remotes that are currently configured for your system. You should see something like what is below which has two branches with a github label.
Fig 1: Result of git remote -v
The next step is to fetch the remote repository. This is done with the following command:
git fetch github
Git will then use that as a repository that can be pulled from. At this step you should see that some new branches were created. The one we need is called 'convert_latin1_to_utf-8'
Fig 2. Fetching the github remote
The final step is to checkout a local branch which contains the same files as that 'convert_latin1_to_utf-8' branch. The following command will do just that:
git checkout -b convert_to_utf-8 github/convert_latin1_to_utf-8
This will create a local branch called 'convert_to_utf-8' which holds the same information as the branch on Github called 'convert_latin1_to_utf-8'. At this point, this repository can be used with the current instructions for importing the routines and globals into your UTF-8 instance.