Working with Medicare Data in Stata

(This is hypotetical documentation for s system that does not yet exist!)

Because the Medicare files are so large, some features of Stata that are not always familiar are needed. This note illustrates these features using an example merging AMI Medpar records and their out-patient records. .

First we read in multiple years of Medpar data.

cd /mxf forvalues year=2008/2009 { use bene_id dgnscd1 admsndt if substr(dgnscd1,1,4)==" 410" using med`year' save /tmp/medami`year',replace }
• The "/mxf" directory contains the largest available datasets. There is an alternate "/mxs" directory with very small subsets, but using the same filenames. It is good for testing programs. A program can switch from test mode to estimation mode with only a change in the directory. If you specify /mx you get the small subset on the compute server and the full files on the MX machine.
• At this stage we are only concerned with creating a list of index patients. By separating the sample selection stage from later stages we can avoid taking up memory with data for non-index patients. The usual -use-, -drop if- sequence might easily take more memory than was available, and is slower. We use the -if- qualifier and a varlist containing only the variables needed for sample selection.
• We have a -use- statement because -append- doesn't allow the -if- qualifier. But that does mean the extra step of saving each year separately.
• /tmp is fast local storage. Stata tempfiles would not survive an error later in the program. We don't want to redo all the work prior to the error, so we keep the intermediate files in /tmp, where they would be available later for another run. clear forvalues year=2008/2009 { append using /tmp/medami`year' } keep if substr(dgnscd1,5,5)!="2" sort bene_id gen meddgnscd = dgnscd1 count duplicates drop bene_id, force save /tmp/medami,replace
• The individual lists of AMI patients are concatenated and sorted.
• The unwanted icd9 "410.X2" patients are dropped.
• The diagnosis from the medpar file is renamed so that it won't be overwritten by the code from the out-patient file.
• Patients with more than one AMI are reduced to a single entry. forvalues year=2008/2009 { clear use /tmp/medami merge 1:m bene_id using opc`year', keep(match) nogenerate keep if abs(admsndt-from_dt)<365 save /tmp/merge`year',replace }
• Stata allows only 2 files in a merge, so we merge the full list of AMI patients with each year of out-patient records, keeping only the matching records, and only if the out-patient date of service is within a year of the AMI event. Because Stata must reserve in advance enough memory to handle the case where every record in the using dataset matches a record in the master dataset, this step can require a lot of memory, even if only a fraction of records are kept. A -keep(match)- option can reduce the memory requirement since Stata will know the maximum number of matches is the size of the smaller file. forvalues year=2008/2009 { append using /tmp/merge`year' } save /tmp/ami,replace <--
• Here we concatenate the outpatient records to create a single multi-year file in the user home directory. capture forvalues i=0/9 { append /mx`i'/tmp/ami } save ~ami
• This step is necessary only on the MX machine, to aggregate the 10 shards into a single file. It can run on MX if ami.dta is small, otherwise it must be run separately on any of other compute servers.
• Here we use -capture- so there won't be an abend while running on a compute server.
• Because the MX divides the data across 10 nodes, it is essential that sample inclusion criteria not be based on any values from another node. Since the data is divided across nodes by patient, this program is valid for the MX. -->

  Running the code

  If the code above is saved as "ami.do" in your home directory. then it can be run on any compute server with sufficient memory in about 30 minutes: mx "do `/ami.do" Here we have used the claims data for two years but we could have used all the years for multiple files within the memory constraints of the larger compute servers. However, to include The much larger revenue center files might strain the memory or require additional effort.