Converting to a New Genealogy Program - RootsMagic and GEDCOM Changes

This is a continuation of my series on converting to RootsMagic version 5 (RM5) genealogy software. Earlier articles can be found at:

1. Software Conversion - Moving to RootsMagic

2. Before Converting to a New Genealogy Program - RootsMagic Conversion Notes

As part of my conversion to RM5 I decided to modify the GEDCOM file directly prior to import to get the cleanest possible import of my data. This should only be attempted by those who understand the GEDCOM spec and how to edit files in a pure text mode (not using a word processor that inserts invisible formatting characters). See the "Better GEDCOM Wiki" for more information on GEDCOM.

I've decided to describe the changes I made generically instead of including the actual code. First, the descriptions will be more readable. Second, I don't want to give enough information to allow novices to cause problems they then take to RootsMagic tech support. Anyone who understands GEDCOM will probably understand exactly what I did based on the descriptions below. If you try this and have problems, it’s your problem, not something the vendor should have to solve for you. Also, if I include all the details from the notes I made as I implemented the changes, this blog post would be many pages longer.

If I had implemented more of the custom features in my old genealogy program I would have needed to do a lot more manual work to get a clean import. Each person will have to look at their own usage and determine what additional GEDCOM changes might be needed. If you can modify a GEDCOM file directly you probably know enough about the customizations you used to figure that out.

This is the list of things I changed in the GEDCOM file before I got a fairly clean import:

1. Moved place details from the place field to place details. The output GEDCOM file includes all place fields at one level. RM5 has a place details subfield for things like "Baylor Hospital" so the master place contains city, county, state, and country if you use it.
   
2. Moved long text notes from the occupation details field. There seems to be a character limit on how much RM5 will import. Or something in my data was causing an issue on the events where I had a long set of sentences. RM5 uses the occupation detail field to hold the occupation name.
   
3. Moved long text notes from the military details field. There seems to be a character limit on how much RM5 will import. Or something in my data was causing an issue on the events where I had a long set of sentences.
   
4. Renamed my custom event tags to a standard one used by RM5, where a standard one will work. I kept custom tags I still use or that contain data I need to transfer inside of RM5 (my Research tags which may become ToDo items, Research Logs, and Research Notes in RM5).
   
5. Changed the path to exhibits (media in RM5 lingo). As part of the conversion I decided to move my document images and photographs to a higher level folder in my document area. It shortens the full path name if I want to use links to these images from other programs.
   
6. Removed references to customizations that don't transfer through GEDCOM such as [WO], [LINDEX], and a few others.
   
7. Removed special characters my old program used for privacy that are not understood by RootsMagic. RootsMagic knows how to handle my sensitivity brackets {} so I can leave those. It does not know how to handle hyphens used as exclusion markers.
   
8. Removed pseudo-people such as the "census" people I added years ago and no longer use.
   
9. Modified the sources so I could get a clean, consistent first footnote or endnote. This will be usable until I convert the sources to formatted source types within RM5. RM5 imports sources as free-form sources. My old program exported them with the data mixed with the field names where I had customized the sources to match Evidence!¹ and later Evidence Explained.²
   
10. Modified repository references in source citations. I also took this opportunity to consistently refer to repositories with the same name. National Archives, NA-Washington, and NA all became NARA. I chose to use the abbreviation NARA (and TSLAC for Texas State Library and Archives Commission, dw. for dwelling, fam. for family, etc.) in the free-form sources as there will be a lot more "subsequent" footnotes than "first" footnotes. If I can remember to change the first abbreviation usage in a report to use the full name and show the abbreviation in parentheses, there will be fewer changes to make in output reports until I convert my sources within RM5. RM5 has a nice way of handling first and subsequent references and the abbreviations within.
    
11. Used a Perl script (a simple programming language) to remove the reference number for each person so I did not have to do this manually. Years ago I used a modified Henry number, as defined in William Dollarhide's Managing a Genealogical Project.³ Once genealogy software was capable of printing an indented descendant list I really didn't need these numbers any longer, but they were still in my old database.
    
12. Fixed a few spelling errors.
    

After importing the GEDCOM file I reviewed the import errors, fixed them, and then imported to a clean database. Repeat until no import errors are reported. I then spent some time reviewing the data and reports in RM5. I exported a GEDCOM file from RM5 and was pleasantly surprised. Not only does it export the basic data, it exports many of the customizations. If you color code people in your RM5 database, even the color is exported. This means imports to another RootsMagic database will be cleaner. Other programs likely won't know how to import all of the information that RM5 exports. But maybe someday all genealogy programs will do this.

RM5 also exports all three types of printed sources—first notes, subsequent notes, and bibliography format sources. Because my sources were imported as free-form sources, all three formats contain identical text right now. I considered taking the time to edit the GEDCOM file output by RM5 to make my citations clean in all three forms. But I decided to wait until I convert from free-form to formatted sources instead of spending a lot of time on the free-form sources.

There was one major issue I had with the way my data printed after my last GEDCOM import. RootsMagic has a different philosophy about how event notes should display in reports. The event data such as date, place, and details print as specified in a sentence template. Then a citation reference is printed. Then any "notes" associated with that event are printed.

Basic writing guidelines say a citation reference should follow all of the information that comes from the cited source. But I now have a lot of information that is printed after the citation reference for the source of the information. At this time there doesn't seem to be a really good workaround for this. This may not be a big deal for those who aren't printing many reports or don't need the report to follow a particular set of style guidelines. But it is a big deal to me. Not big enough to keep me from using RM5 for now.

As the best workaround I could think of after asking experienced users, I made one last set of changes to the GEDCOM file output from RM5. After making this change I then imported it into a clean RM5 database. The change I made was to add the phrase "Additional information: " to every note attached to an event. This won't be pretty when I print a report. But it gives me a unique phrase to search for that marks the places where the text will need to be cleaned up in each report. If (hopefully when) the RootsMagic program is changed to print citation references at the end of a note, I can use the search and replace option inside of RootsMagic to remove this phrase from my notes. For now I can search and replace in my word processor so I know where I need to make changes before sharing a report.

So is my database clean and perfect now? Not by a long shot. But it wasn't perfect in my old program either. Data entry standards change over the decades. Not all of my sources of a similar type were entered the same way in my old program. At least now they are consistent. And I have a usable database I can add to and get output reports from—even in a word processor format in my 64-bit operating system.

One thing I did notice when I imported the RootsMagic GEDCOM back into RM5, DNA events don't seem to be imported and DNA test result values don't seem to be exported via GEDCOM at this time. I had entered DNA results for one person so I could test some things as I played with RM5. A DNA event was included in the GEDCOM file created by RootsMagic. The marker values were not exported. Even the included DNA event was not imported. The lesson learned here is be sure you know what is not included in a GEDCOM export or import—don't assume everything is included even if a GEDCOM file includes a lot data more than some other vendors include.

Good luck if you decide to convert to a different genealogy database. It's not an activity for sissies. If you want more information you can contact me using the e-mail address on my website or add a comment below.

---

1. Elizabeth Shown Mills, Evidence! Citing History Sources from Artifacts to Cyberspace (Baltimore, Maryland: Genealogical Publishing Company, 2007).
2. Elizabeth Shown Mills, Evidence Explained: Citing History Sources from Artifacts to Cyberspace (Baltimore, Maryland: Genealogical Publishing Company, 2007). Evidence Explained expands on and updates concepts presented by Ms. Mills in Evidence! Citation & Analysis for the Family Historian. There is a newer edition of Evidence Explained than the one I have. See evidenceexplained.com for current version and ordering information.
3. William Dollarhide, Managing a Genealogical Project: A Complete Manual for the Management and Organization of Genealogical Materials (Baltimore: Genealogical Publishing Co., rev. ed. 1998).

© 2012, Debbie Parker Wayne, CG, All Rights Reserved

Before Converting to a New Genealogy Program - RootsMagic Conversion Notes

This is a continuation of my series on converting to RootsMagic version 5 (RM5) genealogy software. Earlier articles can be found at:
1. Software Conversion - Moving to RootsMagic

I assume you have your own reasons for wanting to convert to RM5 or a different genealogy program than the one you use now. I gave mine in part one of this series where I discussed my old program, The Master Genealogist (TMG). As part of my decision and testing process I watched all of the free RootsMagic webinars. I played with data I entered directly and data I transferred into RM5. At some point I decided RM4, and later RM5, would meet most of my needs—enough to make me go through the agony of a software conversion process.

As part of my decision-making process, I read (almost) every blog post, mail list message, forum message, and FAQ I could find on converting in general and converting specifically from my old program. Search terms included GEDCOM, import, and similar words, as well as the name of my old program and RootsMagic. In addition to general searches using Google, I did searches on RootsMagic lists and forums.

According to the RootsMagic Features list, RM5 does direct imports from specific versions of some software programs. TMG isn't on the list; the move requires transfer of my data via GEDCOM. This almost always means some data does not transfer properly and requires a lot of cleanup work after the transfer. This is not the fault of any particular software company. Those custom features we all demand were mostly conceived after the GEDCOM spec was written. Each vendor has to interpret how to include data that isn't part of the GEDCOM spec. Each vendor has to decide how to interpret that data when it is imported. All vendors don't agree on the how. See the "Better GEDCOM Wiki, Shortcomings of GEDCOM" for more information.

A typical GEDCOM transfer process would be:

1. Create a GEDCOM file from TMG. Look at all the export options offered and try to choose what looks like it will provide the cleanest import to RM5.
   
   
2. Import the GEDCOM file to RM5.
   
   
3. Review the data on the screen and in printed reports and use RM5 to change everything that didn't import cleanly.
   

If you aren't familiar with GEDCOM specs and editing plain text files, you are better off making changes inside of the RootsMagic program. Because I am comfortable changing raw GEDCOM files, I decided to make some changes and redo my import. (Details on this will be in a future post.)

For those who understand the GEDCOM specs and prefer making the changes in the GEDCOM file then reimporting it, the process might be:

1. Create a GEDCOM file from TMG. Look at all the export options offered and try to choose what looks like it will provide the cleanest import to RM5.
   
   
2. Import the GEDCOM file to RM5.
   
   
3. Review the data on the screen and in printed reports to determine what changes will be easier to make in the GEDCOM file and which can be easily changed after the final import. Check placement of imported data in the person events, notes, and sources—every database field into which your imported data was placed. Print narrative reports with footnotes/endnotes and charts. Look at placement of citation reference numbers, wording on footnotes/endnotes, etc.
   
   
4. If you see things that don't look right on screen or in reports, search to see if there is a way you can modify the GEDCOM file and reimport the data to correct the problem. Whether changes will be needed or not will depend on how much customization you did in TMG. If all you have in your database are basic facts with dates and places and minimal notes, your data may import properly without requiring any changes.
   
   
5. Add a sample family to RM5 with data and source citations properly entered. Create an output GEDCOM file from RM5. (I discovered RM5 GEDCOM output is much more complete than what I got from TMG using what I thought were the best options. I am not sure how many other programs will correctly import all of this RM5 data, but at least transfers to other RM databases would be fairly complete.)
   
   
6. Review the GEDCOM file created by RootsMagic to determine how to modify the GEDCOM file created by TMG so imported data goes into the correct fields in RootsMagic.
   
   
7. Repeat the process from step 2 (import to RM5) until you get an import you can live with in step 3, then stop. If you try to go for a perfect import you could be stuck in this cycle forever. You have to be realistic about how much must be corrected at the import stage and what you can do later as time allows.
   

© 2012, Debbie Parker Wayne, CG, All Rights Reserved

Software Conversion - Moving to RootsMagic

Disclaimer: This post contains opinions based on my understanding of events and my experiences. Others may have different interpretations of these events.

Don’t tell anyone, but I’ve been a secret heretic for almost a year now. At least, according to my genealogy software buddies. I’m switching to RootsMagic as my main genealogy software.

My software of choice changed in the beginning years of my research, but I’ve used one program for over sixteen years now. For a few months I used PAF which was limited in source citation implementation and other ways. I tried FTM and quickly decided it was only good for pretty charts as far as I was concerned. I found Everyone’s Family Tree (EFT) by Bill Dollarhide and happily used it for many years. By 1995 EFT wasn’t being updated and I had been reading about The Master Genealogist (TMG) and its advanced features. As a techie, being able to customize my software appealed to me in a big way. I’ve been a user of TMG since 1995—way back in the DOS days.

At the time TMG was the most sophisticated, advanced, user-customizable genealogy software of all the applications I tested. But in the last sixteen years several other programs have caught up with TMG. Some even surpass TMG in certain areas. I started using RootsMagic when a client wanted the research I performed to be entered into a database they could get at the end of the project. I didn’t want to have to teach a non-techie how to use TMG. While I really liked TMG, too many had trouble understanding it when I tried to teach it to my local society members.

I found features in RootsMagic that I really liked. RootsMagic version 4 has most of the advanced features I use in TMG. RootsMagic version 5 has even more. And both versions have some features that aren’t in TMG—some that are important to me.

Switching from one genealogical software program to another can be an excruciating ordeal. What convinced me to endure the pain of converting my data? Two big things and a few small ones:

• Twice in the last ten years there have been long delays for new releases of TMG.
  
  • TMGv5 was delayed for over a year, in part, while features were added to entice customers of another software program that had folded (UFT). My personal research was at a point where I really needed one of the new features in TMGv5. I can’t blame the company for making a decision that brings them many new customers and more income. But it did impact my use of the program.
    
  • Since Microsoft 64-bit operating systems became widely available and the default installed on new computers, about 2006 or 2007, TMG users with a 64-bit OS have not been able to export genealogy reports to a word processor file. Suggested workarounds included exporting to a PDF or plain text file or keeping an old computer running Windows XP 32-bit version. As a techie I had a 32-bit XP laptop so this worked for me, until it died and had to be replaced with a Windows 7 system. But you can’t tell Grandma she has to keep an old computer when she just got her first computer and it has 64-bit Windows Vista on it. TMGv8 was finally released a few months ago, four years after TMGv7 and two-plus years after TMGv7.04. TMGv8 supports word processor output on a 64-bit operating system, but it comes months and months after I decided to convert.
    
• TMG is based on a database application that Microsoft is dropping support for. At some point a complete rewrite of TMG will be needed to port it to a modern database engine. My programming experience tells me this is likely to result in another long delay before a usable product is available for users. For all I know this port has been in the works for years and could be released tomorrow. But I have to make my decisions based on what I know, not what I hope has been happening behind the scenes.
  
• Some TMG users are so loyal to the TMG way that they don’t seem to see better options offered by other programs. There was a TMG-list discussion a while back about the new “Research Notes” report in RootsMagic. What some call a complicated process was documented to create a similar report in TMG. This demonstrated that the customization features of TMG can allow almost anything to be done. It also demonstrated how hard it is for a less-experienced user to do some things. In my opinion.
  
• By switching to RootsMagic for my personal research, I only have to know how to use one program, not two, for myself and my clients.
  

So, other than the “Research Notes” report, what convinced me to switch to RootsMagic? It is based on an open-source, cross-platform database engine (SQLite 3, according to Wikipedia). RootsMagic has already done the port to a modern database that I think TMG will do in the future.

As with TMG, techie RootsMagic users can write their own database queries (using SQL tools) to access the database. This should allow me to implement my own add-on to pull DNA data and lineage from my database to help with analysis of DNA results. I don’t want to spend all of my time programming instead of performing research, but it is nice to be able to access the underlying database when I need something not yet provided by a software program.

In the future I will post some of the details of modifications to the TMG output GEDCOM so I got the cleanest import to RootsMagic. I’ll also post with other information that may be of interest to RootsMagic users and those converting from other programs.

As an aside, one of the things I will miss the most about moving from TMG is the loss of Second Site by John Cardinal. It is an add-on product that creates a website from a TMG database, better than the website that either TMG or RootsMagic create. Second Site includes some of the best DNA charting tools I have seen for comparing DNA data stored in a genealogy database. I don't put my database online, but made frequent use of Second Site files to access my data without having to start the database program.

© 2012, Debbie Parker Wayne, CG, All Rights Reserved

1940 census presentation and indexing startup

Monday night I gave a presentation on accessing and using the 1940 census to a local society - Angelina County Genealogical Society in Lufkin, Texas. As part of the discussion I encourage everyone to help with the community indexing project sponsored by the U.S. National Archives, FamilySearch, Archives.com, ProQuest, and FindMyPast.com. See https://the1940census.com/ for information on the project and these sponsoring companies.

For those who are interested in indexing, there are videos to explain the process. Go to the website, hover over "Getting Started," then click on the popup for "Video Training." These videos will walk you through the account signup and getting started process:

1. Download FamilySearch Indexing software - This video guide will walk you through downloading the Indexing software.

2. Create a FamilySearch Account - This video guide will show you how to create a FamilySearch Account, which is needed to index.

3. How to Index the 1940 Census - This video guide will provide you with an overview of how to index the 1940 US Census. This video is recommended for all indexing volunteers.

4. Associate yourself with a Group (Society) - This video guide will show you how to associate yourself with a group when you first log in.

I joined the Texas State Genealogical Society indexing project so my work gets counted for Texas. I always select the 1940 Texas census project to work on when I start the indexing program. There are lots of indexing projects for things other than the 1940 census and you have to view the entire list and scroll down to find 1940 census for Texas.

While searching online I found another useful set of step-by-step instructions written for a Buncombe County, North Carolina group. These instructions are also helpful in a general way, but some items are specific to the project this group is working.

Texas is 27% indexed. We need more indexers so we can search the census sooner instead of only being able to browse.

By the way, on my 1940 U.S. Census Tips and Finding Aids page I include all of the links to documents and websites discussed in my 1940 census presentation. You can use the links and save a lot of typing of URLs.

© 2012, Debbie Parker Wayne, CG, All Rights Reserved

DNA presentation in Round Rock, June 21

On Thursday, June 21, I will be doing a presentation on DNA at the Williamson County Genealogical Society in Round Rock, Texas. The topic will be "GATA GACC! DNA and Genetic Genealogy Today." This session covers basics of Y-DNA, mitochondrial DNA, and autosomal DNA tests for genealogical purposes.

I only have a few slides covering the biology of DNA. I spend most of the time using simple examples to help newbies understand how each type of DNA is inherited and from whom. We need to understand this so we can determine who should be tested to contribute the most to our genealogical research goals. The rest of the time is used to demonstrate how you can organize DNA test results to learn as much as possible. I hate to hear someone spent hundreds of dollars getting a DNA test and then just shoved the certificate into a drawer. It isn't difficult to learn how to analyze the DNA data, but it does take an investment of time to make it worthwhile.

The Williamson County Genealogical Society (WCGS) is a vibrant and active group. The have an award winning website at http://www.williamsontxgenealogy.org/?page_id=177. It is clean and simple and easy to navigate. Check it out. They have an annual seminar in the spring with a nationally known speaker. In 2012 it was D. Joshua Taylor. In 2013 it will be John Philip Colletta. WCGS has a large pool of knowledgeable members to call on for presentations. Teri Flack will be discussing "Probate Records" in July.

© 2012, Debbie Parker Wayne, CG, All Rights Reserved

Hot Springs Village, Arkansas, Genealogy Seminars

I did an all-day seminar on April 21st at Hot Springs Village, Arkansas. Hosts were the Village Genealogical Society and the Akansa Chapter NSDAR. Hot Springs Village may be off the beaten path for some, but anyone who is invited to speak there should jump at the chance. These groups go all out to make a speaker feel welcome, prevent problems, and take care of anything that does come up. They even apologized for the poor (read non-existent) Wi-Fi service the hotel had.

There is obviously a very active genealogical community in the heart of Arkansas. They come from all over the U.S. to retire here and are interested in records from all localities, not just Arkansas. All experience levels were represented. We had attendees from Little Rock and other cities an hour or so distant. I feel like I made several new friends. The attendees were attentive, interested, and asked good questions.

We covered a variety of topics:

Using the Website of the U.S. Bureau of Land Management - General Land Office
This was one of the favorite sessions. Some didn't know about ordering the Land Entry Case files from the National Archives after finding initial information on this website. Some really liked seeing a rectangular survey map overlaid on a USGS topographical map to show the terrain as it surrounded an ancestor's home. We forget sometimes how the terrain probably influenced where records are recorded and where associates can be found.

Online Research: Basics and Beyond
This was a potpourri of useful sites and tips on using them. This was the first exposure of some to Steve Morse's fantastic One-Step Webpages with tools to help genealogists locate records more easily than can be done on many commercial and non-profit sites. Learning more about Google Books and Internet Archive also seemed to be popular.

Ours and Theirs: Tax and Land Laws
Introducing researchers to tax records is always fun. New researchers don't realize how much you can learn about an ancestor from tax rolls. I hope experienced researchers learned a few new tricks, too, about using tax laws to help interpret the records.

The 1940 U.S. Federal Census and Finding Aids
This was my first chance to present a session on the 1940 census which had only been available for nineteen days. I loved this session—it gave me an excuse to work on my ancestors in the 1940 census even while I was preparing for the seminar. Even new researchers understand how important it is to know the guidelines given to the enumerators so we can better understand the records.

Once I returned home I put most of the links covered in the presentations on my website so researchers can click a link instead of typing in a long URL. Check out http://debbiewayne.com/lectures.php if you think the links may help you, too.

---

To cite this post:
Debbie Parker Wayne, "Hot Springs Village, Arkansas, Genealogy Seminars," Deb's Delvings in Genealogy, blog, posted 8 May 2012 (http://debsdelvings.blogspot.com/2012/05/hot-springs-village-arkansas-genealogy.html : accessed [access date]).


© 2012, Debbie Parker Wayne, CG, All Rights Reserved

Finding History

Not only have I been pressed for time to blog myself, I haven't been able to keep up reading other's blogs recently. I took some time this morning to check out a few good ones.

Harold Henderson's Midwestern Microhistory has always been a favorite. His latest post on history book reviews is "History books of potential interest to genealogists."¹ If you click on "books" in his right navbar you can see all of his posts related to books. One of the great things about blog labels is that you can easily go back and find things posted years ago on a related topic.

I have learned of quite a few history books that focus on issues, large and small, important to family history and to the history of women from Harold's posts. Check out:

Harold Henderson, “She fought the law, and sometimes won,” Midwestern Microhistory: A Genealogy Blog, posted 29 July 2009 (http://midwesternmicrohistory.blogspot.com/2009/07/she-fought-law-and-sometimes-won.html : accessed 8 May 2012).

Harold Henderson, “4 new books from Chicago,” Midwestern Microhistory: A Genealogy Blog, posted 14 May 2009 (http://midwesternmicrohistory.blogspot.com/2009/05/4-new-books-from-chicago.html : accessed 8 May 2012).

Harold Henderson, “Learning about divorce,” Midwestern Microhistory: A Genealogy Blog, posted 12 January 2009 (http://midwesternmicrohistory.blogspot.com/2009/01/learning-about-divorce.html : accessed 8 May 2012).

You'll find what interests you if you follow the links on his blog.

---

Notes:
1. Harold Henderson, “History books of potential interest to genealogists” Midwestern Microhistory: A Genealogy Blog, posted 3 May 2012 (http://midwesternmicrohistory.blogspot.com/2012/05/history-books-of-potential-interest-to.html : accessed 8 May 2012).

---

To cite this post:
Debbie Parker Wayne, "Finding History," Deb's Delvings in Genealogy, blog, posted 8 May 2012 (http://debsdelvings.blogspot.com/2012/05/finding-history.html : accessed [access date]).

© 2012, Debbie Parker Wayne, CG, All Rights Reserved