Splitting a Large Directory

Once the decision to split "Making This Site" into Designing GeodSoft.com and Building GeodSoft.com the actual mechanics were quite simple and took just about three hours to effect the change on my workstation development site and all three mirror sites. Once again its the fact that all the standard web page components are script generated that made the quick change possible.

Except for one page in the "Style Sheet" section and a few preserved pages from previous designs, every page on the site has at least one link to the affected area that had to become two links with new descriptions. There are over 120 affected pages or 480 pages between the four sites. Within the affected area there were 37 pages each of which had at least one link to every other page in the area and five more links to the preceding, next and contents page. If I hadn't made a couple minor mistakes the entire change would have been made without ever exposing a broken link.

The site maintenance script has completely standard page components hard coded and adjusts other components based on their location within the site. In addition it reads several other "configuration" files which hold site components that change rapidly or are different on the different platforms. The platform specific files were not changed at all for the directory split.

One of the key files is SiteMap.asc which contains all the file names and descriptive text that is used to build the "Site map" on every standard page of the site. This is a tilde delimited file that has two types of lines or record types. Area lines identify directories and individual root level files. File lines identify specific files within the area that precedes them in SiteMap.asc. The script keeps track of where in the site the file being processed is relative to the document root to determine how much to indent the displayed description in the site map.

One line had to be added to SiteMap.asc to indicate the new design area and the "Making This Site" description was changed to "Building GeodSoft". Fourteen files were identified that were to be moved from the making directory to the new design directory. The fourteen lines for these files were already in SiteMap.asc but had to be moved in the correct order under the new "Designing GeodSoft" area entry.

For the directories that have pages that have a sequential order, a navaid.asc file is created in the directory. This file has four parts. There is a file list which has tilde delimited entries. Each line has the actual file name or URL. Typically for the table of contents pages the URL is the directory and the web server provides the default.htm or index.htm file when the directory URL is requested. There is a single line that identifies the table of contents file.

There are two blocks of HTML code with place holders where the script substitutes URLs and link descriptions. The first has the previous and next code for the links that appear at the bottom of the page specific content area. Structured HTML comments are placed in each document where these links are to be located and on each pass the script replaces the content between the comments with up-to-date links generated by merging the URLs and link descriptions into the HTLM code provided in navaid.asc. This block of code may or may not have constant HTML for a link to the table of contents.

The second block of code has the formating for the previous and next links that now appear just under the search form in the navigation and search column. Here a script variable sets the location of the links as they are within the "standard" page components. If the variable was initialized as the script passed through the HTML document, the links are output and if not, there is nothing to output.

To accomodate the new directory the table of contents file (default.htm) and navaid.asc were copied to the new directory. On the table of contents pages in both directories I edited the text to match the content changes as reflected by the files now in each directory.

I did not touch the table of contents links as those would be fixed by the script drawing on the URLs and descriptions found in navaid.asc. In both navaid.asc files I changed the constant text describing the directory and table of contents pages and the URL for the design directory. After identifying the files present in each directory, all the lines for files not present had to be deleted from each navaid.asc file.

I also had to add one new line to the script for the new directory to make "Designing GeodSoft.com" appear in the links showing your current location within the site that starts the navigation and search column. I then ran the script in the new directory. Actually I ran it before making all the changes I've identified here and when I checked the results, fixed the pieces that I'd missed such as the single line in the script just mentioned.

After everything looked clean in the directories that were part of the split, I ran the script on the entire development site and again checked results. I needed be sure that I'd correctly shuffled the lines in SiteMap.asc and navaid.asc to correctly match the files remaining in the respective directories.

Because both the Home Page and About GeodSoft pages had multiple references and links to the affected directories in the page specific content areas these needed to be edited manually. This is one place where database driven site maintenance systems have a clear advantage over a scripted system. Typically pages can be moved and all links, not just those that are part of the standard navigation aids are automatically adjusted.

I can't think of a way to do this automatically with scripts. At some point, I'll probably write a script that reads a file built manually each time a file is moved and scans the entire site fixing each reference.

When I was satisfied that the script and configuration files were right, I transferred them along with the newly modified table of contents pages for the Designing and Building directories. I had to manually create the new design directory on the Linux and OpenBSD sites.

I chose to manually copy the files from the making directory to the design directory. I did this to preserve file modification times. Using the automated ftp would have been faster but it would update the file modification time on all the transferred files. I prefer not to change the file modification times unless there is a change to the unique content of the file. I also left the files that were copied into the design directory in the making directory until all the links that were not part of the navigation aids could be found and fixed, thus avoiding broken links.

I first ran the standardization script only in the design and making directories to verify that I'd copied the right files. I had actually missed one. After these directories verified, I ran the standardization script throughout the entire sites and rechecked results. The final step was to copy the updated Home Page and About GeodSoft pages to the live sites.

Actually Home and About pages were not last. I forgot until after deleting the extra files in the making directories that the What's New page is loaded with references to files is in the making directory, a number of which moved to the design directory. This temporarily created broken links. I had to fix What's New manually. Both links and descriptive text needed to be modified. The simplest approach seemed to be to append "(formerly Making This Site)" to most of the changed descriptions.

I did not count the approximately half hour manually fixing What's New as part of the three hours. I probably should since some site management systems can handle at least part of this. (Where one link becomes two the management system needs to be very sophisticated to automatically handle which of the two to make it.) It's instructive to compare the time to fix one file that required manual fixes to the hundreds that were automated. If a site contains more than a few documents, attempting to create standard site navigation aids without an automated management system to maintain them is a guaranteed prescription for an unmaintainable site.

Top of Page - Site Map

Copyright © 2000 - 2014 by George Shaffer. This material may be distributed only subject to the terms and conditions set forth in https://geodsoft.com/terms.htm (or https://geodsoft.com/cgi-bin/terms.pl). These terms are subject to change. Distribution is subject to the current terms, or at the choice of the distributor, those in an earlier, digitally signed electronic copy of https://geodsoft.com/terms.htm (or cgi-bin/terms.pl) from the time of the distribution. Distribution of substantively modified versions of GeodSoft content is prohibited without the explicit written permission of George Shaffer. Distribution of the work or derivatives of the work, in whole or in part, for commercial purposes is prohibited unless prior written permission is obtained from George Shaffer. Distribution in accordance with these terms, for unrestricted and uncompensated public access, non profit, or internal company use is allowed.