I haven’t posted in awhile because I’ve been working on a very large project that included:
- Simplified information architecture
- New design
- Migrating an existing blog, school, and e-commerce store — all using WordPress — into a single installation
Let’s get into some specifics. The client was What’s Your Grief, a website co-authored by two Baltimore-based mental health professionals that provides grief support and education to grieving individuals and professionals. The project was a collaborative effort between me and the talented team at Drio. Drio forged the relationship with the client and handled the client-facing and project management aspects, as well as providing the design and information architecture guidance.
Blog
WYG had a blog with 450+ posts dating back to 2012, 1,200+ subscribers, and averaged 6,500+ views per day. In addition, the blog’s average views per day has increased approximately 30% year-over-year.
School
WYG had a school powered by WPLMS, offering self-paced and instructor-led courses. These courses offered valuable training to individuals and professionals on topics such as coping with grief during the holidays and parenting while grieving. The school had 500+ students and participants. WooCommerce powered the e-commerce portion of the school where users could purchase course access.
Store
WYG had a store powered by WooCommerce, offering guides and brochures on important topics such as surviving the grief of an overdose death and supporting a grieving teen. The store had almost 1,000 orders.
The Challenge
While functioning well in their own right, each website had a different look and feel, which provided an uneven and oftentimes confusing experience for users, as much of the content was linked to the other websites. For example, blog posts would link to courses on the school and products on the store. Courses would link to supplemental material on the blog and store.
The project entailed making the school the primary site with the most important goal to ensure that the process — from visiting the site, to registration and checkout, to participating in the class — was clean, seamless, and intuitive. The secondary goal was to ensure design consistency and that the aesthetic of the school was inviting and conducive for learning.
So, the pages, posts, users, and media from the blog needed to migrate to the school. Products, customers, and orders from the store needed to migrate to the school. As WordPress treats the overwhelming majority of its content as a post, each with a unique identifier, we’re talking post ID conflict hell.
The Solution
Okay, what was I supposed to do? First off, there’s no easy solution — there’s no one plugin that does this or one knowledgebase article, forum post, or Stack Overflow answer that provides a silver bullet. That said, what I was doing was not unique, so there were plenty of resources out there. Below are some techniques and lessons learned that helped me along the way. I hope they help you too.
1. Budget for Premium Plugins
Good software and good support costs money. They should. There are so many great, free resources available to the WordPress community, but developers within the open-source community should be able to start a business and make a living supporting that community with great resources. Getting off my soapbox now…
When working through your project budget, allocate for purchasing premium plugins and premium support. For example, in order to migrate the WooCommerce products, customers, and orders from the store to the school, without conflicting with the existing products, customers, and orders already on the school, I purchased 3 plugins from WooCommerce:
- Product CSV Import Suite — $199 — Imports AND exports products as a CSV
- Order/Customer CSV Export — $79 — Exports orders and customers as a CSV
- Customer/Order/Coupon CSV Import Suite — $79 — Imports orders and customers from the exported CSV
In addition, support comes with the subscription, and I had a very lengthy ticket with their agents that ultimately resolved one of my bigger issues with the migration. Let me explain.
I had successfully imported over 1,500 customers into the school using the Order/Customer Export/Import plugins. However, these customers were assigned new customer IDs in order to not conflict with existing customer data.
This caused an issue when importing orders, because the customer_id in the orders-export CSV did not align with the newly assigned IDs of those customers. Therefore, I was getting hundreds of orders skipped because the customer ID did not exist.
An agent recommended using the customer billing email column, as the email will uniquely identify users. As instructed, I duplicated the billing email column and renamed it customer_user. Then, when I mapped the fields, I “skipped” the customer_id and selected “Customer user (ID, username or email)” for the customer_user.
Voila!
Not only did I gain this valuable information through the support that comes with the paid subscription, but the import plugins are good software. They offer a “Dry Run” capability to test the results without actually committing the records to the database. Good stuff.
Not to belabor the point, but the modest license cost for WPLMS and the support that comes with it was also invaluable in resolving several issues the client raised, such as integrating bbPress forums within the courses and how to set up a private course.
2. Plugin Audit
Across the blog, school, and store, there were 60 plugins used. Sixty. That said, some of the volume was understandable. Very large plugins like WPLMS and WooCommerce deliberately modularize functionality in order to keep the base installation lean. So WPLMS is bundled with 12 plugins for additional functionality such as event management, user and administrator dashboards, and custom shortcodes. Several plugins were used in conjunction with WooCommerce for things like the Stripe payment gateway, offering bulk discounts, and custom sidebars for specific products.
However, take the time to do a thorough plugin audit. It’s important and has so many benefits:
- Reducing the number of plugins makes your site load faster by eliminating unused Javascript and stylesheet assets. This reduces the page size and the number of HTTP requests.
- Reducing the number of plugins makes your site more secure by reducing the number of ways a hacker can exploit vulnerabilities in your website.
- Reducing the number of plugins makes the migration easier because there’s less to migrate.
- It’s another opportunity to engage your client.
3. Backup Often
Seriously, backup often. If your hosting company offers daily backups of the site files and the database, use that service. But also, make sure that you can backup the site files and the database on an as-needed basis, whether through traditional FTP and phpMyAdmin, or any of the great, free, WordPress plug-ins that bundle the site files and database into a single zip or tarball. Archive these backups for posterity, either with an external USB hard drive or a cloud service like Amazon S3.
In addition, more robust themes offer an Import/Export feature of the configured and customized theme options. WPLMS offers this, storing the social sharing information, Google Analytics code, fonts, logos, header and footer information, etc in a JSON object that can be downloaded and ported to other WPLMS instances. Backup these kinds of configuration files with the same frequency as the site and database backups.
For migrations of this complexity, with the large number of database records being ported and the large number of posts, pages, media attachments, and myriad custom post types transferred, having these backups is essential to restore the site after testing and if there are unforeseen complications during site launch.
4. Test
By the time the site went live, I had performed the migration dozens of times on the development environment. In addition to the “Dry Runs” I mentioned earlier, I repeated the process of exporting and importing the WordPress eXtended RSS files using WordPress’s native Import and Export tools. I had gone through the full add to cart and checkout process with the my Stripe Payment Transaction Mode set to “Test Mode”.
I created an outline in chronological order of each step in the migration, adding more and more detail with each attempt until I had a script that I could follow on launch day. The script gave me the confidence that I had covered all of my bases. If I missed anything during launch, it would be because it was something I had not anticipated (an unknown unknown) and not because I forgot a step or hadn’t thought the process all the way through. It was a long document — 101 lines — consisting of mostly shorthand that is discernible only to me, but I could start at the top and provide accurate estimates on how long the migration would take, and how long the sites would be inaccessible. Below is literally the script I used.
| 1. Complete backups | |
| 2. Maintenance mode for Production Blog, School, Store | |
| 3. Flywheel - baseline SQL dump - 3.sql | |
| 4. Production Store CSV Export | |
| a) Export Customers - 4a.csv | |
| b) Export Orders - 4b.csv | |
| 5. Flywheel - Customer/Order/Coupon CSV Import Suite | |
| a) Import Customers | |
| i) Update existing records if a match is found | |
| ii) Insert as new if a match is not found | |
| b) Dry Run - 5b.png | |
| c) Commit to database - 5c.png | |
| d) Flywheel - SQL dump - 5d.sql | |
| 6. Flywheel - Customer/Order/Coupon CSV Import Suite | |
| a) Modify Orders CSV - 6a.csv | |
| i) Find and replace the following strings: Match whole word only, Wrap around, Normal Search Mode | |
| 1) product_id:18 -> product_id:15864 (261 hits) | |
| 2) product_id:26 -> product_id:15868 (99 hits) | |
| 3) product_id:890 -> product_id:15873 (108 hits) | |
| 4) product_id:1047 -> product_id:15879 (122 hits) | |
| ii) Modify Orders CSV per ticket to address customer_id/customer_user | |
| b) Import Orders | |
| i) Update existing records if a match is found | |
| ii) Insert as new if a match is not found | |
| c) Dry Run - 6c.png | |
| d) Commit to database - 6d.png | |
| e) Flywheel - SQL dump - 6e.sql | |
| 7. Production School CSV Export | |
| a) Export Customers - 7a.csv | |
| i) Start Date - 2017-02-17 | |
| ii) End Date - 2017-04-15 | |
| b) Export Orders - 7b.csv | |
| i) Start Date - 2017-02-17 | |
| ii) End Date - 2017-04-15 | |
| 8. Flywheel - Customer/Order/Coupon CSV Import Suite | |
| a) Import Customers | |
| i) Update existing records if a match is found | |
| ii) Insert as new if a match is not found | |
| b) Dry Run - 8b.png | |
| c) Commit to database - 8c.png | |
| d) Flywheel - SQL dump - 8d.sql | |
| 9. Flywheel - Customer/Order/Coupon CSV Import Suite | |
| a) Modify Orders CSV per ticket to address customer_id/customer_user - 9a.csv | |
| b) Import Orders | |
| i) Update existing records if a match is found | |
| ii) Insert as new if a match is not found | |
| c) Dry Run - 9c.png | |
| d) Commit to database - 9d.png | |
| e) Flywheel - SQL dump - 9e.sql | |
| 10. Production School LMS > Settings > Import/Export | |
| a) Generate Export File for 30 Day Grief Journaling Intensive (ID: 2479) - 10a.csv | |
| i) Export Users | |
| ii) Export Connected User data | |
| b) Generate Export File for A Guide to Navigating Grief (ID: 2599) - 10b.csv | |
| i) Export everything | |
| 11. Flywheel LMS > Settings > Import/Export | |
| a) Import 30 Day Grief Journaling Intensive user data | |
| b) Flywheel - SQL dump - 11b.sql | |
| 12. Flywheel LMS > Settings > Import/Export | |
| a) Import A Guide to Navigating Grief (ID: 18438) | |
| b) Flywheel - SQL dump - 12b.sql | |
| c) Set category as "Collaborative Learning" | |
| d) Delete "Navigation Grief,Collaborative Learning" Course Category | |
| e) Delete "navigating" Unit Tag | |
| f) Edit 2599 slug and set to Draft | |
| g) Edit 18438 slug for consistent permalinks | |
| h) Set Linkage for 18438 units and Trash 2599 units | |
| i) Trash 2599 | |
| j) Edit A Guide To Navigating Grief product (ID: 3544) and set Associated Courses | |
| k) Flywheel - SQL dump - 12k.sql | |
| 13. Tested purchase and access to A Guide To Navigating Grief | |
| 14. Restored 12k.sql | |
| 15. Production School Tools > Export | |
| a) Export Forums - 15a.xml | |
| b) Export Topics - 15b.xml | |
| c) Export Replies - 15c.xml | |
| 16. Flywheel Tools > Import | |
| a) Restore all Forums (previously Trashed by client) | |
| b) Import Forums - 16b.png | |
| i) Assign authors | |
| c) Import Topics - 16c.png | |
| i) Assign 66 authors | |
| 1) Search by username | |
| 2) Search User ID within select listing | |
| d) Import Replies - 16d.png | |
| i) Assign authors | |
| e) Flywheel - SQL dump - 16e.sql | |
| f) Trash all Forums | |
| g) Hide Forum link via stylesheet | |
| 17. Deactivate example Follow-Up Emails | |
| 18. WooCommerce > Settings > Checkout > Stripe Payment | |
| a) Set Transaction Mode to Live Mode | |
| 19. DNS | |
| a) HostGator - Record original settings - 19a.png | |
| b) HostGator - Update A record for "whatsyourgrief.com" | |
| c) FlyWheel - Make whatsyourgrief.com Primary | |
| 20. 301 redirects via Settings > Redirections | |
| 21. JetPack | |
| a) Social sharing | |
| b) Related posts | |
| c) Akismet |
This leads directly into my final piece of advice…
5. Document Everything
This project took several months to complete, with untold emails, Slack messages, and support tickets. Organize and keep all of it. Print important emails and support tickets to PDF so you can access them offline. Make sure to capture all of the research, the trial and error, and the lessons learned.
Documenting your work validates the work you do. And it’s a useful resource for future projects, either when addressing a technical issue or using it as a basis for comparison when quoting other work. And if you or your client has a particular question about a setting or decision that was made a year from now, you can use your documentation as reference material.
Conclusion
The launch of whatsyourgrief.com was largely successful. For me, the unknown unknown was how the site would perform under a significant load. Next time on a site of this caliber, I’ll be sure to look for resources and tools that perform load testing. Harkening back to using premium plugins, I’ve contacted several of the plug-in developers for customizations that would only load their CSS and Javascript assets on the pages that used its functionality. I found a great Gist that only loaded WooCommerce assets on WooCommerce pages. With a blog dating back to 2012, there were over 2,000 revisions that were deleted, speeding up the query times. Working closely with the hosting provider, we’ve tuned the Varnish HTTP cache.
This has been a challenging and rewarding project. Fine-tuning the site performance and adding additional functionality will continue into the foreseeable future, and I’m looking forward to monitoring the site traffic and making the necessary adjustments to continue to grow this important resource for those who are grieving.